2024-09-21T05:40:25.849 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T05:40:25.873 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T05:40:25.968 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913513' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 1843221895adccfb3b692060251dd6a0183bc410 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIcFvSYb0NWh/GafGucXKHWWzVDXHCAyw+0zBqqzo/kzyYObF8+htqyVCdEyK6VOGcO3Ow0f/VjUBeTcTgt689Q= smithi088.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG0fxMAEFMtzuXD0JW+mGVo2ANSv7KgSatFRbCKrI0MV9F9JMBgCCJNiew3r9vrRK1tXUmaGmsIK9RsYL4KG8/U= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgressReporter teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T05:40:25.968 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T05:40:25.969 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T05:40:25.969 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T05:40:25.970 INFO:teuthology.task.internal:Checking packages... 2024-09-21T05:40:25.990 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T05:40:25.991 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T05:40:25.991 INFO:teuthology.packaging:ref: None 2024-09-21T05:40:25.991 INFO:teuthology.packaging:tag: None 2024-09-21T05:40:25.991 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T05:40:25.991 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:40:25.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T05:40:26.133 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120-g18432218-1jammy 2024-09-21T05:40:26.134 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T05:40:26.141 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T05:40:26.141 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T05:40:26.188 INFO:teuthology.task.internal:Saving configuration 2024-09-21T05:40:26.205 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T05:40:26.213 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T05:40:26.234 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:34:23.301045', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOjSVU0Nc6d9FE2yUkNgc7PUG2TDz03PpYX8iR+zHyoY'} 2024-09-21T05:40:26.253 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:34:23.302440', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIO38Y04OwLMynmA1IYKGZNsSM104qcmfLgw0KzeTa180'} 2024-09-21T05:40:26.253 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T05:40:26.260 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-21T05:40:26.260 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-21T05:40:26.260 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T05:40:26.330 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f25981c0160>, signals=[15]) 2024-09-21T05:40:26.330 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T05:40:26.336 INFO:teuthology.task.internal:Opening connections... 2024-09-21T05:40:26.336 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T05:40:26.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:40:26.418 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T05:40:26.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:40:26.495 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T05:40:26.502 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2024-09-21T05:40:26.507 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2024-09-21T05:40:26.507 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-09-21T05:40:26.553 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T05:40:26.553 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="Ubuntu" 2024-09-21T05:40:26.553 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="22.04" 2024-09-21T05:40:26.553 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T05:40:26.553 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_CODENAME=jammy 2024-09-21T05:40:26.554 INFO:teuthology.orchestra.run.smithi050.stdout:ID=ubuntu 2024-09-21T05:40:26.554 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE=debian 2024-09-21T05:40:26.554 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T05:40:26.554 INFO:teuthology.orchestra.run.smithi050.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T05:40:26.554 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T05:40:26.554 INFO:teuthology.orchestra.run.smithi050.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T05:40:26.554 INFO:teuthology.orchestra.run.smithi050.stdout:UBUNTU_CODENAME=jammy 2024-09-21T05:40:26.554 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-09-21T05:40:26.581 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2024-09-21T05:40:26.585 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2024-09-21T05:40:26.585 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-09-21T05:40:26.631 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T05:40:26.631 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="Ubuntu" 2024-09-21T05:40:26.631 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="22.04" 2024-09-21T05:40:26.631 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T05:40:26.631 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_CODENAME=jammy 2024-09-21T05:40:26.631 INFO:teuthology.orchestra.run.smithi088.stdout:ID=ubuntu 2024-09-21T05:40:26.631 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE=debian 2024-09-21T05:40:26.631 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T05:40:26.632 INFO:teuthology.orchestra.run.smithi088.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T05:40:26.632 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T05:40:26.632 INFO:teuthology.orchestra.run.smithi088.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T05:40:26.632 INFO:teuthology.orchestra.run.smithi088.stdout:UBUNTU_CODENAME=jammy 2024-09-21T05:40:26.632 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-09-21T05:40:26.656 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T05:40:26.665 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T05:40:26.688 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T05:40:26.688 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:40:26.690 DEBUG:teuthology.orchestra.run.smithi088:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:40:26.695 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T05:40:26.713 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T05:40:26.713 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:40:26.742 DEBUG:teuthology.orchestra.run.smithi088:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:40:26.750 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T05:40:26.798 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T05:40:26.817 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T05:40:26.817 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-21T05:40:26.817 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-21T05:40:26.818 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:40:26.818 DEBUG:teuthology.orchestra.run.smithi088:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:40:26.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:40:26.823 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-21T05:40:26.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:40:26.826 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-21T05:40:26.984 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T05:40:26.984 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-21T05:40:26.984 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-09-21T05:40:26.988 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T05:40:26.988 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-21T05:40:26.988 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-09-21T05:40:26.990 INFO:teuthology.orchestra.run.smithi050.stdout:5.15.0-56-generic 2024-09-21T05:40:26.990 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T05:40:26.990 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-21T05:40:26.990 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-09-21T05:40:26.990 DEBUG:teuthology.orchestra.run.smithi050:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-21T05:40:26.995 INFO:teuthology.orchestra.run.smithi088.stdout:5.15.0-56-generic 2024-09-21T05:40:26.996 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T05:40:26.996 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-21T05:40:26.996 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-09-21T05:40:26.996 DEBUG:teuthology.orchestra.run.smithi088:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-21T05:40:27.048 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-21T05:40:27.058 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-21T05:40:27.181 INFO:teuthology.orchestra.run.smithi050.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-21T05:40:27.181 INFO:teuthology.orchestra.run.smithi050.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-21T05:40:27.181 INFO:teuthology.orchestra.run.smithi050.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-21T05:40:27.181 INFO:teuthology.orchestra.run.smithi050.stderr:hint: 2024-09-21T05:40:27.181 INFO:teuthology.orchestra.run.smithi050.stderr:hint: git config --global init.defaultBranch 2024-09-21T05:40:27.181 INFO:teuthology.orchestra.run.smithi050.stderr:hint: 2024-09-21T05:40:27.181 INFO:teuthology.orchestra.run.smithi050.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-21T05:40:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-21T05:40:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:hint: 2024-09-21T05:40:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:hint: git branch -m 2024-09-21T05:40:27.182 INFO:teuthology.orchestra.run.smithi050.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-21T05:40:27.185 DEBUG:teuthology.orchestra.run.smithi050:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-21T05:40:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-21T05:40:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-21T05:40:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-21T05:40:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:hint: 2024-09-21T05:40:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:hint: git config --global init.defaultBranch 2024-09-21T05:40:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:hint: 2024-09-21T05:40:27.190 INFO:teuthology.orchestra.run.smithi088.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-21T05:40:27.190 INFO:teuthology.orchestra.run.smithi088.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-21T05:40:27.190 INFO:teuthology.orchestra.run.smithi088.stderr:hint: 2024-09-21T05:40:27.190 INFO:teuthology.orchestra.run.smithi088.stderr:hint: git branch -m 2024-09-21T05:40:27.191 INFO:teuthology.orchestra.run.smithi088.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-21T05:40:27.193 DEBUG:teuthology.orchestra.run.smithi088:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-21T05:40:27.253 DEBUG:teuthology.orchestra.run.smithi050:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-21T05:40:27.268 DEBUG:teuthology.orchestra.run.smithi088:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-21T05:40:27.313 DEBUG:teuthology.orchestra.run.smithi050:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-21T05:40:27.324 DEBUG:teuthology.orchestra.run.smithi088:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-21T05:40:57.058 INFO:teuthology.orchestra.run.smithi050.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-21T05:40:57.058 INFO:teuthology.orchestra.run.smithi050.stderr: * [new branch] main -> origin/main 2024-09-21T05:40:57.096 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20190312 -> 20190312 2024-09-21T05:40:57.096 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20190416 -> 20190416 2024-09-21T05:40:57.096 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20190514 -> 20190514 2024-09-21T05:40:57.096 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20190618 -> 20190618 2024-09-21T05:40:57.096 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20190717 -> 20190717 2024-09-21T05:40:57.096 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20190815 -> 20190815 2024-09-21T05:40:57.096 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20190923 -> 20190923 2024-09-21T05:40:57.098 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20191022 -> 20191022 2024-09-21T05:40:57.098 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20191215 -> 20191215 2024-09-21T05:40:57.098 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20200122 -> 20200122 2024-09-21T05:40:57.098 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20200316 -> 20200316 2024-09-21T05:40:57.098 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20200421 -> 20200421 2024-09-21T05:40:57.098 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20200519 -> 20200519 2024-09-21T05:40:57.098 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20200619 -> 20200619 2024-09-21T05:40:57.098 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20200721 -> 20200721 2024-09-21T05:40:57.099 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20200817 -> 20200817 2024-09-21T05:40:57.099 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20200918 -> 20200918 2024-09-21T05:40:57.099 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20201022 -> 20201022 2024-09-21T05:40:57.099 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20201118 -> 20201118 2024-09-21T05:40:57.099 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20201218 -> 20201218 2024-09-21T05:40:57.099 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20210208 -> 20210208 2024-09-21T05:40:57.099 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20210315 -> 20210315 2024-09-21T05:40:57.101 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20210511 -> 20210511 2024-09-21T05:40:57.101 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20210716 -> 20210716 2024-09-21T05:40:57.101 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20210818 -> 20210818 2024-09-21T05:40:57.101 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20210919 -> 20210919 2024-09-21T05:40:57.101 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20211027 -> 20211027 2024-09-21T05:40:57.101 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20211216 -> 20211216 2024-09-21T05:40:57.101 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20220209 -> 20220209 2024-09-21T05:40:57.101 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20220310 -> 20220310 2024-09-21T05:40:57.101 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20220411 -> 20220411 2024-09-21T05:40:57.102 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20220509 -> 20220509 2024-09-21T05:40:57.102 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20220610 -> 20220610 2024-09-21T05:40:57.102 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20220708 -> 20220708 2024-09-21T05:40:57.102 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20220815 -> 20220815 2024-09-21T05:40:57.102 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20220913 -> 20220913 2024-09-21T05:40:57.102 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20221012 -> 20221012 2024-09-21T05:40:57.102 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20221109 -> 20221109 2024-09-21T05:40:57.102 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20221214 -> 20221214 2024-09-21T05:40:57.102 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20230117 -> 20230117 2024-09-21T05:40:57.104 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20230210 -> 20230210 2024-09-21T05:40:57.104 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20230310 -> 20230310 2024-09-21T05:40:57.104 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20230404 -> 20230404 2024-09-21T05:40:57.104 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20230515 -> 20230515 2024-09-21T05:40:57.104 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20230625 -> 20230625 2024-09-21T05:40:57.104 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20230804 -> 20230804 2024-09-21T05:40:57.104 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20230919 -> 20230919 2024-09-21T05:40:57.105 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20231030 -> 20231030 2024-09-21T05:40:57.105 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20231111 -> 20231111 2024-09-21T05:40:57.105 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20231211 -> 20231211 2024-09-21T05:40:57.105 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20240115 -> 20240115 2024-09-21T05:40:57.105 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20240220 -> 20240220 2024-09-21T05:40:57.105 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20240312 -> 20240312 2024-09-21T05:40:57.105 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20240410 -> 20240410 2024-09-21T05:40:57.105 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20240513 -> 20240513 2024-09-21T05:40:57.105 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20240610 -> 20240610 2024-09-21T05:40:57.105 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20240709 -> 20240709 2024-09-21T05:40:57.105 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20240811 -> 20240811 2024-09-21T05:40:57.105 INFO:teuthology.orchestra.run.smithi050.stderr: * [new tag] 20240909 -> 20240909 2024-09-21T05:40:57.436 INFO:teuthology.orchestra.run.smithi088.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-21T05:40:57.436 INFO:teuthology.orchestra.run.smithi088.stderr: * [new branch] main -> origin/main 2024-09-21T05:40:57.466 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190312 -> 20190312 2024-09-21T05:40:57.466 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190416 -> 20190416 2024-09-21T05:40:57.466 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190514 -> 20190514 2024-09-21T05:40:57.466 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190618 -> 20190618 2024-09-21T05:40:57.466 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190717 -> 20190717 2024-09-21T05:40:57.466 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190815 -> 20190815 2024-09-21T05:40:57.468 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190923 -> 20190923 2024-09-21T05:40:57.468 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20191022 -> 20191022 2024-09-21T05:40:57.468 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20191215 -> 20191215 2024-09-21T05:40:57.468 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200122 -> 20200122 2024-09-21T05:40:57.468 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200316 -> 20200316 2024-09-21T05:40:57.468 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200421 -> 20200421 2024-09-21T05:40:57.468 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200519 -> 20200519 2024-09-21T05:40:57.468 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200619 -> 20200619 2024-09-21T05:40:57.468 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200721 -> 20200721 2024-09-21T05:40:57.468 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200817 -> 20200817 2024-09-21T05:40:57.468 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200918 -> 20200918 2024-09-21T05:40:57.470 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20201022 -> 20201022 2024-09-21T05:40:57.470 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20201118 -> 20201118 2024-09-21T05:40:57.470 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20201218 -> 20201218 2024-09-21T05:40:57.470 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20210208 -> 20210208 2024-09-21T05:40:57.470 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20210315 -> 20210315 2024-09-21T05:40:57.470 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20210511 -> 20210511 2024-09-21T05:40:57.470 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20210716 -> 20210716 2024-09-21T05:40:57.471 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20210818 -> 20210818 2024-09-21T05:40:57.471 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20210919 -> 20210919 2024-09-21T05:40:57.471 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20211027 -> 20211027 2024-09-21T05:40:57.471 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20211216 -> 20211216 2024-09-21T05:40:57.471 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220209 -> 20220209 2024-09-21T05:40:57.471 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220310 -> 20220310 2024-09-21T05:40:57.471 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220411 -> 20220411 2024-09-21T05:40:57.471 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220509 -> 20220509 2024-09-21T05:40:57.471 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220610 -> 20220610 2024-09-21T05:40:57.474 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220708 -> 20220708 2024-09-21T05:40:57.474 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220815 -> 20220815 2024-09-21T05:40:57.474 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220913 -> 20220913 2024-09-21T05:40:57.474 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20221012 -> 20221012 2024-09-21T05:40:57.474 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20221109 -> 20221109 2024-09-21T05:40:57.474 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20221214 -> 20221214 2024-09-21T05:40:57.474 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230117 -> 20230117 2024-09-21T05:40:57.474 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230210 -> 20230210 2024-09-21T05:40:57.474 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230310 -> 20230310 2024-09-21T05:40:57.474 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230404 -> 20230404 2024-09-21T05:40:57.474 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230515 -> 20230515 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230625 -> 20230625 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230804 -> 20230804 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230919 -> 20230919 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20231030 -> 20231030 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20231111 -> 20231111 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20231211 -> 20231211 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20240115 -> 20240115 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20240220 -> 20240220 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20240312 -> 20240312 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20240410 -> 20240410 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20240513 -> 20240513 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20240610 -> 20240610 2024-09-21T05:40:57.475 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20240709 -> 20240709 2024-09-21T05:40:57.476 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20240811 -> 20240811 2024-09-21T05:40:57.476 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20240909 -> 20240909 2024-09-21T05:41:01.964 INFO:teuthology.orchestra.run.smithi050.stderr:Updating files: 28% (1004/3520) Updating files: 29% (1021/3520) Updating files: 30% (1056/3520) Updating files: 31% (1092/3520) Updating files: 32% (1127/3520) Updating files: 33% (1162/3520) Updating files: 34% (1197/3520) Updating files: 35% (1232/3520) Updating files: 36% (1268/3520) Updating files: 37% (1303/3520) Updating files: 38% (1338/3520) Updating files: 39% (1373/3520) Updating files: 40% (1408/3520) Updating files: 41% (1444/3520) Updating files: 42% (1479/3520) Updating files: 43% (1514/3520) Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 58% (2055/3520) Updating files: 59% (2077/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 68% (2426/3520) Updating files: 69% (2429/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 81% (2884/3520) Updating files: 82% (2887/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-21T05:41:01.970 INFO:teuthology.orchestra.run.smithi050.stdout:HEAD is now at 6c88d9b8 Merge branch 'amd-staging' into 'main' 2024-09-21T05:41:01.993 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T05:41:01.993 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T05:41:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:Updating files: 43% (1523/3520) Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 59% (2077/3520) Updating files: 59% (2079/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 69% (2429/3520) Updating files: 69% (2435/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 82% (2887/3520) Updating files: 82% (2902/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-21T05:41:02.076 INFO:teuthology.orchestra.run.smithi088.stdout:HEAD is now at 6c88d9b8 Merge branch 'amd-staging' into 'main' 2024-09-21T05:41:02.099 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-21T05:41:02.099 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T05:41:02.136 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-21T05:41:02.136 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-21T05:41:02.139 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi050.front.sepia.ceph.com to complete... 2024-09-21T05:41:02.242 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-21T05:41:02.242 DEBUG:teuthology.orchestra.run.smithi088:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-21T05:41:02.245 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi088.front.sepia.ceph.com to complete... 2024-09-21T05:41:03.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21 05:41:02 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-21T05:41:03.133 INFO:teuthology.orchestra.run.smithi050.stderr:FINISHED --2024-09-21 05:41:02-- 2024-09-21T05:41:03.133 INFO:teuthology.orchestra.run.smithi050.stderr:Total wall clock time: 1.0s 2024-09-21T05:41:03.133 INFO:teuthology.orchestra.run.smithi050.stderr:Downloaded: 1 files, 43M in 0.9s (50.8 MB/s) 2024-09-21T05:41:03.137 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T05:41:03.137 DEBUG:teuthology.orchestra.run.smithi050:> sudo dpkg -i /tmp/linux-image.deb 2024-09-21T05:41:03.374 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-21T05:41:04.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21 05:41:03 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-21T05:41:04.439 INFO:teuthology.orchestra.run.smithi088.stderr:FINISHED --2024-09-21 05:41:03-- 2024-09-21T05:41:04.439 INFO:teuthology.orchestra.run.smithi088.stderr:Total wall clock time: 1.8s 2024-09-21T05:41:04.439 INFO:teuthology.orchestra.run.smithi088.stderr:Downloaded: 1 files, 43M in 1.6s (26.6 MB/s) 2024-09-21T05:41:04.443 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-21T05:41:04.443 DEBUG:teuthology.orchestra.run.smithi088:> sudo dpkg -i /tmp/linux-image.deb 2024-09-21T05:41:04.692 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-21T05:41:05.380 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-21T05:41:05.385 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-21T05:41:05.496 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-21T05:41:06.705 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-21T05:41:06.709 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-21T05:41:06.816 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-21T05:41:09.174 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-21T05:41:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:10.985 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-21T05:41:11.091 INFO:teuthology.orchestra.run.smithi088.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:32.154 INFO:teuthology.orchestra.run.smithi050.stderr:Sourcing file `/etc/default/grub' 2024-09-21T05:41:32.204 INFO:teuthology.orchestra.run.smithi050.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T05:41:32.233 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2024-09-21T05:41:32.569 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:32.588 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:32.927 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T05:41:32.934 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T05:41:32.996 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T05:41:33.002 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T05:41:33.160 INFO:teuthology.orchestra.run.smithi050.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T05:41:33.160 INFO:teuthology.orchestra.run.smithi050.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T05:41:33.160 INFO:teuthology.orchestra.run.smithi050.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T05:41:33.208 INFO:teuthology.orchestra.run.smithi050.stderr:done 2024-09-21T05:41:33.376 DEBUG:teuthology.orchestra.run.smithi050:> dpkg-deb -c /tmp/linux-image.deb 2024-09-21T05:41:33.384 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-21T05:41:33.384 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-21T05:41:33.384 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:33.403 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:33.404 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:33.427 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-21T05:41:33.427 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-21T05:41:33.428 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-21T05:41:33.429 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-21T05:41:33.429 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-21T05:41:33.429 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-21T05:41:33.429 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-21T05:41:33.429 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-21T05:41:33.431 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-21T05:41:33.431 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-21T05:41:33.431 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-21T05:41:33.431 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-21T05:41:33.431 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-21T05:41:33.431 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-21T05:41:33.431 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-21T05:41:33.431 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-21T05:41:33.431 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-21T05:41:33.432 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-21T05:41:33.433 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-21T05:41:33.449 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-21T05:41:33.450 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-21T05:41:33.450 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-21T05:41:33.450 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-21T05:41:33.450 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-21T05:41:33.450 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-21T05:41:33.450 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-21T05:41:33.450 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-21T05:41:33.450 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-21T05:41:33.450 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-21T05:41:33.450 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-21T05:41:33.451 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-21T05:41:33.452 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-21T05:41:33.453 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-21T05:41:33.453 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-21T05:41:33.453 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-21T05:41:33.453 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-21T05:41:33.453 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-21T05:41:33.453 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-21T05:41:33.453 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-21T05:41:33.453 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-21T05:41:33.453 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-21T05:41:33.453 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-21T05:41:33.453 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-21T05:41:33.453 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-21T05:41:33.454 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-21T05:41:33.454 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-21T05:41:33.454 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-21T05:41:33.454 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-21T05:41:33.454 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-21T05:41:33.454 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-21T05:41:33.454 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-21T05:41:33.455 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-21T05:41:33.456 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-21T05:41:33.457 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-21T05:41:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-21T05:41:33.459 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-21T05:41:33.460 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-21T05:41:33.493 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-21T05:41:33.493 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-21T05:41:33.493 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-21T05:41:33.493 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-21T05:41:33.493 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-21T05:41:33.494 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-21T05:41:33.494 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-21T05:41:33.494 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-21T05:41:33.494 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-21T05:41:33.494 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-21T05:41:33.494 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-21T05:41:33.495 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-21T05:41:33.495 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-21T05:41:33.495 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-21T05:41:33.495 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-21T05:41:33.495 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-21T05:41:33.495 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-21T05:41:33.496 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-21T05:41:33.496 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-21T05:41:33.496 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-21T05:41:33.496 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-21T05:41:33.496 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-21T05:41:33.496 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-21T05:41:33.497 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-21T05:41:33.497 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-21T05:41:33.497 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-21T05:41:33.497 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-21T05:41:33.497 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-21T05:41:33.497 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-21T05:41:33.498 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-21T05:41:33.498 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-21T05:41:33.498 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-21T05:41:33.498 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-21T05:41:33.498 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-21T05:41:33.498 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-21T05:41:33.498 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-21T05:41:33.499 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-21T05:41:33.499 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-21T05:41:33.499 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-21T05:41:33.499 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-21T05:41:33.499 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-21T05:41:33.499 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-21T05:41:33.500 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-21T05:41:33.500 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-21T05:41:33.500 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-21T05:41:33.500 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-21T05:41:33.500 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-21T05:41:33.500 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-21T05:41:33.501 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-21T05:41:33.501 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-21T05:41:33.501 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-21T05:41:33.501 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-21T05:41:33.501 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-21T05:41:33.501 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-21T05:41:33.501 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-21T05:41:33.502 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-21T05:41:33.502 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-21T05:41:33.502 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-21T05:41:33.502 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-21T05:41:33.502 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-21T05:41:33.502 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-21T05:41:33.503 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-21T05:41:33.503 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-21T05:41:33.503 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-21T05:41:33.503 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-21T05:41:33.503 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-21T05:41:33.504 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-21T05:41:33.504 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-21T05:41:33.504 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-21T05:41:33.504 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-21T05:41:33.504 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-21T05:41:33.504 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-21T05:41:33.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-21T05:41:33.505 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-21T05:41:33.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-21T05:41:33.505 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-21T05:41:33.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-21T05:41:33.505 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-21T05:41:33.506 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-21T05:41:33.506 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-21T05:41:33.506 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-21T05:41:33.506 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-21T05:41:33.506 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-21T05:41:33.506 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-21T05:41:33.507 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-21T05:41:33.507 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-21T05:41:33.507 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-21T05:41:33.507 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-21T05:41:33.507 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-21T05:41:33.507 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-21T05:41:33.507 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-21T05:41:33.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-21T05:41:33.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-21T05:41:33.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-21T05:41:33.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-21T05:41:33.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-21T05:41:33.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-21T05:41:33.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-21T05:41:33.509 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-21T05:41:33.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-21T05:41:33.509 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-21T05:41:33.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-21T05:41:33.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-21T05:41:33.510 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-21T05:41:33.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-21T05:41:33.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-21T05:41:33.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-21T05:41:33.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-21T05:41:33.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-21T05:41:33.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-21T05:41:33.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-21T05:41:33.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-21T05:41:33.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-21T05:41:33.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-21T05:41:33.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-21T05:41:33.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-21T05:41:33.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-21T05:41:33.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-21T05:41:33.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-21T05:41:33.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-21T05:41:33.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-21T05:41:33.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-21T05:41:33.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-21T05:41:33.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-21T05:41:33.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-21T05:41:33.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-21T05:41:33.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-21T05:41:33.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-21T05:41:33.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-21T05:41:33.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-21T05:41:33.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-21T05:41:33.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-21T05:41:33.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-21T05:41:33.513 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-21T05:41:33.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-21T05:41:33.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-21T05:41:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-21T05:41:33.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-21T05:41:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-21T05:41:33.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-21T05:41:33.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-21T05:41:33.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-21T05:41:33.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-21T05:41:33.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-21T05:41:33.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-21T05:41:33.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-21T05:41:33.525 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-21T05:41:33.526 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-21T05:41:33.527 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-21T05:41:33.528 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-21T05:41:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-21T05:41:33.530 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-21T05:41:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-21T05:41:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-21T05:41:33.533 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-21T05:41:33.534 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-21T05:41:33.534 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-21T05:41:33.534 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-21T05:41:33.556 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-21T05:41:33.556 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-21T05:41:33.556 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-21T05:41:33.556 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-21T05:41:33.556 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-21T05:41:33.556 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-21T05:41:33.556 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-21T05:41:33.556 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-21T05:41:33.556 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-21T05:41:33.556 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-21T05:41:33.556 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-21T05:41:33.556 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-21T05:41:33.557 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-21T05:41:33.558 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-21T05:41:33.559 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-21T05:41:33.560 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-21T05:41:33.561 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-21T05:41:33.562 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-21T05:41:33.562 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-21T05:41:33.562 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-21T05:41:33.562 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-21T05:41:33.562 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-21T05:41:33.562 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-21T05:41:33.562 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-21T05:41:33.562 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-21T05:41:33.562 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-21T05:41:33.562 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-21T05:41:33.562 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-21T05:41:33.562 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-21T05:41:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-21T05:41:33.564 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-21T05:41:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-21T05:41:33.566 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-21T05:41:33.567 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-21T05:41:33.568 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-21T05:41:33.569 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-21T05:41:33.570 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-21T05:41:33.571 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-21T05:41:33.572 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-21T05:41:33.573 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-21T05:41:33.574 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-21T05:41:33.575 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-21T05:41:33.575 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-21T05:41:33.575 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-21T05:41:33.575 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-21T05:41:33.575 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-21T05:41:33.575 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-21T05:41:33.575 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-21T05:41:33.576 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-21T05:41:33.576 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-21T05:41:33.576 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-21T05:41:33.576 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-21T05:41:33.576 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-21T05:41:33.576 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-21T05:41:33.576 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-21T05:41:33.577 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-21T05:41:33.577 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-21T05:41:33.577 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-21T05:41:33.577 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-21T05:41:33.577 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-21T05:41:33.577 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-21T05:41:33.577 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-21T05:41:33.578 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-21T05:41:33.578 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-21T05:41:33.578 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-21T05:41:33.578 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-21T05:41:33.578 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-21T05:41:33.578 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-21T05:41:33.579 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-21T05:41:33.579 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-21T05:41:33.579 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-21T05:41:33.579 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-21T05:41:33.579 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-21T05:41:33.579 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-21T05:41:33.580 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-21T05:41:33.580 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-21T05:41:33.580 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-21T05:41:33.580 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-21T05:41:33.580 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-21T05:41:33.580 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-21T05:41:33.580 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-21T05:41:33.581 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-21T05:41:33.581 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-21T05:41:33.581 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-21T05:41:33.581 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-21T05:41:33.581 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-21T05:41:33.581 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-21T05:41:33.582 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-21T05:41:33.582 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-21T05:41:33.582 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-21T05:41:33.582 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-21T05:41:33.582 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-21T05:41:33.582 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-21T05:41:33.582 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-21T05:41:33.583 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-21T05:41:33.583 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-21T05:41:33.583 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-21T05:41:33.583 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-21T05:41:33.583 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-21T05:41:33.583 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-21T05:41:33.584 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-21T05:41:33.584 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-21T05:41:33.584 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-21T05:41:33.584 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-21T05:41:33.584 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-21T05:41:33.584 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-21T05:41:33.585 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-21T05:41:33.585 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-21T05:41:33.585 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-21T05:41:33.585 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-21T05:41:33.585 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-21T05:41:33.585 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-21T05:41:33.585 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-21T05:41:33.586 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-21T05:41:33.586 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-21T05:41:33.586 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-21T05:41:33.586 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-21T05:41:33.586 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-21T05:41:33.586 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-21T05:41:33.587 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-21T05:41:33.614 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-21T05:41:33.615 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-21T05:41:33.615 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-21T05:41:33.615 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-21T05:41:33.615 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-21T05:41:33.615 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-21T05:41:33.615 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-21T05:41:33.615 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-21T05:41:33.615 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-21T05:41:33.615 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-21T05:41:33.615 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-21T05:41:33.615 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-21T05:41:33.615 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-21T05:41:33.617 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-21T05:41:33.618 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-21T05:41:33.619 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-21T05:41:33.620 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-21T05:41:33.621 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-21T05:41:33.622 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-21T05:41:33.622 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-21T05:41:33.622 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-21T05:41:33.622 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-21T05:41:33.622 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-21T05:41:33.622 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-21T05:41:33.622 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-21T05:41:33.622 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-21T05:41:33.622 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-21T05:41:33.622 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-21T05:41:33.623 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-21T05:41:33.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-21T05:41:33.625 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-21T05:41:33.626 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-21T05:41:33.626 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-21T05:41:33.626 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-21T05:41:33.626 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-21T05:41:33.626 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-21T05:41:33.626 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-21T05:41:33.626 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-21T05:41:33.626 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-21T05:41:33.626 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-21T05:41:33.626 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-21T05:41:33.626 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-21T05:41:33.626 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-21T05:41:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-21T05:41:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-21T05:41:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-21T05:41:33.632 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-21T05:41:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-21T05:41:33.634 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-21T05:41:33.635 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-21T05:41:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-21T05:41:33.637 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-21T05:41:33.638 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-21T05:41:33.639 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-21T05:41:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-21T05:41:33.641 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-21T05:41:33.642 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-21T05:41:33.643 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-21T05:41:33.643 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-21T05:41:33.643 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-21T05:41:33.643 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-21T05:41:33.643 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-21T05:41:33.643 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-21T05:41:33.686 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-21T05:41:33.686 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-21T05:41:33.687 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-21T05:41:33.688 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-21T05:41:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-21T05:41:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-21T05:41:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-21T05:41:33.692 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-21T05:41:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-21T05:41:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-21T05:41:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-21T05:41:33.696 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-21T05:41:33.696 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-21T05:41:33.696 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-21T05:41:33.696 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-21T05:41:33.696 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-21T05:41:33.696 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-21T05:41:33.696 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-21T05:41:33.696 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-21T05:41:33.696 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-21T05:41:33.696 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-21T05:41:33.696 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-21T05:41:33.696 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-21T05:41:33.697 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-21T05:41:33.698 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-21T05:41:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-21T05:41:33.700 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-21T05:41:33.701 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-21T05:41:33.702 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-21T05:41:33.703 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-21T05:41:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-21T05:41:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-21T05:41:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-21T05:41:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-21T05:41:33.708 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-21T05:41:33.709 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-21T05:41:33.710 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-21T05:41:33.710 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-21T05:41:33.710 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-21T05:41:33.710 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-21T05:41:33.710 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-21T05:41:33.710 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-21T05:41:33.710 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-21T05:41:33.710 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-21T05:41:33.710 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-21T05:41:33.710 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-21T05:41:33.710 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-21T05:41:33.710 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-21T05:41:33.711 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-21T05:41:33.712 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-21T05:41:33.712 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-21T05:41:33.712 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-21T05:41:33.712 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-21T05:41:33.712 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-21T05:41:33.712 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-21T05:41:33.712 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-21T05:41:33.712 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-21T05:41:33.712 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-21T05:41:33.712 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-21T05:41:33.713 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-21T05:41:33.713 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-21T05:41:33.713 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-21T05:41:33.713 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-21T05:41:33.713 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-21T05:41:33.713 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-21T05:41:33.713 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-21T05:41:33.713 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-21T05:41:33.713 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-21T05:41:33.714 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-21T05:41:33.714 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-21T05:41:33.714 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-21T05:41:33.714 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-21T05:41:33.714 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-21T05:41:33.714 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-21T05:41:33.714 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-21T05:41:33.714 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-21T05:41:33.714 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-21T05:41:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-21T05:41:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-21T05:41:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-21T05:41:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-21T05:41:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-21T05:41:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-21T05:41:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-21T05:41:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-21T05:41:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-21T05:41:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-21T05:41:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-21T05:41:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-21T05:41:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-21T05:41:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-21T05:41:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-21T05:41:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-21T05:41:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-21T05:41:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-21T05:41:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-21T05:41:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-21T05:41:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-21T05:41:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-21T05:41:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-21T05:41:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-21T05:41:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-21T05:41:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-21T05:41:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-21T05:41:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-21T05:41:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-21T05:41:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-21T05:41:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-21T05:41:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-21T05:41:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-21T05:41:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-21T05:41:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-21T05:41:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-21T05:41:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-21T05:41:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-21T05:41:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-21T05:41:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-21T05:41:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-21T05:41:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-21T05:41:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-21T05:41:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-21T05:41:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-21T05:41:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-21T05:41:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-21T05:41:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-21T05:41:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-21T05:41:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-21T05:41:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-21T05:41:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-21T05:41:33.722 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-21T05:41:33.722 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-21T05:41:33.722 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-21T05:41:33.722 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-21T05:41:33.722 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-21T05:41:33.722 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-21T05:41:33.722 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-21T05:41:33.722 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-21T05:41:33.722 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-21T05:41:33.723 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-21T05:41:33.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-21T05:41:33.723 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-21T05:41:33.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-21T05:41:33.723 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-21T05:41:33.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-21T05:41:33.723 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-21T05:41:33.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-21T05:41:33.723 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-21T05:41:33.736 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-21T05:41:33.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-21T05:41:33.737 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-21T05:41:33.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-21T05:41:33.737 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-21T05:41:33.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-21T05:41:33.737 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-21T05:41:33.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-21T05:41:33.738 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-21T05:41:33.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-21T05:41:33.738 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-21T05:41:33.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-21T05:41:33.738 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-21T05:41:33.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-21T05:41:33.738 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-21T05:41:33.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-21T05:41:33.738 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-21T05:41:33.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-21T05:41:33.739 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-21T05:41:33.740 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-21T05:41:33.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-21T05:41:33.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-21T05:41:33.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-21T05:41:33.744 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-21T05:41:33.745 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-21T05:41:33.746 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-21T05:41:33.747 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-21T05:41:33.788 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-21T05:41:33.793 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-21T05:41:33.794 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-21T05:41:33.794 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-21T05:41:33.794 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-21T05:41:33.794 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-21T05:41:33.794 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-21T05:41:33.794 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-21T05:41:33.794 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-21T05:41:33.794 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-21T05:41:33.796 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-21T05:41:33.797 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-21T05:41:33.798 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-21T05:41:33.799 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-21T05:41:33.800 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-21T05:41:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-21T05:41:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-21T05:41:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-21T05:41:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-21T05:41:33.805 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-21T05:41:33.805 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-21T05:41:33.805 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-21T05:41:33.805 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-21T05:41:33.805 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-21T05:41:33.805 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-21T05:41:33.805 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-21T05:41:33.806 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-21T05:41:33.806 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-21T05:41:33.806 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-21T05:41:33.806 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-21T05:41:33.806 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-21T05:41:33.806 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-21T05:41:33.806 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-21T05:41:33.807 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-21T05:41:33.807 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-21T05:41:33.807 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-21T05:41:33.807 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-21T05:41:33.807 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-21T05:41:33.807 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-21T05:41:33.807 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-21T05:41:33.807 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-21T05:41:33.807 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-21T05:41:33.808 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-21T05:41:33.808 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-21T05:41:33.808 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-21T05:41:33.808 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-21T05:41:33.808 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-21T05:41:33.808 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-21T05:41:33.808 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-21T05:41:33.808 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-21T05:41:33.808 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-21T05:41:33.808 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-21T05:41:33.808 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-21T05:41:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-21T05:41:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-21T05:41:33.811 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-21T05:41:33.812 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-21T05:41:33.813 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-21T05:41:33.814 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-21T05:41:33.815 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-21T05:41:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-21T05:41:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-21T05:41:33.818 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-21T05:41:33.819 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-21T05:41:33.820 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-21T05:41:33.821 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-21T05:41:33.822 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-21T05:41:33.823 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-21T05:41:33.824 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-21T05:41:33.824 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-21T05:41:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:Sourcing file `/etc/default/grub' 2024-09-21T05:41:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T05:41:33.827 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-21T05:41:33.828 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-21T05:41:33.829 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-21T05:41:33.830 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-21T05:41:33.830 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-21T05:41:33.830 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-21T05:41:33.830 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-21T05:41:33.835 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:33.835 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:33.835 DEBUG:teuthology.orchestra.run.smithi050:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-21T05:41:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:Generating grub configuration file ... 2024-09-21T05:41:33.877 INFO:teuthology.orchestra.run.smithi050.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-21T05:41:33.878 INFO:teuthology.orchestra.run.smithi050.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-21T05:41:33.878 INFO:teuthology.orchestra.run.smithi050.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-21T05:41:33.878 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-21T05:41:33.878 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:33.878 DEBUG:teuthology.orchestra.run.smithi050:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-21T05:41:33.925 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi050.front.sepia.ceph.com to complete... 2024-09-21T05:41:34.212 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:34.228 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:34.381 INFO:teuthology.orchestra.run.smithi050.stderr:Sourcing file `/etc/default/grub' 2024-09-21T05:41:34.382 INFO:teuthology.orchestra.run.smithi050.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T05:41:34.391 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2024-09-21T05:41:34.567 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T05:41:34.579 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T05:41:34.636 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:34.642 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:34.654 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T05:41:34.660 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T05:41:34.803 INFO:teuthology.orchestra.run.smithi088.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T05:41:34.803 INFO:teuthology.orchestra.run.smithi088.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T05:41:34.803 INFO:teuthology.orchestra.run.smithi088.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T05:41:34.849 INFO:teuthology.orchestra.run.smithi088.stderr:done 2024-09-21T05:41:34.945 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T05:41:34.950 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T05:41:34.997 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T05:41:35.003 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T05:41:35.049 DEBUG:teuthology.orchestra.run.smithi088:> dpkg-deb -c /tmp/linux-image.deb 2024-09-21T05:41:35.056 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-21T05:41:35.056 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-21T05:41:35.056 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:35.075 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:35.075 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:35.101 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-21T05:41:35.101 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-21T05:41:35.101 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-21T05:41:35.102 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-21T05:41:35.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-21T05:41:35.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-21T05:41:35.105 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-21T05:41:35.108 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-21T05:41:35.108 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-21T05:41:35.139 INFO:teuthology.orchestra.run.smithi050.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T05:41:35.140 INFO:teuthology.orchestra.run.smithi050.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T05:41:35.140 INFO:teuthology.orchestra.run.smithi050.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T05:41:35.162 INFO:teuthology.orchestra.run.smithi050.stderr:done 2024-09-21T05:41:35.194 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-21T05:41:35.195 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-21T05:41:35.195 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-21T05:41:35.195 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-21T05:41:35.195 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-21T05:41:35.195 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-21T05:41:35.195 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-21T05:41:35.196 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-21T05:41:35.196 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-21T05:41:35.196 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-21T05:41:35.196 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-21T05:41:35.196 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-21T05:41:35.196 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-21T05:41:35.196 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-21T05:41:35.196 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-21T05:41:35.197 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-21T05:41:35.197 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-21T05:41:35.197 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-21T05:41:35.197 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-21T05:41:35.197 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-21T05:41:35.197 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-21T05:41:35.197 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-21T05:41:35.198 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-21T05:41:35.198 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-21T05:41:35.198 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-21T05:41:35.198 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-21T05:41:35.198 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-21T05:41:35.198 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-21T05:41:35.198 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-21T05:41:35.199 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-21T05:41:35.199 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-21T05:41:35.199 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-21T05:41:35.199 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-21T05:41:35.199 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-21T05:41:35.199 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-21T05:41:35.200 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-21T05:41:35.200 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-21T05:41:35.200 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-21T05:41:35.200 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-21T05:41:35.200 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-21T05:41:35.200 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-21T05:41:35.200 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-21T05:41:35.200 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-21T05:41:35.200 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-21T05:41:35.200 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-21T05:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-21T05:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-21T05:41:35.203 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-21T05:41:35.204 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-21T05:41:35.205 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-21T05:41:35.205 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-21T05:41:35.205 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-21T05:41:35.205 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-21T05:41:35.205 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-21T05:41:35.205 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-21T05:41:35.205 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-21T05:41:35.205 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-21T05:41:35.205 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-21T05:41:35.205 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-21T05:41:35.206 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-21T05:41:35.206 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-21T05:41:35.206 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-21T05:41:35.206 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-21T05:41:35.206 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-21T05:41:35.206 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-21T05:41:35.206 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-21T05:41:35.206 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-21T05:41:35.206 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-21T05:41:35.207 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-21T05:41:35.208 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-21T05:41:35.209 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-21T05:41:35.210 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-21T05:41:35.210 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-21T05:41:35.210 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-21T05:41:35.210 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-21T05:41:35.210 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-21T05:41:35.210 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-21T05:41:35.210 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-21T05:41:35.210 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-21T05:41:35.210 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-21T05:41:35.210 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-21T05:41:35.210 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-21T05:41:35.210 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-21T05:41:35.211 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-21T05:41:35.212 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-21T05:41:35.213 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-21T05:41:35.214 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-21T05:41:35.215 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-21T05:41:35.215 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-21T05:41:35.215 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-21T05:41:35.215 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-21T05:41:35.215 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-21T05:41:35.215 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-21T05:41:35.215 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-21T05:41:35.215 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-21T05:41:35.215 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-21T05:41:35.215 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-21T05:41:35.216 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-21T05:41:35.216 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-21T05:41:35.216 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-21T05:41:35.216 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-21T05:41:35.216 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-21T05:41:35.216 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-21T05:41:35.216 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-21T05:41:35.216 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-21T05:41:35.216 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-21T05:41:35.216 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-21T05:41:35.216 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-21T05:41:35.217 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-21T05:41:35.217 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-21T05:41:35.217 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-21T05:41:35.217 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-21T05:41:35.217 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-21T05:41:35.217 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-21T05:41:35.217 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-21T05:41:35.217 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-21T05:41:35.217 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-21T05:41:35.217 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-21T05:41:35.218 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-21T05:41:35.218 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-21T05:41:35.218 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-21T05:41:35.218 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-21T05:41:35.218 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-21T05:41:35.218 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-21T05:41:35.218 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-21T05:41:35.218 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-21T05:41:35.218 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-21T05:41:35.219 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-21T05:41:35.219 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-21T05:41:35.219 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-21T05:41:35.219 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-21T05:41:35.219 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-21T05:41:35.219 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-21T05:41:35.219 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-21T05:41:35.219 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-21T05:41:35.219 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-21T05:41:35.220 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-21T05:41:35.221 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-21T05:41:35.222 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-21T05:41:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-21T05:41:35.224 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-21T05:41:35.225 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-21T05:41:35.226 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-21T05:41:35.227 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-21T05:41:35.228 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-21T05:41:35.229 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-21T05:41:35.230 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-21T05:41:35.231 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-21T05:41:35.232 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-21T05:41:35.233 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-21T05:41:35.234 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-21T05:41:35.235 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-21T05:41:35.236 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-21T05:41:35.237 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-21T05:41:35.238 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-21T05:41:35.239 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-21T05:41:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-21T05:41:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-21T05:41:35.242 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-21T05:41:35.243 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-21T05:41:35.244 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-21T05:41:35.245 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-21T05:41:35.246 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-21T05:41:35.247 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-21T05:41:35.248 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-21T05:41:35.249 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-21T05:41:35.250 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-21T05:41:35.251 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-21T05:41:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-21T05:41:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-21T05:41:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-21T05:41:35.255 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-21T05:41:35.256 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-21T05:41:35.257 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-21T05:41:35.258 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-21T05:41:35.259 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-21T05:41:35.260 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-21T05:41:35.260 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-21T05:41:35.260 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-21T05:41:35.260 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-21T05:41:35.260 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-21T05:41:35.260 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-21T05:41:35.260 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-21T05:41:35.260 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-21T05:41:35.328 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-21T05:41:35.328 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-21T05:41:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-21T05:41:35.330 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-21T05:41:35.331 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-21T05:41:35.332 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-21T05:41:35.333 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-21T05:41:35.334 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-21T05:41:35.335 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-21T05:41:35.336 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-21T05:41:35.337 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-21T05:41:35.338 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-21T05:41:35.339 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-21T05:41:35.340 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-21T05:41:35.341 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-21T05:41:35.342 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-21T05:41:35.343 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-21T05:41:35.344 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-21T05:41:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-21T05:41:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-21T05:41:35.347 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-21T05:41:35.348 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-21T05:41:35.349 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-21T05:41:35.350 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-21T05:41:35.351 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-21T05:41:35.352 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-21T05:41:35.353 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-21T05:41:35.354 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-21T05:41:35.355 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-21T05:41:35.356 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-21T05:41:35.357 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-21T05:41:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-21T05:41:35.359 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-21T05:41:35.360 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-21T05:41:35.361 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-21T05:41:35.361 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-21T05:41:35.361 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-21T05:41:35.361 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-21T05:41:35.361 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-21T05:41:35.361 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-21T05:41:35.361 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-21T05:41:35.361 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-21T05:41:35.361 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-21T05:41:35.361 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-21T05:41:35.361 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-21T05:41:35.361 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-21T05:41:35.362 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-21T05:41:35.363 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-21T05:41:35.364 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-21T05:41:35.364 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-21T05:41:35.364 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-21T05:41:35.364 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-21T05:41:35.364 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-21T05:41:35.364 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-21T05:41:35.364 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-21T05:41:35.364 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-21T05:41:35.364 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-21T05:41:35.365 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-21T05:41:35.365 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-21T05:41:35.365 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-21T05:41:35.365 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-21T05:41:35.365 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-21T05:41:35.365 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-21T05:41:35.365 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-21T05:41:35.365 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-21T05:41:35.365 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-21T05:41:35.365 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-21T05:41:35.366 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-21T05:41:35.367 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-21T05:41:35.368 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-21T05:41:35.369 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-21T05:41:35.370 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-21T05:41:35.371 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-21T05:41:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-21T05:41:35.373 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-21T05:41:35.374 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-21T05:41:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-21T05:41:35.376 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-21T05:41:35.377 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-21T05:41:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-21T05:41:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-21T05:41:35.380 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-21T05:41:35.381 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-21T05:41:35.382 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-21T05:41:35.383 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-21T05:41:35.384 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-21T05:41:35.385 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-21T05:41:35.385 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-21T05:41:35.385 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-21T05:41:35.385 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-21T05:41:35.385 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-21T05:41:35.385 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-21T05:41:35.385 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-21T05:41:35.385 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-21T05:41:35.385 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-21T05:41:35.385 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-21T05:41:35.385 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-21T05:41:35.385 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-21T05:41:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-21T05:41:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-21T05:41:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-21T05:41:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-21T05:41:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-21T05:41:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-21T05:41:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-21T05:41:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-21T05:41:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-21T05:41:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-21T05:41:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-21T05:41:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-21T05:41:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-21T05:41:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-21T05:41:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-21T05:41:35.390 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-21T05:41:35.391 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-21T05:41:35.392 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-21T05:41:35.393 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-21T05:41:35.393 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-21T05:41:35.393 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-21T05:41:35.393 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-21T05:41:35.393 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-21T05:41:35.393 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-21T05:41:35.447 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-21T05:41:35.447 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-21T05:41:35.447 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-21T05:41:35.448 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-21T05:41:35.449 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-21T05:41:35.450 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-21T05:41:35.451 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-21T05:41:35.451 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-21T05:41:35.451 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-21T05:41:35.451 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-21T05:41:35.451 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-21T05:41:35.451 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-21T05:41:35.451 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-21T05:41:35.451 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-21T05:41:35.451 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-21T05:41:35.451 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-21T05:41:35.451 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-21T05:41:35.451 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-21T05:41:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-21T05:41:35.453 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-21T05:41:35.454 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-21T05:41:35.455 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-21T05:41:35.456 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-21T05:41:35.457 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-21T05:41:35.458 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-21T05:41:35.459 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-21T05:41:35.460 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-21T05:41:35.461 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-21T05:41:35.462 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-21T05:41:35.463 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-21T05:41:35.464 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-21T05:41:35.465 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-21T05:41:35.466 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-21T05:41:35.467 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-21T05:41:35.468 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-21T05:41:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-21T05:41:35.470 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-21T05:41:35.471 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-21T05:41:35.472 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-21T05:41:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-21T05:41:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-21T05:41:35.475 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-21T05:41:35.476 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-21T05:41:35.477 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-21T05:41:35.478 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-21T05:41:35.479 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-21T05:41:35.480 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-21T05:41:35.481 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-21T05:41:35.482 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-21T05:41:35.483 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-21T05:41:35.484 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-21T05:41:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-21T05:41:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-21T05:41:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-21T05:41:35.488 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-21T05:41:35.489 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-21T05:41:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-21T05:41:35.491 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-21T05:41:35.492 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-21T05:41:35.493 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-21T05:41:35.493 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-21T05:41:35.493 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-21T05:41:35.493 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-21T05:41:35.493 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-21T05:41:35.493 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-21T05:41:35.493 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-21T05:41:35.493 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-21T05:41:35.497 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:35.498 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:35.498 DEBUG:teuthology.orchestra.run.smithi088:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-21T05:41:35.504 INFO:teuthology.orchestra.run.smithi088.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-21T05:41:35.504 INFO:teuthology.orchestra.run.smithi088.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-21T05:41:35.505 INFO:teuthology.orchestra.run.smithi088.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-21T05:41:35.505 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-21T05:41:35.505 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:35.505 DEBUG:teuthology.orchestra.run.smithi088:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-21T05:41:35.549 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi088.front.sepia.ceph.com to complete... 2024-09-21T05:41:36.047 INFO:teuthology.orchestra.run.smithi088.stderr:Sourcing file `/etc/default/grub' 2024-09-21T05:41:36.048 INFO:teuthology.orchestra.run.smithi088.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T05:41:36.064 INFO:teuthology.orchestra.run.smithi088.stderr:Generating grub configuration file ... 2024-09-21T05:41:36.305 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:36.311 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:41:36.619 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T05:41:36.624 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T05:41:36.668 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T05:41:36.674 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi088.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi088.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi088.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T05:41:36.814 INFO:teuthology.orchestra.run.smithi088.stderr:done 2024-09-21T05:42:06.298 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T05:42:06.299 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-21T05:42:06.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:42:07.933 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-21T05:42:07.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T05:42:07.933 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:42:24.755 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-09-21T05:42:26.291 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-09-21T05:42:33.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-21T05:42:33.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:42:35.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T05:42:35.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:42:38.351 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-09-21T05:42:50.362 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T05:42:50.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:42:53.423 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-09-21T05:43:08.431 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T05:43:08.431 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:43:11.507 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-09-21T05:43:29.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T05:43:29.518 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:43:32.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-09-21T05:43:33.769 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T05:43:45.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-21T05:43:45.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:43:45.990 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-09-21T05:43:46.656 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-21T05:43:46.656 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T05:43:46.656 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-09-21T05:43:46.708 INFO:teuthology.orchestra.run.smithi050.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:43:46.708 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:43:46.708 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T05:43:46.708 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T05:43:47.709 DEBUG:teuthology.parallel:result is None 2024-09-21T05:43:53.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T05:43:53.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:43:53.908 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-09-21T05:43:54.524 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T05:43:54.524 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T05:43:54.524 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-09-21T05:43:54.573 INFO:teuthology.orchestra.run.smithi088.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:43:54.573 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T05:43:54.573 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T05:43:54.573 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T05:43:55.575 DEBUG:teuthology.parallel:result is None 2024-09-21T05:43:55.576 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T05:43:55.582 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T05:43:55.582 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:43:55.586 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:43:55.594 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T05:43:55.616 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T05:43:55.683 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T05:43:55.684 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:43:55.687 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:43:55.715 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T05:43:55.721 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T05:43:55.721 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:43:55.743 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:43:55.831 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:43:55.849 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:43:55.852 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:43:55.872 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:43:55.874 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T05:43:55.880 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T05:43:55.881 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:43:55.884 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:43:55.905 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T05:43:55.913 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T05:43:55.914 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:43:55.947 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:43:55.955 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:43:56.004 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:43:56.056 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:43:56.057 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:43:56.127 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:43:56.135 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:43:56.186 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:43:56.186 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:43:56.256 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2024-09-21T05:43:56.259 DEBUG:teuthology.orchestra.run.smithi088:> sudo service rsyslog restart 2024-09-21T05:43:56.375 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T05:43:56.383 INFO:teuthology.task.internal:Starting timer... 2024-09-21T05:43:56.383 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T05:43:56.413 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T05:43:56.444 DEBUG:teuthology.task.selinux:Excluding smithi050: OS 'ubuntu' does not support SELinux 2024-09-21T05:43:56.444 DEBUG:teuthology.task.selinux:Excluding smithi088: OS 'ubuntu' does not support SELinux 2024-09-21T05:43:56.444 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T05:43:56.444 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-21T05:43:56.445 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T05:43:56.445 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T05:43:56.477 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T05:43:56.479 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T05:43:56.585 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T05:43:56.594 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T05:43:56.595 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi050.front.sepia.ceph.com,smithi088.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T05:52:20.848 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi050.front.sepia.ceph.com'), Remote(name='ubuntu@smithi088.front.sepia.ceph.com')] 2024-09-21T05:52:20.849 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-21T05:52:20.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:52:20.929 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-09-21T05:52:21.037 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-21T05:52:21.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T05:52:21.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:52:21.117 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-09-21T05:52:21.233 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T05:52:21.233 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T05:52:21.245 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T05:52:21.245 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T05:52:21.245 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:52:21.249 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T05:52:21.249 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:52:21.285 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T05:52:21.285 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: Command line: ntpd -gq 2024-09-21T05:52:21.286 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: ---------------------------------------------------- 2024-09-21T05:52:21.286 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T05:52:21.286 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T05:52:21.286 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: corporation. Support and training for ntp-4 are 2024-09-21T05:52:21.286 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: available at https://www.nwtime.org/support 2024-09-21T05:52:21.286 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: ---------------------------------------------------- 2024-09-21T05:52:21.287 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: proto: precision = 0.044 usec (-24) 2024-09-21T05:52:21.287 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: basedate set to 2022-02-04 2024-09-21T05:52:21.287 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: gps base set to 2022-02-06 (week 2196) 2024-09-21T05:52:21.287 INFO:teuthology.orchestra.run.smithi050.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T05:52:21.287 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T05:52:21.287 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T05:52:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T05:52:21.288 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T05:52:21.289 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T05:52:21.289 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T05:52:21.289 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: Listen normally on 3 ens1f0 172.21.15.50:123 2024-09-21T05:52:21.289 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: Listen normally on 4 lo [::1]:123 2024-09-21T05:52:21.289 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1584%4]:123 2024-09-21T05:52:21.289 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:21 ntpd[18376]: Listening on routing socket on fd #22 for interface updates 2024-09-21T05:52:21.315 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T05:52:21.316 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: Command line: ntpd -gq 2024-09-21T05:52:21.316 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: ---------------------------------------------------- 2024-09-21T05:52:21.316 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T05:52:21.316 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T05:52:21.316 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: corporation. Support and training for ntp-4 are 2024-09-21T05:52:21.316 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: available at https://www.nwtime.org/support 2024-09-21T05:52:21.316 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: ---------------------------------------------------- 2024-09-21T05:52:21.316 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: proto: precision = 0.042 usec (-24) 2024-09-21T05:52:21.317 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: basedate set to 2022-02-04 2024-09-21T05:52:21.317 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: gps base set to 2022-02-06 (week 2196) 2024-09-21T05:52:21.317 INFO:teuthology.orchestra.run.smithi088.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T05:52:21.318 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T05:52:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T05:52:21.318 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T05:52:21.319 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T05:52:21.319 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T05:52:21.320 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T05:52:21.320 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: Listen normally on 3 enp3s0f1 172.21.15.88:123 2024-09-21T05:52:21.320 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: Listen normally on 4 lo [::1]:123 2024-09-21T05:52:21.320 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d42d%5]:123 2024-09-21T05:52:21.320 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:21 ntpd[18488]: Listening on routing socket on fd #22 for interface updates 2024-09-21T05:52:22.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:22 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:22.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:22 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:22 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:22 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:22 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:22 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:23.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:23 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:23.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:23 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:23.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:23 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:23.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:23 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:23.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:23 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:23.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:23 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:23.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:23 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:23.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:23 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:24.288 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:24.288 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:24.288 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:24.288 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:24.288 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:24.288 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:24.288 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:24.288 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:24.288 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:24 ntpd[18376]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:24 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:24.356 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:24 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:24.356 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:24 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:24.356 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:24 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:24.356 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:24 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:24.356 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:24 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:25.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:25 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:25.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:25 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:25 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:25 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:25 ntpd[18376]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:25 ntpd[18376]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:25.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:25 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:25.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:25 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:25.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:25 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:25.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:25 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:25.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:25 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:25.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:25 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:25.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:25 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:25.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:25 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:25.318 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:25 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:25.318 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:25 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:25.318 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:25 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:25.318 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:25 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:26.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:26 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:26.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:26 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:26.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:26 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:26.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:26 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:26.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:26 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:26.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:26 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:26.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:26 ntpd[18376]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:26.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:26 ntpd[18376]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:26.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:26 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:26.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:26 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:26.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:26 ntpd[18376]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:26.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:26 ntpd[18376]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:26.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:26 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:26.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:26 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:26.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:26 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:26.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:26 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:26.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:26 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:26.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:26 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:27.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:27 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:27.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:27 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:27.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:27 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:27.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:27 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:27.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:27 ntpd[18376]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:27.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:27 ntpd[18376]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:27.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:27 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:27.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:27 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:27 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:27 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:27 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:27 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:27 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:27 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:27 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:27 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:27 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:27 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:28.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:28 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:28.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:28 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:28.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:28 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:28.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:28 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:28.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:28 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:28.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:28 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:28.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:28 ntpd[18376]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:28.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:28 ntpd[18376]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:28.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:28 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:28.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:28 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:28.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:28 ntpd[18376]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:28.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:28 ntpd[18376]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:28 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:28 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:28 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:28 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:28 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:28 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:28.317 INFO:teuthology.orchestra.run.smithi088.stdout:21 Sep 05:52:28 ntpd[18488]: ntpd: time slew -0.000780 s 2024-09-21T05:52:28.317 INFO:teuthology.orchestra.run.smithi088.stdout:ntpd: time slew -0.000780s 2024-09-21T05:52:28.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:28 ntpd[18488]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T05:52:28.317 INFO:teuthology.orchestra.run.smithi088.stderr:21 Sep 05:52:28 ntpd[18488]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T05:52:28.413 INFO:teuthology.orchestra.run.smithi088.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T05:52:28.413 INFO:teuthology.orchestra.run.smithi088.stdout:============================================================================== 2024-09-21T05:52:28.413 INFO:teuthology.orchestra.run.smithi088.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T05:52:28.413 INFO:teuthology.orchestra.run.smithi088.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T05:52:28.413 INFO:teuthology.orchestra.run.smithi088.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T05:52:28.414 INFO:teuthology.orchestra.run.smithi088.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T05:52:29.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:29 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:29.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:29 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:29.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:29 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:29.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:29 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:29.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:29 ntpd[18376]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:29.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:29 ntpd[18376]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:30.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:30 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:30.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:30 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:30.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:30 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:30.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:30 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:30.286 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:30 ntpd[18376]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T05:52:30.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:30 ntpd[18376]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T05:52:30.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:30 ntpd[18376]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T05:52:30.287 INFO:teuthology.orchestra.run.smithi050.stdout:21 Sep 05:52:30 ntpd[18376]: ntpd: time slew -0.002244 s 2024-09-21T05:52:30.287 INFO:teuthology.orchestra.run.smithi050.stdout:ntpd: time slew -0.002244s 2024-09-21T05:52:30.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:30 ntpd[18376]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T05:52:30.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:30 ntpd[18376]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T05:52:30.287 INFO:teuthology.orchestra.run.smithi050.stderr:21 Sep 05:52:30 ntpd[18376]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T05:52:30.380 INFO:teuthology.orchestra.run.smithi050.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T05:52:30.380 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================== 2024-09-21T05:52:30.380 INFO:teuthology.orchestra.run.smithi050.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T05:52:30.380 INFO:teuthology.orchestra.run.smithi050.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T05:52:30.381 INFO:teuthology.orchestra.run.smithi050.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T05:52:30.381 INFO:teuthology.orchestra.run.smithi050.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T05:52:30.381 INFO:teuthology.run_tasks:Running task install... 2024-09-21T05:52:30.389 DEBUG:teuthology.task.install:project ceph 2024-09-21T05:52:30.389 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T05:52:30.389 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T05:52:30.389 INFO:teuthology.task.install:Using flavor: default 2024-09-21T05:52:30.395 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T05:52:30.395 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T05:52:30.395 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-key list | grep Ceph 2024-09-21T05:52:30.396 DEBUG:teuthology.orchestra.run.smithi088:> sudo apt-key list | grep Ceph 2024-09-21T05:52:30.500 INFO:teuthology.orchestra.run.smithi088.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T05:52:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T05:52:30.551 INFO:teuthology.orchestra.run.smithi088.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T05:52:30.551 INFO:teuthology.orchestra.run.smithi088.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T05:52:30.552 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-21T05:52:30.552 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-21T05:52:30.552 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:52:30.585 INFO:teuthology.orchestra.run.smithi050.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T05:52:30.585 INFO:teuthology.orchestra.run.smithi050.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T05:52:30.586 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-21T05:52:30.586 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-21T05:52:30.586 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:52:30.689 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default/ 2024-09-21T05:52:30.689 INFO:teuthology.task.install.deb:Package version is 19.3.0-5120-g18432218-1jammy 2024-09-21T05:52:30.725 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default/ 2024-09-21T05:52:30.725 INFO:teuthology.task.install.deb:Package version is 19.3.0-5120-g18432218-1jammy 2024-09-21T05:52:30.797 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:52:30.797 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T05:52:30.816 DEBUG:teuthology.orchestra.run.smithi088:> sudo apt-get update 2024-09-21T05:52:30.834 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:52:30.834 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T05:52:30.854 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-get update 2024-09-21T05:52:30.998 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T05:52:31.005 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T05:52:31.022 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T05:52:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T05:52:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T05:52:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T05:52:31.048 INFO:teuthology.orchestra.run.smithi088.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T05:52:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T05:52:31.069 INFO:teuthology.orchestra.run.smithi088.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-21T05:52:31.090 INFO:teuthology.orchestra.run.smithi050.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T05:52:31.091 INFO:teuthology.orchestra.run.smithi088.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T05:52:31.112 INFO:teuthology.orchestra.run.smithi050.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-21T05:52:31.112 INFO:teuthology.orchestra.run.smithi088.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,432 B] 2024-09-21T05:52:31.134 INFO:teuthology.orchestra.run.smithi050.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T05:52:31.136 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T05:52:31.145 INFO:teuthology.orchestra.run.smithi088.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-21T05:52:31.155 INFO:teuthology.orchestra.run.smithi050.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-21T05:52:31.187 INFO:teuthology.orchestra.run.smithi050.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,432 B] 2024-09-21T05:52:31.597 INFO:teuthology.orchestra.run.smithi088.stdout:Fetched 28.6 kB in 1s (51.6 kB/s) 2024-09-21T05:52:31.658 INFO:teuthology.orchestra.run.smithi050.stdout:Fetched 28.6 kB in 1s (50.2 kB/s) 2024-09-21T05:52:32.648 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2024-09-21T05:52:32.663 DEBUG:teuthology.orchestra.run.smithi088:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-5120-g18432218-1jammy cephadm=19.3.0-5120-g18432218-1jammy ceph-mds=19.3.0-5120-g18432218-1jammy ceph-mgr=19.3.0-5120-g18432218-1jammy ceph-common=19.3.0-5120-g18432218-1jammy ceph-fuse=19.3.0-5120-g18432218-1jammy ceph-test=19.3.0-5120-g18432218-1jammy ceph-volume=19.3.0-5120-g18432218-1jammy radosgw=19.3.0-5120-g18432218-1jammy python3-rados=19.3.0-5120-g18432218-1jammy python3-rgw=19.3.0-5120-g18432218-1jammy python3-cephfs=19.3.0-5120-g18432218-1jammy python3-rbd=19.3.0-5120-g18432218-1jammy libcephfs2=19.3.0-5120-g18432218-1jammy libcephfs-dev=19.3.0-5120-g18432218-1jammy librados2=19.3.0-5120-g18432218-1jammy librbd1=19.3.0-5120-g18432218-1jammy rbd-fuse=19.3.0-5120-g18432218-1jammy python3-cephfs=19.3.0-5120-g18432218-1jammy cephfs-shell=19.3.0-5120-g18432218-1jammy cephfs-top=19.3.0-5120-g18432218-1jammy cephfs-mirror=19.3.0-5120-g18432218-1jammy 2024-09-21T05:52:32.739 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2024-09-21T05:52:32.742 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2024-09-21T05:52:32.757 DEBUG:teuthology.orchestra.run.smithi050:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-5120-g18432218-1jammy cephadm=19.3.0-5120-g18432218-1jammy ceph-mds=19.3.0-5120-g18432218-1jammy ceph-mgr=19.3.0-5120-g18432218-1jammy ceph-common=19.3.0-5120-g18432218-1jammy ceph-fuse=19.3.0-5120-g18432218-1jammy ceph-test=19.3.0-5120-g18432218-1jammy ceph-volume=19.3.0-5120-g18432218-1jammy radosgw=19.3.0-5120-g18432218-1jammy python3-rados=19.3.0-5120-g18432218-1jammy python3-rgw=19.3.0-5120-g18432218-1jammy python3-cephfs=19.3.0-5120-g18432218-1jammy python3-rbd=19.3.0-5120-g18432218-1jammy libcephfs2=19.3.0-5120-g18432218-1jammy libcephfs-dev=19.3.0-5120-g18432218-1jammy librados2=19.3.0-5120-g18432218-1jammy librbd1=19.3.0-5120-g18432218-1jammy rbd-fuse=19.3.0-5120-g18432218-1jammy python3-cephfs=19.3.0-5120-g18432218-1jammy cephfs-shell=19.3.0-5120-g18432218-1jammy cephfs-top=19.3.0-5120-g18432218-1jammy cephfs-mirror=19.3.0-5120-g18432218-1jammy 2024-09-21T05:52:32.837 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2024-09-21T05:52:32.927 INFO:teuthology.orchestra.run.smithi088.stdout:Building dependency tree... 2024-09-21T05:52:32.928 INFO:teuthology.orchestra.run.smithi088.stdout:Reading state information... 2024-09-21T05:52:33.030 INFO:teuthology.orchestra.run.smithi050.stdout:Building dependency tree... 2024-09-21T05:52:33.031 INFO:teuthology.orchestra.run.smithi050.stdout:Reading state information... 2024-09-21T05:52:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T05:52:33.116 INFO:teuthology.orchestra.run.smithi088.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T05:52:33.116 INFO:teuthology.orchestra.run.smithi088.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T05:52:33.116 INFO:teuthology.orchestra.run.smithi088.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T05:52:33.116 INFO:teuthology.orchestra.run.smithi088.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T05:52:33.117 INFO:teuthology.orchestra.run.smithi088.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T05:52:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T05:52:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:The following additional packages will be installed: 2024-09-21T05:52:33.118 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T05:52:33.118 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T05:52:33.118 INFO:teuthology.orchestra.run.smithi088.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-21T05:52:33.118 INFO:teuthology.orchestra.run.smithi088.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-21T05:52:33.118 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-21T05:52:33.119 INFO:teuthology.orchestra.run.smithi088.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-21T05:52:33.119 INFO:teuthology.orchestra.run.smithi088.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-21T05:52:33.119 INFO:teuthology.orchestra.run.smithi088.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T05:52:33.119 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-21T05:52:33.119 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-21T05:52:33.120 INFO:teuthology.orchestra.run.smithi088.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-21T05:52:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:Suggested packages: 2024-09-21T05:52:33.121 INFO:teuthology.orchestra.run.smithi088.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T05:52:33.121 INFO:teuthology.orchestra.run.smithi088.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T05:52:33.121 INFO:teuthology.orchestra.run.smithi088.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T05:52:33.121 INFO:teuthology.orchestra.run.smithi088.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T05:52:33.121 INFO:teuthology.orchestra.run.smithi088.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T05:52:33.121 INFO:teuthology.orchestra.run.smithi088.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T05:52:33.121 INFO:teuthology.orchestra.run.smithi088.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T05:52:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:Recommended packages: 2024-09-21T05:52:33.121 INFO:teuthology.orchestra.run.smithi088.stdout: btrfs-tools 2024-09-21T05:52:33.231 INFO:teuthology.orchestra.run.smithi050.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T05:52:33.231 INFO:teuthology.orchestra.run.smithi050.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T05:52:33.232 INFO:teuthology.orchestra.run.smithi050.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T05:52:33.232 INFO:teuthology.orchestra.run.smithi050.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T05:52:33.232 INFO:teuthology.orchestra.run.smithi050.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T05:52:33.233 INFO:teuthology.orchestra.run.smithi050.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T05:52:33.233 INFO:teuthology.orchestra.run.smithi050.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T05:52:33.233 INFO:teuthology.orchestra.run.smithi050.stdout:The following additional packages will be installed: 2024-09-21T05:52:33.233 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T05:52:33.234 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T05:52:33.234 INFO:teuthology.orchestra.run.smithi050.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-21T05:52:33.234 INFO:teuthology.orchestra.run.smithi050.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-21T05:52:33.234 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-21T05:52:33.235 INFO:teuthology.orchestra.run.smithi050.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-21T05:52:33.236 INFO:teuthology.orchestra.run.smithi050.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-21T05:52:33.237 INFO:teuthology.orchestra.run.smithi050.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-21T05:52:33.237 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-21T05:52:33.237 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-21T05:52:33.237 INFO:teuthology.orchestra.run.smithi050.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-21T05:52:33.237 INFO:teuthology.orchestra.run.smithi050.stdout:Suggested packages: 2024-09-21T05:52:33.237 INFO:teuthology.orchestra.run.smithi050.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T05:52:33.237 INFO:teuthology.orchestra.run.smithi050.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T05:52:33.237 INFO:teuthology.orchestra.run.smithi050.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T05:52:33.237 INFO:teuthology.orchestra.run.smithi050.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T05:52:33.237 INFO:teuthology.orchestra.run.smithi050.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T05:52:33.238 INFO:teuthology.orchestra.run.smithi050.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T05:52:33.238 INFO:teuthology.orchestra.run.smithi050.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T05:52:33.238 INFO:teuthology.orchestra.run.smithi050.stdout:Recommended packages: 2024-09-21T05:52:33.238 INFO:teuthology.orchestra.run.smithi050.stdout: btrfs-tools 2024-09-21T05:52:33.274 INFO:teuthology.orchestra.run.smithi088.stdout:The following NEW packages will be installed: 2024-09-21T05:52:33.274 INFO:teuthology.orchestra.run.smithi088.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T05:52:33.274 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T05:52:33.274 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-21T05:52:33.274 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-21T05:52:33.276 INFO:teuthology.orchestra.run.smithi088.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-21T05:52:33.276 INFO:teuthology.orchestra.run.smithi088.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-21T05:52:33.276 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-21T05:52:33.276 INFO:teuthology.orchestra.run.smithi088.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-21T05:52:33.276 INFO:teuthology.orchestra.run.smithi088.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-21T05:52:33.276 INFO:teuthology.orchestra.run.smithi088.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T05:52:33.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-21T05:52:33.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-21T05:52:33.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-21T05:52:33.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-21T05:52:33.277 INFO:teuthology.orchestra.run.smithi088.stdout: socat unzip xmlstarlet zip 2024-09-21T05:52:33.366 INFO:teuthology.orchestra.run.smithi088.stdout:0 upgraded, 125 newly installed, 0 to remove and 335 not upgraded. 2024-09-21T05:52:33.366 INFO:teuthology.orchestra.run.smithi088.stdout:Need to get 239 MB of archives. 2024-09-21T05:52:33.367 INFO:teuthology.orchestra.run.smithi088.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-21T05:52:33.367 INFO:teuthology.orchestra.run.smithi088.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-21T05:52:33.385 INFO:teuthology.orchestra.run.smithi088.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T05:52:33.395 INFO:teuthology.orchestra.run.smithi088.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-21T05:52:33.398 INFO:teuthology.orchestra.run.smithi050.stdout:The following NEW packages will be installed: 2024-09-21T05:52:33.398 INFO:teuthology.orchestra.run.smithi050.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T05:52:33.398 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T05:52:33.398 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-21T05:52:33.401 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-21T05:52:33.401 INFO:teuthology.orchestra.run.smithi050.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-21T05:52:33.401 INFO:teuthology.orchestra.run.smithi050.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-21T05:52:33.401 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-21T05:52:33.401 INFO:teuthology.orchestra.run.smithi050.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-21T05:52:33.401 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-21T05:52:33.401 INFO:teuthology.orchestra.run.smithi050.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T05:52:33.401 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-21T05:52:33.402 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-21T05:52:33.402 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-21T05:52:33.402 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T05:52:33.402 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T05:52:33.402 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T05:52:33.402 INFO:teuthology.orchestra.run.smithi050.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T05:52:33.402 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T05:52:33.402 INFO:teuthology.orchestra.run.smithi050.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T05:52:33.402 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-21T05:52:33.403 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-21T05:52:33.403 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-21T05:52:33.403 INFO:teuthology.orchestra.run.smithi050.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T05:52:33.403 INFO:teuthology.orchestra.run.smithi050.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T05:52:33.403 INFO:teuthology.orchestra.run.smithi050.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T05:52:33.403 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T05:52:33.403 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-21T05:52:33.403 INFO:teuthology.orchestra.run.smithi050.stdout: socat unzip xmlstarlet zip 2024-09-21T05:52:33.426 INFO:teuthology.orchestra.run.smithi088.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-5120-g18432218-1jammy [3,591 kB] 2024-09-21T05:52:33.437 INFO:teuthology.orchestra.run.smithi088.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T05:52:33.459 INFO:teuthology.orchestra.run.smithi088.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T05:52:33.462 INFO:teuthology.orchestra.run.smithi088.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T05:52:33.472 INFO:teuthology.orchestra.run.smithi088.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T05:52:33.511 INFO:teuthology.orchestra.run.smithi088.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T05:52:33.513 INFO:teuthology.orchestra.run.smithi088.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T05:52:33.520 INFO:teuthology.orchestra.run.smithi088.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T05:52:33.523 INFO:teuthology.orchestra.run.smithi088.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-21T05:52:33.524 INFO:teuthology.orchestra.run.smithi088.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T05:52:33.524 INFO:teuthology.orchestra.run.smithi088.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T05:52:33.530 INFO:teuthology.orchestra.run.smithi088.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T05:52:33.532 INFO:teuthology.orchestra.run.smithi088.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T05:52:33.534 INFO:teuthology.orchestra.run.smithi088.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T05:52:33.535 INFO:teuthology.orchestra.run.smithi088.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T05:52:33.548 INFO:teuthology.orchestra.run.smithi088.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T05:52:33.550 INFO:teuthology.orchestra.run.smithi088.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T05:52:33.552 INFO:teuthology.orchestra.run.smithi088.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T05:52:33.553 INFO:teuthology.orchestra.run.smithi088.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T05:52:33.554 INFO:teuthology.orchestra.run.smithi088.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T05:52:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:0 upgraded, 125 newly installed, 0 to remove and 335 not upgraded. 2024-09-21T05:52:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:Need to get 239 MB of archives. 2024-09-21T05:52:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-21T05:52:33.563 INFO:teuthology.orchestra.run.smithi050.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-5120-g18432218-1jammy [3,591 kB] 2024-09-21T05:52:33.566 INFO:teuthology.orchestra.run.smithi088.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T05:52:33.567 INFO:teuthology.orchestra.run.smithi088.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T05:52:33.567 INFO:teuthology.orchestra.run.smithi088.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T05:52:33.568 INFO:teuthology.orchestra.run.smithi088.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T05:52:33.569 INFO:teuthology.orchestra.run.smithi088.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T05:52:33.584 INFO:teuthology.orchestra.run.smithi088.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T05:52:33.586 INFO:teuthology.orchestra.run.smithi088.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T05:52:33.586 INFO:teuthology.orchestra.run.smithi088.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T05:52:33.589 INFO:teuthology.orchestra.run.smithi088.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T05:52:33.590 INFO:teuthology.orchestra.run.smithi088.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T05:52:33.597 INFO:teuthology.orchestra.run.smithi088.stdout:Get:34 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-5120-g18432218-1jammy [3,370 kB] 2024-09-21T05:52:33.599 INFO:teuthology.orchestra.run.smithi050.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-21T05:52:33.601 INFO:teuthology.orchestra.run.smithi088.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T05:52:33.601 INFO:teuthology.orchestra.run.smithi088.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T05:52:33.602 INFO:teuthology.orchestra.run.smithi088.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T05:52:33.602 INFO:teuthology.orchestra.run.smithi088.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T05:52:33.603 INFO:teuthology.orchestra.run.smithi088.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T05:52:33.619 INFO:teuthology.orchestra.run.smithi088.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T05:52:33.624 INFO:teuthology.orchestra.run.smithi088.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T05:52:33.624 INFO:teuthology.orchestra.run.smithi088.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T05:52:33.632 INFO:teuthology.orchestra.run.smithi088.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T05:52:33.633 INFO:teuthology.orchestra.run.smithi088.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T05:52:33.637 INFO:teuthology.orchestra.run.smithi088.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T05:52:33.638 INFO:teuthology.orchestra.run.smithi088.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T05:52:33.639 INFO:teuthology.orchestra.run.smithi088.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T05:52:33.641 INFO:teuthology.orchestra.run.smithi088.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T05:52:33.642 INFO:teuthology.orchestra.run.smithi088.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T05:52:33.648 INFO:teuthology.orchestra.run.smithi088.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-5120-g18432218-1jammy [733 kB] 2024-09-21T05:52:33.654 INFO:teuthology.orchestra.run.smithi088.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T05:52:33.655 INFO:teuthology.orchestra.run.smithi088.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T05:52:33.659 INFO:teuthology.orchestra.run.smithi088.stdout:Get:53 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-5120-g18432218-1jammy [356 kB] 2024-09-21T05:52:33.660 INFO:teuthology.orchestra.run.smithi088.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T05:52:33.661 INFO:teuthology.orchestra.run.smithi088.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T05:52:33.662 INFO:teuthology.orchestra.run.smithi088.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T05:52:33.665 INFO:teuthology.orchestra.run.smithi088.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-5120-g18432218-1jammy [32.8 kB] 2024-09-21T05:52:33.665 INFO:teuthology.orchestra.run.smithi088.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-5120-g18432218-1jammy [180 kB] 2024-09-21T05:52:33.668 INFO:teuthology.orchestra.run.smithi088.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-5120-g18432218-1jammy [72.9 kB] 2024-09-21T05:52:33.669 INFO:teuthology.orchestra.run.smithi088.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-5120-g18432218-1jammy [340 kB] 2024-09-21T05:52:33.672 INFO:teuthology.orchestra.run.smithi088.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T05:52:33.677 INFO:teuthology.orchestra.run.smithi088.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-5120-g18432218-1jammy [8,845 kB] 2024-09-21T05:52:33.681 INFO:teuthology.orchestra.run.smithi050.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T05:52:33.717 INFO:teuthology.orchestra.run.smithi088.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T05:52:33.718 INFO:teuthology.orchestra.run.smithi088.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T05:52:33.719 INFO:teuthology.orchestra.run.smithi088.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T05:52:33.720 INFO:teuthology.orchestra.run.smithi088.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T05:52:33.721 INFO:teuthology.orchestra.run.smithi088.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T05:52:33.726 INFO:teuthology.orchestra.run.smithi050.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-21T05:52:33.740 INFO:teuthology.orchestra.run.smithi088.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T05:52:33.742 INFO:teuthology.orchestra.run.smithi088.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T05:52:33.743 INFO:teuthology.orchestra.run.smithi088.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T05:52:33.760 INFO:teuthology.orchestra.run.smithi088.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T05:52:33.760 INFO:teuthology.orchestra.run.smithi088.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T05:52:33.761 INFO:teuthology.orchestra.run.smithi088.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T05:52:33.761 INFO:teuthology.orchestra.run.smithi088.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T05:52:33.762 INFO:teuthology.orchestra.run.smithi088.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T05:52:33.762 INFO:teuthology.orchestra.run.smithi088.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T05:52:33.771 INFO:teuthology.orchestra.run.smithi088.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T05:52:33.772 INFO:teuthology.orchestra.run.smithi088.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T05:52:33.776 INFO:teuthology.orchestra.run.smithi088.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T05:52:33.788 INFO:teuthology.orchestra.run.smithi088.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T05:52:33.790 INFO:teuthology.orchestra.run.smithi088.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T05:52:33.791 INFO:teuthology.orchestra.run.smithi088.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T05:52:33.794 INFO:teuthology.orchestra.run.smithi088.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T05:52:33.797 INFO:teuthology.orchestra.run.smithi088.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-21T05:52:33.797 INFO:teuthology.orchestra.run.smithi088.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T05:52:33.798 INFO:teuthology.orchestra.run.smithi088.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T05:52:33.806 INFO:teuthology.orchestra.run.smithi088.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T05:52:33.811 INFO:teuthology.orchestra.run.smithi088.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T05:52:33.812 INFO:teuthology.orchestra.run.smithi088.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T05:52:33.824 INFO:teuthology.orchestra.run.smithi088.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T05:52:33.824 INFO:teuthology.orchestra.run.smithi088.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T05:52:33.825 INFO:teuthology.orchestra.run.smithi088.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T05:52:33.825 INFO:teuthology.orchestra.run.smithi088.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T05:52:33.827 INFO:teuthology.orchestra.run.smithi088.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T05:52:33.828 INFO:teuthology.orchestra.run.smithi088.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T05:52:33.835 INFO:teuthology.orchestra.run.smithi088.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T05:52:33.836 INFO:teuthology.orchestra.run.smithi088.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-5120-g18432218-1jammy [112 kB] 2024-09-21T05:52:33.837 INFO:teuthology.orchestra.run.smithi088.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-5120-g18432218-1jammy [478 kB] 2024-09-21T05:52:33.841 INFO:teuthology.orchestra.run.smithi088.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T05:52:33.842 INFO:teuthology.orchestra.run.smithi088.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T05:52:33.843 INFO:teuthology.orchestra.run.smithi088.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-5120-g18432218-1jammy [29.1 MB] 2024-09-21T05:52:33.844 INFO:teuthology.orchestra.run.smithi088.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T05:52:33.859 INFO:teuthology.orchestra.run.smithi088.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T05:52:33.922 INFO:teuthology.orchestra.run.smithi050.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T05:52:34.014 INFO:teuthology.orchestra.run.smithi050.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T05:52:34.026 INFO:teuthology.orchestra.run.smithi050.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T05:52:34.077 INFO:teuthology.orchestra.run.smithi050.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T05:52:34.249 INFO:teuthology.orchestra.run.smithi050.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T05:52:34.258 INFO:teuthology.orchestra.run.smithi050.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T05:52:34.285 INFO:teuthology.orchestra.run.smithi050.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T05:52:34.293 INFO:teuthology.orchestra.run.smithi050.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-21T05:52:34.296 INFO:teuthology.orchestra.run.smithi050.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T05:52:34.297 INFO:teuthology.orchestra.run.smithi050.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T05:52:34.314 INFO:teuthology.orchestra.run.smithi050.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T05:52:34.319 INFO:teuthology.orchestra.run.smithi050.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T05:52:34.322 INFO:teuthology.orchestra.run.smithi050.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T05:52:34.330 INFO:teuthology.orchestra.run.smithi050.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T05:52:34.370 INFO:teuthology.orchestra.run.smithi088.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-5120-g18432218-1jammy [5,023 kB] 2024-09-21T05:52:34.384 INFO:teuthology.orchestra.run.smithi050.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-5120-g18432218-1jammy [3,370 kB] 2024-09-21T05:52:34.407 INFO:teuthology.orchestra.run.smithi050.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T05:52:34.408 INFO:teuthology.orchestra.run.smithi050.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T05:52:34.410 INFO:teuthology.orchestra.run.smithi050.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T05:52:34.412 INFO:teuthology.orchestra.run.smithi050.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T05:52:34.412 INFO:teuthology.orchestra.run.smithi050.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T05:52:34.413 INFO:teuthology.orchestra.run.smithi050.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T05:52:34.414 INFO:teuthology.orchestra.run.smithi050.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T05:52:34.414 INFO:teuthology.orchestra.run.smithi050.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T05:52:34.415 INFO:teuthology.orchestra.run.smithi050.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T05:52:34.455 INFO:teuthology.orchestra.run.smithi088.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-5120-g18432218-1jammy [252 kB] 2024-09-21T05:52:34.458 INFO:teuthology.orchestra.run.smithi088.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-5120-g18432218-1jammy [126 kB] 2024-09-21T05:52:34.464 INFO:teuthology.orchestra.run.smithi088.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-5120-g18432218-1jammy [1,552 kB] 2024-09-21T05:52:34.486 INFO:teuthology.orchestra.run.smithi088.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-5120-g18432218-1jammy [6,198 kB] 2024-09-21T05:52:34.493 INFO:teuthology.orchestra.run.smithi050.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T05:52:34.494 INFO:teuthology.orchestra.run.smithi050.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T05:52:34.494 INFO:teuthology.orchestra.run.smithi050.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T05:52:34.575 INFO:teuthology.orchestra.run.smithi050.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T05:52:34.577 INFO:teuthology.orchestra.run.smithi050.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T05:52:34.578 INFO:teuthology.orchestra.run.smithi050.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T05:52:34.578 INFO:teuthology.orchestra.run.smithi050.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T05:52:34.579 INFO:teuthology.orchestra.run.smithi050.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T05:52:34.580 INFO:teuthology.orchestra.run.smithi050.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T05:52:34.580 INFO:teuthology.orchestra.run.smithi050.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T05:52:34.607 INFO:teuthology.orchestra.run.smithi088.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-5120-g18432218-1jammy [22.9 MB] 2024-09-21T05:52:34.656 INFO:teuthology.orchestra.run.smithi050.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T05:52:34.657 INFO:teuthology.orchestra.run.smithi050.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T05:52:34.658 INFO:teuthology.orchestra.run.smithi050.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T05:52:34.738 INFO:teuthology.orchestra.run.smithi050.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T05:52:34.743 INFO:teuthology.orchestra.run.smithi050.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T05:52:34.743 INFO:teuthology.orchestra.run.smithi050.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T05:52:34.744 INFO:teuthology.orchestra.run.smithi050.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T05:52:34.745 INFO:teuthology.orchestra.run.smithi050.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T05:52:34.746 INFO:teuthology.orchestra.run.smithi050.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T05:52:34.748 INFO:teuthology.orchestra.run.smithi050.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T05:52:34.811 INFO:teuthology.orchestra.run.smithi050.stdout:Get:49 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-5120-g18432218-1jammy [733 kB] 2024-09-21T05:52:34.819 INFO:teuthology.orchestra.run.smithi050.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T05:52:34.821 INFO:teuthology.orchestra.run.smithi050.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T05:52:34.821 INFO:teuthology.orchestra.run.smithi050.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T05:52:34.901 INFO:teuthology.orchestra.run.smithi050.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T05:52:34.901 INFO:teuthology.orchestra.run.smithi050.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T05:52:34.902 INFO:teuthology.orchestra.run.smithi050.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T05:52:34.904 INFO:teuthology.orchestra.run.smithi050.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T05:52:34.915 INFO:teuthology.orchestra.run.smithi050.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-5120-g18432218-1jammy [356 kB] 2024-09-21T05:52:34.962 INFO:teuthology.orchestra.run.smithi050.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-5120-g18432218-1jammy [32.8 kB] 2024-09-21T05:52:34.966 INFO:teuthology.orchestra.run.smithi050.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-5120-g18432218-1jammy [180 kB] 2024-09-21T05:52:34.970 INFO:teuthology.orchestra.run.smithi050.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-5120-g18432218-1jammy [72.9 kB] 2024-09-21T05:52:34.971 INFO:teuthology.orchestra.run.smithi050.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-5120-g18432218-1jammy [340 kB] 2024-09-21T05:52:35.012 INFO:teuthology.orchestra.run.smithi050.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T05:52:35.013 INFO:teuthology.orchestra.run.smithi050.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T05:52:35.014 INFO:teuthology.orchestra.run.smithi050.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T05:52:35.015 INFO:teuthology.orchestra.run.smithi050.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T05:52:35.016 INFO:teuthology.orchestra.run.smithi050.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T05:52:35.023 INFO:teuthology.orchestra.run.smithi050.stdout:Get:67 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-5120-g18432218-1jammy [8,845 kB] 2024-09-21T05:52:35.050 INFO:teuthology.orchestra.run.smithi088.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-5120-g18432218-1jammy [14.0 kB] 2024-09-21T05:52:35.050 INFO:teuthology.orchestra.run.smithi088.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-5120-g18432218-1jammy [865 kB] 2024-09-21T05:52:35.058 INFO:teuthology.orchestra.run.smithi088.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-5120-g18432218-1jammy [2,390 kB] 2024-09-21T05:52:35.087 INFO:teuthology.orchestra.run.smithi050.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T05:52:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T05:52:35.090 INFO:teuthology.orchestra.run.smithi050.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T05:52:35.107 INFO:teuthology.orchestra.run.smithi088.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-5120-g18432218-1jammy [813 kB] 2024-09-21T05:52:35.119 INFO:teuthology.orchestra.run.smithi088.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-5120-g18432218-1jammy [160 kB] 2024-09-21T05:52:35.122 INFO:teuthology.orchestra.run.smithi088.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-5120-g18432218-1jammy [61.5 MB] 2024-09-21T05:52:35.160 INFO:teuthology.orchestra.run.smithi050.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T05:52:35.160 INFO:teuthology.orchestra.run.smithi050.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T05:52:35.161 INFO:teuthology.orchestra.run.smithi050.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T05:52:35.162 INFO:teuthology.orchestra.run.smithi050.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T05:52:35.162 INFO:teuthology.orchestra.run.smithi050.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T05:52:35.163 INFO:teuthology.orchestra.run.smithi050.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T05:52:35.228 INFO:teuthology.orchestra.run.smithi050.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T05:52:35.229 INFO:teuthology.orchestra.run.smithi050.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T05:52:35.233 INFO:teuthology.orchestra.run.smithi050.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T05:52:35.309 INFO:teuthology.orchestra.run.smithi050.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T05:52:35.311 INFO:teuthology.orchestra.run.smithi050.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T05:52:35.311 INFO:teuthology.orchestra.run.smithi050.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T05:52:35.315 INFO:teuthology.orchestra.run.smithi050.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T05:52:35.318 INFO:teuthology.orchestra.run.smithi050.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-21T05:52:35.319 INFO:teuthology.orchestra.run.smithi050.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T05:52:35.319 INFO:teuthology.orchestra.run.smithi050.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T05:52:35.391 INFO:teuthology.orchestra.run.smithi050.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T05:52:35.395 INFO:teuthology.orchestra.run.smithi050.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T05:52:35.396 INFO:teuthology.orchestra.run.smithi050.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T05:52:35.472 INFO:teuthology.orchestra.run.smithi050.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T05:52:35.473 INFO:teuthology.orchestra.run.smithi050.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T05:52:35.473 INFO:teuthology.orchestra.run.smithi050.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T05:52:35.473 INFO:teuthology.orchestra.run.smithi050.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T05:52:35.475 INFO:teuthology.orchestra.run.smithi050.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T05:52:35.476 INFO:teuthology.orchestra.run.smithi050.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T05:52:35.557 INFO:teuthology.orchestra.run.smithi050.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T05:52:35.557 INFO:teuthology.orchestra.run.smithi050.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T05:52:35.557 INFO:teuthology.orchestra.run.smithi050.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T05:52:35.559 INFO:teuthology.orchestra.run.smithi050.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T05:52:35.635 INFO:teuthology.orchestra.run.smithi050.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T05:52:35.685 INFO:teuthology.orchestra.run.smithi050.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-5120-g18432218-1jammy [112 kB] 2024-09-21T05:52:35.728 INFO:teuthology.orchestra.run.smithi050.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-5120-g18432218-1jammy [478 kB] 2024-09-21T05:52:35.736 INFO:teuthology.orchestra.run.smithi050.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-5120-g18432218-1jammy [29.1 MB] 2024-09-21T05:52:36.543 INFO:teuthology.orchestra.run.smithi088.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-5120-g18432218-1jammy [8,625 kB] 2024-09-21T05:52:36.833 INFO:teuthology.orchestra.run.smithi088.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-5120-g18432218-1jammy [14.2 kB] 2024-09-21T05:52:36.833 INFO:teuthology.orchestra.run.smithi088.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-5120-g18432218-1jammy [40.9 MB] 2024-09-21T05:52:36.970 INFO:teuthology.orchestra.run.smithi050.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-5120-g18432218-1jammy [5,023 kB] 2024-09-21T05:52:37.093 INFO:teuthology.orchestra.run.smithi050.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-5120-g18432218-1jammy [252 kB] 2024-09-21T05:52:37.095 INFO:teuthology.orchestra.run.smithi050.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-5120-g18432218-1jammy [126 kB] 2024-09-21T05:52:37.097 INFO:teuthology.orchestra.run.smithi050.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-5120-g18432218-1jammy [1,552 kB] 2024-09-21T05:52:37.130 INFO:teuthology.orchestra.run.smithi050.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-5120-g18432218-1jammy [6,198 kB] 2024-09-21T05:52:37.283 INFO:teuthology.orchestra.run.smithi050.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-5120-g18432218-1jammy [22.9 MB] 2024-09-21T05:52:37.878 INFO:teuthology.orchestra.run.smithi050.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-5120-g18432218-1jammy [14.0 kB] 2024-09-21T05:52:37.879 INFO:teuthology.orchestra.run.smithi050.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-5120-g18432218-1jammy [865 kB] 2024-09-21T05:52:37.905 INFO:teuthology.orchestra.run.smithi050.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-5120-g18432218-1jammy [2,390 kB] 2024-09-21T05:52:37.973 INFO:teuthology.orchestra.run.smithi050.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-5120-g18432218-1jammy [813 kB] 2024-09-21T05:52:37.988 INFO:teuthology.orchestra.run.smithi050.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-5120-g18432218-1jammy [160 kB] 2024-09-21T05:52:37.991 INFO:teuthology.orchestra.run.smithi050.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-5120-g18432218-1jammy [61.5 MB] 2024-09-21T05:52:38.582 INFO:teuthology.orchestra.run.smithi088.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-5120-g18432218-1jammy [130 kB] 2024-09-21T05:52:38.583 INFO:teuthology.orchestra.run.smithi088.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-5120-g18432218-1jammy [237 kB] 2024-09-21T05:52:38.585 INFO:teuthology.orchestra.run.smithi088.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-5120-g18432218-1jammy [34.0 kB] 2024-09-21T05:52:38.586 INFO:teuthology.orchestra.run.smithi088.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-5120-g18432218-1jammy [23.6 kB] 2024-09-21T05:52:38.586 INFO:teuthology.orchestra.run.smithi088.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-5120-g18432218-1jammy [39.0 kB] 2024-09-21T05:52:38.587 INFO:teuthology.orchestra.run.smithi088.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-5120-g18432218-1jammy [17.6 MB] 2024-09-21T05:52:39.381 INFO:teuthology.orchestra.run.smithi088.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-5120-g18432218-1jammy [93.1 kB] 2024-09-21T05:52:39.544 INFO:teuthology.orchestra.run.smithi050.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-5120-g18432218-1jammy [8,625 kB] 2024-09-21T05:52:39.668 INFO:teuthology.orchestra.run.smithi050.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-5120-g18432218-1jammy [14.2 kB] 2024-09-21T05:52:39.669 INFO:teuthology.orchestra.run.smithi050.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-5120-g18432218-1jammy [40.9 MB] 2024-09-21T05:52:40.031 INFO:teuthology.orchestra.run.smithi088.stdout:Fetched 239 MB in 6s (39.3 MB/s) 2024-09-21T05:52:40.181 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-21T05:52:40.245 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-21T05:52:40.248 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-21T05:52:40.296 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-21T05:52:40.351 INFO:teuthology.orchestra.run.smithi050.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-5120-g18432218-1jammy [130 kB] 2024-09-21T05:52:40.352 INFO:teuthology.orchestra.run.smithi050.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-5120-g18432218-1jammy [237 kB] 2024-09-21T05:52:40.354 INFO:teuthology.orchestra.run.smithi050.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-5120-g18432218-1jammy [34.0 kB] 2024-09-21T05:52:40.354 INFO:teuthology.orchestra.run.smithi050.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-5120-g18432218-1jammy [23.6 kB] 2024-09-21T05:52:40.355 INFO:teuthology.orchestra.run.smithi050.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-5120-g18432218-1jammy [39.0 kB] 2024-09-21T05:52:40.356 INFO:teuthology.orchestra.run.smithi050.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-5120-g18432218-1jammy [17.6 MB] 2024-09-21T05:52:40.590 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T05:52:40.603 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T05:52:40.632 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T05:52:40.639 INFO:teuthology.orchestra.run.smithi050.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-5120-g18432218-1jammy [93.1 kB] 2024-09-21T05:52:40.960 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cmd2. 2024-09-21T05:52:40.973 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-21T05:52:41.018 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-21T05:52:41.315 INFO:teuthology.orchestra.run.smithi050.stdout:Fetched 239 MB in 7s (33.1 MB/s) 2024-09-21T05:52:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T05:52:41.401 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T05:52:41.488 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T05:52:41.522 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-21T05:52:41.590 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-21T05:52:41.594 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-21T05:52:41.638 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-21T05:52:41.866 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T05:52:41.881 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T05:52:41.915 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T05:52:41.924 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T05:52:41.930 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T05:52:41.974 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T05:52:42.261 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T05:52:42.276 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T05:52:42.293 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cmd2. 2024-09-21T05:52:42.308 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-21T05:52:42.310 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T05:52:42.352 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-21T05:52:42.730 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T05:52:42.745 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T05:52:42.789 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T05:52:42.804 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T05:52:42.830 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T05:52:42.856 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T05:52:43.192 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T05:52:43.207 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T05:52:43.250 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T05:52:43.501 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T05:52:43.515 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T05:52:43.559 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T05:52:43.561 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T05:52:43.577 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T05:52:43.611 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T05:52:43.912 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T05:52:43.927 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T05:52:43.961 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T05:52:44.090 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T05:52:44.105 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T05:52:44.373 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T05:52:44.440 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T05:52:44.455 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T05:52:44.498 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T05:52:44.816 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T05:52:44.873 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package librados2. 2024-09-21T05:52:44.873 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../010-librados2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:44.873 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T05:52:44.873 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T05:52:44.885 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking librados2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:45.221 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T05:52:45.235 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T05:52:45.271 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T05:52:45.379 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libnbd0. 2024-09-21T05:52:45.394 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-21T05:52:45.437 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-21T05:52:45.632 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T05:52:45.646 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T05:52:45.690 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T05:52:45.756 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package librbd1. 2024-09-21T05:52:45.771 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../012-librbd1_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:45.814 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking librbd1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:46.035 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package librados2. 2024-09-21T05:52:46.049 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../010-librados2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:46.093 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librados2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:46.284 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T05:52:46.299 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:46.342 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libcephfs2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:46.571 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libnbd0. 2024-09-21T05:52:46.585 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-21T05:52:46.638 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-21T05:52:46.661 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-rados. 2024-09-21T05:52:46.676 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../014-python3-rados_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:46.719 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-rados (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package librbd1. 2024-09-21T05:52:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../012-librbd1_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librbd1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:47.039 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T05:52:47.054 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:52:47.097 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-ceph-argparse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:47.333 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T05:52:47.348 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:47.385 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T05:52:47.391 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cephfs (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:47.399 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:47.443 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libcephfs2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:47.736 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T05:52:47.746 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-rados. 2024-09-21T05:52:47.751 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:52:47.759 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../014-python3-rados_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:47.794 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:47.804 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-rados (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:48.123 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T05:52:48.137 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:52:48.181 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-ceph-argparse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:48.189 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T05:52:48.204 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T05:52:48.247 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T05:52:48.393 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T05:52:48.406 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:48.467 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cephfs (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:48.567 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T05:52:48.582 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:48.634 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-rbd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T05:52:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:52:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:48.952 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T05:52:48.967 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T05:52:49.002 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T05:52:49.259 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T05:52:49.272 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T05:52:49.297 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T05:52:49.312 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T05:52:49.315 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T05:52:49.354 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T05:52:49.618 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T05:52:49.631 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:49.685 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-rbd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:49.691 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T05:52:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T05:52:49.741 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T05:52:50.053 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package lua5.1. 2024-09-21T05:52:50.067 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T05:52:50.112 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T05:52:50.126 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T05:52:50.144 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T05:52:50.179 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T05:52:50.455 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package lua-any. 2024-09-21T05:52:50.457 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T05:52:50.470 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T05:52:50.470 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-21T05:52:50.513 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T05:52:50.515 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T05:52:50.791 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package zip. 2024-09-21T05:52:50.806 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-21T05:52:50.818 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T05:52:50.831 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T05:52:50.849 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T05:52:50.876 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T05:52:51.136 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package unzip. 2024-09-21T05:52:51.151 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T05:52:51.202 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T05:52:51.221 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package lua5.1. 2024-09-21T05:52:51.235 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T05:52:51.279 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T05:52:51.589 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package luarocks. 2024-09-21T05:52:51.590 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package lua-any. 2024-09-21T05:52:51.604 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-21T05:52:51.604 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T05:52:51.647 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T05:52:51.665 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T05:52:51.910 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package zip. 2024-09-21T05:52:51.923 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-21T05:52:51.976 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T05:52:52.063 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package librgw2. 2024-09-21T05:52:52.077 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../028-librgw2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:52.121 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking librgw2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:52.288 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package unzip. 2024-09-21T05:52:52.301 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T05:52:52.346 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T05:52:52.775 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package luarocks. 2024-09-21T05:52:52.788 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T05:52:52.799 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T05:52:52.814 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:52.833 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T05:52:52.866 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-rgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:53.219 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T05:52:53.233 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T05:52:53.257 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package librgw2. 2024-09-21T05:52:53.270 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../028-librgw2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:53.277 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T05:52:53.307 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librgw2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:53.571 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T05:52:53.586 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:53.621 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libradosstriper1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:53.927 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T05:52:53.940 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:53.977 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-rgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:54.233 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-common. 2024-09-21T05:52:54.248 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../032-ceph-common_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:54.291 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:54.330 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T05:52:54.343 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T05:52:54.388 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T05:52:54.882 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T05:52:54.892 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:54.932 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libradosstriper1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:55.344 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-common. 2024-09-21T05:52:55.355 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../032-ceph-common_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:55.385 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:55.713 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-base. 2024-09-21T05:52:55.727 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../033-ceph-base_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:55.780 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-base (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:56.383 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T05:52:56.397 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T05:52:56.433 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T05:52:56.757 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-base. 2024-09-21T05:52:56.761 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T05:52:56.769 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../033-ceph-base_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:52:56.775 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T05:52:56.817 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-base (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:52:56.819 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T05:52:57.147 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T05:52:57.161 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T05:52:57.231 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T05:52:57.429 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T05:52:57.437 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T05:52:57.478 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T05:52:57.483 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T05:52:57.497 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T05:52:57.533 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T05:52:57.781 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T05:52:57.790 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T05:52:57.831 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T05:52:57.836 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T05:52:57.850 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T05:52:57.894 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T05:52:58.159 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T05:52:58.169 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T05:52:58.180 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T05:52:58.194 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T05:52:58.209 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T05:52:58.263 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T05:52:58.528 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T05:52:58.541 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T05:52:58.566 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-portend. 2024-09-21T05:52:58.579 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T05:52:58.580 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-21T05:52:58.625 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T05:52:58.865 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T05:52:58.875 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T05:52:58.911 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T05:52:58.924 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T05:52:58.948 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T05:52:58.969 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T05:52:59.239 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T05:52:59.250 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T05:52:59.252 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T05:52:59.261 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T05:52:59.300 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T05:52:59.321 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T05:52:59.596 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-portend. 2024-09-21T05:52:59.606 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-21T05:52:59.637 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T05:52:59.683 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T05:52:59.698 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T05:52:59.741 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T05:52:59.965 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T05:52:59.975 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T05:53:00.014 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T05:53:00.086 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T05:53:00.100 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T05:53:00.144 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T05:53:00.301 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T05:53:00.311 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T05:53:00.350 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T05:53:00.455 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T05:53:00.469 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T05:53:00.513 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T05:53:00.721 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T05:53:00.731 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T05:53:00.770 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T05:53:00.800 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-mako. 2024-09-21T05:53:00.817 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T05:53:00.858 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T05:53:01.116 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T05:53:01.126 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T05:53:01.165 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T05:53:01.220 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T05:53:01.235 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T05:53:01.278 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T05:53:01.435 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T05:53:01.445 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T05:53:01.518 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T05:53:01.606 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T05:53:01.621 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T05:53:01.664 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T05:53:01.804 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-mako. 2024-09-21T05:53:01.815 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T05:53:01.854 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T05:53:01.975 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-webob. 2024-09-21T05:53:01.990 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T05:53:02.025 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T05:53:02.207 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T05:53:02.218 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T05:53:02.257 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T05:53:02.344 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T05:53:02.360 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T05:53:02.397 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T05:53:02.569 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T05:53:02.579 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T05:53:02.618 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T05:53:02.706 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T05:53:02.721 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T05:53:02.764 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T05:53:02.880 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-webob. 2024-09-21T05:53:02.890 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T05:53:02.929 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T05:53:03.100 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-paste. 2024-09-21T05:53:03.115 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T05:53:03.158 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T05:53:03.316 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T05:53:03.326 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T05:53:03.368 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T05:53:03.461 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T05:53:03.476 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T05:53:03.519 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T05:53:03.710 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T05:53:03.721 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T05:53:03.760 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T05:53:03.847 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T05:53:03.862 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T05:53:03.905 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T05:53:04.065 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-paste. 2024-09-21T05:53:04.076 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T05:53:04.115 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T05:53:04.227 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T05:53:04.238 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T05:53:04.435 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T05:53:04.445 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T05:53:04.484 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T05:53:04.787 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T05:53:04.798 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T05:53:04.837 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T05:53:05.044 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T05:53:05.257 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T05:53:05.268 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T05:53:05.298 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T05:53:05.405 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T05:53:05.415 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T05:53:05.455 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T05:53:05.626 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T05:53:05.637 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T05:53:05.676 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T05:53:05.851 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T05:53:05.865 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T05:53:05.909 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T05:53:06.040 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T05:53:06.050 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T05:53:06.123 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T05:53:06.263 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-idna. 2024-09-21T05:53:06.277 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T05:53:06.321 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T05:53:06.615 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T05:53:06.627 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T05:53:06.665 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T05:53:06.985 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-requests. 2024-09-21T05:53:06.997 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T05:53:07.026 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T05:53:07.168 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-idna. 2024-09-21T05:53:07.179 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T05:53:07.218 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T05:53:07.321 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T05:53:07.333 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T05:53:07.371 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T05:53:07.512 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T05:53:07.523 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T05:53:07.554 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T05:53:07.657 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T05:53:07.669 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:07.707 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-modules-core (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:07.899 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-requests. 2024-09-21T05:53:07.909 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T05:53:07.948 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T05:53:08.112 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T05:53:08.123 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:08.162 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:08.260 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T05:53:08.271 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T05:53:08.309 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T05:53:08.506 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T05:53:08.518 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T05:53:08.556 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T05:53:08.621 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T05:53:08.632 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:08.670 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-modules-core (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:08.834 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T05:53:08.846 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T05:53:08.903 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T05:53:09.034 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T05:53:09.044 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:09.076 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:09.420 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T05:53:09.431 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T05:53:09.470 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T05:53:09.529 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T05:53:09.541 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:09.578 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:09.765 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T05:53:09.775 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T05:53:09.814 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T05:53:10.007 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T05:53:10.019 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:10.056 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mon (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:10.143 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T05:53:10.153 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:10.192 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:10.579 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T05:53:10.590 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:10.627 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T05:53:10.628 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mon (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:10.638 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:10.668 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-osd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:11.224 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T05:53:11.235 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:11.274 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-osd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:11.547 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph. 2024-09-21T05:53:11.559 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../069-ceph_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:11.596 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:11.957 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T05:53:11.969 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:12.015 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:12.094 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph. 2024-09-21T05:53:12.104 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../069-ceph_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:12.135 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:12.421 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T05:53:12.432 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:12.469 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T05:53:12.471 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:12.481 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:12.526 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mds (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:12.874 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T05:53:12.885 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:12.924 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mds (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:12.996 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package cephadm. 2024-09-21T05:53:13.009 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../072-cephadm_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:13.054 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:13.344 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package cephadm. 2024-09-21T05:53:13.355 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../072-cephadm_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:13.393 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:13.457 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T05:53:13.470 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T05:53:13.515 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T05:53:13.680 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T05:53:13.691 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T05:53:13.721 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T05:53:13.885 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T05:53:13.897 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T05:53:13.943 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T05:53:14.041 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T05:53:14.051 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T05:53:14.116 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T05:53:14.998 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-babel. 2024-09-21T05:53:15.008 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T05:53:15.047 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T05:53:15.170 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-babel. 2024-09-21T05:53:15.180 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T05:53:15.220 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T05:53:15.409 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T05:53:15.418 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T05:53:15.459 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T05:53:15.748 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T05:53:15.758 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T05:53:15.762 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T05:53:15.771 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:15.798 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T05:53:15.811 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:16.140 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T05:53:16.150 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T05:53:16.168 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T05:53:16.178 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:16.181 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T05:53:16.218 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:16.488 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T05:53:16.498 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T05:53:16.526 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-routes. 2024-09-21T05:53:16.536 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T05:53:16.538 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T05:53:16.576 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T05:53:16.874 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-routes. 2024-09-21T05:53:16.885 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T05:53:16.912 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T05:53:16.923 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:16.923 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T05:53:16.962 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-dashboard (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:17.277 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T05:53:17.288 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:17.326 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-dashboard (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:19.458 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T05:53:19.469 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T05:53:19.499 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T05:53:19.774 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T05:53:19.785 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T05:53:19.831 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T05:53:20.028 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T05:53:20.038 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T05:53:20.163 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T05:53:20.427 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T05:53:20.437 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T05:53:20.509 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T05:53:20.519 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T05:53:20.559 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T05:53:20.594 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T05:53:20.921 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T05:53:20.933 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T05:53:20.970 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T05:53:20.983 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T05:53:20.994 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T05:53:21.033 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T05:53:21.361 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T05:53:21.372 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T05:53:21.411 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T05:53:21.662 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T05:53:21.673 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:21.720 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:22.278 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T05:53:22.289 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:22.328 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:22.807 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T05:53:22.821 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T05:53:22.873 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T05:53:23.234 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T05:53:23.248 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T05:53:23.293 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T05:53:23.373 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T05:53:23.385 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T05:53:23.415 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T05:53:23.688 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T05:53:23.700 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T05:53:23.701 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T05:53:23.712 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T05:53:23.745 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T05:53:23.776 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T05:53:24.096 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T05:53:24.107 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T05:53:24.124 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T05:53:24.137 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-21T05:53:24.145 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T05:53:24.190 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T05:53:24.491 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T05:53:24.502 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-21T05:53:24.541 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T05:53:24.777 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T05:53:24.790 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T05:53:24.835 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T05:53:24.869 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T05:53:24.880 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T05:53:24.918 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T05:53:25.247 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T05:53:25.247 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T05:53:25.258 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T05:53:25.260 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T05:53:25.305 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T05:53:25.322 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T05:53:25.567 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T05:53:25.578 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T05:53:25.608 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T05:53:25.683 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T05:53:25.696 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T05:53:25.733 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T05:53:26.187 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T05:53:26.198 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T05:53:26.203 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T05:53:26.216 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T05:53:26.236 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T05:53:26.269 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T05:53:26.934 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T05:53:26.945 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:26.974 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T05:53:26.983 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:26.986 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:27.033 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:27.303 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T05:53:27.313 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T05:53:27.352 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T05:53:27.365 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T05:53:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T05:53:27.427 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T05:53:27.664 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T05:53:27.675 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T05:53:27.706 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T05:53:27.730 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T05:53:27.745 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T05:53:27.772 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T05:53:27.967 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package jq. 2024-09-21T05:53:27.978 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T05:53:28.033 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T05:53:28.049 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package jq. 2024-09-21T05:53:28.062 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T05:53:28.099 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T05:53:28.286 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package socat. 2024-09-21T05:53:28.297 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T05:53:28.336 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T05:53:28.352 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package socat. 2024-09-21T05:53:28.365 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T05:53:28.435 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T05:53:28.664 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T05:53:28.676 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T05:53:28.714 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T05:53:28.738 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T05:53:28.750 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T05:53:28.788 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T05:53:29.118 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-test. 2024-09-21T05:53:29.129 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../100-ceph-test_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:29.168 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-test (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:29.192 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-test. 2024-09-21T05:53:29.204 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../100-ceph-test_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:29.242 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-test (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:31.097 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T05:53:31.109 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:31.139 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-volume (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:31.221 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T05:53:31.234 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:31.279 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-volume (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:31.510 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-21T05:53:31.522 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:31.551 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking cephfs-mirror (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:31.700 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-21T05:53:31.713 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:31.758 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking cephfs-mirror (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:31.896 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-colorama. 2024-09-21T05:53:31.908 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-21T05:53:31.946 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-21T05:53:32.053 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-colorama. 2024-09-21T05:53:32.065 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-21T05:53:32.094 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-21T05:53:32.207 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package cephfs-shell. 2024-09-21T05:53:32.219 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:32.257 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking cephfs-shell (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:32.305 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package cephfs-shell. 2024-09-21T05:53:32.318 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:32.355 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking cephfs-shell (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:32.526 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package cephfs-top. 2024-09-21T05:53:32.538 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:32.576 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking cephfs-top (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:32.649 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package cephfs-top. 2024-09-21T05:53:32.662 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T05:53:32.699 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking cephfs-top (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:32.803 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T05:53:32.815 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:32.845 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libcephfs-dev (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:32.951 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T05:53:32.965 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libcephfs-dev (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:33.164 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T05:53:33.176 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T05:53:33.214 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T05:53:33.362 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T05:53:33.376 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T05:53:33.420 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T05:53:33.577 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T05:53:33.589 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T05:53:33.627 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T05:53:33.758 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T05:53:33.771 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T05:53:33.808 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T05:53:34.031 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T05:53:34.043 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T05:53:34.080 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T05:53:34.187 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T05:53:34.200 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T05:53:34.245 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T05:53:34.486 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package pkg-config. 2024-09-21T05:53:34.497 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T05:53:34.527 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T05:53:34.700 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package pkg-config. 2024-09-21T05:53:34.713 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T05:53:34.750 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T05:53:34.764 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T05:53:34.775 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T05:53:34.813 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T05:53:35.011 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T05:53:35.024 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T05:53:35.069 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T05:53:35.242 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T05:53:35.254 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T05:53:35.291 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T05:53:35.547 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T05:53:35.560 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T05:53:35.597 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T05:53:35.619 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T05:53:35.631 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T05:53:35.669 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T05:53:35.891 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T05:53:35.904 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T05:53:35.942 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T05:53:36.006 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T05:53:36.018 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T05:53:36.056 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T05:53:36.245 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T05:53:36.258 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T05:53:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T05:53:36.367 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T05:53:36.379 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T05:53:36.417 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T05:53:36.639 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T05:53:36.652 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T05:53:36.697 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T05:53:36.736 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-py. 2024-09-21T05:53:36.749 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-21T05:53:36.778 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T05:53:37.084 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-py. 2024-09-21T05:53:37.090 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T05:53:37.097 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-21T05:53:37.102 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T05:53:37.139 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T05:53:37.142 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T05:53:37.503 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T05:53:37.517 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T05:53:37.561 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T05:53:37.602 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T05:53:37.613 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T05:53:37.652 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T05:53:37.938 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-toml. 2024-09-21T05:53:37.949 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-21T05:53:37.979 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T05:53:38.040 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T05:53:38.052 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T05:53:38.090 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T05:53:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T05:53:38.302 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T05:53:38.340 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T05:53:38.385 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-toml. 2024-09-21T05:53:38.397 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-21T05:53:38.476 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T05:53:38.703 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T05:53:38.714 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T05:53:38.745 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T05:53:38.752 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T05:53:38.757 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T05:53:38.795 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T05:53:39.013 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T05:53:39.025 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T05:53:39.088 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T05:53:39.124 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T05:53:39.135 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T05:53:39.173 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T05:53:39.426 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T05:53:39.438 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T05:53:39.501 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T05:53:39.859 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package radosgw. 2024-09-21T05:53:39.870 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../123-radosgw_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:39.908 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking radosgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:40.505 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package radosgw. 2024-09-21T05:53:40.517 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../123-radosgw_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:40.555 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking radosgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:40.795 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T05:53:40.807 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:40.836 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking rbd-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:41.178 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T05:53:41.499 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T05:53:41.513 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T05:53:41.566 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking rbd-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:41.700 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T05:53:41.826 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T05:53:41.934 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T05:53:42.093 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T05:53:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T05:53:42.356 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T05:53:42.488 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T05:53:42.615 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T05:53:43.010 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T05:53:43.111 INFO:teuthology.orchestra.run.smithi050.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T05:53:43.163 INFO:teuthology.orchestra.run.smithi050.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-21T05:53:43.211 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:43.633 INFO:teuthology.orchestra.run.smithi088.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T05:53:43.646 INFO:teuthology.orchestra.run.smithi088.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-21T05:53:43.691 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:43.747 INFO:teuthology.orchestra.run.smithi050.stdout:Adding system user cephadm....done 2024-09-21T05:53:43.901 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T05:53:44.210 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T05:53:44.259 INFO:teuthology.orchestra.run.smithi088.stdout:Adding system user cephadm....done 2024-09-21T05:53:44.414 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T05:53:44.478 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T05:53:44.604 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T05:53:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T05:53:44.888 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T05:53:45.024 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T05:53:45.139 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T05:53:45.150 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T05:53:45.284 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T05:53:45.435 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T05:53:45.603 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T05:53:45.728 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T05:53:45.854 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T05:53:46.005 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T05:53:46.172 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T05:53:46.265 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-21T05:53:46.533 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T05:53:46.551 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T05:53:46.826 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T05:53:46.894 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-21T05:53:47.053 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T05:53:47.171 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T05:53:47.320 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T05:53:47.446 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T05:53:47.705 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T05:53:47.706 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T05:53:47.974 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T05:53:47.998 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-ceph-argparse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:48.249 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T05:53:48.250 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T05:53:48.391 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-21T05:53:48.518 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T05:53:48.518 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-ceph-argparse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:48.643 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-21T05:53:48.803 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T05:53:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T05:53:48.928 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-21T05:53:49.046 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T05:53:49.046 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T05:53:49.156 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-21T05:53:49.330 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T05:53:49.432 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T05:53:49.557 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T05:53:49.709 INFO:teuthology.orchestra.run.smithi050.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T05:53:49.736 INFO:teuthology.orchestra.run.smithi050.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T05:53:49.825 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T05:53:49.834 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T05:53:49.960 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T05:53:50.214 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T05:53:50.247 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T05:53:50.345 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T05:53:50.353 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T05:53:50.463 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T05:53:50.479 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T05:53:50.772 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T05:53:50.814 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T05:53:50.932 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T05:53:51.056 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T05:53:51.208 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T05:53:51.374 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T05:53:51.484 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T05:53:51.508 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T05:53:51.802 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T05:53:51.953 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T05:53:52.236 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T05:53:52.512 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T05:53:52.638 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T05:53:52.647 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T05:53:52.764 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T05:53:52.898 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T05:53:53.057 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T05:53:53.065 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T05:53:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T05:53:53.458 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T05:53:53.518 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T05:53:53.726 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T05:53:53.911 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T05:53:54.127 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T05:53:54.203 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T05:53:54.445 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-21T05:53:54.613 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T05:53:54.763 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T05:53:54.914 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-21T05:53:55.107 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T05:53:55.224 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T05:53:55.383 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T05:53:55.526 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:55.800 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T05:53:55.852 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T05:53:56.068 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T05:53:56.219 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:53:56.236 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T05:53:56.529 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T05:53:56.888 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T05:53:57.534 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T05:53:57.794 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T05:53:57.928 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T05:53:58.119 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T05:53:58.271 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T05:53:58.372 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T05:53:58.379 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T05:53:58.473 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T05:53:58.513 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T05:53:58.816 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T05:53:58.823 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T05:53:58.956 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T05:53:59.082 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T05:53:59.092 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T05:53:59.369 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T05:53:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T05:53:59.495 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T05:53:59.652 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T05:53:59.838 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T05:53:59.911 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T05:54:00.037 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T05:54:00.106 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T05:54:00.347 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T05:54:00.357 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T05:54:00.608 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T05:54:00.640 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T05:54:00.876 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T05:54:00.899 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T05:54:01.160 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T05:54:01.192 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T05:54:01.286 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T05:54:01.468 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T05:54:01.612 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T05:54:01.727 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T05:54:01.738 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T05:54:01.853 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T05:54:02.023 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T05:54:02.146 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T05:54:02.272 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T05:54:02.358 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T05:54:02.574 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T05:54:02.642 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T05:54:02.777 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T05:54:02.875 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T05:54:02.903 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T05:54:02.991 INFO:teuthology.orchestra.run.smithi050.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T05:54:03.143 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T05:54:03.246 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T05:54:03.268 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T05:54:03.372 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T05:54:03.394 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T05:54:03.483 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T05:54:03.721 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T05:54:03.765 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T05:54:03.847 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T05:54:04.041 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T05:54:04.193 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T05:54:04.248 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T05:54:04.461 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T05:54:04.508 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T05:54:04.586 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T05:54:04.642 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T05:54:04.893 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T05:54:04.938 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T05:54:05.019 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T05:54:05.247 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T05:54:05.346 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T05:54:05.609 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T05:54:05.701 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T05:54:05.760 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T05:54:06.037 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T05:54:06.092 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T05:54:06.268 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T05:54:06.313 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T05:54:06.536 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T05:54:06.572 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T05:54:06.812 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T05:54:06.966 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T05:54:07.080 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T05:54:07.268 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T05:54:07.465 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T05:54:07.629 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T05:54:07.767 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T05:54:07.788 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librados2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:07.906 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libsqlite3-mod-ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:08.015 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T05:54:08.152 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T05:54:08.278 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up librados2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:08.394 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libcephfs2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:08.429 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libsqlite3-mod-ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:08.520 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libradosstriper1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:08.555 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T05:54:08.645 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T05:54:08.917 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libcephfs2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:08.939 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librbd1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:09.026 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libradosstriper1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:09.040 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-modules-core (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:09.136 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T05:54:09.141 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:09.357 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T05:54:09.357 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T05:54:09.461 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up librbd1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:09.571 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-modules-core (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:09.681 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:09.872 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T05:54:09.872 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T05:54:10.001 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libcephfs-dev (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:10.127 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-rados (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:10.253 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librgw2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:10.404 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up cephfs-top (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:10.530 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T05:54:10.540 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libcephfs-dev (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:10.666 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-rados (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:10.776 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up librgw2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:10.893 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up cephfs-top (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:11.019 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T05:54:11.884 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-rbd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:12.010 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up rbd-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-rgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:12.271 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cephfs (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:12.323 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-rbd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:12.438 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up cephfs-shell (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:12.449 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up rbd-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:12.539 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:12.575 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-rgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:12.734 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cephfs (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:12.861 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up cephfs-shell (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:12.995 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:13.101 INFO:teuthology.orchestra.run.smithi050.stdout:Adding group ceph....done 2024-09-21T05:54:13.502 INFO:teuthology.orchestra.run.smithi050.stdout:Adding system user ceph....done 2024-09-21T05:54:13.548 INFO:teuthology.orchestra.run.smithi088.stdout:Adding group ceph....done 2024-09-21T05:54:13.654 INFO:teuthology.orchestra.run.smithi050.stdout:Setting system user ceph properties....done 2024-09-21T05:54:13.671 INFO:teuthology.orchestra.run.smithi050.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T05:54:13.822 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T05:54:13.957 INFO:teuthology.orchestra.run.smithi088.stdout:Adding system user ceph....done 2024-09-21T05:54:14.126 INFO:teuthology.orchestra.run.smithi088.stdout:Setting system user ceph properties....done 2024-09-21T05:54:14.147 INFO:teuthology.orchestra.run.smithi088.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T05:54:14.206 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T05:54:14.290 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T05:54:14.696 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T05:54:14.933 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up cephfs-mirror (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:15.148 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:54:15.148 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:54:15.379 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up cephfs-mirror (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:15.579 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:54:15.579 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:54:15.800 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-test (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:15.960 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up radosgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:16.222 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-test (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:16.390 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up radosgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:16.487 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:54:16.487 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:54:16.915 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:54:16.916 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:54:17.095 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-base (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:17.439 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T05:54:17.583 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-base (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:17.946 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T05:54:18.064 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mds (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:18.593 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mds (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:18.800 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T05:54:18.800 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T05:54:18.841 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T05:54:18.841 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T05:54:19.463 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:19.487 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:19.682 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T05:54:19.682 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T05:54:19.724 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T05:54:19.725 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T05:54:20.289 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-osd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:20.338 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-osd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:20.593 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T05:54:20.593 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T05:54:20.690 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T05:54:20.690 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T05:54:21.232 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-k8sevents (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:21.358 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:21.501 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mon (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:21.717 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T05:54:21.717 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T05:54:21.861 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-k8sevents (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:21.987 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:22.139 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mon (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:22.336 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:22.374 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T05:54:22.375 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T05:54:22.479 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:22.605 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-dashboard (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:22.747 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-volume (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:23.031 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:23.150 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T05:54:23.199 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:23.292 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T05:54:23.350 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-dashboard (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:23.444 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T05:54:23.526 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-volume (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T05:54:23.896 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T05:54:24.030 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T05:54:24.207 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T05:54:24.604 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T05:54:25.420 INFO:teuthology.orchestra.run.smithi050.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T05:54:25.425 DEBUG:teuthology.orchestra.run.smithi050:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-21T05:54:25.504 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2024-09-21T05:54:25.667 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T05:54:25.692 INFO:teuthology.orchestra.run.smithi050.stdout:Building dependency tree... 2024-09-21T05:54:25.692 INFO:teuthology.orchestra.run.smithi050.stdout:Reading state information... 2024-09-21T05:54:25.884 INFO:teuthology.orchestra.run.smithi050.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-21T05:54:25.885 INFO:teuthology.orchestra.run.smithi050.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-21T05:54:25.885 INFO:teuthology.orchestra.run.smithi050.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-21T05:54:25.885 INFO:teuthology.orchestra.run.smithi050.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T05:54:25.885 INFO:teuthology.orchestra.run.smithi050.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T05:54:25.885 INFO:teuthology.orchestra.run.smithi050.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T05:54:25.885 INFO:teuthology.orchestra.run.smithi050.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T05:54:25.886 INFO:teuthology.orchestra.run.smithi050.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T05:54:25.886 INFO:teuthology.orchestra.run.smithi050.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T05:54:25.886 INFO:teuthology.orchestra.run.smithi050.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T05:54:25.887 INFO:teuthology.orchestra.run.smithi050.stdout:The following additional packages will be installed: 2024-09-21T05:54:25.887 INFO:teuthology.orchestra.run.smithi050.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-21T05:54:25.887 INFO:teuthology.orchestra.run.smithi050.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-21T05:54:25.887 INFO:teuthology.orchestra.run.smithi050.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-21T05:54:25.887 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-21T05:54:25.887 INFO:teuthology.orchestra.run.smithi050.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-21T05:54:25.888 INFO:teuthology.orchestra.run.smithi050.stdout:Suggested packages: 2024-09-21T05:54:25.888 INFO:teuthology.orchestra.run.smithi050.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-21T05:54:25.889 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-21T05:54:25.889 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-21T05:54:25.946 INFO:teuthology.orchestra.run.smithi050.stdout:The following NEW packages will be installed: 2024-09-21T05:54:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-21T05:54:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-21T05:54:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-21T05:54:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-21T05:54:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-21T05:54:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-21T05:54:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-21T05:54:25.949 INFO:teuthology.orchestra.run.smithi050.stdout:The following packages will be upgraded: 2024-09-21T05:54:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-21T05:54:26.026 INFO:teuthology.orchestra.run.smithi050.stdout:4 upgraded, 39 newly installed, 0 to remove and 331 not upgraded. 2024-09-21T05:54:26.026 INFO:teuthology.orchestra.run.smithi050.stdout:Need to get 55.8 MB of archives. 2024-09-21T05:54:26.026 INFO:teuthology.orchestra.run.smithi050.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-21T05:54:26.026 INFO:teuthology.orchestra.run.smithi050.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-21T05:54:26.159 INFO:teuthology.orchestra.run.smithi050.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-21T05:54:26.163 INFO:teuthology.orchestra.run.smithi050.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-21T05:54:26.190 INFO:teuthology.orchestra.run.smithi050.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-21T05:54:26.190 INFO:teuthology.orchestra.run.smithi050.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-21T05:54:26.191 INFO:teuthology.orchestra.run.smithi050.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-21T05:54:26.198 INFO:teuthology.orchestra.run.smithi050.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-21T05:54:26.199 INFO:teuthology.orchestra.run.smithi050.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-21T05:54:26.201 INFO:teuthology.orchestra.run.smithi050.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-21T05:54:26.202 INFO:teuthology.orchestra.run.smithi050.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-21T05:54:26.202 INFO:teuthology.orchestra.run.smithi050.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-21T05:54:26.203 INFO:teuthology.orchestra.run.smithi050.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-21T05:54:26.212 INFO:teuthology.orchestra.run.smithi050.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-21T05:54:26.212 INFO:teuthology.orchestra.run.smithi050.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-21T05:54:26.212 INFO:teuthology.orchestra.run.smithi050.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-21T05:54:26.228 INFO:teuthology.orchestra.run.smithi050.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-21T05:54:26.434 INFO:teuthology.orchestra.run.smithi088.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T05:54:26.438 DEBUG:teuthology.orchestra.run.smithi088:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-21T05:54:26.442 INFO:teuthology.orchestra.run.smithi050.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-21T05:54:26.443 INFO:teuthology.orchestra.run.smithi050.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-21T05:54:26.444 INFO:teuthology.orchestra.run.smithi050.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.4 [262 kB] 2024-09-21T05:54:26.446 INFO:teuthology.orchestra.run.smithi050.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-21T05:54:26.447 INFO:teuthology.orchestra.run.smithi050.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-21T05:54:26.451 INFO:teuthology.orchestra.run.smithi050.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-21T05:54:26.453 INFO:teuthology.orchestra.run.smithi050.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-21T05:54:26.460 INFO:teuthology.orchestra.run.smithi050.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-21T05:54:26.460 INFO:teuthology.orchestra.run.smithi050.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-21T05:54:26.482 INFO:teuthology.orchestra.run.smithi050.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-21T05:54:26.482 INFO:teuthology.orchestra.run.smithi050.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.4 [1,094 kB] 2024-09-21T05:54:26.492 INFO:teuthology.orchestra.run.smithi050.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-21T05:54:26.512 INFO:teuthology.orchestra.run.smithi050.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-21T05:54:26.515 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2024-09-21T05:54:26.516 INFO:teuthology.orchestra.run.smithi050.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-21T05:54:26.516 INFO:teuthology.orchestra.run.smithi050.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-21T05:54:26.517 INFO:teuthology.orchestra.run.smithi050.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-21T05:54:26.518 INFO:teuthology.orchestra.run.smithi050.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-21T05:54:26.529 INFO:teuthology.orchestra.run.smithi050.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-21T05:54:26.531 INFO:teuthology.orchestra.run.smithi050.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-21T05:54:26.675 INFO:teuthology.orchestra.run.smithi050.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-21T05:54:26.676 INFO:teuthology.orchestra.run.smithi050.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-21T05:54:26.676 INFO:teuthology.orchestra.run.smithi050.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-21T05:54:26.676 INFO:teuthology.orchestra.run.smithi050.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-21T05:54:26.676 INFO:teuthology.orchestra.run.smithi050.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-21T05:54:26.676 INFO:teuthology.orchestra.run.smithi050.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-21T05:54:26.689 INFO:teuthology.orchestra.run.smithi050.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-21T05:54:26.690 INFO:teuthology.orchestra.run.smithi050.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-21T05:54:26.698 INFO:teuthology.orchestra.run.smithi088.stdout:Building dependency tree... 2024-09-21T05:54:26.698 INFO:teuthology.orchestra.run.smithi088.stdout:Reading state information... 2024-09-21T05:54:26.884 INFO:teuthology.orchestra.run.smithi088.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-21T05:54:26.884 INFO:teuthology.orchestra.run.smithi088.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-21T05:54:26.884 INFO:teuthology.orchestra.run.smithi088.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-21T05:54:26.884 INFO:teuthology.orchestra.run.smithi088.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T05:54:26.884 INFO:teuthology.orchestra.run.smithi088.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T05:54:26.884 INFO:teuthology.orchestra.run.smithi088.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T05:54:26.884 INFO:teuthology.orchestra.run.smithi088.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T05:54:26.885 INFO:teuthology.orchestra.run.smithi088.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T05:54:26.885 INFO:teuthology.orchestra.run.smithi088.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T05:54:26.885 INFO:teuthology.orchestra.run.smithi088.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T05:54:26.886 INFO:teuthology.orchestra.run.smithi088.stdout:The following additional packages will be installed: 2024-09-21T05:54:26.886 INFO:teuthology.orchestra.run.smithi088.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-21T05:54:26.886 INFO:teuthology.orchestra.run.smithi088.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-21T05:54:26.886 INFO:teuthology.orchestra.run.smithi088.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-21T05:54:26.887 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-21T05:54:26.887 INFO:teuthology.orchestra.run.smithi088.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-21T05:54:26.888 INFO:teuthology.orchestra.run.smithi088.stdout:Suggested packages: 2024-09-21T05:54:26.888 INFO:teuthology.orchestra.run.smithi088.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-21T05:54:26.888 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-21T05:54:26.888 INFO:teuthology.orchestra.run.smithi088.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-21T05:54:26.943 INFO:teuthology.orchestra.run.smithi088.stdout:The following NEW packages will be installed: 2024-09-21T05:54:26.944 INFO:teuthology.orchestra.run.smithi088.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-21T05:54:26.944 INFO:teuthology.orchestra.run.smithi088.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-21T05:54:26.945 INFO:teuthology.orchestra.run.smithi088.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-21T05:54:26.945 INFO:teuthology.orchestra.run.smithi088.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-21T05:54:26.945 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-21T05:54:26.945 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-21T05:54:26.945 INFO:teuthology.orchestra.run.smithi088.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-21T05:54:26.946 INFO:teuthology.orchestra.run.smithi088.stdout:The following packages will be upgraded: 2024-09-21T05:54:26.947 INFO:teuthology.orchestra.run.smithi088.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-21T05:54:27.003 INFO:teuthology.orchestra.run.smithi088.stdout:4 upgraded, 39 newly installed, 0 to remove and 331 not upgraded. 2024-09-21T05:54:27.003 INFO:teuthology.orchestra.run.smithi088.stdout:Need to get 55.8 MB of archives. 2024-09-21T05:54:27.003 INFO:teuthology.orchestra.run.smithi088.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-21T05:54:27.003 INFO:teuthology.orchestra.run.smithi088.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-21T05:54:27.090 INFO:teuthology.orchestra.run.smithi050.stdout:Preconfiguring packages ... 2024-09-21T05:54:27.135 INFO:teuthology.orchestra.run.smithi088.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-21T05:54:27.139 INFO:teuthology.orchestra.run.smithi088.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-21T05:54:27.166 INFO:teuthology.orchestra.run.smithi088.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-21T05:54:27.167 INFO:teuthology.orchestra.run.smithi088.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-21T05:54:27.167 INFO:teuthology.orchestra.run.smithi088.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-21T05:54:27.174 INFO:teuthology.orchestra.run.smithi088.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-21T05:54:27.175 INFO:teuthology.orchestra.run.smithi088.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-21T05:54:27.177 INFO:teuthology.orchestra.run.smithi088.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-21T05:54:27.178 INFO:teuthology.orchestra.run.smithi088.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-21T05:54:27.178 INFO:teuthology.orchestra.run.smithi088.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-21T05:54:27.179 INFO:teuthology.orchestra.run.smithi088.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-21T05:54:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-21T05:54:27.188 INFO:teuthology.orchestra.run.smithi088.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-21T05:54:27.188 INFO:teuthology.orchestra.run.smithi088.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-21T05:54:27.205 INFO:teuthology.orchestra.run.smithi088.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-21T05:54:27.246 INFO:teuthology.orchestra.run.smithi050.stdout:Fetched 55.8 MB in 1s (76.2 MB/s) 2024-09-21T05:54:27.414 INFO:teuthology.orchestra.run.smithi088.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-21T05:54:27.416 INFO:teuthology.orchestra.run.smithi088.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-21T05:54:27.417 INFO:teuthology.orchestra.run.smithi088.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.4 [262 kB] 2024-09-21T05:54:27.419 INFO:teuthology.orchestra.run.smithi088.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-21T05:54:27.420 INFO:teuthology.orchestra.run.smithi088.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-21T05:54:27.424 INFO:teuthology.orchestra.run.smithi088.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-21T05:54:27.426 INFO:teuthology.orchestra.run.smithi088.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-21T05:54:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-21T05:54:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-21T05:54:27.453 INFO:teuthology.orchestra.run.smithi088.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-21T05:54:27.459 INFO:teuthology.orchestra.run.smithi088.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.4 [1,094 kB] 2024-09-21T05:54:27.469 INFO:teuthology.orchestra.run.smithi088.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-21T05:54:27.488 INFO:teuthology.orchestra.run.smithi088.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-21T05:54:27.492 INFO:teuthology.orchestra.run.smithi088.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-21T05:54:27.493 INFO:teuthology.orchestra.run.smithi088.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-21T05:54:27.493 INFO:teuthology.orchestra.run.smithi088.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-21T05:54:27.494 INFO:teuthology.orchestra.run.smithi088.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-21T05:54:27.505 INFO:teuthology.orchestra.run.smithi088.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-21T05:54:27.507 INFO:teuthology.orchestra.run.smithi088.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-21T05:54:27.560 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143295 files and directories currently installed.) 2024-09-21T05:54:27.563 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T05:54:27.650 INFO:teuthology.orchestra.run.smithi088.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-21T05:54:27.651 INFO:teuthology.orchestra.run.smithi088.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-21T05:54:27.651 INFO:teuthology.orchestra.run.smithi088.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-21T05:54:27.651 INFO:teuthology.orchestra.run.smithi088.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-21T05:54:27.652 INFO:teuthology.orchestra.run.smithi088.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-21T05:54:27.652 INFO:teuthology.orchestra.run.smithi088.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-21T05:54:27.664 INFO:teuthology.orchestra.run.smithi088.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-21T05:54:27.664 INFO:teuthology.orchestra.run.smithi088.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-21T05:54:27.686 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T05:54:28.049 INFO:teuthology.orchestra.run.smithi088.stdout:Preconfiguring packages ... 2024-09-21T05:54:28.216 INFO:teuthology.orchestra.run.smithi088.stdout:Fetched 55.8 MB in 1s (78.6 MB/s) 2024-09-21T05:54:28.321 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-21T05:54:28.521 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143295 files and directories currently installed.) 2024-09-21T05:54:28.525 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T05:54:28.657 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T05:54:29.331 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-21T05:54:29.447 INFO:teuthology.orchestra.run.smithi050.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-21T05:54:29.716 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package flex. 2024-09-21T05:54:29.789 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143295 files and directories currently installed.) 2024-09-21T05:54:29.793 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-21T05:54:29.840 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-21T05:54:30.336 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-21T05:54:30.485 INFO:teuthology.orchestra.run.smithi088.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-21T05:54:30.538 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-21T05:54:30.754 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package flex. 2024-09-21T05:54:30.824 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143295 files and directories currently installed.) 2024-09-21T05:54:30.828 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-21T05:54:30.877 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-21T05:54:31.080 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T05:54:31.397 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-21T05:54:31.638 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143368 files and directories currently installed.) 2024-09-21T05:54:31.642 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T05:54:31.666 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-21T05:54:31.754 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T05:54:32.149 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T05:54:32.193 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T05:54:32.319 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T05:54:32.645 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-21T05:54:32.736 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143368 files and directories currently installed.) 2024-09-21T05:54:32.739 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T05:54:32.866 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T05:54:33.004 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package bison. 2024-09-21T05:54:33.081 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143368 files and directories currently installed.) 2024-09-21T05:54:33.085 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-21T05:54:33.123 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-21T05:54:33.270 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T05:54:33.414 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T05:54:33.515 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-21T05:54:33.531 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-21T05:54:33.565 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-21T05:54:33.789 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-21T05:54:33.885 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package dump. 2024-09-21T05:54:33.902 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-21T05:54:33.952 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking dump (0.4b47-3) ... 2024-09-21T05:54:34.264 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package indent. 2024-09-21T05:54:34.265 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package bison. 2024-09-21T05:54:34.281 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T05:54:34.322 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-21T05:54:34.342 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143368 files and directories currently installed.) 2024-09-21T05:54:34.346 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-21T05:54:34.401 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-21T05:54:34.617 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-21T05:54:34.634 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-21T05:54:34.683 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-21T05:54:34.910 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-21T05:54:34.925 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-21T05:54:34.976 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-21T05:54:35.038 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-21T05:54:35.055 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-21T05:54:35.096 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-21T05:54:35.329 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package dump. 2024-09-21T05:54:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-21T05:54:35.366 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-21T05:54:35.379 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking dump (0.4b47-3) ... 2024-09-21T05:54:35.383 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-21T05:54:35.424 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-21T05:54:35.683 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package indent. 2024-09-21T05:54:35.700 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T05:54:35.742 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-21T05:54:35.837 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-21T05:54:35.853 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-21T05:54:35.894 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-21T05:54:36.020 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-21T05:54:36.035 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-21T05:54:36.078 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-21T05:54:36.156 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-21T05:54:36.172 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-21T05:54:36.214 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-21T05:54:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-21T05:54:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-21T05:54:36.448 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-21T05:54:36.467 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libjson-perl. 2024-09-21T05:54:36.483 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-21T05:54:36.525 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-21T05:54:36.727 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-21T05:54:36.742 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-21T05:54:36.784 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-21T05:54:36.864 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-21T05:54:36.881 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-21T05:54:36.914 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-21T05:54:37.130 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-21T05:54:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-21T05:54:37.188 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-21T05:54:37.458 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-21T05:54:37.473 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-21T05:54:37.507 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-21T05:54:37.726 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libjson-perl. 2024-09-21T05:54:37.742 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-21T05:54:37.801 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-21T05:54:38.121 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-21T05:54:38.138 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-21T05:54:38.180 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-21T05:54:38.200 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-21T05:54:38.217 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-21T05:54:38.249 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T05:54:38.544 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-21T05:54:38.561 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-21T05:54:38.602 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T05:54:38.922 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-21T05:54:38.939 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.4_amd64.deb ... 2024-09-21T05:54:39.005 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T05:54:39.350 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-21T05:54:39.367 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-21T05:54:39.408 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-21T05:54:39.534 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-21T05:54:39.550 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-21T05:54:39.584 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T05:54:39.811 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-21T05:54:39.828 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-21T05:54:39.877 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-21T05:54:39.887 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-21T05:54:39.903 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-21T05:54:39.945 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T05:54:40.273 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-21T05:54:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-21T05:54:40.289 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.4_amd64.deb ... 2024-09-21T05:54:40.298 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T05:54:40.339 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T05:54:40.356 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T05:54:40.693 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-21T05:54:40.709 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-21T05:54:40.726 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-21T05:54:40.742 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T05:54:40.751 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-21T05:54:40.784 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T05:54:41.096 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-21T05:54:41.111 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-21T05:54:41.120 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-21T05:54:41.137 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T05:54:41.154 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-21T05:54:41.186 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T05:54:41.456 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-21T05:54:41.472 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-21T05:54:41.547 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T05:54:41.590 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-21T05:54:41.606 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T05:54:41.656 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T05:54:42.051 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-21T05:54:42.067 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T05:54:42.068 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-21T05:54:42.084 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-21T05:54:42.110 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T05:54:42.126 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-21T05:54:42.395 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-21T05:54:42.411 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T05:54:42.471 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T05:54:42.479 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package modemmanager. 2024-09-21T05:54:42.496 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.4_amd64.deb ... 2024-09-21T05:54:42.562 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T05:54:42.723 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-21T05:54:42.739 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-21T05:54:42.781 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T05:54:43.210 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package network-manager. 2024-09-21T05:54:43.227 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-21T05:54:43.277 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-21T05:54:43.351 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-21T05:54:43.367 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-21T05:54:43.418 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-21T05:54:43.829 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package modemmanager. 2024-09-21T05:54:43.843 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ppp. 2024-09-21T05:54:43.845 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.4_amd64.deb ... 2024-09-21T05:54:43.860 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-21T05:54:43.896 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T05:54:44.199 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-21T05:54:44.519 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package network-manager. 2024-09-21T05:54:44.535 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-21T05:54:44.603 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-21T05:54:44.614 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package pptp-linux. 2024-09-21T05:54:44.629 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-21T05:54:44.673 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-21T05:54:44.909 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-21T05:54:44.924 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-21T05:54:44.959 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-21T05:54:45.268 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ppp. 2024-09-21T05:54:45.278 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-21T05:54:45.284 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-21T05:54:45.293 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-21T05:54:45.336 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking postgresql-client-common (238) ... 2024-09-21T05:54:45.651 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-21T05:54:45.665 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-21T05:54:45.682 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T05:54:45.724 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T05:54:45.931 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package pptp-linux. 2024-09-21T05:54:45.945 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-21T05:54:45.981 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-21T05:54:46.292 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-21T05:54:46.307 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-21T05:54:46.320 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package postgresql-common. 2024-09-21T05:54:46.337 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-21T05:54:46.342 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-21T05:54:46.397 INFO:teuthology.orchestra.run.smithi050.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-21T05:54:46.427 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking postgresql-common (238) ... 2024-09-21T05:54:46.645 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-21T05:54:46.660 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-21T05:54:46.744 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql-client-common (238) ... 2024-09-21T05:54:46.885 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package postgresql-14. 2024-09-21T05:54:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T05:54:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T05:54:47.015 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-21T05:54:47.031 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T05:54:47.082 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T05:54:47.628 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql-common. 2024-09-21T05:54:47.643 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-21T05:54:47.704 INFO:teuthology.orchestra.run.smithi088.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-21T05:54:47.735 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql-common (238) ... 2024-09-21T05:54:48.219 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql-14. 2024-09-21T05:54:48.233 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T05:54:48.271 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package postgresql. 2024-09-21T05:54:48.280 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T05:54:48.287 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-21T05:54:48.329 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking postgresql (14+238) ... 2024-09-21T05:54:48.557 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package postgresql-client. 2024-09-21T05:54:48.573 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-21T05:54:48.615 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking postgresql-client (14+238) ... 2024-09-21T05:54:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-21T05:54:48.859 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-21T05:54:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-21T05:54:49.245 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package tcl8.6. 2024-09-21T05:54:49.262 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-21T05:54:49.345 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-21T05:54:49.463 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql. 2024-09-21T05:54:49.479 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-21T05:54:49.521 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql (14+238) ... 2024-09-21T05:54:49.749 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql-client. 2024-09-21T05:54:49.764 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-21T05:54:49.798 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql-client (14+238) ... 2024-09-21T05:54:49.982 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package tcl. 2024-09-21T05:54:49.998 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-21T05:54:50.043 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-21T05:54:50.044 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-21T05:54:50.058 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-21T05:54:50.101 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-21T05:54:50.359 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package wpasupplicant. 2024-09-21T05:54:50.376 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-21T05:54:50.417 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-21T05:54:50.436 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package tcl8.6. 2024-09-21T05:54:50.452 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-21T05:54:50.528 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-21T05:54:50.796 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-21T05:54:50.797 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package tcl. 2024-09-21T05:54:50.813 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-21T05:54:50.814 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-21T05:54:50.854 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-21T05:54:50.861 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-21T05:54:51.192 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package wpasupplicant. 2024-09-21T05:54:51.208 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-21T05:54:51.241 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-21T05:54:51.266 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-21T05:54:51.283 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-21T05:54:51.333 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-21T05:54:51.603 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up postgresql-client-common (238) ... 2024-09-21T05:54:51.621 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-21T05:54:51.637 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-21T05:54:51.679 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-21T05:54:51.838 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-21T05:54:51.965 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-21T05:54:52.057 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-21T05:54:52.073 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-21T05:54:52.091 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T05:54:52.157 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-21T05:54:52.217 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-21T05:54:52.359 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-21T05:54:52.460 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql-client-common (238) ... 2024-09-21T05:54:52.460 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T05:54:53.096 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-21T05:54:53.214 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-21T05:54:53.339 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T05:54:53.430 INFO:teuthology.orchestra.run.smithi050.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-21T05:54:53.507 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-21T05:54:53.525 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-21T05:54:53.642 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-21T05:54:53.751 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T05:54:53.884 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-21T05:54:53.884 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-21T05:54:54.527 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-21T05:54:54.653 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-21T05:54:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-21T05:54:54.798 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-21T05:54:54.804 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T05:54:54.947 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-21T05:54:55.065 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-21T05:54:55.191 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-21T05:54:55.212 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-21T05:54:55.212 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-21T05:54:55.317 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-21T05:54:55.576 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-21T05:54:55.935 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-21T05:54:56.035 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-21T05:54:56.186 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T05:54:56.215 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-21T05:54:56.312 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-21T05:54:56.362 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up dump (0.4b47-3) ... 2024-09-21T05:54:56.469 INFO:teuthology.orchestra.run.smithi050.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-21T05:54:56.525 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T05:54:56.546 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-21T05:54:56.635 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-21T05:54:56.697 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-21T05:54:56.761 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-21T05:54:56.823 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-21T05:54:56.920 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-21T05:54:56.958 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-21T05:54:57.046 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-21T05:54:57.136 INFO:teuthology.orchestra.run.smithi050.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-21T05:54:57.198 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-21T05:54:57.324 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-21T05:54:57.525 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-21T05:54:57.621 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-21T05:54:57.634 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T05:54:57.752 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-21T05:54:57.772 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up dump (0.4b47-3) ... 2024-09-21T05:54:57.867 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-21T05:54:57.945 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up postgresql-client (14+238) ... 2024-09-21T05:54:58.088 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-21T05:54:58.349 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T05:54:58.475 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-21T05:54:58.601 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-21T05:54:58.679 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:54:58.679 INFO:teuthology.orchestra.run.smithi050.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-21T05:54:58.679 INFO:teuthology.orchestra.run.smithi050.stdout:which means they are currently configured by ifupdown: 2024-09-21T05:54:58.680 INFO:teuthology.orchestra.run.smithi050.stdout:- ens1f0 2024-09-21T05:54:58.680 INFO:teuthology.orchestra.run.smithi050.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-21T05:54:58.680 INFO:teuthology.orchestra.run.smithi050.stdout:remove their configuration from /etc/network/interfaces. 2024-09-21T05:54:58.680 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:54:58.727 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-21T05:54:58.819 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-21T05:54:58.869 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-21T05:54:58.943 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-21T05:54:59.004 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-21T05:54:59.130 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-21T05:54:59.228 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-21T05:54:59.307 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-21T05:54:59.475 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T05:54:59.601 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-21T05:54:59.634 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-21T05:54:59.785 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql-client (14+238) ... 2024-09-21T05:54:59.903 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-21T05:55:00.374 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T05:55:00.501 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:00.502 INFO:teuthology.orchestra.run.smithi088.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-21T05:55:00.502 INFO:teuthology.orchestra.run.smithi088.stdout:which means they are currently configured by ifupdown: 2024-09-21T05:55:00.502 INFO:teuthology.orchestra.run.smithi088.stdout:- enp3s0f1 2024-09-21T05:55:00.502 INFO:teuthology.orchestra.run.smithi088.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-21T05:55:00.502 INFO:teuthology.orchestra.run.smithi088.stdout:remove their configuration from /etc/network/interfaces. 2024-09-21T05:55:00.502 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:00.542 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-21T05:55:00.634 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-21T05:55:00.651 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-21T05:55:00.752 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up postgresql-common (238) ... 2024-09-21T05:55:01.030 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-21T05:55:01.432 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-21T05:55:01.741 INFO:teuthology.orchestra.run.smithi050.stdout:Adding user postgres to group ssl-cert 2024-09-21T05:55:01.971 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:01.971 INFO:teuthology.orchestra.run.smithi050.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-21T05:55:02.163 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T05:55:02.229 INFO:teuthology.orchestra.run.smithi050.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-21T05:55:02.230 INFO:teuthology.orchestra.run.smithi050.stdout:Removing obsolete dictionary files: 2024-09-21T05:55:02.298 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-21T05:55:02.416 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-21T05:55:02.525 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql-common (238) ... 2024-09-21T05:55:02.939 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-21T05:55:03.371 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T05:55:03.555 INFO:teuthology.orchestra.run.smithi088.stdout:Adding user postgres to group ssl-cert 2024-09-21T05:55:03.839 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:03.839 INFO:teuthology.orchestra.run.smithi088.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-21T05:55:03.873 INFO:teuthology.orchestra.run.smithi050.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-21T05:55:03.890 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-21T05:55:03.944 INFO:teuthology.orchestra.run.smithi050.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-21T05:55:03.944 INFO:teuthology.orchestra.run.smithi050.stdout:This user must also own the server process. 2024-09-21T05:55:03.944 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:03.944 INFO:teuthology.orchestra.run.smithi050.stdout:The database cluster will be initialized with locale "en_US". 2024-09-21T05:55:03.944 INFO:teuthology.orchestra.run.smithi050.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-21T05:55:03.944 INFO:teuthology.orchestra.run.smithi050.stdout:The default text search configuration will be set to "english". 2024-09-21T05:55:03.944 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:03.944 INFO:teuthology.orchestra.run.smithi050.stdout:Data page checksums are disabled. 2024-09-21T05:55:03.944 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:03.945 INFO:teuthology.orchestra.run.smithi050.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-21T05:55:03.945 INFO:teuthology.orchestra.run.smithi050.stdout:creating subdirectories ... ok 2024-09-21T05:55:03.945 INFO:teuthology.orchestra.run.smithi050.stdout:selecting dynamic shared memory implementation ... posix 2024-09-21T05:55:03.970 INFO:teuthology.orchestra.run.smithi050.stdout:selecting default max_connections ... 100 2024-09-21T05:55:04.005 INFO:teuthology.orchestra.run.smithi050.stdout:selecting default shared_buffers ... 128MB 2024-09-21T05:55:04.019 INFO:teuthology.orchestra.run.smithi050.stdout:selecting default time zone ... Etc/UTC 2024-09-21T05:55:04.020 INFO:teuthology.orchestra.run.smithi050.stdout:creating configuration files ... ok 2024-09-21T05:55:04.096 INFO:teuthology.orchestra.run.smithi088.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-21T05:55:04.097 INFO:teuthology.orchestra.run.smithi088.stdout:Removing obsolete dictionary files: 2024-09-21T05:55:04.189 INFO:teuthology.orchestra.run.smithi050.stdout:running bootstrap script ... ok 2024-09-21T05:55:04.830 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-21T05:55:04.834 INFO:teuthology.orchestra.run.smithi050.stdout:performing post-bootstrap initialization ... ok 2024-09-21T05:55:05.375 INFO:teuthology.orchestra.run.smithi050.stdout:syncing data to disk ... ok 2024-09-21T05:55:05.493 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T05:55:05.851 INFO:teuthology.orchestra.run.smithi088.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-21T05:55:05.866 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-21T05:55:05.868 INFO:teuthology.orchestra.run.smithi050.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-21T05:55:05.913 INFO:teuthology.orchestra.run.smithi088.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-21T05:55:05.913 INFO:teuthology.orchestra.run.smithi088.stdout:This user must also own the server process. 2024-09-21T05:55:05.913 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:05.913 INFO:teuthology.orchestra.run.smithi088.stdout:The database cluster will be initialized with locale "en_US". 2024-09-21T05:55:05.913 INFO:teuthology.orchestra.run.smithi088.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-21T05:55:05.913 INFO:teuthology.orchestra.run.smithi088.stdout:The default text search configuration will be set to "english". 2024-09-21T05:55:05.913 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:05.913 INFO:teuthology.orchestra.run.smithi088.stdout:Data page checksums are disabled. 2024-09-21T05:55:05.913 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:05.913 INFO:teuthology.orchestra.run.smithi088.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-21T05:55:05.914 INFO:teuthology.orchestra.run.smithi088.stdout:creating subdirectories ... ok 2024-09-21T05:55:05.914 INFO:teuthology.orchestra.run.smithi088.stdout:selecting dynamic shared memory implementation ... posix 2024-09-21T05:55:05.938 INFO:teuthology.orchestra.run.smithi088.stdout:selecting default max_connections ... 100 2024-09-21T05:55:05.971 INFO:teuthology.orchestra.run.smithi088.stdout:selecting default shared_buffers ... 128MB 2024-09-21T05:55:05.984 INFO:teuthology.orchestra.run.smithi088.stdout:selecting default time zone ... Etc/UTC 2024-09-21T05:55:05.985 INFO:teuthology.orchestra.run.smithi088.stdout:creating configuration files ... ok 2024-09-21T05:55:06.147 INFO:teuthology.orchestra.run.smithi088.stdout:running bootstrap script ... ok 2024-09-21T05:55:06.774 INFO:teuthology.orchestra.run.smithi088.stdout:performing post-bootstrap initialization ... ok 2024-09-21T05:55:07.448 INFO:teuthology.orchestra.run.smithi088.stdout:syncing data to disk ... ok 2024-09-21T05:55:07.967 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-21T05:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T05:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-21T05:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up postgresql (14+238) ... 2024-09-21T05:55:09.215 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T05:55:09.366 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T05:55:09.634 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-21T05:55:09.634 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-21T05:55:10.402 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T05:55:10.494 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-21T05:55:10.787 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T05:55:10.907 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T05:55:10.973 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T05:55:11.033 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-21T05:55:11.184 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql (14+238) ... 2024-09-21T05:55:11.293 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T05:55:11.394 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T05:55:11.654 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-21T05:55:11.655 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-21T05:55:12.413 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T05:55:12.417 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-21T05:55:12.501 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T05:55:12.505 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-21T05:55:12.866 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T05:55:13.018 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T05:55:13.044 INFO:teuthology.orchestra.run.smithi050.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T05:55:13.051 DEBUG:teuthology.parallel:result is None 2024-09-21T05:55:14.462 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-21T05:55:14.562 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T05:55:15.156 INFO:teuthology.orchestra.run.smithi088.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T05:55:15.162 DEBUG:teuthology.parallel:result is None 2024-09-21T05:55:15.162 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:55:15.309 DEBUG:teuthology.orchestra.run.smithi050:> dpkg-query -W -f '${Version}' ceph 2024-09-21T05:55:15.334 INFO:teuthology.orchestra.run.smithi050.stdout:19.3.0-5120-g18432218-1jammy 2024-09-21T05:55:15.334 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120-g18432218-1jammy 2024-09-21T05:55:15.334 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120-g18432218-1jammy is installed. 2024-09-21T05:55:15.336 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:55:15.478 DEBUG:teuthology.orchestra.run.smithi088:> dpkg-query -W -f '${Version}' ceph 2024-09-21T05:55:15.500 INFO:teuthology.orchestra.run.smithi088.stdout:19.3.0-5120-g18432218-1jammy 2024-09-21T05:55:15.500 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120-g18432218-1jammy 2024-09-21T05:55:15.500 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120-g18432218-1jammy is installed. 2024-09-21T05:55:15.502 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T05:55:15.502 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:15.502 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T05:55:15.525 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:15.525 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T05:55:15.559 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T05:55:15.559 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:15.560 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T05:55:15.589 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T05:55:15.652 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:15.652 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T05:55:15.671 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T05:55:15.730 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T05:55:15.730 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:15.730 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T05:55:15.751 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T05:55:15.812 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:15.812 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T05:55:15.831 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T05:55:15.894 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T05:55:15.894 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:15.894 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T05:55:15.915 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T05:55:15.976 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:15.976 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T05:55:15.995 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T05:55:16.054 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T05:55:16.128 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T05:55:16.128 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /var/log/ceph 2024-09-21T05:55:16.131 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 777 /var/log/ceph 2024-09-21T05:55:16.149 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T05:55:16.149 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T05:55:16.194 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T05:55:16.213 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T05:55:16.213 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T05:55:16.258 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T05:55:16.283 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-21T05:55:16.284 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-21T05:55:16.284 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-21T05:55:16.284 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:16.284 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T05:55:16.328 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T05:55:16.421 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:16.421 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T05:55:16.429 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T05:55:16.522 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T05:55:16.522 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T05:55:16.523 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513', 'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7913513', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240920.212106-debug'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7864, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'sleep_before_teardown': 0, 'subset': '67/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240920.212106-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi050.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIcFvSYb0NWh/GafGucXKHWWzVDXHCAyw+0zBqqzo/kzyYObF8+htqyVCdEyK6VOGcO3Ow0f/VjUBeTcTgt689Q=', 'smithi088.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG0fxMAEFMtzuXD0JW+mGVo2ANSv7KgSatFRbCKrI0MV9F9JMBgCCJNiew3r9vrRK1tXUmaGmsIK9RsYL4KG8/U='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgressReporter']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-20_23:48:22', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T05:55:16.523 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T05:55:16.526 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T05:55:16.572 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T05:55:16.578 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T05:55:16.635 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:16.635 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T05:55:16.649 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T05:55:16.649 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2024-09-21T05:55:16.701 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T05:55:16.701 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:55:16.701 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 886 Links: 1 2024-09-21T05:55:16.701 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:55:16.701 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-21 05:48:30.102004662 +0000 2024-09-21T05:55:16.701 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-21 05:48:29.794007412 +0000 2024-09-21T05:55:16.701 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-21 05:48:29.794007412 +0000 2024-09-21T05:55:16.701 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-21 05:48:29.794007412 +0000 2024-09-21T05:55:16.701 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T05:55:16.762 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-21T05:55:16.762 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-21T05:55:16.762 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000267708 s, 1.9 MB/s 2024-09-21T05:55:16.764 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T05:55:16.818 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2024-09-21T05:55:16.868 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T05:55:16.868 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:55:16.868 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 901 Links: 1 2024-09-21T05:55:16.869 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:55:16.869 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-21 05:48:30.414001874 +0000 2024-09-21T05:55:16.869 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-21 05:48:30.410001911 +0000 2024-09-21T05:55:16.869 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-21 05:48:30.410001911 +0000 2024-09-21T05:55:16.869 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-21 05:48:30.410001911 +0000 2024-09-21T05:55:16.870 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T05:55:16.931 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-21T05:55:16.931 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-21T05:55:16.931 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000262854 s, 1.9 MB/s 2024-09-21T05:55:16.933 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T05:55:16.986 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2024-09-21T05:55:17.036 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T05:55:17.036 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:55:17.036 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 921 Links: 1 2024-09-21T05:55:17.037 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:55:17.037 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-21 05:48:31.285994086 +0000 2024-09-21T05:55:17.037 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-21 05:48:31.017996479 +0000 2024-09-21T05:55:17.037 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-21 05:48:31.017996479 +0000 2024-09-21T05:55:17.037 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-21 05:48:31.017996479 +0000 2024-09-21T05:55:17.037 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T05:55:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-21T05:55:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-21T05:55:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000263524 s, 1.9 MB/s 2024-09-21T05:55:17.102 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T05:55:17.154 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2024-09-21T05:55:17.204 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T05:55:17.204 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:55:17.204 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 936 Links: 1 2024-09-21T05:55:17.205 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:55:17.205 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-21 05:48:31.581991442 +0000 2024-09-21T05:55:17.205 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-21 05:48:31.577991478 +0000 2024-09-21T05:55:17.205 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-21 05:48:31.577991478 +0000 2024-09-21T05:55:17.205 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-21 05:48:31.577991478 +0000 2024-09-21T05:55:17.205 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T05:55:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-21T05:55:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-21T05:55:17.267 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000264754 s, 1.9 MB/s 2024-09-21T05:55:17.268 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T05:55:17.322 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T05:55:17.322 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:17.322 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T05:55:17.331 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T05:55:17.331 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_1 2024-09-21T05:55:17.380 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T05:55:17.380 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:55:17.380 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 885 Links: 1 2024-09-21T05:55:17.380 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:55:17.380 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-09-21 05:48:35.942099135 +0000 2024-09-21T05:55:17.380 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 05:48:35.678101493 +0000 2024-09-21T05:55:17.380 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 05:48:35.678101493 +0000 2024-09-21T05:55:17.381 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 05:48:35.678101493 +0000 2024-09-21T05:55:17.381 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T05:55:17.438 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T05:55:17.438 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T05:55:17.438 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000377894 s, 1.4 MB/s 2024-09-21T05:55:17.440 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T05:55:17.490 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_2 2024-09-21T05:55:17.540 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T05:55:17.540 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:55:17.540 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 903 Links: 1 2024-09-21T05:55:17.540 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:55:17.540 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-09-21 05:48:36.542093775 +0000 2024-09-21T05:55:17.540 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 05:48:36.254096347 +0000 2024-09-21T05:55:17.540 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 05:48:36.254096347 +0000 2024-09-21T05:55:17.540 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 05:48:36.254096347 +0000 2024-09-21T05:55:17.540 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T05:55:17.597 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T05:55:17.597 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T05:55:17.598 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000362833 s, 1.4 MB/s 2024-09-21T05:55:17.599 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T05:55:17.650 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_3 2024-09-21T05:55:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T05:55:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:55:17.702 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 920 Links: 1 2024-09-21T05:55:17.702 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:55:17.702 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-09-21 05:48:37.098088808 +0000 2024-09-21T05:55:17.702 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 05:48:36.838091131 +0000 2024-09-21T05:55:17.702 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 05:48:36.838091131 +0000 2024-09-21T05:55:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 05:48:36.838091131 +0000 2024-09-21T05:55:17.702 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T05:55:17.762 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T05:55:17.762 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T05:55:17.762 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000366732 s, 1.4 MB/s 2024-09-21T05:55:17.764 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T05:55:17.818 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_4 2024-09-21T05:55:17.869 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T05:55:17.869 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:55:17.869 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 938 Links: 1 2024-09-21T05:55:17.869 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:55:17.869 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-09-21 05:48:37.714083306 +0000 2024-09-21T05:55:17.869 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 05:48:37.422085915 +0000 2024-09-21T05:55:17.869 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 05:48:37.422085915 +0000 2024-09-21T05:55:17.869 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 05:48:37.422085915 +0000 2024-09-21T05:55:17.869 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T05:55:17.930 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T05:55:17.930 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T05:55:17.930 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000377847 s, 1.4 MB/s 2024-09-21T05:55:17.932 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T05:55:17.983 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T05:55:17.983 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T05:55:17.983 INFO:tasks.ceph:Generating config... 2024-09-21T05:55:17.985 INFO:tasks.ceph:[client] pid file = 2024-09-21T05:55:17.985 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-21T05:55:17.985 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-21T05:55:17.986 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-21T05:55:17.986 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T05:55:17.986 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T05:55:17.986 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T05:55:17.986 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T05:55:17.986 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T05:55:17.986 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-21T05:55:17.987 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-21T05:55:17.987 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-21T05:55:17.987 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-21T05:55:17.987 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T05:55:17.987 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T05:55:17.987 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T05:55:17.988 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T05:55:17.988 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T05:55:17.988 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T05:55:17.988 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T05:55:17.988 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T05:55:17.989 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T05:55:17.989 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T05:55:17.989 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T05:55:17.989 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T05:55:17.989 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T05:55:17.989 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T05:55:18.050 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T05:55:18.060 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T05:55:18.120 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T05:55:18.139 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.50'), ('mon.b', '172.21.15.88'), ('mon.c', '[v2:172.21.15.88:3301,v1:172.21.15.88:6790]')] 2024-09-21T05:55:18.140 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.50,172.21.15.88,[v2:172.21.15.88:3301,v1:172.21.15.88:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-21T05:55:18.140 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:18.140 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T05:55:18.192 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.50 --add b 172.21.15.88 --addv c '[v2:172.21.15.88:3301,v1:172.21.15.88:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:18.294 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:18.294 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool: generated fsid 3f44e77a-b7bc-4ef8-b058-60eaaa9147d6 2024-09-21T05:55:18.294 INFO:teuthology.orchestra.run.smithi050.stdout:setting min_mon_release = quincy 2024-09-21T05:55:18.294 INFO:teuthology.orchestra.run.smithi050.stdout:epoch 0 2024-09-21T05:55:18.295 INFO:teuthology.orchestra.run.smithi050.stdout:fsid 3f44e77a-b7bc-4ef8-b058-60eaaa9147d6 2024-09-21T05:55:18.295 INFO:teuthology.orchestra.run.smithi050.stdout:last_changed 2024-09-21T05:55:18.295535+0000 2024-09-21T05:55:18.295 INFO:teuthology.orchestra.run.smithi050.stdout:created 2024-09-21T05:55:18.295535+0000 2024-09-21T05:55:18.295 INFO:teuthology.orchestra.run.smithi050.stdout:min_mon_release 17 (quincy) 2024-09-21T05:55:18.295 INFO:teuthology.orchestra.run.smithi050.stdout:election_strategy: 1 2024-09-21T05:55:18.295 INFO:teuthology.orchestra.run.smithi050.stdout:0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-09-21T05:55:18.295 INFO:teuthology.orchestra.run.smithi050.stdout:1: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-09-21T05:55:18.295 INFO:teuthology.orchestra.run.smithi050.stdout:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-09-21T05:55:18.295 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T05:55:18.298 DEBUG:teuthology.orchestra.run.smithi050:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T05:55:18.305 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3f44e77a-b7bc-4ef8-b058-60eaaa9147d6... 2024-09-21T05:55:18.306 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T05:55:18.350 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T05:55:18.390 INFO:teuthology.orchestra.run.smithi050.stdout:[global] 2024-09-21T05:55:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: chdir = "" 2024-09-21T05:55:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T05:55:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: auth supported = cephx 2024-09-21T05:55:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: filestore xattr use omap = true 2024-09-21T05:55:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: mon clock drift allowed = 1.000 2024-09-21T05:55:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: osd crush chooseleaf type = 0 2024-09-21T05:55:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: auth debug = true 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on old message = true 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on bug = true 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: mon pg warn max object skew = 0 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default size = 2 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow primary affinity = true 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow pg remap = true 2024-09-21T05:55:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on legacy crush tunables = false 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on crush straw calc version zero = false 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on no sortbitwise = false 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on osd down out interval zero = false 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on too few osds = false 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: mon_allow_pool_size_one = true 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: osd default data pool replay window = 5 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: mon allow pool delete = true 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: mon cluster log file level = debug 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: debug asserts on shutdown = true 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: mon health detail to clog = false 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: mon host = "172.21.15.50,172.21.15.88,[v2:172.21.15.88:3301,v1:172.21.15.88:6790]" 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on skipped message = False 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: fsid = 3f44e77a-b7bc-4ef8-b058-60eaaa9147d6 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout:[osd] 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: osd journal size = 100 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub load threshold = 5.0 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub max interval = 600 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: osd mclock profile = high_recovery_ops 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: osd recover clone overlap = true 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: osd recovery max chunk = 1048576 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug shutdown = true 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug op order = true 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug verify stray on activate = true 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug trim objects = true 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: osd open classes on start = true 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug pg log writeout = true 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: osd deep scrub update digest min age = 30 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: osd map max advance = 10 2024-09-21T05:55:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: journal zero on create = true 2024-09-21T05:55:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: filestore ondisk finisher threads = 3 2024-09-21T05:55:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: filestore apply finisher threads = 3 2024-09-21T05:55:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: bdev debug aio = true 2024-09-21T05:55:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug misdirected ops = true 2024-09-21T05:55:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: bdev async discard = True 2024-09-21T05:55:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: bdev enable discard = True 2024-09-21T05:55:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore allocator = bitmap 2024-09-21T05:55:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore block size = 96636764160 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore fsck on mount = True 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluefs = 1/20 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluestore = 1/20 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: debug osd = 20 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: debug rocksdb = 4/10 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd full ratio = 0.9 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd nearfull ratio = 0.8 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: osd failsafe full ratio = 0.95 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: osd objectstore = bluestore 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr] 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: debug mgr = 20 2024-09-21T05:55:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: mgr/telemetry/nag = false 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: debug client = 20 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: debug finisher = 20 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout:[mon] 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: debug paxos = 20 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: mon data avail warn = 5 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: mon mgr mkfs grace = 240 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:55:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd reporter subtree level = osd 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd prime pg temp = true 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: # 1m isn't quite enough 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: mon_down_mkfs_grace = 2m 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_filestore_osds = false 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: mon down mkfs grace = 300 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout:[client] 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: rgw cache enabled = true 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable ops log = true 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable usage log = true 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = "" 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.a] 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.b] 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.c] 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.a] 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.c] 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.b] 2024-09-21T05:55:18.399 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.d] 2024-09-21T05:55:18.400 INFO:teuthology.orchestra.run.smithi088.stdout:[global] 2024-09-21T05:55:18.400 INFO:teuthology.orchestra.run.smithi088.stdout: chdir = "" 2024-09-21T05:55:18.400 INFO:teuthology.orchestra.run.smithi088.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T05:55:18.400 INFO:teuthology.orchestra.run.smithi088.stdout: auth supported = cephx 2024-09-21T05:55:18.400 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.400 INFO:teuthology.orchestra.run.smithi088.stdout: filestore xattr use omap = true 2024-09-21T05:55:18.400 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.400 INFO:teuthology.orchestra.run.smithi088.stdout: mon clock drift allowed = 1.000 2024-09-21T05:55:18.400 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.400 INFO:teuthology.orchestra.run.smithi088.stdout: osd crush chooseleaf type = 0 2024-09-21T05:55:18.400 INFO:teuthology.orchestra.run.smithi088.stdout: auth debug = true 2024-09-21T05:55:18.400 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on old message = true 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on bug = true 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: mon pg warn max object skew = 0 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default size = 2 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow primary affinity = true 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow pg remap = true 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on legacy crush tunables = false 2024-09-21T05:55:18.401 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on crush straw calc version zero = false 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on no sortbitwise = false 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on osd down out interval zero = false 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on too few osds = false 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: mon_allow_pool_size_one = true 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: osd default data pool replay window = 5 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: mon allow pool delete = true 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.402 INFO:teuthology.orchestra.run.smithi088.stdout: mon cluster log file level = debug 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: debug asserts on shutdown = true 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: mon health detail to clog = false 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: mon host = "172.21.15.50,172.21.15.88,[v2:172.21.15.88:3301,v1:172.21.15.88:6790]" 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on skipped message = False 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: fsid = 3f44e77a-b7bc-4ef8-b058-60eaaa9147d6 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout:[osd] 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: osd journal size = 100 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub load threshold = 5.0 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub max interval = 600 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: osd mclock profile = high_recovery_ops 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.403 INFO:teuthology.orchestra.run.smithi088.stdout: osd recover clone overlap = true 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: osd recovery max chunk = 1048576 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug shutdown = true 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug op order = true 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug verify stray on activate = true 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug trim objects = true 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: osd open classes on start = true 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug pg log writeout = true 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: osd deep scrub update digest min age = 30 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: osd map max advance = 10 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: journal zero on create = true 2024-09-21T05:55:18.404 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.405 INFO:teuthology.orchestra.run.smithi088.stdout: filestore ondisk finisher threads = 3 2024-09-21T05:55:18.405 INFO:teuthology.orchestra.run.smithi088.stdout: filestore apply finisher threads = 3 2024-09-21T05:55:18.405 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.405 INFO:teuthology.orchestra.run.smithi088.stdout: bdev debug aio = true 2024-09-21T05:55:18.405 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug misdirected ops = true 2024-09-21T05:55:18.405 INFO:teuthology.orchestra.run.smithi088.stdout: bdev async discard = True 2024-09-21T05:55:18.405 INFO:teuthology.orchestra.run.smithi088.stdout: bdev enable discard = True 2024-09-21T05:55:18.405 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore allocator = bitmap 2024-09-21T05:55:18.405 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore block size = 96636764160 2024-09-21T05:55:18.405 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore fsck on mount = True 2024-09-21T05:55:18.405 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluefs = 1/20 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluestore = 1/20 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: debug osd = 20 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: debug rocksdb = 4/10 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd full ratio = 0.9 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd nearfull ratio = 0.8 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: osd failsafe full ratio = 0.95 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: osd objectstore = bluestore 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr] 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: debug mgr = 20 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2024-09-21T05:55:18.406 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: mgr/telemetry/nag = false 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: debug client = 20 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: debug finisher = 20 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout:[mon] 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: debug paxos = 20 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: mon data avail warn = 5 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: mon mgr mkfs grace = 240 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd reporter subtree level = osd 2024-09-21T05:55:18.407 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd prime pg temp = true 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: # 1m isn't quite enough 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: mon_down_mkfs_grace = 2m 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_filestore_osds = false 2024-09-21T05:55:18.408 INFO:teuthology.orchestra.run.smithi088.stdout: mon down mkfs grace = 300 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout:[client] 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout: rgw cache enabled = true 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable ops log = true 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable usage log = true 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout: pid file = "" 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.a] 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.b] 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.c] 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.a] 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.c] 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.b] 2024-09-21T05:55:18.409 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.d] 2024-09-21T05:55:18.411 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T05:55:18.412 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T05:55:18.520 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T05:55:18.520 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:18.520 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T05:55:18.529 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:18.529 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T05:55:18.583 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T05:55:18.583 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:18.583 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T05:55:18.583 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T05:55:18.660 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:18.702 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:18.712 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T05:55:18.712 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:18.712 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T05:55:18.712 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T05:55:18.747 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:18.747 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:18.800 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T05:55:18.800 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T05:55:18.801 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T05:55:18.866 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T05:55:18.872 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T05:55:18.943 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T05:55:18.961 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T05:55:18.961 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T05:55:19.027 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T05:55:19.032 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-21T05:55:19.052 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T05:55:19.160 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T05:55:19.165 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-21T05:55:19.185 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T05:55:19.254 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T05:55:19.259 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-21T05:55:19.279 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T05:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T05:55:19.388 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-21T05:55:19.407 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T05:55:19.407 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T05:55:19.463 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T05:55:19.481 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-21T05:55:19.534 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-21T05:55:19.551 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T05:55:19.603 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T05:55:19.620 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-21T05:55:19.671 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-21T05:55:19.688 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T05:55:19.688 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T05:55:19.689 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T05:55:19.708 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T05:55:19.708 INFO:tasks.ceph:role: osd.0 2024-09-21T05:55:19.708 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T05:55:19.709 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T05:55:19.771 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:55:19.771 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:55:19.771 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:55:19.771 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T05:55:19.772 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:55:19.772 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:55:19.772 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:55:19.772 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T05:55:19.772 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:55:19.772 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:55:19.795 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2024-09-21T05:55:19.798 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2024-09-21T05:55:19.798 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T05:55:19.869 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T05:55:19.927 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T05:55:19.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:55:19.928 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T05:55:19.990 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T05:55:19.990 INFO:tasks.ceph:role: osd.1 2024-09-21T05:55:19.990 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T05:55:19.991 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T05:55:20.050 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:55:20.050 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:55:20.050 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:55:20.050 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T05:55:20.050 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:55:20.050 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:55:20.051 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:55:20.051 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T05:55:20.051 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:55:20.051 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:55:20.074 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2024-09-21T05:55:20.077 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2024-09-21T05:55:20.077 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T05:55:20.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T05:55:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T05:55:20.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:55:20.208 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T05:55:20.275 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T05:55:20.275 INFO:tasks.ceph:role: osd.2 2024-09-21T05:55:20.275 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T05:55:20.276 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T05:55:20.334 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:55:20.335 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:55:20.335 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:55:20.335 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T05:55:20.335 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:55:20.335 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:55:20.335 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:55:20.335 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T05:55:20.335 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:55:20.335 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:55:20.361 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2024-09-21T05:55:20.365 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2024-09-21T05:55:20.365 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T05:55:20.436 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T05:55:20.499 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T05:55:20.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:55:20.501 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T05:55:20.563 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T05:55:20.563 INFO:tasks.ceph:role: osd.3 2024-09-21T05:55:20.564 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T05:55:20.564 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T05:55:20.622 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:55:20.622 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:55:20.622 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:55:20.623 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T05:55:20.623 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:55:20.623 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:55:20.623 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:55:20.623 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T05:55:20.623 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:55:20.623 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:55:20.646 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2024-09-21T05:55:20.649 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2024-09-21T05:55:20.649 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T05:55:20.716 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T05:55:20.775 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T05:55:20.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:55:20.776 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:20.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:20.911+0000 7f0c8da727c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T05:55:20.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:20.911+0000 7f0c8da727c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T05:55:20.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:20.911+0000 7f0c8da727c0 -1 bdev(0x5633e86a2a00 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T05:55:20.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:20.911+0000 7f0c8da727c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T05:55:24.130 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T05:55:24.152 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:24.263+0000 7fd15cb847c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T05:55:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:24.263+0000 7fd15cb847c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T05:55:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:24.263+0000 7fd15cb847c0 -1 bdev(0x55d62dd56a00 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T05:55:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:24.263+0000 7fd15cb847c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T05:55:27.437 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T05:55:27.458 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:27.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:27.559+0000 7f9f396547c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T05:55:27.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:27.559+0000 7f9f396547c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T05:55:27.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:27.559+0000 7f9f396547c0 -1 bdev(0x562baa9baa00 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T05:55:27.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:27.559+0000 7f9f396547c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T05:55:30.765 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T05:55:30.787 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:30.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:30.899+0000 7fbbfec957c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T05:55:30.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:30.899+0000 7fbbfec957c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T05:55:30.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:30.899+0000 7fbbfec957c0 -1 bdev(0x56273e1d4a00 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T05:55:30.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:55:30.899+0000 7fbbfec957c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T05:55:34.176 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T05:55:34.198 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T05:55:34.220 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T05:55:34.220 INFO:tasks.ceph:role: osd.4 2024-09-21T05:55:34.220 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T05:55:34.220 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T05:55:34.292 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:55:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:55:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:55:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T05:55:34.292 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:55:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:55:34.293 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:55:34.293 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T05:55:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:55:34.293 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:55:34.688 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-09-21T05:55:34.691 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-09-21T05:55:34.691 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T05:55:34.774 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T05:55:34.832 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T05:55:34.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:55:34.833 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T05:55:34.899 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T05:55:34.899 INFO:tasks.ceph:role: osd.5 2024-09-21T05:55:34.900 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T05:55:34.900 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T05:55:34.972 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:55:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:55:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:55:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T05:55:34.972 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:55:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:55:34.972 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:55:34.972 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T05:55:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:55:34.972 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:55:35.372 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-09-21T05:55:35.375 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-09-21T05:55:35.375 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T05:55:35.496 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T05:55:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T05:55:35.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:55:35.514 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T05:55:35.576 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T05:55:35.576 INFO:tasks.ceph:role: osd.6 2024-09-21T05:55:35.576 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T05:55:35.576 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T05:55:35.648 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:55:35.648 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:55:35.648 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:55:35.648 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T05:55:35.648 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:55:35.648 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:55:35.648 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:55:35.648 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T05:55:35.648 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:55:35.648 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:55:36.062 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-09-21T05:55:36.065 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-09-21T05:55:36.065 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T05:55:36.192 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T05:55:36.209 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T05:55:36.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:55:36.211 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T05:55:36.273 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T05:55:36.273 INFO:tasks.ceph:role: osd.7 2024-09-21T05:55:36.273 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T05:55:36.273 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T05:55:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:55:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:55:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:55:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T05:55:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:55:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:55:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:55:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T05:55:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:55:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:55:36.703 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-09-21T05:55:36.707 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-09-21T05:55:36.707 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T05:55:36.795 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T05:55:36.811 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T05:55:36.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:55:36.812 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:36.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:36.956+0000 7fcb54a4f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T05:55:36.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:36.960+0000 7fcb54a4f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T05:55:36.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:36.960+0000 7fcb54a4f7c0 -1 bdev(0x55ca9fbe0a00 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T05:55:36.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:36.960+0000 7fcb54a4f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T05:55:40.209 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T05:55:40.231 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:40.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:40.340+0000 7f6aa30ab7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T05:55:40.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:40.340+0000 7f6aa30ab7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T05:55:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:40.340+0000 7f6aa30ab7c0 -1 bdev(0x55e4b1f94a00 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T05:55:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:40.340+0000 7f6aa30ab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T05:55:43.520 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T05:55:43.540 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:43.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:43.644+0000 7fb54162b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T05:55:43.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:43.644+0000 7fb54162b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T05:55:43.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:43.644+0000 7fb54162b7c0 -1 bdev(0x55b8bb82ea00 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T05:55:43.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:43.644+0000 7fb54162b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T05:55:46.525 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:55:46.529 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:55:46.921 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T05:55:46.942 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:47.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:47.043+0000 7fc00576e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T05:55:47.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:47.043+0000 7fc00576e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T05:55:47.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:47.043+0000 7fc00576e7c0 -1 bdev(0x55d7533c0a00 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T05:55:47.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:55:47.043+0000 7fc00576e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T05:55:50.304 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T05:55:50.323 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T05:55:50.323 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:50.323 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T05:55:50.346 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:50.346 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-21T05:55:50.409 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:50.409 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-21T05:55:50.473 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:50.473 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T05:55:50.536 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:50.536 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T05:55:50.605 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:50.605 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T05:55:50.672 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:50.672 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T05:55:50.740 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:50.740 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T05:55:50.761 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:50.761 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-21T05:55:50.824 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:50.824 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-21T05:55:50.885 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:50.885 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T05:55:50.948 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:50.948 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T05:55:51.012 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:51.012 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T05:55:51.072 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:51.072 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T05:55:51.132 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:51.132 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T05:55:51.141 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:55:51.141 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-21T05:55:51.192 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:51.192 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T05:55:51.200 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:55:51.201 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-21T05:55:51.252 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T05:55:51.253 DEBUG:teuthology.orchestra.run.smithi050:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T05:55:51.256 DEBUG:teuthology.orchestra.run.smithi088:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T05:55:51.300 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr.y] 2024-09-21T05:55:51.300 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDGX+5mEjC/MxAAHgwIu4CLDgFajwD2rL5WKA== 2024-09-21T05:55:51.300 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.a] 2024-09-21T05:55:51.300 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mSKq0ARAA2egaHGYWhRUEq6oRl1CIMg== 2024-09-21T05:55:51.301 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.c] 2024-09-21T05:55:51.301 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5me6maCRAA4jvHgTLTKpb/mxzzG71SfQ== 2024-09-21T05:55:51.301 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.0] 2024-09-21T05:55:51.301 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDIX+5mrZF/NhAA33M6ummG4r4yXU8/YOL5WA== 2024-09-21T05:55:51.301 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.1] 2024-09-21T05:55:51.301 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDMX+5myVz5DxAAMYfTPep6ubD7Cxnfa9dMCg== 2024-09-21T05:55:51.301 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.2] 2024-09-21T05:55:51.302 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDPX+5mA7ahIRAAf/osnY5l83p8ZWna0ErD2Q== 2024-09-21T05:55:51.302 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.3] 2024-09-21T05:55:51.302 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDSX+5m+rDpNRAApGPZJIlAaA0DyjtPXPgoaQ== 2024-09-21T05:55:51.302 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr.x] 2024-09-21T05:55:51.302 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDGX+5m2F9WOBAAZuQTUyR7oTT01CRD1jo+/Q== 2024-09-21T05:55:51.302 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.b] 2024-09-21T05:55:51.303 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mwv0+DxAAY+eZPxv5uY9PnLl0asdr6w== 2024-09-21T05:55:51.303 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.d] 2024-09-21T05:55:51.303 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mzYnuFhAA/hEigjluXspzs/hyeWxBjQ== 2024-09-21T05:55:51.303 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.4] 2024-09-21T05:55:51.303 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDYX+5mKL9SORAA21ITQUwUSoDUjXaD0Cwb3g== 2024-09-21T05:55:51.303 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.5] 2024-09-21T05:55:51.304 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDcX+5minRqFBAAMJOBQNvJFD4TNdKbPMLBig== 2024-09-21T05:55:51.304 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.6] 2024-09-21T05:55:51.304 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDfX+5m6HadJhAAAv670Tvyi4IKMByFPiBlAw== 2024-09-21T05:55:51.304 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.7] 2024-09-21T05:55:51.304 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDjX+5muPrrAhAAuRB7Muamiht+f7LvSmI6UQ== 2024-09-21T05:55:51.304 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-09-21T05:55:51.305 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw== 2024-09-21T05:55:51.305 INFO:teuthology.orchestra.run.smithi050.stdout:[client.2] 2024-09-21T05:55:51.305 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5m4IrnHxAAFjER3ywuS+FlbuA7bPZVqw== 2024-09-21T05:55:51.305 INFO:teuthology.orchestra.run.smithi050.stdout:[client.1] 2024-09-21T05:55:51.305 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mo94GJBAAMBSb1NPb2L93vzvnAh59kw== 2024-09-21T05:55:51.305 INFO:teuthology.orchestra.run.smithi050.stdout:[client.3] 2024-09-21T05:55:51.306 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mysQeKBAAq5c4bD3hbsU1H7JabaMjwQ== 2024-09-21T05:55:51.314 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.y] 2024-09-21T05:55:51.314 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDGX+5mEjC/MxAAHgwIu4CLDgFajwD2rL5WKA== 2024-09-21T05:55:51.314 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.a] 2024-09-21T05:55:51.315 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDHX+5mSKq0ARAA2egaHGYWhRUEq6oRl1CIMg== 2024-09-21T05:55:51.315 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.c] 2024-09-21T05:55:51.315 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDHX+5me6maCRAA4jvHgTLTKpb/mxzzG71SfQ== 2024-09-21T05:55:51.315 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.0] 2024-09-21T05:55:51.316 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDIX+5mrZF/NhAA33M6ummG4r4yXU8/YOL5WA== 2024-09-21T05:55:51.316 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.1] 2024-09-21T05:55:51.316 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDMX+5myVz5DxAAMYfTPep6ubD7Cxnfa9dMCg== 2024-09-21T05:55:51.316 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.2] 2024-09-21T05:55:51.316 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDPX+5mA7ahIRAAf/osnY5l83p8ZWna0ErD2Q== 2024-09-21T05:55:51.316 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.3] 2024-09-21T05:55:51.317 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDSX+5m+rDpNRAApGPZJIlAaA0DyjtPXPgoaQ== 2024-09-21T05:55:51.317 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.x] 2024-09-21T05:55:51.317 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDGX+5m2F9WOBAAZuQTUyR7oTT01CRD1jo+/Q== 2024-09-21T05:55:51.317 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.b] 2024-09-21T05:55:51.317 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDHX+5mwv0+DxAAY+eZPxv5uY9PnLl0asdr6w== 2024-09-21T05:55:51.318 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.d] 2024-09-21T05:55:51.318 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDHX+5mzYnuFhAA/hEigjluXspzs/hyeWxBjQ== 2024-09-21T05:55:51.318 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.4] 2024-09-21T05:55:51.318 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDYX+5mKL9SORAA21ITQUwUSoDUjXaD0Cwb3g== 2024-09-21T05:55:51.318 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.5] 2024-09-21T05:55:51.319 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDcX+5minRqFBAAMJOBQNvJFD4TNdKbPMLBig== 2024-09-21T05:55:51.319 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.6] 2024-09-21T05:55:51.319 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDfX+5m6HadJhAAAv670Tvyi4IKMByFPiBlAw== 2024-09-21T05:55:51.320 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.7] 2024-09-21T05:55:51.320 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDjX+5muPrrAhAAuRB7Muamiht+f7LvSmI6UQ== 2024-09-21T05:55:51.320 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T05:55:51.320 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw== 2024-09-21T05:55:51.320 INFO:teuthology.orchestra.run.smithi088.stdout:[client.2] 2024-09-21T05:55:51.320 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDHX+5m4IrnHxAAFjER3ywuS+FlbuA7bPZVqw== 2024-09-21T05:55:51.320 INFO:teuthology.orchestra.run.smithi088.stdout:[client.1] 2024-09-21T05:55:51.321 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDHX+5mo94GJBAAMBSb1NPb2L93vzvnAh59kw== 2024-09-21T05:55:51.321 INFO:teuthology.orchestra.run.smithi088.stdout:[client.3] 2024-09-21T05:55:51.321 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDHX+5mysQeKBAAq5c4bD3hbsU1H7JabaMjwQ== 2024-09-21T05:55:51.322 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T05:55:51.350 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T05:55:51.432 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:55:51.435 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:55:51.505 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:55:51.508 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:55:51.576 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:51.579 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:51.643 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:51.646 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:51.711 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:51.714 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:51.778 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:51.781 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:51.845 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T05:55:51.849 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T05:55:51.914 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:55:51.918 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:55:51.985 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:55:51.989 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:55:52.056 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:52.060 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:52.124 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:52.128 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:52.193 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:52.196 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:52.260 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:52.263 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:55:52.327 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:55:52.330 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:55:52.396 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:55:52.399 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:55:52.466 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:55:52.469 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:55:52.537 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:55:52.540 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:55:52.606 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T05:55:52.607 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T05:55:52.626 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T05:55:52.752 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T05:55:52.772 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T05:55:52.791 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T05:55:52.915 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T05:55:52.935 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T05:55:52.999 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T05:55:53.118 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T05:55:53.138 DEBUG:teuthology.orchestra.run.smithi050:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:53.141 DEBUG:teuthology.orchestra.run.smithi088:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:55:53.192 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T05:55:53.193 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T05:55:53.193 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T05:55:53.196 INFO:tasks.ceph.mon.a:Started 2024-09-21T05:55:53.196 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T05:55:53.196 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T05:55:53.239 INFO:tasks.ceph.mon.b:Started 2024-09-21T05:55:53.239 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T05:55:53.239 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T05:55:53.242 INFO:tasks.ceph.mon.c:Started 2024-09-21T05:55:53.242 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T05:55:53.242 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T05:55:53.243 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T05:55:53.245 INFO:tasks.ceph.mgr.y:Started 2024-09-21T05:55:53.246 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T05:55:53.246 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T05:55:53.249 INFO:tasks.ceph.mgr.x:Started 2024-09-21T05:55:53.249 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-21T05:55:53.249 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-21T05:55:53.253 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-21T05:55:53.253 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-21T05:55:53.298 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-21T05:55:53.298 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-21T05:55:53.346 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-21T05:55:53.346 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-21T05:55:53.391 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-21T05:55:53.391 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-21T05:55:53.439 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-21T05:55:53.439 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-21T05:55:53.491 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-21T05:55:53.491 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-21T05:55:53.539 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-21T05:55:53.539 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-21T05:55:53.586 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-21T05:55:53.586 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-21T05:55:53.630 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-21T05:55:53.630 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-21T05:55:53.679 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-21T05:55:53.679 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-21T05:55:53.730 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-21T05:55:53.730 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-21T05:55:53.775 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-21T05:55:53.775 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-21T05:55:53.823 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T05:55:53.823 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-21T05:55:53.875 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T05:55:53.876 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-21T05:55:53.927 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-21T05:55:53.927 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-21T05:55:53.974 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-21T05:55:53.974 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-21T05:55:54.023 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-21T05:55:54.023 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-21T05:55:54.075 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-21T05:55:54.075 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-21T05:55:54.123 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T05:55:54.123 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T05:55:54.171 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-21T05:55:54.171 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-21T05:55:54.219 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-21T05:55:54.219 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-21T05:55:54.266 DEBUG:tasks.ceph:set 22 configs 2024-09-21T05:55:54.266 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.061+0000 7f5717200640 1 -- 172.21.15.50:0/1897025742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56f8004810 msgr2=0x7f571815c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.061+0000 7f5717200640 1 --2- 172.21.15.50:0/1897025742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56f8004810 0x7f571815c9d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5704025110 tx=0x7f56f8005840 comp rx=0 tx=0).stop 2024-09-21T05:56:04.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.061+0000 7f5717200640 1 -- 172.21.15.50:0/1897025742 shutdown_connections 2024-09-21T05:56:04.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.061+0000 7f5717200640 1 --2- 172.21.15.50:0/1897025742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56f8005340 0x7f571815cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.061+0000 7f5717200640 1 --2- 172.21.15.50:0/1897025742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56f8004810 0x7f571815c9d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.061+0000 7f5717200640 1 --2- 172.21.15.50:0/1897025742 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f56f80039c0 0x7f5718153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.061+0000 7f5717200640 1 -- 172.21.15.50:0/1897025742 >> 172.21.15.50:0/1897025742 conn(0x7f571806d3f0 msgr2=0x7f5718155570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.065+0000 7f5717200640 1 -- 172.21.15.50:0/1897025742 shutdown_connections 2024-09-21T05:56:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.065+0000 7f37f7e00640 1 -- 172.21.15.50:0/2922108455 <== mon.1 v2:172.21.15.88:3300/0 12 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v17)=0 v17) ==== 111+0+0 (secure 0 0 0) 0x7f37ec01a040 con 0x7f37e4005100 2024-09-21T05:56:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.065+0000 7f5717200640 1 -- 172.21.15.50:0/1897025742 wait complete. 2024-09-21T05:56:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.065+0000 7f098ac00640 1 -- 172.21.15.50:0/4012000531 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f0958005300 con 0x7f096c004da0 2024-09-21T05:56:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.065+0000 7fdbeca00640 1 -- 172.21.15.50:0/1037703210 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fdbbc005300 con 0x7fdbe809eeb0 2024-09-21T05:56:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.069+0000 7f3806e00640 1 -- 172.21.15.50:0/2922108455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37e4005100 msgr2=0x7f38001f80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.069+0000 7f3806e00640 1 --2- 172.21.15.50:0/2922108455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37e4005100 0x7f38001f80c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f38000d14d0 tx=0x7f37e4005840 comp rx=0 tx=0).stop 2024-09-21T05:56:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.069+0000 7f3806e00640 1 -- 172.21.15.50:0/2922108455 shutdown_connections 2024-09-21T05:56:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.069+0000 7f3806e00640 1 --2- 172.21.15.50:0/2922108455 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f38000afcb0 0x7f38001597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.069+0000 7f3806e00640 1 --2- 172.21.15.50:0/2922108455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37e4005100 0x7f38001f80c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.069+0000 7f3806e00640 1 --2- 172.21.15.50:0/2922108455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37e40039c0 0x7f3800153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.069+0000 7f3806e00640 1 -- 172.21.15.50:0/2922108455 >> 172.21.15.50:0/2922108455 conn(0x7f38000b35f0 msgr2=0x7f380006e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.069+0000 7f3806e00640 1 -- 172.21.15.50:0/2922108455 shutdown_connections 2024-09-21T05:56:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.069+0000 7f3806e00640 1 -- 172.21.15.50:0/2922108455 wait complete. 2024-09-21T05:56:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.073+0000 7f8524a00640 1 -- 172.21.15.50:0/92788404 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f84f4005300 con 0x7f85040039c0 2024-09-21T05:56:04.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.077+0000 7fa46f400640 1 -- 172.21.15.50:0/2360484006 <== mon.1 v2:172.21.15.88:3300/0 12 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v18)=0 v18) ==== 125+0+0 (secure 0 0 0) 0x7fa468012830 con 0x7fa45c003a10 2024-09-21T05:56:04.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.085+0000 7fa47e400640 1 -- 172.21.15.50:0/2360484006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c003a10 msgr2=0x7fa478140d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.085+0000 7fa47e400640 1 --2- 172.21.15.50:0/2360484006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c003a10 0x7fa478140d70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa468023fd0 tx=0x7fa468004d50 comp rx=0 tx=0).stop 2024-09-21T05:56:04.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.085+0000 7fa47e400640 1 -- 172.21.15.50:0/2360484006 shutdown_connections 2024-09-21T05:56:04.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.085+0000 7fa47e400640 1 --2- 172.21.15.50:0/2360484006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa45c0053e0 0x7fa4780b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.085+0000 7fa47e400640 1 --2- 172.21.15.50:0/2360484006 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa45c004810 0x7fa4780b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.085+0000 7fa47e400640 1 --2- 172.21.15.50:0/2360484006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c003a10 0x7fa478140d70 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa468023fd0 tx=0x7fa468004d50 comp rx=0 tx=0).stop 2024-09-21T05:56:04.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.085+0000 7fa47e400640 1 -- 172.21.15.50:0/2360484006 >> 172.21.15.50:0/2360484006 conn(0x7fa4780b35f0 msgr2=0x7fa4780b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.085+0000 7fa47e400640 1 -- 172.21.15.50:0/2360484006 shutdown_connections 2024-09-21T05:56:04.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.085+0000 7fa47e400640 1 -- 172.21.15.50:0/2360484006 wait complete. 2024-09-21T05:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.089+0000 7f0981600640 1 -- 172.21.15.50:0/4012000531 <== mon.0 v2:172.21.15.50:3300/0 12 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f097c020820 con 0x7f096c004da0 2024-09-21T05:56:04.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.093+0000 7f098ac00640 1 -- 172.21.15.50:0/4012000531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f096c004da0 msgr2=0x7f09841e93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.093+0000 7f098ac00640 1 --2- 172.21.15.50:0/4012000531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f096c004da0 0x7f09841e93d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f09840b2b50 tx=0x7f097c003e00 comp rx=0 tx=0).stop 2024-09-21T05:56:04.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7f098ac00640 1 -- 172.21.15.50:0/4012000531 shutdown_connections 2024-09-21T05:56:04.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7f098ac00640 1 --2- 172.21.15.50:0/4012000531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f096c004da0 0x7f09841e93d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7f098ac00640 1 --2- 172.21.15.50:0/4012000531 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f096c003fd0 0x7f09841e84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7f098ac00640 1 --2- 172.21.15.50:0/4012000531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f096400a0e0 0x7f09841576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7f098ac00640 1 -- 172.21.15.50:0/4012000531 >> 172.21.15.50:0/4012000531 conn(0x7f09840b41f0 msgr2=0x7f098415a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7fdbdea00640 1 -- 172.21.15.50:0/1037703210 <== mon.2 v2:172.21.15.88:3301/0 12 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v20)=0 v20) ==== 120+0+0 (secure 0 0 0) 0x7fdbe0026720 con 0x7fdbe809eeb0 2024-09-21T05:56:04.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7f098ac00640 1 -- 172.21.15.50:0/4012000531 shutdown_connections 2024-09-21T05:56:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7f098ac00640 1 -- 172.21.15.50:0/4012000531 wait complete. 2024-09-21T05:56:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7fdbeca00640 1 -- 172.21.15.50:0/1037703210 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdbe809eeb0 msgr2=0x7fdbe80af660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7fdbeca00640 1 --2- 172.21.15.50:0/1037703210 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdbe809eeb0 0x7fdbe80af660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8072fd0 tx=0x7fdbe0011ae0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7fdbeca00640 1 -- 172.21.15.50:0/1037703210 shutdown_connections 2024-09-21T05:56:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7fdbeca00640 1 --2- 172.21.15.50:0/1037703210 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdbe809eeb0 0x7fdbe80af660 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7fdbeca00640 1 --2- 172.21.15.50:0/1037703210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe8158350 0x7fdbe80af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7fdbeca00640 1 --2- 172.21.15.50:0/1037703210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc800f6f0 0x7fdbe815d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.097+0000 7fdbeca00640 1 -- 172.21.15.50:0/1037703210 >> 172.21.15.50:0/1037703210 conn(0x7fdbe80b41f0 msgr2=0x7fdbe8070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.101+0000 7fdbeca00640 1 -- 172.21.15.50:0/1037703210 shutdown_connections 2024-09-21T05:56:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.101+0000 7fdbeca00640 1 -- 172.21.15.50:0/1037703210 wait complete. 2024-09-21T05:56:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.105+0000 7f8516000640 1 -- 172.21.15.50:0/92788404 <== mon.1 v2:172.21.15.88:3300/0 12 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f8510020020 con 0x7f85040039c0 2024-09-21T05:56:04.110 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.110 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.113+0000 7f8524a00640 1 -- 172.21.15.50:0/92788404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85040039c0 msgr2=0x7f85201f3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.113+0000 7f8524a00640 1 --2- 172.21.15.50:0/92788404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85040039c0 0x7f85201f3cd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f85200713e0 tx=0x7f85100016c0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.113+0000 7f8524a00640 1 -- 172.21.15.50:0/92788404 shutdown_connections 2024-09-21T05:56:04.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.113+0000 7f8524a00640 1 --2- 172.21.15.50:0/92788404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8504005360 0x7f85200b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.113+0000 7f8524a00640 1 --2- 172.21.15.50:0/92788404 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8504004810 0x7f85201f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.113+0000 7f8524a00640 1 --2- 172.21.15.50:0/92788404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85040039c0 0x7f85201f3cd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.113+0000 7f8524a00640 1 -- 172.21.15.50:0/92788404 >> 172.21.15.50:0/92788404 conn(0x7f85200708c0 msgr2=0x7f8520070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.113+0000 7f8524a00640 1 -- 172.21.15.50:0/92788404 shutdown_connections 2024-09-21T05:56:04.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.113+0000 7f8524a00640 1 -- 172.21.15.50:0/92788404 wait complete. 2024-09-21T05:56:04.121 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.121 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.121 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.121 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.121 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.121 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.201+0000 7f0c6aa00640 1 -- 172.21.15.50:0/4044146716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c44012040 msgr2=0x7f0c44014550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.201+0000 7f0c6aa00640 1 --2- 172.21.15.50:0/4044146716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c44012040 0x7f0c44014550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c50005dd0 tx=0x7f0c5001e330 comp rx=0 tx=0).stop 2024-09-21T05:56:04.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.201+0000 7f0c68a00640 1 -- 172.21.15.50:0/4044146716 reap_dead start 2024-09-21T05:56:04.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.201+0000 7f0c6aa00640 1 -- 172.21.15.50:0/4044146716 shutdown_connections 2024-09-21T05:56:04.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.201+0000 7f0c6aa00640 1 --2- 172.21.15.50:0/4044146716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c44012040 0x7f0c44014550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.201+0000 7f0c6aa00640 1 -- 172.21.15.50:0/4044146716 >> 172.21.15.50:0/4044146716 conn(0x7f0c640b35f0 msgr2=0x7f0c640b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.201+0000 7f0c6aa00640 1 -- 172.21.15.50:0/4044146716 shutdown_connections 2024-09-21T05:56:04.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.201+0000 7f0c6aa00640 1 -- 172.21.15.50:0/4044146716 wait complete. 2024-09-21T05:56:04.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c6aa00640 1 Processor -- start 2024-09-21T05:56:04.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c6aa00640 1 -- start start 2024-09-21T05:56:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c6aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c44012040 0x7f0c641e8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c6aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c641f8830 0x7f0c640b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c6aa00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c641e8650 0x7f0c640b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c6aa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c640d0ab0 con 0x7f0c641f8830 2024-09-21T05:56:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c6aa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0c640d0930 con 0x7f0c44012040 2024-09-21T05:56:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c6aa00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0c640d0c30 con 0x7f0c641e8650 2024-09-21T05:56:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c63400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c641f8830 0x7f0c640b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c63400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c641f8830 0x7f0c640b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36610/0 (socket says 172.21.15.50:36610) 2024-09-21T05:56:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c63e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c44012040 0x7f0c641e8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c63400640 1 -- 172.21.15.50:0/2175584994 learned_addr learned my addr 172.21.15.50:0/2175584994 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:04.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c68a00640 1 --2- 172.21.15.50:0/2175584994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c641e8650 0x7f0c640b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.205+0000 7f0c63400640 1 -- 172.21.15.50:0/2175584994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c641e8650 msgr2=0x7f0c640b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c63400640 1 --2- 172.21.15.50:0/2175584994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c641e8650 0x7f0c640b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c63400640 1 -- 172.21.15.50:0/2175584994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c44012040 msgr2=0x7f0c641e8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c63400640 1 --2- 172.21.15.50:0/2175584994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c44012040 0x7f0c641e8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c63400640 1 -- 172.21.15.50:0/2175584994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c641e9470 con 0x7f0c641f8830 2024-09-21T05:56:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c63e00640 1 --2- 172.21.15.50:0/2175584994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c44012040 0x7f0c641e8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c63400640 1 --2- 172.21.15.50:0/2175584994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c641f8830 0x7f0c640b5d60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0c500af040 tx=0x7f0c50004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c5be00640 1 -- 172.21.15.50:0/2175584994 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c500a6e60 con 0x7f0c641f8830 2024-09-21T05:56:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c5be00640 1 -- 172.21.15.50:0/2175584994 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c500c0070 con 0x7f0c641f8830 2024-09-21T05:56:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c5be00640 1 -- 172.21.15.50:0/2175584994 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c500b7430 con 0x7f0c641f8830 2024-09-21T05:56:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c6aa00640 1 -- 172.21.15.50:0/2175584994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c641fc290 con 0x7f0c641f8830 2024-09-21T05:56:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c6aa00640 1 -- 172.21.15.50:0/2175584994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c641fc850 con 0x7f0c641f8830 2024-09-21T05:56:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c5be00640 1 -- 172.21.15.50:0/2175584994 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0c500b75d0 con 0x7f0c641f8830 2024-09-21T05:56:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c5be00640 1 -- 172.21.15.50:0/2175584994 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0c500c5460 con 0x7f0c641f8830 2024-09-21T05:56:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.209+0000 7f0c6aa00640 1 -- 172.21.15.50:0/2175584994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6406cdf0 con 0x7f0c641f8830 2024-09-21T05:56:04.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.213+0000 7f0c5be00640 1 -- 172.21.15.50:0/2175584994 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0c50018150 con 0x7f0c641f8830 2024-09-21T05:56:04.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.273+0000 7f0c6aa00640 1 -- 172.21.15.50:0/2175584994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f0c640b1070 con 0x7f0c641f8830 2024-09-21T05:56:04.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.277+0000 7f0c5be00640 1 -- 172.21.15.50:0/2175584994 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v22)=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f0c500b7aa0 con 0x7f0c641f8830 2024-09-21T05:56:04.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.277+0000 7f0c6aa00640 1 -- 172.21.15.50:0/2175584994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c641f8830 msgr2=0x7f0c640b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.277+0000 7f0c6aa00640 1 --2- 172.21.15.50:0/2175584994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c641f8830 0x7f0c640b5d60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0c500af040 tx=0x7f0c50004df0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.277+0000 7f0c6aa00640 1 -- 172.21.15.50:0/2175584994 shutdown_connections 2024-09-21T05:56:04.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.277+0000 7f0c6aa00640 1 --2- 172.21.15.50:0/2175584994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c641e8650 0x7f0c640b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.277+0000 7f0c6aa00640 1 --2- 172.21.15.50:0/2175584994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c641f8830 0x7f0c640b5d60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.277+0000 7f0c6aa00640 1 --2- 172.21.15.50:0/2175584994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c44012040 0x7f0c641e8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.277+0000 7f0c6aa00640 1 -- 172.21.15.50:0/2175584994 >> 172.21.15.50:0/2175584994 conn(0x7f0c640b35f0 msgr2=0x7f0c6415e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.277+0000 7f0c6aa00640 1 -- 172.21.15.50:0/2175584994 shutdown_connections 2024-09-21T05:56:04.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.277+0000 7f0c6aa00640 1 -- 172.21.15.50:0/2175584994 wait complete. 2024-09-21T05:56:04.301 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.301 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.301 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.345+0000 7ffb06c00640 1 -- 172.21.15.50:0/395938867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffae0006dc0 msgr2=0x7ffae0011420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.345+0000 7ffb06c00640 1 --2- 172.21.15.50:0/395938867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffae0006dc0 0x7ffae0011420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0005fd0 tx=0x7ffaf001e3d0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.345+0000 7ffb04c00640 1 -- 172.21.15.50:0/395938867 reap_dead start 2024-09-21T05:56:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.345+0000 7ffb06c00640 1 -- 172.21.15.50:0/395938867 shutdown_connections 2024-09-21T05:56:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.345+0000 7ffb06c00640 1 --2- 172.21.15.50:0/395938867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffae0006dc0 0x7ffae0011420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.345+0000 7ffb06c00640 1 -- 172.21.15.50:0/395938867 >> 172.21.15.50:0/395938867 conn(0x7ffb000708c0 msgr2=0x7ffb00070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.345+0000 7ffb06c00640 1 -- 172.21.15.50:0/395938867 shutdown_connections 2024-09-21T05:56:04.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.345+0000 7ffb06c00640 1 -- 172.21.15.50:0/395938867 wait complete. 2024-09-21T05:56:04.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.349+0000 7ffb06c00640 1 Processor -- start 2024-09-21T05:56:04.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.349+0000 7ffb06c00640 1 -- start start 2024-09-21T05:56:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.349+0000 7ffb06c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffae0006dc0 0x7ffb0013ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.349+0000 7ffb06c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb001f3460 0x7ffb001f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.349+0000 7ffb06c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb00147910 0x7ffb00147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.349+0000 7ffb06c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb000b5e10 con 0x7ffae0006dc0 2024-09-21T05:56:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.349+0000 7ffb06c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb000b5c90 con 0x7ffb00147910 2024-09-21T05:56:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.349+0000 7ffb06c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ffb000b5f90 con 0x7ffb001f3460 2024-09-21T05:56:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.349+0000 7ffaffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffae0006dc0 0x7ffb0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.349+0000 7ffaffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffae0006dc0 0x7ffb0013ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36628/0 (socket says 172.21.15.50:36628) 2024-09-21T05:56:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.349+0000 7ffaffe00640 1 -- 172.21.15.50:0/1063551335 learned_addr learned my addr 172.21.15.50:0/1063551335 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffaff400640 1 --2- 172.21.15.50:0/1063551335 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb001f3460 0x7ffb001f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffb04c00640 1 --2- 172.21.15.50:0/1063551335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb00147910 0x7ffb00147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffaffe00640 1 -- 172.21.15.50:0/1063551335 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb001f3460 msgr2=0x7ffb001f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffaffe00640 1 --2- 172.21.15.50:0/1063551335 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb001f3460 0x7ffb001f3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffaffe00640 1 -- 172.21.15.50:0/1063551335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb00147910 msgr2=0x7ffb00147d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffaffe00640 1 --2- 172.21.15.50:0/1063551335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb00147910 0x7ffb00147d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffaffe00640 1 -- 172.21.15.50:0/1063551335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0013b3c0 con 0x7ffae0006dc0 2024-09-21T05:56:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffaffe00640 1 --2- 172.21.15.50:0/1063551335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffae0006dc0 0x7ffb0013ae80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00b3040 tx=0x7ffaf0005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffaf7e00640 1 -- 172.21.15.50:0/1063551335 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00c0070 con 0x7ffae0006dc0 2024-09-21T05:56:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffaf7e00640 1 -- 172.21.15.50:0/1063551335 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaf0013140 con 0x7ffae0006dc0 2024-09-21T05:56:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffaf7e00640 1 -- 172.21.15.50:0/1063551335 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00b5070 con 0x7ffae0006dc0 2024-09-21T05:56:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffb06c00640 1 -- 172.21.15.50:0/1063551335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb00148260 con 0x7ffae0006dc0 2024-09-21T05:56:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffb06c00640 1 -- 172.21.15.50:0/1063551335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb0013e780 con 0x7ffae0006dc0 2024-09-21T05:56:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffaf7e00640 1 -- 172.21.15.50:0/1063551335 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffaf00b0b50 con 0x7ffae0006dc0 2024-09-21T05:56:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffaf7e00640 1 -- 172.21.15.50:0/1063551335 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffaf00bb480 con 0x7ffae0006dc0 2024-09-21T05:56:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.353+0000 7ffb06c00640 1 -- 172.21.15.50:0/1063551335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb000746b0 con 0x7ffae0006dc0 2024-09-21T05:56:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.357+0000 7ffaf7e00640 1 -- 172.21.15.50:0/1063551335 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ffaf00c9080 con 0x7ffae0006dc0 2024-09-21T05:56:04.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.417+0000 7ffb06c00640 1 -- 172.21.15.50:0/1063551335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7ffb001558e0 con 0x7ffae0006dc0 2024-09-21T05:56:04.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.421+0000 7ffaf7e00640 1 -- 172.21.15.50:0/1063551335 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v23)=0 v23) ==== 119+0+0 (secure 0 0 0) 0x7ffb00148260 con 0x7ffae0006dc0 2024-09-21T05:56:04.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.421+0000 7ffb06c00640 1 -- 172.21.15.50:0/1063551335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffae0006dc0 msgr2=0x7ffb0013ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.421+0000 7ffb06c00640 1 --2- 172.21.15.50:0/1063551335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffae0006dc0 0x7ffb0013ae80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00b3040 tx=0x7ffaf0005a10 comp rx=0 tx=0).stop 2024-09-21T05:56:04.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.421+0000 7ffb06c00640 1 -- 172.21.15.50:0/1063551335 shutdown_connections 2024-09-21T05:56:04.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.421+0000 7ffb06c00640 1 --2- 172.21.15.50:0/1063551335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb00147910 0x7ffb00147d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.421+0000 7ffb06c00640 1 --2- 172.21.15.50:0/1063551335 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb001f3460 0x7ffb001f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.421+0000 7ffb06c00640 1 --2- 172.21.15.50:0/1063551335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffae0006dc0 0x7ffb0013ae80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.421+0000 7ffb06c00640 1 -- 172.21.15.50:0/1063551335 >> 172.21.15.50:0/1063551335 conn(0x7ffb000708c0 msgr2=0x7ffb0015fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.421+0000 7ffb06c00640 1 -- 172.21.15.50:0/1063551335 shutdown_connections 2024-09-21T05:56:04.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.421+0000 7ffb06c00640 1 -- 172.21.15.50:0/1063551335 wait complete. 2024-09-21T05:56:04.445 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.445 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.445 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.446 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.446 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.446 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.446 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.446 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.446 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.446 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi050') 2024-09-21T05:56:04.446 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config dump 2024-09-21T05:56:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.597+0000 7f7914200640 1 -- 172.21.15.50:0/4204511192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f790c0b57c0 msgr2=0x7f790c15f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.597+0000 7f7914200640 1 --2- 172.21.15.50:0/4204511192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f790c0b57c0 0x7f790c15f250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f790c0b0860 tx=0x7f790401cb80 comp rx=0 tx=0).stop 2024-09-21T05:56:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.597+0000 7f7914200640 1 -- 172.21.15.50:0/4204511192 shutdown_connections 2024-09-21T05:56:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.597+0000 7f7914200640 1 --2- 172.21.15.50:0/4204511192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f790c0b57c0 0x7f790c15f250 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.597+0000 7f7914200640 1 --2- 172.21.15.50:0/4204511192 >> v2:172.21.15.88:3300/0 conn(0x7f790c074210 0x7f790c15b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.597+0000 7f7914200640 1 -- 172.21.15.50:0/4204511192 >> 172.21.15.50:0/4204511192 conn(0x7f790c0b3b30 msgr2=0x7f790c0733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.597+0000 7f7914200640 1 -- 172.21.15.50:0/4204511192 shutdown_connections 2024-09-21T05:56:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.597+0000 7f7914200640 1 -- 172.21.15.50:0/4204511192 wait complete. 2024-09-21T05:56:04.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7914200640 1 Processor -- start 2024-09-21T05:56:04.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7914200640 1 -- start start 2024-09-21T05:56:04.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7914200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f790c0b57c0 0x7f790c15bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:04.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7914200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f790c0cf4a0 0x7f790c14fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:04.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7914200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f790c153230 0x7f790c1f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7914200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f790c168db0 con 0x7f790c0b57c0 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7914200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f790c168c30 con 0x7f790c0cf4a0 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7914200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f790c168f30 con 0x7f790c153230 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7911800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f790c0b57c0 0x7f790c15bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7911800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f790c0b57c0 0x7f790c15bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36640/0 (socket says 172.21.15.50:36640) 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7911800640 1 -- 172.21.15.50:0/3419061697 learned_addr learned my addr 172.21.15.50:0/3419061697 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7910e00640 1 --2- 172.21.15.50:0/3419061697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f790c0cf4a0 0x7f790c14fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7912200640 1 --2- 172.21.15.50:0/3419061697 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f790c153230 0x7f790c1f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7911800640 1 -- 172.21.15.50:0/3419061697 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f790c153230 msgr2=0x7f790c1f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7911800640 1 --2- 172.21.15.50:0/3419061697 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f790c153230 0x7f790c1f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7911800640 1 -- 172.21.15.50:0/3419061697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f790c0cf4a0 msgr2=0x7f790c14fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7911800640 1 --2- 172.21.15.50:0/3419061697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f790c0cf4a0 0x7f790c14fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7911800640 1 -- 172.21.15.50:0/3419061697 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c1508d0 con 0x7f790c0b57c0 2024-09-21T05:56:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7910e00640 1 --2- 172.21.15.50:0/3419061697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f790c0cf4a0 0x7f790c14fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:04.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7911800640 1 --2- 172.21.15.50:0/3419061697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f790c0b57c0 0x7f790c15bc30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f790000c400 tx=0x7f790000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:04.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7909600640 1 -- 172.21.15.50:0/3419061697 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900011790 con 0x7f790c0b57c0 2024-09-21T05:56:04.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7909600640 1 -- 172.21.15.50:0/3419061697 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7900017070 con 0x7f790c0b57c0 2024-09-21T05:56:04.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7909600640 1 -- 172.21.15.50:0/3419061697 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900011cc0 con 0x7f790c0b57c0 2024-09-21T05:56:04.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7914200640 1 -- 172.21.15.50:0/3419061697 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790c1fa4a0 con 0x7f790c0b57c0 2024-09-21T05:56:04.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7914200640 1 -- 172.21.15.50:0/3419061697 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790c1faa60 con 0x7f790c0b57c0 2024-09-21T05:56:04.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.601+0000 7f7909600640 1 -- 172.21.15.50:0/3419061697 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f790001c460 con 0x7f790c0b57c0 2024-09-21T05:56:04.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.605+0000 7f7909600640 1 -- 172.21.15.50:0/3419061697 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f79000127d0 con 0x7f790c0b57c0 2024-09-21T05:56:04.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.605+0000 7f7914200640 1 -- 172.21.15.50:0/3419061697 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f790c1faca0 con 0x7f790c0b57c0 2024-09-21T05:56:04.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.605+0000 7f7909600640 1 -- 172.21.15.50:0/3419061697 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f790c1faca0 con 0x7f790c0b57c0 2024-09-21T05:56:04.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.669+0000 7f7914200640 1 -- 172.21.15.50:0/3419061697 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f790c15b570 con 0x7f790c0b57c0 2024-09-21T05:56:04.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.669+0000 7f7909600640 1 -- 172.21.15.50:0/3419061697 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f79000129d0 con 0x7f790c0b57c0 2024-09-21T05:56:04.674 INFO:teuthology.orchestra.run.smithi050.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T05:56:04.674 INFO:teuthology.orchestra.run.smithi050.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T05:56:04.674 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced client_mount_timeout 30 2024-09-21T05:56:04.674 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_client 20/20 2024-09-21T05:56:04.674 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_mgr 20/20 2024-09-21T05:56:04.674 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_ms 1/1 2024-09-21T05:56:04.674 INFO:teuthology.orchestra.run.smithi050.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds 20/20 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_ms 2/2 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_frag true 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_scatterstat true 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_verify_scatter true 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced client_mount_timeout 600 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_client 20/20 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_ms 1/1 2024-09-21T05:56:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T05:56:04.676 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T05:56:04.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.669+0000 7f7914200640 1 -- 172.21.15.50:0/3419061697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f790c0b57c0 msgr2=0x7f790c15bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.669+0000 7f7914200640 1 --2- 172.21.15.50:0/3419061697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f790c0b57c0 0x7f790c15bc30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f790000c400 tx=0x7f790000c8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.669+0000 7f7914200640 1 -- 172.21.15.50:0/3419061697 shutdown_connections 2024-09-21T05:56:04.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.669+0000 7f7914200640 1 --2- 172.21.15.50:0/3419061697 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f790c153230 0x7f790c1f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.669+0000 7f7914200640 1 --2- 172.21.15.50:0/3419061697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f790c0cf4a0 0x7f790c14fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.669+0000 7f7914200640 1 --2- 172.21.15.50:0/3419061697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f790c0b57c0 0x7f790c15bc30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.669+0000 7f7914200640 1 -- 172.21.15.50:0/3419061697 >> 172.21.15.50:0/3419061697 conn(0x7f790c0b3b30 msgr2=0x7f790c073e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.669+0000 7f7914200640 1 -- 172.21.15.50:0/3419061697 shutdown_connections 2024-09-21T05:56:04.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.669+0000 7f7914200640 1 -- 172.21.15.50:0/3419061697 wait complete. 2024-09-21T05:56:04.694 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T05:56:04.694 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T05:56:04.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.845+0000 7f8be4800640 1 -- 172.21.15.50:0/207493883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bdc157030 msgr2=0x7f8bdc0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.845+0000 7f8be4800640 1 --2- 172.21.15.50:0/207493883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bdc157030 0x7f8bdc0b5a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0b1740 tx=0x7f8bd001c830 comp rx=0 tx=0).stop 2024-09-21T05:56:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.845+0000 7f8be4800640 1 -- 172.21.15.50:0/207493883 shutdown_connections 2024-09-21T05:56:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.845+0000 7f8be4800640 1 --2- 172.21.15.50:0/207493883 >> v2:172.21.15.88:3300/0 conn(0x7f8bdc160d50 0x7f8bdc074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.845+0000 7f8be4800640 1 --2- 172.21.15.50:0/207493883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bdc157030 0x7f8bdc0b5a80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.845+0000 7f8be4800640 1 -- 172.21.15.50:0/207493883 >> 172.21.15.50:0/207493883 conn(0x7f8bdc0708c0 msgr2=0x7f8bdc070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.845+0000 7f8be4800640 1 -- 172.21.15.50:0/207493883 shutdown_connections 2024-09-21T05:56:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.845+0000 7f8be4800640 1 -- 172.21.15.50:0/207493883 wait complete. 2024-09-21T05:56:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be4800640 1 Processor -- start 2024-09-21T05:56:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be4800640 1 -- start start 2024-09-21T05:56:04.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be4800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8bdc157030 0x7f8bdc14aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:04.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be4800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bdc160d50 0x7f8bdc06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:04.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be4800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bdc161160 0x7f8bdc150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:04.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be4800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc16a740 con 0x7f8bdc160d50 2024-09-21T05:56:04.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be4800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc16a5c0 con 0x7f8bdc161160 2024-09-21T05:56:04.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be4800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8bdc16a8c0 con 0x7f8bdc157030 2024-09-21T05:56:04.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bdc160d50 0x7f8bdc06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:04.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bdc160d50 0x7f8bdc06e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36660/0 (socket says 172.21.15.50:36660) 2024-09-21T05:56:04.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be1400640 1 -- 172.21.15.50:0/1635653053 learned_addr learned my addr 172.21.15.50:0/1635653053 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:04.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be1e00640 1 --2- 172.21.15.50:0/1635653053 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8bdc157030 0x7f8bdc14aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:04.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be2800640 1 --2- 172.21.15.50:0/1635653053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bdc161160 0x7f8bdc150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:04.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be1400640 1 -- 172.21.15.50:0/1635653053 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8bdc157030 msgr2=0x7f8bdc14aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be1400640 1 --2- 172.21.15.50:0/1635653053 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8bdc157030 0x7f8bdc14aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be1400640 1 -- 172.21.15.50:0/1635653053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bdc161160 msgr2=0x7f8bdc150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be1400640 1 --2- 172.21.15.50:0/1635653053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bdc161160 0x7f8bdc150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be1400640 1 -- 172.21.15.50:0/1635653053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc06cf90 con 0x7f8bdc160d50 2024-09-21T05:56:04.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.849+0000 7f8be1400640 1 --2- 172.21.15.50:0/1635653053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bdc160d50 0x7f8bdc06e3f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00e7f0 tx=0x7f8bcc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:04.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.853+0000 7f8bd6000640 1 -- 172.21.15.50:0/1635653053 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00ccc0 con 0x7f8bdc160d50 2024-09-21T05:56:04.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.853+0000 7f8bd6000640 1 -- 172.21.15.50:0/1635653053 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bcc00ce80 con 0x7f8bdc160d50 2024-09-21T05:56:04.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.853+0000 7f8bd6000640 1 -- 172.21.15.50:0/1635653053 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc005280 con 0x7f8bdc160d50 2024-09-21T05:56:04.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.853+0000 7f8be4800640 1 -- 172.21.15.50:0/1635653053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bdc06d230 con 0x7f8bdc160d50 2024-09-21T05:56:04.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.853+0000 7f8be4800640 1 -- 172.21.15.50:0/1635653053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bdc06d7f0 con 0x7f8bdc160d50 2024-09-21T05:56:04.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.853+0000 7f8bd6000640 1 -- 172.21.15.50:0/1635653053 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8bcc0047b0 con 0x7f8bdc160d50 2024-09-21T05:56:04.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.853+0000 7f8bd6000640 1 -- 172.21.15.50:0/1635653053 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8bcc018430 con 0x7f8bdc160d50 2024-09-21T05:56:04.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.853+0000 7f8be4800640 1 -- 172.21.15.50:0/1635653053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bdc074640 con 0x7f8bdc160d50 2024-09-21T05:56:04.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.853+0000 7f8bd6000640 1 -- 172.21.15.50:0/1635653053 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8bcc004c30 con 0x7f8bdc160d50 2024-09-21T05:56:04.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.921+0000 7f8be4800640 1 -- 172.21.15.50:0/1635653053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8bdc0d2260 con 0x7f8bdc160d50 2024-09-21T05:56:04.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.921+0000 7f8bd6000640 1 -- 172.21.15.50:0/1635653053 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f8bcc004e10 con 0x7f8bdc160d50 2024-09-21T05:56:04.928 INFO:teuthology.orchestra.run.smithi050.stderr:adjusted tunables profile to default 2024-09-21T05:56:04.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.925+0000 7f8be4800640 1 -- 172.21.15.50:0/1635653053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bdc160d50 msgr2=0x7f8bdc06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:04.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.925+0000 7f8be4800640 1 --2- 172.21.15.50:0/1635653053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bdc160d50 0x7f8bdc06e3f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00e7f0 tx=0x7f8bcc00ecb0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.925+0000 7f8be4800640 1 -- 172.21.15.50:0/1635653053 shutdown_connections 2024-09-21T05:56:04.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.925+0000 7f8be4800640 1 --2- 172.21.15.50:0/1635653053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bdc161160 0x7f8bdc150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.925+0000 7f8be4800640 1 --2- 172.21.15.50:0/1635653053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bdc160d50 0x7f8bdc06e3f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.925+0000 7f8be4800640 1 --2- 172.21.15.50:0/1635653053 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8bdc157030 0x7f8bdc14aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:04.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.925+0000 7f8be4800640 1 -- 172.21.15.50:0/1635653053 >> 172.21.15.50:0/1635653053 conn(0x7f8bdc0708c0 msgr2=0x7f8bdc15fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:04.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.925+0000 7f8be4800640 1 -- 172.21.15.50:0/1635653053 shutdown_connections 2024-09-21T05:56:04.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:04.925+0000 7f8be4800640 1 -- 172.21.15.50:0/1635653053 wait complete. 2024-09-21T05:56:04.948 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T05:56:04.948 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T05:56:04.948 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:56:04.948 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T05:56:04.968 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:56:04.968 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T05:56:05.027 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:56:05.027 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T05:56:05.087 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:56:05.087 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T05:56:05.147 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:56:05.147 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T05:56:05.169 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:56:05.170 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T05:56:05.233 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:56:05.233 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T05:56:05.297 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:56:05.297 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T05:56:05.357 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 62980dcb-fdf5-40fb-a2a6-a456110d5727 0 2024-09-21T05:56:05.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.550+0000 7f28b6e00640 1 -- 172.21.15.88:0/3927251698 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28b0157030 msgr2=0x7f28b00b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.550+0000 7f28b6e00640 1 --2- 172.21.15.88:0/3927251698 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28b0157030 0x7f28b00b5a80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009900 tx=0x7f28a001c8e0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.550+0000 7f28b6e00640 1 -- 172.21.15.88:0/3927251698 shutdown_connections 2024-09-21T05:56:05.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.550+0000 7f28b6e00640 1 --2- 172.21.15.88:0/3927251698 >> v2:172.21.15.50:3300/0 conn(0x7f28b0074b20 0x7f28b015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.550+0000 7f28b6e00640 1 --2- 172.21.15.88:0/3927251698 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28b0157030 0x7f28b00b5a80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.550+0000 7f28b6e00640 1 -- 172.21.15.88:0/3927251698 >> 172.21.15.88:0/3927251698 conn(0x7f28b00708c0 msgr2=0x7f28b0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:05.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.550+0000 7f28b6e00640 1 -- 172.21.15.88:0/3927251698 shutdown_connections 2024-09-21T05:56:05.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.550+0000 7f28b6e00640 1 -- 172.21.15.88:0/3927251698 wait complete. 2024-09-21T05:56:05.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.550+0000 7f28b6e00640 1 Processor -- start 2024-09-21T05:56:05.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.550+0000 7f28b6e00640 1 -- start start 2024-09-21T05:56:05.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28b6e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28b0157030 0x7f28b013dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28b6e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b0160d50 0x7f28b0155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28b6e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28b00b4c20 0x7f28b0155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28b6e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f28b016a820 con 0x7f28b00b4c20 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28b6e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28b016a6a0 con 0x7f28b0160d50 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28b6e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f28b016a9a0 con 0x7f28b0157030 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28afe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28b0157030 0x7f28b013dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28afe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28b0157030 0x7f28b013dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.88:49038/0 (socket says 172.21.15.88:49038) 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28afe00640 1 -- 172.21.15.88:0/3405608505 learned_addr learned my addr 172.21.15.88:0/3405608505 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28af400640 1 --2- 172.21.15.88:0/3405608505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b0160d50 0x7f28b0155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28b4e00640 1 --2- 172.21.15.88:0/3405608505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28b00b4c20 0x7f28b0155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28af400640 1 -- 172.21.15.88:0/3405608505 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28b0157030 msgr2=0x7f28b013dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28af400640 1 --2- 172.21.15.88:0/3405608505 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28b0157030 0x7f28b013dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28af400640 1 -- 172.21.15.88:0/3405608505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28b00b4c20 msgr2=0x7f28b0155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28af400640 1 --2- 172.21.15.88:0/3405608505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28b00b4c20 0x7f28b0155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28af400640 1 -- 172.21.15.88:0/3405608505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b01fca90 con 0x7f28b0160d50 2024-09-21T05:56:05.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28b4e00640 1 --2- 172.21.15.88:0/3405608505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28b00b4c20 0x7f28b0155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:05.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.554+0000 7f28af400640 1 --2- 172.21.15.88:0/3405608505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b0160d50 0x7f28b0155a60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f28b00751f0 tx=0x7f289c00c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.558+0000 7f28a7e00640 1 -- 172.21.15.88:0/3405608505 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c017070 con 0x7f28b0160d50 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.558+0000 7f28a7e00640 1 -- 172.21.15.88:0/3405608505 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f289c0053c0 con 0x7f28b0160d50 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.558+0000 7f28a7e00640 1 -- 172.21.15.88:0/3405608505 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c003840 con 0x7f28b0160d50 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.558+0000 7f28b6e00640 1 -- 172.21.15.88:0/3405608505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b01fcd70 con 0x7f28b0160d50 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.558+0000 7f28b6e00640 1 -- 172.21.15.88:0/3405608505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b01fd330 con 0x7f28b0160d50 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.558+0000 7f28a7e00640 1 -- 172.21.15.88:0/3405608505 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f289c01a070 con 0x7f28b0160d50 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.558+0000 7f28a7e00640 1 -- 172.21.15.88:0/3405608505 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f289c01c440 con 0x7f28b0160d50 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.558+0000 7f28b6e00640 1 -- 172.21.15.88:0/3405608505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b015cf90 con 0x7f28b0160d50 2024-09-21T05:56:05.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.562+0000 7f28a7e00640 1 -- 172.21.15.88:0/3405608505 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f289c021020 con 0x7f28b0160d50 2024-09-21T05:56:05.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.630+0000 7f28b6e00640 1 -- 172.21.15.88:0/3405608505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd new", "uuid": "62980dcb-fdf5-40fb-a2a6-a456110d5727", "id": 0} v 0) -- 0x7f28b0075e30 con 0x7f28b0160d50 2024-09-21T05:56:05.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.634+0000 7f28a7e00640 1 -- 172.21.15.88:0/3405608505 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "62980dcb-fdf5-40fb-a2a6-a456110d5727", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f289c011590 con 0x7f28b0160d50 2024-09-21T05:56:05.636 INFO:teuthology.orchestra.run.smithi088.stdout:0 2024-09-21T05:56:05.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.634+0000 7f28b6e00640 1 -- 172.21.15.88:0/3405608505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b0160d50 msgr2=0x7f28b0155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.634+0000 7f28b6e00640 1 --2- 172.21.15.88:0/3405608505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b0160d50 0x7f28b0155a60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f28b00751f0 tx=0x7f289c00c820 comp rx=0 tx=0).stop 2024-09-21T05:56:05.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.634+0000 7f28b6e00640 1 -- 172.21.15.88:0/3405608505 shutdown_connections 2024-09-21T05:56:05.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.634+0000 7f28b6e00640 1 --2- 172.21.15.88:0/3405608505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28b00b4c20 0x7f28b0155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.634+0000 7f28b6e00640 1 --2- 172.21.15.88:0/3405608505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b0160d50 0x7f28b0155a60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.634+0000 7f28b6e00640 1 --2- 172.21.15.88:0/3405608505 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28b0157030 0x7f28b013dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.634+0000 7f28b6e00640 1 -- 172.21.15.88:0/3405608505 >> 172.21.15.88:0/3405608505 conn(0x7f28b00708c0 msgr2=0x7f28b014fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:05.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.634+0000 7f28b6e00640 1 -- 172.21.15.88:0/3405608505 shutdown_connections 2024-09-21T05:56:05.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.634+0000 7f28b6e00640 1 -- 172.21.15.88:0/3405608505 wait complete. 2024-09-21T05:56:05.655 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 56c27f67-47d4-4947-9c33-de2f8289a275 1 2024-09-21T05:56:05.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.806+0000 7f8d1a200640 1 -- 172.21.15.88:0/919267348 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d1415f1e0 msgr2=0x7f8d140b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.806+0000 7f8d1a200640 1 --2- 172.21.15.88:0/919267348 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d1415f1e0 0x7f8d140b3d50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009870 tx=0x7f8d0001c7d0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.806+0000 7f8d1a200640 1 -- 172.21.15.88:0/919267348 shutdown_connections 2024-09-21T05:56:05.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.806+0000 7f8d1a200640 1 --2- 172.21.15.88:0/919267348 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d1415f1e0 0x7f8d140b3d50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.806+0000 7f8d1a200640 1 -- 172.21.15.88:0/919267348 >> 172.21.15.88:0/919267348 conn(0x7f8d1406cfc0 msgr2=0x7f8d1406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:05.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.806+0000 7f8d1a200640 1 -- 172.21.15.88:0/919267348 shutdown_connections 2024-09-21T05:56:05.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.806+0000 7f8d1a200640 1 -- 172.21.15.88:0/919267348 wait complete. 2024-09-21T05:56:05.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.810+0000 7f8d1a200640 1 Processor -- start 2024-09-21T05:56:05.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.810+0000 7f8d1a200640 1 -- start start 2024-09-21T05:56:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d1a200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d140b4290 0x7f8d1409cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d1a200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d140b58e0 0x7f8d14139df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d1a200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1415f1e0 0x7f8d140af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d1a200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14152940 con 0x7f8d140b4290 2024-09-21T05:56:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d1a200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141527c0 con 0x7f8d1415f1e0 2024-09-21T05:56:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d1a200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8d14152ac0 con 0x7f8d140b58e0 2024-09-21T05:56:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d12a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d140b58e0 0x7f8d14139df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d12a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d140b58e0 0x7f8d14139df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.88:49056/0 (socket says 172.21.15.88:49056) 2024-09-21T05:56:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d12a00640 1 -- 172.21.15.88:0/2945567415 learned_addr learned my addr 172.21.15.88:0/2945567415 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T05:56:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d13e00640 1 --2- 172.21.15.88:0/2945567415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1415f1e0 0x7f8d140af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d13400640 1 --2- 172.21.15.88:0/2945567415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d140b4290 0x7f8d1409cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d12a00640 1 -- 172.21.15.88:0/2945567415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1415f1e0 msgr2=0x7f8d140af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d12a00640 1 --2- 172.21.15.88:0/2945567415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1415f1e0 0x7f8d140af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d12a00640 1 -- 172.21.15.88:0/2945567415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d140b4290 msgr2=0x7f8d1409cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d12a00640 1 --2- 172.21.15.88:0/2945567415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d140b4290 0x7f8d1409cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d12a00640 1 -- 172.21.15.88:0/2945567415 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1409d2f0 con 0x7f8d140b58e0 2024-09-21T05:56:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d12a00640 1 --2- 172.21.15.88:0/2945567415 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d140b58e0 0x7f8d14139df0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8d00000c10 tx=0x7f8d000a56b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:05.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d0b400640 1 -- 172.21.15.88:0/2945567415 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d000b2040 con 0x7f8d140b58e0 2024-09-21T05:56:05.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d0b400640 1 -- 172.21.15.88:0/2945567415 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d00004340 con 0x7f8d140b58e0 2024-09-21T05:56:05.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d0b400640 1 -- 172.21.15.88:0/2945567415 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00003860 con 0x7f8d140b58e0 2024-09-21T05:56:05.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d1a200640 1 -- 172.21.15.88:0/2945567415 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1409d5d0 con 0x7f8d140b58e0 2024-09-21T05:56:05.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d1a200640 1 -- 172.21.15.88:0/2945567415 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1414e820 con 0x7f8d140b58e0 2024-09-21T05:56:05.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d0b400640 1 -- 172.21.15.88:0/2945567415 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8d00003a00 con 0x7f8d140b58e0 2024-09-21T05:56:05.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d0b400640 1 -- 172.21.15.88:0/2945567415 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f8d000bf400 con 0x7f8d140b58e0 2024-09-21T05:56:05.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.814+0000 7f8d1a200640 1 -- 172.21.15.88:0/2945567415 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d1409d060 con 0x7f8d140b58e0 2024-09-21T05:56:05.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.818+0000 7f8d0b400640 1 -- 172.21.15.88:0/2945567415 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8d00016450 con 0x7f8d140b58e0 2024-09-21T05:56:05.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.886+0000 7f8d1a200640 1 -- 172.21.15.88:0/2945567415 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd new", "uuid": "56c27f67-47d4-4947-9c33-de2f8289a275", "id": 1} v 0) -- 0x7f8d140b3d50 con 0x7f8d140b58e0 2024-09-21T05:56:05.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.890+0000 7f8d0b400640 1 -- 172.21.15.88:0/2945567415 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "56c27f67-47d4-4947-9c33-de2f8289a275", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f8d000ade10 con 0x7f8d140b58e0 2024-09-21T05:56:05.893 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T05:56:05.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.894+0000 7f8d1a200640 1 -- 172.21.15.88:0/2945567415 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d140b58e0 msgr2=0x7f8d14139df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.894+0000 7f8d1a200640 1 --2- 172.21.15.88:0/2945567415 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d140b58e0 0x7f8d14139df0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8d00000c10 tx=0x7f8d000a56b0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.894+0000 7f8d1a200640 1 -- 172.21.15.88:0/2945567415 shutdown_connections 2024-09-21T05:56:05.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.894+0000 7f8d1a200640 1 --2- 172.21.15.88:0/2945567415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1415f1e0 0x7f8d140af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.894+0000 7f8d1a200640 1 --2- 172.21.15.88:0/2945567415 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d140b58e0 0x7f8d14139df0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.894+0000 7f8d1a200640 1 --2- 172.21.15.88:0/2945567415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d140b4290 0x7f8d1409cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.894+0000 7f8d1a200640 1 -- 172.21.15.88:0/2945567415 >> 172.21.15.88:0/2945567415 conn(0x7f8d1406cfc0 msgr2=0x7f8d14155550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:05.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.894+0000 7f8d1a200640 1 -- 172.21.15.88:0/2945567415 shutdown_connections 2024-09-21T05:56:05.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:05.894+0000 7f8d1a200640 1 -- 172.21.15.88:0/2945567415 wait complete. 2024-09-21T05:56:05.913 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 5c555649-42c2-4954-8b63-c7894b971798 2 2024-09-21T05:56:06.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.062+0000 7fbf7e600640 1 -- 172.21.15.88:0/2419570890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf780cf460 msgr2=0x7fbf780cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.062+0000 7fbf7e600640 1 --2- 172.21.15.88:0/2419570890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf780cf460 0x7fbf780cf870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009bc0 tx=0x7fbf7001cb10 comp rx=0 tx=0).stop 2024-09-21T05:56:06.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.062+0000 7fbf7e600640 1 -- 172.21.15.88:0/2419570890 shutdown_connections 2024-09-21T05:56:06.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.062+0000 7fbf7e600640 1 --2- 172.21.15.88:0/2419570890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf780cf460 0x7fbf780cf870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.062+0000 7fbf7e600640 1 --2- 172.21.15.88:0/2419570890 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf78068df0 0x7fbf78069200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.062+0000 7fbf7e600640 1 -- 172.21.15.88:0/2419570890 >> 172.21.15.88:0/2419570890 conn(0x7fbf780d17f0 msgr2=0x7fbf780d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:06.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.062+0000 7fbf7e600640 1 -- 172.21.15.88:0/2419570890 shutdown_connections 2024-09-21T05:56:06.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.062+0000 7fbf7e600640 1 -- 172.21.15.88:0/2419570890 wait complete. 2024-09-21T05:56:06.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf7e600640 1 Processor -- start 2024-09-21T05:56:06.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf7e600640 1 -- start start 2024-09-21T05:56:06.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf7e600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf78068df0 0x7fbf78152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:06.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf7e600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf780cf460 0x7fbf7814a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:06.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf7e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf78160be0 0x7fbf78070830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:06.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf7e600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbf78169dd0 con 0x7fbf78160be0 2024-09-21T05:56:06.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf7e600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbf78169c50 con 0x7fbf78068df0 2024-09-21T05:56:06.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf7e600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fbf78169f50 con 0x7fbf780cf460 2024-09-21T05:56:06.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf77400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf78068df0 0x7fbf78152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:06.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf76a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf780cf460 0x7fbf7814a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:06.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf77400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf78068df0 0x7fbf78152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50216/0 (socket says 172.21.15.88:50216) 2024-09-21T05:56:06.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf77400640 1 -- 172.21.15.88:0/3437079274 learned_addr learned my addr 172.21.15.88:0/3437079274 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T05:56:06.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf77e00640 1 --2- 172.21.15.88:0/3437079274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf78160be0 0x7fbf78070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:06.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf76a00640 1 -- 172.21.15.88:0/3437079274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf78068df0 msgr2=0x7fbf78152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf76a00640 1 --2- 172.21.15.88:0/3437079274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf78068df0 0x7fbf78152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf76a00640 1 -- 172.21.15.88:0/3437079274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf78160be0 msgr2=0x7fbf78070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf76a00640 1 --2- 172.21.15.88:0/3437079274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf78160be0 0x7fbf78070830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf76a00640 1 -- 172.21.15.88:0/3437079274 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7815c310 con 0x7fbf780cf460 2024-09-21T05:56:06.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.066+0000 7fbf77400640 1 --2- 172.21.15.88:0/3437079274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf78068df0 0x7fbf78152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:06.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.070+0000 7fbf76a00640 1 --2- 172.21.15.88:0/3437079274 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf780cf460 0x7fbf7814a090 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400ea20 tx=0x7fbf6400eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:06.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.070+0000 7fbf6f400640 1 -- 172.21.15.88:0/3437079274 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6400cde0 con 0x7fbf780cf460 2024-09-21T05:56:06.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.070+0000 7fbf6f400640 1 -- 172.21.15.88:0/3437079274 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf64004020 con 0x7fbf780cf460 2024-09-21T05:56:06.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.070+0000 7fbf7e600640 1 -- 172.21.15.88:0/3437079274 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf78151d60 con 0x7fbf780cf460 2024-09-21T05:56:06.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.070+0000 7fbf6f400640 1 -- 172.21.15.88:0/3437079274 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64004470 con 0x7fbf780cf460 2024-09-21T05:56:06.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.070+0000 7fbf7e600640 1 -- 172.21.15.88:0/3437079274 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf78154ba0 con 0x7fbf780cf460 2024-09-21T05:56:06.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.070+0000 7fbf6f400640 1 -- 172.21.15.88:0/3437079274 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbf64003990 con 0x7fbf780cf460 2024-09-21T05:56:06.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.070+0000 7fbf6f400640 1 -- 172.21.15.88:0/3437079274 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fbf64009440 con 0x7fbf780cf460 2024-09-21T05:56:06.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.070+0000 7fbf7e600640 1 -- 172.21.15.88:0/3437079274 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf78069200 con 0x7fbf780cf460 2024-09-21T05:56:06.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.074+0000 7fbf6f400640 1 -- 172.21.15.88:0/3437079274 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fbf64003c80 con 0x7fbf780cf460 2024-09-21T05:56:06.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.142+0000 7fbf7e600640 1 -- 172.21.15.88:0/3437079274 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5c555649-42c2-4954-8b63-c7894b971798", "id": 2} v 0) -- 0x7fbf78069ce0 con 0x7fbf780cf460 2024-09-21T05:56:06.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.146+0000 7fbf6f400640 1 -- 172.21.15.88:0/3437079274 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5c555649-42c2-4954-8b63-c7894b971798", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fbf64021bc0 con 0x7fbf780cf460 2024-09-21T05:56:06.148 INFO:teuthology.orchestra.run.smithi088.stdout:2 2024-09-21T05:56:06.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.146+0000 7fbf7e600640 1 -- 172.21.15.88:0/3437079274 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf780cf460 msgr2=0x7fbf7814a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.146+0000 7fbf7e600640 1 --2- 172.21.15.88:0/3437079274 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf780cf460 0x7fbf7814a090 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400ea20 tx=0x7fbf6400eee0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.146+0000 7fbf7e600640 1 -- 172.21.15.88:0/3437079274 shutdown_connections 2024-09-21T05:56:06.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.146+0000 7fbf7e600640 1 --2- 172.21.15.88:0/3437079274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf78160be0 0x7fbf78070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.146+0000 7fbf7e600640 1 --2- 172.21.15.88:0/3437079274 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf780cf460 0x7fbf7814a090 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.146+0000 7fbf7e600640 1 --2- 172.21.15.88:0/3437079274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf78068df0 0x7fbf78152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.146+0000 7fbf7e600640 1 -- 172.21.15.88:0/3437079274 >> 172.21.15.88:0/3437079274 conn(0x7fbf780d17f0 msgr2=0x7fbf780d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:06.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.146+0000 7fbf7e600640 1 -- 172.21.15.88:0/3437079274 shutdown_connections 2024-09-21T05:56:06.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.146+0000 7fbf7e600640 1 -- 172.21.15.88:0/3437079274 wait complete. 2024-09-21T05:56:06.167 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 6ed0b889-c077-488f-abb5-1918a490c12e 3 2024-09-21T05:56:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.318+0000 7f723c800640 1 -- 172.21.15.88:0/3367978479 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f721c003c40 msgr2=0x7f721c004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.318+0000 7f723c800640 1 --2- 172.21.15.88:0/3367978479 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f721c003c40 0x7f721c004090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7228006f80 tx=0x7f722801cec0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.318+0000 7f723c800640 1 -- 172.21.15.88:0/3367978479 shutdown_connections 2024-09-21T05:56:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.318+0000 7f723c800640 1 --2- 172.21.15.88:0/3367978479 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f721c003c40 0x7f721c004090 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.318+0000 7f723c800640 1 -- 172.21.15.88:0/3367978479 >> 172.21.15.88:0/3367978479 conn(0x7f72340708c0 msgr2=0x7f7234070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.322+0000 7f723c800640 1 -- 172.21.15.88:0/3367978479 shutdown_connections 2024-09-21T05:56:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.322+0000 7f723c800640 1 -- 172.21.15.88:0/3367978479 wait complete. 2024-09-21T05:56:06.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.322+0000 7f723c800640 1 Processor -- start 2024-09-21T05:56:06.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.322+0000 7f723c800640 1 -- start start 2024-09-21T05:56:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.322+0000 7f723c800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f721c003c40 0x7f7234147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.322+0000 7f723c800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72341f9090 0x7f723414a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.322+0000 7f723c800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f723414a7f0 0x7f723414ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.322+0000 7f723c800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f72341610e0 con 0x7f723414a7f0 2024-09-21T05:56:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.322+0000 7f723c800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7234160f60 con 0x7f72341f9090 2024-09-21T05:56:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.322+0000 7f723c800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7234161260 con 0x7f721c003c40 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f7239400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72341f9090 0x7f723414a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f7239e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f721c003c40 0x7f7234147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f7239400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72341f9090 0x7f723414a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50224/0 (socket says 172.21.15.88:50224) 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f7239400640 1 -- 172.21.15.88:0/1558837707 learned_addr learned my addr 172.21.15.88:0/1558837707 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f723a800640 1 --2- 172.21.15.88:0/1558837707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f723414a7f0 0x7f723414ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f7239400640 1 -- 172.21.15.88:0/1558837707 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f721c003c40 msgr2=0x7f7234147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f7239400640 1 --2- 172.21.15.88:0/1558837707 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f721c003c40 0x7f7234147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f7239400640 1 -- 172.21.15.88:0/1558837707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f723414a7f0 msgr2=0x7f723414ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f7239400640 1 --2- 172.21.15.88:0/1558837707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f723414a7f0 0x7f723414ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f7239400640 1 -- 172.21.15.88:0/1558837707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72341f9800 con 0x7f72341f9090 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f7239400640 1 --2- 172.21.15.88:0/1558837707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72341f9090 0x7f723414a2b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7224005e40 tx=0x7f7224011740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f722e000640 1 -- 172.21.15.88:0/1558837707 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224005110 con 0x7f72341f9090 2024-09-21T05:56:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f722e000640 1 -- 172.21.15.88:0/1558837707 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7224018020 con 0x7f72341f9090 2024-09-21T05:56:06.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f722e000640 1 -- 172.21.15.88:0/1558837707 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224003600 con 0x7f72341f9090 2024-09-21T05:56:06.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f723c800640 1 -- 172.21.15.88:0/1558837707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72341f9ae0 con 0x7f72341f9090 2024-09-21T05:56:06.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f723c800640 1 -- 172.21.15.88:0/1558837707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72340c9b90 con 0x7f72341f9090 2024-09-21T05:56:06.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f722e000640 1 -- 172.21.15.88:0/1558837707 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7224003b30 con 0x7f72341f9090 2024-09-21T05:56:06.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f722e000640 1 -- 172.21.15.88:0/1558837707 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f72240138e0 con 0x7f72341f9090 2024-09-21T05:56:06.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.326+0000 7f723c800640 1 -- 172.21.15.88:0/1558837707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7234074700 con 0x7f72341f9090 2024-09-21T05:56:06.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.330+0000 7f722e000640 1 -- 172.21.15.88:0/1558837707 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7224022020 con 0x7f72341f9090 2024-09-21T05:56:06.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.438+0000 7f723c800640 1 -- 172.21.15.88:0/1558837707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6ed0b889-c077-488f-abb5-1918a490c12e", "id": 3} v 0) -- 0x7f72340d2260 con 0x7f72341f9090 2024-09-21T05:56:06.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.442+0000 7f722e000640 1 -- 172.21.15.88:0/1558837707 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6ed0b889-c077-488f-abb5-1918a490c12e", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f7224003e20 con 0x7f72341f9090 2024-09-21T05:56:06.445 INFO:teuthology.orchestra.run.smithi088.stdout:3 2024-09-21T05:56:06.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.446+0000 7f723c800640 1 -- 172.21.15.88:0/1558837707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72341f9090 msgr2=0x7f723414a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.446+0000 7f723c800640 1 --2- 172.21.15.88:0/1558837707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72341f9090 0x7f723414a2b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7224005e40 tx=0x7f7224011740 comp rx=0 tx=0).stop 2024-09-21T05:56:06.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.446+0000 7f723c800640 1 -- 172.21.15.88:0/1558837707 shutdown_connections 2024-09-21T05:56:06.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.446+0000 7f723c800640 1 --2- 172.21.15.88:0/1558837707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f723414a7f0 0x7f723414ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.446+0000 7f723c800640 1 --2- 172.21.15.88:0/1558837707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72341f9090 0x7f723414a2b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.446+0000 7f723c800640 1 --2- 172.21.15.88:0/1558837707 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f721c003c40 0x7f7234147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.446+0000 7f723c800640 1 -- 172.21.15.88:0/1558837707 >> 172.21.15.88:0/1558837707 conn(0x7f72340708c0 msgr2=0x7f72340d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:06.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.446+0000 7f723c800640 1 -- 172.21.15.88:0/1558837707 shutdown_connections 2024-09-21T05:56:06.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.446+0000 7f723c800640 1 -- 172.21.15.88:0/1558837707 wait complete. 2024-09-21T05:56:06.465 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92 4 2024-09-21T05:56:06.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.618+0000 7fad8a600640 1 -- 172.21.15.88:0/999899811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad84157030 msgr2=0x7fad840b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.618+0000 7fad8a600640 1 --2- 172.21.15.88:0/999899811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad84157030 0x7fad840b5a80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fad840713e0 tx=0x7fad7401c830 comp rx=0 tx=0).stop 2024-09-21T05:56:06.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.618+0000 7fad8a600640 1 -- 172.21.15.88:0/999899811 shutdown_connections 2024-09-21T05:56:06.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.618+0000 7fad8a600640 1 --2- 172.21.15.88:0/999899811 >> v2:172.21.15.50:3300/0 conn(0x7fad84074b20 0x7fad8415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.618+0000 7fad8a600640 1 --2- 172.21.15.88:0/999899811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad84157030 0x7fad840b5a80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.618+0000 7fad8a600640 1 -- 172.21.15.88:0/999899811 >> 172.21.15.88:0/999899811 conn(0x7fad840708c0 msgr2=0x7fad84070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:06.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.618+0000 7fad8a600640 1 -- 172.21.15.88:0/999899811 shutdown_connections 2024-09-21T05:56:06.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.618+0000 7fad8a600640 1 -- 172.21.15.88:0/999899811 wait complete. 2024-09-21T05:56:06.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad8a600640 1 Processor -- start 2024-09-21T05:56:06.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad8a600640 1 -- start start 2024-09-21T05:56:06.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad8a600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad84157030 0x7fad8414c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:06.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad8a600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fad84160d50 0x7fad841fa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:06.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad8a600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad8414cee0 0x7fad841fa790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:06.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad8a600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fad8416a740 con 0x7fad84157030 2024-09-21T05:56:06.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad8a600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fad8416a5c0 con 0x7fad8414cee0 2024-09-21T05:56:06.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad8a600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fad8416a8c0 con 0x7fad84160d50 2024-09-21T05:56:06.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad83e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad8414cee0 0x7fad841fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:06.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad7aa00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fad84160d50 0x7fad841fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:06.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad83400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad84157030 0x7fad8414c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:06.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad83400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad84157030 0x7fad8414c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.88:50976/0 (socket says 172.21.15.88:50976) 2024-09-21T05:56:06.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.622+0000 7fad83400640 1 -- 172.21.15.88:0/590414820 learned_addr learned my addr 172.21.15.88:0/590414820 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T05:56:06.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad7aa00640 1 -- 172.21.15.88:0/590414820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad8414cee0 msgr2=0x7fad841fa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad7aa00640 1 --2- 172.21.15.88:0/590414820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad8414cee0 0x7fad841fa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad7aa00640 1 -- 172.21.15.88:0/590414820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad84157030 msgr2=0x7fad8414c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad7aa00640 1 --2- 172.21.15.88:0/590414820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad84157030 0x7fad8414c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad7aa00640 1 -- 172.21.15.88:0/590414820 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad84154fb0 con 0x7fad84160d50 2024-09-21T05:56:06.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad7aa00640 1 --2- 172.21.15.88:0/590414820 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fad84160d50 0x7fad841fa250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c390 tx=0x7fad6c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:06.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad7be00640 1 -- 172.21.15.88:0/590414820 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c017070 con 0x7fad84160d50 2024-09-21T05:56:06.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad7be00640 1 -- 172.21.15.88:0/590414820 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad6c0053c0 con 0x7fad84160d50 2024-09-21T05:56:06.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad8a600640 1 -- 172.21.15.88:0/590414820 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad84155290 con 0x7fad84160d50 2024-09-21T05:56:06.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad7be00640 1 -- 172.21.15.88:0/590414820 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c003840 con 0x7fad84160d50 2024-09-21T05:56:06.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad8a600640 1 -- 172.21.15.88:0/590414820 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fad84155850 con 0x7fad84160d50 2024-09-21T05:56:06.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad7be00640 1 -- 172.21.15.88:0/590414820 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fad6c01a070 con 0x7fad84160d50 2024-09-21T05:56:06.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad7be00640 1 -- 172.21.15.88:0/590414820 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fad6c011590 con 0x7fad84160d50 2024-09-21T05:56:06.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.626+0000 7fad8a600640 1 -- 172.21.15.88:0/590414820 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8415cf90 con 0x7fad84160d50 2024-09-21T05:56:06.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.630+0000 7fad7be00640 1 -- 172.21.15.88:0/590414820 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fad6c021020 con 0x7fad84160d50 2024-09-21T05:56:06.681 INFO:tasks.ceph.mgr.x.smithi088.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T05:56:06.681 INFO:tasks.ceph.mgr.x.smithi088.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T05:56:06.681 INFO:tasks.ceph.mgr.x.smithi088.stderr: from numpy import show_config as show_numpy_config 2024-09-21T05:56:06.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.698+0000 7fad8a600640 1 -- 172.21.15.88:0/590414820 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92", "id": 4} v 0) -- 0x7fad840d2260 con 0x7fad84160d50 2024-09-21T05:56:06.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.702+0000 7fad7be00640 1 -- 172.21.15.88:0/590414820 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fad6c011bd0 con 0x7fad84160d50 2024-09-21T05:56:06.705 INFO:teuthology.orchestra.run.smithi088.stdout:4 2024-09-21T05:56:06.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.706+0000 7fad8a600640 1 -- 172.21.15.88:0/590414820 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fad84160d50 msgr2=0x7fad841fa250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.706+0000 7fad8a600640 1 --2- 172.21.15.88:0/590414820 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fad84160d50 0x7fad841fa250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c390 tx=0x7fad6c00c850 comp rx=0 tx=0).stop 2024-09-21T05:56:06.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.706+0000 7fad8a600640 1 -- 172.21.15.88:0/590414820 shutdown_connections 2024-09-21T05:56:06.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.706+0000 7fad8a600640 1 --2- 172.21.15.88:0/590414820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad8414cee0 0x7fad841fa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.706+0000 7fad8a600640 1 --2- 172.21.15.88:0/590414820 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fad84160d50 0x7fad841fa250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.706+0000 7fad8a600640 1 --2- 172.21.15.88:0/590414820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad84157030 0x7fad8414c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.706+0000 7fad8a600640 1 -- 172.21.15.88:0/590414820 >> 172.21.15.88:0/590414820 conn(0x7fad840708c0 msgr2=0x7fad8415f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:06.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.706+0000 7fad8a600640 1 -- 172.21.15.88:0/590414820 shutdown_connections 2024-09-21T05:56:06.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.706+0000 7fad8a600640 1 -- 172.21.15.88:0/590414820 wait complete. 2024-09-21T05:56:06.724 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 5a0bf4bb-c6d7-48f6-810b-23f01fd7a745 5 2024-09-21T05:56:06.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.874+0000 7f7a6d800640 1 -- 172.21.15.88:0/2314459103 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a68074590 msgr2=0x7f7a6815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.874+0000 7f7a6d800640 1 --2- 172.21.15.88:0/2314459103 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a68074590 0x7f7a6815bcd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009900 tx=0x7f7a5401c8b0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.874+0000 7f7a6d800640 1 -- 172.21.15.88:0/2314459103 shutdown_connections 2024-09-21T05:56:06.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.874+0000 7f7a6d800640 1 --2- 172.21.15.88:0/2314459103 >> v2:172.21.15.50:3300/0 conn(0x7f7a6815c210 0x7f7a6806e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.874+0000 7f7a6d800640 1 --2- 172.21.15.88:0/2314459103 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a68074590 0x7f7a6815bcd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.874+0000 7f7a6d800640 1 -- 172.21.15.88:0/2314459103 >> 172.21.15.88:0/2314459103 conn(0x7f7a680b2c40 msgr2=0x7f7a680b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:06.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.874+0000 7f7a6d800640 1 -- 172.21.15.88:0/2314459103 shutdown_connections 2024-09-21T05:56:06.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.874+0000 7f7a6d800640 1 -- 172.21.15.88:0/2314459103 wait complete. 2024-09-21T05:56:06.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a6d800640 1 Processor -- start 2024-09-21T05:56:06.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a6d800640 1 -- start start 2024-09-21T05:56:06.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a6d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a680cff30 0x7f7a680cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:06.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a6d800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a6815c210 0x7f7a6813f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:06.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a6d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a6815a860 0x7f7a6815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:06.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a6d800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a681525e0 con 0x7f7a680cff30 2024-09-21T05:56:06.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a6d800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68152460 con 0x7f7a6815a860 2024-09-21T05:56:06.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a6d800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7a68152760 con 0x7f7a6815c210 2024-09-21T05:56:06.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a66000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a6815c210 0x7f7a6813f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:06.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a66000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a6815c210 0x7f7a6813f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.88:49126/0 (socket says 172.21.15.88:49126) 2024-09-21T05:56:06.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a66000640 1 -- 172.21.15.88:0/4166773993 learned_addr learned my addr 172.21.15.88:0/4166773993 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T05:56:06.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a67400640 1 --2- 172.21.15.88:0/4166773993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a6815a860 0x7f7a6815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:06.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a66a00640 1 --2- 172.21.15.88:0/4166773993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a680cff30 0x7f7a680cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:06.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a66000640 1 -- 172.21.15.88:0/4166773993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a6815a860 msgr2=0x7f7a6815ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a66000640 1 --2- 172.21.15.88:0/4166773993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a6815a860 0x7f7a6815ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a66000640 1 -- 172.21.15.88:0/4166773993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a680cff30 msgr2=0x7f7a680cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a66000640 1 --2- 172.21.15.88:0/4166773993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a680cff30 0x7f7a680cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a66000640 1 -- 172.21.15.88:0/4166773993 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6815b1b0 con 0x7f7a6815c210 2024-09-21T05:56:06.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a66a00640 1 --2- 172.21.15.88:0/4166773993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a680cff30 0x7f7a680cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:06.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.878+0000 7f7a66000640 1 --2- 172.21.15.88:0/4166773993 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a6815c210 0x7f7a6813f4f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009850 tx=0x7f7a540a57f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:06.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.882+0000 7f7a5ea00640 1 -- 172.21.15.88:0/4166773993 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a540b2040 con 0x7f7a6815c210 2024-09-21T05:56:06.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.882+0000 7f7a5ea00640 1 -- 172.21.15.88:0/4166773993 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a540044b0 con 0x7f7a6815c210 2024-09-21T05:56:06.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.882+0000 7f7a5ea00640 1 -- 172.21.15.88:0/4166773993 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54005270 con 0x7f7a6815c210 2024-09-21T05:56:06.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.882+0000 7f7a6d800640 1 -- 172.21.15.88:0/4166773993 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a68158780 con 0x7f7a6815c210 2024-09-21T05:56:06.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.882+0000 7f7a6d800640 1 -- 172.21.15.88:0/4166773993 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a68158d40 con 0x7f7a6815c210 2024-09-21T05:56:06.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.882+0000 7f7a5ea00640 1 -- 172.21.15.88:0/4166773993 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a54004a60 con 0x7f7a6815c210 2024-09-21T05:56:06.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.882+0000 7f7a5ea00640 1 -- 172.21.15.88:0/4166773993 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f7a54004d50 con 0x7f7a6815c210 2024-09-21T05:56:06.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.882+0000 7f7a6d800640 1 -- 172.21.15.88:0/4166773993 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6815bcd0 con 0x7f7a6815c210 2024-09-21T05:56:06.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.882+0000 7f7a5ea00640 1 -- 172.21.15.88:0/4166773993 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7a540bc060 con 0x7f7a6815c210 2024-09-21T05:56:06.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.954+0000 7f7a6d800640 1 -- 172.21.15.88:0/4166773993 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5a0bf4bb-c6d7-48f6-810b-23f01fd7a745", "id": 5} v 0) -- 0x7f7a6815d2f0 con 0x7f7a6815c210 2024-09-21T05:56:06.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.958+0000 7f7a5ea00640 1 -- 172.21.15.88:0/4166773993 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a0bf4bb-c6d7-48f6-810b-23f01fd7a745", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f7a6815b1b0 con 0x7f7a6815c210 2024-09-21T05:56:06.959 INFO:teuthology.orchestra.run.smithi088.stdout:5 2024-09-21T05:56:06.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.958+0000 7f7a6d800640 1 -- 172.21.15.88:0/4166773993 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a6815c210 msgr2=0x7f7a6813f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:06.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.958+0000 7f7a6d800640 1 --2- 172.21.15.88:0/4166773993 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a6815c210 0x7f7a6813f4f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009850 tx=0x7f7a540a57f0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.958+0000 7f7a6d800640 1 -- 172.21.15.88:0/4166773993 shutdown_connections 2024-09-21T05:56:06.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.958+0000 7f7a6d800640 1 --2- 172.21.15.88:0/4166773993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a6815a860 0x7f7a6815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.958+0000 7f7a6d800640 1 --2- 172.21.15.88:0/4166773993 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a6815c210 0x7f7a6813f4f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.958+0000 7f7a6d800640 1 --2- 172.21.15.88:0/4166773993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a680cff30 0x7f7a680cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:06.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.958+0000 7f7a6d800640 1 -- 172.21.15.88:0/4166773993 >> 172.21.15.88:0/4166773993 conn(0x7f7a680b2c40 msgr2=0x7f7a680d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:06.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.958+0000 7f7a6d800640 1 -- 172.21.15.88:0/4166773993 shutdown_connections 2024-09-21T05:56:06.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:06.958+0000 7f7a6d800640 1 -- 172.21.15.88:0/4166773993 wait complete. 2024-09-21T05:56:06.979 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new a9bbb22e-9d25-49db-9bb4-93ba63f97bb7 6 2024-09-21T05:56:07.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.130+0000 7fc1cea00640 1 -- 172.21.15.88:0/1080079608 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1c815f5c0 msgr2=0x7fc1c815f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.130+0000 7fc1cea00640 1 --2- 172.21.15.88:0/1080079608 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1c815f5c0 0x7fc1c815f9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009c80 tx=0x7fc1c001cd10 comp rx=0 tx=0).stop 2024-09-21T05:56:07.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.130+0000 7fc1cea00640 1 -- 172.21.15.88:0/1080079608 shutdown_connections 2024-09-21T05:56:07.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.130+0000 7fc1cea00640 1 --2- 172.21.15.88:0/1080079608 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1c815f5c0 0x7fc1c815f9d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.130+0000 7fc1cea00640 1 --2- 172.21.15.88:0/1080079608 >> v2:172.21.15.88:3300/0 conn(0x7fc1c806dbc0 0x7fc1c806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.130+0000 7fc1cea00640 1 -- 172.21.15.88:0/1080079608 >> 172.21.15.88:0/1080079608 conn(0x7fc1c80b41f0 msgr2=0x7fc1c80b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:07.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.130+0000 7fc1cea00640 1 -- 172.21.15.88:0/1080079608 shutdown_connections 2024-09-21T05:56:07.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.130+0000 7fc1cea00640 1 -- 172.21.15.88:0/1080079608 wait complete. 2024-09-21T05:56:07.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.134+0000 7fc1cea00640 1 Processor -- start 2024-09-21T05:56:07.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.134+0000 7fc1cea00640 1 -- start start 2024-09-21T05:56:07.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.134+0000 7fc1cea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1c806dbc0 0x7fc1c81fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.134+0000 7fc1cea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1c80d1140 0x7fc1c81fc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.134+0000 7fc1cea00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1c815f5c0 0x7fc1c81fc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.134+0000 7fc1cea00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8069990 con 0x7fc1c806dbc0 2024-09-21T05:56:07.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.134+0000 7fc1cea00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8069810 con 0x7fc1c80d1140 2024-09-21T05:56:07.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.134+0000 7fc1cea00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c8069b10 con 0x7fc1c815f5c0 2024-09-21T05:56:07.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.134+0000 7fc1ce000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1c815f5c0 0x7fc1c81fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:07.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.134+0000 7fc1ccc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1c80d1140 0x7fc1c81fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:07.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1ce000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1c815f5c0 0x7fc1c81fc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.88:49152/0 (socket says 172.21.15.88:49152) 2024-09-21T05:56:07.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1ce000640 1 -- 172.21.15.88:0/1905820106 learned_addr learned my addr 172.21.15.88:0/1905820106 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T05:56:07.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1cd600640 1 --2- 172.21.15.88:0/1905820106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1c806dbc0 0x7fc1c81fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:07.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1ce000640 1 -- 172.21.15.88:0/1905820106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1c80d1140 msgr2=0x7fc1c81fc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1ce000640 1 --2- 172.21.15.88:0/1905820106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1c80d1140 0x7fc1c81fc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1ce000640 1 -- 172.21.15.88:0/1905820106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1c806dbc0 msgr2=0x7fc1c81fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1ce000640 1 --2- 172.21.15.88:0/1905820106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1c806dbc0 0x7fc1c81fb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1ce000640 1 -- 172.21.15.88:0/1905820106 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c815b680 con 0x7fc1c815f5c0 2024-09-21T05:56:07.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1ce000640 1 --2- 172.21.15.88:0/1905820106 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1c815f5c0 0x7fc1c81fc780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc1c001b3c0 tx=0x7fc1c0005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:07.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1c5600640 1 -- 172.21.15.88:0/1905820106 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c00bb070 con 0x7fc1c815f5c0 2024-09-21T05:56:07.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1c5600640 1 -- 172.21.15.88:0/1905820106 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1c0004b40 con 0x7fc1c815f5c0 2024-09-21T05:56:07.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1c5600640 1 -- 172.21.15.88:0/1905820106 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c00b2040 con 0x7fc1c815f5c0 2024-09-21T05:56:07.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1cea00640 1 -- 172.21.15.88:0/1905820106 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c80703f0 con 0x7fc1c815f5c0 2024-09-21T05:56:07.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1cea00640 1 -- 172.21.15.88:0/1905820106 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c81fd050 con 0x7fc1c815f5c0 2024-09-21T05:56:07.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1c5600640 1 -- 172.21.15.88:0/1905820106 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc1c0004000 con 0x7fc1c815f5c0 2024-09-21T05:56:07.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1c5600640 1 -- 172.21.15.88:0/1905820106 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fc1c0005260 con 0x7fc1c815f5c0 2024-09-21T05:56:07.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.138+0000 7fc1cea00640 1 -- 172.21.15.88:0/1905820106 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c806dfd0 con 0x7fc1c815f5c0 2024-09-21T05:56:07.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.142+0000 7fc1c5600640 1 -- 172.21.15.88:0/1905820106 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc1c00c4080 con 0x7fc1c815f5c0 2024-09-21T05:56:07.145 INFO:tasks.ceph.mgr.y.smithi050.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T05:56:07.145 INFO:tasks.ceph.mgr.y.smithi050.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T05:56:07.146 INFO:tasks.ceph.mgr.y.smithi050.stderr: from numpy import show_config as show_numpy_config 2024-09-21T05:56:07.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.210+0000 7fc1cea00640 1 -- 172.21.15.88:0/1905820106 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a9bbb22e-9d25-49db-9bb4-93ba63f97bb7", "id": 6} v 0) -- 0x7fc1c80ae760 con 0x7fc1c815f5c0 2024-09-21T05:56:07.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.214+0000 7fc1c5600640 1 -- 172.21.15.88:0/1905820106 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a9bbb22e-9d25-49db-9bb4-93ba63f97bb7", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fc1c0016990 con 0x7fc1c815f5c0 2024-09-21T05:56:07.216 INFO:teuthology.orchestra.run.smithi088.stdout:6 2024-09-21T05:56:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.214+0000 7fc1cea00640 1 -- 172.21.15.88:0/1905820106 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1c815f5c0 msgr2=0x7fc1c81fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.214+0000 7fc1cea00640 1 --2- 172.21.15.88:0/1905820106 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1c815f5c0 0x7fc1c81fc780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc1c001b3c0 tx=0x7fc1c0005b00 comp rx=0 tx=0).stop 2024-09-21T05:56:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.214+0000 7fc1cea00640 1 -- 172.21.15.88:0/1905820106 shutdown_connections 2024-09-21T05:56:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.214+0000 7fc1cea00640 1 --2- 172.21.15.88:0/1905820106 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1c815f5c0 0x7fc1c81fc780 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.214+0000 7fc1cea00640 1 --2- 172.21.15.88:0/1905820106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1c80d1140 0x7fc1c81fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.214+0000 7fc1cea00640 1 --2- 172.21.15.88:0/1905820106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1c806dbc0 0x7fc1c81fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.214+0000 7fc1cea00640 1 -- 172.21.15.88:0/1905820106 >> 172.21.15.88:0/1905820106 conn(0x7fc1c80b41f0 msgr2=0x7fc1c80d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.214+0000 7fc1cea00640 1 -- 172.21.15.88:0/1905820106 shutdown_connections 2024-09-21T05:56:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.214+0000 7fc1cea00640 1 -- 172.21.15.88:0/1905820106 wait complete. 2024-09-21T05:56:07.235 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 01f3517b-4862-4189-b00f-e2da775385d9 7 2024-09-21T05:56:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.386+0000 7f5e95600640 1 -- 172.21.15.88:0/1805383205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e900749a0 msgr2=0x7f5e90074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.386+0000 7f5e95600640 1 --2- 172.21.15.88:0/1805383205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e900749a0 0x7f5e90074db0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e900b3040 tx=0x7f5e8001c8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.386+0000 7f5e95600640 1 -- 172.21.15.88:0/1805383205 shutdown_connections 2024-09-21T05:56:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.386+0000 7f5e95600640 1 --2- 172.21.15.88:0/1805383205 >> v2:172.21.15.50:3300/0 conn(0x7f5e900cefa0 0x7f5e900cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.386+0000 7f5e95600640 1 --2- 172.21.15.88:0/1805383205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e900749a0 0x7f5e90074db0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.386+0000 7f5e95600640 1 -- 172.21.15.88:0/1805383205 >> 172.21.15.88:0/1805383205 conn(0x7f5e900b2040 msgr2=0x7f5e900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.386+0000 7f5e95600640 1 -- 172.21.15.88:0/1805383205 shutdown_connections 2024-09-21T05:56:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.386+0000 7f5e95600640 1 -- 172.21.15.88:0/1805383205 wait complete. 2024-09-21T05:56:07.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e95600640 1 Processor -- start 2024-09-21T05:56:07.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e95600640 1 -- start start 2024-09-21T05:56:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e95600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e900749a0 0x7f5e90145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e95600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e900cefa0 0x7f5e9013d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e95600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e90160330 0x7f5e9013bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e95600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9016a060 con 0x7f5e90160330 2024-09-21T05:56:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e95600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90169ee0 con 0x7f5e900cefa0 2024-09-21T05:56:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e95600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9016a1e0 con 0x7f5e900749a0 2024-09-21T05:56:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8ea00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e900749a0 0x7f5e90145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e900cefa0 0x7f5e9013d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8ea00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e900749a0 0x7f5e90145b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.88:55332/0 (socket says 172.21.15.88:55332) 2024-09-21T05:56:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8ea00640 1 -- 172.21.15.88:0/3744080835 learned_addr learned my addr 172.21.15.88:0/3744080835 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T05:56:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e900cefa0 0x7f5e9013d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58658/0 (socket says 172.21.15.88:58658) 2024-09-21T05:56:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8f400640 1 --2- 172.21.15.88:0/3744080835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e90160330 0x7f5e9013bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:07.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8ea00640 1 -- 172.21.15.88:0/3744080835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e900cefa0 msgr2=0x7f5e9013d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8ea00640 1 --2- 172.21.15.88:0/3744080835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e900cefa0 0x7f5e9013d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8ea00640 1 -- 172.21.15.88:0/3744080835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e90160330 msgr2=0x7f5e9013bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8ea00640 1 --2- 172.21.15.88:0/3744080835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e90160330 0x7f5e9013bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8ea00640 1 -- 172.21.15.88:0/3744080835 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9013bae0 con 0x7f5e900749a0 2024-09-21T05:56:07.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8e000640 1 --2- 172.21.15.88:0/3744080835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e900cefa0 0x7f5e9013d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:07.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e8ea00640 1 --2- 172.21.15.88:0/3744080835 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e900749a0 0x7f5e90145b30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e800b0040 tx=0x7f5e80004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:07.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e86a00640 1 -- 172.21.15.88:0/3744080835 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e800b2040 con 0x7f5e900749a0 2024-09-21T05:56:07.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e86a00640 1 -- 172.21.15.88:0/3744080835 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e80005310 con 0x7f5e900749a0 2024-09-21T05:56:07.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e86a00640 1 -- 172.21.15.88:0/3744080835 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e800046d0 con 0x7f5e900749a0 2024-09-21T05:56:07.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e95600640 1 -- 172.21.15.88:0/3744080835 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e90140ba0 con 0x7f5e900749a0 2024-09-21T05:56:07.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e95600640 1 -- 172.21.15.88:0/3744080835 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e90141160 con 0x7f5e900749a0 2024-09-21T05:56:07.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e86a00640 1 -- 172.21.15.88:0/3744080835 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5e80004c00 con 0x7f5e900749a0 2024-09-21T05:56:07.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.390+0000 7f5e86a00640 1 -- 172.21.15.88:0/3744080835 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f5e80003c60 con 0x7f5e900749a0 2024-09-21T05:56:07.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.394+0000 7f5e95600640 1 -- 172.21.15.88:0/3744080835 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e90074db0 con 0x7f5e900749a0 2024-09-21T05:56:07.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.394+0000 7f5e86a00640 1 -- 172.21.15.88:0/3744080835 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5e800164b0 con 0x7f5e900749a0 2024-09-21T05:56:07.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.462+0000 7f5e95600640 1 -- 172.21.15.88:0/3744080835 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd new", "uuid": "01f3517b-4862-4189-b00f-e2da775385d9", "id": 7} v 0) -- 0x7f5e9013e380 con 0x7f5e900749a0 2024-09-21T05:56:07.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.470+0000 7f5e86a00640 1 -- 172.21.15.88:0/3744080835 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "01f3517b-4862-4189-b00f-e2da775385d9", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f5e800a5a70 con 0x7f5e900749a0 2024-09-21T05:56:07.470 INFO:teuthology.orchestra.run.smithi088.stdout:7 2024-09-21T05:56:07.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.470+0000 7f5e95600640 1 -- 172.21.15.88:0/3744080835 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e900749a0 msgr2=0x7f5e90145b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.470+0000 7f5e95600640 1 --2- 172.21.15.88:0/3744080835 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e900749a0 0x7f5e90145b30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e800b0040 tx=0x7f5e80004600 comp rx=0 tx=0).stop 2024-09-21T05:56:07.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.470+0000 7f5e95600640 1 -- 172.21.15.88:0/3744080835 shutdown_connections 2024-09-21T05:56:07.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.470+0000 7f5e95600640 1 --2- 172.21.15.88:0/3744080835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e90160330 0x7f5e9013bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.470+0000 7f5e95600640 1 --2- 172.21.15.88:0/3744080835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e900cefa0 0x7f5e9013d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.470+0000 7f5e95600640 1 --2- 172.21.15.88:0/3744080835 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e900749a0 0x7f5e90145b30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.470+0000 7f5e95600640 1 -- 172.21.15.88:0/3744080835 >> 172.21.15.88:0/3744080835 conn(0x7f5e900b2040 msgr2=0x7f5e90141c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:07.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.470+0000 7f5e95600640 1 -- 172.21.15.88:0/3744080835 shutdown_connections 2024-09-21T05:56:07.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:07.470+0000 7f5e95600640 1 -- 172.21.15.88:0/3744080835 wait complete. 2024-09-21T05:56:07.490 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T05:56:07.490 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T05:56:07.493 INFO:tasks.ceph.osd.0:Started 2024-09-21T05:56:07.493 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T05:56:07.493 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T05:56:07.496 INFO:tasks.ceph.osd.1:Started 2024-09-21T05:56:07.496 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T05:56:07.496 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T05:56:07.499 INFO:tasks.ceph.osd.2:Started 2024-09-21T05:56:07.499 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T05:56:07.499 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T05:56:07.502 INFO:tasks.ceph.osd.3:Started 2024-09-21T05:56:07.502 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T05:56:07.503 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T05:56:07.506 INFO:tasks.ceph.osd.4:Started 2024-09-21T05:56:07.506 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T05:56:07.507 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T05:56:07.509 INFO:tasks.ceph.osd.5:Started 2024-09-21T05:56:07.509 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T05:56:07.510 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T05:56:07.512 INFO:tasks.ceph.osd.6:Started 2024-09-21T05:56:07.513 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T05:56:07.513 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T05:56:07.516 INFO:tasks.ceph.osd.7:Started 2024-09-21T05:56:07.517 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:56:07.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.721+0000 7fcccee00640 1 -- 172.21.15.50:0/2970089027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccc80b5a80 msgr2=0x7fccc8074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.721+0000 7fcccee00640 1 --2- 172.21.15.50:0/2970089027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccc80b5a80 0x7fccc8074410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fccc80b6a50 tx=0x7fccb001c830 comp rx=0 tx=0).stop 2024-09-21T05:56:07.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.721+0000 7fcccee00640 1 -- 172.21.15.50:0/2970089027 shutdown_connections 2024-09-21T05:56:07.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.721+0000 7fcccee00640 1 --2- 172.21.15.50:0/2970089027 >> v2:172.21.15.88:3300/0 conn(0x7fccc8074950 0x7fccc815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.721+0000 7fcccee00640 1 --2- 172.21.15.50:0/2970089027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccc80b5a80 0x7fccc8074410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.721+0000 7fcccee00640 1 -- 172.21.15.50:0/2970089027 >> 172.21.15.50:0/2970089027 conn(0x7fccc80708c0 msgr2=0x7fccc8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:07.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.721+0000 7fcccee00640 1 -- 172.21.15.50:0/2970089027 shutdown_connections 2024-09-21T05:56:07.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.721+0000 7fcccee00640 1 -- 172.21.15.50:0/2970089027 wait complete. 2024-09-21T05:56:07.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.721+0000 7fcccee00640 1 Processor -- start 2024-09-21T05:56:07.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.721+0000 7fcccee00640 1 -- start start 2024-09-21T05:56:07.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fcccee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccc8074950 0x7fccc81f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fcccee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc80b5a80 0x7fccc80b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fcccee00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fccc8157030 0x7fccc81fc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fcccee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fccc816a310 con 0x7fccc8074950 2024-09-21T05:56:07.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fcccee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fccc816a190 con 0x7fccc80b5a80 2024-09-21T05:56:07.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fcccee00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fccc816a490 con 0x7fccc8157030 2024-09-21T05:56:07.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccc8074950 0x7fccc81f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:07.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccbf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc80b5a80 0x7fccc80b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:07.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccc8074950 0x7fccc81f9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45182/0 (socket says 172.21.15.50:45182) 2024-09-21T05:56:07.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc7e00640 1 -- 172.21.15.50:0/1943538185 learned_addr learned my addr 172.21.15.50:0/1943538185 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:07.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fcccce00640 1 --2- 172.21.15.50:0/1943538185 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fccc8157030 0x7fccc81fc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:07.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc7e00640 1 -- 172.21.15.50:0/1943538185 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fccc8157030 msgr2=0x7fccc81fc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc7e00640 1 --2- 172.21.15.50:0/1943538185 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fccc8157030 0x7fccc81fc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc7e00640 1 -- 172.21.15.50:0/1943538185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc80b5a80 msgr2=0x7fccc80b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc7e00640 1 --2- 172.21.15.50:0/1943538185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc80b5a80 0x7fccc80b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc7e00640 1 -- 172.21.15.50:0/1943538185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc81fd470 con 0x7fccc8074950 2024-09-21T05:56:07.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccbf400640 1 --2- 172.21.15.50:0/1943538185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc80b5a80 0x7fccc80b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:07.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc7e00640 1 --2- 172.21.15.50:0/1943538185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccc8074950 0x7fccc81f9e60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fccb800c760 tx=0x7fccb800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:07.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc4c00640 1 -- 172.21.15.50:0/1943538185 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8003b80 con 0x7fccc8074950 2024-09-21T05:56:07.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc4c00640 1 -- 172.21.15.50:0/1943538185 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccb8003d40 con 0x7fccc8074950 2024-09-21T05:56:07.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc4c00640 1 -- 172.21.15.50:0/1943538185 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb800aab0 con 0x7fccc8074950 2024-09-21T05:56:07.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fcccee00640 1 -- 172.21.15.50:0/1943538185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc81fd750 con 0x7fccc8074950 2024-09-21T05:56:07.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fcccee00640 1 -- 172.21.15.50:0/1943538185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc81f9660 con 0x7fccc8074950 2024-09-21T05:56:07.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc4c00640 1 -- 172.21.15.50:0/1943538185 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fccb800ac50 con 0x7fccc8074950 2024-09-21T05:56:07.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fccc4c00640 1 -- 172.21.15.50:0/1943538185 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fccb8012c30 con 0x7fccc8074950 2024-09-21T05:56:07.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.725+0000 7fcccee00640 1 -- 172.21.15.50:0/1943538185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc815cf90 con 0x7fccc8074950 2024-09-21T05:56:07.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.729+0000 7fccc4c00640 1 -- 172.21.15.50:0/1943538185 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fccc8074d60 con 0x7fccc8074950 2024-09-21T05:56:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.789+0000 7fcccee00640 1 -- 172.21.15.50:0/1943538185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccc80d2260 con 0x7fccc8074950 2024-09-21T05:56:07.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.789+0000 7fccc4c00640 1 -- 172.21.15.50:0/1943538185 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fccb8012560 con 0x7fccc8074950 2024-09-21T05:56:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:07.796 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":10,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:07.467198+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.793+0000 7fcccee00640 1 -- 172.21.15.50:0/1943538185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccc8074950 msgr2=0x7fccc81f9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.793+0000 7fcccee00640 1 --2- 172.21.15.50:0/1943538185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccc8074950 0x7fccc81f9e60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fccb800c760 tx=0x7fccb800cc20 comp rx=0 tx=0).stop 2024-09-21T05:56:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.793+0000 7fcccee00640 1 -- 172.21.15.50:0/1943538185 shutdown_connections 2024-09-21T05:56:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.793+0000 7fcccee00640 1 --2- 172.21.15.50:0/1943538185 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fccc8157030 0x7fccc81fc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.793+0000 7fcccee00640 1 --2- 172.21.15.50:0/1943538185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc80b5a80 0x7fccc80b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.793+0000 7fcccee00640 1 --2- 172.21.15.50:0/1943538185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccc8074950 0x7fccc81f9e60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.793+0000 7fcccee00640 1 -- 172.21.15.50:0/1943538185 >> 172.21.15.50:0/1943538185 conn(0x7fccc80708c0 msgr2=0x7fccc80d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.793+0000 7fcccee00640 1 -- 172.21.15.50:0/1943538185 shutdown_connections 2024-09-21T05:56:07.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:07.793+0000 7fcccee00640 1 -- 172.21.15.50:0/1943538185 wait complete. 2024-09-21T05:56:07.817 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T05:56:07.817 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T05:56:08.621 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T05:56:08.771 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.765+0000 7f74df200640 1 -- 172.21.15.50:0/3720020522 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74d8074b20 msgr2=0x7f74d815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:08.771 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.765+0000 7f74df200640 1 --2- 172.21.15.50:0/3720020522 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74d8074b20 0x7f74d815cf90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009bc0 tx=0x7f74d001cba0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.771 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.765+0000 7f74df200640 1 -- 172.21.15.50:0/3720020522 shutdown_connections 2024-09-21T05:56:08.771 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.765+0000 7f74df200640 1 --2- 172.21.15.50:0/3720020522 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74d8074b20 0x7f74d815cf90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.771 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.765+0000 7f74df200640 1 --2- 172.21.15.50:0/3720020522 >> v2:172.21.15.88:3300/0 conn(0x7f74d8157030 0x7f74d80b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.771 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.765+0000 7f74df200640 1 -- 172.21.15.50:0/3720020522 >> 172.21.15.50:0/3720020522 conn(0x7f74d80708c0 msgr2=0x7f74d8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:08.771 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.765+0000 7f74df200640 1 -- 172.21.15.50:0/3720020522 shutdown_connections 2024-09-21T05:56:08.771 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.765+0000 7f74df200640 1 -- 172.21.15.50:0/3720020522 wait complete. 2024-09-21T05:56:08.773 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74df200640 1 Processor -- start 2024-09-21T05:56:08.773 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74df200640 1 -- start start 2024-09-21T05:56:08.775 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74df200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74d8157030 0x7f74d8142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:08.775 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74df200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d8160d50 0x7f74d81f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:08.775 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74df200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d806d790 0x7f74d80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:08.775 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74df200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74d816a7e0 con 0x7f74d806d790 2024-09-21T05:56:08.775 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74df200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74d816a660 con 0x7f74d8160d50 2024-09-21T05:56:08.775 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74df200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f74d816a960 con 0x7f74d8157030 2024-09-21T05:56:08.776 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74dd200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d806d790 0x7f74d80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:08.776 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74dd200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d806d790 0x7f74d80b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45192/0 (socket says 172.21.15.50:45192) 2024-09-21T05:56:08.776 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74dd200640 1 -- 172.21.15.50:0/1640389455 learned_addr learned my addr 172.21.15.50:0/1640389455 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:08.776 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74d7e00640 1 --2- 172.21.15.50:0/1640389455 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74d8157030 0x7f74d8142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:08.776 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74cf400640 1 --2- 172.21.15.50:0/1640389455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d8160d50 0x7f74d81f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:08.776 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74dd200640 1 -- 172.21.15.50:0/1640389455 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74d8157030 msgr2=0x7f74d8142ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:08.776 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74dd200640 1 --2- 172.21.15.50:0/1640389455 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74d8157030 0x7f74d8142ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.776 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74dd200640 1 -- 172.21.15.50:0/1640389455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d8160d50 msgr2=0x7f74d81f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:08.776 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74dd200640 1 --2- 172.21.15.50:0/1640389455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d8160d50 0x7f74d81f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.776 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74dd200640 1 -- 172.21.15.50:0/1640389455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d81f95a0 con 0x7f74d806d790 2024-09-21T05:56:08.776 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74dd200640 1 --2- 172.21.15.50:0/1640389455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d806d790 0x7f74d80b2160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009b10 tx=0x7f74d0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:08.777 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74d4c00640 1 -- 172.21.15.50:0/1640389455 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00bb070 con 0x7f74d806d790 2024-09-21T05:56:08.777 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74d4c00640 1 -- 172.21.15.50:0/1640389455 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74d0004d40 con 0x7f74d806d790 2024-09-21T05:56:08.777 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74df200640 1 -- 172.21.15.50:0/1640389455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d81f9ab0 con 0x7f74d806d790 2024-09-21T05:56:08.777 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74d4c00640 1 -- 172.21.15.50:0/1640389455 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00b2040 con 0x7f74d806d790 2024-09-21T05:56:08.777 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74df200640 1 -- 172.21.15.50:0/1640389455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d806cf90 con 0x7f74d806d790 2024-09-21T05:56:08.778 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74d4c00640 1 -- 172.21.15.50:0/1640389455 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f74d00be070 con 0x7f74d806d790 2024-09-21T05:56:08.778 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.769+0000 7f74d4c00640 1 -- 172.21.15.50:0/1640389455 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f74d0005700 con 0x7f74d806d790 2024-09-21T05:56:08.778 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.773+0000 7f74df200640 1 -- 172.21.15.50:0/1640389455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d815cf90 con 0x7f74d806d790 2024-09-21T05:56:08.779 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.773+0000 7f74d4c00640 1 -- 172.21.15.50:0/1640389455 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f74d00d8860 con 0x7f74d806d790 2024-09-21T05:56:08.841 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.837+0000 7f74df200640 1 -- 172.21.15.50:0/1640389455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74d80d2260 con 0x7f74d806d790 2024-09-21T05:56:08.842 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.837+0000 7f74d4c00640 1 -- 172.21.15.50:0/1640389455 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f74d00c6030 con 0x7f74d806d790 2024-09-21T05:56:08.842 INFO:teuthology.misc.health.smithi050.stdout: 2024-09-21T05:56:08.842 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":10,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:07.467198+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:08.843 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.837+0000 7f74df200640 1 -- 172.21.15.50:0/1640389455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d806d790 msgr2=0x7f74d80b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:08.843 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.837+0000 7f74df200640 1 --2- 172.21.15.50:0/1640389455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d806d790 0x7f74d80b2160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009b10 tx=0x7f74d0003f60 comp rx=0 tx=0).stop 2024-09-21T05:56:08.843 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.837+0000 7f74df200640 1 -- 172.21.15.50:0/1640389455 shutdown_connections 2024-09-21T05:56:08.843 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.837+0000 7f74df200640 1 --2- 172.21.15.50:0/1640389455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d806d790 0x7f74d80b2160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.843 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.837+0000 7f74df200640 1 --2- 172.21.15.50:0/1640389455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d8160d50 0x7f74d81f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.843 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.837+0000 7f74df200640 1 --2- 172.21.15.50:0/1640389455 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74d8157030 0x7f74d8142ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.843 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.837+0000 7f74df200640 1 -- 172.21.15.50:0/1640389455 >> 172.21.15.50:0/1640389455 conn(0x7f74d80708c0 msgr2=0x7f74d815f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:08.843 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.837+0000 7f74df200640 1 -- 172.21.15.50:0/1640389455 shutdown_connections 2024-09-21T05:56:08.843 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:08.837+0000 7f74df200640 1 -- 172.21.15.50:0/1640389455 wait complete. 2024-09-21T05:56:08.861 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T05:56:09.158 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-09-21T05:56:09.153+0000 7fd177d657c0 -1 Falling back to public interface 2024-09-21T05:56:09.175 INFO:tasks.ceph.osd.5.smithi088.stderr:2024-09-21T05:56:09.174+0000 7f1ca890b7c0 -1 Falling back to public interface 2024-09-21T05:56:09.218 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-09-21T05:56:09.213+0000 7fa2b37a47c0 -1 Falling back to public interface 2024-09-21T05:56:09.218 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-09-21T05:56:09.213+0000 7fc57ea877c0 -1 Falling back to public interface 2024-09-21T05:56:09.219 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-09-21T05:56:09.213+0000 7fc57d600640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2024-09-21T05:56:09.219 INFO:tasks.ceph.osd.7.smithi088.stderr:2024-09-21T05:56:09.218+0000 7fe2614267c0 -1 Falling back to public interface 2024-09-21T05:56:09.243 INFO:tasks.ceph.osd.6.smithi088.stderr:2024-09-21T05:56:09.242+0000 7f42986807c0 -1 Falling back to public interface 2024-09-21T05:56:09.254 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-09-21T05:56:09.249+0000 7f3f723887c0 -1 Falling back to public interface 2024-09-21T05:56:09.367 INFO:tasks.ceph.osd.4.smithi088.stderr:2024-09-21T05:56:09.366+0000 7fb72723d7c0 -1 Falling back to public interface 2024-09-21T05:56:09.893 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:56:09.890+0000 7f65cb400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T05:56:09.893 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:56:09.890+0000 7f65cb400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T05:56:13.194 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-09-21T05:56:13.193+0000 7fd177d657c0 -1 osd.1 0 log_to_monitors true 2024-09-21T05:56:13.338 INFO:tasks.ceph.osd.5.smithi088.stderr:2024-09-21T05:56:13.337+0000 7f1ca890b7c0 -1 osd.5 0 log_to_monitors true 2024-09-21T05:56:13.598 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-09-21T05:56:13.597+0000 7fa2b37a47c0 -1 osd.0 0 log_to_monitors true 2024-09-21T05:56:13.667 INFO:tasks.ceph.osd.6.smithi088.stderr:2024-09-21T05:56:13.665+0000 7f42986807c0 -1 osd.6 0 log_to_monitors true 2024-09-21T05:56:13.714 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-09-21T05:56:13.713+0000 7f3f723887c0 -1 osd.2 0 log_to_monitors true 2024-09-21T05:56:13.828 INFO:tasks.ceph.osd.7.smithi088.stderr:2024-09-21T05:56:13.825+0000 7fe2614267c0 -1 osd.7 0 log_to_monitors true 2024-09-21T05:56:13.845 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-09-21T05:56:13.845+0000 7fc57ea877c0 -1 osd.3 0 log_to_monitors true 2024-09-21T05:56:13.991 INFO:tasks.ceph.osd.4.smithi088.stderr:2024-09-21T05:56:13.989+0000 7fb72723d7c0 -1 osd.4 0 log_to_monitors true 2024-09-21T05:56:15.449 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-09-21T05:56:15.445+0000 7fd173600640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T05:56:15.483 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-09-21T05:56:15.481+0000 7fd168c00640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:56:15.667 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T05:56:15.887 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.885+0000 7f3d97e00640 1 -- 172.21.15.50:0/1193222031 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d9815f5c0 msgr2=0x7f3d9815f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:15.888 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.885+0000 7f3d97e00640 1 --2- 172.21.15.50:0/1193222031 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d9815f5c0 0x7f3d9815f9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009c80 tx=0x7f3d9001cd10 comp rx=0 tx=0).stop 2024-09-21T05:56:15.888 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.885+0000 7f3d97e00640 1 -- 172.21.15.50:0/1193222031 shutdown_connections 2024-09-21T05:56:15.888 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.885+0000 7f3d97e00640 1 --2- 172.21.15.50:0/1193222031 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d9815f5c0 0x7f3d9815f9d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:15.888 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.885+0000 7f3d97e00640 1 --2- 172.21.15.50:0/1193222031 >> v2:172.21.15.50:3300/0 conn(0x7f3d9806dbc0 0x7f3d9806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:15.888 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.885+0000 7f3d97e00640 1 -- 172.21.15.50:0/1193222031 >> 172.21.15.50:0/1193222031 conn(0x7f3d980b41f0 msgr2=0x7f3d980b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:15.888 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.885+0000 7f3d97e00640 1 -- 172.21.15.50:0/1193222031 shutdown_connections 2024-09-21T05:56:15.888 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.885+0000 7f3d97e00640 1 -- 172.21.15.50:0/1193222031 wait complete. 2024-09-21T05:56:15.890 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.889+0000 7f3d97e00640 1 Processor -- start 2024-09-21T05:56:15.891 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.889+0000 7f3d97e00640 1 -- start start 2024-09-21T05:56:15.892 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.889+0000 7f3d97e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d9806dbc0 0x7f3d981598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:15.892 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.889+0000 7f3d97e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d980d1140 0x7f3d98165d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:15.893 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.889+0000 7f3d97e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d9815f5c0 0x7f3d98070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:15.894 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.889+0000 7f3d97e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3d98166fe0 con 0x7f3d9815f5c0 2024-09-21T05:56:15.894 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.889+0000 7f3d97e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3d98166e60 con 0x7f3d9806dbc0 2024-09-21T05:56:15.894 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.889+0000 7f3d97e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3d98167160 con 0x7f3d980d1140 2024-09-21T05:56:15.895 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.889+0000 7f3d96a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d9806dbc0 0x7f3d981598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:15.895 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.889+0000 7f3d96a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d9806dbc0 0x7f3d981598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:44022/0 (socket says 172.21.15.50:44022) 2024-09-21T05:56:15.895 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.889+0000 7f3d96a00640 1 -- 172.21.15.50:0/3495918061 learned_addr learned my addr 172.21.15.50:0/3495918061 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:15.895 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.889+0000 7f3d97400640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d9815f5c0 0x7f3d98070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:15.895 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d96000640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d980d1140 0x7f3d98165d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:15.895 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d97400640 1 -- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d980d1140 msgr2=0x7f3d98165d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:15.895 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d97400640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d980d1140 0x7f3d98165d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:15.895 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d97400640 1 -- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d9806dbc0 msgr2=0x7f3d981598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:15.895 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d97400640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d9806dbc0 0x7f3d981598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:15.895 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d97400640 1 -- 172.21.15.50:0/3495918061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9815b570 con 0x7f3d9815f5c0 2024-09-21T05:56:15.895 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d96a00640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d9806dbc0 0x7f3d981598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:15.895 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d96000640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d980d1140 0x7f3d98165d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:15.896 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d97400640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d9815f5c0 0x7f3d98070560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d980b2b90 tx=0x7f3d90009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:15.896 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d8f400640 1 -- 172.21.15.50:0/3495918061 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d900bb070 con 0x7f3d9815f5c0 2024-09-21T05:56:15.896 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d8f400640 1 -- 172.21.15.50:0/3495918061 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d90004d40 con 0x7f3d9815f5c0 2024-09-21T05:56:15.896 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d8f400640 1 -- 172.21.15.50:0/3495918061 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d900b2040 con 0x7f3d9815f5c0 2024-09-21T05:56:15.896 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d97e00640 1 -- 172.21.15.50:0/3495918061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d981fa9f0 con 0x7f3d9815f5c0 2024-09-21T05:56:15.896 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d97e00640 1 -- 172.21.15.50:0/3495918061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d98152fa0 con 0x7f3d9815f5c0 2024-09-21T05:56:15.896 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.893+0000 7f3d97e00640 1 -- 172.21.15.50:0/3495918061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d98153200 con 0x7f3d9815f5c0 2024-09-21T05:56:15.903 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.901+0000 7f3d8f400640 1 -- 172.21.15.50:0/3495918061 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 4) ==== 97178+0+0 (secure 0 0 0) 0x7f3d900be070 con 0x7f3d9815f5c0 2024-09-21T05:56:15.904 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.901+0000 7f3d8f400640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d780731f0 0x7f3d78075700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:15.905 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.905+0000 7f3d8f400640 1 -- 172.21.15.50:0/3495918061 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f3d9012d4f0 con 0x7f3d9815f5c0 2024-09-21T05:56:15.905 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.905+0000 7f3d96a00640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d780731f0 0x7f3d78075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:15.906 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.905+0000 7f3d96a00640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d780731f0 0x7f3d78075700 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3d88001c60 tx=0x7f3d88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:15.918 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:15.917+0000 7f3d8f400640 1 -- 172.21.15.50:0/3495918061 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3d900c5020 con 0x7f3d9815f5c0 2024-09-21T05:56:16.246 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.245+0000 7f3d97e00640 1 -- 172.21.15.50:0/3495918061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d9806dfd0 con 0x7f3d9815f5c0 2024-09-21T05:56:16.247 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.245+0000 7f3d8f400640 1 -- 172.21.15.50:0/3495918061 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f3d900169b0 con 0x7f3d9815f5c0 2024-09-21T05:56:16.248 INFO:teuthology.misc.health.smithi050.stdout: 2024-09-21T05:56:16.248 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":13,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:15.253863+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:16.251 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.249+0000 7f3d97e00640 1 -- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d780731f0 msgr2=0x7f3d78075700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:16.251 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.249+0000 7f3d97e00640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d780731f0 0x7f3d78075700 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3d88001c60 tx=0x7f3d88009040 comp rx=0 tx=0).stop 2024-09-21T05:56:16.251 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.249+0000 7f3d97e00640 1 -- 172.21.15.50:0/3495918061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d9815f5c0 msgr2=0x7f3d98070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:16.252 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.249+0000 7f3d97e00640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d9815f5c0 0x7f3d98070560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d980b2b90 tx=0x7f3d90009330 comp rx=0 tx=0).stop 2024-09-21T05:56:16.252 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.249+0000 7f3d97e00640 1 -- 172.21.15.50:0/3495918061 shutdown_connections 2024-09-21T05:56:16.252 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.249+0000 7f3d97e00640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d780731f0 0x7f3d78075700 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.252 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.249+0000 7f3d97e00640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d9815f5c0 0x7f3d98070560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.252 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.249+0000 7f3d97e00640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d980d1140 0x7f3d98165d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.252 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.249+0000 7f3d97e00640 1 --2- 172.21.15.50:0/3495918061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d9806dbc0 0x7f3d981598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.253 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.249+0000 7f3d97e00640 1 -- 172.21.15.50:0/3495918061 >> 172.21.15.50:0/3495918061 conn(0x7f3d980b41f0 msgr2=0x7f3d98157d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:16.253 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.249+0000 7f3d97e00640 1 -- 172.21.15.50:0/3495918061 shutdown_connections 2024-09-21T05:56:16.253 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:16.249+0000 7f3d97e00640 1 -- 172.21.15.50:0/3495918061 wait complete. 2024-09-21T05:56:16.285 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T05:56:16.604 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:56:16.609 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:56:16.895 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-09-21T05:56:16.893+0000 7fa2ae600640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T05:56:16.895 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-09-21T05:56:16.893+0000 7fc579a00640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T05:56:16.896 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-09-21T05:56:16.893+0000 7f3f6d200640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T05:56:16.926 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-09-21T05:56:16.925+0000 7fa2a4600640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:56:16.926 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-09-21T05:56:16.925+0000 7f3f63200640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:56:16.926 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-09-21T05:56:16.925+0000 7fc56fa00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:56:17.566 INFO:tasks.ceph.osd.7.smithi088.stderr:2024-09-21T05:56:17.565+0000 7fe25cc00640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T05:56:17.592 INFO:tasks.ceph.osd.7.smithi088.stderr:2024-09-21T05:56:17.589+0000 7fe252200640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:56:17.608 INFO:tasks.ceph.osd.6.smithi088.stderr:2024-09-21T05:56:17.605+0000 7f4293600640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T05:56:17.615 INFO:tasks.ceph.osd.4.smithi088.stderr:2024-09-21T05:56:17.613+0000 7fb722000640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T05:56:17.619 INFO:tasks.ceph.osd.5.smithi088.stderr:2024-09-21T05:56:17.617+0000 7f1ca4200640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T05:56:17.635 INFO:tasks.ceph.osd.6.smithi088.stderr:2024-09-21T05:56:17.633+0000 7f4289600640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:56:17.651 INFO:tasks.ceph.osd.5.smithi088.stderr:2024-09-21T05:56:17.649+0000 7f1c99800640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:56:17.651 INFO:tasks.ceph.osd.4.smithi088.stderr:2024-09-21T05:56:17.649+0000 7fb718000640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:56:23.092 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T05:56:23.254 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.252+0000 7f3e18000640 1 -- 172.21.15.50:0/2609857949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e100749a0 msgr2=0x7f3e10074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.254 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.252+0000 7f3e18000640 1 --2- 172.21.15.50:0/2609857949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e100749a0 0x7f3e10074db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e100cfc90 tx=0x7f3e0401c800 comp rx=0 tx=0).stop 2024-09-21T05:56:23.254 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.252+0000 7f3e18000640 1 -- 172.21.15.50:0/2609857949 shutdown_connections 2024-09-21T05:56:23.254 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.252+0000 7f3e18000640 1 --2- 172.21.15.50:0/2609857949 >> v2:172.21.15.88:3300/0 conn(0x7f3e100b3340 0x7f3e100b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.255 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.252+0000 7f3e18000640 1 --2- 172.21.15.50:0/2609857949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e100749a0 0x7f3e10074db0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.255 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.252+0000 7f3e18000640 1 -- 172.21.15.50:0/2609857949 >> 172.21.15.50:0/2609857949 conn(0x7f3e100b2040 msgr2=0x7f3e100b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:23.255 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.252+0000 7f3e18000640 1 -- 172.21.15.50:0/2609857949 shutdown_connections 2024-09-21T05:56:23.255 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.252+0000 7f3e18000640 1 -- 172.21.15.50:0/2609857949 wait complete. 2024-09-21T05:56:23.256 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.252+0000 7f3e18000640 1 Processor -- start 2024-09-21T05:56:23.256 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.252+0000 7f3e18000640 1 -- start start 2024-09-21T05:56:23.260 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e18000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e100749a0 0x7f3e101f9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:23.260 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e18000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e100b3340 0x7f3e1012d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:23.261 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e18000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e100cefa0 0x7f3e1013d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:23.261 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e18000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3e10169cd0 con 0x7f3e100b3340 2024-09-21T05:56:23.261 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e18000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3e10169b50 con 0x7f3e100749a0 2024-09-21T05:56:23.261 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e18000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3e10169e50 con 0x7f3e100cefa0 2024-09-21T05:56:23.261 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e14c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e100b3340 0x7f3e1012d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:23.261 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e14c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e100b3340 0x7f3e1012d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37714/0 (socket says 172.21.15.50:37714) 2024-09-21T05:56:23.261 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e14c00640 1 -- 172.21.15.50:0/4088339997 learned_addr learned my addr 172.21.15.50:0/4088339997 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:23.261 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e15600640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e100749a0 0x7f3e101f9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:23.261 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e16000640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e100cefa0 0x7f3e1013d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:23.261 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e14c00640 1 -- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e100cefa0 msgr2=0x7f3e1013d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.261 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e14c00640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e100cefa0 0x7f3e1013d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.262 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e14c00640 1 -- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e100749a0 msgr2=0x7f3e101f9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.262 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e14c00640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e100749a0 0x7f3e101f9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.262 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e14c00640 1 -- 172.21.15.50:0/4088339997 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e101f9d70 con 0x7f3e100b3340 2024-09-21T05:56:23.262 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e16000640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e100cefa0 0x7f3e1013d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:23.262 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e14c00640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e100b3340 0x7f3e1012d890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00c3a0 tx=0x7f3dfc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:23.262 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e0d600640 1 -- 172.21.15.50:0/4088339997 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc017070 con 0x7f3e100b3340 2024-09-21T05:56:23.262 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e0d600640 1 -- 172.21.15.50:0/4088339997 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dfc003600 con 0x7f3e100b3340 2024-09-21T05:56:23.262 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e18000640 1 -- 172.21.15.50:0/4088339997 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e101fa020 con 0x7f3e100b3340 2024-09-21T05:56:23.262 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e0d600640 1 -- 172.21.15.50:0/4088339997 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc003a50 con 0x7f3e100b3340 2024-09-21T05:56:23.262 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e18000640 1 -- 172.21.15.50:0/4088339997 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e101fa5e0 con 0x7f3e100b3340 2024-09-21T05:56:23.263 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.256+0000 7f3e18000640 1 -- 172.21.15.50:0/4088339997 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e10074db0 con 0x7f3e100b3340 2024-09-21T05:56:23.263 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.260+0000 7f3e0d600640 1 -- 172.21.15.50:0/4088339997 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3dfc01a070 con 0x7f3e100b3340 2024-09-21T05:56:23.263 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.260+0000 7f3e0d600640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3df0073200 0x7f3df0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:23.263 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.260+0000 7f3e0d600640 1 -- 172.21.15.50:0/4088339997 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f3dfc093140 con 0x7f3e100b3340 2024-09-21T05:56:23.263 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.260+0000 7f3e15600640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3df0073200 0x7f3df0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:23.264 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.260+0000 7f3e15600640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3df0073200 0x7f3df0075710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3e040af000 tx=0x7f3e0401b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:23.267 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.264+0000 7f3e0d600640 1 -- 172.21.15.50:0/4088339997 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3dfc05de50 con 0x7f3e100b3340 2024-09-21T05:56:23.445 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.444+0000 7f3e18000640 1 -- 172.21.15.50:0/4088339997 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e1006ec30 con 0x7f3e100b3340 2024-09-21T05:56:23.447 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.444+0000 7f3e0d600640 1 -- 172.21.15.50:0/4088339997 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12848 (secure 0 0 0) 0x7f3dfc062880 con 0x7f3e100b3340 2024-09-21T05:56:23.447 INFO:teuthology.misc.health.smithi050.stdout: 2024-09-21T05:56:23.447 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":18,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:20.926722+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:23.449 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.444+0000 7f3e18000640 1 -- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3df0073200 msgr2=0x7f3df0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.449 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.444+0000 7f3e18000640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3df0073200 0x7f3df0075710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3e040af000 tx=0x7f3e0401b040 comp rx=0 tx=0).stop 2024-09-21T05:56:23.449 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.444+0000 7f3e18000640 1 -- 172.21.15.50:0/4088339997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e100b3340 msgr2=0x7f3e1012d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.449 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.444+0000 7f3e18000640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e100b3340 0x7f3e1012d890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00c3a0 tx=0x7f3dfc00c860 comp rx=0 tx=0).stop 2024-09-21T05:56:23.449 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.444+0000 7f3e18000640 1 -- 172.21.15.50:0/4088339997 shutdown_connections 2024-09-21T05:56:23.449 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.444+0000 7f3e18000640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e100cefa0 0x7f3e1013d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.449 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.444+0000 7f3e18000640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3df0073200 0x7f3df0075710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.449 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.444+0000 7f3e18000640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e100b3340 0x7f3e1012d890 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.450 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.444+0000 7f3e18000640 1 --2- 172.21.15.50:0/4088339997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e100749a0 0x7f3e101f9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.450 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.444+0000 7f3e18000640 1 -- 172.21.15.50:0/4088339997 >> 172.21.15.50:0/4088339997 conn(0x7f3e100b2040 msgr2=0x7f3e100b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:23.450 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.448+0000 7f3e18000640 1 -- 172.21.15.50:0/4088339997 shutdown_connections 2024-09-21T05:56:23.450 INFO:teuthology.misc.health.smithi050.stderr:2024-09-21T05:56:23.448+0000 7f3e18000640 1 -- 172.21.15.50:0/4088339997 wait complete. 2024-09-21T05:56:23.467 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T05:56:23.467 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T05:56:23.467 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T05:56:23.468 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T05:56:23.471 INFO:tasks.ceph.mds.a:Started 2024-09-21T05:56:23.471 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T05:56:23.471 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T05:56:23.474 INFO:tasks.ceph.mds.c:Started 2024-09-21T05:56:23.474 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T05:56:23.474 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T05:56:23.477 INFO:tasks.ceph.mds.b:Started 2024-09-21T05:56:23.477 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T05:56:23.477 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T05:56:23.481 INFO:tasks.ceph.mds.d:Started 2024-09-21T05:56:23.481 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-21T05:56:23.481 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:56:23.578 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-09-21T05:56:23.581 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T05:56:23.602 INFO:tasks.ceph.mds.d.smithi088.stdout:starting mds.d at 2024-09-21T05:56:23.604 INFO:tasks.ceph.mds.b.smithi088.stdout:starting mds.b at 2024-09-21T05:56:23.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.644+0000 7fa908c00640 1 -- 172.21.15.50:0/2462918991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90415d280 msgr2=0x7fa90415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.644+0000 7fa908c00640 1 --2- 172.21.15.50:0/2462918991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90415d280 0x7fa90415d690 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009840 tx=0x7fa8f001c820 comp rx=0 tx=0).stop 2024-09-21T05:56:23.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.644+0000 7fa908c00640 1 -- 172.21.15.50:0/2462918991 shutdown_connections 2024-09-21T05:56:23.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.644+0000 7fa908c00640 1 --2- 172.21.15.50:0/2462918991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90415d280 0x7fa90415d690 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.644+0000 7fa908c00640 1 -- 172.21.15.50:0/2462918991 >> 172.21.15.50:0/2462918991 conn(0x7fa9040b2040 msgr2=0x7fa9040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:23.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.644+0000 7fa908c00640 1 -- 172.21.15.50:0/2462918991 shutdown_connections 2024-09-21T05:56:23.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.644+0000 7fa908c00640 1 -- 172.21.15.50:0/2462918991 wait complete. 2024-09-21T05:56:23.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa908c00640 1 Processor -- start 2024-09-21T05:56:23.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa908c00640 1 -- start start 2024-09-21T05:56:23.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa908c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90415d280 0x7fa90413d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:23.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa908c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa90415dbd0 0x7fa904158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:23.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa908c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa90415e8c0 0x7fa9041f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:23.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa908c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa904169480 con 0x7fa90415d280 2024-09-21T05:56:23.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa908c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9040b3b50 con 0x7fa90415dbd0 2024-09-21T05:56:23.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa908c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa904169600 con 0x7fa90415e8c0 2024-09-21T05:56:23.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa901e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90415d280 0x7fa90413d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:23.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa901e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90415d280 0x7fa90413d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37748/0 (socket says 172.21.15.50:37748) 2024-09-21T05:56:23.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa901e00640 1 -- 172.21.15.50:0/2483235485 learned_addr learned my addr 172.21.15.50:0/2483235485 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:23.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa901400640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa90415dbd0 0x7fa904158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:23.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa902800640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa90415e8c0 0x7fa9041f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:23.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa901e00640 1 -- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa90415e8c0 msgr2=0x7fa9041f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa901e00640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa90415e8c0 0x7fa9041f8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa901e00640 1 -- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa90415dbd0 msgr2=0x7fa904158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa901e00640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa90415dbd0 0x7fa904158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa901e00640 1 -- 172.21.15.50:0/2483235485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa904157540 con 0x7fa90415d280 2024-09-21T05:56:23.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa901e00640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90415d280 0x7fa90413d0f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c3c0 tx=0x7fa8f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:23.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa8fa000640 1 -- 172.21.15.50:0/2483235485 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4017070 con 0x7fa90415d280 2024-09-21T05:56:23.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa8fa000640 1 -- 172.21.15.50:0/2483235485 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8f40053c0 con 0x7fa90415d280 2024-09-21T05:56:23.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa908c00640 1 -- 172.21.15.50:0/2483235485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90413e070 con 0x7fa90415d280 2024-09-21T05:56:23.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa8fa000640 1 -- 172.21.15.50:0/2483235485 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4003860 con 0x7fa90415d280 2024-09-21T05:56:23.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.648+0000 7fa908c00640 1 -- 172.21.15.50:0/2483235485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa904158b60 con 0x7fa90415d280 2024-09-21T05:56:23.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.652+0000 7fa908c00640 1 -- 172.21.15.50:0/2483235485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90415d690 con 0x7fa90415d280 2024-09-21T05:56:23.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.652+0000 7fa8fa000640 1 -- 172.21.15.50:0/2483235485 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa8f401a070 con 0x7fa90415d280 2024-09-21T05:56:23.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.652+0000 7fa8fa000640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa8e4073200 0x7fa8e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:23.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.652+0000 7fa8fa000640 1 -- 172.21.15.50:0/2483235485 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fa8f4093180 con 0x7fa90415d280 2024-09-21T05:56:23.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.652+0000 7fa901400640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa8e4073200 0x7fa8e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:23.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.656+0000 7fa901400640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa8e4073200 0x7fa8e4075710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00af000 tx=0x7fa8f001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:23.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.656+0000 7fa8fa000640 1 -- 172.21.15.50:0/2483235485 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa8f405df20 con 0x7fa90415d280 2024-09-21T05:56:23.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.872+0000 7fa908c00640 1 -- 172.21.15.50:0/2483235485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9040b1ad0 con 0x7fa90415d280 2024-09-21T05:56:23.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.872+0000 7fa8fa000640 1 -- 172.21.15.50:0/2483235485 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7fa8f4062950 con 0x7fa90415d280 2024-09-21T05:56:23.876 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:23.876 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":2,"btime":"2024-09-21T05:56:23:584973+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/606533049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":606533049},{"type":"v1","addr":"172.21.15.50:6833","nonce":606533049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T05:56:23.876 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 2 2024-09-21T05:56:23.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.876+0000 7fa908c00640 1 -- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa8e4073200 msgr2=0x7fa8e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.876+0000 7fa908c00640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa8e4073200 0x7fa8e4075710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00af000 tx=0x7fa8f001b060 comp rx=0 tx=0).stop 2024-09-21T05:56:23.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.876+0000 7fa908c00640 1 -- 172.21.15.50:0/2483235485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90415d280 msgr2=0x7fa90413d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.876+0000 7fa908c00640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90415d280 0x7fa90413d0f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c3c0 tx=0x7fa8f400c880 comp rx=0 tx=0).stop 2024-09-21T05:56:23.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.876+0000 7fa908c00640 1 -- 172.21.15.50:0/2483235485 shutdown_connections 2024-09-21T05:56:23.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.876+0000 7fa908c00640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa90415e8c0 0x7fa9041f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.876+0000 7fa908c00640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa90415dbd0 0x7fa904158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.876+0000 7fa908c00640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa8e4073200 0x7fa8e4075710 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.876+0000 7fa908c00640 1 --2- 172.21.15.50:0/2483235485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90415d280 0x7fa90413d0f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.876+0000 7fa908c00640 1 -- 172.21.15.50:0/2483235485 >> 172.21.15.50:0/2483235485 conn(0x7fa9040b2040 msgr2=0x7fa9040cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:23.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.876+0000 7fa908c00640 1 -- 172.21.15.50:0/2483235485 shutdown_connections 2024-09-21T05:56:23.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:23.876+0000 7fa908c00640 1 -- 172.21.15.50:0/2483235485 wait complete. 2024-09-21T05:56:25.899 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:56:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.068+0000 7fc10c400640 1 -- 172.21.15.50:0/1288543463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0ec004810 msgr2=0x7fc0ec004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.068+0000 7fc10c400640 1 --2- 172.21.15.50:0/1288543463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0ec004810 0x7fc0ec004c20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00016f0 tx=0x7fc0f001c720 comp rx=0 tx=0).stop 2024-09-21T05:56:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.068+0000 7fc10c400640 1 -- 172.21.15.50:0/1288543463 shutdown_connections 2024-09-21T05:56:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.068+0000 7fc10c400640 1 --2- 172.21.15.50:0/1288543463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0ec004810 0x7fc0ec004c20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.068+0000 7fc10c400640 1 -- 172.21.15.50:0/1288543463 >> 172.21.15.50:0/1288543463 conn(0x7fc1040b2040 msgr2=0x7fc1040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.068+0000 7fc10c400640 1 -- 172.21.15.50:0/1288543463 shutdown_connections 2024-09-21T05:56:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.068+0000 7fc10c400640 1 -- 172.21.15.50:0/1288543463 wait complete. 2024-09-21T05:56:26.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.068+0000 7fc10c400640 1 Processor -- start 2024-09-21T05:56:26.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc10c400640 1 -- start start 2024-09-21T05:56:26.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc10c400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0ec004810 0x7fc104155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc10c400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10414d010 0x7fc10414c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc10c400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc10414f1f0 0x7fc10414f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc10c400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc10415f510 con 0x7fc10414d010 2024-09-21T05:56:26.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc10c400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc10415f390 con 0x7fc0ec004810 2024-09-21T05:56:26.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc10c400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc10415f690 con 0x7fc10414f1f0 2024-09-21T05:56:26.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc109000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10414d010 0x7fc10414c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc109000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10414d010 0x7fc10414c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37776/0 (socket says 172.21.15.50:37776) 2024-09-21T05:56:26.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc109000640 1 -- 172.21.15.50:0/548903083 learned_addr learned my addr 172.21.15.50:0/548903083 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:26.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc109a00640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0ec004810 0x7fc104155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc10a400640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc10414f1f0 0x7fc10414f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc109000640 1 -- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc10414f1f0 msgr2=0x7fc10414f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc109000640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc10414f1f0 0x7fc10414f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc109000640 1 -- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0ec004810 msgr2=0x7fc104155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc109000640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0ec004810 0x7fc104155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc109000640 1 -- 172.21.15.50:0/548903083 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10415c8b0 con 0x7fc10414d010 2024-09-21T05:56:26.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc109000640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10414d010 0x7fc10414c850 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc0f001ae20 tx=0x7fc0f0003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:26.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc101600640 1 -- 172.21.15.50:0/548903083 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f00b9070 con 0x7fc10414d010 2024-09-21T05:56:26.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc101600640 1 -- 172.21.15.50:0/548903083 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0f00ae040 con 0x7fc10414d010 2024-09-21T05:56:26.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc10c400640 1 -- 172.21.15.50:0/548903083 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc104155bc0 con 0x7fc10414d010 2024-09-21T05:56:26.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc101600640 1 -- 172.21.15.50:0/548903083 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f00ad070 con 0x7fc10414d010 2024-09-21T05:56:26.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.072+0000 7fc10c400640 1 -- 172.21.15.50:0/548903083 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10414cd90 con 0x7fc10414d010 2024-09-21T05:56:26.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.076+0000 7fc10c400640 1 -- 172.21.15.50:0/548903083 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10415fbf0 con 0x7fc10414d010 2024-09-21T05:56:26.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.076+0000 7fc101600640 1 -- 172.21.15.50:0/548903083 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc0f0006350 con 0x7fc10414d010 2024-09-21T05:56:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.076+0000 7fc101600640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc0e4073200 0x7fc0e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.076+0000 7fc101600640 1 -- 172.21.15.50:0/548903083 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fc0f012d390 con 0x7fc10414d010 2024-09-21T05:56:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.076+0000 7fc109a00640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc0e4073200 0x7fc0e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.076+0000 7fc109a00640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc0e4073200 0x7fc0e4075710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8001ec0 tx=0x7fc0f8006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.080+0000 7fc101600640 1 -- 172.21.15.50:0/548903083 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc0f00164b0 con 0x7fc10414d010 2024-09-21T05:56:26.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.292+0000 7fc10c400640 1 -- 172.21.15.50:0/548903083 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc104155d90 con 0x7fc10414d010 2024-09-21T05:56:26.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.292+0000 7fc101600640 1 -- 172.21.15.50:0/548903083 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3900 (secure 0 0 0) 0x7fc0f00f8120 con 0x7fc10414d010 2024-09-21T05:56:26.296 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:26.296 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":3,"btime":"2024-09-21T05:56:24:586391+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3883511106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3883511106},{"type":"v1","addr":"172.21.15.88:6836","nonce":3883511106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/606533049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":606533049},{"type":"v1","addr":"172.21.15.50:6833","nonce":606533049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/183287131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":183287131},{"type":"v1","addr":"172.21.15.50:6835","nonce":183287131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1479043500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1479043500},{"type":"v1","addr":"172.21.15.88:6837","nonce":1479043500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T05:56:26.296 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 3 2024-09-21T05:56:26.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.296+0000 7fc10c400640 1 -- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc0e4073200 msgr2=0x7fc0e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.296+0000 7fc10c400640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc0e4073200 0x7fc0e4075710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8001ec0 tx=0x7fc0f8006040 comp rx=0 tx=0).stop 2024-09-21T05:56:26.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.296+0000 7fc10c400640 1 -- 172.21.15.50:0/548903083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10414d010 msgr2=0x7fc10414c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.296+0000 7fc10c400640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10414d010 0x7fc10414c850 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc0f001ae20 tx=0x7fc0f0003be0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.296+0000 7fc10c400640 1 -- 172.21.15.50:0/548903083 shutdown_connections 2024-09-21T05:56:26.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.296+0000 7fc10c400640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc10414f1f0 0x7fc10414f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.296+0000 7fc10c400640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc0e4073200 0x7fc0e4075710 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.296+0000 7fc10c400640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10414d010 0x7fc10414c850 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.296+0000 7fc10c400640 1 --2- 172.21.15.50:0/548903083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0ec004810 0x7fc104155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.296+0000 7fc10c400640 1 -- 172.21.15.50:0/548903083 >> 172.21.15.50:0/548903083 conn(0x7fc1040b2040 msgr2=0x7fc104159e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:26.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.296+0000 7fc10c400640 1 -- 172.21.15.50:0/548903083 shutdown_connections 2024-09-21T05:56:26.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.296+0000 7fc10c400640 1 -- 172.21.15.50:0/548903083 wait complete. 2024-09-21T05:56:26.318 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T05:56:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.472+0000 7f54b8a00640 1 -- 172.21.15.50:0/3879106598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54b40749a0 msgr2=0x7f54b4074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.472+0000 7f54b8a00640 1 --2- 172.21.15.50:0/3879106598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54b40749a0 0x7f54b4074db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f54b40726f0 tx=0x7f54a401c8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.472+0000 7f54b8a00640 1 -- 172.21.15.50:0/3879106598 shutdown_connections 2024-09-21T05:56:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.472+0000 7f54b8a00640 1 --2- 172.21.15.50:0/3879106598 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54b4160330 0x7f54b4160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.472+0000 7f54b8a00640 1 --2- 172.21.15.50:0/3879106598 >> v2:172.21.15.88:3300/0 conn(0x7f54b40cefa0 0x7f54b40cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.472+0000 7f54b8a00640 1 --2- 172.21.15.50:0/3879106598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54b40749a0 0x7f54b4074db0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.472+0000 7f54b8a00640 1 -- 172.21.15.50:0/3879106598 >> 172.21.15.50:0/3879106598 conn(0x7f54b40b2040 msgr2=0x7f54b40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:26.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.472+0000 7f54b8a00640 1 -- 172.21.15.50:0/3879106598 shutdown_connections 2024-09-21T05:56:26.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.472+0000 7f54b8a00640 1 -- 172.21.15.50:0/3879106598 wait complete. 2024-09-21T05:56:26.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b8a00640 1 Processor -- start 2024-09-21T05:56:26.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b8a00640 1 -- start start 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b8a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54b40749a0 0x7f54b40af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54b40cefa0 0x7f54b41445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b8a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54b4160330 0x7f54b41432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b8a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f54b40b3ee0 con 0x7f54b4160330 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b8a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f54b40b3d60 con 0x7f54b40cefa0 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b8a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f54b40b4060 con 0x7f54b40749a0 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54b4160330 0x7f54b41432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54b4160330 0x7f54b41432c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37790/0 (socket says 172.21.15.50:37790) 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b2800640 1 -- 172.21.15.50:0/1167916004 learned_addr learned my addr 172.21.15.50:0/1167916004 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b1e00640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54b40749a0 0x7f54b40af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b2800640 1 -- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54b40749a0 msgr2=0x7f54b40af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b2800640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54b40749a0 0x7f54b40af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b2800640 1 -- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54b40cefa0 msgr2=0x7f54b41445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b1400640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54b40cefa0 0x7f54b41445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b2800640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54b40cefa0 0x7f54b41445f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b2800640 1 -- 172.21.15.50:0/1167916004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b40ae620 con 0x7f54b4160330 2024-09-21T05:56:26.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b1e00640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54b40749a0 0x7f54b40af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:26.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.476+0000 7f54b1400640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54b40cefa0 0x7f54b41445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:26.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.480+0000 7f54b2800640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54b4160330 0x7f54b41432c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009fd0 tx=0x7f54ac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.480+0000 7f54aa000640 1 -- 172.21.15.50:0/1167916004 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac00ce20 con 0x7f54b4160330 2024-09-21T05:56:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.480+0000 7f54aa000640 1 -- 172.21.15.50:0/1167916004 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54ac004b30 con 0x7f54b4160330 2024-09-21T05:56:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.480+0000 7f54aa000640 1 -- 172.21.15.50:0/1167916004 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac005020 con 0x7f54b4160330 2024-09-21T05:56:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.480+0000 7f54b8a00640 1 -- 172.21.15.50:0/1167916004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b40afba0 con 0x7f54b4160330 2024-09-21T05:56:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.480+0000 7f54b8a00640 1 -- 172.21.15.50:0/1167916004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b40aeb80 con 0x7f54b4160330 2024-09-21T05:56:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.480+0000 7f54b8a00640 1 -- 172.21.15.50:0/1167916004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b40cf3b0 con 0x7f54b4160330 2024-09-21T05:56:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.480+0000 7f54aa000640 1 -- 172.21.15.50:0/1167916004 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f54ac0051c0 con 0x7f54b4160330 2024-09-21T05:56:26.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.480+0000 7f54aa000640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5494073200 0x7f5494075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.480+0000 7f54aa000640 1 -- 172.21.15.50:0/1167916004 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f54ac094c40 con 0x7f54b4160330 2024-09-21T05:56:26.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.480+0000 7f54b1e00640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5494073200 0x7f5494075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.484+0000 7f54b1e00640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5494073200 0x7f5494075710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54a40af000 tx=0x7f54a401b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.484+0000 7f54aa000640 1 -- 172.21.15.50:0/1167916004 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f54b40ae620 con 0x7f54b4160330 2024-09-21T05:56:26.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 -- 172.21.15.50:0/1167916004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f54b40b0ea0 con 0x7f54b4160330 2024-09-21T05:56:26.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54aa000640 1 -- 172.21.15.50:0/1167916004 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f54ac05ff10 con 0x7f54b4160330 2024-09-21T05:56:26.667 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:26.667 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T05:56:26.667 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:56:26.667 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-09-21T05:56:26.667 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-09-21T05:56:26.667 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:56:26.667 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T05:56:26.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 -- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5494073200 msgr2=0x7f5494075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5494073200 0x7f5494075710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54a40af000 tx=0x7f54a401b100 comp rx=0 tx=0).stop 2024-09-21T05:56:26.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 -- 172.21.15.50:0/1167916004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54b4160330 msgr2=0x7f54b41432c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54b4160330 0x7f54b41432c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009fd0 tx=0x7f54ac00ef10 comp rx=0 tx=0).stop 2024-09-21T05:56:26.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 -- 172.21.15.50:0/1167916004 shutdown_connections 2024-09-21T05:56:26.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5494073200 0x7f5494075710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54b4160330 0x7f54b41432c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54b40cefa0 0x7f54b41445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 --2- 172.21.15.50:0/1167916004 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54b40749a0 0x7f54b40af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 -- 172.21.15.50:0/1167916004 >> 172.21.15.50:0/1167916004 conn(0x7f54b40b2040 msgr2=0x7f54b40b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:26.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 -- 172.21.15.50:0/1167916004 shutdown_connections 2024-09-21T05:56:26.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.664+0000 7f54b8a00640 1 -- 172.21.15.50:0/1167916004 wait complete. 2024-09-21T05:56:26.689 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T05:56:26.689 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T05:56:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.848+0000 7fac86c00640 1 -- 172.21.15.50:0/2979453771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac64003a60 msgr2=0x7fac64003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.848+0000 7fac86c00640 1 --2- 172.21.15.50:0/2979453771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac64003a60 0x7fac64003ef0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fac700016f0 tx=0x7fac7001c720 comp rx=0 tx=0).stop 2024-09-21T05:56:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.848+0000 7fac86c00640 1 -- 172.21.15.50:0/2979453771 shutdown_connections 2024-09-21T05:56:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.848+0000 7fac86c00640 1 --2- 172.21.15.50:0/2979453771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac64003a60 0x7fac64003ef0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.848+0000 7fac86c00640 1 -- 172.21.15.50:0/2979453771 >> 172.21.15.50:0/2979453771 conn(0x7fac800b2040 msgr2=0x7fac800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.848+0000 7fac86c00640 1 -- 172.21.15.50:0/2979453771 shutdown_connections 2024-09-21T05:56:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.848+0000 7fac86c00640 1 -- 172.21.15.50:0/2979453771 wait complete. 2024-09-21T05:56:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.852+0000 7fac86c00640 1 Processor -- start 2024-09-21T05:56:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.852+0000 7fac86c00640 1 -- start start 2024-09-21T05:56:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.852+0000 7fac86c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac64003a60 0x7fac800af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.852+0000 7fac86c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac8015a910 0x7fac800ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.852+0000 7fac86c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac800aa4d0 0x7fac800aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.852+0000 7fac86c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fac800b5690 con 0x7fac800aa4d0 2024-09-21T05:56:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.852+0000 7fac86c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fac800b5510 con 0x7fac8015a910 2024-09-21T05:56:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.852+0000 7fac86c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fac8015f180 con 0x7fac64003a60 2024-09-21T05:56:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.852+0000 7fac84c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac800aa4d0 0x7fac800aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.852+0000 7fac84c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac800aa4d0 0x7fac800aa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43088/0 (socket says 172.21.15.50:43088) 2024-09-21T05:56:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.852+0000 7fac84c00640 1 -- 172.21.15.50:0/1530082650 learned_addr learned my addr 172.21.15.50:0/1530082650 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac7fe00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac64003a60 0x7fac800af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac77e00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac8015a910 0x7fac800ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac84c00640 1 -- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac64003a60 msgr2=0x7fac800af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac84c00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac64003a60 0x7fac800af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac84c00640 1 -- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac8015a910 msgr2=0x7fac800ab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac84c00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac8015a910 0x7fac800ab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac84c00640 1 -- 172.21.15.50:0/1530082650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac800ae550 con 0x7fac800aa4d0 2024-09-21T05:56:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac84c00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac800aa4d0 0x7fac800aa8e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fac80152bf0 tx=0x7fac78006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac7cc00640 1 -- 172.21.15.50:0/1530082650 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7800fa80 con 0x7fac800aa4d0 2024-09-21T05:56:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac7cc00640 1 -- 172.21.15.50:0/1530082650 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac7800fc40 con 0x7fac800aa4d0 2024-09-21T05:56:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac7cc00640 1 -- 172.21.15.50:0/1530082650 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78011430 con 0x7fac800aa4d0 2024-09-21T05:56:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac86c00640 1 -- 172.21.15.50:0/1530082650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac800ae830 con 0x7fac800aa4d0 2024-09-21T05:56:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac86c00640 1 -- 172.21.15.50:0/1530082650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac800aedf0 con 0x7fac800aa4d0 2024-09-21T05:56:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac86c00640 1 -- 172.21.15.50:0/1530082650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8015f670 con 0x7fac800aa4d0 2024-09-21T05:56:26.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.856+0000 7fac7cc00640 1 -- 172.21.15.50:0/1530082650 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fac780115d0 con 0x7fac800aa4d0 2024-09-21T05:56:26.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.860+0000 7fac7cc00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac5c073230 0x7fac5c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.860+0000 7fac7cc00640 1 -- 172.21.15.50:0/1530082650 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fac780946a0 con 0x7fac800aa4d0 2024-09-21T05:56:26.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.860+0000 7fac7fe00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac5c073230 0x7fac5c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.860+0000 7fac7fe00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac5c073230 0x7fac5c075740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fac700016c0 tx=0x7fac70019510 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:26.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:26.864+0000 7fac7cc00640 1 -- 172.21.15.50:0/1530082650 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fac7805fa00 con 0x7fac800aa4d0 2024-09-21T05:56:27.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.040+0000 7fac86c00640 1 -- 172.21.15.50:0/1530082650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fac8015ee40 con 0x7fac800aa4d0 2024-09-21T05:56:27.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.748+0000 7fac7cc00640 1 -- 172.21.15.50:0/1530082650 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7fac7805f300 con 0x7fac800aa4d0 2024-09-21T05:56:27.753 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-09-21T05:56:27.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.752+0000 7fac86c00640 1 -- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac5c073230 msgr2=0x7fac5c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:27.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.752+0000 7fac86c00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac5c073230 0x7fac5c075740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fac700016c0 tx=0x7fac70019510 comp rx=0 tx=0).stop 2024-09-21T05:56:27.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.752+0000 7fac86c00640 1 -- 172.21.15.50:0/1530082650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac800aa4d0 msgr2=0x7fac800aa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:27.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.752+0000 7fac86c00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac800aa4d0 0x7fac800aa8e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fac80152bf0 tx=0x7fac78006660 comp rx=0 tx=0).stop 2024-09-21T05:56:27.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.752+0000 7fac86c00640 1 -- 172.21.15.50:0/1530082650 shutdown_connections 2024-09-21T05:56:27.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.752+0000 7fac86c00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac5c073230 0x7fac5c075740 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:27.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.752+0000 7fac86c00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac800aa4d0 0x7fac800aa8e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:27.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.752+0000 7fac86c00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac8015a910 0x7fac800ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:27.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.752+0000 7fac86c00640 1 --2- 172.21.15.50:0/1530082650 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac64003a60 0x7fac800af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:27.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.752+0000 7fac86c00640 1 -- 172.21.15.50:0/1530082650 >> 172.21.15.50:0/1530082650 conn(0x7fac800b2040 msgr2=0x7fac8015d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:27.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.752+0000 7fac86c00640 1 -- 172.21.15.50:0/1530082650 shutdown_connections 2024-09-21T05:56:27.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.752+0000 7fac86c00640 1 -- 172.21.15.50:0/1530082650 wait complete. 2024-09-21T05:56:27.788 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T05:56:27.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.956+0000 7f219a200640 1 -- 172.21.15.50:0/2178710719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2178003fe0 msgr2=0x7f2178004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.956+0000 7f219a200640 1 --2- 172.21.15.50:0/2178710719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2178003fe0 0x7f2178004810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f21940d14d0 tx=0x7f217c01cee0 comp rx=0 tx=0).stop 2024-09-21T05:56:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.956+0000 7f2193e00640 1 -- 172.21.15.50:0/2178710719 reap_dead start 2024-09-21T05:56:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.956+0000 7f219a200640 1 -- 172.21.15.50:0/2178710719 shutdown_connections 2024-09-21T05:56:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.956+0000 7f219a200640 1 -- 172.21.15.50:0/2178710719 >> 172.21.15.50:0/2178710719 conn(0x7f21940b35f0 msgr2=0x7f21940b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.956+0000 7f219a200640 1 -- 172.21.15.50:0/2178710719 shutdown_connections 2024-09-21T05:56:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.956+0000 7f219a200640 1 -- 172.21.15.50:0/2178710719 wait complete. 2024-09-21T05:56:27.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.956+0000 7f219a200640 1 Processor -- start 2024-09-21T05:56:27.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.956+0000 7f219a200640 1 -- start start 2024-09-21T05:56:27.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f219a200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2178003fe0 0x7f21941f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f219a200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21940b0000 0x7f21941633b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f219a200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f219415e500 0x7f21940af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f219a200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2194166c00 con 0x7f21940b0000 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f219a200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2194166a80 con 0x7f2178003fe0 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f219a200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2194166d80 con 0x7f219415e500 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f218aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21940b0000 0x7f21941633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f2193400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2178003fe0 0x7f21941f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f218aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21940b0000 0x7f21941633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43106/0 (socket says 172.21.15.50:43106) 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f218aa00640 1 -- 172.21.15.50:0/619671143 learned_addr learned my addr 172.21.15.50:0/619671143 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f2193e00640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f219415e500 0x7f21940af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f218aa00640 1 -- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f219415e500 msgr2=0x7f21940af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f218aa00640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f219415e500 0x7f21940af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f218aa00640 1 -- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2178003fe0 msgr2=0x7f21941f8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f218aa00640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2178003fe0 0x7f21941f8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:27.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f218aa00640 1 -- 172.21.15.50:0/619671143 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21940ae8e0 con 0x7f21940b0000 2024-09-21T05:56:27.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f2193400640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2178003fe0 0x7f21941f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:27.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f218aa00640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21940b0000 0x7f21941633b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f217c01c9a0 tx=0x7f217c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:27.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f218be00640 1 -- 172.21.15.50:0/619671143 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c001690 con 0x7f21940b0000 2024-09-21T05:56:27.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f218be00640 1 -- 172.21.15.50:0/619671143 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f217c006e50 con 0x7f21940b0000 2024-09-21T05:56:27.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f219a200640 1 -- 172.21.15.50:0/619671143 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21941f98e0 con 0x7f21940b0000 2024-09-21T05:56:27.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f218be00640 1 -- 172.21.15.50:0/619671143 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c006090 con 0x7f21940b0000 2024-09-21T05:56:27.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f219a200640 1 -- 172.21.15.50:0/619671143 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21941f9ea0 con 0x7f21940b0000 2024-09-21T05:56:27.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.960+0000 7f219a200640 1 -- 172.21.15.50:0/619671143 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21941f9650 con 0x7f21940b0000 2024-09-21T05:56:27.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.964+0000 7f218be00640 1 -- 172.21.15.50:0/619671143 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f217c006230 con 0x7f21940b0000 2024-09-21T05:56:27.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.964+0000 7f218be00640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2170073200 0x7f2170075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:27.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.964+0000 7f218be00640 1 -- 172.21.15.50:0/619671143 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f217c12da90 con 0x7f21940b0000 2024-09-21T05:56:27.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.964+0000 7f2193400640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2170073200 0x7f2170075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:27.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.964+0000 7f2193400640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2170073200 0x7f2170075710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2184009fb0 tx=0x7f21840091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:27.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:27.968+0000 7f218be00640 1 -- 172.21.15.50:0/619671143 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f217c0bf030 con 0x7f21940b0000 2024-09-21T05:56:28.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.148+0000 7f219a200640 1 -- 172.21.15.50:0/619671143 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f219406d200 con 0x7f21940b0000 2024-09-21T05:56:28.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.760+0000 7f218be00640 1 -- 172.21.15.50:0/619671143 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7f219415c0d0 con 0x7f21940b0000 2024-09-21T05:56:28.763 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-09-21T05:56:28.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.764+0000 7f219a200640 1 -- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2170073200 msgr2=0x7f2170075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:28.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.764+0000 7f219a200640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2170073200 0x7f2170075710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2184009fb0 tx=0x7f21840091e0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.764+0000 7f219a200640 1 -- 172.21.15.50:0/619671143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21940b0000 msgr2=0x7f21941633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.764+0000 7f219a200640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21940b0000 0x7f21941633b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f217c01c9a0 tx=0x7f217c001260 comp rx=0 tx=0).stop 2024-09-21T05:56:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.764+0000 7f219a200640 1 -- 172.21.15.50:0/619671143 shutdown_connections 2024-09-21T05:56:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.764+0000 7f219a200640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f219415e500 0x7f21940af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.764+0000 7f219a200640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2170073200 0x7f2170075710 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.764+0000 7f219a200640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21940b0000 0x7f21941633b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.764+0000 7f219a200640 1 --2- 172.21.15.50:0/619671143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2178003fe0 0x7f21941f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.764+0000 7f219a200640 1 -- 172.21.15.50:0/619671143 >> 172.21.15.50:0/619671143 conn(0x7f21940b35f0 msgr2=0x7f21940d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:28.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.764+0000 7f219a200640 1 -- 172.21.15.50:0/619671143 shutdown_connections 2024-09-21T05:56:28.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.764+0000 7f219a200640 1 -- 172.21.15.50:0/619671143 wait complete. 2024-09-21T05:56:28.808 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T05:56:28.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.968+0000 7fb549600640 1 -- 172.21.15.50:0/2650026314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb528004d50 msgr2=0x7fb528005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:28.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.968+0000 7fb549600640 1 --2- 172.21.15.50:0/2650026314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb528004d50 0x7fb528005160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb5440ce1e0 tx=0x7fb53c01d590 comp rx=0 tx=0).stop 2024-09-21T05:56:28.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.968+0000 7fb543400640 1 -- 172.21.15.50:0/2650026314 reap_dead start 2024-09-21T05:56:28.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.968+0000 7fb549600640 1 -- 172.21.15.50:0/2650026314 shutdown_connections 2024-09-21T05:56:28.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.968+0000 7fb549600640 1 -- 172.21.15.50:0/2650026314 >> 172.21.15.50:0/2650026314 conn(0x7fb5440b35f0 msgr2=0x7fb5440b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:28.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.968+0000 7fb549600640 1 -- 172.21.15.50:0/2650026314 shutdown_connections 2024-09-21T05:56:28.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.968+0000 7fb549600640 1 -- 172.21.15.50:0/2650026314 wait complete. 2024-09-21T05:56:28.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb549600640 1 Processor -- start 2024-09-21T05:56:28.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb549600640 1 -- start start 2024-09-21T05:56:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb549600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb544153df0 0x7fb5441558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb549600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb544152450 0x7fb54415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb549600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb544152860 0x7fb54415f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb549600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb5440b5b60 con 0x7fb544153df0 2024-09-21T05:56:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb549600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb544070340 con 0x7fb544152450 2024-09-21T05:56:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb549600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb5440b5ce0 con 0x7fb544152860 2024-09-21T05:56:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb542a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb544153df0 0x7fb5441558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb542a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb544153df0 0x7fb5441558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43130/0 (socket says 172.21.15.50:43130) 2024-09-21T05:56:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb542a00640 1 -- 172.21.15.50:0/508549434 learned_addr learned my addr 172.21.15.50:0/508549434 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb542a00640 1 -- 172.21.15.50:0/508549434 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb544152860 msgr2=0x7fb54415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb542a00640 1 --2- 172.21.15.50:0/508549434 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb544152860 0x7fb54415f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb542a00640 1 -- 172.21.15.50:0/508549434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb544152450 msgr2=0x7fb54415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb542a00640 1 --2- 172.21.15.50:0/508549434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb544152450 0x7fb54415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb542a00640 1 -- 172.21.15.50:0/508549434 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5441f64d0 con 0x7fb544153df0 2024-09-21T05:56:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb542a00640 1 --2- 172.21.15.50:0/508549434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb544153df0 0x7fb5441558f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb534012640 tx=0x7fb534012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb53aa00640 1 -- 172.21.15.50:0/508549434 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb534013a40 con 0x7fb544153df0 2024-09-21T05:56:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb53aa00640 1 -- 172.21.15.50:0/508549434 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb534017030 con 0x7fb544153df0 2024-09-21T05:56:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb549600640 1 -- 172.21.15.50:0/508549434 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5441f5950 con 0x7fb544153df0 2024-09-21T05:56:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.972+0000 7fb53aa00640 1 -- 172.21.15.50:0/508549434 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53401d430 con 0x7fb544153df0 2024-09-21T05:56:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.976+0000 7fb549600640 1 -- 172.21.15.50:0/508549434 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5441f61f0 con 0x7fb544153df0 2024-09-21T05:56:28.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.976+0000 7fb549600640 1 -- 172.21.15.50:0/508549434 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5441f5d40 con 0x7fb544153df0 2024-09-21T05:56:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.976+0000 7fb53aa00640 1 -- 172.21.15.50:0/508549434 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb534027440 con 0x7fb544153df0 2024-09-21T05:56:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.976+0000 7fb53aa00640 1 --2- 172.21.15.50:0/508549434 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb520073200 0x7fb520075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.976+0000 7fb53aa00640 1 -- 172.21.15.50:0/508549434 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7fb534098360 con 0x7fb544153df0 2024-09-21T05:56:28.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.976+0000 7fb53be00640 1 --2- 172.21.15.50:0/508549434 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb520073200 0x7fb520075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:28.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.980+0000 7fb53be00640 1 --2- 172.21.15.50:0/508549434 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb520073200 0x7fb520075710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009b40 tx=0x7fb52c009f40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:28.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:28.980+0000 7fb53aa00640 1 -- 172.21.15.50:0/508549434 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb5340634e0 con 0x7fb544153df0 2024-09-21T05:56:29.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.192+0000 7fb549600640 1 -- 172.21.15.50:0/508549434 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb5440704c0 con 0x7fb544153df0 2024-09-21T05:56:29.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.776+0000 7fb53aa00640 1 -- 172.21.15.50:0/508549434 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fb534062de0 con 0x7fb544153df0 2024-09-21T05:56:29.788 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T05:56:29.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.788+0000 7fb549600640 1 -- 172.21.15.50:0/508549434 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb520073200 msgr2=0x7fb520075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:29.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.788+0000 7fb549600640 1 --2- 172.21.15.50:0/508549434 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb520073200 0x7fb520075710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009b40 tx=0x7fb52c009f40 comp rx=0 tx=0).stop 2024-09-21T05:56:29.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.788+0000 7fb549600640 1 -- 172.21.15.50:0/508549434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb544153df0 msgr2=0x7fb5441558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:29.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.788+0000 7fb549600640 1 --2- 172.21.15.50:0/508549434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb544153df0 0x7fb5441558f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb534012640 tx=0x7fb534012b00 comp rx=0 tx=0).stop 2024-09-21T05:56:29.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.788+0000 7fb549600640 1 -- 172.21.15.50:0/508549434 shutdown_connections 2024-09-21T05:56:29.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.788+0000 7fb549600640 1 --2- 172.21.15.50:0/508549434 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb544152860 0x7fb54415f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.788+0000 7fb549600640 1 --2- 172.21.15.50:0/508549434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb544152450 0x7fb54415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.788+0000 7fb549600640 1 --2- 172.21.15.50:0/508549434 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb520073200 0x7fb520075710 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.788+0000 7fb549600640 1 --2- 172.21.15.50:0/508549434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb544153df0 0x7fb5441558f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.788+0000 7fb549600640 1 -- 172.21.15.50:0/508549434 >> 172.21.15.50:0/508549434 conn(0x7fb5440b35f0 msgr2=0x7fb544075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:29.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.792+0000 7fb549600640 1 -- 172.21.15.50:0/508549434 shutdown_connections 2024-09-21T05:56:29.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.792+0000 7fb549600640 1 -- 172.21.15.50:0/508549434 wait complete. 2024-09-21T05:56:29.831 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:56:29.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.988+0000 7fee71c00640 1 -- 172.21.15.50:0/101942963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee6c159130 msgr2=0x7fee6c15bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:29.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.988+0000 7fee71c00640 1 --2- 172.21.15.50:0/101942963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee6c159130 0x7fee6c15bb90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009900 tx=0x7fee5c01c850 comp rx=0 tx=0).stop 2024-09-21T05:56:29.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.988+0000 7fee71c00640 1 -- 172.21.15.50:0/101942963 shutdown_connections 2024-09-21T05:56:29.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.988+0000 7fee71c00640 1 --2- 172.21.15.50:0/101942963 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee6c06cdf0 0x7fee6c06d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.988+0000 7fee71c00640 1 --2- 172.21.15.50:0/101942963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee6c159130 0x7fee6c15bb90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.988+0000 7fee71c00640 1 -- 172.21.15.50:0/101942963 >> 172.21.15.50:0/101942963 conn(0x7fee6c0b35f0 msgr2=0x7fee6c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:29.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.988+0000 7fee71c00640 1 -- 172.21.15.50:0/101942963 shutdown_connections 2024-09-21T05:56:29.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.988+0000 7fee71c00640 1 -- 172.21.15.50:0/101942963 wait complete. 2024-09-21T05:56:29.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee71c00640 1 Processor -- start 2024-09-21T05:56:29.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee71c00640 1 -- start start 2024-09-21T05:56:29.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee71c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee6c06cdf0 0x7fee6c0aa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:29.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee71c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee6c06fc90 0x7fee6c0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:29.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee71c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee6c159130 0x7fee6c165630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:29.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee71c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c168e70 con 0x7fee6c06fc90 2024-09-21T05:56:29.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee71c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c168cf0 con 0x7fee6c159130 2024-09-21T05:56:29.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee71c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fee6c168ff0 con 0x7fee6c06cdf0 2024-09-21T05:56:29.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee6a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee6c06fc90 0x7fee6c0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:29.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee6a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee6c06fc90 0x7fee6c0d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43146/0 (socket says 172.21.15.50:43146) 2024-09-21T05:56:29.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee6a000640 1 -- 172.21.15.50:0/3489885864 learned_addr learned my addr 172.21.15.50:0/3489885864 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:29.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee6aa00640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee6c06cdf0 0x7fee6c0aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:29.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee6b400640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee6c159130 0x7fee6c165630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:29.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee6a000640 1 -- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee6c06cdf0 msgr2=0x7fee6c0aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:29.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee6a000640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee6c06cdf0 0x7fee6c0aa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee6a000640 1 -- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee6c159130 msgr2=0x7fee6c165630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:29.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee6a000640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee6c159130 0x7fee6c165630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee6a000640 1 -- 172.21.15.50:0/3489885864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6c1fa020 con 0x7fee6c06fc90 2024-09-21T05:56:29.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee6a000640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee6c06fc90 0x7fee6c0d0070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fee5800e6c0 tx=0x7fee5800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:29.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee62a00640 1 -- 172.21.15.50:0/3489885864 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58004220 con 0x7fee6c06fc90 2024-09-21T05:56:29.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee62a00640 1 -- 172.21.15.50:0/3489885864 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee5800c8b0 con 0x7fee6c06fc90 2024-09-21T05:56:29.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee71c00640 1 -- 172.21.15.50:0/3489885864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6c1f9260 con 0x7fee6c06fc90 2024-09-21T05:56:29.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.992+0000 7fee62a00640 1 -- 172.21.15.50:0/3489885864 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5800cd00 con 0x7fee6c06fc90 2024-09-21T05:56:29.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.996+0000 7fee71c00640 1 -- 172.21.15.50:0/3489885864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6c1f9820 con 0x7fee6c06fc90 2024-09-21T05:56:30.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.996+0000 7fee71c00640 1 -- 172.21.15.50:0/3489885864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee6c06d200 con 0x7fee6c06fc90 2024-09-21T05:56:30.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.996+0000 7fee62a00640 1 -- 172.21.15.50:0/3489885864 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fee58003600 con 0x7fee6c06fc90 2024-09-21T05:56:30.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.996+0000 7fee62a00640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fee4c073200 0x7fee4c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:30.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.996+0000 7fee62a00640 1 -- 172.21.15.50:0/3489885864 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fee58094770 con 0x7fee6c06fc90 2024-09-21T05:56:30.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:29.996+0000 7fee6aa00640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fee4c073200 0x7fee4c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:30.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.000+0000 7fee6aa00640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fee4c073200 0x7fee4c075710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0af000 tx=0x7fee5c01b090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:30.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.000+0000 7fee62a00640 1 -- 172.21.15.50:0/3489885864 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fee5805f780 con 0x7fee6c06fc90 2024-09-21T05:56:30.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.180+0000 7fee71c00640 1 -- 172.21.15.50:0/3489885864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee6c0ab090 con 0x7fee6c06fc90 2024-09-21T05:56:30.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.180+0000 7fee62a00640 1 -- 172.21.15.50:0/3489885864 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16630 (secure 0 0 0) 0x7fee5805f080 con 0x7fee6c06fc90 2024-09-21T05:56:30.185 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:30.185 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":21,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:29.197522+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:56:27.052826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:56:28.162376+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:30.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.184+0000 7fee71c00640 1 -- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fee4c073200 msgr2=0x7fee4c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:30.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.184+0000 7fee71c00640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fee4c073200 0x7fee4c075710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0af000 tx=0x7fee5c01b090 comp rx=0 tx=0).stop 2024-09-21T05:56:30.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.184+0000 7fee71c00640 1 -- 172.21.15.50:0/3489885864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee6c06fc90 msgr2=0x7fee6c0d0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:30.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.184+0000 7fee71c00640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee6c06fc90 0x7fee6c0d0070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fee5800e6c0 tx=0x7fee5800eb80 comp rx=0 tx=0).stop 2024-09-21T05:56:30.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.184+0000 7fee71c00640 1 -- 172.21.15.50:0/3489885864 shutdown_connections 2024-09-21T05:56:30.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.184+0000 7fee71c00640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee6c159130 0x7fee6c165630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.184+0000 7fee71c00640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fee4c073200 0x7fee4c075710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.184+0000 7fee71c00640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee6c06fc90 0x7fee6c0d0070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.184+0000 7fee71c00640 1 --2- 172.21.15.50:0/3489885864 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee6c06cdf0 0x7fee6c0aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.184+0000 7fee71c00640 1 -- 172.21.15.50:0/3489885864 >> 172.21.15.50:0/3489885864 conn(0x7fee6c0b35f0 msgr2=0x7fee6c070880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:30.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.184+0000 7fee71c00640 1 -- 172.21.15.50:0/3489885864 shutdown_connections 2024-09-21T05:56:30.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.184+0000 7fee71c00640 1 -- 172.21.15.50:0/3489885864 wait complete. 2024-09-21T05:56:30.207 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:56:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.364+0000 7f89dd200640 1 -- 172.21.15.50:0/1629377838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d80b5a80 msgr2=0x7f89d8074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.364+0000 7f89dd200640 1 --2- 172.21.15.50:0/1629377838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d80b5a80 0x7f89d8074410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009900 tx=0x7f89c001c8e0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.364+0000 7f89dd200640 1 -- 172.21.15.50:0/1629377838 shutdown_connections 2024-09-21T05:56:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.364+0000 7f89dd200640 1 --2- 172.21.15.50:0/1629377838 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f89d8074950 0x7f89d815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.364+0000 7f89dd200640 1 --2- 172.21.15.50:0/1629377838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d80b5a80 0x7f89d8074410 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.364+0000 7f89dd200640 1 -- 172.21.15.50:0/1629377838 >> 172.21.15.50:0/1629377838 conn(0x7f89d80708c0 msgr2=0x7f89d8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:30.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.364+0000 7f89dd200640 1 -- 172.21.15.50:0/1629377838 shutdown_connections 2024-09-21T05:56:30.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.364+0000 7f89dd200640 1 -- 172.21.15.50:0/1629377838 wait complete. 2024-09-21T05:56:30.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.364+0000 7f89dd200640 1 Processor -- start 2024-09-21T05:56:30.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.364+0000 7f89dd200640 1 -- start start 2024-09-21T05:56:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89dd200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d8074950 0x7f89d814d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89dd200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f89d80b5a80 0x7f89d8149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89dd200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d8157030 0x7f89d815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89dd200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f89d816a310 con 0x7f89d8074950 2024-09-21T05:56:30.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89dd200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f89d816a190 con 0x7f89d8157030 2024-09-21T05:56:30.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89dd200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f89d816a490 con 0x7f89d80b5a80 2024-09-21T05:56:30.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89d5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d8074950 0x7f89d814d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:30.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89d5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d8074950 0x7f89d814d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43166/0 (socket says 172.21.15.50:43166) 2024-09-21T05:56:30.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89d5e00640 1 -- 172.21.15.50:0/2131636701 learned_addr learned my addr 172.21.15.50:0/2131636701 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:30.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89cd400640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f89d80b5a80 0x7f89d8149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:30.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89d6800640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d8157030 0x7f89d815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:30.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89d5e00640 1 -- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f89d80b5a80 msgr2=0x7f89d8149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:30.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89d5e00640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f89d80b5a80 0x7f89d8149b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89d5e00640 1 -- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d8157030 msgr2=0x7f89d815ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:30.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89d5e00640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d8157030 0x7f89d815ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89d5e00640 1 -- 172.21.15.50:0/2131636701 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d8150f10 con 0x7f89d8074950 2024-09-21T05:56:30.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89d6800640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d8157030 0x7f89d815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:30.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89d5e00640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d8074950 0x7f89d814d690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f89c800c3c0 tx=0x7f89c800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:30.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89cea00640 1 -- 172.21.15.50:0/2131636701 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8017070 con 0x7f89d8074950 2024-09-21T05:56:30.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89cea00640 1 -- 172.21.15.50:0/2131636701 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89c80053c0 con 0x7f89d8074950 2024-09-21T05:56:30.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89cea00640 1 -- 172.21.15.50:0/2131636701 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8003860 con 0x7f89d8074950 2024-09-21T05:56:30.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89dd200640 1 -- 172.21.15.50:0/2131636701 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d814af60 con 0x7f89d8074950 2024-09-21T05:56:30.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89dd200640 1 -- 172.21.15.50:0/2131636701 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d814b520 con 0x7f89d8074950 2024-09-21T05:56:30.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89dd200640 1 -- 172.21.15.50:0/2131636701 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d815cf90 con 0x7f89d8074950 2024-09-21T05:56:30.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.368+0000 7f89cea00640 1 -- 172.21.15.50:0/2131636701 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f89c801a070 con 0x7f89d8074950 2024-09-21T05:56:30.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.372+0000 7f89cea00640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f89b4073200 0x7f89b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:30.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.372+0000 7f89cea00640 1 -- 172.21.15.50:0/2131636701 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f89c8093570 con 0x7f89d8074950 2024-09-21T05:56:30.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.372+0000 7f89cd400640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f89b4073200 0x7f89b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:30.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.372+0000 7f89cd400640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f89b4073200 0x7f89b4075710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89c00af000 tx=0x7f89c001b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:30.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.376+0000 7f89cea00640 1 -- 172.21.15.50:0/2131636701 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f89d8074d60 con 0x7f89d8074950 2024-09-21T05:56:30.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.556+0000 7f89dd200640 1 -- 172.21.15.50:0/2131636701 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89d80d2260 con 0x7f89d8074950 2024-09-21T05:56:30.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.556+0000 7f89cea00640 1 -- 172.21.15.50:0/2131636701 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16630 (secure 0 0 0) 0x7f89c8091e20 con 0x7f89d8074950 2024-09-21T05:56:30.561 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:30.561 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":21,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:29.197522+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:56:27.052826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:56:28.162376+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.560+0000 7f89dd200640 1 -- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f89b4073200 msgr2=0x7f89b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.560+0000 7f89dd200640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f89b4073200 0x7f89b4075710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89c00af000 tx=0x7f89c001b120 comp rx=0 tx=0).stop 2024-09-21T05:56:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.560+0000 7f89dd200640 1 -- 172.21.15.50:0/2131636701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d8074950 msgr2=0x7f89d814d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.560+0000 7f89dd200640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d8074950 0x7f89d814d690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f89c800c3c0 tx=0x7f89c800c880 comp rx=0 tx=0).stop 2024-09-21T05:56:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.560+0000 7f89dd200640 1 -- 172.21.15.50:0/2131636701 shutdown_connections 2024-09-21T05:56:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.560+0000 7f89dd200640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d8157030 0x7f89d815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.560+0000 7f89dd200640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f89d80b5a80 0x7f89d8149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.560+0000 7f89dd200640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f89b4073200 0x7f89b4075710 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.560+0000 7f89dd200640 1 --2- 172.21.15.50:0/2131636701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d8074950 0x7f89d814d690 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.560+0000 7f89dd200640 1 -- 172.21.15.50:0/2131636701 >> 172.21.15.50:0/2131636701 conn(0x7f89d80708c0 msgr2=0x7f89d80d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:30.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.560+0000 7f89dd200640 1 -- 172.21.15.50:0/2131636701 shutdown_connections 2024-09-21T05:56:30.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.560+0000 7f89dd200640 1 -- 172.21.15.50:0/2131636701 wait complete. 2024-09-21T05:56:30.583 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T05:56:30.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.740+0000 7fefb2000640 1 -- 172.21.15.50:0/2307742320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefac0b3340 msgr2=0x7fefac0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:30.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.740+0000 7fefb2000640 1 --2- 172.21.15.50:0/2307742320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefac0b3340 0x7fefac0b3750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009c80 tx=0x7fefa401cda0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.740+0000 7fefb2000640 1 -- 172.21.15.50:0/2307742320 shutdown_connections 2024-09-21T05:56:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.740+0000 7fefb2000640 1 --2- 172.21.15.50:0/2307742320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefac0b3340 0x7fefac0b3750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.740+0000 7fefb2000640 1 --2- 172.21.15.50:0/2307742320 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fefac0749a0 0x7fefac074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.740+0000 7fefb2000640 1 -- 172.21.15.50:0/2307742320 >> 172.21.15.50:0/2307742320 conn(0x7fefac0b2040 msgr2=0x7fefac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.740+0000 7fefb2000640 1 -- 172.21.15.50:0/2307742320 shutdown_connections 2024-09-21T05:56:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.740+0000 7fefb2000640 1 -- 172.21.15.50:0/2307742320 wait complete. 2024-09-21T05:56:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefb2000640 1 Processor -- start 2024-09-21T05:56:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefb2000640 1 -- start start 2024-09-21T05:56:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefb2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefac0749a0 0x7fefac144310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefb2000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fefac0b3340 0x7fefac13aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefb2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefac0cefa0 0x7fefac1f9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefb2000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fefac169dc0 con 0x7fefac0749a0 2024-09-21T05:56:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefb2000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fefac169c40 con 0x7fefac0cefa0 2024-09-21T05:56:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefb2000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fefac169f40 con 0x7fefac0b3340 2024-09-21T05:56:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefab400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefac0749a0 0x7fefac144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefab400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefac0749a0 0x7fefac144310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43198/0 (socket says 172.21.15.50:43198) 2024-09-21T05:56:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefab400640 1 -- 172.21.15.50:0/3630665243 learned_addr learned my addr 172.21.15.50:0/3630665243 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefa3e00640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fefac0b3340 0x7fefac13aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefabe00640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefac0cefa0 0x7fefac1f9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:30.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefab400640 1 -- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fefac0b3340 msgr2=0x7fefac13aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefab400640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fefac0b3340 0x7fefac13aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefab400640 1 -- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefac0cefa0 msgr2=0x7fefac1f9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:30.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefab400640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefac0cefa0 0x7fefac1f9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:30.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.744+0000 7fefab400640 1 -- 172.21.15.50:0/3630665243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac144c40 con 0x7fefac0749a0 2024-09-21T05:56:30.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.748+0000 7fefab400640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefac0749a0 0x7fefac144310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00d720 tx=0x7fef9c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:30.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.748+0000 7fefa3400640 1 -- 172.21.15.50:0/3630665243 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c017070 con 0x7fefac0749a0 2024-09-21T05:56:30.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.748+0000 7fefa3400640 1 -- 172.21.15.50:0/3630665243 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef9c009d20 con 0x7fefac0749a0 2024-09-21T05:56:30.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.748+0000 7fefa3400640 1 -- 172.21.15.50:0/3630665243 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c005280 con 0x7fefac0749a0 2024-09-21T05:56:30.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.748+0000 7fefb2000640 1 -- 172.21.15.50:0/3630665243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefac144f20 con 0x7fefac0749a0 2024-09-21T05:56:30.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.748+0000 7fefb2000640 1 -- 172.21.15.50:0/3630665243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefac1454e0 con 0x7fefac0749a0 2024-09-21T05:56:30.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.748+0000 7fefb2000640 1 -- 172.21.15.50:0/3630665243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefac074db0 con 0x7fefac0749a0 2024-09-21T05:56:30.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.748+0000 7fefa3400640 1 -- 172.21.15.50:0/3630665243 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fef9c005420 con 0x7fefac0749a0 2024-09-21T05:56:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.752+0000 7fefa3400640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef88073200 0x7fef88075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.752+0000 7fefa3400640 1 -- 172.21.15.50:0/3630665243 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fef9c094290 con 0x7fefac0749a0 2024-09-21T05:56:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.752+0000 7fefa3e00640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef88073200 0x7fef88075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.752+0000 7fefa3e00640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef88073200 0x7fef88075710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fefac0afb80 tx=0x7fef94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.756+0000 7fefa3400640 1 -- 172.21.15.50:0/3630665243 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fef9c05ec50 con 0x7fefac0749a0 2024-09-21T05:56:30.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:30.972+0000 7fefb2000640 1 -- 172.21.15.50:0/3630665243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fefac0b0340 con 0x7fefac0749a0 2024-09-21T05:56:31.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.792+0000 7fefa3400640 1 -- 172.21.15.50:0/3630665243 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fef9c063680 con 0x7fefac0749a0 2024-09-21T05:56:31.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.796+0000 7fefb2000640 1 -- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef88073200 msgr2=0x7fef88075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.796+0000 7fefb2000640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef88073200 0x7fef88075710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fefac0afb80 tx=0x7fef94009040 comp rx=0 tx=0).stop 2024-09-21T05:56:31.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.796+0000 7fefb2000640 1 -- 172.21.15.50:0/3630665243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefac0749a0 msgr2=0x7fefac144310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.796+0000 7fefb2000640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefac0749a0 0x7fefac144310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00d720 tx=0x7fef9c00dbe0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.796+0000 7fefb2000640 1 -- 172.21.15.50:0/3630665243 shutdown_connections 2024-09-21T05:56:31.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.796+0000 7fefb2000640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefac0cefa0 0x7fefac1f9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.796+0000 7fefb2000640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fefac0b3340 0x7fefac13aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.796+0000 7fefb2000640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef88073200 0x7fef88075710 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.796+0000 7fefb2000640 1 --2- 172.21.15.50:0/3630665243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefac0749a0 0x7fefac144310 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.796+0000 7fefb2000640 1 -- 172.21.15.50:0/3630665243 >> 172.21.15.50:0/3630665243 conn(0x7fefac0b2040 msgr2=0x7fefac142280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:31.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.796+0000 7fefb2000640 1 -- 172.21.15.50:0/3630665243 shutdown_connections 2024-09-21T05:56:31.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.796+0000 7fefb2000640 1 -- 172.21.15.50:0/3630665243 wait complete. 2024-09-21T05:56:31.824 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T05:56:31.824 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T05:56:31.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.988+0000 7facc5e00640 1 -- 172.21.15.50:0/2680183008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facc015c730 msgr2=0x7facc00b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.988+0000 7facc5e00640 1 --2- 172.21.15.50:0/2680183008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facc015c730 0x7facc00b3710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7facb0009870 tx=0x7facb001c820 comp rx=0 tx=0).stop 2024-09-21T05:56:31.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.988+0000 7facc5e00640 1 -- 172.21.15.50:0/2680183008 shutdown_connections 2024-09-21T05:56:31.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.988+0000 7facc5e00640 1 --2- 172.21.15.50:0/2680183008 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7facc0073f90 0x7facc00743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.988+0000 7facc5e00640 1 --2- 172.21.15.50:0/2680183008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facc015c730 0x7facc00b3710 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.988+0000 7facc5e00640 1 -- 172.21.15.50:0/2680183008 >> 172.21.15.50:0/2680183008 conn(0x7facc00d06e0 msgr2=0x7facc00d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:31.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.988+0000 7facc5e00640 1 -- 172.21.15.50:0/2680183008 shutdown_connections 2024-09-21T05:56:31.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.988+0000 7facc5e00640 1 -- 172.21.15.50:0/2680183008 wait complete. 2024-09-21T05:56:31.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facc5e00640 1 Processor -- start 2024-09-21T05:56:31.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facc5e00640 1 -- start start 2024-09-21T05:56:31.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facc5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facc0073f90 0x7facc015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facc5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facc00af520 0x7facc00cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facc5e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7facc015c730 0x7facc00ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facc5e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7facc006f740 con 0x7facc0073f90 2024-09-21T05:56:31.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facc5e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7facc006f5c0 con 0x7facc00af520 2024-09-21T05:56:31.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facc5e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7facc006f8c0 con 0x7facc015c730 2024-09-21T05:56:31.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facbea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facc0073f90 0x7facc015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facbea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facc0073f90 0x7facc015e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43216/0 (socket says 172.21.15.50:43216) 2024-09-21T05:56:31.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facbea00640 1 -- 172.21.15.50:0/1296061073 learned_addr learned my addr 172.21.15.50:0/1296061073 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:31.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facbea00640 1 -- 172.21.15.50:0/1296061073 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7facc015c730 msgr2=0x7facc00ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facbea00640 1 --2- 172.21.15.50:0/1296061073 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7facc015c730 0x7facc00ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facbea00640 1 -- 172.21.15.50:0/1296061073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facc00af520 msgr2=0x7facc00cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facbea00640 1 --2- 172.21.15.50:0/1296061073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facc00af520 0x7facc00cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facbea00640 1 -- 172.21.15.50:0/1296061073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc01fc360 con 0x7facc0073f90 2024-09-21T05:56:31.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facbea00640 1 --2- 172.21.15.50:0/1296061073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facc0073f90 0x7facc015e7b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7facb0009790 tx=0x7facb00a6760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:31.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facb7400640 1 -- 172.21.15.50:0/1296061073 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0004340 con 0x7facc0073f90 2024-09-21T05:56:31.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facb7400640 1 -- 172.21.15.50:0/1296061073 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facb00bd070 con 0x7facc0073f90 2024-09-21T05:56:31.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facb7400640 1 -- 172.21.15.50:0/1296061073 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0003860 con 0x7facc0073f90 2024-09-21T05:56:31.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facc5e00640 1 -- 172.21.15.50:0/1296061073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc01fc620 con 0x7facc0073f90 2024-09-21T05:56:31.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.992+0000 7facc5e00640 1 -- 172.21.15.50:0/1296061073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc01fcbe0 con 0x7facc0073f90 2024-09-21T05:56:31.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.996+0000 7facc5e00640 1 -- 172.21.15.50:0/1296061073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc00b3710 con 0x7facc0073f90 2024-09-21T05:56:31.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.996+0000 7facb7400640 1 -- 172.21.15.50:0/1296061073 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7facb0003a00 con 0x7facc0073f90 2024-09-21T05:56:31.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.996+0000 7facb7400640 1 --2- 172.21.15.50:0/1296061073 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac9c073200 0x7fac9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.996+0000 7facb7400640 1 -- 172.21.15.50:0/1296061073 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7facb012f3e0 con 0x7facc0073f90 2024-09-21T05:56:31.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.996+0000 7facb7e00640 1 --2- 172.21.15.50:0/1296061073 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac9c073200 0x7fac9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:31.996+0000 7facb7e00640 1 --2- 172.21.15.50:0/1296061073 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac9c073200 0x7fac9c075710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faca80058d0 tx=0x7faca8005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:32.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.000+0000 7facb7400640 1 -- 172.21.15.50:0/1296061073 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7facb0016410 con 0x7facc0073f90 2024-09-21T05:56:32.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.212+0000 7facc5e00640 1 -- 172.21.15.50:0/1296061073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7facc00d0f00 con 0x7facc0073f90 2024-09-21T05:56:32.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.800+0000 7facb7400640 1 -- 172.21.15.50:0/1296061073 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7facb00fa360 con 0x7facc0073f90 2024-09-21T05:56:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.804+0000 7facc5e00640 1 -- 172.21.15.50:0/1296061073 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac9c073200 msgr2=0x7fac9c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.804+0000 7facc5e00640 1 --2- 172.21.15.50:0/1296061073 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac9c073200 0x7fac9c075710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faca80058d0 tx=0x7faca8005860 comp rx=0 tx=0).stop 2024-09-21T05:56:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.804+0000 7facc5e00640 1 -- 172.21.15.50:0/1296061073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facc0073f90 msgr2=0x7facc015e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.804+0000 7facc5e00640 1 --2- 172.21.15.50:0/1296061073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facc0073f90 0x7facc015e7b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7facb0009790 tx=0x7facb00a6760 comp rx=0 tx=0).stop 2024-09-21T05:56:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.804+0000 7facc5e00640 1 -- 172.21.15.50:0/1296061073 shutdown_connections 2024-09-21T05:56:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.804+0000 7facc5e00640 1 --2- 172.21.15.50:0/1296061073 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7facc015c730 0x7facc00ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.804+0000 7facc5e00640 1 --2- 172.21.15.50:0/1296061073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facc00af520 0x7facc00cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.804+0000 7facc5e00640 1 --2- 172.21.15.50:0/1296061073 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac9c073200 0x7fac9c075710 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.804+0000 7facc5e00640 1 --2- 172.21.15.50:0/1296061073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facc0073f90 0x7facc015e7b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.804+0000 7facc5e00640 1 -- 172.21.15.50:0/1296061073 >> 172.21.15.50:0/1296061073 conn(0x7facc00d06e0 msgr2=0x7facc0155080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.804+0000 7facc5e00640 1 -- 172.21.15.50:0/1296061073 shutdown_connections 2024-09-21T05:56:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.804+0000 7facc5e00640 1 -- 172.21.15.50:0/1296061073 wait complete. 2024-09-21T05:56:32.829 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T05:56:32.830 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:56:32.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.988+0000 7fcebd000640 1 -- 172.21.15.50:0/3726392591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb80b0c20 msgr2=0x7fceb8069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.988+0000 7fcebd000640 1 --2- 172.21.15.50:0/3726392591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb80b0c20 0x7fceb8069330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fceb8072d40 tx=0x7fceb001cb80 comp rx=0 tx=0).stop 2024-09-21T05:56:32.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.988+0000 7fcebd000640 1 -- 172.21.15.50:0/3726392591 shutdown_connections 2024-09-21T05:56:32.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.988+0000 7fcebd000640 1 --2- 172.21.15.50:0/3726392591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb80b0c20 0x7fceb8069330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.988+0000 7fcebd000640 1 --2- 172.21.15.50:0/3726392591 >> v2:172.21.15.88:3300/0 conn(0x7fceb80b0810 0x7fceb8068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.988+0000 7fcebd000640 1 -- 172.21.15.50:0/3726392591 >> 172.21.15.50:0/3726392591 conn(0x7fceb80b35f0 msgr2=0x7fceb80b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:32.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.988+0000 7fcebd000640 1 -- 172.21.15.50:0/3726392591 shutdown_connections 2024-09-21T05:56:32.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.988+0000 7fcebd000640 1 -- 172.21.15.50:0/3726392591 wait complete. 2024-09-21T05:56:32.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fcebd000640 1 Processor -- start 2024-09-21T05:56:32.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fcebd000640 1 -- start start 2024-09-21T05:56:32.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fcebd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fceb80b0810 0x7fceb81530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fcebd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb80b0c20 0x7fceb8143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fcebd000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fceb815efc0 0x7fceb8157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fcebd000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fceb8162ec0 con 0x7fceb80b0c20 2024-09-21T05:56:32.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fcebd000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fceb8162d40 con 0x7fceb80b0810 2024-09-21T05:56:32.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fcebd000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fceb8163040 con 0x7fceb815efc0 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fceafe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb80b0c20 0x7fceb8143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fceafe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb80b0c20 0x7fceb8143eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43240/0 (socket says 172.21.15.50:43240) 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fceafe00640 1 -- 172.21.15.50:0/698615924 learned_addr learned my addr 172.21.15.50:0/698615924 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fceb5e00640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fceb80b0810 0x7fceb81530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fceb6800640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fceb815efc0 0x7fceb8157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fceafe00640 1 -- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fceb815efc0 msgr2=0x7fceb8157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fceafe00640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fceb815efc0 0x7fceb8157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fceafe00640 1 -- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fceb80b0810 msgr2=0x7fceb81530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fceafe00640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fceb80b0810 0x7fceb81530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fceafe00640 1 -- 172.21.15.50:0/698615924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb8154390 con 0x7fceb80b0c20 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fceb6800640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fceb815efc0 0x7fceb8157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.992+0000 7fceafe00640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb80b0c20 0x7fceb8143eb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcea000d950 tx=0x7fcea000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:32.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.996+0000 7fceae000640 1 -- 172.21.15.50:0/698615924 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0017070 con 0x7fceb80b0c20 2024-09-21T05:56:32.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.996+0000 7fceae000640 1 -- 172.21.15.50:0/698615924 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcea0009d20 con 0x7fceb80b0c20 2024-09-21T05:56:32.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.996+0000 7fcebd000640 1 -- 172.21.15.50:0/698615924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb8152310 con 0x7fceb80b0c20 2024-09-21T05:56:32.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.996+0000 7fceae000640 1 -- 172.21.15.50:0/698615924 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea00106c0 con 0x7fceb80b0c20 2024-09-21T05:56:32.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.996+0000 7fcebd000640 1 -- 172.21.15.50:0/698615924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb81528d0 con 0x7fceb80b0c20 2024-09-21T05:56:33.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.996+0000 7fcebd000640 1 -- 172.21.15.50:0/698615924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb8070910 con 0x7fceb80b0c20 2024-09-21T05:56:33.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.996+0000 7fceae000640 1 -- 172.21.15.50:0/698615924 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcea001a070 con 0x7fceb80b0c20 2024-09-21T05:56:33.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.996+0000 7fceae000640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fce94073230 0x7fce94075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:33.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.996+0000 7fceae000640 1 -- 172.21.15.50:0/698615924 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcea0094450 con 0x7fceb80b0c20 2024-09-21T05:56:33.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:32.996+0000 7fceb5e00640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fce94073230 0x7fce94075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:33.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.000+0000 7fceb5e00640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fce94073230 0x7fce94075740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcea8000f70 tx=0x7fcea8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:33.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.000+0000 7fceae000640 1 -- 172.21.15.50:0/698615924 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcea005f3e0 con 0x7fceb80b0c20 2024-09-21T05:56:33.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.212+0000 7fcebd000640 1 -- 172.21.15.50:0/698615924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fceb806e5e0 con 0x7fceb80b0c20 2024-09-21T05:56:33.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.212+0000 7fceae000640 1 -- 172.21.15.50:0/698615924 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5211 (secure 0 0 0) 0x7fcea005ece0 con 0x7fceb80b0c20 2024-09-21T05:56:33.217 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:33.217 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":8,"btime":"2024-09-21T05:56:32:796889+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3883511106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3883511106},{"type":"v1","addr":"172.21.15.88:6836","nonce":3883511106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/606533049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":606533049},{"type":"v1","addr":"172.21.15.50:6833","nonce":606533049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/183287131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":183287131},{"type":"v1","addr":"172.21.15.50:6835","nonce":183287131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:56:29.197481+0000","modified":"2024-09-21T05:56:32.215215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4371":{"gid":4371,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/1479043500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1479043500},{"type":"v1","addr":"172.21.15.88:6837","nonce":1479043500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4371,"qdb_cluster":[4371]},"id":1}]} 2024-09-21T05:56:33.217 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 8 2024-09-21T05:56:33.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.216+0000 7fcebd000640 1 -- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fce94073230 msgr2=0x7fce94075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:33.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.216+0000 7fcebd000640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fce94073230 0x7fce94075740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcea8000f70 tx=0x7fcea8009040 comp rx=0 tx=0).stop 2024-09-21T05:56:33.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.216+0000 7fcebd000640 1 -- 172.21.15.50:0/698615924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb80b0c20 msgr2=0x7fceb8143eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:33.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.216+0000 7fcebd000640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb80b0c20 0x7fceb8143eb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcea000d950 tx=0x7fcea000de10 comp rx=0 tx=0).stop 2024-09-21T05:56:33.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.216+0000 7fcebd000640 1 -- 172.21.15.50:0/698615924 shutdown_connections 2024-09-21T05:56:33.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.216+0000 7fcebd000640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fceb815efc0 0x7fceb8157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.216+0000 7fcebd000640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fce94073230 0x7fce94075740 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.216+0000 7fcebd000640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb80b0c20 0x7fceb8143eb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.216+0000 7fcebd000640 1 --2- 172.21.15.50:0/698615924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fceb80b0810 0x7fceb81530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.216+0000 7fcebd000640 1 -- 172.21.15.50:0/698615924 >> 172.21.15.50:0/698615924 conn(0x7fceb80b35f0 msgr2=0x7fceb81619c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:33.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.216+0000 7fcebd000640 1 -- 172.21.15.50:0/698615924 shutdown_connections 2024-09-21T05:56:33.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.216+0000 7fcebd000640 1 -- 172.21.15.50:0/698615924 wait complete. 2024-09-21T05:56:33.239 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:56:33.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.396+0000 7f57cb200640 1 -- 172.21.15.50:0/3589939828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57cc1536f0 msgr2=0x7f57cc15d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:33.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.396+0000 7f57cb200640 1 --2- 172.21.15.50:0/3589939828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57cc1536f0 0x7f57cc15d480 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009840 tx=0x7f57bc01c790 comp rx=0 tx=0).stop 2024-09-21T05:56:33.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.396+0000 7f57cb200640 1 -- 172.21.15.50:0/3589939828 shutdown_connections 2024-09-21T05:56:33.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.396+0000 7f57cb200640 1 --2- 172.21.15.50:0/3589939828 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57cc074950 0x7f57cc0b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.396+0000 7f57cb200640 1 --2- 172.21.15.50:0/3589939828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57cc1536f0 0x7f57cc15d480 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.396+0000 7f57cb200640 1 -- 172.21.15.50:0/3589939828 >> 172.21.15.50:0/3589939828 conn(0x7f57cc0b4560 msgr2=0x7f57cc0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:33.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.396+0000 7f57cb200640 1 -- 172.21.15.50:0/3589939828 shutdown_connections 2024-09-21T05:56:33.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.396+0000 7f57cb200640 1 -- 172.21.15.50:0/3589939828 wait complete. 2024-09-21T05:56:33.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.396+0000 7f57cb200640 1 Processor -- start 2024-09-21T05:56:33.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57cb200640 1 -- start start 2024-09-21T05:56:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57cb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc1536f0 0x7f57cc156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57cb200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57cc160060 0x7f57cc1366b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57cb200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57cc153b90 0x7f57cc153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57cb200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc169e60 con 0x7f57cc160060 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57cb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc169ce0 con 0x7f57cc1536f0 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57cb200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc169fe0 con 0x7f57cc153b90 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57cc160060 0x7f57cc1366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57cc160060 0x7f57cc1366b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43264/0 (socket says 172.21.15.50:43264) 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c9400640 1 -- 172.21.15.50:0/2854989965 learned_addr learned my addr 172.21.15.50:0/2854989965 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c9e00640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc1536f0 0x7f57cc156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57ca800640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57cc153b90 0x7f57cc153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c9400640 1 -- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57cc153b90 msgr2=0x7f57cc153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c9400640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57cc153b90 0x7f57cc153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c9400640 1 -- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc1536f0 msgr2=0x7f57cc156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c9400640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc1536f0 0x7f57cc156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c9400640 1 -- 172.21.15.50:0/2854989965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc1544e0 con 0x7f57cc160060 2024-09-21T05:56:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c9400640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57cc160060 0x7f57cc1366b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f57b800c3a0 tx=0x7f57b800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:33.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c2000640 1 -- 172.21.15.50:0/2854989965 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8017070 con 0x7f57cc160060 2024-09-21T05:56:33.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c2000640 1 -- 172.21.15.50:0/2854989965 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57b8003600 con 0x7f57cc160060 2024-09-21T05:56:33.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57cb200640 1 -- 172.21.15.50:0/2854989965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc154960 con 0x7f57cc160060 2024-09-21T05:56:33.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57c2000640 1 -- 172.21.15.50:0/2854989965 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8003a50 con 0x7f57cc160060 2024-09-21T05:56:33.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57cb200640 1 -- 172.21.15.50:0/2854989965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc14ba30 con 0x7f57cc160060 2024-09-21T05:56:33.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.400+0000 7f57cb200640 1 -- 172.21.15.50:0/2854989965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57cc14b4c0 con 0x7f57cc160060 2024-09-21T05:56:33.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.404+0000 7f57c2000640 1 -- 172.21.15.50:0/2854989965 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f57b801a070 con 0x7f57cc160060 2024-09-21T05:56:33.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.404+0000 7f57c2000640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f57ac073200 0x7f57ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:33.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.404+0000 7f57c2000640 1 -- 172.21.15.50:0/2854989965 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f57b8093530 con 0x7f57cc160060 2024-09-21T05:56:33.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.404+0000 7f57c9e00640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f57ac073200 0x7f57ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:33.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.404+0000 7f57c9e00640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f57ac073200 0x7f57ac075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0af000 tx=0x7f57bc01afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:33.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.408+0000 7f57c2000640 1 -- 172.21.15.50:0/2854989965 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f57cc1544e0 con 0x7f57cc160060 2024-09-21T05:56:33.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 -- 172.21.15.50:0/2854989965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57cc0b5260 con 0x7f57cc160060 2024-09-21T05:56:33.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57c2000640 1 -- 172.21.15.50:0/2854989965 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16617 (secure 0 0 0) 0x7f57cc15d9c0 con 0x7f57cc160060 2024-09-21T05:56:33.591 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:33.591 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":22,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:30.775110+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:56:27.052826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:56:28.162376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:33.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 -- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f57ac073200 msgr2=0x7f57ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:33.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f57ac073200 0x7f57ac075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0af000 tx=0x7f57bc01afd0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 -- 172.21.15.50:0/2854989965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57cc160060 msgr2=0x7f57cc1366b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:33.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57cc160060 0x7f57cc1366b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f57b800c3a0 tx=0x7f57b800c860 comp rx=0 tx=0).stop 2024-09-21T05:56:33.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 -- 172.21.15.50:0/2854989965 shutdown_connections 2024-09-21T05:56:33.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57cc153b90 0x7f57cc153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f57ac073200 0x7f57ac075710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57cc160060 0x7f57cc1366b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 --2- 172.21.15.50:0/2854989965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc1536f0 0x7f57cc156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 -- 172.21.15.50:0/2854989965 >> 172.21.15.50:0/2854989965 conn(0x7f57cc0b4560 msgr2=0x7f57cc15f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:33.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 -- 172.21.15.50:0/2854989965 shutdown_connections 2024-09-21T05:56:33.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.588+0000 7f57cb200640 1 -- 172.21.15.50:0/2854989965 wait complete. 2024-09-21T05:56:33.614 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:56:33.614 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:56:33.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.772+0000 7f76a0a00640 1 -- 172.21.15.50:0/1265866446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f769c1527b0 msgr2=0x7f769c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:33.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.772+0000 7f76a0a00640 1 --2- 172.21.15.50:0/1265866446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f769c1527b0 0x7f769c074e30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7684009840 tx=0x7f768401c760 comp rx=0 tx=0).stop 2024-09-21T05:56:33.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.772+0000 7f76a0a00640 1 -- 172.21.15.50:0/1265866446 shutdown_connections 2024-09-21T05:56:33.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.772+0000 7f76a0a00640 1 --2- 172.21.15.50:0/1265866446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f769c1527b0 0x7f769c074e30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.772+0000 7f76a0a00640 1 --2- 172.21.15.50:0/1265866446 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f769c0b5c70 0x7f769c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.772+0000 7f76a0a00640 1 -- 172.21.15.50:0/1265866446 >> 172.21.15.50:0/1265866446 conn(0x7f769c0b2040 msgr2=0x7f769c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:33.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.772+0000 7f76a0a00640 1 -- 172.21.15.50:0/1265866446 shutdown_connections 2024-09-21T05:56:33.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.772+0000 7f76a0a00640 1 -- 172.21.15.50:0/1265866446 wait complete. 2024-09-21T05:56:33.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.772+0000 7f76a0a00640 1 Processor -- start 2024-09-21T05:56:33.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.772+0000 7f76a0a00640 1 -- start start 2024-09-21T05:56:33.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f76a0a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f769c0b4de0 0x7f769c0cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:33.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f76a0a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f769c0b5c70 0x7f769c1436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:33.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f76a0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c1527b0 0x7f769c1fbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:33.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f76a0a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f769c168c00 con 0x7f769c1527b0 2024-09-21T05:56:33.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f76a0a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f769c168a80 con 0x7f769c0b4de0 2024-09-21T05:56:33.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f76a0a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f769c168d80 con 0x7f769c0b5c70 2024-09-21T05:56:33.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f769a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c1527b0 0x7f769c1fbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:33.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f769a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c1527b0 0x7f769c1fbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43270/0 (socket says 172.21.15.50:43270) 2024-09-21T05:56:33.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f769a800640 1 -- 172.21.15.50:0/78976494 learned_addr learned my addr 172.21.15.50:0/78976494 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:33.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f7699e00640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f769c0b4de0 0x7f769c0cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:33.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f7691400640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f769c0b5c70 0x7f769c1436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:33.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f769a800640 1 -- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f769c0b5c70 msgr2=0x7f769c1436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:33.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f769a800640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f769c0b5c70 0x7f769c1436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f769a800640 1 -- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f769c0b4de0 msgr2=0x7f769c0cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:33.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f769a800640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f769c0b4de0 0x7f769c0cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f769a800640 1 -- 172.21.15.50:0/78976494 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c144010 con 0x7f769c1527b0 2024-09-21T05:56:33.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f769a800640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c1527b0 0x7f769c1fbe30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f769400c3c0 tx=0x7f769400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:33.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f7692a00640 1 -- 172.21.15.50:0/78976494 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694017070 con 0x7f769c1527b0 2024-09-21T05:56:33.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f76a0a00640 1 -- 172.21.15.50:0/78976494 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c141630 con 0x7f769c1527b0 2024-09-21T05:56:33.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f7692a00640 1 -- 172.21.15.50:0/78976494 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7694003600 con 0x7f769c1527b0 2024-09-21T05:56:33.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f7692a00640 1 -- 172.21.15.50:0/78976494 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694003a50 con 0x7f769c1527b0 2024-09-21T05:56:33.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f76a0a00640 1 -- 172.21.15.50:0/78976494 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c1442b0 con 0x7f769c1527b0 2024-09-21T05:56:33.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.776+0000 7f76a0a00640 1 -- 172.21.15.50:0/78976494 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c15bb60 con 0x7f769c1527b0 2024-09-21T05:56:33.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.780+0000 7f7692a00640 1 -- 172.21.15.50:0/78976494 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f769401a070 con 0x7f769c1527b0 2024-09-21T05:56:33.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.780+0000 7f7692a00640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7678073230 0x7f7678075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:33.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.780+0000 7f7692a00640 1 -- 172.21.15.50:0/78976494 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f76940936f0 con 0x7f769c1527b0 2024-09-21T05:56:33.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.780+0000 7f7699e00640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7678073230 0x7f7678075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:33.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.780+0000 7f7699e00640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7678073230 0x7f7678075740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f768c009eb0 tx=0x7f768c009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:33.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.784+0000 7f7692a00640 1 -- 172.21.15.50:0/78976494 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f769405e5f0 con 0x7f769c1527b0 2024-09-21T05:56:33.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.956+0000 7f76a0a00640 1 -- 172.21.15.50:0/78976494 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f769c1444f0 con 0x7f7678073230 2024-09-21T05:56:33.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.964+0000 7f7692a00640 1 -- 172.21.15.50:0/78976494 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241479 (secure 0 0 0) 0x7f768c009b00 con 0x7f7678073230 2024-09-21T05:56:33.967 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:33.971 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:56:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.968+0000 7f76a0a00640 1 -- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7678073230 msgr2=0x7f7678075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.968+0000 7f76a0a00640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7678073230 0x7f7678075740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f768c009eb0 tx=0x7f768c009500 comp rx=0 tx=0).stop 2024-09-21T05:56:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.968+0000 7f76a0a00640 1 -- 172.21.15.50:0/78976494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c1527b0 msgr2=0x7f769c1fbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.968+0000 7f76a0a00640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c1527b0 0x7f769c1fbe30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f769400c3c0 tx=0x7f769400c7c0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.968+0000 7f76a0a00640 1 -- 172.21.15.50:0/78976494 shutdown_connections 2024-09-21T05:56:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.968+0000 7f76a0a00640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7678073230 0x7f7678075740 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.968+0000 7f76a0a00640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c1527b0 0x7f769c1fbe30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.968+0000 7f76a0a00640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f769c0b5c70 0x7f769c1436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.968+0000 7f76a0a00640 1 --2- 172.21.15.50:0/78976494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f769c0b4de0 0x7f769c0cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.968+0000 7f76a0a00640 1 -- 172.21.15.50:0/78976494 >> 172.21.15.50:0/78976494 conn(0x7f769c0b2040 msgr2=0x7f769c151b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.968+0000 7f76a0a00640 1 -- 172.21.15.50:0/78976494 shutdown_connections 2024-09-21T05:56:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:33.968+0000 7f76a0a00640 1 -- 172.21.15.50:0/78976494 wait complete. 2024-09-21T05:56:33.990 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-21T05:56:32.884553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235724,"kb_used_data":2608,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738996,"statfs":{"total":773094113280,"available":772852731904,"internally_reserved":0,"allocated":2670592,"data_stored":1638267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.130423"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780332+0000","last_change":"2024-09-21T05:56:29.802492+0000","last_active":"2024-09-21T05:56:30.780332+0000","last_peered":"2024-09-21T05:56:30.780332+0000","last_clean":"2024-09-21T05:56:30.780332+0000","last_became_active":"2024-09-21T05:56:29.801940+0000","last_became_peered":"2024-09-21T05:56:29.801940+0000","last_unstale":"2024-09-21T05:56:30.780332+0000","last_undegraded":"2024-09-21T05:56:30.780332+0000","last_fullsized":"2024-09-21T05:56:30.780332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:11.434269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779759+0000","last_change":"2024-09-21T05:56:29.789973+0000","last_active":"2024-09-21T05:56:30.779759+0000","last_peered":"2024-09-21T05:56:30.779759+0000","last_clean":"2024-09-21T05:56:30.779759+0000","last_became_active":"2024-09-21T05:56:29.789200+0000","last_became_peered":"2024-09-21T05:56:29.789200+0000","last_unstale":"2024-09-21T05:56:30.779759+0000","last_undegraded":"2024-09-21T05:56:30.779759+0000","last_fullsized":"2024-09-21T05:56:30.779759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:52.718337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779890+0000","last_change":"2024-09-21T05:56:29.788425+0000","last_active":"2024-09-21T05:56:30.779890+0000","last_peered":"2024-09-21T05:56:30.779890+0000","last_clean":"2024-09-21T05:56:30.779890+0000","last_became_active":"2024-09-21T05:56:29.788007+0000","last_became_peered":"2024-09-21T05:56:29.788007+0000","last_unstale":"2024-09-21T05:56:30.779890+0000","last_undegraded":"2024-09-21T05:56:30.779890+0000","last_fullsized":"2024-09-21T05:56:30.779890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:53.445482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.763425+0000","last_change":"2024-09-21T05:56:27.763425+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.763425+0000","last_undegraded":"2024-09-21T05:56:27.763425+0000","last_fullsized":"2024-09-21T05:56:27.763425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.762418+0000","last_change":"2024-09-21T05:56:27.762418+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.762418+0000","last_undegraded":"2024-09-21T05:56:27.762418+0000","last_fullsized":"2024-09-21T05:56:27.762418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.763817+0000","last_change":"2024-09-21T05:56:27.763817+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.763817+0000","last_undegraded":"2024-09-21T05:56:27.763817+0000","last_fullsized":"2024-09-21T05:56:27.763817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758333+0000","last_change":"2024-09-21T05:56:27.758333+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758333+0000","last_undegraded":"2024-09-21T05:56:27.758333+0000","last_fullsized":"2024-09-21T05:56:27.758333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779980+0000","last_change":"2024-09-21T05:56:29.790318+0000","last_active":"2024-09-21T05:56:30.779980+0000","last_peered":"2024-09-21T05:56:30.779980+0000","last_clean":"2024-09-21T05:56:30.779980+0000","last_became_active":"2024-09-21T05:56:29.789628+0000","last_became_peered":"2024-09-21T05:56:29.789628+0000","last_unstale":"2024-09-21T05:56:30.779980+0000","last_undegraded":"2024-09-21T05:56:30.779980+0000","last_fullsized":"2024-09-21T05:56:30.779980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:39.627991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.763171+0000","last_change":"2024-09-21T05:56:27.763171+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.763171+0000","last_undegraded":"2024-09-21T05:56:27.763171+0000","last_fullsized":"2024-09-21T05:56:27.763171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.759298+0000","last_change":"2024-09-21T05:56:27.759298+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.759298+0000","last_undegraded":"2024-09-21T05:56:27.759298+0000","last_fullsized":"2024-09-21T05:56:27.759298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779982+0000","last_change":"2024-09-21T05:56:28.791961+0000","last_active":"2024-09-21T05:56:30.779982+0000","last_peered":"2024-09-21T05:56:30.779982+0000","last_clean":"2024-09-21T05:56:30.779982+0000","last_became_active":"2024-09-21T05:56:28.791654+0000","last_became_peered":"2024-09-21T05:56:28.791654+0000","last_unstale":"2024-09-21T05:56:30.779982+0000","last_undegraded":"2024-09-21T05:56:30.779982+0000","last_fullsized":"2024-09-21T05:56:30.779982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:29.186182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.762326+0000","last_change":"2024-09-21T05:56:27.762326+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.762326+0000","last_undegraded":"2024-09-21T05:56:27.762326+0000","last_fullsized":"2024-09-21T05:56:27.762326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.770548+0000","last_change":"2024-09-21T05:56:27.770548+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.770548+0000","last_undegraded":"2024-09-21T05:56:27.770548+0000","last_fullsized":"2024-09-21T05:56:27.770548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.759135+0000","last_change":"2024-09-21T05:56:27.759135+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.759135+0000","last_undegraded":"2024-09-21T05:56:27.759135+0000","last_fullsized":"2024-09-21T05:56:27.759135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.762448+0000","last_change":"2024-09-21T05:56:27.762448+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.762448+0000","last_undegraded":"2024-09-21T05:56:27.762448+0000","last_fullsized":"2024-09-21T05:56:27.762448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758665+0000","last_change":"2024-09-21T05:56:27.758665+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758665+0000","last_undegraded":"2024-09-21T05:56:27.758665+0000","last_fullsized":"2024-09-21T05:56:27.758665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758019+0000","last_change":"2024-09-21T05:56:27.758019+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758019+0000","last_undegraded":"2024-09-21T05:56:27.758019+0000","last_fullsized":"2024-09-21T05:56:27.758019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758538+0000","last_change":"2024-09-21T05:56:27.758538+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758538+0000","last_undegraded":"2024-09-21T05:56:27.758538+0000","last_fullsized":"2024-09-21T05:56:27.758538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780028+0000","last_change":"2024-09-21T05:56:28.788606+0000","last_active":"2024-09-21T05:56:30.780028+0000","last_peered":"2024-09-21T05:56:30.780028+0000","last_clean":"2024-09-21T05:56:30.780028+0000","last_became_active":"2024-09-21T05:56:28.787648+0000","last_became_peered":"2024-09-21T05:56:28.787648+0000","last_unstale":"2024-09-21T05:56:30.780028+0000","last_undegraded":"2024-09-21T05:56:30.780028+0000","last_fullsized":"2024-09-21T05:56:30.780028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:37.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758669+0000","last_change":"2024-09-21T05:56:27.758669+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758669+0000","last_undegraded":"2024-09-21T05:56:27.758669+0000","last_fullsized":"2024-09-21T05:56:27.758669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T05:56:27.768396+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:56:27.768396+0000","last_peered":"2024-09-21T05:56:27.768396+0000","last_clean":"2024-09-21T05:56:27.768396+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:56:27.768396+0000","last_undegraded":"2024-09-21T05:56:27.768396+0000","last_fullsized":"2024-09-21T05:56:27.768396+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758908+0000","last_change":"2024-09-21T05:56:27.758908+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758908+0000","last_undegraded":"2024-09-21T05:56:27.758908+0000","last_fullsized":"2024-09-21T05:56:27.758908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767011+0000","last_change":"2024-09-21T05:56:27.767011+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767011+0000","last_undegraded":"2024-09-21T05:56:27.767011+0000","last_fullsized":"2024-09-21T05:56:27.767011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758535+0000","last_change":"2024-09-21T05:56:27.758535+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758535+0000","last_undegraded":"2024-09-21T05:56:27.758535+0000","last_fullsized":"2024-09-21T05:56:27.758535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779811+0000","last_change":"2024-09-21T05:56:29.785876+0000","last_active":"2024-09-21T05:56:30.779811+0000","last_peered":"2024-09-21T05:56:30.779811+0000","last_clean":"2024-09-21T05:56:30.779811+0000","last_became_active":"2024-09-21T05:56:29.785564+0000","last_became_peered":"2024-09-21T05:56:29.785564+0000","last_unstale":"2024-09-21T05:56:30.779811+0000","last_undegraded":"2024-09-21T05:56:30.779811+0000","last_fullsized":"2024-09-21T05:56:30.779811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:04.812480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758346+0000","last_change":"2024-09-21T05:56:27.758346+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758346+0000","last_undegraded":"2024-09-21T05:56:27.758346+0000","last_fullsized":"2024-09-21T05:56:27.758346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780096+0000","last_change":"2024-09-21T05:56:28.791893+0000","last_active":"2024-09-21T05:56:30.780096+0000","last_peered":"2024-09-21T05:56:30.780096+0000","last_clean":"2024-09-21T05:56:30.780096+0000","last_became_active":"2024-09-21T05:56:28.791580+0000","last_became_peered":"2024-09-21T05:56:28.791580+0000","last_unstale":"2024-09-21T05:56:30.780096+0000","last_undegraded":"2024-09-21T05:56:30.780096+0000","last_fullsized":"2024-09-21T05:56:30.780096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.359445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767166+0000","last_change":"2024-09-21T05:56:27.767166+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767166+0000","last_undegraded":"2024-09-21T05:56:27.767166+0000","last_fullsized":"2024-09-21T05:56:27.767166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780054+0000","last_change":"2024-09-21T05:56:29.784905+0000","last_active":"2024-09-21T05:56:30.780054+0000","last_peered":"2024-09-21T05:56:30.780054+0000","last_clean":"2024-09-21T05:56:30.780054+0000","last_became_active":"2024-09-21T05:56:29.784562+0000","last_became_peered":"2024-09-21T05:56:29.784562+0000","last_unstale":"2024-09-21T05:56:30.780054+0000","last_undegraded":"2024-09-21T05:56:30.780054+0000","last_fullsized":"2024-09-21T05:56:30.780054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:26.482139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.795965+0000","last_change":"2024-09-21T05:56:28.790111+0000","last_active":"2024-09-21T05:56:30.795965+0000","last_peered":"2024-09-21T05:56:30.795965+0000","last_clean":"2024-09-21T05:56:30.795965+0000","last_became_active":"2024-09-21T05:56:28.789760+0000","last_became_peered":"2024-09-21T05:56:28.789760+0000","last_unstale":"2024-09-21T05:56:30.795965+0000","last_undegraded":"2024-09-21T05:56:30.795965+0000","last_fullsized":"2024-09-21T05:56:30.795965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:51.060754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.762312+0000","last_change":"2024-09-21T05:56:27.762312+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.762312+0000","last_undegraded":"2024-09-21T05:56:27.762312+0000","last_fullsized":"2024-09-21T05:56:27.762312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767893+0000","last_change":"2024-09-21T05:56:27.767893+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767893+0000","last_undegraded":"2024-09-21T05:56:27.767893+0000","last_fullsized":"2024-09-21T05:56:27.767893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779895+0000","last_change":"2024-09-21T05:56:28.801904+0000","last_active":"2024-09-21T05:56:30.779895+0000","last_peered":"2024-09-21T05:56:30.779895+0000","last_clean":"2024-09-21T05:56:30.779895+0000","last_became_active":"2024-09-21T05:56:28.801540+0000","last_became_peered":"2024-09-21T05:56:28.801540+0000","last_unstale":"2024-09-21T05:56:30.779895+0000","last_undegraded":"2024-09-21T05:56:30.779895+0000","last_fullsized":"2024-09-21T05:56:30.779895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:06.676644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779828+0000","last_change":"2024-09-21T05:56:28.784648+0000","last_active":"2024-09-21T05:56:30.779828+0000","last_peered":"2024-09-21T05:56:30.779828+0000","last_clean":"2024-09-21T05:56:30.779828+0000","last_became_active":"2024-09-21T05:56:28.784333+0000","last_became_peered":"2024-09-21T05:56:28.784333+0000","last_unstale":"2024-09-21T05:56:30.779828+0000","last_undegraded":"2024-09-21T05:56:30.779828+0000","last_fullsized":"2024-09-21T05:56:30.779828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:15.892877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767048+0000","last_change":"2024-09-21T05:56:27.767048+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767048+0000","last_undegraded":"2024-09-21T05:56:27.767048+0000","last_fullsized":"2024-09-21T05:56:27.767048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767137+0000","last_change":"2024-09-21T05:56:27.767137+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767137+0000","last_undegraded":"2024-09-21T05:56:27.767137+0000","last_fullsized":"2024-09-21T05:56:27.767137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767964+0000","last_change":"2024-09-21T05:56:27.767964+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767964+0000","last_undegraded":"2024-09-21T05:56:27.767964+0000","last_fullsized":"2024-09-21T05:56:27.767964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.764260+0000","last_change":"2024-09-21T05:56:27.764260+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.764260+0000","last_undegraded":"2024-09-21T05:56:27.764260+0000","last_fullsized":"2024-09-21T05:56:27.764260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.766338+0000","last_change":"2024-09-21T05:56:27.766338+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.766338+0000","last_undegraded":"2024-09-21T05:56:27.766338+0000","last_fullsized":"2024-09-21T05:56:27.766338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:56:33.994 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:56:34.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.188+0000 7f4323800640 1 -- 172.21.15.50:0/3345634203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c157030 msgr2=0x7f431c0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.188+0000 7f4323800640 1 --2- 172.21.15.50:0/3345634203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c157030 0x7f431c0b5a80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f430c009840 tx=0x7f430c01c790 comp rx=0 tx=0).stop 2024-09-21T05:56:34.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.188+0000 7f4323800640 1 -- 172.21.15.50:0/3345634203 shutdown_connections 2024-09-21T05:56:34.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.188+0000 7f4323800640 1 --2- 172.21.15.50:0/3345634203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c157030 0x7f431c0b5a80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.188+0000 7f4323800640 1 -- 172.21.15.50:0/3345634203 >> 172.21.15.50:0/3345634203 conn(0x7f431c0708c0 msgr2=0x7f431c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:34.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.188+0000 7f4323800640 1 -- 172.21.15.50:0/3345634203 shutdown_connections 2024-09-21T05:56:34.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.188+0000 7f4323800640 1 -- 172.21.15.50:0/3345634203 wait complete. 2024-09-21T05:56:34.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.188+0000 7f4323800640 1 Processor -- start 2024-09-21T05:56:34.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.188+0000 7f4323800640 1 -- start start 2024-09-21T05:56:34.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4323800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431c157030 0x7f431c1f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4323800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c160d50 0x7f431c1fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4323800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f431c0b3860 0x7f431c0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4323800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f431c16b820 con 0x7f431c160d50 2024-09-21T05:56:34.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4323800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f431c16b6a0 con 0x7f431c157030 2024-09-21T05:56:34.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4323800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f431c16b9a0 con 0x7f431c0b3860 2024-09-21T05:56:34.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4313e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c160d50 0x7f431c1fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4313e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c160d50 0x7f431c1fb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43290/0 (socket says 172.21.15.50:43290) 2024-09-21T05:56:34.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4313e00640 1 -- 172.21.15.50:0/2417364020 learned_addr learned my addr 172.21.15.50:0/2417364020 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:34.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4320e00640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431c157030 0x7f431c1f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4321800640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f431c0b3860 0x7f431c0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4313e00640 1 -- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f431c0b3860 msgr2=0x7f431c0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4313e00640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f431c0b3860 0x7f431c0b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4313e00640 1 -- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431c157030 msgr2=0x7f431c1f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4313e00640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431c157030 0x7f431c1f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4313e00640 1 -- 172.21.15.50:0/2417364020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c1fbb30 con 0x7f431c160d50 2024-09-21T05:56:34.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4313e00640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c160d50 0x7f431c1fb5f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f430400c3c0 tx=0x7f430400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:34.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4319600640 1 -- 172.21.15.50:0/2417364020 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304017070 con 0x7f431c160d50 2024-09-21T05:56:34.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4319600640 1 -- 172.21.15.50:0/2417364020 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43040053c0 con 0x7f431c160d50 2024-09-21T05:56:34.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4323800640 1 -- 172.21.15.50:0/2417364020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c152280 con 0x7f431c160d50 2024-09-21T05:56:34.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4319600640 1 -- 172.21.15.50:0/2417364020 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304003860 con 0x7f431c160d50 2024-09-21T05:56:34.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4323800640 1 -- 172.21.15.50:0/2417364020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c1f7700 con 0x7f431c160d50 2024-09-21T05:56:34.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.192+0000 7f4323800640 1 -- 172.21.15.50:0/2417364020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431c151d10 con 0x7f431c160d50 2024-09-21T05:56:34.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.196+0000 7f4319600640 1 -- 172.21.15.50:0/2417364020 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f430401a070 con 0x7f431c160d50 2024-09-21T05:56:34.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.196+0000 7f4319600640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f42f8073200 0x7f42f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.196+0000 7f4319600640 1 -- 172.21.15.50:0/2417364020 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f43040934e0 con 0x7f431c160d50 2024-09-21T05:56:34.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.196+0000 7f4320e00640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f42f8073200 0x7f42f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.196+0000 7f4320e00640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f42f8073200 0x7f42f8075710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f430c0af000 tx=0x7f430c0b2040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:34.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.200+0000 7f4319600640 1 -- 172.21.15.50:0/2417364020 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f431c1fbb30 con 0x7f431c160d50 2024-09-21T05:56:34.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.376+0000 7f4323800640 1 -- 172.21.15.50:0/2417364020 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f431c151ae0 con 0x7f42f8073200 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.380+0000 7f4319600640 1 -- 172.21.15.50:0/2417364020 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241479 (secure 0 0 0) 0x7f430c0a68f0 con 0x7f42f8073200 2024-09-21T05:56:34.385 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:34.387 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:56:34.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.384+0000 7f4323800640 1 -- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f42f8073200 msgr2=0x7f42f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.384+0000 7f4323800640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f42f8073200 0x7f42f8075710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f430c0af000 tx=0x7f430c0b2040 comp rx=0 tx=0).stop 2024-09-21T05:56:34.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.384+0000 7f4323800640 1 -- 172.21.15.50:0/2417364020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c160d50 msgr2=0x7f431c1fb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.384+0000 7f4323800640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c160d50 0x7f431c1fb5f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f430400c3c0 tx=0x7f430400c880 comp rx=0 tx=0).stop 2024-09-21T05:56:34.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.384+0000 7f4323800640 1 -- 172.21.15.50:0/2417364020 shutdown_connections 2024-09-21T05:56:34.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.384+0000 7f4323800640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f431c0b3860 0x7f431c0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.384+0000 7f4323800640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f42f8073200 0x7f42f8075710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.384+0000 7f4323800640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c160d50 0x7f431c1fb5f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.384+0000 7f4323800640 1 --2- 172.21.15.50:0/2417364020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431c157030 0x7f431c1f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.384+0000 7f4323800640 1 -- 172.21.15.50:0/2417364020 >> 172.21.15.50:0/2417364020 conn(0x7f431c0708c0 msgr2=0x7f431c0b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:34.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.384+0000 7f4323800640 1 -- 172.21.15.50:0/2417364020 shutdown_connections 2024-09-21T05:56:34.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.384+0000 7f4323800640 1 -- 172.21.15.50:0/2417364020 wait complete. 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-21T05:56:32.884553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235724,"kb_used_data":2608,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738996,"statfs":{"total":773094113280,"available":772852731904,"internally_reserved":0,"allocated":2670592,"data_stored":1638267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.130423"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780332+0000","last_change":"2024-09-21T05:56:29.802492+0000","last_active":"2024-09-21T05:56:30.780332+0000","last_peered":"2024-09-21T05:56:30.780332+0000","last_clean":"2024-09-21T05:56:30.780332+0000","last_became_active":"2024-09-21T05:56:29.801940+0000","last_became_peered":"2024-09-21T05:56:29.801940+0000","last_unstale":"2024-09-21T05:56:30.780332+0000","last_undegraded":"2024-09-21T05:56:30.780332+0000","last_fullsized":"2024-09-21T05:56:30.780332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:11.434269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779759+0000","last_change":"2024-09-21T05:56:29.789973+0000","last_active":"2024-09-21T05:56:30.779759+0000","last_peered":"2024-09-21T05:56:30.779759+0000","last_clean":"2024-09-21T05:56:30.779759+0000","last_became_active":"2024-09-21T05:56:29.789200+0000","last_became_peered":"2024-09-21T05:56:29.789200+0000","last_unstale":"2024-09-21T05:56:30.779759+0000","last_undegraded":"2024-09-21T05:56:30.779759+0000","last_fullsized":"2024-09-21T05:56:30.779759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:52.718337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779890+0000","last_change":"2024-09-21T05:56:29.788425+0000","last_active":"2024-09-21T05:56:30.779890+0000","last_peered":"2024-09-21T05:56:30.779890+0000","last_clean":"2024-09-21T05:56:30.779890+0000","last_became_active":"2024-09-21T05:56:29.788007+0000","last_became_peered":"2024-09-21T05:56:29.788007+0000","last_unstale":"2024-09-21T05:56:30.779890+0000","last_undegraded":"2024-09-21T05:56:30.779890+0000","last_fullsized":"2024-09-21T05:56:30.779890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:53.445482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.763425+0000","last_change":"2024-09-21T05:56:27.763425+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.763425+0000","last_undegraded":"2024-09-21T05:56:27.763425+0000","last_fullsized":"2024-09-21T05:56:27.763425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.762418+0000","last_change":"2024-09-21T05:56:27.762418+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.762418+0000","last_undegraded":"2024-09-21T05:56:27.762418+0000","last_fullsized":"2024-09-21T05:56:27.762418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.763817+0000","last_change":"2024-09-21T05:56:27.763817+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.763817+0000","last_undegraded":"2024-09-21T05:56:27.763817+0000","last_fullsized":"2024-09-21T05:56:27.763817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758333+0000","last_change":"2024-09-21T05:56:27.758333+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758333+0000","last_undegraded":"2024-09-21T05:56:27.758333+0000","last_fullsized":"2024-09-21T05:56:27.758333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779980+0000","last_change":"2024-09-21T05:56:29.790318+0000","last_active":"2024-09-21T05:56:30.779980+0000","last_peered":"2024-09-21T05:56:30.779980+0000","last_clean":"2024-09-21T05:56:30.779980+0000","last_became_active":"2024-09-21T05:56:29.789628+0000","last_became_peered":"2024-09-21T05:56:29.789628+0000","last_unstale":"2024-09-21T05:56:30.779980+0000","last_undegraded":"2024-09-21T05:56:30.779980+0000","last_fullsized":"2024-09-21T05:56:30.779980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:39.627991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.763171+0000","last_change":"2024-09-21T05:56:27.763171+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.763171+0000","last_undegraded":"2024-09-21T05:56:27.763171+0000","last_fullsized":"2024-09-21T05:56:27.763171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.759298+0000","last_change":"2024-09-21T05:56:27.759298+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.759298+0000","last_undegraded":"2024-09-21T05:56:27.759298+0000","last_fullsized":"2024-09-21T05:56:27.759298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779982+0000","last_change":"2024-09-21T05:56:28.791961+0000","last_active":"2024-09-21T05:56:30.779982+0000","last_peered":"2024-09-21T05:56:30.779982+0000","last_clean":"2024-09-21T05:56:30.779982+0000","last_became_active":"2024-09-21T05:56:28.791654+0000","last_became_peered":"2024-09-21T05:56:28.791654+0000","last_unstale":"2024-09-21T05:56:30.779982+0000","last_undegraded":"2024-09-21T05:56:30.779982+0000","last_fullsized":"2024-09-21T05:56:30.779982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:29.186182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.762326+0000","last_change":"2024-09-21T05:56:27.762326+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.762326+0000","last_undegraded":"2024-09-21T05:56:27.762326+0000","last_fullsized":"2024-09-21T05:56:27.762326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.770548+0000","last_change":"2024-09-21T05:56:27.770548+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.770548+0000","last_undegraded":"2024-09-21T05:56:27.770548+0000","last_fullsized":"2024-09-21T05:56:27.770548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.759135+0000","last_change":"2024-09-21T05:56:27.759135+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.759135+0000","last_undegraded":"2024-09-21T05:56:27.759135+0000","last_fullsized":"2024-09-21T05:56:27.759135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.762448+0000","last_change":"2024-09-21T05:56:27.762448+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.762448+0000","last_undegraded":"2024-09-21T05:56:27.762448+0000","last_fullsized":"2024-09-21T05:56:27.762448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758665+0000","last_change":"2024-09-21T05:56:27.758665+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758665+0000","last_undegraded":"2024-09-21T05:56:27.758665+0000","last_fullsized":"2024-09-21T05:56:27.758665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758019+0000","last_change":"2024-09-21T05:56:27.758019+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758019+0000","last_undegraded":"2024-09-21T05:56:27.758019+0000","last_fullsized":"2024-09-21T05:56:27.758019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758538+0000","last_change":"2024-09-21T05:56:27.758538+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758538+0000","last_undegraded":"2024-09-21T05:56:27.758538+0000","last_fullsized":"2024-09-21T05:56:27.758538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780028+0000","last_change":"2024-09-21T05:56:28.788606+0000","last_active":"2024-09-21T05:56:30.780028+0000","last_peered":"2024-09-21T05:56:30.780028+0000","last_clean":"2024-09-21T05:56:30.780028+0000","last_became_active":"2024-09-21T05:56:28.787648+0000","last_became_peered":"2024-09-21T05:56:28.787648+0000","last_unstale":"2024-09-21T05:56:30.780028+0000","last_undegraded":"2024-09-21T05:56:30.780028+0000","last_fullsized":"2024-09-21T05:56:30.780028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:37.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758669+0000","last_change":"2024-09-21T05:56:27.758669+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758669+0000","last_undegraded":"2024-09-21T05:56:27.758669+0000","last_fullsized":"2024-09-21T05:56:27.758669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T05:56:27.768396+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:56:27.768396+0000","last_peered":"2024-09-21T05:56:27.768396+0000","last_clean":"2024-09-21T05:56:27.768396+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:56:27.768396+0000","last_undegraded":"2024-09-21T05:56:27.768396+0000","last_fullsized":"2024-09-21T05:56:27.768396+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758908+0000","last_change":"2024-09-21T05:56:27.758908+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758908+0000","last_undegraded":"2024-09-21T05:56:27.758908+0000","last_fullsized":"2024-09-21T05:56:27.758908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767011+0000","last_change":"2024-09-21T05:56:27.767011+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767011+0000","last_undegraded":"2024-09-21T05:56:27.767011+0000","last_fullsized":"2024-09-21T05:56:27.767011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758535+0000","last_change":"2024-09-21T05:56:27.758535+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758535+0000","last_undegraded":"2024-09-21T05:56:27.758535+0000","last_fullsized":"2024-09-21T05:56:27.758535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779811+0000","last_change":"2024-09-21T05:56:29.785876+0000","last_active":"2024-09-21T05:56:30.779811+0000","last_peered":"2024-09-21T05:56:30.779811+0000","last_clean":"2024-09-21T05:56:30.779811+0000","last_became_active":"2024-09-21T05:56:29.785564+0000","last_became_peered":"2024-09-21T05:56:29.785564+0000","last_unstale":"2024-09-21T05:56:30.779811+0000","last_undegraded":"2024-09-21T05:56:30.779811+0000","last_fullsized":"2024-09-21T05:56:30.779811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:04.812480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758346+0000","last_change":"2024-09-21T05:56:27.758346+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758346+0000","last_undegraded":"2024-09-21T05:56:27.758346+0000","last_fullsized":"2024-09-21T05:56:27.758346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780096+0000","last_change":"2024-09-21T05:56:28.791893+0000","last_active":"2024-09-21T05:56:30.780096+0000","last_peered":"2024-09-21T05:56:30.780096+0000","last_clean":"2024-09-21T05:56:30.780096+0000","last_became_active":"2024-09-21T05:56:28.791580+0000","last_became_peered":"2024-09-21T05:56:28.791580+0000","last_unstale":"2024-09-21T05:56:30.780096+0000","last_undegraded":"2024-09-21T05:56:30.780096+0000","last_fullsized":"2024-09-21T05:56:30.780096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.359445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767166+0000","last_change":"2024-09-21T05:56:27.767166+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767166+0000","last_undegraded":"2024-09-21T05:56:27.767166+0000","last_fullsized":"2024-09-21T05:56:27.767166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780054+0000","last_change":"2024-09-21T05:56:29.784905+0000","last_active":"2024-09-21T05:56:30.780054+0000","last_peered":"2024-09-21T05:56:30.780054+0000","last_clean":"2024-09-21T05:56:30.780054+0000","last_became_active":"2024-09-21T05:56:29.784562+0000","last_became_peered":"2024-09-21T05:56:29.784562+0000","last_unstale":"2024-09-21T05:56:30.780054+0000","last_undegraded":"2024-09-21T05:56:30.780054+0000","last_fullsized":"2024-09-21T05:56:30.780054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:26.482139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.795965+0000","last_change":"2024-09-21T05:56:28.790111+0000","last_active":"2024-09-21T05:56:30.795965+0000","last_peered":"2024-09-21T05:56:30.795965+0000","last_clean":"2024-09-21T05:56:30.795965+0000","last_became_active":"2024-09-21T05:56:28.789760+0000","last_became_peered":"2024-09-21T05:56:28.789760+0000","last_unstale":"2024-09-21T05:56:30.795965+0000","last_undegraded":"2024-09-21T05:56:30.795965+0000","last_fullsized":"2024-09-21T05:56:30.795965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:51.060754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.762312+0000","last_change":"2024-09-21T05:56:27.762312+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.762312+0000","last_undegraded":"2024-09-21T05:56:27.762312+0000","last_fullsized":"2024-09-21T05:56:27.762312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767893+0000","last_change":"2024-09-21T05:56:27.767893+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767893+0000","last_undegraded":"2024-09-21T05:56:27.767893+0000","last_fullsized":"2024-09-21T05:56:27.767893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779895+0000","last_change":"2024-09-21T05:56:28.801904+0000","last_active":"2024-09-21T05:56:30.779895+0000","last_peered":"2024-09-21T05:56:30.779895+0000","last_clean":"2024-09-21T05:56:30.779895+0000","last_became_active":"2024-09-21T05:56:28.801540+0000","last_became_peered":"2024-09-21T05:56:28.801540+0000","last_unstale":"2024-09-21T05:56:30.779895+0000","last_undegraded":"2024-09-21T05:56:30.779895+0000","last_fullsized":"2024-09-21T05:56:30.779895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:06.676644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779828+0000","last_change":"2024-09-21T05:56:28.784648+0000","last_active":"2024-09-21T05:56:30.779828+0000","last_peered":"2024-09-21T05:56:30.779828+0000","last_clean":"2024-09-21T05:56:30.779828+0000","last_became_active":"2024-09-21T05:56:28.784333+0000","last_became_peered":"2024-09-21T05:56:28.784333+0000","last_unstale":"2024-09-21T05:56:30.779828+0000","last_undegraded":"2024-09-21T05:56:30.779828+0000","last_fullsized":"2024-09-21T05:56:30.779828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:15.892877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767048+0000","last_change":"2024-09-21T05:56:27.767048+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767048+0000","last_undegraded":"2024-09-21T05:56:27.767048+0000","last_fullsized":"2024-09-21T05:56:27.767048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767137+0000","last_change":"2024-09-21T05:56:27.767137+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767137+0000","last_undegraded":"2024-09-21T05:56:27.767137+0000","last_fullsized":"2024-09-21T05:56:27.767137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767964+0000","last_change":"2024-09-21T05:56:27.767964+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767964+0000","last_undegraded":"2024-09-21T05:56:27.767964+0000","last_fullsized":"2024-09-21T05:56:27.767964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.764260+0000","last_change":"2024-09-21T05:56:27.764260+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.764260+0000","last_undegraded":"2024-09-21T05:56:27.764260+0000","last_fullsized":"2024-09-21T05:56:27.764260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.766338+0000","last_change":"2024-09-21T05:56:27.766338+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.766338+0000","last_undegraded":"2024-09-21T05:56:27.766338+0000","last_fullsized":"2024-09-21T05:56:27.766338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:56:34.410 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:56:34.598 INFO:tasks.ceph.mds.b.smithi088.stderr:2024-09-21T05:56:34.593+0000 7f967ee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:56:34.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.608+0000 7f2200e00640 1 -- 172.21.15.50:0/2426121510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21fc15d280 msgr2=0x7f21fc15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.608+0000 7f2200e00640 1 --2- 172.21.15.50:0/2426121510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21fc15d280 0x7f21fc15d690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f21e400a950 tx=0x7f21e401db70 comp rx=0 tx=0).stop 2024-09-21T05:56:34.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.608+0000 7f2200e00640 1 -- 172.21.15.50:0/2426121510 shutdown_connections 2024-09-21T05:56:34.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.608+0000 7f2200e00640 1 --2- 172.21.15.50:0/2426121510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21fc15d280 0x7f21fc15d690 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.608+0000 7f2200e00640 1 -- 172.21.15.50:0/2426121510 >> 172.21.15.50:0/2426121510 conn(0x7f21fc0b2040 msgr2=0x7f21fc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:34.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.608+0000 7f2200e00640 1 -- 172.21.15.50:0/2426121510 shutdown_connections 2024-09-21T05:56:34.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.608+0000 7f2200e00640 1 -- 172.21.15.50:0/2426121510 wait complete. 2024-09-21T05:56:34.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.608+0000 7f2200e00640 1 Processor -- start 2024-09-21T05:56:34.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.608+0000 7f2200e00640 1 -- start start 2024-09-21T05:56:34.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f2200e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21fc15d280 0x7f21fc1f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f2200e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc15dbd0 0x7f21fc1610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f2200e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21fc15e8c0 0x7f21fc161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f2200e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc169310 con 0x7f21fc15dbd0 2024-09-21T05:56:34.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f2200e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc169190 con 0x7f21fc15e8c0 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f2200e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f21fc169490 con 0x7f21fc15d280 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc15dbd0 0x7f21fc1610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc15dbd0 0x7f21fc1610d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43296/0 (socket says 172.21.15.50:43296) 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f3e00640 1 -- 172.21.15.50:0/2010456989 learned_addr learned my addr 172.21.15.50:0/2010456989 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f9e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21fc15d280 0x7f21fc1f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21fa800640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21fc15e8c0 0x7f21fc161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f3e00640 1 -- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21fc15d280 msgr2=0x7f21fc1f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f3e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21fc15d280 0x7f21fc1f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f3e00640 1 -- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21fc15e8c0 msgr2=0x7f21fc161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f3e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21fc15e8c0 0x7f21fc161610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f3e00640 1 -- 172.21.15.50:0/2010456989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc13e0c0 con 0x7f21fc15dbd0 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f9e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21fc15d280 0x7f21fc1f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:34.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f3e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc15dbd0 0x7f21fc1610d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f21e400a8a0 tx=0x7f21e40038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:34.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f2000640 1 -- 172.21.15.50:0/2010456989 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e40bd070 con 0x7f21fc15dbd0 2024-09-21T05:56:34.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f2000640 1 -- 172.21.15.50:0/2010456989 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21e4003b90 con 0x7f21fc15dbd0 2024-09-21T05:56:34.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f2200e00640 1 -- 172.21.15.50:0/2010456989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc1580b0 con 0x7f21fc15dbd0 2024-09-21T05:56:34.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f21f2000640 1 -- 172.21.15.50:0/2010456989 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e40b3040 con 0x7f21fc15dbd0 2024-09-21T05:56:34.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f2200e00640 1 -- 172.21.15.50:0/2010456989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc157dd0 con 0x7f21fc15dbd0 2024-09-21T05:56:34.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.612+0000 7f2200e00640 1 -- 172.21.15.50:0/2010456989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21fc15d690 con 0x7f21fc15dbd0 2024-09-21T05:56:34.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.616+0000 7f21f2000640 1 -- 172.21.15.50:0/2010456989 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f21e4005030 con 0x7f21fc15dbd0 2024-09-21T05:56:34.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.616+0000 7f21f2000640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21d8073200 0x7f21d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.616+0000 7f21f2000640 1 -- 172.21.15.50:0/2010456989 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f21e412e530 con 0x7f21fc15dbd0 2024-09-21T05:56:34.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.616+0000 7f21f9e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21d8073200 0x7f21d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.616+0000 7f21f9e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21d8073200 0x7f21d8075710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f21ec000f70 tx=0x7f21ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:34.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.620+0000 7f21f2000640 1 -- 172.21.15.50:0/2010456989 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f21e4017760 con 0x7f21fc15dbd0 2024-09-21T05:56:34.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.792+0000 7f2200e00640 1 -- 172.21.15.50:0/2010456989 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21fc06b540 con 0x7f21d8073200 2024-09-21T05:56:34.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.796+0000 7f21f2000640 1 -- 172.21.15.50:0/2010456989 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241479 (secure 0 0 0) 0x7f21ec0013b0 con 0x7f21d8073200 2024-09-21T05:56:34.803 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:34.804 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:56:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.800+0000 7f2200e00640 1 -- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21d8073200 msgr2=0x7f21d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.800+0000 7f2200e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21d8073200 0x7f21d8075710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f21ec000f70 tx=0x7f21ec009040 comp rx=0 tx=0).stop 2024-09-21T05:56:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.800+0000 7f2200e00640 1 -- 172.21.15.50:0/2010456989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc15dbd0 msgr2=0x7f21fc1610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.800+0000 7f2200e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc15dbd0 0x7f21fc1610d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f21e400a8a0 tx=0x7f21e40038b0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.800+0000 7f2200e00640 1 -- 172.21.15.50:0/2010456989 shutdown_connections 2024-09-21T05:56:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.800+0000 7f2200e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21fc15e8c0 0x7f21fc161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.800+0000 7f2200e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21d8073200 0x7f21d8075710 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.800+0000 7f2200e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc15dbd0 0x7f21fc1610d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.800+0000 7f2200e00640 1 --2- 172.21.15.50:0/2010456989 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21fc15d280 0x7f21fc1f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.800+0000 7f2200e00640 1 -- 172.21.15.50:0/2010456989 >> 172.21.15.50:0/2010456989 conn(0x7f21fc0b2040 msgr2=0x7f21fc15b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.800+0000 7f2200e00640 1 -- 172.21.15.50:0/2010456989 shutdown_connections 2024-09-21T05:56:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:34.800+0000 7f2200e00640 1 -- 172.21.15.50:0/2010456989 wait complete. 2024-09-21T05:56:34.825 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-21T05:56:32.884553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235724,"kb_used_data":2608,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738996,"statfs":{"total":773094113280,"available":772852731904,"internally_reserved":0,"allocated":2670592,"data_stored":1638267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.130423"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780332+0000","last_change":"2024-09-21T05:56:29.802492+0000","last_active":"2024-09-21T05:56:30.780332+0000","last_peered":"2024-09-21T05:56:30.780332+0000","last_clean":"2024-09-21T05:56:30.780332+0000","last_became_active":"2024-09-21T05:56:29.801940+0000","last_became_peered":"2024-09-21T05:56:29.801940+0000","last_unstale":"2024-09-21T05:56:30.780332+0000","last_undegraded":"2024-09-21T05:56:30.780332+0000","last_fullsized":"2024-09-21T05:56:30.780332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:11.434269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779759+0000","last_change":"2024-09-21T05:56:29.789973+0000","last_active":"2024-09-21T05:56:30.779759+0000","last_peered":"2024-09-21T05:56:30.779759+0000","last_clean":"2024-09-21T05:56:30.779759+0000","last_became_active":"2024-09-21T05:56:29.789200+0000","last_became_peered":"2024-09-21T05:56:29.789200+0000","last_unstale":"2024-09-21T05:56:30.779759+0000","last_undegraded":"2024-09-21T05:56:30.779759+0000","last_fullsized":"2024-09-21T05:56:30.779759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:52.718337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779890+0000","last_change":"2024-09-21T05:56:29.788425+0000","last_active":"2024-09-21T05:56:30.779890+0000","last_peered":"2024-09-21T05:56:30.779890+0000","last_clean":"2024-09-21T05:56:30.779890+0000","last_became_active":"2024-09-21T05:56:29.788007+0000","last_became_peered":"2024-09-21T05:56:29.788007+0000","last_unstale":"2024-09-21T05:56:30.779890+0000","last_undegraded":"2024-09-21T05:56:30.779890+0000","last_fullsized":"2024-09-21T05:56:30.779890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:53.445482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.763425+0000","last_change":"2024-09-21T05:56:27.763425+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.763425+0000","last_undegraded":"2024-09-21T05:56:27.763425+0000","last_fullsized":"2024-09-21T05:56:27.763425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.762418+0000","last_change":"2024-09-21T05:56:27.762418+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.762418+0000","last_undegraded":"2024-09-21T05:56:27.762418+0000","last_fullsized":"2024-09-21T05:56:27.762418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.763817+0000","last_change":"2024-09-21T05:56:27.763817+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.763817+0000","last_undegraded":"2024-09-21T05:56:27.763817+0000","last_fullsized":"2024-09-21T05:56:27.763817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758333+0000","last_change":"2024-09-21T05:56:27.758333+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758333+0000","last_undegraded":"2024-09-21T05:56:27.758333+0000","last_fullsized":"2024-09-21T05:56:27.758333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779980+0000","last_change":"2024-09-21T05:56:29.790318+0000","last_active":"2024-09-21T05:56:30.779980+0000","last_peered":"2024-09-21T05:56:30.779980+0000","last_clean":"2024-09-21T05:56:30.779980+0000","last_became_active":"2024-09-21T05:56:29.789628+0000","last_became_peered":"2024-09-21T05:56:29.789628+0000","last_unstale":"2024-09-21T05:56:30.779980+0000","last_undegraded":"2024-09-21T05:56:30.779980+0000","last_fullsized":"2024-09-21T05:56:30.779980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:39.627991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.763171+0000","last_change":"2024-09-21T05:56:27.763171+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.763171+0000","last_undegraded":"2024-09-21T05:56:27.763171+0000","last_fullsized":"2024-09-21T05:56:27.763171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.759298+0000","last_change":"2024-09-21T05:56:27.759298+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.759298+0000","last_undegraded":"2024-09-21T05:56:27.759298+0000","last_fullsized":"2024-09-21T05:56:27.759298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779982+0000","last_change":"2024-09-21T05:56:28.791961+0000","last_active":"2024-09-21T05:56:30.779982+0000","last_peered":"2024-09-21T05:56:30.779982+0000","last_clean":"2024-09-21T05:56:30.779982+0000","last_became_active":"2024-09-21T05:56:28.791654+0000","last_became_peered":"2024-09-21T05:56:28.791654+0000","last_unstale":"2024-09-21T05:56:30.779982+0000","last_undegraded":"2024-09-21T05:56:30.779982+0000","last_fullsized":"2024-09-21T05:56:30.779982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:29.186182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.762326+0000","last_change":"2024-09-21T05:56:27.762326+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.762326+0000","last_undegraded":"2024-09-21T05:56:27.762326+0000","last_fullsized":"2024-09-21T05:56:27.762326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.770548+0000","last_change":"2024-09-21T05:56:27.770548+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.770548+0000","last_undegraded":"2024-09-21T05:56:27.770548+0000","last_fullsized":"2024-09-21T05:56:27.770548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.759135+0000","last_change":"2024-09-21T05:56:27.759135+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.759135+0000","last_undegraded":"2024-09-21T05:56:27.759135+0000","last_fullsized":"2024-09-21T05:56:27.759135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.762448+0000","last_change":"2024-09-21T05:56:27.762448+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.762448+0000","last_undegraded":"2024-09-21T05:56:27.762448+0000","last_fullsized":"2024-09-21T05:56:27.762448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758665+0000","last_change":"2024-09-21T05:56:27.758665+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758665+0000","last_undegraded":"2024-09-21T05:56:27.758665+0000","last_fullsized":"2024-09-21T05:56:27.758665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758019+0000","last_change":"2024-09-21T05:56:27.758019+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758019+0000","last_undegraded":"2024-09-21T05:56:27.758019+0000","last_fullsized":"2024-09-21T05:56:27.758019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758538+0000","last_change":"2024-09-21T05:56:27.758538+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758538+0000","last_undegraded":"2024-09-21T05:56:27.758538+0000","last_fullsized":"2024-09-21T05:56:27.758538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780028+0000","last_change":"2024-09-21T05:56:28.788606+0000","last_active":"2024-09-21T05:56:30.780028+0000","last_peered":"2024-09-21T05:56:30.780028+0000","last_clean":"2024-09-21T05:56:30.780028+0000","last_became_active":"2024-09-21T05:56:28.787648+0000","last_became_peered":"2024-09-21T05:56:28.787648+0000","last_unstale":"2024-09-21T05:56:30.780028+0000","last_undegraded":"2024-09-21T05:56:30.780028+0000","last_fullsized":"2024-09-21T05:56:30.780028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:37.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758669+0000","last_change":"2024-09-21T05:56:27.758669+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758669+0000","last_undegraded":"2024-09-21T05:56:27.758669+0000","last_fullsized":"2024-09-21T05:56:27.758669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T05:56:27.768396+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:56:27.768396+0000","last_peered":"2024-09-21T05:56:27.768396+0000","last_clean":"2024-09-21T05:56:27.768396+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:56:27.768396+0000","last_undegraded":"2024-09-21T05:56:27.768396+0000","last_fullsized":"2024-09-21T05:56:27.768396+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758908+0000","last_change":"2024-09-21T05:56:27.758908+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758908+0000","last_undegraded":"2024-09-21T05:56:27.758908+0000","last_fullsized":"2024-09-21T05:56:27.758908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767011+0000","last_change":"2024-09-21T05:56:27.767011+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767011+0000","last_undegraded":"2024-09-21T05:56:27.767011+0000","last_fullsized":"2024-09-21T05:56:27.767011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758535+0000","last_change":"2024-09-21T05:56:27.758535+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758535+0000","last_undegraded":"2024-09-21T05:56:27.758535+0000","last_fullsized":"2024-09-21T05:56:27.758535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779811+0000","last_change":"2024-09-21T05:56:29.785876+0000","last_active":"2024-09-21T05:56:30.779811+0000","last_peered":"2024-09-21T05:56:30.779811+0000","last_clean":"2024-09-21T05:56:30.779811+0000","last_became_active":"2024-09-21T05:56:29.785564+0000","last_became_peered":"2024-09-21T05:56:29.785564+0000","last_unstale":"2024-09-21T05:56:30.779811+0000","last_undegraded":"2024-09-21T05:56:30.779811+0000","last_fullsized":"2024-09-21T05:56:30.779811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:04.812480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.758346+0000","last_change":"2024-09-21T05:56:27.758346+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.758346+0000","last_undegraded":"2024-09-21T05:56:27.758346+0000","last_fullsized":"2024-09-21T05:56:27.758346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780096+0000","last_change":"2024-09-21T05:56:28.791893+0000","last_active":"2024-09-21T05:56:30.780096+0000","last_peered":"2024-09-21T05:56:30.780096+0000","last_clean":"2024-09-21T05:56:30.780096+0000","last_became_active":"2024-09-21T05:56:28.791580+0000","last_became_peered":"2024-09-21T05:56:28.791580+0000","last_unstale":"2024-09-21T05:56:30.780096+0000","last_undegraded":"2024-09-21T05:56:30.780096+0000","last_fullsized":"2024-09-21T05:56:30.780096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.359445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767166+0000","last_change":"2024-09-21T05:56:27.767166+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767166+0000","last_undegraded":"2024-09-21T05:56:27.767166+0000","last_fullsized":"2024-09-21T05:56:27.767166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780054+0000","last_change":"2024-09-21T05:56:29.784905+0000","last_active":"2024-09-21T05:56:30.780054+0000","last_peered":"2024-09-21T05:56:30.780054+0000","last_clean":"2024-09-21T05:56:30.780054+0000","last_became_active":"2024-09-21T05:56:29.784562+0000","last_became_peered":"2024-09-21T05:56:29.784562+0000","last_unstale":"2024-09-21T05:56:30.780054+0000","last_undegraded":"2024-09-21T05:56:30.780054+0000","last_fullsized":"2024-09-21T05:56:30.780054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:26.482139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.795965+0000","last_change":"2024-09-21T05:56:28.790111+0000","last_active":"2024-09-21T05:56:30.795965+0000","last_peered":"2024-09-21T05:56:30.795965+0000","last_clean":"2024-09-21T05:56:30.795965+0000","last_became_active":"2024-09-21T05:56:28.789760+0000","last_became_peered":"2024-09-21T05:56:28.789760+0000","last_unstale":"2024-09-21T05:56:30.795965+0000","last_undegraded":"2024-09-21T05:56:30.795965+0000","last_fullsized":"2024-09-21T05:56:30.795965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:51.060754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.762312+0000","last_change":"2024-09-21T05:56:27.762312+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.762312+0000","last_undegraded":"2024-09-21T05:56:27.762312+0000","last_fullsized":"2024-09-21T05:56:27.762312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767893+0000","last_change":"2024-09-21T05:56:27.767893+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767893+0000","last_undegraded":"2024-09-21T05:56:27.767893+0000","last_fullsized":"2024-09-21T05:56:27.767893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779895+0000","last_change":"2024-09-21T05:56:28.801904+0000","last_active":"2024-09-21T05:56:30.779895+0000","last_peered":"2024-09-21T05:56:30.779895+0000","last_clean":"2024-09-21T05:56:30.779895+0000","last_became_active":"2024-09-21T05:56:28.801540+0000","last_became_peered":"2024-09-21T05:56:28.801540+0000","last_unstale":"2024-09-21T05:56:30.779895+0000","last_undegraded":"2024-09-21T05:56:30.779895+0000","last_fullsized":"2024-09-21T05:56:30.779895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:06.676644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779828+0000","last_change":"2024-09-21T05:56:28.784648+0000","last_active":"2024-09-21T05:56:30.779828+0000","last_peered":"2024-09-21T05:56:30.779828+0000","last_clean":"2024-09-21T05:56:30.779828+0000","last_became_active":"2024-09-21T05:56:28.784333+0000","last_became_peered":"2024-09-21T05:56:28.784333+0000","last_unstale":"2024-09-21T05:56:30.779828+0000","last_undegraded":"2024-09-21T05:56:30.779828+0000","last_fullsized":"2024-09-21T05:56:30.779828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:15.892877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767048+0000","last_change":"2024-09-21T05:56:27.767048+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767048+0000","last_undegraded":"2024-09-21T05:56:27.767048+0000","last_fullsized":"2024-09-21T05:56:27.767048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767137+0000","last_change":"2024-09-21T05:56:27.767137+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767137+0000","last_undegraded":"2024-09-21T05:56:27.767137+0000","last_fullsized":"2024-09-21T05:56:27.767137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.767964+0000","last_change":"2024-09-21T05:56:27.767964+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.767964+0000","last_undegraded":"2024-09-21T05:56:27.767964+0000","last_fullsized":"2024-09-21T05:56:27.767964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.764260+0000","last_change":"2024-09-21T05:56:27.764260+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.764260+0000","last_undegraded":"2024-09-21T05:56:27.764260+0000","last_fullsized":"2024-09-21T05:56:27.764260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:56:28.753562+0000","last_change":"2024-09-21T05:56:28.753562+0000","last_active":"2024-09-21T05:56:28.753562+0000","last_peered":"2024-09-21T05:56:28.753562+0000","last_clean":"2024-09-21T05:56:28.753562+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:28.753562+0000","last_undegraded":"2024-09-21T05:56:28.753562+0000","last_fullsized":"2024-09-21T05:56:28.753562+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-21T05:56:27.766338+0000","last_change":"2024-09-21T05:56:27.766338+0000","last_active":"2024-09-21T05:56:27.745698+0000","last_peered":"2024-09-21T05:56:27.745698+0000","last_clean":"2024-09-21T05:56:27.745698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:56:27.766338+0000","last_undegraded":"2024-09-21T05:56:27.766338+0000","last_fullsized":"2024-09-21T05:56:27.766338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:56:34.831 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-21T05:56:34.831 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.831 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-21T05:56:34.831 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.831 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-21T05:56:34.831 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.831 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-21T05:56:34.831 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.831 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-21T05:56:34.831 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.832 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-21T05:56:34.832 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.832 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-21T05:56:34.832 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.832 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-21T05:56:34.832 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.832 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-21T05:56:34.832 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.832 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-21T05:56:34.832 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.832 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-21T05:56:34.832 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.833 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-21T05:56:34.833 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.833 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-21T05:56:34.833 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.833 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-21T05:56:34.833 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.833 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-21T05:56:34.833 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.833 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-21T05:56:34.833 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.833 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-21T05:56:34.834 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.834 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-21T05:56:34.834 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.834 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-21T05:56:34.834 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.834 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-21T05:56:34.834 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.834 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-21T05:56:34.834 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.834 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-21T05:56:34.835 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.835 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-21T05:56:34.835 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.835 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-21T05:56:34.835 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.835 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-21T05:56:34.835 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.835 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-21T05:56:34.835 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.835 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-21T05:56:34.835 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.836 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-21T05:56:34.836 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.836 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-21T05:56:34.836 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.836 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-21T05:56:34.836 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.836 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1b is not active+clean 2024-09-21T05:56:34.836 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.763425+0000', 'last_change': '2024-09-21T05:56:27.763425+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.763425+0000', 'last_undegraded': '2024-09-21T05:56:27.763425+0000', 'last_fullsized': '2024-09-21T05:56:27.763425+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T05:56:34.836 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-21T05:56:34.836 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.836 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1a is not active+clean 2024-09-21T05:56:34.836 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.762418+0000', 'last_change': '2024-09-21T05:56:27.762418+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.762418+0000', 'last_undegraded': '2024-09-21T05:56:27.762418+0000', 'last_fullsized': '2024-09-21T05:56:27.762418+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:56:34.837 INFO:tasks.ceph.ceph_manager.ceph:PG 2.19 is not active+clean 2024-09-21T05:56:34.837 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.763817+0000', 'last_change': '2024-09-21T05:56:27.763817+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.763817+0000', 'last_undegraded': '2024-09-21T05:56:27.763817+0000', 'last_fullsized': '2024-09-21T05:56:27.763817+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T05:56:34.837 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-21T05:56:34.837 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.837 INFO:tasks.ceph.ceph_manager.ceph:PG 2.18 is not active+clean 2024-09-21T05:56:34.837 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.758333+0000', 'last_change': '2024-09-21T05:56:27.758333+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.758333+0000', 'last_undegraded': '2024-09-21T05:56:27.758333+0000', 'last_fullsized': '2024-09-21T05:56:27.758333+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T05:56:34.837 INFO:tasks.ceph.ceph_manager.ceph:PG 2.17 is not active+clean 2024-09-21T05:56:34.837 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.763171+0000', 'last_change': '2024-09-21T05:56:27.763171+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.763171+0000', 'last_undegraded': '2024-09-21T05:56:27.763171+0000', 'last_fullsized': '2024-09-21T05:56:27.763171+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:56:34.837 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-21T05:56:34.837 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.837 INFO:tasks.ceph.ceph_manager.ceph:PG 2.16 is not active+clean 2024-09-21T05:56:34.837 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.759298+0000', 'last_change': '2024-09-21T05:56:27.759298+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.759298+0000', 'last_undegraded': '2024-09-21T05:56:27.759298+0000', 'last_fullsized': '2024-09-21T05:56:27.759298+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T05:56:34.838 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-21T05:56:34.838 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.838 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-21T05:56:34.838 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.838 INFO:tasks.ceph.ceph_manager.ceph:PG 2.14 is not active+clean 2024-09-21T05:56:34.838 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.762326+0000', 'last_change': '2024-09-21T05:56:27.762326+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.762326+0000', 'last_undegraded': '2024-09-21T05:56:27.762326+0000', 'last_fullsized': '2024-09-21T05:56:27.762326+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:56:34.838 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-21T05:56:34.838 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.838 INFO:tasks.ceph.ceph_manager.ceph:PG 2.13 is not active+clean 2024-09-21T05:56:34.838 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.770548+0000', 'last_change': '2024-09-21T05:56:27.770548+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.770548+0000', 'last_undegraded': '2024-09-21T05:56:27.770548+0000', 'last_fullsized': '2024-09-21T05:56:27.770548+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:56:34.838 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-21T05:56:34.838 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.839 INFO:tasks.ceph.ceph_manager.ceph:PG 2.12 is not active+clean 2024-09-21T05:56:34.839 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.759135+0000', 'last_change': '2024-09-21T05:56:27.759135+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.759135+0000', 'last_undegraded': '2024-09-21T05:56:27.759135+0000', 'last_fullsized': '2024-09-21T05:56:27.759135+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T05:56:34.839 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-21T05:56:34.839 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.839 INFO:tasks.ceph.ceph_manager.ceph:PG 2.11 is not active+clean 2024-09-21T05:56:34.839 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.762448+0000', 'last_change': '2024-09-21T05:56:27.762448+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.762448+0000', 'last_undegraded': '2024-09-21T05:56:27.762448+0000', 'last_fullsized': '2024-09-21T05:56:27.762448+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:56:34.839 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-21T05:56:34.839 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.839 INFO:tasks.ceph.ceph_manager.ceph:PG 2.10 is not active+clean 2024-09-21T05:56:34.839 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.758665+0000', 'last_change': '2024-09-21T05:56:27.758665+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.758665+0000', 'last_undegraded': '2024-09-21T05:56:27.758665+0000', 'last_fullsized': '2024-09-21T05:56:27.758665+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T05:56:34.839 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-21T05:56:34.839 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.840 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-09-21T05:56:34.840 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.758019+0000', 'last_change': '2024-09-21T05:56:27.758019+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.758019+0000', 'last_undegraded': '2024-09-21T05:56:27.758019+0000', 'last_fullsized': '2024-09-21T05:56:27.758019+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T05:56:34.840 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-21T05:56:34.840 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.840 INFO:tasks.ceph.ceph_manager.ceph:PG 2.e is not active+clean 2024-09-21T05:56:34.840 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.758538+0000', 'last_change': '2024-09-21T05:56:27.758538+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.758538+0000', 'last_undegraded': '2024-09-21T05:56:27.758538+0000', 'last_fullsized': '2024-09-21T05:56:27.758538+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T05:56:34.840 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-21T05:56:34.840 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.840 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-21T05:56:34.840 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.840 INFO:tasks.ceph.ceph_manager.ceph:PG 2.c is not active+clean 2024-09-21T05:56:34.841 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.758669+0000', 'last_change': '2024-09-21T05:56:27.758669+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.758669+0000', 'last_undegraded': '2024-09-21T05:56:27.758669+0000', 'last_fullsized': '2024-09-21T05:56:27.758669+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T05:56:34.841 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-21T05:56:34.841 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.841 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2024-09-21T05:56:34.841 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.758908+0000', 'last_change': '2024-09-21T05:56:27.758908+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.758908+0000', 'last_undegraded': '2024-09-21T05:56:27.758908+0000', 'last_fullsized': '2024-09-21T05:56:27.758908+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T05:56:34.841 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2 is not active+clean 2024-09-21T05:56:34.841 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.841 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-09-21T05:56:34.841 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.767011+0000', 'last_change': '2024-09-21T05:56:27.767011+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.767011+0000', 'last_undegraded': '2024-09-21T05:56:27.767011+0000', 'last_fullsized': '2024-09-21T05:56:27.767011+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:56:34.841 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-21T05:56:34.841 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.842 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2024-09-21T05:56:34.842 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.758535+0000', 'last_change': '2024-09-21T05:56:27.758535+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.758535+0000', 'last_undegraded': '2024-09-21T05:56:27.758535+0000', 'last_fullsized': '2024-09-21T05:56:27.758535+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T05:56:34.842 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2024-09-21T05:56:34.842 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.758346+0000', 'last_change': '2024-09-21T05:56:27.758346+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.758346+0000', 'last_undegraded': '2024-09-21T05:56:27.758346+0000', 'last_fullsized': '2024-09-21T05:56:27.758346+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T05:56:34.842 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-21T05:56:34.842 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.842 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-21T05:56:34.842 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.842 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-09-21T05:56:34.842 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.767166+0000', 'last_change': '2024-09-21T05:56:27.767166+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.767166+0000', 'last_undegraded': '2024-09-21T05:56:27.767166+0000', 'last_fullsized': '2024-09-21T05:56:27.767166+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:56:34.842 INFO:tasks.ceph.ceph_manager.ceph:PG 3.7 is not active+clean 2024-09-21T05:56:34.843 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.843 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2024-09-21T05:56:34.843 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.762312+0000', 'last_change': '2024-09-21T05:56:27.762312+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.762312+0000', 'last_undegraded': '2024-09-21T05:56:27.762312+0000', 'last_fullsized': '2024-09-21T05:56:27.762312+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:56:34.843 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-21T05:56:34.843 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.843 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-09-21T05:56:34.843 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.767893+0000', 'last_change': '2024-09-21T05:56:27.767893+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.767893+0000', 'last_undegraded': '2024-09-21T05:56:27.767893+0000', 'last_fullsized': '2024-09-21T05:56:27.767893+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:56:34.843 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-21T05:56:34.843 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.843 INFO:tasks.ceph.ceph_manager.ceph:PG 3.8 is not active+clean 2024-09-21T05:56:34.843 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.844 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-21T05:56:34.844 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.844 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-09-21T05:56:34.844 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.767048+0000', 'last_change': '2024-09-21T05:56:27.767048+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.767048+0000', 'last_undegraded': '2024-09-21T05:56:27.767048+0000', 'last_fullsized': '2024-09-21T05:56:27.767048+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:56:34.844 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-21T05:56:34.844 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.844 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-21T05:56:34.844 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.844 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-21T05:56:34.844 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.767137+0000', 'last_change': '2024-09-21T05:56:27.767137+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.767137+0000', 'last_undegraded': '2024-09-21T05:56:27.767137+0000', 'last_fullsized': '2024-09-21T05:56:27.767137+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:56:34.844 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-21T05:56:34.844 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.845 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-09-21T05:56:34.845 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.767964+0000', 'last_change': '2024-09-21T05:56:27.767964+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.767964+0000', 'last_undegraded': '2024-09-21T05:56:27.767964+0000', 'last_fullsized': '2024-09-21T05:56:27.767964+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:56:34.845 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-21T05:56:34.845 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.845 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1e is not active+clean 2024-09-21T05:56:34.845 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.764260+0000', 'last_change': '2024-09-21T05:56:27.764260+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.764260+0000', 'last_undegraded': '2024-09-21T05:56:27.764260+0000', 'last_fullsized': '2024-09-21T05:56:27.764260+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T05:56:34.845 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-21T05:56:34.845 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:56:28.753562+0000', 'last_change': '2024-09-21T05:56:28.753562+0000', 'last_active': '2024-09-21T05:56:28.753562+0000', 'last_peered': '2024-09-21T05:56:28.753562+0000', 'last_clean': '2024-09-21T05:56:28.753562+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:28.753562+0000', 'last_undegraded': '2024-09-21T05:56:28.753562+0000', 'last_fullsized': '2024-09-21T05:56:28.753562+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:28.753562+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:56:34.845 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1f is not active+clean 2024-09-21T05:56:34.845 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:56:27.766338+0000', 'last_change': '2024-09-21T05:56:27.766338+0000', 'last_active': '2024-09-21T05:56:27.745698+0000', 'last_peered': '2024-09-21T05:56:27.745698+0000', 'last_clean': '2024-09-21T05:56:27.745698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:56:27.766338+0000', 'last_undegraded': '2024-09-21T05:56:27.766338+0000', 'last_fullsized': '2024-09-21T05:56:27.766338+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'last_clean_scrub_stamp': '2024-09-21T05:56:27.745698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:56:34.846 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T05:56:35.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.020+0000 7f1a07400640 1 -- 172.21.15.50:0/3223977196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a000b5a80 msgr2=0x7f1a00074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.020+0000 7f1a07400640 1 --2- 172.21.15.50:0/3223977196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a000b5a80 0x7f1a00074410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009900 tx=0x7f19ec01c8e0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.024+0000 7f1a07400640 1 -- 172.21.15.50:0/3223977196 shutdown_connections 2024-09-21T05:56:35.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.024+0000 7f1a07400640 1 --2- 172.21.15.50:0/3223977196 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a00074950 0x7f1a0015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.024+0000 7f1a07400640 1 --2- 172.21.15.50:0/3223977196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a000b5a80 0x7f1a00074410 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.024+0000 7f1a07400640 1 -- 172.21.15.50:0/3223977196 >> 172.21.15.50:0/3223977196 conn(0x7f1a000708c0 msgr2=0x7f1a00070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:35.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.024+0000 7f1a07400640 1 -- 172.21.15.50:0/3223977196 shutdown_connections 2024-09-21T05:56:35.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.024+0000 7f1a07400640 1 -- 172.21.15.50:0/3223977196 wait complete. 2024-09-21T05:56:35.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.024+0000 7f1a07400640 1 Processor -- start 2024-09-21T05:56:35.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.024+0000 7f1a07400640 1 -- start start 2024-09-21T05:56:35.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.024+0000 7f1a07400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a00074950 0x7f1a00154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.024+0000 7f1a07400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a000b5a80 0x7f1a000b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f1a07400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a00157030 0x7f1a000b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f1a07400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0016a310 con 0x7f1a00157030 2024-09-21T05:56:35.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f1a07400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0016a190 con 0x7f1a00074950 2024-09-21T05:56:35.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f1a07400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0016a490 con 0x7f1a000b5a80 2024-09-21T05:56:35.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f1a05400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a00157030 0x7f1a000b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f19ffe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a000b5a80 0x7f1a000b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f1a04a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a00074950 0x7f1a00154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f19ffe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a000b5a80 0x7f1a000b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:57736/0 (socket says 172.21.15.50:57736) 2024-09-21T05:56:35.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f19ffe00640 1 -- 172.21.15.50:0/252031386 learned_addr learned my addr 172.21.15.50:0/252031386 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:35.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f1a05400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a00157030 0x7f1a000b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43300/0 (socket says 172.21.15.50:43300) 2024-09-21T05:56:35.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f19ffe00640 1 -- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a00074950 msgr2=0x7f1a00154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f19ffe00640 1 --2- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a00074950 0x7f1a00154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f19ffe00640 1 -- 172.21.15.50:0/252031386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a00157030 msgr2=0x7f1a000b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f19ffe00640 1 --2- 172.21.15.50:0/252031386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a00157030 0x7f1a000b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f19ffe00640 1 -- 172.21.15.50:0/252031386 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a000b4e30 con 0x7f1a000b5a80 2024-09-21T05:56:35.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f19ffe00640 1 --2- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a000b5a80 0x7f1a000b36b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0b0040 tx=0x7f19ec0a5710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:35.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f19fcc00640 1 -- 172.21.15.50:0/252031386 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0ba020 con 0x7f1a000b5a80 2024-09-21T05:56:35.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f19fcc00640 1 -- 172.21.15.50:0/252031386 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19ec0043b0 con 0x7f1a000b5a80 2024-09-21T05:56:35.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f19fcc00640 1 -- 172.21.15.50:0/252031386 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0b2040 con 0x7f1a000b5a80 2024-09-21T05:56:35.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f1a07400640 1 -- 172.21.15.50:0/252031386 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a00156070 con 0x7f1a000b5a80 2024-09-21T05:56:35.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f1a07400640 1 -- 172.21.15.50:0/252031386 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a00156630 con 0x7f1a000b5a80 2024-09-21T05:56:35.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.028+0000 7f1a07400640 1 -- 172.21.15.50:0/252031386 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a00155de0 con 0x7f1a000b5a80 2024-09-21T05:56:35.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.032+0000 7f19fcc00640 1 -- 172.21.15.50:0/252031386 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f19ec005290 con 0x7f1a000b5a80 2024-09-21T05:56:35.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.032+0000 7f19fcc00640 1 --2- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19e0073230 0x7f19e0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.032+0000 7f19fcc00640 1 -- 172.21.15.50:0/252031386 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f19ec12e2c0 con 0x7f1a000b5a80 2024-09-21T05:56:35.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.032+0000 7f1a04a00640 1 --2- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19e0073230 0x7f19e0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.032+0000 7f1a04a00640 1 --2- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19e0073230 0x7f19e0075740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f19f4007fd0 tx=0x7f19f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:35.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.036+0000 7f19fcc00640 1 -- 172.21.15.50:0/252031386 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f19ec0c4060 con 0x7f1a000b5a80 2024-09-21T05:56:35.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.272+0000 7f1a07400640 1 -- 172.21.15.50:0/252031386 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1a0015cf90 con 0x7f1a000b5a80 2024-09-21T05:56:35.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f19fcc00640 1 -- 172.21.15.50:0/252031386 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1212 (secure 0 0 0) 0x7f19ec0164d0 con 0x7f1a000b5a80 2024-09-21T05:56:35.277 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:35.277 INFO:teuthology.orchestra.run.smithi050.stdout:{"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1726898177,"num_in_osds":8,"osd_in_since":1726898167,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":58},{"state_name":"creating+peering","count":26},{"state_name":"active+clean","count":13}],"num_pgs":97,"num_pools":3,"num_objects":6,"data_bytes":590528,"bytes_used":241381376,"bytes_avail":772852731904,"bytes_total":773094113280,"unknown_pgs_ratio":0.59793812036514282,"inactive_pgs_ratio":0.2680412232875824,"write_bytes_sec":399,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":8,"btime":"2024-09-21T05:56:32:796889+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4371}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-21T05:56:03.543664+0000","services":{}},"progress_events":{}} 2024-09-21T05:56:35.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f1a07400640 1 -- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19e0073230 msgr2=0x7f19e0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f1a07400640 1 --2- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19e0073230 0x7f19e0075740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f19f4007fd0 tx=0x7f19f4009040 comp rx=0 tx=0).stop 2024-09-21T05:56:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f1a07400640 1 -- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a000b5a80 msgr2=0x7f1a000b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f1a07400640 1 --2- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a000b5a80 0x7f1a000b36b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0b0040 tx=0x7f19ec0a5710 comp rx=0 tx=0).stop 2024-09-21T05:56:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f1a07400640 1 -- 172.21.15.50:0/252031386 shutdown_connections 2024-09-21T05:56:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f1a07400640 1 --2- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19e0073230 0x7f19e0075740 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f1a07400640 1 --2- 172.21.15.50:0/252031386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a00157030 0x7f1a000b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f1a07400640 1 --2- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a000b5a80 0x7f1a000b36b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f1a07400640 1 --2- 172.21.15.50:0/252031386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a00074950 0x7f1a00154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f1a07400640 1 -- 172.21.15.50:0/252031386 >> 172.21.15.50:0/252031386 conn(0x7f1a000708c0 msgr2=0x7f1a0015f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f1a07400640 1 -- 172.21.15.50:0/252031386 shutdown_connections 2024-09-21T05:56:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.276+0000 7f1a07400640 1 -- 172.21.15.50:0/252031386 wait complete. 2024-09-21T05:56:35.299 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T05:56:35.299 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:56:35.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.452+0000 7feff8000640 1 -- 172.21.15.50:0/1516026835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff0157030 msgr2=0x7feff00b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.452+0000 7feff8000640 1 --2- 172.21.15.50:0/1516026835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff0157030 0x7feff00b5a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009900 tx=0x7fefe401c8e0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.456+0000 7feff8000640 1 -- 172.21.15.50:0/1516026835 shutdown_connections 2024-09-21T05:56:35.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.456+0000 7feff8000640 1 --2- 172.21.15.50:0/1516026835 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff0074b20 0x7feff015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.456+0000 7feff8000640 1 --2- 172.21.15.50:0/1516026835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff0157030 0x7feff00b5a80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.456+0000 7feff8000640 1 -- 172.21.15.50:0/1516026835 >> 172.21.15.50:0/1516026835 conn(0x7feff00708c0 msgr2=0x7feff0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:35.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.456+0000 7feff8000640 1 -- 172.21.15.50:0/1516026835 shutdown_connections 2024-09-21T05:56:35.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.456+0000 7feff8000640 1 -- 172.21.15.50:0/1516026835 wait complete. 2024-09-21T05:56:35.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.456+0000 7feff8000640 1 Processor -- start 2024-09-21T05:56:35.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.456+0000 7feff8000640 1 -- start start 2024-09-21T05:56:35.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.456+0000 7feff8000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff0157030 0x7feff0156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.456+0000 7feff8000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff0160d50 0x7feff013ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.456+0000 7feff8000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff01fcbe0 0x7feff01fcff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff8000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feff016a810 con 0x7feff01fcbe0 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff8000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feff016a690 con 0x7feff0160d50 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff8000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7feff016a990 con 0x7feff0157030 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff01fcbe0 0x7feff01fcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff01fcbe0 0x7feff01fcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43312/0 (socket says 172.21.15.50:43312) 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff6000640 1 -- 172.21.15.50:0/414243718 learned_addr learned my addr 172.21.15.50:0/414243718 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff4c00640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff0160d50 0x7feff013ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff5600640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff0157030 0x7feff0156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff6000640 1 -- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff0157030 msgr2=0x7feff0156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff6000640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff0157030 0x7feff0156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff6000640 1 -- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff0160d50 msgr2=0x7feff013ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff6000640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff0160d50 0x7feff013ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff6000640 1 -- 172.21.15.50:0/414243718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff013efa0 con 0x7feff01fcbe0 2024-09-21T05:56:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff5600640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff0157030 0x7feff0156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff6000640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff01fcbe0 0x7feff01fcff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fefe800b9a0 tx=0x7fefe800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7fefed600640 1 -- 172.21.15.50:0/414243718 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8019070 con 0x7feff01fcbe0 2024-09-21T05:56:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7fefed600640 1 -- 172.21.15.50:0/414243718 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefe800ccb0 con 0x7feff01fcbe0 2024-09-21T05:56:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7fefed600640 1 -- 172.21.15.50:0/414243718 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe80146a0 con 0x7feff01fcbe0 2024-09-21T05:56:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff8000640 1 -- 172.21.15.50:0/414243718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff0155660 con 0x7feff01fcbe0 2024-09-21T05:56:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff8000640 1 -- 172.21.15.50:0/414243718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff0155c20 con 0x7feff01fcbe0 2024-09-21T05:56:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7feff8000640 1 -- 172.21.15.50:0/414243718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff00d2260 con 0x7feff01fcbe0 2024-09-21T05:56:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.460+0000 7fefed600640 1 -- 172.21.15.50:0/414243718 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fefe801c070 con 0x7feff01fcbe0 2024-09-21T05:56:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.464+0000 7fefed600640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fefd0073200 0x7fefd0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.464+0000 7fefed600640 1 -- 172.21.15.50:0/414243718 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fefe8093ee0 con 0x7feff01fcbe0 2024-09-21T05:56:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.464+0000 7feff5600640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fefd0073200 0x7fefd0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.464+0000 7feff5600640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fefd0073200 0x7fefd0075710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefe40af000 tx=0x7fefe401b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:35.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.468+0000 7fefed600640 1 -- 172.21.15.50:0/414243718 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fefe805ef80 con 0x7feff01fcbe0 2024-09-21T05:56:35.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.640+0000 7feff8000640 1 -- 172.21.15.50:0/414243718 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feff006b540 con 0x7fefd0073200 2024-09-21T05:56:35.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.644+0000 7fefed600640 1 -- 172.21.15.50:0/414243718 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251937 (secure 0 0 0) 0x7fefe4004140 con 0x7fefd0073200 2024-09-21T05:56:35.649 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:35.652 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:56:35.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.648+0000 7feff8000640 1 -- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fefd0073200 msgr2=0x7fefd0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.648+0000 7feff8000640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fefd0073200 0x7fefd0075710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefe40af000 tx=0x7fefe401b120 comp rx=0 tx=0).stop 2024-09-21T05:56:35.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.648+0000 7feff8000640 1 -- 172.21.15.50:0/414243718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff01fcbe0 msgr2=0x7feff01fcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.648+0000 7feff8000640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff01fcbe0 0x7feff01fcff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fefe800b9a0 tx=0x7fefe800be60 comp rx=0 tx=0).stop 2024-09-21T05:56:35.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.648+0000 7feff8000640 1 -- 172.21.15.50:0/414243718 shutdown_connections 2024-09-21T05:56:35.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.648+0000 7feff8000640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fefd0073200 0x7fefd0075710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.648+0000 7feff8000640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff01fcbe0 0x7feff01fcff0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.648+0000 7feff8000640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff0160d50 0x7feff013ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.648+0000 7feff8000640 1 --2- 172.21.15.50:0/414243718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff0157030 0x7feff0156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.648+0000 7feff8000640 1 -- 172.21.15.50:0/414243718 >> 172.21.15.50:0/414243718 conn(0x7feff00708c0 msgr2=0x7feff015c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:35.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.648+0000 7feff8000640 1 -- 172.21.15.50:0/414243718 shutdown_connections 2024-09-21T05:56:35.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:35.648+0000 7feff8000640 1 -- 172.21.15.50:0/414243718 wait complete. 2024-09-21T05:56:35.672 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-21T05:56:34.884992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235840,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738880,"statfs":{"total":773094113280,"available":772852613120,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.118866"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176632+0000","last_change":"2024-09-21T05:56:29.801294+0000","last_active":"2024-09-21T05:56:31.176632+0000","last_peered":"2024-09-21T05:56:31.176632+0000","last_clean":"2024-09-21T05:56:31.176632+0000","last_became_active":"2024-09-21T05:56:29.800947+0000","last_became_peered":"2024-09-21T05:56:29.800947+0000","last_unstale":"2024-09-21T05:56:31.176632+0000","last_undegraded":"2024-09-21T05:56:31.176632+0000","last_fullsized":"2024-09-21T05:56:31.176632+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:11.951093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780332+0000","last_change":"2024-09-21T05:56:29.802492+0000","last_active":"2024-09-21T05:56:30.780332+0000","last_peered":"2024-09-21T05:56:30.780332+0000","last_clean":"2024-09-21T05:56:30.780332+0000","last_became_active":"2024-09-21T05:56:29.801940+0000","last_became_peered":"2024-09-21T05:56:29.801940+0000","last_unstale":"2024-09-21T05:56:30.780332+0000","last_undegraded":"2024-09-21T05:56:30.780332+0000","last_fullsized":"2024-09-21T05:56:30.780332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:11.434269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178417+0000","last_change":"2024-09-21T05:56:29.804599+0000","last_active":"2024-09-21T05:56:31.178417+0000","last_peered":"2024-09-21T05:56:31.178417+0000","last_clean":"2024-09-21T05:56:31.178417+0000","last_became_active":"2024-09-21T05:56:29.804231+0000","last_became_peered":"2024-09-21T05:56:29.804231+0000","last_unstale":"2024-09-21T05:56:31.178417+0000","last_undegraded":"2024-09-21T05:56:31.178417+0000","last_fullsized":"2024-09-21T05:56:31.178417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782989+0000","last_change":"2024-09-21T05:56:29.790060+0000","last_active":"2024-09-21T05:56:30.782989+0000","last_peered":"2024-09-21T05:56:30.782989+0000","last_clean":"2024-09-21T05:56:30.782989+0000","last_became_active":"2024-09-21T05:56:29.786267+0000","last_became_peered":"2024-09-21T05:56:29.786267+0000","last_unstale":"2024-09-21T05:56:30.782989+0000","last_undegraded":"2024-09-21T05:56:30.782989+0000","last_fullsized":"2024-09-21T05:56:30.782989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:11.867996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176068+0000","last_change":"2024-09-21T05:56:29.809836+0000","last_active":"2024-09-21T05:56:31.176068+0000","last_peered":"2024-09-21T05:56:31.176068+0000","last_clean":"2024-09-21T05:56:31.176068+0000","last_became_active":"2024-09-21T05:56:29.809429+0000","last_became_peered":"2024-09-21T05:56:29.809429+0000","last_unstale":"2024-09-21T05:56:31.176068+0000","last_undegraded":"2024-09-21T05:56:31.176068+0000","last_fullsized":"2024-09-21T05:56:31.176068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:41.759804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782594+0000","last_change":"2024-09-21T05:56:29.794338+0000","last_active":"2024-09-21T05:56:30.782594+0000","last_peered":"2024-09-21T05:56:30.782594+0000","last_clean":"2024-09-21T05:56:30.782594+0000","last_became_active":"2024-09-21T05:56:29.794011+0000","last_became_peered":"2024-09-21T05:56:29.794011+0000","last_unstale":"2024-09-21T05:56:30.782594+0000","last_undegraded":"2024-09-21T05:56:30.782594+0000","last_fullsized":"2024-09-21T05:56:30.782594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:37.695152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176168+0000","last_change":"2024-09-21T05:56:29.806114+0000","last_active":"2024-09-21T05:56:31.176168+0000","last_peered":"2024-09-21T05:56:31.176168+0000","last_clean":"2024-09-21T05:56:31.176168+0000","last_became_active":"2024-09-21T05:56:29.805711+0000","last_became_peered":"2024-09-21T05:56:29.805711+0000","last_unstale":"2024-09-21T05:56:31.176168+0000","last_undegraded":"2024-09-21T05:56:31.176168+0000","last_fullsized":"2024-09-21T05:56:31.176168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:11.867996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177775+0000","last_change":"2024-09-21T05:56:29.800463+0000","last_active":"2024-09-21T05:56:31.177775+0000","last_peered":"2024-09-21T05:56:31.177775+0000","last_clean":"2024-09-21T05:56:31.177775+0000","last_became_active":"2024-09-21T05:56:29.800084+0000","last_became_peered":"2024-09-21T05:56:29.800084+0000","last_unstale":"2024-09-21T05:56:31.177775+0000","last_undegraded":"2024-09-21T05:56:31.177775+0000","last_fullsized":"2024-09-21T05:56:31.177775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:39.467641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176639+0000","last_change":"2024-09-21T05:56:29.809950+0000","last_active":"2024-09-21T05:56:31.176639+0000","last_peered":"2024-09-21T05:56:31.176639+0000","last_clean":"2024-09-21T05:56:31.176639+0000","last_became_active":"2024-09-21T05:56:29.809337+0000","last_became_peered":"2024-09-21T05:56:29.809337+0000","last_unstale":"2024-09-21T05:56:31.176639+0000","last_undegraded":"2024-09-21T05:56:31.176639+0000","last_fullsized":"2024-09-21T05:56:31.176639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:30.461662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796623+0000","last_change":"2024-09-21T05:56:29.781023+0000","last_active":"2024-09-21T05:56:30.796623+0000","last_peered":"2024-09-21T05:56:30.796623+0000","last_clean":"2024-09-21T05:56:30.796623+0000","last_became_active":"2024-09-21T05:56:29.780729+0000","last_became_peered":"2024-09-21T05:56:29.780729+0000","last_unstale":"2024-09-21T05:56:30.796623+0000","last_undegraded":"2024-09-21T05:56:30.796623+0000","last_fullsized":"2024-09-21T05:56:30.796623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:22.412975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796667+0000","last_change":"2024-09-21T05:56:29.779393+0000","last_active":"2024-09-21T05:56:30.796667+0000","last_peered":"2024-09-21T05:56:30.796667+0000","last_clean":"2024-09-21T05:56:30.796667+0000","last_became_active":"2024-09-21T05:56:29.779016+0000","last_became_peered":"2024-09-21T05:56:29.779016+0000","last_unstale":"2024-09-21T05:56:30.796667+0000","last_undegraded":"2024-09-21T05:56:30.796667+0000","last_fullsized":"2024-09-21T05:56:30.796667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:30.792281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.781580+0000","last_change":"2024-09-21T05:56:29.787708+0000","last_active":"2024-09-21T05:56:30.781580+0000","last_peered":"2024-09-21T05:56:30.781580+0000","last_clean":"2024-09-21T05:56:30.781580+0000","last_became_active":"2024-09-21T05:56:29.787370+0000","last_became_peered":"2024-09-21T05:56:29.787370+0000","last_unstale":"2024-09-21T05:56:30.781580+0000","last_undegraded":"2024-09-21T05:56:30.781580+0000","last_fullsized":"2024-09-21T05:56:30.781580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:11.868532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177932+0000","last_change":"2024-09-21T05:56:29.790480+0000","last_active":"2024-09-21T05:56:31.177932+0000","last_peered":"2024-09-21T05:56:31.177932+0000","last_clean":"2024-09-21T05:56:31.177932+0000","last_became_active":"2024-09-21T05:56:29.790172+0000","last_became_peered":"2024-09-21T05:56:29.790172+0000","last_unstale":"2024-09-21T05:56:31.177932+0000","last_undegraded":"2024-09-21T05:56:31.177932+0000","last_fullsized":"2024-09-21T05:56:31.177932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:14.929983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178355+0000","last_change":"2024-09-21T05:56:29.796096+0000","last_active":"2024-09-21T05:56:31.178355+0000","last_peered":"2024-09-21T05:56:31.178355+0000","last_clean":"2024-09-21T05:56:31.178355+0000","last_became_active":"2024-09-21T05:56:29.795760+0000","last_became_peered":"2024-09-21T05:56:29.795760+0000","last_unstale":"2024-09-21T05:56:31.178355+0000","last_undegraded":"2024-09-21T05:56:31.178355+0000","last_fullsized":"2024-09-21T05:56:31.178355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:33.544641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779759+0000","last_change":"2024-09-21T05:56:29.789973+0000","last_active":"2024-09-21T05:56:30.779759+0000","last_peered":"2024-09-21T05:56:30.779759+0000","last_clean":"2024-09-21T05:56:30.779759+0000","last_became_active":"2024-09-21T05:56:29.789200+0000","last_became_peered":"2024-09-21T05:56:29.789200+0000","last_unstale":"2024-09-21T05:56:30.779759+0000","last_undegraded":"2024-09-21T05:56:30.779759+0000","last_fullsized":"2024-09-21T05:56:30.779759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:52.718337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782539+0000","last_change":"2024-09-21T05:56:29.795924+0000","last_active":"2024-09-21T05:56:30.782539+0000","last_peered":"2024-09-21T05:56:30.782539+0000","last_clean":"2024-09-21T05:56:30.782539+0000","last_became_active":"2024-09-21T05:56:29.795518+0000","last_became_peered":"2024-09-21T05:56:29.795518+0000","last_unstale":"2024-09-21T05:56:30.782539+0000","last_undegraded":"2024-09-21T05:56:30.782539+0000","last_fullsized":"2024-09-21T05:56:30.782539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:56.052565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.784178+0000","last_change":"2024-09-21T05:56:29.803208+0000","last_active":"2024-09-21T05:56:30.784178+0000","last_peered":"2024-09-21T05:56:30.784178+0000","last_clean":"2024-09-21T05:56:30.784178+0000","last_became_active":"2024-09-21T05:56:29.802492+0000","last_became_peered":"2024-09-21T05:56:29.802492+0000","last_unstale":"2024-09-21T05:56:30.784178+0000","last_undegraded":"2024-09-21T05:56:30.784178+0000","last_fullsized":"2024-09-21T05:56:30.784178+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:11.140043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177406+0000","last_change":"2024-09-21T05:56:29.787284+0000","last_active":"2024-09-21T05:56:31.177406+0000","last_peered":"2024-09-21T05:56:31.177406+0000","last_clean":"2024-09-21T05:56:31.177406+0000","last_became_active":"2024-09-21T05:56:29.786932+0000","last_became_peered":"2024-09-21T05:56:29.786932+0000","last_unstale":"2024-09-21T05:56:31.177406+0000","last_undegraded":"2024-09-21T05:56:31.177406+0000","last_fullsized":"2024-09-21T05:56:31.177406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:45.264497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176533+0000","last_change":"2024-09-21T05:56:29.810657+0000","last_active":"2024-09-21T05:56:31.176533+0000","last_peered":"2024-09-21T05:56:31.176533+0000","last_clean":"2024-09-21T05:56:31.176533+0000","last_became_active":"2024-09-21T05:56:29.810300+0000","last_became_peered":"2024-09-21T05:56:29.810300+0000","last_unstale":"2024-09-21T05:56:31.176533+0000","last_undegraded":"2024-09-21T05:56:31.176533+0000","last_fullsized":"2024-09-21T05:56:31.176533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:28.055629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796891+0000","last_change":"2024-09-21T05:56:29.788766+0000","last_active":"2024-09-21T05:56:30.796891+0000","last_peered":"2024-09-21T05:56:30.796891+0000","last_clean":"2024-09-21T05:56:30.796891+0000","last_became_active":"2024-09-21T05:56:29.788430+0000","last_became_peered":"2024-09-21T05:56:29.788430+0000","last_unstale":"2024-09-21T05:56:30.796891+0000","last_undegraded":"2024-09-21T05:56:30.796891+0000","last_fullsized":"2024-09-21T05:56:30.796891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178114+0000","last_change":"2024-09-21T05:56:29.813309+0000","last_active":"2024-09-21T05:56:31.178114+0000","last_peered":"2024-09-21T05:56:31.178114+0000","last_clean":"2024-09-21T05:56:31.178114+0000","last_became_active":"2024-09-21T05:56:29.812998+0000","last_became_peered":"2024-09-21T05:56:29.812998+0000","last_unstale":"2024-09-21T05:56:31.178114+0000","last_undegraded":"2024-09-21T05:56:31.178114+0000","last_fullsized":"2024-09-21T05:56:31.178114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:30.792281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782744+0000","last_change":"2024-09-21T05:56:29.803935+0000","last_active":"2024-09-21T05:56:30.782744+0000","last_peered":"2024-09-21T05:56:30.782744+0000","last_clean":"2024-09-21T05:56:30.782744+0000","last_became_active":"2024-09-21T05:56:29.803639+0000","last_became_peered":"2024-09-21T05:56:29.803639+0000","last_unstale":"2024-09-21T05:56:30.782744+0000","last_undegraded":"2024-09-21T05:56:30.782744+0000","last_fullsized":"2024-09-21T05:56:30.782744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:41.759804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177783+0000","last_change":"2024-09-21T05:56:29.800369+0000","last_active":"2024-09-21T05:56:31.177783+0000","last_peered":"2024-09-21T05:56:31.177783+0000","last_clean":"2024-09-21T05:56:31.177783+0000","last_became_active":"2024-09-21T05:56:29.799794+0000","last_became_peered":"2024-09-21T05:56:29.799794+0000","last_unstale":"2024-09-21T05:56:31.177783+0000","last_undegraded":"2024-09-21T05:56:31.177783+0000","last_fullsized":"2024-09-21T05:56:31.177783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:17.854511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782813+0000","last_change":"2024-09-21T05:56:29.795633+0000","last_active":"2024-09-21T05:56:30.782813+0000","last_peered":"2024-09-21T05:56:30.782813+0000","last_clean":"2024-09-21T05:56:30.782813+0000","last_became_active":"2024-09-21T05:56:29.795285+0000","last_became_peered":"2024-09-21T05:56:29.795285+0000","last_unstale":"2024-09-21T05:56:30.782813+0000","last_undegraded":"2024-09-21T05:56:30.782813+0000","last_fullsized":"2024-09-21T05:56:30.782813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:04.441455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178478+0000","last_change":"2024-09-21T05:56:29.803384+0000","last_active":"2024-09-21T05:56:31.178478+0000","last_peered":"2024-09-21T05:56:31.178478+0000","last_clean":"2024-09-21T05:56:31.178478+0000","last_became_active":"2024-09-21T05:56:29.803152+0000","last_became_peered":"2024-09-21T05:56:29.803152+0000","last_unstale":"2024-09-21T05:56:31.178478+0000","last_undegraded":"2024-09-21T05:56:31.178478+0000","last_fullsized":"2024-09-21T05:56:31.178478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:48.777031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796724+0000","last_change":"2024-09-21T05:56:29.807556+0000","last_active":"2024-09-21T05:56:30.796724+0000","last_peered":"2024-09-21T05:56:30.796724+0000","last_clean":"2024-09-21T05:56:30.796724+0000","last_became_active":"2024-09-21T05:56:29.805826+0000","last_became_peered":"2024-09-21T05:56:29.805826+0000","last_unstale":"2024-09-21T05:56:30.796724+0000","last_undegraded":"2024-09-21T05:56:30.796724+0000","last_fullsized":"2024-09-21T05:56:30.796724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:33.544641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779890+0000","last_change":"2024-09-21T05:56:29.788425+0000","last_active":"2024-09-21T05:56:30.779890+0000","last_peered":"2024-09-21T05:56:30.779890+0000","last_clean":"2024-09-21T05:56:30.779890+0000","last_became_active":"2024-09-21T05:56:29.788007+0000","last_became_peered":"2024-09-21T05:56:29.788007+0000","last_unstale":"2024-09-21T05:56:30.779890+0000","last_undegraded":"2024-09-21T05:56:30.779890+0000","last_fullsized":"2024-09-21T05:56:30.779890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:53.445482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.179201+0000","last_change":"2024-09-21T05:56:29.792314+0000","last_active":"2024-09-21T05:56:31.179201+0000","last_peered":"2024-09-21T05:56:31.179201+0000","last_clean":"2024-09-21T05:56:31.179201+0000","last_became_active":"2024-09-21T05:56:29.791869+0000","last_became_peered":"2024-09-21T05:56:29.791869+0000","last_unstale":"2024-09-21T05:56:31.179201+0000","last_undegraded":"2024-09-21T05:56:31.179201+0000","last_fullsized":"2024-09-21T05:56:31.179201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:20.952370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177862+0000","last_change":"2024-09-21T05:56:29.787078+0000","last_active":"2024-09-21T05:56:31.177862+0000","last_peered":"2024-09-21T05:56:31.177862+0000","last_clean":"2024-09-21T05:56:31.177862+0000","last_became_active":"2024-09-21T05:56:29.786725+0000","last_became_peered":"2024-09-21T05:56:29.786725+0000","last_unstale":"2024-09-21T05:56:31.177862+0000","last_undegraded":"2024-09-21T05:56:31.177862+0000","last_fullsized":"2024-09-21T05:56:31.177862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:17.854511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782870+0000","last_change":"2024-09-21T05:56:29.790180+0000","last_active":"2024-09-21T05:56:30.782870+0000","last_peered":"2024-09-21T05:56:30.782870+0000","last_clean":"2024-09-21T05:56:30.782870+0000","last_became_active":"2024-09-21T05:56:29.789793+0000","last_became_peered":"2024-09-21T05:56:29.789793+0000","last_unstale":"2024-09-21T05:56:30.782870+0000","last_undegraded":"2024-09-21T05:56:30.782870+0000","last_fullsized":"2024-09-21T05:56:30.782870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:48.361541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176727+0000","last_change":"2024-09-21T05:56:29.791764+0000","last_active":"2024-09-21T05:56:31.176727+0000","last_peered":"2024-09-21T05:56:31.176727+0000","last_clean":"2024-09-21T05:56:31.176727+0000","last_became_active":"2024-09-21T05:56:29.791358+0000","last_became_peered":"2024-09-21T05:56:29.791358+0000","last_unstale":"2024-09-21T05:56:31.176727+0000","last_undegraded":"2024-09-21T05:56:31.176727+0000","last_fullsized":"2024-09-21T05:56:31.176727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:35.770789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796700+0000","last_change":"2024-09-21T05:56:29.798644+0000","last_active":"2024-09-21T05:56:30.796700+0000","last_peered":"2024-09-21T05:56:30.796700+0000","last_clean":"2024-09-21T05:56:30.796700+0000","last_became_active":"2024-09-21T05:56:29.798271+0000","last_became_peered":"2024-09-21T05:56:29.798271+0000","last_unstale":"2024-09-21T05:56:30.796700+0000","last_undegraded":"2024-09-21T05:56:30.796700+0000","last_fullsized":"2024-09-21T05:56:30.796700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:39.467641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780441+0000","last_change":"2024-09-21T05:56:29.786401+0000","last_active":"2024-09-21T05:56:30.780441+0000","last_peered":"2024-09-21T05:56:30.780441+0000","last_clean":"2024-09-21T05:56:30.780441+0000","last_became_active":"2024-09-21T05:56:29.786004+0000","last_became_peered":"2024-09-21T05:56:29.786004+0000","last_unstale":"2024-09-21T05:56:30.780441+0000","last_undegraded":"2024-09-21T05:56:30.780441+0000","last_fullsized":"2024-09-21T05:56:30.780441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:32.904074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178010+0000","last_change":"2024-09-21T05:56:28.806999+0000","last_active":"2024-09-21T05:56:31.178010+0000","last_peered":"2024-09-21T05:56:31.178010+0000","last_clean":"2024-09-21T05:56:31.178010+0000","last_became_active":"2024-09-21T05:56:28.806609+0000","last_became_peered":"2024-09-21T05:56:28.806609+0000","last_unstale":"2024-09-21T05:56:31.178010+0000","last_undegraded":"2024-09-21T05:56:31.178010+0000","last_fullsized":"2024-09-21T05:56:31.178010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:11.564758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178105+0000","last_change":"2024-09-21T05:56:29.799191+0000","last_active":"2024-09-21T05:56:31.178105+0000","last_peered":"2024-09-21T05:56:31.178105+0000","last_clean":"2024-09-21T05:56:31.178105+0000","last_became_active":"2024-09-21T05:56:29.798839+0000","last_became_peered":"2024-09-21T05:56:29.798839+0000","last_unstale":"2024-09-21T05:56:31.178105+0000","last_undegraded":"2024-09-21T05:56:31.178105+0000","last_fullsized":"2024-09-21T05:56:31.178105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:28.783969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796226+0000","last_change":"2024-09-21T05:56:28.794189+0000","last_active":"2024-09-21T05:56:30.796226+0000","last_peered":"2024-09-21T05:56:30.796226+0000","last_clean":"2024-09-21T05:56:30.796226+0000","last_became_active":"2024-09-21T05:56:28.793684+0000","last_became_peered":"2024-09-21T05:56:28.793684+0000","last_unstale":"2024-09-21T05:56:30.796226+0000","last_undegraded":"2024-09-21T05:56:30.796226+0000","last_fullsized":"2024-09-21T05:56:30.796226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:08.868280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177368+0000","last_change":"2024-09-21T05:56:28.788375+0000","last_active":"2024-09-21T05:56:31.177368+0000","last_peered":"2024-09-21T05:56:31.177368+0000","last_clean":"2024-09-21T05:56:31.177368+0000","last_became_active":"2024-09-21T05:56:28.788044+0000","last_became_peered":"2024-09-21T05:56:28.788044+0000","last_unstale":"2024-09-21T05:56:31.177368+0000","last_undegraded":"2024-09-21T05:56:31.177368+0000","last_fullsized":"2024-09-21T05:56:31.177368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:11.883400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177781+0000","last_change":"2024-09-21T05:56:29.792845+0000","last_active":"2024-09-21T05:56:31.177781+0000","last_peered":"2024-09-21T05:56:31.177781+0000","last_clean":"2024-09-21T05:56:31.177781+0000","last_became_active":"2024-09-21T05:56:29.792452+0000","last_became_peered":"2024-09-21T05:56:29.792452+0000","last_unstale":"2024-09-21T05:56:31.177781+0000","last_undegraded":"2024-09-21T05:56:31.177781+0000","last_fullsized":"2024-09-21T05:56:31.177781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:14.751206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782637+0000","last_change":"2024-09-21T05:56:28.796778+0000","last_active":"2024-09-21T05:56:30.782637+0000","last_peered":"2024-09-21T05:56:30.782637+0000","last_clean":"2024-09-21T05:56:30.782637+0000","last_became_active":"2024-09-21T05:56:28.792745+0000","last_became_peered":"2024-09-21T05:56:28.792745+0000","last_unstale":"2024-09-21T05:56:30.782637+0000","last_undegraded":"2024-09-21T05:56:30.782637+0000","last_fullsized":"2024-09-21T05:56:30.782637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:33.703504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779980+0000","last_change":"2024-09-21T05:56:29.790318+0000","last_active":"2024-09-21T05:56:30.779980+0000","last_peered":"2024-09-21T05:56:30.779980+0000","last_clean":"2024-09-21T05:56:30.779980+0000","last_became_active":"2024-09-21T05:56:29.789628+0000","last_became_peered":"2024-09-21T05:56:29.789628+0000","last_unstale":"2024-09-21T05:56:30.779980+0000","last_undegraded":"2024-09-21T05:56:30.779980+0000","last_fullsized":"2024-09-21T05:56:30.779980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:39.627991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796583+0000","last_change":"2024-09-21T05:56:28.798412+0000","last_active":"2024-09-21T05:56:30.796583+0000","last_peered":"2024-09-21T05:56:30.796583+0000","last_clean":"2024-09-21T05:56:30.796583+0000","last_became_active":"2024-09-21T05:56:28.796923+0000","last_became_peered":"2024-09-21T05:56:28.796923+0000","last_unstale":"2024-09-21T05:56:30.796583+0000","last_undegraded":"2024-09-21T05:56:30.796583+0000","last_fullsized":"2024-09-21T05:56:30.796583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:31.896210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783435+0000","last_change":"2024-09-21T05:56:29.801946+0000","last_active":"2024-09-21T05:56:30.783435+0000","last_peered":"2024-09-21T05:56:30.783435+0000","last_clean":"2024-09-21T05:56:30.783435+0000","last_became_active":"2024-09-21T05:56:29.801320+0000","last_became_peered":"2024-09-21T05:56:29.801320+0000","last_unstale":"2024-09-21T05:56:30.783435+0000","last_undegraded":"2024-09-21T05:56:30.783435+0000","last_fullsized":"2024-09-21T05:56:30.783435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:28.055629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783473+0000","last_change":"2024-09-21T05:56:28.792022+0000","last_active":"2024-09-21T05:56:30.783473+0000","last_peered":"2024-09-21T05:56:30.783473+0000","last_clean":"2024-09-21T05:56:30.783473+0000","last_became_active":"2024-09-21T05:56:28.789954+0000","last_became_peered":"2024-09-21T05:56:28.789954+0000","last_unstale":"2024-09-21T05:56:30.783473+0000","last_undegraded":"2024-09-21T05:56:30.783473+0000","last_fullsized":"2024-09-21T05:56:30.783473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:31.896210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177113+0000","last_change":"2024-09-21T05:56:29.799322+0000","last_active":"2024-09-21T05:56:31.177113+0000","last_peered":"2024-09-21T05:56:31.177113+0000","last_clean":"2024-09-21T05:56:31.177113+0000","last_became_active":"2024-09-21T05:56:29.798684+0000","last_became_peered":"2024-09-21T05:56:29.798684+0000","last_unstale":"2024-09-21T05:56:31.177113+0000","last_undegraded":"2024-09-21T05:56:31.177113+0000","last_fullsized":"2024-09-21T05:56:31.177113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:13.158342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779982+0000","last_change":"2024-09-21T05:56:28.791961+0000","last_active":"2024-09-21T05:56:30.779982+0000","last_peered":"2024-09-21T05:56:30.779982+0000","last_clean":"2024-09-21T05:56:30.779982+0000","last_became_active":"2024-09-21T05:56:28.791654+0000","last_became_peered":"2024-09-21T05:56:28.791654+0000","last_unstale":"2024-09-21T05:56:30.779982+0000","last_undegraded":"2024-09-21T05:56:30.779982+0000","last_fullsized":"2024-09-21T05:56:30.779982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:29.186182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.781638+0000","last_change":"2024-09-21T05:56:29.808824+0000","last_active":"2024-09-21T05:56:30.781638+0000","last_peered":"2024-09-21T05:56:30.781638+0000","last_clean":"2024-09-21T05:56:30.781638+0000","last_became_active":"2024-09-21T05:56:29.808454+0000","last_became_peered":"2024-09-21T05:56:29.808454+0000","last_unstale":"2024-09-21T05:56:30.781638+0000","last_undegraded":"2024-09-21T05:56:30.781638+0000","last_fullsized":"2024-09-21T05:56:30.781638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:34.185949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796804+0000","last_change":"2024-09-21T05:56:28.785344+0000","last_active":"2024-09-21T05:56:30.796804+0000","last_peered":"2024-09-21T05:56:30.796804+0000","last_clean":"2024-09-21T05:56:30.796804+0000","last_became_active":"2024-09-21T05:56:28.784997+0000","last_became_peered":"2024-09-21T05:56:28.784997+0000","last_unstale":"2024-09-21T05:56:30.796804+0000","last_undegraded":"2024-09-21T05:56:30.796804+0000","last_fullsized":"2024-09-21T05:56:30.796804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176070+0000","last_change":"2024-09-21T05:56:29.801655+0000","last_active":"2024-09-21T05:56:31.176070+0000","last_peered":"2024-09-21T05:56:31.176070+0000","last_clean":"2024-09-21T05:56:31.176070+0000","last_became_active":"2024-09-21T05:56:29.801337+0000","last_became_peered":"2024-09-21T05:56:29.801337+0000","last_unstale":"2024-09-21T05:56:31.176070+0000","last_undegraded":"2024-09-21T05:56:31.176070+0000","last_fullsized":"2024-09-21T05:56:31.176070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:42.929175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178176+0000","last_change":"2024-09-21T05:56:28.788833+0000","last_active":"2024-09-21T05:56:31.178176+0000","last_peered":"2024-09-21T05:56:31.178176+0000","last_clean":"2024-09-21T05:56:31.178176+0000","last_became_active":"2024-09-21T05:56:28.788469+0000","last_became_peered":"2024-09-21T05:56:28.788469+0000","last_unstale":"2024-09-21T05:56:31.178176+0000","last_undegraded":"2024-09-21T05:56:31.178176+0000","last_fullsized":"2024-09-21T05:56:31.178176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:44.013815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177487+0000","last_change":"2024-09-21T05:56:29.807840+0000","last_active":"2024-09-21T05:56:31.177487+0000","last_peered":"2024-09-21T05:56:31.177487+0000","last_clean":"2024-09-21T05:56:31.177487+0000","last_became_active":"2024-09-21T05:56:29.807524+0000","last_became_peered":"2024-09-21T05:56:29.807524+0000","last_unstale":"2024-09-21T05:56:31.177487+0000","last_undegraded":"2024-09-21T05:56:31.177487+0000","last_fullsized":"2024-09-21T05:56:31.177487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:34.812658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782834+0000","last_change":"2024-09-21T05:56:28.790494+0000","last_active":"2024-09-21T05:56:30.782834+0000","last_peered":"2024-09-21T05:56:30.782834+0000","last_clean":"2024-09-21T05:56:30.782834+0000","last_became_active":"2024-09-21T05:56:28.790003+0000","last_became_peered":"2024-09-21T05:56:28.790003+0000","last_unstale":"2024-09-21T05:56:30.782834+0000","last_undegraded":"2024-09-21T05:56:30.782834+0000","last_fullsized":"2024-09-21T05:56:30.782834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176130+0000","last_change":"2024-09-21T05:56:29.810515+0000","last_active":"2024-09-21T05:56:31.176130+0000","last_peered":"2024-09-21T05:56:31.176130+0000","last_clean":"2024-09-21T05:56:31.176130+0000","last_became_active":"2024-09-21T05:56:29.810188+0000","last_became_peered":"2024-09-21T05:56:29.810188+0000","last_unstale":"2024-09-21T05:56:31.176130+0000","last_undegraded":"2024-09-21T05:56:31.176130+0000","last_fullsized":"2024-09-21T05:56:31.176130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:48.361541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796183+0000","last_change":"2024-09-21T05:56:28.794032+0000","last_active":"2024-09-21T05:56:30.796183+0000","last_peered":"2024-09-21T05:56:30.796183+0000","last_clean":"2024-09-21T05:56:30.796183+0000","last_became_active":"2024-09-21T05:56:28.793671+0000","last_became_peered":"2024-09-21T05:56:28.793671+0000","last_unstale":"2024-09-21T05:56:30.796183+0000","last_undegraded":"2024-09-21T05:56:30.796183+0000","last_fullsized":"2024-09-21T05:56:30.796183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:44.652224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796406+0000","last_change":"2024-09-21T05:56:29.798550+0000","last_active":"2024-09-21T05:56:30.796406+0000","last_peered":"2024-09-21T05:56:30.796406+0000","last_clean":"2024-09-21T05:56:30.796406+0000","last_became_active":"2024-09-21T05:56:29.797719+0000","last_became_peered":"2024-09-21T05:56:29.797719+0000","last_unstale":"2024-09-21T05:56:30.796406+0000","last_undegraded":"2024-09-21T05:56:30.796406+0000","last_fullsized":"2024-09-21T05:56:30.796406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:17.854511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176419+0000","last_change":"2024-09-21T05:56:28.777283+0000","last_active":"2024-09-21T05:56:31.176419+0000","last_peered":"2024-09-21T05:56:31.176419+0000","last_clean":"2024-09-21T05:56:31.176419+0000","last_became_active":"2024-09-21T05:56:28.776907+0000","last_became_peered":"2024-09-21T05:56:28.776907+0000","last_unstale":"2024-09-21T05:56:31.176419+0000","last_undegraded":"2024-09-21T05:56:31.176419+0000","last_fullsized":"2024-09-21T05:56:31.176419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176488+0000","last_change":"2024-09-21T05:56:29.810738+0000","last_active":"2024-09-21T05:56:31.176488+0000","last_peered":"2024-09-21T05:56:31.176488+0000","last_clean":"2024-09-21T05:56:31.176488+0000","last_became_active":"2024-09-21T05:56:29.810296+0000","last_became_peered":"2024-09-21T05:56:29.810296+0000","last_unstale":"2024-09-21T05:56:31.176488+0000","last_undegraded":"2024-09-21T05:56:31.176488+0000","last_fullsized":"2024-09-21T05:56:31.176488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:38.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.781591+0000","last_change":"2024-09-21T05:56:28.793250+0000","last_active":"2024-09-21T05:56:30.781591+0000","last_peered":"2024-09-21T05:56:30.781591+0000","last_clean":"2024-09-21T05:56:30.781591+0000","last_became_active":"2024-09-21T05:56:28.792913+0000","last_became_peered":"2024-09-21T05:56:28.792913+0000","last_unstale":"2024-09-21T05:56:30.781591+0000","last_undegraded":"2024-09-21T05:56:30.781591+0000","last_fullsized":"2024-09-21T05:56:30.781591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:39.751978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176567+0000","last_change":"2024-09-21T05:56:29.811980+0000","last_active":"2024-09-21T05:56:31.176567+0000","last_peered":"2024-09-21T05:56:31.176567+0000","last_clean":"2024-09-21T05:56:31.176567+0000","last_became_active":"2024-09-21T05:56:29.811133+0000","last_became_peered":"2024-09-21T05:56:29.811133+0000","last_unstale":"2024-09-21T05:56:31.176567+0000","last_undegraded":"2024-09-21T05:56:31.176567+0000","last_fullsized":"2024-09-21T05:56:31.176567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:16.791089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177124+0000","last_change":"2024-09-21T05:56:28.784954+0000","last_active":"2024-09-21T05:56:31.177124+0000","last_peered":"2024-09-21T05:56:31.177124+0000","last_clean":"2024-09-21T05:56:31.177124+0000","last_became_active":"2024-09-21T05:56:28.784610+0000","last_became_peered":"2024-09-21T05:56:28.784610+0000","last_unstale":"2024-09-21T05:56:31.177124+0000","last_undegraded":"2024-09-21T05:56:31.177124+0000","last_fullsized":"2024-09-21T05:56:31.177124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:08.868280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176832+0000","last_change":"2024-09-21T05:56:29.811442+0000","last_active":"2024-09-21T05:56:31.176832+0000","last_peered":"2024-09-21T05:56:31.176832+0000","last_clean":"2024-09-21T05:56:31.176832+0000","last_became_active":"2024-09-21T05:56:29.809712+0000","last_became_peered":"2024-09-21T05:56:29.809712+0000","last_unstale":"2024-09-21T05:56:31.176832+0000","last_undegraded":"2024-09-21T05:56:31.176832+0000","last_fullsized":"2024-09-21T05:56:31.176832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:11.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780028+0000","last_change":"2024-09-21T05:56:28.788606+0000","last_active":"2024-09-21T05:56:30.780028+0000","last_peered":"2024-09-21T05:56:30.780028+0000","last_clean":"2024-09-21T05:56:30.780028+0000","last_became_active":"2024-09-21T05:56:28.787648+0000","last_became_peered":"2024-09-21T05:56:28.787648+0000","last_unstale":"2024-09-21T05:56:30.780028+0000","last_undegraded":"2024-09-21T05:56:30.780028+0000","last_fullsized":"2024-09-21T05:56:30.780028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:37.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783198+0000","last_change":"2024-09-21T05:56:29.789459+0000","last_active":"2024-09-21T05:56:30.783198+0000","last_peered":"2024-09-21T05:56:30.783198+0000","last_clean":"2024-09-21T05:56:30.783198+0000","last_became_active":"2024-09-21T05:56:29.785168+0000","last_became_peered":"2024-09-21T05:56:29.785168+0000","last_unstale":"2024-09-21T05:56:30.783198+0000","last_undegraded":"2024-09-21T05:56:30.783198+0000","last_fullsized":"2024-09-21T05:56:30.783198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:38.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176888+0000","last_change":"2024-09-21T05:56:28.790714+0000","last_active":"2024-09-21T05:56:31.176888+0000","last_peered":"2024-09-21T05:56:31.176888+0000","last_clean":"2024-09-21T05:56:31.176888+0000","last_became_active":"2024-09-21T05:56:28.790280+0000","last_became_peered":"2024-09-21T05:56:28.790280+0000","last_unstale":"2024-09-21T05:56:31.176888+0000","last_undegraded":"2024-09-21T05:56:31.176888+0000","last_fullsized":"2024-09-21T05:56:31.176888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:43.786792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176561+0000","last_change":"2024-09-21T05:56:29.805253+0000","last_active":"2024-09-21T05:56:31.176561+0000","last_peered":"2024-09-21T05:56:31.176561+0000","last_clean":"2024-09-21T05:56:31.176561+0000","last_became_active":"2024-09-21T05:56:29.801598+0000","last_became_peered":"2024-09-21T05:56:29.801598+0000","last_unstale":"2024-09-21T05:56:31.176561+0000","last_undegraded":"2024-09-21T05:56:31.176561+0000","last_fullsized":"2024-09-21T05:56:31.176561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:22.963835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175921+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:56:31.175921+0000","last_peered":"2024-09-21T05:56:31.175921+0000","last_clean":"2024-09-21T05:56:31.175921+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:56:31.175921+0000","last_undegraded":"2024-09-21T05:56:31.175921+0000","last_fullsized":"2024-09-21T05:56:31.175921+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782518+0000","last_change":"2024-09-21T05:56:28.781190+0000","last_active":"2024-09-21T05:56:30.782518+0000","last_peered":"2024-09-21T05:56:30.782518+0000","last_clean":"2024-09-21T05:56:30.782518+0000","last_became_active":"2024-09-21T05:56:28.780808+0000","last_became_peered":"2024-09-21T05:56:28.780808+0000","last_unstale":"2024-09-21T05:56:30.782518+0000","last_undegraded":"2024-09-21T05:56:30.782518+0000","last_fullsized":"2024-09-21T05:56:30.782518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:44.652224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177804+0000","last_change":"2024-09-21T05:56:29.795930+0000","last_active":"2024-09-21T05:56:31.177804+0000","last_peered":"2024-09-21T05:56:31.177804+0000","last_clean":"2024-09-21T05:56:31.177804+0000","last_became_active":"2024-09-21T05:56:29.795552+0000","last_became_peered":"2024-09-21T05:56:29.795552+0000","last_unstale":"2024-09-21T05:56:31.177804+0000","last_undegraded":"2024-09-21T05:56:31.177804+0000","last_fullsized":"2024-09-21T05:56:31.177804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:22.412975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175735+0000","last_change":"2024-09-21T05:56:28.805633+0000","last_active":"2024-09-21T05:56:31.175735+0000","last_peered":"2024-09-21T05:56:31.175735+0000","last_clean":"2024-09-21T05:56:31.175735+0000","last_became_active":"2024-09-21T05:56:28.804239+0000","last_became_peered":"2024-09-21T05:56:28.804239+0000","last_unstale":"2024-09-21T05:56:31.175735+0000","last_undegraded":"2024-09-21T05:56:31.175735+0000","last_fullsized":"2024-09-21T05:56:31.175735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:53.019618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178389+0000","last_change":"2024-09-21T05:56:29.803538+0000","last_active":"2024-09-21T05:56:31.178389+0000","last_peered":"2024-09-21T05:56:31.178389+0000","last_clean":"2024-09-21T05:56:31.178389+0000","last_became_active":"2024-09-21T05:56:29.803153+0000","last_became_peered":"2024-09-21T05:56:29.803153+0000","last_unstale":"2024-09-21T05:56:31.178389+0000","last_undegraded":"2024-09-21T05:56:31.178389+0000","last_fullsized":"2024-09-21T05:56:31.178389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:18.925867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176800+0000","last_change":"2024-09-21T05:56:28.786025+0000","last_active":"2024-09-21T05:56:31.176800+0000","last_peered":"2024-09-21T05:56:31.176800+0000","last_clean":"2024-09-21T05:56:31.176800+0000","last_became_active":"2024-09-21T05:56:28.785690+0000","last_became_peered":"2024-09-21T05:56:28.785690+0000","last_unstale":"2024-09-21T05:56:31.176800+0000","last_undegraded":"2024-09-21T05:56:31.176800+0000","last_fullsized":"2024-09-21T05:56:31.176800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:44.652224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779811+0000","last_change":"2024-09-21T05:56:29.785876+0000","last_active":"2024-09-21T05:56:30.779811+0000","last_peered":"2024-09-21T05:56:30.779811+0000","last_clean":"2024-09-21T05:56:30.779811+0000","last_became_active":"2024-09-21T05:56:29.785564+0000","last_became_peered":"2024-09-21T05:56:29.785564+0000","last_unstale":"2024-09-21T05:56:30.779811+0000","last_undegraded":"2024-09-21T05:56:30.779811+0000","last_fullsized":"2024-09-21T05:56:30.779811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:04.812480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782960+0000","last_change":"2024-09-21T05:56:28.785911+0000","last_active":"2024-09-21T05:56:30.782960+0000","last_peered":"2024-09-21T05:56:30.782960+0000","last_clean":"2024-09-21T05:56:30.782960+0000","last_became_active":"2024-09-21T05:56:28.785413+0000","last_became_peered":"2024-09-21T05:56:28.785413+0000","last_unstale":"2024-09-21T05:56:30.782960+0000","last_undegraded":"2024-09-21T05:56:30.782960+0000","last_fullsized":"2024-09-21T05:56:30.782960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:08.868280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780540+0000","last_change":"2024-09-21T05:56:29.789624+0000","last_active":"2024-09-21T05:56:30.780540+0000","last_peered":"2024-09-21T05:56:30.780540+0000","last_clean":"2024-09-21T05:56:30.780540+0000","last_became_active":"2024-09-21T05:56:29.789305+0000","last_became_peered":"2024-09-21T05:56:29.789305+0000","last_unstale":"2024-09-21T05:56:30.780540+0000","last_undegraded":"2024-09-21T05:56:30.780540+0000","last_fullsized":"2024-09-21T05:56:30.780540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:34.711369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780096+0000","last_change":"2024-09-21T05:56:28.791893+0000","last_active":"2024-09-21T05:56:30.780096+0000","last_peered":"2024-09-21T05:56:30.780096+0000","last_clean":"2024-09-21T05:56:30.780096+0000","last_became_active":"2024-09-21T05:56:28.791580+0000","last_became_peered":"2024-09-21T05:56:28.791580+0000","last_unstale":"2024-09-21T05:56:30.780096+0000","last_undegraded":"2024-09-21T05:56:30.780096+0000","last_fullsized":"2024-09-21T05:56:30.780096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.359445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.784018+0000","last_change":"2024-09-21T05:56:29.792438+0000","last_active":"2024-09-21T05:56:30.784018+0000","last_peered":"2024-09-21T05:56:30.784018+0000","last_clean":"2024-09-21T05:56:30.784018+0000","last_became_active":"2024-09-21T05:56:29.792034+0000","last_became_peered":"2024-09-21T05:56:29.792034+0000","last_unstale":"2024-09-21T05:56:30.784018+0000","last_undegraded":"2024-09-21T05:56:30.784018+0000","last_fullsized":"2024-09-21T05:56:30.784018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:30.461662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176595+0000","last_change":"2024-09-21T05:56:28.805473+0000","last_active":"2024-09-21T05:56:31.176595+0000","last_peered":"2024-09-21T05:56:31.176595+0000","last_clean":"2024-09-21T05:56:31.176595+0000","last_became_active":"2024-09-21T05:56:28.804850+0000","last_became_peered":"2024-09-21T05:56:28.804850+0000","last_unstale":"2024-09-21T05:56:31.176595+0000","last_undegraded":"2024-09-21T05:56:31.176595+0000","last_fullsized":"2024-09-21T05:56:31.176595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:44.256633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780054+0000","last_change":"2024-09-21T05:56:29.784905+0000","last_active":"2024-09-21T05:56:30.780054+0000","last_peered":"2024-09-21T05:56:30.780054+0000","last_clean":"2024-09-21T05:56:30.780054+0000","last_became_active":"2024-09-21T05:56:29.784562+0000","last_became_peered":"2024-09-21T05:56:29.784562+0000","last_unstale":"2024-09-21T05:56:30.780054+0000","last_undegraded":"2024-09-21T05:56:30.780054+0000","last_fullsized":"2024-09-21T05:56:30.780054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:26.482139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.795965+0000","last_change":"2024-09-21T05:56:28.790111+0000","last_active":"2024-09-21T05:56:30.795965+0000","last_peered":"2024-09-21T05:56:30.795965+0000","last_clean":"2024-09-21T05:56:30.795965+0000","last_became_active":"2024-09-21T05:56:28.789760+0000","last_became_peered":"2024-09-21T05:56:28.789760+0000","last_unstale":"2024-09-21T05:56:30.795965+0000","last_undegraded":"2024-09-21T05:56:30.795965+0000","last_fullsized":"2024-09-21T05:56:30.795965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:51.060754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178344+0000","last_change":"2024-09-21T05:56:29.812261+0000","last_active":"2024-09-21T05:56:31.178344+0000","last_peered":"2024-09-21T05:56:31.178344+0000","last_clean":"2024-09-21T05:56:31.178344+0000","last_became_active":"2024-09-21T05:56:29.811916+0000","last_became_peered":"2024-09-21T05:56:29.811916+0000","last_unstale":"2024-09-21T05:56:31.178344+0000","last_undegraded":"2024-09-21T05:56:31.178344+0000","last_fullsized":"2024-09-21T05:56:31.178344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:42.929175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796707+0000","last_change":"2024-09-21T05:56:28.804549+0000","last_active":"2024-09-21T05:56:30.796707+0000","last_peered":"2024-09-21T05:56:30.796707+0000","last_clean":"2024-09-21T05:56:30.796707+0000","last_became_active":"2024-09-21T05:56:28.804182+0000","last_became_peered":"2024-09-21T05:56:28.804182+0000","last_unstale":"2024-09-21T05:56:30.796707+0000","last_undegraded":"2024-09-21T05:56:30.796707+0000","last_fullsized":"2024-09-21T05:56:30.796707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:33.703504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178183+0000","last_change":"2024-09-21T05:56:29.797061+0000","last_active":"2024-09-21T05:56:31.178183+0000","last_peered":"2024-09-21T05:56:31.178183+0000","last_clean":"2024-09-21T05:56:31.178183+0000","last_became_active":"2024-09-21T05:56:29.796608+0000","last_became_peered":"2024-09-21T05:56:29.796608+0000","last_unstale":"2024-09-21T05:56:31.178183+0000","last_undegraded":"2024-09-21T05:56:31.178183+0000","last_fullsized":"2024-09-21T05:56:31.178183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:14.929983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175725+0000","last_change":"2024-09-21T05:56:28.804158+0000","last_active":"2024-09-21T05:56:31.175725+0000","last_peered":"2024-09-21T05:56:31.175725+0000","last_clean":"2024-09-21T05:56:31.175725+0000","last_became_active":"2024-09-21T05:56:28.803813+0000","last_became_peered":"2024-09-21T05:56:28.803813+0000","last_unstale":"2024-09-21T05:56:31.175725+0000","last_undegraded":"2024-09-21T05:56:31.175725+0000","last_fullsized":"2024-09-21T05:56:31.175725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:49.073829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780422+0000","last_change":"2024-09-21T05:56:29.797845+0000","last_active":"2024-09-21T05:56:30.780422+0000","last_peered":"2024-09-21T05:56:30.780422+0000","last_clean":"2024-09-21T05:56:30.780422+0000","last_became_active":"2024-09-21T05:56:29.797380+0000","last_became_peered":"2024-09-21T05:56:29.797380+0000","last_unstale":"2024-09-21T05:56:30.780422+0000","last_undegraded":"2024-09-21T05:56:30.780422+0000","last_fullsized":"2024-09-21T05:56:30.780422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:09.876144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779895+0000","last_change":"2024-09-21T05:56:28.801904+0000","last_active":"2024-09-21T05:56:30.779895+0000","last_peered":"2024-09-21T05:56:30.779895+0000","last_clean":"2024-09-21T05:56:30.779895+0000","last_became_active":"2024-09-21T05:56:28.801540+0000","last_became_peered":"2024-09-21T05:56:28.801540+0000","last_unstale":"2024-09-21T05:56:30.779895+0000","last_undegraded":"2024-09-21T05:56:30.779895+0000","last_fullsized":"2024-09-21T05:56:30.779895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:06.676644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177960+0000","last_change":"2024-09-21T05:56:29.782893+0000","last_active":"2024-09-21T05:56:31.177960+0000","last_peered":"2024-09-21T05:56:31.177960+0000","last_clean":"2024-09-21T05:56:31.177960+0000","last_became_active":"2024-09-21T05:56:29.782496+0000","last_became_peered":"2024-09-21T05:56:29.782496+0000","last_unstale":"2024-09-21T05:56:31.177960+0000","last_undegraded":"2024-09-21T05:56:31.177960+0000","last_fullsized":"2024-09-21T05:56:31.177960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:33.544641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779828+0000","last_change":"2024-09-21T05:56:28.784648+0000","last_active":"2024-09-21T05:56:30.779828+0000","last_peered":"2024-09-21T05:56:30.779828+0000","last_clean":"2024-09-21T05:56:30.779828+0000","last_became_active":"2024-09-21T05:56:28.784333+0000","last_became_peered":"2024-09-21T05:56:28.784333+0000","last_unstale":"2024-09-21T05:56:30.779828+0000","last_undegraded":"2024-09-21T05:56:30.779828+0000","last_fullsized":"2024-09-21T05:56:30.779828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:15.892877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178255+0000","last_change":"2024-09-21T05:56:29.789722+0000","last_active":"2024-09-21T05:56:31.178255+0000","last_peered":"2024-09-21T05:56:31.178255+0000","last_clean":"2024-09-21T05:56:31.178255+0000","last_became_active":"2024-09-21T05:56:29.789333+0000","last_became_peered":"2024-09-21T05:56:29.789333+0000","last_unstale":"2024-09-21T05:56:31.178255+0000","last_undegraded":"2024-09-21T05:56:31.178255+0000","last_fullsized":"2024-09-21T05:56:31.178255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176267+0000","last_change":"2024-09-21T05:56:28.807252+0000","last_active":"2024-09-21T05:56:31.176267+0000","last_peered":"2024-09-21T05:56:31.176267+0000","last_clean":"2024-09-21T05:56:31.176267+0000","last_became_active":"2024-09-21T05:56:28.806895+0000","last_became_peered":"2024-09-21T05:56:28.806895+0000","last_unstale":"2024-09-21T05:56:31.176267+0000","last_undegraded":"2024-09-21T05:56:31.176267+0000","last_fullsized":"2024-09-21T05:56:31.176267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:34.762925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796596+0000","last_change":"2024-09-21T05:56:29.781497+0000","last_active":"2024-09-21T05:56:30.796596+0000","last_peered":"2024-09-21T05:56:30.796596+0000","last_clean":"2024-09-21T05:56:30.796596+0000","last_became_active":"2024-09-21T05:56:29.780460+0000","last_became_peered":"2024-09-21T05:56:29.780460+0000","last_unstale":"2024-09-21T05:56:30.796596+0000","last_undegraded":"2024-09-21T05:56:30.796596+0000","last_fullsized":"2024-09-21T05:56:30.796596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:55.735758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783852+0000","last_change":"2024-09-21T05:56:29.788038+0000","last_active":"2024-09-21T05:56:30.783852+0000","last_peered":"2024-09-21T05:56:30.783852+0000","last_clean":"2024-09-21T05:56:30.783852+0000","last_became_active":"2024-09-21T05:56:29.784063+0000","last_became_peered":"2024-09-21T05:56:29.784063+0000","last_unstale":"2024-09-21T05:56:30.783852+0000","last_undegraded":"2024-09-21T05:56:30.783852+0000","last_fullsized":"2024-09-21T05:56:30.783852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:22.850164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175903+0000","last_change":"2024-09-21T05:56:28.808759+0000","last_active":"2024-09-21T05:56:31.175903+0000","last_peered":"2024-09-21T05:56:31.175903+0000","last_clean":"2024-09-21T05:56:31.175903+0000","last_became_active":"2024-09-21T05:56:28.808371+0000","last_became_peered":"2024-09-21T05:56:28.808371+0000","last_unstale":"2024-09-21T05:56:31.175903+0000","last_undegraded":"2024-09-21T05:56:31.175903+0000","last_fullsized":"2024-09-21T05:56:31.175903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:23.035025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783562+0000","last_change":"2024-09-21T05:56:29.791927+0000","last_active":"2024-09-21T05:56:30.783562+0000","last_peered":"2024-09-21T05:56:30.783562+0000","last_clean":"2024-09-21T05:56:30.783562+0000","last_became_active":"2024-09-21T05:56:29.790091+0000","last_became_peered":"2024-09-21T05:56:29.790091+0000","last_unstale":"2024-09-21T05:56:30.783562+0000","last_undegraded":"2024-09-21T05:56:30.783562+0000","last_fullsized":"2024-09-21T05:56:30.783562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:22.963835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176216+0000","last_change":"2024-09-21T05:56:28.804798+0000","last_active":"2024-09-21T05:56:31.176216+0000","last_peered":"2024-09-21T05:56:31.176216+0000","last_clean":"2024-09-21T05:56:31.176216+0000","last_became_active":"2024-09-21T05:56:28.803848+0000","last_became_peered":"2024-09-21T05:56:28.803848+0000","last_unstale":"2024-09-21T05:56:31.176216+0000","last_undegraded":"2024-09-21T05:56:31.176216+0000","last_fullsized":"2024-09-21T05:56:31.176216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:32.979720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178080+0000","last_change":"2024-09-21T05:56:29.798957+0000","last_active":"2024-09-21T05:56:31.178080+0000","last_peered":"2024-09-21T05:56:31.178080+0000","last_clean":"2024-09-21T05:56:31.178080+0000","last_became_active":"2024-09-21T05:56:29.798223+0000","last_became_peered":"2024-09-21T05:56:29.798223+0000","last_unstale":"2024-09-21T05:56:31.178080+0000","last_undegraded":"2024-09-21T05:56:31.178080+0000","last_fullsized":"2024-09-21T05:56:31.178080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:39.467641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177351+0000","last_change":"2024-09-21T05:56:28.783936+0000","last_active":"2024-09-21T05:56:31.177351+0000","last_peered":"2024-09-21T05:56:31.177351+0000","last_clean":"2024-09-21T05:56:31.177351+0000","last_became_active":"2024-09-21T05:56:28.783541+0000","last_became_peered":"2024-09-21T05:56:28.783541+0000","last_unstale":"2024-09-21T05:56:31.177351+0000","last_undegraded":"2024-09-21T05:56:31.177351+0000","last_fullsized":"2024-09-21T05:56:31.177351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:38.675175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178320+0000","last_change":"2024-09-21T05:56:29.785856+0000","last_active":"2024-09-21T05:56:31.178320+0000","last_peered":"2024-09-21T05:56:31.178320+0000","last_clean":"2024-09-21T05:56:31.178320+0000","last_became_active":"2024-09-21T05:56:29.785475+0000","last_became_peered":"2024-09-21T05:56:29.785475+0000","last_unstale":"2024-09-21T05:56:31.178320+0000","last_undegraded":"2024-09-21T05:56:31.178320+0000","last_fullsized":"2024-09-21T05:56:31.178320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:21.326466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177339+0000","last_change":"2024-09-21T05:56:28.804387+0000","last_active":"2024-09-21T05:56:31.177339+0000","last_peered":"2024-09-21T05:56:31.177339+0000","last_clean":"2024-09-21T05:56:31.177339+0000","last_became_active":"2024-09-21T05:56:28.804116+0000","last_became_peered":"2024-09-21T05:56:28.804116+0000","last_unstale":"2024-09-21T05:56:31.177339+0000","last_undegraded":"2024-09-21T05:56:31.177339+0000","last_fullsized":"2024-09-21T05:56:31.177339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":667234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":76410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":667290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:56:38.677 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:56:38.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.844+0000 7f116dc00640 1 -- 172.21.15.50:0/2032491675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11680b5c70 msgr2=0x7f116815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:38.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.844+0000 7f116dc00640 1 --2- 172.21.15.50:0/2032491675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11680b5c70 0x7f116815bb60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1150009900 tx=0x7f115001c820 comp rx=0 tx=0).stop 2024-09-21T05:56:38.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.844+0000 7f116dc00640 1 -- 172.21.15.50:0/2032491675 shutdown_connections 2024-09-21T05:56:38.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.844+0000 7f116dc00640 1 --2- 172.21.15.50:0/2032491675 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11681527b0 0x7f1168074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:38.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.844+0000 7f116dc00640 1 --2- 172.21.15.50:0/2032491675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11680b5c70 0x7f116815bb60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:38.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.844+0000 7f116dc00640 1 -- 172.21.15.50:0/2032491675 >> 172.21.15.50:0/2032491675 conn(0x7f11680b2040 msgr2=0x7f11680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:38.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.844+0000 7f116dc00640 1 -- 172.21.15.50:0/2032491675 shutdown_connections 2024-09-21T05:56:38.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.844+0000 7f116dc00640 1 -- 172.21.15.50:0/2032491675 wait complete. 2024-09-21T05:56:38.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f116dc00640 1 Processor -- start 2024-09-21T05:56:38.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f116dc00640 1 -- start start 2024-09-21T05:56:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f116dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11680b4de0 0x7f11681551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f116dc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11680b5c70 0x7f1168149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f116dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11681527b0 0x7f11680b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f116dc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1168168c70 con 0x7f11681527b0 2024-09-21T05:56:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f116dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1168168af0 con 0x7f11680b4de0 2024-09-21T05:56:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f116dc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1168168df0 con 0x7f11680b5c70 2024-09-21T05:56:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f1167400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11681527b0 0x7f11680b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:38.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f1167400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11681527b0 0x7f11680b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48292/0 (socket says 172.21.15.50:48292) 2024-09-21T05:56:38.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f1167400640 1 -- 172.21.15.50:0/3083544116 learned_addr learned my addr 172.21.15.50:0/3083544116 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:38.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f1166a00640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11680b4de0 0x7f11681551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:38.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f115fe00640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11680b5c70 0x7f1168149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:38.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f1167400640 1 -- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11680b5c70 msgr2=0x7f1168149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:38.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f1167400640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11680b5c70 0x7f1168149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:38.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f1167400640 1 -- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11680b4de0 msgr2=0x7f11681551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:38.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f1167400640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11680b4de0 0x7f11681551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:38.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f1167400640 1 -- 172.21.15.50:0/3083544116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116814d240 con 0x7f11681527b0 2024-09-21T05:56:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f1167400640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11681527b0 0x7f11680b1130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f116000c730 tx=0x7f116000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.848+0000 7f115ea00640 1 -- 172.21.15.50:0/3083544116 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160003b80 con 0x7f11681527b0 2024-09-21T05:56:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.852+0000 7f115ea00640 1 -- 172.21.15.50:0/3083544116 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1160003d40 con 0x7f11681527b0 2024-09-21T05:56:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.852+0000 7f116dc00640 1 -- 172.21.15.50:0/3083544116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116814d520 con 0x7f11681527b0 2024-09-21T05:56:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.852+0000 7f115ea00640 1 -- 172.21.15.50:0/3083544116 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11600052d0 con 0x7f11681527b0 2024-09-21T05:56:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.852+0000 7f116dc00640 1 -- 172.21.15.50:0/3083544116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f116814dae0 con 0x7f11681527b0 2024-09-21T05:56:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.852+0000 7f116dc00640 1 -- 172.21.15.50:0/3083544116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11681558b0 con 0x7f11681527b0 2024-09-21T05:56:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.852+0000 7f115ea00640 1 -- 172.21.15.50:0/3083544116 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1160005680 con 0x7f11681527b0 2024-09-21T05:56:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.852+0000 7f115ea00640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1144073200 0x7f1144075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:38.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.852+0000 7f115ea00640 1 -- 172.21.15.50:0/3083544116 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f11600940d0 con 0x7f11681527b0 2024-09-21T05:56:38.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.852+0000 7f1166a00640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1144073200 0x7f1144075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:38.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.856+0000 7f1166a00640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1144073200 0x7f1144075710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f11500af000 tx=0x7f115001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:38.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:38.860+0000 7f115ea00640 1 -- 172.21.15.50:0/3083544116 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f116005eb20 con 0x7f11681527b0 2024-09-21T05:56:39.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.036+0000 7f116dc00640 1 -- 172.21.15.50:0/3083544116 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f116806b540 con 0x7f1144073200 2024-09-21T05:56:39.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.040+0000 7f115ea00640 1 -- 172.21.15.50:0/3083544116 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251825 (secure 0 0 0) 0x7f11500a5700 con 0x7f1144073200 2024-09-21T05:56:39.047 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:56:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.044+0000 7f116dc00640 1 -- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1144073200 msgr2=0x7f1144075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.044+0000 7f116dc00640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1144073200 0x7f1144075710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f11500af000 tx=0x7f115001b060 comp rx=0 tx=0).stop 2024-09-21T05:56:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.044+0000 7f116dc00640 1 -- 172.21.15.50:0/3083544116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11681527b0 msgr2=0x7f11680b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.044+0000 7f116dc00640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11681527b0 0x7f11680b1130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f116000c730 tx=0x7f116000cbf0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.044+0000 7f116dc00640 1 -- 172.21.15.50:0/3083544116 shutdown_connections 2024-09-21T05:56:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.044+0000 7f116dc00640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1144073200 0x7f1144075710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.044+0000 7f116dc00640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11681527b0 0x7f11680b1130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.044+0000 7f116dc00640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11680b5c70 0x7f1168149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.044+0000 7f116dc00640 1 --2- 172.21.15.50:0/3083544116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11680b4de0 0x7f11681551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.044+0000 7f116dc00640 1 -- 172.21.15.50:0/3083544116 >> 172.21.15.50:0/3083544116 conn(0x7f11680b2040 msgr2=0x7f11680b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.044+0000 7f116dc00640 1 -- 172.21.15.50:0/3083544116 shutdown_connections 2024-09-21T05:56:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.044+0000 7f116dc00640 1 -- 172.21.15.50:0/3083544116 wait complete. 2024-09-21T05:56:39.068 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-21T05:56:38.885956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235840,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738880,"statfs":{"total":773094113280,"available":772852613120,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.119467"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176632+0000","last_change":"2024-09-21T05:56:29.801294+0000","last_active":"2024-09-21T05:56:31.176632+0000","last_peered":"2024-09-21T05:56:31.176632+0000","last_clean":"2024-09-21T05:56:31.176632+0000","last_became_active":"2024-09-21T05:56:29.800947+0000","last_became_peered":"2024-09-21T05:56:29.800947+0000","last_unstale":"2024-09-21T05:56:31.176632+0000","last_undegraded":"2024-09-21T05:56:31.176632+0000","last_fullsized":"2024-09-21T05:56:31.176632+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:11.951093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780332+0000","last_change":"2024-09-21T05:56:29.802492+0000","last_active":"2024-09-21T05:56:30.780332+0000","last_peered":"2024-09-21T05:56:30.780332+0000","last_clean":"2024-09-21T05:56:30.780332+0000","last_became_active":"2024-09-21T05:56:29.801940+0000","last_became_peered":"2024-09-21T05:56:29.801940+0000","last_unstale":"2024-09-21T05:56:30.780332+0000","last_undegraded":"2024-09-21T05:56:30.780332+0000","last_fullsized":"2024-09-21T05:56:30.780332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:11.434269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178417+0000","last_change":"2024-09-21T05:56:29.804599+0000","last_active":"2024-09-21T05:56:31.178417+0000","last_peered":"2024-09-21T05:56:31.178417+0000","last_clean":"2024-09-21T05:56:31.178417+0000","last_became_active":"2024-09-21T05:56:29.804231+0000","last_became_peered":"2024-09-21T05:56:29.804231+0000","last_unstale":"2024-09-21T05:56:31.178417+0000","last_undegraded":"2024-09-21T05:56:31.178417+0000","last_fullsized":"2024-09-21T05:56:31.178417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782989+0000","last_change":"2024-09-21T05:56:29.790060+0000","last_active":"2024-09-21T05:56:30.782989+0000","last_peered":"2024-09-21T05:56:30.782989+0000","last_clean":"2024-09-21T05:56:30.782989+0000","last_became_active":"2024-09-21T05:56:29.786267+0000","last_became_peered":"2024-09-21T05:56:29.786267+0000","last_unstale":"2024-09-21T05:56:30.782989+0000","last_undegraded":"2024-09-21T05:56:30.782989+0000","last_fullsized":"2024-09-21T05:56:30.782989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:11.867996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176068+0000","last_change":"2024-09-21T05:56:29.809836+0000","last_active":"2024-09-21T05:56:31.176068+0000","last_peered":"2024-09-21T05:56:31.176068+0000","last_clean":"2024-09-21T05:56:31.176068+0000","last_became_active":"2024-09-21T05:56:29.809429+0000","last_became_peered":"2024-09-21T05:56:29.809429+0000","last_unstale":"2024-09-21T05:56:31.176068+0000","last_undegraded":"2024-09-21T05:56:31.176068+0000","last_fullsized":"2024-09-21T05:56:31.176068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:41.759804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782594+0000","last_change":"2024-09-21T05:56:29.794338+0000","last_active":"2024-09-21T05:56:30.782594+0000","last_peered":"2024-09-21T05:56:30.782594+0000","last_clean":"2024-09-21T05:56:30.782594+0000","last_became_active":"2024-09-21T05:56:29.794011+0000","last_became_peered":"2024-09-21T05:56:29.794011+0000","last_unstale":"2024-09-21T05:56:30.782594+0000","last_undegraded":"2024-09-21T05:56:30.782594+0000","last_fullsized":"2024-09-21T05:56:30.782594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:37.695152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176168+0000","last_change":"2024-09-21T05:56:29.806114+0000","last_active":"2024-09-21T05:56:31.176168+0000","last_peered":"2024-09-21T05:56:31.176168+0000","last_clean":"2024-09-21T05:56:31.176168+0000","last_became_active":"2024-09-21T05:56:29.805711+0000","last_became_peered":"2024-09-21T05:56:29.805711+0000","last_unstale":"2024-09-21T05:56:31.176168+0000","last_undegraded":"2024-09-21T05:56:31.176168+0000","last_fullsized":"2024-09-21T05:56:31.176168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:11.867996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177775+0000","last_change":"2024-09-21T05:56:29.800463+0000","last_active":"2024-09-21T05:56:31.177775+0000","last_peered":"2024-09-21T05:56:31.177775+0000","last_clean":"2024-09-21T05:56:31.177775+0000","last_became_active":"2024-09-21T05:56:29.800084+0000","last_became_peered":"2024-09-21T05:56:29.800084+0000","last_unstale":"2024-09-21T05:56:31.177775+0000","last_undegraded":"2024-09-21T05:56:31.177775+0000","last_fullsized":"2024-09-21T05:56:31.177775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:39.467641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176639+0000","last_change":"2024-09-21T05:56:29.809950+0000","last_active":"2024-09-21T05:56:31.176639+0000","last_peered":"2024-09-21T05:56:31.176639+0000","last_clean":"2024-09-21T05:56:31.176639+0000","last_became_active":"2024-09-21T05:56:29.809337+0000","last_became_peered":"2024-09-21T05:56:29.809337+0000","last_unstale":"2024-09-21T05:56:31.176639+0000","last_undegraded":"2024-09-21T05:56:31.176639+0000","last_fullsized":"2024-09-21T05:56:31.176639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:30.461662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796623+0000","last_change":"2024-09-21T05:56:29.781023+0000","last_active":"2024-09-21T05:56:30.796623+0000","last_peered":"2024-09-21T05:56:30.796623+0000","last_clean":"2024-09-21T05:56:30.796623+0000","last_became_active":"2024-09-21T05:56:29.780729+0000","last_became_peered":"2024-09-21T05:56:29.780729+0000","last_unstale":"2024-09-21T05:56:30.796623+0000","last_undegraded":"2024-09-21T05:56:30.796623+0000","last_fullsized":"2024-09-21T05:56:30.796623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:22.412975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796667+0000","last_change":"2024-09-21T05:56:29.779393+0000","last_active":"2024-09-21T05:56:30.796667+0000","last_peered":"2024-09-21T05:56:30.796667+0000","last_clean":"2024-09-21T05:56:30.796667+0000","last_became_active":"2024-09-21T05:56:29.779016+0000","last_became_peered":"2024-09-21T05:56:29.779016+0000","last_unstale":"2024-09-21T05:56:30.796667+0000","last_undegraded":"2024-09-21T05:56:30.796667+0000","last_fullsized":"2024-09-21T05:56:30.796667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:30.792281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.781580+0000","last_change":"2024-09-21T05:56:29.787708+0000","last_active":"2024-09-21T05:56:30.781580+0000","last_peered":"2024-09-21T05:56:30.781580+0000","last_clean":"2024-09-21T05:56:30.781580+0000","last_became_active":"2024-09-21T05:56:29.787370+0000","last_became_peered":"2024-09-21T05:56:29.787370+0000","last_unstale":"2024-09-21T05:56:30.781580+0000","last_undegraded":"2024-09-21T05:56:30.781580+0000","last_fullsized":"2024-09-21T05:56:30.781580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:11.868532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177932+0000","last_change":"2024-09-21T05:56:29.790480+0000","last_active":"2024-09-21T05:56:31.177932+0000","last_peered":"2024-09-21T05:56:31.177932+0000","last_clean":"2024-09-21T05:56:31.177932+0000","last_became_active":"2024-09-21T05:56:29.790172+0000","last_became_peered":"2024-09-21T05:56:29.790172+0000","last_unstale":"2024-09-21T05:56:31.177932+0000","last_undegraded":"2024-09-21T05:56:31.177932+0000","last_fullsized":"2024-09-21T05:56:31.177932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:14.929983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178355+0000","last_change":"2024-09-21T05:56:29.796096+0000","last_active":"2024-09-21T05:56:31.178355+0000","last_peered":"2024-09-21T05:56:31.178355+0000","last_clean":"2024-09-21T05:56:31.178355+0000","last_became_active":"2024-09-21T05:56:29.795760+0000","last_became_peered":"2024-09-21T05:56:29.795760+0000","last_unstale":"2024-09-21T05:56:31.178355+0000","last_undegraded":"2024-09-21T05:56:31.178355+0000","last_fullsized":"2024-09-21T05:56:31.178355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:33.544641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779759+0000","last_change":"2024-09-21T05:56:29.789973+0000","last_active":"2024-09-21T05:56:30.779759+0000","last_peered":"2024-09-21T05:56:30.779759+0000","last_clean":"2024-09-21T05:56:30.779759+0000","last_became_active":"2024-09-21T05:56:29.789200+0000","last_became_peered":"2024-09-21T05:56:29.789200+0000","last_unstale":"2024-09-21T05:56:30.779759+0000","last_undegraded":"2024-09-21T05:56:30.779759+0000","last_fullsized":"2024-09-21T05:56:30.779759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:52.718337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782539+0000","last_change":"2024-09-21T05:56:29.795924+0000","last_active":"2024-09-21T05:56:30.782539+0000","last_peered":"2024-09-21T05:56:30.782539+0000","last_clean":"2024-09-21T05:56:30.782539+0000","last_became_active":"2024-09-21T05:56:29.795518+0000","last_became_peered":"2024-09-21T05:56:29.795518+0000","last_unstale":"2024-09-21T05:56:30.782539+0000","last_undegraded":"2024-09-21T05:56:30.782539+0000","last_fullsized":"2024-09-21T05:56:30.782539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:56.052565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.784178+0000","last_change":"2024-09-21T05:56:29.803208+0000","last_active":"2024-09-21T05:56:30.784178+0000","last_peered":"2024-09-21T05:56:30.784178+0000","last_clean":"2024-09-21T05:56:30.784178+0000","last_became_active":"2024-09-21T05:56:29.802492+0000","last_became_peered":"2024-09-21T05:56:29.802492+0000","last_unstale":"2024-09-21T05:56:30.784178+0000","last_undegraded":"2024-09-21T05:56:30.784178+0000","last_fullsized":"2024-09-21T05:56:30.784178+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:11.140043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177406+0000","last_change":"2024-09-21T05:56:29.787284+0000","last_active":"2024-09-21T05:56:31.177406+0000","last_peered":"2024-09-21T05:56:31.177406+0000","last_clean":"2024-09-21T05:56:31.177406+0000","last_became_active":"2024-09-21T05:56:29.786932+0000","last_became_peered":"2024-09-21T05:56:29.786932+0000","last_unstale":"2024-09-21T05:56:31.177406+0000","last_undegraded":"2024-09-21T05:56:31.177406+0000","last_fullsized":"2024-09-21T05:56:31.177406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:45.264497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176533+0000","last_change":"2024-09-21T05:56:29.810657+0000","last_active":"2024-09-21T05:56:31.176533+0000","last_peered":"2024-09-21T05:56:31.176533+0000","last_clean":"2024-09-21T05:56:31.176533+0000","last_became_active":"2024-09-21T05:56:29.810300+0000","last_became_peered":"2024-09-21T05:56:29.810300+0000","last_unstale":"2024-09-21T05:56:31.176533+0000","last_undegraded":"2024-09-21T05:56:31.176533+0000","last_fullsized":"2024-09-21T05:56:31.176533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:28.055629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796891+0000","last_change":"2024-09-21T05:56:29.788766+0000","last_active":"2024-09-21T05:56:30.796891+0000","last_peered":"2024-09-21T05:56:30.796891+0000","last_clean":"2024-09-21T05:56:30.796891+0000","last_became_active":"2024-09-21T05:56:29.788430+0000","last_became_peered":"2024-09-21T05:56:29.788430+0000","last_unstale":"2024-09-21T05:56:30.796891+0000","last_undegraded":"2024-09-21T05:56:30.796891+0000","last_fullsized":"2024-09-21T05:56:30.796891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178114+0000","last_change":"2024-09-21T05:56:29.813309+0000","last_active":"2024-09-21T05:56:31.178114+0000","last_peered":"2024-09-21T05:56:31.178114+0000","last_clean":"2024-09-21T05:56:31.178114+0000","last_became_active":"2024-09-21T05:56:29.812998+0000","last_became_peered":"2024-09-21T05:56:29.812998+0000","last_unstale":"2024-09-21T05:56:31.178114+0000","last_undegraded":"2024-09-21T05:56:31.178114+0000","last_fullsized":"2024-09-21T05:56:31.178114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:30.792281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782744+0000","last_change":"2024-09-21T05:56:29.803935+0000","last_active":"2024-09-21T05:56:30.782744+0000","last_peered":"2024-09-21T05:56:30.782744+0000","last_clean":"2024-09-21T05:56:30.782744+0000","last_became_active":"2024-09-21T05:56:29.803639+0000","last_became_peered":"2024-09-21T05:56:29.803639+0000","last_unstale":"2024-09-21T05:56:30.782744+0000","last_undegraded":"2024-09-21T05:56:30.782744+0000","last_fullsized":"2024-09-21T05:56:30.782744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:41.759804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177783+0000","last_change":"2024-09-21T05:56:29.800369+0000","last_active":"2024-09-21T05:56:31.177783+0000","last_peered":"2024-09-21T05:56:31.177783+0000","last_clean":"2024-09-21T05:56:31.177783+0000","last_became_active":"2024-09-21T05:56:29.799794+0000","last_became_peered":"2024-09-21T05:56:29.799794+0000","last_unstale":"2024-09-21T05:56:31.177783+0000","last_undegraded":"2024-09-21T05:56:31.177783+0000","last_fullsized":"2024-09-21T05:56:31.177783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:17.854511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782813+0000","last_change":"2024-09-21T05:56:29.795633+0000","last_active":"2024-09-21T05:56:30.782813+0000","last_peered":"2024-09-21T05:56:30.782813+0000","last_clean":"2024-09-21T05:56:30.782813+0000","last_became_active":"2024-09-21T05:56:29.795285+0000","last_became_peered":"2024-09-21T05:56:29.795285+0000","last_unstale":"2024-09-21T05:56:30.782813+0000","last_undegraded":"2024-09-21T05:56:30.782813+0000","last_fullsized":"2024-09-21T05:56:30.782813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:04.441455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178478+0000","last_change":"2024-09-21T05:56:29.803384+0000","last_active":"2024-09-21T05:56:31.178478+0000","last_peered":"2024-09-21T05:56:31.178478+0000","last_clean":"2024-09-21T05:56:31.178478+0000","last_became_active":"2024-09-21T05:56:29.803152+0000","last_became_peered":"2024-09-21T05:56:29.803152+0000","last_unstale":"2024-09-21T05:56:31.178478+0000","last_undegraded":"2024-09-21T05:56:31.178478+0000","last_fullsized":"2024-09-21T05:56:31.178478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:48.777031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796724+0000","last_change":"2024-09-21T05:56:29.807556+0000","last_active":"2024-09-21T05:56:30.796724+0000","last_peered":"2024-09-21T05:56:30.796724+0000","last_clean":"2024-09-21T05:56:30.796724+0000","last_became_active":"2024-09-21T05:56:29.805826+0000","last_became_peered":"2024-09-21T05:56:29.805826+0000","last_unstale":"2024-09-21T05:56:30.796724+0000","last_undegraded":"2024-09-21T05:56:30.796724+0000","last_fullsized":"2024-09-21T05:56:30.796724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:33.544641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779890+0000","last_change":"2024-09-21T05:56:29.788425+0000","last_active":"2024-09-21T05:56:30.779890+0000","last_peered":"2024-09-21T05:56:30.779890+0000","last_clean":"2024-09-21T05:56:30.779890+0000","last_became_active":"2024-09-21T05:56:29.788007+0000","last_became_peered":"2024-09-21T05:56:29.788007+0000","last_unstale":"2024-09-21T05:56:30.779890+0000","last_undegraded":"2024-09-21T05:56:30.779890+0000","last_fullsized":"2024-09-21T05:56:30.779890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:53.445482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.179201+0000","last_change":"2024-09-21T05:56:29.792314+0000","last_active":"2024-09-21T05:56:31.179201+0000","last_peered":"2024-09-21T05:56:31.179201+0000","last_clean":"2024-09-21T05:56:31.179201+0000","last_became_active":"2024-09-21T05:56:29.791869+0000","last_became_peered":"2024-09-21T05:56:29.791869+0000","last_unstale":"2024-09-21T05:56:31.179201+0000","last_undegraded":"2024-09-21T05:56:31.179201+0000","last_fullsized":"2024-09-21T05:56:31.179201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:20.952370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177862+0000","last_change":"2024-09-21T05:56:29.787078+0000","last_active":"2024-09-21T05:56:31.177862+0000","last_peered":"2024-09-21T05:56:31.177862+0000","last_clean":"2024-09-21T05:56:31.177862+0000","last_became_active":"2024-09-21T05:56:29.786725+0000","last_became_peered":"2024-09-21T05:56:29.786725+0000","last_unstale":"2024-09-21T05:56:31.177862+0000","last_undegraded":"2024-09-21T05:56:31.177862+0000","last_fullsized":"2024-09-21T05:56:31.177862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:17.854511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782870+0000","last_change":"2024-09-21T05:56:29.790180+0000","last_active":"2024-09-21T05:56:30.782870+0000","last_peered":"2024-09-21T05:56:30.782870+0000","last_clean":"2024-09-21T05:56:30.782870+0000","last_became_active":"2024-09-21T05:56:29.789793+0000","last_became_peered":"2024-09-21T05:56:29.789793+0000","last_unstale":"2024-09-21T05:56:30.782870+0000","last_undegraded":"2024-09-21T05:56:30.782870+0000","last_fullsized":"2024-09-21T05:56:30.782870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:48.361541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176727+0000","last_change":"2024-09-21T05:56:29.791764+0000","last_active":"2024-09-21T05:56:31.176727+0000","last_peered":"2024-09-21T05:56:31.176727+0000","last_clean":"2024-09-21T05:56:31.176727+0000","last_became_active":"2024-09-21T05:56:29.791358+0000","last_became_peered":"2024-09-21T05:56:29.791358+0000","last_unstale":"2024-09-21T05:56:31.176727+0000","last_undegraded":"2024-09-21T05:56:31.176727+0000","last_fullsized":"2024-09-21T05:56:31.176727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:35.770789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796700+0000","last_change":"2024-09-21T05:56:29.798644+0000","last_active":"2024-09-21T05:56:30.796700+0000","last_peered":"2024-09-21T05:56:30.796700+0000","last_clean":"2024-09-21T05:56:30.796700+0000","last_became_active":"2024-09-21T05:56:29.798271+0000","last_became_peered":"2024-09-21T05:56:29.798271+0000","last_unstale":"2024-09-21T05:56:30.796700+0000","last_undegraded":"2024-09-21T05:56:30.796700+0000","last_fullsized":"2024-09-21T05:56:30.796700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:39.467641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780441+0000","last_change":"2024-09-21T05:56:29.786401+0000","last_active":"2024-09-21T05:56:30.780441+0000","last_peered":"2024-09-21T05:56:30.780441+0000","last_clean":"2024-09-21T05:56:30.780441+0000","last_became_active":"2024-09-21T05:56:29.786004+0000","last_became_peered":"2024-09-21T05:56:29.786004+0000","last_unstale":"2024-09-21T05:56:30.780441+0000","last_undegraded":"2024-09-21T05:56:30.780441+0000","last_fullsized":"2024-09-21T05:56:30.780441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:32.904074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178010+0000","last_change":"2024-09-21T05:56:28.806999+0000","last_active":"2024-09-21T05:56:31.178010+0000","last_peered":"2024-09-21T05:56:31.178010+0000","last_clean":"2024-09-21T05:56:31.178010+0000","last_became_active":"2024-09-21T05:56:28.806609+0000","last_became_peered":"2024-09-21T05:56:28.806609+0000","last_unstale":"2024-09-21T05:56:31.178010+0000","last_undegraded":"2024-09-21T05:56:31.178010+0000","last_fullsized":"2024-09-21T05:56:31.178010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:11.564758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178105+0000","last_change":"2024-09-21T05:56:29.799191+0000","last_active":"2024-09-21T05:56:31.178105+0000","last_peered":"2024-09-21T05:56:31.178105+0000","last_clean":"2024-09-21T05:56:31.178105+0000","last_became_active":"2024-09-21T05:56:29.798839+0000","last_became_peered":"2024-09-21T05:56:29.798839+0000","last_unstale":"2024-09-21T05:56:31.178105+0000","last_undegraded":"2024-09-21T05:56:31.178105+0000","last_fullsized":"2024-09-21T05:56:31.178105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:28.783969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796226+0000","last_change":"2024-09-21T05:56:28.794189+0000","last_active":"2024-09-21T05:56:30.796226+0000","last_peered":"2024-09-21T05:56:30.796226+0000","last_clean":"2024-09-21T05:56:30.796226+0000","last_became_active":"2024-09-21T05:56:28.793684+0000","last_became_peered":"2024-09-21T05:56:28.793684+0000","last_unstale":"2024-09-21T05:56:30.796226+0000","last_undegraded":"2024-09-21T05:56:30.796226+0000","last_fullsized":"2024-09-21T05:56:30.796226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:08.868280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177368+0000","last_change":"2024-09-21T05:56:28.788375+0000","last_active":"2024-09-21T05:56:31.177368+0000","last_peered":"2024-09-21T05:56:31.177368+0000","last_clean":"2024-09-21T05:56:31.177368+0000","last_became_active":"2024-09-21T05:56:28.788044+0000","last_became_peered":"2024-09-21T05:56:28.788044+0000","last_unstale":"2024-09-21T05:56:31.177368+0000","last_undegraded":"2024-09-21T05:56:31.177368+0000","last_fullsized":"2024-09-21T05:56:31.177368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:11.883400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177781+0000","last_change":"2024-09-21T05:56:29.792845+0000","last_active":"2024-09-21T05:56:31.177781+0000","last_peered":"2024-09-21T05:56:31.177781+0000","last_clean":"2024-09-21T05:56:31.177781+0000","last_became_active":"2024-09-21T05:56:29.792452+0000","last_became_peered":"2024-09-21T05:56:29.792452+0000","last_unstale":"2024-09-21T05:56:31.177781+0000","last_undegraded":"2024-09-21T05:56:31.177781+0000","last_fullsized":"2024-09-21T05:56:31.177781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:14.751206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782637+0000","last_change":"2024-09-21T05:56:28.796778+0000","last_active":"2024-09-21T05:56:30.782637+0000","last_peered":"2024-09-21T05:56:30.782637+0000","last_clean":"2024-09-21T05:56:30.782637+0000","last_became_active":"2024-09-21T05:56:28.792745+0000","last_became_peered":"2024-09-21T05:56:28.792745+0000","last_unstale":"2024-09-21T05:56:30.782637+0000","last_undegraded":"2024-09-21T05:56:30.782637+0000","last_fullsized":"2024-09-21T05:56:30.782637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:33.703504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779980+0000","last_change":"2024-09-21T05:56:29.790318+0000","last_active":"2024-09-21T05:56:30.779980+0000","last_peered":"2024-09-21T05:56:30.779980+0000","last_clean":"2024-09-21T05:56:30.779980+0000","last_became_active":"2024-09-21T05:56:29.789628+0000","last_became_peered":"2024-09-21T05:56:29.789628+0000","last_unstale":"2024-09-21T05:56:30.779980+0000","last_undegraded":"2024-09-21T05:56:30.779980+0000","last_fullsized":"2024-09-21T05:56:30.779980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:39.627991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796583+0000","last_change":"2024-09-21T05:56:28.798412+0000","last_active":"2024-09-21T05:56:30.796583+0000","last_peered":"2024-09-21T05:56:30.796583+0000","last_clean":"2024-09-21T05:56:30.796583+0000","last_became_active":"2024-09-21T05:56:28.796923+0000","last_became_peered":"2024-09-21T05:56:28.796923+0000","last_unstale":"2024-09-21T05:56:30.796583+0000","last_undegraded":"2024-09-21T05:56:30.796583+0000","last_fullsized":"2024-09-21T05:56:30.796583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:31.896210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783435+0000","last_change":"2024-09-21T05:56:29.801946+0000","last_active":"2024-09-21T05:56:30.783435+0000","last_peered":"2024-09-21T05:56:30.783435+0000","last_clean":"2024-09-21T05:56:30.783435+0000","last_became_active":"2024-09-21T05:56:29.801320+0000","last_became_peered":"2024-09-21T05:56:29.801320+0000","last_unstale":"2024-09-21T05:56:30.783435+0000","last_undegraded":"2024-09-21T05:56:30.783435+0000","last_fullsized":"2024-09-21T05:56:30.783435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:28.055629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783473+0000","last_change":"2024-09-21T05:56:28.792022+0000","last_active":"2024-09-21T05:56:30.783473+0000","last_peered":"2024-09-21T05:56:30.783473+0000","last_clean":"2024-09-21T05:56:30.783473+0000","last_became_active":"2024-09-21T05:56:28.789954+0000","last_became_peered":"2024-09-21T05:56:28.789954+0000","last_unstale":"2024-09-21T05:56:30.783473+0000","last_undegraded":"2024-09-21T05:56:30.783473+0000","last_fullsized":"2024-09-21T05:56:30.783473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:31.896210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177113+0000","last_change":"2024-09-21T05:56:29.799322+0000","last_active":"2024-09-21T05:56:31.177113+0000","last_peered":"2024-09-21T05:56:31.177113+0000","last_clean":"2024-09-21T05:56:31.177113+0000","last_became_active":"2024-09-21T05:56:29.798684+0000","last_became_peered":"2024-09-21T05:56:29.798684+0000","last_unstale":"2024-09-21T05:56:31.177113+0000","last_undegraded":"2024-09-21T05:56:31.177113+0000","last_fullsized":"2024-09-21T05:56:31.177113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:13.158342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779982+0000","last_change":"2024-09-21T05:56:28.791961+0000","last_active":"2024-09-21T05:56:30.779982+0000","last_peered":"2024-09-21T05:56:30.779982+0000","last_clean":"2024-09-21T05:56:30.779982+0000","last_became_active":"2024-09-21T05:56:28.791654+0000","last_became_peered":"2024-09-21T05:56:28.791654+0000","last_unstale":"2024-09-21T05:56:30.779982+0000","last_undegraded":"2024-09-21T05:56:30.779982+0000","last_fullsized":"2024-09-21T05:56:30.779982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:29.186182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.781638+0000","last_change":"2024-09-21T05:56:29.808824+0000","last_active":"2024-09-21T05:56:30.781638+0000","last_peered":"2024-09-21T05:56:30.781638+0000","last_clean":"2024-09-21T05:56:30.781638+0000","last_became_active":"2024-09-21T05:56:29.808454+0000","last_became_peered":"2024-09-21T05:56:29.808454+0000","last_unstale":"2024-09-21T05:56:30.781638+0000","last_undegraded":"2024-09-21T05:56:30.781638+0000","last_fullsized":"2024-09-21T05:56:30.781638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:34.185949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796804+0000","last_change":"2024-09-21T05:56:28.785344+0000","last_active":"2024-09-21T05:56:30.796804+0000","last_peered":"2024-09-21T05:56:30.796804+0000","last_clean":"2024-09-21T05:56:30.796804+0000","last_became_active":"2024-09-21T05:56:28.784997+0000","last_became_peered":"2024-09-21T05:56:28.784997+0000","last_unstale":"2024-09-21T05:56:30.796804+0000","last_undegraded":"2024-09-21T05:56:30.796804+0000","last_fullsized":"2024-09-21T05:56:30.796804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176070+0000","last_change":"2024-09-21T05:56:29.801655+0000","last_active":"2024-09-21T05:56:31.176070+0000","last_peered":"2024-09-21T05:56:31.176070+0000","last_clean":"2024-09-21T05:56:31.176070+0000","last_became_active":"2024-09-21T05:56:29.801337+0000","last_became_peered":"2024-09-21T05:56:29.801337+0000","last_unstale":"2024-09-21T05:56:31.176070+0000","last_undegraded":"2024-09-21T05:56:31.176070+0000","last_fullsized":"2024-09-21T05:56:31.176070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:42.929175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178176+0000","last_change":"2024-09-21T05:56:28.788833+0000","last_active":"2024-09-21T05:56:31.178176+0000","last_peered":"2024-09-21T05:56:31.178176+0000","last_clean":"2024-09-21T05:56:31.178176+0000","last_became_active":"2024-09-21T05:56:28.788469+0000","last_became_peered":"2024-09-21T05:56:28.788469+0000","last_unstale":"2024-09-21T05:56:31.178176+0000","last_undegraded":"2024-09-21T05:56:31.178176+0000","last_fullsized":"2024-09-21T05:56:31.178176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:44.013815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177487+0000","last_change":"2024-09-21T05:56:29.807840+0000","last_active":"2024-09-21T05:56:31.177487+0000","last_peered":"2024-09-21T05:56:31.177487+0000","last_clean":"2024-09-21T05:56:31.177487+0000","last_became_active":"2024-09-21T05:56:29.807524+0000","last_became_peered":"2024-09-21T05:56:29.807524+0000","last_unstale":"2024-09-21T05:56:31.177487+0000","last_undegraded":"2024-09-21T05:56:31.177487+0000","last_fullsized":"2024-09-21T05:56:31.177487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:34.812658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782834+0000","last_change":"2024-09-21T05:56:28.790494+0000","last_active":"2024-09-21T05:56:30.782834+0000","last_peered":"2024-09-21T05:56:30.782834+0000","last_clean":"2024-09-21T05:56:30.782834+0000","last_became_active":"2024-09-21T05:56:28.790003+0000","last_became_peered":"2024-09-21T05:56:28.790003+0000","last_unstale":"2024-09-21T05:56:30.782834+0000","last_undegraded":"2024-09-21T05:56:30.782834+0000","last_fullsized":"2024-09-21T05:56:30.782834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176130+0000","last_change":"2024-09-21T05:56:29.810515+0000","last_active":"2024-09-21T05:56:31.176130+0000","last_peered":"2024-09-21T05:56:31.176130+0000","last_clean":"2024-09-21T05:56:31.176130+0000","last_became_active":"2024-09-21T05:56:29.810188+0000","last_became_peered":"2024-09-21T05:56:29.810188+0000","last_unstale":"2024-09-21T05:56:31.176130+0000","last_undegraded":"2024-09-21T05:56:31.176130+0000","last_fullsized":"2024-09-21T05:56:31.176130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:48.361541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796183+0000","last_change":"2024-09-21T05:56:28.794032+0000","last_active":"2024-09-21T05:56:30.796183+0000","last_peered":"2024-09-21T05:56:30.796183+0000","last_clean":"2024-09-21T05:56:30.796183+0000","last_became_active":"2024-09-21T05:56:28.793671+0000","last_became_peered":"2024-09-21T05:56:28.793671+0000","last_unstale":"2024-09-21T05:56:30.796183+0000","last_undegraded":"2024-09-21T05:56:30.796183+0000","last_fullsized":"2024-09-21T05:56:30.796183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:44.652224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796406+0000","last_change":"2024-09-21T05:56:29.798550+0000","last_active":"2024-09-21T05:56:30.796406+0000","last_peered":"2024-09-21T05:56:30.796406+0000","last_clean":"2024-09-21T05:56:30.796406+0000","last_became_active":"2024-09-21T05:56:29.797719+0000","last_became_peered":"2024-09-21T05:56:29.797719+0000","last_unstale":"2024-09-21T05:56:30.796406+0000","last_undegraded":"2024-09-21T05:56:30.796406+0000","last_fullsized":"2024-09-21T05:56:30.796406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:17.854511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176419+0000","last_change":"2024-09-21T05:56:28.777283+0000","last_active":"2024-09-21T05:56:31.176419+0000","last_peered":"2024-09-21T05:56:31.176419+0000","last_clean":"2024-09-21T05:56:31.176419+0000","last_became_active":"2024-09-21T05:56:28.776907+0000","last_became_peered":"2024-09-21T05:56:28.776907+0000","last_unstale":"2024-09-21T05:56:31.176419+0000","last_undegraded":"2024-09-21T05:56:31.176419+0000","last_fullsized":"2024-09-21T05:56:31.176419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176488+0000","last_change":"2024-09-21T05:56:29.810738+0000","last_active":"2024-09-21T05:56:31.176488+0000","last_peered":"2024-09-21T05:56:31.176488+0000","last_clean":"2024-09-21T05:56:31.176488+0000","last_became_active":"2024-09-21T05:56:29.810296+0000","last_became_peered":"2024-09-21T05:56:29.810296+0000","last_unstale":"2024-09-21T05:56:31.176488+0000","last_undegraded":"2024-09-21T05:56:31.176488+0000","last_fullsized":"2024-09-21T05:56:31.176488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:38.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.781591+0000","last_change":"2024-09-21T05:56:28.793250+0000","last_active":"2024-09-21T05:56:30.781591+0000","last_peered":"2024-09-21T05:56:30.781591+0000","last_clean":"2024-09-21T05:56:30.781591+0000","last_became_active":"2024-09-21T05:56:28.792913+0000","last_became_peered":"2024-09-21T05:56:28.792913+0000","last_unstale":"2024-09-21T05:56:30.781591+0000","last_undegraded":"2024-09-21T05:56:30.781591+0000","last_fullsized":"2024-09-21T05:56:30.781591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:39.751978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176567+0000","last_change":"2024-09-21T05:56:29.811980+0000","last_active":"2024-09-21T05:56:31.176567+0000","last_peered":"2024-09-21T05:56:31.176567+0000","last_clean":"2024-09-21T05:56:31.176567+0000","last_became_active":"2024-09-21T05:56:29.811133+0000","last_became_peered":"2024-09-21T05:56:29.811133+0000","last_unstale":"2024-09-21T05:56:31.176567+0000","last_undegraded":"2024-09-21T05:56:31.176567+0000","last_fullsized":"2024-09-21T05:56:31.176567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:16.791089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177124+0000","last_change":"2024-09-21T05:56:28.784954+0000","last_active":"2024-09-21T05:56:31.177124+0000","last_peered":"2024-09-21T05:56:31.177124+0000","last_clean":"2024-09-21T05:56:31.177124+0000","last_became_active":"2024-09-21T05:56:28.784610+0000","last_became_peered":"2024-09-21T05:56:28.784610+0000","last_unstale":"2024-09-21T05:56:31.177124+0000","last_undegraded":"2024-09-21T05:56:31.177124+0000","last_fullsized":"2024-09-21T05:56:31.177124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:08.868280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176832+0000","last_change":"2024-09-21T05:56:29.811442+0000","last_active":"2024-09-21T05:56:31.176832+0000","last_peered":"2024-09-21T05:56:31.176832+0000","last_clean":"2024-09-21T05:56:31.176832+0000","last_became_active":"2024-09-21T05:56:29.809712+0000","last_became_peered":"2024-09-21T05:56:29.809712+0000","last_unstale":"2024-09-21T05:56:31.176832+0000","last_undegraded":"2024-09-21T05:56:31.176832+0000","last_fullsized":"2024-09-21T05:56:31.176832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:11.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780028+0000","last_change":"2024-09-21T05:56:28.788606+0000","last_active":"2024-09-21T05:56:30.780028+0000","last_peered":"2024-09-21T05:56:30.780028+0000","last_clean":"2024-09-21T05:56:30.780028+0000","last_became_active":"2024-09-21T05:56:28.787648+0000","last_became_peered":"2024-09-21T05:56:28.787648+0000","last_unstale":"2024-09-21T05:56:30.780028+0000","last_undegraded":"2024-09-21T05:56:30.780028+0000","last_fullsized":"2024-09-21T05:56:30.780028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:37.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783198+0000","last_change":"2024-09-21T05:56:29.789459+0000","last_active":"2024-09-21T05:56:30.783198+0000","last_peered":"2024-09-21T05:56:30.783198+0000","last_clean":"2024-09-21T05:56:30.783198+0000","last_became_active":"2024-09-21T05:56:29.785168+0000","last_became_peered":"2024-09-21T05:56:29.785168+0000","last_unstale":"2024-09-21T05:56:30.783198+0000","last_undegraded":"2024-09-21T05:56:30.783198+0000","last_fullsized":"2024-09-21T05:56:30.783198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:38.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176888+0000","last_change":"2024-09-21T05:56:28.790714+0000","last_active":"2024-09-21T05:56:31.176888+0000","last_peered":"2024-09-21T05:56:31.176888+0000","last_clean":"2024-09-21T05:56:31.176888+0000","last_became_active":"2024-09-21T05:56:28.790280+0000","last_became_peered":"2024-09-21T05:56:28.790280+0000","last_unstale":"2024-09-21T05:56:31.176888+0000","last_undegraded":"2024-09-21T05:56:31.176888+0000","last_fullsized":"2024-09-21T05:56:31.176888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:43.786792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176561+0000","last_change":"2024-09-21T05:56:29.805253+0000","last_active":"2024-09-21T05:56:31.176561+0000","last_peered":"2024-09-21T05:56:31.176561+0000","last_clean":"2024-09-21T05:56:31.176561+0000","last_became_active":"2024-09-21T05:56:29.801598+0000","last_became_peered":"2024-09-21T05:56:29.801598+0000","last_unstale":"2024-09-21T05:56:31.176561+0000","last_undegraded":"2024-09-21T05:56:31.176561+0000","last_fullsized":"2024-09-21T05:56:31.176561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:22.963835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175921+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:56:31.175921+0000","last_peered":"2024-09-21T05:56:31.175921+0000","last_clean":"2024-09-21T05:56:31.175921+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:56:31.175921+0000","last_undegraded":"2024-09-21T05:56:31.175921+0000","last_fullsized":"2024-09-21T05:56:31.175921+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782518+0000","last_change":"2024-09-21T05:56:28.781190+0000","last_active":"2024-09-21T05:56:30.782518+0000","last_peered":"2024-09-21T05:56:30.782518+0000","last_clean":"2024-09-21T05:56:30.782518+0000","last_became_active":"2024-09-21T05:56:28.780808+0000","last_became_peered":"2024-09-21T05:56:28.780808+0000","last_unstale":"2024-09-21T05:56:30.782518+0000","last_undegraded":"2024-09-21T05:56:30.782518+0000","last_fullsized":"2024-09-21T05:56:30.782518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:44.652224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177804+0000","last_change":"2024-09-21T05:56:29.795930+0000","last_active":"2024-09-21T05:56:31.177804+0000","last_peered":"2024-09-21T05:56:31.177804+0000","last_clean":"2024-09-21T05:56:31.177804+0000","last_became_active":"2024-09-21T05:56:29.795552+0000","last_became_peered":"2024-09-21T05:56:29.795552+0000","last_unstale":"2024-09-21T05:56:31.177804+0000","last_undegraded":"2024-09-21T05:56:31.177804+0000","last_fullsized":"2024-09-21T05:56:31.177804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:22.412975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175735+0000","last_change":"2024-09-21T05:56:28.805633+0000","last_active":"2024-09-21T05:56:31.175735+0000","last_peered":"2024-09-21T05:56:31.175735+0000","last_clean":"2024-09-21T05:56:31.175735+0000","last_became_active":"2024-09-21T05:56:28.804239+0000","last_became_peered":"2024-09-21T05:56:28.804239+0000","last_unstale":"2024-09-21T05:56:31.175735+0000","last_undegraded":"2024-09-21T05:56:31.175735+0000","last_fullsized":"2024-09-21T05:56:31.175735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:53.019618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178389+0000","last_change":"2024-09-21T05:56:29.803538+0000","last_active":"2024-09-21T05:56:31.178389+0000","last_peered":"2024-09-21T05:56:31.178389+0000","last_clean":"2024-09-21T05:56:31.178389+0000","last_became_active":"2024-09-21T05:56:29.803153+0000","last_became_peered":"2024-09-21T05:56:29.803153+0000","last_unstale":"2024-09-21T05:56:31.178389+0000","last_undegraded":"2024-09-21T05:56:31.178389+0000","last_fullsized":"2024-09-21T05:56:31.178389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:18.925867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176800+0000","last_change":"2024-09-21T05:56:28.786025+0000","last_active":"2024-09-21T05:56:31.176800+0000","last_peered":"2024-09-21T05:56:31.176800+0000","last_clean":"2024-09-21T05:56:31.176800+0000","last_became_active":"2024-09-21T05:56:28.785690+0000","last_became_peered":"2024-09-21T05:56:28.785690+0000","last_unstale":"2024-09-21T05:56:31.176800+0000","last_undegraded":"2024-09-21T05:56:31.176800+0000","last_fullsized":"2024-09-21T05:56:31.176800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:44.652224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779811+0000","last_change":"2024-09-21T05:56:29.785876+0000","last_active":"2024-09-21T05:56:30.779811+0000","last_peered":"2024-09-21T05:56:30.779811+0000","last_clean":"2024-09-21T05:56:30.779811+0000","last_became_active":"2024-09-21T05:56:29.785564+0000","last_became_peered":"2024-09-21T05:56:29.785564+0000","last_unstale":"2024-09-21T05:56:30.779811+0000","last_undegraded":"2024-09-21T05:56:30.779811+0000","last_fullsized":"2024-09-21T05:56:30.779811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:04.812480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782960+0000","last_change":"2024-09-21T05:56:28.785911+0000","last_active":"2024-09-21T05:56:30.782960+0000","last_peered":"2024-09-21T05:56:30.782960+0000","last_clean":"2024-09-21T05:56:30.782960+0000","last_became_active":"2024-09-21T05:56:28.785413+0000","last_became_peered":"2024-09-21T05:56:28.785413+0000","last_unstale":"2024-09-21T05:56:30.782960+0000","last_undegraded":"2024-09-21T05:56:30.782960+0000","last_fullsized":"2024-09-21T05:56:30.782960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:08.868280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780540+0000","last_change":"2024-09-21T05:56:29.789624+0000","last_active":"2024-09-21T05:56:30.780540+0000","last_peered":"2024-09-21T05:56:30.780540+0000","last_clean":"2024-09-21T05:56:30.780540+0000","last_became_active":"2024-09-21T05:56:29.789305+0000","last_became_peered":"2024-09-21T05:56:29.789305+0000","last_unstale":"2024-09-21T05:56:30.780540+0000","last_undegraded":"2024-09-21T05:56:30.780540+0000","last_fullsized":"2024-09-21T05:56:30.780540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:34.711369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780096+0000","last_change":"2024-09-21T05:56:28.791893+0000","last_active":"2024-09-21T05:56:30.780096+0000","last_peered":"2024-09-21T05:56:30.780096+0000","last_clean":"2024-09-21T05:56:30.780096+0000","last_became_active":"2024-09-21T05:56:28.791580+0000","last_became_peered":"2024-09-21T05:56:28.791580+0000","last_unstale":"2024-09-21T05:56:30.780096+0000","last_undegraded":"2024-09-21T05:56:30.780096+0000","last_fullsized":"2024-09-21T05:56:30.780096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.359445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.784018+0000","last_change":"2024-09-21T05:56:29.792438+0000","last_active":"2024-09-21T05:56:30.784018+0000","last_peered":"2024-09-21T05:56:30.784018+0000","last_clean":"2024-09-21T05:56:30.784018+0000","last_became_active":"2024-09-21T05:56:29.792034+0000","last_became_peered":"2024-09-21T05:56:29.792034+0000","last_unstale":"2024-09-21T05:56:30.784018+0000","last_undegraded":"2024-09-21T05:56:30.784018+0000","last_fullsized":"2024-09-21T05:56:30.784018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:30.461662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176595+0000","last_change":"2024-09-21T05:56:28.805473+0000","last_active":"2024-09-21T05:56:31.176595+0000","last_peered":"2024-09-21T05:56:31.176595+0000","last_clean":"2024-09-21T05:56:31.176595+0000","last_became_active":"2024-09-21T05:56:28.804850+0000","last_became_peered":"2024-09-21T05:56:28.804850+0000","last_unstale":"2024-09-21T05:56:31.176595+0000","last_undegraded":"2024-09-21T05:56:31.176595+0000","last_fullsized":"2024-09-21T05:56:31.176595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:44.256633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780054+0000","last_change":"2024-09-21T05:56:29.784905+0000","last_active":"2024-09-21T05:56:30.780054+0000","last_peered":"2024-09-21T05:56:30.780054+0000","last_clean":"2024-09-21T05:56:30.780054+0000","last_became_active":"2024-09-21T05:56:29.784562+0000","last_became_peered":"2024-09-21T05:56:29.784562+0000","last_unstale":"2024-09-21T05:56:30.780054+0000","last_undegraded":"2024-09-21T05:56:30.780054+0000","last_fullsized":"2024-09-21T05:56:30.780054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:26.482139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.795965+0000","last_change":"2024-09-21T05:56:28.790111+0000","last_active":"2024-09-21T05:56:30.795965+0000","last_peered":"2024-09-21T05:56:30.795965+0000","last_clean":"2024-09-21T05:56:30.795965+0000","last_became_active":"2024-09-21T05:56:28.789760+0000","last_became_peered":"2024-09-21T05:56:28.789760+0000","last_unstale":"2024-09-21T05:56:30.795965+0000","last_undegraded":"2024-09-21T05:56:30.795965+0000","last_fullsized":"2024-09-21T05:56:30.795965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:51.060754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178344+0000","last_change":"2024-09-21T05:56:29.812261+0000","last_active":"2024-09-21T05:56:31.178344+0000","last_peered":"2024-09-21T05:56:31.178344+0000","last_clean":"2024-09-21T05:56:31.178344+0000","last_became_active":"2024-09-21T05:56:29.811916+0000","last_became_peered":"2024-09-21T05:56:29.811916+0000","last_unstale":"2024-09-21T05:56:31.178344+0000","last_undegraded":"2024-09-21T05:56:31.178344+0000","last_fullsized":"2024-09-21T05:56:31.178344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:42.929175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796707+0000","last_change":"2024-09-21T05:56:28.804549+0000","last_active":"2024-09-21T05:56:30.796707+0000","last_peered":"2024-09-21T05:56:30.796707+0000","last_clean":"2024-09-21T05:56:30.796707+0000","last_became_active":"2024-09-21T05:56:28.804182+0000","last_became_peered":"2024-09-21T05:56:28.804182+0000","last_unstale":"2024-09-21T05:56:30.796707+0000","last_undegraded":"2024-09-21T05:56:30.796707+0000","last_fullsized":"2024-09-21T05:56:30.796707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:33.703504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178183+0000","last_change":"2024-09-21T05:56:29.797061+0000","last_active":"2024-09-21T05:56:31.178183+0000","last_peered":"2024-09-21T05:56:31.178183+0000","last_clean":"2024-09-21T05:56:31.178183+0000","last_became_active":"2024-09-21T05:56:29.796608+0000","last_became_peered":"2024-09-21T05:56:29.796608+0000","last_unstale":"2024-09-21T05:56:31.178183+0000","last_undegraded":"2024-09-21T05:56:31.178183+0000","last_fullsized":"2024-09-21T05:56:31.178183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:14.929983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175725+0000","last_change":"2024-09-21T05:56:28.804158+0000","last_active":"2024-09-21T05:56:31.175725+0000","last_peered":"2024-09-21T05:56:31.175725+0000","last_clean":"2024-09-21T05:56:31.175725+0000","last_became_active":"2024-09-21T05:56:28.803813+0000","last_became_peered":"2024-09-21T05:56:28.803813+0000","last_unstale":"2024-09-21T05:56:31.175725+0000","last_undegraded":"2024-09-21T05:56:31.175725+0000","last_fullsized":"2024-09-21T05:56:31.175725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:49.073829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780422+0000","last_change":"2024-09-21T05:56:29.797845+0000","last_active":"2024-09-21T05:56:30.780422+0000","last_peered":"2024-09-21T05:56:30.780422+0000","last_clean":"2024-09-21T05:56:30.780422+0000","last_became_active":"2024-09-21T05:56:29.797380+0000","last_became_peered":"2024-09-21T05:56:29.797380+0000","last_unstale":"2024-09-21T05:56:30.780422+0000","last_undegraded":"2024-09-21T05:56:30.780422+0000","last_fullsized":"2024-09-21T05:56:30.780422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:09.876144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779895+0000","last_change":"2024-09-21T05:56:28.801904+0000","last_active":"2024-09-21T05:56:30.779895+0000","last_peered":"2024-09-21T05:56:30.779895+0000","last_clean":"2024-09-21T05:56:30.779895+0000","last_became_active":"2024-09-21T05:56:28.801540+0000","last_became_peered":"2024-09-21T05:56:28.801540+0000","last_unstale":"2024-09-21T05:56:30.779895+0000","last_undegraded":"2024-09-21T05:56:30.779895+0000","last_fullsized":"2024-09-21T05:56:30.779895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:06.676644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177960+0000","last_change":"2024-09-21T05:56:29.782893+0000","last_active":"2024-09-21T05:56:31.177960+0000","last_peered":"2024-09-21T05:56:31.177960+0000","last_clean":"2024-09-21T05:56:31.177960+0000","last_became_active":"2024-09-21T05:56:29.782496+0000","last_became_peered":"2024-09-21T05:56:29.782496+0000","last_unstale":"2024-09-21T05:56:31.177960+0000","last_undegraded":"2024-09-21T05:56:31.177960+0000","last_fullsized":"2024-09-21T05:56:31.177960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:33.544641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779828+0000","last_change":"2024-09-21T05:56:28.784648+0000","last_active":"2024-09-21T05:56:30.779828+0000","last_peered":"2024-09-21T05:56:30.779828+0000","last_clean":"2024-09-21T05:56:30.779828+0000","last_became_active":"2024-09-21T05:56:28.784333+0000","last_became_peered":"2024-09-21T05:56:28.784333+0000","last_unstale":"2024-09-21T05:56:30.779828+0000","last_undegraded":"2024-09-21T05:56:30.779828+0000","last_fullsized":"2024-09-21T05:56:30.779828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:15.892877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178255+0000","last_change":"2024-09-21T05:56:29.789722+0000","last_active":"2024-09-21T05:56:31.178255+0000","last_peered":"2024-09-21T05:56:31.178255+0000","last_clean":"2024-09-21T05:56:31.178255+0000","last_became_active":"2024-09-21T05:56:29.789333+0000","last_became_peered":"2024-09-21T05:56:29.789333+0000","last_unstale":"2024-09-21T05:56:31.178255+0000","last_undegraded":"2024-09-21T05:56:31.178255+0000","last_fullsized":"2024-09-21T05:56:31.178255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176267+0000","last_change":"2024-09-21T05:56:28.807252+0000","last_active":"2024-09-21T05:56:31.176267+0000","last_peered":"2024-09-21T05:56:31.176267+0000","last_clean":"2024-09-21T05:56:31.176267+0000","last_became_active":"2024-09-21T05:56:28.806895+0000","last_became_peered":"2024-09-21T05:56:28.806895+0000","last_unstale":"2024-09-21T05:56:31.176267+0000","last_undegraded":"2024-09-21T05:56:31.176267+0000","last_fullsized":"2024-09-21T05:56:31.176267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:34.762925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796596+0000","last_change":"2024-09-21T05:56:29.781497+0000","last_active":"2024-09-21T05:56:30.796596+0000","last_peered":"2024-09-21T05:56:30.796596+0000","last_clean":"2024-09-21T05:56:30.796596+0000","last_became_active":"2024-09-21T05:56:29.780460+0000","last_became_peered":"2024-09-21T05:56:29.780460+0000","last_unstale":"2024-09-21T05:56:30.796596+0000","last_undegraded":"2024-09-21T05:56:30.796596+0000","last_fullsized":"2024-09-21T05:56:30.796596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:55.735758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783852+0000","last_change":"2024-09-21T05:56:29.788038+0000","last_active":"2024-09-21T05:56:30.783852+0000","last_peered":"2024-09-21T05:56:30.783852+0000","last_clean":"2024-09-21T05:56:30.783852+0000","last_became_active":"2024-09-21T05:56:29.784063+0000","last_became_peered":"2024-09-21T05:56:29.784063+0000","last_unstale":"2024-09-21T05:56:30.783852+0000","last_undegraded":"2024-09-21T05:56:30.783852+0000","last_fullsized":"2024-09-21T05:56:30.783852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:22.850164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175903+0000","last_change":"2024-09-21T05:56:28.808759+0000","last_active":"2024-09-21T05:56:31.175903+0000","last_peered":"2024-09-21T05:56:31.175903+0000","last_clean":"2024-09-21T05:56:31.175903+0000","last_became_active":"2024-09-21T05:56:28.808371+0000","last_became_peered":"2024-09-21T05:56:28.808371+0000","last_unstale":"2024-09-21T05:56:31.175903+0000","last_undegraded":"2024-09-21T05:56:31.175903+0000","last_fullsized":"2024-09-21T05:56:31.175903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:23.035025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783562+0000","last_change":"2024-09-21T05:56:29.791927+0000","last_active":"2024-09-21T05:56:30.783562+0000","last_peered":"2024-09-21T05:56:30.783562+0000","last_clean":"2024-09-21T05:56:30.783562+0000","last_became_active":"2024-09-21T05:56:29.790091+0000","last_became_peered":"2024-09-21T05:56:29.790091+0000","last_unstale":"2024-09-21T05:56:30.783562+0000","last_undegraded":"2024-09-21T05:56:30.783562+0000","last_fullsized":"2024-09-21T05:56:30.783562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:22.963835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176216+0000","last_change":"2024-09-21T05:56:28.804798+0000","last_active":"2024-09-21T05:56:31.176216+0000","last_peered":"2024-09-21T05:56:31.176216+0000","last_clean":"2024-09-21T05:56:31.176216+0000","last_became_active":"2024-09-21T05:56:28.803848+0000","last_became_peered":"2024-09-21T05:56:28.803848+0000","last_unstale":"2024-09-21T05:56:31.176216+0000","last_undegraded":"2024-09-21T05:56:31.176216+0000","last_fullsized":"2024-09-21T05:56:31.176216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:32.979720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178080+0000","last_change":"2024-09-21T05:56:29.798957+0000","last_active":"2024-09-21T05:56:31.178080+0000","last_peered":"2024-09-21T05:56:31.178080+0000","last_clean":"2024-09-21T05:56:31.178080+0000","last_became_active":"2024-09-21T05:56:29.798223+0000","last_became_peered":"2024-09-21T05:56:29.798223+0000","last_unstale":"2024-09-21T05:56:31.178080+0000","last_undegraded":"2024-09-21T05:56:31.178080+0000","last_fullsized":"2024-09-21T05:56:31.178080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:39.467641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177351+0000","last_change":"2024-09-21T05:56:28.783936+0000","last_active":"2024-09-21T05:56:31.177351+0000","last_peered":"2024-09-21T05:56:31.177351+0000","last_clean":"2024-09-21T05:56:31.177351+0000","last_became_active":"2024-09-21T05:56:28.783541+0000","last_became_peered":"2024-09-21T05:56:28.783541+0000","last_unstale":"2024-09-21T05:56:31.177351+0000","last_undegraded":"2024-09-21T05:56:31.177351+0000","last_fullsized":"2024-09-21T05:56:31.177351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:38.675175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178320+0000","last_change":"2024-09-21T05:56:29.785856+0000","last_active":"2024-09-21T05:56:31.178320+0000","last_peered":"2024-09-21T05:56:31.178320+0000","last_clean":"2024-09-21T05:56:31.178320+0000","last_became_active":"2024-09-21T05:56:29.785475+0000","last_became_peered":"2024-09-21T05:56:29.785475+0000","last_unstale":"2024-09-21T05:56:31.178320+0000","last_undegraded":"2024-09-21T05:56:31.178320+0000","last_fullsized":"2024-09-21T05:56:31.178320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:21.326466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177339+0000","last_change":"2024-09-21T05:56:28.804387+0000","last_active":"2024-09-21T05:56:31.177339+0000","last_peered":"2024-09-21T05:56:31.177339+0000","last_clean":"2024-09-21T05:56:31.177339+0000","last_became_active":"2024-09-21T05:56:28.804116+0000","last_became_peered":"2024-09-21T05:56:28.804116+0000","last_unstale":"2024-09-21T05:56:31.177339+0000","last_undegraded":"2024-09-21T05:56:31.177339+0000","last_fullsized":"2024-09-21T05:56:31.177339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":667234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":76410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":667290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:56:39.073 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:56:39.073 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-21T05:56:39.073 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T05:56:39.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.268+0000 7f62e9200640 1 -- 172.21.15.50:0/249433176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e40b5c70 msgr2=0x7f62e415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.268+0000 7f62e9200640 1 --2- 172.21.15.50:0/249433176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e40b5c70 0x7f62e415bb60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62e4070710 tx=0x7f62d401ccc0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.268+0000 7f62e9200640 1 -- 172.21.15.50:0/249433176 shutdown_connections 2024-09-21T05:56:39.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.268+0000 7f62e9200640 1 --2- 172.21.15.50:0/249433176 >> v2:172.21.15.88:3300/0 conn(0x7f62e41527b0 0x7f62e4074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.268+0000 7f62e9200640 1 --2- 172.21.15.50:0/249433176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e40b5c70 0x7f62e415bb60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.268+0000 7f62e9200640 1 -- 172.21.15.50:0/249433176 >> 172.21.15.50:0/249433176 conn(0x7f62e40b2040 msgr2=0x7f62e40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:39.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.268+0000 7f62e9200640 1 -- 172.21.15.50:0/249433176 shutdown_connections 2024-09-21T05:56:39.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.268+0000 7f62e9200640 1 -- 172.21.15.50:0/249433176 wait complete. 2024-09-21T05:56:39.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62e9200640 1 Processor -- start 2024-09-21T05:56:39.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62e9200640 1 -- start start 2024-09-21T05:56:39.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62e9200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62e40b4de0 0x7f62e41f9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:39.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62e9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e40b5c70 0x7f62e414fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:39.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62e9200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62e41527b0 0x7f62e41587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:39.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62e9200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4168a80 con 0x7f62e40b5c70 2024-09-21T05:56:39.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62e9200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4168900 con 0x7f62e41527b0 2024-09-21T05:56:39.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62e9200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f62e4168c00 con 0x7f62e40b4de0 2024-09-21T05:56:39.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62d9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e40b5c70 0x7f62e414fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:39.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62d9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e40b5c70 0x7f62e414fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48302/0 (socket says 172.21.15.50:48302) 2024-09-21T05:56:39.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62d9400640 1 -- 172.21.15.50:0/3354380434 learned_addr learned my addr 172.21.15.50:0/3354380434 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:39.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62e2800640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62e41527b0 0x7f62e41587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:39.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62e1e00640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62e40b4de0 0x7f62e41f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:39.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62d9400640 1 -- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62e40b4de0 msgr2=0x7f62e41f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62d9400640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62e40b4de0 0x7f62e41f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62d9400640 1 -- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62e41527b0 msgr2=0x7f62e41587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62d9400640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62e41527b0 0x7f62e41587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62d9400640 1 -- 172.21.15.50:0/3354380434 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e40ceb70 con 0x7f62e40b5c70 2024-09-21T05:56:39.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62d9400640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e40b5c70 0x7f62e414fc40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00e7a0 tx=0x7f62cc00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:39.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62daa00640 1 -- 172.21.15.50:0/3354380434 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00ccc0 con 0x7f62e40b5c70 2024-09-21T05:56:39.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62daa00640 1 -- 172.21.15.50:0/3354380434 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62cc010430 con 0x7f62e40b5c70 2024-09-21T05:56:39.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62e9200640 1 -- 172.21.15.50:0/3354380434 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e40ccbf0 con 0x7f62e40b5c70 2024-09-21T05:56:39.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62daa00640 1 -- 172.21.15.50:0/3354380434 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc010860 con 0x7f62e40b5c70 2024-09-21T05:56:39.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.272+0000 7f62e9200640 1 -- 172.21.15.50:0/3354380434 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e40cee50 con 0x7f62e40b5c70 2024-09-21T05:56:39.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.276+0000 7f62e9200640 1 -- 172.21.15.50:0/3354380434 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e415bb60 con 0x7f62e40b5c70 2024-09-21T05:56:39.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.276+0000 7f62daa00640 1 -- 172.21.15.50:0/3354380434 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f62cc010a00 con 0x7f62e40b5c70 2024-09-21T05:56:39.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.276+0000 7f62daa00640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f62c0073200 0x7f62c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:39.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.276+0000 7f62daa00640 1 -- 172.21.15.50:0/3354380434 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f62cc094bd0 con 0x7f62e40b5c70 2024-09-21T05:56:39.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.276+0000 7f62e1e00640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f62c0073200 0x7f62c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:39.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.276+0000 7f62e1e00640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f62c0073200 0x7f62c0075710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f62d40af000 tx=0x7f62d401b500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:39.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.280+0000 7f62daa00640 1 -- 172.21.15.50:0/3354380434 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f62cc05fc70 con 0x7f62e40b5c70 2024-09-21T05:56:39.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.456+0000 7f62e9200640 1 -- 172.21.15.50:0/3354380434 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f62e40ce900 con 0x7f62e40b5c70 2024-09-21T05:56:39.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62daa00640 1 -- 172.21.15.50:0/3354380434 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7f62cc05f570 con 0x7f62e40b5c70 2024-09-21T05:56:39.462 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:39.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62e9200640 1 -- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f62c0073200 msgr2=0x7f62c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62e9200640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f62c0073200 0x7f62c0075710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f62d40af000 tx=0x7f62d401b500 comp rx=0 tx=0).stop 2024-09-21T05:56:39.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62e9200640 1 -- 172.21.15.50:0/3354380434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e40b5c70 msgr2=0x7f62e414fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62e9200640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e40b5c70 0x7f62e414fc40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00e7a0 tx=0x7f62cc00ec60 comp rx=0 tx=0).stop 2024-09-21T05:56:39.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62e9200640 1 -- 172.21.15.50:0/3354380434 shutdown_connections 2024-09-21T05:56:39.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62e9200640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62e41527b0 0x7f62e41587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62e9200640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f62c0073200 0x7f62c0075710 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62e9200640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e40b5c70 0x7f62e414fc40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62e9200640 1 --2- 172.21.15.50:0/3354380434 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62e40b4de0 0x7f62e41f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62e9200640 1 -- 172.21.15.50:0/3354380434 >> 172.21.15.50:0/3354380434 conn(0x7f62e40b2040 msgr2=0x7f62e4151ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:39.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62e9200640 1 -- 172.21.15.50:0/3354380434 shutdown_connections 2024-09-21T05:56:39.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.460+0000 7f62e9200640 1 -- 172.21.15.50:0/3354380434 wait complete. 2024-09-21T05:56:39.484 INFO:teuthology.orchestra.run.smithi050.stdout:[{"version":31,"timestamp":"2024-09-21T05:56:17.623048+0000","name":"","changes":[{"name":"osd.5/osd_mclock_max_capacity_iops_ssd","new_value":"3661.558528"}]}] 2024-09-21T05:56:39.484 INFO:tasks.ceph_manager:config epoch is 31 2024-09-21T05:56:39.484 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T05:56:39.484 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T05:56:39.484 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T05:56:39.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.640+0000 7fdb1d200640 1 -- 172.21.15.50:0/4008676486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb180703a0 msgr2=0x7fdb1806d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.640+0000 7fdb1d200640 1 --2- 172.21.15.50:0/4008676486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb180703a0 0x7fdb1806d200 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009840 tx=0x7fdb1001c790 comp rx=0 tx=0).stop 2024-09-21T05:56:39.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.640+0000 7fdb1d200640 1 -- 172.21.15.50:0/4008676486 shutdown_connections 2024-09-21T05:56:39.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.640+0000 7fdb1d200640 1 --2- 172.21.15.50:0/4008676486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb180703a0 0x7fdb1806d200 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.640+0000 7fdb1d200640 1 --2- 172.21.15.50:0/4008676486 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdb18159130 0x7fdb1815bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.640+0000 7fdb1d200640 1 -- 172.21.15.50:0/4008676486 >> 172.21.15.50:0/4008676486 conn(0x7fdb180b35f0 msgr2=0x7fdb180b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:39.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.640+0000 7fdb1d200640 1 -- 172.21.15.50:0/4008676486 shutdown_connections 2024-09-21T05:56:39.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.640+0000 7fdb1d200640 1 -- 172.21.15.50:0/4008676486 wait complete. 2024-09-21T05:56:39.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb1d200640 1 Processor -- start 2024-09-21T05:56:39.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb1d200640 1 -- start start 2024-09-21T05:56:39.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb1d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb1806cdf0 0x7fdb181fcb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:39.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb1d200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdb180703a0 0x7fdb18157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:39.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb1d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb18159130 0x7fdb1815f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:39.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb1d200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18168e40 con 0x7fdb1806cdf0 2024-09-21T05:56:39.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb1d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18168cc0 con 0x7fdb18159130 2024-09-21T05:56:39.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb1d200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fdb18168fc0 con 0x7fdb180703a0 2024-09-21T05:56:39.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb15e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb1806cdf0 0x7fdb181fcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:39.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb15e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb1806cdf0 0x7fdb181fcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48324/0 (socket says 172.21.15.50:48324) 2024-09-21T05:56:39.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb15e00640 1 -- 172.21.15.50:0/3152559821 learned_addr learned my addr 172.21.15.50:0/3152559821 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:39.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb0d400640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdb180703a0 0x7fdb18157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:39.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb16800640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb18159130 0x7fdb1815f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:39.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb15e00640 1 -- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdb180703a0 msgr2=0x7fdb18157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb15e00640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdb180703a0 0x7fdb18157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb15e00640 1 -- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb18159130 msgr2=0x7fdb1815f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb15e00640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb18159130 0x7fdb1815f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb15e00640 1 -- 172.21.15.50:0/3152559821 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb180cf1a0 con 0x7fdb1806cdf0 2024-09-21T05:56:39.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb15e00640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb1806cdf0 0x7fdb181fcb10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800eb30 tx=0x7fdb0800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:39.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb0ea00640 1 -- 172.21.15.50:0/3152559821 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08004b30 con 0x7fdb1806cdf0 2024-09-21T05:56:39.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb0ea00640 1 -- 172.21.15.50:0/3152559821 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb08004cf0 con 0x7fdb1806cdf0 2024-09-21T05:56:39.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb1d200640 1 -- 172.21.15.50:0/3152559821 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb180cf480 con 0x7fdb1806cdf0 2024-09-21T05:56:39.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb0ea00640 1 -- 172.21.15.50:0/3152559821 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08005280 con 0x7fdb1806cdf0 2024-09-21T05:56:39.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.644+0000 7fdb1d200640 1 -- 172.21.15.50:0/3152559821 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb180aa6b0 con 0x7fdb1806cdf0 2024-09-21T05:56:39.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.648+0000 7fdb1d200640 1 -- 172.21.15.50:0/3152559821 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb180aa4d0 con 0x7fdb1806cdf0 2024-09-21T05:56:39.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.648+0000 7fdb0ea00640 1 -- 172.21.15.50:0/3152559821 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdb08005420 con 0x7fdb1806cdf0 2024-09-21T05:56:39.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.648+0000 7fdb0ea00640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdaf4073230 0x7fdaf4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:39.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.648+0000 7fdb0ea00640 1 -- 172.21.15.50:0/3152559821 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdb08094eb0 con 0x7fdb1806cdf0 2024-09-21T05:56:39.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.648+0000 7fdb0d400640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdaf4073230 0x7fdaf4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:39.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.648+0000 7fdb0d400640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdaf4073230 0x7fdaf4075740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb180ceef0 tx=0x7fdb00007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.652+0000 7fdb0ea00640 1 -- 172.21.15.50:0/3152559821 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdb180aa4d0 con 0x7fdb1806cdf0 2024-09-21T05:56:39.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.872+0000 7fdb1d200640 1 -- 172.21.15.50:0/3152559821 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fdb1806d200 con 0x7fdb1806cdf0 2024-09-21T05:56:39.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.876+0000 7fdb0ea00640 1 -- 172.21.15.50:0/3152559821 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186457 (secure 0 0 0) 0x7fdb0805f7e0 con 0x7fdb1806cdf0 2024-09-21T05:56:39.882 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.880+0000 7fdb1d200640 1 -- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdaf4073230 msgr2=0x7fdaf4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.880+0000 7fdb1d200640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdaf4073230 0x7fdaf4075740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb180ceef0 tx=0x7fdb00007d00 comp rx=0 tx=0).stop 2024-09-21T05:56:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.880+0000 7fdb1d200640 1 -- 172.21.15.50:0/3152559821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb1806cdf0 msgr2=0x7fdb181fcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.880+0000 7fdb1d200640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb1806cdf0 0x7fdb181fcb10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800eb30 tx=0x7fdb0800c2d0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.880+0000 7fdb1d200640 1 -- 172.21.15.50:0/3152559821 shutdown_connections 2024-09-21T05:56:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.880+0000 7fdb1d200640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb18159130 0x7fdb1815f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.880+0000 7fdb1d200640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdb180703a0 0x7fdb18157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.880+0000 7fdb1d200640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdaf4073230 0x7fdaf4075740 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.880+0000 7fdb1d200640 1 --2- 172.21.15.50:0/3152559821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb1806cdf0 0x7fdb181fcb10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.880+0000 7fdb1d200640 1 -- 172.21.15.50:0/3152559821 >> 172.21.15.50:0/3152559821 conn(0x7fdb180b35f0 msgr2=0x7fdb181646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.880+0000 7fdb1d200640 1 -- 172.21.15.50:0/3152559821 shutdown_connections 2024-09-21T05:56:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:39.880+0000 7fdb1d200640 1 -- 172.21.15.50:0/3152559821 wait complete. 2024-09-21T05:56:39.903 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3010204332},{"type":"v1","addr":"172.21.15.88:6801","nonce":3010204332}]},"active_addr":"172.21.15.88:6801/3010204332","active_change":"2024-09-21T05:56:08.872944+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4171,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":3958351306}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":3269271189}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":3806266428}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":1585742927}]}]} 2024-09-21T05:56:39.907 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T05:56:39.907 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T05:56:39.907 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:56:40.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.064+0000 7f9c72800640 1 -- 172.21.15.50:0/3763920476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c0b4de0 msgr2=0x7f9c6c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.064+0000 7f9c72800640 1 --2- 172.21.15.50:0/3763920476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c0b4de0 0x7f9c6c0b51f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009840 tx=0x7f9c6401c760 comp rx=0 tx=0).stop 2024-09-21T05:56:40.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.064+0000 7f9c72800640 1 -- 172.21.15.50:0/3763920476 shutdown_connections 2024-09-21T05:56:40.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.064+0000 7f9c72800640 1 --2- 172.21.15.50:0/3763920476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c0b4de0 0x7f9c6c0b51f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.064+0000 7f9c72800640 1 --2- 172.21.15.50:0/3763920476 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c6c0b5c70 0x7f9c6c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.064+0000 7f9c72800640 1 -- 172.21.15.50:0/3763920476 >> 172.21.15.50:0/3763920476 conn(0x7f9c6c0b2040 msgr2=0x7f9c6c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.064+0000 7f9c72800640 1 -- 172.21.15.50:0/3763920476 shutdown_connections 2024-09-21T05:56:40.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.064+0000 7f9c72800640 1 -- 172.21.15.50:0/3763920476 wait complete. 2024-09-21T05:56:40.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.064+0000 7f9c72800640 1 Processor -- start 2024-09-21T05:56:40.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.064+0000 7f9c72800640 1 -- start start 2024-09-21T05:56:40.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c72800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c6c0b4de0 0x7f9c6c14fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c72800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c0b5c70 0x7f9c6c1561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c72800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c6c1527b0 0x7f9c6c156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c72800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c168a10 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c72800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c168890 con 0x7f9c6c0b4de0 2024-09-21T05:56:40.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c72800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6c168b90 con 0x7f9c6c1527b0 2024-09-21T05:56:40.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c62a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c0b5c70 0x7f9c6c1561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c62a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c0b5c70 0x7f9c6c1561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48334/0 (socket says 172.21.15.50:48334) 2024-09-21T05:56:40.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c62a00640 1 -- 172.21.15.50:0/4165302775 learned_addr learned my addr 172.21.15.50:0/4165302775 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:40.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c6be00640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c6c1527b0 0x7f9c6c156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c6b400640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c6c0b4de0 0x7f9c6c14fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c62a00640 1 -- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c6c1527b0 msgr2=0x7f9c6c156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c62a00640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c6c1527b0 0x7f9c6c156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c62a00640 1 -- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c6c0b4de0 msgr2=0x7f9c6c14fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c62a00640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c6c0b4de0 0x7f9c6c14fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c62a00640 1 -- 172.21.15.50:0/4165302775 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c159470 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c6b400640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c6c0b4de0 0x7f9c6c14fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:40.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c62a00640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c0b5c70 0x7f9c6c1561c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400c370 tx=0x7f9c5400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c63e00640 1 -- 172.21.15.50:0/4165302775 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54017070 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c63e00640 1 -- 172.21.15.50:0/4165302775 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c54003600 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c72800640 1 -- 172.21.15.50:0/4165302775 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6c0cdbb0 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c63e00640 1 -- 172.21.15.50:0/4165302775 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54003ab0 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c72800640 1 -- 172.21.15.50:0/4165302775 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6c0ce170 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.068+0000 7f9c72800640 1 -- 172.21.15.50:0/4165302775 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6c0b51f0 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.072+0000 7f9c63e00640 1 -- 172.21.15.50:0/4165302775 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9c5401a070 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.072+0000 7f9c63e00640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c48073200 0x7f9c48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.072+0000 7f9c63e00640 1 -- 172.21.15.50:0/4165302775 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9c54093520 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.072+0000 7f9c6b400640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c48073200 0x7f9c48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.072+0000 7f9c6b400640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c48073200 0x7f9c48075710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009ea0 tx=0x7f9c5c009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.076+0000 7f9c63e00640 1 -- 172.21.15.50:0/4165302775 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9c6c0b5730 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.252+0000 7f9c72800640 1 -- 172.21.15.50:0/4165302775 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c6c1570d0 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.252+0000 7f9c63e00640 1 -- 172.21.15.50:0/4165302775 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16617 (secure 0 0 0) 0x7f9c5405e530 con 0x7f9c6c0b5c70 2024-09-21T05:56:40.256 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:40.256 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":22,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:30.775110+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:56:27.052826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:56:28.162376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.256+0000 7f9c72800640 1 -- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c48073200 msgr2=0x7f9c48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.256+0000 7f9c72800640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c48073200 0x7f9c48075710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009ea0 tx=0x7f9c5c009500 comp rx=0 tx=0).stop 2024-09-21T05:56:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.256+0000 7f9c72800640 1 -- 172.21.15.50:0/4165302775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c0b5c70 msgr2=0x7f9c6c1561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.256+0000 7f9c72800640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c0b5c70 0x7f9c6c1561c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400c370 tx=0x7f9c5400c830 comp rx=0 tx=0).stop 2024-09-21T05:56:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.256+0000 7f9c72800640 1 -- 172.21.15.50:0/4165302775 shutdown_connections 2024-09-21T05:56:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.256+0000 7f9c72800640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c6c1527b0 0x7f9c6c156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.256+0000 7f9c72800640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c48073200 0x7f9c48075710 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.256+0000 7f9c72800640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c0b5c70 0x7f9c6c1561c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.256+0000 7f9c72800640 1 --2- 172.21.15.50:0/4165302775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c6c0b4de0 0x7f9c6c14fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.256+0000 7f9c72800640 1 -- 172.21.15.50:0/4165302775 >> 172.21.15.50:0/4165302775 conn(0x7f9c6c0b2040 msgr2=0x7f9c6c151ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.256+0000 7f9c72800640 1 -- 172.21.15.50:0/4165302775 shutdown_connections 2024-09-21T05:56:40.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.256+0000 7f9c72800640 1 -- 172.21.15.50:0/4165302775 wait complete. 2024-09-21T05:56:40.278 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T05:56:40.278 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:56:40.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.432+0000 7f5fcac00640 1 -- 172.21.15.50:0/2083492998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fc40b5a80 msgr2=0x7f5fc4074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.432+0000 7f5fcac00640 1 --2- 172.21.15.50:0/2083492998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fc40b5a80 0x7f5fc4074410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009900 tx=0x7f5fac01c8e0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 -- 172.21.15.50:0/2083492998 shutdown_connections 2024-09-21T05:56:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 --2- 172.21.15.50:0/2083492998 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fc4074950 0x7f5fc415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 --2- 172.21.15.50:0/2083492998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fc40b5a80 0x7f5fc4074410 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 -- 172.21.15.50:0/2083492998 >> 172.21.15.50:0/2083492998 conn(0x7f5fc40708c0 msgr2=0x7f5fc4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 -- 172.21.15.50:0/2083492998 shutdown_connections 2024-09-21T05:56:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 -- 172.21.15.50:0/2083492998 wait complete. 2024-09-21T05:56:40.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 Processor -- start 2024-09-21T05:56:40.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 -- start start 2024-09-21T05:56:40.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fc4074950 0x7f5fc41492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fc40b5a80 0x7f5fc414e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fc4157030 0x7f5fc4150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc416a310 con 0x7f5fc40b5a80 2024-09-21T05:56:40.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc416a190 con 0x7f5fc4074950 2024-09-21T05:56:40.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.436+0000 7f5fcac00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc416a490 con 0x7f5fc4157030 2024-09-21T05:56:40.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fbbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fc40b5a80 0x7f5fc414e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fbbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fc40b5a80 0x7f5fc414e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48358/0 (socket says 172.21.15.50:48358) 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fbbe00640 1 -- 172.21.15.50:0/244290222 learned_addr learned my addr 172.21.15.50:0/244290222 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fc3e00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fc4074950 0x7f5fc41492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fc8c00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fc4157030 0x7f5fc4150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fbbe00640 1 -- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fc4157030 msgr2=0x7f5fc4150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fbbe00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fc4157030 0x7f5fc4150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fbbe00640 1 -- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fc4074950 msgr2=0x7f5fc41492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fbbe00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fc4074950 0x7f5fc41492e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fbbe00640 1 -- 172.21.15.50:0/244290222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc41fb610 con 0x7f5fc40b5a80 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fc3e00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fc4074950 0x7f5fc41492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fbbe00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fc40b5a80 0x7f5fc414e670 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009850 tx=0x7f5fac0a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fc0c00640 1 -- 172.21.15.50:0/244290222 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0b2040 con 0x7f5fc40b5a80 2024-09-21T05:56:40.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fc0c00640 1 -- 172.21.15.50:0/244290222 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fac0043b0 con 0x7f5fc40b5a80 2024-09-21T05:56:40.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fcac00640 1 -- 172.21.15.50:0/244290222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc414df30 con 0x7f5fc40b5a80 2024-09-21T05:56:40.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fc0c00640 1 -- 172.21.15.50:0/244290222 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac005290 con 0x7f5fc40b5a80 2024-09-21T05:56:40.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fcac00640 1 -- 172.21.15.50:0/244290222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc41fad70 con 0x7f5fc40b5a80 2024-09-21T05:56:40.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fcac00640 1 -- 172.21.15.50:0/244290222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc415cf90 con 0x7f5fc40b5a80 2024-09-21T05:56:40.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.440+0000 7f5fc0c00640 1 -- 172.21.15.50:0/244290222 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5fac005430 con 0x7f5fc40b5a80 2024-09-21T05:56:40.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.444+0000 7f5fc0c00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fa0073230 0x7f5fa0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.444+0000 7f5fc0c00640 1 -- 172.21.15.50:0/244290222 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5fac12da60 con 0x7f5fc40b5a80 2024-09-21T05:56:40.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.444+0000 7f5fc3e00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fa0073230 0x7f5fa0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.444+0000 7f5fc3e00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fa0073230 0x7f5fa0075740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40016b0 tx=0x7f5fb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.448+0000 7f5fc0c00640 1 -- 172.21.15.50:0/244290222 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5fac0bc060 con 0x7f5fc40b5a80 2024-09-21T05:56:40.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.628+0000 7f5fcac00640 1 -- 172.21.15.50:0/244290222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fc41fa820 con 0x7f5fc40b5a80 2024-09-21T05:56:40.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fc0c00640 1 -- 172.21.15.50:0/244290222 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16617 (secure 0 0 0) 0x7f5fac0164d0 con 0x7f5fc40b5a80 2024-09-21T05:56:40.633 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:40.634 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":22,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:30.775110+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:56:27.052826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:56:28.162376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:40.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fcac00640 1 -- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fa0073230 msgr2=0x7f5fa0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fcac00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fa0073230 0x7f5fa0075740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40016b0 tx=0x7f5fb4009040 comp rx=0 tx=0).stop 2024-09-21T05:56:40.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fcac00640 1 -- 172.21.15.50:0/244290222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fc40b5a80 msgr2=0x7f5fc414e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fcac00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fc40b5a80 0x7f5fc414e670 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009850 tx=0x7f5fac0a5710 comp rx=0 tx=0).stop 2024-09-21T05:56:40.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fcac00640 1 -- 172.21.15.50:0/244290222 shutdown_connections 2024-09-21T05:56:40.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fcac00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fc4157030 0x7f5fc4150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fcac00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fa0073230 0x7f5fa0075740 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fcac00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fc40b5a80 0x7f5fc414e670 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fcac00640 1 --2- 172.21.15.50:0/244290222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fc4074950 0x7f5fc41492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fcac00640 1 -- 172.21.15.50:0/244290222 >> 172.21.15.50:0/244290222 conn(0x7f5fc40708c0 msgr2=0x7f5fc40d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fcac00640 1 -- 172.21.15.50:0/244290222 shutdown_connections 2024-09-21T05:56:40.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.632+0000 7f5fcac00640 1 -- 172.21.15.50:0/244290222 wait complete. 2024-09-21T05:56:40.656 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T05:56:40.657 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T05:56:40.657 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T05:56:40.657 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T05:56:40.658 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T05:56:40.658 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T05:56:40.659 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T05:56:40.659 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T05:56:40.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.940+0000 7fd8ffe00640 1 -- 172.21.15.50:0/923607124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8f80e0f80 msgr2=0x7fd8f8053280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.940+0000 7fd8ffe00640 1 --2- 172.21.15.50:0/923607124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8f80e0f80 0x7fd8f8053280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800b530 tx=0x7fd8f0020850 comp rx=0 tx=0).stop 2024-09-21T05:56:40.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.940+0000 7fd8ffe00640 1 -- 172.21.15.50:0/923607124 shutdown_connections 2024-09-21T05:56:40.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.940+0000 7fd8ffe00640 1 --2- 172.21.15.50:0/923607124 >> v2:172.21.15.50:3300/0 conn(0x7fd8f80514e0 0x7fd8f80397a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fd8ffe00640 1 --2- 172.21.15.50:0/923607124 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f80537c0 0x7fd8f8050fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fd8ffe00640 1 --2- 172.21.15.50:0/923607124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8f80e0f80 0x7fd8f8053280 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fd8ffe00640 1 -- 172.21.15.50:0/923607124 >> 172.21.15.50:0/923607124 conn(0x7fd8f801fbf0 msgr2=0x7fd8f8056950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fd8ffe00640 1 -- 172.21.15.50:0/923607124 shutdown_connections 2024-09-21T05:56:40.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fd8ffe00640 1 -- 172.21.15.50:0/923607124 wait complete. 2024-09-21T05:56:40.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fbf9ca00640 1 -- 172.21.15.50:0/2935419523 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf900eb010 msgr2=0x7fbf900eb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fbf9ca00640 1 --2- 172.21.15.50:0/2935419523 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf900eb010 0x7fbf900eb420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009840 tx=0x7fbf8401c790 comp rx=0 tx=0).stop 2024-09-21T05:56:40.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fbf9ca00640 1 -- 172.21.15.50:0/2935419523 shutdown_connections 2024-09-21T05:56:40.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fbf9ca00640 1 --2- 172.21.15.50:0/2935419523 >> v2:172.21.15.50:3300/0 conn(0x7fbf90053e20 0x7fbf90054230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fbf9ca00640 1 --2- 172.21.15.50:0/2935419523 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf900eb010 0x7fbf900eb420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fbf9ca00640 1 -- 172.21.15.50:0/2935419523 >> 172.21.15.50:0/2935419523 conn(0x7fbf9000e400 msgr2=0x7fbf9000e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fbf9ca00640 1 -- 172.21.15.50:0/2935419523 shutdown_connections 2024-09-21T05:56:40.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.944+0000 7fbf9ca00640 1 -- 172.21.15.50:0/2935419523 wait complete. 2024-09-21T05:56:40.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.948+0000 7f8c3ec00640 1 -- 172.21.15.50:0/1152719449 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c20004810 msgr2=0x7f8c20004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.948+0000 7f8c3ec00640 1 --2- 172.21.15.50:0/1152719449 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c20004810 0x7f8c20004c20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009940 tx=0x7f8c2401c720 comp rx=0 tx=0).stop 2024-09-21T05:56:40.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.948+0000 7f8c3e200640 1 -- 172.21.15.50:0/1152719449 reap_dead start 2024-09-21T05:56:40.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.948+0000 7fbf9ca00640 1 Processor -- start 2024-09-21T05:56:40.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.948+0000 7fbf9ca00640 1 -- start start 2024-09-21T05:56:40.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7f8c3ec00640 1 -- 172.21.15.50:0/1152719449 shutdown_connections 2024-09-21T05:56:40.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7f8c3ec00640 1 -- 172.21.15.50:0/1152719449 >> 172.21.15.50:0/1152719449 conn(0x7f8c3001eac0 msgr2=0x7f8c300558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7f8c3ec00640 1 -- 172.21.15.50:0/1152719449 shutdown_connections 2024-09-21T05:56:40.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7f8c3ec00640 1 -- 172.21.15.50:0/1152719449 wait complete. 2024-09-21T05:56:40.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fd8ffe00640 1 Processor -- start 2024-09-21T05:56:40.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fd8ffe00640 1 -- start start 2024-09-21T05:56:40.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf9ca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf90053e20 0x7fbf900dc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf9ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900582d0 0x7fbf900dc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf9ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf90052620 0x7fbf900dce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf9ca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90058860 con 0x7fbf90052620 2024-09-21T05:56:40.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf9ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbf900586e0 con 0x7fbf900582d0 2024-09-21T05:56:40.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf9ca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fbf900f49b0 con 0x7fbf90053e20 2024-09-21T05:56:40.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf97e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf90052620 0x7fbf900dce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf97e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf90052620 0x7fbf900dce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48392/0 (socket says 172.21.15.50:48392) 2024-09-21T05:56:40.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf97e00640 1 -- 172.21.15.50:0/4075822017 learned_addr learned my addr 172.21.15.50:0/4075822017 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:40.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf97400640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf90053e20 0x7fbf900dc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf97e00640 1 -- 172.21.15.50:0/4075822017 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf90053e20 msgr2=0x7fbf900dc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf97e00640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbf90053e20 0x7fbf900dc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf97e00640 1 -- 172.21.15.50:0/4075822017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900582d0 msgr2=0x7fbf900dc950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:56:40.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf97e00640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900582d0 0x7fbf900dc950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7fbf97e00640 1 -- 172.21.15.50:0/4075822017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9018e040 con 0x7fbf90052620 2024-09-21T05:56:40.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7f21de000640 1 -- 172.21.15.50:0/90814544 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21d80ce4c0 msgr2=0x7f21d80cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7f21de000640 1 --2- 172.21.15.50:0/90814544 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21d80ce4c0 0x7f21d80cd260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009900 tx=0x7f21c801c8e0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7f21de000640 1 -- 172.21.15.50:0/90814544 shutdown_connections 2024-09-21T05:56:40.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7f21de000640 1 --2- 172.21.15.50:0/90814544 >> v2:172.21.15.88:3300/0 conn(0x7f21d80cdbb0 0x7f21d815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.952+0000 7f9820a00640 1 -- 172.21.15.50:0/3609361769 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98140a66c0 con 0x7f9824159130 2024-09-21T05:56:40.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f21de000640 1 --2- 172.21.15.50:0/90814544 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21d80ce4c0 0x7f21d80cd260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f21de000640 1 -- 172.21.15.50:0/90814544 >> 172.21.15.50:0/90814544 conn(0x7f21d80b2040 msgr2=0x7f21d80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f21de000640 1 -- 172.21.15.50:0/90814544 shutdown_connections 2024-09-21T05:56:40.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f9829200640 1 -- 172.21.15.50:0/3609361769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9824159130 msgr2=0x7f982415bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f8c3ec00640 1 Processor -- start 2024-09-21T05:56:40.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f21de000640 1 -- 172.21.15.50:0/90814544 wait complete. 2024-09-21T05:56:40.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f9829200640 1 --2- 172.21.15.50:0/3609361769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9824159130 0x7f982415bb90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f98240b6bf0 tx=0x7f981401c850 comp rx=0 tx=0).stop 2024-09-21T05:56:40.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f8c3ec00640 1 -- start start 2024-09-21T05:56:40.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f9829200640 1 -- 172.21.15.50:0/3609361769 shutdown_connections 2024-09-21T05:56:40.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f9829200640 1 --2- 172.21.15.50:0/3609361769 >> v2:172.21.15.50:3300/0 conn(0x7f982406fc90 0x7f98240700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f9829200640 1 --2- 172.21.15.50:0/3609361769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f982406cdf0 0x7f982406d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f9829200640 1 --2- 172.21.15.50:0/3609361769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9824159130 0x7f982415bb90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f9829200640 1 -- 172.21.15.50:0/3609361769 >> 172.21.15.50:0/3609361769 conn(0x7f98240b35f0 msgr2=0x7f98240b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f9829200640 1 -- 172.21.15.50:0/3609361769 shutdown_connections 2024-09-21T05:56:40.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f9829200640 1 -- 172.21.15.50:0/3609361769 wait complete. 2024-09-21T05:56:40.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7fbf97e00640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf90052620 0x7fbf900dce90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf98066c60 tx=0x7fbf9806a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7fbf8b400640 1 -- 172.21.15.50:0/4075822017 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9806b540 con 0x7fbf90052620 2024-09-21T05:56:40.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7fbf8b400640 1 -- 172.21.15.50:0/4075822017 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf9806ac60 con 0x7fbf90052620 2024-09-21T05:56:40.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f9074c00640 1 -- 172.21.15.50:0/4094656845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90700d0910 msgr2=0x7f90700b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f9074c00640 1 --2- 172.21.15.50:0/4094656845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90700d0910 0x7f90700b3840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9068009bc0 tx=0x7f906801cba0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7fbf8b400640 1 -- 172.21.15.50:0/4075822017 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9806b030 con 0x7fbf90052620 2024-09-21T05:56:40.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7fbf9ca00640 1 -- 172.21.15.50:0/4075822017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9018e320 con 0x7fbf90052620 2024-09-21T05:56:40.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7fbf9ca00640 1 -- 172.21.15.50:0/4075822017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf900db8c0 con 0x7fbf90052620 2024-09-21T05:56:40.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7fd8ffe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f80514e0 0x7fd8f80e2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7fd8ffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f80537c0 0x7fd8f8176890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f8c3ec00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c20004810 0x7f8c3000c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f8c3ec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c301803a0 0x7f8c30039730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f8c3ec00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c301807b0 0x7f8c30039c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7fd8ffe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8f80e0f80 0x7fd8f8179ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7fd8ffe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f8054a40 con 0x7fd8f80537c0 2024-09-21T05:56:40.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7fd8ffe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f80548c0 con 0x7fd8f80e0f80 2024-09-21T05:56:40.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7fd8ffe00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f8054bc0 con 0x7fd8f80514e0 2024-09-21T05:56:40.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7fd8fea00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f80514e0 0x7fd8f80e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7fd8fea00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f80514e0 0x7fd8f80e2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:50984/0 (socket says 172.21.15.50:50984) 2024-09-21T05:56:40.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7fd8fea00640 1 -- 172.21.15.50:0/939233 learned_addr learned my addr 172.21.15.50:0/939233 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:40.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7fd8ff400640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8f80e0f80 0x7fd8f8179ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f8c3ec00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c300ee090 con 0x7f8c301807b0 2024-09-21T05:56:40.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f8c3ec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8c300edf10 con 0x7f8c301803a0 2024-09-21T05:56:40.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.956+0000 7f8c3ec00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8c300ee210 con 0x7f8c20004810 2024-09-21T05:56:40.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c3d800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c20004810 0x7f8c3000c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c3d800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c20004810 0x7f8c3000c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:50976/0 (socket says 172.21.15.50:50976) 2024-09-21T05:56:40.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7fd8fea00640 1 -- 172.21.15.50:0/939233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8f80e0f80 msgr2=0x7fd8f8179ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7fd8fea00640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8f80e0f80 0x7fd8f8179ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7fd8fea00640 1 -- 172.21.15.50:0/939233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f80537c0 msgr2=0x7fd8f8176890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7fd8fea00640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f80537c0 0x7fd8f8176890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7fd8fea00640 1 -- 172.21.15.50:0/939233 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f8034770 con 0x7fd8f80514e0 2024-09-21T05:56:40.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c3d800640 1 -- 172.21.15.50:0/848922211 learned_addr learned my addr 172.21.15.50:0/848922211 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:40.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c3ce00640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c301803a0 0x7f8c30039730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c3d800640 1 -- 172.21.15.50:0/848922211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c301803a0 msgr2=0x7f8c30039730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c3d800640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c301803a0 0x7f8c30039730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c3d800640 1 -- 172.21.15.50:0/848922211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c301807b0 msgr2=0x7f8c30039c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c3d800640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c301807b0 0x7f8c30039c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c3d800640 1 -- 172.21.15.50:0/848922211 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c30189c00 con 0x7f8c20004810 2024-09-21T05:56:40.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f9074c00640 1 -- 172.21.15.50:0/4094656845 shutdown_connections 2024-09-21T05:56:40.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f9074c00640 1 --2- 172.21.15.50:0/4094656845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90700d0910 0x7f90700b3840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f9074c00640 1 -- 172.21.15.50:0/4094656845 >> 172.21.15.50:0/4094656845 conn(0x7f907006e4a0 msgr2=0x7f907006e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c3d800640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c20004810 0x7f8c3000c160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c000c10 tx=0x7f8c2c003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f21de000640 1 Processor -- start 2024-09-21T05:56:40.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f21de000640 1 -- start start 2024-09-21T05:56:40.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7fd8fea00640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f80514e0 0x7fd8f80e2670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000dc60 tx=0x7fd8f000d3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c35600640 1 -- 172.21.15.50:0/848922211 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c015070 con 0x7f8c20004810 2024-09-21T05:56:40.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c35600640 1 -- 172.21.15.50:0/848922211 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c2c011920 con 0x7f8c20004810 2024-09-21T05:56:40.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c35600640 1 -- 172.21.15.50:0/848922211 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c011d70 con 0x7f8c20004810 2024-09-21T05:56:40.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c3ec00640 1 -- 172.21.15.50:0/848922211 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c30189ee0 con 0x7f8c20004810 2024-09-21T05:56:40.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f8c3ec00640 1 -- 172.21.15.50:0/848922211 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c3018a4a0 con 0x7f8c20004810 2024-09-21T05:56:40.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7fbf9ca00640 1 -- 172.21.15.50:0/4075822017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf900eb420 con 0x7fbf90052620 2024-09-21T05:56:40.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fbf8b400640 1 -- 172.21.15.50:0/4075822017 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbf9806b1d0 con 0x7fbf90052620 2024-09-21T05:56:40.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f9074c00640 1 -- 172.21.15.50:0/4094656845 shutdown_connections 2024-09-21T05:56:40.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f9829200640 1 Processor -- start 2024-09-21T05:56:40.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f9829200640 1 -- start start 2024-09-21T05:56:40.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7efefea00640 1 -- 172.21.15.50:0/3499406254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef8074040 msgr2=0x7efef815f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f21de000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21d80cd7a0 0x7f21d8151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f21de000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21d80cdbb0 0x7f21d814def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f21de000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d80ce4c0 0x7f21d814eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f21de000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f21d8169ee0 con 0x7f21d80cd7a0 2024-09-21T05:56:40.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f21de000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21d8169d60 con 0x7f21d80ce4c0 2024-09-21T05:56:40.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f21de000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f21d816a060 con 0x7f21d80cdbb0 2024-09-21T05:56:40.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f21d6a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21d80cdbb0 0x7f21d814def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f21d6a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21d80cdbb0 0x7f21d814def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:50998/0 (socket says 172.21.15.50:50998) 2024-09-21T05:56:40.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f21d6a00640 1 -- 172.21.15.50:0/2197970352 learned_addr learned my addr 172.21.15.50:0/2197970352 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:40.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21d7e00640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d80ce4c0 0x7f21d814eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fd8f6a00640 1 -- 172.21.15.50:0/939233 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00043d0 con 0x7fd8f80514e0 2024-09-21T05:56:40.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fd8f6a00640 1 -- 172.21.15.50:0/939233 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8f00a95d0 con 0x7fd8f80514e0 2024-09-21T05:56:40.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fd8f6a00640 1 -- 172.21.15.50:0/939233 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00a99e0 con 0x7fd8f80514e0 2024-09-21T05:56:40.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21d6a00640 1 -- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d80ce4c0 msgr2=0x7f21d814eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21d6a00640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d80ce4c0 0x7f21d814eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7efef6a00640 1 -- 172.21.15.50:0/3499406254 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef00b2070 con 0x7efef8074040 2024-09-21T05:56:40.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21d6a00640 1 -- 172.21.15.50:0/2197970352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21d80cd7a0 msgr2=0x7f21d8151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21d6a00640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21d80cd7a0 0x7f21d8151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21d6a00640 1 -- 172.21.15.50:0/2197970352 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d8200960 con 0x7f21d80cdbb0 2024-09-21T05:56:40.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fd8ffe00640 1 -- 172.21.15.50:0/939233 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f80db8a0 con 0x7fd8f80514e0 2024-09-21T05:56:40.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fd8ffe00640 1 -- 172.21.15.50:0/939233 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f80dbe40 con 0x7fd8f80514e0 2024-09-21T05:56:40.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21d6a00640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21d80cdbb0 0x7f21d814def0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f21c400c460 tx=0x7f21c400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21cf400640 1 -- 172.21.15.50:0/2197970352 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c40052d0 con 0x7f21d80cdbb0 2024-09-21T05:56:40.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21cf400640 1 -- 172.21.15.50:0/2197970352 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21c4005470 con 0x7f21d80cdbb0 2024-09-21T05:56:40.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21cf400640 1 -- 172.21.15.50:0/2197970352 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4012870 con 0x7f21d80cdbb0 2024-09-21T05:56:40.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21de000640 1 -- 172.21.15.50:0/2197970352 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d8200c00 con 0x7f21d80cdbb0 2024-09-21T05:56:40.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f9829200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f982406cdf0 0x7f98240cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f9829200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f982406fc90 0x7f98241567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f9829200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9824159130 0x7f98241531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f9829200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f982415f8c0 con 0x7f982406cdf0 2024-09-21T05:56:40.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f9829200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f982415f740 con 0x7f9824159130 2024-09-21T05:56:40.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f9829200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f982415fa40 con 0x7f982406fc90 2024-09-21T05:56:40.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.960+0000 7f9074c00640 1 -- 172.21.15.50:0/4094656845 wait complete. 2024-09-21T05:56:40.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21de000640 1 -- 172.21.15.50:0/2197970352 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d82011c0 con 0x7f21d80cdbb0 2024-09-21T05:56:40.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fbf8b400640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbf78073230 0x7fbf78075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fbf8b400640 1 -- 172.21.15.50:0/4075822017 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbf9810fe20 con 0x7fbf90052620 2024-09-21T05:56:40.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fbf8b400640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] conn(0x7fbf7807b480 0x7fbf7807d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fbf8b400640 1 -- 172.21.15.50:0/4075822017 --> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf7807dfa0 con 0x7fbf7807b480 2024-09-21T05:56:40.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f9821e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f982406cdf0 0x7f98240cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f8c3ec00640 1 -- 172.21.15.50:0/848922211 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c3018a880 con 0x7f8c20004810 2024-09-21T05:56:40.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f9821e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f982406cdf0 0x7f98240cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48452/0 (socket says 172.21.15.50:48452) 2024-09-21T05:56:40.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f9821e00640 1 -- 172.21.15.50:0/2398447448 learned_addr learned my addr 172.21.15.50:0/2398447448 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:40.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f9821400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f982406fc90 0x7f98241567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fbf96a00640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] conn(0x7fbf7807b480 0x7fbf7807d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f21de000640 1 -- 172.21.15.50:0/2197970352 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f21d806b540 con 0x7f21d80cdbb0 2024-09-21T05:56:40.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fbf97400640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbf78073230 0x7fbf78075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f9821400640 1 -- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9824159130 msgr2=0x7f98241531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fbf8b400640 1 -- 172.21.15.50:0/4075822017 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbf980d6eb0 con 0x7fbf90052620 2024-09-21T05:56:40.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f3832600640 1 -- 172.21.15.50:0/161882256 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3810004810 msgr2=0x7f3810004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f3832600640 1 --2- 172.21.15.50:0/161882256 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3810004810 0x7f3810004c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f381c00c260 tx=0x7f381c01f680 comp rx=0 tx=0).stop 2024-09-21T05:56:40.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7f8c35600640 1 -- 172.21.15.50:0/848922211 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8c2c00f430 con 0x7f8c20004810 2024-09-21T05:56:40.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.964+0000 7fbf96a00640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] conn(0x7fbf7807b480 0x7fbf7807d8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7fbf97400640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbf78073230 0x7fbf78075740 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c000f70 tx=0x7fbf8c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f9074c00640 1 Processor -- start 2024-09-21T05:56:40.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f3832600640 1 -- 172.21.15.50:0/161882256 shutdown_connections 2024-09-21T05:56:40.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f3832600640 1 --2- 172.21.15.50:0/161882256 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3810004810 0x7f3810004c20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f3832600640 1 -- 172.21.15.50:0/161882256 >> 172.21.15.50:0/161882256 conn(0x7f382c0b2040 msgr2=0x7f382c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7fd8ffe00640 1 -- 172.21.15.50:0/939233 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd8f8055100 con 0x7fd8f80514e0 2024-09-21T05:56:40.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f9822800640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9824159130 0x7f98241531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f9821400640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9824159130 0x7f98241531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f9821400640 1 -- 172.21.15.50:0/2398447448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f982406cdf0 msgr2=0x7f98240cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f9821400640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f982406cdf0 0x7f98240cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f9074c00640 1 -- start start 2024-09-21T05:56:40.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f9821400640 1 -- 172.21.15.50:0/2398447448 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9824156d00 con 0x7f982406fc90 2024-09-21T05:56:40.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f9821e00640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f982406cdf0 0x7f98240cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:40.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f9821400640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f982406fc90 0x7f98241567c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9824071ac0 tx=0x7f981000cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f21cf400640 1 -- 172.21.15.50:0/2197970352 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f21c4012a10 con 0x7f21d80cdbb0 2024-09-21T05:56:40.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f981a000640 1 -- 172.21.15.50:0/2398447448 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810005220 con 0x7f982406fc90 2024-09-21T05:56:40.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f981a000640 1 -- 172.21.15.50:0/2398447448 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98100054b0 con 0x7f982406fc90 2024-09-21T05:56:40.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f8c35600640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8c180732d0 0x7f8c180757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f9829200640 1 -- 172.21.15.50:0/2398447448 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9824155a00 con 0x7f982406fc90 2024-09-21T05:56:40.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f9829200640 1 -- 172.21.15.50:0/2398447448 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9824155fc0 con 0x7f982406fc90 2024-09-21T05:56:40.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f981a000640 1 -- 172.21.15.50:0/2398447448 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810004a10 con 0x7f982406fc90 2024-09-21T05:56:40.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f9822800640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9824159130 0x7f98241531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:40.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f3832600640 1 -- 172.21.15.50:0/161882256 shutdown_connections 2024-09-21T05:56:40.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f3832600640 1 -- 172.21.15.50:0/161882256 wait complete. 2024-09-21T05:56:40.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7efefea00640 1 --2- 172.21.15.50:0/3499406254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef8074040 0x7efef815f250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efef0009bc0 tx=0x7efef001cb10 comp rx=0 tx=0).stop 2024-09-21T05:56:40.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7efefea00640 1 -- 172.21.15.50:0/3499406254 shutdown_connections 2024-09-21T05:56:40.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7efefea00640 1 --2- 172.21.15.50:0/3499406254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef8074040 0x7efef815f250 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7efefea00640 1 --2- 172.21.15.50:0/3499406254 >> v2:172.21.15.50:3300/0 conn(0x7efef80b57c0 0x7efef8073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7efefea00640 1 -- 172.21.15.50:0/3499406254 >> 172.21.15.50:0/3499406254 conn(0x7efef80b3b30 msgr2=0x7efef80733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f21cf400640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21b80732d0 0x7f21b80757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f21cf400640 1 -- 172.21.15.50:0/2197970352 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f21c4093f20 con 0x7f21d80cdbb0 2024-09-21T05:56:40.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f21cf400640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] conn(0x7f21b807b540 0x7f21b807d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f21cf400640 1 -- 172.21.15.50:0/2197970352 --> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21c4004ec0 con 0x7f21b807b540 2024-09-21T05:56:40.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f21cf400640 1 -- 172.21.15.50:0/2197970352 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f21c4094310 con 0x7f21d80cdbb0 2024-09-21T05:56:40.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f21d7400640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21b80732d0 0x7f21b80757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7efefea00640 1 -- 172.21.15.50:0/3499406254 shutdown_connections 2024-09-21T05:56:40.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f8c35600640 1 -- 172.21.15.50:0/848922211 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8c2c095040 con 0x7f8c20004810 2024-09-21T05:56:40.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f8c3ce00640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8c180732d0 0x7f8c180757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f8c35600640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] conn(0x7f8c1807b5a0 0x7f8c1807d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f8c35600640 1 -- 172.21.15.50:0/848922211 --> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c2c014ec0 con 0x7f8c1807b5a0 2024-09-21T05:56:40.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f8c35600640 1 -- 172.21.15.50:0/848922211 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8c2c09c050 con 0x7f8c20004810 2024-09-21T05:56:40.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7fd8f6a00640 1 -- 172.21.15.50:0/939233 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd8f00c2020 con 0x7fd8f80514e0 2024-09-21T05:56:40.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7efefea00640 1 -- 172.21.15.50:0/3499406254 wait complete. 2024-09-21T05:56:40.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7f21d7e00640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] conn(0x7f21b807b540 0x7f21b807d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.968+0000 7fbf8b400640 1 -- 172.21.15.50:0/4075822017 <== osd.2 v2:172.21.15.50:6824/229343709 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf7807dfa0 con 0x7fbf7807b480 2024-09-21T05:56:40.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7fd8f6a00640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8e0073250 0x7fd8e0075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f8c3e200640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] conn(0x7f8c1807b5a0 0x7f8c1807d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f21d7400640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21b80732d0 0x7f21b80757e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f21c80af000 tx=0x7f21c8019680 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7fd8f6a00640 1 -- 172.21.15.50:0/939233 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd8f01321f0 con 0x7fd8f80514e0 2024-09-21T05:56:40.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7fd8fe000640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8e0073250 0x7fd8e0075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7fd8f6a00640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] conn(0x7fd8e007b520 0x7fd8e007d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7fd8f6a00640 1 -- 172.21.15.50:0/939233 --> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8f0001d40 con 0x7fd8e007b520 2024-09-21T05:56:40.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7fd8f6a00640 1 -- 172.21.15.50:0/939233 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd8f0139050 con 0x7fd8f80514e0 2024-09-21T05:56:40.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f8c3ce00640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8c180732d0 0x7f8c180757e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c240af000 tx=0x7f8c24019520 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f21d7e00640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] conn(0x7f21b807b540 0x7f21b807d990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f9829200640 1 -- 172.21.15.50:0/2398447448 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f97fc001040 con 0x7f982406fc90 2024-09-21T05:56:40.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f3832600640 1 Processor -- start 2024-09-21T05:56:40.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f3832600640 1 -- start start 2024-09-21T05:56:40.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7fd8ff400640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] conn(0x7fd8e007b520 0x7fd8e007d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f9074c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f907006fd70 0x7f9070204880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f8c3e200640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] conn(0x7f8c1807b5a0 0x7f8c1807d9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f9074c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90700d0500 0x7f90701465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f21cf400640 1 -- 172.21.15.50:0/2197970352 <== osd.7 v2:172.21.15.88:6810/4250480352 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21c4004ec0 con 0x7f21b807b540 2024-09-21T05:56:40.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f9074c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f90700d0910 0x7f9070146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f9074c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f90701588a0 con 0x7f907006fd70 2024-09-21T05:56:40.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f9074c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9070158720 con 0x7f90700d0500 2024-09-21T05:56:40.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f9074c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9070158a20 con 0x7f90700d0910 2024-09-21T05:56:40.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f9067e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90700d0500 0x7f90701465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f9067e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90700d0500 0x7f90701465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:47320/0 (socket says 172.21.15.50:47320) 2024-09-21T05:56:40.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f9067e00640 1 -- 172.21.15.50:0/2861875627 learned_addr learned my addr 172.21.15.50:0/2861875627 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:40.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f906fe00640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f90700d0910 0x7f9070146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7fd8fe000640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8e0073250 0x7fd8e0075760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80d8880 tx=0x7fd8ec005cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7efefea00640 1 Processor -- start 2024-09-21T05:56:40.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f8c35600640 1 -- 172.21.15.50:0/848922211 <== osd.1 v2:172.21.15.50:6800/3048348933 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c2c014ec0 con 0x7f8c1807b5a0 2024-09-21T05:56:40.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f906f400640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f907006fd70 0x7f9070204880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7f3832600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3810004810 0x7f382c149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.972+0000 7fd8ff400640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] conn(0x7fd8e007b520 0x7fd8e007d970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f3832600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f382c0ad630 0x7f382c0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f3832600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f382c155130 0x7f382c155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f3832600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f382c0751c0 con 0x7f3810004810 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f3832600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f382c075040 con 0x7f382c155130 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f3832600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f382c075340 con 0x7f382c0ad630 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f382b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3810004810 0x7f382c149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f382b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3810004810 0x7f382c149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48474/0 (socket says 172.21.15.50:48474) 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f382b400640 1 -- 172.21.15.50:0/4156933243 learned_addr learned my addr 172.21.15.50:0/4156933243 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f382aa00640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f382c0ad630 0x7f382c0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f382be00640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f382c155130 0x7f382c155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f382b400640 1 -- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f382c0ad630 msgr2=0x7f382c0ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f382b400640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f382c0ad630 0x7f382c0ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f382b400640 1 -- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f382c155130 msgr2=0x7f382c155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f382b400640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f382c155130 0x7f382c155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f382b400640 1 -- 172.21.15.50:0/4156933243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c155a80 con 0x7f3810004810 2024-09-21T05:56:40.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f906f400640 1 -- 172.21.15.50:0/2861875627 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f90700d0910 msgr2=0x7f9070146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f906f400640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f90700d0910 0x7f9070146b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f906f400640 1 -- 172.21.15.50:0/2861875627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90700d0500 msgr2=0x7f90701465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f906f400640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90700d0500 0x7f90701465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f906f400640 1 -- 172.21.15.50:0/2861875627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90700ae2c0 con 0x7f907006fd70 2024-09-21T05:56:40.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f981a000640 1 -- 172.21.15.50:0/2398447448 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9810004bb0 con 0x7f982406fc90 2024-09-21T05:56:40.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f981a000640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9804073280 0x7f9804075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f981a000640 1 -- 172.21.15.50:0/2398447448 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9810094180 con 0x7f982406fc90 2024-09-21T05:56:40.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f9067e00640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90700d0500 0x7f90701465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:40.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f906f400640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f907006fd70 0x7f9070204880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f906000c3c0 tx=0x7f906000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f9821e00640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9804073280 0x7f9804075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f981a000640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] conn(0x7f980407b4d0 0x7f980407d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f981a000640 1 -- 172.21.15.50:0/2398447448 --> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98100044a0 con 0x7f980407b4d0 2024-09-21T05:56:40.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f981a000640 1 -- 172.21.15.50:0/2398447448 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f981009b050 con 0x7f982406fc90 2024-09-21T05:56:40.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f382b400640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3810004810 0x7f382c149c00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f382400c700 tx=0x7f382400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f3823400640 1 -- 172.21.15.50:0/4156933243 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3824017070 con 0x7f3810004810 2024-09-21T05:56:40.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f3832600640 1 -- 172.21.15.50:0/4156933243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382c0cd020 con 0x7f3810004810 2024-09-21T05:56:40.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f3832600640 1 -- 172.21.15.50:0/4156933243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f382c0cd810 con 0x7f3810004810 2024-09-21T05:56:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f3823400640 1 -- 172.21.15.50:0/4156933243 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3824003c90 con 0x7f3810004810 2024-09-21T05:56:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7f3823400640 1 -- 172.21.15.50:0/4156933243 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38240052d0 con 0x7f3810004810 2024-09-21T05:56:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7efefea00640 1 -- start start 2024-09-21T05:56:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7efefea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef8074040 0x7efef814d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.976+0000 7efefea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef80b57c0 0x7efef815e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efefea00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef80cf4a0 0x7efef8200610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efefea00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efef8166e00 con 0x7efef8074040 2024-09-21T05:56:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efefea00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efef8166c80 con 0x7efef80b57c0 2024-09-21T05:56:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efefea00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7efef8166f80 con 0x7efef80cf4a0 2024-09-21T05:56:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7fd8f6a00640 1 -- 172.21.15.50:0/939233 <== osd.0 v2:172.21.15.50:6808/1152796349 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd8f0001d40 con 0x7fd8e007b520 2024-09-21T05:56:40.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f9067400640 1 -- 172.21.15.50:0/2861875627 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060011790 con 0x7f907006fd70 2024-09-21T05:56:40.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f9067400640 1 -- 172.21.15.50:0/2861875627 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9060017070 con 0x7f907006fd70 2024-09-21T05:56:40.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f9067400640 1 -- 172.21.15.50:0/2861875627 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060011ce0 con 0x7f907006fd70 2024-09-21T05:56:40.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f9074c00640 1 -- 172.21.15.50:0/2861875627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90700ae7c0 con 0x7f907006fd70 2024-09-21T05:56:40.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f3832600640 1 -- 172.21.15.50:0/4156933243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3804001040 con 0x7f3810004810 2024-09-21T05:56:40.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f9074c00640 1 -- 172.21.15.50:0/2861875627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90700aed80 con 0x7f907006fd70 2024-09-21T05:56:40.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efefca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef80cf4a0 0x7efef8200610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efefca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef80cf4a0 0x7efef8200610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:51052/0 (socket says 172.21.15.50:51052) 2024-09-21T05:56:40.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efefca00640 1 -- 172.21.15.50:0/2099638620 learned_addr learned my addr 172.21.15.50:0/2099638620 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:40.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f9064c00640 1 -- 172.21.15.50:0/2861875627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f907006b540 con 0x7f907006fd70 2024-09-21T05:56:41.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f9822800640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] conn(0x7f980407b4d0 0x7f980407d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f9821e00640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9804073280 0x7f9804075790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f981401af00 tx=0x7f98140a8040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efef7400640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef80b57c0 0x7efef815e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efef7e00640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef8074040 0x7efef814d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efef7400640 1 -- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef80cf4a0 msgr2=0x7efef8200610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efef7400640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef80cf4a0 0x7efef8200610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efef7400640 1 -- 172.21.15.50:0/2099638620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef8074040 msgr2=0x7efef814d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7f9822800640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] conn(0x7f980407b4d0 0x7f980407d920 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.980+0000 7efef7400640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef8074040 0x7efef814d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.984+0000 7f981a000640 1 -- 172.21.15.50:0/2398447448 <== osd.4 v2:172.21.15.88:6826/2441482427 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98100044a0 con 0x7f980407b4d0 2024-09-21T05:56:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.984+0000 7f3823400640 1 -- 172.21.15.50:0/4156933243 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f382401a070 con 0x7f3810004810 2024-09-21T05:56:41.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.984+0000 7f9067400640 1 -- 172.21.15.50:0/2861875627 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f906001c460 con 0x7f907006fd70 2024-09-21T05:56:41.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.984+0000 7efef7400640 1 -- 172.21.15.50:0/2099638620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef815e6a0 con 0x7efef80b57c0 2024-09-21T05:56:41.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.984+0000 7efefca00640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef80cf4a0 0x7efef8200610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:41.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.984+0000 7efef7e00640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef8074040 0x7efef814d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:41.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.984+0000 7f3823400640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f380c073200 0x7f380c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.984+0000 7f3823400640 1 -- 172.21.15.50:0/4156933243 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3824093e80 con 0x7f3810004810 2024-09-21T05:56:41.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f3823400640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] conn(0x7f380c07b470 0x7f380c07d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f3823400640 1 -- 172.21.15.50:0/4156933243 --> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f380c07df90 con 0x7f380c07b470 2024-09-21T05:56:41.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f3823400640 1 -- 172.21.15.50:0/4156933243 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3824094270 con 0x7f3810004810 2024-09-21T05:56:41.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7efef7400640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef80b57c0 0x7efef815e160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efee800d6b0 tx=0x7efee800db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f382be00640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] conn(0x7f380c07b470 0x7f380c07d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f9067400640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f904c0732d0 0x7f904c0757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f9067400640 1 -- 172.21.15.50:0/2861875627 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9060094820 con 0x7f907006fd70 2024-09-21T05:56:41.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f9067400640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] conn(0x7f904c07b540 0x7f904c07d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f9067400640 1 -- 172.21.15.50:0/2861875627 --> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9060010e50 con 0x7f904c07b540 2024-09-21T05:56:41.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f9067400640 1 -- 172.21.15.50:0/2861875627 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9060094c10 con 0x7f907006fd70 2024-09-21T05:56:41.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f382aa00640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f380c073200 0x7f380c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7efeefe00640 1 -- 172.21.15.50:0/2099638620 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8017070 con 0x7efef80b57c0 2024-09-21T05:56:41.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7efeefe00640 1 -- 172.21.15.50:0/2099638620 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efee8005310 con 0x7efef80b57c0 2024-09-21T05:56:41.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7efeefe00640 1 -- 172.21.15.50:0/2099638620 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee80097c0 con 0x7efef80b57c0 2024-09-21T05:56:41.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7efefea00640 1 -- 172.21.15.50:0/2099638620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef815c690 con 0x7efef80b57c0 2024-09-21T05:56:41.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7efefea00640 1 -- 172.21.15.50:0/2099638620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef814e340 con 0x7efef80b57c0 2024-09-21T05:56:41.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f9067e00640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f904c0732d0 0x7f904c0757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7efefea00640 1 -- 172.21.15.50:0/2099638620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efef815c1d0 con 0x7efef80b57c0 2024-09-21T05:56:41.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f906fe00640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] conn(0x7f904c07b540 0x7f904c07d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f382be00640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] conn(0x7f380c07b470 0x7f380c07d8c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f382aa00640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f380c073200 0x7f380c075710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f381c0b0000 tx=0x7f381c01c4c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.988+0000 7f9067e00640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f904c0732d0 0x7f904c0757e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9070208580 tx=0x7f9058009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.992+0000 7f3823400640 1 -- 172.21.15.50:0/4156933243 <== osd.5 v2:172.21.15.88:6802/3846197193 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f380c07df90 con 0x7f380c07b470 2024-09-21T05:56:41.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.992+0000 7f906fe00640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] conn(0x7f904c07b540 0x7f904c07d990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.992+0000 7f9067400640 1 -- 172.21.15.50:0/2861875627 <== osd.3 v2:172.21.15.50:6809/599598977 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9060010e50 con 0x7f904c07b540 2024-09-21T05:56:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.992+0000 7efeefe00640 1 -- 172.21.15.50:0/2099638620 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efee8009960 con 0x7efef80b57c0 2024-09-21T05:56:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.992+0000 7efeefe00640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efed8073280 0x7efed8075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.996+0000 7efeefe00640 1 -- 172.21.15.50:0/2099638620 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efee8094630 con 0x7efef80b57c0 2024-09-21T05:56:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.996+0000 7efeefe00640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] conn(0x7efed807b550 0x7efed807d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.996+0000 7efeefe00640 1 -- 172.21.15.50:0/2099638620 --> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efee8007e50 con 0x7efed807b550 2024-09-21T05:56:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.996+0000 7efeefe00640 1 -- 172.21.15.50:0/2099638620 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7efee80949e0 con 0x7efef80b57c0 2024-09-21T05:56:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.996+0000 7efef7e00640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efed8073280 0x7efed8075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.996+0000 7efefca00640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] conn(0x7efed807b550 0x7efed807d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.996+0000 7f21de000640 1 -- 172.21.15.50:0/2197970352 --> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f21d8151980 con 0x7f21b807b540 2024-09-21T05:56:41.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.996+0000 7efefca00640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] conn(0x7efed807b550 0x7efed807d9a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.996+0000 7efef7e00640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efed8073280 0x7efed8075790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efee40016f0 tx=0x7efee4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.996+0000 7efeefe00640 1 -- 172.21.15.50:0/2099638620 <== osd.6 v2:172.21.15.88:6818/2341496649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efee8007e50 con 0x7efed807b550 2024-09-21T05:56:41.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:40.996+0000 7fbf9ca00640 1 -- 172.21.15.50:0/4075822017 --> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf90052260 con 0x7fbf7807b480 2024-09-21T05:56:41.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7fbf8b400640 1 -- 172.21.15.50:0/4075822017 <== osd.2 v2:172.21.15.50:6824/229343709 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbf90052260 con 0x7fbf7807b480 2024-09-21T05:56:41.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7fbf9ca00640 1 -- 172.21.15.50:0/4075822017 >> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] conn(0x7fbf7807b480 msgr2=0x7fbf7807d8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7fbf9ca00640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] conn(0x7fbf7807b480 0x7fbf7807d8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7fbf9ca00640 1 -- 172.21.15.50:0/4075822017 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbf78073230 msgr2=0x7fbf78075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7fbf9ca00640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbf78073230 0x7fbf78075740 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c000f70 tx=0x7fbf8c009040 comp rx=0 tx=0).stop 2024-09-21T05:56:41.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7fbf9ca00640 1 -- 172.21.15.50:0/4075822017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf90052620 msgr2=0x7fbf900dce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7fbf9ca00640 1 --2- 172.21.15.50:0/4075822017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf90052620 0x7fbf900dce90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf98066c60 tx=0x7fbf9806a1d0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7fbf97e00640 1 -- 172.21.15.50:0/4075822017 reap_dead start 2024-09-21T05:56:41.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7f21cf400640 1 -- 172.21.15.50:0/2197970352 <== osd.7 v2:172.21.15.88:6810/4250480352 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f21d8151980 con 0x7f21b807b540 2024-09-21T05:56:41.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7f21de000640 1 -- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] conn(0x7f21b807b540 msgr2=0x7f21b807d990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7f21de000640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] conn(0x7f21b807b540 0x7f21b807d990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7f21de000640 1 -- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21b80732d0 msgr2=0x7f21b80757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7f21de000640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21b80732d0 0x7f21b80757e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f21c80af000 tx=0x7f21c8019680 comp rx=0 tx=0).stop 2024-09-21T05:56:41.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7f21de000640 1 -- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21d80cdbb0 msgr2=0x7f21d814def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7f21de000640 1 --2- 172.21.15.50:0/2197970352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21d80cdbb0 0x7f21d814def0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f21c400c460 tx=0x7f21c400c920 comp rx=0 tx=0).stop 2024-09-21T05:56:41.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7fbf9ca00640 1 -- 172.21.15.50:0/4075822017 shutdown_connections 2024-09-21T05:56:41.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7fbf9ca00640 1 -- 172.21.15.50:0/4075822017 >> 172.21.15.50:0/4075822017 conn(0x7fbf9000e400 msgr2=0x7fbf900de860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7f21d7e00640 1 -- 172.21.15.50:0/2197970352 reap_dead start 2024-09-21T05:56:41.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7f21de000640 1 -- 172.21.15.50:0/2197970352 shutdown_connections 2024-09-21T05:56:41.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7f21de000640 1 -- 172.21.15.50:0/2197970352 >> 172.21.15.50:0/2197970352 conn(0x7f21d80b2040 msgr2=0x7f21d81579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.000+0000 7fbf9ca00640 1 -- 172.21.15.50:0/4075822017 shutdown_connections 2024-09-21T05:56:41.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.004+0000 7fbf9ca00640 1 -- 172.21.15.50:0/4075822017 wait complete. 2024-09-21T05:56:41.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.004+0000 7f21de000640 1 -- 172.21.15.50:0/2197970352 shutdown_connections 2024-09-21T05:56:41.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.004+0000 7f21de000640 1 -- 172.21.15.50:0/2197970352 wait complete. 2024-09-21T05:56:41.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.004+0000 7f8c3ec00640 1 -- 172.21.15.50:0/848922211 --> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8c30056260 con 0x7f8c1807b5a0 2024-09-21T05:56:41.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f8c35600640 1 -- 172.21.15.50:0/848922211 <== osd.1 v2:172.21.15.50:6800/3048348933 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8c30056260 con 0x7f8c1807b5a0 2024-09-21T05:56:41.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f8c3ec00640 1 -- 172.21.15.50:0/848922211 >> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] conn(0x7f8c1807b5a0 msgr2=0x7f8c1807d9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f8c3ec00640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] conn(0x7f8c1807b5a0 0x7f8c1807d9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f8c3ec00640 1 -- 172.21.15.50:0/848922211 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8c180732d0 msgr2=0x7f8c180757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f8c3ec00640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8c180732d0 0x7f8c180757e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c240af000 tx=0x7f8c24019520 comp rx=0 tx=0).stop 2024-09-21T05:56:41.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f8c3ec00640 1 -- 172.21.15.50:0/848922211 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c20004810 msgr2=0x7f8c3000c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f8c3ec00640 1 --2- 172.21.15.50:0/848922211 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c20004810 0x7f8c3000c160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c000c10 tx=0x7f8c2c003a90 comp rx=0 tx=0).stop 2024-09-21T05:56:41.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f8c3e200640 1 -- 172.21.15.50:0/848922211 reap_dead start 2024-09-21T05:56:41.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f8c3ec00640 1 -- 172.21.15.50:0/848922211 shutdown_connections 2024-09-21T05:56:41.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f8c3ec00640 1 -- 172.21.15.50:0/848922211 >> 172.21.15.50:0/848922211 conn(0x7f8c3001eac0 msgr2=0x7f8c300df320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f8c3ec00640 1 -- 172.21.15.50:0/848922211 shutdown_connections 2024-09-21T05:56:41.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f8c3ec00640 1 -- 172.21.15.50:0/848922211 wait complete. 2024-09-21T05:56:41.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.008+0000 7f9829200640 1 -- 172.21.15.50:0/2398447448 --> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f97fc002de0 con 0x7f980407b4d0 2024-09-21T05:56:41.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.012+0000 7f981a000640 1 -- 172.21.15.50:0/2398447448 <== osd.4 v2:172.21.15.88:6826/2441482427 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f97fc002de0 con 0x7f980407b4d0 2024-09-21T05:56:41.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.012+0000 7fd8ffe00640 1 -- 172.21.15.50:0/939233 --> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd8f80e2fa0 con 0x7fd8e007b520 2024-09-21T05:56:41.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.012+0000 7f9829200640 1 -- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] conn(0x7f980407b4d0 msgr2=0x7f980407d920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.012+0000 7f9829200640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] conn(0x7f980407b4d0 0x7f980407d920 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.012+0000 7fd8f6a00640 1 -- 172.21.15.50:0/939233 <== osd.0 v2:172.21.15.50:6808/1152796349 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd8f80e2fa0 con 0x7fd8e007b520 2024-09-21T05:56:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.012+0000 7f9829200640 1 -- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9804073280 msgr2=0x7f9804075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.012+0000 7f9829200640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9804073280 0x7f9804075790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f981401af00 tx=0x7f98140a8040 comp rx=0 tx=0).stop 2024-09-21T05:56:41.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.012+0000 7f9829200640 1 -- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f982406fc90 msgr2=0x7f98241567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.012+0000 7f9829200640 1 --2- 172.21.15.50:0/2398447448 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f982406fc90 0x7f98241567c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9824071ac0 tx=0x7f981000cba0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.012+0000 7f9822800640 1 -- 172.21.15.50:0/2398447448 reap_dead start 2024-09-21T05:56:41.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.012+0000 7f9829200640 1 -- 172.21.15.50:0/2398447448 shutdown_connections 2024-09-21T05:56:41.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.012+0000 7f9829200640 1 -- 172.21.15.50:0/2398447448 >> 172.21.15.50:0/2398447448 conn(0x7f98240b35f0 msgr2=0x7f98240b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7fd8ffe00640 1 -- 172.21.15.50:0/939233 >> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] conn(0x7fd8e007b520 msgr2=0x7fd8e007d970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7fd8ffe00640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] conn(0x7fd8e007b520 0x7fd8e007d970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7fd8ffe00640 1 -- 172.21.15.50:0/939233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8e0073250 msgr2=0x7fd8e0075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7fd8ffe00640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8e0073250 0x7fd8e0075760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80d8880 tx=0x7fd8ec005cc0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7fd8ffe00640 1 -- 172.21.15.50:0/939233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f80514e0 msgr2=0x7fd8f80e2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7fd8ffe00640 1 --2- 172.21.15.50:0/939233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f80514e0 0x7fd8f80e2670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000dc60 tx=0x7fd8f000d3d0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7fd8ff400640 1 -- 172.21.15.50:0/939233 reap_dead start 2024-09-21T05:56:41.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7f9829200640 1 -- 172.21.15.50:0/2398447448 shutdown_connections 2024-09-21T05:56:41.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7f9829200640 1 -- 172.21.15.50:0/2398447448 wait complete. 2024-09-21T05:56:41.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7fd8ffe00640 1 -- 172.21.15.50:0/939233 shutdown_connections 2024-09-21T05:56:41.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7fd8ffe00640 1 -- 172.21.15.50:0/939233 >> 172.21.15.50:0/939233 conn(0x7fd8f801fbf0 msgr2=0x7fd8f80ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7fd8ffe00640 1 -- 172.21.15.50:0/939233 shutdown_connections 2024-09-21T05:56:41.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.016+0000 7fd8ffe00640 1 -- 172.21.15.50:0/939233 wait complete. 2024-09-21T05:56:41.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.020+0000 7f3832600640 1 -- 172.21.15.50:0/4156933243 --> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3804002de0 con 0x7f380c07b470 2024-09-21T05:56:41.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f3823400640 1 -- 172.21.15.50:0/4156933243 <== osd.5 v2:172.21.15.88:6802/3846197193 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3804002de0 con 0x7f380c07b470 2024-09-21T05:56:41.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f9064c00640 1 -- 172.21.15.50:0/2861875627 --> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f907015ff60 con 0x7f904c07b540 2024-09-21T05:56:41.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f9067400640 1 -- 172.21.15.50:0/2861875627 <== osd.3 v2:172.21.15.50:6809/599598977 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f907015ff60 con 0x7f904c07b540 2024-09-21T05:56:41.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7efefea00640 1 -- 172.21.15.50:0/2099638620 --> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efef815c440 con 0x7efed807b550 2024-09-21T05:56:41.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f3832600640 1 -- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] conn(0x7f380c07b470 msgr2=0x7f380c07d8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f3832600640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] conn(0x7f380c07b470 0x7f380c07d8c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f3832600640 1 -- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f380c073200 msgr2=0x7f380c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f3832600640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f380c073200 0x7f380c075710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f381c0b0000 tx=0x7f381c01c4c0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f3832600640 1 -- 172.21.15.50:0/4156933243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3810004810 msgr2=0x7f382c149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f3832600640 1 --2- 172.21.15.50:0/4156933243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3810004810 0x7f382c149c00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f382400c700 tx=0x7f382400cbc0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f382be00640 1 -- 172.21.15.50:0/4156933243 reap_dead start 2024-09-21T05:56:41.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f3832600640 1 -- 172.21.15.50:0/4156933243 shutdown_connections 2024-09-21T05:56:41.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f3832600640 1 -- 172.21.15.50:0/4156933243 >> 172.21.15.50:0/4156933243 conn(0x7f382c0b2040 msgr2=0x7f382c0b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7efeefe00640 1 -- 172.21.15.50:0/2099638620 <== osd.6 v2:172.21.15.88:6818/2341496649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efef815c440 con 0x7efed807b550 2024-09-21T05:56:41.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f9064c00640 1 -- 172.21.15.50:0/2861875627 >> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] conn(0x7f904c07b540 msgr2=0x7f904c07d990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f9064c00640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] conn(0x7f904c07b540 0x7f904c07d990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f9064c00640 1 -- 172.21.15.50:0/2861875627 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f904c0732d0 msgr2=0x7f904c0757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f9064c00640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f904c0732d0 0x7f904c0757e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9070208580 tx=0x7f9058009040 comp rx=0 tx=0).stop 2024-09-21T05:56:41.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f9064c00640 1 -- 172.21.15.50:0/2861875627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f907006fd70 msgr2=0x7f9070204880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f9064c00640 1 --2- 172.21.15.50:0/2861875627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f907006fd70 0x7f9070204880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f906000c3c0 tx=0x7f906000c880 comp rx=0 tx=0).stop 2024-09-21T05:56:41.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.024+0000 7f906fe00640 1 -- 172.21.15.50:0/2861875627 reap_dead start 2024-09-21T05:56:41.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7efefea00640 1 -- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] conn(0x7efed807b550 msgr2=0x7efed807d9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7efefea00640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] conn(0x7efed807b550 0x7efed807d9a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7f3832600640 1 -- 172.21.15.50:0/4156933243 shutdown_connections 2024-09-21T05:56:41.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7f9064c00640 1 -- 172.21.15.50:0/2861875627 shutdown_connections 2024-09-21T05:56:41.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7f9064c00640 1 -- 172.21.15.50:0/2861875627 >> 172.21.15.50:0/2861875627 conn(0x7f907006e4a0 msgr2=0x7f9070076140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7f3832600640 1 -- 172.21.15.50:0/4156933243 wait complete. 2024-09-21T05:56:41.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7efefea00640 1 -- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efed8073280 msgr2=0x7efed8075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7efefea00640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efed8073280 0x7efed8075790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efee40016f0 tx=0x7efee4009040 comp rx=0 tx=0).stop 2024-09-21T05:56:41.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7efefea00640 1 -- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef80b57c0 msgr2=0x7efef815e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7efefea00640 1 --2- 172.21.15.50:0/2099638620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef80b57c0 0x7efef815e160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efee800d6b0 tx=0x7efee800db70 comp rx=0 tx=0).stop 2024-09-21T05:56:41.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7efefca00640 1 -- 172.21.15.50:0/2099638620 reap_dead start 2024-09-21T05:56:41.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7efefea00640 1 -- 172.21.15.50:0/2099638620 shutdown_connections 2024-09-21T05:56:41.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7efefea00640 1 -- 172.21.15.50:0/2099638620 >> 172.21.15.50:0/2099638620 conn(0x7efef80b3b30 msgr2=0x7efef814fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7f9064c00640 1 -- 172.21.15.50:0/2861875627 shutdown_connections 2024-09-21T05:56:41.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7f9064c00640 1 -- 172.21.15.50:0/2861875627 wait complete. 2024-09-21T05:56:41.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7efefea00640 1 -- 172.21.15.50:0/2099638620 shutdown_connections 2024-09-21T05:56:41.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.028+0000 7efefea00640 1 -- 172.21.15.50:0/2099638620 wait complete. 2024-09-21T05:56:41.045 INFO:teuthology.orchestra.run.smithi050.stdout:64424509447 2024-09-21T05:56:41.045 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T05:56:41.054 INFO:teuthology.orchestra.run.smithi050.stdout:64424509448 2024-09-21T05:56:41.054 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T05:56:41.055 INFO:teuthology.orchestra.run.smithi050.stdout:60129542151 2024-09-21T05:56:41.055 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T05:56:41.061 INFO:teuthology.orchestra.run.smithi050.stdout:64424509447 2024-09-21T05:56:41.062 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T05:56:41.065 INFO:teuthology.orchestra.run.smithi050.stdout:64424509446 2024-09-21T05:56:41.066 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T05:56:41.071 INFO:teuthology.orchestra.run.smithi050.stdout:64424509446 2024-09-21T05:56:41.071 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T05:56:41.074 INFO:teuthology.orchestra.run.smithi050.stdout:64424509446 2024-09-21T05:56:41.074 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T05:56:41.075 INFO:teuthology.orchestra.run.smithi050.stdout:64424509447 2024-09-21T05:56:41.075 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T05:56:41.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.308+0000 7f206c800640 1 -- 172.21.15.50:0/3292102918 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f206003ab60 msgr2=0x7f206003af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.308+0000 7f206c800640 1 --2- 172.21.15.50:0/3292102918 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f206003ab60 0x7f206003af70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2054009900 tx=0x7f205401c6b0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.308+0000 7f206c800640 1 -- 172.21.15.50:0/3292102918 shutdown_connections 2024-09-21T05:56:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.308+0000 7f206c800640 1 --2- 172.21.15.50:0/3292102918 >> v2:172.21.15.88:3300/0 conn(0x7f2060034700 0x7f2060034b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.308+0000 7f206c800640 1 --2- 172.21.15.50:0/3292102918 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f206003ab60 0x7f206003af70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.308+0000 7f206c800640 1 -- 172.21.15.50:0/3292102918 >> 172.21.15.50:0/3292102918 conn(0x7f206001fbc0 msgr2=0x7f2060056950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.308+0000 7f206c800640 1 -- 172.21.15.50:0/3292102918 shutdown_connections 2024-09-21T05:56:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.312+0000 7f206c800640 1 -- 172.21.15.50:0/3292102918 wait complete. 2024-09-21T05:56:41.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.316+0000 7f206c800640 1 Processor -- start 2024-09-21T05:56:41.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.316+0000 7f206c800640 1 -- start start 2024-09-21T05:56:41.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.316+0000 7f206c800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2060034700 0x7f20600eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f206c800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f206003a040 0x7f206018acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f2069e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2060034700 0x7f20600eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f2069e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2060034700 0x7f20600eafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:47364/0 (socket says 172.21.15.50:47364) 2024-09-21T05:56:41.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f2069e00640 1 -- 172.21.15.50:0/3266252022 learned_addr learned my addr 172.21.15.50:0/3266252022 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:41.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f206c800640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f206003ab60 0x7f2060189b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2060008450 con 0x7f206003a040 2024-09-21T05:56:41.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f641a200640 1 -- 172.21.15.50:0/2390876821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6414155ad0 msgr2=0x7f641406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f641a200640 1 --2- 172.21.15.50:0/2390876821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6414155ad0 0x7f641406e520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6404009bc0 tx=0x7f640401cbc0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f206a800640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f206003ab60 0x7f2060189b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f2069400640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f206003a040 0x7f206018acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f641a200640 1 -- 172.21.15.50:0/2390876821 shutdown_connections 2024-09-21T05:56:41.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f641a200640 1 --2- 172.21.15.50:0/2390876821 >> v2:172.21.15.50:3300/0 conn(0x7f64140d29c0 0x7f64140d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f641a200640 1 --2- 172.21.15.50:0/2390876821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6414155ad0 0x7f641406e520 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f641a200640 1 -- 172.21.15.50:0/2390876821 >> 172.21.15.50:0/2390876821 conn(0x7f64140708c0 msgr2=0x7f6414070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f20600082d0 con 0x7f2060034700 2024-09-21T05:56:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f20600085d0 con 0x7f206003ab60 2024-09-21T05:56:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f206a800640 1 -- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2060034700 msgr2=0x7f20600eafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f206a800640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2060034700 0x7f20600eafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f641a200640 1 -- 172.21.15.50:0/2390876821 shutdown_connections 2024-09-21T05:56:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f641a200640 1 -- 172.21.15.50:0/2390876821 wait complete. 2024-09-21T05:56:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f206a800640 1 -- 172.21.15.50:0/3266252022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f206003a040 msgr2=0x7f206018acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f206a800640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f206003a040 0x7f206018acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f206a800640 1 -- 172.21.15.50:0/3266252022 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206018b230 con 0x7f206003ab60 2024-09-21T05:56:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.320+0000 7f2069400640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f206003a040 0x7f206018acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.324+0000 7f206a800640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f206003ab60 0x7f2060189b30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2064052910 tx=0x7f206406a850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.324+0000 7f205e000640 1 -- 172.21.15.50:0/3266252022 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206406a980 con 0x7f206003ab60 2024-09-21T05:56:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.324+0000 7f205e000640 1 -- 172.21.15.50:0/3266252022 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2064077240 con 0x7f206003ab60 2024-09-21T05:56:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.324+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20600e9f20 con 0x7f206003ab60 2024-09-21T05:56:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.324+0000 7f205e000640 1 -- 172.21.15.50:0/3266252022 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064095440 con 0x7f206003ab60 2024-09-21T05:56:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.324+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f20600ea4e0 con 0x7f206003ab60 2024-09-21T05:56:41.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.328+0000 7f641a200640 1 Processor -- start 2024-09-21T05:56:41.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.328+0000 7f641a200640 1 -- start start 2024-09-21T05:56:41.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.328+0000 7f205e000640 1 -- 172.21.15.50:0/3266252022 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f20640955e0 con 0x7f206003ab60 2024-09-21T05:56:41.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.328+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2060034b10 con 0x7f206003ab60 2024-09-21T05:56:41.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.328+0000 7f205e000640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2048073200 0x7f2048075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.328+0000 7f641a200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6414074410 0x7f6414156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.328+0000 7f2069e00640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2048073200 0x7f2048075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.328+0000 7f205e000640 1 -- 172.21.15.50:0/3266252022 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f206410ff10 con 0x7f206003ab60 2024-09-21T05:56:41.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.328+0000 7f641a200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64140d29c0 0x7f6414200e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f641a200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6414155ad0 0x7f64141fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f641a200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f641416c4f0 con 0x7f6414155ad0 2024-09-21T05:56:41.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f641a200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f641416c370 con 0x7f64140d29c0 2024-09-21T05:56:41.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f641a200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f641416c670 con 0x7f6414074410 2024-09-21T05:56:41.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f6413400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6414074410 0x7f6414156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f2069e00640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2048073200 0x7f2048075710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f20580018c0 tx=0x7f2058001070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f6412a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64140d29c0 0x7f6414200e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f6412a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64140d29c0 0x7f6414200e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:47394/0 (socket says 172.21.15.50:47394) 2024-09-21T05:56:41.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f6412a00640 1 -- 172.21.15.50:0/175808637 learned_addr learned my addr 172.21.15.50:0/175808637 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:41.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f6413e00640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6414155ad0 0x7f64141fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f6413400640 1 -- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64140d29c0 msgr2=0x7f6414200e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f6413400640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64140d29c0 0x7f6414200e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f6413400640 1 -- 172.21.15.50:0/175808637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6414155ad0 msgr2=0x7f64141fe2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f6413400640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6414155ad0 0x7f64141fe2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f6413400640 1 -- 172.21.15.50:0/175808637 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6414201820 con 0x7f6414074410 2024-09-21T05:56:41.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.332+0000 7f6413400640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6414074410 0x7f6414156610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f64140741e0 tx=0x7f64040a5ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f640b400640 1 -- 172.21.15.50:0/175808637 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040ba020 con 0x7f6414074410 2024-09-21T05:56:41.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f640b400640 1 -- 172.21.15.50:0/175808637 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64040a6e50 con 0x7f6414074410 2024-09-21T05:56:41.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f640b400640 1 -- 172.21.15.50:0/175808637 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040b2040 con 0x7f6414074410 2024-09-21T05:56:41.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f205e000640 1 -- 172.21.15.50:0/3266252022 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f206018b230 con 0x7f206003ab60 2024-09-21T05:56:41.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f641a200640 1 -- 172.21.15.50:0/175808637 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6414201b00 con 0x7f6414074410 2024-09-21T05:56:41.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f641a200640 1 -- 172.21.15.50:0/175808637 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64141589a0 con 0x7f6414074410 2024-09-21T05:56:41.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f8f21c00640 1 -- 172.21.15.50:0/3771911894 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c1625f0 msgr2=0x7f8f1c162a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f641a200640 1 -- 172.21.15.50:0/175808637 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6414158c00 con 0x7f6414074410 2024-09-21T05:56:41.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f8f21c00640 1 --2- 172.21.15.50:0/3771911894 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c1625f0 0x7f8f1c162a00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c15fa90 tx=0x7f8f0801d2d0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f8f21c00640 1 -- 172.21.15.50:0/3771911894 shutdown_connections 2024-09-21T05:56:41.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f8f21c00640 1 --2- 172.21.15.50:0/3771911894 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c1625f0 0x7f8f1c162a00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f8f21c00640 1 --2- 172.21.15.50:0/3771911894 >> v2:172.21.15.88:3300/0 conn(0x7f8f1c0d0ce0 0x7f8f1c0d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f8f21c00640 1 -- 172.21.15.50:0/3771911894 >> 172.21.15.50:0/3771911894 conn(0x7f8f1c073a10 msgr2=0x7f8f1c06cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.336+0000 7f8f21c00640 1 -- 172.21.15.50:0/3771911894 shutdown_connections 2024-09-21T05:56:41.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f01f1200640 1 -- 172.21.15.50:0/3226819499 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e4005cc0 msgr2=0x7f01e40060d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f01f1200640 1 --2- 172.21.15.50:0/3226819499 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e4005cc0 0x7f01e40060d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009900 tx=0x7f01d801c850 comp rx=0 tx=0).stop 2024-09-21T05:56:41.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f640b400640 1 -- 172.21.15.50:0/175808637 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6404004030 con 0x7f6414074410 2024-09-21T05:56:41.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f8f21c00640 1 -- 172.21.15.50:0/3771911894 wait complete. 2024-09-21T05:56:41.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f01f1200640 1 -- 172.21.15.50:0/3226819499 shutdown_connections 2024-09-21T05:56:41.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f01f1200640 1 --2- 172.21.15.50:0/3226819499 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e4005cc0 0x7f01e40060d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f01f1200640 1 --2- 172.21.15.50:0/3226819499 >> v2:172.21.15.88:3300/0 conn(0x7f01e400a400 0x7f01e400a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f01f1200640 1 -- 172.21.15.50:0/3226819499 >> 172.21.15.50:0/3226819499 conn(0x7f01e40513f0 msgr2=0x7f01e4051830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f640b400640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63f0073250 0x7f63f0075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f640b400640 1 -- 172.21.15.50:0/175808637 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6404002020 con 0x7f6414074410 2024-09-21T05:56:41.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f6412a00640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63f0073250 0x7f63f0075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f01f1200640 1 -- 172.21.15.50:0/3226819499 shutdown_connections 2024-09-21T05:56:41.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f6412a00640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63f0073250 0x7f63f0075760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009a00 tx=0x7f63fc009110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.340+0000 7f01f1200640 1 -- 172.21.15.50:0/3226819499 wait complete. 2024-09-21T05:56:41.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.344+0000 7f8f21c00640 1 Processor -- start 2024-09-21T05:56:41.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.344+0000 7f8f21c00640 1 -- start start 2024-09-21T05:56:41.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.344+0000 7f01f1200640 1 Processor -- start 2024-09-21T05:56:41.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.344+0000 7f01f1200640 1 -- start start 2024-09-21T05:56:41.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f640b400640 1 -- 172.21.15.50:0/175808637 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f64040167b0 con 0x7f6414074410 2024-09-21T05:56:41.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f21c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0b5f20 0x7f8f1c159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f21c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c0d0ce0 0x7f8f1c0b9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f21c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f1c1625f0 0x7f8f1c159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f21c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c0d3320 con 0x7f8f1c0b5f20 2024-09-21T05:56:41.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f21c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c0d31a0 con 0x7f8f1c1625f0 2024-09-21T05:56:41.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f21c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1c16c790 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f21200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f1c1625f0 0x7f8f1c159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f1b400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c0d0ce0 0x7f8f1c0b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f1b400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c0d0ce0 0x7f8f1c0b9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:51130/0 (socket says 172.21.15.50:51130) 2024-09-21T05:56:41.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f1b400640 1 -- 172.21.15.50:0/1434327442 learned_addr learned my addr 172.21.15.50:0/1434327442 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:41.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f1be00640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0b5f20 0x7f8f1c159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f1b400640 1 -- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f1c1625f0 msgr2=0x7f8f1c159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f1b400640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f1c1625f0 0x7f8f1c159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f1b400640 1 -- 172.21.15.50:0/1434327442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0b5f20 msgr2=0x7f8f1c159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f1b400640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0b5f20 0x7f8f1c159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f1b400640 1 -- 172.21.15.50:0/1434327442 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c0b84e0 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f1be00640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0b5f20 0x7f8f1c159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:41.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f21200640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f1c1625f0 0x7f8f1c159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:41.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f1b400640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c0d0ce0 0x7f8f1c0b9580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009230 tx=0x7f8f08005130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f13e00640 1 -- 172.21.15.50:0/1434327442 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08005320 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f01f1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01e4005cc0 0x7f01e40e2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f01f1200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e400a400 0x7f01e40da160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f01f1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e40e0f70 0x7f01e40da6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01e9400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e400a400 0x7f01e40da160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01ea800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e40e0f70 0x7f01e40da6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01e9400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e400a400 0x7f01e40da160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:51144/0 (socket says 172.21.15.50:51144) 2024-09-21T05:56:41.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01e9400640 1 -- 172.21.15.50:0/2964771431 learned_addr learned my addr 172.21.15.50:0/2964771431 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:41.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01e9e00640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01e4005cc0 0x7f01e40e2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f01e40f11a0 con 0x7f01e40e0f70 2024-09-21T05:56:41.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f01e40ecc10 con 0x7f01e4005cc0 2024-09-21T05:56:41.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f01e40f1320 con 0x7f01e400a400 2024-09-21T05:56:41.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f13e00640 1 -- 172.21.15.50:0/1434327442 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f080055b0 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.348+0000 7f8f13e00640 1 -- 172.21.15.50:0/1434327442 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08005320 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f8f21c00640 1 -- 172.21.15.50:0/1434327442 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1c0b87c0 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f8f21c00640 1 -- 172.21.15.50:0/1434327442 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1c0b8d80 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01ea800640 1 -- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e400a400 msgr2=0x7f01e40da160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01ea800640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e400a400 0x7f01e40da160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01ea800640 1 -- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01e4005cc0 msgr2=0x7f01e40e2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01ea800640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01e4005cc0 0x7f01e40e2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01ea800640 1 -- 172.21.15.50:0/2964771431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e401e6c0 con 0x7f01e40e0f70 2024-09-21T05:56:41.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01e9400640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e400a400 0x7f01e40da160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:41.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01e9e00640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01e4005cc0 0x7f01e40e2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:41.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f3d85c00640 1 -- 172.21.15.50:0/4121610849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d801557e0 msgr2=0x7f3d800ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f3d85c00640 1 --2- 172.21.15.50:0/4121610849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d801557e0 0x7f3d800ae8e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009900 tx=0x7f3d6c01c5b0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f3d85c00640 1 -- 172.21.15.50:0/4121610849 shutdown_connections 2024-09-21T05:56:41.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f3d85c00640 1 --2- 172.21.15.50:0/4121610849 >> v2:172.21.15.50:3300/0 conn(0x7f3d800aee20 0x7f3d800b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f3d85c00640 1 --2- 172.21.15.50:0/4121610849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d801557e0 0x7f3d800ae8e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f3d85c00640 1 -- 172.21.15.50:0/4121610849 >> 172.21.15.50:0/4121610849 conn(0x7f3d800b2040 msgr2=0x7f3d800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f8f21c00640 1 -- 172.21.15.50:0/1434327442 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1c0d10f0 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01ea800640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e40e0f70 0x7f01e40da6a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f01d80ac000 tx=0x7f01d80a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01e2000640 1 -- 172.21.15.50:0/2964771431 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d80bb070 con 0x7f01e40e0f70 2024-09-21T05:56:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01e2000640 1 -- 172.21.15.50:0/2964771431 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01d8004340 con 0x7f01e40e0f70 2024-09-21T05:56:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.352+0000 7f01e2000640 1 -- 172.21.15.50:0/2964771431 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d80b2040 con 0x7f01e40e0f70 2024-09-21T05:56:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.356+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e41826e0 con 0x7f01e40e0f70 2024-09-21T05:56:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.356+0000 7f3d85c00640 1 -- 172.21.15.50:0/4121610849 shutdown_connections 2024-09-21T05:56:41.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.356+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e4182ca0 con 0x7f01e40e0f70 2024-09-21T05:56:41.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.356+0000 7f3d85c00640 1 -- 172.21.15.50:0/4121610849 wait complete. 2024-09-21T05:56:41.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.356+0000 7f8f13e00640 1 -- 172.21.15.50:0/1434327442 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8f080bf020 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.356+0000 7f8f13e00640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8efc073400 0x7f8efc075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.356+0000 7f8f13e00640 1 -- 172.21.15.50:0/1434327442 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8f0812e5c0 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.356+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e40060d0 con 0x7f01e40e0f70 2024-09-21T05:56:41.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.356+0000 7ff881200640 1 -- 172.21.15.50:0/1343532888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff87c0703a0 msgr2=0x7ff87c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.356+0000 7f8f1be00640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8efc073400 0x7f8efc075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.356+0000 7ff881200640 1 --2- 172.21.15.50:0/1343532888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff87c0703a0 0x7ff87c06d200 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff874009c80 tx=0x7ff87401cb50 comp rx=0 tx=0).stop 2024-09-21T05:56:41.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.356+0000 7ff881200640 1 -- 172.21.15.50:0/1343532888 shutdown_connections 2024-09-21T05:56:41.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.360+0000 7ff881200640 1 --2- 172.21.15.50:0/1343532888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff87c0703a0 0x7ff87c06d200 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.360+0000 7ff881200640 1 --2- 172.21.15.50:0/1343532888 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff87c159130 0x7ff87c15bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.360+0000 7f8f1be00640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8efc073400 0x7f8efc075910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c159ea0 tx=0x7f8f0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.360+0000 7ff881200640 1 -- 172.21.15.50:0/1343532888 >> 172.21.15.50:0/1343532888 conn(0x7ff87c0b35f0 msgr2=0x7ff87c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.360+0000 7f3d85c00640 1 Processor -- start 2024-09-21T05:56:41.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.360+0000 7f3d85c00640 1 -- start start 2024-09-21T05:56:41.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.360+0000 7ff881200640 1 -- 172.21.15.50:0/1343532888 shutdown_connections 2024-09-21T05:56:41.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.360+0000 7ff881200640 1 -- 172.21.15.50:0/1343532888 wait complete. 2024-09-21T05:56:41.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.360+0000 7f8f13e00640 1 -- 172.21.15.50:0/1434327442 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8f080164f0 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.360+0000 7f01e2000640 1 -- 172.21.15.50:0/2964771431 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f01d8003860 con 0x7f01e40e0f70 2024-09-21T05:56:41.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.360+0000 7f01e2000640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f01cc073250 0x7f01cc075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.360+0000 7f01e2000640 1 -- 172.21.15.50:0/2964771431 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f01d812d8d0 con 0x7f01e40e0f70 2024-09-21T05:56:41.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7f3d85c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d800aee20 0x7f3d80154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7ff881200640 1 Processor -- start 2024-09-21T05:56:41.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7f01e9e00640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f01cc073250 0x7f01cc075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7f3d85c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d801557e0 0x7f3d8014cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7ff881200640 1 -- start start 2024-09-21T05:56:41.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7f3d85c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d8015f800 0x7f3d80147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7f3d85c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3d800ccbb0 con 0x7f3d800aee20 2024-09-21T05:56:41.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7f3d85c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3d800cca30 con 0x7f3d801557e0 2024-09-21T05:56:41.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7f3d85c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3d800ccd30 con 0x7f3d8015f800 2024-09-21T05:56:41.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7f3d7f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d8015f800 0x7f3d80147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7f3d7e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d801557e0 0x7f3d8014cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7f3d7e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d801557e0 0x7f3d8014cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:47414/0 (socket says 172.21.15.50:47414) 2024-09-21T05:56:41.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7f3d7e000640 1 -- 172.21.15.50:0/3122281415 learned_addr learned my addr 172.21.15.50:0/3122281415 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:41.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.364+0000 7f3d7ea00640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d800aee20 0x7f3d80154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff881200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c06cdf0 0x7ff87c150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7f01e9e00640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f01cc073250 0x7f01cc075760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f01ec06bb90 tx=0x7f01ec06abb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7f3d7ea00640 1 -- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d8015f800 msgr2=0x7f3d80147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7f3d7ea00640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d8015f800 0x7f3d80147400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7f3d7ea00640 1 -- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d801557e0 msgr2=0x7f3d8014cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7f3d7ea00640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d801557e0 0x7f3d8014cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7f3d7ea00640 1 -- 172.21.15.50:0/3122281415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d801553e0 con 0x7f3d800aee20 2024-09-21T05:56:41.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7f3d7e000640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d801557e0 0x7f3d8014cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:41.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff879e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c06cdf0 0x7ff87c150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7f3d7ea00640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d800aee20 0x7f3d80154ea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000b410 tx=0x7f3d7000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff879e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c06cdf0 0x7ff87c150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48548/0 (socket says 172.21.15.50:48548) 2024-09-21T05:56:41.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff879e00640 1 -- 172.21.15.50:0/441636131 learned_addr learned my addr 172.21.15.50:0/441636131 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:41.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff881200640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff87c0703a0 0x7ff87c154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff881200640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff87c159130 0x7ff87c14b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c168e90 con 0x7ff87c06cdf0 2024-09-21T05:56:41.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c168d10 con 0x7ff87c159130 2024-09-21T05:56:41.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff87c169010 con 0x7ff87c0703a0 2024-09-21T05:56:41.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff879e00640 1 -- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff87c0703a0 msgr2=0x7ff87c154b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:56:41.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff879e00640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff87c0703a0 0x7ff87c154b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff879e00640 1 -- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff87c159130 msgr2=0x7ff87c14b350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:56:41.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff879e00640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff87c159130 0x7ff87c14b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.368+0000 7ff879e00640 1 -- 172.21.15.50:0/441636131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c14c320 con 0x7ff87c06cdf0 2024-09-21T05:56:41.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.372+0000 7ff879e00640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c06cdf0 0x7ff87c150f30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0058d0 tx=0x7ff86c00feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.372+0000 7f3d76a00640 1 -- 172.21.15.50:0/3122281415 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70015020 con 0x7f3d800aee20 2024-09-21T05:56:41.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.372+0000 7f01e2000640 1 -- 172.21.15.50:0/2964771431 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f01d80f2290 con 0x7f01e40e0f70 2024-09-21T05:56:41.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.372+0000 7ff872000640 1 -- 172.21.15.50:0/441636131 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c01b070 con 0x7ff87c06cdf0 2024-09-21T05:56:41.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.372+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87c151470 con 0x7ff87c06cdf0 2024-09-21T05:56:41.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.372+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff87c154080 con 0x7ff87c06cdf0 2024-09-21T05:56:41.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.372+0000 7f3d85c00640 1 -- 172.21.15.50:0/3122281415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d80146c90 con 0x7f3d800aee20 2024-09-21T05:56:41.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.372+0000 7f3d85c00640 1 -- 172.21.15.50:0/3122281415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d80072640 con 0x7f3d800aee20 2024-09-21T05:56:41.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.372+0000 7f3d76a00640 1 -- 172.21.15.50:0/3122281415 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d70011920 con 0x7f3d800aee20 2024-09-21T05:56:41.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.372+0000 7f3d76a00640 1 -- 172.21.15.50:0/3122281415 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70011d10 con 0x7f3d800aee20 2024-09-21T05:56:41.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.372+0000 7ff872000640 1 -- 172.21.15.50:0/441636131 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff86c00f310 con 0x7ff87c06cdf0 2024-09-21T05:56:41.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.372+0000 7ff872000640 1 -- 172.21.15.50:0/441636131 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c01b070 con 0x7ff87c06cdf0 2024-09-21T05:56:41.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.376+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff87c06d200 con 0x7ff87c06cdf0 2024-09-21T05:56:41.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.376+0000 7f3d85c00640 1 -- 172.21.15.50:0/3122281415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d580052a0 con 0x7f3d800aee20 2024-09-21T05:56:41.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.376+0000 7f3d76a00640 1 -- 172.21.15.50:0/3122281415 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3d70018020 con 0x7f3d800aee20 2024-09-21T05:56:41.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.380+0000 7f3d76a00640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d60073200 0x7f3d60075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.380+0000 7f3d76a00640 1 -- 172.21.15.50:0/3122281415 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3d70094140 con 0x7f3d800aee20 2024-09-21T05:56:41.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.380+0000 7f3d7e000640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d60073200 0x7f3d60075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.380+0000 7ff872000640 1 -- 172.21.15.50:0/441636131 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff86c009a30 con 0x7ff87c06cdf0 2024-09-21T05:56:41.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.380+0000 7ff872000640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff85c073600 0x7ff85c075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.384+0000 7ff872000640 1 -- 172.21.15.50:0/441636131 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff86c00f020 con 0x7ff87c06cdf0 2024-09-21T05:56:41.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.384+0000 7ff879400640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff85c073600 0x7ff85c075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.384+0000 7ff879400640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff85c073600 0x7ff85c075b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff868006fd0 tx=0x7ff868006b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.380+0000 7f3d7e000640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d60073200 0x7f3d60075710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c01c3e0 tx=0x7f3d6c01adf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.392+0000 7ff872000640 1 -- 172.21.15.50:0/441636131 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff86c063190 con 0x7ff87c06cdf0 2024-09-21T05:56:41.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.396+0000 7f3d76a00640 1 -- 172.21.15.50:0/3122281415 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3d7001f020 con 0x7f3d800aee20 2024-09-21T05:56:41.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.420+0000 7fd439c00640 1 -- 172.21.15.50:0/1793075031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43415e4e0 msgr2=0x7fd43415e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.420+0000 7fd439c00640 1 --2- 172.21.15.50:0/1793075031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43415e4e0 0x7fd43415e8f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd4340b5020 tx=0x7fd42c01c8e0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.420+0000 7fd439c00640 1 -- 172.21.15.50:0/1793075031 shutdown_connections 2024-09-21T05:56:41.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.420+0000 7fd439c00640 1 --2- 172.21.15.50:0/1793075031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43415e4e0 0x7fd43415e8f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.420+0000 7fd439c00640 1 --2- 172.21.15.50:0/1793075031 >> v2:172.21.15.88:3300/0 conn(0x7fd4340cd7a0 0x7fd4340cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.420+0000 7fd439c00640 1 --2- 172.21.15.50:0/1793075031 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd43415aa50 0x7fd4340cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.420+0000 7fd439c00640 1 -- 172.21.15.50:0/1793075031 >> 172.21.15.50:0/1793075031 conn(0x7fd4340b2040 msgr2=0x7fd4340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.424+0000 7fd439c00640 1 -- 172.21.15.50:0/1793075031 shutdown_connections 2024-09-21T05:56:41.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.424+0000 7fd439c00640 1 -- 172.21.15.50:0/1793075031 wait complete. 2024-09-21T05:56:41.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.424+0000 7f157e400640 1 -- 172.21.15.50:0/2885056384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157815e5a0 msgr2=0x7f157815e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.424+0000 7f157e400640 1 --2- 172.21.15.50:0/2885056384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157815e5a0 0x7f157815e9b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f15780746f0 tx=0x7f157001ccf0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.424+0000 7f157e400640 1 -- 172.21.15.50:0/2885056384 shutdown_connections 2024-09-21T05:56:41.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.424+0000 7f157e400640 1 --2- 172.21.15.50:0/2885056384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157815e5a0 0x7f157815e9b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.424+0000 7f157e400640 1 --2- 172.21.15.50:0/2885056384 >> v2:172.21.15.88:3300/0 conn(0x7f1578073b00 0x7f1578073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.424+0000 7f157e400640 1 --2- 172.21.15.50:0/2885056384 >> v2:172.21.15.50:3300/0 conn(0x7f1578162500 0x7f157806f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.424+0000 7f157e400640 1 -- 172.21.15.50:0/2885056384 >> 172.21.15.50:0/2885056384 conn(0x7f15780b3b30 msgr2=0x7f15780733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.424+0000 7f157e400640 1 -- 172.21.15.50:0/2885056384 shutdown_connections 2024-09-21T05:56:41.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.424+0000 7f157e400640 1 -- 172.21.15.50:0/2885056384 wait complete. 2024-09-21T05:56:41.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.428+0000 7fd439c00640 1 Processor -- start 2024-09-21T05:56:41.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.428+0000 7fd439c00640 1 -- start start 2024-09-21T05:56:41.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.428+0000 7f157e400640 1 Processor -- start 2024-09-21T05:56:41.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.428+0000 7f157e400640 1 -- start start 2024-09-21T05:56:41.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7fd439c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4340cd7a0 0x7fd4341fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7fd439c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43415aa50 0x7fd43413e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7fd42a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43415aa50 0x7fd43413e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f157e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1578073b00 0x7f15780b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7fd42a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43415aa50 0x7fd43413e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48566/0 (socket says 172.21.15.50:48566) 2024-09-21T05:56:41.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7fd42a000640 1 -- 172.21.15.50:0/2640088353 learned_addr learned my addr 172.21.15.50:0/2640088353 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:41.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7fd439c00640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd43415e4e0 0x7fd43415c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f157e400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157815e5a0 0x7f157814e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f157e400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1578162500 0x7f1578157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f157e400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1578069180 con 0x7f1578162500 2024-09-21T05:56:41.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f157e400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1578069000 con 0x7f1578073b00 2024-09-21T05:56:41.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f157e400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1578069300 con 0x7f157815e5a0 2024-09-21T05:56:41.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd4340b41b0 con 0x7fd43415aa50 2024-09-21T05:56:41.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd4340b4030 con 0x7fd4340cd7a0 2024-09-21T05:56:41.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f1577e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1578162500 0x7f1578157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd434167670 con 0x7fd43415e4e0 2024-09-21T05:56:41.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f1577e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1578162500 0x7f1578157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48580/0 (socket says 172.21.15.50:48580) 2024-09-21T05:56:41.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f1577e00640 1 -- 172.21.15.50:0/534890233 learned_addr learned my addr 172.21.15.50:0/534890233 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:41.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7fd433400640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd43415e4e0 0x7fd43415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f1577e00640 1 -- 172.21.15.50:0/534890233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157815e5a0 msgr2=0x7f157814e400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:56:41.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f1577e00640 1 --2- 172.21.15.50:0/534890233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157815e5a0 0x7f157814e400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f1577e00640 1 -- 172.21.15.50:0/534890233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1578073b00 msgr2=0x7f15780b2820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:56:41.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f1577e00640 1 --2- 172.21.15.50:0/534890233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1578073b00 0x7f15780b2820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.432+0000 7f1577e00640 1 -- 172.21.15.50:0/534890233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1578153040 con 0x7f1578162500 2024-09-21T05:56:41.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7f1577e00640 1 --2- 172.21.15.50:0/534890233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1578162500 0x7f1578157470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1570009bd0 tx=0x7f1570003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7fd432a00640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4340cd7a0 0x7fd4341fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7f156f400640 1 -- 172.21.15.50:0/534890233 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15700bb070 con 0x7f1578162500 2024-09-21T05:56:41.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7fd432a00640 1 -- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd43415e4e0 msgr2=0x7fd43415c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7fd432a00640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd43415e4e0 0x7fd43415c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7fd432a00640 1 -- 172.21.15.50:0/2640088353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43415aa50 msgr2=0x7fd43413e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7fd432a00640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43415aa50 0x7fd43413e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7fd432a00640 1 -- 172.21.15.50:0/2640088353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4341536e0 con 0x7fd4340cd7a0 2024-09-21T05:56:41.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7f156f400640 1 -- 172.21.15.50:0/534890233 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1570004c50 con 0x7f1578162500 2024-09-21T05:56:41.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7f157e400640 1 -- 172.21.15.50:0/534890233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1578153320 con 0x7f1578162500 2024-09-21T05:56:41.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7f156f400640 1 -- 172.21.15.50:0/534890233 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15700b2040 con 0x7f1578162500 2024-09-21T05:56:41.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7f157e400640 1 -- 172.21.15.50:0/534890233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15780b1f80 con 0x7f1578162500 2024-09-21T05:56:41.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7fd432a00640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4340cd7a0 0x7fd4341fcca0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd42400ea90 tx=0x7fd42400ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7fd42b400640 1 -- 172.21.15.50:0/2640088353 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42400cea0 con 0x7fd4340cd7a0 2024-09-21T05:56:41.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7fd42b400640 1 -- 172.21.15.50:0/2640088353 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd424010430 con 0x7fd4340cd7a0 2024-09-21T05:56:41.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7fd42b400640 1 -- 172.21.15.50:0/2640088353 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424010860 con 0x7fd4340cd7a0 2024-09-21T05:56:41.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7f157e400640 1 -- 172.21.15.50:0/534890233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15500052a0 con 0x7f1578162500 2024-09-21T05:56:41.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd434141010 con 0x7fd4340cd7a0 2024-09-21T05:56:41.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.436+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd434155b20 con 0x7fd4340cd7a0 2024-09-21T05:56:41.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.440+0000 7f156f400640 1 -- 172.21.15.50:0/534890233 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1570003fd0 con 0x7f1578162500 2024-09-21T05:56:41.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.444+0000 7f156f400640 1 --2- 172.21.15.50:0/534890233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1554073230 0x7f1554075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.444+0000 7f156f400640 1 -- 172.21.15.50:0/534890233 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f157012ddd0 con 0x7f1578162500 2024-09-21T05:56:41.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.440+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c0052a0 con 0x7fd4340cd7a0 2024-09-21T05:56:41.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.444+0000 7fd42b400640 1 -- 172.21.15.50:0/2640088353 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd424010a00 con 0x7fd4340cd7a0 2024-09-21T05:56:41.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.444+0000 7fd42b400640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd410073200 0x7fd410075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:41.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.444+0000 7fd42b400640 1 -- 172.21.15.50:0/2640088353 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd424094ed0 con 0x7fd4340cd7a0 2024-09-21T05:56:41.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.444+0000 7f1577400640 1 --2- 172.21.15.50:0/534890233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1554073230 0x7f1554075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.444+0000 7f1577400640 1 --2- 172.21.15.50:0/534890233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1554073230 0x7f1554075740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f15780b4bf0 tx=0x7f15600091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.444+0000 7fd42a000640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd410073200 0x7fd410075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:41.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.448+0000 7fd42a000640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd410073200 0x7fd410075710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd434150010 tx=0x7fd41c00a250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:41.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.448+0000 7fd42b400640 1 -- 172.21.15.50:0/2640088353 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd42405f880 con 0x7fd4340cd7a0 2024-09-21T05:56:41.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.452+0000 7f156f400640 1 -- 172.21.15.50:0/534890233 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1570016970 con 0x7f1578162500 2024-09-21T05:56:41.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.708+0000 7f641a200640 1 -- 172.21.15.50:0/175808637 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f641406e160 con 0x7f6414074410 2024-09-21T05:56:41.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.708+0000 7f640b400640 1 -- 172.21.15.50:0/175808637 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f64040f9360 con 0x7f6414074410 2024-09-21T05:56:41.713 INFO:teuthology.orchestra.run.smithi050.stdout:64424509446 2024-09-21T05:56:41.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2060054a60 con 0x7f206003ab60 2024-09-21T05:56:41.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f205e000640 1 -- 172.21.15.50:0/3266252022 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f20640da990 con 0x7f206003ab60 2024-09-21T05:56:41.715 INFO:teuthology.orchestra.run.smithi050.stdout:64424509446 2024-09-21T05:56:41.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f641a200640 1 -- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63f0073250 msgr2=0x7f63f0075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f641a200640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63f0073250 0x7f63f0075760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009a00 tx=0x7f63fc009110 comp rx=0 tx=0).stop 2024-09-21T05:56:41.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f641a200640 1 -- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6414074410 msgr2=0x7f6414156610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f641a200640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6414074410 0x7f6414156610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f64140741e0 tx=0x7f64040a5ae0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f641a200640 1 -- 172.21.15.50:0/175808637 shutdown_connections 2024-09-21T05:56:41.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f641a200640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63f0073250 0x7f63f0075760 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009a00 tx=0x7f63fc009110 comp rx=0 tx=0).stop 2024-09-21T05:56:41.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f641a200640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6414155ad0 0x7f64141fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f641a200640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64140d29c0 0x7f6414200e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f641a200640 1 --2- 172.21.15.50:0/175808637 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6414074410 0x7f6414156610 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f641a200640 1 -- 172.21.15.50:0/175808637 >> 172.21.15.50:0/175808637 conn(0x7f64140708c0 msgr2=0x7f641414c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f641a200640 1 -- 172.21.15.50:0/175808637 shutdown_connections 2024-09-21T05:56:41.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.712+0000 7f641a200640 1 -- 172.21.15.50:0/175808637 wait complete. 2024-09-21T05:56:41.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.716+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2048073200 msgr2=0x7f2048075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.716+0000 7f206c800640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2048073200 0x7f2048075710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f20580018c0 tx=0x7f2058001070 comp rx=0 tx=0).stop 2024-09-21T05:56:41.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.716+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f206003ab60 msgr2=0x7f2060189b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.716+0000 7f206c800640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f206003ab60 0x7f2060189b30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2064052910 tx=0x7f206406a850 comp rx=0 tx=0).stop 2024-09-21T05:56:41.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.716+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 shutdown_connections 2024-09-21T05:56:41.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.716+0000 7f206c800640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f206003ab60 0x7f2060189b30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.716+0000 7f206c800640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2048073200 0x7f2048075710 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f20580018c0 tx=0x7f2058001070 comp rx=0 tx=0).stop 2024-09-21T05:56:41.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.716+0000 7f206c800640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f206003a040 0x7f206018acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.716+0000 7f206c800640 1 --2- 172.21.15.50:0/3266252022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2060034700 0x7f20600eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.716+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 >> 172.21.15.50:0/3266252022 conn(0x7f206001fbc0 msgr2=0x7f206000be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.724+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 shutdown_connections 2024-09-21T05:56:41.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.724+0000 7f206c800640 1 -- 172.21.15.50:0/3266252022 wait complete. 2024-09-21T05:56:41.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.736+0000 7f8f21c00640 1 -- 172.21.15.50:0/1434327442 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8f1c0b4430 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.736+0000 7f8f13e00640 1 -- 172.21.15.50:0/1434327442 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8f080f9660 con 0x7f8f1c0d0ce0 2024-09-21T05:56:41.739 INFO:teuthology.orchestra.run.smithi050.stdout:64424509447 2024-09-21T05:56:41.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.740+0000 7f8f21c00640 1 -- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8efc073400 msgr2=0x7f8efc075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.740+0000 7f8f21c00640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8efc073400 0x7f8efc075910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c159ea0 tx=0x7f8f0c008040 comp rx=0 tx=0).stop 2024-09-21T05:56:41.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.740+0000 7f8f21c00640 1 -- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c0d0ce0 msgr2=0x7f8f1c0b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.740+0000 7f8f21c00640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c0d0ce0 0x7f8f1c0b9580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009230 tx=0x7f8f08005130 comp rx=0 tx=0).stop 2024-09-21T05:56:41.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.740+0000 7f8f21c00640 1 -- 172.21.15.50:0/1434327442 shutdown_connections 2024-09-21T05:56:41.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.740+0000 7f8f21c00640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f1c1625f0 0x7f8f1c159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.740+0000 7f8f21c00640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c0d0ce0 0x7f8f1c0b9580 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.740+0000 7f8f21c00640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8efc073400 0x7f8efc075910 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.740+0000 7f8f21c00640 1 --2- 172.21.15.50:0/1434327442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0b5f20 0x7f8f1c159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.740+0000 7f8f21c00640 1 -- 172.21.15.50:0/1434327442 >> 172.21.15.50:0/1434327442 conn(0x7f8f1c073a10 msgr2=0x7f8f1c154cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.740+0000 7f8f21c00640 1 -- 172.21.15.50:0/1434327442 shutdown_connections 2024-09-21T05:56:41.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.740+0000 7f8f21c00640 1 -- 172.21.15.50:0/1434327442 wait complete. 2024-09-21T05:56:41.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-09-21T05:56:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.756+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f01e401de20 con 0x7f01e40e0f70 2024-09-21T05:56:41.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.756+0000 7f01e2000640 1 -- 172.21.15.50:0/2964771431 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f01d80164d0 con 0x7f01e40e0f70 2024-09-21T05:56:41.759 INFO:teuthology.orchestra.run.smithi050.stdout:60129542150 2024-09-21T05:56:41.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-09-21T05:56:41.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.760+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f01cc073250 msgr2=0x7f01cc075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.760+0000 7f01f1200640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f01cc073250 0x7f01cc075760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f01ec06bb90 tx=0x7f01ec06abb0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.760+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e40e0f70 msgr2=0x7f01e40da6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.760+0000 7f01f1200640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e40e0f70 0x7f01e40da6a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f01d80ac000 tx=0x7f01d80a56b0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.760+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 shutdown_connections 2024-09-21T05:56:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.760+0000 7f01f1200640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f01cc073250 0x7f01cc075760 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.760+0000 7f01f1200640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e40e0f70 0x7f01e40da6a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.760+0000 7f01f1200640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e400a400 0x7f01e40da160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.760+0000 7f01f1200640 1 --2- 172.21.15.50:0/2964771431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01e4005cc0 0x7f01e40e2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.760+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 >> 172.21.15.50:0/2964771431 conn(0x7f01e40513f0 msgr2=0x7f01e40052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.760+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 shutdown_connections 2024-09-21T05:56:41.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.760+0000 7f01f1200640 1 -- 172.21.15.50:0/2964771431 wait complete. 2024-09-21T05:56:41.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 -- 172.21.15.50:0/3122281415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3d58005860 con 0x7f3d800aee20 2024-09-21T05:56:41.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d76a00640 1 -- 172.21.15.50:0/3122281415 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d7005f020 con 0x7f3d800aee20 2024-09-21T05:56:41.769 INFO:teuthology.orchestra.run.smithi050.stdout:64424509445 2024-09-21T05:56:41.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 -- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d60073200 msgr2=0x7f3d60075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d60073200 0x7f3d60075710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c01c3e0 tx=0x7f3d6c01adf0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 -- 172.21.15.50:0/3122281415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d800aee20 msgr2=0x7f3d80154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d800aee20 0x7f3d80154ea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000b410 tx=0x7f3d7000b8d0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 -- 172.21.15.50:0/3122281415 shutdown_connections 2024-09-21T05:56:41.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d8015f800 0x7f3d80147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d801557e0 0x7f3d8014cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d60073200 0x7f3d60075710 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 --2- 172.21.15.50:0/3122281415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d800aee20 0x7f3d80154ea0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 -- 172.21.15.50:0/3122281415 >> 172.21.15.50:0/3122281415 conn(0x7f3d800b2040 msgr2=0x7f3d8014bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 -- 172.21.15.50:0/3122281415 shutdown_connections 2024-09-21T05:56:41.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.764+0000 7f3d85c00640 1 -- 172.21.15.50:0/3122281415 wait complete. 2024-09-21T05:56:41.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.772+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff87c152660 con 0x7ff87c06cdf0 2024-09-21T05:56:41.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.772+0000 7ff872000640 1 -- 172.21.15.50:0/441636131 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff86c062a90 con 0x7ff87c06cdf0 2024-09-21T05:56:41.776 INFO:teuthology.orchestra.run.smithi050.stdout:64424509445 2024-09-21T05:56:41.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.776+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff85c073600 msgr2=0x7ff85c075b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.776+0000 7ff881200640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff85c073600 0x7ff85c075b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff868006fd0 tx=0x7ff868006b40 comp rx=0 tx=0).stop 2024-09-21T05:56:41.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.776+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c06cdf0 msgr2=0x7ff87c150f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.776+0000 7ff881200640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c06cdf0 0x7ff87c150f30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0058d0 tx=0x7ff86c00feb0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.776+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 shutdown_connections 2024-09-21T05:56:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.776+0000 7ff881200640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff87c159130 0x7ff87c14b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.776+0000 7ff881200640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff87c0703a0 0x7ff87c154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.776+0000 7ff881200640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff85c073600 0x7ff85c075b10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.776+0000 7ff881200640 1 --2- 172.21.15.50:0/441636131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c06cdf0 0x7ff87c150f30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.776+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 >> 172.21.15.50:0/441636131 conn(0x7ff87c0b35f0 msgr2=0x7ff87c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.780+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 shutdown_connections 2024-09-21T05:56:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.780+0000 7ff881200640 1 -- 172.21.15.50:0/441636131 wait complete. 2024-09-21T05:56:41.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.2 2024-09-21T05:56:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.788+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd40c005860 con 0x7fd4340cd7a0 2024-09-21T05:56:41.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd42b400640 1 -- 172.21.15.50:0/2640088353 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd4240642b0 con 0x7fd4340cd7a0 2024-09-21T05:56:41.793 INFO:teuthology.orchestra.run.smithi050.stdout:64424509446 2024-09-21T05:56:41.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd410073200 msgr2=0x7fd410075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd439c00640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd410073200 0x7fd410075710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd434150010 tx=0x7fd41c00a250 comp rx=0 tx=0).stop 2024-09-21T05:56:41.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4340cd7a0 msgr2=0x7fd4341fcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd439c00640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4340cd7a0 0x7fd4341fcca0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd42400ea90 tx=0x7fd42400ef50 comp rx=0 tx=0).stop 2024-09-21T05:56:41.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.0 2024-09-21T05:56:41.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 shutdown_connections 2024-09-21T05:56:41.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd439c00640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd43415e4e0 0x7fd43415c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd439c00640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd410073200 0x7fd410075710 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd439c00640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43415aa50 0x7fd43413e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd439c00640 1 --2- 172.21.15.50:0/2640088353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4340cd7a0 0x7fd4341fcca0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 >> 172.21.15.50:0/2640088353 conn(0x7fd4340b2040 msgr2=0x7fd4340ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 shutdown_connections 2024-09-21T05:56:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.792+0000 7fd439c00640 1 -- 172.21.15.50:0/2640088353 wait complete. 2024-09-21T05:56:41.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.1 2024-09-21T05:56:41.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.804+0000 7f157e400640 1 -- 172.21.15.50:0/534890233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1550005590 con 0x7f1578162500 2024-09-21T05:56:41.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.804+0000 7f156f400640 1 -- 172.21.15.50:0/534890233 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f15700f8700 con 0x7f1578162500 2024-09-21T05:56:41.808 INFO:teuthology.orchestra.run.smithi050.stdout:64424509445 2024-09-21T05:56:41.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.808+0000 7f157e400640 1 -- 172.21.15.50:0/534890233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1554073230 msgr2=0x7f1554075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.808+0000 7f157e400640 1 --2- 172.21.15.50:0/534890233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1554073230 0x7f1554075740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f15780b4bf0 tx=0x7f15600091e0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.808+0000 7f157e400640 1 -- 172.21.15.50:0/534890233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1578162500 msgr2=0x7f1578157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:41.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.808+0000 7f157e400640 1 --2- 172.21.15.50:0/534890233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1578162500 0x7f1578157470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1570009bd0 tx=0x7f1570003f60 comp rx=0 tx=0).stop 2024-09-21T05:56:41.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.808+0000 7f157e400640 1 -- 172.21.15.50:0/534890233 shutdown_connections 2024-09-21T05:56:41.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.808+0000 7f157e400640 1 --2- 172.21.15.50:0/534890233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1554073230 0x7f1554075740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.808+0000 7f157e400640 1 --2- 172.21.15.50:0/534890233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1578162500 0x7f1578157470 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.808+0000 7f157e400640 1 --2- 172.21.15.50:0/534890233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157815e5a0 0x7f157814e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.808+0000 7f157e400640 1 --2- 172.21.15.50:0/534890233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1578073b00 0x7f15780b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:41.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.808+0000 7f157e400640 1 -- 172.21.15.50:0/534890233 >> 172.21.15.50:0/534890233 conn(0x7f15780b3b30 msgr2=0x7f157815ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:41.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.808+0000 7f157e400640 1 -- 172.21.15.50:0/534890233 shutdown_connections 2024-09-21T05:56:41.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:41.808+0000 7f157e400640 1 -- 172.21.15.50:0/534890233 wait complete. 2024-09-21T05:56:41.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2024-09-21T05:56:41.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-09-21T05:56:41.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2024-09-21T05:56:42.757 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T05:56:42.764 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T05:56:42.783 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T05:56:42.798 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T05:56:42.801 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T05:56:42.814 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T05:56:42.821 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T05:56:42.831 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T05:56:43.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.000+0000 7f8449c00640 1 -- 172.21.15.50:0/1252552608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84440af740 msgr2=0x7f84441546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.000+0000 7f8449c00640 1 --2- 172.21.15.50:0/1252552608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84440af740 0x7f84441546c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f84440ce5b0 tx=0x7f843001c7b0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.000+0000 7f8449c00640 1 -- 172.21.15.50:0/1252552608 shutdown_connections 2024-09-21T05:56:43.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.000+0000 7f8449c00640 1 --2- 172.21.15.50:0/1252552608 >> v2:172.21.15.50:3300/0 conn(0x7f8444154c00 0x7f8444155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.000+0000 7f8449c00640 1 --2- 172.21.15.50:0/1252552608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84440af740 0x7f84441546c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.000+0000 7f8449c00640 1 -- 172.21.15.50:0/1252552608 >> 172.21.15.50:0/1252552608 conn(0x7f84440b2040 msgr2=0x7f84440b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.004+0000 7f8449c00640 1 -- 172.21.15.50:0/1252552608 shutdown_connections 2024-09-21T05:56:43.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.004+0000 7f8449c00640 1 -- 172.21.15.50:0/1252552608 wait complete. 2024-09-21T05:56:43.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.008+0000 7f8449c00640 1 Processor -- start 2024-09-21T05:56:43.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.008+0000 7f8449c00640 1 -- start start 2024-09-21T05:56:43.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8449c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84440af330 0x7f8444074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8449c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84440af740 0x7f844414c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8449c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8444154c00 0x7f84440722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8443e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84440af330 0x7f8444074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8443e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84440af330 0x7f8444074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:47482/0 (socket says 172.21.15.50:47482) 2024-09-21T05:56:43.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8443400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84440af740 0x7f844414c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8443400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84440af740 0x7f844414c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:51204/0 (socket says 172.21.15.50:51204) 2024-09-21T05:56:43.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8449200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8444154c00 0x7f84440722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8449200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8444154c00 0x7f84440722b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48612/0 (socket says 172.21.15.50:48612) 2024-09-21T05:56:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8449c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f84440b5a90 con 0x7f8444154c00 2024-09-21T05:56:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8449c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f84440b5910 con 0x7f84440af330 2024-09-21T05:56:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8449c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f84440b5c10 con 0x7f84440af740 2024-09-21T05:56:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8443e00640 1 -- 172.21.15.50:0/2530731821 learned_addr learned my addr 172.21.15.50:0/2530731821 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8443400640 1 -- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84440af330 msgr2=0x7f8444074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8443400640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84440af330 0x7f8444074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8443400640 1 -- 172.21.15.50:0/2530731821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8444154c00 msgr2=0x7f84440722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8443400640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8444154c00 0x7f84440722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.012+0000 7f8443400640 1 -- 172.21.15.50:0/2530731821 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844414fdc0 con 0x7f84440af740 2024-09-21T05:56:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.016+0000 7f8443e00640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84440af330 0x7f8444074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T05:56:43.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.016+0000 7f8443400640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84440af740 0x7f844414c7f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f84300ac000 tx=0x7f8430005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.016+0000 7f843be00640 1 -- 172.21.15.50:0/2530731821 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84300bc070 con 0x7f84440af740 2024-09-21T05:56:43.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.016+0000 7f8449c00640 1 -- 172.21.15.50:0/2530731821 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84441500a0 con 0x7f84440af740 2024-09-21T05:56:43.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.016+0000 7f8449c00640 1 -- 172.21.15.50:0/2530731821 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8444150b80 con 0x7f84440af740 2024-09-21T05:56:43.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.016+0000 7f843be00640 1 -- 172.21.15.50:0/2530731821 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84300a5850 con 0x7f84440af740 2024-09-21T05:56:43.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.020+0000 7f8439600640 1 -- 172.21.15.50:0/2530731821 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841c0052a0 con 0x7f84440af740 2024-09-21T05:56:43.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.020+0000 7f843be00640 1 -- 172.21.15.50:0/2530731821 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84300bc070 con 0x7f84440af740 2024-09-21T05:56:43.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.024+0000 7f843be00640 1 -- 172.21.15.50:0/2530731821 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8430004240 con 0x7f84440af740 2024-09-21T05:56:43.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.024+0000 7f843be00640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8424073600 0x7f8424075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.028+0000 7f8443e00640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8424073600 0x7f8424075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.028+0000 7f843be00640 1 -- 172.21.15.50:0/2530731821 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8430004040 con 0x7f84440af740 2024-09-21T05:56:43.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.028+0000 7f8443e00640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8424073600 0x7f8424075b10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f843c0015b0 tx=0x7f843c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.028+0000 7f843be00640 1 -- 172.21.15.50:0/2530731821 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f84300c5020 con 0x7f84440af740 2024-09-21T05:56:43.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7f8a81e00640 1 -- 172.21.15.50:0/10755000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a7c0cdf30 msgr2=0x7f8a7c0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7f8a81e00640 1 --2- 172.21.15.50:0/10755000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a7c0cdf30 0x7f8a7c0ce340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009bc0 tx=0x7f8a7401cb30 comp rx=0 tx=0).stop 2024-09-21T05:56:43.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7f8a81e00640 1 -- 172.21.15.50:0/10755000 shutdown_connections 2024-09-21T05:56:43.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7f8a81e00640 1 --2- 172.21.15.50:0/10755000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a7c0cdf30 0x7f8a7c0ce340 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7f8a81e00640 1 --2- 172.21.15.50:0/10755000 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a7c160330 0x7f8a7c160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7f8a81e00640 1 -- 172.21.15.50:0/10755000 >> 172.21.15.50:0/10755000 conn(0x7f8a7c06e4a0 msgr2=0x7f8a7c06e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7fa879400640 1 -- 172.21.15.50:0/1457568493 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0ac070 con 0x7fa87c075160 2024-09-21T05:56:43.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7f8a81e00640 1 -- 172.21.15.50:0/10755000 shutdown_connections 2024-09-21T05:56:43.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7f8a81e00640 1 -- 172.21.15.50:0/10755000 wait complete. 2024-09-21T05:56:43.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7fa880a00640 1 -- 172.21.15.50:0/1457568493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c075160 msgr2=0x7fa87c15ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7fa880a00640 1 --2- 172.21.15.50:0/1457568493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c075160 0x7fa87c15ebd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009900 tx=0x7fa86c01c720 comp rx=0 tx=0).stop 2024-09-21T05:56:43.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7fa880a00640 1 -- 172.21.15.50:0/1457568493 shutdown_connections 2024-09-21T05:56:43.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7fa880a00640 1 --2- 172.21.15.50:0/1457568493 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa87c0ce600 0x7fa87c068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7fa880a00640 1 --2- 172.21.15.50:0/1457568493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c075160 0x7fa87c15ebd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7fa880a00640 1 -- 172.21.15.50:0/1457568493 >> 172.21.15.50:0/1457568493 conn(0x7fa87c0b41f0 msgr2=0x7fa87c0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7fa880a00640 1 -- 172.21.15.50:0/1457568493 shutdown_connections 2024-09-21T05:56:43.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.076+0000 7fa880a00640 1 -- 172.21.15.50:0/1457568493 wait complete. 2024-09-21T05:56:43.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.080+0000 7fa880a00640 1 Processor -- start 2024-09-21T05:56:43.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.080+0000 7f8a81e00640 1 Processor -- start 2024-09-21T05:56:43.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.080+0000 7f8a81e00640 1 -- start start 2024-09-21T05:56:43.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.084+0000 7fa880a00640 1 -- start start 2024-09-21T05:56:43.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.084+0000 7f8a81e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a7c0cdf30 0x7f8a7c1ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.084+0000 7f8a81e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a7c0d0330 0x7f8a7c16a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.084+0000 7f8a81e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a7c160330 0x7f8a7c073600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.084+0000 7f8a80a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a7c0cdf30 0x7f8a7c1ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.084+0000 7f8a7be00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a7c0d0330 0x7f8a7c16a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.084+0000 7f8a80a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a7c0cdf30 0x7f8a7c1ffcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:47508/0 (socket says 172.21.15.50:47508) 2024-09-21T05:56:43.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.084+0000 7f8a80a00640 1 -- 172.21.15.50:0/3140009728 learned_addr learned my addr 172.21.15.50:0/3140009728 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:43.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.084+0000 7f8a7be00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a7c0d0330 0x7f8a7c16a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:51256/0 (socket says 172.21.15.50:51256) 2024-09-21T05:56:43.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.084+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c1677d0 con 0x7f8a7c160330 2024-09-21T05:56:43.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.084+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c167650 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.084+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8a7c167950 con 0x7f8a7c0d0330 2024-09-21T05:56:43.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7f8a80a00640 1 -- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a7c0d0330 msgr2=0x7f8a7c16a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7f8a80a00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a7c0d0330 0x7f8a7c16a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7f8a80a00640 1 -- 172.21.15.50:0/3140009728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a7c160330 msgr2=0x7f8a7c073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7f8a80a00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a7c160330 0x7f8a7c073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa880a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa87c069330 0x7fa87c099f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7f8a80a00640 1 -- 172.21.15.50:0/3140009728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7c200520 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7f8a7be00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a7c0d0330 0x7f8a7c16a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:43.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa880a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c075160 0x7fa87c0d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7f8a80a00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a7c0cdf30 0x7f8a7c1ffcd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00c3f0 tx=0x7f8a6c00c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa879e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c075160 0x7fa87c0d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa880a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa87c0ce600 0x7fa87c15a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7f8a78c00640 1 -- 172.21.15.50:0/3140009728 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c017070 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa879e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c075160 0x7fa87c0d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48658/0 (socket says 172.21.15.50:48658) 2024-09-21T05:56:43.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa880a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c16c650 con 0x7fa87c075160 2024-09-21T05:56:43.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa880a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c16c4d0 con 0x7fa87c069330 2024-09-21T05:56:43.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa880a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa87c16c7d0 con 0x7fa87c0ce600 2024-09-21T05:56:43.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa879e00640 1 -- 172.21.15.50:0/1923903520 learned_addr learned my addr 172.21.15.50:0/1923903520 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:43.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa87a800640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa87c069330 0x7fa87c099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa87b200640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa87c0ce600 0x7fa87c15a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7ff68d200640 1 -- 172.21.15.50:0/677177032 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6881625f0 msgr2=0x7ff688162a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7ff68d200640 1 --2- 172.21.15.50:0/677177032 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6881625f0 0x7ff688162a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff670009820 tx=0x7ff67001c5c0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7f8a78c00640 1 -- 172.21.15.50:0/3140009728 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a6c0053c0 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7c200800 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7ff68d200640 1 -- 172.21.15.50:0/677177032 shutdown_connections 2024-09-21T05:56:43.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7ff68d200640 1 --2- 172.21.15.50:0/677177032 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6881625f0 0x7ff688162a00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa87b200640 1 -- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa87c069330 msgr2=0x7fa87c099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa87b200640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa87c069330 0x7fa87c099f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7ff68d200640 1 -- 172.21.15.50:0/677177032 >> 172.21.15.50:0/677177032 conn(0x7ff688073a10 msgr2=0x7ff68806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7f8a78c00640 1 -- 172.21.15.50:0/3140009728 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c003860 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a7c1fc9f0 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa87b200640 1 -- 172.21.15.50:0/1923903520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c075160 msgr2=0x7fa87c0d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa87b200640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c075160 0x7fa87c0d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.092+0000 7ff68d200640 1 -- 172.21.15.50:0/677177032 shutdown_connections 2024-09-21T05:56:43.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa87b200640 1 -- 172.21.15.50:0/1923903520 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c0cedd0 con 0x7fa87c0ce600 2024-09-21T05:56:43.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.088+0000 7fa879e00640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c075160 0x7fa87c0d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:43.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.092+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a7c167ee0 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.092+0000 7ff68d200640 1 -- 172.21.15.50:0/677177032 wait complete. 2024-09-21T05:56:43.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.092+0000 7f8a78c00640 1 -- 172.21.15.50:0/3140009728 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8a6c01a070 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7fa87b200640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa87c0ce600 0x7fa87c15a250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa87400be20 tx=0x7fa87400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7fa872a00640 1 -- 172.21.15.50:0/1923903520 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874019070 con 0x7fa87c0ce600 2024-09-21T05:56:43.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7fa872a00640 1 -- 172.21.15.50:0/1923903520 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa874014440 con 0x7fa87c0ce600 2024-09-21T05:56:43.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7fa872a00640 1 -- 172.21.15.50:0/1923903520 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874014890 con 0x7fa87c0ce600 2024-09-21T05:56:43.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7fa880a00640 1 -- 172.21.15.50:0/1923903520 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c0cf0b0 con 0x7fa87c0ce600 2024-09-21T05:56:43.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7fa880a00640 1 -- 172.21.15.50:0/1923903520 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c0cf670 con 0x7fa87c0ce600 2024-09-21T05:56:43.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7f8a78c00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a60073200 0x7f8a60075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7f8a78c00640 1 -- 172.21.15.50:0/3140009728 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8a6c0935f0 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7f8a7be00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a60073200 0x7f8a60075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7fa880a00640 1 -- 172.21.15.50:0/1923903520 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8540052a0 con 0x7fa87c0ce600 2024-09-21T05:56:43.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7f446ca00640 1 -- 172.21.15.50:0/2352059787 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f444c005c20 msgr2=0x7f444c006090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7f446ca00640 1 --2- 172.21.15.50:0/2352059787 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f444c005c20 0x7f444c006090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f445800c710 tx=0x7f445801c700 comp rx=0 tx=0).stop 2024-09-21T05:56:43.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7ff68d200640 1 Processor -- start 2024-09-21T05:56:43.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7f4466800640 1 -- 172.21.15.50:0/2352059787 reap_dead start 2024-09-21T05:56:43.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7f8a7be00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a60073200 0x7f8a60075710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c164f00 tx=0x7f8a70009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7f446ca00640 1 -- 172.21.15.50:0/2352059787 shutdown_connections 2024-09-21T05:56:43.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.096+0000 7f446ca00640 1 -- 172.21.15.50:0/2352059787 >> 172.21.15.50:0/2352059787 conn(0x7f44680d06e0 msgr2=0x7f44680d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.100+0000 7f446ca00640 1 -- 172.21.15.50:0/2352059787 shutdown_connections 2024-09-21T05:56:43.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.100+0000 7ff68d200640 1 -- start start 2024-09-21T05:56:43.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.100+0000 7fa872a00640 1 -- 172.21.15.50:0/1923903520 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa87401d3f0 con 0x7fa87c0ce600 2024-09-21T05:56:43.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.100+0000 7f446ca00640 1 -- 172.21.15.50:0/2352059787 wait complete. 2024-09-21T05:56:43.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.100+0000 7fa872a00640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa85c073230 0x7fa85c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.100+0000 7fa872a00640 1 -- 172.21.15.50:0/1923903520 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa874093f90 con 0x7fa87c0ce600 2024-09-21T05:56:43.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.100+0000 7fa87a800640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa85c073230 0x7fa85c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.100+0000 7ff68d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6880cf1c0 0x7ff6881535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.100+0000 7f8a78c00640 1 -- 172.21.15.50:0/3140009728 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8a6c05df20 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f446ca00640 1 Processor -- start 2024-09-21T05:56:43.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f446ca00640 1 -- start start 2024-09-21T05:56:43.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7ff687400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6880cf1c0 0x7ff6881535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7fa87a800640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa85c073230 0x7fa85c075740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0ae000 tx=0x7fa86c0b4040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7ff68d200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6880d0ce0 0x7ff6880c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7ff687400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6880cf1c0 0x7ff6881535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48668/0 (socket says 172.21.15.50:48668) 2024-09-21T05:56:43.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7ff687400640 1 -- 172.21.15.50:0/1485695773 learned_addr learned my addr 172.21.15.50:0/1485695773 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:43.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7ff67fe00640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6880d0ce0 0x7ff6880c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7ff68d200640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6881625f0 0x7ff6880c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff68816d8a0 con 0x7ff6880cf1c0 2024-09-21T05:56:43.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff68816d720 con 0x7ff6881625f0 2024-09-21T05:56:43.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f446ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f444c005c20 0x7f44681f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f446ca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44680ab910 0x7f44681590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f446ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44680aa4d0 0x7f44680aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f446ca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4468073d20 con 0x7f44680aa4d0 2024-09-21T05:56:43.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f446ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4468073ba0 con 0x7f444c005c20 2024-09-21T05:56:43.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f446ca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4468073ea0 con 0x7f44680ab910 2024-09-21T05:56:43.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7ff687e00640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6881625f0 0x7ff6880c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff68816da20 con 0x7ff6880d0ce0 2024-09-21T05:56:43.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f4465400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44680ab910 0x7f44681590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f4465400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44680ab910 0x7f44681590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:51280/0 (socket says 172.21.15.50:51280) 2024-09-21T05:56:43.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f4465400640 1 -- 172.21.15.50:0/4056336714 learned_addr learned my addr 172.21.15.50:0/4056336714 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:43.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7fa872a00640 1 -- 172.21.15.50:0/1923903520 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa87405e8c0 con 0x7fa87c0ce600 2024-09-21T05:56:43.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f4465e00640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f444c005c20 0x7f44681f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f4465400640 1 -- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f444c005c20 msgr2=0x7f44681f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.104+0000 7f4466800640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44680aa4d0 0x7f44680aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff687e00640 1 -- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6880d0ce0 msgr2=0x7ff6880c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff687e00640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6880d0ce0 0x7ff6880c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff687e00640 1 -- 172.21.15.50:0/1485695773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6880cf1c0 msgr2=0x7ff6881535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff687e00640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6880cf1c0 0x7ff6881535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff687e00640 1 -- 172.21.15.50:0/1485695773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6880cdf90 con 0x7ff6881625f0 2024-09-21T05:56:43.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff67fe00640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6880d0ce0 0x7ff6880c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:43.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff687400640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6880cf1c0 0x7ff6881535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:43.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff687e00640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6881625f0 0x7ff6880c9590 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff688162550 tx=0x7ff680009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7f4465400640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f444c005c20 0x7f44681f7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7f4465400640 1 -- 172.21.15.50:0/4056336714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44680aa4d0 msgr2=0x7f44680aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7f4465400640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44680aa4d0 0x7f44680aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7f4465400640 1 -- 172.21.15.50:0/4056336714 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44680ac580 con 0x7f44680ab910 2024-09-21T05:56:43.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7f4466800640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44680aa4d0 0x7f44680aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:43.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7f4465400640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44680ab910 0x7f44681590d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44540039d0 tx=0x7f4454003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7f4465e00640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f444c005c20 0x7f44681f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:43.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff67f400640 1 -- 172.21.15.50:0/1485695773 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680019070 con 0x7ff6881625f0 2024-09-21T05:56:43.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff67f400640 1 -- 172.21.15.50:0/1485695773 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff680014440 con 0x7ff6881625f0 2024-09-21T05:56:43.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6880ce270 con 0x7ff6881625f0 2024-09-21T05:56:43.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6880c7930 con 0x7ff6881625f0 2024-09-21T05:56:43.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7ff67f400640 1 -- 172.21.15.50:0/1485695773 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6800148d0 con 0x7ff6881625f0 2024-09-21T05:56:43.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7f445e000640 1 -- 172.21.15.50:0/4056336714 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445401d070 con 0x7f44680ab910 2024-09-21T05:56:43.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7f445e000640 1 -- 172.21.15.50:0/4056336714 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f445400fc70 con 0x7f44680ab910 2024-09-21T05:56:43.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.108+0000 7f445e000640 1 -- 172.21.15.50:0/4056336714 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445400d6b0 con 0x7f44680ab910 2024-09-21T05:56:43.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.112+0000 7f446ca00640 1 -- 172.21.15.50:0/4056336714 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44681f7400 con 0x7f44680ab910 2024-09-21T05:56:43.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.112+0000 7f446ca00640 1 -- 172.21.15.50:0/4056336714 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f44681f79c0 con 0x7f44680ab910 2024-09-21T05:56:43.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.112+0000 7f446ca00640 1 -- 172.21.15.50:0/4056336714 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4468073490 con 0x7f44680ab910 2024-09-21T05:56:43.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.112+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6600052a0 con 0x7ff6881625f0 2024-09-21T05:56:43.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.112+0000 7f445e000640 1 -- 172.21.15.50:0/4056336714 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4454020070 con 0x7f44680ab910 2024-09-21T05:56:43.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.116+0000 7f445e000640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4448073280 0x7f4448075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.116+0000 7ff67f400640 1 -- 172.21.15.50:0/1485695773 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff68001c070 con 0x7ff6881625f0 2024-09-21T05:56:43.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.116+0000 7f445e000640 1 -- 172.21.15.50:0/4056336714 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f445405b4f0 con 0x7f44680ab910 2024-09-21T05:56:43.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.116+0000 7f4465e00640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4448073280 0x7f4448075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.116+0000 7ff67f400640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff6640732d0 0x7ff6640757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.116+0000 7ff67f400640 1 -- 172.21.15.50:0/1485695773 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff680094150 con 0x7ff6881625f0 2024-09-21T05:56:43.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.116+0000 7f4465e00640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4448073280 0x7f4448075790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f445800c6e0 tx=0x7f44580194a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.116+0000 7ff687400640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff6640732d0 0x7ff6640757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.120+0000 7ff687400640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff6640732d0 0x7ff6640757e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff678000f70 tx=0x7ff678009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.120+0000 7ff67f400640 1 -- 172.21.15.50:0/1485695773 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff68005eb00 con 0x7ff6881625f0 2024-09-21T05:56:43.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.124+0000 7f445e000640 1 -- 172.21.15.50:0/4056336714 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f445400fe20 con 0x7f44680ab910 2024-09-21T05:56:43.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.140+0000 7fb58e400640 1 -- 172.21.15.50:0/4291697075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb58806fd70 msgr2=0x7fb58815f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.140+0000 7fb58e400640 1 --2- 172.21.15.50:0/4291697075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb58806fd70 0x7fb58815f6c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb5880728d0 tx=0x7fb57801c800 comp rx=0 tx=0).stop 2024-09-21T05:56:43.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.140+0000 7fb58e400640 1 -- 172.21.15.50:0/4291697075 shutdown_connections 2024-09-21T05:56:43.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.140+0000 7fb58e400640 1 --2- 172.21.15.50:0/4291697075 >> v2:172.21.15.88:3300/0 conn(0x7fb5880d0910 0x7fb5880b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.140+0000 7fb58e400640 1 --2- 172.21.15.50:0/4291697075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb58806fd70 0x7fb58815f6c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.140+0000 7fb58e400640 1 -- 172.21.15.50:0/4291697075 >> 172.21.15.50:0/4291697075 conn(0x7fb58806e4a0 msgr2=0x7fb58806e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.144+0000 7fb58e400640 1 -- 172.21.15.50:0/4291697075 shutdown_connections 2024-09-21T05:56:43.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.144+0000 7f67d6000640 1 -- 172.21.15.50:0/686227717 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d00b2070 con 0x7f67d815b280 2024-09-21T05:56:43.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.144+0000 7f67dca00640 1 -- 172.21.15.50:0/686227717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d815b280 msgr2=0x7f67d80d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.144+0000 7f67dca00640 1 --2- 172.21.15.50:0/686227717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d815b280 0x7f67d80d0330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009bf0 tx=0x7f67d001c930 comp rx=0 tx=0).stop 2024-09-21T05:56:43.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.144+0000 7f67dca00640 1 -- 172.21.15.50:0/686227717 shutdown_connections 2024-09-21T05:56:43.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.144+0000 7f67dca00640 1 --2- 172.21.15.50:0/686227717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d815b280 0x7f67d80d0330 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.144+0000 7f67dca00640 1 -- 172.21.15.50:0/686227717 >> 172.21.15.50:0/686227717 conn(0x7f67d806e4a0 msgr2=0x7f67d806e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.148+0000 7f67dca00640 1 -- 172.21.15.50:0/686227717 shutdown_connections 2024-09-21T05:56:43.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.144+0000 7fb58e400640 1 -- 172.21.15.50:0/4291697075 wait complete. 2024-09-21T05:56:43.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.148+0000 7f67dca00640 1 -- 172.21.15.50:0/686227717 wait complete. 2024-09-21T05:56:43.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.148+0000 7fb58e400640 1 Processor -- start 2024-09-21T05:56:43.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7f67dca00640 1 Processor -- start 2024-09-21T05:56:43.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.148+0000 7fb58e400640 1 -- start start 2024-09-21T05:56:43.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb58e400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb58806fd70 0x7fb5881fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb58e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5880d0500 0x7fb58816c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb58e400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb5880d0910 0x7fb5882006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb58e400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb588157750 con 0x7fb58806fd70 2024-09-21T05:56:43.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb58e400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881575d0 con 0x7fb5880d0500 2024-09-21T05:56:43.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb58e400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb5881578d0 con 0x7fb5880d0910 2024-09-21T05:56:43.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7f67dca00640 1 -- start start 2024-09-21T05:56:43.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb587e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5880d0500 0x7fb58816c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb587e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5880d0500 0x7fb58816c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:47564/0 (socket says 172.21.15.50:47564) 2024-09-21T05:56:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb587e00640 1 -- 172.21.15.50:0/1318617711 learned_addr learned my addr 172.21.15.50:0/1318617711 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb58da00640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb5880d0910 0x7fb5882006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb587e00640 1 -- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb5880d0910 msgr2=0x7fb5882006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb587e00640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb5880d0910 0x7fb5882006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7fb587e00640 1 -- 172.21.15.50:0/1318617711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb58806fd70 msgr2=0x7fb5881fdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.152+0000 7fb58d000640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb58806fd70 0x7fb5881fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7fb587e00640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb58806fd70 0x7fb5881fdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7fb587e00640 1 -- 172.21.15.50:0/1318617711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb588201220 con 0x7fb5880d0500 2024-09-21T05:56:43.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67dca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d815b280 0x7f67d8150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7fb58d000640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb58806fd70 0x7fb5881fdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:43.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67dca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f67d815b690 0x7f67d814b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f3c5d400640 1 -- 172.21.15.50:0/1231571359 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c580b5c70 msgr2=0x7f3c5815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f3c5d400640 1 --2- 172.21.15.50:0/1231571359 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c580b5c70 0x7f3c5815bb60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009900 tx=0x7f3c4801c840 comp rx=0 tx=0).stop 2024-09-21T05:56:43.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67d7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d815b280 0x7f67d8150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7fb587e00640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5880d0500 0x7fb58816c250 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00b410 tx=0x7fb57c00b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f3c5d400640 1 -- 172.21.15.50:0/1231571359 shutdown_connections 2024-09-21T05:56:43.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f3c5d400640 1 --2- 172.21.15.50:0/1231571359 >> v2:172.21.15.50:3300/0 conn(0x7f3c581527b0 0x7f3c58074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f3c5d400640 1 --2- 172.21.15.50:0/1231571359 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c580b5c70 0x7f3c5815bb60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f3c5d400640 1 -- 172.21.15.50:0/1231571359 >> 172.21.15.50:0/1231571359 conn(0x7f3c580b2040 msgr2=0x7f3c580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67d7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d815b280 0x7f67d8150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:47580/0 (socket says 172.21.15.50:47580) 2024-09-21T05:56:43.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67d7400640 1 -- 172.21.15.50:0/3536677644 learned_addr learned my addr 172.21.15.50:0/3536677644 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:43.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67d6a00640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f67d815b690 0x7f67d814b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67dca00640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67d815f730 0x7f67d8154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8070310 con 0x7f67d815f730 2024-09-21T05:56:43.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8070190 con 0x7f67d815b280 2024-09-21T05:56:43.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f67d8070490 con 0x7f67d815b690 2024-09-21T05:56:43.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f3c5d400640 1 -- 172.21.15.50:0/1231571359 shutdown_connections 2024-09-21T05:56:43.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f3c5d400640 1 -- 172.21.15.50:0/1231571359 wait complete. 2024-09-21T05:56:43.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7fb584c00640 1 -- 172.21.15.50:0/1318617711 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c015030 con 0x7fb5880d0500 2024-09-21T05:56:43.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67d6a00640 1 -- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d815b280 msgr2=0x7f67d8150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67d6a00640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d815b280 0x7f67d8150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67d6a00640 1 -- 172.21.15.50:0/3536677644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67d815f730 msgr2=0x7f67d8154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67d6a00640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67d815f730 0x7f67d8154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7f67d6a00640 1 -- 172.21.15.50:0/3536677644 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d8153d40 con 0x7f67d815b690 2024-09-21T05:56:43.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.160+0000 7f67d6a00640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f67d815b690 0x7f67d814b4e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f67c400c390 tx=0x7f67c400c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.160+0000 7f67cf400640 1 -- 172.21.15.50:0/3536677644 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4017070 con 0x7f67d815b690 2024-09-21T05:56:43.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.160+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d8200dd0 con 0x7f67d815b690 2024-09-21T05:56:43.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.160+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d8154290 con 0x7f67d815b690 2024-09-21T05:56:43.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7fb584c00640 1 -- 172.21.15.50:0/1318617711 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb57c004020 con 0x7fb5880d0500 2024-09-21T05:56:43.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7fb58e400640 1 -- 172.21.15.50:0/1318617711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb588201500 con 0x7fb5880d0500 2024-09-21T05:56:43.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7fb58e400640 1 -- 172.21.15.50:0/1318617711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb588150c70 con 0x7fb5880d0500 2024-09-21T05:56:43.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.156+0000 7fb584c00640 1 -- 172.21.15.50:0/1318617711 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0044b0 con 0x7fb5880d0500 2024-09-21T05:56:43.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.160+0000 7fb58e400640 1 -- 172.21.15.50:0/1318617711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5640052a0 con 0x7fb5880d0500 2024-09-21T05:56:43.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.160+0000 7f67cf400640 1 -- 172.21.15.50:0/3536677644 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67c40053c0 con 0x7f67d815b690 2024-09-21T05:56:43.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.160+0000 7f67cf400640 1 -- 172.21.15.50:0/3536677644 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4017070 con 0x7f67d815b690 2024-09-21T05:56:43.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.164+0000 7f3c5d400640 1 Processor -- start 2024-09-21T05:56:43.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.164+0000 7f3c5d400640 1 -- start start 2024-09-21T05:56:43.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.164+0000 7fb584c00640 1 -- 172.21.15.50:0/1318617711 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb57c018070 con 0x7fb5880d0500 2024-09-21T05:56:43.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.164+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b00052a0 con 0x7f67d815b690 2024-09-21T05:56:43.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.164+0000 7f67cf400640 1 -- 172.21.15.50:0/3536677644 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f67c401a070 con 0x7f67d815b690 2024-09-21T05:56:43.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c5d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c580b4de0 0x7f3c58141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c5d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c580b5c70 0x7f3c580ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c5d400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c581527b0 0x7f3c581589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f67cf400640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67b8073600 0x7f67b8075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c5d400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3c58166cc0 con 0x7f3c580b5c70 2024-09-21T05:56:43.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c55e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c580b5c70 0x7f3c580ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f67cf400640 1 -- 172.21.15.50:0/3536677644 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f67c40050d0 con 0x7f67d815b690 2024-09-21T05:56:43.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c57200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c581527b0 0x7f3c581589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c55e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c580b5c70 0x7f3c580ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48738/0 (socket says 172.21.15.50:48738) 2024-09-21T05:56:43.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c55e00640 1 -- 172.21.15.50:0/2553981060 learned_addr learned my addr 172.21.15.50:0/2553981060 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:43.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f67d7400640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67b8073600 0x7f67b8075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c56800640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c580b4de0 0x7f3c58141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c5d400640 1 -- 172.21.15.50:0/2553981060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3c58166b40 con 0x7f3c580b4de0 2024-09-21T05:56:43.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7fb584c00640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb56c073300 0x7fb56c075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7fb584c00640 1 -- 172.21.15.50:0/1318617711 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb57c093e50 con 0x7fb5880d0500 2024-09-21T05:56:43.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c5d400640 1 -- 172.21.15.50:0/2553981060 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3c58166e40 con 0x7f3c581527b0 2024-09-21T05:56:43.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7fb58d000640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb56c073300 0x7fb56c075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c57200640 1 -- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c580b4de0 msgr2=0x7f3c58141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c57200640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c580b4de0 0x7f3c58141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c57200640 1 -- 172.21.15.50:0/2553981060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c580b5c70 msgr2=0x7f3c580ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c57200640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c580b5c70 0x7f3c580ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c57200640 1 -- 172.21.15.50:0/2553981060 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c580cd0a0 con 0x7f3c581527b0 2024-09-21T05:56:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c55e00640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c580b5c70 0x7f3c580ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f3c57200640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c581527b0 0x7f3c581589e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400b3e0 tx=0x7f3c4400b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.168+0000 7f67d7400640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67b8073600 0x7f67b8075b10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f67d0000f90 tx=0x7f67d00a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.172+0000 7fb584c00640 1 -- 172.21.15.50:0/1318617711 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb57c05eee0 con 0x7fb5880d0500 2024-09-21T05:56:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.172+0000 7fb58d000640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb56c073300 0x7fb56c075810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5780ae000 tx=0x7fb57801b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.172+0000 7f3c4ea00640 1 -- 172.21.15.50:0/2553981060 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44015020 con 0x7f3c581527b0 2024-09-21T05:56:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.172+0000 7f3c4ea00640 1 -- 172.21.15.50:0/2553981060 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c44011920 con 0x7f3c581527b0 2024-09-21T05:56:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.172+0000 7f3c4ea00640 1 -- 172.21.15.50:0/2553981060 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44011d70 con 0x7f3c581527b0 2024-09-21T05:56:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.172+0000 7f3c5d400640 1 -- 172.21.15.50:0/2553981060 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c580cd380 con 0x7f3c581527b0 2024-09-21T05:56:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.172+0000 7f3c5d400640 1 -- 172.21.15.50:0/2553981060 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c580cd940 con 0x7f3c581527b0 2024-09-21T05:56:43.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.176+0000 7f3c5d400640 1 -- 172.21.15.50:0/2553981060 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c300052a0 con 0x7f3c581527b0 2024-09-21T05:56:43.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.172+0000 7f67cf400640 1 -- 172.21.15.50:0/3536677644 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f67c405d590 con 0x7f67d815b690 2024-09-21T05:56:43.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.180+0000 7f3c4ea00640 1 -- 172.21.15.50:0/2553981060 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3c44018020 con 0x7f3c581527b0 2024-09-21T05:56:43.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.184+0000 7f3c4ea00640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3c38073300 0x7f3c38075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.184+0000 7f3c4ea00640 1 -- 172.21.15.50:0/2553981060 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3c44093d30 con 0x7f3c581527b0 2024-09-21T05:56:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.188+0000 7f3c56800640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3c38073300 0x7f3c38075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.188+0000 7f3c4ea00640 1 -- 172.21.15.50:0/2553981060 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3c4405ed30 con 0x7f3c581527b0 2024-09-21T05:56:43.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.188+0000 7f3c56800640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3c38073300 0x7f3c38075810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3c480af000 tx=0x7f3c4801b080 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.396+0000 7f8439600640 1 -- 172.21.15.50:0/2530731821 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f841c005860 con 0x7f84440af740 2024-09-21T05:56:43.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.396+0000 7f843be00640 1 -- 172.21.15.50:0/2530731821 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8430016430 con 0x7f84440af740 2024-09-21T05:56:43.399 INFO:teuthology.orchestra.run.smithi050.stdout:64424509447 2024-09-21T05:56:43.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.400+0000 7f8439600640 1 -- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8424073600 msgr2=0x7f8424075b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.400+0000 7f8439600640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8424073600 0x7f8424075b10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f843c0015b0 tx=0x7f843c009040 comp rx=0 tx=0).stop 2024-09-21T05:56:43.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.400+0000 7f8439600640 1 -- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84440af740 msgr2=0x7f844414c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.400+0000 7f8439600640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84440af740 0x7f844414c7f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f84300ac000 tx=0x7f8430005760 comp rx=0 tx=0).stop 2024-09-21T05:56:43.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.400+0000 7f8439600640 1 -- 172.21.15.50:0/2530731821 shutdown_connections 2024-09-21T05:56:43.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.400+0000 7f8439600640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8424073600 0x7f8424075b10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.400+0000 7f8439600640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8444154c00 0x7f84440722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.400+0000 7f8439600640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84440af740 0x7f844414c7f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.400+0000 7f8439600640 1 --2- 172.21.15.50:0/2530731821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84440af330 0x7f8444074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.400+0000 7f8439600640 1 -- 172.21.15.50:0/2530731821 >> 172.21.15.50:0/2530731821 conn(0x7f84440b2040 msgr2=0x7f8444160af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.400+0000 7f8439600640 1 -- 172.21.15.50:0/2530731821 shutdown_connections 2024-09-21T05:56:43.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.400+0000 7f8439600640 1 -- 172.21.15.50:0/2530731821 wait complete. 2024-09-21T05:56:43.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-09-21T05:56:43.437 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:43.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.468+0000 7fa880a00640 1 -- 172.21.15.50:0/1923903520 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa854005590 con 0x7fa87c0ce600 2024-09-21T05:56:43.472 INFO:teuthology.orchestra.run.smithi050.stdout:64424509448 2024-09-21T05:56:43.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.468+0000 7fa872a00640 1 -- 172.21.15.50:0/1923903520 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa8740632f0 con 0x7fa87c0ce600 2024-09-21T05:56:43.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.472+0000 7fa880a00640 1 -- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa85c073230 msgr2=0x7fa85c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.472+0000 7fa880a00640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa85c073230 0x7fa85c075740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0ae000 tx=0x7fa86c0b4040 comp rx=0 tx=0).stop 2024-09-21T05:56:43.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.472+0000 7fa880a00640 1 -- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa87c0ce600 msgr2=0x7fa87c15a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.472+0000 7fa880a00640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa87c0ce600 0x7fa87c15a250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa87400be20 tx=0x7fa87400bf10 comp rx=0 tx=0).stop 2024-09-21T05:56:43.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.472+0000 7fa880a00640 1 -- 172.21.15.50:0/1923903520 shutdown_connections 2024-09-21T05:56:43.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.472+0000 7fa880a00640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa87c0ce600 0x7fa87c15a250 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.472+0000 7fa880a00640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa85c073230 0x7fa85c075740 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.472+0000 7fa880a00640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c075160 0x7fa87c0d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.472+0000 7fa880a00640 1 --2- 172.21.15.50:0/1923903520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa87c069330 0x7fa87c099f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.472+0000 7fa880a00640 1 -- 172.21.15.50:0/1923903520 >> 172.21.15.50:0/1923903520 conn(0x7fa87c0b41f0 msgr2=0x7fa87c0b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.472+0000 7fa880a00640 1 -- 172.21.15.50:0/1923903520 shutdown_connections 2024-09-21T05:56:43.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.472+0000 7fa880a00640 1 -- 172.21.15.50:0/1923903520 wait complete. 2024-09-21T05:56:43.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.476+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8a7c15e4a0 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.476+0000 7f8a78c00640 1 -- 172.21.15.50:0/3140009728 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8a6c062950 con 0x7f8a7c0cdf30 2024-09-21T05:56:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:64424509446 2024-09-21T05:56:43.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.480+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a60073200 msgr2=0x7f8a60075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.480+0000 7f8a81e00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a60073200 0x7f8a60075710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c164f00 tx=0x7f8a70009260 comp rx=0 tx=0).stop 2024-09-21T05:56:43.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.480+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a7c0cdf30 msgr2=0x7f8a7c1ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.480+0000 7f8a81e00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a7c0cdf30 0x7f8a7c1ffcd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00c3f0 tx=0x7f8a6c00c8b0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.484+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 shutdown_connections 2024-09-21T05:56:43.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.484+0000 7f8a81e00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a60073200 0x7f8a60075710 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.484+0000 7f8a81e00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a7c160330 0x7f8a7c073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.484+0000 7f8a81e00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a7c0d0330 0x7f8a7c16a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.484+0000 7f8a81e00640 1 --2- 172.21.15.50:0/3140009728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a7c0cdf30 0x7f8a7c1ffcd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.484+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 >> 172.21.15.50:0/3140009728 conn(0x7f8a7c06e4a0 msgr2=0x7f8a7c0d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.484+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 shutdown_connections 2024-09-21T05:56:43.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.484+0000 7f8a81e00640 1 -- 172.21.15.50:0/3140009728 wait complete. 2024-09-21T05:56:43.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.496+0000 7f446ca00640 1 -- 172.21.15.50:0/4056336714 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f44681663d0 con 0x7f44680ab910 2024-09-21T05:56:43.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.496+0000 7f445e000640 1 -- 172.21.15.50:0/4056336714 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4454068bf0 con 0x7f44680ab910 2024-09-21T05:56:43.501 INFO:teuthology.orchestra.run.smithi050.stdout:64424509447 2024-09-21T05:56:43.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f446ca00640 1 -- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4448073280 msgr2=0x7f4448075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f446ca00640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4448073280 0x7f4448075790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f445800c6e0 tx=0x7f44580194a0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f446ca00640 1 -- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44680ab910 msgr2=0x7f44681590d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f3c5d400640 1 -- 172.21.15.50:0/2553981060 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3c30005590 con 0x7f3c581527b0 2024-09-21T05:56:43.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f446ca00640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44680ab910 0x7f44681590d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44540039d0 tx=0x7f4454003e90 comp rx=0 tx=0).stop 2024-09-21T05:56:43.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f446ca00640 1 -- 172.21.15.50:0/4056336714 shutdown_connections 2024-09-21T05:56:43.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f446ca00640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4448073280 0x7f4448075790 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f446ca00640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44680aa4d0 0x7f44680aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f446ca00640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44680ab910 0x7f44681590d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f446ca00640 1 --2- 172.21.15.50:0/4056336714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f444c005c20 0x7f44681f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f446ca00640 1 -- 172.21.15.50:0/4056336714 >> 172.21.15.50:0/4056336714 conn(0x7f44680d06e0 msgr2=0x7f44680b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f446ca00640 1 -- 172.21.15.50:0/4056336714 shutdown_connections 2024-09-21T05:56:43.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.504+0000 7f446ca00640 1 -- 172.21.15.50:0/4056336714 wait complete. 2024-09-21T05:56:43.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.500+0000 7f3c4ea00640 1 -- 172.21.15.50:0/2553981060 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3c4405e630 con 0x7f3c581527b0 2024-09-21T05:56:43.507 INFO:teuthology.orchestra.run.smithi050.stdout:64424509447 2024-09-21T05:56:43.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.504+0000 7f3c5d400640 1 -- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3c38073300 msgr2=0x7f3c38075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.504+0000 7f3c5d400640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3c38073300 0x7f3c38075810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3c480af000 tx=0x7f3c4801b080 comp rx=0 tx=0).stop 2024-09-21T05:56:43.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.504+0000 7f3c5d400640 1 -- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c581527b0 msgr2=0x7f3c581589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.504+0000 7f3c5d400640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c581527b0 0x7f3c581589e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400b3e0 tx=0x7f3c4400b8a0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.504+0000 7f3c5d400640 1 -- 172.21.15.50:0/2553981060 shutdown_connections 2024-09-21T05:56:43.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.504+0000 7f3c5d400640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c581527b0 0x7f3c581589e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.504+0000 7f3c5d400640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3c38073300 0x7f3c38075810 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.504+0000 7f3c5d400640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c580b5c70 0x7f3c580ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.504+0000 7f3c5d400640 1 --2- 172.21.15.50:0/2553981060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c580b4de0 0x7f3c58141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.504+0000 7f3c5d400640 1 -- 172.21.15.50:0/2553981060 >> 172.21.15.50:0/2553981060 conn(0x7f3c580b2040 msgr2=0x7f3c5815b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.2 2024-09-21T05:56:43.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.508+0000 7f3c5d400640 1 -- 172.21.15.50:0/2553981060 shutdown_connections 2024-09-21T05:56:43.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.508+0000 7f3c5d400640 1 -- 172.21.15.50:0/2553981060 wait complete. 2024-09-21T05:56:43.512 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:43.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.508+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff660005590 con 0x7ff6881625f0 2024-09-21T05:56:43.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.508+0000 7ff67f400640 1 -- 172.21.15.50:0/1485695773 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff680063530 con 0x7ff6881625f0 2024-09-21T05:56:43.512 INFO:teuthology.orchestra.run.smithi050.stdout:60129542152 2024-09-21T05:56:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.512+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff6640732d0 msgr2=0x7ff6640757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.512+0000 7ff68d200640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff6640732d0 0x7ff6640757e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff678000f70 tx=0x7ff678009040 comp rx=0 tx=0).stop 2024-09-21T05:56:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.512+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6881625f0 msgr2=0x7ff6880c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.512+0000 7ff68d200640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6881625f0 0x7ff6880c9590 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff688162550 tx=0x7ff680009ef0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.512+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 shutdown_connections 2024-09-21T05:56:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.512+0000 7ff68d200640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6881625f0 0x7ff6880c9590 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.512+0000 7ff68d200640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6880d0ce0 0x7ff6880c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.512+0000 7ff68d200640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff6640732d0 0x7ff6640757e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.512+0000 7ff68d200640 1 --2- 172.21.15.50:0/1485695773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6880cf1c0 0x7ff6881535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.512+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 >> 172.21.15.50:0/1485695773 conn(0x7ff688073a10 msgr2=0x7ff68814b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.512+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 shutdown_connections 2024-09-21T05:56:43.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.512+0000 7ff68d200640 1 -- 172.21.15.50:0/1485695773 wait complete. 2024-09-21T05:56:43.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.0 2024-09-21T05:56:43.520 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:43.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.528+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f67b0005590 con 0x7f67d815b690 2024-09-21T05:56:43.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.528+0000 7f67cf400640 1 -- 172.21.15.50:0/3536677644 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f67c4061fc0 con 0x7f67d815b690 2024-09-21T05:56:43.532 INFO:teuthology.orchestra.run.smithi050.stdout:64424509446 2024-09-21T05:56:43.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.532+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67b8073600 msgr2=0x7f67b8075b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.532+0000 7f67dca00640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67b8073600 0x7f67b8075b10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f67d0000f90 tx=0x7f67d00a7040 comp rx=0 tx=0).stop 2024-09-21T05:56:43.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.532+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f67d815b690 msgr2=0x7f67d814b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.532+0000 7f67dca00640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f67d815b690 0x7f67d814b4e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f67c400c390 tx=0x7f67c400c850 comp rx=0 tx=0).stop 2024-09-21T05:56:43.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.532+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 shutdown_connections 2024-09-21T05:56:43.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.532+0000 7f67dca00640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67b8073600 0x7f67b8075b10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.532+0000 7f67dca00640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67d815f730 0x7f67d8154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.532+0000 7f67dca00640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f67d815b690 0x7f67d814b4e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.532+0000 7f67dca00640 1 --2- 172.21.15.50:0/3536677644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d815b280 0x7f67d8150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.532+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 >> 172.21.15.50:0/3536677644 conn(0x7f67d806e4a0 msgr2=0x7f67d80cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.532+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 shutdown_connections 2024-09-21T05:56:43.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.532+0000 7f67dca00640 1 -- 172.21.15.50:0/3536677644 wait complete. 2024-09-21T05:56:43.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-09-21T05:56:43.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.540+0000 7fb58e400640 1 -- 172.21.15.50:0/1318617711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb564005860 con 0x7fb5880d0500 2024-09-21T05:56:43.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.540+0000 7fb584c00640 1 -- 172.21.15.50:0/1318617711 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb57c05e7e0 con 0x7fb5880d0500 2024-09-21T05:56:43.543 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:43.543 INFO:teuthology.orchestra.run.smithi050.stdout:64424509446 2024-09-21T05:56:43.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.540+0000 7fb58e400640 1 -- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb56c073300 msgr2=0x7fb56c075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.540+0000 7fb58e400640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb56c073300 0x7fb56c075810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5780ae000 tx=0x7fb57801b040 comp rx=0 tx=0).stop 2024-09-21T05:56:43.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.544+0000 7fb58e400640 1 -- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5880d0500 msgr2=0x7fb58816c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.544+0000 7fb58e400640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5880d0500 0x7fb58816c250 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00b410 tx=0x7fb57c00b8d0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-09-21T05:56:43.548 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:43.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.544+0000 7fb58e400640 1 -- 172.21.15.50:0/1318617711 shutdown_connections 2024-09-21T05:56:43.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.544+0000 7fb58e400640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb5880d0910 0x7fb5882006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.544+0000 7fb58e400640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5880d0500 0x7fb58816c250 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.544+0000 7fb58e400640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb56c073300 0x7fb56c075810 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.544+0000 7fb58e400640 1 --2- 172.21.15.50:0/1318617711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb58806fd70 0x7fb5881fdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.544+0000 7fb58e400640 1 -- 172.21.15.50:0/1318617711 >> 172.21.15.50:0/1318617711 conn(0x7fb58806e4a0 msgr2=0x7fb5880b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.544+0000 7fb58e400640 1 -- 172.21.15.50:0/1318617711 shutdown_connections 2024-09-21T05:56:43.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.544+0000 7fb58e400640 1 -- 172.21.15.50:0/1318617711 wait complete. 2024-09-21T05:56:43.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542152 for osd.1 2024-09-21T05:56:43.551 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:43.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2024-09-21T05:56:43.560 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:43.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-09-21T05:56:43.570 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:43.570 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:56:43.570 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:56:43.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.772+0000 7fe4d3a00640 1 -- 172.21.15.50:0/2353018673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4cc0749a0 msgr2=0x7fe4cc074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.772+0000 7fe4d3a00640 1 --2- 172.21.15.50:0/2353018673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4cc0749a0 0x7fe4cc074db0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009840 tx=0x7fe4bc01c820 comp rx=0 tx=0).stop 2024-09-21T05:56:43.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.772+0000 7fe4d3a00640 1 -- 172.21.15.50:0/2353018673 shutdown_connections 2024-09-21T05:56:43.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.772+0000 7fe4d3a00640 1 --2- 172.21.15.50:0/2353018673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4cc0749a0 0x7fe4cc074db0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.772+0000 7fe4d3a00640 1 -- 172.21.15.50:0/2353018673 >> 172.21.15.50:0/2353018673 conn(0x7fe4cc0b2040 msgr2=0x7fe4cc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.772+0000 7fe4d3a00640 1 -- 172.21.15.50:0/2353018673 shutdown_connections 2024-09-21T05:56:43.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.772+0000 7fe4d3a00640 1 -- 172.21.15.50:0/2353018673 wait complete. 2024-09-21T05:56:43.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.772+0000 7fe4d3a00640 1 Processor -- start 2024-09-21T05:56:43.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.772+0000 7fe4d3a00640 1 -- start start 2024-09-21T05:56:43.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d3a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe4cc0749a0 0x7fe4cc15a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d3a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4cc0b3340 0x7fe4cc139960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d3a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4cc0cefa0 0x7fe4cc139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d3a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc16b2e0 con 0x7fe4cc0b3340 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d3a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc16b160 con 0x7fe4cc0cefa0 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d3a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe4cc16b460 con 0x7fe4cc0749a0 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4cc0b3340 0x7fe4cc139960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4cc0b3340 0x7fe4cc139960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48756/0 (socket says 172.21.15.50:48756) 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4c3e00640 1 -- 172.21.15.50:0/497005007 learned_addr learned my addr 172.21.15.50:0/497005007 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d1a00640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4cc0cefa0 0x7fe4cc139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d1000640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe4cc0749a0 0x7fe4cc15a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4c3e00640 1 -- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe4cc0749a0 msgr2=0x7fe4cc15a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4c3e00640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe4cc0749a0 0x7fe4cc15a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4c3e00640 1 -- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4cc0cefa0 msgr2=0x7fe4cc139ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4c3e00640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4cc0cefa0 0x7fe4cc139ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4c3e00640 1 -- 172.21.15.50:0/497005007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc143e70 con 0x7fe4cc0b3340 2024-09-21T05:56:43.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d1a00640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4cc0cefa0 0x7fe4cc139ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:43.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d1000640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe4cc0749a0 0x7fe4cc15a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:43.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4c3e00640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4cc0b3340 0x7fe4cc139960 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400c400 tx=0x7fe4b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4c9600640 1 -- 172.21.15.50:0/497005007 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4011790 con 0x7fe4cc0b3340 2024-09-21T05:56:43.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4c9600640 1 -- 172.21.15.50:0/497005007 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4b4017070 con 0x7fe4cc0b3340 2024-09-21T05:56:43.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d3a00640 1 -- 172.21.15.50:0/497005007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc15a6f0 con 0x7fe4cc0b3340 2024-09-21T05:56:43.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4c9600640 1 -- 172.21.15.50:0/497005007 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4011cc0 con 0x7fe4cc0b3340 2024-09-21T05:56:43.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d3a00640 1 -- 172.21.15.50:0/497005007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc1443f0 con 0x7fe4cc0b3340 2024-09-21T05:56:43.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.776+0000 7fe4d3a00640 1 -- 172.21.15.50:0/497005007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4cc13e480 con 0x7fe4cc0b3340 2024-09-21T05:56:43.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.780+0000 7fe4c9600640 1 -- 172.21.15.50:0/497005007 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe4b401c460 con 0x7fe4cc0b3340 2024-09-21T05:56:43.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.780+0000 7fe4c9600640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe4a8073200 0x7fe4a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.780+0000 7fe4c9600640 1 -- 172.21.15.50:0/497005007 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe4b4094110 con 0x7fe4cc0b3340 2024-09-21T05:56:43.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.780+0000 7fe4d1000640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe4a8073200 0x7fe4a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.780+0000 7fe4d1000640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe4a8073200 0x7fe4a8075710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0af000 tx=0x7fe4bc01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.784+0000 7fe4c9600640 1 -- 172.21.15.50:0/497005007 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe4b405f090 con 0x7fe4cc0b3340 2024-09-21T05:56:43.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.956+0000 7fe4d3a00640 1 -- 172.21.15.50:0/497005007 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4cc06b540 con 0x7fe4a8073200 2024-09-21T05:56:43.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4c9600640 1 -- 172.21.15.50:0/497005007 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7fe4bc004040 con 0x7fe4a8073200 2024-09-21T05:56:43.968 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:43.969 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:56:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4d3a00640 1 -- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe4a8073200 msgr2=0x7fe4a8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4d3a00640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe4a8073200 0x7fe4a8075710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0af000 tx=0x7fe4bc01b060 comp rx=0 tx=0).stop 2024-09-21T05:56:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4d3a00640 1 -- 172.21.15.50:0/497005007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4cc0b3340 msgr2=0x7fe4cc139960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4d3a00640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4cc0b3340 0x7fe4cc139960 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400c400 tx=0x7fe4b400c8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4d3a00640 1 -- 172.21.15.50:0/497005007 shutdown_connections 2024-09-21T05:56:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4d3a00640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4cc0cefa0 0x7fe4cc139ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4d3a00640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe4a8073200 0x7fe4a8075710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4d3a00640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4cc0b3340 0x7fe4cc139960 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4d3a00640 1 --2- 172.21.15.50:0/497005007 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe4cc0749a0 0x7fe4cc15a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4d3a00640 1 -- 172.21.15.50:0/497005007 >> 172.21.15.50:0/497005007 conn(0x7fe4cc0b2040 msgr2=0x7fe4cc0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4d3a00640 1 -- 172.21.15.50:0/497005007 shutdown_connections 2024-09-21T05:56:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:43.964+0000 7fe4d3a00640 1 -- 172.21.15.50:0/497005007 wait complete. 2024-09-21T05:56:43.990 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-21T05:56:42.887046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235840,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738880,"statfs":{"total":773094113280,"available":772852613120,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002893"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176632+0000","last_change":"2024-09-21T05:56:29.801294+0000","last_active":"2024-09-21T05:56:31.176632+0000","last_peered":"2024-09-21T05:56:31.176632+0000","last_clean":"2024-09-21T05:56:31.176632+0000","last_became_active":"2024-09-21T05:56:29.800947+0000","last_became_peered":"2024-09-21T05:56:29.800947+0000","last_unstale":"2024-09-21T05:56:31.176632+0000","last_undegraded":"2024-09-21T05:56:31.176632+0000","last_fullsized":"2024-09-21T05:56:31.176632+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:11.951093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780332+0000","last_change":"2024-09-21T05:56:29.802492+0000","last_active":"2024-09-21T05:56:30.780332+0000","last_peered":"2024-09-21T05:56:30.780332+0000","last_clean":"2024-09-21T05:56:30.780332+0000","last_became_active":"2024-09-21T05:56:29.801940+0000","last_became_peered":"2024-09-21T05:56:29.801940+0000","last_unstale":"2024-09-21T05:56:30.780332+0000","last_undegraded":"2024-09-21T05:56:30.780332+0000","last_fullsized":"2024-09-21T05:56:30.780332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:11.434269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178417+0000","last_change":"2024-09-21T05:56:29.804599+0000","last_active":"2024-09-21T05:56:31.178417+0000","last_peered":"2024-09-21T05:56:31.178417+0000","last_clean":"2024-09-21T05:56:31.178417+0000","last_became_active":"2024-09-21T05:56:29.804231+0000","last_became_peered":"2024-09-21T05:56:29.804231+0000","last_unstale":"2024-09-21T05:56:31.178417+0000","last_undegraded":"2024-09-21T05:56:31.178417+0000","last_fullsized":"2024-09-21T05:56:31.178417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782989+0000","last_change":"2024-09-21T05:56:29.790060+0000","last_active":"2024-09-21T05:56:30.782989+0000","last_peered":"2024-09-21T05:56:30.782989+0000","last_clean":"2024-09-21T05:56:30.782989+0000","last_became_active":"2024-09-21T05:56:29.786267+0000","last_became_peered":"2024-09-21T05:56:29.786267+0000","last_unstale":"2024-09-21T05:56:30.782989+0000","last_undegraded":"2024-09-21T05:56:30.782989+0000","last_fullsized":"2024-09-21T05:56:30.782989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:11.867996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176068+0000","last_change":"2024-09-21T05:56:29.809836+0000","last_active":"2024-09-21T05:56:31.176068+0000","last_peered":"2024-09-21T05:56:31.176068+0000","last_clean":"2024-09-21T05:56:31.176068+0000","last_became_active":"2024-09-21T05:56:29.809429+0000","last_became_peered":"2024-09-21T05:56:29.809429+0000","last_unstale":"2024-09-21T05:56:31.176068+0000","last_undegraded":"2024-09-21T05:56:31.176068+0000","last_fullsized":"2024-09-21T05:56:31.176068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:41.759804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782594+0000","last_change":"2024-09-21T05:56:29.794338+0000","last_active":"2024-09-21T05:56:30.782594+0000","last_peered":"2024-09-21T05:56:30.782594+0000","last_clean":"2024-09-21T05:56:30.782594+0000","last_became_active":"2024-09-21T05:56:29.794011+0000","last_became_peered":"2024-09-21T05:56:29.794011+0000","last_unstale":"2024-09-21T05:56:30.782594+0000","last_undegraded":"2024-09-21T05:56:30.782594+0000","last_fullsized":"2024-09-21T05:56:30.782594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:37.695152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176168+0000","last_change":"2024-09-21T05:56:29.806114+0000","last_active":"2024-09-21T05:56:31.176168+0000","last_peered":"2024-09-21T05:56:31.176168+0000","last_clean":"2024-09-21T05:56:31.176168+0000","last_became_active":"2024-09-21T05:56:29.805711+0000","last_became_peered":"2024-09-21T05:56:29.805711+0000","last_unstale":"2024-09-21T05:56:31.176168+0000","last_undegraded":"2024-09-21T05:56:31.176168+0000","last_fullsized":"2024-09-21T05:56:31.176168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:11.867996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177775+0000","last_change":"2024-09-21T05:56:29.800463+0000","last_active":"2024-09-21T05:56:31.177775+0000","last_peered":"2024-09-21T05:56:31.177775+0000","last_clean":"2024-09-21T05:56:31.177775+0000","last_became_active":"2024-09-21T05:56:29.800084+0000","last_became_peered":"2024-09-21T05:56:29.800084+0000","last_unstale":"2024-09-21T05:56:31.177775+0000","last_undegraded":"2024-09-21T05:56:31.177775+0000","last_fullsized":"2024-09-21T05:56:31.177775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:39.467641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176639+0000","last_change":"2024-09-21T05:56:29.809950+0000","last_active":"2024-09-21T05:56:31.176639+0000","last_peered":"2024-09-21T05:56:31.176639+0000","last_clean":"2024-09-21T05:56:31.176639+0000","last_became_active":"2024-09-21T05:56:29.809337+0000","last_became_peered":"2024-09-21T05:56:29.809337+0000","last_unstale":"2024-09-21T05:56:31.176639+0000","last_undegraded":"2024-09-21T05:56:31.176639+0000","last_fullsized":"2024-09-21T05:56:31.176639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:30.461662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796623+0000","last_change":"2024-09-21T05:56:29.781023+0000","last_active":"2024-09-21T05:56:30.796623+0000","last_peered":"2024-09-21T05:56:30.796623+0000","last_clean":"2024-09-21T05:56:30.796623+0000","last_became_active":"2024-09-21T05:56:29.780729+0000","last_became_peered":"2024-09-21T05:56:29.780729+0000","last_unstale":"2024-09-21T05:56:30.796623+0000","last_undegraded":"2024-09-21T05:56:30.796623+0000","last_fullsized":"2024-09-21T05:56:30.796623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:22.412975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796667+0000","last_change":"2024-09-21T05:56:29.779393+0000","last_active":"2024-09-21T05:56:30.796667+0000","last_peered":"2024-09-21T05:56:30.796667+0000","last_clean":"2024-09-21T05:56:30.796667+0000","last_became_active":"2024-09-21T05:56:29.779016+0000","last_became_peered":"2024-09-21T05:56:29.779016+0000","last_unstale":"2024-09-21T05:56:30.796667+0000","last_undegraded":"2024-09-21T05:56:30.796667+0000","last_fullsized":"2024-09-21T05:56:30.796667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:30.792281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.781580+0000","last_change":"2024-09-21T05:56:29.787708+0000","last_active":"2024-09-21T05:56:30.781580+0000","last_peered":"2024-09-21T05:56:30.781580+0000","last_clean":"2024-09-21T05:56:30.781580+0000","last_became_active":"2024-09-21T05:56:29.787370+0000","last_became_peered":"2024-09-21T05:56:29.787370+0000","last_unstale":"2024-09-21T05:56:30.781580+0000","last_undegraded":"2024-09-21T05:56:30.781580+0000","last_fullsized":"2024-09-21T05:56:30.781580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:11.868532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177932+0000","last_change":"2024-09-21T05:56:29.790480+0000","last_active":"2024-09-21T05:56:31.177932+0000","last_peered":"2024-09-21T05:56:31.177932+0000","last_clean":"2024-09-21T05:56:31.177932+0000","last_became_active":"2024-09-21T05:56:29.790172+0000","last_became_peered":"2024-09-21T05:56:29.790172+0000","last_unstale":"2024-09-21T05:56:31.177932+0000","last_undegraded":"2024-09-21T05:56:31.177932+0000","last_fullsized":"2024-09-21T05:56:31.177932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:14.929983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178355+0000","last_change":"2024-09-21T05:56:29.796096+0000","last_active":"2024-09-21T05:56:31.178355+0000","last_peered":"2024-09-21T05:56:31.178355+0000","last_clean":"2024-09-21T05:56:31.178355+0000","last_became_active":"2024-09-21T05:56:29.795760+0000","last_became_peered":"2024-09-21T05:56:29.795760+0000","last_unstale":"2024-09-21T05:56:31.178355+0000","last_undegraded":"2024-09-21T05:56:31.178355+0000","last_fullsized":"2024-09-21T05:56:31.178355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:33.544641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779759+0000","last_change":"2024-09-21T05:56:29.789973+0000","last_active":"2024-09-21T05:56:30.779759+0000","last_peered":"2024-09-21T05:56:30.779759+0000","last_clean":"2024-09-21T05:56:30.779759+0000","last_became_active":"2024-09-21T05:56:29.789200+0000","last_became_peered":"2024-09-21T05:56:29.789200+0000","last_unstale":"2024-09-21T05:56:30.779759+0000","last_undegraded":"2024-09-21T05:56:30.779759+0000","last_fullsized":"2024-09-21T05:56:30.779759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:52.718337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782539+0000","last_change":"2024-09-21T05:56:29.795924+0000","last_active":"2024-09-21T05:56:30.782539+0000","last_peered":"2024-09-21T05:56:30.782539+0000","last_clean":"2024-09-21T05:56:30.782539+0000","last_became_active":"2024-09-21T05:56:29.795518+0000","last_became_peered":"2024-09-21T05:56:29.795518+0000","last_unstale":"2024-09-21T05:56:30.782539+0000","last_undegraded":"2024-09-21T05:56:30.782539+0000","last_fullsized":"2024-09-21T05:56:30.782539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:56.052565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.784178+0000","last_change":"2024-09-21T05:56:29.803208+0000","last_active":"2024-09-21T05:56:30.784178+0000","last_peered":"2024-09-21T05:56:30.784178+0000","last_clean":"2024-09-21T05:56:30.784178+0000","last_became_active":"2024-09-21T05:56:29.802492+0000","last_became_peered":"2024-09-21T05:56:29.802492+0000","last_unstale":"2024-09-21T05:56:30.784178+0000","last_undegraded":"2024-09-21T05:56:30.784178+0000","last_fullsized":"2024-09-21T05:56:30.784178+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:11.140043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177406+0000","last_change":"2024-09-21T05:56:29.787284+0000","last_active":"2024-09-21T05:56:31.177406+0000","last_peered":"2024-09-21T05:56:31.177406+0000","last_clean":"2024-09-21T05:56:31.177406+0000","last_became_active":"2024-09-21T05:56:29.786932+0000","last_became_peered":"2024-09-21T05:56:29.786932+0000","last_unstale":"2024-09-21T05:56:31.177406+0000","last_undegraded":"2024-09-21T05:56:31.177406+0000","last_fullsized":"2024-09-21T05:56:31.177406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:45.264497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176533+0000","last_change":"2024-09-21T05:56:29.810657+0000","last_active":"2024-09-21T05:56:31.176533+0000","last_peered":"2024-09-21T05:56:31.176533+0000","last_clean":"2024-09-21T05:56:31.176533+0000","last_became_active":"2024-09-21T05:56:29.810300+0000","last_became_peered":"2024-09-21T05:56:29.810300+0000","last_unstale":"2024-09-21T05:56:31.176533+0000","last_undegraded":"2024-09-21T05:56:31.176533+0000","last_fullsized":"2024-09-21T05:56:31.176533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:28.055629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796891+0000","last_change":"2024-09-21T05:56:29.788766+0000","last_active":"2024-09-21T05:56:30.796891+0000","last_peered":"2024-09-21T05:56:30.796891+0000","last_clean":"2024-09-21T05:56:30.796891+0000","last_became_active":"2024-09-21T05:56:29.788430+0000","last_became_peered":"2024-09-21T05:56:29.788430+0000","last_unstale":"2024-09-21T05:56:30.796891+0000","last_undegraded":"2024-09-21T05:56:30.796891+0000","last_fullsized":"2024-09-21T05:56:30.796891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178114+0000","last_change":"2024-09-21T05:56:29.813309+0000","last_active":"2024-09-21T05:56:31.178114+0000","last_peered":"2024-09-21T05:56:31.178114+0000","last_clean":"2024-09-21T05:56:31.178114+0000","last_became_active":"2024-09-21T05:56:29.812998+0000","last_became_peered":"2024-09-21T05:56:29.812998+0000","last_unstale":"2024-09-21T05:56:31.178114+0000","last_undegraded":"2024-09-21T05:56:31.178114+0000","last_fullsized":"2024-09-21T05:56:31.178114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:30.792281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782744+0000","last_change":"2024-09-21T05:56:29.803935+0000","last_active":"2024-09-21T05:56:30.782744+0000","last_peered":"2024-09-21T05:56:30.782744+0000","last_clean":"2024-09-21T05:56:30.782744+0000","last_became_active":"2024-09-21T05:56:29.803639+0000","last_became_peered":"2024-09-21T05:56:29.803639+0000","last_unstale":"2024-09-21T05:56:30.782744+0000","last_undegraded":"2024-09-21T05:56:30.782744+0000","last_fullsized":"2024-09-21T05:56:30.782744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:41.759804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177783+0000","last_change":"2024-09-21T05:56:29.800369+0000","last_active":"2024-09-21T05:56:31.177783+0000","last_peered":"2024-09-21T05:56:31.177783+0000","last_clean":"2024-09-21T05:56:31.177783+0000","last_became_active":"2024-09-21T05:56:29.799794+0000","last_became_peered":"2024-09-21T05:56:29.799794+0000","last_unstale":"2024-09-21T05:56:31.177783+0000","last_undegraded":"2024-09-21T05:56:31.177783+0000","last_fullsized":"2024-09-21T05:56:31.177783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:17.854511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782813+0000","last_change":"2024-09-21T05:56:29.795633+0000","last_active":"2024-09-21T05:56:30.782813+0000","last_peered":"2024-09-21T05:56:30.782813+0000","last_clean":"2024-09-21T05:56:30.782813+0000","last_became_active":"2024-09-21T05:56:29.795285+0000","last_became_peered":"2024-09-21T05:56:29.795285+0000","last_unstale":"2024-09-21T05:56:30.782813+0000","last_undegraded":"2024-09-21T05:56:30.782813+0000","last_fullsized":"2024-09-21T05:56:30.782813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:04.441455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178478+0000","last_change":"2024-09-21T05:56:29.803384+0000","last_active":"2024-09-21T05:56:31.178478+0000","last_peered":"2024-09-21T05:56:31.178478+0000","last_clean":"2024-09-21T05:56:31.178478+0000","last_became_active":"2024-09-21T05:56:29.803152+0000","last_became_peered":"2024-09-21T05:56:29.803152+0000","last_unstale":"2024-09-21T05:56:31.178478+0000","last_undegraded":"2024-09-21T05:56:31.178478+0000","last_fullsized":"2024-09-21T05:56:31.178478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:48.777031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796724+0000","last_change":"2024-09-21T05:56:29.807556+0000","last_active":"2024-09-21T05:56:30.796724+0000","last_peered":"2024-09-21T05:56:30.796724+0000","last_clean":"2024-09-21T05:56:30.796724+0000","last_became_active":"2024-09-21T05:56:29.805826+0000","last_became_peered":"2024-09-21T05:56:29.805826+0000","last_unstale":"2024-09-21T05:56:30.796724+0000","last_undegraded":"2024-09-21T05:56:30.796724+0000","last_fullsized":"2024-09-21T05:56:30.796724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:33.544641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779890+0000","last_change":"2024-09-21T05:56:29.788425+0000","last_active":"2024-09-21T05:56:30.779890+0000","last_peered":"2024-09-21T05:56:30.779890+0000","last_clean":"2024-09-21T05:56:30.779890+0000","last_became_active":"2024-09-21T05:56:29.788007+0000","last_became_peered":"2024-09-21T05:56:29.788007+0000","last_unstale":"2024-09-21T05:56:30.779890+0000","last_undegraded":"2024-09-21T05:56:30.779890+0000","last_fullsized":"2024-09-21T05:56:30.779890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:53.445482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.179201+0000","last_change":"2024-09-21T05:56:29.792314+0000","last_active":"2024-09-21T05:56:31.179201+0000","last_peered":"2024-09-21T05:56:31.179201+0000","last_clean":"2024-09-21T05:56:31.179201+0000","last_became_active":"2024-09-21T05:56:29.791869+0000","last_became_peered":"2024-09-21T05:56:29.791869+0000","last_unstale":"2024-09-21T05:56:31.179201+0000","last_undegraded":"2024-09-21T05:56:31.179201+0000","last_fullsized":"2024-09-21T05:56:31.179201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:20.952370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177862+0000","last_change":"2024-09-21T05:56:29.787078+0000","last_active":"2024-09-21T05:56:31.177862+0000","last_peered":"2024-09-21T05:56:31.177862+0000","last_clean":"2024-09-21T05:56:31.177862+0000","last_became_active":"2024-09-21T05:56:29.786725+0000","last_became_peered":"2024-09-21T05:56:29.786725+0000","last_unstale":"2024-09-21T05:56:31.177862+0000","last_undegraded":"2024-09-21T05:56:31.177862+0000","last_fullsized":"2024-09-21T05:56:31.177862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:17.854511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782870+0000","last_change":"2024-09-21T05:56:29.790180+0000","last_active":"2024-09-21T05:56:30.782870+0000","last_peered":"2024-09-21T05:56:30.782870+0000","last_clean":"2024-09-21T05:56:30.782870+0000","last_became_active":"2024-09-21T05:56:29.789793+0000","last_became_peered":"2024-09-21T05:56:29.789793+0000","last_unstale":"2024-09-21T05:56:30.782870+0000","last_undegraded":"2024-09-21T05:56:30.782870+0000","last_fullsized":"2024-09-21T05:56:30.782870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:48.361541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176727+0000","last_change":"2024-09-21T05:56:29.791764+0000","last_active":"2024-09-21T05:56:31.176727+0000","last_peered":"2024-09-21T05:56:31.176727+0000","last_clean":"2024-09-21T05:56:31.176727+0000","last_became_active":"2024-09-21T05:56:29.791358+0000","last_became_peered":"2024-09-21T05:56:29.791358+0000","last_unstale":"2024-09-21T05:56:31.176727+0000","last_undegraded":"2024-09-21T05:56:31.176727+0000","last_fullsized":"2024-09-21T05:56:31.176727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:35.770789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796700+0000","last_change":"2024-09-21T05:56:29.798644+0000","last_active":"2024-09-21T05:56:30.796700+0000","last_peered":"2024-09-21T05:56:30.796700+0000","last_clean":"2024-09-21T05:56:30.796700+0000","last_became_active":"2024-09-21T05:56:29.798271+0000","last_became_peered":"2024-09-21T05:56:29.798271+0000","last_unstale":"2024-09-21T05:56:30.796700+0000","last_undegraded":"2024-09-21T05:56:30.796700+0000","last_fullsized":"2024-09-21T05:56:30.796700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:39.467641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780441+0000","last_change":"2024-09-21T05:56:29.786401+0000","last_active":"2024-09-21T05:56:30.780441+0000","last_peered":"2024-09-21T05:56:30.780441+0000","last_clean":"2024-09-21T05:56:30.780441+0000","last_became_active":"2024-09-21T05:56:29.786004+0000","last_became_peered":"2024-09-21T05:56:29.786004+0000","last_unstale":"2024-09-21T05:56:30.780441+0000","last_undegraded":"2024-09-21T05:56:30.780441+0000","last_fullsized":"2024-09-21T05:56:30.780441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:32.904074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178010+0000","last_change":"2024-09-21T05:56:28.806999+0000","last_active":"2024-09-21T05:56:31.178010+0000","last_peered":"2024-09-21T05:56:31.178010+0000","last_clean":"2024-09-21T05:56:31.178010+0000","last_became_active":"2024-09-21T05:56:28.806609+0000","last_became_peered":"2024-09-21T05:56:28.806609+0000","last_unstale":"2024-09-21T05:56:31.178010+0000","last_undegraded":"2024-09-21T05:56:31.178010+0000","last_fullsized":"2024-09-21T05:56:31.178010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:11.564758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178105+0000","last_change":"2024-09-21T05:56:29.799191+0000","last_active":"2024-09-21T05:56:31.178105+0000","last_peered":"2024-09-21T05:56:31.178105+0000","last_clean":"2024-09-21T05:56:31.178105+0000","last_became_active":"2024-09-21T05:56:29.798839+0000","last_became_peered":"2024-09-21T05:56:29.798839+0000","last_unstale":"2024-09-21T05:56:31.178105+0000","last_undegraded":"2024-09-21T05:56:31.178105+0000","last_fullsized":"2024-09-21T05:56:31.178105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:28.783969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796226+0000","last_change":"2024-09-21T05:56:28.794189+0000","last_active":"2024-09-21T05:56:30.796226+0000","last_peered":"2024-09-21T05:56:30.796226+0000","last_clean":"2024-09-21T05:56:30.796226+0000","last_became_active":"2024-09-21T05:56:28.793684+0000","last_became_peered":"2024-09-21T05:56:28.793684+0000","last_unstale":"2024-09-21T05:56:30.796226+0000","last_undegraded":"2024-09-21T05:56:30.796226+0000","last_fullsized":"2024-09-21T05:56:30.796226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:08.868280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177368+0000","last_change":"2024-09-21T05:56:28.788375+0000","last_active":"2024-09-21T05:56:31.177368+0000","last_peered":"2024-09-21T05:56:31.177368+0000","last_clean":"2024-09-21T05:56:31.177368+0000","last_became_active":"2024-09-21T05:56:28.788044+0000","last_became_peered":"2024-09-21T05:56:28.788044+0000","last_unstale":"2024-09-21T05:56:31.177368+0000","last_undegraded":"2024-09-21T05:56:31.177368+0000","last_fullsized":"2024-09-21T05:56:31.177368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:11.883400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177781+0000","last_change":"2024-09-21T05:56:29.792845+0000","last_active":"2024-09-21T05:56:31.177781+0000","last_peered":"2024-09-21T05:56:31.177781+0000","last_clean":"2024-09-21T05:56:31.177781+0000","last_became_active":"2024-09-21T05:56:29.792452+0000","last_became_peered":"2024-09-21T05:56:29.792452+0000","last_unstale":"2024-09-21T05:56:31.177781+0000","last_undegraded":"2024-09-21T05:56:31.177781+0000","last_fullsized":"2024-09-21T05:56:31.177781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:14.751206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782637+0000","last_change":"2024-09-21T05:56:28.796778+0000","last_active":"2024-09-21T05:56:30.782637+0000","last_peered":"2024-09-21T05:56:30.782637+0000","last_clean":"2024-09-21T05:56:30.782637+0000","last_became_active":"2024-09-21T05:56:28.792745+0000","last_became_peered":"2024-09-21T05:56:28.792745+0000","last_unstale":"2024-09-21T05:56:30.782637+0000","last_undegraded":"2024-09-21T05:56:30.782637+0000","last_fullsized":"2024-09-21T05:56:30.782637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:33.703504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779980+0000","last_change":"2024-09-21T05:56:29.790318+0000","last_active":"2024-09-21T05:56:30.779980+0000","last_peered":"2024-09-21T05:56:30.779980+0000","last_clean":"2024-09-21T05:56:30.779980+0000","last_became_active":"2024-09-21T05:56:29.789628+0000","last_became_peered":"2024-09-21T05:56:29.789628+0000","last_unstale":"2024-09-21T05:56:30.779980+0000","last_undegraded":"2024-09-21T05:56:30.779980+0000","last_fullsized":"2024-09-21T05:56:30.779980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:39.627991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796583+0000","last_change":"2024-09-21T05:56:28.798412+0000","last_active":"2024-09-21T05:56:30.796583+0000","last_peered":"2024-09-21T05:56:30.796583+0000","last_clean":"2024-09-21T05:56:30.796583+0000","last_became_active":"2024-09-21T05:56:28.796923+0000","last_became_peered":"2024-09-21T05:56:28.796923+0000","last_unstale":"2024-09-21T05:56:30.796583+0000","last_undegraded":"2024-09-21T05:56:30.796583+0000","last_fullsized":"2024-09-21T05:56:30.796583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:31.896210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783435+0000","last_change":"2024-09-21T05:56:29.801946+0000","last_active":"2024-09-21T05:56:30.783435+0000","last_peered":"2024-09-21T05:56:30.783435+0000","last_clean":"2024-09-21T05:56:30.783435+0000","last_became_active":"2024-09-21T05:56:29.801320+0000","last_became_peered":"2024-09-21T05:56:29.801320+0000","last_unstale":"2024-09-21T05:56:30.783435+0000","last_undegraded":"2024-09-21T05:56:30.783435+0000","last_fullsized":"2024-09-21T05:56:30.783435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:28.055629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783473+0000","last_change":"2024-09-21T05:56:28.792022+0000","last_active":"2024-09-21T05:56:30.783473+0000","last_peered":"2024-09-21T05:56:30.783473+0000","last_clean":"2024-09-21T05:56:30.783473+0000","last_became_active":"2024-09-21T05:56:28.789954+0000","last_became_peered":"2024-09-21T05:56:28.789954+0000","last_unstale":"2024-09-21T05:56:30.783473+0000","last_undegraded":"2024-09-21T05:56:30.783473+0000","last_fullsized":"2024-09-21T05:56:30.783473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:31.896210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177113+0000","last_change":"2024-09-21T05:56:29.799322+0000","last_active":"2024-09-21T05:56:31.177113+0000","last_peered":"2024-09-21T05:56:31.177113+0000","last_clean":"2024-09-21T05:56:31.177113+0000","last_became_active":"2024-09-21T05:56:29.798684+0000","last_became_peered":"2024-09-21T05:56:29.798684+0000","last_unstale":"2024-09-21T05:56:31.177113+0000","last_undegraded":"2024-09-21T05:56:31.177113+0000","last_fullsized":"2024-09-21T05:56:31.177113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:13.158342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779982+0000","last_change":"2024-09-21T05:56:28.791961+0000","last_active":"2024-09-21T05:56:30.779982+0000","last_peered":"2024-09-21T05:56:30.779982+0000","last_clean":"2024-09-21T05:56:30.779982+0000","last_became_active":"2024-09-21T05:56:28.791654+0000","last_became_peered":"2024-09-21T05:56:28.791654+0000","last_unstale":"2024-09-21T05:56:30.779982+0000","last_undegraded":"2024-09-21T05:56:30.779982+0000","last_fullsized":"2024-09-21T05:56:30.779982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:29.186182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.781638+0000","last_change":"2024-09-21T05:56:29.808824+0000","last_active":"2024-09-21T05:56:30.781638+0000","last_peered":"2024-09-21T05:56:30.781638+0000","last_clean":"2024-09-21T05:56:30.781638+0000","last_became_active":"2024-09-21T05:56:29.808454+0000","last_became_peered":"2024-09-21T05:56:29.808454+0000","last_unstale":"2024-09-21T05:56:30.781638+0000","last_undegraded":"2024-09-21T05:56:30.781638+0000","last_fullsized":"2024-09-21T05:56:30.781638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:34.185949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796804+0000","last_change":"2024-09-21T05:56:28.785344+0000","last_active":"2024-09-21T05:56:30.796804+0000","last_peered":"2024-09-21T05:56:30.796804+0000","last_clean":"2024-09-21T05:56:30.796804+0000","last_became_active":"2024-09-21T05:56:28.784997+0000","last_became_peered":"2024-09-21T05:56:28.784997+0000","last_unstale":"2024-09-21T05:56:30.796804+0000","last_undegraded":"2024-09-21T05:56:30.796804+0000","last_fullsized":"2024-09-21T05:56:30.796804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176070+0000","last_change":"2024-09-21T05:56:29.801655+0000","last_active":"2024-09-21T05:56:31.176070+0000","last_peered":"2024-09-21T05:56:31.176070+0000","last_clean":"2024-09-21T05:56:31.176070+0000","last_became_active":"2024-09-21T05:56:29.801337+0000","last_became_peered":"2024-09-21T05:56:29.801337+0000","last_unstale":"2024-09-21T05:56:31.176070+0000","last_undegraded":"2024-09-21T05:56:31.176070+0000","last_fullsized":"2024-09-21T05:56:31.176070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:42.929175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178176+0000","last_change":"2024-09-21T05:56:28.788833+0000","last_active":"2024-09-21T05:56:31.178176+0000","last_peered":"2024-09-21T05:56:31.178176+0000","last_clean":"2024-09-21T05:56:31.178176+0000","last_became_active":"2024-09-21T05:56:28.788469+0000","last_became_peered":"2024-09-21T05:56:28.788469+0000","last_unstale":"2024-09-21T05:56:31.178176+0000","last_undegraded":"2024-09-21T05:56:31.178176+0000","last_fullsized":"2024-09-21T05:56:31.178176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:44.013815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177487+0000","last_change":"2024-09-21T05:56:29.807840+0000","last_active":"2024-09-21T05:56:31.177487+0000","last_peered":"2024-09-21T05:56:31.177487+0000","last_clean":"2024-09-21T05:56:31.177487+0000","last_became_active":"2024-09-21T05:56:29.807524+0000","last_became_peered":"2024-09-21T05:56:29.807524+0000","last_unstale":"2024-09-21T05:56:31.177487+0000","last_undegraded":"2024-09-21T05:56:31.177487+0000","last_fullsized":"2024-09-21T05:56:31.177487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:34.812658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782834+0000","last_change":"2024-09-21T05:56:28.790494+0000","last_active":"2024-09-21T05:56:30.782834+0000","last_peered":"2024-09-21T05:56:30.782834+0000","last_clean":"2024-09-21T05:56:30.782834+0000","last_became_active":"2024-09-21T05:56:28.790003+0000","last_became_peered":"2024-09-21T05:56:28.790003+0000","last_unstale":"2024-09-21T05:56:30.782834+0000","last_undegraded":"2024-09-21T05:56:30.782834+0000","last_fullsized":"2024-09-21T05:56:30.782834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176130+0000","last_change":"2024-09-21T05:56:29.810515+0000","last_active":"2024-09-21T05:56:31.176130+0000","last_peered":"2024-09-21T05:56:31.176130+0000","last_clean":"2024-09-21T05:56:31.176130+0000","last_became_active":"2024-09-21T05:56:29.810188+0000","last_became_peered":"2024-09-21T05:56:29.810188+0000","last_unstale":"2024-09-21T05:56:31.176130+0000","last_undegraded":"2024-09-21T05:56:31.176130+0000","last_fullsized":"2024-09-21T05:56:31.176130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:48.361541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796183+0000","last_change":"2024-09-21T05:56:28.794032+0000","last_active":"2024-09-21T05:56:30.796183+0000","last_peered":"2024-09-21T05:56:30.796183+0000","last_clean":"2024-09-21T05:56:30.796183+0000","last_became_active":"2024-09-21T05:56:28.793671+0000","last_became_peered":"2024-09-21T05:56:28.793671+0000","last_unstale":"2024-09-21T05:56:30.796183+0000","last_undegraded":"2024-09-21T05:56:30.796183+0000","last_fullsized":"2024-09-21T05:56:30.796183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:44.652224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796406+0000","last_change":"2024-09-21T05:56:29.798550+0000","last_active":"2024-09-21T05:56:30.796406+0000","last_peered":"2024-09-21T05:56:30.796406+0000","last_clean":"2024-09-21T05:56:30.796406+0000","last_became_active":"2024-09-21T05:56:29.797719+0000","last_became_peered":"2024-09-21T05:56:29.797719+0000","last_unstale":"2024-09-21T05:56:30.796406+0000","last_undegraded":"2024-09-21T05:56:30.796406+0000","last_fullsized":"2024-09-21T05:56:30.796406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:17.854511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176419+0000","last_change":"2024-09-21T05:56:28.777283+0000","last_active":"2024-09-21T05:56:31.176419+0000","last_peered":"2024-09-21T05:56:31.176419+0000","last_clean":"2024-09-21T05:56:31.176419+0000","last_became_active":"2024-09-21T05:56:28.776907+0000","last_became_peered":"2024-09-21T05:56:28.776907+0000","last_unstale":"2024-09-21T05:56:31.176419+0000","last_undegraded":"2024-09-21T05:56:31.176419+0000","last_fullsized":"2024-09-21T05:56:31.176419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176488+0000","last_change":"2024-09-21T05:56:29.810738+0000","last_active":"2024-09-21T05:56:31.176488+0000","last_peered":"2024-09-21T05:56:31.176488+0000","last_clean":"2024-09-21T05:56:31.176488+0000","last_became_active":"2024-09-21T05:56:29.810296+0000","last_became_peered":"2024-09-21T05:56:29.810296+0000","last_unstale":"2024-09-21T05:56:31.176488+0000","last_undegraded":"2024-09-21T05:56:31.176488+0000","last_fullsized":"2024-09-21T05:56:31.176488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:38.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.781591+0000","last_change":"2024-09-21T05:56:28.793250+0000","last_active":"2024-09-21T05:56:30.781591+0000","last_peered":"2024-09-21T05:56:30.781591+0000","last_clean":"2024-09-21T05:56:30.781591+0000","last_became_active":"2024-09-21T05:56:28.792913+0000","last_became_peered":"2024-09-21T05:56:28.792913+0000","last_unstale":"2024-09-21T05:56:30.781591+0000","last_undegraded":"2024-09-21T05:56:30.781591+0000","last_fullsized":"2024-09-21T05:56:30.781591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:39.751978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176567+0000","last_change":"2024-09-21T05:56:29.811980+0000","last_active":"2024-09-21T05:56:31.176567+0000","last_peered":"2024-09-21T05:56:31.176567+0000","last_clean":"2024-09-21T05:56:31.176567+0000","last_became_active":"2024-09-21T05:56:29.811133+0000","last_became_peered":"2024-09-21T05:56:29.811133+0000","last_unstale":"2024-09-21T05:56:31.176567+0000","last_undegraded":"2024-09-21T05:56:31.176567+0000","last_fullsized":"2024-09-21T05:56:31.176567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:16.791089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177124+0000","last_change":"2024-09-21T05:56:28.784954+0000","last_active":"2024-09-21T05:56:31.177124+0000","last_peered":"2024-09-21T05:56:31.177124+0000","last_clean":"2024-09-21T05:56:31.177124+0000","last_became_active":"2024-09-21T05:56:28.784610+0000","last_became_peered":"2024-09-21T05:56:28.784610+0000","last_unstale":"2024-09-21T05:56:31.177124+0000","last_undegraded":"2024-09-21T05:56:31.177124+0000","last_fullsized":"2024-09-21T05:56:31.177124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:08.868280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176832+0000","last_change":"2024-09-21T05:56:29.811442+0000","last_active":"2024-09-21T05:56:31.176832+0000","last_peered":"2024-09-21T05:56:31.176832+0000","last_clean":"2024-09-21T05:56:31.176832+0000","last_became_active":"2024-09-21T05:56:29.809712+0000","last_became_peered":"2024-09-21T05:56:29.809712+0000","last_unstale":"2024-09-21T05:56:31.176832+0000","last_undegraded":"2024-09-21T05:56:31.176832+0000","last_fullsized":"2024-09-21T05:56:31.176832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:11.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780028+0000","last_change":"2024-09-21T05:56:28.788606+0000","last_active":"2024-09-21T05:56:30.780028+0000","last_peered":"2024-09-21T05:56:30.780028+0000","last_clean":"2024-09-21T05:56:30.780028+0000","last_became_active":"2024-09-21T05:56:28.787648+0000","last_became_peered":"2024-09-21T05:56:28.787648+0000","last_unstale":"2024-09-21T05:56:30.780028+0000","last_undegraded":"2024-09-21T05:56:30.780028+0000","last_fullsized":"2024-09-21T05:56:30.780028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:37.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783198+0000","last_change":"2024-09-21T05:56:29.789459+0000","last_active":"2024-09-21T05:56:30.783198+0000","last_peered":"2024-09-21T05:56:30.783198+0000","last_clean":"2024-09-21T05:56:30.783198+0000","last_became_active":"2024-09-21T05:56:29.785168+0000","last_became_peered":"2024-09-21T05:56:29.785168+0000","last_unstale":"2024-09-21T05:56:30.783198+0000","last_undegraded":"2024-09-21T05:56:30.783198+0000","last_fullsized":"2024-09-21T05:56:30.783198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:38.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176888+0000","last_change":"2024-09-21T05:56:28.790714+0000","last_active":"2024-09-21T05:56:31.176888+0000","last_peered":"2024-09-21T05:56:31.176888+0000","last_clean":"2024-09-21T05:56:31.176888+0000","last_became_active":"2024-09-21T05:56:28.790280+0000","last_became_peered":"2024-09-21T05:56:28.790280+0000","last_unstale":"2024-09-21T05:56:31.176888+0000","last_undegraded":"2024-09-21T05:56:31.176888+0000","last_fullsized":"2024-09-21T05:56:31.176888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:43.786792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176561+0000","last_change":"2024-09-21T05:56:29.805253+0000","last_active":"2024-09-21T05:56:31.176561+0000","last_peered":"2024-09-21T05:56:31.176561+0000","last_clean":"2024-09-21T05:56:31.176561+0000","last_became_active":"2024-09-21T05:56:29.801598+0000","last_became_peered":"2024-09-21T05:56:29.801598+0000","last_unstale":"2024-09-21T05:56:31.176561+0000","last_undegraded":"2024-09-21T05:56:31.176561+0000","last_fullsized":"2024-09-21T05:56:31.176561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:22.963835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175921+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:56:31.175921+0000","last_peered":"2024-09-21T05:56:31.175921+0000","last_clean":"2024-09-21T05:56:31.175921+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:56:31.175921+0000","last_undegraded":"2024-09-21T05:56:31.175921+0000","last_fullsized":"2024-09-21T05:56:31.175921+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782518+0000","last_change":"2024-09-21T05:56:28.781190+0000","last_active":"2024-09-21T05:56:30.782518+0000","last_peered":"2024-09-21T05:56:30.782518+0000","last_clean":"2024-09-21T05:56:30.782518+0000","last_became_active":"2024-09-21T05:56:28.780808+0000","last_became_peered":"2024-09-21T05:56:28.780808+0000","last_unstale":"2024-09-21T05:56:30.782518+0000","last_undegraded":"2024-09-21T05:56:30.782518+0000","last_fullsized":"2024-09-21T05:56:30.782518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:44.652224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177804+0000","last_change":"2024-09-21T05:56:29.795930+0000","last_active":"2024-09-21T05:56:31.177804+0000","last_peered":"2024-09-21T05:56:31.177804+0000","last_clean":"2024-09-21T05:56:31.177804+0000","last_became_active":"2024-09-21T05:56:29.795552+0000","last_became_peered":"2024-09-21T05:56:29.795552+0000","last_unstale":"2024-09-21T05:56:31.177804+0000","last_undegraded":"2024-09-21T05:56:31.177804+0000","last_fullsized":"2024-09-21T05:56:31.177804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:22.412975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175735+0000","last_change":"2024-09-21T05:56:28.805633+0000","last_active":"2024-09-21T05:56:31.175735+0000","last_peered":"2024-09-21T05:56:31.175735+0000","last_clean":"2024-09-21T05:56:31.175735+0000","last_became_active":"2024-09-21T05:56:28.804239+0000","last_became_peered":"2024-09-21T05:56:28.804239+0000","last_unstale":"2024-09-21T05:56:31.175735+0000","last_undegraded":"2024-09-21T05:56:31.175735+0000","last_fullsized":"2024-09-21T05:56:31.175735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:53.019618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178389+0000","last_change":"2024-09-21T05:56:29.803538+0000","last_active":"2024-09-21T05:56:31.178389+0000","last_peered":"2024-09-21T05:56:31.178389+0000","last_clean":"2024-09-21T05:56:31.178389+0000","last_became_active":"2024-09-21T05:56:29.803153+0000","last_became_peered":"2024-09-21T05:56:29.803153+0000","last_unstale":"2024-09-21T05:56:31.178389+0000","last_undegraded":"2024-09-21T05:56:31.178389+0000","last_fullsized":"2024-09-21T05:56:31.178389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:18.925867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176800+0000","last_change":"2024-09-21T05:56:28.786025+0000","last_active":"2024-09-21T05:56:31.176800+0000","last_peered":"2024-09-21T05:56:31.176800+0000","last_clean":"2024-09-21T05:56:31.176800+0000","last_became_active":"2024-09-21T05:56:28.785690+0000","last_became_peered":"2024-09-21T05:56:28.785690+0000","last_unstale":"2024-09-21T05:56:31.176800+0000","last_undegraded":"2024-09-21T05:56:31.176800+0000","last_fullsized":"2024-09-21T05:56:31.176800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:44.652224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779811+0000","last_change":"2024-09-21T05:56:29.785876+0000","last_active":"2024-09-21T05:56:30.779811+0000","last_peered":"2024-09-21T05:56:30.779811+0000","last_clean":"2024-09-21T05:56:30.779811+0000","last_became_active":"2024-09-21T05:56:29.785564+0000","last_became_peered":"2024-09-21T05:56:29.785564+0000","last_unstale":"2024-09-21T05:56:30.779811+0000","last_undegraded":"2024-09-21T05:56:30.779811+0000","last_fullsized":"2024-09-21T05:56:30.779811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:04.812480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782960+0000","last_change":"2024-09-21T05:56:28.785911+0000","last_active":"2024-09-21T05:56:30.782960+0000","last_peered":"2024-09-21T05:56:30.782960+0000","last_clean":"2024-09-21T05:56:30.782960+0000","last_became_active":"2024-09-21T05:56:28.785413+0000","last_became_peered":"2024-09-21T05:56:28.785413+0000","last_unstale":"2024-09-21T05:56:30.782960+0000","last_undegraded":"2024-09-21T05:56:30.782960+0000","last_fullsized":"2024-09-21T05:56:30.782960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:08.868280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780540+0000","last_change":"2024-09-21T05:56:29.789624+0000","last_active":"2024-09-21T05:56:30.780540+0000","last_peered":"2024-09-21T05:56:30.780540+0000","last_clean":"2024-09-21T05:56:30.780540+0000","last_became_active":"2024-09-21T05:56:29.789305+0000","last_became_peered":"2024-09-21T05:56:29.789305+0000","last_unstale":"2024-09-21T05:56:30.780540+0000","last_undegraded":"2024-09-21T05:56:30.780540+0000","last_fullsized":"2024-09-21T05:56:30.780540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:34.711369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780096+0000","last_change":"2024-09-21T05:56:28.791893+0000","last_active":"2024-09-21T05:56:30.780096+0000","last_peered":"2024-09-21T05:56:30.780096+0000","last_clean":"2024-09-21T05:56:30.780096+0000","last_became_active":"2024-09-21T05:56:28.791580+0000","last_became_peered":"2024-09-21T05:56:28.791580+0000","last_unstale":"2024-09-21T05:56:30.780096+0000","last_undegraded":"2024-09-21T05:56:30.780096+0000","last_fullsized":"2024-09-21T05:56:30.780096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.359445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.784018+0000","last_change":"2024-09-21T05:56:29.792438+0000","last_active":"2024-09-21T05:56:30.784018+0000","last_peered":"2024-09-21T05:56:30.784018+0000","last_clean":"2024-09-21T05:56:30.784018+0000","last_became_active":"2024-09-21T05:56:29.792034+0000","last_became_peered":"2024-09-21T05:56:29.792034+0000","last_unstale":"2024-09-21T05:56:30.784018+0000","last_undegraded":"2024-09-21T05:56:30.784018+0000","last_fullsized":"2024-09-21T05:56:30.784018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:30.461662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176595+0000","last_change":"2024-09-21T05:56:28.805473+0000","last_active":"2024-09-21T05:56:31.176595+0000","last_peered":"2024-09-21T05:56:31.176595+0000","last_clean":"2024-09-21T05:56:31.176595+0000","last_became_active":"2024-09-21T05:56:28.804850+0000","last_became_peered":"2024-09-21T05:56:28.804850+0000","last_unstale":"2024-09-21T05:56:31.176595+0000","last_undegraded":"2024-09-21T05:56:31.176595+0000","last_fullsized":"2024-09-21T05:56:31.176595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:44.256633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780054+0000","last_change":"2024-09-21T05:56:29.784905+0000","last_active":"2024-09-21T05:56:30.780054+0000","last_peered":"2024-09-21T05:56:30.780054+0000","last_clean":"2024-09-21T05:56:30.780054+0000","last_became_active":"2024-09-21T05:56:29.784562+0000","last_became_peered":"2024-09-21T05:56:29.784562+0000","last_unstale":"2024-09-21T05:56:30.780054+0000","last_undegraded":"2024-09-21T05:56:30.780054+0000","last_fullsized":"2024-09-21T05:56:30.780054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:26.482139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.795965+0000","last_change":"2024-09-21T05:56:28.790111+0000","last_active":"2024-09-21T05:56:30.795965+0000","last_peered":"2024-09-21T05:56:30.795965+0000","last_clean":"2024-09-21T05:56:30.795965+0000","last_became_active":"2024-09-21T05:56:28.789760+0000","last_became_peered":"2024-09-21T05:56:28.789760+0000","last_unstale":"2024-09-21T05:56:30.795965+0000","last_undegraded":"2024-09-21T05:56:30.795965+0000","last_fullsized":"2024-09-21T05:56:30.795965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:51.060754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178344+0000","last_change":"2024-09-21T05:56:29.812261+0000","last_active":"2024-09-21T05:56:31.178344+0000","last_peered":"2024-09-21T05:56:31.178344+0000","last_clean":"2024-09-21T05:56:31.178344+0000","last_became_active":"2024-09-21T05:56:29.811916+0000","last_became_peered":"2024-09-21T05:56:29.811916+0000","last_unstale":"2024-09-21T05:56:31.178344+0000","last_undegraded":"2024-09-21T05:56:31.178344+0000","last_fullsized":"2024-09-21T05:56:31.178344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:42.929175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796707+0000","last_change":"2024-09-21T05:56:28.804549+0000","last_active":"2024-09-21T05:56:30.796707+0000","last_peered":"2024-09-21T05:56:30.796707+0000","last_clean":"2024-09-21T05:56:30.796707+0000","last_became_active":"2024-09-21T05:56:28.804182+0000","last_became_peered":"2024-09-21T05:56:28.804182+0000","last_unstale":"2024-09-21T05:56:30.796707+0000","last_undegraded":"2024-09-21T05:56:30.796707+0000","last_fullsized":"2024-09-21T05:56:30.796707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:33.703504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178183+0000","last_change":"2024-09-21T05:56:29.797061+0000","last_active":"2024-09-21T05:56:31.178183+0000","last_peered":"2024-09-21T05:56:31.178183+0000","last_clean":"2024-09-21T05:56:31.178183+0000","last_became_active":"2024-09-21T05:56:29.796608+0000","last_became_peered":"2024-09-21T05:56:29.796608+0000","last_unstale":"2024-09-21T05:56:31.178183+0000","last_undegraded":"2024-09-21T05:56:31.178183+0000","last_fullsized":"2024-09-21T05:56:31.178183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:14.929983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175725+0000","last_change":"2024-09-21T05:56:28.804158+0000","last_active":"2024-09-21T05:56:31.175725+0000","last_peered":"2024-09-21T05:56:31.175725+0000","last_clean":"2024-09-21T05:56:31.175725+0000","last_became_active":"2024-09-21T05:56:28.803813+0000","last_became_peered":"2024-09-21T05:56:28.803813+0000","last_unstale":"2024-09-21T05:56:31.175725+0000","last_undegraded":"2024-09-21T05:56:31.175725+0000","last_fullsized":"2024-09-21T05:56:31.175725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:49.073829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780422+0000","last_change":"2024-09-21T05:56:29.797845+0000","last_active":"2024-09-21T05:56:30.780422+0000","last_peered":"2024-09-21T05:56:30.780422+0000","last_clean":"2024-09-21T05:56:30.780422+0000","last_became_active":"2024-09-21T05:56:29.797380+0000","last_became_peered":"2024-09-21T05:56:29.797380+0000","last_unstale":"2024-09-21T05:56:30.780422+0000","last_undegraded":"2024-09-21T05:56:30.780422+0000","last_fullsized":"2024-09-21T05:56:30.780422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:09.876144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779895+0000","last_change":"2024-09-21T05:56:28.801904+0000","last_active":"2024-09-21T05:56:30.779895+0000","last_peered":"2024-09-21T05:56:30.779895+0000","last_clean":"2024-09-21T05:56:30.779895+0000","last_became_active":"2024-09-21T05:56:28.801540+0000","last_became_peered":"2024-09-21T05:56:28.801540+0000","last_unstale":"2024-09-21T05:56:30.779895+0000","last_undegraded":"2024-09-21T05:56:30.779895+0000","last_fullsized":"2024-09-21T05:56:30.779895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:06.676644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177960+0000","last_change":"2024-09-21T05:56:29.782893+0000","last_active":"2024-09-21T05:56:31.177960+0000","last_peered":"2024-09-21T05:56:31.177960+0000","last_clean":"2024-09-21T05:56:31.177960+0000","last_became_active":"2024-09-21T05:56:29.782496+0000","last_became_peered":"2024-09-21T05:56:29.782496+0000","last_unstale":"2024-09-21T05:56:31.177960+0000","last_undegraded":"2024-09-21T05:56:31.177960+0000","last_fullsized":"2024-09-21T05:56:31.177960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:33.544641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779828+0000","last_change":"2024-09-21T05:56:28.784648+0000","last_active":"2024-09-21T05:56:30.779828+0000","last_peered":"2024-09-21T05:56:30.779828+0000","last_clean":"2024-09-21T05:56:30.779828+0000","last_became_active":"2024-09-21T05:56:28.784333+0000","last_became_peered":"2024-09-21T05:56:28.784333+0000","last_unstale":"2024-09-21T05:56:30.779828+0000","last_undegraded":"2024-09-21T05:56:30.779828+0000","last_fullsized":"2024-09-21T05:56:30.779828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:15.892877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178255+0000","last_change":"2024-09-21T05:56:29.789722+0000","last_active":"2024-09-21T05:56:31.178255+0000","last_peered":"2024-09-21T05:56:31.178255+0000","last_clean":"2024-09-21T05:56:31.178255+0000","last_became_active":"2024-09-21T05:56:29.789333+0000","last_became_peered":"2024-09-21T05:56:29.789333+0000","last_unstale":"2024-09-21T05:56:31.178255+0000","last_undegraded":"2024-09-21T05:56:31.178255+0000","last_fullsized":"2024-09-21T05:56:31.178255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176267+0000","last_change":"2024-09-21T05:56:28.807252+0000","last_active":"2024-09-21T05:56:31.176267+0000","last_peered":"2024-09-21T05:56:31.176267+0000","last_clean":"2024-09-21T05:56:31.176267+0000","last_became_active":"2024-09-21T05:56:28.806895+0000","last_became_peered":"2024-09-21T05:56:28.806895+0000","last_unstale":"2024-09-21T05:56:31.176267+0000","last_undegraded":"2024-09-21T05:56:31.176267+0000","last_fullsized":"2024-09-21T05:56:31.176267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:34.762925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796596+0000","last_change":"2024-09-21T05:56:29.781497+0000","last_active":"2024-09-21T05:56:30.796596+0000","last_peered":"2024-09-21T05:56:30.796596+0000","last_clean":"2024-09-21T05:56:30.796596+0000","last_became_active":"2024-09-21T05:56:29.780460+0000","last_became_peered":"2024-09-21T05:56:29.780460+0000","last_unstale":"2024-09-21T05:56:30.796596+0000","last_undegraded":"2024-09-21T05:56:30.796596+0000","last_fullsized":"2024-09-21T05:56:30.796596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:55.735758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783852+0000","last_change":"2024-09-21T05:56:29.788038+0000","last_active":"2024-09-21T05:56:30.783852+0000","last_peered":"2024-09-21T05:56:30.783852+0000","last_clean":"2024-09-21T05:56:30.783852+0000","last_became_active":"2024-09-21T05:56:29.784063+0000","last_became_peered":"2024-09-21T05:56:29.784063+0000","last_unstale":"2024-09-21T05:56:30.783852+0000","last_undegraded":"2024-09-21T05:56:30.783852+0000","last_fullsized":"2024-09-21T05:56:30.783852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:22.850164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175903+0000","last_change":"2024-09-21T05:56:28.808759+0000","last_active":"2024-09-21T05:56:31.175903+0000","last_peered":"2024-09-21T05:56:31.175903+0000","last_clean":"2024-09-21T05:56:31.175903+0000","last_became_active":"2024-09-21T05:56:28.808371+0000","last_became_peered":"2024-09-21T05:56:28.808371+0000","last_unstale":"2024-09-21T05:56:31.175903+0000","last_undegraded":"2024-09-21T05:56:31.175903+0000","last_fullsized":"2024-09-21T05:56:31.175903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:23.035025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783562+0000","last_change":"2024-09-21T05:56:29.791927+0000","last_active":"2024-09-21T05:56:30.783562+0000","last_peered":"2024-09-21T05:56:30.783562+0000","last_clean":"2024-09-21T05:56:30.783562+0000","last_became_active":"2024-09-21T05:56:29.790091+0000","last_became_peered":"2024-09-21T05:56:29.790091+0000","last_unstale":"2024-09-21T05:56:30.783562+0000","last_undegraded":"2024-09-21T05:56:30.783562+0000","last_fullsized":"2024-09-21T05:56:30.783562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:22.963835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176216+0000","last_change":"2024-09-21T05:56:28.804798+0000","last_active":"2024-09-21T05:56:31.176216+0000","last_peered":"2024-09-21T05:56:31.176216+0000","last_clean":"2024-09-21T05:56:31.176216+0000","last_became_active":"2024-09-21T05:56:28.803848+0000","last_became_peered":"2024-09-21T05:56:28.803848+0000","last_unstale":"2024-09-21T05:56:31.176216+0000","last_undegraded":"2024-09-21T05:56:31.176216+0000","last_fullsized":"2024-09-21T05:56:31.176216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:32.979720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178080+0000","last_change":"2024-09-21T05:56:29.798957+0000","last_active":"2024-09-21T05:56:31.178080+0000","last_peered":"2024-09-21T05:56:31.178080+0000","last_clean":"2024-09-21T05:56:31.178080+0000","last_became_active":"2024-09-21T05:56:29.798223+0000","last_became_peered":"2024-09-21T05:56:29.798223+0000","last_unstale":"2024-09-21T05:56:31.178080+0000","last_undegraded":"2024-09-21T05:56:31.178080+0000","last_fullsized":"2024-09-21T05:56:31.178080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:39.467641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177351+0000","last_change":"2024-09-21T05:56:28.783936+0000","last_active":"2024-09-21T05:56:31.177351+0000","last_peered":"2024-09-21T05:56:31.177351+0000","last_clean":"2024-09-21T05:56:31.177351+0000","last_became_active":"2024-09-21T05:56:28.783541+0000","last_became_peered":"2024-09-21T05:56:28.783541+0000","last_unstale":"2024-09-21T05:56:31.177351+0000","last_undegraded":"2024-09-21T05:56:31.177351+0000","last_fullsized":"2024-09-21T05:56:31.177351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:38.675175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178320+0000","last_change":"2024-09-21T05:56:29.785856+0000","last_active":"2024-09-21T05:56:31.178320+0000","last_peered":"2024-09-21T05:56:31.178320+0000","last_clean":"2024-09-21T05:56:31.178320+0000","last_became_active":"2024-09-21T05:56:29.785475+0000","last_became_peered":"2024-09-21T05:56:29.785475+0000","last_unstale":"2024-09-21T05:56:31.178320+0000","last_undegraded":"2024-09-21T05:56:31.178320+0000","last_fullsized":"2024-09-21T05:56:31.178320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:21.326466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177339+0000","last_change":"2024-09-21T05:56:28.804387+0000","last_active":"2024-09-21T05:56:31.177339+0000","last_peered":"2024-09-21T05:56:31.177339+0000","last_clean":"2024-09-21T05:56:31.177339+0000","last_became_active":"2024-09-21T05:56:28.804116+0000","last_became_peered":"2024-09-21T05:56:28.804116+0000","last_unstale":"2024-09-21T05:56:31.177339+0000","last_undegraded":"2024-09-21T05:56:31.177339+0000","last_fullsized":"2024-09-21T05:56:31.177339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":667234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":76410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":667290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:56:43.994 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:56:44.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.188+0000 7f102ee00640 1 -- 172.21.15.50:0/1039941664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10280703a0 msgr2=0x7f102806d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:44.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.188+0000 7f102ee00640 1 --2- 172.21.15.50:0/1039941664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10280703a0 0x7f102806d200 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f10280b6bf0 tx=0x7f102001caf0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.188+0000 7f102ee00640 1 -- 172.21.15.50:0/1039941664 shutdown_connections 2024-09-21T05:56:44.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.188+0000 7f102ee00640 1 --2- 172.21.15.50:0/1039941664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10280703a0 0x7f102806d200 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.188+0000 7f102ee00640 1 --2- 172.21.15.50:0/1039941664 >> v2:172.21.15.88:3300/0 conn(0x7f1028159130 0x7f102815bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.188+0000 7f102ee00640 1 -- 172.21.15.50:0/1039941664 >> 172.21.15.50:0/1039941664 conn(0x7f10280b35f0 msgr2=0x7f10280b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:44.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.188+0000 7f102ee00640 1 -- 172.21.15.50:0/1039941664 shutdown_connections 2024-09-21T05:56:44.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.188+0000 7f102ee00640 1 -- 172.21.15.50:0/1039941664 wait complete. 2024-09-21T05:56:44.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.188+0000 7f102ee00640 1 Processor -- start 2024-09-21T05:56:44.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.188+0000 7f102ee00640 1 -- start start 2024-09-21T05:56:44.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f102ee00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f102806cdf0 0x7f10281597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:44.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f102ee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10280703a0 0x7f10280d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:44.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f102ee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1028159130 0x7f102815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:44.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f102ee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1028168cb0 con 0x7f10280703a0 2024-09-21T05:56:44.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f102ee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1028168b30 con 0x7f1028159130 2024-09-21T05:56:44.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f102ee00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1028168e30 con 0x7f102806cdf0 2024-09-21T05:56:44.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f1027400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10280703a0 0x7f10280d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:44.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f1027400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10280703a0 0x7f10280d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48764/0 (socket says 172.21.15.50:48764) 2024-09-21T05:56:44.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f1027400640 1 -- 172.21.15.50:0/2019392674 learned_addr learned my addr 172.21.15.50:0/2019392674 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:44.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f1027e00640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f102806cdf0 0x7f10281597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:44.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f102ce00640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1028159130 0x7f102815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:44.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f1027400640 1 -- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f102806cdf0 msgr2=0x7f10281597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:44.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f1027400640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f102806cdf0 0x7f10281597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f1027400640 1 -- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1028159130 msgr2=0x7f102815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:44.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f1027400640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1028159130 0x7f102815a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f1027400640 1 -- 172.21.15.50:0/2019392674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10281fc630 con 0x7f10280703a0 2024-09-21T05:56:44.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f1027400640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10280703a0 0x7f10280d00f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f101400c760 tx=0x7f101400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f101fe00640 1 -- 172.21.15.50:0/2019392674 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014016020 con 0x7f10280703a0 2024-09-21T05:56:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f101fe00640 1 -- 172.21.15.50:0/2019392674 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f101400a950 con 0x7f10280703a0 2024-09-21T05:56:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f102ee00640 1 -- 172.21.15.50:0/2019392674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10281fc910 con 0x7f10280703a0 2024-09-21T05:56:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f101fe00640 1 -- 172.21.15.50:0/2019392674 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101400ada0 con 0x7f10280703a0 2024-09-21T05:56:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f102ee00640 1 -- 172.21.15.50:0/2019392674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10281fced0 con 0x7f10280703a0 2024-09-21T05:56:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.192+0000 7f102ee00640 1 -- 172.21.15.50:0/2019392674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102806d200 con 0x7f10280703a0 2024-09-21T05:56:44.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.196+0000 7f101fe00640 1 -- 172.21.15.50:0/2019392674 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1014019020 con 0x7f10280703a0 2024-09-21T05:56:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.196+0000 7f101fe00640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1008073200 0x7f1008075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.196+0000 7f101fe00640 1 -- 172.21.15.50:0/2019392674 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1014093fe0 con 0x7f10280703a0 2024-09-21T05:56:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.196+0000 7f1027e00640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1008073200 0x7f1008075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:44.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.196+0000 7f1027e00640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1008073200 0x7f1008075710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1018004630 tx=0x7f101800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:44.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.200+0000 7f101fe00640 1 -- 172.21.15.50:0/2019392674 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1014020020 con 0x7f10280703a0 2024-09-21T05:56:44.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.372+0000 7f102ee00640 1 -- 172.21.15.50:0/2019392674 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10281fd2b0 con 0x7f1008073200 2024-09-21T05:56:44.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.380+0000 7f101fe00640 1 -- 172.21.15.50:0/2019392674 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f10281fd2b0 con 0x7f1008073200 2024-09-21T05:56:44.385 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:44.386 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:56:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.384+0000 7f102ee00640 1 -- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1008073200 msgr2=0x7f1008075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.384+0000 7f102ee00640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1008073200 0x7f1008075710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1018004630 tx=0x7f101800b2e0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.384+0000 7f102ee00640 1 -- 172.21.15.50:0/2019392674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10280703a0 msgr2=0x7f10280d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.384+0000 7f102ee00640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10280703a0 0x7f10280d00f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f101400c760 tx=0x7f101400cc20 comp rx=0 tx=0).stop 2024-09-21T05:56:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.384+0000 7f102ee00640 1 -- 172.21.15.50:0/2019392674 shutdown_connections 2024-09-21T05:56:44.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.384+0000 7f102ee00640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1028159130 0x7f102815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.384+0000 7f102ee00640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1008073200 0x7f1008075710 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.384+0000 7f102ee00640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10280703a0 0x7f10280d00f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.384+0000 7f102ee00640 1 --2- 172.21.15.50:0/2019392674 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f102806cdf0 0x7f10281597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.384+0000 7f102ee00640 1 -- 172.21.15.50:0/2019392674 >> 172.21.15.50:0/2019392674 conn(0x7f10280b35f0 msgr2=0x7f10281572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:44.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.384+0000 7f102ee00640 1 -- 172.21.15.50:0/2019392674 shutdown_connections 2024-09-21T05:56:44.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.384+0000 7f102ee00640 1 -- 172.21.15.50:0/2019392674 wait complete. 2024-09-21T05:56:44.406 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-21T05:56:42.887046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235840,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738880,"statfs":{"total":773094113280,"available":772852613120,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002893"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176632+0000","last_change":"2024-09-21T05:56:29.801294+0000","last_active":"2024-09-21T05:56:31.176632+0000","last_peered":"2024-09-21T05:56:31.176632+0000","last_clean":"2024-09-21T05:56:31.176632+0000","last_became_active":"2024-09-21T05:56:29.800947+0000","last_became_peered":"2024-09-21T05:56:29.800947+0000","last_unstale":"2024-09-21T05:56:31.176632+0000","last_undegraded":"2024-09-21T05:56:31.176632+0000","last_fullsized":"2024-09-21T05:56:31.176632+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:11.951093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780332+0000","last_change":"2024-09-21T05:56:29.802492+0000","last_active":"2024-09-21T05:56:30.780332+0000","last_peered":"2024-09-21T05:56:30.780332+0000","last_clean":"2024-09-21T05:56:30.780332+0000","last_became_active":"2024-09-21T05:56:29.801940+0000","last_became_peered":"2024-09-21T05:56:29.801940+0000","last_unstale":"2024-09-21T05:56:30.780332+0000","last_undegraded":"2024-09-21T05:56:30.780332+0000","last_fullsized":"2024-09-21T05:56:30.780332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:11.434269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178417+0000","last_change":"2024-09-21T05:56:29.804599+0000","last_active":"2024-09-21T05:56:31.178417+0000","last_peered":"2024-09-21T05:56:31.178417+0000","last_clean":"2024-09-21T05:56:31.178417+0000","last_became_active":"2024-09-21T05:56:29.804231+0000","last_became_peered":"2024-09-21T05:56:29.804231+0000","last_unstale":"2024-09-21T05:56:31.178417+0000","last_undegraded":"2024-09-21T05:56:31.178417+0000","last_fullsized":"2024-09-21T05:56:31.178417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782989+0000","last_change":"2024-09-21T05:56:29.790060+0000","last_active":"2024-09-21T05:56:30.782989+0000","last_peered":"2024-09-21T05:56:30.782989+0000","last_clean":"2024-09-21T05:56:30.782989+0000","last_became_active":"2024-09-21T05:56:29.786267+0000","last_became_peered":"2024-09-21T05:56:29.786267+0000","last_unstale":"2024-09-21T05:56:30.782989+0000","last_undegraded":"2024-09-21T05:56:30.782989+0000","last_fullsized":"2024-09-21T05:56:30.782989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:11.867996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176068+0000","last_change":"2024-09-21T05:56:29.809836+0000","last_active":"2024-09-21T05:56:31.176068+0000","last_peered":"2024-09-21T05:56:31.176068+0000","last_clean":"2024-09-21T05:56:31.176068+0000","last_became_active":"2024-09-21T05:56:29.809429+0000","last_became_peered":"2024-09-21T05:56:29.809429+0000","last_unstale":"2024-09-21T05:56:31.176068+0000","last_undegraded":"2024-09-21T05:56:31.176068+0000","last_fullsized":"2024-09-21T05:56:31.176068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:41.759804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782594+0000","last_change":"2024-09-21T05:56:29.794338+0000","last_active":"2024-09-21T05:56:30.782594+0000","last_peered":"2024-09-21T05:56:30.782594+0000","last_clean":"2024-09-21T05:56:30.782594+0000","last_became_active":"2024-09-21T05:56:29.794011+0000","last_became_peered":"2024-09-21T05:56:29.794011+0000","last_unstale":"2024-09-21T05:56:30.782594+0000","last_undegraded":"2024-09-21T05:56:30.782594+0000","last_fullsized":"2024-09-21T05:56:30.782594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:37.695152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176168+0000","last_change":"2024-09-21T05:56:29.806114+0000","last_active":"2024-09-21T05:56:31.176168+0000","last_peered":"2024-09-21T05:56:31.176168+0000","last_clean":"2024-09-21T05:56:31.176168+0000","last_became_active":"2024-09-21T05:56:29.805711+0000","last_became_peered":"2024-09-21T05:56:29.805711+0000","last_unstale":"2024-09-21T05:56:31.176168+0000","last_undegraded":"2024-09-21T05:56:31.176168+0000","last_fullsized":"2024-09-21T05:56:31.176168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:11.867996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177775+0000","last_change":"2024-09-21T05:56:29.800463+0000","last_active":"2024-09-21T05:56:31.177775+0000","last_peered":"2024-09-21T05:56:31.177775+0000","last_clean":"2024-09-21T05:56:31.177775+0000","last_became_active":"2024-09-21T05:56:29.800084+0000","last_became_peered":"2024-09-21T05:56:29.800084+0000","last_unstale":"2024-09-21T05:56:31.177775+0000","last_undegraded":"2024-09-21T05:56:31.177775+0000","last_fullsized":"2024-09-21T05:56:31.177775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:39.467641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176639+0000","last_change":"2024-09-21T05:56:29.809950+0000","last_active":"2024-09-21T05:56:31.176639+0000","last_peered":"2024-09-21T05:56:31.176639+0000","last_clean":"2024-09-21T05:56:31.176639+0000","last_became_active":"2024-09-21T05:56:29.809337+0000","last_became_peered":"2024-09-21T05:56:29.809337+0000","last_unstale":"2024-09-21T05:56:31.176639+0000","last_undegraded":"2024-09-21T05:56:31.176639+0000","last_fullsized":"2024-09-21T05:56:31.176639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:30.461662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796623+0000","last_change":"2024-09-21T05:56:29.781023+0000","last_active":"2024-09-21T05:56:30.796623+0000","last_peered":"2024-09-21T05:56:30.796623+0000","last_clean":"2024-09-21T05:56:30.796623+0000","last_became_active":"2024-09-21T05:56:29.780729+0000","last_became_peered":"2024-09-21T05:56:29.780729+0000","last_unstale":"2024-09-21T05:56:30.796623+0000","last_undegraded":"2024-09-21T05:56:30.796623+0000","last_fullsized":"2024-09-21T05:56:30.796623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:22.412975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796667+0000","last_change":"2024-09-21T05:56:29.779393+0000","last_active":"2024-09-21T05:56:30.796667+0000","last_peered":"2024-09-21T05:56:30.796667+0000","last_clean":"2024-09-21T05:56:30.796667+0000","last_became_active":"2024-09-21T05:56:29.779016+0000","last_became_peered":"2024-09-21T05:56:29.779016+0000","last_unstale":"2024-09-21T05:56:30.796667+0000","last_undegraded":"2024-09-21T05:56:30.796667+0000","last_fullsized":"2024-09-21T05:56:30.796667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:30.792281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.781580+0000","last_change":"2024-09-21T05:56:29.787708+0000","last_active":"2024-09-21T05:56:30.781580+0000","last_peered":"2024-09-21T05:56:30.781580+0000","last_clean":"2024-09-21T05:56:30.781580+0000","last_became_active":"2024-09-21T05:56:29.787370+0000","last_became_peered":"2024-09-21T05:56:29.787370+0000","last_unstale":"2024-09-21T05:56:30.781580+0000","last_undegraded":"2024-09-21T05:56:30.781580+0000","last_fullsized":"2024-09-21T05:56:30.781580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:11.868532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177932+0000","last_change":"2024-09-21T05:56:29.790480+0000","last_active":"2024-09-21T05:56:31.177932+0000","last_peered":"2024-09-21T05:56:31.177932+0000","last_clean":"2024-09-21T05:56:31.177932+0000","last_became_active":"2024-09-21T05:56:29.790172+0000","last_became_peered":"2024-09-21T05:56:29.790172+0000","last_unstale":"2024-09-21T05:56:31.177932+0000","last_undegraded":"2024-09-21T05:56:31.177932+0000","last_fullsized":"2024-09-21T05:56:31.177932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:14.929983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178355+0000","last_change":"2024-09-21T05:56:29.796096+0000","last_active":"2024-09-21T05:56:31.178355+0000","last_peered":"2024-09-21T05:56:31.178355+0000","last_clean":"2024-09-21T05:56:31.178355+0000","last_became_active":"2024-09-21T05:56:29.795760+0000","last_became_peered":"2024-09-21T05:56:29.795760+0000","last_unstale":"2024-09-21T05:56:31.178355+0000","last_undegraded":"2024-09-21T05:56:31.178355+0000","last_fullsized":"2024-09-21T05:56:31.178355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:33.544641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779759+0000","last_change":"2024-09-21T05:56:29.789973+0000","last_active":"2024-09-21T05:56:30.779759+0000","last_peered":"2024-09-21T05:56:30.779759+0000","last_clean":"2024-09-21T05:56:30.779759+0000","last_became_active":"2024-09-21T05:56:29.789200+0000","last_became_peered":"2024-09-21T05:56:29.789200+0000","last_unstale":"2024-09-21T05:56:30.779759+0000","last_undegraded":"2024-09-21T05:56:30.779759+0000","last_fullsized":"2024-09-21T05:56:30.779759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:52.718337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782539+0000","last_change":"2024-09-21T05:56:29.795924+0000","last_active":"2024-09-21T05:56:30.782539+0000","last_peered":"2024-09-21T05:56:30.782539+0000","last_clean":"2024-09-21T05:56:30.782539+0000","last_became_active":"2024-09-21T05:56:29.795518+0000","last_became_peered":"2024-09-21T05:56:29.795518+0000","last_unstale":"2024-09-21T05:56:30.782539+0000","last_undegraded":"2024-09-21T05:56:30.782539+0000","last_fullsized":"2024-09-21T05:56:30.782539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:56.052565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.784178+0000","last_change":"2024-09-21T05:56:29.803208+0000","last_active":"2024-09-21T05:56:30.784178+0000","last_peered":"2024-09-21T05:56:30.784178+0000","last_clean":"2024-09-21T05:56:30.784178+0000","last_became_active":"2024-09-21T05:56:29.802492+0000","last_became_peered":"2024-09-21T05:56:29.802492+0000","last_unstale":"2024-09-21T05:56:30.784178+0000","last_undegraded":"2024-09-21T05:56:30.784178+0000","last_fullsized":"2024-09-21T05:56:30.784178+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:11.140043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177406+0000","last_change":"2024-09-21T05:56:29.787284+0000","last_active":"2024-09-21T05:56:31.177406+0000","last_peered":"2024-09-21T05:56:31.177406+0000","last_clean":"2024-09-21T05:56:31.177406+0000","last_became_active":"2024-09-21T05:56:29.786932+0000","last_became_peered":"2024-09-21T05:56:29.786932+0000","last_unstale":"2024-09-21T05:56:31.177406+0000","last_undegraded":"2024-09-21T05:56:31.177406+0000","last_fullsized":"2024-09-21T05:56:31.177406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:45.264497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176533+0000","last_change":"2024-09-21T05:56:29.810657+0000","last_active":"2024-09-21T05:56:31.176533+0000","last_peered":"2024-09-21T05:56:31.176533+0000","last_clean":"2024-09-21T05:56:31.176533+0000","last_became_active":"2024-09-21T05:56:29.810300+0000","last_became_peered":"2024-09-21T05:56:29.810300+0000","last_unstale":"2024-09-21T05:56:31.176533+0000","last_undegraded":"2024-09-21T05:56:31.176533+0000","last_fullsized":"2024-09-21T05:56:31.176533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:28.055629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796891+0000","last_change":"2024-09-21T05:56:29.788766+0000","last_active":"2024-09-21T05:56:30.796891+0000","last_peered":"2024-09-21T05:56:30.796891+0000","last_clean":"2024-09-21T05:56:30.796891+0000","last_became_active":"2024-09-21T05:56:29.788430+0000","last_became_peered":"2024-09-21T05:56:29.788430+0000","last_unstale":"2024-09-21T05:56:30.796891+0000","last_undegraded":"2024-09-21T05:56:30.796891+0000","last_fullsized":"2024-09-21T05:56:30.796891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178114+0000","last_change":"2024-09-21T05:56:29.813309+0000","last_active":"2024-09-21T05:56:31.178114+0000","last_peered":"2024-09-21T05:56:31.178114+0000","last_clean":"2024-09-21T05:56:31.178114+0000","last_became_active":"2024-09-21T05:56:29.812998+0000","last_became_peered":"2024-09-21T05:56:29.812998+0000","last_unstale":"2024-09-21T05:56:31.178114+0000","last_undegraded":"2024-09-21T05:56:31.178114+0000","last_fullsized":"2024-09-21T05:56:31.178114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:30.792281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782744+0000","last_change":"2024-09-21T05:56:29.803935+0000","last_active":"2024-09-21T05:56:30.782744+0000","last_peered":"2024-09-21T05:56:30.782744+0000","last_clean":"2024-09-21T05:56:30.782744+0000","last_became_active":"2024-09-21T05:56:29.803639+0000","last_became_peered":"2024-09-21T05:56:29.803639+0000","last_unstale":"2024-09-21T05:56:30.782744+0000","last_undegraded":"2024-09-21T05:56:30.782744+0000","last_fullsized":"2024-09-21T05:56:30.782744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:41.759804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177783+0000","last_change":"2024-09-21T05:56:29.800369+0000","last_active":"2024-09-21T05:56:31.177783+0000","last_peered":"2024-09-21T05:56:31.177783+0000","last_clean":"2024-09-21T05:56:31.177783+0000","last_became_active":"2024-09-21T05:56:29.799794+0000","last_became_peered":"2024-09-21T05:56:29.799794+0000","last_unstale":"2024-09-21T05:56:31.177783+0000","last_undegraded":"2024-09-21T05:56:31.177783+0000","last_fullsized":"2024-09-21T05:56:31.177783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:17.854511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782813+0000","last_change":"2024-09-21T05:56:29.795633+0000","last_active":"2024-09-21T05:56:30.782813+0000","last_peered":"2024-09-21T05:56:30.782813+0000","last_clean":"2024-09-21T05:56:30.782813+0000","last_became_active":"2024-09-21T05:56:29.795285+0000","last_became_peered":"2024-09-21T05:56:29.795285+0000","last_unstale":"2024-09-21T05:56:30.782813+0000","last_undegraded":"2024-09-21T05:56:30.782813+0000","last_fullsized":"2024-09-21T05:56:30.782813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:04.441455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178478+0000","last_change":"2024-09-21T05:56:29.803384+0000","last_active":"2024-09-21T05:56:31.178478+0000","last_peered":"2024-09-21T05:56:31.178478+0000","last_clean":"2024-09-21T05:56:31.178478+0000","last_became_active":"2024-09-21T05:56:29.803152+0000","last_became_peered":"2024-09-21T05:56:29.803152+0000","last_unstale":"2024-09-21T05:56:31.178478+0000","last_undegraded":"2024-09-21T05:56:31.178478+0000","last_fullsized":"2024-09-21T05:56:31.178478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:48.777031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796724+0000","last_change":"2024-09-21T05:56:29.807556+0000","last_active":"2024-09-21T05:56:30.796724+0000","last_peered":"2024-09-21T05:56:30.796724+0000","last_clean":"2024-09-21T05:56:30.796724+0000","last_became_active":"2024-09-21T05:56:29.805826+0000","last_became_peered":"2024-09-21T05:56:29.805826+0000","last_unstale":"2024-09-21T05:56:30.796724+0000","last_undegraded":"2024-09-21T05:56:30.796724+0000","last_fullsized":"2024-09-21T05:56:30.796724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:33.544641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779890+0000","last_change":"2024-09-21T05:56:29.788425+0000","last_active":"2024-09-21T05:56:30.779890+0000","last_peered":"2024-09-21T05:56:30.779890+0000","last_clean":"2024-09-21T05:56:30.779890+0000","last_became_active":"2024-09-21T05:56:29.788007+0000","last_became_peered":"2024-09-21T05:56:29.788007+0000","last_unstale":"2024-09-21T05:56:30.779890+0000","last_undegraded":"2024-09-21T05:56:30.779890+0000","last_fullsized":"2024-09-21T05:56:30.779890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:53.445482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.179201+0000","last_change":"2024-09-21T05:56:29.792314+0000","last_active":"2024-09-21T05:56:31.179201+0000","last_peered":"2024-09-21T05:56:31.179201+0000","last_clean":"2024-09-21T05:56:31.179201+0000","last_became_active":"2024-09-21T05:56:29.791869+0000","last_became_peered":"2024-09-21T05:56:29.791869+0000","last_unstale":"2024-09-21T05:56:31.179201+0000","last_undegraded":"2024-09-21T05:56:31.179201+0000","last_fullsized":"2024-09-21T05:56:31.179201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:20.952370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177862+0000","last_change":"2024-09-21T05:56:29.787078+0000","last_active":"2024-09-21T05:56:31.177862+0000","last_peered":"2024-09-21T05:56:31.177862+0000","last_clean":"2024-09-21T05:56:31.177862+0000","last_became_active":"2024-09-21T05:56:29.786725+0000","last_became_peered":"2024-09-21T05:56:29.786725+0000","last_unstale":"2024-09-21T05:56:31.177862+0000","last_undegraded":"2024-09-21T05:56:31.177862+0000","last_fullsized":"2024-09-21T05:56:31.177862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:17.854511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782870+0000","last_change":"2024-09-21T05:56:29.790180+0000","last_active":"2024-09-21T05:56:30.782870+0000","last_peered":"2024-09-21T05:56:30.782870+0000","last_clean":"2024-09-21T05:56:30.782870+0000","last_became_active":"2024-09-21T05:56:29.789793+0000","last_became_peered":"2024-09-21T05:56:29.789793+0000","last_unstale":"2024-09-21T05:56:30.782870+0000","last_undegraded":"2024-09-21T05:56:30.782870+0000","last_fullsized":"2024-09-21T05:56:30.782870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:48.361541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176727+0000","last_change":"2024-09-21T05:56:29.791764+0000","last_active":"2024-09-21T05:56:31.176727+0000","last_peered":"2024-09-21T05:56:31.176727+0000","last_clean":"2024-09-21T05:56:31.176727+0000","last_became_active":"2024-09-21T05:56:29.791358+0000","last_became_peered":"2024-09-21T05:56:29.791358+0000","last_unstale":"2024-09-21T05:56:31.176727+0000","last_undegraded":"2024-09-21T05:56:31.176727+0000","last_fullsized":"2024-09-21T05:56:31.176727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:35.770789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796700+0000","last_change":"2024-09-21T05:56:29.798644+0000","last_active":"2024-09-21T05:56:30.796700+0000","last_peered":"2024-09-21T05:56:30.796700+0000","last_clean":"2024-09-21T05:56:30.796700+0000","last_became_active":"2024-09-21T05:56:29.798271+0000","last_became_peered":"2024-09-21T05:56:29.798271+0000","last_unstale":"2024-09-21T05:56:30.796700+0000","last_undegraded":"2024-09-21T05:56:30.796700+0000","last_fullsized":"2024-09-21T05:56:30.796700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:39.467641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780441+0000","last_change":"2024-09-21T05:56:29.786401+0000","last_active":"2024-09-21T05:56:30.780441+0000","last_peered":"2024-09-21T05:56:30.780441+0000","last_clean":"2024-09-21T05:56:30.780441+0000","last_became_active":"2024-09-21T05:56:29.786004+0000","last_became_peered":"2024-09-21T05:56:29.786004+0000","last_unstale":"2024-09-21T05:56:30.780441+0000","last_undegraded":"2024-09-21T05:56:30.780441+0000","last_fullsized":"2024-09-21T05:56:30.780441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:32.904074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178010+0000","last_change":"2024-09-21T05:56:28.806999+0000","last_active":"2024-09-21T05:56:31.178010+0000","last_peered":"2024-09-21T05:56:31.178010+0000","last_clean":"2024-09-21T05:56:31.178010+0000","last_became_active":"2024-09-21T05:56:28.806609+0000","last_became_peered":"2024-09-21T05:56:28.806609+0000","last_unstale":"2024-09-21T05:56:31.178010+0000","last_undegraded":"2024-09-21T05:56:31.178010+0000","last_fullsized":"2024-09-21T05:56:31.178010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:11.564758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178105+0000","last_change":"2024-09-21T05:56:29.799191+0000","last_active":"2024-09-21T05:56:31.178105+0000","last_peered":"2024-09-21T05:56:31.178105+0000","last_clean":"2024-09-21T05:56:31.178105+0000","last_became_active":"2024-09-21T05:56:29.798839+0000","last_became_peered":"2024-09-21T05:56:29.798839+0000","last_unstale":"2024-09-21T05:56:31.178105+0000","last_undegraded":"2024-09-21T05:56:31.178105+0000","last_fullsized":"2024-09-21T05:56:31.178105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:28.783969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796226+0000","last_change":"2024-09-21T05:56:28.794189+0000","last_active":"2024-09-21T05:56:30.796226+0000","last_peered":"2024-09-21T05:56:30.796226+0000","last_clean":"2024-09-21T05:56:30.796226+0000","last_became_active":"2024-09-21T05:56:28.793684+0000","last_became_peered":"2024-09-21T05:56:28.793684+0000","last_unstale":"2024-09-21T05:56:30.796226+0000","last_undegraded":"2024-09-21T05:56:30.796226+0000","last_fullsized":"2024-09-21T05:56:30.796226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:08.868280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177368+0000","last_change":"2024-09-21T05:56:28.788375+0000","last_active":"2024-09-21T05:56:31.177368+0000","last_peered":"2024-09-21T05:56:31.177368+0000","last_clean":"2024-09-21T05:56:31.177368+0000","last_became_active":"2024-09-21T05:56:28.788044+0000","last_became_peered":"2024-09-21T05:56:28.788044+0000","last_unstale":"2024-09-21T05:56:31.177368+0000","last_undegraded":"2024-09-21T05:56:31.177368+0000","last_fullsized":"2024-09-21T05:56:31.177368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:11.883400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177781+0000","last_change":"2024-09-21T05:56:29.792845+0000","last_active":"2024-09-21T05:56:31.177781+0000","last_peered":"2024-09-21T05:56:31.177781+0000","last_clean":"2024-09-21T05:56:31.177781+0000","last_became_active":"2024-09-21T05:56:29.792452+0000","last_became_peered":"2024-09-21T05:56:29.792452+0000","last_unstale":"2024-09-21T05:56:31.177781+0000","last_undegraded":"2024-09-21T05:56:31.177781+0000","last_fullsized":"2024-09-21T05:56:31.177781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:14.751206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782637+0000","last_change":"2024-09-21T05:56:28.796778+0000","last_active":"2024-09-21T05:56:30.782637+0000","last_peered":"2024-09-21T05:56:30.782637+0000","last_clean":"2024-09-21T05:56:30.782637+0000","last_became_active":"2024-09-21T05:56:28.792745+0000","last_became_peered":"2024-09-21T05:56:28.792745+0000","last_unstale":"2024-09-21T05:56:30.782637+0000","last_undegraded":"2024-09-21T05:56:30.782637+0000","last_fullsized":"2024-09-21T05:56:30.782637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:33.703504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779980+0000","last_change":"2024-09-21T05:56:29.790318+0000","last_active":"2024-09-21T05:56:30.779980+0000","last_peered":"2024-09-21T05:56:30.779980+0000","last_clean":"2024-09-21T05:56:30.779980+0000","last_became_active":"2024-09-21T05:56:29.789628+0000","last_became_peered":"2024-09-21T05:56:29.789628+0000","last_unstale":"2024-09-21T05:56:30.779980+0000","last_undegraded":"2024-09-21T05:56:30.779980+0000","last_fullsized":"2024-09-21T05:56:30.779980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:39.627991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796583+0000","last_change":"2024-09-21T05:56:28.798412+0000","last_active":"2024-09-21T05:56:30.796583+0000","last_peered":"2024-09-21T05:56:30.796583+0000","last_clean":"2024-09-21T05:56:30.796583+0000","last_became_active":"2024-09-21T05:56:28.796923+0000","last_became_peered":"2024-09-21T05:56:28.796923+0000","last_unstale":"2024-09-21T05:56:30.796583+0000","last_undegraded":"2024-09-21T05:56:30.796583+0000","last_fullsized":"2024-09-21T05:56:30.796583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:31.896210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783435+0000","last_change":"2024-09-21T05:56:29.801946+0000","last_active":"2024-09-21T05:56:30.783435+0000","last_peered":"2024-09-21T05:56:30.783435+0000","last_clean":"2024-09-21T05:56:30.783435+0000","last_became_active":"2024-09-21T05:56:29.801320+0000","last_became_peered":"2024-09-21T05:56:29.801320+0000","last_unstale":"2024-09-21T05:56:30.783435+0000","last_undegraded":"2024-09-21T05:56:30.783435+0000","last_fullsized":"2024-09-21T05:56:30.783435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:28.055629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783473+0000","last_change":"2024-09-21T05:56:28.792022+0000","last_active":"2024-09-21T05:56:30.783473+0000","last_peered":"2024-09-21T05:56:30.783473+0000","last_clean":"2024-09-21T05:56:30.783473+0000","last_became_active":"2024-09-21T05:56:28.789954+0000","last_became_peered":"2024-09-21T05:56:28.789954+0000","last_unstale":"2024-09-21T05:56:30.783473+0000","last_undegraded":"2024-09-21T05:56:30.783473+0000","last_fullsized":"2024-09-21T05:56:30.783473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:31.896210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177113+0000","last_change":"2024-09-21T05:56:29.799322+0000","last_active":"2024-09-21T05:56:31.177113+0000","last_peered":"2024-09-21T05:56:31.177113+0000","last_clean":"2024-09-21T05:56:31.177113+0000","last_became_active":"2024-09-21T05:56:29.798684+0000","last_became_peered":"2024-09-21T05:56:29.798684+0000","last_unstale":"2024-09-21T05:56:31.177113+0000","last_undegraded":"2024-09-21T05:56:31.177113+0000","last_fullsized":"2024-09-21T05:56:31.177113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:13.158342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779982+0000","last_change":"2024-09-21T05:56:28.791961+0000","last_active":"2024-09-21T05:56:30.779982+0000","last_peered":"2024-09-21T05:56:30.779982+0000","last_clean":"2024-09-21T05:56:30.779982+0000","last_became_active":"2024-09-21T05:56:28.791654+0000","last_became_peered":"2024-09-21T05:56:28.791654+0000","last_unstale":"2024-09-21T05:56:30.779982+0000","last_undegraded":"2024-09-21T05:56:30.779982+0000","last_fullsized":"2024-09-21T05:56:30.779982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:29.186182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.781638+0000","last_change":"2024-09-21T05:56:29.808824+0000","last_active":"2024-09-21T05:56:30.781638+0000","last_peered":"2024-09-21T05:56:30.781638+0000","last_clean":"2024-09-21T05:56:30.781638+0000","last_became_active":"2024-09-21T05:56:29.808454+0000","last_became_peered":"2024-09-21T05:56:29.808454+0000","last_unstale":"2024-09-21T05:56:30.781638+0000","last_undegraded":"2024-09-21T05:56:30.781638+0000","last_fullsized":"2024-09-21T05:56:30.781638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:34.185949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796804+0000","last_change":"2024-09-21T05:56:28.785344+0000","last_active":"2024-09-21T05:56:30.796804+0000","last_peered":"2024-09-21T05:56:30.796804+0000","last_clean":"2024-09-21T05:56:30.796804+0000","last_became_active":"2024-09-21T05:56:28.784997+0000","last_became_peered":"2024-09-21T05:56:28.784997+0000","last_unstale":"2024-09-21T05:56:30.796804+0000","last_undegraded":"2024-09-21T05:56:30.796804+0000","last_fullsized":"2024-09-21T05:56:30.796804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176070+0000","last_change":"2024-09-21T05:56:29.801655+0000","last_active":"2024-09-21T05:56:31.176070+0000","last_peered":"2024-09-21T05:56:31.176070+0000","last_clean":"2024-09-21T05:56:31.176070+0000","last_became_active":"2024-09-21T05:56:29.801337+0000","last_became_peered":"2024-09-21T05:56:29.801337+0000","last_unstale":"2024-09-21T05:56:31.176070+0000","last_undegraded":"2024-09-21T05:56:31.176070+0000","last_fullsized":"2024-09-21T05:56:31.176070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:42.929175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178176+0000","last_change":"2024-09-21T05:56:28.788833+0000","last_active":"2024-09-21T05:56:31.178176+0000","last_peered":"2024-09-21T05:56:31.178176+0000","last_clean":"2024-09-21T05:56:31.178176+0000","last_became_active":"2024-09-21T05:56:28.788469+0000","last_became_peered":"2024-09-21T05:56:28.788469+0000","last_unstale":"2024-09-21T05:56:31.178176+0000","last_undegraded":"2024-09-21T05:56:31.178176+0000","last_fullsized":"2024-09-21T05:56:31.178176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:44.013815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177487+0000","last_change":"2024-09-21T05:56:29.807840+0000","last_active":"2024-09-21T05:56:31.177487+0000","last_peered":"2024-09-21T05:56:31.177487+0000","last_clean":"2024-09-21T05:56:31.177487+0000","last_became_active":"2024-09-21T05:56:29.807524+0000","last_became_peered":"2024-09-21T05:56:29.807524+0000","last_unstale":"2024-09-21T05:56:31.177487+0000","last_undegraded":"2024-09-21T05:56:31.177487+0000","last_fullsized":"2024-09-21T05:56:31.177487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:34.812658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782834+0000","last_change":"2024-09-21T05:56:28.790494+0000","last_active":"2024-09-21T05:56:30.782834+0000","last_peered":"2024-09-21T05:56:30.782834+0000","last_clean":"2024-09-21T05:56:30.782834+0000","last_became_active":"2024-09-21T05:56:28.790003+0000","last_became_peered":"2024-09-21T05:56:28.790003+0000","last_unstale":"2024-09-21T05:56:30.782834+0000","last_undegraded":"2024-09-21T05:56:30.782834+0000","last_fullsized":"2024-09-21T05:56:30.782834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176130+0000","last_change":"2024-09-21T05:56:29.810515+0000","last_active":"2024-09-21T05:56:31.176130+0000","last_peered":"2024-09-21T05:56:31.176130+0000","last_clean":"2024-09-21T05:56:31.176130+0000","last_became_active":"2024-09-21T05:56:29.810188+0000","last_became_peered":"2024-09-21T05:56:29.810188+0000","last_unstale":"2024-09-21T05:56:31.176130+0000","last_undegraded":"2024-09-21T05:56:31.176130+0000","last_fullsized":"2024-09-21T05:56:31.176130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:48.361541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796183+0000","last_change":"2024-09-21T05:56:28.794032+0000","last_active":"2024-09-21T05:56:30.796183+0000","last_peered":"2024-09-21T05:56:30.796183+0000","last_clean":"2024-09-21T05:56:30.796183+0000","last_became_active":"2024-09-21T05:56:28.793671+0000","last_became_peered":"2024-09-21T05:56:28.793671+0000","last_unstale":"2024-09-21T05:56:30.796183+0000","last_undegraded":"2024-09-21T05:56:30.796183+0000","last_fullsized":"2024-09-21T05:56:30.796183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:44.652224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796406+0000","last_change":"2024-09-21T05:56:29.798550+0000","last_active":"2024-09-21T05:56:30.796406+0000","last_peered":"2024-09-21T05:56:30.796406+0000","last_clean":"2024-09-21T05:56:30.796406+0000","last_became_active":"2024-09-21T05:56:29.797719+0000","last_became_peered":"2024-09-21T05:56:29.797719+0000","last_unstale":"2024-09-21T05:56:30.796406+0000","last_undegraded":"2024-09-21T05:56:30.796406+0000","last_fullsized":"2024-09-21T05:56:30.796406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:17.854511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176419+0000","last_change":"2024-09-21T05:56:28.777283+0000","last_active":"2024-09-21T05:56:31.176419+0000","last_peered":"2024-09-21T05:56:31.176419+0000","last_clean":"2024-09-21T05:56:31.176419+0000","last_became_active":"2024-09-21T05:56:28.776907+0000","last_became_peered":"2024-09-21T05:56:28.776907+0000","last_unstale":"2024-09-21T05:56:31.176419+0000","last_undegraded":"2024-09-21T05:56:31.176419+0000","last_fullsized":"2024-09-21T05:56:31.176419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176488+0000","last_change":"2024-09-21T05:56:29.810738+0000","last_active":"2024-09-21T05:56:31.176488+0000","last_peered":"2024-09-21T05:56:31.176488+0000","last_clean":"2024-09-21T05:56:31.176488+0000","last_became_active":"2024-09-21T05:56:29.810296+0000","last_became_peered":"2024-09-21T05:56:29.810296+0000","last_unstale":"2024-09-21T05:56:31.176488+0000","last_undegraded":"2024-09-21T05:56:31.176488+0000","last_fullsized":"2024-09-21T05:56:31.176488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:38.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.781591+0000","last_change":"2024-09-21T05:56:28.793250+0000","last_active":"2024-09-21T05:56:30.781591+0000","last_peered":"2024-09-21T05:56:30.781591+0000","last_clean":"2024-09-21T05:56:30.781591+0000","last_became_active":"2024-09-21T05:56:28.792913+0000","last_became_peered":"2024-09-21T05:56:28.792913+0000","last_unstale":"2024-09-21T05:56:30.781591+0000","last_undegraded":"2024-09-21T05:56:30.781591+0000","last_fullsized":"2024-09-21T05:56:30.781591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:39.751978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176567+0000","last_change":"2024-09-21T05:56:29.811980+0000","last_active":"2024-09-21T05:56:31.176567+0000","last_peered":"2024-09-21T05:56:31.176567+0000","last_clean":"2024-09-21T05:56:31.176567+0000","last_became_active":"2024-09-21T05:56:29.811133+0000","last_became_peered":"2024-09-21T05:56:29.811133+0000","last_unstale":"2024-09-21T05:56:31.176567+0000","last_undegraded":"2024-09-21T05:56:31.176567+0000","last_fullsized":"2024-09-21T05:56:31.176567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:16.791089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177124+0000","last_change":"2024-09-21T05:56:28.784954+0000","last_active":"2024-09-21T05:56:31.177124+0000","last_peered":"2024-09-21T05:56:31.177124+0000","last_clean":"2024-09-21T05:56:31.177124+0000","last_became_active":"2024-09-21T05:56:28.784610+0000","last_became_peered":"2024-09-21T05:56:28.784610+0000","last_unstale":"2024-09-21T05:56:31.177124+0000","last_undegraded":"2024-09-21T05:56:31.177124+0000","last_fullsized":"2024-09-21T05:56:31.177124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:08.868280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176832+0000","last_change":"2024-09-21T05:56:29.811442+0000","last_active":"2024-09-21T05:56:31.176832+0000","last_peered":"2024-09-21T05:56:31.176832+0000","last_clean":"2024-09-21T05:56:31.176832+0000","last_became_active":"2024-09-21T05:56:29.809712+0000","last_became_peered":"2024-09-21T05:56:29.809712+0000","last_unstale":"2024-09-21T05:56:31.176832+0000","last_undegraded":"2024-09-21T05:56:31.176832+0000","last_fullsized":"2024-09-21T05:56:31.176832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:11.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780028+0000","last_change":"2024-09-21T05:56:28.788606+0000","last_active":"2024-09-21T05:56:30.780028+0000","last_peered":"2024-09-21T05:56:30.780028+0000","last_clean":"2024-09-21T05:56:30.780028+0000","last_became_active":"2024-09-21T05:56:28.787648+0000","last_became_peered":"2024-09-21T05:56:28.787648+0000","last_unstale":"2024-09-21T05:56:30.780028+0000","last_undegraded":"2024-09-21T05:56:30.780028+0000","last_fullsized":"2024-09-21T05:56:30.780028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:37.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783198+0000","last_change":"2024-09-21T05:56:29.789459+0000","last_active":"2024-09-21T05:56:30.783198+0000","last_peered":"2024-09-21T05:56:30.783198+0000","last_clean":"2024-09-21T05:56:30.783198+0000","last_became_active":"2024-09-21T05:56:29.785168+0000","last_became_peered":"2024-09-21T05:56:29.785168+0000","last_unstale":"2024-09-21T05:56:30.783198+0000","last_undegraded":"2024-09-21T05:56:30.783198+0000","last_fullsized":"2024-09-21T05:56:30.783198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:38.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176888+0000","last_change":"2024-09-21T05:56:28.790714+0000","last_active":"2024-09-21T05:56:31.176888+0000","last_peered":"2024-09-21T05:56:31.176888+0000","last_clean":"2024-09-21T05:56:31.176888+0000","last_became_active":"2024-09-21T05:56:28.790280+0000","last_became_peered":"2024-09-21T05:56:28.790280+0000","last_unstale":"2024-09-21T05:56:31.176888+0000","last_undegraded":"2024-09-21T05:56:31.176888+0000","last_fullsized":"2024-09-21T05:56:31.176888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:43.786792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176561+0000","last_change":"2024-09-21T05:56:29.805253+0000","last_active":"2024-09-21T05:56:31.176561+0000","last_peered":"2024-09-21T05:56:31.176561+0000","last_clean":"2024-09-21T05:56:31.176561+0000","last_became_active":"2024-09-21T05:56:29.801598+0000","last_became_peered":"2024-09-21T05:56:29.801598+0000","last_unstale":"2024-09-21T05:56:31.176561+0000","last_undegraded":"2024-09-21T05:56:31.176561+0000","last_fullsized":"2024-09-21T05:56:31.176561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:22.963835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175921+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:56:31.175921+0000","last_peered":"2024-09-21T05:56:31.175921+0000","last_clean":"2024-09-21T05:56:31.175921+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:56:31.175921+0000","last_undegraded":"2024-09-21T05:56:31.175921+0000","last_fullsized":"2024-09-21T05:56:31.175921+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782518+0000","last_change":"2024-09-21T05:56:28.781190+0000","last_active":"2024-09-21T05:56:30.782518+0000","last_peered":"2024-09-21T05:56:30.782518+0000","last_clean":"2024-09-21T05:56:30.782518+0000","last_became_active":"2024-09-21T05:56:28.780808+0000","last_became_peered":"2024-09-21T05:56:28.780808+0000","last_unstale":"2024-09-21T05:56:30.782518+0000","last_undegraded":"2024-09-21T05:56:30.782518+0000","last_fullsized":"2024-09-21T05:56:30.782518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:44.652224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177804+0000","last_change":"2024-09-21T05:56:29.795930+0000","last_active":"2024-09-21T05:56:31.177804+0000","last_peered":"2024-09-21T05:56:31.177804+0000","last_clean":"2024-09-21T05:56:31.177804+0000","last_became_active":"2024-09-21T05:56:29.795552+0000","last_became_peered":"2024-09-21T05:56:29.795552+0000","last_unstale":"2024-09-21T05:56:31.177804+0000","last_undegraded":"2024-09-21T05:56:31.177804+0000","last_fullsized":"2024-09-21T05:56:31.177804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:22.412975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175735+0000","last_change":"2024-09-21T05:56:28.805633+0000","last_active":"2024-09-21T05:56:31.175735+0000","last_peered":"2024-09-21T05:56:31.175735+0000","last_clean":"2024-09-21T05:56:31.175735+0000","last_became_active":"2024-09-21T05:56:28.804239+0000","last_became_peered":"2024-09-21T05:56:28.804239+0000","last_unstale":"2024-09-21T05:56:31.175735+0000","last_undegraded":"2024-09-21T05:56:31.175735+0000","last_fullsized":"2024-09-21T05:56:31.175735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:53.019618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178389+0000","last_change":"2024-09-21T05:56:29.803538+0000","last_active":"2024-09-21T05:56:31.178389+0000","last_peered":"2024-09-21T05:56:31.178389+0000","last_clean":"2024-09-21T05:56:31.178389+0000","last_became_active":"2024-09-21T05:56:29.803153+0000","last_became_peered":"2024-09-21T05:56:29.803153+0000","last_unstale":"2024-09-21T05:56:31.178389+0000","last_undegraded":"2024-09-21T05:56:31.178389+0000","last_fullsized":"2024-09-21T05:56:31.178389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:18.925867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176800+0000","last_change":"2024-09-21T05:56:28.786025+0000","last_active":"2024-09-21T05:56:31.176800+0000","last_peered":"2024-09-21T05:56:31.176800+0000","last_clean":"2024-09-21T05:56:31.176800+0000","last_became_active":"2024-09-21T05:56:28.785690+0000","last_became_peered":"2024-09-21T05:56:28.785690+0000","last_unstale":"2024-09-21T05:56:31.176800+0000","last_undegraded":"2024-09-21T05:56:31.176800+0000","last_fullsized":"2024-09-21T05:56:31.176800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:44.652224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779811+0000","last_change":"2024-09-21T05:56:29.785876+0000","last_active":"2024-09-21T05:56:30.779811+0000","last_peered":"2024-09-21T05:56:30.779811+0000","last_clean":"2024-09-21T05:56:30.779811+0000","last_became_active":"2024-09-21T05:56:29.785564+0000","last_became_peered":"2024-09-21T05:56:29.785564+0000","last_unstale":"2024-09-21T05:56:30.779811+0000","last_undegraded":"2024-09-21T05:56:30.779811+0000","last_fullsized":"2024-09-21T05:56:30.779811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:04.812480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.782960+0000","last_change":"2024-09-21T05:56:28.785911+0000","last_active":"2024-09-21T05:56:30.782960+0000","last_peered":"2024-09-21T05:56:30.782960+0000","last_clean":"2024-09-21T05:56:30.782960+0000","last_became_active":"2024-09-21T05:56:28.785413+0000","last_became_peered":"2024-09-21T05:56:28.785413+0000","last_unstale":"2024-09-21T05:56:30.782960+0000","last_undegraded":"2024-09-21T05:56:30.782960+0000","last_fullsized":"2024-09-21T05:56:30.782960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:08.868280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780540+0000","last_change":"2024-09-21T05:56:29.789624+0000","last_active":"2024-09-21T05:56:30.780540+0000","last_peered":"2024-09-21T05:56:30.780540+0000","last_clean":"2024-09-21T05:56:30.780540+0000","last_became_active":"2024-09-21T05:56:29.789305+0000","last_became_peered":"2024-09-21T05:56:29.789305+0000","last_unstale":"2024-09-21T05:56:30.780540+0000","last_undegraded":"2024-09-21T05:56:30.780540+0000","last_fullsized":"2024-09-21T05:56:30.780540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:34.711369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780096+0000","last_change":"2024-09-21T05:56:28.791893+0000","last_active":"2024-09-21T05:56:30.780096+0000","last_peered":"2024-09-21T05:56:30.780096+0000","last_clean":"2024-09-21T05:56:30.780096+0000","last_became_active":"2024-09-21T05:56:28.791580+0000","last_became_peered":"2024-09-21T05:56:28.791580+0000","last_unstale":"2024-09-21T05:56:30.780096+0000","last_undegraded":"2024-09-21T05:56:30.780096+0000","last_fullsized":"2024-09-21T05:56:30.780096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.359445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.784018+0000","last_change":"2024-09-21T05:56:29.792438+0000","last_active":"2024-09-21T05:56:30.784018+0000","last_peered":"2024-09-21T05:56:30.784018+0000","last_clean":"2024-09-21T05:56:30.784018+0000","last_became_active":"2024-09-21T05:56:29.792034+0000","last_became_peered":"2024-09-21T05:56:29.792034+0000","last_unstale":"2024-09-21T05:56:30.784018+0000","last_undegraded":"2024-09-21T05:56:30.784018+0000","last_fullsized":"2024-09-21T05:56:30.784018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:30.461662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176595+0000","last_change":"2024-09-21T05:56:28.805473+0000","last_active":"2024-09-21T05:56:31.176595+0000","last_peered":"2024-09-21T05:56:31.176595+0000","last_clean":"2024-09-21T05:56:31.176595+0000","last_became_active":"2024-09-21T05:56:28.804850+0000","last_became_peered":"2024-09-21T05:56:28.804850+0000","last_unstale":"2024-09-21T05:56:31.176595+0000","last_undegraded":"2024-09-21T05:56:31.176595+0000","last_fullsized":"2024-09-21T05:56:31.176595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:44.256633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780054+0000","last_change":"2024-09-21T05:56:29.784905+0000","last_active":"2024-09-21T05:56:30.780054+0000","last_peered":"2024-09-21T05:56:30.780054+0000","last_clean":"2024-09-21T05:56:30.780054+0000","last_became_active":"2024-09-21T05:56:29.784562+0000","last_became_peered":"2024-09-21T05:56:29.784562+0000","last_unstale":"2024-09-21T05:56:30.780054+0000","last_undegraded":"2024-09-21T05:56:30.780054+0000","last_fullsized":"2024-09-21T05:56:30.780054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:26.482139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.795965+0000","last_change":"2024-09-21T05:56:28.790111+0000","last_active":"2024-09-21T05:56:30.795965+0000","last_peered":"2024-09-21T05:56:30.795965+0000","last_clean":"2024-09-21T05:56:30.795965+0000","last_became_active":"2024-09-21T05:56:28.789760+0000","last_became_peered":"2024-09-21T05:56:28.789760+0000","last_unstale":"2024-09-21T05:56:30.795965+0000","last_undegraded":"2024-09-21T05:56:30.795965+0000","last_fullsized":"2024-09-21T05:56:30.795965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:51.060754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178344+0000","last_change":"2024-09-21T05:56:29.812261+0000","last_active":"2024-09-21T05:56:31.178344+0000","last_peered":"2024-09-21T05:56:31.178344+0000","last_clean":"2024-09-21T05:56:31.178344+0000","last_became_active":"2024-09-21T05:56:29.811916+0000","last_became_peered":"2024-09-21T05:56:29.811916+0000","last_unstale":"2024-09-21T05:56:31.178344+0000","last_undegraded":"2024-09-21T05:56:31.178344+0000","last_fullsized":"2024-09-21T05:56:31.178344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:42.929175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796707+0000","last_change":"2024-09-21T05:56:28.804549+0000","last_active":"2024-09-21T05:56:30.796707+0000","last_peered":"2024-09-21T05:56:30.796707+0000","last_clean":"2024-09-21T05:56:30.796707+0000","last_became_active":"2024-09-21T05:56:28.804182+0000","last_became_peered":"2024-09-21T05:56:28.804182+0000","last_unstale":"2024-09-21T05:56:30.796707+0000","last_undegraded":"2024-09-21T05:56:30.796707+0000","last_fullsized":"2024-09-21T05:56:30.796707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:33.703504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178183+0000","last_change":"2024-09-21T05:56:29.797061+0000","last_active":"2024-09-21T05:56:31.178183+0000","last_peered":"2024-09-21T05:56:31.178183+0000","last_clean":"2024-09-21T05:56:31.178183+0000","last_became_active":"2024-09-21T05:56:29.796608+0000","last_became_peered":"2024-09-21T05:56:29.796608+0000","last_unstale":"2024-09-21T05:56:31.178183+0000","last_undegraded":"2024-09-21T05:56:31.178183+0000","last_fullsized":"2024-09-21T05:56:31.178183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:14.929983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175725+0000","last_change":"2024-09-21T05:56:28.804158+0000","last_active":"2024-09-21T05:56:31.175725+0000","last_peered":"2024-09-21T05:56:31.175725+0000","last_clean":"2024-09-21T05:56:31.175725+0000","last_became_active":"2024-09-21T05:56:28.803813+0000","last_became_peered":"2024-09-21T05:56:28.803813+0000","last_unstale":"2024-09-21T05:56:31.175725+0000","last_undegraded":"2024-09-21T05:56:31.175725+0000","last_fullsized":"2024-09-21T05:56:31.175725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:49.073829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.780422+0000","last_change":"2024-09-21T05:56:29.797845+0000","last_active":"2024-09-21T05:56:30.780422+0000","last_peered":"2024-09-21T05:56:30.780422+0000","last_clean":"2024-09-21T05:56:30.780422+0000","last_became_active":"2024-09-21T05:56:29.797380+0000","last_became_peered":"2024-09-21T05:56:29.797380+0000","last_unstale":"2024-09-21T05:56:30.780422+0000","last_undegraded":"2024-09-21T05:56:30.780422+0000","last_fullsized":"2024-09-21T05:56:30.780422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:09.876144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779895+0000","last_change":"2024-09-21T05:56:28.801904+0000","last_active":"2024-09-21T05:56:30.779895+0000","last_peered":"2024-09-21T05:56:30.779895+0000","last_clean":"2024-09-21T05:56:30.779895+0000","last_became_active":"2024-09-21T05:56:28.801540+0000","last_became_peered":"2024-09-21T05:56:28.801540+0000","last_unstale":"2024-09-21T05:56:30.779895+0000","last_undegraded":"2024-09-21T05:56:30.779895+0000","last_fullsized":"2024-09-21T05:56:30.779895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:06.676644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177960+0000","last_change":"2024-09-21T05:56:29.782893+0000","last_active":"2024-09-21T05:56:31.177960+0000","last_peered":"2024-09-21T05:56:31.177960+0000","last_clean":"2024-09-21T05:56:31.177960+0000","last_became_active":"2024-09-21T05:56:29.782496+0000","last_became_peered":"2024-09-21T05:56:29.782496+0000","last_unstale":"2024-09-21T05:56:31.177960+0000","last_undegraded":"2024-09-21T05:56:31.177960+0000","last_fullsized":"2024-09-21T05:56:31.177960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:33.544641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.779828+0000","last_change":"2024-09-21T05:56:28.784648+0000","last_active":"2024-09-21T05:56:30.779828+0000","last_peered":"2024-09-21T05:56:30.779828+0000","last_clean":"2024-09-21T05:56:30.779828+0000","last_became_active":"2024-09-21T05:56:28.784333+0000","last_became_peered":"2024-09-21T05:56:28.784333+0000","last_unstale":"2024-09-21T05:56:30.779828+0000","last_undegraded":"2024-09-21T05:56:30.779828+0000","last_fullsized":"2024-09-21T05:56:30.779828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:15.892877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178255+0000","last_change":"2024-09-21T05:56:29.789722+0000","last_active":"2024-09-21T05:56:31.178255+0000","last_peered":"2024-09-21T05:56:31.178255+0000","last_clean":"2024-09-21T05:56:31.178255+0000","last_became_active":"2024-09-21T05:56:29.789333+0000","last_became_peered":"2024-09-21T05:56:29.789333+0000","last_unstale":"2024-09-21T05:56:31.178255+0000","last_undegraded":"2024-09-21T05:56:31.178255+0000","last_fullsized":"2024-09-21T05:56:31.178255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176267+0000","last_change":"2024-09-21T05:56:28.807252+0000","last_active":"2024-09-21T05:56:31.176267+0000","last_peered":"2024-09-21T05:56:31.176267+0000","last_clean":"2024-09-21T05:56:31.176267+0000","last_became_active":"2024-09-21T05:56:28.806895+0000","last_became_peered":"2024-09-21T05:56:28.806895+0000","last_unstale":"2024-09-21T05:56:31.176267+0000","last_undegraded":"2024-09-21T05:56:31.176267+0000","last_fullsized":"2024-09-21T05:56:31.176267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:34.762925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.796596+0000","last_change":"2024-09-21T05:56:29.781497+0000","last_active":"2024-09-21T05:56:30.796596+0000","last_peered":"2024-09-21T05:56:30.796596+0000","last_clean":"2024-09-21T05:56:30.796596+0000","last_became_active":"2024-09-21T05:56:29.780460+0000","last_became_peered":"2024-09-21T05:56:29.780460+0000","last_unstale":"2024-09-21T05:56:30.796596+0000","last_undegraded":"2024-09-21T05:56:30.796596+0000","last_fullsized":"2024-09-21T05:56:30.796596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:55.735758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783852+0000","last_change":"2024-09-21T05:56:29.788038+0000","last_active":"2024-09-21T05:56:30.783852+0000","last_peered":"2024-09-21T05:56:30.783852+0000","last_clean":"2024-09-21T05:56:30.783852+0000","last_became_active":"2024-09-21T05:56:29.784063+0000","last_became_peered":"2024-09-21T05:56:29.784063+0000","last_unstale":"2024-09-21T05:56:30.783852+0000","last_undegraded":"2024-09-21T05:56:30.783852+0000","last_fullsized":"2024-09-21T05:56:30.783852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:22.850164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.175903+0000","last_change":"2024-09-21T05:56:28.808759+0000","last_active":"2024-09-21T05:56:31.175903+0000","last_peered":"2024-09-21T05:56:31.175903+0000","last_clean":"2024-09-21T05:56:31.175903+0000","last_became_active":"2024-09-21T05:56:28.808371+0000","last_became_peered":"2024-09-21T05:56:28.808371+0000","last_unstale":"2024-09-21T05:56:31.175903+0000","last_undegraded":"2024-09-21T05:56:31.175903+0000","last_fullsized":"2024-09-21T05:56:31.175903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:23.035025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:30.783562+0000","last_change":"2024-09-21T05:56:29.791927+0000","last_active":"2024-09-21T05:56:30.783562+0000","last_peered":"2024-09-21T05:56:30.783562+0000","last_clean":"2024-09-21T05:56:30.783562+0000","last_became_active":"2024-09-21T05:56:29.790091+0000","last_became_peered":"2024-09-21T05:56:29.790091+0000","last_unstale":"2024-09-21T05:56:30.783562+0000","last_undegraded":"2024-09-21T05:56:30.783562+0000","last_fullsized":"2024-09-21T05:56:30.783562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:22.963835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.176216+0000","last_change":"2024-09-21T05:56:28.804798+0000","last_active":"2024-09-21T05:56:31.176216+0000","last_peered":"2024-09-21T05:56:31.176216+0000","last_clean":"2024-09-21T05:56:31.176216+0000","last_became_active":"2024-09-21T05:56:28.803848+0000","last_became_peered":"2024-09-21T05:56:28.803848+0000","last_unstale":"2024-09-21T05:56:31.176216+0000","last_undegraded":"2024-09-21T05:56:31.176216+0000","last_fullsized":"2024-09-21T05:56:31.176216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:32.979720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178080+0000","last_change":"2024-09-21T05:56:29.798957+0000","last_active":"2024-09-21T05:56:31.178080+0000","last_peered":"2024-09-21T05:56:31.178080+0000","last_clean":"2024-09-21T05:56:31.178080+0000","last_became_active":"2024-09-21T05:56:29.798223+0000","last_became_peered":"2024-09-21T05:56:29.798223+0000","last_unstale":"2024-09-21T05:56:31.178080+0000","last_undegraded":"2024-09-21T05:56:31.178080+0000","last_fullsized":"2024-09-21T05:56:31.178080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:39.467641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177351+0000","last_change":"2024-09-21T05:56:28.783936+0000","last_active":"2024-09-21T05:56:31.177351+0000","last_peered":"2024-09-21T05:56:31.177351+0000","last_clean":"2024-09-21T05:56:31.177351+0000","last_became_active":"2024-09-21T05:56:28.783541+0000","last_became_peered":"2024-09-21T05:56:28.783541+0000","last_unstale":"2024-09-21T05:56:31.177351+0000","last_undegraded":"2024-09-21T05:56:31.177351+0000","last_fullsized":"2024-09-21T05:56:31.177351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:38.675175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.178320+0000","last_change":"2024-09-21T05:56:29.785856+0000","last_active":"2024-09-21T05:56:31.178320+0000","last_peered":"2024-09-21T05:56:31.178320+0000","last_clean":"2024-09-21T05:56:31.178320+0000","last_became_active":"2024-09-21T05:56:29.785475+0000","last_became_peered":"2024-09-21T05:56:29.785475+0000","last_unstale":"2024-09-21T05:56:31.178320+0000","last_undegraded":"2024-09-21T05:56:31.178320+0000","last_fullsized":"2024-09-21T05:56:31.178320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:28.753562+0000","last_clean_scrub_stamp":"2024-09-21T05:56:28.753562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:21.326466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T05:56:31.177339+0000","last_change":"2024-09-21T05:56:28.804387+0000","last_active":"2024-09-21T05:56:31.177339+0000","last_peered":"2024-09-21T05:56:31.177339+0000","last_clean":"2024-09-21T05:56:31.177339+0000","last_became_active":"2024-09-21T05:56:28.804116+0000","last_became_peered":"2024-09-21T05:56:28.804116+0000","last_unstale":"2024-09-21T05:56:31.177339+0000","last_undegraded":"2024-09-21T05:56:31.177339+0000","last_fullsized":"2024-09-21T05:56:31.177339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:27.745698+0000","last_clean_scrub_stamp":"2024-09-21T05:56:27.745698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:19.806590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":667234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":76410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":667290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:56:44.410 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:56:44.410 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T05:56:44.410 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T05:56:44.411 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T05:56:44.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.604+0000 7fc2d9000640 1 -- 172.21.15.50:0/591942436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2d4074b20 msgr2=0x7fc2d415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:44.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.604+0000 7fc2d9000640 1 --2- 172.21.15.50:0/591942436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2d4074b20 0x7fc2d415cf90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009bc0 tx=0x7fc2cc01cba0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.604+0000 7fc2d9000640 1 -- 172.21.15.50:0/591942436 shutdown_connections 2024-09-21T05:56:44.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.604+0000 7fc2d9000640 1 --2- 172.21.15.50:0/591942436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2d4074b20 0x7fc2d415cf90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.604+0000 7fc2d9000640 1 --2- 172.21.15.50:0/591942436 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2d4157030 0x7fc2d40b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.604+0000 7fc2d9000640 1 -- 172.21.15.50:0/591942436 >> 172.21.15.50:0/591942436 conn(0x7fc2d40708c0 msgr2=0x7fc2d4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:44.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.604+0000 7fc2d9000640 1 -- 172.21.15.50:0/591942436 shutdown_connections 2024-09-21T05:56:44.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.604+0000 7fc2d9000640 1 -- 172.21.15.50:0/591942436 wait complete. 2024-09-21T05:56:44.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.604+0000 7fc2d9000640 1 Processor -- start 2024-09-21T05:56:44.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.604+0000 7fc2d9000640 1 -- start start 2024-09-21T05:56:44.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2d9000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2d4157030 0x7fc2d41f9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:44.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2d9000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2d4160d50 0x7fc2d41fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:44.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2d9000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2d414ce00 0x7fc2d41551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:44.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2d9000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d416a7d0 con 0x7fc2d4160d50 2024-09-21T05:56:44.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2d9000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d416a650 con 0x7fc2d414ce00 2024-09-21T05:56:44.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2d9000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d416a950 con 0x7fc2d4157030 2024-09-21T05:56:44.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2cbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2d4160d50 0x7fc2d41fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2d1e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2d4157030 0x7fc2d41f9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2cbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2d4160d50 0x7fc2d41fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48780/0 (socket says 172.21.15.50:48780) 2024-09-21T05:56:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2cbe00640 1 -- 172.21.15.50:0/1210068806 learned_addr learned my addr 172.21.15.50:0/1210068806 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2d2800640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2d414ce00 0x7fc2d41551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2cbe00640 1 -- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2d4157030 msgr2=0x7fc2d41f9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2cbe00640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2d4157030 0x7fc2d41f9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2cbe00640 1 -- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2d414ce00 msgr2=0x7fc2d41551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2cbe00640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2d414ce00 0x7fc2d41551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2cbe00640 1 -- 172.21.15.50:0/1210068806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d41fd930 con 0x7fc2d4160d50 2024-09-21T05:56:44.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2cbe00640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2d4160d50 0x7fc2d41fd3f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00c3c0 tx=0x7fc2bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:44.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2ca000640 1 -- 172.21.15.50:0/1210068806 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc011790 con 0x7fc2d4160d50 2024-09-21T05:56:44.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2ca000640 1 -- 172.21.15.50:0/1210068806 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2bc017070 con 0x7fc2d4160d50 2024-09-21T05:56:44.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2ca000640 1 -- 172.21.15.50:0/1210068806 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc011cc0 con 0x7fc2d4160d50 2024-09-21T05:56:44.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2d9000640 1 -- 172.21.15.50:0/1210068806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d415c3e0 con 0x7fc2d4160d50 2024-09-21T05:56:44.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2d9000640 1 -- 172.21.15.50:0/1210068806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d41f9500 con 0x7fc2d4160d50 2024-09-21T05:56:44.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.608+0000 7fc2d9000640 1 -- 172.21.15.50:0/1210068806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d415cf90 con 0x7fc2d4160d50 2024-09-21T05:56:44.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.612+0000 7fc2ca000640 1 -- 172.21.15.50:0/1210068806 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc2bc01c460 con 0x7fc2d4160d50 2024-09-21T05:56:44.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.612+0000 7fc2ca000640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc2b0073200 0x7fc2b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:44.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.612+0000 7fc2ca000640 1 -- 172.21.15.50:0/1210068806 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc2bc093ec0 con 0x7fc2d4160d50 2024-09-21T05:56:44.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.612+0000 7fc2d1e00640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc2b0073200 0x7fc2b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:44.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.612+0000 7fc2d1e00640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc2b0073200 0x7fc2b0075710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009600 tx=0x7fc2c40091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:44.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.616+0000 7fc2ca000640 1 -- 172.21.15.50:0/1210068806 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc2d41fd930 con 0x7fc2d4160d50 2024-09-21T05:56:44.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.832+0000 7fc2d9000640 1 -- 172.21.15.50:0/1210068806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc2d415c140 con 0x7fc2d4160d50 2024-09-21T05:56:44.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.832+0000 7fc2ca000640 1 -- 172.21.15.50:0/1210068806 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc2bc05e7f0 con 0x7fc2d4160d50 2024-09-21T05:56:44.835 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:44.835 INFO:teuthology.orchestra.run.smithi050.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T05:56:44.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.836+0000 7fc2d9000640 1 -- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc2b0073200 msgr2=0x7fc2b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:44.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.836+0000 7fc2d9000640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc2b0073200 0x7fc2b0075710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009600 tx=0x7fc2c40091e0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.836+0000 7fc2d9000640 1 -- 172.21.15.50:0/1210068806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2d4160d50 msgr2=0x7fc2d41fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:44.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.836+0000 7fc2d9000640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2d4160d50 0x7fc2d41fd3f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00c3c0 tx=0x7fc2bc00c880 comp rx=0 tx=0).stop 2024-09-21T05:56:44.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.836+0000 7fc2d9000640 1 -- 172.21.15.50:0/1210068806 shutdown_connections 2024-09-21T05:56:44.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.836+0000 7fc2d9000640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2d414ce00 0x7fc2d41551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.836+0000 7fc2d9000640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc2b0073200 0x7fc2b0075710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.836+0000 7fc2d9000640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2d4160d50 0x7fc2d41fd3f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.836+0000 7fc2d9000640 1 --2- 172.21.15.50:0/1210068806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2d4157030 0x7fc2d41f9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:44.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.836+0000 7fc2d9000640 1 -- 172.21.15.50:0/1210068806 >> 172.21.15.50:0/1210068806 conn(0x7fc2d40708c0 msgr2=0x7fc2d40d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:44.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.836+0000 7fc2d9000640 1 -- 172.21.15.50:0/1210068806 shutdown_connections 2024-09-21T05:56:44.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:44.836+0000 7fc2d9000640 1 -- 172.21.15.50:0/1210068806 wait complete. 2024-09-21T05:56:44.857 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T05:56:44.858 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:56:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.012+0000 7f859c400640 1 -- 172.21.15.50:0/430871694 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85940b5a80 msgr2=0x7f8594074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.012+0000 7f859c400640 1 --2- 172.21.15.50:0/430871694 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85940b5a80 0x7f8594074410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8580009870 tx=0x7f858001c860 comp rx=0 tx=0).stop 2024-09-21T05:56:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.012+0000 7f859c400640 1 -- 172.21.15.50:0/430871694 shutdown_connections 2024-09-21T05:56:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.012+0000 7f859c400640 1 --2- 172.21.15.50:0/430871694 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85940b5a80 0x7f8594074410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.012+0000 7f859c400640 1 -- 172.21.15.50:0/430871694 >> 172.21.15.50:0/430871694 conn(0x7f85940708c0 msgr2=0x7f8594070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.012+0000 7f859c400640 1 -- 172.21.15.50:0/430871694 shutdown_connections 2024-09-21T05:56:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.012+0000 7f859c400640 1 -- 172.21.15.50:0/430871694 wait complete. 2024-09-21T05:56:45.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f859c400640 1 Processor -- start 2024-09-21T05:56:45.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f859c400640 1 -- start start 2024-09-21T05:56:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f859c400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8594074950 0x7f859414f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f859c400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85940b5a80 0x7f859414a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f859c400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8594157030 0x7f85940b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f859c400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f859416b6a0 con 0x7f8594074950 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f859c400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f859416b520 con 0x7f8594157030 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f859c400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f859416b820 con 0x7f85940b5a80 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f8599a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8594074950 0x7f859414f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f8599a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8594074950 0x7f859414f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48796/0 (socket says 172.21.15.50:48796) 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f8599a00640 1 -- 172.21.15.50:0/1640267682 learned_addr learned my addr 172.21.15.50:0/1640267682 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f859a400640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8594157030 0x7f85940b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f8599000640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85940b5a80 0x7f859414a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f8599a00640 1 -- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85940b5a80 msgr2=0x7f859414a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f8599a00640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85940b5a80 0x7f859414a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f8599a00640 1 -- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8594157030 msgr2=0x7f85940b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f8599a00640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8594157030 0x7f85940b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f8599a00640 1 -- 172.21.15.50:0/1640267682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85940b2400 con 0x7f8594074950 2024-09-21T05:56:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f8599a00640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8594074950 0x7f859414f3a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f858800c760 tx=0x7f858800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:45.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.016+0000 7f8591600640 1 -- 172.21.15.50:0/1640267682 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588016020 con 0x7f8594074950 2024-09-21T05:56:45.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.020+0000 7f8591600640 1 -- 172.21.15.50:0/1640267682 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f858800a950 con 0x7f8594074950 2024-09-21T05:56:45.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.020+0000 7f859c400640 1 -- 172.21.15.50:0/1640267682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85940b26e0 con 0x7f8594074950 2024-09-21T05:56:45.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.020+0000 7f8591600640 1 -- 172.21.15.50:0/1640267682 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858800ada0 con 0x7f8594074950 2024-09-21T05:56:45.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.020+0000 7f859c400640 1 -- 172.21.15.50:0/1640267682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85940b2ca0 con 0x7f8594074950 2024-09-21T05:56:45.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.020+0000 7f859c400640 1 -- 172.21.15.50:0/1640267682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8594074410 con 0x7f8594074950 2024-09-21T05:56:45.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.020+0000 7f8591600640 1 -- 172.21.15.50:0/1640267682 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8588019020 con 0x7f8594074950 2024-09-21T05:56:45.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.020+0000 7f8591600640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8574073200 0x7f8574075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.020+0000 7f8591600640 1 -- 172.21.15.50:0/1640267682 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8588094320 con 0x7f8594074950 2024-09-21T05:56:45.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.020+0000 7f8599000640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8574073200 0x7f8574075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.024+0000 7f8599000640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8574073200 0x7f8574075710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85800af000 tx=0x7f858001b0a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:45.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.024+0000 7f8591600640 1 -- 172.21.15.50:0/1640267682 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8588020020 con 0x7f8594074950 2024-09-21T05:56:45.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.240+0000 7f859c400640 1 -- 172.21.15.50:0/1640267682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85940d2260 con 0x7f8594074950 2024-09-21T05:56:45.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.240+0000 7f8591600640 1 -- 172.21.15.50:0/1640267682 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5211 (secure 0 0 0) 0x7f8594074d60 con 0x7f8594074950 2024-09-21T05:56:45.243 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:45.244 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":8,"btime":"2024-09-21T05:56:32:796889+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3883511106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3883511106},{"type":"v1","addr":"172.21.15.88:6836","nonce":3883511106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/606533049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":606533049},{"type":"v1","addr":"172.21.15.50:6833","nonce":606533049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/183287131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":183287131},{"type":"v1","addr":"172.21.15.50:6835","nonce":183287131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:56:29.197481+0000","modified":"2024-09-21T05:56:32.215215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4371":{"gid":4371,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/1479043500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1479043500},{"type":"v1","addr":"172.21.15.88:6837","nonce":1479043500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4371,"qdb_cluster":[4371]},"id":1}]} 2024-09-21T05:56:45.244 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 8 2024-09-21T05:56:45.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.244+0000 7f859c400640 1 -- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8574073200 msgr2=0x7f8574075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.244+0000 7f859c400640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8574073200 0x7f8574075710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85800af000 tx=0x7f858001b0a0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.244+0000 7f859c400640 1 -- 172.21.15.50:0/1640267682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8594074950 msgr2=0x7f859414f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.244+0000 7f859c400640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8594074950 0x7f859414f3a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f858800c760 tx=0x7f858800cc20 comp rx=0 tx=0).stop 2024-09-21T05:56:45.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.244+0000 7f859c400640 1 -- 172.21.15.50:0/1640267682 shutdown_connections 2024-09-21T05:56:45.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.244+0000 7f859c400640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8594157030 0x7f85940b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.244+0000 7f859c400640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85940b5a80 0x7f859414a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.244+0000 7f859c400640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8574073200 0x7f8574075710 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.244+0000 7f859c400640 1 --2- 172.21.15.50:0/1640267682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8594074950 0x7f859414f3a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.244+0000 7f859c400640 1 -- 172.21.15.50:0/1640267682 >> 172.21.15.50:0/1640267682 conn(0x7f85940708c0 msgr2=0x7f85941506f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:45.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.244+0000 7f859c400640 1 -- 172.21.15.50:0/1640267682 shutdown_connections 2024-09-21T05:56:45.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.244+0000 7f859c400640 1 -- 172.21.15.50:0/1640267682 wait complete. 2024-09-21T05:56:45.266 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:56:45.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.424+0000 7fbd51e00640 1 -- 172.21.15.50:0/793848669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd300052f0 msgr2=0x7fbd3000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.424+0000 7fbd51e00640 1 --2- 172.21.15.50:0/793848669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd300052f0 0x7fbd3000b9d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009b30 tx=0x7fbd4401d270 comp rx=0 tx=0).stop 2024-09-21T05:56:45.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.424+0000 7fbd51e00640 1 -- 172.21.15.50:0/793848669 shutdown_connections 2024-09-21T05:56:45.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.424+0000 7fbd51e00640 1 --2- 172.21.15.50:0/793848669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd300052f0 0x7fbd3000b9d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.424+0000 7fbd51e00640 1 -- 172.21.15.50:0/793848669 >> 172.21.15.50:0/793848669 conn(0x7fbd4c0b35f0 msgr2=0x7fbd4c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:45.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.424+0000 7fbd51e00640 1 -- 172.21.15.50:0/793848669 shutdown_connections 2024-09-21T05:56:45.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.424+0000 7fbd51e00640 1 -- 172.21.15.50:0/793848669 wait complete. 2024-09-21T05:56:45.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.424+0000 7fbd51e00640 1 Processor -- start 2024-09-21T05:56:45.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.424+0000 7fbd51e00640 1 -- start start 2024-09-21T05:56:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd51e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd300052f0 0x7fbd4c1f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd51e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd4c1f6b30 0x7fbd4c1f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd51e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbd4c14dce0 0x7fbd4c14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd51e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c15d780 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd51e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c15d600 con 0x7fbd300052f0 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd51e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4c15d900 con 0x7fbd4c14dce0 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd42000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd4c1f6b30 0x7fbd4c1f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd42000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd4c1f6b30 0x7fbd4c1f6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48816/0 (socket says 172.21.15.50:48816) 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd42000640 1 -- 172.21.15.50:0/2200324702 learned_addr learned my addr 172.21.15.50:0/2200324702 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd4aa00640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd300052f0 0x7fbd4c1f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd4b400640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbd4c14dce0 0x7fbd4c14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd42000640 1 -- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbd4c14dce0 msgr2=0x7fbd4c14e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd42000640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbd4c14dce0 0x7fbd4c14e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd42000640 1 -- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd300052f0 msgr2=0x7fbd4c1f65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd42000640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd300052f0 0x7fbd4c1f65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd42000640 1 -- 172.21.15.50:0/2200324702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c13c4b0 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd4b400640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbd4c14dce0 0x7fbd4c14e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd4aa00640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd300052f0 0x7fbd4c1f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd42000640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd4c1f6b30 0x7fbd4c1f6f40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbd340016e0 tx=0x7fbd340017d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd43e00640 1 -- 172.21.15.50:0/2200324702 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34004240 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd43e00640 1 -- 172.21.15.50:0/2200324702 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd34017020 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd51e00640 1 -- 172.21.15.50:0/2200324702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c13c790 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd43e00640 1 -- 172.21.15.50:0/2200324702 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34003600 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd51e00640 1 -- 172.21.15.50:0/2200324702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c1f3d10 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.428+0000 7fbd51e00640 1 -- 172.21.15.50:0/2200324702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4c068e40 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.432+0000 7fbd43e00640 1 -- 172.21.15.50:0/2200324702 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbd3400d070 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.432+0000 7fbd43e00640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbd28073200 0x7fbd28075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.432+0000 7fbd43e00640 1 -- 172.21.15.50:0/2200324702 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbd340977b0 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.432+0000 7fbd4aa00640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbd28073200 0x7fbd28075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.432+0000 7fbd4aa00640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbd28073200 0x7fbd28075710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c006fd0 tx=0x7fbd3c004b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:45.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.436+0000 7fbd43e00640 1 -- 172.21.15.50:0/2200324702 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbd34021020 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.648+0000 7fbd51e00640 1 -- 172.21.15.50:0/2200324702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd4c13c240 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.648+0000 7fbd43e00640 1 -- 172.21.15.50:0/2200324702 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5211 (secure 0 0 0) 0x7fbd340628c0 con 0x7fbd4c1f6b30 2024-09-21T05:56:45.651 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:45.651 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":8,"btime":"2024-09-21T05:56:32:796889+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3883511106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3883511106},{"type":"v1","addr":"172.21.15.88:6836","nonce":3883511106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/606533049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":606533049},{"type":"v1","addr":"172.21.15.50:6833","nonce":606533049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/183287131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":183287131},{"type":"v1","addr":"172.21.15.50:6835","nonce":183287131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:56:29.197481+0000","modified":"2024-09-21T05:56:32.215215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4371":{"gid":4371,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/1479043500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1479043500},{"type":"v1","addr":"172.21.15.88:6837","nonce":1479043500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4371,"qdb_cluster":[4371]},"id":1}]} 2024-09-21T05:56:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 8 2024-09-21T05:56:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.652+0000 7fbd51e00640 1 -- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbd28073200 msgr2=0x7fbd28075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.652+0000 7fbd51e00640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbd28073200 0x7fbd28075710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c006fd0 tx=0x7fbd3c004b40 comp rx=0 tx=0).stop 2024-09-21T05:56:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.652+0000 7fbd51e00640 1 -- 172.21.15.50:0/2200324702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd4c1f6b30 msgr2=0x7fbd4c1f6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.652+0000 7fbd51e00640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd4c1f6b30 0x7fbd4c1f6f40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbd340016e0 tx=0x7fbd340017d0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.652+0000 7fbd51e00640 1 -- 172.21.15.50:0/2200324702 shutdown_connections 2024-09-21T05:56:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.652+0000 7fbd51e00640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbd4c14dce0 0x7fbd4c14e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.652+0000 7fbd51e00640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbd28073200 0x7fbd28075710 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.652+0000 7fbd51e00640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd4c1f6b30 0x7fbd4c1f6f40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.652+0000 7fbd51e00640 1 --2- 172.21.15.50:0/2200324702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd300052f0 0x7fbd4c1f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.652+0000 7fbd51e00640 1 -- 172.21.15.50:0/2200324702 >> 172.21.15.50:0/2200324702 conn(0x7fbd4c0b35f0 msgr2=0x7fbd4c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.652+0000 7fbd51e00640 1 -- 172.21.15.50:0/2200324702 shutdown_connections 2024-09-21T05:56:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.652+0000 7fbd51e00640 1 -- 172.21.15.50:0/2200324702 wait complete. 2024-09-21T05:56:45.674 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:56:45.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.832+0000 7ff99e200640 1 -- 172.21.15.50:0/3731918722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9980cefa0 msgr2=0x7ff9980cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.832+0000 7ff99e200640 1 --2- 172.21.15.50:0/3731918722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9980cefa0 0x7ff9980cf3b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff980009bc0 tx=0x7ff98001cba0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.832+0000 7ff99e200640 1 -- 172.21.15.50:0/3731918722 shutdown_connections 2024-09-21T05:56:45.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.832+0000 7ff99e200640 1 --2- 172.21.15.50:0/3731918722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9980cefa0 0x7ff9980cf3b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.832+0000 7ff99e200640 1 -- 172.21.15.50:0/3731918722 >> 172.21.15.50:0/3731918722 conn(0x7ff9980b2040 msgr2=0x7ff9980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:45.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.832+0000 7ff99e200640 1 -- 172.21.15.50:0/3731918722 shutdown_connections 2024-09-21T05:56:45.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.832+0000 7ff99e200640 1 -- 172.21.15.50:0/3731918722 wait complete. 2024-09-21T05:56:45.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.832+0000 7ff99e200640 1 Processor -- start 2024-09-21T05:56:45.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.832+0000 7ff99e200640 1 -- start start 2024-09-21T05:56:45.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff99e200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9980749a0 0x7ff998159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff99e200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff9980b3340 0x7ff9980ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff99e200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9980cefa0 0x7ff9980ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff99e200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff99816b350 con 0x7ff9980cefa0 2024-09-21T05:56:45.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff99e200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff99816b1d0 con 0x7ff9980749a0 2024-09-21T05:56:45.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff99e200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff99816b4d0 con 0x7ff9980b3340 2024-09-21T05:56:45.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff997e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9980cefa0 0x7ff9980ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff997e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9980cefa0 0x7ff9980ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48826/0 (socket says 172.21.15.50:48826) 2024-09-21T05:56:45.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff997e00640 1 -- 172.21.15.50:0/591036399 learned_addr learned my addr 172.21.15.50:0/591036399 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:45.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff997400640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9980749a0 0x7ff998159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff997e00640 1 -- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff9980b3340 msgr2=0x7ff9980ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff98ea00640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff9980b3340 0x7ff9980ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff997e00640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff9980b3340 0x7ff9980ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff997e00640 1 -- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9980749a0 msgr2=0x7ff998159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff997e00640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9980749a0 0x7ff998159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff997e00640 1 -- 172.21.15.50:0/591036399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff998143f20 con 0x7ff9980cefa0 2024-09-21T05:56:45.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff98ea00640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff9980b3340 0x7ff9980ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:45.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff997e00640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9980cefa0 0x7ff9980ce730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff99000c3f0 tx=0x7ff99000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:45.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff98fe00640 1 -- 172.21.15.50:0/591036399 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990017070 con 0x7ff9980cefa0 2024-09-21T05:56:45.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff99e200640 1 -- 172.21.15.50:0/591036399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff998144200 con 0x7ff9980cefa0 2024-09-21T05:56:45.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff98fe00640 1 -- 172.21.15.50:0/591036399 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9900053c0 con 0x7ff9980cefa0 2024-09-21T05:56:45.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff98fe00640 1 -- 172.21.15.50:0/591036399 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990003860 con 0x7ff9980cefa0 2024-09-21T05:56:45.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff99e200640 1 -- 172.21.15.50:0/591036399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99815a1a0 con 0x7ff9980cefa0 2024-09-21T05:56:45.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.836+0000 7ff99e200640 1 -- 172.21.15.50:0/591036399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9980cf3b0 con 0x7ff9980cefa0 2024-09-21T05:56:45.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.840+0000 7ff98fe00640 1 -- 172.21.15.50:0/591036399 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff99001a070 con 0x7ff9980cefa0 2024-09-21T05:56:45.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.840+0000 7ff98fe00640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff974073230 0x7ff974075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.840+0000 7ff98fe00640 1 -- 172.21.15.50:0/591036399 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff990093550 con 0x7ff9980cefa0 2024-09-21T05:56:45.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.840+0000 7ff997400640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff974073230 0x7ff974075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.840+0000 7ff997400640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff974073230 0x7ff974075740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff9880046c0 tx=0x7ff988007cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:45.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:45.844+0000 7ff98fe00640 1 -- 172.21.15.50:0/591036399 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff99005dec0 con 0x7ff9980cefa0 2024-09-21T05:56:46.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.024+0000 7ff99e200640 1 -- 172.21.15.50:0/591036399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff99813fec0 con 0x7ff9980cefa0 2024-09-21T05:56:46.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.024+0000 7ff98fe00640 1 -- 172.21.15.50:0/591036399 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16617 (secure 0 0 0) 0x7ff9900628f0 con 0x7ff9980cefa0 2024-09-21T05:56:46.027 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:46.028 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":22,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:30.775110+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:56:27.052826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:56:28.162376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:46.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.028+0000 7ff99e200640 1 -- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff974073230 msgr2=0x7ff974075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:46.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.028+0000 7ff99e200640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff974073230 0x7ff974075740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff9880046c0 tx=0x7ff988007cd0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.028+0000 7ff99e200640 1 -- 172.21.15.50:0/591036399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9980cefa0 msgr2=0x7ff9980ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:46.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.028+0000 7ff99e200640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9980cefa0 0x7ff9980ce730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff99000c3f0 tx=0x7ff99000c8b0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.028+0000 7ff99e200640 1 -- 172.21.15.50:0/591036399 shutdown_connections 2024-09-21T05:56:46.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.028+0000 7ff99e200640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff974073230 0x7ff974075740 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.028+0000 7ff99e200640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9980cefa0 0x7ff9980ce730 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.028+0000 7ff99e200640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff9980b3340 0x7ff9980ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.028+0000 7ff99e200640 1 --2- 172.21.15.50:0/591036399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9980749a0 0x7ff998159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.028+0000 7ff99e200640 1 -- 172.21.15.50:0/591036399 >> 172.21.15.50:0/591036399 conn(0x7ff9980b2040 msgr2=0x7ff99813bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:46.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.028+0000 7ff99e200640 1 -- 172.21.15.50:0/591036399 shutdown_connections 2024-09-21T05:56:46.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.028+0000 7ff99e200640 1 -- 172.21.15.50:0/591036399 wait complete. 2024-09-21T05:56:46.051 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:56:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.204+0000 7f73a1c00640 1 -- 172.21.15.50:0/2765226587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c06fa70 msgr2=0x7f739c15f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.204+0000 7f73a1c00640 1 --2- 172.21.15.50:0/2765226587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c06fa70 0x7f739c15f880 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f739c071970 tx=0x7f739401cb50 comp rx=0 tx=0).stop 2024-09-21T05:56:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.204+0000 7f73a1c00640 1 -- 172.21.15.50:0/2765226587 shutdown_connections 2024-09-21T05:56:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.204+0000 7f73a1c00640 1 --2- 172.21.15.50:0/2765226587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c06fa70 0x7f739c15f880 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.204+0000 7f73a1c00640 1 --2- 172.21.15.50:0/2765226587 >> v2:172.21.15.88:3300/0 conn(0x7f739c15c5d0 0x7f739c073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.204+0000 7f73a1c00640 1 -- 172.21.15.50:0/2765226587 >> 172.21.15.50:0/2765226587 conn(0x7f739c0b2c40 msgr2=0x7f739c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.204+0000 7f73a1c00640 1 -- 172.21.15.50:0/2765226587 shutdown_connections 2024-09-21T05:56:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.204+0000 7f73a1c00640 1 -- 172.21.15.50:0/2765226587 wait complete. 2024-09-21T05:56:46.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.208+0000 7f73a1c00640 1 Processor -- start 2024-09-21T05:56:46.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.208+0000 7f73a1c00640 1 -- start start 2024-09-21T05:56:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.208+0000 7f73a1c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f739c06fa70 0x7f739c14d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.208+0000 7f73a1c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074300 0x7f739c150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.208+0000 7f73a1c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f739c15c5d0 0x7f739c14f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.208+0000 7f73a1c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f739c15ff40 con 0x7f739c074300 2024-09-21T05:56:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.208+0000 7f73a1c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f739c15fdc0 con 0x7f739c15c5d0 2024-09-21T05:56:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.208+0000 7f73a1c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f739c168b10 con 0x7f739c06fa70 2024-09-21T05:56:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.208+0000 7f7393e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074300 0x7f739c150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.208+0000 7f7393e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074300 0x7f739c150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48844/0 (socket says 172.21.15.50:48844) 2024-09-21T05:56:46.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.208+0000 7f7393e00640 1 -- 172.21.15.50:0/4081826399 learned_addr learned my addr 172.21.15.50:0/4081826399 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:46.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.208+0000 7f739aa00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f739c06fa70 0x7f739c14d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:46.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f739b400640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f739c15c5d0 0x7f739c14f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:46.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f7393e00640 1 -- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f739c06fa70 msgr2=0x7f739c14d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:46.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f7393e00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f739c06fa70 0x7f739c14d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f7393e00640 1 -- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f739c15c5d0 msgr2=0x7f739c14f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:46.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f7393e00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f739c15c5d0 0x7f739c14f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f7393e00640 1 -- 172.21.15.50:0/4081826399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739c14d630 con 0x7f739c074300 2024-09-21T05:56:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f7393e00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074300 0x7f739c150320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f738400c3a0 tx=0x7f738400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f7392a00640 1 -- 172.21.15.50:0/4081826399 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384017070 con 0x7f739c074300 2024-09-21T05:56:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f7392a00640 1 -- 172.21.15.50:0/4081826399 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7384003600 con 0x7f739c074300 2024-09-21T05:56:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f73a1c00640 1 -- 172.21.15.50:0/4081826399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739c1fb920 con 0x7f739c074300 2024-09-21T05:56:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f7392a00640 1 -- 172.21.15.50:0/4081826399 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384003a70 con 0x7f739c074300 2024-09-21T05:56:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f73a1c00640 1 -- 172.21.15.50:0/4081826399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f739c1fbee0 con 0x7f739c074300 2024-09-21T05:56:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f73a1c00640 1 -- 172.21.15.50:0/4081826399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f739c073dc0 con 0x7f739c074300 2024-09-21T05:56:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.212+0000 7f7392a00640 1 -- 172.21.15.50:0/4081826399 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f738401a070 con 0x7f739c074300 2024-09-21T05:56:46.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.216+0000 7f7392a00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7378073200 0x7f7378075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:46.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.216+0000 7f7392a00640 1 -- 172.21.15.50:0/4081826399 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7384093470 con 0x7f739c074300 2024-09-21T05:56:46.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.216+0000 7f739aa00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7378073200 0x7f7378075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:46.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.216+0000 7f739aa00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7378073200 0x7f7378075710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f738c009b40 tx=0x7f738c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:46.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.220+0000 7f7392a00640 1 -- 172.21.15.50:0/4081826399 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f738405de30 con 0x7f739c074300 2024-09-21T05:56:46.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.432+0000 7f73a1c00640 1 -- 172.21.15.50:0/4081826399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f739c076200 con 0x7f739c074300 2024-09-21T05:56:46.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.432+0000 7f7392a00640 1 -- 172.21.15.50:0/4081826399 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5211 (secure 0 0 0) 0x7f7384062860 con 0x7f739c074300 2024-09-21T05:56:46.435 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:46.436 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":8,"btime":"2024-09-21T05:56:32:796889+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3883511106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3883511106},{"type":"v1","addr":"172.21.15.88:6836","nonce":3883511106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/606533049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":606533049},{"type":"v1","addr":"172.21.15.50:6833","nonce":606533049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/183287131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":183287131},{"type":"v1","addr":"172.21.15.50:6835","nonce":183287131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:56:29.197481+0000","modified":"2024-09-21T05:56:32.215215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4371":{"gid":4371,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/1479043500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1479043500},{"type":"v1","addr":"172.21.15.88:6837","nonce":1479043500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4371,"qdb_cluster":[4371]},"id":1}]} 2024-09-21T05:56:46.436 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 8 2024-09-21T05:56:46.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.436+0000 7f73a1c00640 1 -- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7378073200 msgr2=0x7f7378075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:46.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.436+0000 7f73a1c00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7378073200 0x7f7378075710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f738c009b40 tx=0x7f738c0091e0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.436+0000 7f73a1c00640 1 -- 172.21.15.50:0/4081826399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074300 msgr2=0x7f739c150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:46.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.436+0000 7f73a1c00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074300 0x7f739c150320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f738400c3a0 tx=0x7f738400c860 comp rx=0 tx=0).stop 2024-09-21T05:56:46.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.436+0000 7f73a1c00640 1 -- 172.21.15.50:0/4081826399 shutdown_connections 2024-09-21T05:56:46.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.436+0000 7f73a1c00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f739c15c5d0 0x7f739c14f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.436+0000 7f73a1c00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7378073200 0x7f7378075710 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.436+0000 7f73a1c00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074300 0x7f739c150320 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.436+0000 7f73a1c00640 1 --2- 172.21.15.50:0/4081826399 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f739c06fa70 0x7f739c14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:46.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.436+0000 7f73a1c00640 1 -- 172.21.15.50:0/4081826399 >> 172.21.15.50:0/4081826399 conn(0x7f739c0b2c40 msgr2=0x7f739c15a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:46.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.436+0000 7f73a1c00640 1 -- 172.21.15.50:0/4081826399 shutdown_connections 2024-09-21T05:56:46.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:46.436+0000 7f73a1c00640 1 -- 172.21.15.50:0/4081826399 wait complete. 2024-09-21T05:56:46.672 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:56:46.675 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:56:46.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:56:29.197481+0000', 'modified': '2024-09-21T05:56:32.215215+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4371}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4371': {'gid': 4371, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6837/1479043500', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6835', 'nonce': 1479043500}, {'type': 'v1', 'addr': '172.21.15.88:6837', 'nonce': 1479043500}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4371, 'qdb_cluster': [4371]} 2024-09-21T05:56:46.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:56:46.861 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T05:56:47.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.020+0000 7f9afb400640 1 -- 172.21.15.50:0/3957810074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9af4159130 msgr2=0x7f9af415bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.020+0000 7f9afb400640 1 --2- 172.21.15.50:0/3957810074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9af4159130 0x7f9af415bb90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009900 tx=0x7f9ae801c850 comp rx=0 tx=0).stop 2024-09-21T05:56:47.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.020+0000 7f9afb400640 1 -- 172.21.15.50:0/3957810074 shutdown_connections 2024-09-21T05:56:47.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.020+0000 7f9afb400640 1 --2- 172.21.15.50:0/3957810074 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9af406cdf0 0x7f9af406d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.020+0000 7f9afb400640 1 --2- 172.21.15.50:0/3957810074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9af4159130 0x7f9af415bb90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.020+0000 7f9afb400640 1 -- 172.21.15.50:0/3957810074 >> 172.21.15.50:0/3957810074 conn(0x7f9af40b35f0 msgr2=0x7f9af40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:47.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.020+0000 7f9afb400640 1 -- 172.21.15.50:0/3957810074 shutdown_connections 2024-09-21T05:56:47.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.020+0000 7f9afb400640 1 -- 172.21.15.50:0/3957810074 wait complete. 2024-09-21T05:56:47.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9afb400640 1 Processor -- start 2024-09-21T05:56:47.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9afb400640 1 -- start start 2024-09-21T05:56:47.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9afb400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9af406cdf0 0x7f9af40aa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9afb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9af406fc90 0x7f9af40d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9afb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9af4159130 0x7f9af4165630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9afb400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4168dd0 con 0x7f9af4159130 2024-09-21T05:56:47.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9afb400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4168c50 con 0x7f9af406fc90 2024-09-21T05:56:47.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9afb400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9af4168f50 con 0x7f9af406cdf0 2024-09-21T05:56:47.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9af4159130 0x7f9af4165630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9af4159130 0x7f9af4165630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54188/0 (socket says 172.21.15.50:54188) 2024-09-21T05:56:47.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af9400640 1 -- 172.21.15.50:0/720533323 learned_addr learned my addr 172.21.15.50:0/720533323 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:47.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af8a00640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9af406cdf0 0x7f9af40aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af3e00640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9af406fc90 0x7f9af40d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af9400640 1 -- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9af406cdf0 msgr2=0x7f9af40aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af9400640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9af406cdf0 0x7f9af40aa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af9400640 1 -- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9af406fc90 msgr2=0x7f9af40d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af9400640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9af406fc90 0x7f9af40d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af9400640 1 -- 172.21.15.50:0/720533323 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af41f9f80 con 0x7f9af4159130 2024-09-21T05:56:47.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af9400640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9af4159130 0x7f9af4165630 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00c710 tx=0x7f9aec00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:47.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af0c00640 1 -- 172.21.15.50:0/720533323 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec003b80 con 0x7f9af4159130 2024-09-21T05:56:47.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af0c00640 1 -- 172.21.15.50:0/720533323 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9aec003d20 con 0x7f9af4159130 2024-09-21T05:56:47.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9afb400640 1 -- 172.21.15.50:0/720533323 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af41f91c0 con 0x7f9af4159130 2024-09-21T05:56:47.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9af0c00640 1 -- 172.21.15.50:0/720533323 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00aab0 con 0x7f9af4159130 2024-09-21T05:56:47.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.024+0000 7f9afb400640 1 -- 172.21.15.50:0/720533323 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af41f9780 con 0x7f9af4159130 2024-09-21T05:56:47.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.028+0000 7f9af0c00640 1 -- 172.21.15.50:0/720533323 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9aec00ac50 con 0x7f9af4159130 2024-09-21T05:56:47.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af0c00640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ad4073200 0x7f9ad4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af0c00640 1 -- 172.21.15.50:0/720533323 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9aec093e60 con 0x7f9af4159130 2024-09-21T05:56:47.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 Processor -- start 2024-09-21T05:56:47.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 -- start start 2024-09-21T05:56:47.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af8a00640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ad4073200 0x7f9ad4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af8a00640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ad4073200 0x7f9ad4075710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80af000 tx=0x7f9ae801b090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:47.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 0x5619f5a396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58e8950 0x5619f5a39be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 0x5619f5a3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5619f590e410 con 0x5619f58d1ca0 2024-09-21T05:56:47.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5619f584bb40 con 0x5619f54f8460 2024-09-21T05:56:47.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x5619f583af70 con 0x5619f58e8950 2024-09-21T05:56:47.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 0x5619f5a3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 0x5619f5a3a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54202/0 (socket says 172.21.15.50:54202) 2024-09-21T05:56:47.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af9400640 1 -- 172.21.15.50:0/592290006 learned_addr learned my addr 172.21.15.50:0/592290006 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:47.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af9400640 1 -- 172.21.15.50:0/592290006 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58e8950 msgr2=0x5619f5a39be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:47.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af9400640 1 --2- 172.21.15.50:0/592290006 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58e8950 0x5619f5a39be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af9400640 1 -- 172.21.15.50:0/592290006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 msgr2=0x5619f5a396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:47.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af9400640 1 --2- 172.21.15.50:0/592290006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 0x5619f5a396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af9400640 1 -- 172.21.15.50:0/592290006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619f58025b0 con 0x5619f58d1ca0 2024-09-21T05:56:47.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9af9400640 1 --2- 172.21.15.50:0/592290006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 0x5619f5a3a120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00c6e0 tx=0x7f9aec05e210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397908928c603019 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:47.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9ae5200640 1 -- 172.21.15.50:0/592290006 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00ce20 con 0x5619f58d1ca0 2024-09-21T05:56:47.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9ae5200640 1 -- 172.21.15.50:0/592290006 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9aec05e8c0 con 0x5619f58d1ca0 2024-09-21T05:56:47.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9ae5200640 1 -- 172.21.15.50:0/592290006 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec05d350 con 0x5619f58d1ca0 2024-09-21T05:56:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 -- 172.21.15.50:0/592290006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 msgr2=0x5619f5a3a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 --2- 172.21.15.50:0/592290006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 0x5619f5a3a120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00c6e0 tx=0x7f9aec05e210 comp rx=0 tx=0).stop 2024-09-21T05:56:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 -- 172.21.15.50:0/592290006 shutdown_connections 2024-09-21T05:56:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 --2- 172.21.15.50:0/592290006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 0x5619f5a3a120 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 --2- 172.21.15.50:0/592290006 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58e8950 0x5619f5a39be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 --2- 172.21.15.50:0/592290006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 0x5619f5a396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 -- 172.21.15.50:0/592290006 >> 172.21.15.50:0/592290006 conn(0x5619f59bed90 msgr2=0x5619f59ee690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 -- 172.21.15.50:0/592290006 shutdown_connections 2024-09-21T05:56:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.032+0000 7f9b0296a000 1 -- 172.21.15.50:0/592290006 wait complete. 2024-09-21T05:56:47.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9b0296a000 1 -- start start 2024-09-21T05:56:47.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9b0296a000 1 Processor -- start 2024-09-21T05:56:47.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9b0296a000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 0x5619f5a40920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9b0296a000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58d1ca0 0x5619f5a40e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9b0296a000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 0x5619f5a413a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9b0296a000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5619f57d55f0 con 0x5619f58e8950 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9b0296a000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5619f583ed80 con 0x5619f54f8460 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9b0296a000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x5619f57e1520 con 0x5619f58d1ca0 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9b0296a000 10 client.0 _ll_register_callbacks cb 0x5619f57e5830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9af9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 0x5619f5a413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9af9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 0x5619f5a413a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54204/0 (socket says 172.21.15.50:54204) 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9af9400640 1 -- 172.21.15.50:0/34532117 learned_addr learned my addr 172.21.15.50:0/34532117 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9af8a00640 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 0x5619f5a40920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9af3e00640 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58d1ca0 0x5619f5a40e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9af9400640 1 -- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58d1ca0 msgr2=0x5619f5a40e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9af9400640 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58d1ca0 0x5619f5a40e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9af9400640 1 -- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 msgr2=0x5619f5a40920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9af9400640 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 0x5619f5a40920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9af9400640 1 -- 172.21.15.50:0/34532117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619f57d9190 con 0x5619f58e8950 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9af3e00640 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58d1ca0 0x5619f5a40e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9af9400640 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 0x5619f5a413a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9aec05e570 tx=0x7f9aec05ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e6dbff9f520169 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9ad9600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9ad9600640 1 -- 172.21.15.50:0/34532117 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0632f0 con 0x5619f58e8950 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9ad9600640 1 -- 172.21.15.50:0/34532117 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9aec05fb40 con 0x5619f58e8950 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.036+0000 7f9b0296a000 1 -- 172.21.15.50:0/34532117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619f57d3110 con 0x5619f58e8950 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9ad9600640 1 -- 172.21.15.50:0/34532117 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f9aec07d050 con 0x5619f58e8950 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 10 client.4674 fetch_fsmap learned FSMap version 8 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 -- 172.21.15.50:0/34532117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5619f58bff10 con 0x5619f58e8950 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9ad9600640 1 -- 172.21.15.50:0/34532117 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5619f57d3110 con 0x5619f58e8950 2024-09-21T05:56:47.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9ad9600640 1 -- 172.21.15.50:0/34532117 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f9aec05db20 con 0x5619f58e8950 2024-09-21T05:56:47.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 10 client.4674 fetch_fsmap finished waiting for FSMap version 8 2024-09-21T05:56:47.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 10 client.4674 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T05:56:47.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 20 client.4674 populate_metadata read hostname 'smithi050' 2024-09-21T05:56:47.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:6835/1479043500,v1:172.21.15.88:6837/1479043500] conn(0x5619f54dd7d0 0x5619f5a4ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 4 client.4674 mds_command: new command op to 4371 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:56:47.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 -- 172.21.15.50:0/34532117 --> [v2:172.21.15.88:6835/1479043500,v1:172.21.15.88:6837/1479043500] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619f584da90 con 0x5619f54dd7d0 2024-09-21T05:56:47.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9af8a00640 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:6835/1479043500,v1:172.21.15.88:6837/1479043500] conn(0x5619f54dd7d0 0x5619f5a4ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9af8a00640 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:6835/1479043500,v1:172.21.15.88:6837/1479043500] conn(0x5619f54dd7d0 0x5619f5a4ad00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94d28f2cc459ea1 server_cookie=b4b4a3be2c045578 in_seq=0 out_seq=0 2024-09-21T05:56:47.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9ad9600640 10 client.4674 ms_handle_connect on v2:172.21.15.88:6835/1479043500 2024-09-21T05:56:47.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9ad9600640 1 -- 172.21.15.50:0/34532117 <== mds.0 v2:172.21.15.88:6835/1479043500 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5619f584da90 con 0x5619f54dd7d0 2024-09-21T05:56:47.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9ad9600640 10 client.4674 handle_command_reply: tid=0 2024-09-21T05:56:47.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 client.4674 shutdown 2024-09-21T05:56:47.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 -- 172.21.15.50:0/34532117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 msgr2=0x5619f5a413a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 0x5619f5a413a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9aec05e570 tx=0x7f9aec05ae40 comp rx=0 tx=0).stop 2024-09-21T05:56:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 -- 172.21.15.50:0/34532117 shutdown_connections 2024-09-21T05:56:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:6835/1479043500,v1:172.21.15.88:6837/1479043500] conn(0x5619f54dd7d0 0x5619f5a4ad00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 0x5619f5a413a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58d1ca0 0x5619f5a40e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 --2- 172.21.15.50:0/34532117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 0x5619f5a40920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9ad9600640 0 client.4674 ms_handle_reset on v2:172.21.15.88:6835/1479043500 2024-09-21T05:56:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 -- 172.21.15.50:0/34532117 >> 172.21.15.50:0/34532117 conn(0x5619f59bed90 msgr2=0x5619f59f5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 -- 172.21.15.50:0/34532117 shutdown_connections 2024-09-21T05:56:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 1 -- 172.21.15.50:0/34532117 wait complete. 2024-09-21T05:56:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.040+0000 7f9b0296a000 20 client.4674 trim_cache size 0 max 16384 2024-09-21T05:56:47.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.060+0000 7f9b0296a000 1 Processor -- start 2024-09-21T05:56:47.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.060+0000 7f9b0296a000 1 -- start start 2024-09-21T05:56:47.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f54f8460 0x5619f5aab620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 0x5619f5aabb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f58e8950 0x5619f5aac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4205870 con 0x5619f58d1ca0 2024-09-21T05:56:47.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4204950 con 0x5619f58e8950 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9af415abe0 con 0x5619f54f8460 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 0x5619f5aabb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 0x5619f5aabb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54210/0 (socket says 172.21.15.50:54210) 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af3e00640 1 -- 172.21.15.50:0/3894219102 learned_addr learned my addr 172.21.15.50:0/3894219102 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af9400640 1 --2- 172.21.15.50:0/3894219102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f58e8950 0x5619f5aac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af3e00640 1 -- 172.21.15.50:0/3894219102 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f54f8460 msgr2=0x5619f5aab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af8a00640 1 --2- 172.21.15.50:0/3894219102 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f54f8460 0x5619f5aab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af3e00640 1 --2- 172.21.15.50:0/3894219102 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f54f8460 0x5619f5aab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af3e00640 1 -- 172.21.15.50:0/3894219102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f58e8950 msgr2=0x5619f5aac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af3e00640 1 --2- 172.21.15.50:0/3894219102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f58e8950 0x5619f5aac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af3e00640 1 -- 172.21.15.50:0/3894219102 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619f5896db0 con 0x5619f58d1ca0 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af9400640 1 --2- 172.21.15.50:0/3894219102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f58e8950 0x5619f5aac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:47.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af8a00640 1 --2- 172.21.15.50:0/3894219102 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f54f8460 0x5619f5aab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:47.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9af3e00640 1 --2- 172.21.15.50:0/3894219102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 0x5619f5aabb60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0004710 tx=0x7f9ae0003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0e0d47b6dc1eae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:47.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9adbe00640 1 -- 172.21.15.50:0/3894219102 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000b980 con 0x5619f58d1ca0 2024-09-21T05:56:47.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9adbe00640 1 -- 172.21.15.50:0/3894219102 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ae0016070 con 0x5619f58d1ca0 2024-09-21T05:56:47.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9adbe00640 1 -- 172.21.15.50:0/3894219102 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae001b470 con 0x5619f58d1ca0 2024-09-21T05:56:47.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 -- 172.21.15.50:0/3894219102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 msgr2=0x5619f5aabb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 --2- 172.21.15.50:0/3894219102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 0x5619f5aabb60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0004710 tx=0x7f9ae0003a90 comp rx=0 tx=0).stop 2024-09-21T05:56:47.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 -- 172.21.15.50:0/3894219102 shutdown_connections 2024-09-21T05:56:47.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 --2- 172.21.15.50:0/3894219102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f58e8950 0x5619f5aac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 --2- 172.21.15.50:0/3894219102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58d1ca0 0x5619f5aabb60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 --2- 172.21.15.50:0/3894219102 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f54f8460 0x5619f5aab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 -- 172.21.15.50:0/3894219102 >> 172.21.15.50:0/3894219102 conn(0x5619f59bed90 msgr2=0x5619f5a37620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:47.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 -- 172.21.15.50:0/3894219102 shutdown_connections 2024-09-21T05:56:47.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 -- 172.21.15.50:0/3894219102 wait complete. 2024-09-21T05:56:47.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 -- start start 2024-09-21T05:56:47.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.064+0000 7f9b0296a000 1 Processor -- start 2024-09-21T05:56:47.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 0x5619f5aaf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58d1ca0 0x5619f5aaf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 0x5619f5aafe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5619f57e1520 con 0x5619f58e8950 2024-09-21T05:56:47.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9af415b230 con 0x5619f54f8460 2024-09-21T05:56:47.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x5619f57d55f0 con 0x5619f58d1ca0 2024-09-21T05:56:47.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 10 client.0 _ll_register_callbacks cb 0x7f9aec063070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:47.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 0x5619f5aafe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 0x5619f5aaf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 0x5619f5aafe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54220/0 (socket says 172.21.15.50:54220) 2024-09-21T05:56:47.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af9400640 1 -- 172.21.15.50:0/375976730 learned_addr learned my addr 172.21.15.50:0/375976730 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:47.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af3e00640 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58d1ca0 0x5619f5aaf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af9400640 1 -- 172.21.15.50:0/375976730 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58d1ca0 msgr2=0x5619f5aaf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:47.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af9400640 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58d1ca0 0x5619f5aaf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af9400640 1 -- 172.21.15.50:0/375976730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 msgr2=0x5619f5aaf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:47.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af9400640 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 0x5619f5aaf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af9400640 1 -- 172.21.15.50:0/375976730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619f5801ab0 con 0x5619f58e8950 2024-09-21T05:56:47.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af8a00640 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 0x5619f5aaf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:47.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af9400640 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 0x5619f5aafe10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9aec05e380 tx=0x7f9aec05a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7e9bbb244abb21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:47.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9ad9600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T05:56:47.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9ad9600640 1 -- 172.21.15.50:0/375976730 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec05af70 con 0x5619f58e8950 2024-09-21T05:56:47.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9ad9600640 1 -- 172.21.15.50:0/375976730 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9aec05fd50 con 0x5619f58e8950 2024-09-21T05:56:47.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 1 -- 172.21.15.50:0/375976730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae0016070 con 0x5619f58e8950 2024-09-21T05:56:47.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9ad9600640 1 -- 172.21.15.50:0/375976730 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f9aec06c050 con 0x5619f58e8950 2024-09-21T05:56:47.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 10 client.4680 fetch_fsmap learned FSMap version 8 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 1 -- 172.21.15.50:0/375976730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5619f57e0070 con 0x5619f58e8950 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9ad9600640 1 -- 172.21.15.50:0/375976730 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ae0016070 con 0x5619f58e8950 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9ad9600640 1 -- 172.21.15.50:0/375976730 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f9aec05d590 con 0x5619f58e8950 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 10 client.4680 fetch_fsmap finished waiting for FSMap version 8 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 10 client.4680 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 20 client.4680 populate_metadata read hostname 'smithi050' 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.88:6835/1479043500,v1:172.21.15.88:6837/1479043500] conn(0x5619f54cb830 0x5619f5a6b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 4 client.4680 mds_command: new command op to 4371 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9b0296a000 1 -- 172.21.15.50:0/375976730 --> [v2:172.21.15.88:6835/1479043500,v1:172.21.15.88:6837/1479043500] -- command(tid 0: {"prefix": "status"}) -- 0x5619f5849540 con 0x5619f54cb830 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af8a00640 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.88:6835/1479043500,v1:172.21.15.88:6837/1479043500] conn(0x5619f54cb830 0x5619f5a6b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9af8a00640 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.88:6835/1479043500,v1:172.21.15.88:6837/1479043500] conn(0x5619f54cb830 0x5619f5a6b5e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe8fb5b79f8b8671 server_cookie=e35df084cb3f0f28 in_seq=0 out_seq=0 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.068+0000 7f9ad9600640 10 client.4680 ms_handle_connect on v2:172.21.15.88:6835/1479043500 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9ad9600640 1 -- 172.21.15.50:0/375976730 <== mds.0 v2:172.21.15.88:6835/1479043500 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5619f5849540 con 0x5619f54cb830 2024-09-21T05:56:47.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9ad9600640 10 client.4680 handle_command_reply: tid=0 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9b0296a000 1 client.4680 shutdown 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9b0296a000 1 -- 172.21.15.50:0/375976730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 msgr2=0x5619f5aafe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9b0296a000 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 0x5619f5aafe10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9aec05e380 tx=0x7f9aec05a4c0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9b0296a000 1 -- 172.21.15.50:0/375976730 shutdown_connections 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9b0296a000 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.88:6835/1479043500,v1:172.21.15.88:6837/1479043500] conn(0x5619f54cb830 0x5619f5a6b5e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9b0296a000 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619f58e8950 0x5619f5aafe10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9b0296a000 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x5619f58d1ca0 0x5619f5aaf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9ad9600640 0 client.4680 ms_handle_reset on v2:172.21.15.88:6835/1479043500 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9b0296a000 1 --2- 172.21.15.50:0/375976730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619f54f8460 0x5619f5aaf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9b0296a000 1 -- 172.21.15.50:0/375976730 >> 172.21.15.50:0/375976730 conn(0x5619f59bed90 msgr2=0x5619f5a37fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9b0296a000 1 -- 172.21.15.50:0/375976730 shutdown_connections 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9b0296a000 1 -- 172.21.15.50:0/375976730 wait complete. 2024-09-21T05:56:47.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9b0296a000 20 client.4680 trim_cache size 0 max 16384 2024-09-21T05:56:47.077 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T05:56:47.077 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T05:56:47.077 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-21T05:56:47.077 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4371, 2024-09-21T05:56:47.077 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-21T05:56:47.077 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-21T05:56:47.077 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-21T05:56:47.077 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 17.293806179000001, 2024-09-21T05:56:47.077 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 8, 2024-09-21T05:56:47.077 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 22, 2024-09-21T05:56:47.077 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 22, 2024-09-21T05:56:47.077 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 23.465454057999999 2024-09-21T05:56:47.078 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T05:56:47.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9afb400640 1 -- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ad4073200 msgr2=0x7f9ad4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9afb400640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ad4073200 0x7f9ad4075710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80af000 tx=0x7f9ae801b090 comp rx=0 tx=0).stop 2024-09-21T05:56:47.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9afb400640 1 -- 172.21.15.50:0/720533323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9af4159130 msgr2=0x7f9af4165630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9afb400640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9af4159130 0x7f9af4165630 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00c710 tx=0x7f9aec00cbd0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9afb400640 1 -- 172.21.15.50:0/720533323 shutdown_connections 2024-09-21T05:56:47.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9afb400640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ad4073200 0x7f9ad4075710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9afb400640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9af4159130 0x7f9af4165630 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9afb400640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9af406fc90 0x7f9af40d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9afb400640 1 --2- 172.21.15.50:0/720533323 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9af406cdf0 0x7f9af40aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9afb400640 1 -- 172.21.15.50:0/720533323 >> 172.21.15.50:0/720533323 conn(0x7f9af40b35f0 msgr2=0x7f9af4070880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:47.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9afb400640 1 -- 172.21.15.50:0/720533323 shutdown_connections 2024-09-21T05:56:47.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.072+0000 7f9afb400640 1 -- 172.21.15.50:0/720533323 wait complete. 2024-09-21T05:56:47.094 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T05:56:47.122 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T05:56:47.122 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-21T05:56:47.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:47.123 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T05:56:47.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:47.132 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-09-21T05:56:47.194 INFO:teuthology.orchestra.run.smithi050.stderr:Cannot find device "ceph-brx" 2024-09-21T05:56:47.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:56:47.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:47.197 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T05:56:47.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:47.254 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-09-21T05:56:47.319 INFO:teuthology.orchestra.run.smithi050.stderr:Cannot find device "ceph-brx" 2024-09-21T05:56:47.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:56:47.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:47.321 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T05:56:47.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:47.331 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-09-21T05:56:47.403 INFO:teuthology.orchestra.run.smithi088.stderr:Cannot find device "ceph-brx" 2024-09-21T05:56:47.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:56:47.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:47.406 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T05:56:47.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:47.456 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-09-21T05:56:47.523 INFO:teuthology.orchestra.run.smithi088.stderr:Cannot find device "ceph-brx" 2024-09-21T05:56:47.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:56:47.526 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-21T05:56:47.526 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T05:56:47.526 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:56:47.526 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:56:47.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.680+0000 7f7a0ec00640 1 -- 172.21.15.50:0/1955525064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a080d29c0 msgr2=0x7f7a080d2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.680+0000 7f7a0ec00640 1 --2- 172.21.15.50:0/1955525064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a080d29c0 0x7f7a080d2dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009840 tx=0x7f79f001c820 comp rx=0 tx=0).stop 2024-09-21T05:56:47.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.680+0000 7f7a0ec00640 1 -- 172.21.15.50:0/1955525064 shutdown_connections 2024-09-21T05:56:47.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.680+0000 7f7a0ec00640 1 --2- 172.21.15.50:0/1955525064 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a0815f180 0x7f7a0815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.680+0000 7f7a0ec00640 1 --2- 172.21.15.50:0/1955525064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a080d29c0 0x7f7a080d2dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.680+0000 7f7a0ec00640 1 -- 172.21.15.50:0/1955525064 >> 172.21.15.50:0/1955525064 conn(0x7f7a080708c0 msgr2=0x7f7a08070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:47.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.684+0000 7f7a0ec00640 1 -- 172.21.15.50:0/1955525064 shutdown_connections 2024-09-21T05:56:47.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.684+0000 7f7a0ec00640 1 -- 172.21.15.50:0/1955525064 wait complete. 2024-09-21T05:56:47.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.684+0000 7f7a0ec00640 1 Processor -- start 2024-09-21T05:56:47.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.684+0000 7f7a0ec00640 1 -- start start 2024-09-21T05:56:47.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.684+0000 7f7a0ec00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a080d29c0 0x7f7a08147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.684+0000 7f7a0ec00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a08155ad0 0x7f7a08153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.684+0000 7f7a0ec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a0815f180 0x7f7a08154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.684+0000 7f7a0ec00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0816c4a0 con 0x7f7a08155ad0 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.684+0000 7f7a0ec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0816c320 con 0x7f7a0815f180 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.684+0000 7f7a0ec00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0816c620 con 0x7f7a080d29c0 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.684+0000 7f79ffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a08155ad0 0x7f7a08153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f79ffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a08155ad0 0x7f7a08153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54232/0 (socket says 172.21.15.50:54232) 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f79ffe00640 1 -- 172.21.15.50:0/3243883668 learned_addr learned my addr 172.21.15.50:0/3243883668 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f79ffe00640 1 -- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a080d29c0 msgr2=0x7f7a08147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f7a07e00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a080d29c0 0x7f7a08147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f7a0cc00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a0815f180 0x7f7a08154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f79ffe00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a080d29c0 0x7f7a08147eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f79ffe00640 1 -- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a0815f180 msgr2=0x7f7a08154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f79ffe00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a0815f180 0x7f7a08154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f79ffe00640 1 -- 172.21.15.50:0/3243883668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a080b3120 con 0x7f7a08155ad0 2024-09-21T05:56:47.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f7a0cc00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a0815f180 0x7f7a08154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:47.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f7a07e00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a080d29c0 0x7f7a08147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:47.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f79ffe00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a08155ad0 0x7f7a08153cc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009790 tx=0x7f79f00a55d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:47.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f7a04c00640 1 -- 172.21.15.50:0/3243883668 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f00b2040 con 0x7f7a08155ad0 2024-09-21T05:56:47.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f7a04c00640 1 -- 172.21.15.50:0/3243883668 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79f00043b0 con 0x7f7a08155ad0 2024-09-21T05:56:47.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f7a0ec00640 1 -- 172.21.15.50:0/3243883668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a081592a0 con 0x7f7a08155ad0 2024-09-21T05:56:47.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f7a04c00640 1 -- 172.21.15.50:0/3243883668 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0003820 con 0x7f7a08155ad0 2024-09-21T05:56:47.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f7a0ec00640 1 -- 172.21.15.50:0/3243883668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a081574b0 con 0x7f7a08155ad0 2024-09-21T05:56:47.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f7a0ec00640 1 -- 172.21.15.50:0/3243883668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a08159010 con 0x7f7a08155ad0 2024-09-21T05:56:47.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.688+0000 7f7a04c00640 1 -- 172.21.15.50:0/3243883668 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f79f00bd060 con 0x7f7a08155ad0 2024-09-21T05:56:47.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.692+0000 7f7a04c00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79e4073200 0x7f79e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.692+0000 7f7a04c00640 1 -- 172.21.15.50:0/3243883668 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f79f012d7e0 con 0x7f7a08155ad0 2024-09-21T05:56:47.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.692+0000 7f7a07e00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79e4073200 0x7f79e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.692+0000 7f7a07e00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79e4073200 0x7f79e4075710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f79f8000f70 tx=0x7f79f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:47.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.696+0000 7f7a04c00640 1 -- 172.21.15.50:0/3243883668 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f79f0016410 con 0x7f7a08155ad0 2024-09-21T05:56:47.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.908+0000 7f7a0ec00640 1 -- 172.21.15.50:0/3243883668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7a080d2dd0 con 0x7f7a08155ad0 2024-09-21T05:56:47.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.908+0000 7f7a04c00640 1 -- 172.21.15.50:0/3243883668 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f79f00f8880 con 0x7f7a08155ad0 2024-09-21T05:56:47.912 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:56:47.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.912+0000 7f7a0ec00640 1 -- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79e4073200 msgr2=0x7f79e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.912+0000 7f7a0ec00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79e4073200 0x7f79e4075710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f79f8000f70 tx=0x7f79f8009040 comp rx=0 tx=0).stop 2024-09-21T05:56:47.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.912+0000 7f7a0ec00640 1 -- 172.21.15.50:0/3243883668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a08155ad0 msgr2=0x7f7a08153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.912+0000 7f7a0ec00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a08155ad0 0x7f7a08153cc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009790 tx=0x7f79f00a55d0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.912+0000 7f7a0ec00640 1 -- 172.21.15.50:0/3243883668 shutdown_connections 2024-09-21T05:56:47.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.912+0000 7f7a0ec00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a0815f180 0x7f7a08154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.912+0000 7f7a0ec00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79e4073200 0x7f79e4075710 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.912+0000 7f7a0ec00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a08155ad0 0x7f7a08153cc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.912+0000 7f7a0ec00640 1 --2- 172.21.15.50:0/3243883668 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a080d29c0 0x7f7a08147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.912+0000 7f7a0ec00640 1 -- 172.21.15.50:0/3243883668 >> 172.21.15.50:0/3243883668 conn(0x7f7a080708c0 msgr2=0x7f7a08155120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:47.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.912+0000 7f7a0ec00640 1 -- 172.21.15.50:0/3243883668 shutdown_connections 2024-09-21T05:56:47.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:47.912+0000 7f7a0ec00640 1 -- 172.21.15.50:0/3243883668 wait complete. 2024-09-21T05:56:47.934 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T05:56:47.934 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:56:47.934 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-21T05:56:47.935 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:56:47.935 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:56:47.935 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T05:56:47.935 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T05:56:47.935 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:56:47.935 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:56:47.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:47.935 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-21T05:56:47.944 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:56:47.945 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:56:47.945 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:56:47.945 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:47.945 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T05:56:47.945 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:47.945 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:56:47.945 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:47.945 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-21T05:56:47.945 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:56:47.945 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:47.945 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42387sec preferred_lft 42387sec 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:8b:d0:0a:5d brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:47.946 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T05:56:47.947 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:47.947 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T05:56:47.947 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:47.947 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T05:56:47.947 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add name ceph-brx type bridge 2024-09-21T05:56:47.947 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr flush dev ceph-brx 2024-09-21T05:56:47.947 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set ceph-brx up 2024-09-21T05:56:47.947 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T05:56:47.948 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T05:56:48.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:48.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:48.105 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:56:48.124 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-21T05:56:48.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:48.126 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-21T05:56:48.203 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-21T05:56:48.203 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:56:48.203 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-21T05:56:48.203 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T05:56:48.203 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-21T05:56:48.203 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:56:48.204 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:48.204 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T05:56:48.204 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:56:48.204 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:56:48.204 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:56:48.204 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T05:56:48.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:48.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:48.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:48.340 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T05:56:48.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:48.349 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2024-09-21T05:56:48.405 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:48.405 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T05:56:48.405 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:56:48.406 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T05:56:48.406 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T05:56:48.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:48.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:48.532 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T05:56:48.533 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:48.533 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T05:56:48.533 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T05:56:48.533 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T05:56:48.533 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T05:56:48.533 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T05:56:48.533 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T05:56:48.533 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T05:56:48.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:48.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:48.716 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:48.716 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T05:56:48.716 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-09-21T05:56:48.716 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T05:56:48.716 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T05:56:48.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:48.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:48.803 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:56:48.804 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:56:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:56:48.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:56:48.812 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:56:48.860 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:56:48.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:48.860 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:56:48.962 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:56:48.962 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:56:48.962 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T05:56:49.620 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:56:49.620 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:56:49.620 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T05:56:49.620 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T05:56:49.620 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:56:49.635 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:56:49.635 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:56:49.635 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T05:56:49.636 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T05:56:49.636 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:56:49.636 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T05:56:49.636 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T05:56:49.636 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T05:56:49.639 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:56:49.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:49.639 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:56:49.661 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:56:49.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:56:49.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:49.790 INFO:teuthology.orchestra.run.smithi050.stdout:client4559 2024-09-21T05:56:49.790 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:56:49.790 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:56:49.799 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:56:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:56:49.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:49.914 INFO:teuthology.orchestra.run.smithi050.stdout:client4559 2024-09-21T05:56:49.915 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:56:49.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:56:49.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:49.985 INFO:teuthology.orchestra.run.smithi050.stdout:client4559 2024-09-21T05:56:49.985 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/3f44e77a-b7bc-4ef8-b058-60eaaa9147d6.client4559/status') 2024-09-21T05:56:50.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:50.054 INFO:teuthology.orchestra.run.smithi050.stdout:instance: client.4559 (3)172.21.15.50:0/3145856872 2024-09-21T05:56:50.054 INFO:teuthology.orchestra.run.smithi050.stdout:blocklisted: false 2024-09-21T05:56:50.055 INFO:teuthology.orchestra.run.smithi050.stderr:0+1 records in 2024-09-21T05:56:50.055 INFO:teuthology.orchestra.run.smithi050.stderr:0+1 records out 2024-09-21T05:56:50.055 INFO:teuthology.orchestra.run.smithi050.stderr:70 bytes copied, 5.4888e-05 s, 1.3 MB/s 2024-09-21T05:56:50.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:50.063 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-21T05:56:50.063 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T05:56:50.063 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T05:56:50.063 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:56:50.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.220+0000 7f4597600640 1 -- 172.21.15.50:0/4137914020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b54f0 msgr2=0x7f459015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:50.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.220+0000 7f4597600640 1 --2- 172.21.15.50:0/4137914020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b54f0 0x7f459015bb60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4588009bc0 tx=0x7f458801cae0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.220+0000 7f4597600640 1 -- 172.21.15.50:0/4137914020 shutdown_connections 2024-09-21T05:56:50.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.220+0000 7f4597600640 1 --2- 172.21.15.50:0/4137914020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b54f0 0x7f459015bb60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.220+0000 7f4597600640 1 -- 172.21.15.50:0/4137914020 >> 172.21.15.50:0/4137914020 conn(0x7f45900b2040 msgr2=0x7f45900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:50.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.220+0000 7f4597600640 1 -- 172.21.15.50:0/4137914020 shutdown_connections 2024-09-21T05:56:50.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.220+0000 7f4597600640 1 -- 172.21.15.50:0/4137914020 wait complete. 2024-09-21T05:56:50.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4597600640 1 Processor -- start 2024-09-21T05:56:50.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4597600640 1 -- start start 2024-09-21T05:56:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4597600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45900b5c70 0x7f459014b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4597600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45901527b0 0x7f4590156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4597600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f459014ba20 0x7f4590156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4597600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4590169b30 con 0x7f45901527b0 2024-09-21T05:56:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4597600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f45901699b0 con 0x7f459014ba20 2024-09-21T05:56:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4597600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4590169cb0 con 0x7f45900b5c70 2024-09-21T05:56:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f458fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45901527b0 0x7f4590156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f458fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45901527b0 0x7f4590156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54244/0 (socket says 172.21.15.50:54244) 2024-09-21T05:56:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f458fe00640 1 -- 172.21.15.50:0/17626362 learned_addr learned my addr 172.21.15.50:0/17626362 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4595600640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f459014ba20 0x7f4590156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4594c00640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45900b5c70 0x7f459014b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f458fe00640 1 -- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45900b5c70 msgr2=0x7f459014b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f458fe00640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45900b5c70 0x7f459014b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f458fe00640 1 -- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f459014ba20 msgr2=0x7f4590156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f458fe00640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f459014ba20 0x7f4590156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f458fe00640 1 -- 172.21.15.50:0/17626362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45901543e0 con 0x7f45901527b0 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4595600640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f459014ba20 0x7f4590156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f458fe00640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45901527b0 0x7f4590156ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f457c00c3c0 tx=0x7f457c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f458cc00640 1 -- 172.21.15.50:0/17626362 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c017070 con 0x7f45901527b0 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f458cc00640 1 -- 172.21.15.50:0/17626362 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f457c003600 con 0x7f45901527b0 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f458cc00640 1 -- 172.21.15.50:0/17626362 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c003a50 con 0x7f45901527b0 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4597600640 1 -- 172.21.15.50:0/17626362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4590158c70 con 0x7f45901527b0 2024-09-21T05:56:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4597600640 1 -- 172.21.15.50:0/17626362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4590159230 con 0x7f45901527b0 2024-09-21T05:56:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.224+0000 7f4597600640 1 -- 172.21.15.50:0/17626362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4590154150 con 0x7f45901527b0 2024-09-21T05:56:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.228+0000 7f458cc00640 1 -- 172.21.15.50:0/17626362 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f457c01a070 con 0x7f45901527b0 2024-09-21T05:56:50.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.228+0000 7f458cc00640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4570073230 0x7f4570075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:50.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.228+0000 7f458cc00640 1 -- 172.21.15.50:0/17626362 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f457c093490 con 0x7f45901527b0 2024-09-21T05:56:50.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.228+0000 7f4594c00640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4570073230 0x7f4570075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:50.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.228+0000 7f4594c00640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4570073230 0x7f4570075740 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4584000f70 tx=0x7f4584009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.232+0000 7f458cc00640 1 -- 172.21.15.50:0/17626362 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f457c05e530 con 0x7f45901527b0 2024-09-21T05:56:50.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.451+0000 7f4597600640 1 -- 172.21.15.50:0/17626362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f459015bb60 con 0x7f45901527b0 2024-09-21T05:56:50.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.451+0000 7f458cc00640 1 -- 172.21.15.50:0/17626362 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f457c05de30 con 0x7f45901527b0 2024-09-21T05:56:50.457 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:56:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.455+0000 7f4597600640 1 -- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4570073230 msgr2=0x7f4570075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.455+0000 7f4597600640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4570073230 0x7f4570075740 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4584000f70 tx=0x7f4584009040 comp rx=0 tx=0).stop 2024-09-21T05:56:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.455+0000 7f4597600640 1 -- 172.21.15.50:0/17626362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45901527b0 msgr2=0x7f4590156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.455+0000 7f4597600640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45901527b0 0x7f4590156ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f457c00c3c0 tx=0x7f457c00c880 comp rx=0 tx=0).stop 2024-09-21T05:56:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.455+0000 7f4597600640 1 -- 172.21.15.50:0/17626362 shutdown_connections 2024-09-21T05:56:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.455+0000 7f4597600640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f459014ba20 0x7f4590156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.455+0000 7f4597600640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4570073230 0x7f4570075740 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.455+0000 7f4597600640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45901527b0 0x7f4590156ab0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.455+0000 7f4597600640 1 --2- 172.21.15.50:0/17626362 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45900b5c70 0x7f459014b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.455+0000 7f4597600640 1 -- 172.21.15.50:0/17626362 >> 172.21.15.50:0/17626362 conn(0x7f45900b2040 msgr2=0x7f4590159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.455+0000 7f4597600640 1 -- 172.21.15.50:0/17626362 shutdown_connections 2024-09-21T05:56:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50.455+0000 7f4597600640 1 -- 172.21.15.50:0/17626362 wait complete. 2024-09-21T05:56:50.479 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T05:56:50.479 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:56:50.479 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-21T05:56:50.479 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-21T05:56:50.479 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T05:56:50.479 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T05:56:50.479 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T05:56:50.479 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:56:50.479 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-21T05:56:50.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:50.479 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-21T05:56:50.488 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:56:50.488 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-21T05:56:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42385sec preferred_lft 42385sec 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:8b:d0:0a:5d brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:56:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:50.491 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T05:56:50.491 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:50.491 INFO:teuthology.orchestra.run.smithi050.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:56:50.491 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:56:50.491 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-09-21T05:56:50.491 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:50.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:50.492 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:56:50.555 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-21T05:56:50.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:50.557 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-21T05:56:50.610 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-21T05:56:50.610 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:56:50.610 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-21T05:56:50.610 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T05:56:50.610 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-21T05:56:50.610 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:56:50.611 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:50.611 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T05:56:50.611 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:56:50.611 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:56:50.611 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:56:50.611 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T05:56:50.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:50.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:50.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:50.747 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T05:56:50.757 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:56:50.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:50.757 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2024-09-21T05:56:50.812 INFO:teuthology.orchestra.run.smithi050.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-21T05:56:50.813 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:50.814 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T05:56:50.814 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:56:50.814 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-21T05:56:50.814 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T05:56:50.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:50.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:50.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:50.941 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T05:56:50.962 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:56:50.962 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:56:50.962 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:56:50.963 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:50.963 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T05:56:50.963 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:50.963 INFO:teuthology.orchestra.run.smithi050.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:56:50.963 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 92:42:ec:9a:56:6c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T05:56:50.963 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T05:56:50.963 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:50.963 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::9042:ecff:fe9a:566c/64 scope link 2024-09-21T05:56:50.963 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:50.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:50.965 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T05:56:51.033 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:56:51.033 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:56:51.033 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:56:51.033 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:51.033 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T05:56:51.033 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:51.033 INFO:teuthology.orchestra.run.smithi050.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:56:51.033 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 92:42:ec:9a:56:6c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T05:56:51.033 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T05:56:51.034 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:51.034 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::9042:ecff:fe9a:566c/64 scope link 2024-09-21T05:56:51.034 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:51.035 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-21T05:56:51.035 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:51.035 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T05:56:51.035 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-21T05:56:51.035 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T05:56:51.035 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-21T05:56:51.035 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-21T05:56:51.036 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-21T05:56:51.036 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T05:56:51.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:51.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:51.219 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:51.219 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T05:56:51.219 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.1 up 2024-09-21T05:56:51.219 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T05:56:51.220 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T05:56:51.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:51.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:51.304 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:56:51.304 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-21T05:56:51.313 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-21T05:56:51.313 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:56:51.313 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-21T05:56:51.364 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:56:51.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:51.364 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T05:56:51.466 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T05:56:51.466 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T05:56:51.466 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T05:56:51.484 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T05:56:51.485 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T05:56:51.485 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T05:56:51.485 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T05:56:51.485 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:56:51.495 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T05:56:51.495 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T05:56:51.496 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T05:56:51.496 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T05:56:51.496 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:56:51.496 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T05:56:51.496 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T05:56:51.496 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T05:56:51.499 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:56:51.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:51.499 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-21T05:56:51.519 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:56:51.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:56:51.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:51.623 INFO:teuthology.orchestra.run.smithi050.stdout:client4565 2024-09-21T05:56:51.624 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:56:51.624 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:56:51.632 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:56:51.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:56:51.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:51.730 INFO:teuthology.orchestra.run.smithi050.stdout:client4565 2024-09-21T05:56:51.730 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:56:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:56:51.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:51.788 INFO:teuthology.orchestra.run.smithi050.stdout:client4565 2024-09-21T05:56:51.788 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/3f44e77a-b7bc-4ef8-b058-60eaaa9147d6.client4565/status') 2024-09-21T05:56:51.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:51.848 INFO:teuthology.orchestra.run.smithi050.stdout:instance: client.4565 (3)172.21.15.50:0/338191602 2024-09-21T05:56:51.848 INFO:teuthology.orchestra.run.smithi050.stdout:blocklisted: false 2024-09-21T05:56:51.848 INFO:teuthology.orchestra.run.smithi050.stderr:0+1 records in 2024-09-21T05:56:51.848 INFO:teuthology.orchestra.run.smithi050.stderr:0+1 records out 2024-09-21T05:56:51.848 INFO:teuthology.orchestra.run.smithi050.stderr:69 bytes copied, 5.2644e-05 s, 1.3 MB/s 2024-09-21T05:56:51.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:51.856 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-21T05:56:51.856 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T05:56:51.856 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T05:56:51.856 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:56:52.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.011+0000 7f24a8c00640 1 -- 172.21.15.50:0/3615239994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24a4160d50 msgr2=0x7f24a4074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:52.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.011+0000 7f24a8c00640 1 --2- 172.21.15.50:0/3615239994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24a4160d50 0x7f24a4074410 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f248c009bc0 tx=0x7f248c01cba0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.011+0000 7f24a8c00640 1 -- 172.21.15.50:0/3615239994 shutdown_connections 2024-09-21T05:56:52.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.011+0000 7f24a8c00640 1 --2- 172.21.15.50:0/3615239994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24a4160d50 0x7f24a4074410 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.011+0000 7f24a8c00640 1 --2- 172.21.15.50:0/3615239994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24a4157030 0x7f24a40b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.011+0000 7f24a8c00640 1 -- 172.21.15.50:0/3615239994 >> 172.21.15.50:0/3615239994 conn(0x7f24a40708c0 msgr2=0x7f24a4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:52.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.011+0000 7f24a8c00640 1 -- 172.21.15.50:0/3615239994 shutdown_connections 2024-09-21T05:56:52.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.011+0000 7f24a8c00640 1 -- 172.21.15.50:0/3615239994 wait complete. 2024-09-21T05:56:52.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.015+0000 7f24a8c00640 1 Processor -- start 2024-09-21T05:56:52.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.015+0000 7f24a8c00640 1 -- start start 2024-09-21T05:56:52.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.015+0000 7f24a8c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a4157030 0x7f24a41fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:52.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.015+0000 7f24a8c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24a4160d50 0x7f24a4143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:52.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.015+0000 7f24a8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24a4161160 0x7f24a41f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:52.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.015+0000 7f24a8c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24a416a850 con 0x7f24a4161160 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.015+0000 7f24a8c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f24a416a6d0 con 0x7f24a4157030 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.015+0000 7f24a8c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f24a416a9d0 con 0x7f24a4160d50 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.015+0000 7f24a2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24a4161160 0x7f24a41f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f24a2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24a4161160 0x7f24a41f9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54248/0 (socket says 172.21.15.50:54248) 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f24a2800640 1 -- 172.21.15.50:0/358804688 learned_addr learned my addr 172.21.15.50:0/358804688 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f24a1e00640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a4157030 0x7f24a41fd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f2499400640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24a4160d50 0x7f24a4143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f24a2800640 1 -- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24a4160d50 msgr2=0x7f24a4143e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f24a2800640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24a4160d50 0x7f24a4143e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f24a2800640 1 -- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a4157030 msgr2=0x7f24a41fd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f24a2800640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a4157030 0x7f24a41fd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f24a2800640 1 -- 172.21.15.50:0/358804688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a41f9270 con 0x7f24a4161160 2024-09-21T05:56:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f24a2800640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24a4161160 0x7f24a41f9a30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f249c00c3c0 tx=0x7f249c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:52.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f249aa00640 1 -- 172.21.15.50:0/358804688 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c011790 con 0x7f24a4161160 2024-09-21T05:56:52.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f249aa00640 1 -- 172.21.15.50:0/358804688 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f249c017070 con 0x7f24a4161160 2024-09-21T05:56:52.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f24a8c00640 1 -- 172.21.15.50:0/358804688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a41443b0 con 0x7f24a4161160 2024-09-21T05:56:52.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f249aa00640 1 -- 172.21.15.50:0/358804688 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c011ce0 con 0x7f24a4161160 2024-09-21T05:56:52.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f24a8c00640 1 -- 172.21.15.50:0/358804688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a4162360 con 0x7f24a4161160 2024-09-21T05:56:52.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f24a8c00640 1 -- 172.21.15.50:0/358804688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a40d2260 con 0x7f24a4161160 2024-09-21T05:56:52.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.019+0000 7f249aa00640 1 -- 172.21.15.50:0/358804688 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f249c01c460 con 0x7f24a4161160 2024-09-21T05:56:52.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.023+0000 7f249aa00640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2480073200 0x7f2480075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:52.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.023+0000 7f249aa00640 1 -- 172.21.15.50:0/358804688 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f249c093ea0 con 0x7f24a4161160 2024-09-21T05:56:52.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.023+0000 7f24a1e00640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2480073200 0x7f2480075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:52.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.023+0000 7f24a1e00640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2480073200 0x7f2480075710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2494009b40 tx=0x7f24940091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:52.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.027+0000 7f249aa00640 1 -- 172.21.15.50:0/358804688 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f249c05e860 con 0x7f24a4161160 2024-09-21T05:56:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.239+0000 7f24a8c00640 1 -- 172.21.15.50:0/358804688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f24a41f9f70 con 0x7f24a4161160 2024-09-21T05:56:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.239+0000 7f249aa00640 1 -- 172.21.15.50:0/358804688 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f249c063290 con 0x7f24a4161160 2024-09-21T05:56:52.245 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:56:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.243+0000 7f24a8c00640 1 -- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2480073200 msgr2=0x7f2480075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.243+0000 7f24a8c00640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2480073200 0x7f2480075710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2494009b40 tx=0x7f24940091e0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.243+0000 7f24a8c00640 1 -- 172.21.15.50:0/358804688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24a4161160 msgr2=0x7f24a41f9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.243+0000 7f24a8c00640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24a4161160 0x7f24a41f9a30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f249c00c3c0 tx=0x7f249c00c880 comp rx=0 tx=0).stop 2024-09-21T05:56:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.243+0000 7f24a8c00640 1 -- 172.21.15.50:0/358804688 shutdown_connections 2024-09-21T05:56:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.243+0000 7f24a8c00640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2480073200 0x7f2480075710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.243+0000 7f24a8c00640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24a4161160 0x7f24a41f9a30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.243+0000 7f24a8c00640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24a4160d50 0x7f24a4143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.243+0000 7f24a8c00640 1 --2- 172.21.15.50:0/358804688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a4157030 0x7f24a41fd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.243+0000 7f24a8c00640 1 -- 172.21.15.50:0/358804688 >> 172.21.15.50:0/358804688 conn(0x7f24a40708c0 msgr2=0x7f24a4155fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:52.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.243+0000 7f24a8c00640 1 -- 172.21.15.50:0/358804688 shutdown_connections 2024-09-21T05:56:52.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:52.243+0000 7f24a8c00640 1 -- 172.21.15.50:0/358804688 wait complete. 2024-09-21T05:56:52.267 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T05:56:52.267 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:56:52.267 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T05:56:52.267 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T05:56:52.267 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T05:56:52.267 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T05:56:52.267 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T05:56:52.267 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:56:52.267 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T05:56:52.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:52.268 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T05:56:52.277 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:56:52.277 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:56:52.277 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:56:52.277 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T05:56:52.278 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42402sec preferred_lft 42402sec 2024-09-21T05:56:52.279 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2024-09-21T05:56:52.279 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:52.279 INFO:teuthology.orchestra.run.smithi088.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T05:56:52.279 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 02:42:42:c8:9d:20 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:52.279 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T05:56:52.279 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:52.280 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T05:56:52.280 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:52.280 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T05:56:52.280 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add name ceph-brx type bridge 2024-09-21T05:56:52.280 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr flush dev ceph-brx 2024-09-21T05:56:52.280 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set ceph-brx up 2024-09-21T05:56:52.280 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T05:56:52.280 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T05:56:52.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:52.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:52.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:52.444 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:56:52.462 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T05:56:52.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:52.464 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T05:56:52.539 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T05:56:52.540 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:56:52.540 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T05:56:52.540 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T05:56:52.540 INFO:teuthology.orchestra.run.smithi088.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T05:56:52.540 INFO:teuthology.orchestra.run.smithi088.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T05:56:52.540 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T05:56:52.540 INFO:teuthology.orchestra.run.smithi088.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T05:56:52.540 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T05:56:52.540 INFO:teuthology.orchestra.run.smithi088.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T05:56:52.540 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:56:52.541 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:52.541 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T05:56:52.541 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T05:56:52.541 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T05:56:52.541 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T05:56:52.541 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T05:56:52.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:52.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:52.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:52.680 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T05:56:52.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:52.689 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list-id 2024-09-21T05:56:52.738 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:52.739 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T05:56:52.739 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T05:56:52.739 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-21T05:56:52.739 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T05:56:52.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:52.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:52.870 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-21T05:56:52.870 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:52.870 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T05:56:52.870 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-21T05:56:52.870 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T05:56:52.870 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-21T05:56:52.870 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-21T05:56:52.871 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-21T05:56:52.871 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T05:56:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:53.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:53.047 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:53.047 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T05:56:53.047 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T05:56:53.047 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T05:56:53.047 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T05:56:53.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:53.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:53.177 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:56:53.177 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T05:56:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T05:56:53.186 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:56:53.186 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T05:56:53.238 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:56:53.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:53.238 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T05:56:53.342 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T05:56:53.342 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T05:56:53.342 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T05:56:54.026 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T05:56:54.026 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T05:56:54.026 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T05:56:54.026 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T05:56:54.026 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:56:54.040 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T05:56:54.040 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T05:56:54.040 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T05:56:54.040 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T05:56:54.040 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:56:54.040 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T05:56:54.040 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T05:56:54.040 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T05:56:54.044 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:56:54.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:54.044 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T05:56:54.063 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:56:54.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:56:54.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:54.179 INFO:teuthology.orchestra.run.smithi088.stdout:client4657 2024-09-21T05:56:54.179 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:56:54.179 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:56:54.230 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:56:54.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:56:54.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:54.329 INFO:teuthology.orchestra.run.smithi088.stdout:client4657 2024-09-21T05:56:54.329 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:56:54.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:56:54.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:54.425 INFO:teuthology.orchestra.run.smithi088.stdout:client4657 2024-09-21T05:56:54.426 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/3f44e77a-b7bc-4ef8-b058-60eaaa9147d6.client4657/status') 2024-09-21T05:56:54.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:54.484 INFO:teuthology.orchestra.run.smithi088.stdout:instance: client.4657 (3)192.168.0.1:0/1045138089 2024-09-21T05:56:54.484 INFO:teuthology.orchestra.run.smithi088.stdout:blocklisted: false 2024-09-21T05:56:54.485 INFO:teuthology.orchestra.run.smithi088.stderr:0+1 records in 2024-09-21T05:56:54.485 INFO:teuthology.orchestra.run.smithi088.stderr:0+1 records out 2024-09-21T05:56:54.485 INFO:teuthology.orchestra.run.smithi088.stderr:69 bytes copied, 6.0498e-05 s, 1.1 MB/s 2024-09-21T05:56:54.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:54.492 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-21T05:56:54.492 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T05:56:54.492 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T05:56:54.492 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:56:54.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.659+0000 7f6802400640 1 -- 172.21.15.50:0/987206731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67fc074950 msgr2=0x7f67fc15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:54.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.659+0000 7f6802400640 1 --2- 172.21.15.50:0/987206731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67fc074950 0x7f67fc15cf90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009c80 tx=0x7f67f401cbc0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.659+0000 7f6802400640 1 -- 172.21.15.50:0/987206731 shutdown_connections 2024-09-21T05:56:54.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.659+0000 7f6802400640 1 --2- 172.21.15.50:0/987206731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67fc074950 0x7f67fc15cf90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.659+0000 7f6802400640 1 --2- 172.21.15.50:0/987206731 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f67fc0b5a80 0x7f67fc074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.659+0000 7f6802400640 1 -- 172.21.15.50:0/987206731 >> 172.21.15.50:0/987206731 conn(0x7f67fc0708c0 msgr2=0x7f67fc070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:54.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.659+0000 7f6802400640 1 -- 172.21.15.50:0/987206731 shutdown_connections 2024-09-21T05:56:54.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.659+0000 7f6802400640 1 -- 172.21.15.50:0/987206731 wait complete. 2024-09-21T05:56:54.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.663+0000 7f6802400640 1 Processor -- start 2024-09-21T05:56:54.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.663+0000 7f6802400640 1 -- start start 2024-09-21T05:56:54.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.663+0000 7f6802400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f67fc074950 0x7f67fc1459d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:54.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.663+0000 7f6802400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67fc0b5a80 0x7f67fc15eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:54.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.663+0000 7f6802400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc157030 0x7f67fc15efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:54.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.663+0000 7f6802400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc16a3c0 con 0x7f67fc0b5a80 2024-09-21T05:56:54.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.663+0000 7f6802400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc16a240 con 0x7f67fc157030 2024-09-21T05:56:54.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.663+0000 7f6802400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f67fc16a540 con 0x7f67fc074950 2024-09-21T05:56:54.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.663+0000 7f67f3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67fc0b5a80 0x7f67fc15eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:54.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.663+0000 7f67f3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67fc0b5a80 0x7f67fc15eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54258/0 (socket says 172.21.15.50:54258) 2024-09-21T05:56:54.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.663+0000 7f67f3e00640 1 -- 172.21.15.50:0/3830132718 learned_addr learned my addr 172.21.15.50:0/3830132718 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.663+0000 7f67fb400640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f67fc074950 0x7f67fc1459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f67fbe00640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc157030 0x7f67fc15efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f67f3e00640 1 -- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f67fc074950 msgr2=0x7f67fc1459d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f67f3e00640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f67fc074950 0x7f67fc1459d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f67f3e00640 1 -- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc157030 msgr2=0x7f67fc15efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f67f3e00640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc157030 0x7f67fc15efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f67f3e00640 1 -- 172.21.15.50:0/3830132718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67fc1fd3a0 con 0x7f67fc0b5a80 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f67f3e00640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67fc0b5a80 0x7f67fc15eaa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f67e400d6a0 tx=0x7f67e400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f67f3400640 1 -- 172.21.15.50:0/3830132718 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4017070 con 0x7f67fc0b5a80 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f67f3400640 1 -- 172.21.15.50:0/3830132718 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67e4005310 con 0x7f67fc0b5a80 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f6802400640 1 -- 172.21.15.50:0/3830132718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67fc1fd680 con 0x7f67fc0b5a80 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f67f3400640 1 -- 172.21.15.50:0/3830132718 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4009820 con 0x7f67fc0b5a80 2024-09-21T05:56:54.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f6802400640 1 -- 172.21.15.50:0/3830132718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67fc0b4fb0 con 0x7f67fc0b5a80 2024-09-21T05:56:54.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f6802400640 1 -- 172.21.15.50:0/3830132718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67fc151b10 con 0x7f67fc0b5a80 2024-09-21T05:56:54.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.667+0000 7f67f3400640 1 -- 172.21.15.50:0/3830132718 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f67e40099c0 con 0x7f67fc0b5a80 2024-09-21T05:56:54.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.671+0000 7f67f3400640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67d8073230 0x7f67d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:54.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.671+0000 7f67f3400640 1 -- 172.21.15.50:0/3830132718 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f67e4094330 con 0x7f67fc0b5a80 2024-09-21T05:56:54.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.671+0000 7f67fb400640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67d8073230 0x7f67d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:54.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.671+0000 7f67fb400640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67d8073230 0x7f67d8075740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0098e0 tx=0x7f67ec009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:54.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.675+0000 7f67f3400640 1 -- 172.21.15.50:0/3830132718 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f67e405ec70 con 0x7f67fc0b5a80 2024-09-21T05:56:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.891+0000 7f6802400640 1 -- 172.21.15.50:0/3830132718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f67fc15cf90 con 0x7f67fc0b5a80 2024-09-21T05:56:54.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.891+0000 7f67f3400640 1 -- 172.21.15.50:0/3830132718 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f67e40636a0 con 0x7f67fc0b5a80 2024-09-21T05:56:54.896 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:56:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.895+0000 7f6802400640 1 -- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67d8073230 msgr2=0x7f67d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.895+0000 7f6802400640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67d8073230 0x7f67d8075740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0098e0 tx=0x7f67ec009060 comp rx=0 tx=0).stop 2024-09-21T05:56:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.895+0000 7f6802400640 1 -- 172.21.15.50:0/3830132718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67fc0b5a80 msgr2=0x7f67fc15eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.895+0000 7f6802400640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67fc0b5a80 0x7f67fc15eaa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f67e400d6a0 tx=0x7f67e400db60 comp rx=0 tx=0).stop 2024-09-21T05:56:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.895+0000 7f6802400640 1 -- 172.21.15.50:0/3830132718 shutdown_connections 2024-09-21T05:56:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.895+0000 7f6802400640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc157030 0x7f67fc15efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.895+0000 7f6802400640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67d8073230 0x7f67d8075740 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.895+0000 7f6802400640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67fc0b5a80 0x7f67fc15eaa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.895+0000 7f6802400640 1 --2- 172.21.15.50:0/3830132718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f67fc074950 0x7f67fc1459d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.895+0000 7f6802400640 1 -- 172.21.15.50:0/3830132718 >> 172.21.15.50:0/3830132718 conn(0x7f67fc0708c0 msgr2=0x7f67fc154050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.895+0000 7f6802400640 1 -- 172.21.15.50:0/3830132718 shutdown_connections 2024-09-21T05:56:54.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:54.895+0000 7f6802400640 1 -- 172.21.15.50:0/3830132718 wait complete. 2024-09-21T05:56:54.920 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T05:56:54.920 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:56:54.920 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T05:56:54.920 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-21T05:56:54.920 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T05:56:54.920 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T05:56:54.920 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T05:56:54.920 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:56:54.921 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-21T05:56:54.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:54.921 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T05:56:54.930 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:56:54.930 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:56:54.930 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:56:54.930 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:54.930 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T05:56:54.930 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:54.930 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:56:54.930 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:54.930 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T05:56:54.930 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:56:54.930 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42399sec preferred_lft 42399sec 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 02:42:42:c8:9d:20 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:54.931 INFO:teuthology.orchestra.run.smithi088.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:56:54.932 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T05:56:54.932 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:56:54.932 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:54.932 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T05:56:54.932 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:54.932 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:56:54.932 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T05:56:54.932 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T05:56:54.932 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:54.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:54.933 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:56:54.996 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T05:56:54.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:54.998 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T05:56:55.054 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T05:56:55.054 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:56:55.054 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T05:56:55.054 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T05:56:55.054 INFO:teuthology.orchestra.run.smithi088.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T05:56:55.054 INFO:teuthology.orchestra.run.smithi088.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T05:56:55.055 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T05:56:55.055 INFO:teuthology.orchestra.run.smithi088.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T05:56:55.055 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T05:56:55.055 INFO:teuthology.orchestra.run.smithi088.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T05:56:55.055 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:56:55.055 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:55.055 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T05:56:55.055 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T05:56:55.055 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T05:56:55.055 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T05:56:55.055 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T05:56:55.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:55.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:55.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:55.192 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T05:56:55.242 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T05:56:55.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:55.243 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list-id 2024-09-21T05:56:55.298 INFO:teuthology.orchestra.run.smithi088.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-21T05:56:55.299 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:55.299 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T05:56:55.299 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T05:56:55.299 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-21T05:56:55.299 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T05:56:55.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:55.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:55.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:55.424 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T05:56:55.444 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:56:55.444 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:56:55.444 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:56:55.444 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:55.444 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T05:56:55.445 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:55.445 INFO:teuthology.orchestra.run.smithi088.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:56:55.445 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ce:48:45:53:1e:b2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T05:56:55.445 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T05:56:55.445 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:55.445 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::cc48:45ff:fe53:1eb2/64 scope link 2024-09-21T05:56:55.445 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:55.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:55.446 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi088.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ce:48:45:53:1e:b2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::cc48:45ff:fe53:1eb2/64 scope link 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:56:55.511 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-21T05:56:55.511 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:55.511 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T05:56:55.512 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-21T05:56:55.512 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T05:56:55.512 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-21T05:56:55.512 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-21T05:56:55.512 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-21T05:56:55.512 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T05:56:55.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:55.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:55.695 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:56:55.695 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T05:56:55.695 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.1 up 2024-09-21T05:56:55.695 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T05:56:55.695 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T05:56:55.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:55.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:55.776 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:56:55.777 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-21T05:56:55.826 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-21T05:56:55.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:56:55.827 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-21T05:56:55.882 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:56:55.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:55.882 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T05:56:55.982 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T05:56:55.982 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T05:56:55.982 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T05:56:56.000 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T05:56:56.000 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T05:56:56.000 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T05:56:56.000 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T05:56:56.000 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:56:56.012 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T05:56:56.012 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T05:56:56.012 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T05:56:56.012 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T05:56:56.012 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:56:56.012 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T05:56:56.013 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T05:56:56.013 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T05:56:56.016 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:56:56.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:56:56.016 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-21T05:56:56.035 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:56:56.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:56:56.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:56.138 INFO:teuthology.orchestra.run.smithi088.stdout:client4716 2024-09-21T05:56:56.138 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:56:56.138 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:56:56.186 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:56:56.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:56:56.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:56.288 INFO:teuthology.orchestra.run.smithi088.stdout:client4716 2024-09-21T05:56:56.288 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:56:56.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:56:56.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:56.385 INFO:teuthology.orchestra.run.smithi088.stdout:client4716 2024-09-21T05:56:56.385 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/3f44e77a-b7bc-4ef8-b058-60eaaa9147d6.client4716/status') 2024-09-21T05:56:56.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:56:56.443 INFO:teuthology.orchestra.run.smithi088.stdout:instance: client.4716 (3)172.21.15.88:0/2765617200 2024-09-21T05:56:56.443 INFO:teuthology.orchestra.run.smithi088.stdout:blocklisted: false 2024-09-21T05:56:56.443 INFO:teuthology.orchestra.run.smithi088.stderr:0+1 records in 2024-09-21T05:56:56.444 INFO:teuthology.orchestra.run.smithi088.stderr:0+1 records out 2024-09-21T05:56:56.444 INFO:teuthology.orchestra.run.smithi088.stderr:70 bytes copied, 5.9718e-05 s, 1.2 MB/s 2024-09-21T05:56:56.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T05:56:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:56:56.451 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-21T05:56:57.715 INFO:tasks.cephfs_test_runner:Starting test: test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Initiate cloning where clone subvolume and source subvolume are located 2024-09-21T05:56:57.715 INFO:tasks.cephfs_test_runner:test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-21T05:56:57.715 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:56:57.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.883+0000 7fd1a8e00640 1 -- 172.21.15.50:0/1250900888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a41527b0 msgr2=0x7fd1a4074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:57.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.883+0000 7fd1a8e00640 1 --2- 172.21.15.50:0/1250900888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a41527b0 0x7fd1a4074e30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009900 tx=0x7fd18c01c820 comp rx=0 tx=0).stop 2024-09-21T05:56:57.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.883+0000 7fd1a8e00640 1 -- 172.21.15.50:0/1250900888 shutdown_connections 2024-09-21T05:56:57.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.883+0000 7fd1a8e00640 1 --2- 172.21.15.50:0/1250900888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a41527b0 0x7fd1a4074e30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.883+0000 7fd1a8e00640 1 --2- 172.21.15.50:0/1250900888 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1a40b5c70 0x7fd1a415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.883+0000 7fd1a8e00640 1 -- 172.21.15.50:0/1250900888 >> 172.21.15.50:0/1250900888 conn(0x7fd1a40b2040 msgr2=0x7fd1a40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:57.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.883+0000 7fd1a8e00640 1 -- 172.21.15.50:0/1250900888 shutdown_connections 2024-09-21T05:56:57.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.883+0000 7fd1a8e00640 1 -- 172.21.15.50:0/1250900888 wait complete. 2024-09-21T05:56:57.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd1a8e00640 1 Processor -- start 2024-09-21T05:56:57.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd1a8e00640 1 -- start start 2024-09-21T05:56:57.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd1a8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a40b4de0 0x7fd1a40cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd1a8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a40b5c70 0x7fd1a41507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd1a8e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1a41527b0 0x7fd1a4150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd1a8e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a4168bb0 con 0x7fd1a40b5c70 2024-09-21T05:56:57.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd1a8e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a4168a30 con 0x7fd1a40b4de0 2024-09-21T05:56:57.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd1a8e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a4168d30 con 0x7fd1a41527b0 2024-09-21T05:56:57.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd19be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a40b5c70 0x7fd1a41507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd19be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a40b5c70 0x7fd1a41507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48728/0 (socket says 172.21.15.50:48728) 2024-09-21T05:56:57.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd19be00640 1 -- 172.21.15.50:0/3395781889 learned_addr learned my addr 172.21.15.50:0/3395781889 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:57.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd1a1e00640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a40b4de0 0x7fd1a40cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd1a2800640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1a41527b0 0x7fd1a4150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd19be00640 1 -- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1a41527b0 msgr2=0x7fd1a4150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:57.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd19be00640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1a41527b0 0x7fd1a4150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd19be00640 1 -- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a40b4de0 msgr2=0x7fd1a40cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:57.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd19be00640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a40b4de0 0x7fd1a40cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd19be00640 1 -- 172.21.15.50:0/3395781889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a4151220 con 0x7fd1a40b5c70 2024-09-21T05:56:57.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd19be00640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a40b5c70 0x7fd1a41507a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009850 tx=0x7fd18c0a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:57.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd19a000640 1 -- 172.21.15.50:0/3395781889 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c0b2040 con 0x7fd1a40b5c70 2024-09-21T05:56:57.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd19a000640 1 -- 172.21.15.50:0/3395781889 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd18c004430 con 0x7fd1a40b5c70 2024-09-21T05:56:57.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd1a8e00640 1 -- 172.21.15.50:0/3395781889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a41564f0 con 0x7fd1a40b5c70 2024-09-21T05:56:57.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd19a000640 1 -- 172.21.15.50:0/3395781889 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c005290 con 0x7fd1a40b5c70 2024-09-21T05:56:57.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.887+0000 7fd1a8e00640 1 -- 172.21.15.50:0/3395781889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a4159080 con 0x7fd1a40b5c70 2024-09-21T05:56:57.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.891+0000 7fd1a8e00640 1 -- 172.21.15.50:0/3395781889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a4156bd0 con 0x7fd1a40b5c70 2024-09-21T05:56:57.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.891+0000 7fd19a000640 1 -- 172.21.15.50:0/3395781889 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd18c005430 con 0x7fd1a40b5c70 2024-09-21T05:56:57.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.891+0000 7fd19a000640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd180073200 0x7fd180075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.891+0000 7fd19a000640 1 -- 172.21.15.50:0/3395781889 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd18c12e120 con 0x7fd1a40b5c70 2024-09-21T05:56:57.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.891+0000 7fd1a1e00640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd180073200 0x7fd180075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.895+0000 7fd1a1e00640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd180073200 0x7fd180075710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd194009b40 tx=0x7fd1940091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:57.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:57.895+0000 7fd19a000640 1 -- 172.21.15.50:0/3395781889 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd1a4075580 con 0x7fd1a40b5c70 2024-09-21T05:56:58.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 -- 172.21.15.50:0/3395781889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1a415bb60 con 0x7fd1a40b5c70 2024-09-21T05:56:58.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd19a000640 1 -- 172.21.15.50:0/3395781889 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16617 (secure 0 0 0) 0x7fd18c0164d0 con 0x7fd1a40b5c70 2024-09-21T05:56:58.079 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:56:58.079 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":22,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:30.775110+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:56:27.052826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:56:28.162376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:58.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 -- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd180073200 msgr2=0x7fd180075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd180073200 0x7fd180075710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd194009b40 tx=0x7fd1940091e0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 -- 172.21.15.50:0/3395781889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a40b5c70 msgr2=0x7fd1a41507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a40b5c70 0x7fd1a41507a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009850 tx=0x7fd18c0a57f0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 -- 172.21.15.50:0/3395781889 shutdown_connections 2024-09-21T05:56:58.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1a41527b0 0x7fd1a4150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd180073200 0x7fd180075710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a40b5c70 0x7fd1a41507a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 --2- 172.21.15.50:0/3395781889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a40b4de0 0x7fd1a40cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 -- 172.21.15.50:0/3395781889 >> 172.21.15.50:0/3395781889 conn(0x7fd1a40b2040 msgr2=0x7fd1a40b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:58.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 -- 172.21.15.50:0/3395781889 shutdown_connections 2024-09-21T05:56:58.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.075+0000 7fd1a8e00640 1 -- 172.21.15.50:0/3395781889 wait complete. 2024-09-21T05:56:58.101 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:56:18.921197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T05:56:27.052826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T05:56:28.162376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T05:56:58.101 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T05:56:58.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.259+0000 7f7a2ea00640 1 -- 172.21.15.50:0/1210024040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a28073bb0 msgr2=0x7f7a281662b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.259+0000 7f7a2ea00640 1 --2- 172.21.15.50:0/1210024040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a28073bb0 0x7f7a281662b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009bc0 tx=0x7f7a2001ca80 comp rx=0 tx=0).stop 2024-09-21T05:56:58.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.259+0000 7f7a2ea00640 1 -- 172.21.15.50:0/1210024040 shutdown_connections 2024-09-21T05:56:58.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.259+0000 7f7a2ea00640 1 --2- 172.21.15.50:0/1210024040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a28073bb0 0x7f7a281662b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.259+0000 7f7a2ea00640 1 -- 172.21.15.50:0/1210024040 >> 172.21.15.50:0/1210024040 conn(0x7f7a2806d3f0 msgr2=0x7f7a280b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:58.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.259+0000 7f7a2ea00640 1 -- 172.21.15.50:0/1210024040 shutdown_connections 2024-09-21T05:56:58.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.259+0000 7f7a2ea00640 1 -- 172.21.15.50:0/1210024040 wait complete. 2024-09-21T05:56:58.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2ea00640 1 Processor -- start 2024-09-21T05:56:58.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2ea00640 1 -- start start 2024-09-21T05:56:58.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2806f940 0x7f7a2814ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a28073bb0 0x7f7a280cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2ea00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a280af5b0 0x7f7a280ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2ea00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28169f50 con 0x7f7a2806f940 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2ea00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28169dd0 con 0x7f7a28073bb0 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2ea00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7a2816a0d0 con 0x7f7a280af5b0 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2806f940 0x7f7a2814ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2806f940 0x7f7a2814ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48750/0 (socket says 172.21.15.50:48750) 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2d600640 1 -- 172.21.15.50:0/2783734155 learned_addr learned my addr 172.21.15.50:0/2783734155 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2e000640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a280af5b0 0x7f7a280ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2cc00640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a28073bb0 0x7f7a280cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2d600640 1 -- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a280af5b0 msgr2=0x7f7a280ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2d600640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a280af5b0 0x7f7a280ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2d600640 1 -- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a28073bb0 msgr2=0x7f7a280cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2d600640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a28073bb0 0x7f7a280cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2d600640 1 -- 172.21.15.50:0/2783734155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a280cca70 con 0x7f7a2806f940 2024-09-21T05:56:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2e000640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a280af5b0 0x7f7a280ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:58.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2cc00640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a28073bb0 0x7f7a280cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:58.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a2d600640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2806f940 0x7f7a2814ef30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00c400 tx=0x7f7a1c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:58.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a25600640 1 -- 172.21.15.50:0/2783734155 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c017070 con 0x7f7a2806f940 2024-09-21T05:56:58.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a25600640 1 -- 172.21.15.50:0/2783734155 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a1c003600 con 0x7f7a2806f940 2024-09-21T05:56:58.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.263+0000 7f7a25600640 1 -- 172.21.15.50:0/2783734155 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c003a50 con 0x7f7a2806f940 2024-09-21T05:56:58.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.267+0000 7f7a2ea00640 1 -- 172.21.15.50:0/2783734155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a28159bf0 con 0x7f7a2806f940 2024-09-21T05:56:58.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.267+0000 7f7a2ea00640 1 -- 172.21.15.50:0/2783734155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a281fc470 con 0x7f7a2806f940 2024-09-21T05:56:58.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.267+0000 7f7a2ea00640 1 -- 172.21.15.50:0/2783734155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a28159dc0 con 0x7f7a2806f940 2024-09-21T05:56:58.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.267+0000 7f7a25600640 1 -- 172.21.15.50:0/2783734155 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7a1c01a070 con 0x7f7a2806f940 2024-09-21T05:56:58.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.267+0000 7f7a25600640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a08073200 0x7f7a08075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.267+0000 7f7a25600640 1 -- 172.21.15.50:0/2783734155 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7a1c0934f0 con 0x7f7a2806f940 2024-09-21T05:56:58.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.267+0000 7f7a2cc00640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a08073200 0x7f7a08075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.271+0000 7f7a2cc00640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a08073200 0x7f7a08075710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7a14001230 tx=0x7f7a14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:58.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.271+0000 7f7a25600640 1 -- 172.21.15.50:0/2783734155 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7a1c05e590 con 0x7f7a2806f940 2024-09-21T05:56:58.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 -- 172.21.15.50:0/2783734155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a2815b310 con 0x7f7a2806f940 2024-09-21T05:56:58.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a25600640 1 -- 172.21.15.50:0/2783734155 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f7a1c05de90 con 0x7f7a2806f940 2024-09-21T05:56:58.454 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-09-21T05:56:58.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 -- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a08073200 msgr2=0x7f7a08075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a08073200 0x7f7a08075710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7a14001230 tx=0x7f7a14009040 comp rx=0 tx=0).stop 2024-09-21T05:56:58.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 -- 172.21.15.50:0/2783734155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2806f940 msgr2=0x7f7a2814ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2806f940 0x7f7a2814ef30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00c400 tx=0x7f7a1c00c8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 -- 172.21.15.50:0/2783734155 shutdown_connections 2024-09-21T05:56:58.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a280af5b0 0x7f7a280ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a28073bb0 0x7f7a280cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a08073200 0x7f7a08075710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 --2- 172.21.15.50:0/2783734155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2806f940 0x7f7a2814ef30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 -- 172.21.15.50:0/2783734155 >> 172.21.15.50:0/2783734155 conn(0x7f7a2806d3f0 msgr2=0x7f7a28072b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:58.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 -- 172.21.15.50:0/2783734155 shutdown_connections 2024-09-21T05:56:58.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.451+0000 7f7a2ea00640 1 -- 172.21.15.50:0/2783734155 wait complete. 2024-09-21T05:56:58.476 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T05:56:58.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.635+0000 7f1584800640 1 -- 172.21.15.50:0/1275797637 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157c0703a0 msgr2=0x7f157c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.635+0000 7f1584800640 1 --2- 172.21.15.50:0/1275797637 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157c0703a0 0x7f157c06d200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1578009bc0 tx=0x7f157801cb10 comp rx=0 tx=0).stop 2024-09-21T05:56:58.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.635+0000 7f1584800640 1 -- 172.21.15.50:0/1275797637 shutdown_connections 2024-09-21T05:56:58.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.635+0000 7f1584800640 1 --2- 172.21.15.50:0/1275797637 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157c0703a0 0x7f157c06d200 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.635+0000 7f1584800640 1 --2- 172.21.15.50:0/1275797637 >> v2:172.21.15.88:3300/0 conn(0x7f157c159130 0x7f157c15bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.635+0000 7f1584800640 1 -- 172.21.15.50:0/1275797637 >> 172.21.15.50:0/1275797637 conn(0x7f157c0b35f0 msgr2=0x7f157c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:58.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.635+0000 7f1584800640 1 -- 172.21.15.50:0/1275797637 shutdown_connections 2024-09-21T05:56:58.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.635+0000 7f1584800640 1 -- 172.21.15.50:0/1275797637 wait complete. 2024-09-21T05:56:58.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.635+0000 7f1584800640 1 Processor -- start 2024-09-21T05:56:58.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.635+0000 7f1584800640 1 -- start start 2024-09-21T05:56:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1584800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f157c06cdf0 0x7f157c1f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1584800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f157c0703a0 0x7f157c0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1584800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157c159130 0x7f157c15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1584800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f157c168d50 con 0x7f157c06cdf0 2024-09-21T05:56:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1584800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f157c168bd0 con 0x7f157c0703a0 2024-09-21T05:56:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1584800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f157c168ed0 con 0x7f157c159130 2024-09-21T05:56:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1581e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f157c06cdf0 0x7f157c1f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1581e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f157c06cdf0 0x7f157c1f8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48766/0 (socket says 172.21.15.50:48766) 2024-09-21T05:56:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1581e00640 1 -- 172.21.15.50:0/10724153 learned_addr learned my addr 172.21.15.50:0/10724153 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1581400640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f157c0703a0 0x7f157c0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1582800640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157c159130 0x7f157c15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1581e00640 1 -- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157c159130 msgr2=0x7f157c15adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1581e00640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157c159130 0x7f157c15adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1581e00640 1 -- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f157c0703a0 msgr2=0x7f157c0d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1581e00640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f157c0703a0 0x7f157c0d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1581e00640 1 -- 172.21.15.50:0/10724153 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c159d20 con 0x7f157c06cdf0 2024-09-21T05:56:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1581e00640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f157c06cdf0 0x7f157c1f8db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f157000e970 tx=0x7f157000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:58.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1576000640 1 -- 172.21.15.50:0/10724153 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157000cd20 con 0x7f157c06cdf0 2024-09-21T05:56:58.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1576000640 1 -- 172.21.15.50:0/10724153 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1570004020 con 0x7f157c06cdf0 2024-09-21T05:56:58.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1584800640 1 -- 172.21.15.50:0/10724153 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157c15a000 con 0x7f157c06cdf0 2024-09-21T05:56:58.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1576000640 1 -- 172.21.15.50:0/10724153 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570004470 con 0x7f157c06cdf0 2024-09-21T05:56:58.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1584800640 1 -- 172.21.15.50:0/10724153 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f157c15a5c0 con 0x7f157c06cdf0 2024-09-21T05:56:58.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.639+0000 7f1584800640 1 -- 172.21.15.50:0/10724153 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157c06d200 con 0x7f157c06cdf0 2024-09-21T05:56:58.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.643+0000 7f1576000640 1 -- 172.21.15.50:0/10724153 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1570003600 con 0x7f157c06cdf0 2024-09-21T05:56:58.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.643+0000 7f1576000640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1560073200 0x7f1560075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.643+0000 7f1576000640 1 -- 172.21.15.50:0/10724153 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1570094c60 con 0x7f157c06cdf0 2024-09-21T05:56:58.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.643+0000 7f1581400640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1560073200 0x7f1560075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.643+0000 7f1581400640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1560073200 0x7f1560075710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f157c15b630 tx=0x7f156c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:58.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.647+0000 7f1576000640 1 -- 172.21.15.50:0/10724153 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f157005fbe0 con 0x7f157c06cdf0 2024-09-21T05:56:58.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.823+0000 7f1584800640 1 -- 172.21.15.50:0/10724153 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f157c0b3160 con 0x7f157c06cdf0 2024-09-21T05:56:58.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.823+0000 7f1576000640 1 -- 172.21.15.50:0/10724153 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f157005f4e0 con 0x7f157c06cdf0 2024-09-21T05:56:58.827 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-09-21T05:56:58.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.823+0000 7f1584800640 1 -- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1560073200 msgr2=0x7f1560075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.823+0000 7f1584800640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1560073200 0x7f1560075710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f157c15b630 tx=0x7f156c009040 comp rx=0 tx=0).stop 2024-09-21T05:56:58.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.823+0000 7f1584800640 1 -- 172.21.15.50:0/10724153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f157c06cdf0 msgr2=0x7f157c1f8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.823+0000 7f1584800640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f157c06cdf0 0x7f157c1f8db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f157000e970 tx=0x7f157000ee30 comp rx=0 tx=0).stop 2024-09-21T05:56:58.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.823+0000 7f1584800640 1 -- 172.21.15.50:0/10724153 shutdown_connections 2024-09-21T05:56:58.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.827+0000 7f1584800640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f157c159130 0x7f157c15adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.827+0000 7f1584800640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f157c0703a0 0x7f157c0d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.827+0000 7f1584800640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1560073200 0x7f1560075710 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.827+0000 7f1584800640 1 --2- 172.21.15.50:0/10724153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f157c06cdf0 0x7f157c1f8db0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.827+0000 7f1584800640 1 -- 172.21.15.50:0/10724153 >> 172.21.15.50:0/10724153 conn(0x7f157c0b35f0 msgr2=0x7f157c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:58.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.827+0000 7f1584800640 1 -- 172.21.15.50:0/10724153 shutdown_connections 2024-09-21T05:56:58.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:58.827+0000 7f1584800640 1 -- 172.21.15.50:0/10724153 wait complete. 2024-09-21T05:56:58.849 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T05:56:59.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.007+0000 7f3b7c000640 1 -- 172.21.15.50:0/2389141326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b740b5c70 msgr2=0x7f3b7415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:59.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.007+0000 7f3b7c000640 1 --2- 172.21.15.50:0/2389141326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b740b5c70 0x7f3b7415bb60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3b740cca90 tx=0x7f3b6801c800 comp rx=0 tx=0).stop 2024-09-21T05:56:59.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.007+0000 7f3b7c000640 1 -- 172.21.15.50:0/2389141326 shutdown_connections 2024-09-21T05:56:59.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.007+0000 7f3b7c000640 1 --2- 172.21.15.50:0/2389141326 >> v2:172.21.15.88:3300/0 conn(0x7f3b740b4de0 0x7f3b740b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:59.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.007+0000 7f3b7c000640 1 --2- 172.21.15.50:0/2389141326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b740b5c70 0x7f3b7415bb60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:59.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.007+0000 7f3b7c000640 1 -- 172.21.15.50:0/2389141326 >> 172.21.15.50:0/2389141326 conn(0x7f3b740b2040 msgr2=0x7f3b740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:59.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.007+0000 7f3b7c000640 1 -- 172.21.15.50:0/2389141326 shutdown_connections 2024-09-21T05:56:59.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.007+0000 7f3b7c000640 1 -- 172.21.15.50:0/2389141326 wait complete. 2024-09-21T05:56:59.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7c000640 1 Processor -- start 2024-09-21T05:56:59.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7c000640 1 -- start start 2024-09-21T05:56:59.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7c000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b740b4de0 0x7f3b74154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:59.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7c000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b740b5c70 0x7f3b741fbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:59.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7c000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b741527b0 0x7f3b741f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:59.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7c000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3b74168970 con 0x7f3b741527b0 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7c000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3b741687f0 con 0x7f3b740b4de0 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7c000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3b74168af0 con 0x7f3b740b5c70 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b741527b0 0x7f3b741f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b741527b0 0x7f3b741f7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48776/0 (socket says 172.21.15.50:48776) 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7a000640 1 -- 172.21.15.50:0/670044457 learned_addr learned my addr 172.21.15.50:0/670044457 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b79600640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b740b4de0 0x7f3b74154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b78c00640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b740b5c70 0x7f3b741fbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7a000640 1 -- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b740b5c70 msgr2=0x7f3b741fbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7a000640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b740b5c70 0x7f3b741fbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7a000640 1 -- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b740b4de0 msgr2=0x7f3b74154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7a000640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b740b4de0 0x7f3b74154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7a000640 1 -- 172.21.15.50:0/670044457 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b741fcca0 con 0x7f3b741527b0 2024-09-21T05:56:59.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b79600640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b740b4de0 0x7f3b74154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:59.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7a000640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b741527b0 0x7f3b741f7d70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00caa0 tx=0x7f3b6c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:59.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b71600640 1 -- 172.21.15.50:0/670044457 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c00adf0 con 0x7f3b741527b0 2024-09-21T05:56:59.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b71600640 1 -- 172.21.15.50:0/670044457 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b6c003a70 con 0x7f3b741527b0 2024-09-21T05:56:59.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7c000640 1 -- 172.21.15.50:0/670044457 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b74163d40 con 0x7f3b741527b0 2024-09-21T05:56:59.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b71600640 1 -- 172.21.15.50:0/670044457 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c004030 con 0x7f3b741527b0 2024-09-21T05:56:59.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7c000640 1 -- 172.21.15.50:0/670044457 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b74164300 con 0x7f3b741527b0 2024-09-21T05:56:59.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.011+0000 7f3b7c000640 1 -- 172.21.15.50:0/670044457 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b741fca10 con 0x7f3b741527b0 2024-09-21T05:56:59.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.015+0000 7f3b71600640 1 -- 172.21.15.50:0/670044457 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3b6c0041d0 con 0x7f3b741527b0 2024-09-21T05:56:59.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.015+0000 7f3b71600640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b54073200 0x7f3b54075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:59.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.015+0000 7f3b71600640 1 -- 172.21.15.50:0/670044457 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3b6c094ad0 con 0x7f3b741527b0 2024-09-21T05:56:59.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.015+0000 7f3b79600640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b54073200 0x7f3b54075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:59.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.015+0000 7f3b79600640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b54073200 0x7f3b54075710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3b680af000 tx=0x7f3b6801b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:59.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.019+0000 7f3b71600640 1 -- 172.21.15.50:0/670044457 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3b6c014070 con 0x7f3b741527b0 2024-09-21T05:56:59.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.199+0000 7f3b7c000640 1 -- 172.21.15.50:0/670044457 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3b740b51f0 con 0x7f3b741527b0 2024-09-21T05:56:59.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b71600640 1 -- 172.21.15.50:0/670044457 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f3b6c068020 con 0x7f3b741527b0 2024-09-21T05:56:59.207 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-09-21T05:56:59.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b7c000640 1 -- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b54073200 msgr2=0x7f3b54075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:59.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b7c000640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b54073200 0x7f3b54075710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3b680af000 tx=0x7f3b6801b040 comp rx=0 tx=0).stop 2024-09-21T05:56:59.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b7c000640 1 -- 172.21.15.50:0/670044457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b741527b0 msgr2=0x7f3b741f7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:59.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b7c000640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b741527b0 0x7f3b741f7d70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00caa0 tx=0x7f3b6c00cf60 comp rx=0 tx=0).stop 2024-09-21T05:56:59.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b7c000640 1 -- 172.21.15.50:0/670044457 shutdown_connections 2024-09-21T05:56:59.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b7c000640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b54073200 0x7f3b54075710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:59.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b7c000640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b741527b0 0x7f3b741f7d70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:59.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b7c000640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b740b5c70 0x7f3b741fbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:59.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b7c000640 1 --2- 172.21.15.50:0/670044457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b740b4de0 0x7f3b74154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:59.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b7c000640 1 -- 172.21.15.50:0/670044457 >> 172.21.15.50:0/670044457 conn(0x7f3b740b2040 msgr2=0x7f3b74159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:59.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b7c000640 1 -- 172.21.15.50:0/670044457 shutdown_connections 2024-09-21T05:56:59.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.203+0000 7f3b7c000640 1 -- 172.21.15.50:0/670044457 wait complete. 2024-09-21T05:56:59.228 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-21T05:56:59.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.383+0000 7fc3ab800640 1 -- 172.21.15.50:0/1932346532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc38c004810 msgr2=0x7fc38c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:59.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.383+0000 7fc3ab800640 1 --2- 172.21.15.50:0/1932346532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc38c004810 0x7fc38c004c20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc39000c4a0 tx=0x7fc39001cf40 comp rx=0 tx=0).stop 2024-09-21T05:56:59.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.383+0000 7fc3ab800640 1 -- 172.21.15.50:0/1932346532 shutdown_connections 2024-09-21T05:56:59.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.383+0000 7fc3ab800640 1 --2- 172.21.15.50:0/1932346532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc38c004810 0x7fc38c004c20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:59.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.383+0000 7fc3ab800640 1 -- 172.21.15.50:0/1932346532 >> 172.21.15.50:0/1932346532 conn(0x7fc3a40b35f0 msgr2=0x7fc3a40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:59.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.383+0000 7fc3ab800640 1 -- 172.21.15.50:0/1932346532 shutdown_connections 2024-09-21T05:56:59.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.383+0000 7fc3ab800640 1 -- 172.21.15.50:0/1932346532 wait complete. 2024-09-21T05:56:59.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3ab800640 1 Processor -- start 2024-09-21T05:56:59.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3ab800640 1 -- start start 2024-09-21T05:56:59.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3ab800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc38c004810 0x7fc3a41456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3ab800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3a40cf420 0x7fc3a4068df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3ab800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3a4069330 0x7fc3a41f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3ab800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a4070000 con 0x7fc38c004810 2024-09-21T05:56:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3ab800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a406fe80 con 0x7fc3a40cf420 2024-09-21T05:56:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3ab800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a4070180 con 0x7fc3a4069330 2024-09-21T05:56:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3a8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc38c004810 0x7fc3a41456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3a8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc38c004810 0x7fc3a41456d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48806/0 (socket says 172.21.15.50:48806) 2024-09-21T05:56:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3a8e00640 1 -- 172.21.15.50:0/3516502580 learned_addr learned my addr 172.21.15.50:0/3516502580 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:56:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3a3e00640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3a40cf420 0x7fc3a4068df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3a9800640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3a4069330 0x7fc3a41f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3a8e00640 1 -- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3a4069330 msgr2=0x7fc3a41f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3a8e00640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3a4069330 0x7fc3a41f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:59.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3a8e00640 1 -- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3a40cf420 msgr2=0x7fc3a4068df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:59.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3a8e00640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3a40cf420 0x7fc3a4068df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:59.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3a8e00640 1 -- 172.21.15.50:0/3516502580 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a4069740 con 0x7fc38c004810 2024-09-21T05:56:59.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.387+0000 7fc3a8e00640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc38c004810 0x7fc3a41456d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc398011ab0 tx=0x7fc398011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:59.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.391+0000 7fc3a0c00640 1 -- 172.21.15.50:0/3516502580 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39801e070 con 0x7fc38c004810 2024-09-21T05:56:59.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.391+0000 7fc3a0c00640 1 -- 172.21.15.50:0/3516502580 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc39800fdf0 con 0x7fc38c004810 2024-09-21T05:56:59.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.391+0000 7fc3ab800640 1 -- 172.21.15.50:0/3516502580 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a41f7eb0 con 0x7fc38c004810 2024-09-21T05:56:59.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.391+0000 7fc3a0c00640 1 -- 172.21.15.50:0/3516502580 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398004290 con 0x7fc38c004810 2024-09-21T05:56:59.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.391+0000 7fc3ab800640 1 -- 172.21.15.50:0/3516502580 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a40cec20 con 0x7fc38c004810 2024-09-21T05:56:59.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.391+0000 7fc3ab800640 1 -- 172.21.15.50:0/3516502580 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a415fcc0 con 0x7fc38c004810 2024-09-21T05:56:59.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.391+0000 7fc3a0c00640 1 -- 172.21.15.50:0/3516502580 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc398021070 con 0x7fc38c004810 2024-09-21T05:56:59.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.391+0000 7fc3a0c00640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc384073200 0x7fc384075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:59.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.395+0000 7fc3a0c00640 1 -- 172.21.15.50:0/3516502580 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc398094b20 con 0x7fc38c004810 2024-09-21T05:56:59.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.395+0000 7fc3a3e00640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc384073200 0x7fc384075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:59.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.395+0000 7fc3a3e00640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc384073200 0x7fc384075710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc3900aa000 tx=0x7fc3900b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:59.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.399+0000 7fc3a0c00640 1 -- 172.21.15.50:0/3516502580 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc3a4069740 con 0x7fc38c004810 2024-09-21T05:56:59.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:56:59.635+0000 7fc3ab800640 1 -- 172.21.15.50:0/3516502580 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7fc3a415ed40 con 0x7fc38c004810 2024-09-21T05:57:00.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.027+0000 7fc3a0c00640 1 -- 172.21.15.50:0/3516502580 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v53) ==== 187+0+0 (secure 0 0 0) 0x7fc39805f570 con 0x7fc38c004810 2024-09-21T05:57:00.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.031+0000 7fc3ab800640 1 -- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc384073200 msgr2=0x7fc384075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.031+0000 7fc3ab800640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc384073200 0x7fc384075710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc3900aa000 tx=0x7fc3900b1040 comp rx=0 tx=0).stop 2024-09-21T05:57:00.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.031+0000 7fc3ab800640 1 -- 172.21.15.50:0/3516502580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc38c004810 msgr2=0x7fc3a41456d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.031+0000 7fc3ab800640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc38c004810 0x7fc3a41456d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc398011ab0 tx=0x7fc398011f70 comp rx=0 tx=0).stop 2024-09-21T05:57:00.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.031+0000 7fc3ab800640 1 -- 172.21.15.50:0/3516502580 shutdown_connections 2024-09-21T05:57:00.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.031+0000 7fc3ab800640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3a4069330 0x7fc3a41f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.031+0000 7fc3ab800640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3a40cf420 0x7fc3a4068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.031+0000 7fc3ab800640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc384073200 0x7fc384075710 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.031+0000 7fc3ab800640 1 --2- 172.21.15.50:0/3516502580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc38c004810 0x7fc3a41456d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.031+0000 7fc3ab800640 1 -- 172.21.15.50:0/3516502580 >> 172.21.15.50:0/3516502580 conn(0x7fc3a40b35f0 msgr2=0x7fc3a415a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:00.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.031+0000 7fc3ab800640 1 -- 172.21.15.50:0/3516502580 shutdown_connections 2024-09-21T05:57:00.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.031+0000 7fc3ab800640 1 -- 172.21.15.50:0/3516502580 wait complete. 2024-09-21T05:57:00.056 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T05:57:00.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.211+0000 7f7ab5200640 1 -- 172.21.15.50:0/4291790558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ab0157030 msgr2=0x7f7ab00b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.211+0000 7f7ab5200640 1 --2- 172.21.15.50:0/4291790558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ab0157030 0x7f7ab00b5a80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009840 tx=0x7f7aa001c820 comp rx=0 tx=0).stop 2024-09-21T05:57:00.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.211+0000 7f7ab5200640 1 -- 172.21.15.50:0/4291790558 shutdown_connections 2024-09-21T05:57:00.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.211+0000 7f7ab5200640 1 --2- 172.21.15.50:0/4291790558 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ab0160d50 0x7f7ab0074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.211+0000 7f7ab5200640 1 --2- 172.21.15.50:0/4291790558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ab0157030 0x7f7ab00b5a80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.211+0000 7f7ab5200640 1 -- 172.21.15.50:0/4291790558 >> 172.21.15.50:0/4291790558 conn(0x7f7ab00708c0 msgr2=0x7f7ab0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:00.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.211+0000 7f7ab5200640 1 -- 172.21.15.50:0/4291790558 shutdown_connections 2024-09-21T05:57:00.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.211+0000 7f7ab5200640 1 -- 172.21.15.50:0/4291790558 wait complete. 2024-09-21T05:57:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.211+0000 7f7ab5200640 1 Processor -- start 2024-09-21T05:57:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.211+0000 7f7ab5200640 1 -- start start 2024-09-21T05:57:00.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7ab5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ab0157030 0x7f7ab0152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7ab5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ab0160d50 0x7f7ab0155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7ab5200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ab0161160 0x7f7ab00d1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7ab5200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab016a7d0 con 0x7f7ab0157030 2024-09-21T05:57:00.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7ab5200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab016a650 con 0x7f7ab0160d50 2024-09-21T05:57:00.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7ab5200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab016a950 con 0x7f7ab0161160 2024-09-21T05:57:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aade00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ab0157030 0x7f7ab0152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aa7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ab0160d50 0x7f7ab0155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aade00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ab0157030 0x7f7ab0152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48822/0 (socket says 172.21.15.50:48822) 2024-09-21T05:57:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aade00640 1 -- 172.21.15.50:0/79696573 learned_addr learned my addr 172.21.15.50:0/79696573 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aae800640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ab0161160 0x7f7ab00d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aa7e00640 1 -- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ab0161160 msgr2=0x7f7ab00d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aa7e00640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ab0161160 0x7f7ab00d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aa7e00640 1 -- 172.21.15.50:0/79696573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ab0157030 msgr2=0x7f7ab0152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aa7e00640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ab0157030 0x7f7ab0152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aa7e00640 1 -- 172.21.15.50:0/79696573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab01f9920 con 0x7f7ab0160d50 2024-09-21T05:57:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aade00640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ab0157030 0x7f7ab0152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aa7e00640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ab0160d50 0x7f7ab0155f50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00b1740 tx=0x7f7a9800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:00.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aa6000640 1 -- 172.21.15.50:0/79696573 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9800cd00 con 0x7f7ab0160d50 2024-09-21T05:57:00.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aa6000640 1 -- 172.21.15.50:0/79696573 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a98004020 con 0x7f7ab0160d50 2024-09-21T05:57:00.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7aa6000640 1 -- 172.21.15.50:0/79696573 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98004470 con 0x7f7ab0160d50 2024-09-21T05:57:00.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7ab5200640 1 -- 172.21.15.50:0/79696573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab01f9c00 con 0x7f7ab0160d50 2024-09-21T05:57:00.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7ab5200640 1 -- 172.21.15.50:0/79696573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab01fa1c0 con 0x7f7ab0160d50 2024-09-21T05:57:00.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.215+0000 7f7ab5200640 1 -- 172.21.15.50:0/79696573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab00d2260 con 0x7f7ab0160d50 2024-09-21T05:57:00.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.219+0000 7f7aa6000640 1 -- 172.21.15.50:0/79696573 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7a98003600 con 0x7f7ab0160d50 2024-09-21T05:57:00.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.219+0000 7f7aa6000640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a8c073230 0x7f7a8c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.219+0000 7f7aa6000640 1 -- 172.21.15.50:0/79696573 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7a98094bd0 con 0x7f7ab0160d50 2024-09-21T05:57:00.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.219+0000 7f7aade00640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a8c073230 0x7f7a8c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.219+0000 7f7aade00640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a8c073230 0x7f7a8c075740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7aa001a450 tx=0x7f7aa001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:00.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.223+0000 7f7aa6000640 1 -- 172.21.15.50:0/79696573 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7a9805f610 con 0x7f7ab0160d50 2024-09-21T05:57:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.399+0000 7f7ab5200640 1 -- 172.21.15.50:0/79696573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f7ab00b2940 con 0x7f7ab0160d50 2024-09-21T05:57:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.399+0000 7f7aa6000640 1 -- 172.21.15.50:0/79696573 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7f7a98064040 con 0x7f7ab0160d50 2024-09-21T05:57:00.404 INFO:teuthology.orchestra.run.smithi050.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T05:57:00.404 INFO:teuthology.orchestra.run.smithi050.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T05:57:00.404 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced client_mount_timeout 30 2024-09-21T05:57:00.404 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_client 20/20 2024-09-21T05:57:00.404 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_mgr 20/20 2024-09-21T05:57:00.404 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_ms 1/1 2024-09-21T05:57:00.404 INFO:teuthology.orchestra.run.smithi050.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3735.149299 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4471.400731 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3640.131623 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3663.645087 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3762.055672 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3661.558528 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3661.456695 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3796.187852 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds 20/20 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_ms 2/2 2024-09-21T05:57:00.405 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_frag true 2024-09-21T05:57:00.406 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_scatterstat true 2024-09-21T05:57:00.406 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T05:57:00.406 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_verify_scatter true 2024-09-21T05:57:00.406 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T05:57:00.406 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T05:57:00.406 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T05:57:00.406 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced client_mount_timeout 600 2024-09-21T05:57:00.406 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_client 20/20 2024-09-21T05:57:00.406 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_ms 1/1 2024-09-21T05:57:00.406 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T05:57:00.406 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T05:57:00.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.403+0000 7f7ab5200640 1 -- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a8c073230 msgr2=0x7f7a8c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.403+0000 7f7ab5200640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a8c073230 0x7f7a8c075740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7aa001a450 tx=0x7f7aa001b060 comp rx=0 tx=0).stop 2024-09-21T05:57:00.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.403+0000 7f7ab5200640 1 -- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ab0160d50 msgr2=0x7f7ab0155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.403+0000 7f7ab5200640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ab0160d50 0x7f7ab0155f50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00b1740 tx=0x7f7a9800ee50 comp rx=0 tx=0).stop 2024-09-21T05:57:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.403+0000 7f7ab5200640 1 -- 172.21.15.50:0/79696573 shutdown_connections 2024-09-21T05:57:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.403+0000 7f7ab5200640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ab0161160 0x7f7ab00d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.403+0000 7f7ab5200640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ab0160d50 0x7f7ab0155f50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.403+0000 7f7ab5200640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a8c073230 0x7f7a8c075740 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.403+0000 7f7ab5200640 1 --2- 172.21.15.50:0/79696573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ab0157030 0x7f7ab0152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.403+0000 7f7ab5200640 1 -- 172.21.15.50:0/79696573 >> 172.21.15.50:0/79696573 conn(0x7f7ab00708c0 msgr2=0x7f7ab015c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.403+0000 7f7ab5200640 1 -- 172.21.15.50:0/79696573 shutdown_connections 2024-09-21T05:57:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.403+0000 7f7ab5200640 1 -- 172.21.15.50:0/79696573 wait complete. 2024-09-21T05:57:00.426 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T05:57:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.583+0000 7fe8a5200640 1 -- 172.21.15.50:0/1121260062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a0160470 msgr2=0x7fe8a0074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.583+0000 7fe8a5200640 1 --2- 172.21.15.50:0/1121260062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a0160470 0x7fe8a0074780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe898009bc0 tx=0x7fe89801cb10 comp rx=0 tx=0).stop 2024-09-21T05:57:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.583+0000 7fe8a5200640 1 -- 172.21.15.50:0/1121260062 shutdown_connections 2024-09-21T05:57:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.583+0000 7fe8a5200640 1 --2- 172.21.15.50:0/1121260062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a0160470 0x7fe8a0074780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.583+0000 7fe8a5200640 1 --2- 172.21.15.50:0/1121260062 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8a0160060 0x7fe8a0074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.583+0000 7fe8a5200640 1 -- 172.21.15.50:0/1121260062 >> 172.21.15.50:0/1121260062 conn(0x7fe8a00b4560 msgr2=0x7fe8a00d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.583+0000 7fe8a5200640 1 -- 172.21.15.50:0/1121260062 shutdown_connections 2024-09-21T05:57:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.583+0000 7fe8a5200640 1 -- 172.21.15.50:0/1121260062 wait complete. 2024-09-21T05:57:00.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe8a5200640 1 Processor -- start 2024-09-21T05:57:00.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe8a5200640 1 -- start start 2024-09-21T05:57:00.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe8a5200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8a01536f0 0x7fe8a0152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe8a5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8a0160060 0x7fe8a0148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe8a5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a0160470 0x7fe8a0150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe8a5200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01696e0 con 0x7fe8a0160060 2024-09-21T05:57:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe8a5200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0169560 con 0x7fe8a0160470 2024-09-21T05:57:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe8a5200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a0169860 con 0x7fe8a01536f0 2024-09-21T05:57:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe897e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8a0160060 0x7fe8a0148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe89fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a0160470 0x7fe8a0150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe89f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8a01536f0 0x7fe8a0152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe897e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8a0160060 0x7fe8a0148dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48828/0 (socket says 172.21.15.50:48828) 2024-09-21T05:57:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe897e00640 1 -- 172.21.15.50:0/1837987689 learned_addr learned my addr 172.21.15.50:0/1837987689 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe897e00640 1 -- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8a01536f0 msgr2=0x7fe8a0152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe897e00640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8a01536f0 0x7fe8a0152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe897e00640 1 -- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a0160470 msgr2=0x7fe8a0150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe897e00640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a0160470 0x7fe8a0150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe897e00640 1 -- 172.21.15.50:0/1837987689 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a0149310 con 0x7fe8a0160060 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe89f400640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8a01536f0 0x7fe8a0152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.587+0000 7fe89fe00640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a0160470 0x7fe8a0150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.591+0000 7fe897e00640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8a0160060 0x7fe8a0148dd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe88800e9f0 tx=0x7fe88800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.591+0000 7fe897400640 1 -- 172.21.15.50:0/1837987689 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88800cd80 con 0x7fe8a0160060 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.591+0000 7fe897400640 1 -- 172.21.15.50:0/1837987689 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe888004020 con 0x7fe8a0160060 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.591+0000 7fe897400640 1 -- 172.21.15.50:0/1837987689 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888004470 con 0x7fe8a0160060 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.591+0000 7fe8a5200640 1 -- 172.21.15.50:0/1837987689 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a0151da0 con 0x7fe8a0160060 2024-09-21T05:57:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.591+0000 7fe8a5200640 1 -- 172.21.15.50:0/1837987689 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a0152360 con 0x7fe8a0160060 2024-09-21T05:57:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.591+0000 7fe8a5200640 1 -- 172.21.15.50:0/1837987689 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a015d480 con 0x7fe8a0160060 2024-09-21T05:57:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.591+0000 7fe897400640 1 -- 172.21.15.50:0/1837987689 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe888003600 con 0x7fe8a0160060 2024-09-21T05:57:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.591+0000 7fe897400640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe87c073200 0x7fe87c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.591+0000 7fe897400640 1 -- 172.21.15.50:0/1837987689 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe8880949d0 con 0x7fe8a0160060 2024-09-21T05:57:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.595+0000 7fe89f400640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe87c073200 0x7fe87c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.595+0000 7fe89f400640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe87c073200 0x7fe87c075710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe890001b70 tx=0x7fe890009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:00.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.599+0000 7fe897400640 1 -- 172.21.15.50:0/1837987689 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe88805f420 con 0x7fe8a0160060 2024-09-21T05:57:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.775+0000 7fe8a5200640 1 -- 172.21.15.50:0/1837987689 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe8a0151b30 con 0x7fe8a0160060 2024-09-21T05:57:00.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.779+0000 7fe897400640 1 -- 172.21.15.50:0/1837987689 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fe888063e50 con 0x7fe8a0160060 2024-09-21T05:57:00.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.783+0000 7fe8a5200640 1 -- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe87c073200 msgr2=0x7fe87c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.783+0000 7fe8a5200640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe87c073200 0x7fe87c075710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe890001b70 tx=0x7fe890009040 comp rx=0 tx=0).stop 2024-09-21T05:57:00.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.783+0000 7fe8a5200640 1 -- 172.21.15.50:0/1837987689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8a0160060 msgr2=0x7fe8a0148dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.783+0000 7fe8a5200640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8a0160060 0x7fe8a0148dd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe88800e9f0 tx=0x7fe88800edf0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.783+0000 7fe8a5200640 1 -- 172.21.15.50:0/1837987689 shutdown_connections 2024-09-21T05:57:00.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.783+0000 7fe8a5200640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a0160470 0x7fe8a0150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.783+0000 7fe8a5200640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe87c073200 0x7fe87c075710 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.783+0000 7fe8a5200640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8a0160060 0x7fe8a0148dd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.783+0000 7fe8a5200640 1 --2- 172.21.15.50:0/1837987689 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8a01536f0 0x7fe8a0152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.783+0000 7fe8a5200640 1 -- 172.21.15.50:0/1837987689 >> 172.21.15.50:0/1837987689 conn(0x7fe8a00b4560 msgr2=0x7fe8a0072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:00.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.783+0000 7fe8a5200640 1 -- 172.21.15.50:0/1837987689 shutdown_connections 2024-09-21T05:57:00.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:00.783+0000 7fe8a5200640 1 -- 172.21.15.50:0/1837987689 wait complete. 2024-09-21T05:57:00.808 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:57:00.809 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:57:00.908 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:57:01.005 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:57:01.005 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:57:01.006 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:01.015 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:57:01.015 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:01.065 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:57:01.065 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:01.117 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:57:01.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.315+0000 7fa533000640 1 -- 172.21.15.50:0/2020831947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa52c0cd970 msgr2=0x7fa52c0b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.315+0000 7fa533000640 1 --2- 172.21.15.50:0/2020831947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa52c0cd970 0x7fa52c0b3340 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa514009900 tx=0x7fa51401c8e0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.315+0000 7fa533000640 1 -- 172.21.15.50:0/2020831947 shutdown_connections 2024-09-21T05:57:01.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.315+0000 7fa533000640 1 --2- 172.21.15.50:0/2020831947 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa52c0cdd80 0x7fa52c0b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.315+0000 7fa533000640 1 --2- 172.21.15.50:0/2020831947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa52c0cd970 0x7fa52c0b3340 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.315+0000 7fa533000640 1 -- 172.21.15.50:0/2020831947 >> 172.21.15.50:0/2020831947 conn(0x7fa52c0b2040 msgr2=0x7fa52c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:01.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.315+0000 7fa533000640 1 -- 172.21.15.50:0/2020831947 shutdown_connections 2024-09-21T05:57:01.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.315+0000 7fa533000640 1 -- 172.21.15.50:0/2020831947 wait complete. 2024-09-21T05:57:01.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa533000640 1 Processor -- start 2024-09-21T05:57:01.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa533000640 1 -- start start 2024-09-21T05:57:01.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa533000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa52c0cd970 0x7fa52c1443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa533000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa52c0cdd80 0x7fa52c13da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa533000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa52c15aa50 0x7fa52c15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa533000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c1660c0 con 0x7fa52c15aa50 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa533000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c165f40 con 0x7fa52c0cdd80 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa533000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa52c166240 con 0x7fa52c0cd970 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa531000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa52c15aa50 0x7fa52c15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa531000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa52c15aa50 0x7fa52c15cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48848/0 (socket says 172.21.15.50:48848) 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa531000640 1 -- 172.21.15.50:0/2617146589 learned_addr learned my addr 172.21.15.50:0/2617146589 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa523e00640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa52c0cdd80 0x7fa52c13da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa52be00640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa52c0cd970 0x7fa52c1443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa531000640 1 -- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa52c0cd970 msgr2=0x7fa52c1443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa531000640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa52c0cd970 0x7fa52c1443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa531000640 1 -- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa52c0cdd80 msgr2=0x7fa52c13da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa531000640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa52c0cdd80 0x7fa52c13da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa531000640 1 -- 172.21.15.50:0/2617146589 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c1520a0 con 0x7fa52c15aa50 2024-09-21T05:57:01.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa531000640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa52c15aa50 0x7fa52c15cfb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa52400e9a0 tx=0x7fa52400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:01.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa528c00640 1 -- 172.21.15.50:0/2617146589 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524004d30 con 0x7fa52c15aa50 2024-09-21T05:57:01.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa528c00640 1 -- 172.21.15.50:0/2617146589 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa52400cb20 con 0x7fa52c15aa50 2024-09-21T05:57:01.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa528c00640 1 -- 172.21.15.50:0/2617146589 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524005020 con 0x7fa52c15aa50 2024-09-21T05:57:01.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa533000640 1 -- 172.21.15.50:0/2617146589 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c152380 con 0x7fa52c15aa50 2024-09-21T05:57:01.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.319+0000 7fa533000640 1 -- 172.21.15.50:0/2617146589 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c152940 con 0x7fa52c15aa50 2024-09-21T05:57:01.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.323+0000 7fa533000640 1 -- 172.21.15.50:0/2617146589 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c0b3550 con 0x7fa52c15aa50 2024-09-21T05:57:01.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.323+0000 7fa528c00640 1 -- 172.21.15.50:0/2617146589 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa5240051c0 con 0x7fa52c15aa50 2024-09-21T05:57:01.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.323+0000 7fa528c00640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa508073200 0x7fa508075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.323+0000 7fa528c00640 1 -- 172.21.15.50:0/2617146589 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa524094a80 con 0x7fa52c15aa50 2024-09-21T05:57:01.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.323+0000 7fa52be00640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa508073200 0x7fa508075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.323+0000 7fa52be00640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa508073200 0x7fa508075710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa51c007fd0 tx=0x7fa51c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:01.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.327+0000 7fa528c00640 1 -- 172.21.15.50:0/2617146589 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa524096e20 con 0x7fa52c15aa50 2024-09-21T05:57:01.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 -- 172.21.15.50:0/2617146589 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa52c151e30 con 0x7fa52c15aa50 2024-09-21T05:57:01.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa528c00640 1 -- 172.21.15.50:0/2617146589 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fa524016030 con 0x7fa52c15aa50 2024-09-21T05:57:01.519 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T05:57:01.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 -- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa508073200 msgr2=0x7fa508075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa508073200 0x7fa508075710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa51c007fd0 tx=0x7fa51c009040 comp rx=0 tx=0).stop 2024-09-21T05:57:01.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 -- 172.21.15.50:0/2617146589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa52c15aa50 msgr2=0x7fa52c15cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa52c15aa50 0x7fa52c15cfb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa52400e9a0 tx=0x7fa52400ee60 comp rx=0 tx=0).stop 2024-09-21T05:57:01.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 -- 172.21.15.50:0/2617146589 shutdown_connections 2024-09-21T05:57:01.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa508073200 0x7fa508075710 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa52c15aa50 0x7fa52c15cfb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa52c0cdd80 0x7fa52c13da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 --2- 172.21.15.50:0/2617146589 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa52c0cd970 0x7fa52c1443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 -- 172.21.15.50:0/2617146589 >> 172.21.15.50:0/2617146589 conn(0x7fa52c0b2040 msgr2=0x7fa52c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:01.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 -- 172.21.15.50:0/2617146589 shutdown_connections 2024-09-21T05:57:01.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.515+0000 7fa533000640 1 -- 172.21.15.50:0/2617146589 wait complete. 2024-09-21T05:57:01.541 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:57:01.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:01.541 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T05:57:01.577 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T05:57:01.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:01.577 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:57:01.642 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:57:01.642 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:01.651 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:57:01.651 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:01.702 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:57:01.703 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:01.754 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:57:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.915+0000 7f4275200640 1 -- 172.21.15.50:0/1093542084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270160060 msgr2=0x7f4270074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.915+0000 7f4275200640 1 --2- 172.21.15.50:0/1093542084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270160060 0x7f4270074240 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4258009840 tx=0x7f425801c5b0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.915+0000 7f4275200640 1 -- 172.21.15.50:0/1093542084 shutdown_connections 2024-09-21T05:57:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.915+0000 7f4275200640 1 --2- 172.21.15.50:0/1093542084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270160060 0x7f4270074240 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.915+0000 7f4275200640 1 -- 172.21.15.50:0/1093542084 >> 172.21.15.50:0/1093542084 conn(0x7f42700b4560 msgr2=0x7f42700d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.915+0000 7f4275200640 1 -- 172.21.15.50:0/1093542084 shutdown_connections 2024-09-21T05:57:01.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.915+0000 7f4275200640 1 -- 172.21.15.50:0/1093542084 wait complete. 2024-09-21T05:57:01.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.915+0000 7f4275200640 1 Processor -- start 2024-09-21T05:57:01.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.915+0000 7f4275200640 1 -- start start 2024-09-21T05:57:01.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.915+0000 7f4275200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42701536f0 0x7f4270147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.915+0000 7f4275200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4270160060 0x7f42701fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4275200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4270160470 0x7f427006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4275200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f427016b520 con 0x7f42701536f0 2024-09-21T05:57:01.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4275200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f427016b3a0 con 0x7f4270160470 2024-09-21T05:57:01.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4275200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f427016b6a0 con 0x7f4270160060 2024-09-21T05:57:01.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f426f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42701536f0 0x7f4270147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f426f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42701536f0 0x7f4270147fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48858/0 (socket says 172.21.15.50:48858) 2024-09-21T05:57:01.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f426f400640 1 -- 172.21.15.50:0/2964747757 learned_addr learned my addr 172.21.15.50:0/2964747757 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:01.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4266a00640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4270160060 0x7f42701fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f426f400640 1 -- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4270160060 msgr2=0x7f42701fcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f426f400640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4270160060 0x7f42701fcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f426fe00640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4270160470 0x7f427006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f426f400640 1 -- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4270160470 msgr2=0x7f427006dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f426f400640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4270160470 0x7f427006dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f426f400640 1 -- 172.21.15.50:0/2964747757 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42700ced70 con 0x7f42701536f0 2024-09-21T05:57:01.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4266a00640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4270160060 0x7f42701fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:01.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f426fe00640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4270160470 0x7f427006dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:01.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f426f400640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42701536f0 0x7f4270147fe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f426000c3f0 tx=0x7f426000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:01.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4267e00640 1 -- 172.21.15.50:0/2964747757 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260017070 con 0x7f42701536f0 2024-09-21T05:57:01.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4267e00640 1 -- 172.21.15.50:0/2964747757 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4260003600 con 0x7f42701536f0 2024-09-21T05:57:01.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4267e00640 1 -- 172.21.15.50:0/2964747757 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260003a50 con 0x7f42701536f0 2024-09-21T05:57:01.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4275200640 1 -- 172.21.15.50:0/2964747757 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427006cf20 con 0x7f42701536f0 2024-09-21T05:57:01.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4275200640 1 -- 172.21.15.50:0/2964747757 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427006d4e0 con 0x7f42701536f0 2024-09-21T05:57:01.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4275200640 1 -- 172.21.15.50:0/2964747757 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4270074240 con 0x7f42701536f0 2024-09-21T05:57:01.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.919+0000 7f4267e00640 1 -- 172.21.15.50:0/2964747757 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f426001a070 con 0x7f42701536f0 2024-09-21T05:57:01.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.923+0000 7f4267e00640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f424c073230 0x7f424c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.923+0000 7f4267e00640 1 -- 172.21.15.50:0/2964747757 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f42600934c0 con 0x7f42701536f0 2024-09-21T05:57:01.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.923+0000 7f4266a00640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f424c073230 0x7f424c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.923+0000 7f4266a00640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f424c073230 0x7f424c075740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f42580ae000 tx=0x7f425801adf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:01.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:01.927+0000 7f4267e00640 1 -- 172.21.15.50:0/2964747757 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f426005e4d0 con 0x7f42701536f0 2024-09-21T05:57:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 -- 172.21.15.50:0/2964747757 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f42701532a0 con 0x7f42701536f0 2024-09-21T05:57:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4267e00640 1 -- 172.21.15.50:0/2964747757 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f426005ddd0 con 0x7f42701536f0 2024-09-21T05:57:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T05:57:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 -- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f424c073230 msgr2=0x7f424c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f424c073230 0x7f424c075740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f42580ae000 tx=0x7f425801adf0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 -- 172.21.15.50:0/2964747757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42701536f0 msgr2=0x7f4270147fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42701536f0 0x7f4270147fe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f426000c3f0 tx=0x7f426000c8b0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 -- 172.21.15.50:0/2964747757 shutdown_connections 2024-09-21T05:57:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4270160470 0x7f427006dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4270160060 0x7f42701fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f424c073230 0x7f424c075740 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 --2- 172.21.15.50:0/2964747757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42701536f0 0x7f4270147fe0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 -- 172.21.15.50:0/2964747757 >> 172.21.15.50:0/2964747757 conn(0x7f42700b4560 msgr2=0x7f42700b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 -- 172.21.15.50:0/2964747757 shutdown_connections 2024-09-21T05:57:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.115+0000 7f4275200640 1 -- 172.21.15.50:0/2964747757 wait complete. 2024-09-21T05:57:02.141 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T05:57:02.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:02.141 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-21T05:57:02.181 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T05:57:02.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:02.181 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T05:57:02.241 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:57:02.241 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:02.250 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:57:02.251 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:02.301 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:57:02.301 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:02.357 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:57:02.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.555+0000 7f2163000640 1 -- 172.21.15.50:0/1653678732 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f215c1527b0 msgr2=0x7f215c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:02.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.555+0000 7f2163000640 1 --2- 172.21.15.50:0/1653678732 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f215c1527b0 0x7f215c15bb60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2144009900 tx=0x7f214401c820 comp rx=0 tx=0).stop 2024-09-21T05:57:02.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.555+0000 7f2163000640 1 -- 172.21.15.50:0/1653678732 shutdown_connections 2024-09-21T05:57:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.555+0000 7f2163000640 1 --2- 172.21.15.50:0/1653678732 >> v2:172.21.15.88:3300/0 conn(0x7f215c0b4de0 0x7f215c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.555+0000 7f2163000640 1 --2- 172.21.15.50:0/1653678732 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f215c1527b0 0x7f215c15bb60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.555+0000 7f2163000640 1 -- 172.21.15.50:0/1653678732 >> 172.21.15.50:0/1653678732 conn(0x7f215c0b2040 msgr2=0x7f215c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.555+0000 7f2163000640 1 -- 172.21.15.50:0/1653678732 shutdown_connections 2024-09-21T05:57:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.555+0000 7f2163000640 1 -- 172.21.15.50:0/1653678732 wait complete. 2024-09-21T05:57:02.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2163000640 1 Processor -- start 2024-09-21T05:57:02.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2163000640 1 -- start start 2024-09-21T05:57:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2163000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f215c0b4de0 0x7f215c14e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2163000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f215c0b5c70 0x7f215c1f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2163000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f215c1527b0 0x7f215c0ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2163000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1689b0 con 0x7f215c0b4de0 2024-09-21T05:57:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2163000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f215c168830 con 0x7f215c1527b0 2024-09-21T05:57:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2163000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f215c168b30 con 0x7f215c0b5c70 2024-09-21T05:57:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f215be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f215c0b4de0 0x7f215c14e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f215be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f215c0b4de0 0x7f215c14e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48862/0 (socket says 172.21.15.50:48862) 2024-09-21T05:57:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f215be00640 1 -- 172.21.15.50:0/3267309392 learned_addr learned my addr 172.21.15.50:0/3267309392 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2153400640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f215c0b5c70 0x7f215c1f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2161000640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f215c1527b0 0x7f215c0ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f215be00640 1 -- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f215c0b5c70 msgr2=0x7f215c1f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f215be00640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f215c0b5c70 0x7f215c1f66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f215be00640 1 -- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f215c1527b0 msgr2=0x7f215c0ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f215be00640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f215c1527b0 0x7f215c0ab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f215be00640 1 -- 172.21.15.50:0/3267309392 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c1f6c30 con 0x7f215c0b4de0 2024-09-21T05:57:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2161000640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f215c1527b0 0x7f215c0ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f215be00640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f215c0b4de0 0x7f215c14e920 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f214c00c760 tx=0x7f214c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2158c00640 1 -- 172.21.15.50:0/3267309392 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c0052d0 con 0x7f215c0b4de0 2024-09-21T05:57:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2158c00640 1 -- 172.21.15.50:0/3267309392 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f214c001d20 con 0x7f215c0b4de0 2024-09-21T05:57:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2163000640 1 -- 172.21.15.50:0/3267309392 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c1f76e0 con 0x7f215c0b4de0 2024-09-21T05:57:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2158c00640 1 -- 172.21.15.50:0/3267309392 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c00a880 con 0x7f215c0b4de0 2024-09-21T05:57:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.559+0000 7f2163000640 1 -- 172.21.15.50:0/3267309392 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c1f7400 con 0x7f215c0b4de0 2024-09-21T05:57:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.563+0000 7f2163000640 1 -- 172.21.15.50:0/3267309392 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f215c0b51f0 con 0x7f215c0b4de0 2024-09-21T05:57:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.563+0000 7f2158c00640 1 -- 172.21.15.50:0/3267309392 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f214c00aa20 con 0x7f215c0b4de0 2024-09-21T05:57:02.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.563+0000 7f2158c00640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2138073200 0x7f2138075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:02.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.563+0000 7f2158c00640 1 -- 172.21.15.50:0/3267309392 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f214c093c60 con 0x7f215c0b4de0 2024-09-21T05:57:02.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.563+0000 7f2153400640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2138073200 0x7f2138075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:02.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.567+0000 7f2153400640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2138073200 0x7f2138075710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f21440af000 tx=0x7f214401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:02.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.567+0000 7f2158c00640 1 -- 172.21.15.50:0/3267309392 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f215c0b5730 con 0x7f215c0b4de0 2024-09-21T05:57:02.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 -- 172.21.15.50:0/3267309392 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f215c1516e0 con 0x7f215c0b4de0 2024-09-21T05:57:02.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2158c00640 1 -- 172.21.15.50:0/3267309392 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f214c05ed10 con 0x7f215c0b4de0 2024-09-21T05:57:02.763 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T05:57:02.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 -- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2138073200 msgr2=0x7f2138075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:02.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2138073200 0x7f2138075710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f21440af000 tx=0x7f214401b060 comp rx=0 tx=0).stop 2024-09-21T05:57:02.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 -- 172.21.15.50:0/3267309392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f215c0b4de0 msgr2=0x7f215c14e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:02.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f215c0b4de0 0x7f215c14e920 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f214c00c760 tx=0x7f214c00cc20 comp rx=0 tx=0).stop 2024-09-21T05:57:02.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 -- 172.21.15.50:0/3267309392 shutdown_connections 2024-09-21T05:57:02.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f215c1527b0 0x7f215c0ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f215c0b5c70 0x7f215c1f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2138073200 0x7f2138075710 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 --2- 172.21.15.50:0/3267309392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f215c0b4de0 0x7f215c14e920 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:02.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 -- 172.21.15.50:0/3267309392 >> 172.21.15.50:0/3267309392 conn(0x7f215c0b2040 msgr2=0x7f215c151c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:02.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 -- 172.21.15.50:0/3267309392 shutdown_connections 2024-09-21T05:57:02.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:02.759+0000 7f2163000640 1 -- 172.21.15.50:0/3267309392 wait complete. 2024-09-21T05:57:02.785 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-21T05:57:02.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:02.785 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-21T05:57:02.808 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T05:57:02.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:02.809 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-21T05:57:02.862 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:57:02.862 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:02.872 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:57:02.872 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:02.923 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:57:02.923 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:02.975 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:57:03.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.131+0000 7f0356a00640 1 -- 172.21.15.50:0/3626918380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03500b5c70 msgr2=0x7f035015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.131+0000 7f0356a00640 1 --2- 172.21.15.50:0/3626918380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03500b5c70 0x7f035015bb60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0340009900 tx=0x7f034001c8b0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.131+0000 7f0356a00640 1 -- 172.21.15.50:0/3626918380 shutdown_connections 2024-09-21T05:57:03.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.131+0000 7f0356a00640 1 --2- 172.21.15.50:0/3626918380 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f03500b4de0 0x7f03500b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.131+0000 7f0356a00640 1 --2- 172.21.15.50:0/3626918380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03500b5c70 0x7f035015bb60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.131+0000 7f0356a00640 1 -- 172.21.15.50:0/3626918380 >> 172.21.15.50:0/3626918380 conn(0x7f03500b2040 msgr2=0x7f03500b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:03.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.131+0000 7f0356a00640 1 -- 172.21.15.50:0/3626918380 shutdown_connections 2024-09-21T05:57:03.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.131+0000 7f0356a00640 1 -- 172.21.15.50:0/3626918380 wait complete. 2024-09-21T05:57:03.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.131+0000 7f0356a00640 1 Processor -- start 2024-09-21T05:57:03.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.131+0000 7f0356a00640 1 -- start start 2024-09-21T05:57:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0356a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03500b4de0 0x7f035013cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0356a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03500b5c70 0x7f03501f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0356a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f03501527b0 0x7f0350153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0356a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0350168a20 con 0x7f03500b5c70 2024-09-21T05:57:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0356a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f03501688a0 con 0x7f03500b4de0 2024-09-21T05:57:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0356a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0350168ba0 con 0x7f03501527b0 2024-09-21T05:57:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0347e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03500b5c70 0x7f03501f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f034fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03500b4de0 0x7f035013cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0347e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03500b5c70 0x7f03501f8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48872/0 (socket says 172.21.15.50:48872) 2024-09-21T05:57:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0347e00640 1 -- 172.21.15.50:0/2158871333 learned_addr learned my addr 172.21.15.50:0/2158871333 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0354a00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f03501527b0 0x7f0350153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0347e00640 1 -- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f03501527b0 msgr2=0x7f0350153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0347e00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f03501527b0 0x7f0350153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0347e00640 1 -- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03500b4de0 msgr2=0x7f035013cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0347e00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03500b4de0 0x7f035013cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0347e00640 1 -- 172.21.15.50:0/2158871333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035013d100 con 0x7f03500b5c70 2024-09-21T05:57:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0347e00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03500b5c70 0x7f03501f8a90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f033800c3f0 tx=0x7f033800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f034cc00640 1 -- 172.21.15.50:0/2158871333 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338017070 con 0x7f03500b5c70 2024-09-21T05:57:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f034cc00640 1 -- 172.21.15.50:0/2158871333 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03380053c0 con 0x7f03500b5c70 2024-09-21T05:57:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f034cc00640 1 -- 172.21.15.50:0/2158871333 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338003860 con 0x7f03500b5c70 2024-09-21T05:57:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0356a00640 1 -- 172.21.15.50:0/2158871333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03501f94c0 con 0x7f03500b5c70 2024-09-21T05:57:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0356a00640 1 -- 172.21.15.50:0/2158871333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0350152d30 con 0x7f03500b5c70 2024-09-21T05:57:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.135+0000 7f0356a00640 1 -- 172.21.15.50:0/2158871333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03500b51f0 con 0x7f03500b5c70 2024-09-21T05:57:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.139+0000 7f034cc00640 1 -- 172.21.15.50:0/2158871333 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f033801a070 con 0x7f03500b5c70 2024-09-21T05:57:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.139+0000 7f034cc00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f032c073200 0x7f032c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.139+0000 7f034cc00640 1 -- 172.21.15.50:0/2158871333 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0338093430 con 0x7f03500b5c70 2024-09-21T05:57:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.139+0000 7f034fe00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f032c073200 0x7f032c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.139+0000 7f034fe00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f032c073200 0x7f032c075710 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03400af000 tx=0x7f034001b0f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:03.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.143+0000 7f034cc00640 1 -- 172.21.15.50:0/2158871333 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f03500b5730 con 0x7f03500b5c70 2024-09-21T05:57:03.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 -- 172.21.15.50:0/2158871333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f03501511f0 con 0x7f03500b5c70 2024-09-21T05:57:03.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f034cc00640 1 -- 172.21.15.50:0/2158871333 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f033805ddf0 con 0x7f03500b5c70 2024-09-21T05:57:03.335 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T05:57:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 -- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f032c073200 msgr2=0x7f032c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f032c073200 0x7f032c075710 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03400af000 tx=0x7f034001b0f0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 -- 172.21.15.50:0/2158871333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03500b5c70 msgr2=0x7f03501f8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03500b5c70 0x7f03501f8a90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f033800c3f0 tx=0x7f033800c8b0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 -- 172.21.15.50:0/2158871333 shutdown_connections 2024-09-21T05:57:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f03501527b0 0x7f0350153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f032c073200 0x7f032c075710 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03500b5c70 0x7f03501f8a90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 --2- 172.21.15.50:0/2158871333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03500b4de0 0x7f035013cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 -- 172.21.15.50:0/2158871333 >> 172.21.15.50:0/2158871333 conn(0x7f03500b2040 msgr2=0x7f0350154e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:03.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 -- 172.21.15.50:0/2158871333 shutdown_connections 2024-09-21T05:57:03.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.331+0000 7f0356a00640 1 -- 172.21.15.50:0/2158871333 wait complete. 2024-09-21T05:57:03.358 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-21T05:57:03.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:03.358 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-21T05:57:03.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T05:57:03.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:03.385 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-21T05:57:03.439 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T05:57:03.440 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:57:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.595+0000 7fb499600640 1 -- 172.21.15.50:0/413936133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb478003a10 msgr2=0x7fb478003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.595+0000 7fb499600640 1 --2- 172.21.15.50:0/413936133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb478003a10 0x7fb478003e80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb48400c260 tx=0x7fb48401cf60 comp rx=0 tx=0).stop 2024-09-21T05:57:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.595+0000 7fb493400640 1 -- 172.21.15.50:0/413936133 reap_dead start 2024-09-21T05:57:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.595+0000 7fb499600640 1 -- 172.21.15.50:0/413936133 shutdown_connections 2024-09-21T05:57:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.595+0000 7fb499600640 1 -- 172.21.15.50:0/413936133 >> 172.21.15.50:0/413936133 conn(0x7fb4940d06e0 msgr2=0x7fb4940d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.595+0000 7fb499600640 1 -- 172.21.15.50:0/413936133 shutdown_connections 2024-09-21T05:57:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.595+0000 7fb499600640 1 -- 172.21.15.50:0/413936133 wait complete. 2024-09-21T05:57:03.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb499600640 1 Processor -- start 2024-09-21T05:57:03.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb499600640 1 -- start start 2024-09-21T05:57:03.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb499600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb478003a10 0x7fb494145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:03.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb499600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4941f9fc0 0x7fb49415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:03.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb499600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb494145970 0x7fb49415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb499600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb494167e30 con 0x7fb494145970 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb499600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb494167cb0 con 0x7fb478003a10 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb499600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb494167fb0 con 0x7fb4941f9fc0 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb493400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb494145970 0x7fb49415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb493400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb494145970 0x7fb49415f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48892/0 (socket says 172.21.15.50:48892) 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb493400640 1 -- 172.21.15.50:0/736980375 learned_addr learned my addr 172.21.15.50:0/736980375 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb492a00640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb478003a10 0x7fb494145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb48be00640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4941f9fc0 0x7fb49415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb493400640 1 -- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4941f9fc0 msgr2=0x7fb49415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb493400640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4941f9fc0 0x7fb49415eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb493400640 1 -- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb478003a10 msgr2=0x7fb494145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb493400640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb478003a10 0x7fb494145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb493400640 1 -- 172.21.15.50:0/736980375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4940b56a0 con 0x7fb494145970 2024-09-21T05:57:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb493400640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb494145970 0x7fb49415f2e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb494146800 tx=0x7fb48c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb48aa00640 1 -- 172.21.15.50:0/736980375 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c004b30 con 0x7fb494145970 2024-09-21T05:57:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb48aa00640 1 -- 172.21.15.50:0/736980375 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb48c004cd0 con 0x7fb494145970 2024-09-21T05:57:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb48aa00640 1 -- 172.21.15.50:0/736980375 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c003be0 con 0x7fb494145970 2024-09-21T05:57:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.599+0000 7fb499600640 1 -- 172.21.15.50:0/736980375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb494143320 con 0x7fb494145970 2024-09-21T05:57:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.603+0000 7fb499600640 1 -- 172.21.15.50:0/736980375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4941438e0 con 0x7fb494145970 2024-09-21T05:57:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.603+0000 7fb499600640 1 -- 172.21.15.50:0/736980375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb494143ae0 con 0x7fb494145970 2024-09-21T05:57:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.603+0000 7fb48aa00640 1 -- 172.21.15.50:0/736980375 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb48c004030 con 0x7fb494145970 2024-09-21T05:57:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.603+0000 7fb48aa00640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb470073230 0x7fb470075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.603+0000 7fb48aa00640 1 -- 172.21.15.50:0/736980375 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb48c094d40 con 0x7fb494145970 2024-09-21T05:57:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.603+0000 7fb492a00640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb470073230 0x7fb470075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.607+0000 7fb492a00640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb470073230 0x7fb470075740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb4840ab000 tx=0x7fb4840a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:03.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.607+0000 7fb48aa00640 1 -- 172.21.15.50:0/736980375 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb48c05fde0 con 0x7fb494145970 2024-09-21T05:57:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.823+0000 7fb499600640 1 -- 172.21.15.50:0/736980375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4940b5410 con 0x7fb494145970 2024-09-21T05:57:03.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.823+0000 7fb48aa00640 1 -- 172.21.15.50:0/736980375 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5211 (secure 0 0 0) 0x7fb48c05f6e0 con 0x7fb494145970 2024-09-21T05:57:03.827 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:03.827 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":8,"btime":"2024-09-21T05:56:32:796889+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3883511106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3883511106},{"type":"v1","addr":"172.21.15.88:6836","nonce":3883511106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/606533049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":606533049},{"type":"v1","addr":"172.21.15.50:6833","nonce":606533049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/183287131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":183287131},{"type":"v1","addr":"172.21.15.50:6835","nonce":183287131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:56:29.197481+0000","modified":"2024-09-21T05:56:32.215215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4371":{"gid":4371,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/1479043500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1479043500},{"type":"v1","addr":"172.21.15.88:6837","nonce":1479043500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4371,"qdb_cluster":[4371]},"id":1}]} 2024-09-21T05:57:03.827 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 8 2024-09-21T05:57:03.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.823+0000 7fb499600640 1 -- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb470073230 msgr2=0x7fb470075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.823+0000 7fb499600640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb470073230 0x7fb470075740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb4840ab000 tx=0x7fb4840a7040 comp rx=0 tx=0).stop 2024-09-21T05:57:03.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.823+0000 7fb499600640 1 -- 172.21.15.50:0/736980375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb494145970 msgr2=0x7fb49415f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.823+0000 7fb499600640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb494145970 0x7fb49415f2e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb494146800 tx=0x7fb48c011f10 comp rx=0 tx=0).stop 2024-09-21T05:57:03.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.823+0000 7fb499600640 1 -- 172.21.15.50:0/736980375 shutdown_connections 2024-09-21T05:57:03.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.823+0000 7fb499600640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb470073230 0x7fb470075740 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.827+0000 7fb499600640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb494145970 0x7fb49415f2e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.827+0000 7fb499600640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4941f9fc0 0x7fb49415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.827+0000 7fb499600640 1 --2- 172.21.15.50:0/736980375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb478003a10 0x7fb494145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.827+0000 7fb499600640 1 -- 172.21.15.50:0/736980375 >> 172.21.15.50:0/736980375 conn(0x7fb4940d06e0 msgr2=0x7fb494161670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:03.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.827+0000 7fb499600640 1 -- 172.21.15.50:0/736980375 shutdown_connections 2024-09-21T05:57:03.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:03.827+0000 7fb499600640 1 -- 172.21.15.50:0/736980375 wait complete. 2024-09-21T05:57:03.850 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:57:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.003+0000 7f0046c00640 1 -- 172.21.15.50:0/1972781648 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0040074b20 msgr2=0x7f004015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.003+0000 7f0046c00640 1 --2- 172.21.15.50:0/1972781648 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0040074b20 0x7f004015cf90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0038009c80 tx=0x7f003801cda0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.003+0000 7f0046c00640 1 -- 172.21.15.50:0/1972781648 shutdown_connections 2024-09-21T05:57:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.003+0000 7f0046c00640 1 --2- 172.21.15.50:0/1972781648 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0040074b20 0x7f004015cf90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.003+0000 7f0046c00640 1 --2- 172.21.15.50:0/1972781648 >> v2:172.21.15.88:3300/0 conn(0x7f0040157030 0x7f00400b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.003+0000 7f0046c00640 1 -- 172.21.15.50:0/1972781648 >> 172.21.15.50:0/1972781648 conn(0x7f00400708c0 msgr2=0x7f0040070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.003+0000 7f0046c00640 1 -- 172.21.15.50:0/1972781648 shutdown_connections 2024-09-21T05:57:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.003+0000 7f0046c00640 1 -- 172.21.15.50:0/1972781648 wait complete. 2024-09-21T05:57:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0046c00640 1 Processor -- start 2024-09-21T05:57:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0046c00640 1 -- start start 2024-09-21T05:57:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0046c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0040157030 0x7f0040141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0046c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0040160d50 0x7f00401513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0046c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0040151900 0x7f004006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0046c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f004016a740 con 0x7f0040160d50 2024-09-21T05:57:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0046c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f004016a5c0 con 0x7f0040157030 2024-09-21T05:57:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0046c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f004016a8c0 con 0x7f0040151900 2024-09-21T05:57:04.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0037400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0040160d50 0x7f00401513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:04.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0037400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0040160d50 0x7f00401513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48906/0 (socket says 172.21.15.50:48906) 2024-09-21T05:57:04.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0037400640 1 -- 172.21.15.50:0/3056747112 learned_addr learned my addr 172.21.15.50:0/3056747112 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:04.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f003fe00640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0040157030 0x7f0040141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:04.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0044c00640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0040151900 0x7f004006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:04.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0037400640 1 -- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0040151900 msgr2=0x7f004006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0037400640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0040151900 0x7f004006d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0037400640 1 -- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0040157030 msgr2=0x7f0040141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0037400640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0040157030 0x7f0040141a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0037400640 1 -- 172.21.15.50:0/3056747112 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0040141fd0 con 0x7f0040160d50 2024-09-21T05:57:04.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0037400640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0040160d50 0x7f00401513c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f002800c3c0 tx=0x7f002800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:04.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f003cc00640 1 -- 172.21.15.50:0/3056747112 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028017070 con 0x7f0040160d50 2024-09-21T05:57:04.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f003cc00640 1 -- 172.21.15.50:0/3056747112 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00280053c0 con 0x7f0040160d50 2024-09-21T05:57:04.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0046c00640 1 -- 172.21.15.50:0/3056747112 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004014fcf0 con 0x7f0040160d50 2024-09-21T05:57:04.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f003cc00640 1 -- 172.21.15.50:0/3056747112 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028003860 con 0x7f0040160d50 2024-09-21T05:57:04.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.007+0000 7f0046c00640 1 -- 172.21.15.50:0/3056747112 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00401502b0 con 0x7f0040160d50 2024-09-21T05:57:04.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.011+0000 7f0046c00640 1 -- 172.21.15.50:0/3056747112 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00400d2260 con 0x7f0040160d50 2024-09-21T05:57:04.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.011+0000 7f003cc00640 1 -- 172.21.15.50:0/3056747112 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f002801a070 con 0x7f0040160d50 2024-09-21T05:57:04.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.011+0000 7f003cc00640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f001c073200 0x7f001c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:04.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.011+0000 7f003cc00640 1 -- 172.21.15.50:0/3056747112 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0028093560 con 0x7f0040160d50 2024-09-21T05:57:04.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.011+0000 7f003fe00640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f001c073200 0x7f001c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:04.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.015+0000 7f003fe00640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f001c073200 0x7f001c075710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0030009600 tx=0x7f00300091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:04.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.015+0000 7f003cc00640 1 -- 172.21.15.50:0/3056747112 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f002805de90 con 0x7f0040160d50 2024-09-21T05:57:04.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 -- 172.21.15.50:0/3056747112 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f004015bf40 con 0x7f0040160d50 2024-09-21T05:57:04.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f003cc00640 1 -- 172.21.15.50:0/3056747112 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5211 (secure 0 0 0) 0x7f00280628c0 con 0x7f0040160d50 2024-09-21T05:57:04.235 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:04.235 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":8,"btime":"2024-09-21T05:56:32:796889+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3883511106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3883511106},{"type":"v1","addr":"172.21.15.88:6836","nonce":3883511106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/606533049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":606533049},{"type":"v1","addr":"172.21.15.50:6833","nonce":606533049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/183287131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":183287131},{"type":"v1","addr":"172.21.15.50:6835","nonce":183287131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:56:29.197481+0000","modified":"2024-09-21T05:56:32.215215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4371":{"gid":4371,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/1479043500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1479043500},{"type":"v1","addr":"172.21.15.88:6837","nonce":1479043500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4371,"qdb_cluster":[4371]},"id":1}]} 2024-09-21T05:57:04.235 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 8 2024-09-21T05:57:04.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 -- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f001c073200 msgr2=0x7f001c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f001c073200 0x7f001c075710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0030009600 tx=0x7f00300091e0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 -- 172.21.15.50:0/3056747112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0040160d50 msgr2=0x7f00401513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0040160d50 0x7f00401513c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f002800c3c0 tx=0x7f002800c880 comp rx=0 tx=0).stop 2024-09-21T05:57:04.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 -- 172.21.15.50:0/3056747112 shutdown_connections 2024-09-21T05:57:04.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0040151900 0x7f004006d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f001c073200 0x7f001c075710 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0040160d50 0x7f00401513c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 --2- 172.21.15.50:0/3056747112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0040157030 0x7f0040141a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 -- 172.21.15.50:0/3056747112 >> 172.21.15.50:0/3056747112 conn(0x7f00400708c0 msgr2=0x7f004015e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:04.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 -- 172.21.15.50:0/3056747112 shutdown_connections 2024-09-21T05:57:04.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.231+0000 7f0046c00640 1 -- 172.21.15.50:0/3056747112 wait complete. 2024-09-21T05:57:04.257 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:57:04.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.411+0000 7f2324200640 1 -- 172.21.15.50:0/2769096892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f231c0b5c70 msgr2=0x7f231c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.411+0000 7f2324200640 1 --2- 172.21.15.50:0/2769096892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f231c0b5c70 0x7f231c15bb60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f231c070710 tx=0x7f231001c890 comp rx=0 tx=0).stop 2024-09-21T05:57:04.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.411+0000 7f2324200640 1 -- 172.21.15.50:0/2769096892 shutdown_connections 2024-09-21T05:57:04.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.411+0000 7f2324200640 1 --2- 172.21.15.50:0/2769096892 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f231c0b4de0 0x7f231c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.411+0000 7f2324200640 1 --2- 172.21.15.50:0/2769096892 >> v2:172.21.15.88:3300/0 conn(0x7f231c1527b0 0x7f231c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.411+0000 7f2324200640 1 --2- 172.21.15.50:0/2769096892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f231c0b5c70 0x7f231c15bb60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.411+0000 7f2324200640 1 -- 172.21.15.50:0/2769096892 >> 172.21.15.50:0/2769096892 conn(0x7f231c0b2040 msgr2=0x7f231c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:04.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.411+0000 7f2324200640 1 -- 172.21.15.50:0/2769096892 shutdown_connections 2024-09-21T05:57:04.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.411+0000 7f2324200640 1 -- 172.21.15.50:0/2769096892 wait complete. 2024-09-21T05:57:04.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2324200640 1 Processor -- start 2024-09-21T05:57:04.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2324200640 1 -- start start 2024-09-21T05:57:04.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2324200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f231c0b4de0 0x7f231c153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:04.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2324200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f231c0b5c70 0x7f231c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:04.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2324200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f231c1527b0 0x7f231c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:04.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2324200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f231c15f650 con 0x7f231c1527b0 2024-09-21T05:57:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2324200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f231c15f4d0 con 0x7f231c0b4de0 2024-09-21T05:57:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2324200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f231c15f7d0 con 0x7f231c0b5c70 2024-09-21T05:57:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2322200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f231c1527b0 0x7f231c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2322200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f231c1527b0 0x7f231c0ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48928/0 (socket says 172.21.15.50:48928) 2024-09-21T05:57:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2322200640 1 -- 172.21.15.50:0/3726513773 learned_addr learned my addr 172.21.15.50:0/3726513773 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2321800640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f231c0b4de0 0x7f231c153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2320e00640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f231c0b5c70 0x7f231c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2322200640 1 -- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f231c0b5c70 msgr2=0x7f231c1515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2322200640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f231c0b5c70 0x7f231c1515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2322200640 1 -- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f231c0b4de0 msgr2=0x7f231c153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2322200640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f231c0b4de0 0x7f231c153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2322200640 1 -- 172.21.15.50:0/3726513773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231c157210 con 0x7f231c1527b0 2024-09-21T05:57:04.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2322200640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f231c1527b0 0x7f231c0ae8e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f231400cae0 tx=0x7f2314007220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:04.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2319600640 1 -- 172.21.15.50:0/3726513773 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314007da0 con 0x7f231c1527b0 2024-09-21T05:57:04.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2319600640 1 -- 172.21.15.50:0/3726513773 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f231400f560 con 0x7f231c1527b0 2024-09-21T05:57:04.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2319600640 1 -- 172.21.15.50:0/3726513773 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231400f9b0 con 0x7f231c1527b0 2024-09-21T05:57:04.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.415+0000 7f2324200640 1 -- 172.21.15.50:0/3726513773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231c1f89c0 con 0x7f231c1527b0 2024-09-21T05:57:04.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.419+0000 7f2324200640 1 -- 172.21.15.50:0/3726513773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231c1f8f40 con 0x7f231c1527b0 2024-09-21T05:57:04.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.419+0000 7f2324200640 1 -- 172.21.15.50:0/3726513773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f231c1573e0 con 0x7f231c1527b0 2024-09-21T05:57:04.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.419+0000 7f2319600640 1 -- 172.21.15.50:0/3726513773 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f231400fb50 con 0x7f231c1527b0 2024-09-21T05:57:04.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.419+0000 7f2319600640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f22fc073200 0x7f22fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:04.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.419+0000 7f2319600640 1 -- 172.21.15.50:0/3726513773 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2314094d00 con 0x7f231c1527b0 2024-09-21T05:57:04.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.419+0000 7f2321800640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f22fc073200 0x7f22fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:04.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.423+0000 7f2321800640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f22fc073200 0x7f22fc075710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f23100af000 tx=0x7f231001b0d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:04.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.423+0000 7f2319600640 1 -- 172.21.15.50:0/3726513773 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2314014070 con 0x7f231c1527b0 2024-09-21T05:57:04.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.607+0000 7f2324200640 1 -- 172.21.15.50:0/3726513773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f231c154470 con 0x7f231c1527b0 2024-09-21T05:57:04.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.607+0000 7f2319600640 1 -- 172.21.15.50:0/3726513773 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16617 (secure 0 0 0) 0x7f231405f630 con 0x7f231c1527b0 2024-09-21T05:57:04.611 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:04.612 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":22,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:30.775110+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:56:27.052826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:56:28.162376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:57:04.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.611+0000 7f2324200640 1 -- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f22fc073200 msgr2=0x7f22fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.611+0000 7f2324200640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f22fc073200 0x7f22fc075710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f23100af000 tx=0x7f231001b0d0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.611+0000 7f2324200640 1 -- 172.21.15.50:0/3726513773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f231c1527b0 msgr2=0x7f231c0ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.611+0000 7f2324200640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f231c1527b0 0x7f231c0ae8e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f231400cae0 tx=0x7f2314007220 comp rx=0 tx=0).stop 2024-09-21T05:57:04.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.611+0000 7f2324200640 1 -- 172.21.15.50:0/3726513773 shutdown_connections 2024-09-21T05:57:04.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.611+0000 7f2324200640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f22fc073200 0x7f22fc075710 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.611+0000 7f2324200640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f231c1527b0 0x7f231c0ae8e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.611+0000 7f2324200640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f231c0b5c70 0x7f231c1515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.611+0000 7f2324200640 1 --2- 172.21.15.50:0/3726513773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f231c0b4de0 0x7f231c153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.611+0000 7f2324200640 1 -- 172.21.15.50:0/3726513773 >> 172.21.15.50:0/3726513773 conn(0x7f231c0b2040 msgr2=0x7f231c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:04.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.611+0000 7f2324200640 1 -- 172.21.15.50:0/3726513773 shutdown_connections 2024-09-21T05:57:04.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.611+0000 7f2324200640 1 -- 172.21.15.50:0/3726513773 wait complete. 2024-09-21T05:57:04.635 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T05:57:04.635 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:57:04.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.787+0000 7feb5a800640 1 -- 172.21.15.50:0/260225855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb540b5040 msgr2=0x7feb540b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.787+0000 7feb5a800640 1 --2- 172.21.15.50:0/260225855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb540b5040 0x7feb540b1e30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feb54071de0 tx=0x7feb4801cd50 comp rx=0 tx=0).stop 2024-09-21T05:57:04.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.787+0000 7feb5a800640 1 -- 172.21.15.50:0/260225855 shutdown_connections 2024-09-21T05:57:04.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.787+0000 7feb5a800640 1 --2- 172.21.15.50:0/260225855 >> v2:172.21.15.88:3300/0 conn(0x7feb540b3130 0x7feb540b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.787+0000 7feb5a800640 1 --2- 172.21.15.50:0/260225855 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feb540cfe40 0x7feb540d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.787+0000 7feb5a800640 1 --2- 172.21.15.50:0/260225855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb540b5040 0x7feb540b1e30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.787+0000 7feb5a800640 1 -- 172.21.15.50:0/260225855 >> 172.21.15.50:0/260225855 conn(0x7feb5406cdf0 msgr2=0x7feb5406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:04.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.787+0000 7feb5a800640 1 -- 172.21.15.50:0/260225855 shutdown_connections 2024-09-21T05:57:04.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.787+0000 7feb5a800640 1 -- 172.21.15.50:0/260225855 wait complete. 2024-09-21T05:57:04.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb5a800640 1 Processor -- start 2024-09-21T05:57:04.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb5a800640 1 -- start start 2024-09-21T05:57:04.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb5a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb540b3130 0x7feb5413abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:04.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb5a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb540b5040 0x7feb540ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:04.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb5a800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feb540cfe40 0x7feb540aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:04.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb5a800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feb540729c0 con 0x7feb540b5040 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb5a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feb54072840 con 0x7feb540b3130 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb5a800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7feb5415e830 con 0x7feb540cfe40 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb58a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb540b5040 0x7feb540ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb58a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb540b5040 0x7feb540ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48948/0 (socket says 172.21.15.50:48948) 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb58a00640 1 -- 172.21.15.50:0/451626769 learned_addr learned my addr 172.21.15.50:0/451626769 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb59400640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb540b3130 0x7feb5413abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.791+0000 7feb59e00640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feb540cfe40 0x7feb540aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb58a00640 1 -- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feb540cfe40 msgr2=0x7feb540aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb58a00640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feb540cfe40 0x7feb540aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb58a00640 1 -- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb540b3130 msgr2=0x7feb5413abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb58a00640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb540b3130 0x7feb5413abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb58a00640 1 -- 172.21.15.50:0/451626769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb54154800 con 0x7feb540b5040 2024-09-21T05:57:04.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb58a00640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb540b5040 0x7feb540ae8e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feb4000b420 tx=0x7feb4000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:04.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb51600640 1 -- 172.21.15.50:0/451626769 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40011830 con 0x7feb540b5040 2024-09-21T05:57:04.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb51600640 1 -- 172.21.15.50:0/451626769 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb40016070 con 0x7feb540b5040 2024-09-21T05:57:04.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb51600640 1 -- 172.21.15.50:0/451626769 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40011d60 con 0x7feb540b5040 2024-09-21T05:57:04.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb5a800640 1 -- 172.21.15.50:0/451626769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb54154ae0 con 0x7feb540b5040 2024-09-21T05:57:04.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb5a800640 1 -- 172.21.15.50:0/451626769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb541550a0 con 0x7feb540b5040 2024-09-21T05:57:04.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb5a800640 1 -- 172.21.15.50:0/451626769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb54154550 con 0x7feb540b5040 2024-09-21T05:57:04.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb51600640 1 -- 172.21.15.50:0/451626769 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feb4001b460 con 0x7feb540b5040 2024-09-21T05:57:04.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.795+0000 7feb51600640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feb34073200 0x7feb34075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:04.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.799+0000 7feb51600640 1 -- 172.21.15.50:0/451626769 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7feb40093de0 con 0x7feb540b5040 2024-09-21T05:57:04.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.799+0000 7feb59400640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feb34073200 0x7feb34075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:04.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.799+0000 7feb59400640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feb34073200 0x7feb34075710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb480af000 tx=0x7feb4801b590 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:04.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:04.803+0000 7feb51600640 1 -- 172.21.15.50:0/451626769 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7feb4005e7a0 con 0x7feb540b5040 2024-09-21T05:57:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 -- 172.21.15.50:0/451626769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb540cf8c0 con 0x7feb540b5040 2024-09-21T05:57:05.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb51600640 1 -- 172.21.15.50:0/451626769 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5211 (secure 0 0 0) 0x7feb400631d0 con 0x7feb540b5040 2024-09-21T05:57:05.018 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":8,"btime":"2024-09-21T05:56:32:796889+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3883511106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3883511106},{"type":"v1","addr":"172.21.15.88:6836","nonce":3883511106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/606533049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":606533049},{"type":"v1","addr":"172.21.15.50:6833","nonce":606533049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/183287131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":183287131},{"type":"v1","addr":"172.21.15.50:6835","nonce":183287131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:56:29.197481+0000","modified":"2024-09-21T05:56:32.215215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4371":{"gid":4371,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/1479043500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1479043500},{"type":"v1","addr":"172.21.15.88:6837","nonce":1479043500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4371,"qdb_cluster":[4371]},"id":1}]} 2024-09-21T05:57:05.019 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 8 2024-09-21T05:57:05.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 -- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feb34073200 msgr2=0x7feb34075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feb34073200 0x7feb34075710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb480af000 tx=0x7feb4801b590 comp rx=0 tx=0).stop 2024-09-21T05:57:05.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 -- 172.21.15.50:0/451626769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb540b5040 msgr2=0x7feb540ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb540b5040 0x7feb540ae8e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feb4000b420 tx=0x7feb4000b8e0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 -- 172.21.15.50:0/451626769 shutdown_connections 2024-09-21T05:57:05.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feb540cfe40 0x7feb540aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feb34073200 0x7feb34075710 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb540b5040 0x7feb540ae8e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 --2- 172.21.15.50:0/451626769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb540b3130 0x7feb5413abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 -- 172.21.15.50:0/451626769 >> 172.21.15.50:0/451626769 conn(0x7feb5406cdf0 msgr2=0x7feb5415e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:05.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 -- 172.21.15.50:0/451626769 shutdown_connections 2024-09-21T05:57:05.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.015+0000 7feb5a800640 1 -- 172.21.15.50:0/451626769 wait complete. 2024-09-21T05:57:05.041 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:57:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.195+0000 7f0e97600640 1 -- 172.21.15.50:0/591508765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e78003a60 msgr2=0x7f0e78003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.195+0000 7f0e97600640 1 --2- 172.21.15.50:0/591508765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e78003a60 0x7f0e78003ed0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009b40 tx=0x7f0e8401d250 comp rx=0 tx=0).stop 2024-09-21T05:57:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.195+0000 7f0e97600640 1 -- 172.21.15.50:0/591508765 shutdown_connections 2024-09-21T05:57:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.195+0000 7f0e97600640 1 --2- 172.21.15.50:0/591508765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e78003a60 0x7f0e78003ed0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.195+0000 7f0e97600640 1 -- 172.21.15.50:0/591508765 >> 172.21.15.50:0/591508765 conn(0x7f0e900b2040 msgr2=0x7f0e900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.195+0000 7f0e97600640 1 -- 172.21.15.50:0/591508765 shutdown_connections 2024-09-21T05:57:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.195+0000 7f0e97600640 1 -- 172.21.15.50:0/591508765 wait complete. 2024-09-21T05:57:05.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e97600640 1 Processor -- start 2024-09-21T05:57:05.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e97600640 1 -- start start 2024-09-21T05:57:05.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e97600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e78003a60 0x7f0e901f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:05.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e97600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e900ce060 0x7f0e90140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:05.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e97600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0e901415a0 0x7f0e900b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:05.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e97600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e900b3ae0 con 0x7f0e900ce060 2024-09-21T05:57:05.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e97600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0e900b3960 con 0x7f0e78003a60 2024-09-21T05:57:05.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e97600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0e900b3c60 con 0x7f0e901415a0 2024-09-21T05:57:05.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e8fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e900ce060 0x7f0e90140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:05.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e8fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e900ce060 0x7f0e90140e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48954/0 (socket says 172.21.15.50:48954) 2024-09-21T05:57:05.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e8fe00640 1 -- 172.21.15.50:0/3060978656 learned_addr learned my addr 172.21.15.50:0/3060978656 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:05.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e95600640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0e901415a0 0x7f0e900b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:05.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e94c00640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e78003a60 0x7f0e901f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:05.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e8fe00640 1 -- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0e901415a0 msgr2=0x7f0e900b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e8fe00640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0e901415a0 0x7f0e900b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e8fe00640 1 -- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e78003a60 msgr2=0x7f0e901f83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e8fe00640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e78003a60 0x7f0e901f83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e8fe00640 1 -- 172.21.15.50:0/3060978656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e900ce7d0 con 0x7f0e900ce060 2024-09-21T05:57:05.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e95600640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0e901415a0 0x7f0e900b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:05.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e94c00640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e78003a60 0x7f0e901f83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:05.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e8fe00640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e900ce060 0x7f0e90140e00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c000fc0 tx=0x7f0e7c00e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:05.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e8cc00640 1 -- 172.21.15.50:0/3060978656 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c015070 con 0x7f0e900ce060 2024-09-21T05:57:05.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e8cc00640 1 -- 172.21.15.50:0/3060978656 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e7c010980 con 0x7f0e900ce060 2024-09-21T05:57:05.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e8cc00640 1 -- 172.21.15.50:0/3060978656 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c010dd0 con 0x7f0e900ce060 2024-09-21T05:57:05.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.199+0000 7f0e97600640 1 -- 172.21.15.50:0/3060978656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e900ceab0 con 0x7f0e900ce060 2024-09-21T05:57:05.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.203+0000 7f0e97600640 1 -- 172.21.15.50:0/3060978656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e90141340 con 0x7f0e900ce060 2024-09-21T05:57:05.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.203+0000 7f0e97600640 1 -- 172.21.15.50:0/3060978656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e900b41c0 con 0x7f0e900ce060 2024-09-21T05:57:05.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.203+0000 7f0e8cc00640 1 -- 172.21.15.50:0/3060978656 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0e7c018070 con 0x7f0e900ce060 2024-09-21T05:57:05.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.203+0000 7f0e8cc00640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e70073200 0x7f0e70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:05.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.203+0000 7f0e8cc00640 1 -- 172.21.15.50:0/3060978656 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0e7c094560 con 0x7f0e900ce060 2024-09-21T05:57:05.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.203+0000 7f0e94c00640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e70073200 0x7f0e70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:05.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.207+0000 7f0e94c00640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e70073200 0x7f0e70075710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e840ad000 tx=0x7f0e84019ff0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:05.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.207+0000 7f0e8cc00640 1 -- 172.21.15.50:0/3060978656 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0e7c05f570 con 0x7f0e900ce060 2024-09-21T05:57:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 -- 172.21.15.50:0/3060978656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e900ce560 con 0x7f0e900ce060 2024-09-21T05:57:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e8cc00640 1 -- 172.21.15.50:0/3060978656 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5211 (secure 0 0 0) 0x7f0e7c001bc0 con 0x7f0e900ce060 2024-09-21T05:57:05.426 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:05.427 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":8,"btime":"2024-09-21T05:56:32:796889+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3883511106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3883511106},{"type":"v1","addr":"172.21.15.88:6836","nonce":3883511106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/606533049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":606533049},{"type":"v1","addr":"172.21.15.50:6833","nonce":606533049}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/183287131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":183287131},{"type":"v1","addr":"172.21.15.50:6835","nonce":183287131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:56:29.197481+0000","modified":"2024-09-21T05:56:32.215215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4371":{"gid":4371,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/1479043500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1479043500},{"type":"v1","addr":"172.21.15.88:6837","nonce":1479043500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4371,"qdb_cluster":[4371]},"id":1}]} 2024-09-21T05:57:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 8 2024-09-21T05:57:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 -- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e70073200 msgr2=0x7f0e70075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e70073200 0x7f0e70075710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e840ad000 tx=0x7f0e84019ff0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 -- 172.21.15.50:0/3060978656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e900ce060 msgr2=0x7f0e90140e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e900ce060 0x7f0e90140e00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c000fc0 tx=0x7f0e7c00e9a0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 -- 172.21.15.50:0/3060978656 shutdown_connections 2024-09-21T05:57:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0e901415a0 0x7f0e900b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e70073200 0x7f0e70075710 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e900ce060 0x7f0e90140e00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 --2- 172.21.15.50:0/3060978656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e78003a60 0x7f0e901f83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 -- 172.21.15.50:0/3060978656 >> 172.21.15.50:0/3060978656 conn(0x7f0e900b2040 msgr2=0x7f0e90141d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 -- 172.21.15.50:0/3060978656 shutdown_connections 2024-09-21T05:57:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.423+0000 7f0e97600640 1 -- 172.21.15.50:0/3060978656 wait complete. 2024-09-21T05:57:05.449 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:57:05.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.603+0000 7fa828600640 1 -- 172.21.15.50:0/14003768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa808004810 msgr2=0x7fa808004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.603+0000 7fa828600640 1 --2- 172.21.15.50:0/14003768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa808004810 0x7fa808004c20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0016f0 tx=0x7fa80c01c750 comp rx=0 tx=0).stop 2024-09-21T05:57:05.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.603+0000 7fa828600640 1 -- 172.21.15.50:0/14003768 shutdown_connections 2024-09-21T05:57:05.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.603+0000 7fa828600640 1 --2- 172.21.15.50:0/14003768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa808004810 0x7fa808004c20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.603+0000 7fa828600640 1 -- 172.21.15.50:0/14003768 >> 172.21.15.50:0/14003768 conn(0x7fa8200b35f0 msgr2=0x7fa8200b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:05.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.607+0000 7fa828600640 1 -- 172.21.15.50:0/14003768 shutdown_connections 2024-09-21T05:57:05.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.607+0000 7fa828600640 1 -- 172.21.15.50:0/14003768 wait complete. 2024-09-21T05:57:05.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.607+0000 7fa828600640 1 Processor -- start 2024-09-21T05:57:05.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.607+0000 7fa828600640 1 -- start start 2024-09-21T05:57:05.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.607+0000 7fa828600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa808004810 0x7fa82015a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:05.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.607+0000 7fa828600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8201f9ed0 0x7fa8201531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:05.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.607+0000 7fa828600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8201536e0 0x7fa820150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:05.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.607+0000 7fa828600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa820070080 con 0x7fa808004810 2024-09-21T05:57:05.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.607+0000 7fa828600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa82006ff00 con 0x7fa8201f9ed0 2024-09-21T05:57:05.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.607+0000 7fa828600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa820070200 con 0x7fa8201536e0 2024-09-21T05:57:05.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa825c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa808004810 0x7fa82015a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:05.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa825c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa808004810 0x7fa82015a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48980/0 (socket says 172.21.15.50:48980) 2024-09-21T05:57:05.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa825c00640 1 -- 172.21.15.50:0/1557326081 learned_addr learned my addr 172.21.15.50:0/1557326081 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:05.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa825c00640 1 -- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8201536e0 msgr2=0x7fa820150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa826600640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8201536e0 0x7fa820150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:05.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa825c00640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8201536e0 0x7fa820150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa825c00640 1 -- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8201f9ed0 msgr2=0x7fa8201531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa825200640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8201f9ed0 0x7fa8201531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:05.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa825c00640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8201f9ed0 0x7fa8201531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa825c00640 1 -- 172.21.15.50:0/1557326081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82015ab90 con 0x7fa808004810 2024-09-21T05:57:05.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa825200640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8201f9ed0 0x7fa8201531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:05.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa825c00640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa808004810 0x7fa82015a290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa814006630 tx=0x7fa8140039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:05.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa81d600640 1 -- 172.21.15.50:0/1557326081 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814015070 con 0x7fa808004810 2024-09-21T05:57:05.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa81d600640 1 -- 172.21.15.50:0/1557326081 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa814011920 con 0x7fa808004810 2024-09-21T05:57:05.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa81d600640 1 -- 172.21.15.50:0/1557326081 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814011d70 con 0x7fa808004810 2024-09-21T05:57:05.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa828600640 1 -- 172.21.15.50:0/1557326081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82015ae70 con 0x7fa808004810 2024-09-21T05:57:05.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa828600640 1 -- 172.21.15.50:0/1557326081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82015b430 con 0x7fa808004810 2024-09-21T05:57:05.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa828600640 1 -- 172.21.15.50:0/1557326081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82015f560 con 0x7fa808004810 2024-09-21T05:57:05.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.611+0000 7fa81d600640 1 -- 172.21.15.50:0/1557326081 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa814018070 con 0x7fa808004810 2024-09-21T05:57:05.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.615+0000 7fa81d600640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa800073200 0x7fa800075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:05.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.615+0000 7fa81d600640 1 -- 172.21.15.50:0/1557326081 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa814094f40 con 0x7fa808004810 2024-09-21T05:57:05.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.615+0000 7fa825200640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa800073200 0x7fa800075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:05.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.615+0000 7fa825200640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa800073200 0x7fa800075710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0016c0 tx=0x7fa80c019540 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:05.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.619+0000 7fa81d600640 1 -- 172.21.15.50:0/1557326081 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa81405ff60 con 0x7fa808004810 2024-09-21T05:57:05.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.799+0000 7fa828600640 1 -- 172.21.15.50:0/1557326081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8200709f0 con 0x7fa808004810 2024-09-21T05:57:05.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.799+0000 7fa81d600640 1 -- 172.21.15.50:0/1557326081 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16617 (secure 0 0 0) 0x7fa81405f860 con 0x7fa808004810 2024-09-21T05:57:05.803 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:05.803 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":22,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:56:30.775110+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:56:27.052826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:56:28.162376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:57:05.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.799+0000 7fa828600640 1 -- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa800073200 msgr2=0x7fa800075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.799+0000 7fa828600640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa800073200 0x7fa800075710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0016c0 tx=0x7fa80c019540 comp rx=0 tx=0).stop 2024-09-21T05:57:05.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.803+0000 7fa828600640 1 -- 172.21.15.50:0/1557326081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa808004810 msgr2=0x7fa82015a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.803+0000 7fa828600640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa808004810 0x7fa82015a290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa814006630 tx=0x7fa8140039d0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.803+0000 7fa828600640 1 -- 172.21.15.50:0/1557326081 shutdown_connections 2024-09-21T05:57:05.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.803+0000 7fa828600640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8201536e0 0x7fa820150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.803+0000 7fa828600640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8201f9ed0 0x7fa8201531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.803+0000 7fa828600640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa800073200 0x7fa800075710 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.803+0000 7fa828600640 1 --2- 172.21.15.50:0/1557326081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa808004810 0x7fa82015a290 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.803+0000 7fa828600640 1 -- 172.21.15.50:0/1557326081 >> 172.21.15.50:0/1557326081 conn(0x7fa8200b35f0 msgr2=0x7fa8200af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:05.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.803+0000 7fa828600640 1 -- 172.21.15.50:0/1557326081 shutdown_connections 2024-09-21T05:57:05.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.803+0000 7fa828600640 1 -- 172.21.15.50:0/1557326081 wait complete. 2024-09-21T05:57:05.826 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:57:05.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.979+0000 7f24fd400640 1 -- 172.21.15.50:0/2215585233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f80744d0 msgr2=0x7f24f806ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.979+0000 7f24fd400640 1 --2- 172.21.15.50:0/2215585233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f80744d0 0x7f24f806ea60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009840 tx=0x7f24e001c7f0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.979+0000 7f24fd400640 1 -- 172.21.15.50:0/2215585233 shutdown_connections 2024-09-21T05:57:05.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.979+0000 7f24fd400640 1 --2- 172.21.15.50:0/2215585233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f80744d0 0x7f24f806ea60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.979+0000 7f24fd400640 1 -- 172.21.15.50:0/2215585233 >> 172.21.15.50:0/2215585233 conn(0x7f24f80b2c40 msgr2=0x7f24f80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:05.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.983+0000 7f24fd400640 1 -- 172.21.15.50:0/2215585233 shutdown_connections 2024-09-21T05:57:05.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.983+0000 7f24fd400640 1 -- 172.21.15.50:0/2215585233 wait complete. 2024-09-21T05:57:05.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.983+0000 7f24fd400640 1 Processor -- start 2024-09-21T05:57:05.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.983+0000 7f24fd400640 1 -- start start 2024-09-21T05:57:05.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.983+0000 7f24fd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24f806efa0 0x7f24f814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:05.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.983+0000 7f24fd400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24f815c5d0 0x7f24f81fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:05.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.983+0000 7f24fd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f81fc480 0x7f24f80ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:05.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.983+0000 7f24fd400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24f8169f00 con 0x7f24f81fc480 2024-09-21T05:57:05.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.983+0000 7f24fd400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f24f8169d80 con 0x7f24f806efa0 2024-09-21T05:57:05.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.983+0000 7f24fd400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f24f816a080 con 0x7f24f815c5d0 2024-09-21T05:57:05.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24f7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f81fc480 0x7f24f80ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:05.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24f7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f81fc480 0x7f24f80ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49008/0 (socket says 172.21.15.50:49008) 2024-09-21T05:57:05.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24f7200640 1 -- 172.21.15.50:0/1116399921 learned_addr learned my addr 172.21.15.50:0/1116399921 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:05.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24f6800640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24f806efa0 0x7f24f814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:05.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24efe00640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24f815c5d0 0x7f24f81fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:05.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24f7200640 1 -- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24f815c5d0 msgr2=0x7f24f81fbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24f7200640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24f815c5d0 0x7f24f81fbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24f7200640 1 -- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24f806efa0 msgr2=0x7f24f814ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:05.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24f7200640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24f806efa0 0x7f24f814ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:05.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24f7200640 1 -- 172.21.15.50:0/1116399921 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f81fc890 con 0x7f24f81fc480 2024-09-21T05:57:05.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24f7200640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f81fc480 0x7f24f80ce580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f24f000cb10 tx=0x7f24f000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:05.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24eea00640 1 -- 172.21.15.50:0/1116399921 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0016020 con 0x7f24f81fc480 2024-09-21T05:57:05.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24eea00640 1 -- 172.21.15.50:0/1116399921 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24f000ce50 con 0x7f24f81fc480 2024-09-21T05:57:05.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24fd400640 1 -- 172.21.15.50:0/1116399921 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f8154cc0 con 0x7f24f81fc480 2024-09-21T05:57:05.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24eea00640 1 -- 172.21.15.50:0/1116399921 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f000ade0 con 0x7f24f81fc480 2024-09-21T05:57:05.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24fd400640 1 -- 172.21.15.50:0/1116399921 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f8154fa0 con 0x7f24f81fc480 2024-09-21T05:57:05.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24fd400640 1 -- 172.21.15.50:0/1116399921 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f80aa520 con 0x7f24f81fc480 2024-09-21T05:57:05.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.987+0000 7f24eea00640 1 -- 172.21.15.50:0/1116399921 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f24f0019020 con 0x7f24f81fc480 2024-09-21T05:57:05.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.991+0000 7f24eea00640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f24d4073200 0x7f24d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:05.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.991+0000 7f24eea00640 1 -- 172.21.15.50:0/1116399921 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f24f0093ff0 con 0x7f24f81fc480 2024-09-21T05:57:05.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.991+0000 7f24f6800640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f24d4073200 0x7f24d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:05.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.991+0000 7f24f6800640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f24d4073200 0x7f24d4075710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f24e80098e0 tx=0x7f24e8009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:05.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:05.995+0000 7f24eea00640 1 -- 172.21.15.50:0/1116399921 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f24f0020030 con 0x7f24f81fc480 2024-09-21T05:57:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.207+0000 7f24fd400640 1 -- 172.21.15.50:0/1116399921 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f24f80b3260 con 0x7f24f81fc480 2024-09-21T05:57:06.213 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-09-21T05:57:06.211+0000 7f0d45000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:57:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.211+0000 7f24eea00640 1 -- 172.21.15.50:0/1116399921 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f24f005e9f0 con 0x7f24f81fc480 2024-09-21T05:57:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:57:06.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.215+0000 7f24fd400640 1 -- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f24d4073200 msgr2=0x7f24d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.215+0000 7f24fd400640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f24d4073200 0x7f24d4075710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f24e80098e0 tx=0x7f24e8009060 comp rx=0 tx=0).stop 2024-09-21T05:57:06.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.215+0000 7f24fd400640 1 -- 172.21.15.50:0/1116399921 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f81fc480 msgr2=0x7f24f80ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.215+0000 7f24fd400640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f81fc480 0x7f24f80ce580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f24f000cb10 tx=0x7f24f000cc00 comp rx=0 tx=0).stop 2024-09-21T05:57:06.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.215+0000 7f24fd400640 1 -- 172.21.15.50:0/1116399921 shutdown_connections 2024-09-21T05:57:06.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.215+0000 7f24fd400640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f24d4073200 0x7f24d4075710 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.215+0000 7f24fd400640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f81fc480 0x7f24f80ce580 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.215+0000 7f24fd400640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24f815c5d0 0x7f24f81fbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.215+0000 7f24fd400640 1 --2- 172.21.15.50:0/1116399921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24f806efa0 0x7f24f814ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.215+0000 7f24fd400640 1 -- 172.21.15.50:0/1116399921 >> 172.21.15.50:0/1116399921 conn(0x7f24f80b2c40 msgr2=0x7f24f8155550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:06.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.215+0000 7f24fd400640 1 -- 172.21.15.50:0/1116399921 shutdown_connections 2024-09-21T05:57:06.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.215+0000 7f24fd400640 1 -- 172.21.15.50:0/1116399921 wait complete. 2024-09-21T05:57:06.232 INFO:tasks.ceph.mds.b.smithi088.stderr: -7187> 2024-09-21T05:56:34.593+0000 7f967ee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:57:06.238 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T05:57:06.354 INFO:tasks.ceph.mds.b.smithi088.stdout:starting mds.b at 2024-09-21T05:57:06.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.395+0000 7f50b9a00640 1 -- 172.21.15.50:0/825954638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b41527b0 msgr2=0x7f50b4074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.395+0000 7f50b9a00640 1 --2- 172.21.15.50:0/825954638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b41527b0 0x7f50b4074e30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f509c009840 tx=0x7f509c01c760 comp rx=0 tx=0).stop 2024-09-21T05:57:06.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.395+0000 7f50b9a00640 1 -- 172.21.15.50:0/825954638 shutdown_connections 2024-09-21T05:57:06.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.395+0000 7f50b9a00640 1 --2- 172.21.15.50:0/825954638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b41527b0 0x7f50b4074e30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.395+0000 7f50b9a00640 1 --2- 172.21.15.50:0/825954638 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50b40b5c70 0x7f50b415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.395+0000 7f50b9a00640 1 -- 172.21.15.50:0/825954638 >> 172.21.15.50:0/825954638 conn(0x7f50b40b2040 msgr2=0x7f50b40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:06.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.395+0000 7f50b9a00640 1 -- 172.21.15.50:0/825954638 shutdown_connections 2024-09-21T05:57:06.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.395+0000 7f50b9a00640 1 -- 172.21.15.50:0/825954638 wait complete. 2024-09-21T05:57:06.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.395+0000 7f50b9a00640 1 Processor -- start 2024-09-21T05:57:06.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b9a00640 1 -- start start 2024-09-21T05:57:06.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b9a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50b40b4de0 0x7f50b4152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:06.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b40b5c70 0x7f50b4149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:06.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50b41527b0 0x7f50b414ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b9a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f50b4168bb0 con 0x7f50b41527b0 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b9a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f50b4168a30 con 0x7f50b40b5c70 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b9a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f50b4168d30 con 0x7f50b40b4de0 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50b41527b0 0x7f50b414ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50b41527b0 0x7f50b414ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49018/0 (socket says 172.21.15.50:49018) 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b3400640 1 -- 172.21.15.50:0/637072915 learned_addr learned my addr 172.21.15.50:0/637072915 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b2a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50b40b4de0 0x7f50b4152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50abe00640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b40b5c70 0x7f50b4149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b3400640 1 -- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50b40b4de0 msgr2=0x7f50b4152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b3400640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50b40b4de0 0x7f50b4152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b3400640 1 -- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b40b5c70 msgr2=0x7f50b4149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b3400640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b40b5c70 0x7f50b4149940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b3400640 1 -- 172.21.15.50:0/637072915 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b41f9020 con 0x7f50b41527b0 2024-09-21T05:57:06.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b3400640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50b41527b0 0x7f50b414ece0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00cb10 tx=0x7f50ac00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:06.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50aaa00640 1 -- 172.21.15.50:0/637072915 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac016020 con 0x7f50b41527b0 2024-09-21T05:57:06.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50aaa00640 1 -- 172.21.15.50:0/637072915 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50ac00ce50 con 0x7f50b41527b0 2024-09-21T05:57:06.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50aaa00640 1 -- 172.21.15.50:0/637072915 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac00ae00 con 0x7f50b41527b0 2024-09-21T05:57:06.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b9a00640 1 -- 172.21.15.50:0/637072915 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b41f9300 con 0x7f50b41527b0 2024-09-21T05:57:06.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.399+0000 7f50b9a00640 1 -- 172.21.15.50:0/637072915 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b4153510 con 0x7f50b41527b0 2024-09-21T05:57:06.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.403+0000 7f50b9a00640 1 -- 172.21.15.50:0/637072915 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b41f8d90 con 0x7f50b41527b0 2024-09-21T05:57:06.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.403+0000 7f50aaa00640 1 -- 172.21.15.50:0/637072915 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f50ac019020 con 0x7f50b41527b0 2024-09-21T05:57:06.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.403+0000 7f50aaa00640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5090073200 0x7f5090075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:06.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.403+0000 7f50aaa00640 1 -- 172.21.15.50:0/637072915 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7f50ac094900 con 0x7f50b41527b0 2024-09-21T05:57:06.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.403+0000 7f50b2a00640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5090073200 0x7f5090075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:06.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.407+0000 7f50b2a00640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5090073200 0x7f5090075710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009b40 tx=0x7f50a40091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:06.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.407+0000 7f50aaa00640 1 -- 172.21.15.50:0/637072915 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f50ac020020 con 0x7f50b41527b0 2024-09-21T05:57:06.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:06.623+0000 7f50b9a00640 1 -- 172.21.15.50:0/637072915 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f50b415bb60 con 0x7f50b41527b0 2024-09-21T05:57:07.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50aaa00640 1 -- 172.21.15.50:0/637072915 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f50ac05f250 con 0x7f50b41527b0 2024-09-21T05:57:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50b9a00640 1 -- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5090073200 msgr2=0x7f5090075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50b9a00640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5090073200 0x7f5090075710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009b40 tx=0x7f50a40091e0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50b9a00640 1 -- 172.21.15.50:0/637072915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50b41527b0 msgr2=0x7f50b414ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50b9a00640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50b41527b0 0x7f50b414ece0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00cb10 tx=0x7f50ac00cc00 comp rx=0 tx=0).stop 2024-09-21T05:57:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50b9a00640 1 -- 172.21.15.50:0/637072915 shutdown_connections 2024-09-21T05:57:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50b9a00640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5090073200 0x7f5090075710 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50b9a00640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50b41527b0 0x7f50b414ece0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50b9a00640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b40b5c70 0x7f50b4149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50b9a00640 1 --2- 172.21.15.50:0/637072915 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50b40b4de0 0x7f50b4152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50b9a00640 1 -- 172.21.15.50:0/637072915 >> 172.21.15.50:0/637072915 conn(0x7f50b40b2040 msgr2=0x7f50b40b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:07.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50b9a00640 1 -- 172.21.15.50:0/637072915 shutdown_connections 2024-09-21T05:57:07.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.219+0000 7f50b9a00640 1 -- 172.21.15.50:0/637072915 wait complete. 2024-09-21T05:57:07.249 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T05:57:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.411+0000 7f5fdde00640 1 -- 172.21.15.50:0/498801918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd80cf7f0 msgr2=0x7f5fd815d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.411+0000 7f5fdde00640 1 --2- 172.21.15.50:0/498801918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd80cf7f0 0x7f5fd815d180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5fd815cc60 tx=0x7f5fc401c800 comp rx=0 tx=0).stop 2024-09-21T05:57:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.411+0000 7f5fdde00640 1 -- 172.21.15.50:0/498801918 shutdown_connections 2024-09-21T05:57:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.411+0000 7f5fdde00640 1 --2- 172.21.15.50:0/498801918 >> v2:172.21.15.88:3300/0 conn(0x7f5fd815d6c0 0x7f5fd815dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.411+0000 7f5fdde00640 1 --2- 172.21.15.50:0/498801918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd80cf7f0 0x7f5fd815d180 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.411+0000 7f5fdde00640 1 -- 172.21.15.50:0/498801918 >> 172.21.15.50:0/498801918 conn(0x7f5fd80b2040 msgr2=0x7f5fd80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.411+0000 7f5fdde00640 1 -- 172.21.15.50:0/498801918 shutdown_connections 2024-09-21T05:57:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.411+0000 7f5fdde00640 1 -- 172.21.15.50:0/498801918 wait complete. 2024-09-21T05:57:07.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fdde00640 1 Processor -- start 2024-09-21T05:57:07.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fdde00640 1 -- start start 2024-09-21T05:57:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fdde00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fd80cf7f0 0x7f5fd8154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fdde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd815d6c0 0x7f5fd81f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fdde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fd815eeb0 0x7f5fd80b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fdde00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd8169440 con 0x7f5fd815d6c0 2024-09-21T05:57:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fdde00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd81692c0 con 0x7f5fd815eeb0 2024-09-21T05:57:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fdde00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd81695c0 con 0x7f5fd80cf7f0 2024-09-21T05:57:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fd6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd815d6c0 0x7f5fd81f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fd6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd815d6c0 0x7f5fd81f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56436/0 (socket says 172.21.15.50:56436) 2024-09-21T05:57:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fd6000640 1 -- 172.21.15.50:0/298546093 learned_addr learned my addr 172.21.15.50:0/298546093 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fd6a00640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fd80cf7f0 0x7f5fd8154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fd7400640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fd815eeb0 0x7f5fd80b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fd6000640 1 -- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fd80cf7f0 msgr2=0x7f5fd8154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fd6000640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fd80cf7f0 0x7f5fd8154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fd6000640 1 -- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fd815eeb0 msgr2=0x7f5fd80b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fd6000640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fd815eeb0 0x7f5fd80b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.415+0000 7f5fd6000640 1 -- 172.21.15.50:0/298546093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd8154670 con 0x7f5fd815d6c0 2024-09-21T05:57:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.419+0000 7f5fd6000640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd815d6c0 0x7f5fd81f6e90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4019410 tx=0x7f5fc40a66f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.419+0000 7f5fcf400640 1 -- 172.21.15.50:0/298546093 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40040c0 con 0x7f5fd815d6c0 2024-09-21T05:57:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.419+0000 7f5fcf400640 1 -- 172.21.15.50:0/298546093 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fc40be070 con 0x7f5fd815d6c0 2024-09-21T05:57:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.419+0000 7f5fcf400640 1 -- 172.21.15.50:0/298546093 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4005030 con 0x7f5fd815d6c0 2024-09-21T05:57:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.419+0000 7f5fdde00640 1 -- 172.21.15.50:0/298546093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd81f7c00 con 0x7f5fd815d6c0 2024-09-21T05:57:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.419+0000 7f5fdde00640 1 -- 172.21.15.50:0/298546093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd81f81e0 con 0x7f5fd815d6c0 2024-09-21T05:57:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.419+0000 7f5fdde00640 1 -- 172.21.15.50:0/298546093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd81f7970 con 0x7f5fd815d6c0 2024-09-21T05:57:07.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.419+0000 7f5fcf400640 1 -- 172.21.15.50:0/298546093 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5fc40051d0 con 0x7f5fd815d6c0 2024-09-21T05:57:07.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.419+0000 7f5fcf400640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fb8073200 0x7f5fb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.419+0000 7f5fcf400640 1 -- 172.21.15.50:0/298546093 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7f5fc412e950 con 0x7f5fd815d6c0 2024-09-21T05:57:07.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.423+0000 7f5fd6a00640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fb8073200 0x7f5fb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.423+0000 7f5fd6a00640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fb8073200 0x7f5fb8075710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8001870 tx=0x7f5fc8001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:07.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.427+0000 7f5fcf400640 1 -- 172.21.15.50:0/298546093 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5fc40c7020 con 0x7f5fd815d6c0 2024-09-21T05:57:07.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:07.603+0000 7f5fdde00640 1 -- 172.21.15.50:0/298546093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5fd815d180 con 0x7f5fd815d6c0 2024-09-21T05:57:08.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.219+0000 7f5fcf400640 1 -- 172.21.15.50:0/298546093 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f5fc40163f0 con 0x7f5fd815d6c0 2024-09-21T05:57:08.223 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-09-21T05:57:08.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.223+0000 7f5fdde00640 1 -- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fb8073200 msgr2=0x7f5fb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:08.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.223+0000 7f5fdde00640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fb8073200 0x7f5fb8075710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8001870 tx=0x7f5fc8001380 comp rx=0 tx=0).stop 2024-09-21T05:57:08.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.223+0000 7f5fdde00640 1 -- 172.21.15.50:0/298546093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd815d6c0 msgr2=0x7f5fd81f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:08.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.223+0000 7f5fdde00640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd815d6c0 0x7f5fd81f6e90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4019410 tx=0x7f5fc40a66f0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.223+0000 7f5fdde00640 1 -- 172.21.15.50:0/298546093 shutdown_connections 2024-09-21T05:57:08.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.223+0000 7f5fdde00640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fd815eeb0 0x7f5fd80b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.223+0000 7f5fdde00640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fb8073200 0x7f5fb8075710 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.223+0000 7f5fdde00640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd815d6c0 0x7f5fd81f6e90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.223+0000 7f5fdde00640 1 --2- 172.21.15.50:0/298546093 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fd80cf7f0 0x7f5fd8154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.223+0000 7f5fdde00640 1 -- 172.21.15.50:0/298546093 >> 172.21.15.50:0/298546093 conn(0x7f5fd80b2040 msgr2=0x7f5fd815b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:08.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.223+0000 7f5fdde00640 1 -- 172.21.15.50:0/298546093 shutdown_connections 2024-09-21T05:57:08.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.223+0000 7f5fdde00640 1 -- 172.21.15.50:0/298546093 wait complete. 2024-09-21T05:57:08.253 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T05:57:08.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.415+0000 7f43b4000640 1 -- 172.21.15.50:0/2444246705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ac0cefa0 msgr2=0x7f43ac0cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:08.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.415+0000 7f43b4000640 1 --2- 172.21.15.50:0/2444246705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ac0cefa0 0x7f43ac0cf3b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4398009840 tx=0x7f439801c820 comp rx=0 tx=0).stop 2024-09-21T05:57:08.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.415+0000 7f43b4000640 1 -- 172.21.15.50:0/2444246705 shutdown_connections 2024-09-21T05:57:08.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.415+0000 7f43b4000640 1 --2- 172.21.15.50:0/2444246705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ac0cefa0 0x7f43ac0cf3b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.415+0000 7f43b4000640 1 --2- 172.21.15.50:0/2444246705 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43ac0749a0 0x7f43ac074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.415+0000 7f43b4000640 1 -- 172.21.15.50:0/2444246705 >> 172.21.15.50:0/2444246705 conn(0x7f43ac0b2040 msgr2=0x7f43ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:08.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.415+0000 7f43b4000640 1 -- 172.21.15.50:0/2444246705 shutdown_connections 2024-09-21T05:57:08.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.415+0000 7f43b4000640 1 -- 172.21.15.50:0/2444246705 wait complete. 2024-09-21T05:57:08.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b4000640 1 Processor -- start 2024-09-21T05:57:08.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b4000640 1 -- start start 2024-09-21T05:57:08.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b4000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43ac0749a0 0x7f43ac15dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:08.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b4000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ac0cefa0 0x7f43ac15a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:08.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b4000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ac160330 0x7f43ac0afe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:08.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b4000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac16a060 con 0x7f43ac160330 2024-09-21T05:57:08.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b4000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac169ee0 con 0x7f43ac0cefa0 2024-09-21T05:57:08.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b4000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f43ac16a1e0 con 0x7f43ac0749a0 2024-09-21T05:57:08.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ac160330 0x7f43ac0afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:08.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ac160330 0x7f43ac0afe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56458/0 (socket says 172.21.15.50:56458) 2024-09-21T05:57:08.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b2000640 1 -- 172.21.15.50:0/1121075376 learned_addr learned my addr 172.21.15.50:0/1121075376 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:08.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b1600640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43ac0749a0 0x7f43ac15dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:08.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b0c00640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ac0cefa0 0x7f43ac15a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:08.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b2000640 1 -- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43ac0749a0 msgr2=0x7f43ac15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:08.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b2000640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43ac0749a0 0x7f43ac15dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b2000640 1 -- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ac0cefa0 msgr2=0x7f43ac15a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:08.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b2000640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ac0cefa0 0x7f43ac15a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b2000640 1 -- 172.21.15.50:0/1121075376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac15a760 con 0x7f43ac160330 2024-09-21T05:57:08.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b0c00640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ac0cefa0 0x7f43ac15a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b2000640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ac160330 0x7f43ac0afe60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f43a400cb00 tx=0x7f43a400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43a9600640 1 -- 172.21.15.50:0/1121075376 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4016020 con 0x7f43ac160330 2024-09-21T05:57:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43a9600640 1 -- 172.21.15.50:0/1121075376 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43a400aae0 con 0x7f43ac160330 2024-09-21T05:57:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b4000640 1 -- 172.21.15.50:0/1121075376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ac15e7c0 con 0x7f43ac160330 2024-09-21T05:57:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43a9600640 1 -- 172.21.15.50:0/1121075376 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a400ce40 con 0x7f43ac160330 2024-09-21T05:57:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.419+0000 7f43b4000640 1 -- 172.21.15.50:0/1121075376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ac15ed80 con 0x7f43ac160330 2024-09-21T05:57:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.423+0000 7f43b4000640 1 -- 172.21.15.50:0/1121075376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ac074db0 con 0x7f43ac160330 2024-09-21T05:57:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.423+0000 7f43a9600640 1 -- 172.21.15.50:0/1121075376 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f43a4019020 con 0x7f43ac160330 2024-09-21T05:57:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.423+0000 7f43a9600640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f438c073230 0x7f438c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.423+0000 7f43a9600640 1 -- 172.21.15.50:0/1121075376 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5357+0+0 (secure 0 0 0) 0x7f43a4094310 con 0x7f43ac160330 2024-09-21T05:57:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.423+0000 7f43b1600640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f438c073230 0x7f438c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.427+0000 7f43b1600640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f438c073230 0x7f438c075740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43a000bb50 tx=0x7f43a000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.431+0000 7f43a9600640 1 -- 172.21.15.50:0/1121075376 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f43a4020020 con 0x7f43ac160330 2024-09-21T05:57:08.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:08.611+0000 7f43b4000640 1 -- 172.21.15.50:0/1121075376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f43ac15afc0 con 0x7f43ac160330 2024-09-21T05:57:09.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.227+0000 7f43a9600640 1 -- 172.21.15.50:0/1121075376 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f43a405f500 con 0x7f43ac160330 2024-09-21T05:57:09.231 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-09-21T05:57:09.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.231+0000 7f43b4000640 1 -- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f438c073230 msgr2=0x7f438c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:09.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.231+0000 7f43b4000640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f438c073230 0x7f438c075740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43a000bb50 tx=0x7f43a000b2e0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.231+0000 7f43b4000640 1 -- 172.21.15.50:0/1121075376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ac160330 msgr2=0x7f43ac0afe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:09.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.231+0000 7f43b4000640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ac160330 0x7f43ac0afe60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f43a400cb00 tx=0x7f43a400cbf0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.231+0000 7f43b4000640 1 -- 172.21.15.50:0/1121075376 shutdown_connections 2024-09-21T05:57:09.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.231+0000 7f43b4000640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f438c073230 0x7f438c075740 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.231+0000 7f43b4000640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ac160330 0x7f43ac0afe60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.231+0000 7f43b4000640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ac0cefa0 0x7f43ac15a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.231+0000 7f43b4000640 1 --2- 172.21.15.50:0/1121075376 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43ac0749a0 0x7f43ac15dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.231+0000 7f43b4000640 1 -- 172.21.15.50:0/1121075376 >> 172.21.15.50:0/1121075376 conn(0x7f43ac0b2040 msgr2=0x7f43ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:09.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.231+0000 7f43b4000640 1 -- 172.21.15.50:0/1121075376 shutdown_connections 2024-09-21T05:57:09.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.231+0000 7f43b4000640 1 -- 172.21.15.50:0/1121075376 wait complete. 2024-09-21T05:57:09.257 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T05:57:09.257 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T05:57:09.257 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T05:57:09.257 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:57:09.257 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T05:57:09.257 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T05:57:09.258 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T05:57:09.258 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:57:09.258 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T05:57:09.258 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T05:57:09.258 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T05:57:09.258 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:57:09.258 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T05:57:09.258 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T05:57:09.259 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T05:57:09.259 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:57:09.270 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T05:57:09.270 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T05:57:09.271 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T05:57:09.271 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T05:57:09.274 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T05:57:09.274 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T05:57:09.281 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T05:57:09.282 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T05:57:09.315 INFO:tasks.ceph.mds.a:Started 2024-09-21T05:57:09.315 DEBUG:teuthology.parallel:result is None 2024-09-21T05:57:09.317 INFO:tasks.ceph.mds.c:Started 2024-09-21T05:57:09.317 DEBUG:teuthology.parallel:result is None 2024-09-21T05:57:09.325 INFO:tasks.ceph.mds.d:Started 2024-09-21T05:57:09.325 DEBUG:teuthology.parallel:result is None 2024-09-21T05:57:09.327 INFO:tasks.ceph.mds.b:Started 2024-09-21T05:57:09.327 DEBUG:teuthology.parallel:result is None 2024-09-21T05:57:09.327 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T05:57:09.415 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T05:57:09.418 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-09-21T05:57:09.425 INFO:tasks.ceph.mds.d.smithi088.stdout:starting mds.d at 2024-09-21T05:57:09.428 INFO:tasks.ceph.mds.b.smithi088.stdout:starting mds.b at 2024-09-21T05:57:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.491+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1059091459 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb2815c920 msgr2=0x7ffb280ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.491+0000 7ffb2ee00640 1 --2- 172.21.15.50:0/1059091459 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb2815c920 0x7ffb280ce190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009bf0 tx=0x7ffb2001cb50 comp rx=0 tx=0).stop 2024-09-21T05:57:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.491+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1059091459 shutdown_connections 2024-09-21T05:57:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.491+0000 7ffb2ee00640 1 --2- 172.21.15.50:0/1059091459 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb2815c920 0x7ffb280ce190 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.491+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1059091459 >> 172.21.15.50:0/1059091459 conn(0x7ffb280b41f0 msgr2=0x7ffb280b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.491+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1059091459 shutdown_connections 2024-09-21T05:57:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.491+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1059091459 wait complete. 2024-09-21T05:57:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.491+0000 7ffb2ee00640 1 Processor -- start 2024-09-21T05:57:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.491+0000 7ffb2ee00640 1 -- start start 2024-09-21T05:57:09.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.491+0000 7ffb2ee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb280746e0 0x7ffb2809a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:09.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2ee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb2815c920 0x7ffb280cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:09.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2ee00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb280d0b50 0x7ffb280b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:09.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2ee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2816b490 con 0x7ffb2815c920 2024-09-21T05:57:09.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2ee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2816b310 con 0x7ffb280746e0 2024-09-21T05:57:09.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2ee00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2816b610 con 0x7ffb280d0b50 2024-09-21T05:57:09.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb2815c920 0x7ffb280cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:09.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb280746e0 0x7ffb2809a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:09.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb2815c920 0x7ffb280cedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56500/0 (socket says 172.21.15.50:56500) 2024-09-21T05:57:09.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2d000640 1 -- 172.21.15.50:0/1193248435 learned_addr learned my addr 172.21.15.50:0/1193248435 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:09.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2e400640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb280d0b50 0x7ffb280b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:09.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2d000640 1 -- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb280d0b50 msgr2=0x7ffb280b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:09.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2d000640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb280d0b50 0x7ffb280b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2d000640 1 -- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb280746e0 msgr2=0x7ffb2809a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:09.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2d000640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb280746e0 0x7ffb2809a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2d000640 1 -- 172.21.15.50:0/1193248435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb280cfa90 con 0x7ffb2815c920 2024-09-21T05:57:09.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2da00640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb280746e0 0x7ffb2809a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:09.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2d000640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb2815c920 0x7ffb280cedd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400c410 tx=0x7ffb1400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:09.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb25600640 1 -- 172.21.15.50:0/1193248435 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14017070 con 0x7ffb2815c920 2024-09-21T05:57:09.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb25600640 1 -- 172.21.15.50:0/1193248435 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb140053c0 con 0x7ffb2815c920 2024-09-21T05:57:09.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1193248435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb280d0070 con 0x7ffb2815c920 2024-09-21T05:57:09.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb25600640 1 -- 172.21.15.50:0/1193248435 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14003860 con 0x7ffb2815c920 2024-09-21T05:57:09.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1193248435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2809a5c0 con 0x7ffb2815c920 2024-09-21T05:57:09.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1193248435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb280cf800 con 0x7ffb2815c920 2024-09-21T05:57:09.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.495+0000 7ffb25600640 1 -- 172.21.15.50:0/1193248435 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffb1401a070 con 0x7ffb2815c920 2024-09-21T05:57:09.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.499+0000 7ffb25600640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ffb08073200 0x7ffb08075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:09.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.499+0000 7ffb25600640 1 -- 172.21.15.50:0/1193248435 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7ffb14093310 con 0x7ffb2815c920 2024-09-21T05:57:09.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.499+0000 7ffb2da00640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ffb08073200 0x7ffb08075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:09.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.499+0000 7ffb2da00640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ffb08073200 0x7ffb08075710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c007b60 tx=0x7ffb1c007ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.503+0000 7ffb25600640 1 -- 172.21.15.50:0/1193248435 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ffb1405df40 con 0x7ffb2815c920 2024-09-21T05:57:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:09.691+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1193248435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ffb28074c30 con 0x7ffb2815c920 2024-09-21T05:57:10.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.239+0000 7ffb25600640 1 -- 172.21.15.50:0/1193248435 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7ffb14062970 con 0x7ffb2815c920 2024-09-21T05:57:10.243 INFO:teuthology.orchestra.run.smithi050.stderr: removed all blocklist entries 2024-09-21T05:57:10.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.243+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ffb08073200 msgr2=0x7ffb08075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.243+0000 7ffb2ee00640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ffb08073200 0x7ffb08075710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c007b60 tx=0x7ffb1c007ab0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.243+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1193248435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb2815c920 msgr2=0x7ffb280cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.243+0000 7ffb2ee00640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb2815c920 0x7ffb280cedd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400c410 tx=0x7ffb1400c8d0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.243+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1193248435 shutdown_connections 2024-09-21T05:57:10.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.243+0000 7ffb2ee00640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb280d0b50 0x7ffb280b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.243+0000 7ffb2ee00640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ffb08073200 0x7ffb08075710 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.243+0000 7ffb2ee00640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb2815c920 0x7ffb280cedd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.243+0000 7ffb2ee00640 1 --2- 172.21.15.50:0/1193248435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb280746e0 0x7ffb2809a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.243+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1193248435 >> 172.21.15.50:0/1193248435 conn(0x7ffb280b41f0 msgr2=0x7ffb280cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:10.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.243+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1193248435 shutdown_connections 2024-09-21T05:57:10.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.243+0000 7ffb2ee00640 1 -- 172.21.15.50:0/1193248435 wait complete. 2024-09-21T05:57:10.270 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T05:57:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.423+0000 7f85bf200640 1 -- 172.21.15.50:0/3380885471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85c0160470 msgr2=0x7f85c0074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.423+0000 7f85bf200640 1 --2- 172.21.15.50:0/3380885471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85c0160470 0x7f85c0074780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f85c00ced40 tx=0x7f85b801caf0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.423+0000 7f85bf200640 1 -- 172.21.15.50:0/3380885471 shutdown_connections 2024-09-21T05:57:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.423+0000 7f85bf200640 1 --2- 172.21.15.50:0/3380885471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85c0160470 0x7f85c0074780 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.423+0000 7f85bf200640 1 --2- 172.21.15.50:0/3380885471 >> v2:172.21.15.88:3300/0 conn(0x7f85c0160060 0x7f85c0074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.423+0000 7f85bf200640 1 -- 172.21.15.50:0/3380885471 >> 172.21.15.50:0/3380885471 conn(0x7f85c00b4560 msgr2=0x7f85c00d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.423+0000 7f85bf200640 1 -- 172.21.15.50:0/3380885471 shutdown_connections 2024-09-21T05:57:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.423+0000 7f85bf200640 1 -- 172.21.15.50:0/3380885471 wait complete. 2024-09-21T05:57:10.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85bf200640 1 Processor -- start 2024-09-21T05:57:10.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85bf200640 1 -- start start 2024-09-21T05:57:10.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85bf200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85c01536f0 0x7f85c014fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85bf200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85c0160060 0x7f85c014cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85bf200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85c0160470 0x7f85c0151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85bf200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f85c01696e0 con 0x7f85c0160470 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85bf200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85c0169560 con 0x7f85c01536f0 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85bf200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f85c0169860 con 0x7f85c0160060 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85be800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85c0160470 0x7f85c0151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85be800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85c0160470 0x7f85c0151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56512/0 (socket says 172.21.15.50:56512) 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85bde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85c01536f0 0x7f85c014fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85be800640 1 -- 172.21.15.50:0/3569633003 learned_addr learned my addr 172.21.15.50:0/3569633003 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85b5400640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85c0160060 0x7f85c014cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85be800640 1 -- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85c0160060 msgr2=0x7f85c014cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.427+0000 7f85be800640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85c0160060 0x7f85c014cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.431+0000 7f85be800640 1 -- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85c01536f0 msgr2=0x7f85c014fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.431+0000 7f85be800640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85c01536f0 0x7f85c014fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.431+0000 7f85be800640 1 -- 172.21.15.50:0/3569633003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c01509a0 con 0x7f85c0160470 2024-09-21T05:57:10.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.431+0000 7f85be800640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85c0160470 0x7f85c0151a40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f85b80984e0 tx=0x7f85b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:10.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.431+0000 7f85b6a00640 1 -- 172.21.15.50:0/3569633003 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b80bb070 con 0x7f85c0160470 2024-09-21T05:57:10.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.431+0000 7f85b6a00640 1 -- 172.21.15.50:0/3569633003 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85b8004a50 con 0x7f85c0160470 2024-09-21T05:57:10.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.431+0000 7f85b6a00640 1 -- 172.21.15.50:0/3569633003 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b80b2040 con 0x7f85c0160470 2024-09-21T05:57:10.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.431+0000 7f85bf200640 1 -- 172.21.15.50:0/3569633003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c0150c80 con 0x7f85c0160470 2024-09-21T05:57:10.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.431+0000 7f85bf200640 1 -- 172.21.15.50:0/3569633003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c0150420 con 0x7f85c0160470 2024-09-21T05:57:10.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.431+0000 7f85bf200640 1 -- 172.21.15.50:0/3569633003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c0150710 con 0x7f85c0160470 2024-09-21T05:57:10.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.431+0000 7f85b6a00640 1 -- 172.21.15.50:0/3569633003 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f85b8004460 con 0x7f85c0160470 2024-09-21T05:57:10.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.431+0000 7f85b6a00640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f859c073230 0x7f859c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.435+0000 7f85b6a00640 1 -- 172.21.15.50:0/3569633003 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f85b812de30 con 0x7f85c0160470 2024-09-21T05:57:10.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.435+0000 7f85bde00640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f859c073230 0x7f859c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.435+0000 7f85bde00640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f859c073230 0x7f859c075740 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f85b0001870 tx=0x7f85b0001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:10.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.439+0000 7f85b6a00640 1 -- 172.21.15.50:0/3569633003 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f85b8016770 con 0x7f85c0160470 2024-09-21T05:57:10.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.671+0000 7f85bf200640 1 -- 172.21.15.50:0/3569633003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f85c015e600 con 0x7f85c0160470 2024-09-21T05:57:10.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85b6a00640 1 -- 172.21.15.50:0/3569633003 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+6645 (secure 0 0 0) 0x7f85b80f8b40 con 0x7f85c0160470 2024-09-21T05:57:10.678 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:10.678 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T05:57:10.678 INFO:teuthology.orchestra.run.smithi050.stdout: "auth_dump": [ 2024-09-21T05:57:10.678 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.678 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.a", 2024-09-21T05:57:10.678 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mSKq0ARAA2egaHGYWhRUEq6oRl1CIMg==", 2024-09-21T05:57:10.678 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.678 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:57:10.678 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T05:57:10.678 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.b", 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mwv0+DxAAY+eZPxv5uY9PnLl0asdr6w==", 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.679 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.c", 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5me6maCRAA4jvHgTLTKpb/mxzzG71SfQ==", 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.d", 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mzYnuFhAA/hEigjluXspzs/hyeWxBjQ==", 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.680 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.0", 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDIX+5mrZF/NhAA33M6ummG4r4yXU8/YOL5WA==", 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.681 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.1", 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDMX+5myVz5DxAAMYfTPep6ubD7Cxnfa9dMCg==", 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.2", 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDPX+5mA7ahIRAAf/osnY5l83p8ZWna0ErD2Q==", 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.682 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:57:10.683 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:57:10.683 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.683 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.683 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.683 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.683 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.3", 2024-09-21T05:57:10.683 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDSX+5m+rDpNRAApGPZJIlAaA0DyjtPXPgoaQ==", 2024-09-21T05:57:10.683 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.683 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:57:10.683 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:57:10.683 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.684 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.684 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.684 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.684 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.4", 2024-09-21T05:57:10.685 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDYX+5mKL9SORAA21ITQUwUSoDUjXaD0Cwb3g==", 2024-09-21T05:57:10.685 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.685 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:57:10.685 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:57:10.685 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.5", 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDcX+5minRqFBAAMJOBQNvJFD4TNdKbPMLBig==", 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.686 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.6", 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDfX+5m6HadJhAAAv670Tvyi4IKMByFPiBlAw==", 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.7", 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDjX+5muPrrAhAAuRB7Muamiht+f7LvSmI6UQ==", 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:57:10.687 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.688 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.688 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.688 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.688 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.0", 2024-09-21T05:57:10.688 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw==", 2024-09-21T05:57:10.688 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.688 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:57:10.688 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T05:57:10.688 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T05:57:10.688 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T05:57:10.688 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.1", 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mo94GJBAAMBSb1NPb2L93vzvnAh59kw==", 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.2", 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5m4IrnHxAAFjER3ywuS+FlbuA7bPZVqw==", 2024-09-21T05:57:10.689 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.690 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:57:10.690 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T05:57:10.690 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T05:57:10.690 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T05:57:10.690 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.690 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.690 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.690 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.3", 2024-09-21T05:57:10.690 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mysQeKBAAq5c4bD3hbsU1H7JabaMjwQ==", 2024-09-21T05:57:10.690 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.690 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.admin", 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mcUXKHhAAV1Rs0OMNVPeAsRiAVt67jA==", 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.691 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-mds", 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDzX+5mFKtNIBAAl1VEOleD6c80j6l9YuXQyA==", 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-mds" 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-mgr", 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDzX+5m07VNIBAA0TP+nYSRvl9GullbjjWzbg==", 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.692 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-osd", 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDzX+5mab9NIBAAQ6Bj8bzdzB/1XKKfvGbqdA==", 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-osd" 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-rbd", 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDzX+5me8hNIBAAdflfnkqSvdrYOP+e3Bbd+A==", 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.693 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDzX+5meNNNIBAA6omgM7bNNUtSXaIBTbynSw==", 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-rgw", 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDzX+5mWd1NIBAAIw+kxKVjLgopNWIsBf7rIQ==", 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.x", 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5m2F9WOBAAZuQTUyR7oTT01CRD1jo+/Q==", 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.y", 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mEjC/MxAAHgwIu4CLDgFajwD2rL5WKA==", 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T05:57:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T05:57:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:57:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-21T05:57:10.696 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T05:57:10.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85bf200640 1 -- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f859c073230 msgr2=0x7f859c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85bf200640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f859c073230 0x7f859c075740 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f85b0001870 tx=0x7f85b0001380 comp rx=0 tx=0).stop 2024-09-21T05:57:10.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85bf200640 1 -- 172.21.15.50:0/3569633003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85c0160470 msgr2=0x7f85c0151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85bf200640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85c0160470 0x7f85c0151a40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f85b80984e0 tx=0x7f85b8004270 comp rx=0 tx=0).stop 2024-09-21T05:57:10.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85bf200640 1 -- 172.21.15.50:0/3569633003 shutdown_connections 2024-09-21T05:57:10.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85bf200640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f859c073230 0x7f859c075740 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85bf200640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85c0160470 0x7f85c0151a40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85bf200640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85c0160060 0x7f85c014cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85bf200640 1 --2- 172.21.15.50:0/3569633003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85c01536f0 0x7f85c014fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85bf200640 1 -- 172.21.15.50:0/3569633003 >> 172.21.15.50:0/3569633003 conn(0x7f85c00b4560 msgr2=0x7f85c00715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:10.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85bf200640 1 -- 172.21.15.50:0/3569633003 shutdown_connections 2024-09-21T05:57:10.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.675+0000 7f85bf200640 1 -- 172.21.15.50:0/3569633003 wait complete. 2024-09-21T05:57:10.700 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-21T05:57:10.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.855+0000 7f9a26e00640 1 -- 172.21.15.50:0/2915859074 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9a2015efd0 msgr2=0x7f9a2015d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.855+0000 7f9a26e00640 1 --2- 172.21.15.50:0/2915859074 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9a2015efd0 0x7f9a2015d180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00a950 tx=0x7f9a0c01dae0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.855+0000 7f9a26e00640 1 -- 172.21.15.50:0/2915859074 shutdown_connections 2024-09-21T05:57:10.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.855+0000 7f9a26e00640 1 --2- 172.21.15.50:0/2915859074 >> v2:172.21.15.88:3300/0 conn(0x7f9a2015d6c0 0x7f9a2015dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.855+0000 7f9a26e00640 1 --2- 172.21.15.50:0/2915859074 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9a2015efd0 0x7f9a2015d180 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.855+0000 7f9a26e00640 1 -- 172.21.15.50:0/2915859074 >> 172.21.15.50:0/2915859074 conn(0x7f9a200b2040 msgr2=0x7f9a200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:10.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.855+0000 7f9a26e00640 1 -- 172.21.15.50:0/2915859074 shutdown_connections 2024-09-21T05:57:10.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.855+0000 7f9a26e00640 1 -- 172.21.15.50:0/2915859074 wait complete. 2024-09-21T05:57:10.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a26e00640 1 Processor -- start 2024-09-21T05:57:10.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a26e00640 1 -- start start 2024-09-21T05:57:10.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a26e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9a200cf900 0x7f9a2013d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a26e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2015d6c0 0x7f9a20165ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a26e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a2013d940 0x7f9a20166220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a26e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20168110 con 0x7f9a2013d940 2024-09-21T05:57:10.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a26e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20167f90 con 0x7f9a2015d6c0 2024-09-21T05:57:10.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a26e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9a20168290 con 0x7f9a200cf900 2024-09-21T05:57:10.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a24e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a2013d940 0x7f9a20166220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a24e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a2013d940 0x7f9a20166220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56528/0 (socket says 172.21.15.50:56528) 2024-09-21T05:57:10.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a24e00640 1 -- 172.21.15.50:0/3235413305 learned_addr learned my addr 172.21.15.50:0/3235413305 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:10.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a1f400640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2015d6c0 0x7f9a20165ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a1fe00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9a200cf900 0x7f9a2013d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a24e00640 1 -- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9a200cf900 msgr2=0x7f9a2013d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a24e00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9a200cf900 0x7f9a2013d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a24e00640 1 -- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2015d6c0 msgr2=0x7f9a20165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a24e00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2015d6c0 0x7f9a20165ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a24e00640 1 -- 172.21.15.50:0/3235413305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a20152440 con 0x7f9a2013d940 2024-09-21T05:57:10.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.859+0000 7f9a1f400640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2015d6c0 0x7f9a20165ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:10.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.863+0000 7f9a24e00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a2013d940 0x7f9a20166220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800ec30 tx=0x7f9a1800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:10.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.863+0000 7f9a17e00640 1 -- 172.21.15.50:0/3235413305 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18003b80 con 0x7f9a2013d940 2024-09-21T05:57:10.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.863+0000 7f9a17e00640 1 -- 172.21.15.50:0/3235413305 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a18003e10 con 0x7f9a2013d940 2024-09-21T05:57:10.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.863+0000 7f9a26e00640 1 -- 172.21.15.50:0/3235413305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a20151e80 con 0x7f9a2013d940 2024-09-21T05:57:10.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.863+0000 7f9a17e00640 1 -- 172.21.15.50:0/3235413305 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18004280 con 0x7f9a2013d940 2024-09-21T05:57:10.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.863+0000 7f9a26e00640 1 -- 172.21.15.50:0/3235413305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2013d6c0 con 0x7f9a2013d940 2024-09-21T05:57:10.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.863+0000 7f9a26e00640 1 -- 172.21.15.50:0/3235413305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a201fce00 con 0x7f9a2013d940 2024-09-21T05:57:10.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.863+0000 7f9a17e00640 1 -- 172.21.15.50:0/3235413305 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9a18004420 con 0x7f9a2013d940 2024-09-21T05:57:10.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.863+0000 7f9a17e00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9a00073230 0x7f9a00075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.863+0000 7f9a17e00640 1 -- 172.21.15.50:0/3235413305 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f9a18094c60 con 0x7f9a2013d940 2024-09-21T05:57:10.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.867+0000 7f9a1fe00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9a00073230 0x7f9a00075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.867+0000 7f9a1fe00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9a00073230 0x7f9a00075740 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9a10001640 tx=0x7f9a10009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:10.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:10.867+0000 7f9a17e00640 1 -- 172.21.15.50:0/3235413305 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9a20152440 con 0x7f9a2013d940 2024-09-21T05:57:11.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.111+0000 7f9a26e00640 1 -- 172.21.15.50:0/3235413305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f9a2015dad0 con 0x7f9a2013d940 2024-09-21T05:57:11.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.111+0000 7f9a17e00640 1 -- 172.21.15.50:0/3235413305 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v23) ==== 90+0+0 (secure 0 0 0) 0x7f9a18060060 con 0x7f9a2013d940 2024-09-21T05:57:11.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.115+0000 7f9a26e00640 1 -- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9a00073230 msgr2=0x7f9a00075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.115+0000 7f9a26e00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9a00073230 0x7f9a00075740 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9a10001640 tx=0x7f9a10009040 comp rx=0 tx=0).stop 2024-09-21T05:57:11.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.115+0000 7f9a26e00640 1 -- 172.21.15.50:0/3235413305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a2013d940 msgr2=0x7f9a20166220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.115+0000 7f9a26e00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a2013d940 0x7f9a20166220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800ec30 tx=0x7f9a1800c6a0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.115+0000 7f9a26e00640 1 -- 172.21.15.50:0/3235413305 shutdown_connections 2024-09-21T05:57:11.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.115+0000 7f9a26e00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9a00073230 0x7f9a00075740 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.115+0000 7f9a26e00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a2013d940 0x7f9a20166220 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.115+0000 7f9a26e00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2015d6c0 0x7f9a20165ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.115+0000 7f9a26e00640 1 --2- 172.21.15.50:0/3235413305 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9a200cf900 0x7f9a2013d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.115+0000 7f9a26e00640 1 -- 172.21.15.50:0/3235413305 >> 172.21.15.50:0/3235413305 conn(0x7f9a200b2040 msgr2=0x7f9a200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:11.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.115+0000 7f9a26e00640 1 -- 172.21.15.50:0/3235413305 shutdown_connections 2024-09-21T05:57:11.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.115+0000 7f9a26e00640 1 -- 172.21.15.50:0/3235413305 wait complete. 2024-09-21T05:57:11.139 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-21T05:57:11.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.299+0000 7fb02c800640 1 -- 172.21.15.50:0/262186427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c005260 msgr2=0x7fb00c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.299+0000 7fb02c800640 1 --2- 172.21.15.50:0/262186427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c005260 0x7fb00c00b9d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb020001dc0 tx=0x7fb02001cb30 comp rx=0 tx=0).stop 2024-09-21T05:57:11.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.299+0000 7fb02c800640 1 -- 172.21.15.50:0/262186427 shutdown_connections 2024-09-21T05:57:11.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.299+0000 7fb02c800640 1 --2- 172.21.15.50:0/262186427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c005260 0x7fb00c00b9d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.299+0000 7fb02c800640 1 -- 172.21.15.50:0/262186427 >> 172.21.15.50:0/262186427 conn(0x7fb0240b2040 msgr2=0x7fb0240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:11.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.299+0000 7fb02c800640 1 -- 172.21.15.50:0/262186427 shutdown_connections 2024-09-21T05:57:11.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.299+0000 7fb02c800640 1 -- 172.21.15.50:0/262186427 wait complete. 2024-09-21T05:57:11.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02c800640 1 Processor -- start 2024-09-21T05:57:11.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02c800640 1 -- start start 2024-09-21T05:57:11.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02c800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb00c005260 0x7fb024157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:11.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02c800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0241f9130 0x7fb0241f9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:11.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02c800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb024143ba0 0x7fb024143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:11.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02c800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0240b5f50 con 0x7fb024143ba0 2024-09-21T05:57:11.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02c800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb024070500 con 0x7fb0241f9130 2024-09-21T05:57:11.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02c800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb0240cd260 con 0x7fb00c005260 2024-09-21T05:57:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb024143ba0 0x7fb024143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb024143ba0 0x7fb024143fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56552/0 (socket says 172.21.15.50:56552) 2024-09-21T05:57:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02a800640 1 -- 172.21.15.50:0/154801364 learned_addr learned my addr 172.21.15.50:0/154801364 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb029400640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0241f9130 0x7fb0241f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb029e00640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb00c005260 0x7fb024157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02a800640 1 -- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb00c005260 msgr2=0x7fb024157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02a800640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb00c005260 0x7fb024157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02a800640 1 -- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0241f9130 msgr2=0x7fb0241f9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02a800640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0241f9130 0x7fb0241f9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02a800640 1 -- 172.21.15.50:0/154801364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0241f9a80 con 0x7fb024143ba0 2024-09-21T05:57:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02a800640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb024143ba0 0x7fb024143fb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb02001d3d0 tx=0x7fb020004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb01e000640 1 -- 172.21.15.50:0/154801364 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0200019f0 con 0x7fb024143ba0 2024-09-21T05:57:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb01e000640 1 -- 172.21.15.50:0/154801364 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0200b5070 con 0x7fb024143ba0 2024-09-21T05:57:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02c800640 1 -- 172.21.15.50:0/154801364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0241eb490 con 0x7fb024143ba0 2024-09-21T05:57:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb01e000640 1 -- 172.21.15.50:0/154801364 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0200065f0 con 0x7fb024143ba0 2024-09-21T05:57:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.303+0000 7fb02c800640 1 -- 172.21.15.50:0/154801364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0241e5e30 con 0x7fb024143ba0 2024-09-21T05:57:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.307+0000 7fb02c800640 1 -- 172.21.15.50:0/154801364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb024141410 con 0x7fb024143ba0 2024-09-21T05:57:11.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.307+0000 7fb01e000640 1 -- 172.21.15.50:0/154801364 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb0200ba470 con 0x7fb024143ba0 2024-09-21T05:57:11.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.307+0000 7fb01e000640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb008073230 0x7fb008075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:11.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.307+0000 7fb01e000640 1 -- 172.21.15.50:0/154801364 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fb02012eee0 con 0x7fb024143ba0 2024-09-21T05:57:11.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.307+0000 7fb029e00640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb008073230 0x7fb008075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:11.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.311+0000 7fb029e00640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb008073230 0x7fb008075740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb018006cc0 tx=0x7fb0180067b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:11.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.311+0000 7fb01e000640 1 -- 172.21.15.50:0/154801364 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb024141410 con 0x7fb024143ba0 2024-09-21T05:57:11.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.551+0000 7fb02c800640 1 -- 172.21.15.50:0/154801364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fb0240cd760 con 0x7fb024143ba0 2024-09-21T05:57:11.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb01e000640 1 -- 172.21.15.50:0/154801364 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v24) ==== 90+0+0 (secure 0 0 0) 0x7fb0200bf020 con 0x7fb024143ba0 2024-09-21T05:57:11.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb02c800640 1 -- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb008073230 msgr2=0x7fb008075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb02c800640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb008073230 0x7fb008075740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb018006cc0 tx=0x7fb0180067b0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb02c800640 1 -- 172.21.15.50:0/154801364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb024143ba0 msgr2=0x7fb024143fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb02c800640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb024143ba0 0x7fb024143fb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb02001d3d0 tx=0x7fb020004f90 comp rx=0 tx=0).stop 2024-09-21T05:57:11.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb02c800640 1 -- 172.21.15.50:0/154801364 shutdown_connections 2024-09-21T05:57:11.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb02c800640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb008073230 0x7fb008075740 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb02c800640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb024143ba0 0x7fb024143fb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb02c800640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0241f9130 0x7fb0241f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb02c800640 1 --2- 172.21.15.50:0/154801364 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb00c005260 0x7fb024157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb02c800640 1 -- 172.21.15.50:0/154801364 >> 172.21.15.50:0/154801364 conn(0x7fb0240b2040 msgr2=0x7fb0240b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:11.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb02c800640 1 -- 172.21.15.50:0/154801364 shutdown_connections 2024-09-21T05:57:11.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.555+0000 7fb02c800640 1 -- 172.21.15.50:0/154801364 wait complete. 2024-09-21T05:57:11.586 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-21T05:57:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.739+0000 7fb0d9800640 1 -- 172.21.15.50:0/1990567740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d40744d0 msgr2=0x7fb0d406ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.739+0000 7fb0d9800640 1 --2- 172.21.15.50:0/1990567740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d40744d0 0x7fb0d406ea60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009840 tx=0x7fb0bc01c7f0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 -- 172.21.15.50:0/1990567740 shutdown_connections 2024-09-21T05:57:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 --2- 172.21.15.50:0/1990567740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d40744d0 0x7fb0d406ea60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 -- 172.21.15.50:0/1990567740 >> 172.21.15.50:0/1990567740 conn(0x7fb0d40b2c40 msgr2=0x7fb0d40b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 -- 172.21.15.50:0/1990567740 shutdown_connections 2024-09-21T05:57:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 -- 172.21.15.50:0/1990567740 wait complete. 2024-09-21T05:57:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 Processor -- start 2024-09-21T05:57:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 -- start start 2024-09-21T05:57:11.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d406efa0 0x7fb0d41f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:11.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0d415c5d0 0x7fb0d41426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:11.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0d4150d30 0x7fb0d4154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:11.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4169f60 con 0x7fb0d406efa0 2024-09-21T05:57:11.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4169de0 con 0x7fb0d415c5d0 2024-09-21T05:57:11.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.743+0000 7fb0d9800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d416a0e0 con 0x7fb0d4150d30 2024-09-21T05:57:11.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d406efa0 0x7fb0d41f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:11.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d406efa0 0x7fb0d41f93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56570/0 (socket says 172.21.15.50:56570) 2024-09-21T05:57:11.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d2a00640 1 -- 172.21.15.50:0/573213134 learned_addr learned my addr 172.21.15.50:0/573213134 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:11.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0cbe00640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0d415c5d0 0x7fb0d41426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:11.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d3400640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0d4150d30 0x7fb0d4154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:11.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d2a00640 1 -- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0d4150d30 msgr2=0x7fb0d4154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d2a00640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0d4150d30 0x7fb0d4154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d2a00640 1 -- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0d415c5d0 msgr2=0x7fb0d41426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d2a00640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0d415c5d0 0x7fb0d41426b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d2a00640 1 -- 172.21.15.50:0/573213134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d41f98f0 con 0x7fb0d406efa0 2024-09-21T05:57:11.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d2a00640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d406efa0 0x7fb0d41f93b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400c3f0 tx=0x7fb0c400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:11.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0caa00640 1 -- 172.21.15.50:0/573213134 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4017070 con 0x7fb0d406efa0 2024-09-21T05:57:11.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0caa00640 1 -- 172.21.15.50:0/573213134 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0c40053c0 con 0x7fb0d406efa0 2024-09-21T05:57:11.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0caa00640 1 -- 172.21.15.50:0/573213134 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4003860 con 0x7fb0d406efa0 2024-09-21T05:57:11.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d9800640 1 -- 172.21.15.50:0/573213134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d4142bf0 con 0x7fb0d406efa0 2024-09-21T05:57:11.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d9800640 1 -- 172.21.15.50:0/573213134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d4150250 con 0x7fb0d406efa0 2024-09-21T05:57:11.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0d9800640 1 -- 172.21.15.50:0/573213134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d406ea60 con 0x7fb0d406efa0 2024-09-21T05:57:11.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.747+0000 7fb0caa00640 1 -- 172.21.15.50:0/573213134 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb0c401a070 con 0x7fb0d406efa0 2024-09-21T05:57:11.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.751+0000 7fb0caa00640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0b0073200 0x7fb0b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:11.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.751+0000 7fb0caa00640 1 -- 172.21.15.50:0/573213134 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fb0c4093190 con 0x7fb0d406efa0 2024-09-21T05:57:11.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.751+0000 7fb0cbe00640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0b0073200 0x7fb0b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:11.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.751+0000 7fb0cbe00640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0b0073200 0x7fb0b0075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0af000 tx=0x7fb0bc01b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:11.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.755+0000 7fb0caa00640 1 -- 172.21.15.50:0/573213134 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb0d41f98f0 con 0x7fb0d406efa0 2024-09-21T05:57:11.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.995+0000 7fb0d9800640 1 -- 172.21.15.50:0/573213134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fb0d4072de0 con 0x7fb0d406efa0 2024-09-21T05:57:12.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.995+0000 7fb0caa00640 1 -- 172.21.15.50:0/573213134 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v25) ==== 90+0+0 (secure 0 0 0) 0x7fb0c405df30 con 0x7fb0d406efa0 2024-09-21T05:57:12.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.999+0000 7fb0d9800640 1 -- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0b0073200 msgr2=0x7fb0b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:12.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.999+0000 7fb0d9800640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0b0073200 0x7fb0b0075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0af000 tx=0x7fb0bc01b030 comp rx=0 tx=0).stop 2024-09-21T05:57:12.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.999+0000 7fb0d9800640 1 -- 172.21.15.50:0/573213134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d406efa0 msgr2=0x7fb0d41f93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:12.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.999+0000 7fb0d9800640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d406efa0 0x7fb0d41f93b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400c3f0 tx=0x7fb0c400c8b0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.999+0000 7fb0d9800640 1 -- 172.21.15.50:0/573213134 shutdown_connections 2024-09-21T05:57:12.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.999+0000 7fb0d9800640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0d4150d30 0x7fb0d4154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.999+0000 7fb0d9800640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0d415c5d0 0x7fb0d41426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.999+0000 7fb0d9800640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0b0073200 0x7fb0b0075710 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.999+0000 7fb0d9800640 1 --2- 172.21.15.50:0/573213134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d406efa0 0x7fb0d41f93b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.999+0000 7fb0d9800640 1 -- 172.21.15.50:0/573213134 >> 172.21.15.50:0/573213134 conn(0x7fb0d40b2c40 msgr2=0x7fb0d40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:12.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.999+0000 7fb0d9800640 1 -- 172.21.15.50:0/573213134 shutdown_connections 2024-09-21T05:57:12.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:11.999+0000 7fb0d9800640 1 -- 172.21.15.50:0/573213134 wait complete. 2024-09-21T05:57:12.025 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-21T05:57:12.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.179+0000 7f983b200640 1 -- 172.21.15.50:0/3944734558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98341527b0 msgr2=0x7f9834074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:12.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.179+0000 7f983b200640 1 --2- 172.21.15.50:0/3944734558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98341527b0 0x7f9834074e30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9834071cb0 tx=0x7f982001c740 comp rx=0 tx=0).stop 2024-09-21T05:57:12.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.179+0000 7f983b200640 1 -- 172.21.15.50:0/3944734558 shutdown_connections 2024-09-21T05:57:12.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.179+0000 7f983b200640 1 --2- 172.21.15.50:0/3944734558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98341527b0 0x7f9834074e30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.179+0000 7f983b200640 1 --2- 172.21.15.50:0/3944734558 >> v2:172.21.15.88:3300/0 conn(0x7f98340b5c70 0x7f983415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.179+0000 7f983b200640 1 -- 172.21.15.50:0/3944734558 >> 172.21.15.50:0/3944734558 conn(0x7f98340b2040 msgr2=0x7f98340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:12.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.179+0000 7f983b200640 1 -- 172.21.15.50:0/3944734558 shutdown_connections 2024-09-21T05:57:12.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.179+0000 7f983b200640 1 -- 172.21.15.50:0/3944734558 wait complete. 2024-09-21T05:57:12.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f983b200640 1 Processor -- start 2024-09-21T05:57:12.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f983b200640 1 -- start start 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f983b200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f98340b4de0 0x7f983413e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f983b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98340b5c70 0x7f9834159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f983b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98341527b0 0x7f983415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f983b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9834168a80 con 0x7f98341527b0 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f983b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9834168900 con 0x7f98340b5c70 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f983b200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9834168c00 con 0x7f98340b4de0 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f9839200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98341527b0 0x7f983415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f9839200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98341527b0 0x7f983415a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56592/0 (socket says 172.21.15.50:56592) 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f9839200640 1 -- 172.21.15.50:0/724544144 learned_addr learned my addr 172.21.15.50:0/724544144 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f9833e00640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f98340b4de0 0x7f983413e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f9833400640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98340b5c70 0x7f9834159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f9839200640 1 -- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f98340b4de0 msgr2=0x7f983413e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:12.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f9839200640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f98340b4de0 0x7f983413e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f9839200640 1 -- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98340b5c70 msgr2=0x7f9834159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:12.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f9839200640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98340b5c70 0x7f9834159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.183+0000 7f9839200640 1 -- 172.21.15.50:0/724544144 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98341fbd60 con 0x7f98341527b0 2024-09-21T05:57:12.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.187+0000 7f9839200640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98341527b0 0x7f983415a240 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f982c00caa0 tx=0x7f982c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:12.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.187+0000 7f982be00640 1 -- 172.21.15.50:0/724544144 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c016020 con 0x7f98341527b0 2024-09-21T05:57:12.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.187+0000 7f982be00640 1 -- 172.21.15.50:0/724544144 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f982c00a930 con 0x7f98341527b0 2024-09-21T05:57:12.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.187+0000 7f982be00640 1 -- 172.21.15.50:0/724544144 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c00adc0 con 0x7f98341527b0 2024-09-21T05:57:12.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.187+0000 7f983b200640 1 -- 172.21.15.50:0/724544144 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98341fc040 con 0x7f98341527b0 2024-09-21T05:57:12.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.187+0000 7f983b200640 1 -- 172.21.15.50:0/724544144 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98341fc600 con 0x7f98341527b0 2024-09-21T05:57:12.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.187+0000 7f983b200640 1 -- 172.21.15.50:0/724544144 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f983415bb60 con 0x7f98341527b0 2024-09-21T05:57:12.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.187+0000 7f982be00640 1 -- 172.21.15.50:0/724544144 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f982c019020 con 0x7f98341527b0 2024-09-21T05:57:12.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.187+0000 7f982be00640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9814073200 0x7f9814075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:12.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.187+0000 7f982be00640 1 -- 172.21.15.50:0/724544144 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f982c0941f0 con 0x7f98341527b0 2024-09-21T05:57:12.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.191+0000 7f9833e00640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9814073200 0x7f9814075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:12.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.191+0000 7f9833e00640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9814073200 0x7f9814075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9824004630 tx=0x7f982400b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:12.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.191+0000 7f982be00640 1 -- 172.21.15.50:0/724544144 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f982c020020 con 0x7f98341527b0 2024-09-21T05:57:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.435+0000 7f983b200640 1 -- 172.21.15.50:0/724544144 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f9834142090 con 0x7f98341527b0 2024-09-21T05:57:12.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f982be00640 1 -- 172.21.15.50:0/724544144 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v26) ==== 90+0+0 (secure 0 0 0) 0x7f982c05ee70 con 0x7f98341527b0 2024-09-21T05:57:12.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f983b200640 1 -- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9814073200 msgr2=0x7f9814075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f983b200640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9814073200 0x7f9814075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9824004630 tx=0x7f982400b2e0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f983b200640 1 -- 172.21.15.50:0/724544144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98341527b0 msgr2=0x7f983415a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f983b200640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98341527b0 0x7f983415a240 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f982c00caa0 tx=0x7f982c00cb90 comp rx=0 tx=0).stop 2024-09-21T05:57:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f983b200640 1 -- 172.21.15.50:0/724544144 shutdown_connections 2024-09-21T05:57:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f983b200640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9814073200 0x7f9814075710 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f983b200640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98341527b0 0x7f983415a240 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f983b200640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98340b5c70 0x7f9834159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f983b200640 1 --2- 172.21.15.50:0/724544144 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f98340b4de0 0x7f983413e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f983b200640 1 -- 172.21.15.50:0/724544144 >> 172.21.15.50:0/724544144 conn(0x7f98340b2040 msgr2=0x7f98341581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f983b200640 1 -- 172.21.15.50:0/724544144 shutdown_connections 2024-09-21T05:57:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.439+0000 7f983b200640 1 -- 172.21.15.50:0/724544144 wait complete. 2024-09-21T05:57:12.465 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-21T05:57:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.619+0000 7f401a800640 1 -- 172.21.15.50:0/53634819 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f401415aa50 msgr2=0x7f40140cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.619+0000 7f401a800640 1 --2- 172.21.15.50:0/53634819 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f401415aa50 0x7f40140cd260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f400400a980 tx=0x7f400401dab0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.619+0000 7f401a800640 1 -- 172.21.15.50:0/53634819 shutdown_connections 2024-09-21T05:57:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.619+0000 7f401a800640 1 --2- 172.21.15.50:0/53634819 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f401415aa50 0x7f40140cd260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.619+0000 7f401a800640 1 -- 172.21.15.50:0/53634819 >> 172.21.15.50:0/53634819 conn(0x7f40140b2040 msgr2=0x7f40140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.619+0000 7f401a800640 1 -- 172.21.15.50:0/53634819 shutdown_connections 2024-09-21T05:57:12.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.619+0000 7f401a800640 1 -- 172.21.15.50:0/53634819 wait complete. 2024-09-21T05:57:12.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f401a800640 1 Processor -- start 2024-09-21T05:57:12.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f401a800640 1 -- start start 2024-09-21T05:57:12.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f401a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40140cd7a0 0x7f401414f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:12.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f401a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40140cdbb0 0x7f40141397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:12.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f401a800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f401415aa50 0x7f40140affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f401a800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40141690a0 con 0x7f40140cd7a0 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f401a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4014169220 con 0x7f40140cdbb0 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f401a800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f40141693a0 con 0x7f401415aa50 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f4013400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40140cd7a0 0x7f401414f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f4013400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40140cd7a0 0x7f401414f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56604/0 (socket says 172.21.15.50:56604) 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f4013400640 1 -- 172.21.15.50:0/3053201292 learned_addr learned my addr 172.21.15.50:0/3053201292 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f4013e00640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f401415aa50 0x7f40140affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.623+0000 7f4012a00640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40140cdbb0 0x7f40141397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f4013400640 1 -- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f401415aa50 msgr2=0x7f40140affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f4013400640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f401415aa50 0x7f40140affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f4013400640 1 -- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40140cdbb0 msgr2=0x7f40141397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f4013400640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40140cdbb0 0x7f40141397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f4013400640 1 -- 172.21.15.50:0/3053201292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401414f9a0 con 0x7f40140cd7a0 2024-09-21T05:57:12.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f4012a00640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40140cdbb0 0x7f40141397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:12.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f4013400640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40140cd7a0 0x7f401414f110 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4004000c10 tx=0x7f40040040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:12.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f400b400640 1 -- 172.21.15.50:0/3053201292 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40040b3040 con 0x7f40140cd7a0 2024-09-21T05:57:12.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f400b400640 1 -- 172.21.15.50:0/3053201292 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4004005030 con 0x7f40140cd7a0 2024-09-21T05:57:12.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f401a800640 1 -- 172.21.15.50:0/3053201292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401414fc80 con 0x7f40140cd7a0 2024-09-21T05:57:12.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f400b400640 1 -- 172.21.15.50:0/3053201292 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004005480 con 0x7f40140cd7a0 2024-09-21T05:57:12.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f401a800640 1 -- 172.21.15.50:0/3053201292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4014150240 con 0x7f40140cd7a0 2024-09-21T05:57:12.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f401a800640 1 -- 172.21.15.50:0/3053201292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4014158280 con 0x7f40140cd7a0 2024-09-21T05:57:12.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f400b400640 1 -- 172.21.15.50:0/3053201292 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f40040bf060 con 0x7f40140cd7a0 2024-09-21T05:57:12.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.627+0000 7f400b400640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3ff4073200 0x7f3ff4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:12.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.631+0000 7f400b400640 1 -- 172.21.15.50:0/3053201292 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f400412db40 con 0x7f40140cd7a0 2024-09-21T05:57:12.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.631+0000 7f4012a00640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3ff4073200 0x7f3ff4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:12.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.631+0000 7f4012a00640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3ff4073200 0x7f3ff4075710 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f40140af5f0 tx=0x7f4000009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:12.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.631+0000 7f400b400640 1 -- 172.21.15.50:0/3053201292 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f40040176a0 con 0x7f40140cd7a0 2024-09-21T05:57:12.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.875+0000 7f401a800640 1 -- 172.21.15.50:0/3053201292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f40140b5f70 con 0x7f40140cd7a0 2024-09-21T05:57:12.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.879+0000 7f400b400640 1 -- 172.21.15.50:0/3053201292 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v27) ==== 97+0+0 (secure 0 0 0) 0x7f40040f8e20 con 0x7f40140cd7a0 2024-09-21T05:57:12.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.883+0000 7f401a800640 1 -- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3ff4073200 msgr2=0x7f3ff4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:12.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.883+0000 7f401a800640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3ff4073200 0x7f3ff4075710 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f40140af5f0 tx=0x7f4000009040 comp rx=0 tx=0).stop 2024-09-21T05:57:12.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.883+0000 7f401a800640 1 -- 172.21.15.50:0/3053201292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40140cd7a0 msgr2=0x7f401414f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:12.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.883+0000 7f401a800640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40140cd7a0 0x7f401414f110 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4004000c10 tx=0x7f40040040c0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.883+0000 7f401a800640 1 -- 172.21.15.50:0/3053201292 shutdown_connections 2024-09-21T05:57:12.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.883+0000 7f401a800640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f401415aa50 0x7f40140affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.883+0000 7f401a800640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40140cdbb0 0x7f40141397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.883+0000 7f401a800640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3ff4073200 0x7f3ff4075710 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.883+0000 7f401a800640 1 --2- 172.21.15.50:0/3053201292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40140cd7a0 0x7f401414f110 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:12.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.883+0000 7f401a800640 1 -- 172.21.15.50:0/3053201292 >> 172.21.15.50:0/3053201292 conn(0x7f40140b2040 msgr2=0x7f4014151e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:12.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.883+0000 7f401a800640 1 -- 172.21.15.50:0/3053201292 shutdown_connections 2024-09-21T05:57:12.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:12.883+0000 7f401a800640 1 -- 172.21.15.50:0/3053201292 wait complete. 2024-09-21T05:57:12.907 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-21T05:57:13.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.063+0000 7fcde8600640 1 -- 172.21.15.50:0/3967019834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde00b5c70 msgr2=0x7fcde015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.063+0000 7fcde8600640 1 --2- 172.21.15.50:0/3967019834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde00b5c70 0x7fcde015bb60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009840 tx=0x7fcdd401c760 comp rx=0 tx=0).stop 2024-09-21T05:57:13.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.063+0000 7fcde8600640 1 -- 172.21.15.50:0/3967019834 shutdown_connections 2024-09-21T05:57:13.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.063+0000 7fcde8600640 1 --2- 172.21.15.50:0/3967019834 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcde01527b0 0x7fcde0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.063+0000 7fcde8600640 1 --2- 172.21.15.50:0/3967019834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde00b5c70 0x7fcde015bb60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.063+0000 7fcde8600640 1 -- 172.21.15.50:0/3967019834 >> 172.21.15.50:0/3967019834 conn(0x7fcde00b2040 msgr2=0x7fcde00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:13.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.063+0000 7fcde8600640 1 -- 172.21.15.50:0/3967019834 shutdown_connections 2024-09-21T05:57:13.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.063+0000 7fcde8600640 1 -- 172.21.15.50:0/3967019834 wait complete. 2024-09-21T05:57:13.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde8600640 1 Processor -- start 2024-09-21T05:57:13.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde8600640 1 -- start start 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde8600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcde00b4de0 0x7fcde01fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde8600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde00b5c70 0x7fcde015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde8600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde01527b0 0x7fcde00aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde8600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0168bb0 con 0x7fcde00b5c70 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde8600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0168a30 con 0x7fcde01527b0 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde8600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fcde0168d30 con 0x7fcde00b4de0 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde00b5c70 0x7fcde015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde00b5c70 0x7fcde015a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56622/0 (socket says 172.21.15.50:56622) 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde5200640 1 -- 172.21.15.50:0/982096311 learned_addr learned my addr 172.21.15.50:0/982096311 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde5c00640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcde00b4de0 0x7fcde01fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde5200640 1 -- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcde00b4de0 msgr2=0x7fcde01fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde5200640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcde00b4de0 0x7fcde01fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde6600640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde01527b0 0x7fcde00aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde5200640 1 -- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde01527b0 msgr2=0x7fcde00aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde5200640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde01527b0 0x7fcde00aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde5200640 1 -- 172.21.15.50:0/982096311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde015c0a0 con 0x7fcde00b5c70 2024-09-21T05:57:13.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde5c00640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcde00b4de0 0x7fcde01fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:13.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde6600640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde01527b0 0x7fcde00aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:13.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde5200640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde00b5c70 0x7fcde015a3a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e790 tx=0x7fcdcc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:13.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcddd600640 1 -- 172.21.15.50:0/982096311 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00eea0 con 0x7fcde00b5c70 2024-09-21T05:57:13.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcddd600640 1 -- 172.21.15.50:0/982096311 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdcc00cc60 con 0x7fcde00b5c70 2024-09-21T05:57:13.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcddd600640 1 -- 172.21.15.50:0/982096311 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc005280 con 0x7fcde00b5c70 2024-09-21T05:57:13.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde8600640 1 -- 172.21.15.50:0/982096311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde01fc150 con 0x7fcde00b5c70 2024-09-21T05:57:13.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.067+0000 7fcde8600640 1 -- 172.21.15.50:0/982096311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde015af40 con 0x7fcde00b5c70 2024-09-21T05:57:13.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.071+0000 7fcde8600640 1 -- 172.21.15.50:0/982096311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde015bb60 con 0x7fcde00b5c70 2024-09-21T05:57:13.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.071+0000 7fcddd600640 1 -- 172.21.15.50:0/982096311 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcdcc005420 con 0x7fcde00b5c70 2024-09-21T05:57:13.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.071+0000 7fcddd600640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcdc0073200 0x7fcdc0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.071+0000 7fcddd600640 1 -- 172.21.15.50:0/982096311 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fcdcc0949a0 con 0x7fcde00b5c70 2024-09-21T05:57:13.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.071+0000 7fcde5c00640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcdc0073200 0x7fcdc0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.071+0000 7fcde5c00640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcdc0073200 0x7fcdc0075710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40af000 tx=0x7fcdd401afa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:13.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.075+0000 7fcddd600640 1 -- 172.21.15.50:0/982096311 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcdcc05fc70 con 0x7fcde00b5c70 2024-09-21T05:57:13.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.315+0000 7fcde8600640 1 -- 172.21.15.50:0/982096311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fcde00b2630 con 0x7fcde00b5c70 2024-09-21T05:57:13.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcddd600640 1 -- 172.21.15.50:0/982096311 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v28) ==== 90+0+0 (secure 0 0 0) 0x7fcdcc05f570 con 0x7fcde00b5c70 2024-09-21T05:57:13.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcde8600640 1 -- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcdc0073200 msgr2=0x7fcdc0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcde8600640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcdc0073200 0x7fcdc0075710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40af000 tx=0x7fcdd401afa0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcde8600640 1 -- 172.21.15.50:0/982096311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde00b5c70 msgr2=0x7fcde015a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcde8600640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde00b5c70 0x7fcde015a3a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e790 tx=0x7fcdcc00ec50 comp rx=0 tx=0).stop 2024-09-21T05:57:13.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcde8600640 1 -- 172.21.15.50:0/982096311 shutdown_connections 2024-09-21T05:57:13.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcde8600640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde01527b0 0x7fcde00aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcde8600640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcdc0073200 0x7fcdc0075710 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcde8600640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde00b5c70 0x7fcde015a3a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcde8600640 1 --2- 172.21.15.50:0/982096311 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcde00b4de0 0x7fcde01fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcde8600640 1 -- 172.21.15.50:0/982096311 >> 172.21.15.50:0/982096311 conn(0x7fcde00b2040 msgr2=0x7fcde0151cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:13.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcde8600640 1 -- 172.21.15.50:0/982096311 shutdown_connections 2024-09-21T05:57:13.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.319+0000 7fcde8600640 1 -- 172.21.15.50:0/982096311 wait complete. 2024-09-21T05:57:13.345 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T05:57:13.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.499+0000 7f7254000640 1 -- 172.21.15.50:0/3085001496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f724c160d50 msgr2=0x7f724c074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.499+0000 7f7254000640 1 --2- 172.21.15.50:0/3085001496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f724c160d50 0x7f724c074410 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f724c0713e0 tx=0x7f723801c8c0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.499+0000 7f7254000640 1 -- 172.21.15.50:0/3085001496 shutdown_connections 2024-09-21T05:57:13.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.499+0000 7f7254000640 1 --2- 172.21.15.50:0/3085001496 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f724c161160 0x7f724c074950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.499+0000 7f7254000640 1 --2- 172.21.15.50:0/3085001496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f724c160d50 0x7f724c074410 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.499+0000 7f7254000640 1 --2- 172.21.15.50:0/3085001496 >> v2:172.21.15.88:3300/0 conn(0x7f724c157030 0x7f724c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.499+0000 7f7254000640 1 -- 172.21.15.50:0/3085001496 >> 172.21.15.50:0/3085001496 conn(0x7f724c0708c0 msgr2=0x7f724c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:13.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.499+0000 7f7254000640 1 -- 172.21.15.50:0/3085001496 shutdown_connections 2024-09-21T05:57:13.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.499+0000 7f7254000640 1 -- 172.21.15.50:0/3085001496 wait complete. 2024-09-21T05:57:13.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7254000640 1 Processor -- start 2024-09-21T05:57:13.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7254000640 1 -- start start 2024-09-21T05:57:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7254000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f724c157030 0x7f724c1fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7254000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f724c160d50 0x7f724c13dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7254000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f724c161160 0x7f724c150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7254000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f724c072230 con 0x7f724c160d50 2024-09-21T05:57:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7254000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f724c0720b0 con 0x7f724c161160 2024-09-21T05:57:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7254000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f724c0723b0 con 0x7f724c157030 2024-09-21T05:57:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7250c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f724c160d50 0x7f724c13dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7250c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f724c160d50 0x7f724c13dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56644/0 (socket says 172.21.15.50:56644) 2024-09-21T05:57:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7250c00640 1 -- 172.21.15.50:0/622717962 learned_addr learned my addr 172.21.15.50:0/622717962 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7251600640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f724c157030 0x7f724c1fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7252000640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f724c161160 0x7f724c150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7250c00640 1 -- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f724c157030 msgr2=0x7f724c1fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7250c00640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f724c157030 0x7f724c1fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7250c00640 1 -- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f724c161160 msgr2=0x7f724c150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7250c00640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f724c161160 0x7f724c150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7250c00640 1 -- 172.21.15.50:0/622717962 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c06d8a0 con 0x7f724c160d50 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.503+0000 7f7251600640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f724c157030 0x7f724c1fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.507+0000 7f7250c00640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f724c160d50 0x7f724c13dc00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f723801af70 tx=0x7f72380a55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.507+0000 7f7249600640 1 -- 172.21.15.50:0/622717962 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72380b2040 con 0x7f724c160d50 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.507+0000 7f7249600640 1 -- 172.21.15.50:0/622717962 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72380043b0 con 0x7f724c160d50 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.507+0000 7f7254000640 1 -- 172.21.15.50:0/622717962 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724c06db80 con 0x7f724c160d50 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.507+0000 7f7249600640 1 -- 172.21.15.50:0/622717962 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238003860 con 0x7f724c160d50 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.507+0000 7f7254000640 1 -- 172.21.15.50:0/622717962 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724c06e140 con 0x7f724c160d50 2024-09-21T05:57:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.507+0000 7f7254000640 1 -- 172.21.15.50:0/622717962 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724c072860 con 0x7f724c160d50 2024-09-21T05:57:13.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.507+0000 7f7249600640 1 -- 172.21.15.50:0/622717962 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f72380bf060 con 0x7f724c160d50 2024-09-21T05:57:13.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.507+0000 7f7249600640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f722c073200 0x7f722c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.507+0000 7f7249600640 1 -- 172.21.15.50:0/622717962 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f723812da30 con 0x7f724c160d50 2024-09-21T05:57:13.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.511+0000 7f7251600640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f722c073200 0x7f722c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.511+0000 7f7251600640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f722c073200 0x7f722c075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7240005fd0 tx=0x7f7240005e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:13.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.515+0000 7f7249600640 1 -- 172.21.15.50:0/622717962 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f72380164b0 con 0x7f724c160d50 2024-09-21T05:57:13.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 -- 172.21.15.50:0/622717962 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f724c0d2260 con 0x7f724c160d50 2024-09-21T05:57:13.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7249600640 1 -- 172.21.15.50:0/622717962 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f72380f8da0 con 0x7f724c160d50 2024-09-21T05:57:13.699 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:13.699 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T05:57:13.699 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:13.699 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-09-21T05:57:13.699 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-09-21T05:57:13.699 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:13.699 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T05:57:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 -- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f722c073200 msgr2=0x7f722c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f722c073200 0x7f722c075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7240005fd0 tx=0x7f7240005e40 comp rx=0 tx=0).stop 2024-09-21T05:57:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 -- 172.21.15.50:0/622717962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f724c160d50 msgr2=0x7f724c13dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f724c160d50 0x7f724c13dc00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f723801af70 tx=0x7f72380a55c0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 -- 172.21.15.50:0/622717962 shutdown_connections 2024-09-21T05:57:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f724c161160 0x7f724c150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f722c073200 0x7f722c075710 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f724c160d50 0x7f724c13dc00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 --2- 172.21.15.50:0/622717962 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f724c157030 0x7f724c1fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 -- 172.21.15.50:0/622717962 >> 172.21.15.50:0/622717962 conn(0x7f724c0708c0 msgr2=0x7f724c155720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:13.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 -- 172.21.15.50:0/622717962 shutdown_connections 2024-09-21T05:57:13.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.695+0000 7f7254000640 1 -- 172.21.15.50:0/622717962 wait complete. 2024-09-21T05:57:13.722 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T05:57:13.722 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T05:57:13.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.879+0000 7fd058e00640 1 -- 172.21.15.50:0/219677934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0540b5c70 msgr2=0x7fd05415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.879+0000 7fd058e00640 1 --2- 172.21.15.50:0/219677934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0540b5c70 0x7fd05415bb60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd054074b20 tx=0x7fd04401c760 comp rx=0 tx=0).stop 2024-09-21T05:57:13.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.879+0000 7fd058e00640 1 -- 172.21.15.50:0/219677934 shutdown_connections 2024-09-21T05:57:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.879+0000 7fd058e00640 1 --2- 172.21.15.50:0/219677934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0540b5c70 0x7fd05415bb60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.879+0000 7fd058e00640 1 -- 172.21.15.50:0/219677934 >> 172.21.15.50:0/219677934 conn(0x7fd0540b2040 msgr2=0x7fd0540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.879+0000 7fd058e00640 1 -- 172.21.15.50:0/219677934 shutdown_connections 2024-09-21T05:57:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.879+0000 7fd058e00640 1 -- 172.21.15.50:0/219677934 wait complete. 2024-09-21T05:57:13.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.883+0000 7fd058e00640 1 Processor -- start 2024-09-21T05:57:13.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.883+0000 7fd058e00640 1 -- start start 2024-09-21T05:57:13.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.883+0000 7fd058e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0540b4de0 0x7fd054161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.883+0000 7fd058e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0540b5c70 0x7fd0541f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.883+0000 7fd058e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0541527b0 0x7fd0541f9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.883+0000 7fd058e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd054169b30 con 0x7fd0540b4de0 2024-09-21T05:57:13.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.883+0000 7fd058e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd0541699b0 con 0x7fd0540b5c70 2024-09-21T05:57:13.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.883+0000 7fd058e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd054169cb0 con 0x7fd0541527b0 2024-09-21T05:57:13.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd051e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0540b4de0 0x7fd054161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd051e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0540b4de0 0x7fd054161ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56664/0 (socket says 172.21.15.50:56664) 2024-09-21T05:57:13.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd051e00640 1 -- 172.21.15.50:0/1118269331 learned_addr learned my addr 172.21.15.50:0/1118269331 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:13.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd052800640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0541527b0 0x7fd0541f9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd04be00640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0540b5c70 0x7fd0541f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd051e00640 1 -- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0541527b0 msgr2=0x7fd0541f9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd051e00640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0541527b0 0x7fd0541f9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd051e00640 1 -- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0540b5c70 msgr2=0x7fd0541f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd051e00640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0540b5c70 0x7fd0541f9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd051e00640 1 -- 172.21.15.50:0/1118269331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05415d990 con 0x7fd0540b4de0 2024-09-21T05:57:13.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd051e00640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0540b4de0 0x7fd054161ae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd044098340 tx=0x7fd044005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:13.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd04a000640 1 -- 172.21.15.50:0/1118269331 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0440bb070 con 0x7fd0540b4de0 2024-09-21T05:57:13.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd04a000640 1 -- 172.21.15.50:0/1118269331 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd044004330 con 0x7fd0540b4de0 2024-09-21T05:57:13.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd04a000640 1 -- 172.21.15.50:0/1118269331 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0440b2040 con 0x7fd0540b4de0 2024-09-21T05:57:13.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd058e00640 1 -- 172.21.15.50:0/1118269331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05415dc70 con 0x7fd0540b4de0 2024-09-21T05:57:13.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd058e00640 1 -- 172.21.15.50:0/1118269331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05415e230 con 0x7fd0540b4de0 2024-09-21T05:57:13.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd058e00640 1 -- 172.21.15.50:0/1118269331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05415bb60 con 0x7fd0540b4de0 2024-09-21T05:57:13.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.887+0000 7fd04a000640 1 -- 172.21.15.50:0/1118269331 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd044005270 con 0x7fd0540b4de0 2024-09-21T05:57:13.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.891+0000 7fd04a000640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd030073230 0x7fd030075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.891+0000 7fd04a000640 1 -- 172.21.15.50:0/1118269331 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fd04412d7e0 con 0x7fd0540b4de0 2024-09-21T05:57:13.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.891+0000 7fd04be00640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd030073230 0x7fd030075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.891+0000 7fd04be00640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd030073230 0x7fd030075740 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0046c0 tx=0x7fd03c007cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:13.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:13.895+0000 7fd04a000640 1 -- 172.21.15.50:0/1118269331 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd044016410 con 0x7fd0540b4de0 2024-09-21T05:57:14.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.071+0000 7fd058e00640 1 -- 172.21.15.50:0/1118269331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd054159420 con 0x7fd0540b4de0 2024-09-21T05:57:14.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.883+0000 7fd04a000640 1 -- 172.21.15.50:0/1118269331 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7fd044005560 con 0x7fd0540b4de0 2024-09-21T05:57:14.888 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-09-21T05:57:14.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.887+0000 7fd058e00640 1 -- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd030073230 msgr2=0x7fd030075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.887+0000 7fd058e00640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd030073230 0x7fd030075740 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0046c0 tx=0x7fd03c007cd0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.887+0000 7fd058e00640 1 -- 172.21.15.50:0/1118269331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0540b4de0 msgr2=0x7fd054161ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.887+0000 7fd058e00640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0540b4de0 0x7fd054161ae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd044098340 tx=0x7fd044005d50 comp rx=0 tx=0).stop 2024-09-21T05:57:14.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.887+0000 7fd058e00640 1 -- 172.21.15.50:0/1118269331 shutdown_connections 2024-09-21T05:57:14.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.887+0000 7fd058e00640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0541527b0 0x7fd0541f9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.887+0000 7fd058e00640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0540b5c70 0x7fd0541f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.887+0000 7fd058e00640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd030073230 0x7fd030075740 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.887+0000 7fd058e00640 1 --2- 172.21.15.50:0/1118269331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0540b4de0 0x7fd054161ae0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.887+0000 7fd058e00640 1 -- 172.21.15.50:0/1118269331 >> 172.21.15.50:0/1118269331 conn(0x7fd0540b2040 msgr2=0x7fd054152c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:14.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.887+0000 7fd058e00640 1 -- 172.21.15.50:0/1118269331 shutdown_connections 2024-09-21T05:57:14.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:14.887+0000 7fd058e00640 1 -- 172.21.15.50:0/1118269331 wait complete. 2024-09-21T05:57:14.920 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T05:57:15.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.087+0000 7f77ee800640 1 -- 172.21.15.50:0/910748074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77e80b5c70 msgr2=0x7f77e815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.087+0000 7f77ee800640 1 --2- 172.21.15.50:0/910748074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77e80b5c70 0x7f77e815bb60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f77e8074b20 tx=0x7f77d801c760 comp rx=0 tx=0).stop 2024-09-21T05:57:15.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.087+0000 7f77ee800640 1 -- 172.21.15.50:0/910748074 shutdown_connections 2024-09-21T05:57:15.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.087+0000 7f77ee800640 1 --2- 172.21.15.50:0/910748074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77e80b5c70 0x7f77e815bb60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.087+0000 7f77ee800640 1 -- 172.21.15.50:0/910748074 >> 172.21.15.50:0/910748074 conn(0x7f77e80b2040 msgr2=0x7f77e80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:15.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.087+0000 7f77ee800640 1 -- 172.21.15.50:0/910748074 shutdown_connections 2024-09-21T05:57:15.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.087+0000 7f77ee800640 1 -- 172.21.15.50:0/910748074 wait complete. 2024-09-21T05:57:15.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.087+0000 7f77ee800640 1 Processor -- start 2024-09-21T05:57:15.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.087+0000 7f77ee800640 1 -- start start 2024-09-21T05:57:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77ee800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77e80b4de0 0x7f77e80abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77ee800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f77e80b5c70 0x7f77e813da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77ee800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81527b0 0x7f77e80b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77ee800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8169b30 con 0x7f77e80b4de0 2024-09-21T05:57:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77ee800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81699b0 con 0x7f77e81527b0 2024-09-21T05:57:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77ee800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f77e8169cb0 con 0x7f77e80b5c70 2024-09-21T05:57:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77e7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77e80b4de0 0x7f77e80abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77e7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77e80b4de0 0x7f77e80abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56684/0 (socket says 172.21.15.50:56684) 2024-09-21T05:57:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77e7400640 1 -- 172.21.15.50:0/2967962680 learned_addr learned my addr 172.21.15.50:0/2967962680 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:15.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77e7e00640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81527b0 0x7f77e80b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:15.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77dea00640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f77e80b5c70 0x7f77e813da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:15.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77e7400640 1 -- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f77e80b5c70 msgr2=0x7f77e813da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77e7400640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f77e80b5c70 0x7f77e813da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77e7400640 1 -- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81527b0 msgr2=0x7f77e80b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77e7400640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81527b0 0x7f77e80b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77e7400640 1 -- 172.21.15.50:0/2967962680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e80ac2e0 con 0x7f77e80b4de0 2024-09-21T05:57:15.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77e7400640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77e80b4de0 0x7f77e80abda0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f77d801cc40 tx=0x7f77d80a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:15.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77dfe00640 1 -- 172.21.15.50:0/2967962680 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d80b2040 con 0x7f77e80b4de0 2024-09-21T05:57:15.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77dfe00640 1 -- 172.21.15.50:0/2967962680 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77d80042f0 con 0x7f77e80b4de0 2024-09-21T05:57:15.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77dfe00640 1 -- 172.21.15.50:0/2967962680 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8003840 con 0x7f77e80b4de0 2024-09-21T05:57:15.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77ee800640 1 -- 172.21.15.50:0/2967962680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e814f570 con 0x7f77e80b4de0 2024-09-21T05:57:15.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77ee800640 1 -- 172.21.15.50:0/2967962680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e814fb30 con 0x7f77e80b4de0 2024-09-21T05:57:15.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.091+0000 7f77ee800640 1 -- 172.21.15.50:0/2967962680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e815bb60 con 0x7f77e80b4de0 2024-09-21T05:57:15.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.095+0000 7f77dfe00640 1 -- 172.21.15.50:0/2967962680 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f77d80bd030 con 0x7f77e80b4de0 2024-09-21T05:57:15.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.095+0000 7f77dfe00640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77c4073230 0x7f77c4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:15.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.095+0000 7f77dfe00640 1 -- 172.21.15.50:0/2967962680 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f77d812de20 con 0x7f77e80b4de0 2024-09-21T05:57:15.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.095+0000 7f77dea00640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77c4073230 0x7f77c4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.095+0000 7f77dea00640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77c4073230 0x7f77c4075740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77d0000f70 tx=0x7f77d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:15.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.099+0000 7f77dfe00640 1 -- 172.21.15.50:0/2967962680 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f77d80c4060 con 0x7f77e80b4de0 2024-09-21T05:57:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.279+0000 7f77ee800640 1 -- 172.21.15.50:0/2967962680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f77e80b25f0 con 0x7f77e80b4de0 2024-09-21T05:57:15.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.887+0000 7f77dfe00640 1 -- 172.21.15.50:0/2967962680 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f77e80ac2e0 con 0x7f77e80b4de0 2024-09-21T05:57:15.892 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-09-21T05:57:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.891+0000 7f77ee800640 1 -- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77c4073230 msgr2=0x7f77c4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.891+0000 7f77ee800640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77c4073230 0x7f77c4075740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77d0000f70 tx=0x7f77d0009040 comp rx=0 tx=0).stop 2024-09-21T05:57:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.891+0000 7f77ee800640 1 -- 172.21.15.50:0/2967962680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77e80b4de0 msgr2=0x7f77e80abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.891+0000 7f77ee800640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77e80b4de0 0x7f77e80abda0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f77d801cc40 tx=0x7f77d80a5780 comp rx=0 tx=0).stop 2024-09-21T05:57:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.891+0000 7f77ee800640 1 -- 172.21.15.50:0/2967962680 shutdown_connections 2024-09-21T05:57:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.891+0000 7f77ee800640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81527b0 0x7f77e80b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.891+0000 7f77ee800640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f77e80b5c70 0x7f77e813da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.891+0000 7f77ee800640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77c4073230 0x7f77c4075740 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.891+0000 7f77ee800640 1 --2- 172.21.15.50:0/2967962680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77e80b4de0 0x7f77e80abda0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.891+0000 7f77ee800640 1 -- 172.21.15.50:0/2967962680 >> 172.21.15.50:0/2967962680 conn(0x7f77e80b2040 msgr2=0x7f77e80cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:15.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.891+0000 7f77ee800640 1 -- 172.21.15.50:0/2967962680 shutdown_connections 2024-09-21T05:57:15.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:15.891+0000 7f77ee800640 1 -- 172.21.15.50:0/2967962680 wait complete. 2024-09-21T05:57:15.936 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T05:57:16.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.095+0000 7fbb52600640 1 -- 172.21.15.50:0/2865883641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb30004d50 msgr2=0x7fbb30005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.095+0000 7fbb52600640 1 --2- 172.21.15.50:0/2865883641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb30004d50 0x7fbb30005160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0ced80 tx=0x7fbb4401fa40 comp rx=0 tx=0).stop 2024-09-21T05:57:16.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.095+0000 7fbb51c00640 1 -- 172.21.15.50:0/2865883641 reap_dead start 2024-09-21T05:57:16.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.095+0000 7fbb52600640 1 -- 172.21.15.50:0/2865883641 shutdown_connections 2024-09-21T05:57:16.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.095+0000 7fbb52600640 1 -- 172.21.15.50:0/2865883641 >> 172.21.15.50:0/2865883641 conn(0x7fbb4c0b4560 msgr2=0x7fbb4c0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:16.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.095+0000 7fbb52600640 1 -- 172.21.15.50:0/2865883641 shutdown_connections 2024-09-21T05:57:16.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.095+0000 7fbb52600640 1 -- 172.21.15.50:0/2865883641 wait complete. 2024-09-21T05:57:16.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb52600640 1 Processor -- start 2024-09-21T05:57:16.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb52600640 1 -- start start 2024-09-21T05:57:16.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb52600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbb30004d50 0x7fbb4c06d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb52600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb4c1e6e30 0x7fbb4c143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb52600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb4c15b4b0 0x7fbb4c15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb52600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c074b10 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb52600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c074990 con 0x7fbb4c1e6e30 2024-09-21T05:57:16.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb52600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fbb4c0b5970 con 0x7fbb30004d50 2024-09-21T05:57:16.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb51c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb4c15b4b0 0x7fbb4c15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb51c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb4c15b4b0 0x7fbb4c15b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56704/0 (socket says 172.21.15.50:56704) 2024-09-21T05:57:16.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb51c00640 1 -- 172.21.15.50:0/4005240170 learned_addr learned my addr 172.21.15.50:0/4005240170 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:16.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb51200640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbb30004d50 0x7fbb4c06d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb43e00640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb4c1e6e30 0x7fbb4c143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.099+0000 7fbb51c00640 1 -- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbb30004d50 msgr2=0x7fbb4c06d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb51c00640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbb30004d50 0x7fbb4c06d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb51c00640 1 -- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb4c1e6e30 msgr2=0x7fbb4c143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb51c00640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb4c1e6e30 0x7fbb4c143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb51c00640 1 -- 172.21.15.50:0/4005240170 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4c06e9b0 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb43e00640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb4c1e6e30 0x7fbb4c143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:16.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb51c00640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb4c15b4b0 0x7fbb4c15b8c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbb44021120 tx=0x7fbb440a9410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:16.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb49600640 1 -- 172.21.15.50:0/4005240170 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c009730 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb49600640 1 -- 172.21.15.50:0/4005240170 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb44003980 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb52600640 1 -- 172.21.15.50:0/4005240170 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4c15be00 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb49600640 1 -- 172.21.15.50:0/4005240170 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400c3c0 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb52600640 1 -- 172.21.15.50:0/4005240170 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb4c1e92f0 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb52600640 1 -- 172.21.15.50:0/4005240170 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb4c074240 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.103+0000 7fbb49600640 1 -- 172.21.15.50:0/4005240170 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbb440a9610 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.107+0000 7fbb49600640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbb28073230 0x7fbb28075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.107+0000 7fbb49600640 1 -- 172.21.15.50:0/4005240170 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7fbb440b8030 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.107+0000 7fbb51200640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbb28073230 0x7fbb28075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.107+0000 7fbb51200640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbb28073230 0x7fbb28075740 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009d40 tx=0x7fbb3c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:16.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.107+0000 7fbb49600640 1 -- 172.21.15.50:0/4005240170 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbb4c15be00 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.319+0000 7fbb52600640 1 -- 172.21.15.50:0/4005240170 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbb4c0766a0 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.729 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:57:16.733 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:57:16.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.899+0000 7fbb49600640 1 -- 172.21.15.50:0/4005240170 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fbb4c06e9b0 con 0x7fbb4c15b4b0 2024-09-21T05:57:16.905 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 4 and data pool 5 2024-09-21T05:57:16.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.903+0000 7fbb52600640 1 -- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbb28073230 msgr2=0x7fbb28075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.903+0000 7fbb52600640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbb28073230 0x7fbb28075740 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009d40 tx=0x7fbb3c0091e0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.903+0000 7fbb52600640 1 -- 172.21.15.50:0/4005240170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb4c15b4b0 msgr2=0x7fbb4c15b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.903+0000 7fbb52600640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb4c15b4b0 0x7fbb4c15b8c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbb44021120 tx=0x7fbb440a9410 comp rx=0 tx=0).stop 2024-09-21T05:57:16.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.907+0000 7fbb52600640 1 -- 172.21.15.50:0/4005240170 shutdown_connections 2024-09-21T05:57:16.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.907+0000 7fbb52600640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbb28073230 0x7fbb28075740 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.907+0000 7fbb52600640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb4c15b4b0 0x7fbb4c15b8c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.907+0000 7fbb52600640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb4c1e6e30 0x7fbb4c143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.907+0000 7fbb52600640 1 --2- 172.21.15.50:0/4005240170 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbb30004d50 0x7fbb4c06d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.907+0000 7fbb52600640 1 -- 172.21.15.50:0/4005240170 >> 172.21.15.50:0/4005240170 conn(0x7fbb4c0b4560 msgr2=0x7fbb4c0d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:16.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.907+0000 7fbb52600640 1 -- 172.21.15.50:0/4005240170 shutdown_connections 2024-09-21T05:57:16.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:16.907+0000 7fbb52600640 1 -- 172.21.15.50:0/4005240170 wait complete. 2024-09-21T05:57:16.961 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:57:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.119+0000 7fac99200640 1 -- 172.21.15.50:0/171993800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac940b4de0 msgr2=0x7fac940b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.119+0000 7fac99200640 1 --2- 172.21.15.50:0/171993800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac940b4de0 0x7fac940b51f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fac94074b20 tx=0x7fac8c01cb50 comp rx=0 tx=0).stop 2024-09-21T05:57:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.119+0000 7fac99200640 1 -- 172.21.15.50:0/171993800 shutdown_connections 2024-09-21T05:57:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.119+0000 7fac99200640 1 --2- 172.21.15.50:0/171993800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac940b4de0 0x7fac940b51f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.119+0000 7fac99200640 1 --2- 172.21.15.50:0/171993800 >> v2:172.21.15.88:3300/0 conn(0x7fac940b5c70 0x7fac9415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.119+0000 7fac99200640 1 -- 172.21.15.50:0/171993800 >> 172.21.15.50:0/171993800 conn(0x7fac940b2040 msgr2=0x7fac940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.119+0000 7fac99200640 1 -- 172.21.15.50:0/171993800 shutdown_connections 2024-09-21T05:57:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.119+0000 7fac99200640 1 -- 172.21.15.50:0/171993800 wait complete. 2024-09-21T05:57:17.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.119+0000 7fac99200640 1 Processor -- start 2024-09-21T05:57:17.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.119+0000 7fac99200640 1 -- start start 2024-09-21T05:57:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac99200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac940b4de0 0x7fac94153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:17.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac99200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac940b5c70 0x7fac9414d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:17.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac99200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac941527b0 0x7fac9414e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:17.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac99200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fac941689d0 con 0x7fac940b4de0 2024-09-21T05:57:17.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac99200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fac94168850 con 0x7fac941527b0 2024-09-21T05:57:17.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac99200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fac94168b50 con 0x7fac940b5c70 2024-09-21T05:57:17.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac91e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac940b4de0 0x7fac94153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:17.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac91e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac940b4de0 0x7fac94153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58808/0 (socket says 172.21.15.50:58808) 2024-09-21T05:57:17.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac91e00640 1 -- 172.21.15.50:0/4051274068 learned_addr learned my addr 172.21.15.50:0/4051274068 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac8be00640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac940b5c70 0x7fac9414d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac92800640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac941527b0 0x7fac9414e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac91e00640 1 -- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac940b5c70 msgr2=0x7fac9414d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac91e00640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac940b5c70 0x7fac9414d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac91e00640 1 -- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac941527b0 msgr2=0x7fac9414e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac91e00640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac941527b0 0x7fac9414e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac91e00640 1 -- 172.21.15.50:0/4051274068 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac941fc3a0 con 0x7fac940b4de0 2024-09-21T05:57:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac8be00640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac940b5c70 0x7fac9414d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac92800640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac941527b0 0x7fac9414e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac91e00640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac940b4de0 0x7fac94153130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fac8400b440 tx=0x7fac8400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac8a000640 1 -- 172.21.15.50:0/4051274068 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84015020 con 0x7fac940b4de0 2024-09-21T05:57:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac8a000640 1 -- 172.21.15.50:0/4051274068 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac840119a0 con 0x7fac940b4de0 2024-09-21T05:57:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac8a000640 1 -- 172.21.15.50:0/4051274068 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84011df0 con 0x7fac940b4de0 2024-09-21T05:57:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac99200640 1 -- 172.21.15.50:0/4051274068 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac941fc680 con 0x7fac940b4de0 2024-09-21T05:57:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac99200640 1 -- 172.21.15.50:0/4051274068 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac941f8900 con 0x7fac940b4de0 2024-09-21T05:57:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac99200640 1 -- 172.21.15.50:0/4051274068 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac940b51f0 con 0x7fac940b4de0 2024-09-21T05:57:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.123+0000 7fac8a000640 1 -- 172.21.15.50:0/4051274068 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fac84018020 con 0x7fac940b4de0 2024-09-21T05:57:17.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.127+0000 7fac8a000640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac70073200 0x7fac70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:17.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.127+0000 7fac8a000640 1 -- 172.21.15.50:0/4051274068 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fac84094440 con 0x7fac940b4de0 2024-09-21T05:57:17.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.127+0000 7fac8be00640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac70073200 0x7fac70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:17.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.127+0000 7fac8be00640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac70073200 0x7fac70075710 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00a830 tx=0x7fac7c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:17.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.131+0000 7fac8a000640 1 -- 172.21.15.50:0/4051274068 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fac8401f020 con 0x7fac940b4de0 2024-09-21T05:57:17.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.307+0000 7fac99200640 1 -- 172.21.15.50:0/4051274068 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac940760c0 con 0x7fac940b4de0 2024-09-21T05:57:17.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.307+0000 7fac8a000640 1 -- 172.21.15.50:0/4051274068 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16694 (secure 0 0 0) 0x7fac8400d040 con 0x7fac940b4de0 2024-09-21T05:57:17.313 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:17.313 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":32,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:57:16.325278+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:57:14.083279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T05:57:15.292209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:57:17.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.311+0000 7fac99200640 1 -- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac70073200 msgr2=0x7fac70075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.311+0000 7fac99200640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac70073200 0x7fac70075710 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00a830 tx=0x7fac7c008040 comp rx=0 tx=0).stop 2024-09-21T05:57:17.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.311+0000 7fac99200640 1 -- 172.21.15.50:0/4051274068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac940b4de0 msgr2=0x7fac94153130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.311+0000 7fac99200640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac940b4de0 0x7fac94153130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fac8400b440 tx=0x7fac8400b900 comp rx=0 tx=0).stop 2024-09-21T05:57:17.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.311+0000 7fac99200640 1 -- 172.21.15.50:0/4051274068 shutdown_connections 2024-09-21T05:57:17.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.311+0000 7fac99200640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac941527b0 0x7fac9414e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.311+0000 7fac99200640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac940b5c70 0x7fac9414d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.311+0000 7fac99200640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac70073200 0x7fac70075710 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.311+0000 7fac99200640 1 --2- 172.21.15.50:0/4051274068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac940b4de0 0x7fac94153130 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.311+0000 7fac99200640 1 -- 172.21.15.50:0/4051274068 >> 172.21.15.50:0/4051274068 conn(0x7fac940b2040 msgr2=0x7fac940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:17.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.311+0000 7fac99200640 1 -- 172.21.15.50:0/4051274068 shutdown_connections 2024-09-21T05:57:17.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.311+0000 7fac99200640 1 -- 172.21.15.50:0/4051274068 wait complete. 2024-09-21T05:57:17.335 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:57:17.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.486+0000 7f25a3e00640 1 -- 172.21.15.50:0/4010211151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259c0b4de0 msgr2=0x7f259c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.486+0000 7f25a3e00640 1 --2- 172.21.15.50:0/4010211151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259c0b4de0 0x7f259c0b51f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2594009c80 tx=0x7f259401cb00 comp rx=0 tx=0).stop 2024-09-21T05:57:17.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.490+0000 7f25a3e00640 1 -- 172.21.15.50:0/4010211151 shutdown_connections 2024-09-21T05:57:17.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.490+0000 7f25a3e00640 1 --2- 172.21.15.50:0/4010211151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259c0b4de0 0x7f259c0b51f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.490+0000 7f25a3e00640 1 --2- 172.21.15.50:0/4010211151 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f259c1527b0 0x7f259c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.490+0000 7f25a3e00640 1 -- 172.21.15.50:0/4010211151 >> 172.21.15.50:0/4010211151 conn(0x7f259c0b2040 msgr2=0x7f259c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:17.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.490+0000 7f25a3e00640 1 -- 172.21.15.50:0/4010211151 shutdown_connections 2024-09-21T05:57:17.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.490+0000 7f25a3e00640 1 -- 172.21.15.50:0/4010211151 wait complete. 2024-09-21T05:57:17.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.490+0000 7f25a3e00640 1 Processor -- start 2024-09-21T05:57:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.490+0000 7f25a3e00640 1 -- start start 2024-09-21T05:57:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.490+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c0b4de0 0x7f259c151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.490+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259c0b5c70 0x7f259c0af450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.490+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f259c1527b0 0x7f259c15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f259c168ac0 con 0x7f259c0b5c70 2024-09-21T05:57:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f259c168940 con 0x7f259c0b4de0 2024-09-21T05:57:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a3e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f259c168c40 con 0x7f259c1527b0 2024-09-21T05:57:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259c0b5c70 0x7f259c0af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259c0b5c70 0x7f259c0af450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58814/0 (socket says 172.21.15.50:58814) 2024-09-21T05:57:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a0a00640 1 -- 172.21.15.50:0/3242295523 learned_addr learned my addr 172.21.15.50:0/3242295523 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:17.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a1400640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c0b4de0 0x7f259c151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:17.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a1e00640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f259c1527b0 0x7f259c15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:17.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a0a00640 1 -- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f259c1527b0 msgr2=0x7f259c15a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a0a00640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f259c1527b0 0x7f259c15a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a0a00640 1 -- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c0b4de0 msgr2=0x7f259c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a0a00640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c0b4de0 0x7f259c151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a0a00640 1 -- 172.21.15.50:0/3242295523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c0aa4d0 con 0x7f259c0b5c70 2024-09-21T05:57:17.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a1400640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c0b4de0 0x7f259c151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:17.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a0a00640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259c0b5c70 0x7f259c0af450 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f258800c9a0 tx=0x7f258800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:17.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f2599600640 1 -- 172.21.15.50:0/3242295523 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588014070 con 0x7f259c0b5c70 2024-09-21T05:57:17.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f2599600640 1 -- 172.21.15.50:0/3242295523 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25880077a0 con 0x7f259c0b5c70 2024-09-21T05:57:17.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a3e00640 1 -- 172.21.15.50:0/3242295523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259c0aa7b0 con 0x7f259c0b5c70 2024-09-21T05:57:17.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f2599600640 1 -- 172.21.15.50:0/3242295523 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588007bd0 con 0x7f259c0b5c70 2024-09-21T05:57:17.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a3e00640 1 -- 172.21.15.50:0/3242295523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259c0aad70 con 0x7f259c0b5c70 2024-09-21T05:57:17.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f25a3e00640 1 -- 172.21.15.50:0/3242295523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259c0b51f0 con 0x7f259c0b5c70 2024-09-21T05:57:17.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.494+0000 7f2599600640 1 -- 172.21.15.50:0/3242295523 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2588017070 con 0x7f259c0b5c70 2024-09-21T05:57:17.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.498+0000 7f2599600640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f257c073230 0x7f257c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:17.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.498+0000 7f2599600640 1 -- 172.21.15.50:0/3242295523 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f25880944f0 con 0x7f259c0b5c70 2024-09-21T05:57:17.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.498+0000 7f25a1400640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f257c073230 0x7f257c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:17.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.498+0000 7f25a1400640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f257c073230 0x7f257c075740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25900098b0 tx=0x7f2590008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:17.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.502+0000 7f2599600640 1 -- 172.21.15.50:0/3242295523 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f258805f4f0 con 0x7f259c0b5c70 2024-09-21T05:57:17.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.682+0000 7f25a3e00640 1 -- 172.21.15.50:0/3242295523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f259c0af990 con 0x7f259c0b5c70 2024-09-21T05:57:17.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.682+0000 7f2599600640 1 -- 172.21.15.50:0/3242295523 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16694 (secure 0 0 0) 0x7f2588092e20 con 0x7f259c0b5c70 2024-09-21T05:57:17.689 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:17.689 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":32,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:57:16.325278+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:57:14.083279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T05:57:15.292209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:57:17.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.686+0000 7f25a3e00640 1 -- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f257c073230 msgr2=0x7f257c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.686+0000 7f25a3e00640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f257c073230 0x7f257c075740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25900098b0 tx=0x7f2590008040 comp rx=0 tx=0).stop 2024-09-21T05:57:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.686+0000 7f25a3e00640 1 -- 172.21.15.50:0/3242295523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259c0b5c70 msgr2=0x7f259c0af450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.686+0000 7f25a3e00640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259c0b5c70 0x7f259c0af450 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f258800c9a0 tx=0x7f258800ce60 comp rx=0 tx=0).stop 2024-09-21T05:57:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.686+0000 7f25a3e00640 1 -- 172.21.15.50:0/3242295523 shutdown_connections 2024-09-21T05:57:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.686+0000 7f25a3e00640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f259c1527b0 0x7f259c15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.686+0000 7f25a3e00640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f257c073230 0x7f257c075740 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.686+0000 7f25a3e00640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259c0b5c70 0x7f259c0af450 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.686+0000 7f25a3e00640 1 --2- 172.21.15.50:0/3242295523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c0b4de0 0x7f259c151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.686+0000 7f25a3e00640 1 -- 172.21.15.50:0/3242295523 >> 172.21.15.50:0/3242295523 conn(0x7f259c0b2040 msgr2=0x7f259c0ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.686+0000 7f25a3e00640 1 -- 172.21.15.50:0/3242295523 shutdown_connections 2024-09-21T05:57:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.686+0000 7f25a3e00640 1 -- 172.21.15.50:0/3242295523 wait complete. 2024-09-21T05:57:17.711 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T05:57:17.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.866+0000 7fe866c00640 1 -- 172.21.15.50:0/3007706803 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe86015f5c0 msgr2=0x7fe86015f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.866+0000 7fe866c00640 1 --2- 172.21.15.50:0/3007706803 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe86015f5c0 0x7fe86015f9d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe858009bc0 tx=0x7fe85801cb10 comp rx=0 tx=0).stop 2024-09-21T05:57:17.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 -- 172.21.15.50:0/3007706803 shutdown_connections 2024-09-21T05:57:17.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 --2- 172.21.15.50:0/3007706803 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe86015f5c0 0x7fe86015f9d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 --2- 172.21.15.50:0/3007706803 >> v2:172.21.15.88:3300/0 conn(0x7fe8600d1140 0x7fe8600b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 -- 172.21.15.50:0/3007706803 >> 172.21.15.50:0/3007706803 conn(0x7fe8600b41f0 msgr2=0x7fe8600b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:17.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 -- 172.21.15.50:0/3007706803 shutdown_connections 2024-09-21T05:57:17.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 -- 172.21.15.50:0/3007706803 wait complete. 2024-09-21T05:57:17.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 Processor -- start 2024-09-21T05:57:17.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 -- start start 2024-09-21T05:57:17.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe86006dbc0 0x7fe86009c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:17.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8600d1140 0x7fe86009cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:17.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe86015f5c0 0x7fe860154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:17.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe860168ff0 con 0x7fe8600d1140 2024-09-21T05:57:17.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe860168e70 con 0x7fe86015f5c0 2024-09-21T05:57:17.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.870+0000 7fe866c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe860169170 con 0x7fe86006dbc0 2024-09-21T05:57:17.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe864e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8600d1140 0x7fe86009cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:17.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe864e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8600d1140 0x7fe86009cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58822/0 (socket says 172.21.15.50:58822) 2024-09-21T05:57:17.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe864e00640 1 -- 172.21.15.50:0/1464996951 learned_addr learned my addr 172.21.15.50:0/1464996951 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:17.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe865800640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe86006dbc0 0x7fe86009c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:17.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe866200640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe86015f5c0 0x7fe860154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:17.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe864e00640 1 -- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe86006dbc0 msgr2=0x7fe86009c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe864e00640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe86006dbc0 0x7fe86009c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe864e00640 1 -- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe86015f5c0 msgr2=0x7fe860154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:17.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe864e00640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe86015f5c0 0x7fe860154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:17.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe864e00640 1 -- 172.21.15.50:0/1464996951 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8600b3370 con 0x7fe8600d1140 2024-09-21T05:57:17.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe864e00640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8600d1140 0x7fe86009cb40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00c460 tx=0x7fe84c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:17.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe85d600640 1 -- 172.21.15.50:0/1464996951 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c016020 con 0x7fe8600d1140 2024-09-21T05:57:17.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe85d600640 1 -- 172.21.15.50:0/1464996951 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe84c012940 con 0x7fe8600d1140 2024-09-21T05:57:17.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe866c00640 1 -- 172.21.15.50:0/1464996951 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86015cf90 con 0x7fe8600d1140 2024-09-21T05:57:17.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe85d600640 1 -- 172.21.15.50:0/1464996951 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c012d90 con 0x7fe8600d1140 2024-09-21T05:57:17.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe866c00640 1 -- 172.21.15.50:0/1464996951 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86015d550 con 0x7fe8600d1140 2024-09-21T05:57:17.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe866c00640 1 -- 172.21.15.50:0/1464996951 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8600b3540 con 0x7fe8600d1140 2024-09-21T05:57:17.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.874+0000 7fe85d600640 1 -- 172.21.15.50:0/1464996951 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe84c019020 con 0x7fe8600d1140 2024-09-21T05:57:17.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.878+0000 7fe85d600640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe840073200 0x7fe840075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:17.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.878+0000 7fe85d600640 1 -- 172.21.15.50:0/1464996951 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe84c093f80 con 0x7fe8600d1140 2024-09-21T05:57:17.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.878+0000 7fe865800640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe840073200 0x7fe840075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:17.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.878+0000 7fe865800640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe840073200 0x7fe840075710 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe854009ad0 tx=0x7fe8540091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:17.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:17.882+0000 7fe85d600640 1 -- 172.21.15.50:0/1464996951 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe8600b3540 con 0x7fe8600d1140 2024-09-21T05:57:18.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.098+0000 7fe866c00640 1 -- 172.21.15.50:0/1464996951 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe8600b0730 con 0x7fe8600d1140 2024-09-21T05:57:18.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.930+0000 7fe85d600640 1 -- 172.21.15.50:0/1464996951 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fe84c020030 con 0x7fe8600d1140 2024-09-21T05:57:18.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.934+0000 7fe866c00640 1 -- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe840073200 msgr2=0x7fe840075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:18.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.934+0000 7fe866c00640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe840073200 0x7fe840075710 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe854009ad0 tx=0x7fe8540091e0 comp rx=0 tx=0).stop 2024-09-21T05:57:18.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.934+0000 7fe866c00640 1 -- 172.21.15.50:0/1464996951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8600d1140 msgr2=0x7fe86009cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:18.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.934+0000 7fe866c00640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8600d1140 0x7fe86009cb40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00c460 tx=0x7fe84c00c920 comp rx=0 tx=0).stop 2024-09-21T05:57:18.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.934+0000 7fe866c00640 1 -- 172.21.15.50:0/1464996951 shutdown_connections 2024-09-21T05:57:18.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.934+0000 7fe866c00640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe86015f5c0 0x7fe860154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:18.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.934+0000 7fe866c00640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe840073200 0x7fe840075710 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:18.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.934+0000 7fe866c00640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8600d1140 0x7fe86009cb40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:18.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.934+0000 7fe866c00640 1 --2- 172.21.15.50:0/1464996951 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe86006dbc0 0x7fe86009c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:18.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.934+0000 7fe866c00640 1 -- 172.21.15.50:0/1464996951 >> 172.21.15.50:0/1464996951 conn(0x7fe8600b41f0 msgr2=0x7fe86006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:18.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.934+0000 7fe866c00640 1 -- 172.21.15.50:0/1464996951 shutdown_connections 2024-09-21T05:57:18.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:18.934+0000 7fe866c00640 1 -- 172.21.15.50:0/1464996951 wait complete. 2024-09-21T05:57:18.963 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T05:57:18.963 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T05:57:19.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.126+0000 7f9de4800640 1 -- 172.21.15.50:0/3869455908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ddc15eeb0 msgr2=0x7f9ddc0cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:19.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.126+0000 7f9de4800640 1 --2- 172.21.15.50:0/3869455908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ddc15eeb0 0x7f9ddc0cf0e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000aa10 tx=0x7f9dd001dc30 comp rx=0 tx=0).stop 2024-09-21T05:57:19.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.126+0000 7f9de4800640 1 -- 172.21.15.50:0/3869455908 shutdown_connections 2024-09-21T05:57:19.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.126+0000 7f9de4800640 1 --2- 172.21.15.50:0/3869455908 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ddc0cf620 0x7f9ddc15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:19.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.126+0000 7f9de4800640 1 --2- 172.21.15.50:0/3869455908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ddc15eeb0 0x7f9ddc0cf0e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:19.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.126+0000 7f9de4800640 1 -- 172.21.15.50:0/3869455908 >> 172.21.15.50:0/3869455908 conn(0x7f9ddc0b2040 msgr2=0x7f9ddc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:19.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.126+0000 7f9de4800640 1 -- 172.21.15.50:0/3869455908 shutdown_connections 2024-09-21T05:57:19.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.126+0000 7f9de4800640 1 -- 172.21.15.50:0/3869455908 wait complete. 2024-09-21T05:57:19.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.126+0000 7f9de4800640 1 Processor -- start 2024-09-21T05:57:19.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.126+0000 7f9de4800640 1 -- start start 2024-09-21T05:57:19.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de4800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ddc0cf620 0x7f9ddc158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:19.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de4800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ddc15b810 0x7f9ddc150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:19.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de4800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ddc15eeb0 0x7f9ddc15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:19.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de4800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc1681f0 con 0x7f9ddc0cf620 2024-09-21T05:57:19.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de4800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc168070 con 0x7f9ddc15eeb0 2024-09-21T05:57:19.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de4800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9ddc168370 con 0x7f9ddc15b810 2024-09-21T05:57:19.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ddc0cf620 0x7f9ddc158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:19.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ddc0cf620 0x7f9ddc158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58840/0 (socket says 172.21.15.50:58840) 2024-09-21T05:57:19.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de1e00640 1 -- 172.21.15.50:0/3805676688 learned_addr learned my addr 172.21.15.50:0/3805676688 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:19.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de1400640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ddc15b810 0x7f9ddc150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:19.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de2800640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ddc15eeb0 0x7f9ddc15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:19.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de1e00640 1 -- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ddc15b810 msgr2=0x7f9ddc150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:19.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de1e00640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ddc15b810 0x7f9ddc150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:19.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de1e00640 1 -- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ddc15eeb0 msgr2=0x7f9ddc15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:19.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de1e00640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ddc15eeb0 0x7f9ddc15a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:19.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de1e00640 1 -- 172.21.15.50:0/3805676688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc06ee30 con 0x7f9ddc0cf620 2024-09-21T05:57:19.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de2800640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ddc15eeb0 0x7f9ddc15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:19.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de1e00640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ddc0cf620 0x7f9ddc158420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000a960 tx=0x7f9dd00038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:19.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9dd6000640 1 -- 172.21.15.50:0/3805676688 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd00b3070 con 0x7f9ddc0cf620 2024-09-21T05:57:19.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9dd6000640 1 -- 172.21.15.50:0/3805676688 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dd0003cb0 con 0x7f9ddc0cf620 2024-09-21T05:57:19.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de4800640 1 -- 172.21.15.50:0/3805676688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc158960 con 0x7f9ddc0cf620 2024-09-21T05:57:19.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9dd6000640 1 -- 172.21.15.50:0/3805676688 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0003e60 con 0x7f9ddc0cf620 2024-09-21T05:57:19.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de4800640 1 -- 172.21.15.50:0/3805676688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc073480 con 0x7f9ddc0cf620 2024-09-21T05:57:19.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.130+0000 7f9de4800640 1 -- 172.21.15.50:0/3805676688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ddc15b2d0 con 0x7f9ddc0cf620 2024-09-21T05:57:19.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.134+0000 7f9dd6000640 1 -- 172.21.15.50:0/3805676688 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9dd0005280 con 0x7f9ddc0cf620 2024-09-21T05:57:19.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.134+0000 7f9dd6000640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9dc0073200 0x7f9dc0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:19.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.134+0000 7f9dd6000640 1 -- 172.21.15.50:0/3805676688 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9dd012e650 con 0x7f9ddc0cf620 2024-09-21T05:57:19.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.134+0000 7f9de1400640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9dc0073200 0x7f9dc0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:19.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.134+0000 7f9de1400640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9dc0073200 0x7f9dc0075710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc13e260 tx=0x7f9dcc0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:19.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.138+0000 7f9dd6000640 1 -- 172.21.15.50:0/3805676688 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9dd00be030 con 0x7f9ddc0cf620 2024-09-21T05:57:19.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.354+0000 7f9de4800640 1 -- 172.21.15.50:0/3805676688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9ddc150570 con 0x7f9ddc0cf620 2024-09-21T05:57:19.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.930+0000 7f9dd6000640 1 -- 172.21.15.50:0/3805676688 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f9dd0017820 con 0x7f9ddc0cf620 2024-09-21T05:57:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.934+0000 7f9de4800640 1 -- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9dc0073200 msgr2=0x7f9dc0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.934+0000 7f9de4800640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9dc0073200 0x7f9dc0075710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc13e260 tx=0x7f9dcc0091e0 comp rx=0 tx=0).stop 2024-09-21T05:57:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.934+0000 7f9de4800640 1 -- 172.21.15.50:0/3805676688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ddc0cf620 msgr2=0x7f9ddc158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.934+0000 7f9de4800640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ddc0cf620 0x7f9ddc158420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000a960 tx=0x7f9dd00038b0 comp rx=0 tx=0).stop 2024-09-21T05:57:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.934+0000 7f9de4800640 1 -- 172.21.15.50:0/3805676688 shutdown_connections 2024-09-21T05:57:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.934+0000 7f9de4800640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ddc15eeb0 0x7f9ddc15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.934+0000 7f9de4800640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ddc15b810 0x7f9ddc150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.934+0000 7f9de4800640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9dc0073200 0x7f9dc0075710 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.934+0000 7f9de4800640 1 --2- 172.21.15.50:0/3805676688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ddc0cf620 0x7f9ddc158420 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.934+0000 7f9de4800640 1 -- 172.21.15.50:0/3805676688 >> 172.21.15.50:0/3805676688 conn(0x7f9ddc0b2040 msgr2=0x7f9ddc152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.934+0000 7f9de4800640 1 -- 172.21.15.50:0/3805676688 shutdown_connections 2024-09-21T05:57:19.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:19.934+0000 7f9de4800640 1 -- 172.21.15.50:0/3805676688 wait complete. 2024-09-21T05:57:19.966 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T05:57:19.966 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:57:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.126+0000 7efd54400640 1 -- 172.21.15.50:0/1570114346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd4c0b4de0 msgr2=0x7efd4c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.126+0000 7efd54400640 1 --2- 172.21.15.50:0/1570114346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd4c0b4de0 0x7efd4c0b51f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efd44009c80 tx=0x7efd4401cce0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.126+0000 7efd54400640 1 -- 172.21.15.50:0/1570114346 shutdown_connections 2024-09-21T05:57:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.126+0000 7efd54400640 1 --2- 172.21.15.50:0/1570114346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd4c0b4de0 0x7efd4c0b51f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.126+0000 7efd54400640 1 --2- 172.21.15.50:0/1570114346 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efd4c0b5c70 0x7efd4c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.126+0000 7efd54400640 1 -- 172.21.15.50:0/1570114346 >> 172.21.15.50:0/1570114346 conn(0x7efd4c0b2040 msgr2=0x7efd4c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:20.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.126+0000 7efd54400640 1 -- 172.21.15.50:0/1570114346 shutdown_connections 2024-09-21T05:57:20.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.126+0000 7efd54400640 1 -- 172.21.15.50:0/1570114346 wait complete. 2024-09-21T05:57:20.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.126+0000 7efd54400640 1 Processor -- start 2024-09-21T05:57:20.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.126+0000 7efd54400640 1 -- start start 2024-09-21T05:57:20.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd54400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd4c0b4de0 0x7efd4c1516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:20.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd54400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd4c0b5c70 0x7efd4c14fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:20.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd54400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efd4c1527b0 0x7efd4c1f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:20.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd54400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c168ab0 con 0x7efd4c0b5c70 2024-09-21T05:57:20.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd54400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c168930 con 0x7efd4c0b4de0 2024-09-21T05:57:20.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd54400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7efd4c168c30 con 0x7efd4c1527b0 2024-09-21T05:57:20.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd51000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd4c0b5c70 0x7efd4c14fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:20.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd51000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd4c0b5c70 0x7efd4c14fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58858/0 (socket says 172.21.15.50:58858) 2024-09-21T05:57:20.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd51000640 1 -- 172.21.15.50:0/455185875 learned_addr learned my addr 172.21.15.50:0/455185875 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:20.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd51a00640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd4c0b4de0 0x7efd4c1516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:20.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd52400640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efd4c1527b0 0x7efd4c1f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:20.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd51000640 1 -- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efd4c1527b0 msgr2=0x7efd4c1f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd51000640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efd4c1527b0 0x7efd4c1f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd51000640 1 -- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd4c0b4de0 msgr2=0x7efd4c1516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd51000640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd4c0b4de0 0x7efd4c1516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd51000640 1 -- 172.21.15.50:0/455185875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4c1505e0 con 0x7efd4c0b5c70 2024-09-21T05:57:20.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd51000640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd4c0b5c70 0x7efd4c14fdb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efd3800c390 tx=0x7efd3800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:20.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd49600640 1 -- 172.21.15.50:0/455185875 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38017070 con 0x7efd4c0b5c70 2024-09-21T05:57:20.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd49600640 1 -- 172.21.15.50:0/455185875 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd380053c0 con 0x7efd4c0b5c70 2024-09-21T05:57:20.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd54400640 1 -- 172.21.15.50:0/455185875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4c1508c0 con 0x7efd4c0b5c70 2024-09-21T05:57:20.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd49600640 1 -- 172.21.15.50:0/455185875 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38003840 con 0x7efd4c0b5c70 2024-09-21T05:57:20.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd54400640 1 -- 172.21.15.50:0/455185875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4c0ceba0 con 0x7efd4c0b5c70 2024-09-21T05:57:20.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.130+0000 7efd54400640 1 -- 172.21.15.50:0/455185875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd4c0b51f0 con 0x7efd4c0b5c70 2024-09-21T05:57:20.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.134+0000 7efd49600640 1 -- 172.21.15.50:0/455185875 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efd3801a070 con 0x7efd4c0b5c70 2024-09-21T05:57:20.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.134+0000 7efd49600640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efd2c073200 0x7efd2c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:20.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.134+0000 7efd49600640 1 -- 172.21.15.50:0/455185875 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efd380938a0 con 0x7efd4c0b5c70 2024-09-21T05:57:20.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.134+0000 7efd51a00640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efd2c073200 0x7efd2c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:20.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.134+0000 7efd51a00640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efd2c073200 0x7efd2c075710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd40009b40 tx=0x7efd400091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:20.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.138+0000 7efd49600640 1 -- 172.21.15.50:0/455185875 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efd38021020 con 0x7efd4c0b5c70 2024-09-21T05:57:20.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.350+0000 7efd54400640 1 -- 172.21.15.50:0/455185875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd4c150370 con 0x7efd4c0b5c70 2024-09-21T05:57:20.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.350+0000 7efd49600640 1 -- 172.21.15.50:0/455185875 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5217 (secure 0 0 0) 0x7efd4c0b5730 con 0x7efd4c0b5c70 2024-09-21T05:57:20.356 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:20.357 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"btime":"2024-09-21T05:57:19:934324+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3475589656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3475589656},{"type":"v1","addr":"172.21.15.88:6835","nonce":3475589656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/88740447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":88740447},{"type":"v1","addr":"172.21.15.88:6837","nonce":88740447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4798,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/4086930068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":4086930068},{"type":"v1","addr":"172.21.15.50:6833","nonce":4086930068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:57:16.325238+0000","modified":"2024-09-21T05:57:19.359942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4813},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4813":{"gid":4813,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/2722641195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2722641195},{"type":"v1","addr":"172.21.15.50:6835","nonce":2722641195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4813,"qdb_cluster":[4813]},"id":2}]} 2024-09-21T05:57:20.357 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2024-09-21T05:57:20.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.354+0000 7efd54400640 1 -- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efd2c073200 msgr2=0x7efd2c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.354+0000 7efd54400640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efd2c073200 0x7efd2c075710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd40009b40 tx=0x7efd400091e0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.354+0000 7efd54400640 1 -- 172.21.15.50:0/455185875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd4c0b5c70 msgr2=0x7efd4c14fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.354+0000 7efd54400640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd4c0b5c70 0x7efd4c14fdb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efd3800c390 tx=0x7efd3800c850 comp rx=0 tx=0).stop 2024-09-21T05:57:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.354+0000 7efd54400640 1 -- 172.21.15.50:0/455185875 shutdown_connections 2024-09-21T05:57:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.354+0000 7efd54400640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efd4c1527b0 0x7efd4c1f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.354+0000 7efd54400640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efd2c073200 0x7efd2c075710 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.354+0000 7efd54400640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd4c0b5c70 0x7efd4c14fdb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.354+0000 7efd54400640 1 --2- 172.21.15.50:0/455185875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd4c0b4de0 0x7efd4c1516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.354+0000 7efd54400640 1 -- 172.21.15.50:0/455185875 >> 172.21.15.50:0/455185875 conn(0x7efd4c0b2040 msgr2=0x7efd4c151dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.354+0000 7efd54400640 1 -- 172.21.15.50:0/455185875 shutdown_connections 2024-09-21T05:57:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.354+0000 7efd54400640 1 -- 172.21.15.50:0/455185875 wait complete. 2024-09-21T05:57:20.379 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:57:20.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.534+0000 7fedc3c00640 1 -- 172.21.15.50:0/1903128856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedbc0703a0 msgr2=0x7fedbc06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.534+0000 7fedc3c00640 1 --2- 172.21.15.50:0/1903128856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedbc0703a0 0x7fedbc06d200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009bc0 tx=0x7fedb401cb30 comp rx=0 tx=0).stop 2024-09-21T05:57:20.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.534+0000 7fedc3c00640 1 -- 172.21.15.50:0/1903128856 shutdown_connections 2024-09-21T05:57:20.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.534+0000 7fedc3c00640 1 --2- 172.21.15.50:0/1903128856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedbc0703a0 0x7fedbc06d200 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.534+0000 7fedc3c00640 1 --2- 172.21.15.50:0/1903128856 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fedbc159130 0x7fedbc15bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.534+0000 7fedc3c00640 1 -- 172.21.15.50:0/1903128856 >> 172.21.15.50:0/1903128856 conn(0x7fedbc0b35f0 msgr2=0x7fedbc0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:20.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.534+0000 7fedc3c00640 1 -- 172.21.15.50:0/1903128856 shutdown_connections 2024-09-21T05:57:20.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.534+0000 7fedc3c00640 1 -- 172.21.15.50:0/1903128856 wait complete. 2024-09-21T05:57:20.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.534+0000 7fedc3c00640 1 Processor -- start 2024-09-21T05:57:20.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.534+0000 7fedc3c00640 1 -- start start 2024-09-21T05:57:20.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc3c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedbc06cdf0 0x7fedbc1fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:20.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc3c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc0703a0 0x7fedbc157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:20.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc3c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fedbc159130 0x7fedbc15f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:20.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc3c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc168df0 con 0x7fedbc06cdf0 2024-09-21T05:57:20.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc3c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc168c70 con 0x7fedbc0703a0 2024-09-21T05:57:20.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc3c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fedbc168f70 con 0x7fedbc159130 2024-09-21T05:57:20.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedbc06cdf0 0x7fedbc1fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:20.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedbc06cdf0 0x7fedbc1fcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58874/0 (socket says 172.21.15.50:58874) 2024-09-21T05:57:20.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc1200640 1 -- 172.21.15.50:0/971092753 learned_addr learned my addr 172.21.15.50:0/971092753 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:20.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedb3e00640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc0703a0 0x7fedbc157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:20.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc1c00640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fedbc159130 0x7fedbc15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:20.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc1200640 1 -- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fedbc159130 msgr2=0x7fedbc15f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc1200640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fedbc159130 0x7fedbc15f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc1200640 1 -- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc0703a0 msgr2=0x7fedbc157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc1200640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc0703a0 0x7fedbc157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc1200640 1 -- 172.21.15.50:0/971092753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedbc0cf1a0 con 0x7fedbc06cdf0 2024-09-21T05:57:20.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc1200640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedbc06cdf0 0x7fedbc1fcac0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fedac00e7f0 tx=0x7fedac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:20.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedb9600640 1 -- 172.21.15.50:0/971092753 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac00ccc0 con 0x7fedbc06cdf0 2024-09-21T05:57:20.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedb9600640 1 -- 172.21.15.50:0/971092753 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedac00ce80 con 0x7fedbc06cdf0 2024-09-21T05:57:20.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedb9600640 1 -- 172.21.15.50:0/971092753 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac005280 con 0x7fedbc06cdf0 2024-09-21T05:57:20.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc3c00640 1 -- 172.21.15.50:0/971092753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedbc0cf480 con 0x7fedbc06cdf0 2024-09-21T05:57:20.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc3c00640 1 -- 172.21.15.50:0/971092753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedbc0aa6b0 con 0x7fedbc06cdf0 2024-09-21T05:57:20.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.538+0000 7fedc3c00640 1 -- 172.21.15.50:0/971092753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedbc0aa4d0 con 0x7fedbc06cdf0 2024-09-21T05:57:20.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.542+0000 7fedb9600640 1 -- 172.21.15.50:0/971092753 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fedac005420 con 0x7fedbc06cdf0 2024-09-21T05:57:20.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.542+0000 7fedb9600640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fed98073230 0x7fed98075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:20.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.542+0000 7fedb9600640 1 -- 172.21.15.50:0/971092753 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fedac094ab0 con 0x7fedbc06cdf0 2024-09-21T05:57:20.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.542+0000 7fedb3e00640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fed98073230 0x7fed98075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:20.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.542+0000 7fedb3e00640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fed98073230 0x7fed98075740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feda4007fd0 tx=0x7feda4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:20.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.546+0000 7fedb9600640 1 -- 172.21.15.50:0/971092753 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fedbc0aa4d0 con 0x7fedbc06cdf0 2024-09-21T05:57:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.726+0000 7fedc3c00640 1 -- 172.21.15.50:0/971092753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedbc06d200 con 0x7fedbc06cdf0 2024-09-21T05:57:20.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.726+0000 7fedb9600640 1 -- 172.21.15.50:0/971092753 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16681 (secure 0 0 0) 0x7fedac05f480 con 0x7fedbc06cdf0 2024-09-21T05:57:20.732 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:20.732 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":33,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:57:17.917372+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:57:14.083279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T05:57:15.292209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:57:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.730+0000 7fedc3c00640 1 -- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fed98073230 msgr2=0x7fed98075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.730+0000 7fedc3c00640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fed98073230 0x7fed98075740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feda4007fd0 tx=0x7feda4009040 comp rx=0 tx=0).stop 2024-09-21T05:57:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.730+0000 7fedc3c00640 1 -- 172.21.15.50:0/971092753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedbc06cdf0 msgr2=0x7fedbc1fcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.730+0000 7fedc3c00640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedbc06cdf0 0x7fedbc1fcac0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fedac00e7f0 tx=0x7fedac00ecb0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.730+0000 7fedc3c00640 1 -- 172.21.15.50:0/971092753 shutdown_connections 2024-09-21T05:57:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.730+0000 7fedc3c00640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fedbc159130 0x7fedbc15f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.730+0000 7fedc3c00640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc0703a0 0x7fedbc157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.730+0000 7fedc3c00640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fed98073230 0x7fed98075740 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.730+0000 7fedc3c00640 1 --2- 172.21.15.50:0/971092753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedbc06cdf0 0x7fedbc1fcac0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.730+0000 7fedc3c00640 1 -- 172.21.15.50:0/971092753 >> 172.21.15.50:0/971092753 conn(0x7fedbc0b35f0 msgr2=0x7fedbc1646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.730+0000 7fedc3c00640 1 -- 172.21.15.50:0/971092753 shutdown_connections 2024-09-21T05:57:20.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.730+0000 7fedc3c00640 1 -- 172.21.15.50:0/971092753 wait complete. 2024-09-21T05:57:20.755 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:57:20.755 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:57:20.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.906+0000 7fabfc600640 1 -- 172.21.15.50:0/3148404100 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fabf40b5a80 msgr2=0x7fabf4074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.906+0000 7fabfc600640 1 --2- 172.21.15.50:0/3148404100 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fabf40b5a80 0x7fabf4074410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009840 tx=0x7fabe001c820 comp rx=0 tx=0).stop 2024-09-21T05:57:20.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.910+0000 7fabfc600640 1 -- 172.21.15.50:0/3148404100 shutdown_connections 2024-09-21T05:57:20.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.910+0000 7fabfc600640 1 --2- 172.21.15.50:0/3148404100 >> v2:172.21.15.88:3300/0 conn(0x7fabf4074950 0x7fabf415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.910+0000 7fabfc600640 1 --2- 172.21.15.50:0/3148404100 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fabf40b5a80 0x7fabf4074410 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.910+0000 7fabfc600640 1 -- 172.21.15.50:0/3148404100 >> 172.21.15.50:0/3148404100 conn(0x7fabf40708c0 msgr2=0x7fabf4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:20.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.910+0000 7fabfc600640 1 -- 172.21.15.50:0/3148404100 shutdown_connections 2024-09-21T05:57:20.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.910+0000 7fabfc600640 1 -- 172.21.15.50:0/3148404100 wait complete. 2024-09-21T05:57:20.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.910+0000 7fabfc600640 1 Processor -- start 2024-09-21T05:57:20.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.910+0000 7fabfc600640 1 -- start start 2024-09-21T05:57:20.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabfc600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabf4074950 0x7fabf40b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabfc600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf40b5a80 0x7fabf41f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabfc600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fabf4157030 0x7fabf4149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabfc600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fabf416a310 con 0x7fabf40b5a80 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabfc600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fabf416a190 con 0x7fabf4074950 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabfc600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fabf416a490 con 0x7fabf4157030 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf40b5a80 0x7fabf41f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf40b5a80 0x7fabf41f9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58886/0 (socket says 172.21.15.50:58886) 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf9200640 1 -- 172.21.15.50:0/3918320983 learned_addr learned my addr 172.21.15.50:0/3918320983 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf9c00640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabf4074950 0x7fabf40b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabfa600640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fabf4157030 0x7fabf4149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf9200640 1 -- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fabf4157030 msgr2=0x7fabf4149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf9200640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fabf4157030 0x7fabf4149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf9200640 1 -- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabf4074950 msgr2=0x7fabf40b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:20.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf9200640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabf4074950 0x7fabf40b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:20.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf9200640 1 -- 172.21.15.50:0/3918320983 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf4155e10 con 0x7fabf40b5a80 2024-09-21T05:57:20.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabfa600640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fabf4157030 0x7fabf4149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:20.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf9200640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf40b5a80 0x7fabf41f9eb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fabe001cd00 tx=0x7fabe0005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:20.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf1600640 1 -- 172.21.15.50:0/3918320983 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe00b2040 con 0x7fabf40b5a80 2024-09-21T05:57:20.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf1600640 1 -- 172.21.15.50:0/3918320983 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabe0004040 con 0x7fabf40b5a80 2024-09-21T05:57:20.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf1600640 1 -- 172.21.15.50:0/3918320983 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0004410 con 0x7fabf40b5a80 2024-09-21T05:57:20.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabfc600640 1 -- 172.21.15.50:0/3918320983 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf41560f0 con 0x7fabf40b5a80 2024-09-21T05:57:20.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabfc600640 1 -- 172.21.15.50:0/3918320983 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf41fa3f0 con 0x7fabf40b5a80 2024-09-21T05:57:20.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabfc600640 1 -- 172.21.15.50:0/3918320983 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf415cf90 con 0x7fabf40b5a80 2024-09-21T05:57:20.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.914+0000 7fabf1600640 1 -- 172.21.15.50:0/3918320983 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fabe00bd030 con 0x7fabf40b5a80 2024-09-21T05:57:20.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.918+0000 7fabf1600640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabd4073230 0x7fabd4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:20.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.918+0000 7fabf1600640 1 -- 172.21.15.50:0/3918320983 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fabe012dcb0 con 0x7fabf40b5a80 2024-09-21T05:57:20.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.918+0000 7fabf9c00640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabd4073230 0x7fabd4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:20.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.918+0000 7fabf9c00640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabd4073230 0x7fabd4075740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fabe8007fd0 tx=0x7fabe8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:20.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:20.922+0000 7fabf1600640 1 -- 172.21.15.50:0/3918320983 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fabe00c4060 con 0x7fabf40b5a80 2024-09-21T05:57:21.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.094+0000 7fabfc600640 1 -- 172.21.15.50:0/3918320983 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fabf406b540 con 0x7fabd4073230 2024-09-21T05:57:21.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.102+0000 7fabf1600640 1 -- 172.21.15.50:0/3918320983 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7fabe8001ca0 con 0x7fabd4073230 2024-09-21T05:57:21.109 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:21.110 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:57:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.106+0000 7fabfc600640 1 -- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabd4073230 msgr2=0x7fabd4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.106+0000 7fabfc600640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabd4073230 0x7fabd4075740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fabe8007fd0 tx=0x7fabe8009040 comp rx=0 tx=0).stop 2024-09-21T05:57:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.106+0000 7fabfc600640 1 -- 172.21.15.50:0/3918320983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf40b5a80 msgr2=0x7fabf41f9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.106+0000 7fabfc600640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf40b5a80 0x7fabf41f9eb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fabe001cd00 tx=0x7fabe0005ca0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.106+0000 7fabfc600640 1 -- 172.21.15.50:0/3918320983 shutdown_connections 2024-09-21T05:57:21.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.106+0000 7fabfc600640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fabf4157030 0x7fabf4149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.106+0000 7fabfc600640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabd4073230 0x7fabd4075740 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.106+0000 7fabfc600640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf40b5a80 0x7fabf41f9eb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.106+0000 7fabfc600640 1 --2- 172.21.15.50:0/3918320983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabf4074950 0x7fabf40b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.106+0000 7fabfc600640 1 -- 172.21.15.50:0/3918320983 >> 172.21.15.50:0/3918320983 conn(0x7fabf40708c0 msgr2=0x7fabf4150440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:21.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.106+0000 7fabfc600640 1 -- 172.21.15.50:0/3918320983 shutdown_connections 2024-09-21T05:57:21.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.106+0000 7fabfc600640 1 -- 172.21.15.50:0/3918320983 wait complete. 2024-09-21T05:57:21.133 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-21T05:57:20.896506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236868,"kb_used_data":3984,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737852,"statfs":{"total":773094113280,"available":772851560448,"internally_reserved":0,"allocated":4079616,"data_stored":2261930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001594"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.945107+0000","last_change":"2024-09-21T05:57:16.940234+0000","last_active":"2024-09-21T05:57:16.945107+0000","last_peered":"2024-09-21T05:57:16.945107+0000","last_clean":"2024-09-21T05:57:16.945107+0000","last_became_active":"2024-09-21T05:57:16.939897+0000","last_became_peered":"2024-09-21T05:57:16.939897+0000","last_unstale":"2024-09-21T05:57:16.945107+0000","last_undegraded":"2024-09-21T05:57:16.945107+0000","last_fullsized":"2024-09-21T05:57:16.945107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:13.788363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927762+0000","last_change":"2024-09-21T05:57:16.943546+0000","last_active":"2024-09-21T05:57:17.927762+0000","last_peered":"2024-09-21T05:57:17.927762+0000","last_clean":"2024-09-21T05:57:17.927762+0000","last_became_active":"2024-09-21T05:57:16.941470+0000","last_became_peered":"2024-09-21T05:57:16.941470+0000","last_unstale":"2024-09-21T05:57:17.927762+0000","last_undegraded":"2024-09-21T05:57:17.927762+0000","last_fullsized":"2024-09-21T05:57:17.927762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.164703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928920+0000","last_change":"2024-09-21T05:57:16.944278+0000","last_active":"2024-09-21T05:57:17.928920+0000","last_peered":"2024-09-21T05:57:17.928920+0000","last_clean":"2024-09-21T05:57:17.928920+0000","last_became_active":"2024-09-21T05:57:16.943563+0000","last_became_peered":"2024-09-21T05:57:16.943563+0000","last_unstale":"2024-09-21T05:57:17.928920+0000","last_undegraded":"2024-09-21T05:57:17.928920+0000","last_fullsized":"2024-09-21T05:57:17.928920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:24.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.949159+0000","last_change":"2024-09-21T05:57:16.947157+0000","last_active":"2024-09-21T05:57:16.949159+0000","last_peered":"2024-09-21T05:57:16.949159+0000","last_clean":"2024-09-21T05:57:16.949159+0000","last_became_active":"2024-09-21T05:57:16.946490+0000","last_became_peered":"2024-09-21T05:57:16.946490+0000","last_unstale":"2024-09-21T05:57:16.949159+0000","last_undegraded":"2024-09-21T05:57:16.949159+0000","last_fullsized":"2024-09-21T05:57:16.949159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:45.056061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.944661+0000","last_change":"2024-09-21T05:57:16.937915+0000","last_active":"2024-09-21T05:57:16.944661+0000","last_peered":"2024-09-21T05:57:16.944661+0000","last_clean":"2024-09-21T05:57:16.944661+0000","last_became_active":"2024-09-21T05:57:16.937607+0000","last_became_peered":"2024-09-21T05:57:16.937607+0000","last_unstale":"2024-09-21T05:57:16.944661+0000","last_undegraded":"2024-09-21T05:57:16.944661+0000","last_fullsized":"2024-09-21T05:57:16.944661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:30.785047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927182+0000","last_change":"2024-09-21T05:57:16.950111+0000","last_active":"2024-09-21T05:57:17.927182+0000","last_peered":"2024-09-21T05:57:17.927182+0000","last_clean":"2024-09-21T05:57:17.927182+0000","last_became_active":"2024-09-21T05:57:16.945063+0000","last_became_peered":"2024-09-21T05:57:16.945063+0000","last_unstale":"2024-09-21T05:57:17.927182+0000","last_undegraded":"2024-09-21T05:57:17.927182+0000","last_fullsized":"2024-09-21T05:57:17.927182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:40.125474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.940660+0000","last_change":"2024-09-21T05:57:16.937457+0000","last_active":"2024-09-21T05:57:16.940660+0000","last_peered":"2024-09-21T05:57:16.940660+0000","last_clean":"2024-09-21T05:57:16.940660+0000","last_became_active":"2024-09-21T05:57:16.937121+0000","last_became_peered":"2024-09-21T05:57:16.937121+0000","last_unstale":"2024-09-21T05:57:16.940660+0000","last_undegraded":"2024-09-21T05:57:16.940660+0000","last_fullsized":"2024-09-21T05:57:16.940660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:09.248511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.939399+0000","last_change":"2024-09-21T05:57:16.936161+0000","last_active":"2024-09-21T05:57:16.939399+0000","last_peered":"2024-09-21T05:57:16.939399+0000","last_clean":"2024-09-21T05:57:16.939399+0000","last_became_active":"2024-09-21T05:57:16.935741+0000","last_became_peered":"2024-09-21T05:57:16.935741+0000","last_unstale":"2024-09-21T05:57:16.939399+0000","last_undegraded":"2024-09-21T05:57:16.939399+0000","last_fullsized":"2024-09-21T05:57:16.939399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:41.787430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930695+0000","last_change":"2024-09-21T05:57:16.944136+0000","last_active":"2024-09-21T05:57:17.930695+0000","last_peered":"2024-09-21T05:57:17.930695+0000","last_clean":"2024-09-21T05:57:17.930695+0000","last_became_active":"2024-09-21T05:57:16.943762+0000","last_became_peered":"2024-09-21T05:57:16.943762+0000","last_unstale":"2024-09-21T05:57:17.930695+0000","last_undegraded":"2024-09-21T05:57:17.930695+0000","last_fullsized":"2024-09-21T05:57:17.930695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:09.580318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.954693+0000","last_change":"2024-09-21T05:57:16.953289+0000","last_active":"2024-09-21T05:57:16.954693+0000","last_peered":"2024-09-21T05:57:16.954693+0000","last_clean":"2024-09-21T05:57:16.954693+0000","last_became_active":"2024-09-21T05:57:16.952949+0000","last_became_peered":"2024-09-21T05:57:16.952949+0000","last_unstale":"2024-09-21T05:57:16.954693+0000","last_undegraded":"2024-09-21T05:57:16.954693+0000","last_fullsized":"2024-09-21T05:57:16.954693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:04.044328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.958308+0000","last_change":"2024-09-21T05:57:16.952792+0000","last_active":"2024-09-21T05:57:16.958308+0000","last_peered":"2024-09-21T05:57:16.958308+0000","last_clean":"2024-09-21T05:57:16.958308+0000","last_became_active":"2024-09-21T05:57:16.952468+0000","last_became_peered":"2024-09-21T05:57:16.952468+0000","last_unstale":"2024-09-21T05:57:16.958308+0000","last_undegraded":"2024-09-21T05:57:16.958308+0000","last_fullsized":"2024-09-21T05:57:16.958308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:37.101273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928545+0000","last_change":"2024-09-21T05:57:16.943251+0000","last_active":"2024-09-21T05:57:17.928545+0000","last_peered":"2024-09-21T05:57:17.928545+0000","last_clean":"2024-09-21T05:57:17.928545+0000","last_became_active":"2024-09-21T05:57:16.941982+0000","last_became_peered":"2024-09-21T05:57:16.941982+0000","last_unstale":"2024-09-21T05:57:17.928545+0000","last_undegraded":"2024-09-21T05:57:17.928545+0000","last_fullsized":"2024-09-21T05:57:17.928545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:40.125474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.955663+0000","last_change":"2024-09-21T05:57:16.949896+0000","last_active":"2024-09-21T05:57:16.955663+0000","last_peered":"2024-09-21T05:57:16.955663+0000","last_clean":"2024-09-21T05:57:16.955663+0000","last_became_active":"2024-09-21T05:57:16.949651+0000","last_became_peered":"2024-09-21T05:57:16.949651+0000","last_unstale":"2024-09-21T05:57:16.955663+0000","last_undegraded":"2024-09-21T05:57:16.955663+0000","last_fullsized":"2024-09-21T05:57:16.955663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:19.925729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927353+0000","last_change":"2024-09-21T05:57:16.951013+0000","last_active":"2024-09-21T05:57:17.927353+0000","last_peered":"2024-09-21T05:57:17.927353+0000","last_clean":"2024-09-21T05:57:17.927353+0000","last_became_active":"2024-09-21T05:57:16.945539+0000","last_became_peered":"2024-09-21T05:57:16.945539+0000","last_unstale":"2024-09-21T05:57:17.927353+0000","last_undegraded":"2024-09-21T05:57:17.927353+0000","last_fullsized":"2024-09-21T05:57:17.927353+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:31.707738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927822+0000","last_change":"2024-09-21T05:57:16.938984+0000","last_active":"2024-09-21T05:57:17.927822+0000","last_peered":"2024-09-21T05:57:17.927822+0000","last_clean":"2024-09-21T05:57:17.927822+0000","last_became_active":"2024-09-21T05:57:16.938621+0000","last_became_peered":"2024-09-21T05:57:16.938621+0000","last_unstale":"2024-09-21T05:57:17.927822+0000","last_undegraded":"2024-09-21T05:57:17.927822+0000","last_fullsized":"2024-09-21T05:57:17.927822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:27.248981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.945437+0000","last_change":"2024-09-21T05:57:16.944128+0000","last_active":"2024-09-21T05:57:16.945437+0000","last_peered":"2024-09-21T05:57:16.945437+0000","last_clean":"2024-09-21T05:57:16.945437+0000","last_became_active":"2024-09-21T05:57:16.940434+0000","last_became_peered":"2024-09-21T05:57:16.940434+0000","last_unstale":"2024-09-21T05:57:16.945437+0000","last_undegraded":"2024-09-21T05:57:16.945437+0000","last_fullsized":"2024-09-21T05:57:16.945437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:52.641317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.950436+0000","last_change":"2024-09-21T05:57:16.944547+0000","last_active":"2024-09-21T05:57:16.950436+0000","last_peered":"2024-09-21T05:57:16.950436+0000","last_clean":"2024-09-21T05:57:16.950436+0000","last_became_active":"2024-09-21T05:57:16.944248+0000","last_became_peered":"2024-09-21T05:57:16.944248+0000","last_unstale":"2024-09-21T05:57:16.950436+0000","last_undegraded":"2024-09-21T05:57:16.950436+0000","last_fullsized":"2024-09-21T05:57:16.950436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:09.580318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927806+0000","last_change":"2024-09-21T05:57:16.956746+0000","last_active":"2024-09-21T05:57:17.927806+0000","last_peered":"2024-09-21T05:57:17.927806+0000","last_clean":"2024-09-21T05:57:17.927806+0000","last_became_active":"2024-09-21T05:57:16.956448+0000","last_became_peered":"2024-09-21T05:57:16.956448+0000","last_unstale":"2024-09-21T05:57:17.927806+0000","last_undegraded":"2024-09-21T05:57:17.927806+0000","last_fullsized":"2024-09-21T05:57:17.927806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:36.304709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927778+0000","last_change":"2024-09-21T05:57:16.955706+0000","last_active":"2024-09-21T05:57:17.927778+0000","last_peered":"2024-09-21T05:57:17.927778+0000","last_clean":"2024-09-21T05:57:17.927778+0000","last_became_active":"2024-09-21T05:57:16.955379+0000","last_became_peered":"2024-09-21T05:57:16.955379+0000","last_unstale":"2024-09-21T05:57:17.927778+0000","last_undegraded":"2024-09-21T05:57:17.927778+0000","last_fullsized":"2024-09-21T05:57:17.927778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:51.231280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.931634+0000","last_change":"2024-09-21T05:57:16.921955+0000","last_active":"2024-09-21T05:57:16.931634+0000","last_peered":"2024-09-21T05:57:16.931634+0000","last_clean":"2024-09-21T05:57:16.931634+0000","last_became_active":"2024-09-21T05:57:16.921593+0000","last_became_peered":"2024-09-21T05:57:16.921593+0000","last_unstale":"2024-09-21T05:57:16.931634+0000","last_undegraded":"2024-09-21T05:57:16.931634+0000","last_fullsized":"2024-09-21T05:57:16.931634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:08.460664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929755+0000","last_change":"2024-09-21T05:57:16.923990+0000","last_active":"2024-09-21T05:57:17.929755+0000","last_peered":"2024-09-21T05:57:17.929755+0000","last_clean":"2024-09-21T05:57:17.929755+0000","last_became_active":"2024-09-21T05:57:16.922933+0000","last_became_peered":"2024-09-21T05:57:16.922933+0000","last_unstale":"2024-09-21T05:57:17.929755+0000","last_undegraded":"2024-09-21T05:57:17.929755+0000","last_fullsized":"2024-09-21T05:57:17.929755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:41.787430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929784+0000","last_change":"2024-09-21T05:57:16.950653+0000","last_active":"2024-09-21T05:57:17.929784+0000","last_peered":"2024-09-21T05:57:17.929784+0000","last_clean":"2024-09-21T05:57:17.929784+0000","last_became_active":"2024-09-21T05:57:16.950231+0000","last_became_peered":"2024-09-21T05:57:16.950231+0000","last_unstale":"2024-09-21T05:57:17.929784+0000","last_undegraded":"2024-09-21T05:57:17.929784+0000","last_fullsized":"2024-09-21T05:57:17.929784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:09.248511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.928919+0000","last_change":"2024-09-21T05:57:16.924101+0000","last_active":"2024-09-21T05:57:16.928919+0000","last_peered":"2024-09-21T05:57:16.928919+0000","last_clean":"2024-09-21T05:57:16.928919+0000","last_became_active":"2024-09-21T05:57:16.923492+0000","last_became_peered":"2024-09-21T05:57:16.923492+0000","last_unstale":"2024-09-21T05:57:16.928919+0000","last_undegraded":"2024-09-21T05:57:16.928919+0000","last_fullsized":"2024-09-21T05:57:16.928919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:29.785298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927688+0000","last_change":"2024-09-21T05:57:16.929813+0000","last_active":"2024-09-21T05:57:17.927688+0000","last_peered":"2024-09-21T05:57:17.927688+0000","last_clean":"2024-09-21T05:57:17.927688+0000","last_became_active":"2024-09-21T05:57:16.929465+0000","last_became_peered":"2024-09-21T05:57:16.929465+0000","last_unstale":"2024-09-21T05:57:17.927688+0000","last_undegraded":"2024-09-21T05:57:17.927688+0000","last_fullsized":"2024-09-21T05:57:17.927688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:53.736157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.945984+0000","last_change":"2024-09-21T05:57:16.943530+0000","last_active":"2024-09-21T05:57:16.945984+0000","last_peered":"2024-09-21T05:57:16.945984+0000","last_clean":"2024-09-21T05:57:16.945984+0000","last_became_active":"2024-09-21T05:57:16.943189+0000","last_became_peered":"2024-09-21T05:57:16.943189+0000","last_unstale":"2024-09-21T05:57:16.945984+0000","last_undegraded":"2024-09-21T05:57:16.945984+0000","last_fullsized":"2024-09-21T05:57:16.945984+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:58.234837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927990+0000","last_change":"2024-09-21T05:57:16.943892+0000","last_active":"2024-09-21T05:57:17.927990+0000","last_peered":"2024-09-21T05:57:17.927990+0000","last_clean":"2024-09-21T05:57:17.927990+0000","last_became_active":"2024-09-21T05:57:16.942897+0000","last_became_peered":"2024-09-21T05:57:16.942897+0000","last_unstale":"2024-09-21T05:57:17.927990+0000","last_undegraded":"2024-09-21T05:57:17.927990+0000","last_fullsized":"2024-09-21T05:57:17.927990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:07.255500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.952663+0000","last_change":"2024-09-21T05:57:16.946812+0000","last_active":"2024-09-21T05:57:16.952663+0000","last_peered":"2024-09-21T05:57:16.952663+0000","last_clean":"2024-09-21T05:57:16.952663+0000","last_became_active":"2024-09-21T05:57:16.946462+0000","last_became_peered":"2024-09-21T05:57:16.946462+0000","last_unstale":"2024-09-21T05:57:16.952663+0000","last_undegraded":"2024-09-21T05:57:16.952663+0000","last_fullsized":"2024-09-21T05:57:16.952663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:37.284888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928566+0000","last_change":"2024-09-21T05:57:16.918874+0000","last_active":"2024-09-21T05:57:17.928566+0000","last_peered":"2024-09-21T05:57:17.928566+0000","last_clean":"2024-09-21T05:57:17.928566+0000","last_became_active":"2024-09-21T05:57:16.918524+0000","last_became_peered":"2024-09-21T05:57:16.918524+0000","last_unstale":"2024-09-21T05:57:17.928566+0000","last_undegraded":"2024-09-21T05:57:17.928566+0000","last_fullsized":"2024-09-21T05:57:17.928566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:31.707738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930137+0000","last_change":"2024-09-21T05:57:16.941517+0000","last_active":"2024-09-21T05:57:17.930137+0000","last_peered":"2024-09-21T05:57:17.930137+0000","last_clean":"2024-09-21T05:57:17.930137+0000","last_became_active":"2024-09-21T05:57:16.941208+0000","last_became_peered":"2024-09-21T05:57:16.941208+0000","last_unstale":"2024-09-21T05:57:17.930137+0000","last_undegraded":"2024-09-21T05:57:17.930137+0000","last_fullsized":"2024-09-21T05:57:17.930137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:39.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.940411+0000","last_change":"2024-09-21T05:57:16.937583+0000","last_active":"2024-09-21T05:57:16.940411+0000","last_peered":"2024-09-21T05:57:16.940411+0000","last_clean":"2024-09-21T05:57:16.940411+0000","last_became_active":"2024-09-21T05:57:16.936955+0000","last_became_peered":"2024-09-21T05:57:16.936955+0000","last_unstale":"2024-09-21T05:57:16.940411+0000","last_undegraded":"2024-09-21T05:57:16.940411+0000","last_fullsized":"2024-09-21T05:57:16.940411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:39.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.945716+0000","last_change":"2024-09-21T05:57:16.940868+0000","last_active":"2024-09-21T05:57:16.945716+0000","last_peered":"2024-09-21T05:57:16.945716+0000","last_clean":"2024-09-21T05:57:16.945716+0000","last_became_active":"2024-09-21T05:57:16.939925+0000","last_became_peered":"2024-09-21T05:57:16.939925+0000","last_unstale":"2024-09-21T05:57:16.945716+0000","last_undegraded":"2024-09-21T05:57:16.945716+0000","last_fullsized":"2024-09-21T05:57:16.945716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:29.785298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.939371+0000","last_change":"2024-09-21T05:57:16.935337+0000","last_active":"2024-09-21T05:57:16.939371+0000","last_peered":"2024-09-21T05:57:16.939371+0000","last_clean":"2024-09-21T05:57:16.939371+0000","last_became_active":"2024-09-21T05:57:16.932030+0000","last_became_peered":"2024-09-21T05:57:16.932030+0000","last_unstale":"2024-09-21T05:57:16.939371+0000","last_undegraded":"2024-09-21T05:57:16.939371+0000","last_fullsized":"2024-09-21T05:57:16.939371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:08.762203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.950197+0000","last_change":"2024-09-21T05:57:16.933797+0000","last_active":"2024-09-21T05:57:16.950197+0000","last_peered":"2024-09-21T05:57:16.950197+0000","last_clean":"2024-09-21T05:57:16.950197+0000","last_became_active":"2024-09-21T05:57:16.933532+0000","last_became_peered":"2024-09-21T05:57:16.933532+0000","last_unstale":"2024-09-21T05:57:16.950197+0000","last_undegraded":"2024-09-21T05:57:16.950197+0000","last_fullsized":"2024-09-21T05:57:16.950197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:44.012450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.946078+0000","last_change":"2024-09-21T05:57:15.917205+0000","last_active":"2024-09-21T05:57:16.946078+0000","last_peered":"2024-09-21T05:57:16.946078+0000","last_clean":"2024-09-21T05:57:16.946078+0000","last_became_active":"2024-09-21T05:57:15.916908+0000","last_became_peered":"2024-09-21T05:57:15.916908+0000","last_unstale":"2024-09-21T05:57:16.946078+0000","last_undegraded":"2024-09-21T05:57:16.946078+0000","last_fullsized":"2024-09-21T05:57:16.946078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:43.575259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.942327+0000","last_change":"2024-09-21T05:57:16.936756+0000","last_active":"2024-09-21T05:57:16.942327+0000","last_peered":"2024-09-21T05:57:16.942327+0000","last_clean":"2024-09-21T05:57:16.942327+0000","last_became_active":"2024-09-21T05:57:16.936439+0000","last_became_peered":"2024-09-21T05:57:16.936439+0000","last_unstale":"2024-09-21T05:57:16.942327+0000","last_undegraded":"2024-09-21T05:57:16.942327+0000","last_fullsized":"2024-09-21T05:57:16.942327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:23.957773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.950236+0000","last_change":"2024-09-21T05:57:15.920694+0000","last_active":"2024-09-21T05:57:16.950236+0000","last_peered":"2024-09-21T05:57:16.950236+0000","last_clean":"2024-09-21T05:57:16.950236+0000","last_became_active":"2024-09-21T05:57:15.920344+0000","last_became_peered":"2024-09-21T05:57:15.920344+0000","last_unstale":"2024-09-21T05:57:16.950236+0000","last_undegraded":"2024-09-21T05:57:16.950236+0000","last_fullsized":"2024-09-21T05:57:16.950236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:59.310164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930447+0000","last_change":"2024-09-21T05:57:16.941408+0000","last_active":"2024-09-21T05:57:17.930447+0000","last_peered":"2024-09-21T05:57:17.930447+0000","last_clean":"2024-09-21T05:57:17.930447+0000","last_became_active":"2024-09-21T05:57:16.941088+0000","last_became_peered":"2024-09-21T05:57:16.941088+0000","last_unstale":"2024-09-21T05:57:17.930447+0000","last_undegraded":"2024-09-21T05:57:17.930447+0000","last_fullsized":"2024-09-21T05:57:17.930447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:08.599070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927814+0000","last_change":"2024-09-21T05:57:15.925553+0000","last_active":"2024-09-21T05:57:17.927814+0000","last_peered":"2024-09-21T05:57:17.927814+0000","last_clean":"2024-09-21T05:57:17.927814+0000","last_became_active":"2024-09-21T05:57:15.924815+0000","last_became_peered":"2024-09-21T05:57:15.924815+0000","last_unstale":"2024-09-21T05:57:17.927814+0000","last_undegraded":"2024-09-21T05:57:17.927814+0000","last_fullsized":"2024-09-21T05:57:17.927814+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:33.245150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.952635+0000","last_change":"2024-09-21T05:57:16.946642+0000","last_active":"2024-09-21T05:57:16.952635+0000","last_peered":"2024-09-21T05:57:16.952635+0000","last_clean":"2024-09-21T05:57:16.952635+0000","last_became_active":"2024-09-21T05:57:16.945993+0000","last_became_peered":"2024-09-21T05:57:16.945993+0000","last_unstale":"2024-09-21T05:57:16.952635+0000","last_undegraded":"2024-09-21T05:57:16.952635+0000","last_fullsized":"2024-09-21T05:57:16.952635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:19.582982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.953079+0000","last_change":"2024-09-21T05:57:15.918528+0000","last_active":"2024-09-21T05:57:16.953079+0000","last_peered":"2024-09-21T05:57:16.953079+0000","last_clean":"2024-09-21T05:57:16.953079+0000","last_became_active":"2024-09-21T05:57:15.918222+0000","last_became_peered":"2024-09-21T05:57:15.918222+0000","last_unstale":"2024-09-21T05:57:16.953079+0000","last_undegraded":"2024-09-21T05:57:16.953079+0000","last_fullsized":"2024-09-21T05:57:16.953079+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:01.061385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.935933+0000","last_change":"2024-09-21T05:57:16.932739+0000","last_active":"2024-09-21T05:57:16.935933+0000","last_peered":"2024-09-21T05:57:16.935933+0000","last_clean":"2024-09-21T05:57:16.935933+0000","last_became_active":"2024-09-21T05:57:16.932406+0000","last_became_peered":"2024-09-21T05:57:16.932406+0000","last_unstale":"2024-09-21T05:57:16.935933+0000","last_undegraded":"2024-09-21T05:57:16.935933+0000","last_fullsized":"2024-09-21T05:57:16.935933+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:18.466136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930003+0000","last_change":"2024-09-21T05:57:15.926077+0000","last_active":"2024-09-21T05:57:17.930003+0000","last_peered":"2024-09-21T05:57:17.930003+0000","last_clean":"2024-09-21T05:57:17.930003+0000","last_became_active":"2024-09-21T05:57:15.925722+0000","last_became_peered":"2024-09-21T05:57:15.925722+0000","last_unstale":"2024-09-21T05:57:17.930003+0000","last_undegraded":"2024-09-21T05:57:17.930003+0000","last_fullsized":"2024-09-21T05:57:17.930003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:43.575259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928305+0000","last_change":"2024-09-21T05:57:16.917429+0000","last_active":"2024-09-21T05:57:17.928305+0000","last_peered":"2024-09-21T05:57:17.928305+0000","last_clean":"2024-09-21T05:57:17.928305+0000","last_became_active":"2024-09-21T05:57:16.917002+0000","last_became_peered":"2024-09-21T05:57:16.917002+0000","last_unstale":"2024-09-21T05:57:17.928305+0000","last_undegraded":"2024-09-21T05:57:17.928305+0000","last_fullsized":"2024-09-21T05:57:17.928305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:12.234166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929591+0000","last_change":"2024-09-21T05:57:15.925516+0000","last_active":"2024-09-21T05:57:17.929591+0000","last_peered":"2024-09-21T05:57:17.929591+0000","last_clean":"2024-09-21T05:57:17.929591+0000","last_became_active":"2024-09-21T05:57:15.924926+0000","last_became_peered":"2024-09-21T05:57:15.924926+0000","last_unstale":"2024-09-21T05:57:17.929591+0000","last_undegraded":"2024-09-21T05:57:17.929591+0000","last_fullsized":"2024-09-21T05:57:17.929591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:58.082495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928287+0000","last_change":"2024-09-21T05:57:16.941362+0000","last_active":"2024-09-21T05:57:17.928287+0000","last_peered":"2024-09-21T05:57:17.928287+0000","last_clean":"2024-09-21T05:57:17.928287+0000","last_became_active":"2024-09-21T05:57:16.940941+0000","last_became_peered":"2024-09-21T05:57:16.940941+0000","last_unstale":"2024-09-21T05:57:17.928287+0000","last_undegraded":"2024-09-21T05:57:17.928287+0000","last_fullsized":"2024-09-21T05:57:17.928287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:31.954378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927060+0000","last_change":"2024-09-21T05:57:15.929926+0000","last_active":"2024-09-21T05:57:17.927060+0000","last_peered":"2024-09-21T05:57:17.927060+0000","last_clean":"2024-09-21T05:57:17.927060+0000","last_became_active":"2024-09-21T05:57:15.928734+0000","last_became_peered":"2024-09-21T05:57:15.928734+0000","last_unstale":"2024-09-21T05:57:17.927060+0000","last_undegraded":"2024-09-21T05:57:17.927060+0000","last_fullsized":"2024-09-21T05:57:17.927060+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.033103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.951218+0000","last_change":"2024-09-21T05:57:16.944919+0000","last_active":"2024-09-21T05:57:16.951218+0000","last_peered":"2024-09-21T05:57:16.951218+0000","last_clean":"2024-09-21T05:57:16.951218+0000","last_became_active":"2024-09-21T05:57:16.944535+0000","last_became_peered":"2024-09-21T05:57:16.944535+0000","last_unstale":"2024-09-21T05:57:16.951218+0000","last_undegraded":"2024-09-21T05:57:16.951218+0000","last_fullsized":"2024-09-21T05:57:16.951218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:06.976050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928255+0000","last_change":"2024-09-21T05:57:15.909641+0000","last_active":"2024-09-21T05:57:17.928255+0000","last_peered":"2024-09-21T05:57:17.928255+0000","last_clean":"2024-09-21T05:57:17.928255+0000","last_became_active":"2024-09-21T05:57:15.909277+0000","last_became_peered":"2024-09-21T05:57:15.909277+0000","last_unstale":"2024-09-21T05:57:17.928255+0000","last_undegraded":"2024-09-21T05:57:17.928255+0000","last_fullsized":"2024-09-21T05:57:17.928255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:29.782251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929252+0000","last_change":"2024-09-21T05:57:16.941383+0000","last_active":"2024-09-21T05:57:17.929252+0000","last_peered":"2024-09-21T05:57:17.929252+0000","last_clean":"2024-09-21T05:57:17.929252+0000","last_became_active":"2024-09-21T05:57:16.941079+0000","last_became_peered":"2024-09-21T05:57:16.941079+0000","last_unstale":"2024-09-21T05:57:17.929252+0000","last_undegraded":"2024-09-21T05:57:17.929252+0000","last_fullsized":"2024-09-21T05:57:17.929252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.164703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927894+0000","last_change":"2024-09-21T05:57:15.926406+0000","last_active":"2024-09-21T05:57:17.927894+0000","last_peered":"2024-09-21T05:57:17.927894+0000","last_clean":"2024-09-21T05:57:17.927894+0000","last_became_active":"2024-09-21T05:57:15.924778+0000","last_became_peered":"2024-09-21T05:57:15.924778+0000","last_unstale":"2024-09-21T05:57:17.927894+0000","last_undegraded":"2024-09-21T05:57:17.927894+0000","last_fullsized":"2024-09-21T05:57:17.927894+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:40.784634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927895+0000","last_change":"2024-09-21T05:57:16.954250+0000","last_active":"2024-09-21T05:57:17.927895+0000","last_peered":"2024-09-21T05:57:17.927895+0000","last_clean":"2024-09-21T05:57:17.927895+0000","last_became_active":"2024-09-21T05:57:16.953568+0000","last_became_peered":"2024-09-21T05:57:16.953568+0000","last_unstale":"2024-09-21T05:57:17.927895+0000","last_undegraded":"2024-09-21T05:57:17.927895+0000","last_fullsized":"2024-09-21T05:57:17.927895+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:28.060065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.908946+0000","last_change":"2024-09-21T05:57:15.926232+0000","last_active":"2024-09-21T05:57:16.908946+0000","last_peered":"2024-09-21T05:57:16.908946+0000","last_clean":"2024-09-21T05:57:16.908946+0000","last_became_active":"2024-09-21T05:57:15.925837+0000","last_became_peered":"2024-09-21T05:57:15.925837+0000","last_unstale":"2024-09-21T05:57:16.908946+0000","last_undegraded":"2024-09-21T05:57:16.908946+0000","last_fullsized":"2024-09-21T05:57:16.908946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:27:15.049834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927550+0000","last_change":"2024-09-21T05:57:16.941143+0000","last_active":"2024-09-21T05:57:17.927550+0000","last_peered":"2024-09-21T05:57:17.927550+0000","last_clean":"2024-09-21T05:57:17.927550+0000","last_became_active":"2024-09-21T05:57:16.940798+0000","last_became_peered":"2024-09-21T05:57:16.940798+0000","last_unstale":"2024-09-21T05:57:17.927550+0000","last_undegraded":"2024-09-21T05:57:17.927550+0000","last_fullsized":"2024-09-21T05:57:17.927550+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:17.045500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929100+0000","last_change":"2024-09-21T05:57:15.919832+0000","last_active":"2024-09-21T05:57:17.929100+0000","last_peered":"2024-09-21T05:57:17.929100+0000","last_clean":"2024-09-21T05:57:17.929100+0000","last_became_active":"2024-09-21T05:57:15.919510+0000","last_became_peered":"2024-09-21T05:57:15.919510+0000","last_unstale":"2024-09-21T05:57:17.929100+0000","last_undegraded":"2024-09-21T05:57:17.929100+0000","last_fullsized":"2024-09-21T05:57:17.929100+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:16.765386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928938+0000","last_change":"2024-09-21T05:57:16.944659+0000","last_active":"2024-09-21T05:57:17.928938+0000","last_peered":"2024-09-21T05:57:17.928938+0000","last_clean":"2024-09-21T05:57:17.928938+0000","last_became_active":"2024-09-21T05:57:16.943952+0000","last_became_peered":"2024-09-21T05:57:16.943952+0000","last_unstale":"2024-09-21T05:57:17.928938+0000","last_undegraded":"2024-09-21T05:57:17.928938+0000","last_fullsized":"2024-09-21T05:57:17.928938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:17.045500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927623+0000","last_change":"2024-09-21T05:57:15.927479+0000","last_active":"2024-09-21T05:57:17.927623+0000","last_peered":"2024-09-21T05:57:17.927623+0000","last_clean":"2024-09-21T05:57:17.927623+0000","last_became_active":"2024-09-21T05:57:15.927178+0000","last_became_peered":"2024-09-21T05:57:15.927178+0000","last_unstale":"2024-09-21T05:57:17.927623+0000","last_undegraded":"2024-09-21T05:57:17.927623+0000","last_fullsized":"2024-09-21T05:57:17.927623+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:07.759407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.946829+0000","last_change":"2024-09-21T05:57:16.944107+0000","last_active":"2024-09-21T05:57:16.946829+0000","last_peered":"2024-09-21T05:57:16.946829+0000","last_clean":"2024-09-21T05:57:16.946829+0000","last_became_active":"2024-09-21T05:57:16.943762+0000","last_became_peered":"2024-09-21T05:57:16.943762+0000","last_unstale":"2024-09-21T05:57:16.946829+0000","last_undegraded":"2024-09-21T05:57:16.946829+0000","last_fullsized":"2024-09-21T05:57:16.946829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:28.339309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930159+0000","last_change":"2024-09-21T05:57:15.909732+0000","last_active":"2024-09-21T05:57:17.930159+0000","last_peered":"2024-09-21T05:57:17.930159+0000","last_clean":"2024-09-21T05:57:17.930159+0000","last_became_active":"2024-09-21T05:57:15.909380+0000","last_became_peered":"2024-09-21T05:57:15.909380+0000","last_unstale":"2024-09-21T05:57:17.930159+0000","last_undegraded":"2024-09-21T05:57:17.930159+0000","last_fullsized":"2024-09-21T05:57:17.930159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:59.310164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927553+0000","last_change":"2024-09-21T05:57:16.934731+0000","last_active":"2024-09-21T05:57:17.927553+0000","last_peered":"2024-09-21T05:57:17.927553+0000","last_clean":"2024-09-21T05:57:17.927553+0000","last_became_active":"2024-09-21T05:57:16.934215+0000","last_became_peered":"2024-09-21T05:57:16.934215+0000","last_unstale":"2024-09-21T05:57:17.927553+0000","last_undegraded":"2024-09-21T05:57:17.927553+0000","last_fullsized":"2024-09-21T05:57:17.927553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:24.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.929158+0000","last_change":"2024-09-21T05:57:15.908297+0000","last_active":"2024-09-21T05:57:16.929158+0000","last_peered":"2024-09-21T05:57:16.929158+0000","last_clean":"2024-09-21T05:57:16.929158+0000","last_became_active":"2024-09-21T05:57:15.907487+0000","last_became_peered":"2024-09-21T05:57:15.907487+0000","last_unstale":"2024-09-21T05:57:16.929158+0000","last_undegraded":"2024-09-21T05:57:16.929158+0000","last_fullsized":"2024-09-21T05:57:16.929158+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:55.280545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.939280+0000","last_change":"2024-09-21T05:57:16.935017+0000","last_active":"2024-09-21T05:57:16.939280+0000","last_peered":"2024-09-21T05:57:16.939280+0000","last_clean":"2024-09-21T05:57:16.939280+0000","last_became_active":"2024-09-21T05:57:16.932497+0000","last_became_peered":"2024-09-21T05:57:16.932497+0000","last_unstale":"2024-09-21T05:57:16.939280+0000","last_undegraded":"2024-09-21T05:57:16.939280+0000","last_fullsized":"2024-09-21T05:57:16.939280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:31.954378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927284+0000","last_change":"2024-09-21T05:57:15.926773+0000","last_active":"2024-09-21T05:57:17.927284+0000","last_peered":"2024-09-21T05:57:17.927284+0000","last_clean":"2024-09-21T05:57:17.927284+0000","last_became_active":"2024-09-21T05:57:15.925391+0000","last_became_peered":"2024-09-21T05:57:15.925391+0000","last_unstale":"2024-09-21T05:57:17.927284+0000","last_undegraded":"2024-09-21T05:57:17.927284+0000","last_fullsized":"2024-09-21T05:57:17.927284+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:17.463340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.941149+0000","last_change":"2024-09-21T05:57:16.934230+0000","last_active":"2024-09-21T05:57:16.941149+0000","last_peered":"2024-09-21T05:57:16.941149+0000","last_clean":"2024-09-21T05:57:16.941149+0000","last_became_active":"2024-09-21T05:57:16.933060+0000","last_became_peered":"2024-09-21T05:57:16.933060+0000","last_unstale":"2024-09-21T05:57:16.941149+0000","last_undegraded":"2024-09-21T05:57:16.941149+0000","last_fullsized":"2024-09-21T05:57:16.941149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:27:16.052630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927059+0000","last_change":"2024-09-21T05:57:15.923890+0000","last_active":"2024-09-21T05:57:17.927059+0000","last_peered":"2024-09-21T05:57:17.927059+0000","last_clean":"2024-09-21T05:57:17.927059+0000","last_became_active":"2024-09-21T05:57:15.923618+0000","last_became_peered":"2024-09-21T05:57:15.923618+0000","last_unstale":"2024-09-21T05:57:17.927059+0000","last_undegraded":"2024-09-21T05:57:17.927059+0000","last_fullsized":"2024-09-21T05:57:17.927059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:50.185505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930449+0000","last_change":"2024-09-21T05:57:16.917415+0000","last_active":"2024-09-21T05:57:17.930449+0000","last_peered":"2024-09-21T05:57:17.930449+0000","last_clean":"2024-09-21T05:57:17.930449+0000","last_became_active":"2024-09-21T05:57:16.917133+0000","last_became_peered":"2024-09-21T05:57:16.917133+0000","last_unstale":"2024-09-21T05:57:17.930449+0000","last_undegraded":"2024-09-21T05:57:17.930449+0000","last_fullsized":"2024-09-21T05:57:17.930449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:01.510305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.959725+0000","last_change":"2024-09-21T05:57:15.925125+0000","last_active":"2024-09-21T05:57:16.959725+0000","last_peered":"2024-09-21T05:57:16.959725+0000","last_clean":"2024-09-21T05:57:16.959725+0000","last_became_active":"2024-09-21T05:57:15.924760+0000","last_became_peered":"2024-09-21T05:57:15.924760+0000","last_unstale":"2024-09-21T05:57:16.959725+0000","last_undegraded":"2024-09-21T05:57:16.959725+0000","last_fullsized":"2024-09-21T05:57:16.959725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:13.041145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929558+0000","last_change":"2024-09-21T05:57:16.918473+0000","last_active":"2024-09-21T05:57:17.929558+0000","last_peered":"2024-09-21T05:57:17.929558+0000","last_clean":"2024-09-21T05:57:17.929558+0000","last_became_active":"2024-09-21T05:57:16.918141+0000","last_became_peered":"2024-09-21T05:57:16.918141+0000","last_unstale":"2024-09-21T05:57:17.929558+0000","last_undegraded":"2024-09-21T05:57:17.929558+0000","last_fullsized":"2024-09-21T05:57:17.929558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:51.231280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930274+0000","last_change":"2024-09-21T05:57:15.912772+0000","last_active":"2024-09-21T05:57:17.930274+0000","last_peered":"2024-09-21T05:57:17.930274+0000","last_clean":"2024-09-21T05:57:17.930274+0000","last_became_active":"2024-09-21T05:57:15.912439+0000","last_became_peered":"2024-09-21T05:57:15.912439+0000","last_unstale":"2024-09-21T05:57:17.930274+0000","last_undegraded":"2024-09-21T05:57:17.930274+0000","last_fullsized":"2024-09-21T05:57:17.930274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:05.879005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.949130+0000","last_change":"2024-09-21T05:57:16.946819+0000","last_active":"2024-09-21T05:57:16.949130+0000","last_peered":"2024-09-21T05:57:16.949130+0000","last_clean":"2024-09-21T05:57:16.949130+0000","last_became_active":"2024-09-21T05:57:16.946089+0000","last_became_peered":"2024-09-21T05:57:16.946089+0000","last_unstale":"2024-09-21T05:57:16.949130+0000","last_undegraded":"2024-09-21T05:57:16.949130+0000","last_fullsized":"2024-09-21T05:57:16.949130+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:14.707859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930326+0000","last_change":"2024-09-21T05:57:15.920592+0000","last_active":"2024-09-21T05:57:17.930326+0000","last_peered":"2024-09-21T05:57:17.930326+0000","last_clean":"2024-09-21T05:57:17.930326+0000","last_became_active":"2024-09-21T05:57:15.920223+0000","last_became_peered":"2024-09-21T05:57:15.920223+0000","last_unstale":"2024-09-21T05:57:17.930326+0000","last_undegraded":"2024-09-21T05:57:17.930326+0000","last_fullsized":"2024-09-21T05:57:17.930326+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:02.922491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.934667+0000","last_change":"2024-09-21T05:57:16.932868+0000","last_active":"2024-09-21T05:57:16.934667+0000","last_peered":"2024-09-21T05:57:16.934667+0000","last_clean":"2024-09-21T05:57:16.934667+0000","last_became_active":"2024-09-21T05:57:16.932529+0000","last_became_peered":"2024-09-21T05:57:16.932529+0000","last_unstale":"2024-09-21T05:57:16.934667+0000","last_undegraded":"2024-09-21T05:57:16.934667+0000","last_fullsized":"2024-09-21T05:57:16.934667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:08.086568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930232+0000","last_change":"2024-09-21T05:57:15.928375+0000","last_active":"2024-09-21T05:57:17.930232+0000","last_peered":"2024-09-21T05:57:17.930232+0000","last_clean":"2024-09-21T05:57:17.930232+0000","last_became_active":"2024-09-21T05:57:15.928053+0000","last_became_peered":"2024-09-21T05:57:15.928053+0000","last_unstale":"2024-09-21T05:57:17.930232+0000","last_undegraded":"2024-09-21T05:57:17.930232+0000","last_fullsized":"2024-09-21T05:57:17.930232+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:24.553890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.933292+0000","last_change":"2024-09-21T05:57:16.928785+0000","last_active":"2024-09-21T05:57:16.933292+0000","last_peered":"2024-09-21T05:57:16.933292+0000","last_clean":"2024-09-21T05:57:16.933292+0000","last_became_active":"2024-09-21T05:57:16.928453+0000","last_became_peered":"2024-09-21T05:57:16.928453+0000","last_unstale":"2024-09-21T05:57:16.933292+0000","last_undegraded":"2024-09-21T05:57:16.933292+0000","last_fullsized":"2024-09-21T05:57:16.933292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:35.911229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.955958+0000","last_change":"2024-09-21T05:57:15.912938+0000","last_active":"2024-09-21T05:57:16.955958+0000","last_peered":"2024-09-21T05:57:16.955958+0000","last_clean":"2024-09-21T05:57:16.955958+0000","last_became_active":"2024-09-21T05:57:15.912610+0000","last_became_peered":"2024-09-21T05:57:15.912610+0000","last_unstale":"2024-09-21T05:57:16.955958+0000","last_undegraded":"2024-09-21T05:57:16.955958+0000","last_fullsized":"2024-09-21T05:57:16.955958+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:12.785567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.933182+0000","last_change":"2024-09-21T05:57:16.925512+0000","last_active":"2024-09-21T05:57:16.933182+0000","last_peered":"2024-09-21T05:57:16.933182+0000","last_clean":"2024-09-21T05:57:16.933182+0000","last_became_active":"2024-09-21T05:57:16.925047+0000","last_became_peered":"2024-09-21T05:57:16.925047+0000","last_unstale":"2024-09-21T05:57:16.933182+0000","last_undegraded":"2024-09-21T05:57:16.933182+0000","last_fullsized":"2024-09-21T05:57:16.933182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:30.063373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.930909+0000","last_change":"2024-09-21T05:57:15.912332+0000","last_active":"2024-09-21T05:57:16.930909+0000","last_peered":"2024-09-21T05:57:16.930909+0000","last_clean":"2024-09-21T05:57:16.930909+0000","last_became_active":"2024-09-21T05:57:15.911414+0000","last_became_peered":"2024-09-21T05:57:15.911414+0000","last_unstale":"2024-09-21T05:57:16.930909+0000","last_undegraded":"2024-09-21T05:57:16.930909+0000","last_fullsized":"2024-09-21T05:57:16.930909+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:58.082495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.928718+0000","last_change":"2024-09-21T05:57:16.919184+0000","last_active":"2024-09-21T05:57:16.928718+0000","last_peered":"2024-09-21T05:57:16.928718+0000","last_clean":"2024-09-21T05:57:16.928718+0000","last_became_active":"2024-09-21T05:57:16.918738+0000","last_became_peered":"2024-09-21T05:57:16.918738+0000","last_unstale":"2024-09-21T05:57:16.928718+0000","last_undegraded":"2024-09-21T05:57:16.928718+0000","last_fullsized":"2024-09-21T05:57:16.928718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:20.678838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928896+0000","last_change":"2024-09-21T05:57:15.913419+0000","last_active":"2024-09-21T05:57:17.928896+0000","last_peered":"2024-09-21T05:57:17.928896+0000","last_clean":"2024-09-21T05:57:17.928896+0000","last_became_active":"2024-09-21T05:57:15.913056+0000","last_became_peered":"2024-09-21T05:57:15.913056+0000","last_unstale":"2024-09-21T05:57:17.928896+0000","last_undegraded":"2024-09-21T05:57:17.928896+0000","last_fullsized":"2024-09-21T05:57:17.928896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:11.273821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.940685+0000","last_change":"2024-09-21T05:57:16.938390+0000","last_active":"2024-09-21T05:57:16.940685+0000","last_peered":"2024-09-21T05:57:16.940685+0000","last_clean":"2024-09-21T05:57:16.940685+0000","last_became_active":"2024-09-21T05:57:16.938074+0000","last_became_peered":"2024-09-21T05:57:16.938074+0000","last_unstale":"2024-09-21T05:57:16.940685+0000","last_undegraded":"2024-09-21T05:57:16.940685+0000","last_fullsized":"2024-09-21T05:57:16.940685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:08.599070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927811+0000","last_change":"2024-09-21T05:57:15.919361+0000","last_active":"2024-09-21T05:57:17.927811+0000","last_peered":"2024-09-21T05:57:17.927811+0000","last_clean":"2024-09-21T05:57:17.927811+0000","last_became_active":"2024-09-21T05:57:15.919009+0000","last_became_peered":"2024-09-21T05:57:15.919009+0000","last_unstale":"2024-09-21T05:57:17.927811+0000","last_undegraded":"2024-09-21T05:57:17.927811+0000","last_fullsized":"2024-09-21T05:57:17.927811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:42.914075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.925589+0000","last_change":"2024-09-21T05:57:16.921956+0000","last_active":"2024-09-21T05:57:16.925589+0000","last_peered":"2024-09-21T05:57:16.925589+0000","last_clean":"2024-09-21T05:57:16.925589+0000","last_became_active":"2024-09-21T05:57:16.921631+0000","last_became_peered":"2024-09-21T05:57:16.921631+0000","last_unstale":"2024-09-21T05:57:16.925589+0000","last_undegraded":"2024-09-21T05:57:16.925589+0000","last_fullsized":"2024-09-21T05:57:16.925589+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:22.936196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.953503+0000","last_change":"2024-09-21T05:57:15.913998+0000","last_active":"2024-09-21T05:57:16.953503+0000","last_peered":"2024-09-21T05:57:16.953503+0000","last_clean":"2024-09-21T05:57:16.953503+0000","last_became_active":"2024-09-21T05:57:15.912647+0000","last_became_peered":"2024-09-21T05:57:15.912647+0000","last_unstale":"2024-09-21T05:57:16.953503+0000","last_undegraded":"2024-09-21T05:57:16.953503+0000","last_fullsized":"2024-09-21T05:57:16.953503+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:24.588116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929572+0000","last_change":"2024-09-21T05:57:16.946973+0000","last_active":"2024-09-21T05:57:17.929572+0000","last_peered":"2024-09-21T05:57:17.929572+0000","last_clean":"2024-09-21T05:57:17.929572+0000","last_became_active":"2024-09-21T05:57:16.946617+0000","last_became_peered":"2024-09-21T05:57:16.946617+0000","last_unstale":"2024-09-21T05:57:17.929572+0000","last_undegraded":"2024-09-21T05:57:17.929572+0000","last_fullsized":"2024-09-21T05:57:17.929572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:36.304709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.909161+0000","last_change":"2024-09-21T05:57:15.909184+0000","last_active":"2024-09-21T05:57:16.909161+0000","last_peered":"2024-09-21T05:57:16.909161+0000","last_clean":"2024-09-21T05:57:16.909161+0000","last_became_active":"2024-09-21T05:57:15.908859+0000","last_became_peered":"2024-09-21T05:57:15.908859+0000","last_unstale":"2024-09-21T05:57:16.909161+0000","last_undegraded":"2024-09-21T05:57:16.909161+0000","last_fullsized":"2024-09-21T05:57:16.909161+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:22.954976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.940286+0000","last_change":"2024-09-21T05:57:16.938486+0000","last_active":"2024-09-21T05:57:16.940286+0000","last_peered":"2024-09-21T05:57:16.940286+0000","last_clean":"2024-09-21T05:57:16.940286+0000","last_became_active":"2024-09-21T05:57:16.938159+0000","last_became_peered":"2024-09-21T05:57:16.938159+0000","last_unstale":"2024-09-21T05:57:16.940286+0000","last_undegraded":"2024-09-21T05:57:16.940286+0000","last_fullsized":"2024-09-21T05:57:16.940286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:14:42.049159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.928369+0000","last_change":"2024-09-21T05:57:15.921703+0000","last_active":"2024-09-21T05:57:16.928369+0000","last_peered":"2024-09-21T05:57:16.928369+0000","last_clean":"2024-09-21T05:57:16.928369+0000","last_became_active":"2024-09-21T05:57:15.921323+0000","last_became_peered":"2024-09-21T05:57:15.921323+0000","last_unstale":"2024-09-21T05:57:16.928369+0000","last_undegraded":"2024-09-21T05:57:16.928369+0000","last_fullsized":"2024-09-21T05:57:16.928369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:53.815717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.951665+0000","last_change":"2024-09-21T05:57:16.948625+0000","last_active":"2024-09-21T05:57:16.951665+0000","last_peered":"2024-09-21T05:57:16.951665+0000","last_clean":"2024-09-21T05:57:16.951665+0000","last_became_active":"2024-09-21T05:57:16.948321+0000","last_became_peered":"2024-09-21T05:57:16.948321+0000","last_unstale":"2024-09-21T05:57:16.951665+0000","last_undegraded":"2024-09-21T05:57:16.951665+0000","last_fullsized":"2024-09-21T05:57:16.951665+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:44.012450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.908710+0000","last_change":"2024-09-21T05:57:15.913517+0000","last_active":"2024-09-21T05:57:16.908710+0000","last_peered":"2024-09-21T05:57:16.908710+0000","last_clean":"2024-09-21T05:57:16.908710+0000","last_became_active":"2024-09-21T05:57:15.912035+0000","last_became_peered":"2024-09-21T05:57:15.912035+0000","last_unstale":"2024-09-21T05:57:16.908710+0000","last_undegraded":"2024-09-21T05:57:16.908710+0000","last_fullsized":"2024-09-21T05:57:16.908710+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:43.575259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.955904+0000","last_change":"2024-09-21T05:57:16.951076+0000","last_active":"2024-09-21T05:57:16.955904+0000","last_peered":"2024-09-21T05:57:16.955904+0000","last_clean":"2024-09-21T05:57:16.955904+0000","last_became_active":"2024-09-21T05:57:16.950589+0000","last_became_peered":"2024-09-21T05:57:16.950589+0000","last_unstale":"2024-09-21T05:57:16.955904+0000","last_undegraded":"2024-09-21T05:57:16.955904+0000","last_fullsized":"2024-09-21T05:57:16.955904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:03.568849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.929521+0000","last_change":"2024-09-21T05:57:15.914840+0000","last_active":"2024-09-21T05:57:16.929521+0000","last_peered":"2024-09-21T05:57:16.929521+0000","last_clean":"2024-09-21T05:57:16.929521+0000","last_became_active":"2024-09-21T05:57:15.912953+0000","last_became_peered":"2024-09-21T05:57:15.912953+0000","last_unstale":"2024-09-21T05:57:16.929521+0000","last_undegraded":"2024-09-21T05:57:16.929521+0000","last_fullsized":"2024-09-21T05:57:16.929521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:45.073451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927087+0000","last_change":"2024-09-21T05:57:16.943627+0000","last_active":"2024-09-21T05:57:17.927087+0000","last_peered":"2024-09-21T05:57:17.927087+0000","last_clean":"2024-09-21T05:57:17.927087+0000","last_became_active":"2024-09-21T05:57:16.942858+0000","last_became_peered":"2024-09-21T05:57:16.942858+0000","last_unstale":"2024-09-21T05:57:17.927087+0000","last_undegraded":"2024-09-21T05:57:17.927087+0000","last_fullsized":"2024-09-21T05:57:17.927087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:12.234166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929146+0000","last_change":"2024-09-21T05:57:15.914024+0000","last_active":"2024-09-21T05:57:17.929146+0000","last_peered":"2024-09-21T05:57:17.929146+0000","last_clean":"2024-09-21T05:57:17.929146+0000","last_became_active":"2024-09-21T05:57:15.913698+0000","last_became_peered":"2024-09-21T05:57:15.913698+0000","last_unstale":"2024-09-21T05:57:17.929146+0000","last_undegraded":"2024-09-21T05:57:17.929146+0000","last_fullsized":"2024-09-21T05:57:17.929146+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:27:15.049834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928709+0000","last_change":"2024-09-21T05:57:16.941067+0000","last_active":"2024-09-21T05:57:17.928709+0000","last_peered":"2024-09-21T05:57:17.928709+0000","last_clean":"2024-09-21T05:57:17.928709+0000","last_became_active":"2024-09-21T05:57:16.940701+0000","last_became_peered":"2024-09-21T05:57:16.940701+0000","last_unstale":"2024-09-21T05:57:17.928709+0000","last_undegraded":"2024-09-21T05:57:17.928709+0000","last_fullsized":"2024-09-21T05:57:17.928709+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:08.762203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927189+0000","last_change":"2024-09-21T05:57:15.927747+0000","last_active":"2024-09-21T05:57:17.927189+0000","last_peered":"2024-09-21T05:57:17.927189+0000","last_clean":"2024-09-21T05:57:17.927189+0000","last_became_active":"2024-09-21T05:57:15.925820+0000","last_became_peered":"2024-09-21T05:57:15.925820+0000","last_unstale":"2024-09-21T05:57:17.927189+0000","last_undegraded":"2024-09-21T05:57:17.927189+0000","last_fullsized":"2024-09-21T05:57:17.927189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:30.951582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.939595+0000","last_change":"2024-09-21T05:57:16.934655+0000","last_active":"2024-09-21T05:57:16.939595+0000","last_peered":"2024-09-21T05:57:16.939595+0000","last_clean":"2024-09-21T05:57:16.939595+0000","last_became_active":"2024-09-21T05:57:16.934349+0000","last_became_peered":"2024-09-21T05:57:16.934349+0000","last_unstale":"2024-09-21T05:57:16.939595+0000","last_undegraded":"2024-09-21T05:57:16.939595+0000","last_fullsized":"2024-09-21T05:57:16.939595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:51.188301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.934147+0000","last_change":"2024-09-21T05:57:15.916369+0000","last_active":"2024-09-21T05:57:16.934147+0000","last_peered":"2024-09-21T05:57:16.934147+0000","last_clean":"2024-09-21T05:57:16.934147+0000","last_became_active":"2024-09-21T05:57:15.915996+0000","last_became_peered":"2024-09-21T05:57:15.915996+0000","last_unstale":"2024-09-21T05:57:16.934147+0000","last_undegraded":"2024-09-21T05:57:16.934147+0000","last_fullsized":"2024-09-21T05:57:16.934147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:59.310164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":129,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.920139+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:57:16.920139+0000","last_peered":"2024-09-21T05:57:16.920139+0000","last_clean":"2024-09-21T05:57:16.920139+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:57:16.920139+0000","last_undegraded":"2024-09-21T05:57:16.920139+0000","last_fullsized":"2024-09-21T05:57:16.920139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":950272,"data_stored":723634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":133792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":372736,"data_stored":140148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":372736,"data_stored":139580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":368640,"data_stored":134810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31540,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":356352,"data_stored":133246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":723544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":133176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:57:21.138 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:57:21.252 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-09-21T05:57:21.246+0000 7f4532e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:57:21.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.326+0000 7fd6bfa00640 1 -- 172.21.15.50:0/3670734364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b80b5c70 msgr2=0x7fd6b815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:21.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.326+0000 7fd6bfa00640 1 --2- 172.21.15.50:0/3670734364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b80b5c70 0x7fd6b815bb60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80cca90 tx=0x7fd6ac01c740 comp rx=0 tx=0).stop 2024-09-21T05:57:21.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.326+0000 7fd6bfa00640 1 -- 172.21.15.50:0/3670734364 shutdown_connections 2024-09-21T05:57:21.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.326+0000 7fd6bfa00640 1 --2- 172.21.15.50:0/3670734364 >> v2:172.21.15.88:3300/0 conn(0x7fd6b80b4de0 0x7fd6b80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.326+0000 7fd6bfa00640 1 --2- 172.21.15.50:0/3670734364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b80b5c70 0x7fd6b815bb60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.326+0000 7fd6bfa00640 1 -- 172.21.15.50:0/3670734364 >> 172.21.15.50:0/3670734364 conn(0x7fd6b80b2040 msgr2=0x7fd6b80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:21.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.326+0000 7fd6bfa00640 1 -- 172.21.15.50:0/3670734364 shutdown_connections 2024-09-21T05:57:21.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.326+0000 7fd6bfa00640 1 -- 172.21.15.50:0/3670734364 wait complete. 2024-09-21T05:57:21.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.326+0000 7fd6bfa00640 1 Processor -- start 2024-09-21T05:57:21.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bfa00640 1 -- start start 2024-09-21T05:57:21.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bfa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b80b4de0 0x7fd6b81f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:21.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bfa00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b80b5c70 0x7fd6b815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bfa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b81527b0 0x7fd6b815ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bfa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8168970 con 0x7fd6b80b4de0 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bfa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81687f0 con 0x7fd6b81527b0 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bfa00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b8168af0 con 0x7fd6b80b5c70 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b80b4de0 0x7fd6b81f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b80b4de0 0x7fd6b81f8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58908/0 (socket says 172.21.15.50:58908) 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bd000640 1 -- 172.21.15.50:0/577605337 learned_addr learned my addr 172.21.15.50:0/577605337 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bda00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b81527b0 0x7fd6b815ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bd000640 1 -- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b80b5c70 msgr2=0x7fd6b815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6b7e00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b80b5c70 0x7fd6b815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bd000640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b80b5c70 0x7fd6b815a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bd000640 1 -- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b81527b0 msgr2=0x7fd6b815ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bd000640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b81527b0 0x7fd6b815ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bd000640 1 -- 172.21.15.50:0/577605337 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b81f93f0 con 0x7fd6b80b4de0 2024-09-21T05:57:21.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bda00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b81527b0 0x7fd6b815ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:21.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6b7e00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b80b5c70 0x7fd6b815a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:21.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bd000640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b80b4de0 0x7fd6b81f8eb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac000c10 tx=0x7fd6ac0a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:21.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6b4c00640 1 -- 172.21.15.50:0/577605337 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac0bb070 con 0x7fd6b80b4de0 2024-09-21T05:57:21.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6b4c00640 1 -- 172.21.15.50:0/577605337 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6ac005030 con 0x7fd6b80b4de0 2024-09-21T05:57:21.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6b4c00640 1 -- 172.21.15.50:0/577605337 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac0b2040 con 0x7fd6b80b4de0 2024-09-21T05:57:21.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bfa00640 1 -- 172.21.15.50:0/577605337 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b8157210 con 0x7fd6b80b4de0 2024-09-21T05:57:21.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.330+0000 7fd6bfa00640 1 -- 172.21.15.50:0/577605337 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b80af300 con 0x7fd6b80b4de0 2024-09-21T05:57:21.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.334+0000 7fd6bfa00640 1 -- 172.21.15.50:0/577605337 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b80b51f0 con 0x7fd6b80b4de0 2024-09-21T05:57:21.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.334+0000 7fd6b4c00640 1 -- 172.21.15.50:0/577605337 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd6ac0be070 con 0x7fd6b80b4de0 2024-09-21T05:57:21.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.334+0000 7fd6b4c00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd698073230 0x7fd698075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:21.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.334+0000 7fd6b4c00640 1 -- 172.21.15.50:0/577605337 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd6ac12de00 con 0x7fd6b80b4de0 2024-09-21T05:57:21.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.334+0000 7fd6b7e00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd698073230 0x7fd698075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:21.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.334+0000 7fd6b7e00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd698073230 0x7fd698075740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd6b81662d0 tx=0x7fd6a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:21.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.338+0000 7fd6b4c00640 1 -- 172.21.15.50:0/577605337 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd6b81f93f0 con 0x7fd6b80b4de0 2024-09-21T05:57:21.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.514+0000 7fd6bfa00640 1 -- 172.21.15.50:0/577605337 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6b806b540 con 0x7fd698073230 2024-09-21T05:57:21.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.518+0000 7fd6b4c00640 1 -- 172.21.15.50:0/577605337 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7fd6a40050d0 con 0x7fd698073230 2024-09-21T05:57:21.525 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:21.527 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:57:21.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.522+0000 7fd6bfa00640 1 -- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd698073230 msgr2=0x7fd698075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:21.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.522+0000 7fd6bfa00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd698073230 0x7fd698075740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd6b81662d0 tx=0x7fd6a4009040 comp rx=0 tx=0).stop 2024-09-21T05:57:21.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.522+0000 7fd6bfa00640 1 -- 172.21.15.50:0/577605337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b80b4de0 msgr2=0x7fd6b81f8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:21.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.522+0000 7fd6bfa00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b80b4de0 0x7fd6b81f8eb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac000c10 tx=0x7fd6ac0a5710 comp rx=0 tx=0).stop 2024-09-21T05:57:21.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.522+0000 7fd6bfa00640 1 -- 172.21.15.50:0/577605337 shutdown_connections 2024-09-21T05:57:21.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.522+0000 7fd6bfa00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b81527b0 0x7fd6b815ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.522+0000 7fd6bfa00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b80b5c70 0x7fd6b815a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.522+0000 7fd6bfa00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd698073230 0x7fd698075740 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.522+0000 7fd6bfa00640 1 --2- 172.21.15.50:0/577605337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b80b4de0 0x7fd6b81f8eb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.522+0000 7fd6bfa00640 1 -- 172.21.15.50:0/577605337 >> 172.21.15.50:0/577605337 conn(0x7fd6b80b2040 msgr2=0x7fd6b81500a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:21.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.522+0000 7fd6bfa00640 1 -- 172.21.15.50:0/577605337 shutdown_connections 2024-09-21T05:57:21.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.522+0000 7fd6bfa00640 1 -- 172.21.15.50:0/577605337 wait complete. 2024-09-21T05:57:21.547 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-21T05:57:20.896506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236868,"kb_used_data":3984,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737852,"statfs":{"total":773094113280,"available":772851560448,"internally_reserved":0,"allocated":4079616,"data_stored":2261930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001594"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.945107+0000","last_change":"2024-09-21T05:57:16.940234+0000","last_active":"2024-09-21T05:57:16.945107+0000","last_peered":"2024-09-21T05:57:16.945107+0000","last_clean":"2024-09-21T05:57:16.945107+0000","last_became_active":"2024-09-21T05:57:16.939897+0000","last_became_peered":"2024-09-21T05:57:16.939897+0000","last_unstale":"2024-09-21T05:57:16.945107+0000","last_undegraded":"2024-09-21T05:57:16.945107+0000","last_fullsized":"2024-09-21T05:57:16.945107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:13.788363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927762+0000","last_change":"2024-09-21T05:57:16.943546+0000","last_active":"2024-09-21T05:57:17.927762+0000","last_peered":"2024-09-21T05:57:17.927762+0000","last_clean":"2024-09-21T05:57:17.927762+0000","last_became_active":"2024-09-21T05:57:16.941470+0000","last_became_peered":"2024-09-21T05:57:16.941470+0000","last_unstale":"2024-09-21T05:57:17.927762+0000","last_undegraded":"2024-09-21T05:57:17.927762+0000","last_fullsized":"2024-09-21T05:57:17.927762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.164703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928920+0000","last_change":"2024-09-21T05:57:16.944278+0000","last_active":"2024-09-21T05:57:17.928920+0000","last_peered":"2024-09-21T05:57:17.928920+0000","last_clean":"2024-09-21T05:57:17.928920+0000","last_became_active":"2024-09-21T05:57:16.943563+0000","last_became_peered":"2024-09-21T05:57:16.943563+0000","last_unstale":"2024-09-21T05:57:17.928920+0000","last_undegraded":"2024-09-21T05:57:17.928920+0000","last_fullsized":"2024-09-21T05:57:17.928920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:24.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.949159+0000","last_change":"2024-09-21T05:57:16.947157+0000","last_active":"2024-09-21T05:57:16.949159+0000","last_peered":"2024-09-21T05:57:16.949159+0000","last_clean":"2024-09-21T05:57:16.949159+0000","last_became_active":"2024-09-21T05:57:16.946490+0000","last_became_peered":"2024-09-21T05:57:16.946490+0000","last_unstale":"2024-09-21T05:57:16.949159+0000","last_undegraded":"2024-09-21T05:57:16.949159+0000","last_fullsized":"2024-09-21T05:57:16.949159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:45.056061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.944661+0000","last_change":"2024-09-21T05:57:16.937915+0000","last_active":"2024-09-21T05:57:16.944661+0000","last_peered":"2024-09-21T05:57:16.944661+0000","last_clean":"2024-09-21T05:57:16.944661+0000","last_became_active":"2024-09-21T05:57:16.937607+0000","last_became_peered":"2024-09-21T05:57:16.937607+0000","last_unstale":"2024-09-21T05:57:16.944661+0000","last_undegraded":"2024-09-21T05:57:16.944661+0000","last_fullsized":"2024-09-21T05:57:16.944661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:30.785047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927182+0000","last_change":"2024-09-21T05:57:16.950111+0000","last_active":"2024-09-21T05:57:17.927182+0000","last_peered":"2024-09-21T05:57:17.927182+0000","last_clean":"2024-09-21T05:57:17.927182+0000","last_became_active":"2024-09-21T05:57:16.945063+0000","last_became_peered":"2024-09-21T05:57:16.945063+0000","last_unstale":"2024-09-21T05:57:17.927182+0000","last_undegraded":"2024-09-21T05:57:17.927182+0000","last_fullsized":"2024-09-21T05:57:17.927182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:40.125474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.940660+0000","last_change":"2024-09-21T05:57:16.937457+0000","last_active":"2024-09-21T05:57:16.940660+0000","last_peered":"2024-09-21T05:57:16.940660+0000","last_clean":"2024-09-21T05:57:16.940660+0000","last_became_active":"2024-09-21T05:57:16.937121+0000","last_became_peered":"2024-09-21T05:57:16.937121+0000","last_unstale":"2024-09-21T05:57:16.940660+0000","last_undegraded":"2024-09-21T05:57:16.940660+0000","last_fullsized":"2024-09-21T05:57:16.940660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:09.248511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.939399+0000","last_change":"2024-09-21T05:57:16.936161+0000","last_active":"2024-09-21T05:57:16.939399+0000","last_peered":"2024-09-21T05:57:16.939399+0000","last_clean":"2024-09-21T05:57:16.939399+0000","last_became_active":"2024-09-21T05:57:16.935741+0000","last_became_peered":"2024-09-21T05:57:16.935741+0000","last_unstale":"2024-09-21T05:57:16.939399+0000","last_undegraded":"2024-09-21T05:57:16.939399+0000","last_fullsized":"2024-09-21T05:57:16.939399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:41.787430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930695+0000","last_change":"2024-09-21T05:57:16.944136+0000","last_active":"2024-09-21T05:57:17.930695+0000","last_peered":"2024-09-21T05:57:17.930695+0000","last_clean":"2024-09-21T05:57:17.930695+0000","last_became_active":"2024-09-21T05:57:16.943762+0000","last_became_peered":"2024-09-21T05:57:16.943762+0000","last_unstale":"2024-09-21T05:57:17.930695+0000","last_undegraded":"2024-09-21T05:57:17.930695+0000","last_fullsized":"2024-09-21T05:57:17.930695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:09.580318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.954693+0000","last_change":"2024-09-21T05:57:16.953289+0000","last_active":"2024-09-21T05:57:16.954693+0000","last_peered":"2024-09-21T05:57:16.954693+0000","last_clean":"2024-09-21T05:57:16.954693+0000","last_became_active":"2024-09-21T05:57:16.952949+0000","last_became_peered":"2024-09-21T05:57:16.952949+0000","last_unstale":"2024-09-21T05:57:16.954693+0000","last_undegraded":"2024-09-21T05:57:16.954693+0000","last_fullsized":"2024-09-21T05:57:16.954693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:04.044328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.958308+0000","last_change":"2024-09-21T05:57:16.952792+0000","last_active":"2024-09-21T05:57:16.958308+0000","last_peered":"2024-09-21T05:57:16.958308+0000","last_clean":"2024-09-21T05:57:16.958308+0000","last_became_active":"2024-09-21T05:57:16.952468+0000","last_became_peered":"2024-09-21T05:57:16.952468+0000","last_unstale":"2024-09-21T05:57:16.958308+0000","last_undegraded":"2024-09-21T05:57:16.958308+0000","last_fullsized":"2024-09-21T05:57:16.958308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:37.101273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928545+0000","last_change":"2024-09-21T05:57:16.943251+0000","last_active":"2024-09-21T05:57:17.928545+0000","last_peered":"2024-09-21T05:57:17.928545+0000","last_clean":"2024-09-21T05:57:17.928545+0000","last_became_active":"2024-09-21T05:57:16.941982+0000","last_became_peered":"2024-09-21T05:57:16.941982+0000","last_unstale":"2024-09-21T05:57:17.928545+0000","last_undegraded":"2024-09-21T05:57:17.928545+0000","last_fullsized":"2024-09-21T05:57:17.928545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:40.125474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.955663+0000","last_change":"2024-09-21T05:57:16.949896+0000","last_active":"2024-09-21T05:57:16.955663+0000","last_peered":"2024-09-21T05:57:16.955663+0000","last_clean":"2024-09-21T05:57:16.955663+0000","last_became_active":"2024-09-21T05:57:16.949651+0000","last_became_peered":"2024-09-21T05:57:16.949651+0000","last_unstale":"2024-09-21T05:57:16.955663+0000","last_undegraded":"2024-09-21T05:57:16.955663+0000","last_fullsized":"2024-09-21T05:57:16.955663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:19.925729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927353+0000","last_change":"2024-09-21T05:57:16.951013+0000","last_active":"2024-09-21T05:57:17.927353+0000","last_peered":"2024-09-21T05:57:17.927353+0000","last_clean":"2024-09-21T05:57:17.927353+0000","last_became_active":"2024-09-21T05:57:16.945539+0000","last_became_peered":"2024-09-21T05:57:16.945539+0000","last_unstale":"2024-09-21T05:57:17.927353+0000","last_undegraded":"2024-09-21T05:57:17.927353+0000","last_fullsized":"2024-09-21T05:57:17.927353+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:31.707738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927822+0000","last_change":"2024-09-21T05:57:16.938984+0000","last_active":"2024-09-21T05:57:17.927822+0000","last_peered":"2024-09-21T05:57:17.927822+0000","last_clean":"2024-09-21T05:57:17.927822+0000","last_became_active":"2024-09-21T05:57:16.938621+0000","last_became_peered":"2024-09-21T05:57:16.938621+0000","last_unstale":"2024-09-21T05:57:17.927822+0000","last_undegraded":"2024-09-21T05:57:17.927822+0000","last_fullsized":"2024-09-21T05:57:17.927822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:27.248981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.945437+0000","last_change":"2024-09-21T05:57:16.944128+0000","last_active":"2024-09-21T05:57:16.945437+0000","last_peered":"2024-09-21T05:57:16.945437+0000","last_clean":"2024-09-21T05:57:16.945437+0000","last_became_active":"2024-09-21T05:57:16.940434+0000","last_became_peered":"2024-09-21T05:57:16.940434+0000","last_unstale":"2024-09-21T05:57:16.945437+0000","last_undegraded":"2024-09-21T05:57:16.945437+0000","last_fullsized":"2024-09-21T05:57:16.945437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:52.641317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.950436+0000","last_change":"2024-09-21T05:57:16.944547+0000","last_active":"2024-09-21T05:57:16.950436+0000","last_peered":"2024-09-21T05:57:16.950436+0000","last_clean":"2024-09-21T05:57:16.950436+0000","last_became_active":"2024-09-21T05:57:16.944248+0000","last_became_peered":"2024-09-21T05:57:16.944248+0000","last_unstale":"2024-09-21T05:57:16.950436+0000","last_undegraded":"2024-09-21T05:57:16.950436+0000","last_fullsized":"2024-09-21T05:57:16.950436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:09.580318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927806+0000","last_change":"2024-09-21T05:57:16.956746+0000","last_active":"2024-09-21T05:57:17.927806+0000","last_peered":"2024-09-21T05:57:17.927806+0000","last_clean":"2024-09-21T05:57:17.927806+0000","last_became_active":"2024-09-21T05:57:16.956448+0000","last_became_peered":"2024-09-21T05:57:16.956448+0000","last_unstale":"2024-09-21T05:57:17.927806+0000","last_undegraded":"2024-09-21T05:57:17.927806+0000","last_fullsized":"2024-09-21T05:57:17.927806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:36.304709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927778+0000","last_change":"2024-09-21T05:57:16.955706+0000","last_active":"2024-09-21T05:57:17.927778+0000","last_peered":"2024-09-21T05:57:17.927778+0000","last_clean":"2024-09-21T05:57:17.927778+0000","last_became_active":"2024-09-21T05:57:16.955379+0000","last_became_peered":"2024-09-21T05:57:16.955379+0000","last_unstale":"2024-09-21T05:57:17.927778+0000","last_undegraded":"2024-09-21T05:57:17.927778+0000","last_fullsized":"2024-09-21T05:57:17.927778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:51.231280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.931634+0000","last_change":"2024-09-21T05:57:16.921955+0000","last_active":"2024-09-21T05:57:16.931634+0000","last_peered":"2024-09-21T05:57:16.931634+0000","last_clean":"2024-09-21T05:57:16.931634+0000","last_became_active":"2024-09-21T05:57:16.921593+0000","last_became_peered":"2024-09-21T05:57:16.921593+0000","last_unstale":"2024-09-21T05:57:16.931634+0000","last_undegraded":"2024-09-21T05:57:16.931634+0000","last_fullsized":"2024-09-21T05:57:16.931634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:08.460664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929755+0000","last_change":"2024-09-21T05:57:16.923990+0000","last_active":"2024-09-21T05:57:17.929755+0000","last_peered":"2024-09-21T05:57:17.929755+0000","last_clean":"2024-09-21T05:57:17.929755+0000","last_became_active":"2024-09-21T05:57:16.922933+0000","last_became_peered":"2024-09-21T05:57:16.922933+0000","last_unstale":"2024-09-21T05:57:17.929755+0000","last_undegraded":"2024-09-21T05:57:17.929755+0000","last_fullsized":"2024-09-21T05:57:17.929755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:41.787430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929784+0000","last_change":"2024-09-21T05:57:16.950653+0000","last_active":"2024-09-21T05:57:17.929784+0000","last_peered":"2024-09-21T05:57:17.929784+0000","last_clean":"2024-09-21T05:57:17.929784+0000","last_became_active":"2024-09-21T05:57:16.950231+0000","last_became_peered":"2024-09-21T05:57:16.950231+0000","last_unstale":"2024-09-21T05:57:17.929784+0000","last_undegraded":"2024-09-21T05:57:17.929784+0000","last_fullsized":"2024-09-21T05:57:17.929784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:09.248511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.928919+0000","last_change":"2024-09-21T05:57:16.924101+0000","last_active":"2024-09-21T05:57:16.928919+0000","last_peered":"2024-09-21T05:57:16.928919+0000","last_clean":"2024-09-21T05:57:16.928919+0000","last_became_active":"2024-09-21T05:57:16.923492+0000","last_became_peered":"2024-09-21T05:57:16.923492+0000","last_unstale":"2024-09-21T05:57:16.928919+0000","last_undegraded":"2024-09-21T05:57:16.928919+0000","last_fullsized":"2024-09-21T05:57:16.928919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:29.785298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927688+0000","last_change":"2024-09-21T05:57:16.929813+0000","last_active":"2024-09-21T05:57:17.927688+0000","last_peered":"2024-09-21T05:57:17.927688+0000","last_clean":"2024-09-21T05:57:17.927688+0000","last_became_active":"2024-09-21T05:57:16.929465+0000","last_became_peered":"2024-09-21T05:57:16.929465+0000","last_unstale":"2024-09-21T05:57:17.927688+0000","last_undegraded":"2024-09-21T05:57:17.927688+0000","last_fullsized":"2024-09-21T05:57:17.927688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:53.736157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.945984+0000","last_change":"2024-09-21T05:57:16.943530+0000","last_active":"2024-09-21T05:57:16.945984+0000","last_peered":"2024-09-21T05:57:16.945984+0000","last_clean":"2024-09-21T05:57:16.945984+0000","last_became_active":"2024-09-21T05:57:16.943189+0000","last_became_peered":"2024-09-21T05:57:16.943189+0000","last_unstale":"2024-09-21T05:57:16.945984+0000","last_undegraded":"2024-09-21T05:57:16.945984+0000","last_fullsized":"2024-09-21T05:57:16.945984+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:58.234837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927990+0000","last_change":"2024-09-21T05:57:16.943892+0000","last_active":"2024-09-21T05:57:17.927990+0000","last_peered":"2024-09-21T05:57:17.927990+0000","last_clean":"2024-09-21T05:57:17.927990+0000","last_became_active":"2024-09-21T05:57:16.942897+0000","last_became_peered":"2024-09-21T05:57:16.942897+0000","last_unstale":"2024-09-21T05:57:17.927990+0000","last_undegraded":"2024-09-21T05:57:17.927990+0000","last_fullsized":"2024-09-21T05:57:17.927990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:07.255500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.952663+0000","last_change":"2024-09-21T05:57:16.946812+0000","last_active":"2024-09-21T05:57:16.952663+0000","last_peered":"2024-09-21T05:57:16.952663+0000","last_clean":"2024-09-21T05:57:16.952663+0000","last_became_active":"2024-09-21T05:57:16.946462+0000","last_became_peered":"2024-09-21T05:57:16.946462+0000","last_unstale":"2024-09-21T05:57:16.952663+0000","last_undegraded":"2024-09-21T05:57:16.952663+0000","last_fullsized":"2024-09-21T05:57:16.952663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:37.284888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928566+0000","last_change":"2024-09-21T05:57:16.918874+0000","last_active":"2024-09-21T05:57:17.928566+0000","last_peered":"2024-09-21T05:57:17.928566+0000","last_clean":"2024-09-21T05:57:17.928566+0000","last_became_active":"2024-09-21T05:57:16.918524+0000","last_became_peered":"2024-09-21T05:57:16.918524+0000","last_unstale":"2024-09-21T05:57:17.928566+0000","last_undegraded":"2024-09-21T05:57:17.928566+0000","last_fullsized":"2024-09-21T05:57:17.928566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:31.707738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930137+0000","last_change":"2024-09-21T05:57:16.941517+0000","last_active":"2024-09-21T05:57:17.930137+0000","last_peered":"2024-09-21T05:57:17.930137+0000","last_clean":"2024-09-21T05:57:17.930137+0000","last_became_active":"2024-09-21T05:57:16.941208+0000","last_became_peered":"2024-09-21T05:57:16.941208+0000","last_unstale":"2024-09-21T05:57:17.930137+0000","last_undegraded":"2024-09-21T05:57:17.930137+0000","last_fullsized":"2024-09-21T05:57:17.930137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:39.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.940411+0000","last_change":"2024-09-21T05:57:16.937583+0000","last_active":"2024-09-21T05:57:16.940411+0000","last_peered":"2024-09-21T05:57:16.940411+0000","last_clean":"2024-09-21T05:57:16.940411+0000","last_became_active":"2024-09-21T05:57:16.936955+0000","last_became_peered":"2024-09-21T05:57:16.936955+0000","last_unstale":"2024-09-21T05:57:16.940411+0000","last_undegraded":"2024-09-21T05:57:16.940411+0000","last_fullsized":"2024-09-21T05:57:16.940411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:39.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.945716+0000","last_change":"2024-09-21T05:57:16.940868+0000","last_active":"2024-09-21T05:57:16.945716+0000","last_peered":"2024-09-21T05:57:16.945716+0000","last_clean":"2024-09-21T05:57:16.945716+0000","last_became_active":"2024-09-21T05:57:16.939925+0000","last_became_peered":"2024-09-21T05:57:16.939925+0000","last_unstale":"2024-09-21T05:57:16.945716+0000","last_undegraded":"2024-09-21T05:57:16.945716+0000","last_fullsized":"2024-09-21T05:57:16.945716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:29.785298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.939371+0000","last_change":"2024-09-21T05:57:16.935337+0000","last_active":"2024-09-21T05:57:16.939371+0000","last_peered":"2024-09-21T05:57:16.939371+0000","last_clean":"2024-09-21T05:57:16.939371+0000","last_became_active":"2024-09-21T05:57:16.932030+0000","last_became_peered":"2024-09-21T05:57:16.932030+0000","last_unstale":"2024-09-21T05:57:16.939371+0000","last_undegraded":"2024-09-21T05:57:16.939371+0000","last_fullsized":"2024-09-21T05:57:16.939371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:08.762203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.950197+0000","last_change":"2024-09-21T05:57:16.933797+0000","last_active":"2024-09-21T05:57:16.950197+0000","last_peered":"2024-09-21T05:57:16.950197+0000","last_clean":"2024-09-21T05:57:16.950197+0000","last_became_active":"2024-09-21T05:57:16.933532+0000","last_became_peered":"2024-09-21T05:57:16.933532+0000","last_unstale":"2024-09-21T05:57:16.950197+0000","last_undegraded":"2024-09-21T05:57:16.950197+0000","last_fullsized":"2024-09-21T05:57:16.950197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:44.012450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.946078+0000","last_change":"2024-09-21T05:57:15.917205+0000","last_active":"2024-09-21T05:57:16.946078+0000","last_peered":"2024-09-21T05:57:16.946078+0000","last_clean":"2024-09-21T05:57:16.946078+0000","last_became_active":"2024-09-21T05:57:15.916908+0000","last_became_peered":"2024-09-21T05:57:15.916908+0000","last_unstale":"2024-09-21T05:57:16.946078+0000","last_undegraded":"2024-09-21T05:57:16.946078+0000","last_fullsized":"2024-09-21T05:57:16.946078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:43.575259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.942327+0000","last_change":"2024-09-21T05:57:16.936756+0000","last_active":"2024-09-21T05:57:16.942327+0000","last_peered":"2024-09-21T05:57:16.942327+0000","last_clean":"2024-09-21T05:57:16.942327+0000","last_became_active":"2024-09-21T05:57:16.936439+0000","last_became_peered":"2024-09-21T05:57:16.936439+0000","last_unstale":"2024-09-21T05:57:16.942327+0000","last_undegraded":"2024-09-21T05:57:16.942327+0000","last_fullsized":"2024-09-21T05:57:16.942327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:23.957773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.950236+0000","last_change":"2024-09-21T05:57:15.920694+0000","last_active":"2024-09-21T05:57:16.950236+0000","last_peered":"2024-09-21T05:57:16.950236+0000","last_clean":"2024-09-21T05:57:16.950236+0000","last_became_active":"2024-09-21T05:57:15.920344+0000","last_became_peered":"2024-09-21T05:57:15.920344+0000","last_unstale":"2024-09-21T05:57:16.950236+0000","last_undegraded":"2024-09-21T05:57:16.950236+0000","last_fullsized":"2024-09-21T05:57:16.950236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:59.310164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930447+0000","last_change":"2024-09-21T05:57:16.941408+0000","last_active":"2024-09-21T05:57:17.930447+0000","last_peered":"2024-09-21T05:57:17.930447+0000","last_clean":"2024-09-21T05:57:17.930447+0000","last_became_active":"2024-09-21T05:57:16.941088+0000","last_became_peered":"2024-09-21T05:57:16.941088+0000","last_unstale":"2024-09-21T05:57:17.930447+0000","last_undegraded":"2024-09-21T05:57:17.930447+0000","last_fullsized":"2024-09-21T05:57:17.930447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:08.599070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927814+0000","last_change":"2024-09-21T05:57:15.925553+0000","last_active":"2024-09-21T05:57:17.927814+0000","last_peered":"2024-09-21T05:57:17.927814+0000","last_clean":"2024-09-21T05:57:17.927814+0000","last_became_active":"2024-09-21T05:57:15.924815+0000","last_became_peered":"2024-09-21T05:57:15.924815+0000","last_unstale":"2024-09-21T05:57:17.927814+0000","last_undegraded":"2024-09-21T05:57:17.927814+0000","last_fullsized":"2024-09-21T05:57:17.927814+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:33.245150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.952635+0000","last_change":"2024-09-21T05:57:16.946642+0000","last_active":"2024-09-21T05:57:16.952635+0000","last_peered":"2024-09-21T05:57:16.952635+0000","last_clean":"2024-09-21T05:57:16.952635+0000","last_became_active":"2024-09-21T05:57:16.945993+0000","last_became_peered":"2024-09-21T05:57:16.945993+0000","last_unstale":"2024-09-21T05:57:16.952635+0000","last_undegraded":"2024-09-21T05:57:16.952635+0000","last_fullsized":"2024-09-21T05:57:16.952635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:19.582982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.953079+0000","last_change":"2024-09-21T05:57:15.918528+0000","last_active":"2024-09-21T05:57:16.953079+0000","last_peered":"2024-09-21T05:57:16.953079+0000","last_clean":"2024-09-21T05:57:16.953079+0000","last_became_active":"2024-09-21T05:57:15.918222+0000","last_became_peered":"2024-09-21T05:57:15.918222+0000","last_unstale":"2024-09-21T05:57:16.953079+0000","last_undegraded":"2024-09-21T05:57:16.953079+0000","last_fullsized":"2024-09-21T05:57:16.953079+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:01.061385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.935933+0000","last_change":"2024-09-21T05:57:16.932739+0000","last_active":"2024-09-21T05:57:16.935933+0000","last_peered":"2024-09-21T05:57:16.935933+0000","last_clean":"2024-09-21T05:57:16.935933+0000","last_became_active":"2024-09-21T05:57:16.932406+0000","last_became_peered":"2024-09-21T05:57:16.932406+0000","last_unstale":"2024-09-21T05:57:16.935933+0000","last_undegraded":"2024-09-21T05:57:16.935933+0000","last_fullsized":"2024-09-21T05:57:16.935933+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:18.466136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930003+0000","last_change":"2024-09-21T05:57:15.926077+0000","last_active":"2024-09-21T05:57:17.930003+0000","last_peered":"2024-09-21T05:57:17.930003+0000","last_clean":"2024-09-21T05:57:17.930003+0000","last_became_active":"2024-09-21T05:57:15.925722+0000","last_became_peered":"2024-09-21T05:57:15.925722+0000","last_unstale":"2024-09-21T05:57:17.930003+0000","last_undegraded":"2024-09-21T05:57:17.930003+0000","last_fullsized":"2024-09-21T05:57:17.930003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:43.575259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928305+0000","last_change":"2024-09-21T05:57:16.917429+0000","last_active":"2024-09-21T05:57:17.928305+0000","last_peered":"2024-09-21T05:57:17.928305+0000","last_clean":"2024-09-21T05:57:17.928305+0000","last_became_active":"2024-09-21T05:57:16.917002+0000","last_became_peered":"2024-09-21T05:57:16.917002+0000","last_unstale":"2024-09-21T05:57:17.928305+0000","last_undegraded":"2024-09-21T05:57:17.928305+0000","last_fullsized":"2024-09-21T05:57:17.928305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:12.234166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929591+0000","last_change":"2024-09-21T05:57:15.925516+0000","last_active":"2024-09-21T05:57:17.929591+0000","last_peered":"2024-09-21T05:57:17.929591+0000","last_clean":"2024-09-21T05:57:17.929591+0000","last_became_active":"2024-09-21T05:57:15.924926+0000","last_became_peered":"2024-09-21T05:57:15.924926+0000","last_unstale":"2024-09-21T05:57:17.929591+0000","last_undegraded":"2024-09-21T05:57:17.929591+0000","last_fullsized":"2024-09-21T05:57:17.929591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:58.082495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928287+0000","last_change":"2024-09-21T05:57:16.941362+0000","last_active":"2024-09-21T05:57:17.928287+0000","last_peered":"2024-09-21T05:57:17.928287+0000","last_clean":"2024-09-21T05:57:17.928287+0000","last_became_active":"2024-09-21T05:57:16.940941+0000","last_became_peered":"2024-09-21T05:57:16.940941+0000","last_unstale":"2024-09-21T05:57:17.928287+0000","last_undegraded":"2024-09-21T05:57:17.928287+0000","last_fullsized":"2024-09-21T05:57:17.928287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:31.954378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927060+0000","last_change":"2024-09-21T05:57:15.929926+0000","last_active":"2024-09-21T05:57:17.927060+0000","last_peered":"2024-09-21T05:57:17.927060+0000","last_clean":"2024-09-21T05:57:17.927060+0000","last_became_active":"2024-09-21T05:57:15.928734+0000","last_became_peered":"2024-09-21T05:57:15.928734+0000","last_unstale":"2024-09-21T05:57:17.927060+0000","last_undegraded":"2024-09-21T05:57:17.927060+0000","last_fullsized":"2024-09-21T05:57:17.927060+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.033103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.951218+0000","last_change":"2024-09-21T05:57:16.944919+0000","last_active":"2024-09-21T05:57:16.951218+0000","last_peered":"2024-09-21T05:57:16.951218+0000","last_clean":"2024-09-21T05:57:16.951218+0000","last_became_active":"2024-09-21T05:57:16.944535+0000","last_became_peered":"2024-09-21T05:57:16.944535+0000","last_unstale":"2024-09-21T05:57:16.951218+0000","last_undegraded":"2024-09-21T05:57:16.951218+0000","last_fullsized":"2024-09-21T05:57:16.951218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:06.976050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928255+0000","last_change":"2024-09-21T05:57:15.909641+0000","last_active":"2024-09-21T05:57:17.928255+0000","last_peered":"2024-09-21T05:57:17.928255+0000","last_clean":"2024-09-21T05:57:17.928255+0000","last_became_active":"2024-09-21T05:57:15.909277+0000","last_became_peered":"2024-09-21T05:57:15.909277+0000","last_unstale":"2024-09-21T05:57:17.928255+0000","last_undegraded":"2024-09-21T05:57:17.928255+0000","last_fullsized":"2024-09-21T05:57:17.928255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:29.782251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929252+0000","last_change":"2024-09-21T05:57:16.941383+0000","last_active":"2024-09-21T05:57:17.929252+0000","last_peered":"2024-09-21T05:57:17.929252+0000","last_clean":"2024-09-21T05:57:17.929252+0000","last_became_active":"2024-09-21T05:57:16.941079+0000","last_became_peered":"2024-09-21T05:57:16.941079+0000","last_unstale":"2024-09-21T05:57:17.929252+0000","last_undegraded":"2024-09-21T05:57:17.929252+0000","last_fullsized":"2024-09-21T05:57:17.929252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.164703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927894+0000","last_change":"2024-09-21T05:57:15.926406+0000","last_active":"2024-09-21T05:57:17.927894+0000","last_peered":"2024-09-21T05:57:17.927894+0000","last_clean":"2024-09-21T05:57:17.927894+0000","last_became_active":"2024-09-21T05:57:15.924778+0000","last_became_peered":"2024-09-21T05:57:15.924778+0000","last_unstale":"2024-09-21T05:57:17.927894+0000","last_undegraded":"2024-09-21T05:57:17.927894+0000","last_fullsized":"2024-09-21T05:57:17.927894+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:40.784634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927895+0000","last_change":"2024-09-21T05:57:16.954250+0000","last_active":"2024-09-21T05:57:17.927895+0000","last_peered":"2024-09-21T05:57:17.927895+0000","last_clean":"2024-09-21T05:57:17.927895+0000","last_became_active":"2024-09-21T05:57:16.953568+0000","last_became_peered":"2024-09-21T05:57:16.953568+0000","last_unstale":"2024-09-21T05:57:17.927895+0000","last_undegraded":"2024-09-21T05:57:17.927895+0000","last_fullsized":"2024-09-21T05:57:17.927895+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:28.060065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.908946+0000","last_change":"2024-09-21T05:57:15.926232+0000","last_active":"2024-09-21T05:57:16.908946+0000","last_peered":"2024-09-21T05:57:16.908946+0000","last_clean":"2024-09-21T05:57:16.908946+0000","last_became_active":"2024-09-21T05:57:15.925837+0000","last_became_peered":"2024-09-21T05:57:15.925837+0000","last_unstale":"2024-09-21T05:57:16.908946+0000","last_undegraded":"2024-09-21T05:57:16.908946+0000","last_fullsized":"2024-09-21T05:57:16.908946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:27:15.049834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927550+0000","last_change":"2024-09-21T05:57:16.941143+0000","last_active":"2024-09-21T05:57:17.927550+0000","last_peered":"2024-09-21T05:57:17.927550+0000","last_clean":"2024-09-21T05:57:17.927550+0000","last_became_active":"2024-09-21T05:57:16.940798+0000","last_became_peered":"2024-09-21T05:57:16.940798+0000","last_unstale":"2024-09-21T05:57:17.927550+0000","last_undegraded":"2024-09-21T05:57:17.927550+0000","last_fullsized":"2024-09-21T05:57:17.927550+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:17.045500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929100+0000","last_change":"2024-09-21T05:57:15.919832+0000","last_active":"2024-09-21T05:57:17.929100+0000","last_peered":"2024-09-21T05:57:17.929100+0000","last_clean":"2024-09-21T05:57:17.929100+0000","last_became_active":"2024-09-21T05:57:15.919510+0000","last_became_peered":"2024-09-21T05:57:15.919510+0000","last_unstale":"2024-09-21T05:57:17.929100+0000","last_undegraded":"2024-09-21T05:57:17.929100+0000","last_fullsized":"2024-09-21T05:57:17.929100+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:16.765386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928938+0000","last_change":"2024-09-21T05:57:16.944659+0000","last_active":"2024-09-21T05:57:17.928938+0000","last_peered":"2024-09-21T05:57:17.928938+0000","last_clean":"2024-09-21T05:57:17.928938+0000","last_became_active":"2024-09-21T05:57:16.943952+0000","last_became_peered":"2024-09-21T05:57:16.943952+0000","last_unstale":"2024-09-21T05:57:17.928938+0000","last_undegraded":"2024-09-21T05:57:17.928938+0000","last_fullsized":"2024-09-21T05:57:17.928938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:17.045500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927623+0000","last_change":"2024-09-21T05:57:15.927479+0000","last_active":"2024-09-21T05:57:17.927623+0000","last_peered":"2024-09-21T05:57:17.927623+0000","last_clean":"2024-09-21T05:57:17.927623+0000","last_became_active":"2024-09-21T05:57:15.927178+0000","last_became_peered":"2024-09-21T05:57:15.927178+0000","last_unstale":"2024-09-21T05:57:17.927623+0000","last_undegraded":"2024-09-21T05:57:17.927623+0000","last_fullsized":"2024-09-21T05:57:17.927623+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:07.759407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.946829+0000","last_change":"2024-09-21T05:57:16.944107+0000","last_active":"2024-09-21T05:57:16.946829+0000","last_peered":"2024-09-21T05:57:16.946829+0000","last_clean":"2024-09-21T05:57:16.946829+0000","last_became_active":"2024-09-21T05:57:16.943762+0000","last_became_peered":"2024-09-21T05:57:16.943762+0000","last_unstale":"2024-09-21T05:57:16.946829+0000","last_undegraded":"2024-09-21T05:57:16.946829+0000","last_fullsized":"2024-09-21T05:57:16.946829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:28.339309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930159+0000","last_change":"2024-09-21T05:57:15.909732+0000","last_active":"2024-09-21T05:57:17.930159+0000","last_peered":"2024-09-21T05:57:17.930159+0000","last_clean":"2024-09-21T05:57:17.930159+0000","last_became_active":"2024-09-21T05:57:15.909380+0000","last_became_peered":"2024-09-21T05:57:15.909380+0000","last_unstale":"2024-09-21T05:57:17.930159+0000","last_undegraded":"2024-09-21T05:57:17.930159+0000","last_fullsized":"2024-09-21T05:57:17.930159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:59.310164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927553+0000","last_change":"2024-09-21T05:57:16.934731+0000","last_active":"2024-09-21T05:57:17.927553+0000","last_peered":"2024-09-21T05:57:17.927553+0000","last_clean":"2024-09-21T05:57:17.927553+0000","last_became_active":"2024-09-21T05:57:16.934215+0000","last_became_peered":"2024-09-21T05:57:16.934215+0000","last_unstale":"2024-09-21T05:57:17.927553+0000","last_undegraded":"2024-09-21T05:57:17.927553+0000","last_fullsized":"2024-09-21T05:57:17.927553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:24.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.929158+0000","last_change":"2024-09-21T05:57:15.908297+0000","last_active":"2024-09-21T05:57:16.929158+0000","last_peered":"2024-09-21T05:57:16.929158+0000","last_clean":"2024-09-21T05:57:16.929158+0000","last_became_active":"2024-09-21T05:57:15.907487+0000","last_became_peered":"2024-09-21T05:57:15.907487+0000","last_unstale":"2024-09-21T05:57:16.929158+0000","last_undegraded":"2024-09-21T05:57:16.929158+0000","last_fullsized":"2024-09-21T05:57:16.929158+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:55.280545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.939280+0000","last_change":"2024-09-21T05:57:16.935017+0000","last_active":"2024-09-21T05:57:16.939280+0000","last_peered":"2024-09-21T05:57:16.939280+0000","last_clean":"2024-09-21T05:57:16.939280+0000","last_became_active":"2024-09-21T05:57:16.932497+0000","last_became_peered":"2024-09-21T05:57:16.932497+0000","last_unstale":"2024-09-21T05:57:16.939280+0000","last_undegraded":"2024-09-21T05:57:16.939280+0000","last_fullsized":"2024-09-21T05:57:16.939280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:31.954378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927284+0000","last_change":"2024-09-21T05:57:15.926773+0000","last_active":"2024-09-21T05:57:17.927284+0000","last_peered":"2024-09-21T05:57:17.927284+0000","last_clean":"2024-09-21T05:57:17.927284+0000","last_became_active":"2024-09-21T05:57:15.925391+0000","last_became_peered":"2024-09-21T05:57:15.925391+0000","last_unstale":"2024-09-21T05:57:17.927284+0000","last_undegraded":"2024-09-21T05:57:17.927284+0000","last_fullsized":"2024-09-21T05:57:17.927284+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:17.463340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.941149+0000","last_change":"2024-09-21T05:57:16.934230+0000","last_active":"2024-09-21T05:57:16.941149+0000","last_peered":"2024-09-21T05:57:16.941149+0000","last_clean":"2024-09-21T05:57:16.941149+0000","last_became_active":"2024-09-21T05:57:16.933060+0000","last_became_peered":"2024-09-21T05:57:16.933060+0000","last_unstale":"2024-09-21T05:57:16.941149+0000","last_undegraded":"2024-09-21T05:57:16.941149+0000","last_fullsized":"2024-09-21T05:57:16.941149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:27:16.052630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927059+0000","last_change":"2024-09-21T05:57:15.923890+0000","last_active":"2024-09-21T05:57:17.927059+0000","last_peered":"2024-09-21T05:57:17.927059+0000","last_clean":"2024-09-21T05:57:17.927059+0000","last_became_active":"2024-09-21T05:57:15.923618+0000","last_became_peered":"2024-09-21T05:57:15.923618+0000","last_unstale":"2024-09-21T05:57:17.927059+0000","last_undegraded":"2024-09-21T05:57:17.927059+0000","last_fullsized":"2024-09-21T05:57:17.927059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:50.185505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930449+0000","last_change":"2024-09-21T05:57:16.917415+0000","last_active":"2024-09-21T05:57:17.930449+0000","last_peered":"2024-09-21T05:57:17.930449+0000","last_clean":"2024-09-21T05:57:17.930449+0000","last_became_active":"2024-09-21T05:57:16.917133+0000","last_became_peered":"2024-09-21T05:57:16.917133+0000","last_unstale":"2024-09-21T05:57:17.930449+0000","last_undegraded":"2024-09-21T05:57:17.930449+0000","last_fullsized":"2024-09-21T05:57:17.930449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:01.510305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.959725+0000","last_change":"2024-09-21T05:57:15.925125+0000","last_active":"2024-09-21T05:57:16.959725+0000","last_peered":"2024-09-21T05:57:16.959725+0000","last_clean":"2024-09-21T05:57:16.959725+0000","last_became_active":"2024-09-21T05:57:15.924760+0000","last_became_peered":"2024-09-21T05:57:15.924760+0000","last_unstale":"2024-09-21T05:57:16.959725+0000","last_undegraded":"2024-09-21T05:57:16.959725+0000","last_fullsized":"2024-09-21T05:57:16.959725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:13.041145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929558+0000","last_change":"2024-09-21T05:57:16.918473+0000","last_active":"2024-09-21T05:57:17.929558+0000","last_peered":"2024-09-21T05:57:17.929558+0000","last_clean":"2024-09-21T05:57:17.929558+0000","last_became_active":"2024-09-21T05:57:16.918141+0000","last_became_peered":"2024-09-21T05:57:16.918141+0000","last_unstale":"2024-09-21T05:57:17.929558+0000","last_undegraded":"2024-09-21T05:57:17.929558+0000","last_fullsized":"2024-09-21T05:57:17.929558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:51.231280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930274+0000","last_change":"2024-09-21T05:57:15.912772+0000","last_active":"2024-09-21T05:57:17.930274+0000","last_peered":"2024-09-21T05:57:17.930274+0000","last_clean":"2024-09-21T05:57:17.930274+0000","last_became_active":"2024-09-21T05:57:15.912439+0000","last_became_peered":"2024-09-21T05:57:15.912439+0000","last_unstale":"2024-09-21T05:57:17.930274+0000","last_undegraded":"2024-09-21T05:57:17.930274+0000","last_fullsized":"2024-09-21T05:57:17.930274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:05.879005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.949130+0000","last_change":"2024-09-21T05:57:16.946819+0000","last_active":"2024-09-21T05:57:16.949130+0000","last_peered":"2024-09-21T05:57:16.949130+0000","last_clean":"2024-09-21T05:57:16.949130+0000","last_became_active":"2024-09-21T05:57:16.946089+0000","last_became_peered":"2024-09-21T05:57:16.946089+0000","last_unstale":"2024-09-21T05:57:16.949130+0000","last_undegraded":"2024-09-21T05:57:16.949130+0000","last_fullsized":"2024-09-21T05:57:16.949130+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:14.707859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930326+0000","last_change":"2024-09-21T05:57:15.920592+0000","last_active":"2024-09-21T05:57:17.930326+0000","last_peered":"2024-09-21T05:57:17.930326+0000","last_clean":"2024-09-21T05:57:17.930326+0000","last_became_active":"2024-09-21T05:57:15.920223+0000","last_became_peered":"2024-09-21T05:57:15.920223+0000","last_unstale":"2024-09-21T05:57:17.930326+0000","last_undegraded":"2024-09-21T05:57:17.930326+0000","last_fullsized":"2024-09-21T05:57:17.930326+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:02.922491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.934667+0000","last_change":"2024-09-21T05:57:16.932868+0000","last_active":"2024-09-21T05:57:16.934667+0000","last_peered":"2024-09-21T05:57:16.934667+0000","last_clean":"2024-09-21T05:57:16.934667+0000","last_became_active":"2024-09-21T05:57:16.932529+0000","last_became_peered":"2024-09-21T05:57:16.932529+0000","last_unstale":"2024-09-21T05:57:16.934667+0000","last_undegraded":"2024-09-21T05:57:16.934667+0000","last_fullsized":"2024-09-21T05:57:16.934667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:08.086568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.930232+0000","last_change":"2024-09-21T05:57:15.928375+0000","last_active":"2024-09-21T05:57:17.930232+0000","last_peered":"2024-09-21T05:57:17.930232+0000","last_clean":"2024-09-21T05:57:17.930232+0000","last_became_active":"2024-09-21T05:57:15.928053+0000","last_became_peered":"2024-09-21T05:57:15.928053+0000","last_unstale":"2024-09-21T05:57:17.930232+0000","last_undegraded":"2024-09-21T05:57:17.930232+0000","last_fullsized":"2024-09-21T05:57:17.930232+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:24.553890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.933292+0000","last_change":"2024-09-21T05:57:16.928785+0000","last_active":"2024-09-21T05:57:16.933292+0000","last_peered":"2024-09-21T05:57:16.933292+0000","last_clean":"2024-09-21T05:57:16.933292+0000","last_became_active":"2024-09-21T05:57:16.928453+0000","last_became_peered":"2024-09-21T05:57:16.928453+0000","last_unstale":"2024-09-21T05:57:16.933292+0000","last_undegraded":"2024-09-21T05:57:16.933292+0000","last_fullsized":"2024-09-21T05:57:16.933292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:35.911229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.955958+0000","last_change":"2024-09-21T05:57:15.912938+0000","last_active":"2024-09-21T05:57:16.955958+0000","last_peered":"2024-09-21T05:57:16.955958+0000","last_clean":"2024-09-21T05:57:16.955958+0000","last_became_active":"2024-09-21T05:57:15.912610+0000","last_became_peered":"2024-09-21T05:57:15.912610+0000","last_unstale":"2024-09-21T05:57:16.955958+0000","last_undegraded":"2024-09-21T05:57:16.955958+0000","last_fullsized":"2024-09-21T05:57:16.955958+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:12.785567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.933182+0000","last_change":"2024-09-21T05:57:16.925512+0000","last_active":"2024-09-21T05:57:16.933182+0000","last_peered":"2024-09-21T05:57:16.933182+0000","last_clean":"2024-09-21T05:57:16.933182+0000","last_became_active":"2024-09-21T05:57:16.925047+0000","last_became_peered":"2024-09-21T05:57:16.925047+0000","last_unstale":"2024-09-21T05:57:16.933182+0000","last_undegraded":"2024-09-21T05:57:16.933182+0000","last_fullsized":"2024-09-21T05:57:16.933182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:30.063373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.930909+0000","last_change":"2024-09-21T05:57:15.912332+0000","last_active":"2024-09-21T05:57:16.930909+0000","last_peered":"2024-09-21T05:57:16.930909+0000","last_clean":"2024-09-21T05:57:16.930909+0000","last_became_active":"2024-09-21T05:57:15.911414+0000","last_became_peered":"2024-09-21T05:57:15.911414+0000","last_unstale":"2024-09-21T05:57:16.930909+0000","last_undegraded":"2024-09-21T05:57:16.930909+0000","last_fullsized":"2024-09-21T05:57:16.930909+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:58.082495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.928718+0000","last_change":"2024-09-21T05:57:16.919184+0000","last_active":"2024-09-21T05:57:16.928718+0000","last_peered":"2024-09-21T05:57:16.928718+0000","last_clean":"2024-09-21T05:57:16.928718+0000","last_became_active":"2024-09-21T05:57:16.918738+0000","last_became_peered":"2024-09-21T05:57:16.918738+0000","last_unstale":"2024-09-21T05:57:16.928718+0000","last_undegraded":"2024-09-21T05:57:16.928718+0000","last_fullsized":"2024-09-21T05:57:16.928718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:20.678838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928896+0000","last_change":"2024-09-21T05:57:15.913419+0000","last_active":"2024-09-21T05:57:17.928896+0000","last_peered":"2024-09-21T05:57:17.928896+0000","last_clean":"2024-09-21T05:57:17.928896+0000","last_became_active":"2024-09-21T05:57:15.913056+0000","last_became_peered":"2024-09-21T05:57:15.913056+0000","last_unstale":"2024-09-21T05:57:17.928896+0000","last_undegraded":"2024-09-21T05:57:17.928896+0000","last_fullsized":"2024-09-21T05:57:17.928896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:11.273821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.940685+0000","last_change":"2024-09-21T05:57:16.938390+0000","last_active":"2024-09-21T05:57:16.940685+0000","last_peered":"2024-09-21T05:57:16.940685+0000","last_clean":"2024-09-21T05:57:16.940685+0000","last_became_active":"2024-09-21T05:57:16.938074+0000","last_became_peered":"2024-09-21T05:57:16.938074+0000","last_unstale":"2024-09-21T05:57:16.940685+0000","last_undegraded":"2024-09-21T05:57:16.940685+0000","last_fullsized":"2024-09-21T05:57:16.940685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:08.599070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927811+0000","last_change":"2024-09-21T05:57:15.919361+0000","last_active":"2024-09-21T05:57:17.927811+0000","last_peered":"2024-09-21T05:57:17.927811+0000","last_clean":"2024-09-21T05:57:17.927811+0000","last_became_active":"2024-09-21T05:57:15.919009+0000","last_became_peered":"2024-09-21T05:57:15.919009+0000","last_unstale":"2024-09-21T05:57:17.927811+0000","last_undegraded":"2024-09-21T05:57:17.927811+0000","last_fullsized":"2024-09-21T05:57:17.927811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:42.914075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.925589+0000","last_change":"2024-09-21T05:57:16.921956+0000","last_active":"2024-09-21T05:57:16.925589+0000","last_peered":"2024-09-21T05:57:16.925589+0000","last_clean":"2024-09-21T05:57:16.925589+0000","last_became_active":"2024-09-21T05:57:16.921631+0000","last_became_peered":"2024-09-21T05:57:16.921631+0000","last_unstale":"2024-09-21T05:57:16.925589+0000","last_undegraded":"2024-09-21T05:57:16.925589+0000","last_fullsized":"2024-09-21T05:57:16.925589+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:22.936196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.953503+0000","last_change":"2024-09-21T05:57:15.913998+0000","last_active":"2024-09-21T05:57:16.953503+0000","last_peered":"2024-09-21T05:57:16.953503+0000","last_clean":"2024-09-21T05:57:16.953503+0000","last_became_active":"2024-09-21T05:57:15.912647+0000","last_became_peered":"2024-09-21T05:57:15.912647+0000","last_unstale":"2024-09-21T05:57:16.953503+0000","last_undegraded":"2024-09-21T05:57:16.953503+0000","last_fullsized":"2024-09-21T05:57:16.953503+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:24.588116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929572+0000","last_change":"2024-09-21T05:57:16.946973+0000","last_active":"2024-09-21T05:57:17.929572+0000","last_peered":"2024-09-21T05:57:17.929572+0000","last_clean":"2024-09-21T05:57:17.929572+0000","last_became_active":"2024-09-21T05:57:16.946617+0000","last_became_peered":"2024-09-21T05:57:16.946617+0000","last_unstale":"2024-09-21T05:57:17.929572+0000","last_undegraded":"2024-09-21T05:57:17.929572+0000","last_fullsized":"2024-09-21T05:57:17.929572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:36.304709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.909161+0000","last_change":"2024-09-21T05:57:15.909184+0000","last_active":"2024-09-21T05:57:16.909161+0000","last_peered":"2024-09-21T05:57:16.909161+0000","last_clean":"2024-09-21T05:57:16.909161+0000","last_became_active":"2024-09-21T05:57:15.908859+0000","last_became_peered":"2024-09-21T05:57:15.908859+0000","last_unstale":"2024-09-21T05:57:16.909161+0000","last_undegraded":"2024-09-21T05:57:16.909161+0000","last_fullsized":"2024-09-21T05:57:16.909161+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:22.954976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.940286+0000","last_change":"2024-09-21T05:57:16.938486+0000","last_active":"2024-09-21T05:57:16.940286+0000","last_peered":"2024-09-21T05:57:16.940286+0000","last_clean":"2024-09-21T05:57:16.940286+0000","last_became_active":"2024-09-21T05:57:16.938159+0000","last_became_peered":"2024-09-21T05:57:16.938159+0000","last_unstale":"2024-09-21T05:57:16.940286+0000","last_undegraded":"2024-09-21T05:57:16.940286+0000","last_fullsized":"2024-09-21T05:57:16.940286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:14:42.049159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.928369+0000","last_change":"2024-09-21T05:57:15.921703+0000","last_active":"2024-09-21T05:57:16.928369+0000","last_peered":"2024-09-21T05:57:16.928369+0000","last_clean":"2024-09-21T05:57:16.928369+0000","last_became_active":"2024-09-21T05:57:15.921323+0000","last_became_peered":"2024-09-21T05:57:15.921323+0000","last_unstale":"2024-09-21T05:57:16.928369+0000","last_undegraded":"2024-09-21T05:57:16.928369+0000","last_fullsized":"2024-09-21T05:57:16.928369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:53.815717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.951665+0000","last_change":"2024-09-21T05:57:16.948625+0000","last_active":"2024-09-21T05:57:16.951665+0000","last_peered":"2024-09-21T05:57:16.951665+0000","last_clean":"2024-09-21T05:57:16.951665+0000","last_became_active":"2024-09-21T05:57:16.948321+0000","last_became_peered":"2024-09-21T05:57:16.948321+0000","last_unstale":"2024-09-21T05:57:16.951665+0000","last_undegraded":"2024-09-21T05:57:16.951665+0000","last_fullsized":"2024-09-21T05:57:16.951665+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:44.012450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.908710+0000","last_change":"2024-09-21T05:57:15.913517+0000","last_active":"2024-09-21T05:57:16.908710+0000","last_peered":"2024-09-21T05:57:16.908710+0000","last_clean":"2024-09-21T05:57:16.908710+0000","last_became_active":"2024-09-21T05:57:15.912035+0000","last_became_peered":"2024-09-21T05:57:15.912035+0000","last_unstale":"2024-09-21T05:57:16.908710+0000","last_undegraded":"2024-09-21T05:57:16.908710+0000","last_fullsized":"2024-09-21T05:57:16.908710+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:43.575259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.955904+0000","last_change":"2024-09-21T05:57:16.951076+0000","last_active":"2024-09-21T05:57:16.955904+0000","last_peered":"2024-09-21T05:57:16.955904+0000","last_clean":"2024-09-21T05:57:16.955904+0000","last_became_active":"2024-09-21T05:57:16.950589+0000","last_became_peered":"2024-09-21T05:57:16.950589+0000","last_unstale":"2024-09-21T05:57:16.955904+0000","last_undegraded":"2024-09-21T05:57:16.955904+0000","last_fullsized":"2024-09-21T05:57:16.955904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:03.568849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.929521+0000","last_change":"2024-09-21T05:57:15.914840+0000","last_active":"2024-09-21T05:57:16.929521+0000","last_peered":"2024-09-21T05:57:16.929521+0000","last_clean":"2024-09-21T05:57:16.929521+0000","last_became_active":"2024-09-21T05:57:15.912953+0000","last_became_peered":"2024-09-21T05:57:15.912953+0000","last_unstale":"2024-09-21T05:57:16.929521+0000","last_undegraded":"2024-09-21T05:57:16.929521+0000","last_fullsized":"2024-09-21T05:57:16.929521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:45.073451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927087+0000","last_change":"2024-09-21T05:57:16.943627+0000","last_active":"2024-09-21T05:57:17.927087+0000","last_peered":"2024-09-21T05:57:17.927087+0000","last_clean":"2024-09-21T05:57:17.927087+0000","last_became_active":"2024-09-21T05:57:16.942858+0000","last_became_peered":"2024-09-21T05:57:16.942858+0000","last_unstale":"2024-09-21T05:57:17.927087+0000","last_undegraded":"2024-09-21T05:57:17.927087+0000","last_fullsized":"2024-09-21T05:57:17.927087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:12.234166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.929146+0000","last_change":"2024-09-21T05:57:15.914024+0000","last_active":"2024-09-21T05:57:17.929146+0000","last_peered":"2024-09-21T05:57:17.929146+0000","last_clean":"2024-09-21T05:57:17.929146+0000","last_became_active":"2024-09-21T05:57:15.913698+0000","last_became_peered":"2024-09-21T05:57:15.913698+0000","last_unstale":"2024-09-21T05:57:17.929146+0000","last_undegraded":"2024-09-21T05:57:17.929146+0000","last_fullsized":"2024-09-21T05:57:17.929146+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:27:15.049834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.928709+0000","last_change":"2024-09-21T05:57:16.941067+0000","last_active":"2024-09-21T05:57:17.928709+0000","last_peered":"2024-09-21T05:57:17.928709+0000","last_clean":"2024-09-21T05:57:17.928709+0000","last_became_active":"2024-09-21T05:57:16.940701+0000","last_became_peered":"2024-09-21T05:57:16.940701+0000","last_unstale":"2024-09-21T05:57:17.928709+0000","last_undegraded":"2024-09-21T05:57:17.928709+0000","last_fullsized":"2024-09-21T05:57:17.928709+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:08.762203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T05:57:17.927189+0000","last_change":"2024-09-21T05:57:15.927747+0000","last_active":"2024-09-21T05:57:17.927189+0000","last_peered":"2024-09-21T05:57:17.927189+0000","last_clean":"2024-09-21T05:57:17.927189+0000","last_became_active":"2024-09-21T05:57:15.925820+0000","last_became_peered":"2024-09-21T05:57:15.925820+0000","last_unstale":"2024-09-21T05:57:17.927189+0000","last_undegraded":"2024-09-21T05:57:17.927189+0000","last_fullsized":"2024-09-21T05:57:17.927189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:30.951582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.939595+0000","last_change":"2024-09-21T05:57:16.934655+0000","last_active":"2024-09-21T05:57:16.939595+0000","last_peered":"2024-09-21T05:57:16.939595+0000","last_clean":"2024-09-21T05:57:16.939595+0000","last_became_active":"2024-09-21T05:57:16.934349+0000","last_became_peered":"2024-09-21T05:57:16.934349+0000","last_unstale":"2024-09-21T05:57:16.939595+0000","last_undegraded":"2024-09-21T05:57:16.939595+0000","last_fullsized":"2024-09-21T05:57:16.939595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:15.887760+0000","last_clean_scrub_stamp":"2024-09-21T05:57:15.887760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:51.188301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.934147+0000","last_change":"2024-09-21T05:57:15.916369+0000","last_active":"2024-09-21T05:57:16.934147+0000","last_peered":"2024-09-21T05:57:16.934147+0000","last_clean":"2024-09-21T05:57:16.934147+0000","last_became_active":"2024-09-21T05:57:15.915996+0000","last_became_peered":"2024-09-21T05:57:15.915996+0000","last_unstale":"2024-09-21T05:57:16.934147+0000","last_undegraded":"2024-09-21T05:57:16.934147+0000","last_fullsized":"2024-09-21T05:57:16.934147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:57:14.884964+0000","last_clean_scrub_stamp":"2024-09-21T05:57:14.884964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:59.310164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":129,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:57:16.920139+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:57:16.920139+0000","last_peered":"2024-09-21T05:57:16.920139+0000","last_clean":"2024-09-21T05:57:16.920139+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:57:16.920139+0000","last_undegraded":"2024-09-21T05:57:16.920139+0000","last_fullsized":"2024-09-21T05:57:16.920139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":950272,"data_stored":723634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":133792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":372736,"data_stored":140148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":372736,"data_stored":139580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":368640,"data_stored":134810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31540,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":356352,"data_stored":133246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":723544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":133176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:57:21.551 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:57:21.552 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T05:57:21.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.754+0000 7f8cfac00640 1 -- 172.21.15.50:0/2033347110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf40b4de0 msgr2=0x7f8cf40b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:21.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.754+0000 7f8cfac00640 1 --2- 172.21.15.50:0/2033347110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf40b4de0 0x7f8cf40b51f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009840 tx=0x7f8ce001c7f0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.754+0000 7f8cfac00640 1 -- 172.21.15.50:0/2033347110 shutdown_connections 2024-09-21T05:57:21.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.754+0000 7f8cfac00640 1 --2- 172.21.15.50:0/2033347110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf40b4de0 0x7f8cf40b51f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.754+0000 7f8cfac00640 1 -- 172.21.15.50:0/2033347110 >> 172.21.15.50:0/2033347110 conn(0x7f8cf40b2040 msgr2=0x7f8cf40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:21.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.754+0000 7f8cfac00640 1 -- 172.21.15.50:0/2033347110 shutdown_connections 2024-09-21T05:57:21.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.754+0000 7f8cfac00640 1 -- 172.21.15.50:0/2033347110 wait complete. 2024-09-21T05:57:21.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cfac00640 1 Processor -- start 2024-09-21T05:57:21.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cfac00640 1 -- start start 2024-09-21T05:57:21.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cfac00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8cf40738b0 0x7f8cf414c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:21.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cfac00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf40b4de0 0x7f8cf41fa550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:21.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cfac00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf415cc60 0x7f8cf415a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:21.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cfac00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4169d40 con 0x7f8cf415cc60 2024-09-21T05:57:21.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cfac00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4152a30 con 0x7f8cf40b4de0 2024-09-21T05:57:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cfac00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf4169ec0 con 0x7f8cf40738b0 2024-09-21T05:57:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf415cc60 0x7f8cf415a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf415cc60 0x7f8cf415a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58938/0 (socket says 172.21.15.50:58938) 2024-09-21T05:57:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf8c00640 1 -- 172.21.15.50:0/1925546398 learned_addr learned my addr 172.21.15.50:0/1925546398 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:21.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf3e00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8cf40738b0 0x7f8cf414c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:21.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf3400640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf40b4de0 0x7f8cf41fa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:21.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf8c00640 1 -- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8cf40738b0 msgr2=0x7f8cf414c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf8c00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8cf40738b0 0x7f8cf414c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf8c00640 1 -- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf40b4de0 msgr2=0x7f8cf41fa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf8c00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf40b4de0 0x7f8cf41fa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf8c00640 1 -- 172.21.15.50:0/1925546398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf41faa90 con 0x7f8cf415cc60 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf3e00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8cf40738b0 0x7f8cf414c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf3400640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf40b4de0 0x7f8cf41fa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cf8c00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf415cc60 0x7f8cf415a430 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00cb40 tx=0x7f8cec00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cebe00640 1 -- 172.21.15.50:0/1925546398 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec016020 con 0x7f8cf415cc60 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cebe00640 1 -- 172.21.15.50:0/1925546398 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cec00aae0 con 0x7f8cf415cc60 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cebe00640 1 -- 172.21.15.50:0/1925546398 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec012420 con 0x7f8cf415cc60 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cfac00640 1 -- 172.21.15.50:0/1925546398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf41f6c70 con 0x7f8cf415cc60 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.758+0000 7f8cfac00640 1 -- 172.21.15.50:0/1925546398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf41f7230 con 0x7f8cf415cc60 2024-09-21T05:57:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.762+0000 7f8cfac00640 1 -- 172.21.15.50:0/1925546398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf40cf1d0 con 0x7f8cf415cc60 2024-09-21T05:57:21.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.762+0000 7f8cebe00640 1 -- 172.21.15.50:0/1925546398 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8cec019020 con 0x7f8cf415cc60 2024-09-21T05:57:21.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.762+0000 7f8cebe00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8cd4073230 0x7f8cd4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:21.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.762+0000 7f8cebe00640 1 -- 172.21.15.50:0/1925546398 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8cec094160 con 0x7f8cf415cc60 2024-09-21T05:57:21.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.762+0000 7f8cf3e00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8cd4073230 0x7f8cd4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:21.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.762+0000 7f8cf3e00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8cd4073230 0x7f8cd4075740 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4001630 tx=0x7f8ce4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:21.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:21.766+0000 7f8cebe00640 1 -- 172.21.15.50:0/1925546398 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8cf40cf1d0 con 0x7f8cf415cc60 2024-09-21T05:57:22.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.006+0000 7f8cfac00640 1 -- 172.21.15.50:0/1925546398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f8cf40b51f0 con 0x7f8cf415cc60 2024-09-21T05:57:22.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cebe00640 1 -- 172.21.15.50:0/1925546398 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v29) ==== 190+0+147 (secure 0 0 0) 0x7f8cec020030 con 0x7f8cf415cc60 2024-09-21T05:57:22.014 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-09-21T05:57:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw== 2024-09-21T05:57:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: caps mds = "allow" 2024-09-21T05:57:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: caps mon = "allow r" 2024-09-21T05:57:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T05:57:22.014 INFO:teuthology.orchestra.run.smithi050.stderr:updated caps for client.0 2024-09-21T05:57:22.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cfac00640 1 -- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8cd4073230 msgr2=0x7f8cd4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:22.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cfac00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8cd4073230 0x7f8cd4075740 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4001630 tx=0x7f8ce4009040 comp rx=0 tx=0).stop 2024-09-21T05:57:22.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cfac00640 1 -- 172.21.15.50:0/1925546398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf415cc60 msgr2=0x7f8cf415a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:22.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cfac00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf415cc60 0x7f8cf415a430 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00cb40 tx=0x7f8cec00cc30 comp rx=0 tx=0).stop 2024-09-21T05:57:22.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cfac00640 1 -- 172.21.15.50:0/1925546398 shutdown_connections 2024-09-21T05:57:22.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cfac00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8cd4073230 0x7f8cd4075740 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cfac00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf415cc60 0x7f8cf415a430 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cfac00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf40b4de0 0x7f8cf41fa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cfac00640 1 --2- 172.21.15.50:0/1925546398 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8cf40738b0 0x7f8cf414c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cfac00640 1 -- 172.21.15.50:0/1925546398 >> 172.21.15.50:0/1925546398 conn(0x7f8cf40b2040 msgr2=0x7f8cf4159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:22.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cfac00640 1 -- 172.21.15.50:0/1925546398 shutdown_connections 2024-09-21T05:57:22.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.010+0000 7f8cfac00640 1 -- 172.21.15.50:0/1925546398 wait complete. 2024-09-21T05:57:22.037 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:57:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.190+0000 7f2a70a00640 1 -- 172.21.15.50:0/1866221045 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a6c1527b0 msgr2=0x7f2a6c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.190+0000 7f2a70a00640 1 --2- 172.21.15.50:0/1866221045 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a6c1527b0 0x7f2a6c15bb60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009900 tx=0x7f2a5401c820 comp rx=0 tx=0).stop 2024-09-21T05:57:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 -- 172.21.15.50:0/1866221045 shutdown_connections 2024-09-21T05:57:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 --2- 172.21.15.50:0/1866221045 >> v2:172.21.15.88:3300/0 conn(0x7f2a6c0b4de0 0x7f2a6c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 --2- 172.21.15.50:0/1866221045 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a6c1527b0 0x7f2a6c15bb60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 -- 172.21.15.50:0/1866221045 >> 172.21.15.50:0/1866221045 conn(0x7f2a6c0b2040 msgr2=0x7f2a6c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:22.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 -- 172.21.15.50:0/1866221045 shutdown_connections 2024-09-21T05:57:22.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 -- 172.21.15.50:0/1866221045 wait complete. 2024-09-21T05:57:22.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 Processor -- start 2024-09-21T05:57:22.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 -- start start 2024-09-21T05:57:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6c0b4de0 0x7f2a6c149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0b5c70 0x7f2a6c14ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a6c1527b0 0x7f2a6c15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c168a60 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1688e0 con 0x7f2a6c0b5c70 2024-09-21T05:57:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.194+0000 7f2a70a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6c168be0 con 0x7f2a6c1527b0 2024-09-21T05:57:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a69e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6c0b4de0 0x7f2a6c149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a61400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0b5c70 0x7f2a6c14ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:22.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a69e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6c0b4de0 0x7f2a6c149a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58950/0 (socket says 172.21.15.50:58950) 2024-09-21T05:57:22.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a69e00640 1 -- 172.21.15.50:0/3040626595 learned_addr learned my addr 172.21.15.50:0/3040626595 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:22.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a6a800640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a6c1527b0 0x7f2a6c15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:22.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a69e00640 1 -- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a6c1527b0 msgr2=0x7f2a6c15a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:22.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a69e00640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a6c1527b0 0x7f2a6c15a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a69e00640 1 -- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0b5c70 msgr2=0x7f2a6c14ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:22.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a69e00640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0b5c70 0x7f2a6c14ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a69e00640 1 -- 172.21.15.50:0/3040626595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c1f6a10 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a69e00640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6c0b4de0 0x7f2a6c149a40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00c730 tx=0x7f2a5c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:22.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a62a00640 1 -- 172.21.15.50:0/3040626595 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c003b80 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a62a00640 1 -- 172.21.15.50:0/3040626595 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a5c003d40 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a62a00640 1 -- 172.21.15.50:0/3040626595 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00aab0 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a70a00640 1 -- 172.21.15.50:0/3040626595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c14faf0 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a70a00640 1 -- 172.21.15.50:0/3040626595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c1f6fb0 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a70a00640 1 -- 172.21.15.50:0/3040626595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a6c1515d0 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.198+0000 7f2a62a00640 1 -- 172.21.15.50:0/3040626595 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2a5c00ac50 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.202+0000 7f2a62a00640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a48073230 0x7f2a48075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:22.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.202+0000 7f2a62a00640 1 -- 172.21.15.50:0/3040626595 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2a5c093ec0 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.202+0000 7f2a61400640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a48073230 0x7f2a48075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:22.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.202+0000 7f2a61400640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a48073230 0x7f2a48075740 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2a540af000 tx=0x7f2a5401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:22.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.206+0000 7f2a62a00640 1 -- 172.21.15.50:0/3040626595 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2a6c0b5730 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.418+0000 7f2a70a00640 1 -- 172.21.15.50:0/3040626595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a6c1512f0 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.418+0000 7f2a62a00640 1 -- 172.21.15.50:0/3040626595 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5217 (secure 0 0 0) 0x7f2a5c05e880 con 0x7f2a6c0b4de0 2024-09-21T05:57:22.423 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:57:22.423 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"btime":"2024-09-21T05:57:19:934324+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3475589656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3475589656},{"type":"v1","addr":"172.21.15.88:6835","nonce":3475589656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/88740447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":88740447},{"type":"v1","addr":"172.21.15.88:6837","nonce":88740447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4798,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/4086930068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":4086930068},{"type":"v1","addr":"172.21.15.50:6833","nonce":4086930068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:57:16.325238+0000","modified":"2024-09-21T05:57:19.359942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4813},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4813":{"gid":4813,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/2722641195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2722641195},{"type":"v1","addr":"172.21.15.50:6835","nonce":2722641195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4813,"qdb_cluster":[4813]},"id":2}]} 2024-09-21T05:57:22.424 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2024-09-21T05:57:22.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.418+0000 7f2a70a00640 1 -- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a48073230 msgr2=0x7f2a48075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:22.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.418+0000 7f2a70a00640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a48073230 0x7f2a48075740 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2a540af000 tx=0x7f2a5401b060 comp rx=0 tx=0).stop 2024-09-21T05:57:22.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.418+0000 7f2a70a00640 1 -- 172.21.15.50:0/3040626595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6c0b4de0 msgr2=0x7f2a6c149a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:22.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.418+0000 7f2a70a00640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6c0b4de0 0x7f2a6c149a40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00c730 tx=0x7f2a5c00cbf0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.418+0000 7f2a70a00640 1 -- 172.21.15.50:0/3040626595 shutdown_connections 2024-09-21T05:57:22.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.418+0000 7f2a70a00640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a6c1527b0 0x7f2a6c15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.418+0000 7f2a70a00640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0b5c70 0x7f2a6c14ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.418+0000 7f2a70a00640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a48073230 0x7f2a48075740 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.418+0000 7f2a70a00640 1 --2- 172.21.15.50:0/3040626595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6c0b4de0 0x7f2a6c149a40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:22.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.418+0000 7f2a70a00640 1 -- 172.21.15.50:0/3040626595 >> 172.21.15.50:0/3040626595 conn(0x7f2a6c0b2040 msgr2=0x7f2a6c151af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:22.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.422+0000 7f2a70a00640 1 -- 172.21.15.50:0/3040626595 shutdown_connections 2024-09-21T05:57:22.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:22.422+0000 7f2a70a00640 1 -- 172.21.15.50:0/3040626595 wait complete. 2024-09-21T05:57:22.848 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:57:16.325238+0000', 'modified': '2024-09-21T05:57:19.359942+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4813}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4813': {'gid': 4813, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6835/2722641195', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6834', 'nonce': 2722641195}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 2722641195}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4813, 'qdb_cluster': [4813]} 2024-09-21T05:57:22.848 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:57:22.848 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T05:57:23.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.010+0000 7f5e16000640 1 -- 172.21.15.50:0/900975251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5df4003a10 msgr2=0x7f5df4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.010+0000 7f5e16000640 1 --2- 172.21.15.50:0/900975251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5df4003a10 0x7f5df4003e80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000c470 tx=0x7f5e0001cf70 comp rx=0 tx=0).stop 2024-09-21T05:57:23.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.010+0000 7f5e0fe00640 1 -- 172.21.15.50:0/900975251 reap_dead start 2024-09-21T05:57:23.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.010+0000 7f5e16000640 1 -- 172.21.15.50:0/900975251 shutdown_connections 2024-09-21T05:57:23.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.010+0000 7f5e16000640 1 -- 172.21.15.50:0/900975251 >> 172.21.15.50:0/900975251 conn(0x7f5e100b2040 msgr2=0x7f5e100b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:23.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.010+0000 7f5e16000640 1 -- 172.21.15.50:0/900975251 shutdown_connections 2024-09-21T05:57:23.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.010+0000 7f5e16000640 1 -- 172.21.15.50:0/900975251 wait complete. 2024-09-21T05:57:23.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.010+0000 7f5e16000640 1 Processor -- start 2024-09-21T05:57:23.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e16000640 1 -- start start 2024-09-21T05:57:23.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e16000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5df4003a10 0x7f5e101f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e16000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e100cf370 0x7f5e10158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e16000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e10152c20 0x7f5e10153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e16000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5e101668a0 con 0x7f5df4003a10 2024-09-21T05:57:23.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e16000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e10166720 con 0x7f5e100cf370 2024-09-21T05:57:23.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e16000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5e10166a20 con 0x7f5e10152c20 2024-09-21T05:57:23.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e0f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5df4003a10 0x7f5e101f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e0f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5df4003a10 0x7f5e101f93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58970/0 (socket says 172.21.15.50:58970) 2024-09-21T05:57:23.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e0f400640 1 -- 172.21.15.50:0/141735242 learned_addr learned my addr 172.21.15.50:0/141735242 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:23.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e06a00640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e100cf370 0x7f5e10158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e0fe00640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e10152c20 0x7f5e10153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e0f400640 1 -- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e10152c20 msgr2=0x7f5e10153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e0f400640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e10152c20 0x7f5e10153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e0f400640 1 -- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e100cf370 msgr2=0x7f5e10158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e0f400640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e100cf370 0x7f5e10158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e0f400640 1 -- 172.21.15.50:0/141735242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e10154530 con 0x7f5df4003a10 2024-09-21T05:57:23.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e0f400640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5df4003a10 0x7f5e101f93e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5e0001b670 tx=0x7f5e00006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:23.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e07e00640 1 -- 172.21.15.50:0/141735242 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00006620 con 0x7f5df4003a10 2024-09-21T05:57:23.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e07e00640 1 -- 172.21.15.50:0/141735242 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e00003600 con 0x7f5df4003a10 2024-09-21T05:57:23.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e07e00640 1 -- 172.21.15.50:0/141735242 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e000039f0 con 0x7f5df4003a10 2024-09-21T05:57:23.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e16000640 1 -- 172.21.15.50:0/141735242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e101549a0 con 0x7f5df4003a10 2024-09-21T05:57:23.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.014+0000 7f5e16000640 1 -- 172.21.15.50:0/141735242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e10157b80 con 0x7f5df4003a10 2024-09-21T05:57:23.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.018+0000 7f5e07e00640 1 -- 172.21.15.50:0/141735242 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5e00003b90 con 0x7f5df4003a10 2024-09-21T05:57:23.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.018+0000 7f5e07e00640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5dec073230 0x7f5dec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.018+0000 7f5e07e00640 1 -- 172.21.15.50:0/141735242 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5e0012e6f0 con 0x7f5df4003a10 2024-09-21T05:57:23.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.018+0000 7f5e06a00640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5dec073230 0x7f5dec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.018+0000 7f5e1d749000 1 Processor -- start 2024-09-21T05:57:23.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.018+0000 7f5e1d749000 1 -- start start 2024-09-21T05:57:23.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.018+0000 7f5e06a00640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5dec073230 0x7f5dec075740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5e100af470 tx=0x7f5df8009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:23.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694ede5460 0x55694f3266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 0x55694f326be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694f1beca0 0x55694f327120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55694f1fb410 con 0x55694f1beca0 2024-09-21T05:57:23.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55694f138b40 con 0x55694f1d5950 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55694f127f70 con 0x55694ede5460 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e0fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694f1beca0 0x55694f327120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e0f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694ede5460 0x55694f3266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e0fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694f1beca0 0x55694f327120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58982/0 (socket says 172.21.15.50:58982) 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e0fe00640 1 -- 172.21.15.50:0/3857730618 learned_addr learned my addr 172.21.15.50:0/3857730618 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e06a00640 1 --2- 172.21.15.50:0/3857730618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 0x55694f326be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e0f400640 1 -- 172.21.15.50:0/3857730618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 msgr2=0x55694f326be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e0f400640 1 --2- 172.21.15.50:0/3857730618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 0x55694f326be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e0f400640 1 -- 172.21.15.50:0/3857730618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694f1beca0 msgr2=0x55694f327120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e0f400640 1 --2- 172.21.15.50:0/3857730618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694f1beca0 0x55694f327120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e0f400640 1 -- 172.21.15.50:0/3857730618 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55694f0ef5b0 con 0x55694ede5460 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e0fe00640 1 --2- 172.21.15.50:0/3857730618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694f1beca0 0x55694f327120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e0f400640 1 --2- 172.21.15.50:0/3857730618 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694ede5460 0x55694f3266a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5e00098a30 tx=0x7f5e000f88a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dad565daf4d1f343 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5dff400640 1 -- 172.21.15.50:0/3857730618 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e001000c0 con 0x55694ede5460 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5dff400640 1 -- 172.21.15.50:0/3857730618 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e000fdb60 con 0x55694ede5460 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5dff400640 1 -- 172.21.15.50:0/3857730618 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00102a10 con 0x55694ede5460 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 -- 172.21.15.50:0/3857730618 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694ede5460 msgr2=0x55694f3266a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 --2- 172.21.15.50:0/3857730618 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694ede5460 0x55694f3266a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5e00098a30 tx=0x7f5e000f88a0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 -- 172.21.15.50:0/3857730618 shutdown_connections 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 --2- 172.21.15.50:0/3857730618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694f1beca0 0x55694f327120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 --2- 172.21.15.50:0/3857730618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 0x55694f326be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 --2- 172.21.15.50:0/3857730618 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694ede5460 0x55694f3266a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 -- 172.21.15.50:0/3857730618 >> 172.21.15.50:0/3857730618 conn(0x55694f2abd90 msgr2=0x55694f2db690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 -- 172.21.15.50:0/3857730618 shutdown_connections 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 -- 172.21.15.50:0/3857730618 wait complete. 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.022+0000 7f5e1d749000 1 -- start start 2024-09-21T05:57:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 1 Processor -- start 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f32d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1beca0 0x55694f32de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1d5950 0x55694f32e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55694f0c25f0 con 0x55694ede5460 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55694f12bd80 con 0x55694f1beca0 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55694f0ce520 con 0x55694f1d5950 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 10 client.0 _ll_register_callbacks cb 0x55694f0d2830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e0f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f32d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e0f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f32d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58986/0 (socket says 172.21.15.50:58986) 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e0f400640 1 -- 172.21.15.50:0/2623054243 learned_addr learned my addr 172.21.15.50:0/2623054243 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e06a00640 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1beca0 0x55694f32de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e0fe00640 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1d5950 0x55694f32e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e0f400640 1 -- 172.21.15.50:0/2623054243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1d5950 msgr2=0x55694f32e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e0f400640 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1d5950 0x55694f32e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e0f400640 1 -- 172.21.15.50:0/2623054243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1beca0 msgr2=0x55694f32de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e0f400640 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1beca0 0x55694f32de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e0f400640 1 -- 172.21.15.50:0/2623054243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55694f0c6190 con 0x55694ede5460 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e0f400640 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f32d920 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5e0001d450 tx=0x7f5e000012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a2bd4f09ec7761 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5df3e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5df3e00640 1 -- 172.21.15.50:0/2623054243 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00111030 con 0x55694ede5460 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 1 -- 172.21.15.50:0/2623054243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55694f0c0110 con 0x55694ede5460 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5df3e00640 1 -- 172.21.15.50:0/2623054243 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5e000f53a0 con 0x55694ede5460 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5df3e00640 1 -- 172.21.15.50:0/2623054243 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5e0011a050 con 0x55694ede5460 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 10 client.5043 fetch_fsmap learned FSMap version 17 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 1 -- 172.21.15.50:0/2623054243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55694f1acf10 con 0x55694ede5460 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5df3e00640 1 -- 172.21.15.50:0/2623054243 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55694f0c0110 con 0x55694ede5460 2024-09-21T05:57:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5df3e00640 1 -- 172.21.15.50:0/2623054243 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f5e000c1030 con 0x55694ede5460 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 10 client.5043 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 10 client.5043 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 20 client.5043 populate_metadata read hostname 'smithi050' 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.50:6834/2722641195,v1:172.21.15.50:6835/2722641195] conn(0x55694edca7d0 0x55694f337d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 4 client.5043 mds_command: new command op to 4813 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.026+0000 7f5e1d749000 1 -- 172.21.15.50:0/2623054243 --> [v2:172.21.15.50:6834/2722641195,v1:172.21.15.50:6835/2722641195] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55694f13aa90 con 0x55694edca7d0 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e06a00640 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.50:6834/2722641195,v1:172.21.15.50:6835/2722641195] conn(0x55694edca7d0 0x55694f337d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e06a00640 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.50:6834/2722641195,v1:172.21.15.50:6835/2722641195] conn(0x55694edca7d0 0x55694f337d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c2fb3186e70a354 server_cookie=31b969e640ca466f in_seq=0 out_seq=0 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5df3e00640 10 client.5043 ms_handle_connect on v2:172.21.15.50:6834/2722641195 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5df3e00640 1 -- 172.21.15.50:0/2623054243 <== mds.0 v2:172.21.15.50:6834/2722641195 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55694f13aa90 con 0x55694edca7d0 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5df3e00640 10 client.5043 handle_command_reply: tid=0 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e1d749000 1 client.5043 shutdown 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e1d749000 1 -- 172.21.15.50:0/2623054243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 msgr2=0x55694f32d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e1d749000 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f32d920 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5e0001d450 tx=0x7f5e000012d0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e1d749000 1 -- 172.21.15.50:0/2623054243 shutdown_connections 2024-09-21T05:57:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e1d749000 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.50:6834/2722641195,v1:172.21.15.50:6835/2722641195] conn(0x55694edca7d0 0x55694f337d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e1d749000 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1d5950 0x55694f32e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e1d749000 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1beca0 0x55694f32de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e1d749000 1 --2- 172.21.15.50:0/2623054243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f32d920 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5df3e00640 0 client.5043 ms_handle_reset on v2:172.21.15.50:6834/2722641195 2024-09-21T05:57:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e1d749000 1 -- 172.21.15.50:0/2623054243 >> 172.21.15.50:0/2623054243 conn(0x55694f2abd90 msgr2=0x55694f2e2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e1d749000 1 -- 172.21.15.50:0/2623054243 shutdown_connections 2024-09-21T05:57:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e1d749000 1 -- 172.21.15.50:0/2623054243 wait complete. 2024-09-21T05:57:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.030+0000 7f5e1d749000 20 client.5043 trim_cache size 0 max 16384 2024-09-21T05:57:23.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e1d749000 1 Processor -- start 2024-09-21T05:57:23.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e1d749000 1 -- start start 2024-09-21T05:57:23.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e1d749000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f398620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e1d749000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1beca0 0x55694f398b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e1d749000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 0x55694f3990a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e1d749000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5e10204d50 con 0x55694ede5460 2024-09-21T05:57:23.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e1d749000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e101fcd60 con 0x55694f1d5950 2024-09-21T05:57:23.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e1d749000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5e10204700 con 0x55694f1beca0 2024-09-21T05:57:23.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e0f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f398620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e0f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f398620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58988/0 (socket says 172.21.15.50:58988) 2024-09-21T05:57:23.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e0f400640 1 -- 172.21.15.50:0/1202256576 learned_addr learned my addr 172.21.15.50:0/1202256576 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:23.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e06a00640 1 --2- 172.21.15.50:0/1202256576 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1beca0 0x55694f398b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e0f400640 1 -- 172.21.15.50:0/1202256576 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1beca0 msgr2=0x55694f398b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:23.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e0f400640 1 --2- 172.21.15.50:0/1202256576 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1beca0 0x55694f398b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e0fe00640 1 --2- 172.21.15.50:0/1202256576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 0x55694f3990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e0f400640 1 -- 172.21.15.50:0/1202256576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 msgr2=0x55694f3990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:23.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e0f400640 1 --2- 172.21.15.50:0/1202256576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 0x55694f3990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e0f400640 1 -- 172.21.15.50:0/1202256576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55694f183db0 con 0x55694ede5460 2024-09-21T05:57:23.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5e0f400640 1 --2- 172.21.15.50:0/1202256576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f398620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5e0001ca30 tx=0x7f5e00001380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e5c1c4f0939426 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5dfea00640 1 -- 172.21.15.50:0/1202256576 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00111030 con 0x55694ede5460 2024-09-21T05:57:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5dfea00640 1 -- 172.21.15.50:0/1202256576 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e00100e50 con 0x55694ede5460 2024-09-21T05:57:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.050+0000 7f5dfea00640 1 -- 172.21.15.50:0/1202256576 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e000f8c40 con 0x55694ede5460 2024-09-21T05:57:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 -- 172.21.15.50:0/1202256576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 msgr2=0x55694f398620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 --2- 172.21.15.50:0/1202256576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f398620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5e0001ca30 tx=0x7f5e00001380 comp rx=0 tx=0).stop 2024-09-21T05:57:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 -- 172.21.15.50:0/1202256576 shutdown_connections 2024-09-21T05:57:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 --2- 172.21.15.50:0/1202256576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 0x55694f3990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 --2- 172.21.15.50:0/1202256576 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1beca0 0x55694f398b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 --2- 172.21.15.50:0/1202256576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f398620 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 -- 172.21.15.50:0/1202256576 >> 172.21.15.50:0/1202256576 conn(0x55694f2abd90 msgr2=0x55694f324620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:23.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 -- 172.21.15.50:0/1202256576 shutdown_connections 2024-09-21T05:57:23.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 -- 172.21.15.50:0/1202256576 wait complete. 2024-09-21T05:57:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 -- start start 2024-09-21T05:57:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 Processor -- start 2024-09-21T05:57:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f39c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1beca0 0x55694f39c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 0x55694f39cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55694f0c25f0 con 0x55694ede5460 2024-09-21T05:57:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55694f1aca80 con 0x55694f1d5950 2024-09-21T05:57:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55694f12bd80 con 0x55694f1beca0 2024-09-21T05:57:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e1d749000 10 client.0 _ll_register_callbacks cb 0x7f5e080096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e0f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f39c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e0f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f39c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59002/0 (socket says 172.21.15.50:59002) 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e0f400640 1 -- 172.21.15.50:0/1388795422 learned_addr learned my addr 172.21.15.50:0/1388795422 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e06a00640 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1beca0 0x55694f39c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e0fe00640 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 0x55694f39cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e0f400640 1 -- 172.21.15.50:0/1388795422 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1beca0 msgr2=0x55694f39c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e0f400640 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1beca0 0x55694f39c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e0f400640 1 -- 172.21.15.50:0/1388795422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 msgr2=0x55694f39cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e0f400640 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 0x55694f39cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.054+0000 7f5e0f400640 1 -- 172.21.15.50:0/1388795422 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55694f0eeab0 con 0x55694ede5460 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e0f400640 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f39c180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5e000f51e0 tx=0x7f5e000f89d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c138e737f8c7933a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5df3e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5df3e00640 1 -- 172.21.15.50:0/1388795422 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e000f8c40 con 0x55694ede5460 2024-09-21T05:57:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5df3e00640 1 -- 172.21.15.50:0/1388795422 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5e000f85a0 con 0x55694ede5460 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 -- 172.21.15.50:0/1388795422 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e00100e50 con 0x55694ede5460 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5df3e00640 1 -- 172.21.15.50:0/1388795422 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5e00102a10 con 0x55694ede5460 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 10 client.5049 fetch_fsmap learned FSMap version 17 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 -- 172.21.15.50:0/1388795422 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55694f0cd070 con 0x55694ede5460 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5df3e00640 1 -- 172.21.15.50:0/1388795422 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e00100e50 con 0x55694ede5460 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5df3e00640 1 -- 172.21.15.50:0/1388795422 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f5e000c1040 con 0x55694ede5460 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 10 client.5049 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 10 client.5049 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 20 client.5049 populate_metadata read hostname 'smithi050' 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.50:6834/2722641195,v1:172.21.15.50:6835/2722641195] conn(0x55694edb8830 0x55694f358640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 4 client.5049 mds_command: new command op to 4813 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 -- 172.21.15.50:0/1388795422 --> [v2:172.21.15.50:6834/2722641195,v1:172.21.15.50:6835/2722641195] -- command(tid 0: {"prefix": "status"}) -- 0x55694f136540 con 0x55694edb8830 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e06a00640 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.50:6834/2722641195,v1:172.21.15.50:6835/2722641195] conn(0x55694edb8830 0x55694f358640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e06a00640 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.50:6834/2722641195,v1:172.21.15.50:6835/2722641195] conn(0x55694edb8830 0x55694f358640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1601560b0bb3ef server_cookie=636138e0fa374239 in_seq=0 out_seq=0 2024-09-21T05:57:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5df3e00640 10 client.5049 ms_handle_connect on v2:172.21.15.50:6834/2722641195 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5df3e00640 1 -- 172.21.15.50:0/1388795422 <== mds.0 v2:172.21.15.50:6834/2722641195 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55694f136540 con 0x55694edb8830 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5df3e00640 10 client.5049 handle_command_reply: tid=0 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 client.5049 shutdown 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 -- 172.21.15.50:0/1388795422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 msgr2=0x55694f39c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f39c180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5e000f51e0 tx=0x7f5e000f89d0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 -- 172.21.15.50:0/1388795422 shutdown_connections 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.50:6834/2722641195,v1:172.21.15.50:6835/2722641195] conn(0x55694edb8830 0x55694f358640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55694f1d5950 0x55694f39cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55694f1beca0 0x55694f39c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 --2- 172.21.15.50:0/1388795422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55694ede5460 0x55694f39c180 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5df3e00640 0 client.5049 ms_handle_reset on v2:172.21.15.50:6834/2722641195 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 -- 172.21.15.50:0/1388795422 >> 172.21.15.50:0/1388795422 conn(0x55694f2abd90 msgr2=0x55694f3376a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 -- 172.21.15.50:0/1388795422 shutdown_connections 2024-09-21T05:57:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 1 -- 172.21.15.50:0/1388795422 wait complete. 2024-09-21T05:57:23.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e1d749000 20 client.5049 trim_cache size 0 max 16384 2024-09-21T05:57:23.069 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T05:57:23.069 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T05:57:23.069 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-21T05:57:23.069 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4813, 2024-09-21T05:57:23.069 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-21T05:57:23.069 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-21T05:57:23.069 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-21T05:57:23.069 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 6.1564422619999997, 2024-09-21T05:57:23.069 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 17, 2024-09-21T05:57:23.070 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 33, 2024-09-21T05:57:23.070 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 33, 2024-09-21T05:57:23.070 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 13.640918127999999 2024-09-21T05:57:23.070 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T05:57:23.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e16000640 1 -- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5dec073230 msgr2=0x7f5dec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e16000640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5dec073230 0x7f5dec075740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5e100af470 tx=0x7f5df8009500 comp rx=0 tx=0).stop 2024-09-21T05:57:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e16000640 1 -- 172.21.15.50:0/141735242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5df4003a10 msgr2=0x7f5e101f93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e16000640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5df4003a10 0x7f5e101f93e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5e0001b670 tx=0x7f5e00006040 comp rx=0 tx=0).stop 2024-09-21T05:57:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e16000640 1 -- 172.21.15.50:0/141735242 shutdown_connections 2024-09-21T05:57:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e16000640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e10152c20 0x7f5e10153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e16000640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e100cf370 0x7f5e10158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e16000640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5dec073230 0x7f5dec075740 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e16000640 1 --2- 172.21.15.50:0/141735242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5df4003a10 0x7f5e101f93e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e16000640 1 -- 172.21.15.50:0/141735242 >> 172.21.15.50:0/141735242 conn(0x7f5e100b2040 msgr2=0x7f5e100affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.058+0000 7f5e16000640 1 -- 172.21.15.50:0/141735242 shutdown_connections 2024-09-21T05:57:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.062+0000 7f5e16000640 1 -- 172.21.15.50:0/141735242 wait complete. 2024-09-21T05:57:23.085 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:57:23.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.238+0000 7fb808000640 1 -- 172.21.15.50:0/446697658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8000749a0 msgr2=0x7fb800074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.238+0000 7fb808000640 1 --2- 172.21.15.50:0/446697658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8000749a0 0x7fb800074db0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb8000cfc90 tx=0x7fb7f401c800 comp rx=0 tx=0).stop 2024-09-21T05:57:23.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.238+0000 7fb808000640 1 -- 172.21.15.50:0/446697658 shutdown_connections 2024-09-21T05:57:23.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.238+0000 7fb808000640 1 --2- 172.21.15.50:0/446697658 >> v2:172.21.15.88:3300/0 conn(0x7fb8000b3340 0x7fb8000b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.238+0000 7fb808000640 1 --2- 172.21.15.50:0/446697658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8000749a0 0x7fb800074db0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.238+0000 7fb808000640 1 -- 172.21.15.50:0/446697658 >> 172.21.15.50:0/446697658 conn(0x7fb8000b2040 msgr2=0x7fb8000b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:23.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.238+0000 7fb808000640 1 -- 172.21.15.50:0/446697658 shutdown_connections 2024-09-21T05:57:23.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.238+0000 7fb808000640 1 -- 172.21.15.50:0/446697658 wait complete. 2024-09-21T05:57:23.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb808000640 1 Processor -- start 2024-09-21T05:57:23.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb808000640 1 -- start start 2024-09-21T05:57:23.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb808000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8000749a0 0x7fb8000ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb808000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8000b3340 0x7fb80013d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb808000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8000cefa0 0x7fb80015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb808000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb800169cd0 con 0x7fb8000749a0 2024-09-21T05:57:23.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb808000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb800169b50 con 0x7fb8000b3340 2024-09-21T05:57:23.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb808000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb800169e50 con 0x7fb8000cefa0 2024-09-21T05:57:23.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb805600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8000749a0 0x7fb8000ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb805600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8000749a0 0x7fb8000ab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59010/0 (socket says 172.21.15.50:59010) 2024-09-21T05:57:23.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb805600640 1 -- 172.21.15.50:0/2091622393 learned_addr learned my addr 172.21.15.50:0/2091622393 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:23.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb806000640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8000cefa0 0x7fb80015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb804c00640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8000b3340 0x7fb80013d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb805600640 1 -- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8000cefa0 msgr2=0x7fb80015dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb805600640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8000cefa0 0x7fb80015dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb805600640 1 -- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8000b3340 msgr2=0x7fb80013d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb805600640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8000b3340 0x7fb80013d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb805600640 1 -- 172.21.15.50:0/2091622393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8001448f0 con 0x7fb8000749a0 2024-09-21T05:57:23.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb804c00640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8000b3340 0x7fb80013d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:23.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb805600640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8000749a0 0x7fb8000ab380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb7f401cce0 tx=0x7fb7f4005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:23.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb7fd600640 1 -- 172.21.15.50:0/2091622393 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40b2040 con 0x7fb8000749a0 2024-09-21T05:57:23.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb7fd600640 1 -- 172.21.15.50:0/2091622393 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7f4004040 con 0x7fb8000749a0 2024-09-21T05:57:23.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb808000640 1 -- 172.21.15.50:0/2091622393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb800144bd0 con 0x7fb8000749a0 2024-09-21T05:57:23.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.242+0000 7fb7fd600640 1 -- 172.21.15.50:0/2091622393 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4004490 con 0x7fb8000749a0 2024-09-21T05:57:23.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.246+0000 7fb808000640 1 -- 172.21.15.50:0/2091622393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb800145190 con 0x7fb8000749a0 2024-09-21T05:57:23.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.246+0000 7fb808000640 1 -- 172.21.15.50:0/2091622393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb800074db0 con 0x7fb8000749a0 2024-09-21T05:57:23.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.246+0000 7fb7fd600640 1 -- 172.21.15.50:0/2091622393 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb7f40bd030 con 0x7fb8000749a0 2024-09-21T05:57:23.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.246+0000 7fb7fd600640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7e0073200 0x7fb7e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:23.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.246+0000 7fb7fd600640 1 -- 172.21.15.50:0/2091622393 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb7f412dba0 con 0x7fb8000749a0 2024-09-21T05:57:23.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.246+0000 7fb804c00640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7e0073200 0x7fb7e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:23.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.250+0000 7fb804c00640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7e0073200 0x7fb7e0075710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb8000b6340 tx=0x7fb7ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:23.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.250+0000 7fb7fd600640 1 -- 172.21.15.50:0/2091622393 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb7f40c4060 con 0x7fb8000749a0 2024-09-21T05:57:23.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.466+0000 7fb808000640 1 -- 172.21.15.50:0/2091622393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb80015b900 con 0x7fb8000749a0 2024-09-21T05:57:23.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.466+0000 7fb7fd600640 1 -- 172.21.15.50:0/2091622393 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fb7f40ff030 con 0x7fb8000749a0 2024-09-21T05:57:23.472 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:57:23.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.470+0000 7fb808000640 1 -- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7e0073200 msgr2=0x7fb7e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.470+0000 7fb808000640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7e0073200 0x7fb7e0075710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb8000b6340 tx=0x7fb7ec009040 comp rx=0 tx=0).stop 2024-09-21T05:57:23.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.470+0000 7fb808000640 1 -- 172.21.15.50:0/2091622393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8000749a0 msgr2=0x7fb8000ab380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:23.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.470+0000 7fb808000640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8000749a0 0x7fb8000ab380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb7f401cce0 tx=0x7fb7f4005ca0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.470+0000 7fb808000640 1 -- 172.21.15.50:0/2091622393 shutdown_connections 2024-09-21T05:57:23.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.470+0000 7fb808000640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8000cefa0 0x7fb80015dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.470+0000 7fb808000640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8000b3340 0x7fb80013d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.470+0000 7fb808000640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7e0073200 0x7fb7e0075710 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.470+0000 7fb808000640 1 --2- 172.21.15.50:0/2091622393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8000749a0 0x7fb8000ab380 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:23.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.470+0000 7fb808000640 1 -- 172.21.15.50:0/2091622393 >> 172.21.15.50:0/2091622393 conn(0x7fb8000b2040 msgr2=0x7fb80013f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:23.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.470+0000 7fb808000640 1 -- 172.21.15.50:0/2091622393 shutdown_connections 2024-09-21T05:57:23.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23.470+0000 7fb808000640 1 -- 172.21.15.50:0/2091622393 wait complete. 2024-09-21T05:57:23.493 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:57:23.493 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:57:23.494 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-21T05:57:23.494 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:57:23.494 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:57:23.494 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:57:23.494 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:57:23.494 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:57:23.494 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:57:23.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:23.494 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-21T05:57:23.503 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:57:23.503 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:57:23.503 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:57:23.503 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:57:23.503 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T05:57:23.503 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:57:23.503 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:57:23.503 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:57:23.503 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42352sec preferred_lft 42352sec 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-21T05:57:23.504 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:8b:d0:0a:5d brd ff:ff:ff:ff:ff:ff 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T05:57:23.505 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:57:23.506 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:57:23.506 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:57:23.506 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T05:57:23.506 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:57:23.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:23.507 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:57:23.567 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-21T05:57:23.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:23.569 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-21T05:57:23.622 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-21T05:57:23.622 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:57:23.622 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-21T05:57:23.622 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T05:57:23.622 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-21T05:57:23.622 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:57:23.623 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:57:23.623 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T05:57:23.623 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:57:23.623 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:57:23.623 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:57:23.623 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T05:57:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:57:23.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:57:23.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:23.762 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T05:57:23.771 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:57:23.771 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:57:23.772 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:57:23.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:23.772 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-09-21T05:57:23.833 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:57:23.833 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:57:23.885 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:57:23.885 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:57:23.885 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:57:23.936 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:57:23.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:23.937 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:57:24.038 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:57:24.038 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:57:24.038 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T05:57:24.056 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:57:24.057 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:57:24.057 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T05:57:24.057 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T05:57:24.057 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:57:24.067 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:57:24.067 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:57:24.067 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T05:57:24.067 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T05:57:24.068 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:57:24.068 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T05:57:24.068 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T05:57:24.068 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T05:57:24.071 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:57:24.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:24.071 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:57:24.091 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:57:24.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:57:24.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:57:24.197 INFO:teuthology.orchestra.run.smithi050.stdout:client4945 2024-09-21T05:57:24.197 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:57:24.197 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:24.206 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T05:57:24.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.406+0000 7f54db600640 1 -- 172.21.15.50:0/515978763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54d40b4de0 msgr2=0x7f54d40b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:24.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.406+0000 7f54db600640 1 --2- 172.21.15.50:0/515978763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54d40b4de0 0x7f54d40b51f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f54cc009c80 tx=0x7f54cc01cce0 comp rx=0 tx=0).stop 2024-09-21T05:57:24.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.406+0000 7f54db600640 1 -- 172.21.15.50:0/515978763 shutdown_connections 2024-09-21T05:57:24.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.406+0000 7f54db600640 1 --2- 172.21.15.50:0/515978763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54d40b4de0 0x7f54d40b51f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:24.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.406+0000 7f54db600640 1 --2- 172.21.15.50:0/515978763 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54d41527b0 0x7f54d415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:24.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.406+0000 7f54db600640 1 -- 172.21.15.50:0/515978763 >> 172.21.15.50:0/515978763 conn(0x7f54d40b2040 msgr2=0x7f54d40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:24.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.406+0000 7f54db600640 1 -- 172.21.15.50:0/515978763 shutdown_connections 2024-09-21T05:57:24.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.406+0000 7f54db600640 1 -- 172.21.15.50:0/515978763 wait complete. 2024-09-21T05:57:24.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.406+0000 7f54db600640 1 Processor -- start 2024-09-21T05:57:24.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.406+0000 7f54db600640 1 -- start start 2024-09-21T05:57:24.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54db600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54d40b4de0 0x7f54d4146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54db600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54d40b5c70 0x7f54d4152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54db600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54d41527b0 0x7f54d4153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54db600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4168ab0 con 0x7f54d40b5c70 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54db600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4168930 con 0x7f54d41527b0 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54db600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f54d4168c30 con 0x7f54d40b4de0 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54cbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54d40b5c70 0x7f54d4152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54cbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54d40b5c70 0x7f54d4152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59026/0 (socket says 172.21.15.50:59026) 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54cbe00640 1 -- 172.21.15.50:0/4142018693 learned_addr learned my addr 172.21.15.50:0/4142018693 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54d8c00640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54d40b4de0 0x7f54d4146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54d9600640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54d41527b0 0x7f54d4153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54cbe00640 1 -- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54d40b4de0 msgr2=0x7f54d4146bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54cbe00640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54d40b4de0 0x7f54d4146bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:24.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54cbe00640 1 -- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54d41527b0 msgr2=0x7f54d4153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:24.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54cbe00640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54d41527b0 0x7f54d4153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:24.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54cbe00640 1 -- 172.21.15.50:0/4142018693 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d4156d40 con 0x7f54d40b5c70 2024-09-21T05:57:24.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54d9600640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54d41527b0 0x7f54d4153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:24.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54cbe00640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54d40b5c70 0x7f54d4152bc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00d720 tx=0x7f54bc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:24.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54d1600640 1 -- 172.21.15.50:0/4142018693 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc017070 con 0x7f54d40b5c70 2024-09-21T05:57:24.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54d1600640 1 -- 172.21.15.50:0/4142018693 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54bc009b10 con 0x7f54d40b5c70 2024-09-21T05:57:24.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54db600640 1 -- 172.21.15.50:0/4142018693 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d4154aa0 con 0x7f54d40b5c70 2024-09-21T05:57:24.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54d1600640 1 -- 172.21.15.50:0/4142018693 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc010460 con 0x7f54d40b5c70 2024-09-21T05:57:24.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54db600640 1 -- 172.21.15.50:0/4142018693 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d41551f0 con 0x7f54d40b5c70 2024-09-21T05:57:24.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.410+0000 7f54db600640 1 -- 172.21.15.50:0/4142018693 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d40b51f0 con 0x7f54d40b5c70 2024-09-21T05:57:24.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.414+0000 7f54d1600640 1 -- 172.21.15.50:0/4142018693 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f54bc01a070 con 0x7f54d40b5c70 2024-09-21T05:57:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.414+0000 7f54d1600640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54b0073200 0x7f54b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.414+0000 7f54d1600640 1 -- 172.21.15.50:0/4142018693 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f54bc0943c0 con 0x7f54d40b5c70 2024-09-21T05:57:24.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.414+0000 7f54d8c00640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54b0073200 0x7f54b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.414+0000 7f54d8c00640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54b0073200 0x7f54b0075710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f54c4001650 tx=0x7f54c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:24.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.418+0000 7f54d1600640 1 -- 172.21.15.50:0/4142018693 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f54bc05f3d0 con 0x7f54d40b5c70 2024-09-21T05:57:24.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:24.634+0000 7f54db600640 1 -- 172.21.15.50:0/4142018693 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f54d4154c70 con 0x7f54d40b5c70 2024-09-21T05:57:25.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.022+0000 7f54d1600640 1 -- 172.21.15.50:0/4142018693 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7f54bc05ecd0 con 0x7f54d40b5c70 2024-09-21T05:57:25.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.026+0000 7f54db600640 1 -- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54b0073200 msgr2=0x7f54b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.026+0000 7f54db600640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54b0073200 0x7f54b0075710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f54c4001650 tx=0x7f54c4009040 comp rx=0 tx=0).stop 2024-09-21T05:57:25.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.026+0000 7f54db600640 1 -- 172.21.15.50:0/4142018693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54d40b5c70 msgr2=0x7f54d4152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.026+0000 7f54db600640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54d40b5c70 0x7f54d4152bc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00d720 tx=0x7f54bc00dbe0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.026+0000 7f54db600640 1 -- 172.21.15.50:0/4142018693 shutdown_connections 2024-09-21T05:57:25.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.026+0000 7f54db600640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54d41527b0 0x7f54d4153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.026+0000 7f54db600640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54b0073200 0x7f54b0075710 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.026+0000 7f54db600640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54d40b5c70 0x7f54d4152bc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.026+0000 7f54db600640 1 --2- 172.21.15.50:0/4142018693 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54d40b4de0 0x7f54d4146bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.026+0000 7f54db600640 1 -- 172.21.15.50:0/4142018693 >> 172.21.15.50:0/4142018693 conn(0x7f54d40b2040 msgr2=0x7f54d4151960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:25.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.026+0000 7f54db600640 1 -- 172.21.15.50:0/4142018693 shutdown_connections 2024-09-21T05:57:25.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.026+0000 7f54db600640 1 -- 172.21.15.50:0/4142018693 wait complete. 2024-09-21T05:57:25.052 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T05:57:25.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.206+0000 7fd835c00640 1 -- 172.21.15.50:0/82673406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8300cd2e0 msgr2=0x7fd830070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.206+0000 7fd835c00640 1 --2- 172.21.15.50:0/82673406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8300cd2e0 0x7fd830070e80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd820009840 tx=0x7fd82001c820 comp rx=0 tx=0).stop 2024-09-21T05:57:25.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.206+0000 7fd835c00640 1 -- 172.21.15.50:0/82673406 shutdown_connections 2024-09-21T05:57:25.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.206+0000 7fd835c00640 1 --2- 172.21.15.50:0/82673406 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8300cf1b0 0x7fd8300cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.206+0000 7fd835c00640 1 --2- 172.21.15.50:0/82673406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8300cd2e0 0x7fd830070e80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.206+0000 7fd835c00640 1 -- 172.21.15.50:0/82673406 >> 172.21.15.50:0/82673406 conn(0x7fd8300b2c40 msgr2=0x7fd8300b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:25.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.206+0000 7fd835c00640 1 -- 172.21.15.50:0/82673406 shutdown_connections 2024-09-21T05:57:25.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.206+0000 7fd835c00640 1 -- 172.21.15.50:0/82673406 wait complete. 2024-09-21T05:57:25.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd835c00640 1 Processor -- start 2024-09-21T05:57:25.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd835c00640 1 -- start start 2024-09-21T05:57:25.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd835c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8300cd2e0 0x7fd83013d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:25.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd835c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8300cf1b0 0x7fd830152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:25.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd835c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8301602a0 0x7fd830150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:25.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd835c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd830169e70 con 0x7fd8300cf1b0 2024-09-21T05:57:25.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd835c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd830169cf0 con 0x7fd8300cd2e0 2024-09-21T05:57:25.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd835c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd830169ff0 con 0x7fd8301602a0 2024-09-21T05:57:25.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd827e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8300cf1b0 0x7fd830152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:25.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd827e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8300cf1b0 0x7fd830152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59058/0 (socket says 172.21.15.50:59058) 2024-09-21T05:57:25.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd827e00640 1 -- 172.21.15.50:0/2145585885 learned_addr learned my addr 172.21.15.50:0/2145585885 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:25.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd82f400640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8301602a0 0x7fd830150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:25.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd82ea00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8300cd2e0 0x7fd83013d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:25.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd827e00640 1 -- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8301602a0 msgr2=0x7fd830150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd827e00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8301602a0 0x7fd830150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd827e00640 1 -- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8300cd2e0 msgr2=0x7fd83013d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd827e00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8300cd2e0 0x7fd83013d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd827e00640 1 -- 172.21.15.50:0/2145585885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd830137c60 con 0x7fd8300cf1b0 2024-09-21T05:57:25.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd827e00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8300cf1b0 0x7fd830152670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd818001ec0 tx=0x7fd81800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:25.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd826a00640 1 -- 172.21.15.50:0/2145585885 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818017070 con 0x7fd8300cf1b0 2024-09-21T05:57:25.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd826a00640 1 -- 172.21.15.50:0/2145585885 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd818003600 con 0x7fd8300cf1b0 2024-09-21T05:57:25.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd835c00640 1 -- 172.21.15.50:0/2145585885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd830136ec0 con 0x7fd8300cf1b0 2024-09-21T05:57:25.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd826a00640 1 -- 172.21.15.50:0/2145585885 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818003a50 con 0x7fd8300cf1b0 2024-09-21T05:57:25.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.210+0000 7fd835c00640 1 -- 172.21.15.50:0/2145585885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd830137480 con 0x7fd8300cf1b0 2024-09-21T05:57:25.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.214+0000 7fd835c00640 1 -- 172.21.15.50:0/2145585885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8300cf5c0 con 0x7fd8300cf1b0 2024-09-21T05:57:25.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.214+0000 7fd826a00640 1 -- 172.21.15.50:0/2145585885 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd81801a070 con 0x7fd8300cf1b0 2024-09-21T05:57:25.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.214+0000 7fd826a00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd80c073200 0x7fd80c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:25.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.214+0000 7fd826a00640 1 -- 172.21.15.50:0/2145585885 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd818093580 con 0x7fd8300cf1b0 2024-09-21T05:57:25.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.214+0000 7fd82ea00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd80c073200 0x7fd80c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:25.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.218+0000 7fd82ea00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd80c073200 0x7fd80c075710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd8200af000 tx=0x7fd82001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:25.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.218+0000 7fd826a00640 1 -- 172.21.15.50:0/2145585885 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd81805deb0 con 0x7fd8300cf1b0 2024-09-21T05:57:25.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.418+0000 7fd835c00640 1 -- 172.21.15.50:0/2145585885 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fd8300b2540 con 0x7fd80c073200 2024-09-21T05:57:25.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.422+0000 7fd826a00640 1 -- 172.21.15.50:0/2145585885 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fd8300b2540 con 0x7fd80c073200 2024-09-21T05:57:25.427 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T05:57:25.427 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:57:25.427 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T05:57:25.427 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:57:25.427 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T05:57:25.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.422+0000 7fd835c00640 1 -- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd80c073200 msgr2=0x7fd80c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.422+0000 7fd835c00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd80c073200 0x7fd80c075710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd8200af000 tx=0x7fd82001b060 comp rx=0 tx=0).stop 2024-09-21T05:57:25.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.422+0000 7fd835c00640 1 -- 172.21.15.50:0/2145585885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8300cf1b0 msgr2=0x7fd830152670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.422+0000 7fd835c00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8300cf1b0 0x7fd830152670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd818001ec0 tx=0x7fd81800c890 comp rx=0 tx=0).stop 2024-09-21T05:57:25.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.426+0000 7fd835c00640 1 -- 172.21.15.50:0/2145585885 shutdown_connections 2024-09-21T05:57:25.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.426+0000 7fd835c00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8301602a0 0x7fd830150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.426+0000 7fd835c00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd80c073200 0x7fd80c075710 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.426+0000 7fd835c00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8300cf1b0 0x7fd830152670 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.426+0000 7fd835c00640 1 --2- 172.21.15.50:0/2145585885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8300cd2e0 0x7fd83013d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.426+0000 7fd835c00640 1 -- 172.21.15.50:0/2145585885 >> 172.21.15.50:0/2145585885 conn(0x7fd8300b2c40 msgr2=0x7fd830153030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:25.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.426+0000 7fd835c00640 1 -- 172.21.15.50:0/2145585885 shutdown_connections 2024-09-21T05:57:25.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.426+0000 7fd835c00640 1 -- 172.21.15.50:0/2145585885 wait complete. 2024-09-21T05:57:25.450 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T05:57:25.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.606+0000 7f3812c00640 1 -- 172.21.15.50:0/3920094263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f380c0b5c70 msgr2=0x7f380c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.606+0000 7f3812c00640 1 --2- 172.21.15.50:0/3920094263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f380c0b5c70 0x7f380c15bb60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009840 tx=0x7f37fc01c7f0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.606+0000 7f3812c00640 1 -- 172.21.15.50:0/3920094263 shutdown_connections 2024-09-21T05:57:25.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.606+0000 7f3812c00640 1 --2- 172.21.15.50:0/3920094263 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f380c0b4de0 0x7f380c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.606+0000 7f3812c00640 1 --2- 172.21.15.50:0/3920094263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f380c0b5c70 0x7f380c15bb60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.606+0000 7f3812c00640 1 -- 172.21.15.50:0/3920094263 >> 172.21.15.50:0/3920094263 conn(0x7f380c0b2040 msgr2=0x7f380c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:25.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.606+0000 7f3812c00640 1 -- 172.21.15.50:0/3920094263 shutdown_connections 2024-09-21T05:57:25.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.606+0000 7f3812c00640 1 -- 172.21.15.50:0/3920094263 wait complete. 2024-09-21T05:57:25.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.606+0000 7f3812c00640 1 Processor -- start 2024-09-21T05:57:25.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3812c00640 1 -- start start 2024-09-21T05:57:25.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3812c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f380c0b4de0 0x7f380c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:25.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3812c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f380c0b5c70 0x7f380c1f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:25.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3812c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f380c1527b0 0x7f380c0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3812c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f380c168970 con 0x7f380c0b4de0 2024-09-21T05:57:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3812c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f380c1687f0 con 0x7f380c0b5c70 2024-09-21T05:57:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3812c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f380c168af0 con 0x7f380c1527b0 2024-09-21T05:57:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f380be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f380c0b4de0 0x7f380c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f380be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f380c0b4de0 0x7f380c0b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59070/0 (socket says 172.21.15.50:59070) 2024-09-21T05:57:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f380be00640 1 -- 172.21.15.50:0/353116904 learned_addr learned my addr 172.21.15.50:0/353116904 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f380be00640 1 -- 172.21.15.50:0/353116904 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f380c1527b0 msgr2=0x7f380c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f380be00640 1 --2- 172.21.15.50:0/353116904 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f380c1527b0 0x7f380c0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f380be00640 1 -- 172.21.15.50:0/353116904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f380c0b5c70 msgr2=0x7f380c1f81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f380be00640 1 --2- 172.21.15.50:0/353116904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f380c0b5c70 0x7f380c1f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f380be00640 1 -- 172.21.15.50:0/353116904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c0c5350 con 0x7f380c0b4de0 2024-09-21T05:57:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f380be00640 1 --2- 172.21.15.50:0/353116904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f380c0b4de0 0x7f380c0b1750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f37fc000c10 tx=0x7f37fc005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3808c00640 1 -- 172.21.15.50:0/353116904 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0b2040 con 0x7f380c0b4de0 2024-09-21T05:57:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3808c00640 1 -- 172.21.15.50:0/353116904 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37fc004330 con 0x7f380c0b4de0 2024-09-21T05:57:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3808c00640 1 -- 172.21.15.50:0/353116904 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc003800 con 0x7f380c0b4de0 2024-09-21T05:57:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3812c00640 1 -- 172.21.15.50:0/353116904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c0c5630 con 0x7f380c0b4de0 2024-09-21T05:57:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3812c00640 1 -- 172.21.15.50:0/353116904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c152f80 con 0x7f380c0b4de0 2024-09-21T05:57:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.610+0000 7f3812c00640 1 -- 172.21.15.50:0/353116904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f380c0b51f0 con 0x7f380c0b4de0 2024-09-21T05:57:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.614+0000 7f3808c00640 1 -- 172.21.15.50:0/353116904 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f37fc0bd060 con 0x7f380c0b4de0 2024-09-21T05:57:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.614+0000 7f3808c00640 1 --2- 172.21.15.50:0/353116904 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f37e8073200 0x7f37e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.614+0000 7f3808c00640 1 -- 172.21.15.50:0/353116904 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f37fc12db00 con 0x7f380c0b4de0 2024-09-21T05:57:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.614+0000 7f3803e00640 1 --2- 172.21.15.50:0/353116904 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f37e8073200 0x7f37e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.614+0000 7f3803e00640 1 --2- 172.21.15.50:0/353116904 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f37e8073200 0x7f37e8075710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f37f40046c0 tx=0x7f37f40012c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.618+0000 7f3808c00640 1 -- 172.21.15.50:0/353116904 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f37fc016410 con 0x7f380c0b4de0 2024-09-21T05:57:25.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.794+0000 7f3812c00640 1 -- 172.21.15.50:0/353116904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f380c159a00 con 0x7f380c0b4de0 2024-09-21T05:57:25.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3808c00640 1 -- 172.21.15.50:0/353116904 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f37fc0f8b10 con 0x7f380c0b4de0 2024-09-21T05:57:25.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3812c00640 1 -- 172.21.15.50:0/353116904 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f37e8073200 msgr2=0x7f37e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3812c00640 1 --2- 172.21.15.50:0/353116904 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f37e8073200 0x7f37e8075710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f37f40046c0 tx=0x7f37f40012c0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3812c00640 1 -- 172.21.15.50:0/353116904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f380c0b4de0 msgr2=0x7f380c0b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3812c00640 1 --2- 172.21.15.50:0/353116904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f380c0b4de0 0x7f380c0b1750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f37fc000c10 tx=0x7f37fc005db0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3812c00640 1 -- 172.21.15.50:0/353116904 shutdown_connections 2024-09-21T05:57:25.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3812c00640 1 --2- 172.21.15.50:0/353116904 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f380c1527b0 0x7f380c0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3812c00640 1 --2- 172.21.15.50:0/353116904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f380c0b5c70 0x7f380c1f81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3812c00640 1 --2- 172.21.15.50:0/353116904 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f37e8073200 0x7f37e8075710 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3812c00640 1 --2- 172.21.15.50:0/353116904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f380c0b4de0 0x7f380c0b1750 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3812c00640 1 -- 172.21.15.50:0/353116904 >> 172.21.15.50:0/353116904 conn(0x7f380c0b2040 msgr2=0x7f380c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:25.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3812c00640 1 -- 172.21.15.50:0/353116904 shutdown_connections 2024-09-21T05:57:25.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.798+0000 7f3812c00640 1 -- 172.21.15.50:0/353116904 wait complete. 2024-09-21T05:57:25.825 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-21T05:57:25.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.978+0000 7fd770000640 1 -- 172.21.15.50:0/1604444848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd76815e8c0 msgr2=0x7fd76815ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.978+0000 7fd770000640 1 --2- 172.21.15.50:0/1604444848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd76815e8c0 0x7fd76815ecd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd76815cd60 tx=0x7fd75c01c8c0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.978+0000 7fd770000640 1 -- 172.21.15.50:0/1604444848 shutdown_connections 2024-09-21T05:57:25.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.978+0000 7fd770000640 1 --2- 172.21.15.50:0/1604444848 >> v2:172.21.15.88:3300/0 conn(0x7fd76815e290 0x7fd76815bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.978+0000 7fd770000640 1 --2- 172.21.15.50:0/1604444848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd76815e8c0 0x7fd76815ecd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.978+0000 7fd770000640 1 -- 172.21.15.50:0/1604444848 >> 172.21.15.50:0/1604444848 conn(0x7fd7680b2040 msgr2=0x7fd7680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:25.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.978+0000 7fd770000640 1 -- 172.21.15.50:0/1604444848 shutdown_connections 2024-09-21T05:57:25.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.978+0000 7fd770000640 1 -- 172.21.15.50:0/1604444848 wait complete. 2024-09-21T05:57:25.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.982+0000 7fd770000640 1 Processor -- start 2024-09-21T05:57:25.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.982+0000 7fd770000640 1 -- start start 2024-09-21T05:57:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.982+0000 7fd770000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd76815b2d0 0x7fd76813d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.982+0000 7fd770000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd76815e290 0x7fd7681579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.982+0000 7fd770000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd76815e8c0 0x7fd768157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.982+0000 7fd770000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd768169d70 con 0x7fd76815e8c0 2024-09-21T05:57:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.982+0000 7fd770000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd768167890 con 0x7fd76815e290 2024-09-21T05:57:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.982+0000 7fd770000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd768169ef0 con 0x7fd76815b2d0 2024-09-21T05:57:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.982+0000 7fd76e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd76815e8c0 0x7fd768157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.982+0000 7fd76e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd76815e8c0 0x7fd768157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59092/0 (socket says 172.21.15.50:59092) 2024-09-21T05:57:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.982+0000 7fd76e000640 1 -- 172.21.15.50:0/1634552655 learned_addr learned my addr 172.21.15.50:0/1634552655 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd76d600640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd76815b2d0 0x7fd76813d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:25.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd76cc00640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd76815e290 0x7fd7681579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:25.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd76e000640 1 -- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd76815b2d0 msgr2=0x7fd76813d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd76e000640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd76815b2d0 0x7fd76813d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd76e000640 1 -- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd76815e290 msgr2=0x7fd7681579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:25.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd76e000640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd76815e290 0x7fd7681579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:25.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd76e000640 1 -- 172.21.15.50:0/1634552655 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd768164a20 con 0x7fd76815e8c0 2024-09-21T05:57:25.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd76e000640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd76815e8c0 0x7fd768157ee0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd76000cab0 tx=0x7fd76000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:25.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd765600640 1 -- 172.21.15.50:0/1634552655 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76000ae20 con 0x7fd76815e8c0 2024-09-21T05:57:25.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd765600640 1 -- 172.21.15.50:0/1634552655 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd76000f560 con 0x7fd76815e8c0 2024-09-21T05:57:25.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd765600640 1 -- 172.21.15.50:0/1634552655 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76000f9b0 con 0x7fd76815e8c0 2024-09-21T05:57:25.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd770000640 1 -- 172.21.15.50:0/1634552655 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd768167c00 con 0x7fd76815e8c0 2024-09-21T05:57:25.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd770000640 1 -- 172.21.15.50:0/1634552655 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7681697e0 con 0x7fd76815e8c0 2024-09-21T05:57:25.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd770000640 1 -- 172.21.15.50:0/1634552655 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76815ecd0 con 0x7fd76815e8c0 2024-09-21T05:57:25.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.986+0000 7fd765600640 1 -- 172.21.15.50:0/1634552655 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd76000fb50 con 0x7fd76815e8c0 2024-09-21T05:57:25.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.990+0000 7fd765600640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd748073200 0x7fd748075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:25.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.990+0000 7fd765600640 1 -- 172.21.15.50:0/1634552655 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd760094c60 con 0x7fd76815e8c0 2024-09-21T05:57:25.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.990+0000 7fd76d600640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd748073200 0x7fd748075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:25.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.990+0000 7fd76d600640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd748073200 0x7fd748075710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd75c0af000 tx=0x7fd75c01b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:25.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:25.994+0000 7fd765600640 1 -- 172.21.15.50:0/1634552655 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd760014030 con 0x7fd76815e8c0 2024-09-21T05:57:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 -- 172.21.15.50:0/1634552655 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fd768169a50 con 0x7fd76815e8c0 2024-09-21T05:57:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd765600640 1 -- 172.21.15.50:0/1634552655 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v32) ==== 116+0+2 (secure 0 0 0) 0x7fd76005f620 con 0x7fd76815e8c0 2024-09-21T05:57:26.179 INFO:teuthology.orchestra.run.smithi050.stdout:4 2024-09-21T05:57:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 -- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd748073200 msgr2=0x7fd748075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd748073200 0x7fd748075710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd75c0af000 tx=0x7fd75c01b100 comp rx=0 tx=0).stop 2024-09-21T05:57:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 -- 172.21.15.50:0/1634552655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd76815e8c0 msgr2=0x7fd768157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd76815e8c0 0x7fd768157ee0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd76000cab0 tx=0x7fd76000cf70 comp rx=0 tx=0).stop 2024-09-21T05:57:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 -- 172.21.15.50:0/1634552655 shutdown_connections 2024-09-21T05:57:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd748073200 0x7fd748075710 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd76815e8c0 0x7fd768157ee0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd76815e290 0x7fd7681579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 --2- 172.21.15.50:0/1634552655 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd76815b2d0 0x7fd76813d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 -- 172.21.15.50:0/1634552655 >> 172.21.15.50:0/1634552655 conn(0x7fd7680b2040 msgr2=0x7fd768165ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 -- 172.21.15.50:0/1634552655 shutdown_connections 2024-09-21T05:57:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.174+0000 7fd770000640 1 -- 172.21.15.50:0/1634552655 wait complete. 2024-09-21T05:57:26.201 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T05:57:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.358+0000 7f9631200640 1 -- 172.21.15.50:0/1345108805 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f962c15aa50 msgr2=0x7f962c0cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.358+0000 7f9631200640 1 --2- 172.21.15.50:0/1345108805 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f962c15aa50 0x7f962c0cd260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f961c00a980 tx=0x7f961c01db90 comp rx=0 tx=0).stop 2024-09-21T05:57:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.358+0000 7f9631200640 1 -- 172.21.15.50:0/1345108805 shutdown_connections 2024-09-21T05:57:26.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.358+0000 7f9631200640 1 --2- 172.21.15.50:0/1345108805 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f962c15aa50 0x7f962c0cd260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.358+0000 7f9631200640 1 -- 172.21.15.50:0/1345108805 >> 172.21.15.50:0/1345108805 conn(0x7f962c0b2040 msgr2=0x7f962c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:26.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.358+0000 7f9631200640 1 -- 172.21.15.50:0/1345108805 shutdown_connections 2024-09-21T05:57:26.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.358+0000 7f9631200640 1 -- 172.21.15.50:0/1345108805 wait complete. 2024-09-21T05:57:26.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9631200640 1 Processor -- start 2024-09-21T05:57:26.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9631200640 1 -- start start 2024-09-21T05:57:26.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9631200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f962c0cd7a0 0x7f962c0b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:26.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9631200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f962c0cdbb0 0x7f962c1fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:26.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9631200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c15aa50 0x7f962c142a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:26.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9631200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1690a0 con 0x7f962c0cd7a0 2024-09-21T05:57:26.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9631200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f962c0b5f50 con 0x7f962c15aa50 2024-09-21T05:57:26.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9631200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f962c169220 con 0x7f962c0cdbb0 2024-09-21T05:57:26.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9629e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f962c0cd7a0 0x7f962c0b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:26.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9629e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f962c0cd7a0 0x7f962c0b1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59108/0 (socket says 172.21.15.50:59108) 2024-09-21T05:57:26.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9629e00640 1 -- 172.21.15.50:0/3602924754 learned_addr learned my addr 172.21.15.50:0/3602924754 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:26.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9621400640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f962c0cdbb0 0x7f962c1fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:26.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f962a800640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c15aa50 0x7f962c142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:26.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9629e00640 1 -- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f962c0cdbb0 msgr2=0x7f962c1fc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:26.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9629e00640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f962c0cdbb0 0x7f962c1fc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9629e00640 1 -- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c15aa50 msgr2=0x7f962c142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:26.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9629e00640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c15aa50 0x7f962c142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9629e00640 1 -- 172.21.15.50:0/3602924754 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c1fd0b0 con 0x7f962c0cd7a0 2024-09-21T05:57:26.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9629e00640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f962c0cd7a0 0x7f962c0b1140 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f961c01b7c0 tx=0x7f961c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9622a00640 1 -- 172.21.15.50:0/3602924754 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c0bd070 con 0x7f962c0cd7a0 2024-09-21T05:57:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9622a00640 1 -- 172.21.15.50:0/3602924754 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f961c005320 con 0x7f962c0cd7a0 2024-09-21T05:57:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9631200640 1 -- 172.21.15.50:0/3602924754 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962c1fcaf0 con 0x7f962c0cd7a0 2024-09-21T05:57:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9622a00640 1 -- 172.21.15.50:0/3602924754 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c0b3040 con 0x7f962c0cd7a0 2024-09-21T05:57:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9631200640 1 -- 172.21.15.50:0/3602924754 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962c1548c0 con 0x7f962c0cd7a0 2024-09-21T05:57:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.362+0000 7f9631200640 1 -- 172.21.15.50:0/3602924754 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f962c0cc9a0 con 0x7f962c0cd7a0 2024-09-21T05:57:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.366+0000 7f9622a00640 1 -- 172.21.15.50:0/3602924754 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f961c0046d0 con 0x7f962c0cd7a0 2024-09-21T05:57:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.366+0000 7f9622a00640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9608073230 0x7f9608075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.366+0000 7f9622a00640 1 -- 172.21.15.50:0/3602924754 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f961c12df80 con 0x7f962c0cd7a0 2024-09-21T05:57:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.366+0000 7f9621400640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9608073230 0x7f9608075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.366+0000 7f9621400640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9608073230 0x7f9608075740 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f962c159fa0 tx=0x7f9614009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.370+0000 7f9622a00640 1 -- 172.21.15.50:0/3602924754 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f961c017780 con 0x7f962c0cd7a0 2024-09-21T05:57:26.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.550+0000 7f9631200640 1 -- 172.21.15.50:0/3602924754 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f962c0cc6c0 con 0x7f962c0cd7a0 2024-09-21T05:57:26.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9622a00640 1 -- 172.21.15.50:0/3602924754 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v33)=0 v33) ==== 131+0+0 (secure 0 0 0) 0x7f961c0f8f00 con 0x7f962c0cd7a0 2024-09-21T05:57:26.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9631200640 1 -- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9608073230 msgr2=0x7f9608075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:26.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9631200640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9608073230 0x7f9608075740 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f962c159fa0 tx=0x7f9614009040 comp rx=0 tx=0).stop 2024-09-21T05:57:26.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9631200640 1 -- 172.21.15.50:0/3602924754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f962c0cd7a0 msgr2=0x7f962c0b1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:26.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9631200640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f962c0cd7a0 0x7f962c0b1140 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f961c01b7c0 tx=0x7f961c004600 comp rx=0 tx=0).stop 2024-09-21T05:57:26.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9631200640 1 -- 172.21.15.50:0/3602924754 shutdown_connections 2024-09-21T05:57:26.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9631200640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c15aa50 0x7f962c142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9631200640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f962c0cdbb0 0x7f962c1fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9631200640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9608073230 0x7f9608075740 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9631200640 1 --2- 172.21.15.50:0/3602924754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f962c0cd7a0 0x7f962c0b1140 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9631200640 1 -- 172.21.15.50:0/3602924754 >> 172.21.15.50:0/3602924754 conn(0x7f962c0b2040 msgr2=0x7f962c153c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:26.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9631200640 1 -- 172.21.15.50:0/3602924754 shutdown_connections 2024-09-21T05:57:26.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.554+0000 7f9631200640 1 -- 172.21.15.50:0/3602924754 wait complete. 2024-09-21T05:57:26.581 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group1 2024-09-21T05:57:26.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.738+0000 7fddbde00640 1 -- 172.21.15.50:0/3154214901 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fddb80b5c70 msgr2=0x7fddb815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:26.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.738+0000 7fddbde00640 1 --2- 172.21.15.50:0/3154214901 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fddb80b5c70 0x7fddb815bb60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009870 tx=0x7fdda801c6c0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.738+0000 7fddbde00640 1 -- 172.21.15.50:0/3154214901 shutdown_connections 2024-09-21T05:57:26.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.738+0000 7fddbde00640 1 --2- 172.21.15.50:0/3154214901 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fddb80b5c70 0x7fddb815bb60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.738+0000 7fddbde00640 1 -- 172.21.15.50:0/3154214901 >> 172.21.15.50:0/3154214901 conn(0x7fddb80b2040 msgr2=0x7fddb80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:26.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.738+0000 7fddbde00640 1 -- 172.21.15.50:0/3154214901 shutdown_connections 2024-09-21T05:57:26.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.738+0000 7fddbde00640 1 -- 172.21.15.50:0/3154214901 wait complete. 2024-09-21T05:57:26.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddbde00640 1 Processor -- start 2024-09-21T05:57:26.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddbde00640 1 -- start start 2024-09-21T05:57:26.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddbde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddb80b4de0 0x7fddb814f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:26.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddbde00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fddb80b5c70 0x7fddb81f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:26.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddbde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb81527b0 0x7fddb81593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:26.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddbde00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8169cb0 con 0x7fddb81527b0 2024-09-21T05:57:26.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddbde00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8169b30 con 0x7fddb80b4de0 2024-09-21T05:57:26.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddbde00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fddb8169e30 con 0x7fddb80b5c70 2024-09-21T05:57:26.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddb7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb81527b0 0x7fddb81593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:26.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddb6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddb80b4de0 0x7fddb814f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:26.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddb7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb81527b0 0x7fddb81593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45452/0 (socket says 172.21.15.50:45452) 2024-09-21T05:57:26.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddb7400640 1 -- 172.21.15.50:0/4217669324 learned_addr learned my addr 172.21.15.50:0/4217669324 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:26.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddafe00640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fddb80b5c70 0x7fddb81f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:26.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddb7400640 1 -- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fddb80b5c70 msgr2=0x7fddb81f86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:26.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddb7400640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fddb80b5c70 0x7fddb81f86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddb7400640 1 -- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddb80b4de0 msgr2=0x7fddb814f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:26.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddb7400640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddb80b4de0 0x7fddb814f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.742+0000 7fddb7400640 1 -- 172.21.15.50:0/4217669324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb8159930 con 0x7fddb81527b0 2024-09-21T05:57:26.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.746+0000 7fddb7400640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb81527b0 0x7fddb81593f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c700 tx=0x7fddb000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:26.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.746+0000 7fddaf400640 1 -- 172.21.15.50:0/4217669324 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0003b80 con 0x7fddb81527b0 2024-09-21T05:57:26.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.746+0000 7fddaf400640 1 -- 172.21.15.50:0/4217669324 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddb0003d20 con 0x7fddb81527b0 2024-09-21T05:57:26.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.746+0000 7fddbde00640 1 -- 172.21.15.50:0/4217669324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb81f8f80 con 0x7fddb81527b0 2024-09-21T05:57:26.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.746+0000 7fddaf400640 1 -- 172.21.15.50:0/4217669324 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0005330 con 0x7fddb81527b0 2024-09-21T05:57:26.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.746+0000 7fddbde00640 1 -- 172.21.15.50:0/4217669324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb81f9540 con 0x7fddb81527b0 2024-09-21T05:57:26.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.746+0000 7fddbde00640 1 -- 172.21.15.50:0/4217669324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb815bb60 con 0x7fddb81527b0 2024-09-21T05:57:26.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.746+0000 7fddaf400640 1 -- 172.21.15.50:0/4217669324 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fddb00054d0 con 0x7fddb81527b0 2024-09-21T05:57:26.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.746+0000 7fddaf400640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdd94073230 0x7fdd94075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:26.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.746+0000 7fddaf400640 1 -- 172.21.15.50:0/4217669324 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fddb0093ed0 con 0x7fddb81527b0 2024-09-21T05:57:26.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.750+0000 7fddb6a00640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdd94073230 0x7fdd94075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:26.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.750+0000 7fddb6a00640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdd94073230 0x7fdd94075740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdda80af000 tx=0x7fdda801af00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:26.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.754+0000 7fddaf400640 1 -- 172.21.15.50:0/4217669324 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fddb005e800 con 0x7fddb81527b0 2024-09-21T05:57:26.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.954+0000 7fddbde00640 1 -- 172.21.15.50:0/4217669324 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7fddb806b540 con 0x7fdd94073230 2024-09-21T05:57:26.966 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:57:26.959+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:57:26.966 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:57:26.959+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:57:26.966 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:57:26.959+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:57:26.966 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:57:26.959+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:57:26.966 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:57:26.959+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:57:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.978+0000 7fddaf400640 1 -- 172.21.15.50:0/4217669324 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdda80040c0 con 0x7fdd94073230 2024-09-21T05:57:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.982+0000 7fddbde00640 1 -- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdd94073230 msgr2=0x7fdd94075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.982+0000 7fddbde00640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdd94073230 0x7fdd94075740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdda80af000 tx=0x7fdda801af00 comp rx=0 tx=0).stop 2024-09-21T05:57:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.982+0000 7fddbde00640 1 -- 172.21.15.50:0/4217669324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb81527b0 msgr2=0x7fddb81593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.982+0000 7fddbde00640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb81527b0 0x7fddb81593f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c700 tx=0x7fddb000cbc0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.982+0000 7fddbde00640 1 -- 172.21.15.50:0/4217669324 shutdown_connections 2024-09-21T05:57:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.982+0000 7fddbde00640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdd94073230 0x7fdd94075740 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.982+0000 7fddbde00640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb81527b0 0x7fddb81593f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.982+0000 7fddbde00640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fddb80b5c70 0x7fddb81f86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.982+0000 7fddbde00640 1 --2- 172.21.15.50:0/4217669324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddb80b4de0 0x7fddb814f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.982+0000 7fddbde00640 1 -- 172.21.15.50:0/4217669324 >> 172.21.15.50:0/4217669324 conn(0x7fddb80b2040 msgr2=0x7fddb80cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.982+0000 7fddbde00640 1 -- 172.21.15.50:0/4217669324 shutdown_connections 2024-09-21T05:57:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:26.982+0000 7fddbde00640 1 -- 172.21.15.50:0/4217669324 wait complete. 2024-09-21T05:57:27.008 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 group1 --mode=777 2024-09-21T05:57:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.166+0000 7fa2a1e00640 1 -- 172.21.15.50:0/3070412156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa29c0cefa0 msgr2=0x7fa29c0cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.166+0000 7fa2a1e00640 1 --2- 172.21.15.50:0/3070412156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa29c0cefa0 0x7fa29c0cf3b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0cfc90 tx=0x7fa28401c800 comp rx=0 tx=0).stop 2024-09-21T05:57:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.166+0000 7fa2a1e00640 1 -- 172.21.15.50:0/3070412156 shutdown_connections 2024-09-21T05:57:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.166+0000 7fa2a1e00640 1 --2- 172.21.15.50:0/3070412156 >> v2:172.21.15.88:3300/0 conn(0x7fa29c0b3340 0x7fa29c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.166+0000 7fa2a1e00640 1 --2- 172.21.15.50:0/3070412156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa29c0cefa0 0x7fa29c0cf3b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.166+0000 7fa2a1e00640 1 -- 172.21.15.50:0/3070412156 >> 172.21.15.50:0/3070412156 conn(0x7fa29c0b2040 msgr2=0x7fa29c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:27.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.166+0000 7fa2a1e00640 1 -- 172.21.15.50:0/3070412156 shutdown_connections 2024-09-21T05:57:27.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.166+0000 7fa2a1e00640 1 -- 172.21.15.50:0/3070412156 wait complete. 2024-09-21T05:57:27.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa2a1e00640 1 Processor -- start 2024-09-21T05:57:27.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa2a1e00640 1 -- start start 2024-09-21T05:57:27.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa2a1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c0749a0 0x7fa29c160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:27.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa2a1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa29c0b3340 0x7fa29c141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:27.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa2a1e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa29c0cefa0 0x7fa29c141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:27.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa2a1e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c169d70 con 0x7fa29c0b3340 2024-09-21T05:57:27.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa2a1e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c169bf0 con 0x7fa29c0749a0 2024-09-21T05:57:27.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa2a1e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa29c169ef0 con 0x7fa29c0cefa0 2024-09-21T05:57:27.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa293e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa29c0b3340 0x7fa29c141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:27.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa293e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa29c0b3340 0x7fa29c141830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45474/0 (socket says 172.21.15.50:45474) 2024-09-21T05:57:27.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa293e00640 1 -- 172.21.15.50:0/298737870 learned_addr learned my addr 172.21.15.50:0/298737870 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:27.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa29b400640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa29c0cefa0 0x7fa29c141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:27.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa29aa00640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c0749a0 0x7fa29c160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:27.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa293e00640 1 -- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa29c0cefa0 msgr2=0x7fa29c141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:27.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa293e00640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa29c0cefa0 0x7fa29c141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa293e00640 1 -- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c0749a0 msgr2=0x7fa29c160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:27.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa293e00640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c0749a0 0x7fa29c160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa293e00640 1 -- 172.21.15.50:0/298737870 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c143840 con 0x7fa29c0b3340 2024-09-21T05:57:27.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa293e00640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa29c0b3340 0x7fa29c141830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa284000c10 tx=0x7fa284005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:27.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa293400640 1 -- 172.21.15.50:0/298737870 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2840b2040 con 0x7fa29c0b3340 2024-09-21T05:57:27.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa293400640 1 -- 172.21.15.50:0/298737870 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2840043e0 con 0x7fa29c0b3340 2024-09-21T05:57:27.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.170+0000 7fa293400640 1 -- 172.21.15.50:0/298737870 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284005270 con 0x7fa29c0b3340 2024-09-21T05:57:27.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.174+0000 7fa2a1e00640 1 -- 172.21.15.50:0/298737870 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29c1408f0 con 0x7fa29c0b3340 2024-09-21T05:57:27.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.174+0000 7fa2a1e00640 1 -- 172.21.15.50:0/298737870 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa29c13e290 con 0x7fa29c0b3340 2024-09-21T05:57:27.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.174+0000 7fa2a1e00640 1 -- 172.21.15.50:0/298737870 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa29c140b60 con 0x7fa29c0b3340 2024-09-21T05:57:27.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.174+0000 7fa293400640 1 -- 172.21.15.50:0/298737870 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa284005410 con 0x7fa29c0b3340 2024-09-21T05:57:27.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.174+0000 7fa293400640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa278073200 0x7fa278075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:27.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.174+0000 7fa293400640 1 -- 172.21.15.50:0/298737870 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa28412dcb0 con 0x7fa29c0b3340 2024-09-21T05:57:27.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.174+0000 7fa29aa00640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa278073200 0x7fa278075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.178+0000 7fa29aa00640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa278073200 0x7fa278075710 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa28c001640 tx=0x7fa28c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.178+0000 7fa293400640 1 -- 172.21.15.50:0/298737870 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa2840bd060 con 0x7fa29c0b3340 2024-09-21T05:57:27.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.386+0000 7fa2a1e00640 1 -- 172.21.15.50:0/298737870 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fa29c06b540 con 0x7fa278073200 2024-09-21T05:57:27.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa293400640 1 -- 172.21.15.50:0/298737870 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa28c005020 con 0x7fa278073200 2024-09-21T05:57:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa2a1e00640 1 -- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa278073200 msgr2=0x7fa278075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa2a1e00640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa278073200 0x7fa278075710 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa28c001640 tx=0x7fa28c009040 comp rx=0 tx=0).stop 2024-09-21T05:57:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa2a1e00640 1 -- 172.21.15.50:0/298737870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa29c0b3340 msgr2=0x7fa29c141830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa2a1e00640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa29c0b3340 0x7fa29c141830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa284000c10 tx=0x7fa284005dd0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa2a1e00640 1 -- 172.21.15.50:0/298737870 shutdown_connections 2024-09-21T05:57:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa2a1e00640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa29c0cefa0 0x7fa29c141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa2a1e00640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa278073200 0x7fa278075710 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa2a1e00640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa29c0b3340 0x7fa29c141830 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa2a1e00640 1 --2- 172.21.15.50:0/298737870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c0749a0 0x7fa29c160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa2a1e00640 1 -- 172.21.15.50:0/298737870 >> 172.21.15.50:0/298737870 conn(0x7fa29c0b2040 msgr2=0x7fa29c074db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa2a1e00640 1 -- 172.21.15.50:0/298737870 shutdown_connections 2024-09-21T05:57:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.518+0000 7fa2a1e00640 1 -- 172.21.15.50:0/298737870 wait complete. 2024-09-21T05:57:27.545 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-21T05:57:27.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.702+0000 7f99a9c00640 1 -- 172.21.15.50:0/3592815008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a415f5b0 msgr2=0x7f99a40b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:27.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.702+0000 7f99a9c00640 1 --2- 172.21.15.50:0/3592815008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a415f5b0 0x7f99a40b1dc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f998c009840 tx=0x7f998c01c720 comp rx=0 tx=0).stop 2024-09-21T05:57:27.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.702+0000 7f99a9c00640 1 -- 172.21.15.50:0/3592815008 shutdown_connections 2024-09-21T05:57:27.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.702+0000 7f99a9c00640 1 --2- 172.21.15.50:0/3592815008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a415f5b0 0x7f99a40b1dc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.702+0000 7f99a9c00640 1 --2- 172.21.15.50:0/3592815008 >> v2:172.21.15.88:3300/0 conn(0x7f99a4155a00 0x7f99a415f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.702+0000 7f99a9c00640 1 -- 172.21.15.50:0/3592815008 >> 172.21.15.50:0/3592815008 conn(0x7f99a406cdf0 msgr2=0x7f99a406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:27.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.702+0000 7f99a9c00640 1 -- 172.21.15.50:0/3592815008 shutdown_connections 2024-09-21T05:57:27.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.702+0000 7f99a9c00640 1 -- 172.21.15.50:0/3592815008 wait complete. 2024-09-21T05:57:27.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f99a9c00640 1 Processor -- start 2024-09-21T05:57:27.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f99a9c00640 1 -- start start 2024-09-21T05:57:27.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f99a9c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f99a40b2300 0x7f99a414cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:27.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f99a9c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a4155a00 0x7f99a41f9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:27.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f99a9c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99a415f5b0 0x7f99a41f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f99a9c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4168810 con 0x7f99a4155a00 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f99a9c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4168690 con 0x7f99a415f5b0 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f99a9c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f99a4168990 con 0x7f99a40b2300 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f999be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a4155a00 0x7f99a41f9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f999be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a4155a00 0x7f99a41f9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45496/0 (socket says 172.21.15.50:45496) 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f999be00640 1 -- 172.21.15.50:0/2906902806 learned_addr learned my addr 172.21.15.50:0/2906902806 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f99a3e00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f99a40b2300 0x7f99a414cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.706+0000 7f99a9200640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99a415f5b0 0x7f99a41f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f999be00640 1 -- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f99a40b2300 msgr2=0x7f99a414cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f999be00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f99a40b2300 0x7f99a414cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f999be00640 1 -- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99a415f5b0 msgr2=0x7f99a41f9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f999be00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99a415f5b0 0x7f99a41f9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f999be00640 1 -- 172.21.15.50:0/2906902806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a414deb0 con 0x7f99a4155a00 2024-09-21T05:57:27.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f99a3e00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f99a40b2300 0x7f99a414cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:27.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f999be00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a4155a00 0x7f99a41f9560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f998c009790 tx=0x7f998c0a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:27.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f99a0c00640 1 -- 172.21.15.50:0/2906902806 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c0b2040 con 0x7f99a4155a00 2024-09-21T05:57:27.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f99a9c00640 1 -- 172.21.15.50:0/2906902806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a414e190 con 0x7f99a4155a00 2024-09-21T05:57:27.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f99a0c00640 1 -- 172.21.15.50:0/2906902806 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f998c005030 con 0x7f99a4155a00 2024-09-21T05:57:27.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f99a0c00640 1 -- 172.21.15.50:0/2906902806 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c0044f0 con 0x7f99a4155a00 2024-09-21T05:57:27.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f99a9c00640 1 -- 172.21.15.50:0/2906902806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a414e750 con 0x7f99a4155a00 2024-09-21T05:57:27.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f99a9c00640 1 -- 172.21.15.50:0/2906902806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a415f070 con 0x7f99a4155a00 2024-09-21T05:57:27.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f99a0c00640 1 -- 172.21.15.50:0/2906902806 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f998c005460 con 0x7f99a4155a00 2024-09-21T05:57:27.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.710+0000 7f99a0c00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9980073200 0x7f9980075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:27.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.714+0000 7f99a0c00640 1 -- 172.21.15.50:0/2906902806 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f998c12df00 con 0x7f99a4155a00 2024-09-21T05:57:27.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.714+0000 7f99a3e00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9980073200 0x7f9980075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:27.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.714+0000 7f99a3e00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9980073200 0x7f9980075710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f999400bbd0 tx=0x7f999400b5b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:27.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.718+0000 7f99a0c00640 1 -- 172.21.15.50:0/2906902806 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f998c0163d0 con 0x7f99a4155a00 2024-09-21T05:57:27.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.922+0000 7f99a9c00640 1 -- 172.21.15.50:0/2906902806 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f99a406b540 con 0x7f9980073200 2024-09-21T05:57:27.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a0c00640 1 -- 172.21.15.50:0/2906902806 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7f999400fb90 con 0x7f9980073200 2024-09-21T05:57:27.934 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b 2024-09-21T05:57:27.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a9c00640 1 -- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9980073200 msgr2=0x7f9980075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:27.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a9c00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9980073200 0x7f9980075710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f999400bbd0 tx=0x7f999400b5b0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a9c00640 1 -- 172.21.15.50:0/2906902806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a4155a00 msgr2=0x7f99a41f9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:27.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a9c00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a4155a00 0x7f99a41f9560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f998c009790 tx=0x7f998c0a56f0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a9c00640 1 -- 172.21.15.50:0/2906902806 shutdown_connections 2024-09-21T05:57:27.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a9c00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99a415f5b0 0x7f99a41f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a9c00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9980073200 0x7f9980075710 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a9c00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a4155a00 0x7f99a41f9560 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a9c00640 1 --2- 172.21.15.50:0/2906902806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f99a40b2300 0x7f99a414cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:27.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a9c00640 1 -- 172.21.15.50:0/2906902806 >> 172.21.15.50:0/2906902806 conn(0x7f99a406cdf0 msgr2=0x7f99a41503d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:27.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a9c00640 1 -- 172.21.15.50:0/2906902806 shutdown_connections 2024-09-21T05:57:27.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:57:27.930+0000 7f99a9c00640 1 -- 172.21.15.50:0/2906902806 wait complete. 2024-09-21T05:57:27.956 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b 2024-09-21T05:57:27.957 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:57:27.957 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:27.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:27.966 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T05:57:37.571 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T05:57:37.571 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T05:57:37.571 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.54673 s, 112 MB/s 2024-09-21T05:57:37.572 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:57:37.572 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:37.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:37.581 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T05:57:46.785 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:57:46.790 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:57:47.386 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T05:57:47.386 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T05:57:47.386 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.75142 s, 110 MB/s 2024-09-21T05:57:47.388 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:57:47.388 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:47.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:47.397 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T05:57:56.883 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T05:57:56.883 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T05:57:56.884 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.43302 s, 114 MB/s 2024-09-21T05:57:56.885 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:57:56.885 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:56.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:56.893 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T05:58:06.526 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T05:58:06.526 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T05:58:06.527 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.57794 s, 112 MB/s 2024-09-21T05:58:06.528 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:58:06.528 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:58:06.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:58:06.537 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T05:58:16.330 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T05:58:16.330 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T05:58:16.330 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.74009 s, 110 MB/s 2024-09-21T05:58:16.331 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:58:16.331 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:58:16.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:58:16.341 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T05:58:16.855 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:58:16.859 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:58:26.441 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T05:58:26.441 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T05:58:26.441 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.0508 s, 107 MB/s 2024-09-21T05:58:26.442 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:58:26.442 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:58:26.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:58:26.451 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T05:58:36.418 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T05:58:36.419 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T05:58:36.419 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.91572 s, 108 MB/s 2024-09-21T05:58:36.420 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:58:36.420 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:58:36.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:58:36.429 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T05:58:46.160 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T05:58:46.160 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T05:58:46.160 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.68151 s, 111 MB/s 2024-09-21T05:58:46.161 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:58:46.161 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:58:46.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:58:46.170 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T05:58:46.909 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:58:46.912 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:58:56.110 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T05:58:56.111 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T05:58:56.111 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.88786 s, 109 MB/s 2024-09-21T05:58:56.112 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:58:56.112 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:58:56.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:58:56.122 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T05:59:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T05:59:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T05:59:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.88716 s, 109 MB/s 2024-09-21T05:59:06.062 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 group1 2024-09-21T05:59:06.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.228+0000 7f3053c00640 1 -- 172.21.15.50:0/3946753798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f304c069330 msgr2=0x7f304c06cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.228+0000 7f3053c00640 1 --2- 172.21.15.50:0/3946753798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f304c069330 0x7f304c06cdf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3034009840 tx=0x7f303401c7f0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.228+0000 7f3053c00640 1 -- 172.21.15.50:0/3946753798 shutdown_connections 2024-09-21T05:59:06.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.228+0000 7f3053c00640 1 --2- 172.21.15.50:0/3946753798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f304c069330 0x7f304c06cdf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.228+0000 7f3053c00640 1 --2- 172.21.15.50:0/3946753798 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f304c160c20 0x7f304c068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.228+0000 7f3053c00640 1 -- 172.21.15.50:0/3946753798 >> 172.21.15.50:0/3946753798 conn(0x7f304c0b35f0 msgr2=0x7f304c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:06.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.228+0000 7f3053c00640 1 -- 172.21.15.50:0/3946753798 shutdown_connections 2024-09-21T05:59:06.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.228+0000 7f3053c00640 1 -- 172.21.15.50:0/3946753798 wait complete. 2024-09-21T05:59:06.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.228+0000 7f3053c00640 1 Processor -- start 2024-09-21T05:59:06.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.228+0000 7f3053c00640 1 -- start start 2024-09-21T05:59:06.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3053c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f304c069330 0x7f304c1f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3053c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f304c06d330 0x7f304c1fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3053c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f304c160c20 0x7f304c0b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3053c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f304c0d1040 con 0x7f304c160c20 2024-09-21T05:59:06.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3053c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f304c0d0ec0 con 0x7f304c06d330 2024-09-21T05:59:06.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3053c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f304c153c50 con 0x7f304c069330 2024-09-21T05:59:06.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3051c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f304c160c20 0x7f304c0b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3051c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f304c160c20 0x7f304c0b2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36444/0 (socket says 172.21.15.50:36444) 2024-09-21T05:59:06.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3051c00640 1 -- 172.21.15.50:0/3383097670 learned_addr learned my addr 172.21.15.50:0/3383097670 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:06.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3051200640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f304c069330 0x7f304c1f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3043e00640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f304c06d330 0x7f304c1fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3051c00640 1 -- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f304c069330 msgr2=0x7f304c1f8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3051c00640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f304c069330 0x7f304c1f8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3051c00640 1 -- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f304c06d330 msgr2=0x7f304c1fbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3051c00640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f304c06d330 0x7f304c1fbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3051c00640 1 -- 172.21.15.50:0/3383097670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c14b220 con 0x7f304c160c20 2024-09-21T05:59:06.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3043e00640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f304c06d330 0x7f304c1fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:59:06.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3051c00640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f304c160c20 0x7f304c0b2260 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f304400cb10 tx=0x7f304400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:06.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3049600640 1 -- 172.21.15.50:0/3383097670 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044016020 con 0x7f304c160c20 2024-09-21T05:59:06.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3049600640 1 -- 172.21.15.50:0/3383097670 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f304400ce50 con 0x7f304c160c20 2024-09-21T05:59:06.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3049600640 1 -- 172.21.15.50:0/3383097670 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304400ae00 con 0x7f304c160c20 2024-09-21T05:59:06.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3053c00640 1 -- 172.21.15.50:0/3383097670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c150810 con 0x7f304c160c20 2024-09-21T05:59:06.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3053c00640 1 -- 172.21.15.50:0/3383097670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c150530 con 0x7f304c160c20 2024-09-21T05:59:06.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.232+0000 7f3053c00640 1 -- 172.21.15.50:0/3383097670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304c06cdf0 con 0x7f304c160c20 2024-09-21T05:59:06.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.236+0000 7f3049600640 1 -- 172.21.15.50:0/3383097670 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 7) ==== 97234+0+0 (secure 0 0 0) 0x7f3044019020 con 0x7f304c160c20 2024-09-21T05:59:06.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.236+0000 7f3049600640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3028073200 0x7f3028075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.236+0000 7f3049600640 1 -- 172.21.15.50:0/3383097670 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f30440942d0 con 0x7f304c160c20 2024-09-21T05:59:06.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.236+0000 7f3051200640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3028073200 0x7f3028075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.236+0000 7f3051200640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3028073200 0x7f3028075710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f303c009600 tx=0x7f303c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:06.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.240+0000 7f3049600640 1 -- 172.21.15.50:0/3383097670 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3044020020 con 0x7f304c160c20 2024-09-21T05:59:06.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.448+0000 7f3053c00640 1 -- 172.21.15.50:0/3383097670 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f304c06b540 con 0x7f3028073200 2024-09-21T05:59:06.455 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:06.453+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:06.455 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:06.453+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:06.455 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:06.453+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:06.455 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:06.453+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:06.456 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:06.453+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:06.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.472+0000 7f3049600640 1 -- 172.21.15.50:0/3383097670 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f303c011cf0 con 0x7f3028073200 2024-09-21T05:59:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.476+0000 7f3053c00640 1 -- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3028073200 msgr2=0x7f3028075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.476+0000 7f3053c00640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3028073200 0x7f3028075710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f303c009600 tx=0x7f303c0091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.476+0000 7f3053c00640 1 -- 172.21.15.50:0/3383097670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f304c160c20 msgr2=0x7f304c0b2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.476+0000 7f3053c00640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f304c160c20 0x7f304c0b2260 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f304400cb10 tx=0x7f304400cc00 comp rx=0 tx=0).stop 2024-09-21T05:59:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.476+0000 7f3053c00640 1 -- 172.21.15.50:0/3383097670 shutdown_connections 2024-09-21T05:59:06.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.476+0000 7f3053c00640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3028073200 0x7f3028075710 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.476+0000 7f3053c00640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f304c160c20 0x7f304c0b2260 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.476+0000 7f3053c00640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f304c06d330 0x7f304c1fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.476+0000 7f3053c00640 1 --2- 172.21.15.50:0/3383097670 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f304c069330 0x7f304c1f8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.476+0000 7f3053c00640 1 -- 172.21.15.50:0/3383097670 >> 172.21.15.50:0/3383097670 conn(0x7f304c0b35f0 msgr2=0x7f304c15c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:06.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.476+0000 7f3053c00640 1 -- 172.21.15.50:0/3383097670 shutdown_connections 2024-09-21T05:59:06.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.476+0000 7f3053c00640 1 -- 172.21.15.50:0/3383097670 wait complete. 2024-09-21T05:59:06.499 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-21T05:59:06.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.660+0000 7f5a75200640 1 -- 172.21.15.50:0/3081566617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a70160060 msgr2=0x7f5a70074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.660+0000 7f5a75200640 1 --2- 172.21.15.50:0/3081566617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a70160060 0x7f5a70074240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009840 tx=0x7f5a5801c4d0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.660+0000 7f5a75200640 1 -- 172.21.15.50:0/3081566617 shutdown_connections 2024-09-21T05:59:06.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.660+0000 7f5a75200640 1 --2- 172.21.15.50:0/3081566617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a70160060 0x7f5a70074240 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.660+0000 7f5a75200640 1 -- 172.21.15.50:0/3081566617 >> 172.21.15.50:0/3081566617 conn(0x7f5a700b4560 msgr2=0x7f5a700d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:06.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.660+0000 7f5a75200640 1 -- 172.21.15.50:0/3081566617 shutdown_connections 2024-09-21T05:59:06.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.660+0000 7f5a75200640 1 -- 172.21.15.50:0/3081566617 wait complete. 2024-09-21T05:59:06.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.660+0000 7f5a75200640 1 Processor -- start 2024-09-21T05:59:06.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.660+0000 7f5a75200640 1 -- start start 2024-09-21T05:59:06.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a75200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a701536f0 0x7f5a7015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a75200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a70160060 0x7f5a7014ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a75200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a70160470 0x7f5a70153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a75200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7016b520 con 0x7f5a70160060 2024-09-21T05:59:06.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a75200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7016b3a0 con 0x7f5a70160470 2024-09-21T05:59:06.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a75200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7016b6a0 con 0x7f5a701536f0 2024-09-21T05:59:06.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a66a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a70160060 0x7f5a7014ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a66a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a70160060 0x7f5a7014ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36458/0 (socket says 172.21.15.50:36458) 2024-09-21T05:59:06.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a66a00640 1 -- 172.21.15.50:0/1987000150 learned_addr learned my addr 172.21.15.50:0/1987000150 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:06.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a6fe00640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a70160470 0x7f5a70153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a6f400640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a701536f0 0x7f5a7015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a66a00640 1 -- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a701536f0 msgr2=0x7f5a7015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a66a00640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a701536f0 0x7f5a7015de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a66a00640 1 -- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a70160470 msgr2=0x7f5a70153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a66a00640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a70160470 0x7f5a70153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a66a00640 1 -- 172.21.15.50:0/1987000150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7015e340 con 0x7f5a70160060 2024-09-21T05:59:06.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a6fe00640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a70160470 0x7f5a70153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:06.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a66a00640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a70160060 0x7f5a7014ecc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009790 tx=0x7f5a580038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:06.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a67e00640 1 -- 172.21.15.50:0/1987000150 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a580b2040 con 0x7f5a70160060 2024-09-21T05:59:06.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a67e00640 1 -- 172.21.15.50:0/1987000150 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a58003aa0 con 0x7f5a70160060 2024-09-21T05:59:06.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a75200640 1 -- 172.21.15.50:0/1987000150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a701f9b60 con 0x7f5a70160060 2024-09-21T05:59:06.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a67e00640 1 -- 172.21.15.50:0/1987000150 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58005030 con 0x7f5a70160060 2024-09-21T05:59:06.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a75200640 1 -- 172.21.15.50:0/1987000150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7015d8e0 con 0x7f5a70160060 2024-09-21T05:59:06.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.664+0000 7f5a75200640 1 -- 172.21.15.50:0/1987000150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a7015f900 con 0x7f5a70160060 2024-09-21T05:59:06.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.668+0000 7f5a67e00640 1 -- 172.21.15.50:0/1987000150 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 7) ==== 97234+0+0 (secure 0 0 0) 0x7f5a580be020 con 0x7f5a70160060 2024-09-21T05:59:06.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.668+0000 7f5a67e00640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a4c073230 0x7f5a4c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.668+0000 7f5a67e00640 1 -- 172.21.15.50:0/1987000150 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5a5812dd10 con 0x7f5a70160060 2024-09-21T05:59:06.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.668+0000 7f5a6f400640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a4c073230 0x7f5a4c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.668+0000 7f5a6f400640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a4c073230 0x7f5a4c075740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5a600014b0 tx=0x7f5a60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:06.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.672+0000 7f5a67e00640 1 -- 172.21.15.50:0/1987000150 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5a58133020 con 0x7f5a70160060 2024-09-21T05:59:06.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.872+0000 7f5a75200640 1 -- 172.21.15.50:0/1987000150 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f5a7006b540 con 0x7f5a4c073230 2024-09-21T05:59:06.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.880+0000 7f5a67e00640 1 -- 172.21.15.50:0/1987000150 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7f5a60002000 con 0x7f5a4c073230 2024-09-21T05:59:06.883 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b 2024-09-21T05:59:06.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.884+0000 7f5a75200640 1 -- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a4c073230 msgr2=0x7f5a4c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.884+0000 7f5a75200640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a4c073230 0x7f5a4c075740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5a600014b0 tx=0x7f5a60009040 comp rx=0 tx=0).stop 2024-09-21T05:59:06.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.884+0000 7f5a75200640 1 -- 172.21.15.50:0/1987000150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a70160060 msgr2=0x7f5a7014ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.884+0000 7f5a75200640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a70160060 0x7f5a7014ecc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009790 tx=0x7f5a580038b0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.884+0000 7f5a75200640 1 -- 172.21.15.50:0/1987000150 shutdown_connections 2024-09-21T05:59:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.884+0000 7f5a75200640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a70160470 0x7f5a70153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.884+0000 7f5a75200640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a4c073230 0x7f5a4c075740 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.884+0000 7f5a75200640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a70160060 0x7f5a7014ecc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.884+0000 7f5a75200640 1 --2- 172.21.15.50:0/1987000150 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a701536f0 0x7f5a7015de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.884+0000 7f5a75200640 1 -- 172.21.15.50:0/1987000150 >> 172.21.15.50:0/1987000150 conn(0x7f5a700b4560 msgr2=0x7f5a700b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.884+0000 7f5a75200640 1 -- 172.21.15.50:0/1987000150 shutdown_connections 2024-09-21T05:59:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:06.884+0000 7f5a75200640 1 -- 172.21.15.50:0/1987000150 wait complete. 2024-09-21T05:59:06.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:06.905 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b 2024-09-21T05:59:06.905 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T05:59:06.915 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b 2024-09-21T05:59:06.915 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="10412687360" 2024-09-21T05:59:06.915 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:06.916 INFO:tasks.cephfs.test_volumes:r_size = 10412687360 exp_size = 10737418240 2024-09-21T05:59:08.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:08.916 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b 2024-09-21T05:59:08.916 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T05:59:08.927 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b 2024-09-21T05:59:08.927 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="10412687360" 2024-09-21T05:59:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:08.928 INFO:tasks.cephfs.test_volumes:r_size = 10412687360 exp_size = 10737418240 2024-09-21T05:59:10.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:10.929 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b 2024-09-21T05:59:10.929 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T05:59:10.940 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/group1/sv1/3e1dea30-c311-44e5-87ef-735039e0455b 2024-09-21T05:59:10.940 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="10737418240" 2024-09-21T05:59:10.940 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:10.941 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-21T05:59:10.942 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 --group-name group1 2024-09-21T05:59:11.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.160+0000 7fe3a6200640 1 -- 172.21.15.50:0/120178319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe388003a80 msgr2=0x7fe388003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:11.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.160+0000 7fe3a6200640 1 --2- 172.21.15.50:0/120178319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe388003a80 0x7fe388003ef0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe39400c310 tx=0x7fe39401cfb0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.160+0000 7fe3a5800640 1 -- 172.21.15.50:0/120178319 reap_dead start 2024-09-21T05:59:11.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.160+0000 7fe3a6200640 1 -- 172.21.15.50:0/120178319 shutdown_connections 2024-09-21T05:59:11.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.160+0000 7fe3a6200640 1 -- 172.21.15.50:0/120178319 >> 172.21.15.50:0/120178319 conn(0x7fe3a00b4460 msgr2=0x7fe3a00b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:11.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.160+0000 7fe3a6200640 1 -- 172.21.15.50:0/120178319 shutdown_connections 2024-09-21T05:59:11.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.160+0000 7fe3a6200640 1 -- 172.21.15.50:0/120178319 wait complete. 2024-09-21T05:59:11.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a6200640 1 Processor -- start 2024-09-21T05:59:11.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a6200640 1 -- start start 2024-09-21T05:59:11.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a6200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe388003a80 0x7fe3a00b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a6200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015f860 0x7fe3a014dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a6200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3a014e220 0x7fe3a00b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a6200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a0167290 con 0x7fe388003a80 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a6200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a0153960 con 0x7fe3a015f860 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a6200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a0167410 con 0x7fe3a014e220 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe388003a80 0x7fe3a00b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe388003a80 0x7fe3a00b7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58250/0 (socket says 172.21.15.50:58250) 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a4e00640 1 -- 172.21.15.50:0/2393956048 learned_addr learned my addr 172.21.15.50:0/2393956048 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a5800640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3a014e220 0x7fe3a00b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe39fe00640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015f860 0x7fe3a014dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a4e00640 1 -- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3a014e220 msgr2=0x7fe3a00b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a4e00640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3a014e220 0x7fe3a00b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a4e00640 1 -- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015f860 msgr2=0x7fe3a014dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:11.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a4e00640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015f860 0x7fe3a014dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a4e00640 1 -- 172.21.15.50:0/2393956048 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a014e630 con 0x7fe388003a80 2024-09-21T05:59:11.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe39fe00640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015f860 0x7fe3a014dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:11.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a4e00640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe388003a80 0x7fe3a00b7490 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe39400c310 tx=0x7fe394001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:11.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe39cc00640 1 -- 172.21.15.50:0/2393956048 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394003600 con 0x7fe388003a80 2024-09-21T05:59:11.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe39cc00640 1 -- 172.21.15.50:0/2393956048 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe394003890 con 0x7fe388003a80 2024-09-21T05:59:11.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a6200640 1 -- 172.21.15.50:0/2393956048 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a00b2bf0 con 0x7fe388003a80 2024-09-21T05:59:11.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe39cc00640 1 -- 172.21.15.50:0/2393956048 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394003c80 con 0x7fe388003a80 2024-09-21T05:59:11.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.164+0000 7fe3a6200640 1 -- 172.21.15.50:0/2393956048 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a00b31b0 con 0x7fe388003a80 2024-09-21T05:59:11.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.168+0000 7fe3a6200640 1 -- 172.21.15.50:0/2393956048 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a00d0ab0 con 0x7fe388003a80 2024-09-21T05:59:11.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.168+0000 7fe39cc00640 1 -- 172.21.15.50:0/2393956048 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 97285+0+0 (secure 0 0 0) 0x7fe3940ba460 con 0x7fe388003a80 2024-09-21T05:59:11.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.168+0000 7fe39cc00640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe380073290 0x7fe3800757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:11.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.168+0000 7fe39cc00640 1 -- 172.21.15.50:0/2393956048 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe39412ed50 con 0x7fe388003a80 2024-09-21T05:59:11.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.168+0000 7fe39fe00640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe380073290 0x7fe3800757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:11.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.168+0000 7fe39fe00640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe380073290 0x7fe3800757a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe3a015e130 tx=0x7fe38c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:11.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.172+0000 7fe39cc00640 1 -- 172.21.15.50:0/2393956048 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe3a014e630 con 0x7fe388003a80 2024-09-21T05:59:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.384+0000 7fe3a6200640 1 -- 172.21.15.50:0/2393956048 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7fe3a006b540 con 0x7fe380073290 2024-09-21T05:59:11.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe39cc00640 1 -- 172.21.15.50:0/2393956048 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe38c013b50 con 0x7fe380073290 2024-09-21T05:59:11.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe3a6200640 1 -- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe380073290 msgr2=0x7fe3800757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:11.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe3a6200640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe380073290 0x7fe3800757a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe3a015e130 tx=0x7fe38c0091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe3a6200640 1 -- 172.21.15.50:0/2393956048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe388003a80 msgr2=0x7fe3a00b7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:11.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe3a6200640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe388003a80 0x7fe3a00b7490 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe39400c310 tx=0x7fe394001d70 comp rx=0 tx=0).stop 2024-09-21T05:59:11.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe3a6200640 1 -- 172.21.15.50:0/2393956048 shutdown_connections 2024-09-21T05:59:11.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe3a6200640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3a014e220 0x7fe3a00b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe3a6200640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015f860 0x7fe3a014dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe3a6200640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe380073290 0x7fe3800757a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe3a6200640 1 --2- 172.21.15.50:0/2393956048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe388003a80 0x7fe3a00b7490 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe3a6200640 1 -- 172.21.15.50:0/2393956048 >> 172.21.15.50:0/2393956048 conn(0x7fe3a00b4460 msgr2=0x7fe3a0151fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:11.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe3a6200640 1 -- 172.21.15.50:0/2393956048 shutdown_connections 2024-09-21T05:59:11.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.464+0000 7fe3a6200640 1 -- 172.21.15.50:0/2393956048 wait complete. 2024-09-21T05:59:11.469 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:11.465+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:11.469 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:11.465+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:11.469 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:11.465+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:11.469 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:11.465+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:11.469 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:11.465+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:11.490 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:11.485+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:11.490 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:11.485+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:11.490 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:11.485+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:11.490 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:11.485+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:11.490 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:11.485+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:11.492 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T05:59:11.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.652+0000 7f2109200640 1 -- 172.21.15.50:0/306983808 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21040749a0 msgr2=0x7f2104074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:11.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.652+0000 7f2109200640 1 --2- 172.21.15.50:0/306983808 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21040749a0 0x7f2104074db0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009840 tx=0x7f20f401c820 comp rx=0 tx=0).stop 2024-09-21T05:59:11.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.652+0000 7f2109200640 1 -- 172.21.15.50:0/306983808 shutdown_connections 2024-09-21T05:59:11.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.652+0000 7f2109200640 1 --2- 172.21.15.50:0/306983808 >> v2:172.21.15.88:3300/0 conn(0x7f21040cefa0 0x7f21040cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.652+0000 7f2109200640 1 --2- 172.21.15.50:0/306983808 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21040749a0 0x7f2104074db0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.652+0000 7f2109200640 1 -- 172.21.15.50:0/306983808 >> 172.21.15.50:0/306983808 conn(0x7f21040b2040 msgr2=0x7f21040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:11.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.652+0000 7f2109200640 1 -- 172.21.15.50:0/306983808 shutdown_connections 2024-09-21T05:59:11.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.652+0000 7f2109200640 1 -- 172.21.15.50:0/306983808 wait complete. 2024-09-21T05:59:11.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2109200640 1 Processor -- start 2024-09-21T05:59:11.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2109200640 1 -- start start 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2109200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21040749a0 0x7f2104145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2109200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21040cefa0 0x7f2104166e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2109200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2104160330 0x7f210412fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2109200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f210416a060 con 0x7f2104160330 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2109200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2104169ee0 con 0x7f21040cefa0 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2109200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f210416a1e0 con 0x7f21040749a0 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2102800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2104160330 0x7f210412fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2102800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2104160330 0x7f210412fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58260/0 (socket says 172.21.15.50:58260) 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2102800640 1 -- 172.21.15.50:0/2855356614 learned_addr learned my addr 172.21.15.50:0/2855356614 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2101e00640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21040749a0 0x7f2104145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2101400640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21040cefa0 0x7f2104166e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2102800640 1 -- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21040749a0 msgr2=0x7f2104145a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:11.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2102800640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21040749a0 0x7f2104145a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2102800640 1 -- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21040cefa0 msgr2=0x7f2104166e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:11.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.656+0000 7f2102800640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21040cefa0 0x7f2104166e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.660+0000 7f2102800640 1 -- 172.21.15.50:0/2855356614 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2104130240 con 0x7f2104160330 2024-09-21T05:59:11.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.660+0000 7f2101e00640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21040749a0 0x7f2104145a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:11.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.660+0000 7f2102800640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2104160330 0x7f210412fd00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00caa0 tx=0x7f20fc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:11.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.660+0000 7f20fa000640 1 -- 172.21.15.50:0/2855356614 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc016020 con 0x7f2104160330 2024-09-21T05:59:11.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.660+0000 7f20fa000640 1 -- 172.21.15.50:0/2855356614 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20fc00a8d0 con 0x7f2104160330 2024-09-21T05:59:11.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.660+0000 7f2109200640 1 -- 172.21.15.50:0/2855356614 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210412d300 con 0x7f2104160330 2024-09-21T05:59:11.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.660+0000 7f20fa000640 1 -- 172.21.15.50:0/2855356614 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00ad20 con 0x7f2104160330 2024-09-21T05:59:11.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.660+0000 7f2109200640 1 -- 172.21.15.50:0/2855356614 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f210412d8c0 con 0x7f2104160330 2024-09-21T05:59:11.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.660+0000 7f2109200640 1 -- 172.21.15.50:0/2855356614 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2104074db0 con 0x7f2104160330 2024-09-21T05:59:11.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.660+0000 7f20fa000640 1 -- 172.21.15.50:0/2855356614 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 97285+0+0 (secure 0 0 0) 0x7f20fc019020 con 0x7f2104160330 2024-09-21T05:59:11.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.660+0000 7f20fa000640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f20e40732c0 0x7f20e40757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:11.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.664+0000 7f20fa000640 1 -- 172.21.15.50:0/2855356614 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f20fc094830 con 0x7f2104160330 2024-09-21T05:59:11.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.664+0000 7f2101e00640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f20e40732c0 0x7f20e40757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:11.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.664+0000 7f2101e00640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f20e40732c0 0x7f20e40757d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f20f40af000 tx=0x7f20f401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:11.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.668+0000 7f20fa000640 1 -- 172.21.15.50:0/2855356614 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f20fc020020 con 0x7f2104160330 2024-09-21T05:59:11.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.908+0000 7f2109200640 1 -- 172.21.15.50:0/2855356614 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f21040b27d0 con 0x7f2104160330 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.908+0000 7f20fa000640 1 -- 172.21.15.50:0/2855356614 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1911 (secure 0 0 0) 0x7f2104130240 con 0x7f2104160330 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T05:59:11.912 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 188, 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 33, 2024-09-21T05:59:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T05:59:11.914 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 2585, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 10738570793, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 21744107520, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 751350005760, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 85, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 89460428, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 0, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 22 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 18, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T05:57:25:016672+0000", 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T05:59:11.915 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 2, 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "a", 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 4813 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T05:59:11.916 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 7, 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T05:58:22.913083+0000", 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": {} 2024-09-21T05:59:11.917 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T05:59:11.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.912+0000 7f2109200640 1 -- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f20e40732c0 msgr2=0x7f20e40757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:11.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.912+0000 7f2109200640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f20e40732c0 0x7f20e40757d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f20f40af000 tx=0x7f20f401b060 comp rx=0 tx=0).stop 2024-09-21T05:59:11.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.912+0000 7f2109200640 1 -- 172.21.15.50:0/2855356614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2104160330 msgr2=0x7f210412fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:11.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.912+0000 7f2109200640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2104160330 0x7f210412fd00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00caa0 tx=0x7f20fc00cb90 comp rx=0 tx=0).stop 2024-09-21T05:59:11.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.912+0000 7f2109200640 1 -- 172.21.15.50:0/2855356614 shutdown_connections 2024-09-21T05:59:11.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.912+0000 7f2109200640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f20e40732c0 0x7f20e40757d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.912+0000 7f2109200640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2104160330 0x7f210412fd00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.912+0000 7f2109200640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21040cefa0 0x7f2104166e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.912+0000 7f2109200640 1 --2- 172.21.15.50:0/2855356614 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21040749a0 0x7f2104145a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:11.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.912+0000 7f2109200640 1 -- 172.21.15.50:0/2855356614 >> 172.21.15.50:0/2855356614 conn(0x7f21040b2040 msgr2=0x7f210406e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:11.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.912+0000 7f2109200640 1 -- 172.21.15.50:0/2855356614 shutdown_connections 2024-09-21T05:59:11.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:11.912+0000 7f2109200640 1 -- 172.21.15.50:0/2855356614 wait complete. 2024-09-21T05:59:12.475 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:12.473+0000 7f6594400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:12.475 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:12.473+0000 7f6594400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:12.475 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:12.473+0000 7f6594400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:12.475 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:12.473+0000 7f6594400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:12.475 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:12.473+0000 7f6594400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:12.935 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T05:59:13.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.208+0000 7fb525c00640 1 -- 172.21.15.50:0/3498920661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb504004810 msgr2=0x7fb504004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:13.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.208+0000 7fb525c00640 1 --2- 172.21.15.50:0/3498920661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb504004810 0x7fb504004c20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb520072120 tx=0x7fb50c01ce80 comp rx=0 tx=0).stop 2024-09-21T05:59:13.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.212+0000 7fb51f400640 1 -- 172.21.15.50:0/3498920661 reap_dead start 2024-09-21T05:59:13.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.212+0000 7fb525c00640 1 -- 172.21.15.50:0/3498920661 shutdown_connections 2024-09-21T05:59:13.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.212+0000 7fb525c00640 1 -- 172.21.15.50:0/3498920661 >> 172.21.15.50:0/3498920661 conn(0x7fb5200b35f0 msgr2=0x7fb5200b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:13.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.212+0000 7fb525c00640 1 -- 172.21.15.50:0/3498920661 shutdown_connections 2024-09-21T05:59:13.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.212+0000 7fb525c00640 1 -- 172.21.15.50:0/3498920661 wait complete. 2024-09-21T05:59:13.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.212+0000 7fb525c00640 1 Processor -- start 2024-09-21T05:59:13.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.212+0000 7fb525c00640 1 -- start start 2024-09-21T05:59:13.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb525c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb504004810 0x7fb5201e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:13.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb525c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb520146190 0x7fb52015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:13.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb525c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5201465a0 0x7fb52015d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:13.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb525c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb520069770 con 0x7fb5201465a0 2024-09-21T05:59:13.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb525c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5200695f0 con 0x7fb504004810 2024-09-21T05:59:13.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb525c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb5200698f0 con 0x7fb520146190 2024-09-21T05:59:13.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb51f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5201465a0 0x7fb52015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:13.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb51f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5201465a0 0x7fb52015d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58278/0 (socket says 172.21.15.50:58278) 2024-09-21T05:59:13.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb51f400640 1 -- 172.21.15.50:0/1838528833 learned_addr learned my addr 172.21.15.50:0/1838528833 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:13.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb51ea00640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb504004810 0x7fb5201e8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:13.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb51e000640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb520146190 0x7fb52015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:13.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb51f400640 1 -- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb520146190 msgr2=0x7fb52015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:13.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb51f400640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb520146190 0x7fb52015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:13.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb51f400640 1 -- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb504004810 msgr2=0x7fb5201e8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:13.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb51f400640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb504004810 0x7fb5201e8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:13.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb51f400640 1 -- 172.21.15.50:0/1838528833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5200772a0 con 0x7fb5201465a0 2024-09-21T05:59:13.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb51e000640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb520146190 0x7fb52015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:13.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb51f400640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5201465a0 0x7fb52015d660 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb520073020 tx=0x7fb51800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:13.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb516a00640 1 -- 172.21.15.50:0/1838528833 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51800ec50 con 0x7fb5201465a0 2024-09-21T05:59:13.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb516a00640 1 -- 172.21.15.50:0/1838528833 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb518003980 con 0x7fb5201465a0 2024-09-21T05:59:13.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb525c00640 1 -- 172.21.15.50:0/1838528833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5201e90e0 con 0x7fb5201465a0 2024-09-21T05:59:13.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb516a00640 1 -- 172.21.15.50:0/1838528833 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518003dd0 con 0x7fb5201465a0 2024-09-21T05:59:13.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.216+0000 7fb525c00640 1 -- 172.21.15.50:0/1838528833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5200ab210 con 0x7fb5201465a0 2024-09-21T05:59:13.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.220+0000 7fb525c00640 1 -- 172.21.15.50:0/1838528833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5200aa8d0 con 0x7fb5201465a0 2024-09-21T05:59:13.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.220+0000 7fb516a00640 1 -- 172.21.15.50:0/1838528833 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 97438+0+0 (secure 0 0 0) 0x7fb51801b030 con 0x7fb5201465a0 2024-09-21T05:59:13.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.224+0000 7fb516a00640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb500073470 0x7fb500075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:13.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.224+0000 7fb516a00640 1 -- 172.21.15.50:0/1838528833 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb5180955a0 con 0x7fb5201465a0 2024-09-21T05:59:13.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.224+0000 7fb51ea00640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb500073470 0x7fb500075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:13.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.228+0000 7fb51ea00640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb500073470 0x7fb500075980 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb510005790 tx=0x7fb51000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:13.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.232+0000 7fb516a00640 1 -- 172.21.15.50:0/1838528833 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb518022020 con 0x7fb5201465a0 2024-09-21T05:59:13.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.604+0000 7fb525c00640 1 -- 172.21.15.50:0/1838528833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb52006d200 con 0x7fb5201465a0 2024-09-21T05:59:13.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.604+0000 7fb516a00640 1 -- 172.21.15.50:0/1838528833 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2137 (secure 0 0 0) 0x7fb518060250 con 0x7fb5201465a0 2024-09-21T05:59:13.607 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:13.607 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T05:59:13.607 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T05:59:13.607 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T05:59:13.607 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T05:59:13.607 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T05:59:13.607 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T05:59:13.607 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:13.607 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T05:59:13.607 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T05:59:13.607 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T05:59:13.607 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 190, 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 33, 2024-09-21T05:59:13.608 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T05:59:13.609 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 2585, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 10738570793, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 21744181248, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 751349932032, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 85, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 89461024, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 0, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 22 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 18, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T05:57:25:016672+0000", 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 2, 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "a", 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 4813 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T05:59:13.611 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 7, 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T05:58:22.913083+0000", 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": { 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr-vol-ongoing-clones": { 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "message": "1 ongoing clones - average progress is 0.0% (0s)\n [............................] ", 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "progress": 0, 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: "add_to_ceph_s": true 2024-09-21T05:59:13.612 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:13.613 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:13.613 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T05:59:13.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.608+0000 7fb525c00640 1 -- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb500073470 msgr2=0x7fb500075980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:13.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.608+0000 7fb525c00640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb500073470 0x7fb500075980 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb510005790 tx=0x7fb51000b2e0 comp rx=0 tx=0).stop 2024-09-21T05:59:13.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.608+0000 7fb525c00640 1 -- 172.21.15.50:0/1838528833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5201465a0 msgr2=0x7fb52015d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:13.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.608+0000 7fb525c00640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5201465a0 0x7fb52015d660 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb520073020 tx=0x7fb51800c6a0 comp rx=0 tx=0).stop 2024-09-21T05:59:13.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.612+0000 7fb525c00640 1 -- 172.21.15.50:0/1838528833 shutdown_connections 2024-09-21T05:59:13.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.612+0000 7fb525c00640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb500073470 0x7fb500075980 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:13.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.612+0000 7fb525c00640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5201465a0 0x7fb52015d660 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:13.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.612+0000 7fb525c00640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb520146190 0x7fb52015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:13.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.612+0000 7fb525c00640 1 --2- 172.21.15.50:0/1838528833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb504004810 0x7fb5201e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:13.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.612+0000 7fb525c00640 1 -- 172.21.15.50:0/1838528833 >> 172.21.15.50:0/1838528833 conn(0x7fb5200b35f0 msgr2=0x7fb520070880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:13.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.612+0000 7fb525c00640 1 -- 172.21.15.50:0/1838528833 shutdown_connections 2024-09-21T05:59:13.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.612+0000 7fb525c00640 1 -- 172.21.15.50:0/1838528833 wait complete. 2024-09-21T05:59:13.668 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-21T05:59:13.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.912+0000 7ff419800640 1 -- 172.21.15.50:0/1797802020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff41415f9c0 msgr2=0x7ff4140ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:13.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.912+0000 7ff419800640 1 --2- 172.21.15.50:0/1797802020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff41415f9c0 0x7ff4140ceef0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff4140d0540 tx=0x7ff40c01cd80 comp rx=0 tx=0).stop 2024-09-21T05:59:13.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.912+0000 7ff419800640 1 -- 172.21.15.50:0/1797802020 shutdown_connections 2024-09-21T05:59:13.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.912+0000 7ff419800640 1 --2- 172.21.15.50:0/1797802020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff41415f9c0 0x7ff4140ceef0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:13.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.912+0000 7ff419800640 1 --2- 172.21.15.50:0/1797802020 >> v2:172.21.15.88:3300/0 conn(0x7ff41406e180 0x7ff41415b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:13.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.912+0000 7ff419800640 1 -- 172.21.15.50:0/1797802020 >> 172.21.15.50:0/1797802020 conn(0x7ff41406d1f0 msgr2=0x7ff41406d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:13.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.912+0000 7ff419800640 1 -- 172.21.15.50:0/1797802020 shutdown_connections 2024-09-21T05:59:13.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.912+0000 7ff419800640 1 -- 172.21.15.50:0/1797802020 wait complete. 2024-09-21T05:59:13.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.916+0000 7ff419800640 1 Processor -- start 2024-09-21T05:59:13.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.916+0000 7ff419800640 1 -- start start 2024-09-21T05:59:13.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.916+0000 7ff419800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff41406e180 0x7ff414158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:13.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff419800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff414073f40 0x7ff414150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:13.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff419800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff414151460 0x7ff4140ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:13.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff419800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff4141692e0 con 0x7ff414151460 2024-09-21T05:59:13.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff419800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff414169160 con 0x7ff41406e180 2024-09-21T05:59:13.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff419800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff414169460 con 0x7ff414073f40 2024-09-21T05:59:13.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff418e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff414151460 0x7ff4140ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:13.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff418e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff414151460 0x7ff4140ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58294/0 (socket says 172.21.15.50:58294) 2024-09-21T05:59:13.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff418e00640 1 -- 172.21.15.50:0/495857706 learned_addr learned my addr 172.21.15.50:0/495857706 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:13.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff413e00640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff41406e180 0x7ff414158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:13.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff413400640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff414073f40 0x7ff414150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:13.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff418e00640 1 -- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff414073f40 msgr2=0x7ff414150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:13.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff418e00640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff414073f40 0x7ff414150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:13.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff418e00640 1 -- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff41406e180 msgr2=0x7ff414158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:13.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff418e00640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff41406e180 0x7ff414158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:13.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff418e00640 1 -- 172.21.15.50:0/495857706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4141f8ef0 con 0x7ff414151460 2024-09-21T05:59:13.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff418e00640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff414151460 0x7ff4140ae8e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009bd0 tx=0x7ff40c009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:13.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff40be00640 1 -- 172.21.15.50:0/495857706 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0bc070 con 0x7ff414151460 2024-09-21T05:59:13.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff40be00640 1 -- 172.21.15.50:0/495857706 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff40c0ad8e0 con 0x7ff414151460 2024-09-21T05:59:13.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff419800640 1 -- 172.21.15.50:0/495857706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41415aac0 con 0x7ff414151460 2024-09-21T05:59:13.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff40be00640 1 -- 172.21.15.50:0/495857706 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0b2040 con 0x7ff414151460 2024-09-21T05:59:13.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff419800640 1 -- 172.21.15.50:0/495857706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff414158020 con 0x7ff414151460 2024-09-21T05:59:13.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.920+0000 7ff419800640 1 -- 172.21.15.50:0/495857706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff41415b4a0 con 0x7ff414151460 2024-09-21T05:59:13.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.924+0000 7ff40be00640 1 -- 172.21.15.50:0/495857706 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 97438+0+0 (secure 0 0 0) 0x7ff40c0bf070 con 0x7ff414151460 2024-09-21T05:59:13.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.924+0000 7ff40be00640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff3f40734a0 0x7ff3f40759b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:13.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.928+0000 7ff40be00640 1 -- 172.21.15.50:0/495857706 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff40c12e6c0 con 0x7ff414151460 2024-09-21T05:59:13.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.928+0000 7ff413e00640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff3f40734a0 0x7ff3f40759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:13.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.932+0000 7ff413e00640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff3f40734a0 0x7ff3f40759b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff40400a490 tx=0x7ff40400a420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:13.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:13.932+0000 7ff40be00640 1 -- 172.21.15.50:0/495857706 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff40c0c6080 con 0x7ff414151460 2024-09-21T05:59:14.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.256+0000 7ff419800640 1 -- 172.21.15.50:0/495857706 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7ff41415ada0 con 0x7ff3f40734a0 2024-09-21T05:59:14.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.424+0000 7ff40be00640 1 -- 172.21.15.50:0/495857706 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff41415ada0 con 0x7ff3f40734a0 2024-09-21T05:59:14.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.428+0000 7ff419800640 1 -- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff3f40734a0 msgr2=0x7ff3f40759b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:14.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.428+0000 7ff419800640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff3f40734a0 0x7ff3f40759b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff40400a490 tx=0x7ff40400a420 comp rx=0 tx=0).stop 2024-09-21T05:59:14.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.428+0000 7ff419800640 1 -- 172.21.15.50:0/495857706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff414151460 msgr2=0x7ff4140ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:14.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.428+0000 7ff419800640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff414151460 0x7ff4140ae8e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009bd0 tx=0x7ff40c009340 comp rx=0 tx=0).stop 2024-09-21T05:59:14.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.428+0000 7ff419800640 1 -- 172.21.15.50:0/495857706 shutdown_connections 2024-09-21T05:59:14.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.428+0000 7ff419800640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff3f40734a0 0x7ff3f40759b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:14.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.428+0000 7ff419800640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff414151460 0x7ff4140ae8e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:14.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.428+0000 7ff419800640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff414073f40 0x7ff414150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:14.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.428+0000 7ff419800640 1 --2- 172.21.15.50:0/495857706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff41406e180 0x7ff414158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:14.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.428+0000 7ff419800640 1 -- 172.21.15.50:0/495857706 >> 172.21.15.50:0/495857706 conn(0x7ff41406d1f0 msgr2=0x7ff4140b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:14.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.428+0000 7ff419800640 1 -- 172.21.15.50:0/495857706 shutdown_connections 2024-09-21T05:59:14.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.428+0000 7ff419800640 1 -- 172.21.15.50:0/495857706 wait complete. 2024-09-21T05:59:14.455 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T05:59:14.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.612+0000 7f5a73200640 1 -- 172.21.15.50:0/1668348187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a74155be0 msgr2=0x7f5a7415c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:14.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.612+0000 7f5a73200640 1 --2- 172.21.15.50:0/1668348187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a74155be0 0x7f5a7415c640 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009840 tx=0x7f5a6401c820 comp rx=0 tx=0).stop 2024-09-21T05:59:14.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.612+0000 7f5a73200640 1 -- 172.21.15.50:0/1668348187 shutdown_connections 2024-09-21T05:59:14.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.612+0000 7f5a73200640 1 --2- 172.21.15.50:0/1668348187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a74155be0 0x7f5a7415c640 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:14.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.612+0000 7f5a73200640 1 -- 172.21.15.50:0/1668348187 >> 172.21.15.50:0/1668348187 conn(0x7f5a7406d920 msgr2=0x7f5a740b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:14.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.612+0000 7f5a73200640 1 -- 172.21.15.50:0/1668348187 shutdown_connections 2024-09-21T05:59:14.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.612+0000 7f5a73200640 1 -- 172.21.15.50:0/1668348187 wait complete. 2024-09-21T05:59:14.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.616+0000 7f5a73200640 1 Processor -- start 2024-09-21T05:59:14.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.616+0000 7f5a73200640 1 -- start start 2024-09-21T05:59:14.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.616+0000 7f5a73200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a74072360 0x7f5a74151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:14.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.616+0000 7f5a73200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a74072770 0x7f5a74143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:14.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.616+0000 7f5a73200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a74155be0 0x7f5a74153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:14.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.616+0000 7f5a73200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74169d30 con 0x7f5a74155be0 2024-09-21T05:59:14.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.616+0000 7f5a73200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74169bb0 con 0x7f5a74072770 2024-09-21T05:59:14.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.616+0000 7f5a73200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5a74169eb0 con 0x7f5a74072360 2024-09-21T05:59:14.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a72800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a74155be0 0x7f5a74153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:14.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a72800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a74155be0 0x7f5a74153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58316/0 (socket says 172.21.15.50:58316) 2024-09-21T05:59:14.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a72800640 1 -- 172.21.15.50:0/3325948280 learned_addr learned my addr 172.21.15.50:0/3325948280 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:14.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a6be00640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a74072770 0x7f5a74143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:14.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a71e00640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a74072360 0x7f5a74151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:14.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a72800640 1 -- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a74072360 msgr2=0x7f5a74151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:14.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a72800640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a74072360 0x7f5a74151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:14.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a72800640 1 -- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a74072770 msgr2=0x7f5a74143ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:14.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a72800640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a74072770 0x7f5a74143ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:14.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a72800640 1 -- 172.21.15.50:0/3325948280 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a741fad90 con 0x7f5a74155be0 2024-09-21T05:59:14.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a72800640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a74155be0 0x7f5a74153f00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00cab0 tx=0x7f5a6c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:14.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a6a000640 1 -- 172.21.15.50:0/3325948280 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c016020 con 0x7f5a74155be0 2024-09-21T05:59:14.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a6a000640 1 -- 172.21.15.50:0/3325948280 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a6c00a950 con 0x7f5a74155be0 2024-09-21T05:59:14.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a6a000640 1 -- 172.21.15.50:0/3325948280 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c00ada0 con 0x7f5a74155be0 2024-09-21T05:59:14.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a73200640 1 -- 172.21.15.50:0/3325948280 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a741519d0 con 0x7f5a74155be0 2024-09-21T05:59:14.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a73200640 1 -- 172.21.15.50:0/3325948280 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a741fa2b0 con 0x7f5a74155be0 2024-09-21T05:59:14.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a73200640 1 -- 172.21.15.50:0/3325948280 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a74151740 con 0x7f5a74155be0 2024-09-21T05:59:14.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.620+0000 7f5a6a000640 1 -- 172.21.15.50:0/3325948280 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 97438+0+0 (secure 0 0 0) 0x7f5a6c019020 con 0x7f5a74155be0 2024-09-21T05:59:14.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.624+0000 7f5a6a000640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a50073440 0x7f5a50075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:14.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.624+0000 7f5a6a000640 1 -- 172.21.15.50:0/3325948280 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5a6c094820 con 0x7f5a74155be0 2024-09-21T05:59:14.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.624+0000 7f5a71e00640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a50073440 0x7f5a50075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:14.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.624+0000 7f5a71e00640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a50073440 0x7f5a50075950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5a640af000 tx=0x7f5a6401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:14.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.628+0000 7f5a6a000640 1 -- 172.21.15.50:0/3325948280 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5a6c020020 con 0x7f5a74155be0 2024-09-21T05:59:14.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.824+0000 7f5a73200640 1 -- 172.21.15.50:0/3325948280 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f5a7406b540 con 0x7f5a50073440 2024-09-21T05:59:14.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a6a000640 1 -- 172.21.15.50:0/3325948280 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f5a640a55d0 con 0x7f5a50073440 2024-09-21T05:59:14.830 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T05:59:14.830 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:14.830 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T05:59:14.830 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:14.830 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T05:59:14.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a73200640 1 -- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a50073440 msgr2=0x7f5a50075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:14.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a73200640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a50073440 0x7f5a50075950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5a640af000 tx=0x7f5a6401b060 comp rx=0 tx=0).stop 2024-09-21T05:59:14.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a73200640 1 -- 172.21.15.50:0/3325948280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a74155be0 msgr2=0x7f5a74153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:14.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a73200640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a74155be0 0x7f5a74153f00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00cab0 tx=0x7f5a6c00cba0 comp rx=0 tx=0).stop 2024-09-21T05:59:14.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a73200640 1 -- 172.21.15.50:0/3325948280 shutdown_connections 2024-09-21T05:59:14.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a73200640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a50073440 0x7f5a50075950 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:14.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a73200640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a74155be0 0x7f5a74153f00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:14.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a73200640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a74072770 0x7f5a74143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:14.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a73200640 1 --2- 172.21.15.50:0/3325948280 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a74072360 0x7f5a74151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:14.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a73200640 1 -- 172.21.15.50:0/3325948280 >> 172.21.15.50:0/3325948280 conn(0x7f5a7406d920 msgr2=0x7f5a74075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:14.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a73200640 1 -- 172.21.15.50:0/3325948280 shutdown_connections 2024-09-21T05:59:14.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:14.828+0000 7f5a73200640 1 -- 172.21.15.50:0/3325948280 wait complete. 2024-09-21T05:59:14.852 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-21T05:59:15.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.012+0000 7fb03ae00640 1 -- 172.21.15.50:0/27838821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb034157030 msgr2=0x7fb0340b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:15.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.012+0000 7fb03ae00640 1 --2- 172.21.15.50:0/27838821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb034157030 0x7fb0340b5a80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb024009840 tx=0x7fb02401c820 comp rx=0 tx=0).stop 2024-09-21T05:59:15.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.012+0000 7fb03ae00640 1 -- 172.21.15.50:0/27838821 shutdown_connections 2024-09-21T05:59:15.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.012+0000 7fb03ae00640 1 --2- 172.21.15.50:0/27838821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb034074b20 0x7fb03415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.012+0000 7fb03ae00640 1 --2- 172.21.15.50:0/27838821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb034157030 0x7fb0340b5a80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.012+0000 7fb03ae00640 1 -- 172.21.15.50:0/27838821 >> 172.21.15.50:0/27838821 conn(0x7fb0340708c0 msgr2=0x7fb034070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:15.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.012+0000 7fb03ae00640 1 -- 172.21.15.50:0/27838821 shutdown_connections 2024-09-21T05:59:15.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.012+0000 7fb03ae00640 1 -- 172.21.15.50:0/27838821 wait complete. 2024-09-21T05:59:15.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.012+0000 7fb03ae00640 1 Processor -- start 2024-09-21T05:59:15.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.012+0000 7fb03ae00640 1 -- start start 2024-09-21T05:59:15.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb03ae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb034157030 0x7fb034154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:15.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb03ae00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb034160d50 0x7fb03406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:15.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb03ae00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb03414d2a0 0x7fb034151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb03ae00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb03416a740 con 0x7fb034160d50 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb03ae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb03416a5c0 con 0x7fb034157030 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb03ae00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb03416a8c0 con 0x7fb03414d2a0 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb02b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb034160d50 0x7fb03406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb02b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb034160d50 0x7fb03406e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58336/0 (socket says 172.21.15.50:58336) 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb02b400640 1 -- 172.21.15.50:0/3196240937 learned_addr learned my addr 172.21.15.50:0/3196240937 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb033e00640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb034157030 0x7fb034154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb038e00640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb03414d2a0 0x7fb034151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb02b400640 1 -- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb03414d2a0 msgr2=0x7fb034151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb02b400640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb03414d2a0 0x7fb034151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb02b400640 1 -- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb034157030 msgr2=0x7fb034154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb02b400640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb034157030 0x7fb034154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb02b400640 1 -- 172.21.15.50:0/3196240937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb034154410 con 0x7fb034160d50 2024-09-21T05:59:15.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb033e00640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb034157030 0x7fb034154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:59:15.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb02b400640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb034160d50 0x7fb03406e2a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00c400 tx=0x7fb01c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:15.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb030c00640 1 -- 172.21.15.50:0/3196240937 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c017070 con 0x7fb034160d50 2024-09-21T05:59:15.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb030c00640 1 -- 172.21.15.50:0/3196240937 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb01c003600 con 0x7fb034160d50 2024-09-21T05:59:15.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb03ae00640 1 -- 172.21.15.50:0/3196240937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0341401e0 con 0x7fb034160d50 2024-09-21T05:59:15.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb030c00640 1 -- 172.21.15.50:0/3196240937 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c003a50 con 0x7fb034160d50 2024-09-21T05:59:15.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb03ae00640 1 -- 172.21.15.50:0/3196240937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0341546b0 con 0x7fb034160d50 2024-09-21T05:59:15.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.016+0000 7fb03ae00640 1 -- 172.21.15.50:0/3196240937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb03406e910 con 0x7fb034160d50 2024-09-21T05:59:15.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.020+0000 7fb030c00640 1 -- 172.21.15.50:0/3196240937 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 97438+0+0 (secure 0 0 0) 0x7fb01c01a070 con 0x7fb034160d50 2024-09-21T05:59:15.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.020+0000 7fb030c00640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb010073470 0x7fb010075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:15.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.020+0000 7fb030c00640 1 -- 172.21.15.50:0/3196240937 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb01c093a00 con 0x7fb034160d50 2024-09-21T05:59:15.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.020+0000 7fb033e00640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb010073470 0x7fb010075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:15.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.020+0000 7fb033e00640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb010073470 0x7fb010075980 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb0240af000 tx=0x7fb02401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:15.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.024+0000 7fb030c00640 1 -- 172.21.15.50:0/3196240937 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb01c05df50 con 0x7fb034160d50 2024-09-21T05:59:15.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.224+0000 7fb03ae00640 1 -- 172.21.15.50:0/3196240937 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb03406b540 con 0x7fb010073470 2024-09-21T05:59:15.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.228+0000 7fb030c00640 1 -- 172.21.15.50:0/3196240937 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+44 (secure 0 0 0) 0x7fb024009480 con 0x7fb010073470 2024-09-21T05:59:15.232 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:15.232 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T05:59:15.232 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:15.232 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ss1clone1" 2024-09-21T05:59:15.232 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:15.232 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T05:59:15.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.232+0000 7fb03ae00640 1 -- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb010073470 msgr2=0x7fb010075980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:15.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.232+0000 7fb03ae00640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb010073470 0x7fb010075980 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb0240af000 tx=0x7fb02401b060 comp rx=0 tx=0).stop 2024-09-21T05:59:15.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.232+0000 7fb03ae00640 1 -- 172.21.15.50:0/3196240937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb034160d50 msgr2=0x7fb03406e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:15.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.232+0000 7fb03ae00640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb034160d50 0x7fb03406e2a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00c400 tx=0x7fb01c00c8c0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.232+0000 7fb03ae00640 1 -- 172.21.15.50:0/3196240937 shutdown_connections 2024-09-21T05:59:15.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.232+0000 7fb03ae00640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb03414d2a0 0x7fb034151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.232+0000 7fb03ae00640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb010073470 0x7fb010075980 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.232+0000 7fb03ae00640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb034160d50 0x7fb03406e2a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.232+0000 7fb03ae00640 1 --2- 172.21.15.50:0/3196240937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb034157030 0x7fb034154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.232+0000 7fb03ae00640 1 -- 172.21.15.50:0/3196240937 >> 172.21.15.50:0/3196240937 conn(0x7fb0340708c0 msgr2=0x7fb03415fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:15.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.232+0000 7fb03ae00640 1 -- 172.21.15.50:0/3196240937 shutdown_connections 2024-09-21T05:59:15.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.232+0000 7fb03ae00640 1 -- 172.21.15.50:0/3196240937 wait complete. 2024-09-21T05:59:15.254 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-21T05:59:15.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 -- 172.21.15.50:0/3982940784 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3e406f170 msgr2=0x7fc3e406f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:15.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 --2- 172.21.15.50:0/3982940784 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3e406f170 0x7fc3e406f580 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009bf0 tx=0x7fc3dc01cad0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 -- 172.21.15.50:0/3982940784 shutdown_connections 2024-09-21T05:59:15.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 --2- 172.21.15.50:0/3982940784 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3e406f170 0x7fc3e406f580 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 -- 172.21.15.50:0/3982940784 >> 172.21.15.50:0/3982940784 conn(0x7fc3e40b2c40 msgr2=0x7fc3e40b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:15.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 -- 172.21.15.50:0/3982940784 shutdown_connections 2024-09-21T05:59:15.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 -- 172.21.15.50:0/3982940784 wait complete. 2024-09-21T05:59:15.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 Processor -- start 2024-09-21T05:59:15.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 -- start start 2024-09-21T05:59:15.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e406f170 0x7fc3e4142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:15.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3e415c5d0 0x7fc3e40aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:15.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3e4151b80 0x7fc3e40ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:15.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e4169f00 con 0x7fc3e406f170 2024-09-21T05:59:15.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e4169d80 con 0x7fc3e415c5d0 2024-09-21T05:59:15.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.416+0000 7fc3e9800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e416a080 con 0x7fc3e4151b80 2024-09-21T05:59:15.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e406f170 0x7fc3e4142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:15.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e406f170 0x7fc3e4142590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58340/0 (socket says 172.21.15.50:58340) 2024-09-21T05:59:15.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e2a00640 1 -- 172.21.15.50:0/3304051233 learned_addr learned my addr 172.21.15.50:0/3304051233 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:15.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3dbe00640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3e415c5d0 0x7fc3e40aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:15.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e3400640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3e4151b80 0x7fc3e40ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:15.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e2a00640 1 -- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3e4151b80 msgr2=0x7fc3e40ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:15.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e2a00640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3e4151b80 0x7fc3e40ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e2a00640 1 -- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3e415c5d0 msgr2=0x7fc3e40aaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:15.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e2a00640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3e415c5d0 0x7fc3e40aaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:15.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e2a00640 1 -- 172.21.15.50:0/3304051233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e415b400 con 0x7fc3e406f170 2024-09-21T05:59:15.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e2a00640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e406f170 0x7fc3e4142590 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400c6e0 tx=0x7fc3d400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:15.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3daa00640 1 -- 172.21.15.50:0/3304051233 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4016020 con 0x7fc3e406f170 2024-09-21T05:59:15.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3daa00640 1 -- 172.21.15.50:0/3304051233 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3d400a930 con 0x7fc3e406f170 2024-09-21T05:59:15.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e9800640 1 -- 172.21.15.50:0/3304051233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e4157f40 con 0x7fc3e406f170 2024-09-21T05:59:15.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3daa00640 1 -- 172.21.15.50:0/3304051233 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d400ad80 con 0x7fc3e406f170 2024-09-21T05:59:15.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e9800640 1 -- 172.21.15.50:0/3304051233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e4158500 con 0x7fc3e406f170 2024-09-21T05:59:15.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3e9800640 1 -- 172.21.15.50:0/3304051233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e406f580 con 0x7fc3e406f170 2024-09-21T05:59:15.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.420+0000 7fc3daa00640 1 -- 172.21.15.50:0/3304051233 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 97438+0+0 (secure 0 0 0) 0x7fc3d4019020 con 0x7fc3e406f170 2024-09-21T05:59:15.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.424+0000 7fc3daa00640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc3c0073440 0x7fc3c0075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:15.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.424+0000 7fc3daa00640 1 -- 172.21.15.50:0/3304051233 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc3d4094850 con 0x7fc3e406f170 2024-09-21T05:59:15.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.424+0000 7fc3dbe00640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc3c0073440 0x7fc3c0075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:15.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.424+0000 7fc3dbe00640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc3c0073440 0x7fc3c0075950 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc007fd0 tx=0x7fc3cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:15.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.428+0000 7fc3daa00640 1 -- 172.21.15.50:0/3304051233 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc3d4020020 con 0x7fc3e406f170 2024-09-21T05:59:15.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:15.628+0000 7fc3e9800640 1 -- 172.21.15.50:0/3304051233 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fc3e406b540 con 0x7fc3c0073440 2024-09-21T05:59:16.982 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:59:16.985 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:59:19.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.456+0000 7fc3daa00640 1 -- 172.21.15.50:0/3304051233 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc3cc001440 con 0x7fc3c0073440 2024-09-21T05:59:19.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.460+0000 7fc3e9800640 1 -- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc3c0073440 msgr2=0x7fc3c0075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:19.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.460+0000 7fc3e9800640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc3c0073440 0x7fc3c0075950 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc007fd0 tx=0x7fc3cc009040 comp rx=0 tx=0).stop 2024-09-21T05:59:19.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.460+0000 7fc3e9800640 1 -- 172.21.15.50:0/3304051233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e406f170 msgr2=0x7fc3e4142590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:19.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.460+0000 7fc3e9800640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e406f170 0x7fc3e4142590 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400c6e0 tx=0x7fc3d400cba0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.460+0000 7fc3e9800640 1 -- 172.21.15.50:0/3304051233 shutdown_connections 2024-09-21T05:59:19.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.460+0000 7fc3e9800640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3e4151b80 0x7fc3e40ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.460+0000 7fc3e9800640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3e415c5d0 0x7fc3e40aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.460+0000 7fc3e9800640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc3c0073440 0x7fc3c0075950 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.460+0000 7fc3e9800640 1 --2- 172.21.15.50:0/3304051233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e406f170 0x7fc3e4142590 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.460+0000 7fc3e9800640 1 -- 172.21.15.50:0/3304051233 >> 172.21.15.50:0/3304051233 conn(0x7fc3e40b2c40 msgr2=0x7fc3e415e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:19.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.460+0000 7fc3e9800640 1 -- 172.21.15.50:0/3304051233 shutdown_connections 2024-09-21T05:59:19.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.460+0000 7fc3e9800640 1 -- 172.21.15.50:0/3304051233 wait complete. 2024-09-21T05:59:19.464 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:19.461+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:19.464 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:19.461+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:19.464 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:19.461+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:19.465 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:19.461+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:19.465 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:19.461+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:19.486 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:19.481+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:19.486 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:19.481+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:19.486 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:19.481+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:19.486 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:19.481+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:19.486 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:19.481+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:19.488 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T05:59:19.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:59:19.550 INFO:teuthology.orchestra.run.smithi050.stdout:2 2024-09-21T05:59:19.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:59:19.557 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T05:59:19.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.716+0000 7fa4ea200640 1 -- 172.21.15.50:0/1699381136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4e41527b0 msgr2=0x7fa4e4074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:19.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.716+0000 7fa4ea200640 1 --2- 172.21.15.50:0/1699381136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4e41527b0 0x7fa4e4074e30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4072e30 tx=0x7fa4cc01c800 comp rx=0 tx=0).stop 2024-09-21T05:59:19.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.716+0000 7fa4ea200640 1 -- 172.21.15.50:0/1699381136 shutdown_connections 2024-09-21T05:59:19.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.716+0000 7fa4ea200640 1 --2- 172.21.15.50:0/1699381136 >> v2:172.21.15.88:3300/0 conn(0x7fa4e40b4de0 0x7fa4e40b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.716+0000 7fa4ea200640 1 --2- 172.21.15.50:0/1699381136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4e41527b0 0x7fa4e4074e30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.716+0000 7fa4ea200640 1 --2- 172.21.15.50:0/1699381136 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4e40b5c70 0x7fa4e415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.716+0000 7fa4ea200640 1 -- 172.21.15.50:0/1699381136 >> 172.21.15.50:0/1699381136 conn(0x7fa4e40b2040 msgr2=0x7fa4e40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:19.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.716+0000 7fa4ea200640 1 -- 172.21.15.50:0/1699381136 shutdown_connections 2024-09-21T05:59:19.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.716+0000 7fa4ea200640 1 -- 172.21.15.50:0/1699381136 wait complete. 2024-09-21T05:59:19.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4ea200640 1 Processor -- start 2024-09-21T05:59:19.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4ea200640 1 -- start start 2024-09-21T05:59:19.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4ea200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4e40b4de0 0x7fa4e41466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:19.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4ea200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e40b5c70 0x7fa4e414b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4ea200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4e41527b0 0x7fa4e41f9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4ea200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e415f700 con 0x7fa4e41527b0 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4ea200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e415f580 con 0x7fa4e40b5c70 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4ea200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e415f880 con 0x7fa4e40b4de0 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4e41527b0 0x7fa4e41f9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4e41527b0 0x7fa4e41f9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59596/0 (socket says 172.21.15.50:59596) 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4e3e00640 1 -- 172.21.15.50:0/2718305522 learned_addr learned my addr 172.21.15.50:0/2718305522 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4dbe00640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e40b5c70 0x7fa4e414b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4e3400640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4e40b4de0 0x7fa4e41466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4e3e00640 1 -- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4e40b4de0 msgr2=0x7fa4e41466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4e3e00640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4e40b4de0 0x7fa4e41466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4e3e00640 1 -- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e40b5c70 msgr2=0x7fa4e414b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4e3e00640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e40b5c70 0x7fa4e414b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4e3e00640 1 -- 172.21.15.50:0/2718305522 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e41fa530 con 0x7fa4e41527b0 2024-09-21T05:59:19.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4dbe00640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e40b5c70 0x7fa4e414b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:19.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4e3e00640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4e41527b0 0x7fa4e41f9a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00ea30 tx=0x7fa4dc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:19.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4db400640 1 -- 172.21.15.50:0/2718305522 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00ce40 con 0x7fa4e41527b0 2024-09-21T05:59:19.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4db400640 1 -- 172.21.15.50:0/2718305522 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4dc004c20 con 0x7fa4e41527b0 2024-09-21T05:59:19.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4ea200640 1 -- 172.21.15.50:0/2718305522 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e41fa810 con 0x7fa4e41527b0 2024-09-21T05:59:19.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.720+0000 7fa4db400640 1 -- 172.21.15.50:0/2718305522 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc005020 con 0x7fa4e41527b0 2024-09-21T05:59:19.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.724+0000 7fa4ea200640 1 -- 172.21.15.50:0/2718305522 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e414af40 con 0x7fa4e41527b0 2024-09-21T05:59:19.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.724+0000 7fa4ea200640 1 -- 172.21.15.50:0/2718305522 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e415fda0 con 0x7fa4e41527b0 2024-09-21T05:59:19.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.724+0000 7fa4db400640 1 -- 172.21.15.50:0/2718305522 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 97438+0+0 (secure 0 0 0) 0x7fa4dc0051c0 con 0x7fa4e41527b0 2024-09-21T05:59:19.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.724+0000 7fa4db400640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4c0073440 0x7fa4c0075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:19.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.724+0000 7fa4db400640 1 -- 172.21.15.50:0/2718305522 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa4dc0955e0 con 0x7fa4e41527b0 2024-09-21T05:59:19.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.728+0000 7fa4e3400640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4c0073440 0x7fa4c0075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:19.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.728+0000 7fa4e3400640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4c0073440 0x7fa4c0075950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40058d0 tx=0x7fa4d4005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:19.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.732+0000 7fa4db400640 1 -- 172.21.15.50:0/2718305522 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa4dc060180 con 0x7fa4e41527b0 2024-09-21T05:59:19.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 -- 172.21.15.50:0/2718305522 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fa4e415ebe0 con 0x7fa4e41527b0 2024-09-21T05:59:19.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4db400640 1 -- 172.21.15.50:0/2718305522 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v33)=0 v33) ==== 131+0+0 (secure 0 0 0) 0x7fa4dc05fa80 con 0x7fa4e41527b0 2024-09-21T05:59:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 -- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4c0073440 msgr2=0x7fa4c0075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4c0073440 0x7fa4c0075950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40058d0 tx=0x7fa4d4005860 comp rx=0 tx=0).stop 2024-09-21T05:59:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 -- 172.21.15.50:0/2718305522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4e41527b0 msgr2=0x7fa4e41f9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4e41527b0 0x7fa4e41f9a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00ea30 tx=0x7fa4dc00eef0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 -- 172.21.15.50:0/2718305522 shutdown_connections 2024-09-21T05:59:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4c0073440 0x7fa4c0075950 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4e41527b0 0x7fa4e41f9a90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e40b5c70 0x7fa4e414b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 --2- 172.21.15.50:0/2718305522 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4e40b4de0 0x7fa4e41466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:19.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 -- 172.21.15.50:0/2718305522 >> 172.21.15.50:0/2718305522 conn(0x7fa4e40b2040 msgr2=0x7fa4e406e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:19.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 -- 172.21.15.50:0/2718305522 shutdown_connections 2024-09-21T05:59:19.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:19.908+0000 7fa4ea200640 1 -- 172.21.15.50:0/2718305522 wait complete. 2024-09-21T05:59:19.934 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:59:19.934 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:59:20.008 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:59:20.085 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:59:20.085 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:59:20.085 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:59:20.085 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:59:20.094 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:59:20.094 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:59:20.145 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:59:20.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.348+0000 7f69e3c00640 1 -- 172.21.15.50:0/1470983243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc1527b0 msgr2=0x7f69dc074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:20.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.348+0000 7f69e3c00640 1 --2- 172.21.15.50:0/1470983243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc1527b0 0x7f69dc074e30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009840 tx=0x7f69c401c760 comp rx=0 tx=0).stop 2024-09-21T05:59:20.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.348+0000 7f69e3c00640 1 -- 172.21.15.50:0/1470983243 shutdown_connections 2024-09-21T05:59:20.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.348+0000 7f69e3c00640 1 --2- 172.21.15.50:0/1470983243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc1527b0 0x7f69dc074e30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:20.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.348+0000 7f69e3c00640 1 --2- 172.21.15.50:0/1470983243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69dc0b5c70 0x7f69dc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:20.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.348+0000 7f69e3c00640 1 -- 172.21.15.50:0/1470983243 >> 172.21.15.50:0/1470983243 conn(0x7f69dc0b2040 msgr2=0x7f69dc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:20.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.348+0000 7f69e3c00640 1 -- 172.21.15.50:0/1470983243 shutdown_connections 2024-09-21T05:59:20.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.348+0000 7f69e3c00640 1 -- 172.21.15.50:0/1470983243 wait complete. 2024-09-21T05:59:20.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e3c00640 1 Processor -- start 2024-09-21T05:59:20.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e3c00640 1 -- start start 2024-09-21T05:59:20.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e3c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69dc0b4de0 0x7f69dc1f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:20.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e3c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69dc0b5c70 0x7f69dc0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:20.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e3c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc1527b0 0x7f69dc14ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:20.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e3c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc168c00 con 0x7f69dc0b4de0 2024-09-21T05:59:20.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e3c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc168a80 con 0x7f69dc1527b0 2024-09-21T05:59:20.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e3c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f69dc168d80 con 0x7f69dc0b5c70 2024-09-21T05:59:20.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69dc0b4de0 0x7f69dc1f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:20.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69dc0b4de0 0x7f69dc1f6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59604/0 (socket says 172.21.15.50:59604) 2024-09-21T05:59:20.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e1200640 1 -- 172.21.15.50:0/3633618235 learned_addr learned my addr 172.21.15.50:0/3633618235 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:20.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69d3e00640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69dc0b5c70 0x7f69dc0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:20.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e1c00640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc1527b0 0x7f69dc14ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:20.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e1200640 1 -- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69dc0b5c70 msgr2=0x7f69dc0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:20.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e1200640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69dc0b5c70 0x7f69dc0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:20.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e1200640 1 -- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc1527b0 msgr2=0x7f69dc14ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:20.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e1200640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc1527b0 0x7f69dc14ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:20.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e1200640 1 -- 172.21.15.50:0/3633618235 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc1f74e0 con 0x7f69dc0b4de0 2024-09-21T05:59:20.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e1200640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69dc0b4de0 0x7f69dc1f6fa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00e970 tx=0x7f69cc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:20.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69d9600640 1 -- 172.21.15.50:0/3633618235 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc00cd80 con 0x7f69dc0b4de0 2024-09-21T05:59:20.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69d9600640 1 -- 172.21.15.50:0/3633618235 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69cc004020 con 0x7f69dc0b4de0 2024-09-21T05:59:20.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69e3c00640 1 -- 172.21.15.50:0/3633618235 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69dc0ced40 con 0x7f69dc0b4de0 2024-09-21T05:59:20.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.352+0000 7f69d9600640 1 -- 172.21.15.50:0/3633618235 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc004470 con 0x7f69dc0b4de0 2024-09-21T05:59:20.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.356+0000 7f69e3c00640 1 -- 172.21.15.50:0/3633618235 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69dc0cf300 con 0x7f69dc0b4de0 2024-09-21T05:59:20.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.356+0000 7f69e3c00640 1 -- 172.21.15.50:0/3633618235 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69dc15bb60 con 0x7f69dc0b4de0 2024-09-21T05:59:20.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.356+0000 7f69d9600640 1 -- 172.21.15.50:0/3633618235 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 97438+0+0 (secure 0 0 0) 0x7f69cc003600 con 0x7f69dc0b4de0 2024-09-21T05:59:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.356+0000 7f69d9600640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f69b8073440 0x7f69b8075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.356+0000 7f69d9600640 1 -- 172.21.15.50:0/3633618235 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f69cc094fe0 con 0x7f69dc0b4de0 2024-09-21T05:59:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.360+0000 7f69d3e00640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f69b8073440 0x7f69b8075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:20.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.360+0000 7f69d3e00640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f69b8073440 0x7f69b8075950 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f69c40af000 tx=0x7f69c401afa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:20.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.364+0000 7f69d9600640 1 -- 172.21.15.50:0/3633618235 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f69cc05fc10 con 0x7f69dc0b4de0 2024-09-21T05:59:20.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.548+0000 7f69e3c00640 1 -- 172.21.15.50:0/3633618235 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f69dc155640 con 0x7f69dc0b4de0 2024-09-21T05:59:20.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69d9600640 1 -- 172.21.15.50:0/3633618235 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f69cc05f510 con 0x7f69dc0b4de0 2024-09-21T05:59:20.553 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T05:59:20.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69e3c00640 1 -- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f69b8073440 msgr2=0x7f69b8075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:20.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69e3c00640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f69b8073440 0x7f69b8075950 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f69c40af000 tx=0x7f69c401afa0 comp rx=0 tx=0).stop 2024-09-21T05:59:20.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69e3c00640 1 -- 172.21.15.50:0/3633618235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69dc0b4de0 msgr2=0x7f69dc1f6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:20.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69e3c00640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69dc0b4de0 0x7f69dc1f6fa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00e970 tx=0x7f69cc00ee30 comp rx=0 tx=0).stop 2024-09-21T05:59:20.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69e3c00640 1 -- 172.21.15.50:0/3633618235 shutdown_connections 2024-09-21T05:59:20.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69e3c00640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc1527b0 0x7f69dc14ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:20.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69e3c00640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69dc0b5c70 0x7f69dc0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:20.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69e3c00640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f69b8073440 0x7f69b8075950 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:20.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69e3c00640 1 --2- 172.21.15.50:0/3633618235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69dc0b4de0 0x7f69dc1f6fa0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:20.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69e3c00640 1 -- 172.21.15.50:0/3633618235 >> 172.21.15.50:0/3633618235 conn(0x7f69dc0b2040 msgr2=0x7f69dc14fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:20.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69e3c00640 1 -- 172.21.15.50:0/3633618235 shutdown_connections 2024-09-21T05:59:20.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:20.552+0000 7f69e3c00640 1 -- 172.21.15.50:0/3633618235 wait complete. 2024-09-21T05:59:20.576 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:59:20.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:20.576 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:59:22.439 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T05:59:22.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:22.439 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:59:22.456 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:59:22.456 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:59:22.456 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:59:22.466 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:59:22.466 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:59:22.466 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:59:22.467 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:59:22.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:59:22.505 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:59:22.505 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:59:22.505 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:59:22.522 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:59:22.522 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:59:22.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.708+0000 7f7237800640 1 -- 172.21.15.50:0/4230825954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72300749a0 msgr2=0x7f7230074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:22.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.708+0000 7f7237800640 1 --2- 172.21.15.50:0/4230825954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72300749a0 0x7f7230074db0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f72300cfc90 tx=0x7f722001c8c0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.708+0000 7f7237800640 1 -- 172.21.15.50:0/4230825954 shutdown_connections 2024-09-21T05:59:22.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.708+0000 7f7237800640 1 --2- 172.21.15.50:0/4230825954 >> v2:172.21.15.88:3300/0 conn(0x7f72300b3340 0x7f72300b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.708+0000 7f7237800640 1 --2- 172.21.15.50:0/4230825954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72300749a0 0x7f7230074db0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.708+0000 7f7237800640 1 -- 172.21.15.50:0/4230825954 >> 172.21.15.50:0/4230825954 conn(0x7f72300b2040 msgr2=0x7f72300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:22.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.708+0000 7f7237800640 1 -- 172.21.15.50:0/4230825954 shutdown_connections 2024-09-21T05:59:22.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.708+0000 7f7237800640 1 -- 172.21.15.50:0/4230825954 wait complete. 2024-09-21T05:59:22.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.708+0000 7f7237800640 1 Processor -- start 2024-09-21T05:59:22.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.708+0000 7f7237800640 1 -- start start 2024-09-21T05:59:22.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7237800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72300749a0 0x7f72300ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7237800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72300b3340 0x7f723012d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7237800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72300cefa0 0x7f723012d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7237800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7230169cd0 con 0x7f72300b3340 2024-09-21T05:59:22.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7237800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7230169b50 con 0x7f72300749a0 2024-09-21T05:59:22.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7237800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7230169e50 con 0x7f72300cefa0 2024-09-21T05:59:22.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7227e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72300b3340 0x7f723012d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7235800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72300cefa0 0x7f723012d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7234e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72300749a0 0x7f72300ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7227e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72300b3340 0x7f723012d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59632/0 (socket says 172.21.15.50:59632) 2024-09-21T05:59:22.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7227e00640 1 -- 172.21.15.50:0/2267774875 learned_addr learned my addr 172.21.15.50:0/2267774875 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:22.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7227e00640 1 -- 172.21.15.50:0/2267774875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72300cefa0 msgr2=0x7f723012d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:22.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7227e00640 1 --2- 172.21.15.50:0/2267774875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72300cefa0 0x7f723012d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7227e00640 1 -- 172.21.15.50:0/2267774875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72300749a0 msgr2=0x7f72300ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:22.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7227e00640 1 --2- 172.21.15.50:0/2267774875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72300749a0 0x7f72300ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7227e00640 1 -- 172.21.15.50:0/2267774875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72301fd240 con 0x7f72300b3340 2024-09-21T05:59:22.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7227e00640 1 --2- 172.21.15.50:0/2267774875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72300b3340 0x7f723012d490 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f721800b3e0 tx=0x7f721800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:22.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f722d600640 1 -- 172.21.15.50:0/2267774875 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218015020 con 0x7f72300b3340 2024-09-21T05:59:22.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f722d600640 1 -- 172.21.15.50:0/2267774875 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7218011920 con 0x7f72300b3340 2024-09-21T05:59:22.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7237800640 1 -- 172.21.15.50:0/2267774875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72300ccd90 con 0x7f72300b3340 2024-09-21T05:59:22.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f722d600640 1 -- 172.21.15.50:0/2267774875 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218011d70 con 0x7f72300b3340 2024-09-21T05:59:22.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7237800640 1 -- 172.21.15.50:0/2267774875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72300cd570 con 0x7f72300b3340 2024-09-21T05:59:22.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.712+0000 7f7237800640 1 -- 172.21.15.50:0/2267774875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72301f6830 con 0x7f72300b3340 2024-09-21T05:59:22.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.716+0000 7f722d600640 1 -- 172.21.15.50:0/2267774875 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f7218018020 con 0x7f72300b3340 2024-09-21T05:59:22.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.716+0000 7f722d600640 1 --2- 172.21.15.50:0/2267774875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f720c073560 0x7f720c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.716+0000 7f722d600640 1 -- 172.21.15.50:0/2267774875 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f72180943a0 con 0x7f72300b3340 2024-09-21T05:59:22.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.716+0000 7f7234e00640 1 --2- 172.21.15.50:0/2267774875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f720c073560 0x7f720c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.716+0000 7f7234e00640 1 --2- 172.21.15.50:0/2267774875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f720c073560 0x7f720c075a70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f72200af000 tx=0x7f722001b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:22.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.720+0000 7f722d600640 1 -- 172.21.15.50:0/2267774875 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f72301f6830 con 0x7f72300b3340 2024-09-21T05:59:22.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.944+0000 7f7237800640 1 -- 172.21.15.50:0/2267774875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72300cd020 con 0x7f72300b3340 2024-09-21T05:59:22.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.944+0000 7f722d600640 1 -- 172.21.15.50:0/2267774875 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5217 (secure 0 0 0) 0x7f721801f030 con 0x7f72300b3340 2024-09-21T05:59:22.948 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:22.948 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":18,"btime":"2024-09-21T05:57:25:016672+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3475589656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3475589656},{"type":"v1","addr":"172.21.15.88:6835","nonce":3475589656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/88740447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":88740447},{"type":"v1","addr":"172.21.15.88:6837","nonce":88740447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4798,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/4086930068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":4086930068},{"type":"v1","addr":"172.21.15.50:6833","nonce":4086930068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:57:16.325238+0000","modified":"2024-09-21T05:57:19.359942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4813},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4813":{"gid":4813,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/2722641195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2722641195},{"type":"v1","addr":"172.21.15.50:6835","nonce":2722641195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4813,"qdb_cluster":[4813]},"id":2}]} 2024-09-21T05:59:22.948 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 18 2024-09-21T05:59:22.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.948+0000 7f7237800640 1 -- 172.21.15.50:0/2267774875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f720c073560 msgr2=0x7f720c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:22.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.948+0000 7f7237800640 1 --2- 172.21.15.50:0/2267774875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f720c073560 0x7f720c075a70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f72200af000 tx=0x7f722001b100 comp rx=0 tx=0).stop 2024-09-21T05:59:22.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.948+0000 7f7237800640 1 -- 172.21.15.50:0/2267774875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72300b3340 msgr2=0x7f723012d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:22.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.948+0000 7f7237800640 1 --2- 172.21.15.50:0/2267774875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72300b3340 0x7f723012d490 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f721800b3e0 tx=0x7f721800b8a0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.948+0000 7f7237800640 1 -- 172.21.15.50:0/2267774875 shutdown_connections 2024-09-21T05:59:22.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.948+0000 7f7237800640 1 --2- 172.21.15.50:0/2267774875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72300cefa0 0x7f723012d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.948+0000 7f7237800640 1 --2- 172.21.15.50:0/2267774875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f720c073560 0x7f720c075a70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.948+0000 7f7237800640 1 --2- 172.21.15.50:0/2267774875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72300b3340 0x7f723012d490 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.948+0000 7f7237800640 1 --2- 172.21.15.50:0/2267774875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72300749a0 0x7f72300ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.948+0000 7f7237800640 1 -- 172.21.15.50:0/2267774875 >> 172.21.15.50:0/2267774875 conn(0x7f72300b2040 msgr2=0x7f723015b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:22.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.948+0000 7f7237800640 1 -- 172.21.15.50:0/2267774875 shutdown_connections 2024-09-21T05:59:22.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:22.948+0000 7f7237800640 1 -- 172.21.15.50:0/2267774875 wait complete. 2024-09-21T05:59:22.971 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:59:23.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.127+0000 7fea2a400640 1 -- 172.21.15.50:0/2643067279 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea24074950 msgr2=0x7fea240b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.127+0000 7fea2a400640 1 --2- 172.21.15.50:0/2643067279 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea24074950 0x7fea240b5260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009bf0 tx=0x7fea1c01ca70 comp rx=0 tx=0).stop 2024-09-21T05:59:23.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.127+0000 7fea2a400640 1 -- 172.21.15.50:0/2643067279 shutdown_connections 2024-09-21T05:59:23.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.127+0000 7fea2a400640 1 --2- 172.21.15.50:0/2643067279 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea24074950 0x7fea240b5260 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.127+0000 7fea2a400640 1 -- 172.21.15.50:0/2643067279 >> 172.21.15.50:0/2643067279 conn(0x7fea240b4560 msgr2=0x7fea240d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:23.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.127+0000 7fea2a400640 1 -- 172.21.15.50:0/2643067279 shutdown_connections 2024-09-21T05:59:23.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.127+0000 7fea2a400640 1 -- 172.21.15.50:0/2643067279 wait complete. 2024-09-21T05:59:23.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.127+0000 7fea2a400640 1 Processor -- start 2024-09-21T05:59:23.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.127+0000 7fea2a400640 1 -- start start 2024-09-21T05:59:23.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea2a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea241536f0 0x7fea24147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea2a400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea24160060 0x7fea240d1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea2a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea241565b0 0x7fea240d1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea2a400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fea2416ae70 con 0x7fea241565b0 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea2a400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fea2416acf0 con 0x7fea241536f0 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea2a400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fea2416aff0 con 0x7fea24160060 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea29a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea241565b0 0x7fea240d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea29a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea241565b0 0x7fea240d1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59648/0 (socket says 172.21.15.50:59648) 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea29a00640 1 -- 172.21.15.50:0/6556923 learned_addr learned my addr 172.21.15.50:0/6556923 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea29000640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea241536f0 0x7fea24147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea23e00640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea24160060 0x7fea240d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea29a00640 1 -- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea24160060 msgr2=0x7fea240d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea29a00640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea24160060 0x7fea240d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea29a00640 1 -- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea241536f0 msgr2=0x7fea24147ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea29a00640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea241536f0 0x7fea24147ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea29a00640 1 -- 172.21.15.50:0/6556923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2414b200 con 0x7fea241565b0 2024-09-21T05:59:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea29000640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea241536f0 0x7fea24147ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:23.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea23e00640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea24160060 0x7fea240d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:23.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea29a00640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea241565b0 0x7fea240d1660 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fea1c01a6a0 tx=0x7fea1c0a5b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:23.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea20c00640 1 -- 172.21.15.50:0/6556923 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0bb070 con 0x7fea241565b0 2024-09-21T05:59:23.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea2a400640 1 -- 172.21.15.50:0/6556923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea240d22e0 con 0x7fea241565b0 2024-09-21T05:59:23.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea20c00640 1 -- 172.21.15.50:0/6556923 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea1c004d40 con 0x7fea241565b0 2024-09-21T05:59:23.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea20c00640 1 -- 172.21.15.50:0/6556923 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0b2040 con 0x7fea241565b0 2024-09-21T05:59:23.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea2a400640 1 -- 172.21.15.50:0/6556923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea24151f80 con 0x7fea241565b0 2024-09-21T05:59:23.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.131+0000 7fea2a400640 1 -- 172.21.15.50:0/6556923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea24151a90 con 0x7fea241565b0 2024-09-21T05:59:23.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.135+0000 7fea20c00640 1 -- 172.21.15.50:0/6556923 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fea1c0041a0 con 0x7fea241565b0 2024-09-21T05:59:23.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.135+0000 7fea20c00640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea04073590 0x7fea04075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.135+0000 7fea20c00640 1 -- 172.21.15.50:0/6556923 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fea1c12e5c0 con 0x7fea241565b0 2024-09-21T05:59:23.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.135+0000 7fea29000640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea04073590 0x7fea04075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.135+0000 7fea29000640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea04073590 0x7fea04075aa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea18001230 tx=0x7fea18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:23.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.139+0000 7fea20c00640 1 -- 172.21.15.50:0/6556923 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fea2414b200 con 0x7fea241565b0 2024-09-21T05:59:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.351+0000 7fea2a400640 1 -- 172.21.15.50:0/6556923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea240b5260 con 0x7fea241565b0 2024-09-21T05:59:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea20c00640 1 -- 172.21.15.50:0/6556923 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5217 (secure 0 0 0) 0x7fea1c0166f0 con 0x7fea241565b0 2024-09-21T05:59:23.358 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:23.358 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":18,"btime":"2024-09-21T05:57:25:016672+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3475589656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3475589656},{"type":"v1","addr":"172.21.15.88:6835","nonce":3475589656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/88740447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":88740447},{"type":"v1","addr":"172.21.15.88:6837","nonce":88740447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4798,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/4086930068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":4086930068},{"type":"v1","addr":"172.21.15.50:6833","nonce":4086930068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:57:16.325238+0000","modified":"2024-09-21T05:57:19.359942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4813},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4813":{"gid":4813,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/2722641195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2722641195},{"type":"v1","addr":"172.21.15.50:6835","nonce":2722641195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4813,"qdb_cluster":[4813]},"id":2}]} 2024-09-21T05:59:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 18 2024-09-21T05:59:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea2a400640 1 -- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea04073590 msgr2=0x7fea04075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea2a400640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea04073590 0x7fea04075aa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea18001230 tx=0x7fea18009040 comp rx=0 tx=0).stop 2024-09-21T05:59:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea2a400640 1 -- 172.21.15.50:0/6556923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea241565b0 msgr2=0x7fea240d1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea2a400640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea241565b0 0x7fea240d1660 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fea1c01a6a0 tx=0x7fea1c0a5b90 comp rx=0 tx=0).stop 2024-09-21T05:59:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea2a400640 1 -- 172.21.15.50:0/6556923 shutdown_connections 2024-09-21T05:59:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea2a400640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea04073590 0x7fea04075aa0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea2a400640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea241565b0 0x7fea240d1660 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea2a400640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea24160060 0x7fea240d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea2a400640 1 --2- 172.21.15.50:0/6556923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea241536f0 0x7fea24147ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea2a400640 1 -- 172.21.15.50:0/6556923 >> 172.21.15.50:0/6556923 conn(0x7fea240b4560 msgr2=0x7fea2414f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea2a400640 1 -- 172.21.15.50:0/6556923 shutdown_connections 2024-09-21T05:59:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.355+0000 7fea2a400640 1 -- 172.21.15.50:0/6556923 wait complete. 2024-09-21T05:59:23.380 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:59:23.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.535+0000 7fc4aba00640 1 -- 172.21.15.50:0/3730455626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a40b4de0 msgr2=0x7fc4a40b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.535+0000 7fc4aba00640 1 --2- 172.21.15.50:0/3730455626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a40b4de0 0x7fc4a40b51f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009c80 tx=0x7fc49c01cce0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.535+0000 7fc4aba00640 1 -- 172.21.15.50:0/3730455626 shutdown_connections 2024-09-21T05:59:23.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.535+0000 7fc4aba00640 1 --2- 172.21.15.50:0/3730455626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a40b4de0 0x7fc4a40b51f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.535+0000 7fc4aba00640 1 --2- 172.21.15.50:0/3730455626 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4a41527b0 0x7fc4a415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.535+0000 7fc4aba00640 1 -- 172.21.15.50:0/3730455626 >> 172.21.15.50:0/3730455626 conn(0x7fc4a40b2040 msgr2=0x7fc4a40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:23.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.535+0000 7fc4aba00640 1 -- 172.21.15.50:0/3730455626 shutdown_connections 2024-09-21T05:59:23.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.535+0000 7fc4aba00640 1 -- 172.21.15.50:0/3730455626 wait complete. 2024-09-21T05:59:23.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.535+0000 7fc4aba00640 1 Processor -- start 2024-09-21T05:59:23.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.535+0000 7fc4aba00640 1 -- start start 2024-09-21T05:59:23.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4aba00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4a40b4de0 0x7fc4a4141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4aba00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a40b5c70 0x7fc4a40cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4aba00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a41527b0 0x7fc4a41f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4aba00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a4168a00 con 0x7fc4a41527b0 2024-09-21T05:59:23.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4aba00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a4168880 con 0x7fc4a40b5c70 2024-09-21T05:59:23.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4aba00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a4168b80 con 0x7fc4a40b4de0 2024-09-21T05:59:23.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a41527b0 0x7fc4a41f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a41527b0 0x7fc4a41f8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59662/0 (socket says 172.21.15.50:59662) 2024-09-21T05:59:23.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a9a00640 1 -- 172.21.15.50:0/1833915959 learned_addr learned my addr 172.21.15.50:0/1833915959 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:23.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a9000640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4a40b4de0 0x7fc4a4141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a3e00640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a40b5c70 0x7fc4a40cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a9a00640 1 -- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4a40b4de0 msgr2=0x7fc4a4141d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a9a00640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4a40b4de0 0x7fc4a4141d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a9a00640 1 -- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a40b5c70 msgr2=0x7fc4a40cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a9a00640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a40b5c70 0x7fc4a40cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a9a00640 1 -- 172.21.15.50:0/1833915959 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a41f8560 con 0x7fc4a41527b0 2024-09-21T05:59:23.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a9a00640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a41527b0 0x7fc4a41f8d00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc49c01c750 tx=0x7fc49c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:23.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a0c00640 1 -- 172.21.15.50:0/1833915959 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0b2040 con 0x7fc4a41527b0 2024-09-21T05:59:23.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a0c00640 1 -- 172.21.15.50:0/1833915959 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc49c004040 con 0x7fc4a41527b0 2024-09-21T05:59:23.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4a0c00640 1 -- 172.21.15.50:0/1833915959 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0043f0 con 0x7fc4a41527b0 2024-09-21T05:59:23.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4aba00640 1 -- 172.21.15.50:0/1833915959 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a41fc330 con 0x7fc4a41527b0 2024-09-21T05:59:23.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4aba00640 1 -- 172.21.15.50:0/1833915959 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a41fc8f0 con 0x7fc4a41527b0 2024-09-21T05:59:23.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.539+0000 7fc4aba00640 1 -- 172.21.15.50:0/1833915959 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a41fc0a0 con 0x7fc4a41527b0 2024-09-21T05:59:23.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.543+0000 7fc4a0c00640 1 -- 172.21.15.50:0/1833915959 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fc49c0bd030 con 0x7fc4a41527b0 2024-09-21T05:59:23.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.543+0000 7fc4a0c00640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc484073560 0x7fc484075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.543+0000 7fc4a0c00640 1 -- 172.21.15.50:0/1833915959 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc49c12e930 con 0x7fc4a41527b0 2024-09-21T05:59:23.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.543+0000 7fc4a9000640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc484073560 0x7fc484075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.543+0000 7fc4a9000640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc484073560 0x7fc484075a70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc498007fd0 tx=0x7fc498009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:23.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.547+0000 7fc4a0c00640 1 -- 172.21.15.50:0/1833915959 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc49c0c4060 con 0x7fc4a41527b0 2024-09-21T05:59:23.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.723+0000 7fc4aba00640 1 -- 172.21.15.50:0/1833915959 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4a40b51f0 con 0x7fc4a41527b0 2024-09-21T05:59:23.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4a0c00640 1 -- 172.21.15.50:0/1833915959 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16681 (secure 0 0 0) 0x7fc4a41f8560 con 0x7fc4a41527b0 2024-09-21T05:59:23.730 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:23.730 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":33,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:57:17.917372+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:57:14.083279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T05:57:15.292209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:59:23.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4aba00640 1 -- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc484073560 msgr2=0x7fc484075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4aba00640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc484073560 0x7fc484075a70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc498007fd0 tx=0x7fc498009040 comp rx=0 tx=0).stop 2024-09-21T05:59:23.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4aba00640 1 -- 172.21.15.50:0/1833915959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a41527b0 msgr2=0x7fc4a41f8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4aba00640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a41527b0 0x7fc4a41f8d00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc49c01c750 tx=0x7fc49c005ae0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4aba00640 1 -- 172.21.15.50:0/1833915959 shutdown_connections 2024-09-21T05:59:23.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4aba00640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc484073560 0x7fc484075a70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4aba00640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a41527b0 0x7fc4a41f8d00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4aba00640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a40b5c70 0x7fc4a40cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4aba00640 1 --2- 172.21.15.50:0/1833915959 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4a40b4de0 0x7fc4a4141d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4aba00640 1 -- 172.21.15.50:0/1833915959 >> 172.21.15.50:0/1833915959 conn(0x7fc4a40b2040 msgr2=0x7fc4a4154bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:23.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4aba00640 1 -- 172.21.15.50:0/1833915959 shutdown_connections 2024-09-21T05:59:23.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.727+0000 7fc4aba00640 1 -- 172.21.15.50:0/1833915959 wait complete. 2024-09-21T05:59:23.752 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T05:59:23.753 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:59:23.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.907+0000 7f15c2e00640 1 -- 172.21.15.50:0/1806672546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15bc0b4de0 msgr2=0x7f15bc0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.907+0000 7f15c2e00640 1 --2- 172.21.15.50:0/1806672546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15bc0b4de0 0x7f15bc0b51f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009c80 tx=0x7f15b401cce0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.907+0000 7f15c2e00640 1 -- 172.21.15.50:0/1806672546 shutdown_connections 2024-09-21T05:59:23.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.907+0000 7f15c2e00640 1 --2- 172.21.15.50:0/1806672546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15bc0b4de0 0x7f15bc0b51f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.907+0000 7f15c2e00640 1 --2- 172.21.15.50:0/1806672546 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15bc1527b0 0x7f15bc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.907+0000 7f15c2e00640 1 -- 172.21.15.50:0/1806672546 >> 172.21.15.50:0/1806672546 conn(0x7f15bc0b2040 msgr2=0x7f15bc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:23.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.907+0000 7f15c2e00640 1 -- 172.21.15.50:0/1806672546 shutdown_connections 2024-09-21T05:59:23.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.907+0000 7f15c2e00640 1 -- 172.21.15.50:0/1806672546 wait complete. 2024-09-21T05:59:23.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.907+0000 7f15c2e00640 1 Processor -- start 2024-09-21T05:59:23.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.907+0000 7f15c2e00640 1 -- start start 2024-09-21T05:59:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15c2e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15bc0b4de0 0x7f15bc149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15c2e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15bc0b5c70 0x7f15bc150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15c2e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15bc1527b0 0x7f15bc0cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15c2e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc168ab0 con 0x7f15bc0b5c70 2024-09-21T05:59:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15c2e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc168930 con 0x7f15bc1527b0 2024-09-21T05:59:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15c2e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f15bc168c30 con 0x7f15bc0b4de0 2024-09-21T05:59:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15b3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15bc0b5c70 0x7f15bc150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15b3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15bc0b5c70 0x7f15bc150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59676/0 (socket says 172.21.15.50:59676) 2024-09-21T05:59:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15b3e00640 1 -- 172.21.15.50:0/3624134340 learned_addr learned my addr 172.21.15.50:0/3624134340 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15bbe00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15bc0b4de0 0x7f15bc149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15c0e00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15bc1527b0 0x7f15bc0cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15b3e00640 1 -- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15bc0b4de0 msgr2=0x7f15bc149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15b3e00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15bc0b4de0 0x7f15bc149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15b3e00640 1 -- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15bc1527b0 msgr2=0x7f15bc0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15b3e00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15bc1527b0 0x7f15bc0cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15b3e00640 1 -- 172.21.15.50:0/3624134340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc149c60 con 0x7f15bc0b5c70 2024-09-21T05:59:23.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15b3e00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15bc0b5c70 0x7f15bc150210 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f15a400d720 tx=0x7f15a400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:23.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15b8c00640 1 -- 172.21.15.50:0/3624134340 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4017070 con 0x7f15bc0b5c70 2024-09-21T05:59:23.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15b8c00640 1 -- 172.21.15.50:0/3624134340 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15a4009b10 con 0x7f15bc0b5c70 2024-09-21T05:59:23.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15b8c00640 1 -- 172.21.15.50:0/3624134340 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4010460 con 0x7f15bc0b5c70 2024-09-21T05:59:23.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15c2e00640 1 -- 172.21.15.50:0/3624134340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc1562f0 con 0x7f15bc0b5c70 2024-09-21T05:59:23.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.911+0000 7f15c2e00640 1 -- 172.21.15.50:0/3624134340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc1568b0 con 0x7f15bc0b5c70 2024-09-21T05:59:23.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.915+0000 7f15c2e00640 1 -- 172.21.15.50:0/3624134340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15bc0b51f0 con 0x7f15bc0b5c70 2024-09-21T05:59:23.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.915+0000 7f15b8c00640 1 -- 172.21.15.50:0/3624134340 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f15a401a070 con 0x7f15bc0b5c70 2024-09-21T05:59:23.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.915+0000 7f15b8c00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1598073590 0x7f1598075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.915+0000 7f15b8c00640 1 -- 172.21.15.50:0/3624134340 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f15a4094960 con 0x7f15bc0b5c70 2024-09-21T05:59:23.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.915+0000 7f15bbe00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1598073590 0x7f1598075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.915+0000 7f15bbe00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1598073590 0x7f1598075aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f15ac007fd0 tx=0x7f15ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:23.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:23.919+0000 7f15b8c00640 1 -- 172.21.15.50:0/3624134340 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f15a405eda0 con 0x7f15bc0b5c70 2024-09-21T05:59:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.135+0000 7f15c2e00640 1 -- 172.21.15.50:0/3624134340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15bc0b2910 con 0x7f15bc0b5c70 2024-09-21T05:59:24.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.135+0000 7f15b8c00640 1 -- 172.21.15.50:0/3624134340 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5217 (secure 0 0 0) 0x7f15a40637d0 con 0x7f15bc0b5c70 2024-09-21T05:59:24.140 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:24.140 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":18,"btime":"2024-09-21T05:57:25:016672+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3475589656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3475589656},{"type":"v1","addr":"172.21.15.88:6835","nonce":3475589656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/88740447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":88740447},{"type":"v1","addr":"172.21.15.88:6837","nonce":88740447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4798,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/4086930068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":4086930068},{"type":"v1","addr":"172.21.15.50:6833","nonce":4086930068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:57:16.325238+0000","modified":"2024-09-21T05:57:19.359942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4813},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4813":{"gid":4813,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/2722641195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2722641195},{"type":"v1","addr":"172.21.15.50:6835","nonce":2722641195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4813,"qdb_cluster":[4813]},"id":2}]} 2024-09-21T05:59:24.141 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 18 2024-09-21T05:59:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.139+0000 7f15c2e00640 1 -- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1598073590 msgr2=0x7f1598075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.139+0000 7f15c2e00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1598073590 0x7f1598075aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f15ac007fd0 tx=0x7f15ac009040 comp rx=0 tx=0).stop 2024-09-21T05:59:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.139+0000 7f15c2e00640 1 -- 172.21.15.50:0/3624134340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15bc0b5c70 msgr2=0x7f15bc150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.139+0000 7f15c2e00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15bc0b5c70 0x7f15bc150210 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f15a400d720 tx=0x7f15a400dbe0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.139+0000 7f15c2e00640 1 -- 172.21.15.50:0/3624134340 shutdown_connections 2024-09-21T05:59:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.139+0000 7f15c2e00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15bc1527b0 0x7f15bc0cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.139+0000 7f15c2e00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1598073590 0x7f1598075aa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.139+0000 7f15c2e00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15bc0b5c70 0x7f15bc150210 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.139+0000 7f15c2e00640 1 --2- 172.21.15.50:0/3624134340 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15bc0b4de0 0x7f15bc149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.139+0000 7f15c2e00640 1 -- 172.21.15.50:0/3624134340 >> 172.21.15.50:0/3624134340 conn(0x7f15bc0b2040 msgr2=0x7f15bc154dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.139+0000 7f15c2e00640 1 -- 172.21.15.50:0/3624134340 shutdown_connections 2024-09-21T05:59:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.139+0000 7f15c2e00640 1 -- 172.21.15.50:0/3624134340 wait complete. 2024-09-21T05:59:24.163 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:59:24.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.319+0000 7f56c5400640 1 -- 172.21.15.50:0/3276792249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56c00b5c70 msgr2=0x7f56c015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.319+0000 7f56c5400640 1 --2- 172.21.15.50:0/3276792249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56c00b5c70 0x7f56c015bb60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f56b0009900 tx=0x7f56b001c820 comp rx=0 tx=0).stop 2024-09-21T05:59:24.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.319+0000 7f56c5400640 1 -- 172.21.15.50:0/3276792249 shutdown_connections 2024-09-21T05:59:24.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.319+0000 7f56c5400640 1 --2- 172.21.15.50:0/3276792249 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f56c01527b0 0x7f56c0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.319+0000 7f56c5400640 1 --2- 172.21.15.50:0/3276792249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56c00b5c70 0x7f56c015bb60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.319+0000 7f56c5400640 1 -- 172.21.15.50:0/3276792249 >> 172.21.15.50:0/3276792249 conn(0x7f56c00b2040 msgr2=0x7f56c00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:24.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.319+0000 7f56c5400640 1 -- 172.21.15.50:0/3276792249 shutdown_connections 2024-09-21T05:59:24.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.319+0000 7f56c5400640 1 -- 172.21.15.50:0/3276792249 wait complete. 2024-09-21T05:59:24.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.319+0000 7f56c5400640 1 Processor -- start 2024-09-21T05:59:24.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.319+0000 7f56c5400640 1 -- start start 2024-09-21T05:59:24.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56c5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56c00b4de0 0x7f56c014f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56c5400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f56c00b5c70 0x7f56c015a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56c5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c01527b0 0x7f56c015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56c5400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f56c0168c50 con 0x7f56c00b4de0 2024-09-21T05:59:24.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56c5400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f56c0168ad0 con 0x7f56c01527b0 2024-09-21T05:59:24.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56c5400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f56c0168dd0 con 0x7f56c00b5c70 2024-09-21T05:59:24.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56be800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56c00b4de0 0x7f56c014f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56be800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56c00b4de0 0x7f56c014f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59684/0 (socket says 172.21.15.50:59684) 2024-09-21T05:59:24.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56be800640 1 -- 172.21.15.50:0/1728759953 learned_addr learned my addr 172.21.15.50:0/1728759953 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:24.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56b7e00640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f56c00b5c70 0x7f56c015a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56bf200640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c01527b0 0x7f56c015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56be800640 1 -- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f56c00b5c70 msgr2=0x7f56c015a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56be800640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f56c00b5c70 0x7f56c015a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56be800640 1 -- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c01527b0 msgr2=0x7f56c015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56be800640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c01527b0 0x7f56c015ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56be800640 1 -- 172.21.15.50:0/1728759953 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c014fb30 con 0x7f56c00b4de0 2024-09-21T05:59:24.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56be800640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56c00b4de0 0x7f56c014f5f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f56b00b0040 tx=0x7f56b0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:24.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56b6a00640 1 -- 172.21.15.50:0/1728759953 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b00b2040 con 0x7f56c00b4de0 2024-09-21T05:59:24.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56b6a00640 1 -- 172.21.15.50:0/1728759953 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56b0005030 con 0x7f56c00b4de0 2024-09-21T05:59:24.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56b6a00640 1 -- 172.21.15.50:0/1728759953 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0005420 con 0x7f56c00b4de0 2024-09-21T05:59:24.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56c5400640 1 -- 172.21.15.50:0/1728759953 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c01f9cb0 con 0x7f56c00b4de0 2024-09-21T05:59:24.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56c5400640 1 -- 172.21.15.50:0/1728759953 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c01fa270 con 0x7f56c00b4de0 2024-09-21T05:59:24.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.323+0000 7f56c5400640 1 -- 172.21.15.50:0/1728759953 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c015bb60 con 0x7f56c00b4de0 2024-09-21T05:59:24.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.327+0000 7f56b6a00640 1 -- 172.21.15.50:0/1728759953 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f56b00be060 con 0x7f56c00b4de0 2024-09-21T05:59:24.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.327+0000 7f56b6a00640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f569c073590 0x7f569c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.327+0000 7f56b6a00640 1 -- 172.21.15.50:0/1728759953 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f56b012ddd0 con 0x7f56c00b4de0 2024-09-21T05:59:24.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.327+0000 7f56b7e00640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f569c073590 0x7f569c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.327+0000 7f56b7e00640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f569c073590 0x7f569c075aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f56c0150320 tx=0x7f56a800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:24.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.331+0000 7f56b6a00640 1 -- 172.21.15.50:0/1728759953 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f56c0075580 con 0x7f56c00b4de0 2024-09-21T05:59:24.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.547+0000 7f56c5400640 1 -- 172.21.15.50:0/1728759953 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56c0155cd0 con 0x7f56c00b4de0 2024-09-21T05:59:24.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.547+0000 7f56b6a00640 1 -- 172.21.15.50:0/1728759953 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5217 (secure 0 0 0) 0x7f56b00164d0 con 0x7f56c00b4de0 2024-09-21T05:59:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:24.553 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":18,"btime":"2024-09-21T05:57:25:016672+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3475589656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3475589656},{"type":"v1","addr":"172.21.15.88:6835","nonce":3475589656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/88740447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":88740447},{"type":"v1","addr":"172.21.15.88:6837","nonce":88740447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4798,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/4086930068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":4086930068},{"type":"v1","addr":"172.21.15.50:6833","nonce":4086930068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:57:16.325238+0000","modified":"2024-09-21T05:57:19.359942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4813},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4813":{"gid":4813,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/2722641195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2722641195},{"type":"v1","addr":"172.21.15.50:6835","nonce":2722641195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4813,"qdb_cluster":[4813]},"id":2}]} 2024-09-21T05:59:24.554 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 18 2024-09-21T05:59:24.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.551+0000 7f56c5400640 1 -- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f569c073590 msgr2=0x7f569c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.551+0000 7f56c5400640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f569c073590 0x7f569c075aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f56c0150320 tx=0x7f56a800b2e0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.551+0000 7f56c5400640 1 -- 172.21.15.50:0/1728759953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56c00b4de0 msgr2=0x7f56c014f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.551+0000 7f56c5400640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56c00b4de0 0x7f56c014f5f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f56b00b0040 tx=0x7f56b0004040 comp rx=0 tx=0).stop 2024-09-21T05:59:24.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.551+0000 7f56c5400640 1 -- 172.21.15.50:0/1728759953 shutdown_connections 2024-09-21T05:59:24.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.551+0000 7f56c5400640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c01527b0 0x7f56c015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.551+0000 7f56c5400640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f56c00b5c70 0x7f56c015a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.551+0000 7f56c5400640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f569c073590 0x7f569c075aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.551+0000 7f56c5400640 1 --2- 172.21.15.50:0/1728759953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56c00b4de0 0x7f56c014f5f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.551+0000 7f56c5400640 1 -- 172.21.15.50:0/1728759953 >> 172.21.15.50:0/1728759953 conn(0x7f56c00b2040 msgr2=0x7f56c00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:24.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.551+0000 7f56c5400640 1 -- 172.21.15.50:0/1728759953 shutdown_connections 2024-09-21T05:59:24.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.551+0000 7f56c5400640 1 -- 172.21.15.50:0/1728759953 wait complete. 2024-09-21T05:59:24.576 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:59:24.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.731+0000 7efee6e00640 1 -- 172.21.15.50:0/944313497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efee006fa70 msgr2=0x7efee015f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.731+0000 7efee6e00640 1 --2- 172.21.15.50:0/944313497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efee006fa70 0x7efee015f880 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efee0071970 tx=0x7efed801cd50 comp rx=0 tx=0).stop 2024-09-21T05:59:24.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.731+0000 7efee6e00640 1 -- 172.21.15.50:0/944313497 shutdown_connections 2024-09-21T05:59:24.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.731+0000 7efee6e00640 1 --2- 172.21.15.50:0/944313497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efee006fa70 0x7efee015f880 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.731+0000 7efee6e00640 1 --2- 172.21.15.50:0/944313497 >> v2:172.21.15.88:3300/0 conn(0x7efee015c5d0 0x7efee0073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.731+0000 7efee6e00640 1 -- 172.21.15.50:0/944313497 >> 172.21.15.50:0/944313497 conn(0x7efee00b2c40 msgr2=0x7efee00b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:24.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.731+0000 7efee6e00640 1 -- 172.21.15.50:0/944313497 shutdown_connections 2024-09-21T05:59:24.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.731+0000 7efee6e00640 1 -- 172.21.15.50:0/944313497 wait complete. 2024-09-21T05:59:24.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.731+0000 7efee6e00640 1 Processor -- start 2024-09-21T05:59:24.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.731+0000 7efee6e00640 1 -- start start 2024-09-21T05:59:24.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efee6e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efee006fa70 0x7efee014d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efee6e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee0074300 0x7efee014ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efee6e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efee015c5d0 0x7efee01f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efee6e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efee0168b10 con 0x7efee006fa70 2024-09-21T05:59:24.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efee6e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efee015ffd0 con 0x7efee0074300 2024-09-21T05:59:24.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efee6e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7efee0168c90 con 0x7efee015c5d0 2024-09-21T05:59:24.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efedfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efee006fa70 0x7efee014d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efedfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efee006fa70 0x7efee014d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59708/0 (socket says 172.21.15.50:59708) 2024-09-21T05:59:24.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efedfe00640 1 -- 172.21.15.50:0/1379957751 learned_addr learned my addr 172.21.15.50:0/1379957751 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:24.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efed7e00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee0074300 0x7efee014ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efee4e00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efee015c5d0 0x7efee01f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efedfe00640 1 -- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efee015c5d0 msgr2=0x7efee01f5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efedfe00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efee015c5d0 0x7efee01f5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efedfe00640 1 -- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee0074300 msgr2=0x7efee014ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efedfe00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee0074300 0x7efee014ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efedfe00640 1 -- 172.21.15.50:0/1379957751 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee00705e0 con 0x7efee006fa70 2024-09-21T05:59:24.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efed7e00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee0074300 0x7efee014ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:24.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efedfe00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efee006fa70 0x7efee014d130 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efed000b440 tx=0x7efed000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:24.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efedcc00640 1 -- 172.21.15.50:0/1379957751 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0015020 con 0x7efee006fa70 2024-09-21T05:59:24.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efedcc00640 1 -- 172.21.15.50:0/1379957751 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efed00119a0 con 0x7efee006fa70 2024-09-21T05:59:24.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efedcc00640 1 -- 172.21.15.50:0/1379957751 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0011df0 con 0x7efee006fa70 2024-09-21T05:59:24.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efee6e00640 1 -- 172.21.15.50:0/1379957751 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee014da30 con 0x7efee006fa70 2024-09-21T05:59:24.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efee6e00640 1 -- 172.21.15.50:0/1379957751 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee014dff0 con 0x7efee006fa70 2024-09-21T05:59:24.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.735+0000 7efee6e00640 1 -- 172.21.15.50:0/1379957751 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee0073dc0 con 0x7efee006fa70 2024-09-21T05:59:24.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.739+0000 7efedcc00640 1 -- 172.21.15.50:0/1379957751 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7efed0018020 con 0x7efee006fa70 2024-09-21T05:59:24.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.739+0000 7efedcc00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efebc073560 0x7efebc075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.739+0000 7efedcc00640 1 -- 172.21.15.50:0/1379957751 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efed0094a00 con 0x7efee006fa70 2024-09-21T05:59:24.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.739+0000 7efed7e00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efebc073560 0x7efebc075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.739+0000 7efed7e00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efebc073560 0x7efebc075a70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efee00b1fb0 tx=0x7efec8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:24.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.743+0000 7efedcc00640 1 -- 172.21.15.50:0/1379957751 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efed001f020 con 0x7efee006fa70 2024-09-21T05:59:24.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.919+0000 7efee6e00640 1 -- 172.21.15.50:0/1379957751 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efee00b0880 con 0x7efee006fa70 2024-09-21T05:59:24.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efedcc00640 1 -- 172.21.15.50:0/1379957751 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16681 (secure 0 0 0) 0x7efed000d040 con 0x7efee006fa70 2024-09-21T05:59:24.926 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:24.926 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":33,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:57:17.917372+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:57:14.083279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T05:57:15.292209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:59:24.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efee6e00640 1 -- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efebc073560 msgr2=0x7efebc075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efee6e00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efebc073560 0x7efebc075a70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efee00b1fb0 tx=0x7efec8008040 comp rx=0 tx=0).stop 2024-09-21T05:59:24.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efee6e00640 1 -- 172.21.15.50:0/1379957751 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efee006fa70 msgr2=0x7efee014d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efee6e00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efee006fa70 0x7efee014d130 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efed000b440 tx=0x7efed000b900 comp rx=0 tx=0).stop 2024-09-21T05:59:24.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efee6e00640 1 -- 172.21.15.50:0/1379957751 shutdown_connections 2024-09-21T05:59:24.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efee6e00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efee015c5d0 0x7efee01f5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efee6e00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee0074300 0x7efee014ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efee6e00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efebc073560 0x7efebc075a70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efee6e00640 1 --2- 172.21.15.50:0/1379957751 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efee006fa70 0x7efee014d130 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efee6e00640 1 -- 172.21.15.50:0/1379957751 >> 172.21.15.50:0/1379957751 conn(0x7efee00b2c40 msgr2=0x7efee014f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:24.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efee6e00640 1 -- 172.21.15.50:0/1379957751 shutdown_connections 2024-09-21T05:59:24.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:24.923+0000 7efee6e00640 1 -- 172.21.15.50:0/1379957751 wait complete. 2024-09-21T05:59:24.952 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:59:25.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.107+0000 7fcc5d600640 1 -- 172.21.15.50:0/1180752015 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc580b4de0 msgr2=0x7fcc580b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:25.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.107+0000 7fcc5d600640 1 --2- 172.21.15.50:0/1180752015 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc580b4de0 0x7fcc580b51f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009c80 tx=0x7fcc5001cd70 comp rx=0 tx=0).stop 2024-09-21T05:59:25.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.107+0000 7fcc5d600640 1 -- 172.21.15.50:0/1180752015 shutdown_connections 2024-09-21T05:59:25.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.107+0000 7fcc5d600640 1 --2- 172.21.15.50:0/1180752015 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc580b4de0 0x7fcc580b51f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.107+0000 7fcc5d600640 1 --2- 172.21.15.50:0/1180752015 >> v2:172.21.15.88:3300/0 conn(0x7fcc580b5c70 0x7fcc5815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.107+0000 7fcc5d600640 1 -- 172.21.15.50:0/1180752015 >> 172.21.15.50:0/1180752015 conn(0x7fcc580b2040 msgr2=0x7fcc580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:25.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.107+0000 7fcc5d600640 1 -- 172.21.15.50:0/1180752015 shutdown_connections 2024-09-21T05:59:25.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.107+0000 7fcc5d600640 1 -- 172.21.15.50:0/1180752015 wait complete. 2024-09-21T05:59:25.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc5d600640 1 Processor -- start 2024-09-21T05:59:25.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc5d600640 1 -- start start 2024-09-21T05:59:25.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc5d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc580b4de0 0x7fcc58149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:25.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc5d600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc580b5c70 0x7fcc58158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:25.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc5d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc581527b0 0x7fcc581504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:25.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc5d600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581689d0 con 0x7fcc580b4de0 2024-09-21T05:59:25.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc5d600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58168850 con 0x7fcc581527b0 2024-09-21T05:59:25.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc5d600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fcc58168b50 con 0x7fcc580b5c70 2024-09-21T05:59:25.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc56a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc580b4de0 0x7fcc58149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:25.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc56a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc580b4de0 0x7fcc58149a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59724/0 (socket says 172.21.15.50:59724) 2024-09-21T05:59:25.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc56a00640 1 -- 172.21.15.50:0/2036851949 learned_addr learned my addr 172.21.15.50:0/2036851949 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:25.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc57400640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc581527b0 0x7fcc581504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:25.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.111+0000 7fcc4fe00640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc580b5c70 0x7fcc58158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:25.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc56a00640 1 -- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc580b5c70 msgr2=0x7fcc58158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:25.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc56a00640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc580b5c70 0x7fcc58158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc56a00640 1 -- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc581527b0 msgr2=0x7fcc581504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:25.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc56a00640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc581527b0 0x7fcc581504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc56a00640 1 -- 172.21.15.50:0/2036851949 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5814a9c0 con 0x7fcc580b4de0 2024-09-21T05:59:25.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc56a00640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc580b4de0 0x7fcc58149a40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800e9a0 tx=0x7fcc4800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:25.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc4ea00640 1 -- 172.21.15.50:0/2036851949 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4800cd80 con 0x7fcc580b4de0 2024-09-21T05:59:25.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc4ea00640 1 -- 172.21.15.50:0/2036851949 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc48004020 con 0x7fcc580b4de0 2024-09-21T05:59:25.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc5d600640 1 -- 172.21.15.50:0/2036851949 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5814aca0 con 0x7fcc580b4de0 2024-09-21T05:59:25.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc4ea00640 1 -- 172.21.15.50:0/2036851949 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48004450 con 0x7fcc580b4de0 2024-09-21T05:59:25.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc5d600640 1 -- 172.21.15.50:0/2036851949 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5814b260 con 0x7fcc580b4de0 2024-09-21T05:59:25.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc5d600640 1 -- 172.21.15.50:0/2036851949 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc580b51f0 con 0x7fcc580b4de0 2024-09-21T05:59:25.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.115+0000 7fcc4ea00640 1 -- 172.21.15.50:0/2036851949 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fcc48003600 con 0x7fcc580b4de0 2024-09-21T05:59:25.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.119+0000 7fcc4ea00640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc34073590 0x7fcc34075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:25.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.119+0000 7fcc4ea00640 1 -- 172.21.15.50:0/2036851949 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fcc48095170 con 0x7fcc580b4de0 2024-09-21T05:59:25.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.119+0000 7fcc4fe00640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc34073590 0x7fcc34075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:25.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.119+0000 7fcc4fe00640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc34073590 0x7fcc34075aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcc580cdcb0 tx=0x7fcc40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:25.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.123+0000 7fcc4ea00640 1 -- 172.21.15.50:0/2036851949 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcc4805f580 con 0x7fcc580b4de0 2024-09-21T05:59:25.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.335+0000 7fcc5d600640 1 -- 172.21.15.50:0/2036851949 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcc5815a4c0 con 0x7fcc580b4de0 2024-09-21T05:59:25.339 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-09-21T05:59:25.335+0000 7f0d45000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:59:25.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.339+0000 7fcc4ea00640 1 -- 172.21.15.50:0/2036851949 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7fcc48063fb0 con 0x7fcc580b4de0 2024-09-21T05:59:25.341 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:59:25.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.339+0000 7fcc5d600640 1 -- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc34073590 msgr2=0x7fcc34075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:25.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.339+0000 7fcc5d600640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc34073590 0x7fcc34075aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcc580cdcb0 tx=0x7fcc40009040 comp rx=0 tx=0).stop 2024-09-21T05:59:25.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.339+0000 7fcc5d600640 1 -- 172.21.15.50:0/2036851949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc580b4de0 msgr2=0x7fcc58149a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:25.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.339+0000 7fcc5d600640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc580b4de0 0x7fcc58149a40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800e9a0 tx=0x7fcc4800ee60 comp rx=0 tx=0).stop 2024-09-21T05:59:25.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.339+0000 7fcc5d600640 1 -- 172.21.15.50:0/2036851949 shutdown_connections 2024-09-21T05:59:25.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.339+0000 7fcc5d600640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc581527b0 0x7fcc581504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.339+0000 7fcc5d600640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc580b5c70 0x7fcc58158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.339+0000 7fcc5d600640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc34073590 0x7fcc34075aa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.343+0000 7fcc5d600640 1 --2- 172.21.15.50:0/2036851949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc580b4de0 0x7fcc58149a40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.343+0000 7fcc5d600640 1 -- 172.21.15.50:0/2036851949 >> 172.21.15.50:0/2036851949 conn(0x7fcc580b2040 msgr2=0x7fcc580b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:25.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.343+0000 7fcc5d600640 1 -- 172.21.15.50:0/2036851949 shutdown_connections 2024-09-21T05:59:25.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.343+0000 7fcc5d600640 1 -- 172.21.15.50:0/2036851949 wait complete. 2024-09-21T05:59:25.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T05:59:25.500 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-09-21T05:59:25.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.543+0000 7fb2de200640 1 -- 172.21.15.50:0/3794281388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d80cd2e0 msgr2=0x7fb2d8070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:25.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.543+0000 7fb2de200640 1 --2- 172.21.15.50:0/3794281388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d80cd2e0 0x7fb2d8070e80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009900 tx=0x7fb2c801c8e0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.543+0000 7fb2de200640 1 -- 172.21.15.50:0/3794281388 shutdown_connections 2024-09-21T05:59:25.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.543+0000 7fb2de200640 1 --2- 172.21.15.50:0/3794281388 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2d80cf1b0 0x7fb2d80cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.543+0000 7fb2de200640 1 --2- 172.21.15.50:0/3794281388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d80cd2e0 0x7fb2d8070e80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.543+0000 7fb2de200640 1 -- 172.21.15.50:0/3794281388 >> 172.21.15.50:0/3794281388 conn(0x7fb2d80b2c40 msgr2=0x7fb2d80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:25.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.543+0000 7fb2de200640 1 -- 172.21.15.50:0/3794281388 shutdown_connections 2024-09-21T05:59:25.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.543+0000 7fb2de200640 1 -- 172.21.15.50:0/3794281388 wait complete. 2024-09-21T05:59:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.543+0000 7fb2de200640 1 Processor -- start 2024-09-21T05:59:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.543+0000 7fb2de200640 1 -- start start 2024-09-21T05:59:25.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2de200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2d80cd2e0 0x7fb2d80b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:25.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2de200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d80cf1b0 0x7fb2d813c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:25.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2de200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2d81602a0 0x7fb2d80b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:25.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2de200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8169e20 con 0x7fb2d80cf1b0 2024-09-21T05:59:25.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2de200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8169ca0 con 0x7fb2d81602a0 2024-09-21T05:59:25.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2de200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d8169fa0 con 0x7fb2d80cd2e0 2024-09-21T05:59:25.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2cfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d80cf1b0 0x7fb2d813c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:25.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2cfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d80cf1b0 0x7fb2d813c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59780/0 (socket says 172.21.15.50:59780) 2024-09-21T05:59:25.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2cfe00640 1 -- 172.21.15.50:0/739331485 learned_addr learned my addr 172.21.15.50:0/739331485 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:25.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2d7e00640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2d81602a0 0x7fb2d80b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:25.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2d7400640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2d80cd2e0 0x7fb2d80b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:25.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2cfe00640 1 -- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2d80cd2e0 msgr2=0x7fb2d80b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:25.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2cfe00640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2d80cd2e0 0x7fb2d80b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2cfe00640 1 -- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2d81602a0 msgr2=0x7fb2d80b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:25.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2cfe00640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2d81602a0 0x7fb2d80b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:25.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2cfe00640 1 -- 172.21.15.50:0/739331485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d813c740 con 0x7fb2d80cf1b0 2024-09-21T05:59:25.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2d7e00640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2d81602a0 0x7fb2d80b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:25.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2d7400640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2d80cd2e0 0x7fb2d80b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:25.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2cfe00640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d80cf1b0 0x7fb2d813c200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c400 tx=0x7fb2c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:25.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2cf400640 1 -- 172.21.15.50:0/739331485 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0017070 con 0x7fb2d80cf1b0 2024-09-21T05:59:25.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2cf400640 1 -- 172.21.15.50:0/739331485 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2c0003600 con 0x7fb2d80cf1b0 2024-09-21T05:59:25.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2cf400640 1 -- 172.21.15.50:0/739331485 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0003a50 con 0x7fb2d80cf1b0 2024-09-21T05:59:25.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2de200640 1 -- 172.21.15.50:0/739331485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d8070b40 con 0x7fb2d80cf1b0 2024-09-21T05:59:25.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2de200640 1 -- 172.21.15.50:0/739331485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d81fc450 con 0x7fb2d80cf1b0 2024-09-21T05:59:25.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.547+0000 7fb2de200640 1 -- 172.21.15.50:0/739331485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d80cf5c0 con 0x7fb2d80cf1b0 2024-09-21T05:59:25.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.551+0000 7fb2cf400640 1 -- 172.21.15.50:0/739331485 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fb2c001a070 con 0x7fb2d80cf1b0 2024-09-21T05:59:25.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.551+0000 7fb2cf400640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb2b4073560 0x7fb2b4075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:25.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.551+0000 7fb2cf400640 1 -- 172.21.15.50:0/739331485 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fb2c0093bd0 con 0x7fb2d80cf1b0 2024-09-21T05:59:25.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.551+0000 7fb2d7400640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb2b4073560 0x7fb2b4075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:25.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.551+0000 7fb2d7400640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb2b4073560 0x7fb2b4075a70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80af000 tx=0x7fb2c801b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:25.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.555+0000 7fb2cf400640 1 -- 172.21.15.50:0/739331485 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb2c005e700 con 0x7fb2d80cf1b0 2024-09-21T05:59:25.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:25.767+0000 7fb2de200640 1 -- 172.21.15.50:0/739331485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb2d8075f60 con 0x7fb2d80cf1b0 2024-09-21T05:59:26.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.339+0000 7fb2cf400640 1 -- 172.21.15.50:0/739331485 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7fb2c005e000 con 0x7fb2d80cf1b0 2024-09-21T05:59:26.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.343+0000 7fb2de200640 1 -- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb2b4073560 msgr2=0x7fb2b4075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:26.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.343+0000 7fb2de200640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb2b4073560 0x7fb2b4075a70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80af000 tx=0x7fb2c801b120 comp rx=0 tx=0).stop 2024-09-21T05:59:26.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.343+0000 7fb2de200640 1 -- 172.21.15.50:0/739331485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d80cf1b0 msgr2=0x7fb2d813c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:26.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.343+0000 7fb2de200640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d80cf1b0 0x7fb2d813c200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c400 tx=0x7fb2c000c8c0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.343+0000 7fb2de200640 1 -- 172.21.15.50:0/739331485 shutdown_connections 2024-09-21T05:59:26.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.343+0000 7fb2de200640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2d81602a0 0x7fb2d80b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.343+0000 7fb2de200640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb2b4073560 0x7fb2b4075a70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.343+0000 7fb2de200640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d80cf1b0 0x7fb2d813c200 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.343+0000 7fb2de200640 1 --2- 172.21.15.50:0/739331485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2d80cd2e0 0x7fb2d80b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.343+0000 7fb2de200640 1 -- 172.21.15.50:0/739331485 >> 172.21.15.50:0/739331485 conn(0x7fb2d80b2c40 msgr2=0x7fb2d815f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:26.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.343+0000 7fb2de200640 1 -- 172.21.15.50:0/739331485 shutdown_connections 2024-09-21T05:59:26.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.343+0000 7fb2de200640 1 -- 172.21.15.50:0/739331485 wait complete. 2024-09-21T05:59:26.369 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T05:59:26.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.527+0000 7f4b76400640 1 -- 172.21.15.50:0/2566538108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b700cefa0 msgr2=0x7f4b700cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:26.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.527+0000 7f4b76400640 1 --2- 172.21.15.50:0/2566538108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b700cefa0 0x7f4b700cf3b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009900 tx=0x7f4b5801c8e0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.527+0000 7f4b76400640 1 -- 172.21.15.50:0/2566538108 shutdown_connections 2024-09-21T05:59:26.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.527+0000 7f4b76400640 1 --2- 172.21.15.50:0/2566538108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b700cefa0 0x7f4b700cf3b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.527+0000 7f4b76400640 1 --2- 172.21.15.50:0/2566538108 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b700749a0 0x7f4b70074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.527+0000 7f4b76400640 1 -- 172.21.15.50:0/2566538108 >> 172.21.15.50:0/2566538108 conn(0x7f4b700b2040 msgr2=0x7f4b700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:26.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.527+0000 7f4b76400640 1 -- 172.21.15.50:0/2566538108 shutdown_connections 2024-09-21T05:59:26.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.527+0000 7f4b76400640 1 -- 172.21.15.50:0/2566538108 wait complete. 2024-09-21T05:59:26.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b76400640 1 Processor -- start 2024-09-21T05:59:26.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b76400640 1 -- start start 2024-09-21T05:59:26.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b76400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b700749a0 0x7f4b7015a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:26.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b76400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b700cefa0 0x7f4b70143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:26.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b76400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b70160330 0x7f4b700ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:26.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b76400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7016a060 con 0x7f4b70160330 2024-09-21T05:59:26.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b76400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70169ee0 con 0x7f4b700749a0 2024-09-21T05:59:26.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b76400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4b7016a1e0 con 0x7f4b700cefa0 2024-09-21T05:59:26.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b6fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b70160330 0x7f4b700ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:26.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b6fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b70160330 0x7f4b700ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59788/0 (socket says 172.21.15.50:59788) 2024-09-21T05:59:26.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b6fe00640 1 -- 172.21.15.50:0/950683447 learned_addr learned my addr 172.21.15.50:0/950683447 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:26.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b6f400640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b700749a0 0x7f4b7015a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:26.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.531+0000 7f4b67e00640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b700cefa0 0x7f4b70143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:26.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b6fe00640 1 -- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b700cefa0 msgr2=0x7f4b70143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:26.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b6fe00640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b700cefa0 0x7f4b70143ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b6fe00640 1 -- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b700749a0 msgr2=0x7f4b7015a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:26.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b6fe00640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b700749a0 0x7f4b7015a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b6fe00640 1 -- 172.21.15.50:0/950683447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b70144430 con 0x7f4b70160330 2024-09-21T05:59:26.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b6fe00640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b70160330 0x7f4b700ccd90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c760 tx=0x7f4b6800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:26.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b67400640 1 -- 172.21.15.50:0/950683447 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68003b80 con 0x7f4b70160330 2024-09-21T05:59:26.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b67400640 1 -- 172.21.15.50:0/950683447 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b68003d20 con 0x7f4b70160330 2024-09-21T05:59:26.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b67400640 1 -- 172.21.15.50:0/950683447 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6800aab0 con 0x7f4b70160330 2024-09-21T05:59:26.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b76400640 1 -- 172.21.15.50:0/950683447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b700cd900 con 0x7f4b70160330 2024-09-21T05:59:26.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b76400640 1 -- 172.21.15.50:0/950683447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b700aec60 con 0x7f4b70160330 2024-09-21T05:59:26.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b76400640 1 -- 172.21.15.50:0/950683447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b700cd670 con 0x7f4b70160330 2024-09-21T05:59:26.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.535+0000 7f4b67400640 1 -- 172.21.15.50:0/950683447 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f4b6800ac50 con 0x7f4b70160330 2024-09-21T05:59:26.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.539+0000 7f4b67400640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4b4c073560 0x7f4b4c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:26.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.539+0000 7f4b67400640 1 -- 172.21.15.50:0/950683447 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f4b68094610 con 0x7f4b70160330 2024-09-21T05:59:26.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.539+0000 7f4b6f400640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4b4c073560 0x7f4b4c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:26.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.539+0000 7f4b6f400640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4b4c073560 0x7f4b4c075a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009b40 tx=0x7f4b600091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:26.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.543+0000 7f4b67400640 1 -- 172.21.15.50:0/950683447 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4b6805e920 con 0x7f4b70160330 2024-09-21T05:59:26.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:26.719+0000 7f4b76400640 1 -- 172.21.15.50:0/950683447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4b70074db0 con 0x7f4b70160330 2024-09-21T05:59:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.347+0000 7f4b67400640 1 -- 172.21.15.50:0/950683447 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7f4b68063350 con 0x7f4b70160330 2024-09-21T05:59:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-09-21T05:59:27.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.351+0000 7f4b76400640 1 -- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4b4c073560 msgr2=0x7f4b4c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:27.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.351+0000 7f4b76400640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4b4c073560 0x7f4b4c075a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009b40 tx=0x7f4b600091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:27.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.351+0000 7f4b76400640 1 -- 172.21.15.50:0/950683447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b70160330 msgr2=0x7f4b700ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:27.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.351+0000 7f4b76400640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b70160330 0x7f4b700ccd90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c760 tx=0x7f4b6800cc20 comp rx=0 tx=0).stop 2024-09-21T05:59:27.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.351+0000 7f4b76400640 1 -- 172.21.15.50:0/950683447 shutdown_connections 2024-09-21T05:59:27.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.351+0000 7f4b76400640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4b4c073560 0x7f4b4c075a70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:27.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.351+0000 7f4b76400640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b70160330 0x7f4b700ccd90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:27.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.351+0000 7f4b76400640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b700cefa0 0x7f4b70143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:27.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.351+0000 7f4b76400640 1 --2- 172.21.15.50:0/950683447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b700749a0 0x7f4b7015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:27.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.351+0000 7f4b76400640 1 -- 172.21.15.50:0/950683447 >> 172.21.15.50:0/950683447 conn(0x7f4b700b2040 msgr2=0x7f4b70160740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:27.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.351+0000 7f4b76400640 1 -- 172.21.15.50:0/950683447 shutdown_connections 2024-09-21T05:59:27.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.351+0000 7f4b76400640 1 -- 172.21.15.50:0/950683447 wait complete. 2024-09-21T05:59:27.381 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T05:59:27.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.547+0000 7fb965000640 1 -- 172.21.15.50:0/1876487515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb944003c40 msgr2=0x7fb944004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:27.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.547+0000 7fb965000640 1 --2- 172.21.15.50:0/1876487515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb944003c40 0x7fb944004090 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb950009b40 tx=0x7fb95001d250 comp rx=0 tx=0).stop 2024-09-21T05:59:27.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.547+0000 7fb965000640 1 -- 172.21.15.50:0/1876487515 shutdown_connections 2024-09-21T05:59:27.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.547+0000 7fb965000640 1 --2- 172.21.15.50:0/1876487515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb944003c40 0x7fb944004090 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:27.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.547+0000 7fb965000640 1 -- 172.21.15.50:0/1876487515 >> 172.21.15.50:0/1876487515 conn(0x7fb9600b4560 msgr2=0x7fb9600d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:27.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.547+0000 7fb965000640 1 -- 172.21.15.50:0/1876487515 shutdown_connections 2024-09-21T05:59:27.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.547+0000 7fb965000640 1 -- 172.21.15.50:0/1876487515 wait complete. 2024-09-21T05:59:27.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.547+0000 7fb965000640 1 Processor -- start 2024-09-21T05:59:27.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.547+0000 7fb965000640 1 -- start start 2024-09-21T05:59:27.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb965000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb944003c40 0x7fb960154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:27.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb965000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9600cb450 0x7fb960151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:27.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb965000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9601566a0 0x7fb9600b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:27.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb965000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb9600b5970 con 0x7fb9600cb450 2024-09-21T05:59:27.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb965000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9600b57f0 con 0x7fb944003c40 2024-09-21T05:59:27.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb965000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb9600b5af0 con 0x7fb9601566a0 2024-09-21T05:59:27.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb956a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9600cb450 0x7fb960151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:27.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb956a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9600cb450 0x7fb960151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49036/0 (socket says 172.21.15.50:49036) 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb956a00640 1 -- 172.21.15.50:0/2032024469 learned_addr learned my addr 172.21.15.50:0/2032024469 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb95fe00640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9601566a0 0x7fb9600b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb95f400640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb944003c40 0x7fb960154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb956a00640 1 -- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9601566a0 msgr2=0x7fb9600b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb956a00640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9601566a0 0x7fb9600b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb956a00640 1 -- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb944003c40 msgr2=0x7fb960154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb956a00640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb944003c40 0x7fb960154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb956a00640 1 -- 172.21.15.50:0/2032024469 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb960155470 con 0x7fb9600cb450 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb956a00640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9600cb450 0x7fb960151ff0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb948006630 tx=0x7fb9480039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb957e00640 1 -- 172.21.15.50:0/2032024469 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948015070 con 0x7fb9600cb450 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb957e00640 1 -- 172.21.15.50:0/2032024469 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb94800f960 con 0x7fb9600cb450 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb957e00640 1 -- 172.21.15.50:0/2032024469 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94800fdb0 con 0x7fb9600cb450 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb965000640 1 -- 172.21.15.50:0/2032024469 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb960155750 con 0x7fb9600cb450 2024-09-21T05:59:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb965000640 1 -- 172.21.15.50:0/2032024469 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9600c72d0 con 0x7fb9600cb450 2024-09-21T05:59:27.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.551+0000 7fb965000640 1 -- 172.21.15.50:0/2032024469 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9601551e0 con 0x7fb9600cb450 2024-09-21T05:59:27.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.555+0000 7fb957e00640 1 -- 172.21.15.50:0/2032024469 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fb948011430 con 0x7fb9600cb450 2024-09-21T05:59:27.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.555+0000 7fb957e00640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb93c073500 0x7fb93c075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:27.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.555+0000 7fb957e00640 1 -- 172.21.15.50:0/2032024469 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5357+0+0 (secure 0 0 0) 0x7fb9480955e0 con 0x7fb9600cb450 2024-09-21T05:59:27.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.555+0000 7fb95f400640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb93c073500 0x7fb93c075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:27.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.555+0000 7fb95f400640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb93c073500 0x7fb93c075a10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb9500ad000 tx=0x7fb950019ff0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:27.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.559+0000 7fb957e00640 1 -- 172.21.15.50:0/2032024469 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb96015d9c0 con 0x7fb9600cb450 2024-09-21T05:59:27.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:27.739+0000 7fb965000640 1 -- 172.21.15.50:0/2032024469 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb9600718b0 con 0x7fb9600cb450 2024-09-21T05:59:28.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.347+0000 7fb957e00640 1 -- 172.21.15.50:0/2032024469 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7fb9480601c0 con 0x7fb9600cb450 2024-09-21T05:59:28.351 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-09-21T05:59:28.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.351+0000 7fb965000640 1 -- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb93c073500 msgr2=0x7fb93c075a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:28.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.351+0000 7fb965000640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb93c073500 0x7fb93c075a10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb9500ad000 tx=0x7fb950019ff0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.351+0000 7fb965000640 1 -- 172.21.15.50:0/2032024469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9600cb450 msgr2=0x7fb960151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:28.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.351+0000 7fb965000640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9600cb450 0x7fb960151ff0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb948006630 tx=0x7fb9480039d0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.351+0000 7fb965000640 1 -- 172.21.15.50:0/2032024469 shutdown_connections 2024-09-21T05:59:28.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.351+0000 7fb965000640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9601566a0 0x7fb9600b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.351+0000 7fb965000640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb93c073500 0x7fb93c075a10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.351+0000 7fb965000640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9600cb450 0x7fb960151ff0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.351+0000 7fb965000640 1 --2- 172.21.15.50:0/2032024469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb944003c40 0x7fb960154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.351+0000 7fb965000640 1 -- 172.21.15.50:0/2032024469 >> 172.21.15.50:0/2032024469 conn(0x7fb9600b4560 msgr2=0x7fb96015f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:28.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.355+0000 7fb965000640 1 -- 172.21.15.50:0/2032024469 shutdown_connections 2024-09-21T05:59:28.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.355+0000 7fb965000640 1 -- 172.21.15.50:0/2032024469 wait complete. 2024-09-21T05:59:28.397 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T05:59:28.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.675+0000 7f74f5600640 1 -- 172.21.15.50:0/1584737833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74f015ebf0 msgr2=0x7f74f0165660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:28.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.675+0000 7f74f5600640 1 --2- 172.21.15.50:0/1584737833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74f015ebf0 0x7f74f0165660 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009840 tx=0x7f74e801c6b0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.675+0000 7f74f5600640 1 -- 172.21.15.50:0/1584737833 shutdown_connections 2024-09-21T05:59:28.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.675+0000 7f74f5600640 1 --2- 172.21.15.50:0/1584737833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74f015ebf0 0x7f74f0165660 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.675+0000 7f74f5600640 1 --2- 172.21.15.50:0/1584737833 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74f00cdaf0 0x7f74f015aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.675+0000 7f74f5600640 1 -- 172.21.15.50:0/1584737833 >> 172.21.15.50:0/1584737833 conn(0x7f74f00b2040 msgr2=0x7f74f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:28.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.679+0000 7f74f5600640 1 -- 172.21.15.50:0/1584737833 shutdown_connections 2024-09-21T05:59:28.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.679+0000 7f74f5600640 1 -- 172.21.15.50:0/1584737833 wait complete. 2024-09-21T05:59:28.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.683+0000 7f74f5600640 1 Processor -- start 2024-09-21T05:59:28.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.683+0000 7f74f5600640 1 -- start start 2024-09-21T05:59:28.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.687+0000 7f74f5600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74f00cdaf0 0x7f74f0157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:28.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.687+0000 7f74f5600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74f00ce4c0 0x7f74f014bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:28.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.687+0000 7f74f5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74f015ebf0 0x7f74f014a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:28.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.687+0000 7f74eea00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74f00cdaf0 0x7f74f0157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:28.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.687+0000 7f74eea00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74f00cdaf0 0x7f74f0157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:55552/0 (socket says 172.21.15.50:55552) 2024-09-21T05:59:28.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.687+0000 7f74e6000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74f00ce4c0 0x7f74f014bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:28.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.687+0000 7f74e6000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74f00ce4c0 0x7f74f014bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:47198/0 (socket says 172.21.15.50:47198) 2024-09-21T05:59:28.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.687+0000 7f74eea00640 1 -- 172.21.15.50:0/783813986 learned_addr learned my addr 172.21.15.50:0/783813986 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:28.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.687+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74f016a2b0 con 0x7f74f015ebf0 2024-09-21T05:59:28.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.687+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74f016a130 con 0x7f74f00ce4c0 2024-09-21T05:59:28.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.687+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f74f016a430 con 0x7f74f00cdaf0 2024-09-21T05:59:28.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.691+0000 7f74ef400640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74f015ebf0 0x7f74f014a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:28.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.691+0000 7f74e6000640 1 -- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74f00cdaf0 msgr2=0x7f74f0157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:28.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.691+0000 7f74e6000640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74f00cdaf0 0x7f74f0157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.691+0000 7f74e6000640 1 -- 172.21.15.50:0/783813986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74f015ebf0 msgr2=0x7f74f014a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:28.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.691+0000 7f74e6000640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74f015ebf0 0x7f74f014a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.691+0000 7f74e6000640 1 -- 172.21.15.50:0/783813986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f014c2c0 con 0x7f74f00ce4c0 2024-09-21T05:59:28.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.691+0000 7f74ef400640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74f015ebf0 0x7f74f014a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:28.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.695+0000 7f74e6000640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74f00ce4c0 0x7f74f014bd80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e990 tx=0x7f74d800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:28.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.699+0000 7f74e7400640 1 -- 172.21.15.50:0/783813986 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8004d30 con 0x7f74f00ce4c0 2024-09-21T05:59:28.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.699+0000 7f74e7400640 1 -- 172.21.15.50:0/783813986 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74d800cb00 con 0x7f74f00ce4c0 2024-09-21T05:59:28.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.699+0000 7f74e7400640 1 -- 172.21.15.50:0/783813986 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8005020 con 0x7f74f00ce4c0 2024-09-21T05:59:28.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.699+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f014b000 con 0x7f74f00ce4c0 2024-09-21T05:59:28.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.699+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f014b580 con 0x7f74f00ce4c0 2024-09-21T05:59:28.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.707+0000 7f74e7400640 1 -- 172.21.15.50:0/783813986 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f74d80051c0 con 0x7f74f00ce4c0 2024-09-21T05:59:28.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.707+0000 7f74e7400640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74cc0735e0 0x7f74cc075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:28.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.707+0000 7f74e7400640 1 -- 172.21.15.50:0/783813986 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f74d8094f20 con 0x7f74f00ce4c0 2024-09-21T05:59:28.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.707+0000 7f74eea00640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74cc0735e0 0x7f74cc075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:28.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.711+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f0157dc0 con 0x7f74f00ce4c0 2024-09-21T05:59:28.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.711+0000 7f74eea00640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74cc0735e0 0x7f74cc075af0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f74e0001880 tx=0x7f74e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:28.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.723+0000 7f74e7400640 1 -- 172.21.15.50:0/783813986 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f74f014c2c0 con 0x7f74f00ce4c0 2024-09-21T05:59:28.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.967+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f74f015aa50 con 0x7f74f00ce4c0 2024-09-21T05:59:28.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.971+0000 7f74e7400640 1 -- 172.21.15.50:0/783813986 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v34) ==== 71+0+0 (secure 0 0 0) 0x7f74d8016030 con 0x7f74f00ce4c0 2024-09-21T05:59:28.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.975+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74cc0735e0 msgr2=0x7f74cc075af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:28.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.975+0000 7f74f5600640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74cc0735e0 0x7f74cc075af0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f74e0001880 tx=0x7f74e0009040 comp rx=0 tx=0).stop 2024-09-21T05:59:28.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.975+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74f00ce4c0 msgr2=0x7f74f014bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:28.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.975+0000 7f74f5600640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74f00ce4c0 0x7f74f014bd80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e990 tx=0x7f74d800ee50 comp rx=0 tx=0).stop 2024-09-21T05:59:28.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.975+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 shutdown_connections 2024-09-21T05:59:28.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.975+0000 7f74f5600640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74cc0735e0 0x7f74cc075af0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.975+0000 7f74f5600640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74f015ebf0 0x7f74f014a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.975+0000 7f74f5600640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74f00ce4c0 0x7f74f014bd80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.975+0000 7f74f5600640 1 --2- 172.21.15.50:0/783813986 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74f00cdaf0 0x7f74f0157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.975+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 >> 172.21.15.50:0/783813986 conn(0x7f74f00b2040 msgr2=0x7f74f015cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.975+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 shutdown_connections 2024-09-21T05:59:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:28.975+0000 7f74f5600640 1 -- 172.21.15.50:0/783813986 wait complete. 2024-09-21T05:59:28.999 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-21T05:59:29.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.155+0000 7fe848e00640 1 -- 172.21.15.50:0/3216749188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8440749a0 msgr2=0x7fe844074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:29.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.155+0000 7fe848e00640 1 --2- 172.21.15.50:0/3216749188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8440749a0 0x7fe844074db0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe8440726f0 tx=0x7fe83401c8c0 comp rx=0 tx=0).stop 2024-09-21T05:59:29.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.155+0000 7fe848e00640 1 -- 172.21.15.50:0/3216749188 shutdown_connections 2024-09-21T05:59:29.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.155+0000 7fe848e00640 1 --2- 172.21.15.50:0/3216749188 >> v2:172.21.15.88:3300/0 conn(0x7fe8440cefa0 0x7fe8440cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:29.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.155+0000 7fe848e00640 1 --2- 172.21.15.50:0/3216749188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8440749a0 0x7fe844074db0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:29.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.155+0000 7fe848e00640 1 -- 172.21.15.50:0/3216749188 >> 172.21.15.50:0/3216749188 conn(0x7fe8440b2040 msgr2=0x7fe8440b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:29.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.155+0000 7fe848e00640 1 -- 172.21.15.50:0/3216749188 shutdown_connections 2024-09-21T05:59:29.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.155+0000 7fe848e00640 1 -- 172.21.15.50:0/3216749188 wait complete. 2024-09-21T05:59:29.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe848e00640 1 Processor -- start 2024-09-21T05:59:29.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe848e00640 1 -- start start 2024-09-21T05:59:29.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe848e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8440749a0 0x7fe8440ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:29.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe848e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8440cefa0 0x7fe8440b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:29.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe848e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe844160330 0x7fe8440b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:29.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe848e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe84416a060 con 0x7fe8440cefa0 2024-09-21T05:59:29.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe848e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe844169ee0 con 0x7fe8440749a0 2024-09-21T05:59:29.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe848e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe84416a1e0 con 0x7fe844160330 2024-09-21T05:59:29.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe83be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8440cefa0 0x7fe8440b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:29.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe83be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8440cefa0 0x7fe8440b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49084/0 (socket says 172.21.15.50:49084) 2024-09-21T05:59:29.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe83be00640 1 -- 172.21.15.50:0/2856398623 learned_addr learned my addr 172.21.15.50:0/2856398623 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:29.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe841e00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8440749a0 0x7fe8440ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:29.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe83be00640 1 -- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe844160330 msgr2=0x7fe8440b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:29.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe842800640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe844160330 0x7fe8440b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:29.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe83be00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe844160330 0x7fe8440b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:29.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe83be00640 1 -- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8440749a0 msgr2=0x7fe8440ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:29.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe83be00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8440749a0 0x7fe8440ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:29.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe83be00640 1 -- 172.21.15.50:0/2856398623 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84413d570 con 0x7fe8440cefa0 2024-09-21T05:59:29.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe841e00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8440749a0 0x7fe8440ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:29.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe842800640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe844160330 0x7fe8440b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:29.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe83be00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8440cefa0 0x7fe8440b0060 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00e7f0 tx=0x7fe82c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:29.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe83a000640 1 -- 172.21.15.50:0/2856398623 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c00cc60 con 0x7fe8440cefa0 2024-09-21T05:59:29.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe83a000640 1 -- 172.21.15.50:0/2856398623 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe82c00ce00 con 0x7fe8440cefa0 2024-09-21T05:59:29.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe83a000640 1 -- 172.21.15.50:0/2856398623 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c005280 con 0x7fe8440cefa0 2024-09-21T05:59:29.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe848e00640 1 -- 172.21.15.50:0/2856398623 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe84413d850 con 0x7fe8440cefa0 2024-09-21T05:59:29.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.159+0000 7fe848e00640 1 -- 172.21.15.50:0/2856398623 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe84413de10 con 0x7fe8440cefa0 2024-09-21T05:59:29.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.163+0000 7fe848e00640 1 -- 172.21.15.50:0/2856398623 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe844145440 con 0x7fe8440cefa0 2024-09-21T05:59:29.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.163+0000 7fe83a000640 1 -- 172.21.15.50:0/2856398623 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fe82c005420 con 0x7fe8440cefa0 2024-09-21T05:59:29.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.163+0000 7fe83a000640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe820073560 0x7fe820075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:29.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.163+0000 7fe83a000640 1 -- 172.21.15.50:0/2856398623 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fe82c0950b0 con 0x7fe8440cefa0 2024-09-21T05:59:29.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.163+0000 7fe841e00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe820073560 0x7fe820075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:29.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.163+0000 7fe841e00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe820073560 0x7fe820075a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8340af000 tx=0x7fe83401b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:29.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.167+0000 7fe83a000640 1 -- 172.21.15.50:0/2856398623 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe82c05fd70 con 0x7fe8440cefa0 2024-09-21T05:59:29.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:29.411+0000 7fe848e00640 1 -- 172.21.15.50:0/2856398623 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7fe844074db0 con 0x7fe8440cefa0 2024-09-21T05:59:30.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.363+0000 7fe83a000640 1 -- 172.21.15.50:0/2856398623 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v150) ==== 184+0+0 (secure 0 0 0) 0x7fe82c05f670 con 0x7fe8440cefa0 2024-09-21T05:59:30.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.367+0000 7fe848e00640 1 -- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe820073560 msgr2=0x7fe820075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:30.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.367+0000 7fe848e00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe820073560 0x7fe820075a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8340af000 tx=0x7fe83401b100 comp rx=0 tx=0).stop 2024-09-21T05:59:30.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.367+0000 7fe848e00640 1 -- 172.21.15.50:0/2856398623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8440cefa0 msgr2=0x7fe8440b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:30.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.367+0000 7fe848e00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8440cefa0 0x7fe8440b0060 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00e7f0 tx=0x7fe82c00ecb0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.367+0000 7fe848e00640 1 -- 172.21.15.50:0/2856398623 shutdown_connections 2024-09-21T05:59:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.367+0000 7fe848e00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe844160330 0x7fe8440b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.367+0000 7fe848e00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe820073560 0x7fe820075a70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.367+0000 7fe848e00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8440cefa0 0x7fe8440b0060 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.367+0000 7fe848e00640 1 --2- 172.21.15.50:0/2856398623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8440749a0 0x7fe8440ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.367+0000 7fe848e00640 1 -- 172.21.15.50:0/2856398623 >> 172.21.15.50:0/2856398623 conn(0x7fe8440b2040 msgr2=0x7fe84415a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.367+0000 7fe848e00640 1 -- 172.21.15.50:0/2856398623 shutdown_connections 2024-09-21T05:59:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.367+0000 7fe848e00640 1 -- 172.21.15.50:0/2856398623 wait complete. 2024-09-21T05:59:30.396 INFO:tasks.cephfs_test_runner:Initiate cloning where clone subvolume and source subvolume are located ... ok 2024-09-21T05:59:30.396 INFO:tasks.cephfs_test_runner:Starting test: test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-21T05:59:30.396 INFO:tasks.cephfs_test_runner:test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-21T05:59:30.396 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:59:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.555+0000 7f87de600640 1 -- 172.21.15.50:0/3736401997 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87d80cefa0 msgr2=0x7f87d80cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.555+0000 7f87de600640 1 --2- 172.21.15.50:0/3736401997 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87d80cefa0 0x7f87d80cf3b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009840 tx=0x7f87c001c820 comp rx=0 tx=0).stop 2024-09-21T05:59:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.555+0000 7f87de600640 1 -- 172.21.15.50:0/3736401997 shutdown_connections 2024-09-21T05:59:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.555+0000 7f87de600640 1 --2- 172.21.15.50:0/3736401997 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87d80cefa0 0x7f87d80cf3b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.555+0000 7f87de600640 1 --2- 172.21.15.50:0/3736401997 >> v2:172.21.15.88:3300/0 conn(0x7f87d80749a0 0x7f87d8074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.555+0000 7f87de600640 1 -- 172.21.15.50:0/3736401997 >> 172.21.15.50:0/3736401997 conn(0x7f87d80b2040 msgr2=0x7f87d80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.555+0000 7f87de600640 1 -- 172.21.15.50:0/3736401997 shutdown_connections 2024-09-21T05:59:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.555+0000 7f87de600640 1 -- 172.21.15.50:0/3736401997 wait complete. 2024-09-21T05:59:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87de600640 1 Processor -- start 2024-09-21T05:59:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87de600640 1 -- start start 2024-09-21T05:59:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87de600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87d80749a0 0x7f87d815f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87de600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87d80cefa0 0x7f87d8159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87de600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87d8160330 0x7f87d813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87de600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f87d816a060 con 0x7f87d80749a0 2024-09-21T05:59:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87de600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f87d8169ee0 con 0x7f87d8160330 2024-09-21T05:59:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87de600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f87d816a1e0 con 0x7f87d80cefa0 2024-09-21T05:59:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87d7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87d80749a0 0x7f87d815f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87d7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87d80749a0 0x7f87d815f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49098/0 (socket says 172.21.15.50:49098) 2024-09-21T05:59:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87d7400640 1 -- 172.21.15.50:0/3754892339 learned_addr learned my addr 172.21.15.50:0/3754892339 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87d7e00640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87d8160330 0x7f87d813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.559+0000 7f87cfe00640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87d80cefa0 0x7f87d8159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:30.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87d7400640 1 -- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87d80cefa0 msgr2=0x7f87d8159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:30.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87d7400640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87d80cefa0 0x7f87d8159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87d7400640 1 -- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87d8160330 msgr2=0x7f87d813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87d7400640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87d8160330 0x7f87d813f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87d7400640 1 -- 172.21.15.50:0/3754892339 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d815fa30 con 0x7f87d80749a0 2024-09-21T05:59:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87d7400640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87d80749a0 0x7f87d815f4f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f87c800e7d0 tx=0x7f87c800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87cf400640 1 -- 172.21.15.50:0/3754892339 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c800ccc0 con 0x7f87d80749a0 2024-09-21T05:59:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87cf400640 1 -- 172.21.15.50:0/3754892339 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87c8010430 con 0x7f87d80749a0 2024-09-21T05:59:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87cf400640 1 -- 172.21.15.50:0/3754892339 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8010860 con 0x7f87d80749a0 2024-09-21T05:59:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87de600640 1 -- 172.21.15.50:0/3754892339 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d8159d50 con 0x7f87d80749a0 2024-09-21T05:59:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87de600640 1 -- 172.21.15.50:0/3754892339 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d80aaa90 con 0x7f87d80749a0 2024-09-21T05:59:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87de600640 1 -- 172.21.15.50:0/3754892339 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d8074db0 con 0x7f87d80749a0 2024-09-21T05:59:30.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.563+0000 7f87cf400640 1 -- 172.21.15.50:0/3754892339 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f87c8010a00 con 0x7f87d80749a0 2024-09-21T05:59:30.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.567+0000 7f87cf400640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87b4073560 0x7f87b4075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:30.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.567+0000 7f87cf400640 1 -- 172.21.15.50:0/3754892339 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f87c8094f40 con 0x7f87d80749a0 2024-09-21T05:59:30.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.567+0000 7f87cfe00640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87b4073560 0x7f87b4075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:30.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.567+0000 7f87cfe00640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87b4073560 0x7f87b4075a70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f87c00af000 tx=0x7f87c001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.571+0000 7f87cf400640 1 -- 172.21.15.50:0/3754892339 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f87c805f740 con 0x7f87d80749a0 2024-09-21T05:59:30.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.747+0000 7f87de600640 1 -- 172.21.15.50:0/3754892339 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87d815c880 con 0x7f87d80749a0 2024-09-21T05:59:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87cf400640 1 -- 172.21.15.50:0/3754892339 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+12984 (secure 0 0 0) 0x7f87c8064170 con 0x7f87d80749a0 2024-09-21T05:59:30.754 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:30.754 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":36,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:59:28.349258+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6835/2722641195":"2024-09-22T05:59:25.339681+0000","172.21.15.50:6834/2722641195":"2024-09-22T05:59:25.339681+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:59:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87de600640 1 -- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87b4073560 msgr2=0x7f87b4075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87de600640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87b4073560 0x7f87b4075a70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f87c00af000 tx=0x7f87c001b060 comp rx=0 tx=0).stop 2024-09-21T05:59:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87de600640 1 -- 172.21.15.50:0/3754892339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87d80749a0 msgr2=0x7f87d815f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87de600640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87d80749a0 0x7f87d815f4f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f87c800e7d0 tx=0x7f87c800ec90 comp rx=0 tx=0).stop 2024-09-21T05:59:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87de600640 1 -- 172.21.15.50:0/3754892339 shutdown_connections 2024-09-21T05:59:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87de600640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87d8160330 0x7f87d813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87de600640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87d80cefa0 0x7f87d8159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87de600640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87b4073560 0x7f87b4075a70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87de600640 1 --2- 172.21.15.50:0/3754892339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87d80749a0 0x7f87d815f4f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87de600640 1 -- 172.21.15.50:0/3754892339 >> 172.21.15.50:0/3754892339 conn(0x7f87d80b2040 msgr2=0x7f87d80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87de600640 1 -- 172.21.15.50:0/3754892339 shutdown_connections 2024-09-21T05:59:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.751+0000 7f87de600640 1 -- 172.21.15.50:0/3754892339 wait complete. 2024-09-21T05:59:30.776 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:56:18.921197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T05:59:30.776 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T05:59:30.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.935+0000 7fda52c00640 1 -- 172.21.15.50:0/3603773104 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fda4c0b4de0 msgr2=0x7fda4c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:30.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.935+0000 7fda52c00640 1 --2- 172.21.15.50:0/3603773104 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fda4c0b4de0 0x7fda4c0b51f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda44009bc0 tx=0x7fda4401ca90 comp rx=0 tx=0).stop 2024-09-21T05:59:30.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.935+0000 7fda52c00640 1 -- 172.21.15.50:0/3603773104 shutdown_connections 2024-09-21T05:59:30.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.935+0000 7fda52c00640 1 --2- 172.21.15.50:0/3603773104 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fda4c0b4de0 0x7fda4c0b51f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.935+0000 7fda52c00640 1 --2- 172.21.15.50:0/3603773104 >> v2:172.21.15.88:3300/0 conn(0x7fda4c0b5c70 0x7fda4c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.935+0000 7fda52c00640 1 -- 172.21.15.50:0/3603773104 >> 172.21.15.50:0/3603773104 conn(0x7fda4c0b2040 msgr2=0x7fda4c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:30.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.935+0000 7fda52c00640 1 -- 172.21.15.50:0/3603773104 shutdown_connections 2024-09-21T05:59:30.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.935+0000 7fda52c00640 1 -- 172.21.15.50:0/3603773104 wait complete. 2024-09-21T05:59:30.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda52c00640 1 Processor -- start 2024-09-21T05:59:30.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda52c00640 1 -- start start 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda52c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fda4c0b4de0 0x7fda4c156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda52c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0b5c70 0x7fda4c13ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda52c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda4c1527b0 0x7fda4c0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda52c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c168980 con 0x7fda4c1527b0 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda52c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c168800 con 0x7fda4c0b5c70 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda52c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fda4c168b00 con 0x7fda4c0b4de0 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda50c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda4c1527b0 0x7fda4c0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda50c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda4c1527b0 0x7fda4c0b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49100/0 (socket says 172.21.15.50:49100) 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda50c00640 1 -- 172.21.15.50:0/2884701263 learned_addr learned my addr 172.21.15.50:0/2884701263 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda4be00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fda4c0b4de0 0x7fda4c156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda43e00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0b5c70 0x7fda4c13ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda50c00640 1 -- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fda4c0b4de0 msgr2=0x7fda4c156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:30.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda50c00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fda4c0b4de0 0x7fda4c156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda50c00640 1 -- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0b5c70 msgr2=0x7fda4c13ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:30.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda50c00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0b5c70 0x7fda4c13ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:30.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda50c00640 1 -- 172.21.15.50:0/2884701263 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4c0ae980 con 0x7fda4c1527b0 2024-09-21T05:59:30.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda50c00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda4c1527b0 0x7fda4c0b3cd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fda4401b140 tx=0x7fda44009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda48c00640 1 -- 172.21.15.50:0/2884701263 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440bb070 con 0x7fda4c1527b0 2024-09-21T05:59:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda48c00640 1 -- 172.21.15.50:0/2884701263 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda44003df0 con 0x7fda4c1527b0 2024-09-21T05:59:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda52c00640 1 -- 172.21.15.50:0/2884701263 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4c0aec60 con 0x7fda4c1527b0 2024-09-21T05:59:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda48c00640 1 -- 172.21.15.50:0/2884701263 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440b2040 con 0x7fda4c1527b0 2024-09-21T05:59:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.939+0000 7fda52c00640 1 -- 172.21.15.50:0/2884701263 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda4c0af220 con 0x7fda4c1527b0 2024-09-21T05:59:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.943+0000 7fda52c00640 1 -- 172.21.15.50:0/2884701263 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4c0b51f0 con 0x7fda4c1527b0 2024-09-21T05:59:30.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.943+0000 7fda48c00640 1 -- 172.21.15.50:0/2884701263 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fda44004cb0 con 0x7fda4c1527b0 2024-09-21T05:59:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.943+0000 7fda48c00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fda28073590 0x7fda28075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.943+0000 7fda48c00640 1 -- 172.21.15.50:0/2884701263 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fda4412e070 con 0x7fda4c1527b0 2024-09-21T05:59:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.943+0000 7fda4be00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fda28073590 0x7fda28075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:30.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.943+0000 7fda4be00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fda28073590 0x7fda28075aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009600 tx=0x7fda3c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:30.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:30.947+0000 7fda48c00640 1 -- 172.21.15.50:0/2884701263 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fda440166b0 con 0x7fda4c1527b0 2024-09-21T05:59:31.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.127+0000 7fda52c00640 1 -- 172.21.15.50:0/2884701263 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda4c156a60 con 0x7fda4c1527b0 2024-09-21T05:59:31.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.127+0000 7fda48c00640 1 -- 172.21.15.50:0/2884701263 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7fda440f87f0 con 0x7fda4c1527b0 2024-09-21T05:59:31.131 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-09-21T05:59:31.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.127+0000 7fda52c00640 1 -- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fda28073590 msgr2=0x7fda28075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.127+0000 7fda52c00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fda28073590 0x7fda28075aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009600 tx=0x7fda3c0091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.127+0000 7fda52c00640 1 -- 172.21.15.50:0/2884701263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda4c1527b0 msgr2=0x7fda4c0b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.127+0000 7fda52c00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda4c1527b0 0x7fda4c0b3cd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fda4401b140 tx=0x7fda44009330 comp rx=0 tx=0).stop 2024-09-21T05:59:31.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.131+0000 7fda52c00640 1 -- 172.21.15.50:0/2884701263 shutdown_connections 2024-09-21T05:59:31.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.131+0000 7fda52c00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fda28073590 0x7fda28075aa0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.131+0000 7fda52c00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda4c1527b0 0x7fda4c0b3cd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.131+0000 7fda52c00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0b5c70 0x7fda4c13ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.131+0000 7fda52c00640 1 --2- 172.21.15.50:0/2884701263 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fda4c0b4de0 0x7fda4c156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.131+0000 7fda52c00640 1 -- 172.21.15.50:0/2884701263 >> 172.21.15.50:0/2884701263 conn(0x7fda4c0b2040 msgr2=0x7fda4c159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:31.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.131+0000 7fda52c00640 1 -- 172.21.15.50:0/2884701263 shutdown_connections 2024-09-21T05:59:31.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.131+0000 7fda52c00640 1 -- 172.21.15.50:0/2884701263 wait complete. 2024-09-21T05:59:31.153 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-21T05:59:31.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.307+0000 7fcac4c00640 1 -- 172.21.15.50:0/2153145914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcaa4004810 msgr2=0x7fcaa4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.307+0000 7fcac4c00640 1 --2- 172.21.15.50:0/2153145914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcaa4004810 0x7fcaa4004c20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009b30 tx=0x7fcaa801d270 comp rx=0 tx=0).stop 2024-09-21T05:59:31.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.307+0000 7fcabe800640 1 -- 172.21.15.50:0/2153145914 reap_dead start 2024-09-21T05:59:31.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.307+0000 7fcac4c00640 1 -- 172.21.15.50:0/2153145914 shutdown_connections 2024-09-21T05:59:31.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.307+0000 7fcac4c00640 1 -- 172.21.15.50:0/2153145914 >> 172.21.15.50:0/2153145914 conn(0x7fcac00b2c40 msgr2=0x7fcac00b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:31.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.307+0000 7fcac4c00640 1 -- 172.21.15.50:0/2153145914 shutdown_connections 2024-09-21T05:59:31.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.307+0000 7fcac4c00640 1 -- 172.21.15.50:0/2153145914 wait complete. 2024-09-21T05:59:31.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.311+0000 7fcac4c00640 1 Processor -- start 2024-09-21T05:59:31.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.311+0000 7fcac4c00640 1 -- start start 2024-09-21T05:59:31.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.311+0000 7fcac4c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcaa4004810 0x7fcac0144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.311+0000 7fcac4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcac0137d40 0x7fcac0144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.311+0000 7fcac4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcac0159f00 0x7fcac0144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.311+0000 7fcac4c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcac015d840 con 0x7fcac0137d40 2024-09-21T05:59:31.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.311+0000 7fcac4c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcac00d03d0 con 0x7fcac0159f00 2024-09-21T05:59:31.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.311+0000 7fcac4c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fcac015d9c0 con 0x7fcaa4004810 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.311+0000 7fcab7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcac0137d40 0x7fcac0144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.311+0000 7fcab7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcac0137d40 0x7fcac0144760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49124/0 (socket says 172.21.15.50:49124) 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.311+0000 7fcab7e00640 1 -- 172.21.15.50:0/1127076917 learned_addr learned my addr 172.21.15.50:0/1127076917 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.311+0000 7fcabde00640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcaa4004810 0x7fcac0144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcabe800640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcac0159f00 0x7fcac0144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcab7e00640 1 -- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcaa4004810 msgr2=0x7fcac0144220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcab7e00640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcaa4004810 0x7fcac0144220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcab7e00640 1 -- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcac0159f00 msgr2=0x7fcac0144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcab7e00640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcac0159f00 0x7fcac0144ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcab7e00640 1 -- 172.21.15.50:0/1127076917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac00b5c20 con 0x7fcac0137d40 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcabe800640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcac0159f00 0x7fcac0144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcab7e00640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcac0137d40 0x7fcac0144760 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcaa801d0a0 tx=0x7fcaa80055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcab6000640 1 -- 172.21.15.50:0/1127076917 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80b1040 con 0x7fcac0137d40 2024-09-21T05:59:31.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcab6000640 1 -- 172.21.15.50:0/1127076917 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaa8004120 con 0x7fcac0137d40 2024-09-21T05:59:31.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcac4c00640 1 -- 172.21.15.50:0/1127076917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac00b3de0 con 0x7fcac0137d40 2024-09-21T05:59:31.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcab6000640 1 -- 172.21.15.50:0/1127076917 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80046d0 con 0x7fcac0137d40 2024-09-21T05:59:31.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcac4c00640 1 -- 172.21.15.50:0/1127076917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac00b43a0 con 0x7fcac0137d40 2024-09-21T05:59:31.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcac4c00640 1 -- 172.21.15.50:0/1127076917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac0070ff0 con 0x7fcac0137d40 2024-09-21T05:59:31.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcab6000640 1 -- 172.21.15.50:0/1127076917 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fcaa80bd020 con 0x7fcac0137d40 2024-09-21T05:59:31.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.315+0000 7fcab6000640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca9c073590 0x7fca9c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.319+0000 7fcab6000640 1 -- 172.21.15.50:0/1127076917 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fcaa812ed80 con 0x7fcac0137d40 2024-09-21T05:59:31.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.319+0000 7fcabde00640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca9c073590 0x7fca9c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.319+0000 7fcabde00640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca9c073590 0x7fca9c075aa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005ec0 tx=0x7fcab00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:31.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.323+0000 7fcab6000640 1 -- 172.21.15.50:0/1127076917 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcaa80c4020 con 0x7fcac0137d40 2024-09-21T05:59:31.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:31.563+0000 7fcac4c00640 1 -- 172.21.15.50:0/1127076917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7fcac0069f10 con 0x7fcac0137d40 2024-09-21T05:59:32.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.371+0000 7fcab6000640 1 -- 172.21.15.50:0/1127076917 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v152) ==== 169+0+0 (secure 0 0 0) 0x7fcaa8016f80 con 0x7fcac0137d40 2024-09-21T05:59:32.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.375+0000 7fcac4c00640 1 -- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca9c073590 msgr2=0x7fca9c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:32.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.375+0000 7fcac4c00640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca9c073590 0x7fca9c075aa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005ec0 tx=0x7fcab00091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.375+0000 7fcac4c00640 1 -- 172.21.15.50:0/1127076917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcac0137d40 msgr2=0x7fcac0144760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:32.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.375+0000 7fcac4c00640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcac0137d40 0x7fcac0144760 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcaa801d0a0 tx=0x7fcaa80055d0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.375+0000 7fcac4c00640 1 -- 172.21.15.50:0/1127076917 shutdown_connections 2024-09-21T05:59:32.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.375+0000 7fcac4c00640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcac0159f00 0x7fcac0144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.375+0000 7fcac4c00640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca9c073590 0x7fca9c075aa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.375+0000 7fcac4c00640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcac0137d40 0x7fcac0144760 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.375+0000 7fcac4c00640 1 --2- 172.21.15.50:0/1127076917 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcaa4004810 0x7fcac0144220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.375+0000 7fcac4c00640 1 -- 172.21.15.50:0/1127076917 >> 172.21.15.50:0/1127076917 conn(0x7fcac00b2c40 msgr2=0x7fcac006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:32.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.375+0000 7fcac4c00640 1 -- 172.21.15.50:0/1127076917 shutdown_connections 2024-09-21T05:59:32.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.375+0000 7fcac4c00640 1 -- 172.21.15.50:0/1127076917 wait complete. 2024-09-21T05:59:32.402 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T05:59:32.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.559+0000 7fc486400640 1 -- 172.21.15.50:0/4120756891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4801527b0 msgr2=0x7fc48015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:32.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.559+0000 7fc486400640 1 --2- 172.21.15.50:0/4120756891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4801527b0 0x7fc48015bb60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc468009840 tx=0x7fc46801c7f0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.559+0000 7fc486400640 1 -- 172.21.15.50:0/4120756891 shutdown_connections 2024-09-21T05:59:32.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.559+0000 7fc486400640 1 --2- 172.21.15.50:0/4120756891 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4800b4de0 0x7fc4800b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.559+0000 7fc486400640 1 --2- 172.21.15.50:0/4120756891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4801527b0 0x7fc48015bb60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.559+0000 7fc486400640 1 -- 172.21.15.50:0/4120756891 >> 172.21.15.50:0/4120756891 conn(0x7fc4800b2040 msgr2=0x7fc4800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:32.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.559+0000 7fc486400640 1 -- 172.21.15.50:0/4120756891 shutdown_connections 2024-09-21T05:59:32.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.559+0000 7fc486400640 1 -- 172.21.15.50:0/4120756891 wait complete. 2024-09-21T05:59:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.559+0000 7fc486400640 1 Processor -- start 2024-09-21T05:59:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc486400640 1 -- start start 2024-09-21T05:59:32.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc486400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4800b4de0 0x7fc480158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:32.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc486400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4800b5c70 0x7fc4800ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:32.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc486400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4801527b0 0x7fc4800af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:32.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc486400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc480168a30 con 0x7fc4801527b0 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc486400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4801688b0 con 0x7fc4800b4de0 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc486400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc480168bb0 con 0x7fc4800b5c70 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc47fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4801527b0 0x7fc4800af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc47fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4801527b0 0x7fc4800af2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49144/0 (socket says 172.21.15.50:49144) 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc47fe00640 1 -- 172.21.15.50:0/2098904536 learned_addr learned my addr 172.21.15.50:0/2098904536 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc476a00640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4800b5c70 0x7fc4800ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc47f400640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4800b4de0 0x7fc480158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc47fe00640 1 -- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4800b5c70 msgr2=0x7fc4800ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc47fe00640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4800b5c70 0x7fc4800ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc47fe00640 1 -- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4800b4de0 msgr2=0x7fc480158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc47fe00640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4800b4de0 0x7fc480158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc47fe00640 1 -- 172.21.15.50:0/2098904536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc480158250 con 0x7fc4801527b0 2024-09-21T05:59:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc47fe00640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4801527b0 0x7fc4800af2a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc47800e940 tx=0x7fc47800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:32.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc477e00640 1 -- 172.21.15.50:0/2098904536 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478004d30 con 0x7fc4801527b0 2024-09-21T05:59:32.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc477e00640 1 -- 172.21.15.50:0/2098904536 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc47800cb20 con 0x7fc4801527b0 2024-09-21T05:59:32.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc477e00640 1 -- 172.21.15.50:0/2098904536 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478005020 con 0x7fc4801527b0 2024-09-21T05:59:32.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc486400640 1 -- 172.21.15.50:0/2098904536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4801f93d0 con 0x7fc4801527b0 2024-09-21T05:59:32.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc486400640 1 -- 172.21.15.50:0/2098904536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4801f96b0 con 0x7fc4801527b0 2024-09-21T05:59:32.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.563+0000 7fc486400640 1 -- 172.21.15.50:0/2098904536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4800b51f0 con 0x7fc4801527b0 2024-09-21T05:59:32.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.567+0000 7fc477e00640 1 -- 172.21.15.50:0/2098904536 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fc4780051c0 con 0x7fc4801527b0 2024-09-21T05:59:32.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.567+0000 7fc477e00640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc45c073590 0x7fc45c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:32.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.567+0000 7fc477e00640 1 -- 172.21.15.50:0/2098904536 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fc478094f80 con 0x7fc4801527b0 2024-09-21T05:59:32.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.567+0000 7fc47f400640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc45c073590 0x7fc45c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:32.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.567+0000 7fc47f400640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc45c073590 0x7fc45c075aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc470000f70 tx=0x7fc470009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:32.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.571+0000 7fc477e00640 1 -- 172.21.15.50:0/2098904536 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc478016030 con 0x7fc4801527b0 2024-09-21T05:59:32.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 -- 172.21.15.50:0/2098904536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc4800aecf0 con 0x7fc4801527b0 2024-09-21T05:59:32.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc477e00640 1 -- 172.21.15.50:0/2098904536 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v34) ==== 59+0+2294 (secure 0 0 0) 0x7fc47805fcd0 con 0x7fc4801527b0 2024-09-21T05:59:32.754 INFO:teuthology.orchestra.run.smithi050.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T05:59:32.754 INFO:teuthology.orchestra.run.smithi050.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced client_mount_timeout 30 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_client 20/20 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_mgr 20/20 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_ms 1/1 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3735.149299 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4471.400731 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3640.131623 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3663.645087 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3762.055672 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3661.558528 2024-09-21T05:59:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3661.456695 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3796.187852 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds 20/20 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_ms 2/2 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_frag true 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_scatterstat true 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_verify_scatter true 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced client_mount_timeout 600 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_client 20/20 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_ms 1/1 2024-09-21T05:59:32.756 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T05:59:32.757 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T05:59:32.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 -- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc45c073590 msgr2=0x7fc45c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:32.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc45c073590 0x7fc45c075aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc470000f70 tx=0x7fc470009040 comp rx=0 tx=0).stop 2024-09-21T05:59:32.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 -- 172.21.15.50:0/2098904536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4801527b0 msgr2=0x7fc4800af2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4801527b0 0x7fc4800af2a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc47800e940 tx=0x7fc47800ee00 comp rx=0 tx=0).stop 2024-09-21T05:59:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 -- 172.21.15.50:0/2098904536 shutdown_connections 2024-09-21T05:59:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc45c073590 0x7fc45c075aa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4801527b0 0x7fc4800af2a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4800b5c70 0x7fc4800ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 --2- 172.21.15.50:0/2098904536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4800b4de0 0x7fc480158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 -- 172.21.15.50:0/2098904536 >> 172.21.15.50:0/2098904536 conn(0x7fc4800b2040 msgr2=0x7fc4800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 -- 172.21.15.50:0/2098904536 shutdown_connections 2024-09-21T05:59:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.751+0000 7fc486400640 1 -- 172.21.15.50:0/2098904536 wait complete. 2024-09-21T05:59:32.777 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T05:59:32.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.935+0000 7f86a3e00640 1 -- 172.21.15.50:0/3110582339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8688004810 msgr2=0x7f8688004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:32.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.935+0000 7f86a3e00640 1 --2- 172.21.15.50:0/3110582339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8688004810 0x7f8688004c20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f868c001860 tx=0x7f868c01c8c0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.935+0000 7f86a3e00640 1 -- 172.21.15.50:0/3110582339 shutdown_connections 2024-09-21T05:59:32.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.935+0000 7f86a3e00640 1 --2- 172.21.15.50:0/3110582339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8688004810 0x7f8688004c20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.935+0000 7f86a3e00640 1 -- 172.21.15.50:0/3110582339 >> 172.21.15.50:0/3110582339 conn(0x7f86a40b4560 msgr2=0x7f86a40d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:32.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.935+0000 7f86a3e00640 1 -- 172.21.15.50:0/3110582339 shutdown_connections 2024-09-21T05:59:32.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.935+0000 7f86a3e00640 1 -- 172.21.15.50:0/3110582339 wait complete. 2024-09-21T05:59:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f86a3e00640 1 Processor -- start 2024-09-21T05:59:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f86a3e00640 1 -- start start 2024-09-21T05:59:32.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f86a3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8688004810 0x7f86a414b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:32.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f86a3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a414bea0 0x7f86a4161350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f86a3e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86a4161890 0x7f86a41592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f86a3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4071860 con 0x7f86a414bea0 2024-09-21T05:59:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f86a3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f86a40b7540 con 0x7f8688004810 2024-09-21T05:59:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f86a3e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f86a40719e0 con 0x7f86a4161890 2024-09-21T05:59:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f869be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a414bea0 0x7f86a4161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f869be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a414bea0 0x7f86a4161350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49158/0 (socket says 172.21.15.50:49158) 2024-09-21T05:59:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f869be00640 1 -- 172.21.15.50:0/1356509650 learned_addr learned my addr 172.21.15.50:0/1356509650 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f86a3400640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86a4161890 0x7f86a41592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f86a2a00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8688004810 0x7f86a414b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f869be00640 1 -- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86a4161890 msgr2=0x7f86a41592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f869be00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86a4161890 0x7f86a41592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f869be00640 1 -- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8688004810 msgr2=0x7f86a414b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:32.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f869be00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8688004810 0x7f86a414b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:32.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f869be00640 1 -- 172.21.15.50:0/1356509650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a4161ca0 con 0x7f86a414bea0 2024-09-21T05:59:32.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f86a2a00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8688004810 0x7f86a414b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:32.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f869be00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a414bea0 0x7f86a4161350 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f868c005040 tx=0x7f868c003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:32.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.939+0000 7f869aa00640 1 -- 172.21.15.50:0/1356509650 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c0068d0 con 0x7f86a414bea0 2024-09-21T05:59:32.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.943+0000 7f869aa00640 1 -- 172.21.15.50:0/1356509650 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f868c003e50 con 0x7f86a414bea0 2024-09-21T05:59:32.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.943+0000 7f86a3e00640 1 -- 172.21.15.50:0/1356509650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a415aa00 con 0x7f86a414bea0 2024-09-21T05:59:32.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.943+0000 7f869aa00640 1 -- 172.21.15.50:0/1356509650 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c001480 con 0x7f86a414bea0 2024-09-21T05:59:32.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.943+0000 7f86a3e00640 1 -- 172.21.15.50:0/1356509650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a4158d40 con 0x7f86a414bea0 2024-09-21T05:59:32.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.943+0000 7f86a3e00640 1 -- 172.21.15.50:0/1356509650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86a4071ee0 con 0x7f86a414bea0 2024-09-21T05:59:32.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.943+0000 7f869aa00640 1 -- 172.21.15.50:0/1356509650 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f868c006150 con 0x7f86a414bea0 2024-09-21T05:59:32.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.943+0000 7f869aa00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8680073560 0x7f8680075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:32.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.943+0000 7f869aa00640 1 -- 172.21.15.50:0/1356509650 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f868c12e7d0 con 0x7f86a414bea0 2024-09-21T05:59:32.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.943+0000 7f86a2a00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8680073560 0x7f8680075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:32.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.947+0000 7f86a2a00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8680073560 0x7f8680075a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8694005ec0 tx=0x7f86940091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:32.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:32.947+0000 7f869aa00640 1 -- 172.21.15.50:0/1356509650 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f86a4161ca0 con 0x7f86a414bea0 2024-09-21T05:59:33.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.127+0000 7f86a3e00640 1 -- 172.21.15.50:0/1356509650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f86a4159070 con 0x7f86a414bea0 2024-09-21T05:59:33.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f869aa00640 1 -- 172.21.15.50:0/1356509650 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7f868c016670 con 0x7f86a414bea0 2024-09-21T05:59:33.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f86a3e00640 1 -- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8680073560 msgr2=0x7f8680075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:33.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f86a3e00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8680073560 0x7f8680075a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8694005ec0 tx=0x7f86940091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:33.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f86a3e00640 1 -- 172.21.15.50:0/1356509650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a414bea0 msgr2=0x7f86a4161350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:33.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f86a3e00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a414bea0 0x7f86a4161350 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f868c005040 tx=0x7f868c003c60 comp rx=0 tx=0).stop 2024-09-21T05:59:33.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f86a3e00640 1 -- 172.21.15.50:0/1356509650 shutdown_connections 2024-09-21T05:59:33.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f86a3e00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86a4161890 0x7f86a41592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:33.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f86a3e00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8680073560 0x7f8680075a70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:33.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f86a3e00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a414bea0 0x7f86a4161350 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:33.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f86a3e00640 1 --2- 172.21.15.50:0/1356509650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8688004810 0x7f86a414b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:33.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f86a3e00640 1 -- 172.21.15.50:0/1356509650 >> 172.21.15.50:0/1356509650 conn(0x7f86a40b4560 msgr2=0x7f86a415cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:33.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f86a3e00640 1 -- 172.21.15.50:0/1356509650 shutdown_connections 2024-09-21T05:59:33.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.131+0000 7f86a3e00640 1 -- 172.21.15.50:0/1356509650 wait complete. 2024-09-21T05:59:33.157 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:59:33.157 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:59:33.240 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:59:33.337 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:59:33.337 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:59:33.337 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:59:33.346 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:59:33.346 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:59:33.346 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:59:33.358 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:59:33.358 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:59:33.358 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:59:33.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:59:33.397 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T05:59:33.397 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:59:33.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:59:33.411 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T05:59:33.411 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:59:33.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.599+0000 7fdd24200640 1 -- 172.21.15.50:0/3116816660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd04005160 msgr2=0x7fdd0400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:33.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.599+0000 7fdd24200640 1 --2- 172.21.15.50:0/3116816660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd04005160 0x7fdd0400b9d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009ec0 tx=0x7fdd1401d5f0 comp rx=0 tx=0).stop 2024-09-21T05:59:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.599+0000 7fdd22200640 1 -- 172.21.15.50:0/3116816660 reap_dead start 2024-09-21T05:59:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.599+0000 7fdd24200640 1 -- 172.21.15.50:0/3116816660 shutdown_connections 2024-09-21T05:59:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.599+0000 7fdd24200640 1 -- 172.21.15.50:0/3116816660 >> 172.21.15.50:0/3116816660 conn(0x7fdd1c0708c0 msgr2=0x7fdd1c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.599+0000 7fdd24200640 1 -- 172.21.15.50:0/3116816660 shutdown_connections 2024-09-21T05:59:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.599+0000 7fdd24200640 1 -- 172.21.15.50:0/3116816660 wait complete. 2024-09-21T05:59:33.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd24200640 1 Processor -- start 2024-09-21T05:59:33.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd24200640 1 -- start start 2024-09-21T05:59:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd24200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdd04005160 0x7fdd1c143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd24200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c14fde0 0x7fdd1c0b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd24200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1c0b4080 0x7fdd1c0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd24200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c15cf90 con 0x7fdd1c14fde0 2024-09-21T05:59:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd24200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c074d30 con 0x7fdd1c0b4080 2024-09-21T05:59:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd24200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fdd1c15d110 con 0x7fdd04005160 2024-09-21T05:59:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd20e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c14fde0 0x7fdd1c0b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd20e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c14fde0 0x7fdd1c0b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49166/0 (socket says 172.21.15.50:49166) 2024-09-21T05:59:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd20e00640 1 -- 172.21.15.50:0/724465129 learned_addr learned my addr 172.21.15.50:0/724465129 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd21800640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdd04005160 0x7fdd1c143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd22200640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1c0b4080 0x7fdd1c0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd20e00640 1 -- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdd04005160 msgr2=0x7fdd1c143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:33.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd20e00640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdd04005160 0x7fdd1c143080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:33.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd20e00640 1 -- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1c0b4080 msgr2=0x7fdd1c0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:33.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd20e00640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1c0b4080 0x7fdd1c0b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:33.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd20e00640 1 -- 172.21.15.50:0/724465129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1c0b1da0 con 0x7fdd1c14fde0 2024-09-21T05:59:33.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd21800640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdd04005160 0x7fdd1c143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:33.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd22200640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1c0b4080 0x7fdd1c0b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:33.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.603+0000 7fdd20e00640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c14fde0 0x7fdd1c0b2e40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdd080126c0 tx=0x7fdd08012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:33.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.607+0000 7fdd19600640 1 -- 172.21.15.50:0/724465129 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08010ae0 con 0x7fdd1c14fde0 2024-09-21T05:59:33.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.607+0000 7fdd19600640 1 -- 172.21.15.50:0/724465129 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd08017030 con 0x7fdd1c14fde0 2024-09-21T05:59:33.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.607+0000 7fdd19600640 1 -- 172.21.15.50:0/724465129 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08013450 con 0x7fdd1c14fde0 2024-09-21T05:59:33.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.607+0000 7fdd24200640 1 -- 172.21.15.50:0/724465129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1c0b2080 con 0x7fdd1c14fde0 2024-09-21T05:59:33.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.607+0000 7fdd24200640 1 -- 172.21.15.50:0/724465129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1c0b2640 con 0x7fdd1c14fde0 2024-09-21T05:59:33.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.607+0000 7fdd24200640 1 -- 172.21.15.50:0/724465129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd1c0b60d0 con 0x7fdd1c14fde0 2024-09-21T05:59:33.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.607+0000 7fdd19600640 1 -- 172.21.15.50:0/724465129 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fdd080135f0 con 0x7fdd1c14fde0 2024-09-21T05:59:33.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.611+0000 7fdd19600640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdcfc073590 0x7fdcfc075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:33.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.611+0000 7fdd19600640 1 -- 172.21.15.50:0/724465129 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fdd08098260 con 0x7fdd1c14fde0 2024-09-21T05:59:33.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.611+0000 7fdd21800640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdcfc073590 0x7fdcfc075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:33.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.611+0000 7fdd21800640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdcfc073590 0x7fdcfc075aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdd10006fd0 tx=0x7fdd10006170 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:33.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.615+0000 7fdd19600640 1 -- 172.21.15.50:0/724465129 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdd08062fb0 con 0x7fdd1c14fde0 2024-09-21T05:59:33.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.827+0000 7fdd24200640 1 -- 172.21.15.50:0/724465129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd1c0d2260 con 0x7fdd1c14fde0 2024-09-21T05:59:33.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.827+0000 7fdd19600640 1 -- 172.21.15.50:0/724465129 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3905 (secure 0 0 0) 0x7fdd08098080 con 0x7fdd1c14fde0 2024-09-21T05:59:33.831 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:33.832 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":20,"btime":"2024-09-21T05:59:26:341728+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3475589656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3475589656},{"type":"v1","addr":"172.21.15.88:6835","nonce":3475589656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/88740447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":88740447},{"type":"v1","addr":"172.21.15.88:6837","nonce":88740447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4798,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/4086930068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":4086930068},{"type":"v1","addr":"172.21.15.50:6833","nonce":4086930068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5277,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3304183459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3304183459},{"type":"v1","addr":"172.21.15.50:6835","nonce":3304183459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-21T05:59:33.832 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 20 2024-09-21T05:59:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.831+0000 7fdd24200640 1 -- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdcfc073590 msgr2=0x7fdcfc075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.831+0000 7fdd24200640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdcfc073590 0x7fdcfc075aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdd10006fd0 tx=0x7fdd10006170 comp rx=0 tx=0).stop 2024-09-21T05:59:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.831+0000 7fdd24200640 1 -- 172.21.15.50:0/724465129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c14fde0 msgr2=0x7fdd1c0b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.831+0000 7fdd24200640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c14fde0 0x7fdd1c0b2e40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdd080126c0 tx=0x7fdd08012b80 comp rx=0 tx=0).stop 2024-09-21T05:59:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.831+0000 7fdd24200640 1 -- 172.21.15.50:0/724465129 shutdown_connections 2024-09-21T05:59:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.831+0000 7fdd24200640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1c0b4080 0x7fdd1c0b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.831+0000 7fdd24200640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdcfc073590 0x7fdcfc075aa0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.831+0000 7fdd24200640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c14fde0 0x7fdd1c0b2e40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.831+0000 7fdd24200640 1 --2- 172.21.15.50:0/724465129 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdd04005160 0x7fdd1c143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.831+0000 7fdd24200640 1 -- 172.21.15.50:0/724465129 >> 172.21.15.50:0/724465129 conn(0x7fdd1c0708c0 msgr2=0x7fdd1c15c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:33.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.831+0000 7fdd24200640 1 -- 172.21.15.50:0/724465129 shutdown_connections 2024-09-21T05:59:33.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:33.831+0000 7fdd24200640 1 -- 172.21.15.50:0/724465129 wait complete. 2024-09-21T05:59:33.855 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T05:59:33.855 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T05:59:33.855 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T05:59:33.855 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:59:33.855 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T05:59:33.855 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T05:59:33.856 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T05:59:33.856 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:59:33.856 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T05:59:33.856 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T05:59:33.856 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T05:59:33.856 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:59:33.856 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T05:59:33.857 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T05:59:33.857 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T05:59:33.857 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:59:33.868 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T05:59:33.868 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T05:59:33.871 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T05:59:33.871 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T05:59:33.872 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T05:59:33.872 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T05:59:33.875 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T05:59:33.875 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T05:59:33.921 INFO:tasks.ceph.mds.b:Started 2024-09-21T05:59:33.921 DEBUG:teuthology.parallel:result is None 2024-09-21T05:59:33.922 INFO:tasks.ceph.mds.d:Started 2024-09-21T05:59:33.923 DEBUG:teuthology.parallel:result is None 2024-09-21T05:59:33.923 INFO:tasks.ceph.mds.c:Started 2024-09-21T05:59:33.924 DEBUG:teuthology.parallel:result is None 2024-09-21T05:59:33.925 INFO:tasks.ceph.mds.a:Started 2024-09-21T05:59:33.926 DEBUG:teuthology.parallel:result is None 2024-09-21T05:59:33.926 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T05:59:34.023 INFO:tasks.ceph.mds.b.smithi088.stdout:starting mds.b at 2024-09-21T05:59:34.024 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T05:59:34.024 INFO:tasks.ceph.mds.d.smithi088.stdout:starting mds.d at 2024-09-21T05:59:34.025 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-09-21T05:59:34.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.091+0000 7f1f7be00640 1 -- 172.21.15.50:0/2333447453 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1f7c160470 msgr2=0x7f1f7c074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:34.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.091+0000 7f1f7be00640 1 --2- 172.21.15.50:0/2333447453 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1f7c160470 0x7f1f7c074780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009bc0 tx=0x7f1f7401cb10 comp rx=0 tx=0).stop 2024-09-21T05:59:34.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.091+0000 7f1f7be00640 1 -- 172.21.15.50:0/2333447453 shutdown_connections 2024-09-21T05:59:34.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.091+0000 7f1f7be00640 1 --2- 172.21.15.50:0/2333447453 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1f7c160470 0x7f1f7c074780 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.091+0000 7f1f7be00640 1 --2- 172.21.15.50:0/2333447453 >> v2:172.21.15.88:3300/0 conn(0x7f1f7c160060 0x7f1f7c074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.091+0000 7f1f7be00640 1 -- 172.21.15.50:0/2333447453 >> 172.21.15.50:0/2333447453 conn(0x7f1f7c0b4560 msgr2=0x7f1f7c0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:34.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.091+0000 7f1f7be00640 1 -- 172.21.15.50:0/2333447453 shutdown_connections 2024-09-21T05:59:34.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.091+0000 7f1f7be00640 1 -- 172.21.15.50:0/2333447453 wait complete. 2024-09-21T05:59:34.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.091+0000 7f1f7be00640 1 Processor -- start 2024-09-21T05:59:34.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.091+0000 7f1f7be00640 1 -- start start 2024-09-21T05:59:34.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f7c1536f0 0x7f1f7c150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:34.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f7c160060 0x7f1f7c14ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:34.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7be00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1f7c160470 0x7f1f7c142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:34.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7be00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c169790 con 0x7f1f7c1536f0 2024-09-21T05:59:34.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c169610 con 0x7f1f7c160060 2024-09-21T05:59:34.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7be00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1f7c169910 con 0x7f1f7c160470 2024-09-21T05:59:34.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f7c1536f0 0x7f1f7c150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:34.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f7c1536f0 0x7f1f7c150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49190/0 (socket says 172.21.15.50:49190) 2024-09-21T05:59:34.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7aa00640 1 -- 172.21.15.50:0/2554998782 learned_addr learned my addr 172.21.15.50:0/2554998782 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:34.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f72000640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f7c160060 0x7f1f7c14ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:34.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7b400640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1f7c160470 0x7f1f7c142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:34.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7aa00640 1 -- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1f7c160470 msgr2=0x7f1f7c142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:34.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7aa00640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1f7c160470 0x7f1f7c142100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7aa00640 1 -- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f7c160060 msgr2=0x7f1f7c14ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:34.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7aa00640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f7c160060 0x7f1f7c14ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7aa00640 1 -- 172.21.15.50:0/2554998782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7c0d0550 con 0x7f1f7c1536f0 2024-09-21T05:59:34.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7aa00640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f7c1536f0 0x7f1f7c150d10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00c400 tx=0x7f1f6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:34.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f73400640 1 -- 172.21.15.50:0/2554998782 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c017070 con 0x7f1f7c1536f0 2024-09-21T05:59:34.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f73400640 1 -- 172.21.15.50:0/2554998782 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f6c003600 con 0x7f1f7c1536f0 2024-09-21T05:59:34.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7be00640 1 -- 172.21.15.50:0/2554998782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7c142e80 con 0x7f1f7c1536f0 2024-09-21T05:59:34.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f73400640 1 -- 172.21.15.50:0/2554998782 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c003a50 con 0x7f1f7c1536f0 2024-09-21T05:59:34.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7be00640 1 -- 172.21.15.50:0/2554998782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f7c0d0aa0 con 0x7f1f7c1536f0 2024-09-21T05:59:34.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f7be00640 1 -- 172.21.15.50:0/2554998782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f7c15d480 con 0x7f1f7c1536f0 2024-09-21T05:59:34.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.095+0000 7f1f73400640 1 -- 172.21.15.50:0/2554998782 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f1f6c01a070 con 0x7f1f7c1536f0 2024-09-21T05:59:34.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.099+0000 7f1f73400640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1f58073560 0x7f1f58075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:34.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.099+0000 7f1f73400640 1 -- 172.21.15.50:0/2554998782 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f1f6c093820 con 0x7f1f7c1536f0 2024-09-21T05:59:34.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.099+0000 7f1f72000640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1f58073560 0x7f1f58075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:34.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.099+0000 7f1f72000640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1f58073560 0x7f1f58075a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c14a3c0 tx=0x7f1f640091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:34.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.103+0000 7f1f73400640 1 -- 172.21.15.50:0/2554998782 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1f6c05df90 con 0x7f1f7c1536f0 2024-09-21T05:59:34.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.291+0000 7f1f7be00640 1 -- 172.21.15.50:0/2554998782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1f7c0b6c00 con 0x7f1f7c1536f0 2024-09-21T05:59:34.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.387+0000 7f1f73400640 1 -- 172.21.15.50:0/2554998782 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) ==== 97+0+0 (secure 0 0 0) 0x7f1f6c0629c0 con 0x7f1f7c1536f0 2024-09-21T05:59:34.393 INFO:teuthology.orchestra.run.smithi050.stderr: removed all blocklist entries 2024-09-21T05:59:34.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.391+0000 7f1f7be00640 1 -- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1f58073560 msgr2=0x7f1f58075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:34.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.391+0000 7f1f7be00640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1f58073560 0x7f1f58075a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c14a3c0 tx=0x7f1f640091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.391+0000 7f1f7be00640 1 -- 172.21.15.50:0/2554998782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f7c1536f0 msgr2=0x7f1f7c150d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:34.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.391+0000 7f1f7be00640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f7c1536f0 0x7f1f7c150d10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00c400 tx=0x7f1f6c00c8c0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.391+0000 7f1f7be00640 1 -- 172.21.15.50:0/2554998782 shutdown_connections 2024-09-21T05:59:34.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.391+0000 7f1f7be00640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1f7c160470 0x7f1f7c142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.391+0000 7f1f7be00640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f7c160060 0x7f1f7c14ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.391+0000 7f1f7be00640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1f58073560 0x7f1f58075a70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.391+0000 7f1f7be00640 1 --2- 172.21.15.50:0/2554998782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f7c1536f0 0x7f1f7c150d10 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.391+0000 7f1f7be00640 1 -- 172.21.15.50:0/2554998782 >> 172.21.15.50:0/2554998782 conn(0x7f1f7c0b4560 msgr2=0x7f1f7c15c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:34.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.391+0000 7f1f7be00640 1 -- 172.21.15.50:0/2554998782 shutdown_connections 2024-09-21T05:59:34.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.391+0000 7f1f7be00640 1 -- 172.21.15.50:0/2554998782 wait complete. 2024-09-21T05:59:34.420 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T05:59:34.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.575+0000 7f1d8ec00640 1 -- 172.21.15.50:0/266789574 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1d881527b0 msgr2=0x7f1d8815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:34.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.575+0000 7f1d8ec00640 1 --2- 172.21.15.50:0/266789574 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1d881527b0 0x7f1d8815bb60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009870 tx=0x7f1d7001c6c0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.575+0000 7f1d8ec00640 1 -- 172.21.15.50:0/266789574 shutdown_connections 2024-09-21T05:59:34.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.575+0000 7f1d8ec00640 1 --2- 172.21.15.50:0/266789574 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1d881527b0 0x7f1d8815bb60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.575+0000 7f1d8ec00640 1 -- 172.21.15.50:0/266789574 >> 172.21.15.50:0/266789574 conn(0x7f1d880b2040 msgr2=0x7f1d880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:34.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.575+0000 7f1d8ec00640 1 -- 172.21.15.50:0/266789574 shutdown_connections 2024-09-21T05:59:34.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.575+0000 7f1d8ec00640 1 -- 172.21.15.50:0/266789574 wait complete. 2024-09-21T05:59:34.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8ec00640 1 Processor -- start 2024-09-21T05:59:34.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8ec00640 1 -- start start 2024-09-21T05:59:34.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8ec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d880b4de0 0x7f1d88146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:34.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8ec00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1d880b5c70 0x7f1d8814cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:34.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8ec00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d881527b0 0x7f1d8814d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:34.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8ec00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1d88169d00 con 0x7f1d881527b0 2024-09-21T05:59:34.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8ec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1d88169b80 con 0x7f1d880b4de0 2024-09-21T05:59:34.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8ec00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1d88169e80 con 0x7f1d880b5c70 2024-09-21T05:59:34.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d881527b0 0x7f1d8814d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:34.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d881527b0 0x7f1d8814d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49192/0 (socket says 172.21.15.50:49192) 2024-09-21T05:59:34.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8cc00640 1 -- 172.21.15.50:0/244267818 learned_addr learned my addr 172.21.15.50:0/244267818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:34.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d87e00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d880b4de0 0x7f1d88146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:34.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d7fe00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1d880b5c70 0x7f1d8814cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:34.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8cc00640 1 -- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1d880b5c70 msgr2=0x7f1d8814cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:34.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8cc00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1d880b5c70 0x7f1d8814cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8cc00640 1 -- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d880b4de0 msgr2=0x7f1d88146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:34.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8cc00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d880b4de0 0x7f1d88146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8cc00640 1 -- 172.21.15.50:0/244267818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d881f6730 con 0x7f1d881527b0 2024-09-21T05:59:34.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8cc00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d881527b0 0x7f1d8814d4b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000cbc0 tx=0x7f1d8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:34.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d84c00640 1 -- 172.21.15.50:0/244267818 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80016020 con 0x7f1d881527b0 2024-09-21T05:59:34.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d84c00640 1 -- 172.21.15.50:0/244267818 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d8000ab60 con 0x7f1d881527b0 2024-09-21T05:59:34.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d84c00640 1 -- 172.21.15.50:0/244267818 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80012420 con 0x7f1d881527b0 2024-09-21T05:59:34.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8ec00640 1 -- 172.21.15.50:0/244267818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d881f6a10 con 0x7f1d881527b0 2024-09-21T05:59:34.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.579+0000 7f1d8ec00640 1 -- 172.21.15.50:0/244267818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d881f6fd0 con 0x7f1d881527b0 2024-09-21T05:59:34.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.583+0000 7f1d8ec00640 1 -- 172.21.15.50:0/244267818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d8815bb60 con 0x7f1d881527b0 2024-09-21T05:59:34.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.583+0000 7f1d84c00640 1 -- 172.21.15.50:0/244267818 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f1d80019020 con 0x7f1d881527b0 2024-09-21T05:59:34.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.583+0000 7f1d84c00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1d64073590 0x7f1d64075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:34.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.583+0000 7f1d84c00640 1 -- 172.21.15.50:0/244267818 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f1d80094b90 con 0x7f1d881527b0 2024-09-21T05:59:34.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.583+0000 7f1d87e00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1d64073590 0x7f1d64075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:34.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.583+0000 7f1d87e00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1d64073590 0x7f1d64075aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1d78007fd0 tx=0x7f1d78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:34.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.587+0000 7f1d84c00640 1 -- 172.21.15.50:0/244267818 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1d80020020 con 0x7f1d881527b0 2024-09-21T05:59:34.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.823+0000 7f1d8ec00640 1 -- 172.21.15.50:0/244267818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1d8815ad00 con 0x7f1d881527b0 2024-09-21T05:59:34.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.823+0000 7f1d84c00640 1 -- 172.21.15.50:0/244267818 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v29) ==== 80+0+5304 (secure 0 0 0) 0x7f1d8005f460 con 0x7f1d881527b0 2024-09-21T05:59:34.828 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:34.828 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T05:59:34.828 INFO:teuthology.orchestra.run.smithi050.stdout: "auth_dump": [ 2024-09-21T05:59:34.828 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.828 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.a", 2024-09-21T05:59:34.828 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mSKq0ARAA2egaHGYWhRUEq6oRl1CIMg==", 2024-09-21T05:59:34.828 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.828 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:59:34.828 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.b", 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mwv0+DxAAY+eZPxv5uY9PnLl0asdr6w==", 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.829 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.c", 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5me6maCRAA4jvHgTLTKpb/mxzzG71SfQ==", 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.d", 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mzYnuFhAA/hEigjluXspzs/hyeWxBjQ==", 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.830 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.0", 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDIX+5mrZF/NhAA33M6ummG4r4yXU8/YOL5WA==", 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.831 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.1", 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDMX+5myVz5DxAAMYfTPep6ubD7Cxnfa9dMCg==", 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.2", 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDPX+5mA7ahIRAAf/osnY5l83p8ZWna0ErD2Q==", 2024-09-21T05:59:34.832 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.3", 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDSX+5m+rDpNRAApGPZJIlAaA0DyjtPXPgoaQ==", 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.833 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.4", 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDYX+5mKL9SORAA21ITQUwUSoDUjXaD0Cwb3g==", 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.5", 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDcX+5minRqFBAAMJOBQNvJFD4TNdKbPMLBig==", 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.834 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.6", 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDfX+5m6HadJhAAAv670Tvyi4IKMByFPiBlAw==", 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.835 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.7", 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDjX+5muPrrAhAAuRB7Muamiht+f7LvSmI6UQ==", 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.0", 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw==", 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.836 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow r", 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.1", 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mo94GJBAAMBSb1NPb2L93vzvnAh59kw==", 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T05:59:34.837 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.2", 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5m4IrnHxAAFjER3ywuS+FlbuA7bPZVqw==", 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.3", 2024-09-21T05:59:34.838 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mysQeKBAAq5c4bD3hbsU1H7JabaMjwQ==", 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.admin", 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mcUXKHhAAV1Rs0OMNVPeAsRiAVt67jA==", 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T05:59:34.839 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.x", 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5m2F9WOBAAZuQTUyR7oTT01CRD1jo+/Q==", 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:34.840 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.y", 2024-09-21T05:59:34.841 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mEjC/MxAAHgwIu4CLDgFajwD2rL5WKA==", 2024-09-21T05:59:34.841 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T05:59:34.841 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T05:59:34.841 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T05:59:34.841 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T05:59:34.841 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.841 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:34.841 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-21T05:59:34.841 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T05:59:34.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.827+0000 7f1d8ec00640 1 -- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1d64073590 msgr2=0x7f1d64075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:34.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.827+0000 7f1d8ec00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1d64073590 0x7f1d64075aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1d78007fd0 tx=0x7f1d78009040 comp rx=0 tx=0).stop 2024-09-21T05:59:34.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.827+0000 7f1d8ec00640 1 -- 172.21.15.50:0/244267818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d881527b0 msgr2=0x7f1d8814d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:34.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.827+0000 7f1d8ec00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d881527b0 0x7f1d8814d4b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000cbc0 tx=0x7f1d8000ccb0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.827+0000 7f1d8ec00640 1 -- 172.21.15.50:0/244267818 shutdown_connections 2024-09-21T05:59:34.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.827+0000 7f1d8ec00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1d64073590 0x7f1d64075aa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.827+0000 7f1d8ec00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d881527b0 0x7f1d8814d4b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.827+0000 7f1d8ec00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1d880b5c70 0x7f1d8814cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.827+0000 7f1d8ec00640 1 --2- 172.21.15.50:0/244267818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d880b4de0 0x7f1d88146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:34.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.827+0000 7f1d8ec00640 1 -- 172.21.15.50:0/244267818 >> 172.21.15.50:0/244267818 conn(0x7f1d880b2040 msgr2=0x7f1d8815afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:34.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.827+0000 7f1d8ec00640 1 -- 172.21.15.50:0/244267818 shutdown_connections 2024-09-21T05:59:34.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:34.827+0000 7f1d8ec00640 1 -- 172.21.15.50:0/244267818 wait complete. 2024-09-21T05:59:34.852 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T05:59:35.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.007+0000 7f8204000640 1 -- 172.21.15.50:0/2816015296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81e4003c20 msgr2=0x7f81e4004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:35.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.007+0000 7f8204000640 1 --2- 172.21.15.50:0/2816015296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81e4003c20 0x7f81e4004070 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009b60 tx=0x7f81f001d260 comp rx=0 tx=0).stop 2024-09-21T05:59:35.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.007+0000 7f8202000640 1 -- 172.21.15.50:0/2816015296 reap_dead start 2024-09-21T05:59:35.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.007+0000 7f8204000640 1 -- 172.21.15.50:0/2816015296 shutdown_connections 2024-09-21T05:59:35.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.007+0000 7f8204000640 1 -- 172.21.15.50:0/2816015296 >> 172.21.15.50:0/2816015296 conn(0x7f81fc0b2c40 msgr2=0x7f81fc0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:35.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.007+0000 7f8204000640 1 -- 172.21.15.50:0/2816015296 shutdown_connections 2024-09-21T05:59:35.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.007+0000 7f8204000640 1 -- 172.21.15.50:0/2816015296 wait complete. 2024-09-21T05:59:35.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.007+0000 7f8204000640 1 Processor -- start 2024-09-21T05:59:35.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.007+0000 7f8204000640 1 -- start start 2024-09-21T05:59:35.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8204000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81e4003c20 0x7f81fc14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:35.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8204000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81fc14d430 0x7f81fc143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:35.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8204000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81fc1504c0 0x7f81fc1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:35.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8204000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc167b10 con 0x7f81e4003c20 2024-09-21T05:59:35.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8204000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0b3d50 con 0x7f81fc14d430 2024-09-21T05:59:35.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8204000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc167c90 con 0x7f81fc1504c0 2024-09-21T05:59:35.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8201600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81e4003c20 0x7f81fc14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:35.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8201600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81e4003c20 0x7f81fc14ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49216/0 (socket says 172.21.15.50:49216) 2024-09-21T05:59:35.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8201600640 1 -- 172.21.15.50:0/3808360177 learned_addr learned my addr 172.21.15.50:0/3808360177 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:35.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8202000640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81fc1504c0 0x7f81fc1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:35.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8200c00640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81fc14d430 0x7f81fc143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:35.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8201600640 1 -- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81fc1504c0 msgr2=0x7f81fc1508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:35.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8201600640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81fc1504c0 0x7f81fc1508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:35.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8201600640 1 -- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81fc14d430 msgr2=0x7f81fc143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:35.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8201600640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81fc14d430 0x7f81fc143370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:35.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8201600640 1 -- 172.21.15.50:0/3808360177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc14f420 con 0x7f81e4003c20 2024-09-21T05:59:35.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8200c00640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81fc14d430 0x7f81fc143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:35.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8201600640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81e4003c20 0x7f81fc14ebf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f81f00af040 tx=0x7f81f0005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:35.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f81f9600640 1 -- 172.21.15.50:0/3808360177 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f00b1040 con 0x7f81e4003c20 2024-09-21T05:59:35.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f81f9600640 1 -- 172.21.15.50:0/3808360177 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81f00bb070 con 0x7f81e4003c20 2024-09-21T05:59:35.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f81f9600640 1 -- 172.21.15.50:0/3808360177 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0004290 con 0x7f81e4003c20 2024-09-21T05:59:35.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8204000640 1 -- 172.21.15.50:0/3808360177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc14f700 con 0x7f81e4003c20 2024-09-21T05:59:35.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8204000640 1 -- 172.21.15.50:0/3808360177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc14fcc0 con 0x7f81e4003c20 2024-09-21T05:59:35.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.011+0000 7f8204000640 1 -- 172.21.15.50:0/3808360177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc15f170 con 0x7f81e4003c20 2024-09-21T05:59:35.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.015+0000 7f81f9600640 1 -- 172.21.15.50:0/3808360177 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f81f00046d0 con 0x7f81e4003c20 2024-09-21T05:59:35.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.015+0000 7f81f9600640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f81dc073560 0x7f81dc075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:35.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.015+0000 7f81f9600640 1 -- 172.21.15.50:0/3808360177 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f81f012f670 con 0x7f81e4003c20 2024-09-21T05:59:35.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.015+0000 7f8200c00640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f81dc073560 0x7f81dc075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:35.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.015+0000 7f8200c00640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f81dc073560 0x7f81dc075a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f81e8001ce0 tx=0x7f81e8001bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:35.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.019+0000 7f81f9600640 1 -- 172.21.15.50:0/3808360177 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f81f0016fa0 con 0x7f81e4003c20 2024-09-21T05:59:35.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.199+0000 7f8204000640 1 -- 172.21.15.50:0/3808360177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f81fc0b2430 con 0x7f81e4003c20 2024-09-21T05:59:35.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f81f9600640 1 -- 172.21.15.50:0/3808360177 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) ==== 84+0+65 (secure 0 0 0) 0x7f81f00fa510 con 0x7f81e4003c20 2024-09-21T05:59:35.205 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:35.206 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T05:59:35.206 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:35.206 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-09-21T05:59:35.206 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-09-21T05:59:35.206 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:35.206 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T05:59:35.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f8204000640 1 -- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f81dc073560 msgr2=0x7f81dc075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:35.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f8204000640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f81dc073560 0x7f81dc075a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f81e8001ce0 tx=0x7f81e8001bd0 comp rx=0 tx=0).stop 2024-09-21T05:59:35.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f8204000640 1 -- 172.21.15.50:0/3808360177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81e4003c20 msgr2=0x7f81fc14ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:35.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f8204000640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81e4003c20 0x7f81fc14ebf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f81f00af040 tx=0x7f81f0005590 comp rx=0 tx=0).stop 2024-09-21T05:59:35.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f8204000640 1 -- 172.21.15.50:0/3808360177 shutdown_connections 2024-09-21T05:59:35.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f8204000640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81fc1504c0 0x7f81fc1508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:35.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f8204000640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81fc14d430 0x7f81fc143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:35.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f8204000640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f81dc073560 0x7f81dc075a70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:35.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f8204000640 1 --2- 172.21.15.50:0/3808360177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81e4003c20 0x7f81fc14ebf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:35.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f8204000640 1 -- 172.21.15.50:0/3808360177 >> 172.21.15.50:0/3808360177 conn(0x7f81fc0b2c40 msgr2=0x7f81fc0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:35.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f8204000640 1 -- 172.21.15.50:0/3808360177 shutdown_connections 2024-09-21T05:59:35.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.203+0000 7f8204000640 1 -- 172.21.15.50:0/3808360177 wait complete. 2024-09-21T05:59:35.228 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T05:59:35.228 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T05:59:35.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.387+0000 7f1950000640 1 -- 172.21.15.50:0/349197076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1948160d50 msgr2=0x7f1948074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:35.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.387+0000 7f1950000640 1 --2- 172.21.15.50:0/349197076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1948160d50 0x7f1948074410 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f19480713e0 tx=0x7f193401c830 comp rx=0 tx=0).stop 2024-09-21T05:59:35.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.387+0000 7f1950000640 1 -- 172.21.15.50:0/349197076 shutdown_connections 2024-09-21T05:59:35.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.387+0000 7f1950000640 1 --2- 172.21.15.50:0/349197076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1948160d50 0x7f1948074410 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:35.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.387+0000 7f1950000640 1 --2- 172.21.15.50:0/349197076 >> v2:172.21.15.88:3300/0 conn(0x7f1948157030 0x7f19480b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:35.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.387+0000 7f1950000640 1 -- 172.21.15.50:0/349197076 >> 172.21.15.50:0/349197076 conn(0x7f19480708c0 msgr2=0x7f1948070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:35.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.387+0000 7f1950000640 1 -- 172.21.15.50:0/349197076 shutdown_connections 2024-09-21T05:59:35.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.387+0000 7f1950000640 1 -- 172.21.15.50:0/349197076 wait complete. 2024-09-21T05:59:35.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f1950000640 1 Processor -- start 2024-09-21T05:59:35.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f1950000640 1 -- start start 2024-09-21T05:59:35.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f1950000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1948157030 0x7f194814f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:35.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f1950000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1948160d50 0x7f1948146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f1950000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1948161160 0x7f194814fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f1950000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f194816a740 con 0x7f1948157030 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f1950000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f194816a5c0 con 0x7f1948161160 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f1950000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f194816a8c0 con 0x7f1948160d50 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f194d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1948157030 0x7f194814f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f194d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1948157030 0x7f194814f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49226/0 (socket says 172.21.15.50:49226) 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f194d600640 1 -- 172.21.15.50:0/3884550876 learned_addr learned my addr 172.21.15.50:0/3884550876 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f194cc00640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1948160d50 0x7f1948146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f194e000640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1948161160 0x7f194814fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f194d600640 1 -- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1948160d50 msgr2=0x7f1948146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f194d600640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1948160d50 0x7f1948146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f194d600640 1 -- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1948161160 msgr2=0x7f194814fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f194d600640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1948161160 0x7f194814fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f194d600640 1 -- 172.21.15.50:0/3884550876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194814bac0 con 0x7f1948157030 2024-09-21T05:59:35.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.391+0000 7f194e000640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1948161160 0x7f194814fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:35.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.395+0000 7f194d600640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1948157030 0x7f194814f7e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f193c00d720 tx=0x7f193c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:35.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.395+0000 7f1945600640 1 -- 172.21.15.50:0/3884550876 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c017070 con 0x7f1948157030 2024-09-21T05:59:35.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.395+0000 7f1945600640 1 -- 172.21.15.50:0/3884550876 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f193c009d20 con 0x7f1948157030 2024-09-21T05:59:35.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.395+0000 7f1945600640 1 -- 172.21.15.50:0/3884550876 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c005280 con 0x7f1948157030 2024-09-21T05:59:35.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.395+0000 7f1950000640 1 -- 172.21.15.50:0/3884550876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f194814bda0 con 0x7f1948157030 2024-09-21T05:59:35.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.395+0000 7f1950000640 1 -- 172.21.15.50:0/3884550876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1948147770 con 0x7f1948157030 2024-09-21T05:59:35.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.395+0000 7f1950000640 1 -- 172.21.15.50:0/3884550876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19480d2260 con 0x7f1948157030 2024-09-21T05:59:35.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.395+0000 7f1945600640 1 -- 172.21.15.50:0/3884550876 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f193c005420 con 0x7f1948157030 2024-09-21T05:59:35.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.395+0000 7f1945600640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1928073590 0x7f1928075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:35.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.399+0000 7f1945600640 1 -- 172.21.15.50:0/3884550876 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f193c0945c0 con 0x7f1948157030 2024-09-21T05:59:35.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.399+0000 7f194cc00640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1928073590 0x7f1928075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:35.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.399+0000 7f194cc00640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1928073590 0x7f1928075aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f19340af000 tx=0x7f193401b070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:35.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.403+0000 7f1945600640 1 -- 172.21.15.50:0/3884550876 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f193c05ed50 con 0x7f1948157030 2024-09-21T05:59:35.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:35.583+0000 7f1950000640 1 -- 172.21.15.50:0/3884550876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f19480b16d0 con 0x7f1948157030 2024-09-21T05:59:36.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.407+0000 7f1945600640 1 -- 172.21.15.50:0/3884550876 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) ==== 138+0+0 (secure 0 0 0) 0x7f193c063780 con 0x7f1948157030 2024-09-21T05:59:36.411 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-09-21T05:59:36.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.411+0000 7f1950000640 1 -- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1928073590 msgr2=0x7f1928075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:36.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.411+0000 7f1950000640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1928073590 0x7f1928075aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f19340af000 tx=0x7f193401b070 comp rx=0 tx=0).stop 2024-09-21T05:59:36.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.411+0000 7f1950000640 1 -- 172.21.15.50:0/3884550876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1948157030 msgr2=0x7f194814f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:36.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.411+0000 7f1950000640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1948157030 0x7f194814f7e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f193c00d720 tx=0x7f193c00dbe0 comp rx=0 tx=0).stop 2024-09-21T05:59:36.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.411+0000 7f1950000640 1 -- 172.21.15.50:0/3884550876 shutdown_connections 2024-09-21T05:59:36.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.411+0000 7f1950000640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1948161160 0x7f194814fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:36.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.411+0000 7f1950000640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1948160d50 0x7f1948146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:36.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.411+0000 7f1950000640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1928073590 0x7f1928075aa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:36.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.411+0000 7f1950000640 1 --2- 172.21.15.50:0/3884550876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1948157030 0x7f194814f7e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:36.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.411+0000 7f1950000640 1 -- 172.21.15.50:0/3884550876 >> 172.21.15.50:0/3884550876 conn(0x7f19480708c0 msgr2=0x7f1948152bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:36.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.411+0000 7f1950000640 1 -- 172.21.15.50:0/3884550876 shutdown_connections 2024-09-21T05:59:36.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.411+0000 7f1950000640 1 -- 172.21.15.50:0/3884550876 wait complete. 2024-09-21T05:59:36.446 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T05:59:36.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.611+0000 7f780fc00640 1 -- 172.21.15.50:0/3810909263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780813e790 msgr2=0x7f78080cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:36.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.611+0000 7f780fc00640 1 --2- 172.21.15.50:0/3810909263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780813e790 0x7f78080cc320 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7808072bb0 tx=0x7f77f801c800 comp rx=0 tx=0).stop 2024-09-21T05:59:36.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.611+0000 7f780fc00640 1 -- 172.21.15.50:0/3810909263 shutdown_connections 2024-09-21T05:59:36.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.611+0000 7f780fc00640 1 --2- 172.21.15.50:0/3810909263 >> v2:172.21.15.88:3300/0 conn(0x7f780815edd0 0x7f78080cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:36.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.611+0000 7f780fc00640 1 --2- 172.21.15.50:0/3810909263 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f78080cc860 0x7f780815e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:36.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.611+0000 7f780fc00640 1 --2- 172.21.15.50:0/3810909263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780813e790 0x7f78080cc320 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:36.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.611+0000 7f780fc00640 1 -- 172.21.15.50:0/3810909263 >> 172.21.15.50:0/3810909263 conn(0x7f78080b2040 msgr2=0x7f78080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:36.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.611+0000 7f780fc00640 1 -- 172.21.15.50:0/3810909263 shutdown_connections 2024-09-21T05:59:36.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.611+0000 7f780fc00640 1 -- 172.21.15.50:0/3810909263 wait complete. 2024-09-21T05:59:36.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.615+0000 7f780fc00640 1 Processor -- start 2024-09-21T05:59:36.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.615+0000 7f780fc00640 1 -- start start 2024-09-21T05:59:36.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.615+0000 7f780fc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78080cc860 0x7f78081fd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:36.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.615+0000 7f780fc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f780813e790 0x7f78080ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:36.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.615+0000 7f780fc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f780815edd0 0x7f78080aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:36.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.615+0000 7f780fc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7808140590 con 0x7f78080cc860 2024-09-21T05:59:36.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.615+0000 7f780fc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78080b55b0 con 0x7f780813e790 2024-09-21T05:59:36.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.615+0000 7f780fc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7808140710 con 0x7f780815edd0 2024-09-21T05:59:36.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.615+0000 7f780d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78080cc860 0x7f78081fd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:36.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.615+0000 7f780d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78080cc860 0x7f78081fd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49254/0 (socket says 172.21.15.50:49254) 2024-09-21T05:59:36.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.615+0000 7f780d200640 1 -- 172.21.15.50:0/2654105894 learned_addr learned my addr 172.21.15.50:0/2654105894 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:36.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f77ffe00640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f780813e790 0x7f78080ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:36.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f780dc00640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f780815edd0 0x7f78080aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:36.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f780d200640 1 -- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f780815edd0 msgr2=0x7f78080aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:36.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f780d200640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f780815edd0 0x7f78080aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:36.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f780d200640 1 -- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f780813e790 msgr2=0x7f78080ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:36.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f780d200640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f780813e790 0x7f78080ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:36.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f780d200640 1 -- 172.21.15.50:0/2654105894 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78081f9450 con 0x7f78080cc860 2024-09-21T05:59:36.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f780dc00640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f780815edd0 0x7f78080aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:36.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f780d200640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78080cc860 0x7f78081fd190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f77f80b0040 tx=0x7f77f80a5680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:36.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f7805600640 1 -- 172.21.15.50:0/2654105894 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f80b2040 con 0x7f78080cc860 2024-09-21T05:59:36.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f7805600640 1 -- 172.21.15.50:0/2654105894 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77f80042c0 con 0x7f78080cc860 2024-09-21T05:59:36.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f780fc00640 1 -- 172.21.15.50:0/2654105894 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78081f9730 con 0x7f78080cc860 2024-09-21T05:59:36.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f7805600640 1 -- 172.21.15.50:0/2654105894 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8005290 con 0x7f78080cc860 2024-09-21T05:59:36.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f780fc00640 1 -- 172.21.15.50:0/2654105894 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78081f9cf0 con 0x7f78080cc860 2024-09-21T05:59:36.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f780fc00640 1 -- 172.21.15.50:0/2654105894 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78080cc320 con 0x7f78080cc860 2024-09-21T05:59:36.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.619+0000 7f7805600640 1 -- 172.21.15.50:0/2654105894 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f77f8005430 con 0x7f78080cc860 2024-09-21T05:59:36.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.623+0000 7f7805600640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77e4073560 0x7f77e4075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:36.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.623+0000 7f7805600640 1 -- 172.21.15.50:0/2654105894 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5227+0+0 (secure 0 0 0) 0x7f77f812e4c0 con 0x7f78080cc860 2024-09-21T05:59:36.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.623+0000 7f77ffe00640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77e4073560 0x7f77e4075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:36.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.623+0000 7f77ffe00640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77e4073560 0x7f77e4075a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f77f0005fd0 tx=0x7f77f0005d80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:36.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.627+0000 7f7805600640 1 -- 172.21.15.50:0/2654105894 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f780815f1e0 con 0x7f78080cc860 2024-09-21T05:59:36.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:36.803+0000 7f780fc00640 1 -- 172.21.15.50:0/2654105894 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7808051050 con 0x7f78080cc860 2024-09-21T05:59:37.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.415+0000 7f7805600640 1 -- 172.21.15.50:0/2654105894 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) ==== 170+0+0 (secure 0 0 0) 0x7f77f80164b0 con 0x7f78080cc860 2024-09-21T05:59:37.418 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-09-21T05:59:37.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.419+0000 7f780fc00640 1 -- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77e4073560 msgr2=0x7f77e4075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:37.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.419+0000 7f780fc00640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77e4073560 0x7f77e4075a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f77f0005fd0 tx=0x7f77f0005d80 comp rx=0 tx=0).stop 2024-09-21T05:59:37.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.419+0000 7f780fc00640 1 -- 172.21.15.50:0/2654105894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78080cc860 msgr2=0x7f78081fd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:37.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.419+0000 7f780fc00640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78080cc860 0x7f78081fd190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f77f80b0040 tx=0x7f77f80a5680 comp rx=0 tx=0).stop 2024-09-21T05:59:37.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.419+0000 7f780fc00640 1 -- 172.21.15.50:0/2654105894 shutdown_connections 2024-09-21T05:59:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.419+0000 7f780fc00640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f780815edd0 0x7f78080aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.419+0000 7f780fc00640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f780813e790 0x7f78080ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.419+0000 7f780fc00640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77e4073560 0x7f77e4075a70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.419+0000 7f780fc00640 1 --2- 172.21.15.50:0/2654105894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78080cc860 0x7f78081fd190 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.419+0000 7f780fc00640 1 -- 172.21.15.50:0/2654105894 >> 172.21.15.50:0/2654105894 conn(0x7f78080b2040 msgr2=0x7f78081ed740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.419+0000 7f780fc00640 1 -- 172.21.15.50:0/2654105894 shutdown_connections 2024-09-21T05:59:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.419+0000 7f780fc00640 1 -- 172.21.15.50:0/2654105894 wait complete. 2024-09-21T05:59:37.463 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T05:59:37.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.623+0000 7fe441e00640 1 -- 172.21.15.50:0/173353973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe43c1527b0 msgr2=0x7fe43c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:37.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.623+0000 7fe441e00640 1 --2- 172.21.15.50:0/173353973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe43c1527b0 0x7fe43c074e30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe424009900 tx=0x7fe42401c820 comp rx=0 tx=0).stop 2024-09-21T05:59:37.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.623+0000 7fe441e00640 1 -- 172.21.15.50:0/173353973 shutdown_connections 2024-09-21T05:59:37.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.623+0000 7fe441e00640 1 --2- 172.21.15.50:0/173353973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe43c1527b0 0x7fe43c074e30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:37.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.623+0000 7fe441e00640 1 --2- 172.21.15.50:0/173353973 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe43c0b5c70 0x7fe43c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:37.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.623+0000 7fe441e00640 1 -- 172.21.15.50:0/173353973 >> 172.21.15.50:0/173353973 conn(0x7fe43c0b2040 msgr2=0x7fe43c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:37.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.623+0000 7fe441e00640 1 -- 172.21.15.50:0/173353973 shutdown_connections 2024-09-21T05:59:37.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.623+0000 7fe441e00640 1 -- 172.21.15.50:0/173353973 wait complete. 2024-09-21T05:59:37.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.623+0000 7fe441e00640 1 Processor -- start 2024-09-21T05:59:37.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.623+0000 7fe441e00640 1 -- start start 2024-09-21T05:59:37.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe441e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe43c0b4de0 0x7fe43c1f8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:37.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe441e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe43c0b5c70 0x7fe43c15ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:37.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe441e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe43c1527b0 0x7fe43c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:37.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe441e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c168c50 con 0x7fe43c1527b0 2024-09-21T05:59:37.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe441e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c168ad0 con 0x7fe43c0b5c70 2024-09-21T05:59:37.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe441e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe43c168dd0 con 0x7fe43c0b4de0 2024-09-21T05:59:37.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe43b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe43c1527b0 0x7fe43c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:37.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe43b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe43c1527b0 0x7fe43c0ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46512/0 (socket says 172.21.15.50:46512) 2024-09-21T05:59:37.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe43b400640 1 -- 172.21.15.50:0/2386377133 learned_addr learned my addr 172.21.15.50:0/2386377133 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:37.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe433e00640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe43c0b5c70 0x7fe43c15ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:37.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe43aa00640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe43c0b4de0 0x7fe43c1f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:37.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe43b400640 1 -- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe43c0b4de0 msgr2=0x7fe43c1f8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:37.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe43b400640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe43c0b4de0 0x7fe43c1f8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:37.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe43b400640 1 -- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe43c0b5c70 msgr2=0x7fe43c15ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:37.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe43b400640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe43c0b5c70 0x7fe43c15ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:37.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe43b400640 1 -- 172.21.15.50:0/2386377133 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c1f90e0 con 0x7fe43c1527b0 2024-09-21T05:59:37.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe43b400640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe43c1527b0 0x7fe43c0ae8e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c740 tx=0x7fe43400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:37.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe433400640 1 -- 172.21.15.50:0/2386377133 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434003b80 con 0x7fe43c1527b0 2024-09-21T05:59:37.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe433400640 1 -- 172.21.15.50:0/2386377133 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe43400ce50 con 0x7fe43c1527b0 2024-09-21T05:59:37.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe441e00640 1 -- 172.21.15.50:0/2386377133 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c1f93c0 con 0x7fe43c1527b0 2024-09-21T05:59:37.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe433400640 1 -- 172.21.15.50:0/2386377133 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43400a850 con 0x7fe43c1527b0 2024-09-21T05:59:37.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe441e00640 1 -- 172.21.15.50:0/2386377133 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c1f9980 con 0x7fe43c1527b0 2024-09-21T05:59:37.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.627+0000 7fe441e00640 1 -- 172.21.15.50:0/2386377133 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43c1f8e20 con 0x7fe43c1527b0 2024-09-21T05:59:37.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.631+0000 7fe433400640 1 -- 172.21.15.50:0/2386377133 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fe43400a9f0 con 0x7fe43c1527b0 2024-09-21T05:59:37.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.631+0000 7fe433400640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe418073590 0x7fe418075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:37.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.631+0000 7fe433400640 1 -- 172.21.15.50:0/2386377133 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5609+0+0 (secure 0 0 0) 0x7fe4340950b0 con 0x7fe43c1527b0 2024-09-21T05:59:37.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.631+0000 7fe43aa00640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe418073590 0x7fe418075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:37.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.631+0000 7fe43aa00640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe418073590 0x7fe418075aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009b40 tx=0x7fe42c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:37.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.635+0000 7fe433400640 1 -- 172.21.15.50:0/2386377133 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe43405f500 con 0x7fe43c1527b0 2024-09-21T05:59:37.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:37.847+0000 7fe441e00640 1 -- 172.21.15.50:0/2386377133 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe43c14a4a0 con 0x7fe43c1527b0 2024-09-21T05:59:38.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.415+0000 7fe433400640 1 -- 172.21.15.50:0/2386377133 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) ==== 172+0+0 (secure 0 0 0) 0x7fe434063f30 con 0x7fe43c1527b0 2024-09-21T05:59:38.420 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 6 and data pool 7 2024-09-21T05:59:38.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.419+0000 7fe441e00640 1 -- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe418073590 msgr2=0x7fe418075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.419+0000 7fe441e00640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe418073590 0x7fe418075aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009b40 tx=0x7fe42c0091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.419+0000 7fe441e00640 1 -- 172.21.15.50:0/2386377133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe43c1527b0 msgr2=0x7fe43c0ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.419+0000 7fe441e00640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe43c1527b0 0x7fe43c0ae8e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c740 tx=0x7fe43400cc00 comp rx=0 tx=0).stop 2024-09-21T05:59:38.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.419+0000 7fe441e00640 1 -- 172.21.15.50:0/2386377133 shutdown_connections 2024-09-21T05:59:38.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.419+0000 7fe441e00640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe418073590 0x7fe418075aa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.419+0000 7fe441e00640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe43c1527b0 0x7fe43c0ae8e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.419+0000 7fe441e00640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe43c0b5c70 0x7fe43c15ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.419+0000 7fe441e00640 1 --2- 172.21.15.50:0/2386377133 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe43c0b4de0 0x7fe43c1f8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.419+0000 7fe441e00640 1 -- 172.21.15.50:0/2386377133 >> 172.21.15.50:0/2386377133 conn(0x7fe43c0b2040 msgr2=0x7fe43c14cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:38.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.419+0000 7fe441e00640 1 -- 172.21.15.50:0/2386377133 shutdown_connections 2024-09-21T05:59:38.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.419+0000 7fe441e00640 1 -- 172.21.15.50:0/2386377133 wait complete. 2024-09-21T05:59:38.469 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:59:38.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.627+0000 7f3e83400640 1 -- 172.21.15.50:0/650433423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e7c157030 msgr2=0x7f3e7c0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.627+0000 7f3e83400640 1 --2- 172.21.15.50:0/650433423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e7c157030 0x7f3e7c0b5a80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009840 tx=0x7f3e6c01c6b0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.627+0000 7f3e83400640 1 -- 172.21.15.50:0/650433423 shutdown_connections 2024-09-21T05:59:38.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.627+0000 7f3e83400640 1 --2- 172.21.15.50:0/650433423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e7c157030 0x7f3e7c0b5a80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.627+0000 7f3e83400640 1 -- 172.21.15.50:0/650433423 >> 172.21.15.50:0/650433423 conn(0x7f3e7c0708c0 msgr2=0x7f3e7c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:38.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.627+0000 7f3e83400640 1 -- 172.21.15.50:0/650433423 shutdown_connections 2024-09-21T05:59:38.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.627+0000 7f3e83400640 1 -- 172.21.15.50:0/650433423 wait complete. 2024-09-21T05:59:38.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.627+0000 7f3e83400640 1 Processor -- start 2024-09-21T05:59:38.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.627+0000 7f3e83400640 1 -- start start 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e83400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e7c157030 0x7f3e7c13de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e83400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e7c160d50 0x7f3e7c1551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e83400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e7c152860 0x7f3e7c155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e83400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c16b8d0 con 0x7f3e7c160d50 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e83400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c16b750 con 0x7f3e7c157030 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e83400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7c16ba50 con 0x7f3e7c152860 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e73e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e7c160d50 0x7f3e7c1551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e80a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e7c157030 0x7f3e7c13de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e73e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e7c160d50 0x7f3e7c1551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46514/0 (socket says 172.21.15.50:46514) 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e73e00640 1 -- 172.21.15.50:0/3524284149 learned_addr learned my addr 172.21.15.50:0/3524284149 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e80a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e7c157030 0x7f3e7c13de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:38236/0 (socket says 172.21.15.50:38236) 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e81400640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e7c152860 0x7f3e7c155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e73e00640 1 -- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e7c152860 msgr2=0x7f3e7c155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e73e00640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e7c152860 0x7f3e7c155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e73e00640 1 -- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e7c157030 msgr2=0x7f3e7c13de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e73e00640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e7c157030 0x7f3e7c13de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e73e00640 1 -- 172.21.15.50:0/3524284149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7c13e370 con 0x7f3e7c160d50 2024-09-21T05:59:38.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e73e00640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e7c160d50 0x7f3e7c1551f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400c4f0 tx=0x7f3e6400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:38.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e79600640 1 -- 172.21.15.50:0/3524284149 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64016020 con 0x7f3e7c160d50 2024-09-21T05:59:38.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e83400640 1 -- 172.21.15.50:0/3524284149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7c1517c0 con 0x7f3e7c160d50 2024-09-21T05:59:38.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e79600640 1 -- 172.21.15.50:0/3524284149 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e64012940 con 0x7f3e7c160d50 2024-09-21T05:59:38.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e79600640 1 -- 172.21.15.50:0/3524284149 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64012d90 con 0x7f3e7c160d50 2024-09-21T05:59:38.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e83400640 1 -- 172.21.15.50:0/3524284149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e7c151d80 con 0x7f3e7c160d50 2024-09-21T05:59:38.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.631+0000 7f3e83400640 1 -- 172.21.15.50:0/3524284149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7c151530 con 0x7f3e7c160d50 2024-09-21T05:59:38.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.635+0000 7f3e79600640 1 -- 172.21.15.50:0/3524284149 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f3e64019020 con 0x7f3e7c160d50 2024-09-21T05:59:38.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.635+0000 7f3e79600640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3e58073560 0x7f3e58075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.635+0000 7f3e79600640 1 -- 172.21.15.50:0/3524284149 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f3e64094d60 con 0x7f3e7c160d50 2024-09-21T05:59:38.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.635+0000 7f3e80a00640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3e58073560 0x7f3e58075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.635+0000 7f3e80a00640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3e58073560 0x7f3e58075a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0af000 tx=0x7f3e6c01aef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:38.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.639+0000 7f3e79600640 1 -- 172.21.15.50:0/3524284149 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3e64020020 con 0x7f3e7c160d50 2024-09-21T05:59:38.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.815+0000 7f3e83400640 1 -- 172.21.15.50:0/3524284149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e7c0d2260 con 0x7f3e7c160d50 2024-09-21T05:59:38.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.815+0000 7f3e79600640 1 -- 172.21.15.50:0/3524284149 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16638 (secure 0 0 0) 0x7f3e6405f210 con 0x7f3e7c160d50 2024-09-21T05:59:38.821 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:38.821 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":43,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:59:37.853348+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:59:35.596192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T05:59:36.818395+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:59:38.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.819+0000 7f3e83400640 1 -- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3e58073560 msgr2=0x7f3e58075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.819+0000 7f3e83400640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3e58073560 0x7f3e58075a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0af000 tx=0x7f3e6c01aef0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.819+0000 7f3e83400640 1 -- 172.21.15.50:0/3524284149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e7c160d50 msgr2=0x7f3e7c1551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.819+0000 7f3e83400640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e7c160d50 0x7f3e7c1551f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400c4f0 tx=0x7f3e6400c9b0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.819+0000 7f3e83400640 1 -- 172.21.15.50:0/3524284149 shutdown_connections 2024-09-21T05:59:38.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.819+0000 7f3e83400640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e7c152860 0x7f3e7c155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.819+0000 7f3e83400640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3e58073560 0x7f3e58075a70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.819+0000 7f3e83400640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e7c160d50 0x7f3e7c1551f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.819+0000 7f3e83400640 1 --2- 172.21.15.50:0/3524284149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e7c157030 0x7f3e7c13de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.819+0000 7f3e83400640 1 -- 172.21.15.50:0/3524284149 >> 172.21.15.50:0/3524284149 conn(0x7f3e7c0708c0 msgr2=0x7f3e7c0b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:38.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.819+0000 7f3e83400640 1 -- 172.21.15.50:0/3524284149 shutdown_connections 2024-09-21T05:59:38.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.819+0000 7f3e83400640 1 -- 172.21.15.50:0/3524284149 wait complete. 2024-09-21T05:59:38.844 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:59:39.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.999+0000 7f15cf200640 1 -- 172.21.15.50:0/2557209162 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15c80b5c70 msgr2=0x7f15c815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.999+0000 7f15cf200640 1 --2- 172.21.15.50:0/2557209162 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15c80b5c70 0x7f15c815bb60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009bf0 tx=0x7f15b801ca40 comp rx=0 tx=0).stop 2024-09-21T05:59:39.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.999+0000 7f15cf200640 1 -- 172.21.15.50:0/2557209162 shutdown_connections 2024-09-21T05:59:39.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.999+0000 7f15cf200640 1 --2- 172.21.15.50:0/2557209162 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15c80b5c70 0x7f15c815bb60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.999+0000 7f15cf200640 1 -- 172.21.15.50:0/2557209162 >> 172.21.15.50:0/2557209162 conn(0x7f15c80b2040 msgr2=0x7f15c80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:39.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.999+0000 7f15cf200640 1 -- 172.21.15.50:0/2557209162 shutdown_connections 2024-09-21T05:59:39.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.999+0000 7f15cf200640 1 -- 172.21.15.50:0/2557209162 wait complete. 2024-09-21T05:59:39.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:38.999+0000 7f15cf200640 1 Processor -- start 2024-09-21T05:59:39.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15cf200640 1 -- start start 2024-09-21T05:59:39.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15cf200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15c80b4de0 0x7f15c815aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15cf200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15c80b5c70 0x7f15c813fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15cf200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15c81527b0 0x7f15c81fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15cf200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f15c8169cb0 con 0x7f15c80b5c70 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15cf200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f15c8169b30 con 0x7f15c80b4de0 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15cf200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f15c8169e30 con 0x7f15c81527b0 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15bf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15c80b5c70 0x7f15c813fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15bf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15c80b5c70 0x7f15c813fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46530/0 (socket says 172.21.15.50:46530) 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15bf400640 1 -- 172.21.15.50:0/2709888744 learned_addr learned my addr 172.21.15.50:0/2709888744 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15c7e00640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15c80b4de0 0x7f15c815aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15cd200640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15c81527b0 0x7f15c81fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15bf400640 1 -- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15c81527b0 msgr2=0x7f15c81fc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15bf400640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15c81527b0 0x7f15c81fc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15bf400640 1 -- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15c80b4de0 msgr2=0x7f15c815aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15bf400640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15c80b4de0 0x7f15c815aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15bf400640 1 -- 172.21.15.50:0/2709888744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c81fcd00 con 0x7f15c80b5c70 2024-09-21T05:59:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15c7e00640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15c80b4de0 0x7f15c815aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:59:39.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15bf400640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15c80b5c70 0x7f15c813fbd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c3c0 tx=0x7f15b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:39.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15c4c00640 1 -- 172.21.15.50:0/2709888744 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0017070 con 0x7f15c80b5c70 2024-09-21T05:59:39.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15c4c00640 1 -- 172.21.15.50:0/2709888744 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15b00053c0 con 0x7f15c80b5c70 2024-09-21T05:59:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15c4c00640 1 -- 172.21.15.50:0/2709888744 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0003860 con 0x7f15c80b5c70 2024-09-21T05:59:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15cf200640 1 -- 172.21.15.50:0/2709888744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c8154670 con 0x7f15c80b5c70 2024-09-21T05:59:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.003+0000 7f15cf200640 1 -- 172.21.15.50:0/2709888744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c81fbfc0 con 0x7f15c80b5c70 2024-09-21T05:59:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.007+0000 7f15cf200640 1 -- 172.21.15.50:0/2709888744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c815bb60 con 0x7f15c80b5c70 2024-09-21T05:59:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.007+0000 7f15c4c00640 1 -- 172.21.15.50:0/2709888744 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f15b001a070 con 0x7f15c80b5c70 2024-09-21T05:59:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.007+0000 7f15c4c00640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15a4073560 0x7f15a4075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.007+0000 7f15c4c00640 1 -- 172.21.15.50:0/2709888744 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f15b0093bb0 con 0x7f15c80b5c70 2024-09-21T05:59:39.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.007+0000 7f15c7e00640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15a4073560 0x7f15a4075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.007+0000 7f15c7e00640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15a4073560 0x7f15a4075a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15b80af000 tx=0x7f15b801b280 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:39.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.011+0000 7f15c4c00640 1 -- 172.21.15.50:0/2709888744 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f15b005e030 con 0x7f15c80b5c70 2024-09-21T05:59:39.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.191+0000 7f15cf200640 1 -- 172.21.15.50:0/2709888744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15c80b2730 con 0x7f15c80b5c70 2024-09-21T05:59:39.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.191+0000 7f15c4c00640 1 -- 172.21.15.50:0/2709888744 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16638 (secure 0 0 0) 0x7f15b0062a60 con 0x7f15c80b5c70 2024-09-21T05:59:39.197 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:39.197 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":43,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:59:37.853348+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:59:35.596192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T05:59:36.818395+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:59:39.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.195+0000 7f15cf200640 1 -- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15a4073560 msgr2=0x7f15a4075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.195+0000 7f15cf200640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15a4073560 0x7f15a4075a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15b80af000 tx=0x7f15b801b280 comp rx=0 tx=0).stop 2024-09-21T05:59:39.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.195+0000 7f15cf200640 1 -- 172.21.15.50:0/2709888744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15c80b5c70 msgr2=0x7f15c813fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.195+0000 7f15cf200640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15c80b5c70 0x7f15c813fbd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c3c0 tx=0x7f15b000c880 comp rx=0 tx=0).stop 2024-09-21T05:59:39.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.195+0000 7f15cf200640 1 -- 172.21.15.50:0/2709888744 shutdown_connections 2024-09-21T05:59:39.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.195+0000 7f15cf200640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15c81527b0 0x7f15c81fc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.195+0000 7f15cf200640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15a4073560 0x7f15a4075a70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.195+0000 7f15cf200640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15c80b5c70 0x7f15c813fbd0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.195+0000 7f15cf200640 1 --2- 172.21.15.50:0/2709888744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15c80b4de0 0x7f15c815aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.195+0000 7f15cf200640 1 -- 172.21.15.50:0/2709888744 >> 172.21.15.50:0/2709888744 conn(0x7f15c80b2040 msgr2=0x7f15c8154df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:39.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.195+0000 7f15cf200640 1 -- 172.21.15.50:0/2709888744 shutdown_connections 2024-09-21T05:59:39.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.195+0000 7f15cf200640 1 -- 172.21.15.50:0/2709888744 wait complete. 2024-09-21T05:59:39.221 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T05:59:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.379+0000 7f57afc00640 1 -- 172.21.15.50:0/192662731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a80b5c70 msgr2=0x7f57a815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.379+0000 7f57afc00640 1 --2- 172.21.15.50:0/192662731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a80b5c70 0x7f57a815bb60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f57a8074b20 tx=0x7f579c01c820 comp rx=0 tx=0).stop 2024-09-21T05:59:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.379+0000 7f57afc00640 1 -- 172.21.15.50:0/192662731 shutdown_connections 2024-09-21T05:59:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.379+0000 7f57afc00640 1 --2- 172.21.15.50:0/192662731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a80b5c70 0x7f57a815bb60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.379+0000 7f57afc00640 1 -- 172.21.15.50:0/192662731 >> 172.21.15.50:0/192662731 conn(0x7f57a80b2040 msgr2=0x7f57a80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.379+0000 7f57afc00640 1 -- 172.21.15.50:0/192662731 shutdown_connections 2024-09-21T05:59:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.379+0000 7f57afc00640 1 -- 172.21.15.50:0/192662731 wait complete. 2024-09-21T05:59:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.379+0000 7f57afc00640 1 Processor -- start 2024-09-21T05:59:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.379+0000 7f57afc00640 1 -- start start 2024-09-21T05:59:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57afc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57a80b4de0 0x7f57a8154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57afc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a80b5c70 0x7f57a8140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57afc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57a81527b0 0x7f57a813d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57afc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57a8169bd0 con 0x7f57a80b5c70 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57afc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57a8169a50 con 0x7f57a80b4de0 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57afc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f57a8169d50 con 0x7f57a81527b0 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57a7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a80b5c70 0x7f57a8140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57ad200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57a80b4de0 0x7f57a8154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57ad200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57a80b4de0 0x7f57a8154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:38262/0 (socket says 172.21.15.50:38262) 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57ad200640 1 -- 172.21.15.50:0/3003466524 learned_addr learned my addr 172.21.15.50:0/3003466524 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57adc00640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57a81527b0 0x7f57a813d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57ad200640 1 -- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57a81527b0 msgr2=0x7f57a813d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57ad200640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57a81527b0 0x7f57a813d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57ad200640 1 -- 172.21.15.50:0/3003466524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a80b5c70 msgr2=0x7f57a8140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57ad200640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a80b5c70 0x7f57a8140b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57ad200640 1 -- 172.21.15.50:0/3003466524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a8154ed0 con 0x7f57a80b4de0 2024-09-21T05:59:39.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57ad200640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57a80b4de0 0x7f57a8154680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f579c0b0040 tx=0x7f579c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:39.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57a4c00640 1 -- 172.21.15.50:0/3003466524 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c0b2040 con 0x7f57a80b4de0 2024-09-21T05:59:39.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57a4c00640 1 -- 172.21.15.50:0/3003466524 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f579c003600 con 0x7f57a80b4de0 2024-09-21T05:59:39.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57afc00640 1 -- 172.21.15.50:0/3003466524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a81551b0 con 0x7f57a80b4de0 2024-09-21T05:59:39.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57a4c00640 1 -- 172.21.15.50:0/3003466524 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c003a50 con 0x7f57a80b4de0 2024-09-21T05:59:39.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57afc00640 1 -- 172.21.15.50:0/3003466524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a813e0f0 con 0x7f57a80b4de0 2024-09-21T05:59:39.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.383+0000 7f57afc00640 1 -- 172.21.15.50:0/3003466524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a813dee0 con 0x7f57a80b4de0 2024-09-21T05:59:39.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.387+0000 7f57a4c00640 1 -- 172.21.15.50:0/3003466524 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f579c003bf0 con 0x7f57a80b4de0 2024-09-21T05:59:39.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.387+0000 7f57a4c00640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5788073560 0x7f5788075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.387+0000 7f57a4c00640 1 -- 172.21.15.50:0/3003466524 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f579c12ea00 con 0x7f57a80b4de0 2024-09-21T05:59:39.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.387+0000 7f57a7e00640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5788073560 0x7f5788075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.387+0000 7f57a7e00640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5788073560 0x7f5788075a70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f57a80b4d80 tx=0x7f5794009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:39.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.391+0000 7f57a4c00640 1 -- 172.21.15.50:0/3003466524 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f579c0164d0 con 0x7f57a80b4de0 2024-09-21T05:59:39.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:39.607+0000 7f57afc00640 1 -- 172.21.15.50:0/3003466524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f57a815bb60 con 0x7f57a80b4de0 2024-09-21T05:59:40.407 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-09-21T05:59:40.403+0000 7fa50d400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:59:40.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.443+0000 7f57a4c00640 1 -- 172.21.15.50:0/3003466524 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f579c0f9010 con 0x7f57a80b4de0 2024-09-21T05:59:40.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.443+0000 7f57afc00640 1 -- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5788073560 msgr2=0x7f5788075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:40.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.443+0000 7f57afc00640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5788073560 0x7f5788075a70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f57a80b4d80 tx=0x7f5794009040 comp rx=0 tx=0).stop 2024-09-21T05:59:40.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.443+0000 7f57afc00640 1 -- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57a80b4de0 msgr2=0x7f57a8154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:40.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.443+0000 7f57afc00640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57a80b4de0 0x7f57a8154680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f579c0b0040 tx=0x7f579c0040c0 comp rx=0 tx=0).stop 2024-09-21T05:59:40.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.443+0000 7f57afc00640 1 -- 172.21.15.50:0/3003466524 shutdown_connections 2024-09-21T05:59:40.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.443+0000 7f57afc00640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57a81527b0 0x7f57a813d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:40.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.447+0000 7f57afc00640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5788073560 0x7f5788075a70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:40.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.447+0000 7f57afc00640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a80b5c70 0x7f57a8140b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:40.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.447+0000 7f57afc00640 1 --2- 172.21.15.50:0/3003466524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57a80b4de0 0x7f57a8154680 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:40.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.447+0000 7f57afc00640 1 -- 172.21.15.50:0/3003466524 >> 172.21.15.50:0/3003466524 conn(0x7f57a80b2040 msgr2=0x7f57a80ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:40.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.447+0000 7f57afc00640 1 -- 172.21.15.50:0/3003466524 shutdown_connections 2024-09-21T05:59:40.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.447+0000 7f57afc00640 1 -- 172.21.15.50:0/3003466524 wait complete. 2024-09-21T05:59:40.474 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T05:59:40.475 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T05:59:40.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.639+0000 7fec75200640 1 -- 172.21.15.50:0/997012701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec54004810 msgr2=0x7fec54004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:40.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.639+0000 7fec75200640 1 --2- 172.21.15.50:0/997012701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec54004810 0x7fec54004c20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c310 tx=0x7fec5801cf30 comp rx=0 tx=0).stop 2024-09-21T05:59:40.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.639+0000 7fec75200640 1 -- 172.21.15.50:0/997012701 shutdown_connections 2024-09-21T05:59:40.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.639+0000 7fec75200640 1 --2- 172.21.15.50:0/997012701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec54004810 0x7fec54004c20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:40.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.639+0000 7fec75200640 1 -- 172.21.15.50:0/997012701 >> 172.21.15.50:0/997012701 conn(0x7fec700b2040 msgr2=0x7fec700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:40.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.639+0000 7fec75200640 1 -- 172.21.15.50:0/997012701 shutdown_connections 2024-09-21T05:59:40.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.639+0000 7fec75200640 1 -- 172.21.15.50:0/997012701 wait complete. 2024-09-21T05:59:40.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec75200640 1 Processor -- start 2024-09-21T05:59:40.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec75200640 1 -- start start 2024-09-21T05:59:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec75200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec54004810 0x7fec7014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec75200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec700cfc20 0x7fec7014ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec75200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701fa530 0x7fec7015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec75200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fec700b55d0 con 0x7fec701fa530 2024-09-21T05:59:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec75200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec700b5450 con 0x7fec700cfc20 2024-09-21T05:59:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec75200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fec700b5750 con 0x7fec54004810 2024-09-21T05:59:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec6e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701fa530 0x7fec7015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec6e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701fa530 0x7fec7015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46556/0 (socket says 172.21.15.50:46556) 2024-09-21T05:59:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec6e800640 1 -- 172.21.15.50:0/668040536 learned_addr learned my addr 172.21.15.50:0/668040536 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec6de00640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec54004810 0x7fec7014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec67e00640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec700cfc20 0x7fec7014ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:40.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec6e800640 1 -- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec54004810 msgr2=0x7fec7014e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:40.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec6e800640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec54004810 0x7fec7014e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:40.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec6e800640 1 -- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec700cfc20 msgr2=0x7fec7014ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:40.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec6e800640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec700cfc20 0x7fec7014ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:40.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec6e800640 1 -- 172.21.15.50:0/668040536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7014ef20 con 0x7fec701fa530 2024-09-21T05:59:40.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec6e800640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701fa530 0x7fec7015d860 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fec70152bf0 tx=0x7fec680019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:40.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec66000640 1 -- 172.21.15.50:0/668040536 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68018070 con 0x7fec701fa530 2024-09-21T05:59:40.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec66000640 1 -- 172.21.15.50:0/668040536 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec68011960 con 0x7fec701fa530 2024-09-21T05:59:40.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec75200640 1 -- 172.21.15.50:0/668040536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7014e040 con 0x7fec701fa530 2024-09-21T05:59:40.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec66000640 1 -- 172.21.15.50:0/668040536 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68011d70 con 0x7fec701fa530 2024-09-21T05:59:40.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.643+0000 7fec75200640 1 -- 172.21.15.50:0/668040536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7014e600 con 0x7fec701fa530 2024-09-21T05:59:40.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.647+0000 7fec75200640 1 -- 172.21.15.50:0/668040536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7015f560 con 0x7fec701fa530 2024-09-21T05:59:40.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.647+0000 7fec66000640 1 -- 172.21.15.50:0/668040536 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fec6801b070 con 0x7fec701fa530 2024-09-21T05:59:40.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.647+0000 7fec66000640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec4c073590 0x7fec4c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:40.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.647+0000 7fec66000640 1 -- 172.21.15.50:0/668040536 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fec68094470 con 0x7fec701fa530 2024-09-21T05:59:40.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.647+0000 7fec6de00640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec4c073590 0x7fec4c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:40.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.651+0000 7fec6de00640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec4c073590 0x7fec4c075aa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec7006a420 tx=0x7fec600091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:40.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.651+0000 7fec66000640 1 -- 172.21.15.50:0/668040536 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fec6805e970 con 0x7fec701fa530 2024-09-21T05:59:40.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:40.867+0000 7fec75200640 1 -- 172.21.15.50:0/668040536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fec7014ddd0 con 0x7fec701fa530 2024-09-21T05:59:41.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.447+0000 7fec66000640 1 -- 172.21.15.50:0/668040536 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7fec680633a0 con 0x7fec701fa530 2024-09-21T05:59:41.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.447+0000 7fec75200640 1 -- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec4c073590 msgr2=0x7fec4c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:41.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.447+0000 7fec75200640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec4c073590 0x7fec4c075aa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec7006a420 tx=0x7fec600091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.447+0000 7fec75200640 1 -- 172.21.15.50:0/668040536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701fa530 msgr2=0x7fec7015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:41.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.447+0000 7fec75200640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701fa530 0x7fec7015d860 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fec70152bf0 tx=0x7fec680019c0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.447+0000 7fec75200640 1 -- 172.21.15.50:0/668040536 shutdown_connections 2024-09-21T05:59:41.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.447+0000 7fec75200640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec4c073590 0x7fec4c075aa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.447+0000 7fec75200640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701fa530 0x7fec7015d860 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.447+0000 7fec75200640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec700cfc20 0x7fec7014ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.447+0000 7fec75200640 1 --2- 172.21.15.50:0/668040536 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec54004810 0x7fec7014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.447+0000 7fec75200640 1 -- 172.21.15.50:0/668040536 >> 172.21.15.50:0/668040536 conn(0x7fec700b2040 msgr2=0x7fec7015a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:41.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.447+0000 7fec75200640 1 -- 172.21.15.50:0/668040536 shutdown_connections 2024-09-21T05:59:41.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.451+0000 7fec75200640 1 -- 172.21.15.50:0/668040536 wait complete. 2024-09-21T05:59:41.477 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T05:59:41.477 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:59:41.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.631+0000 7f147b400640 1 -- 172.21.15.50:0/234598959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14740b4de0 msgr2=0x7f14740b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:41.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.631+0000 7f147b400640 1 --2- 172.21.15.50:0/234598959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14740b4de0 0x7f14740b51f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1474070710 tx=0x7f146c01cae0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.635+0000 7f147b400640 1 -- 172.21.15.50:0/234598959 shutdown_connections 2024-09-21T05:59:41.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.635+0000 7f147b400640 1 --2- 172.21.15.50:0/234598959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14740b4de0 0x7f14740b51f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.635+0000 7f147b400640 1 --2- 172.21.15.50:0/234598959 >> v2:172.21.15.88:3300/0 conn(0x7f14741527b0 0x7f147415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.635+0000 7f147b400640 1 -- 172.21.15.50:0/234598959 >> 172.21.15.50:0/234598959 conn(0x7f14740b2040 msgr2=0x7f14740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:41.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.635+0000 7f147b400640 1 -- 172.21.15.50:0/234598959 shutdown_connections 2024-09-21T05:59:41.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.635+0000 7f147b400640 1 -- 172.21.15.50:0/234598959 wait complete. 2024-09-21T05:59:41.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.635+0000 7f147b400640 1 Processor -- start 2024-09-21T05:59:41.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.635+0000 7f147b400640 1 -- start start 2024-09-21T05:59:41.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.635+0000 7f147b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14740b4de0 0x7f14741fc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:41.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f147b400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f14740b5c70 0x7f14741f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:41.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f147b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14741527b0 0x7f147414bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:41.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f147b400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1474168970 con 0x7f14740b4de0 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f147b400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14741687f0 con 0x7f14741527b0 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f147b400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1474168af0 con 0x7f14740b5c70 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1478a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14740b4de0 0x7f14741fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1478a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14740b4de0 0x7f14741fc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46582/0 (socket says 172.21.15.50:46582) 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1478a00640 1 -- 172.21.15.50:0/480580610 learned_addr learned my addr 172.21.15.50:0/480580610 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1479400640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14741527b0 0x7f147414bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1473e00640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f14740b5c70 0x7f14741f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1478a00640 1 -- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f14740b5c70 msgr2=0x7f14741f95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1478a00640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f14740b5c70 0x7f14741f95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1478a00640 1 -- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14741527b0 msgr2=0x7f147414bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1478a00640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14741527b0 0x7f147414bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1478a00640 1 -- 172.21.15.50:0/480580610 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14741f9b20 con 0x7f14740b4de0 2024-09-21T05:59:41.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1479400640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14741527b0 0x7f147414bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:41.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1478a00640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14740b4de0 0x7f14741fc230 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f146800b410 tx=0x7f146800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:41.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1470c00640 1 -- 172.21.15.50:0/480580610 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468015030 con 0x7f14740b4de0 2024-09-21T05:59:41.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1470c00640 1 -- 172.21.15.50:0/480580610 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1468004020 con 0x7f14740b4de0 2024-09-21T05:59:41.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1470c00640 1 -- 172.21.15.50:0/480580610 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468004470 con 0x7f14740b4de0 2024-09-21T05:59:41.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f147b400640 1 -- 172.21.15.50:0/480580610 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14741fcb30 con 0x7f14740b4de0 2024-09-21T05:59:41.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f147b400640 1 -- 172.21.15.50:0/480580610 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14741f8de0 con 0x7f14740b4de0 2024-09-21T05:59:41.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f147b400640 1 -- 172.21.15.50:0/480580610 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14740b51f0 con 0x7f14740b4de0 2024-09-21T05:59:41.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.639+0000 7f1470c00640 1 -- 172.21.15.50:0/480580610 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f1468018070 con 0x7f14740b4de0 2024-09-21T05:59:41.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.643+0000 7f1470c00640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1454073560 0x7f1454075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:41.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.643+0000 7f1470c00640 1 -- 172.21.15.50:0/480580610 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f1468094710 con 0x7f14740b4de0 2024-09-21T05:59:41.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.643+0000 7f1473e00640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1454073560 0x7f1454075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:41.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.643+0000 7f1473e00640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1454073560 0x7f1454075a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14600058d0 tx=0x7f1460005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:41.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.647+0000 7f1470c00640 1 -- 172.21.15.50:0/480580610 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f146805f390 con 0x7f14740b4de0 2024-09-21T05:59:41.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.855+0000 7f147b400640 1 -- 172.21.15.50:0/480580610 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14740cc710 con 0x7f14740b4de0 2024-09-21T05:59:41.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.855+0000 7f1470c00640 1 -- 172.21.15.50:0/480580610 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5217 (secure 0 0 0) 0x7f146805ec90 con 0x7f14740b4de0 2024-09-21T05:59:41.860 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:41.860 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"btime":"2024-09-21T05:59:41:444356+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/723044219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":723044219},{"type":"v1","addr":"172.21.15.88:6836","nonce":723044219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1966998855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1966998855},{"type":"v1","addr":"172.21.15.88:6837","nonce":1966998855}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2536951841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2536951841},{"type":"v1","addr":"172.21.15.50:6834","nonce":2536951841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:59:37.853304+0000","modified":"2024-09-21T05:59:40.872885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5370":{"gid":5370,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/988935074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":988935074},{"type":"v1","addr":"172.21.15.50:6835","nonce":988935074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5370,"qdb_cluster":[5370]},"id":3}]} 2024-09-21T05:59:41.860 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 27 2024-09-21T05:59:41.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.859+0000 7f147b400640 1 -- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1454073560 msgr2=0x7f1454075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:41.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.859+0000 7f147b400640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1454073560 0x7f1454075a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14600058d0 tx=0x7f1460005860 comp rx=0 tx=0).stop 2024-09-21T05:59:41.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.859+0000 7f147b400640 1 -- 172.21.15.50:0/480580610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14740b4de0 msgr2=0x7f14741fc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:41.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.859+0000 7f147b400640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14740b4de0 0x7f14741fc230 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f146800b410 tx=0x7f146800b8d0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.859+0000 7f147b400640 1 -- 172.21.15.50:0/480580610 shutdown_connections 2024-09-21T05:59:41.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.859+0000 7f147b400640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14741527b0 0x7f147414bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.859+0000 7f147b400640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f14740b5c70 0x7f14741f95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.859+0000 7f147b400640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1454073560 0x7f1454075a70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.859+0000 7f147b400640 1 --2- 172.21.15.50:0/480580610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14740b4de0 0x7f14741fc230 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:41.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.859+0000 7f147b400640 1 -- 172.21.15.50:0/480580610 >> 172.21.15.50:0/480580610 conn(0x7f14740b2040 msgr2=0x7f14741511b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:41.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.859+0000 7f147b400640 1 -- 172.21.15.50:0/480580610 shutdown_connections 2024-09-21T05:59:41.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:41.859+0000 7f147b400640 1 -- 172.21.15.50:0/480580610 wait complete. 2024-09-21T05:59:41.883 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:59:42.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.039+0000 7fbda3800640 1 -- 172.21.15.50:0/1492124581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd9c0b54f0 msgr2=0x7fbd9c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.039+0000 7fbda3800640 1 --2- 172.21.15.50:0/1492124581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd9c0b54f0 0x7fbd9c15bb60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009bc0 tx=0x7fbd9401cae0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.039+0000 7fbda3800640 1 -- 172.21.15.50:0/1492124581 shutdown_connections 2024-09-21T05:59:42.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.039+0000 7fbda3800640 1 --2- 172.21.15.50:0/1492124581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd9c0b54f0 0x7fbd9c15bb60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.039+0000 7fbda3800640 1 -- 172.21.15.50:0/1492124581 >> 172.21.15.50:0/1492124581 conn(0x7fbd9c0b2040 msgr2=0x7fbd9c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:42.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.039+0000 7fbda3800640 1 -- 172.21.15.50:0/1492124581 shutdown_connections 2024-09-21T05:59:42.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.039+0000 7fbda3800640 1 -- 172.21.15.50:0/1492124581 wait complete. 2024-09-21T05:59:42.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.039+0000 7fbda3800640 1 Processor -- start 2024-09-21T05:59:42.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.039+0000 7fbda3800640 1 -- start start 2024-09-21T05:59:42.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda3800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbd9c0b5c70 0x7fbd9c14c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:42.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda3800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd9c1527b0 0x7fbd9c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:42.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda3800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd9c14c860 0x7fbd9c0aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:42.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda3800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c169b30 con 0x7fbd9c1527b0 2024-09-21T05:59:42.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda3800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1699b0 con 0x7fbd9c14c860 2024-09-21T05:59:42.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda3800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c169cb0 con 0x7fbd9c0b5c70 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd9be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd9c1527b0 0x7fbd9c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd9be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd9c1527b0 0x7fbd9c0ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46600/0 (socket says 172.21.15.50:46600) 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd9be00640 1 -- 172.21.15.50:0/2680063942 learned_addr learned my addr 172.21.15.50:0/2680063942 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd9be00640 1 -- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbd9c0b5c70 msgr2=0x7fbd9c14c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda1800640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd9c14c860 0x7fbd9c0aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda0e00640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbd9c0b5c70 0x7fbd9c14c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd9be00640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbd9c0b5c70 0x7fbd9c14c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd9be00640 1 -- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd9c14c860 msgr2=0x7fbd9c0aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd9be00640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd9c14c860 0x7fbd9c0aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd9be00640 1 -- 172.21.15.50:0/2680063942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c14eca0 con 0x7fbd9c1527b0 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda1800640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd9c14c860 0x7fbd9c0aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda0e00640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbd9c0b5c70 0x7fbd9c14c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd9be00640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd9c1527b0 0x7fbd9c0ae8e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800c400 tx=0x7fbd8800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:42.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd98c00640 1 -- 172.21.15.50:0/2680063942 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88017070 con 0x7fbd9c1527b0 2024-09-21T05:59:42.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd98c00640 1 -- 172.21.15.50:0/2680063942 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd88003600 con 0x7fbd9c1527b0 2024-09-21T05:59:42.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd98c00640 1 -- 172.21.15.50:0/2680063942 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88003a50 con 0x7fbd9c1527b0 2024-09-21T05:59:42.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda3800640 1 -- 172.21.15.50:0/2680063942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c14ef80 con 0x7fbd9c1527b0 2024-09-21T05:59:42.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda3800640 1 -- 172.21.15.50:0/2680063942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c14f540 con 0x7fbd9c1527b0 2024-09-21T05:59:42.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbda3800640 1 -- 172.21.15.50:0/2680063942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9c14ea10 con 0x7fbd9c1527b0 2024-09-21T05:59:42.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.043+0000 7fbd98c00640 1 -- 172.21.15.50:0/2680063942 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fbd8801a070 con 0x7fbd9c1527b0 2024-09-21T05:59:42.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.047+0000 7fbd98c00640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbd7c073560 0x7fbd7c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:42.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.047+0000 7fbd98c00640 1 -- 172.21.15.50:0/2680063942 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fbd88093a80 con 0x7fbd9c1527b0 2024-09-21T05:59:42.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.047+0000 7fbda0e00640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbd7c073560 0x7fbd7c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:42.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.047+0000 7fbda0e00640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbd7c073560 0x7fbd7c075a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000a850 tx=0x7fbd90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:42.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.051+0000 7fbd98c00640 1 -- 172.21.15.50:0/2680063942 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbd88095e20 con 0x7fbd9c1527b0 2024-09-21T05:59:42.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.227+0000 7fbda3800640 1 -- 172.21.15.50:0/2680063942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd9c15bb60 con 0x7fbd9c1527b0 2024-09-21T05:59:42.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbd98c00640 1 -- 172.21.15.50:0/2680063942 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16625 (secure 0 0 0) 0x7fbd8805e660 con 0x7fbd9c1527b0 2024-09-21T05:59:42.234 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:42.235 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":44,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T05:59:39.431486+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:59:35.596192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T05:59:36.818395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:59:42.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbda3800640 1 -- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbd7c073560 msgr2=0x7fbd7c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbda3800640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbd7c073560 0x7fbd7c075a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000a850 tx=0x7fbd90008040 comp rx=0 tx=0).stop 2024-09-21T05:59:42.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbda3800640 1 -- 172.21.15.50:0/2680063942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd9c1527b0 msgr2=0x7fbd9c0ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbda3800640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd9c1527b0 0x7fbd9c0ae8e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800c400 tx=0x7fbd8800c8c0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbda3800640 1 -- 172.21.15.50:0/2680063942 shutdown_connections 2024-09-21T05:59:42.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbda3800640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd9c14c860 0x7fbd9c0aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbda3800640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbd7c073560 0x7fbd7c075a70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbda3800640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd9c1527b0 0x7fbd9c0ae8e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbda3800640 1 --2- 172.21.15.50:0/2680063942 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbd9c0b5c70 0x7fbd9c14c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbda3800640 1 -- 172.21.15.50:0/2680063942 >> 172.21.15.50:0/2680063942 conn(0x7fbd9c0b2040 msgr2=0x7fbd9c0b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:42.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbda3800640 1 -- 172.21.15.50:0/2680063942 shutdown_connections 2024-09-21T05:59:42.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.231+0000 7fbda3800640 1 -- 172.21.15.50:0/2680063942 wait complete. 2024-09-21T05:59:42.258 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:59:42.258 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:59:42.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.411+0000 7f76b3000640 1 -- 172.21.15.50:0/3061796557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76ac0b24d0 msgr2=0x7f76ac0b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.411+0000 7f76b3000640 1 --2- 172.21.15.50:0/3061796557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76ac0b24d0 0x7f76ac0b3130 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009c80 tx=0x7f76a401cca0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.411+0000 7f76b3000640 1 -- 172.21.15.50:0/3061796557 shutdown_connections 2024-09-21T05:59:42.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.411+0000 7f76b3000640 1 --2- 172.21.15.50:0/3061796557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76ac0b24d0 0x7f76ac0b3130 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.411+0000 7f76b3000640 1 --2- 172.21.15.50:0/3061796557 >> v2:172.21.15.88:3300/0 conn(0x7f76ac155a00 0x7f76ac15f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.411+0000 7f76b3000640 1 -- 172.21.15.50:0/3061796557 >> 172.21.15.50:0/3061796557 conn(0x7f76ac06cdf0 msgr2=0x7f76ac06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:42.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.411+0000 7f76b3000640 1 -- 172.21.15.50:0/3061796557 shutdown_connections 2024-09-21T05:59:42.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.411+0000 7f76b3000640 1 -- 172.21.15.50:0/3061796557 wait complete. 2024-09-21T05:59:42.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b3000640 1 Processor -- start 2024-09-21T05:59:42.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b3000640 1 -- start start 2024-09-21T05:59:42.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b3000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f76ac155a00 0x7f76ac1365a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:42.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b3000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76ac15f5b0 0x7f76ac1fc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:42.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b3000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76ac073500 0x7f76ac1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:42.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b3000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac168a20 con 0x7f76ac15f5b0 2024-09-21T05:59:42.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b3000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac1688a0 con 0x7f76ac073500 2024-09-21T05:59:42.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b3000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac168ba0 con 0x7f76ac155a00 2024-09-21T05:59:42.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76ac15f5b0 0x7f76ac1fc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:42.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76ac15f5b0 0x7f76ac1fc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46620/0 (socket says 172.21.15.50:46620) 2024-09-21T05:59:42.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b1200640 1 -- 172.21.15.50:0/3465069209 learned_addr learned my addr 172.21.15.50:0/3465069209 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:42.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b1c00640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f76ac155a00 0x7f76ac1365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:42.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.415+0000 7f76b2600640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76ac073500 0x7f76ac1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:42.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76b1200640 1 -- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f76ac155a00 msgr2=0x7f76ac1365a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76b1200640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f76ac155a00 0x7f76ac1365a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76b1200640 1 -- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76ac073500 msgr2=0x7f76ac1534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76b1200640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76ac073500 0x7f76ac1534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76b1200640 1 -- 172.21.15.50:0/3465069209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac151870 con 0x7f76ac15f5b0 2024-09-21T05:59:42.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76b1200640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76ac15f5b0 0x7f76ac1fc3f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f769800c3c0 tx=0x7f769800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:42.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76a9600640 1 -- 172.21.15.50:0/3465069209 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698017070 con 0x7f76ac15f5b0 2024-09-21T05:59:42.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76a9600640 1 -- 172.21.15.50:0/3465069209 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76980053e0 con 0x7f76ac15f5b0 2024-09-21T05:59:42.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76a9600640 1 -- 172.21.15.50:0/3465069209 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698003860 con 0x7f76ac15f5b0 2024-09-21T05:59:42.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76b3000640 1 -- 172.21.15.50:0/3465069209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac151b50 con 0x7f76ac15f5b0 2024-09-21T05:59:42.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76b3000640 1 -- 172.21.15.50:0/3465069209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac1f8500 con 0x7f76ac15f5b0 2024-09-21T05:59:42.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76b3000640 1 -- 172.21.15.50:0/3465069209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ac154ab0 con 0x7f76ac15f5b0 2024-09-21T05:59:42.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.419+0000 7f76a9600640 1 -- 172.21.15.50:0/3465069209 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f769801a070 con 0x7f76ac15f5b0 2024-09-21T05:59:42.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.423+0000 7f76a9600640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f768c073560 0x7f768c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:42.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.423+0000 7f76a9600640 1 -- 172.21.15.50:0/3465069209 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f7698093ad0 con 0x7f76ac15f5b0 2024-09-21T05:59:42.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.423+0000 7f76b1c00640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f768c073560 0x7f768c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:42.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.423+0000 7f76b1c00640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f768c073560 0x7f768c075a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f76a000bb50 tx=0x7f76a000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:42.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.427+0000 7f76a9600640 1 -- 172.21.15.50:0/3465069209 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f76ac154ab0 con 0x7f76ac15f5b0 2024-09-21T05:59:42.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.599+0000 7f76b3000640 1 -- 172.21.15.50:0/3465069209 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76ac06b540 con 0x7f768c073560 2024-09-21T05:59:42.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.607+0000 7f76a9600640 1 -- 172.21.15.50:0/3465069209 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246096 (secure 0 0 0) 0x7f76a0015070 con 0x7f768c073560 2024-09-21T05:59:42.612 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:42.613 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:59:42.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.607+0000 7f76b3000640 1 -- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f768c073560 msgr2=0x7f768c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.607+0000 7f76b3000640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f768c073560 0x7f768c075a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f76a000bb50 tx=0x7f76a000b2e0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.607+0000 7f76b3000640 1 -- 172.21.15.50:0/3465069209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76ac15f5b0 msgr2=0x7f76ac1fc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.607+0000 7f76b3000640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76ac15f5b0 0x7f76ac1fc3f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f769800c3c0 tx=0x7f769800c880 comp rx=0 tx=0).stop 2024-09-21T05:59:42.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.607+0000 7f76b3000640 1 -- 172.21.15.50:0/3465069209 shutdown_connections 2024-09-21T05:59:42.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.607+0000 7f76b3000640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76ac073500 0x7f76ac1534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.607+0000 7f76b3000640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f768c073560 0x7f768c075a70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.607+0000 7f76b3000640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76ac15f5b0 0x7f76ac1fc3f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.607+0000 7f76b3000640 1 --2- 172.21.15.50:0/3465069209 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f76ac155a00 0x7f76ac1365a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.607+0000 7f76b3000640 1 -- 172.21.15.50:0/3465069209 >> 172.21.15.50:0/3465069209 conn(0x7f76ac06cdf0 msgr2=0x7f76ac15d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:42.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.611+0000 7f76b3000640 1 -- 172.21.15.50:0/3465069209 shutdown_connections 2024-09-21T05:59:42.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.611+0000 7f76b3000640 1 -- 172.21.15.50:0/3465069209 wait complete. 2024-09-21T05:59:42.634 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-21T05:59:40.934133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":188,"acting":188,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271092,"kb_used_data":4912,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754703628,"statfs":{"total":773094113280,"available":772816515072,"internally_reserved":0,"allocated":5029888,"data_stored":2684946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.521851"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430439+0000","last_change":"2024-09-21T05:59:37.430439+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430439+0000","last_undegraded":"2024-09-21T05:59:37.430439+0000","last_fullsized":"2024-09-21T05:59:37.430439+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424624+0000","last_change":"2024-09-21T05:59:37.424624+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424624+0000","last_undegraded":"2024-09-21T05:59:37.424624+0000","last_fullsized":"2024-09-21T05:59:37.424624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424101+0000","last_change":"2024-09-21T05:59:37.424101+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424101+0000","last_undegraded":"2024-09-21T05:59:37.424101+0000","last_fullsized":"2024-09-21T05:59:37.424101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431709+0000","last_change":"2024-09-21T05:59:37.431709+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431709+0000","last_undegraded":"2024-09-21T05:59:37.431709+0000","last_fullsized":"2024-09-21T05:59:37.431709+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423190+0000","last_change":"2024-09-21T05:59:37.423190+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423190+0000","last_undegraded":"2024-09-21T05:59:37.423190+0000","last_fullsized":"2024-09-21T05:59:37.423190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.428041+0000","last_change":"2024-09-21T05:59:37.428041+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.428041+0000","last_undegraded":"2024-09-21T05:59:37.428041+0000","last_fullsized":"2024-09-21T05:59:37.428041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430128+0000","last_change":"2024-09-21T05:59:37.430128+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430128+0000","last_undegraded":"2024-09-21T05:59:37.430128+0000","last_fullsized":"2024-09-21T05:59:37.430128+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433589+0000","last_change":"2024-09-21T05:59:37.433589+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433589+0000","last_undegraded":"2024-09-21T05:59:37.433589+0000","last_fullsized":"2024-09-21T05:59:37.433589+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430329+0000","last_change":"2024-09-21T05:59:37.430329+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430329+0000","last_undegraded":"2024-09-21T05:59:37.430329+0000","last_fullsized":"2024-09-21T05:59:37.430329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431028+0000","last_change":"2024-09-21T05:59:37.431028+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431028+0000","last_undegraded":"2024-09-21T05:59:37.431028+0000","last_fullsized":"2024-09-21T05:59:37.431028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429854+0000","last_change":"2024-09-21T05:59:37.429854+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429854+0000","last_undegraded":"2024-09-21T05:59:37.429854+0000","last_fullsized":"2024-09-21T05:59:37.429854+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423767+0000","last_change":"2024-09-21T05:59:37.423767+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423767+0000","last_undegraded":"2024-09-21T05:59:37.423767+0000","last_fullsized":"2024-09-21T05:59:37.423767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.435082+0000","last_change":"2024-09-21T05:59:37.435082+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.435082+0000","last_undegraded":"2024-09-21T05:59:37.435082+0000","last_fullsized":"2024-09-21T05:59:37.435082+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432295+0000","last_change":"2024-09-21T05:59:37.432295+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432295+0000","last_undegraded":"2024-09-21T05:59:37.432295+0000","last_fullsized":"2024-09-21T05:59:37.432295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.427649+0000","last_change":"2024-09-21T05:59:37.427649+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.427649+0000","last_undegraded":"2024-09-21T05:59:37.427649+0000","last_fullsized":"2024-09-21T05:59:37.427649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430585+0000","last_change":"2024-09-21T05:59:37.430585+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430585+0000","last_undegraded":"2024-09-21T05:59:37.430585+0000","last_fullsized":"2024-09-21T05:59:37.430585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424708+0000","last_change":"2024-09-21T05:59:37.424708+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424708+0000","last_undegraded":"2024-09-21T05:59:37.424708+0000","last_fullsized":"2024-09-21T05:59:37.424708+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425629+0000","last_change":"2024-09-21T05:59:37.425629+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425629+0000","last_undegraded":"2024-09-21T05:59:37.425629+0000","last_fullsized":"2024-09-21T05:59:37.425629+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432796+0000","last_change":"2024-09-21T05:59:37.432796+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432796+0000","last_undegraded":"2024-09-21T05:59:37.432796+0000","last_fullsized":"2024-09-21T05:59:37.432796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.434597+0000","last_change":"2024-09-21T05:59:37.434597+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.434597+0000","last_undegraded":"2024-09-21T05:59:37.434597+0000","last_fullsized":"2024-09-21T05:59:37.434597+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.427426+0000","last_change":"2024-09-21T05:59:37.427426+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.427426+0000","last_undegraded":"2024-09-21T05:59:37.427426+0000","last_fullsized":"2024-09-21T05:59:37.427426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429310+0000","last_change":"2024-09-21T05:59:37.429310+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429310+0000","last_undegraded":"2024-09-21T05:59:37.429310+0000","last_fullsized":"2024-09-21T05:59:37.429310+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.434559+0000","last_change":"2024-09-21T05:59:37.434559+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.434559+0000","last_undegraded":"2024-09-21T05:59:37.434559+0000","last_fullsized":"2024-09-21T05:59:37.434559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429894+0000","last_change":"2024-09-21T05:59:37.429894+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429894+0000","last_undegraded":"2024-09-21T05:59:37.429894+0000","last_fullsized":"2024-09-21T05:59:37.429894+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433207+0000","last_change":"2024-09-21T05:59:37.433207+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433207+0000","last_undegraded":"2024-09-21T05:59:37.433207+0000","last_fullsized":"2024-09-21T05:59:37.433207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.439276+0000","last_change":"2024-09-21T05:59:37.439276+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.439276+0000","last_undegraded":"2024-09-21T05:59:37.439276+0000","last_fullsized":"2024-09-21T05:59:37.439276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423541+0000","last_change":"2024-09-21T05:59:37.423541+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423541+0000","last_undegraded":"2024-09-21T05:59:37.423541+0000","last_fullsized":"2024-09-21T05:59:37.423541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423216+0000","last_change":"2024-09-21T05:59:37.423216+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423216+0000","last_undegraded":"2024-09-21T05:59:37.423216+0000","last_fullsized":"2024-09-21T05:59:37.423216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431328+0000","last_change":"2024-09-21T05:59:37.431328+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431328+0000","last_undegraded":"2024-09-21T05:59:37.431328+0000","last_fullsized":"2024-09-21T05:59:37.431328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423470+0000","last_change":"2024-09-21T05:59:37.423470+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423470+0000","last_undegraded":"2024-09-21T05:59:37.423470+0000","last_fullsized":"2024-09-21T05:59:37.423470+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.426152+0000","last_change":"2024-09-21T05:59:37.426152+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.426152+0000","last_undegraded":"2024-09-21T05:59:37.426152+0000","last_fullsized":"2024-09-21T05:59:37.426152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.438506+0000","last_change":"2024-09-21T05:59:37.438506+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.438506+0000","last_undegraded":"2024-09-21T05:59:37.438506+0000","last_fullsized":"2024-09-21T05:59:37.438506+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:59:37.411723+0000","last_change":"2024-09-21T05:59:37.411723+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.411723+0000","last_undegraded":"2024-09-21T05:59:37.411723+0000","last_fullsized":"2024-09-21T05:59:37.411723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451150+0000","last_change":"2024-09-21T05:59:37.447751+0000","last_active":"2024-09-21T05:59:37.451150+0000","last_peered":"2024-09-21T05:59:37.451150+0000","last_clean":"2024-09-21T05:59:37.451150+0000","last_became_active":"2024-09-21T05:59:37.447392+0000","last_became_peered":"2024-09-21T05:59:37.447392+0000","last_unstale":"2024-09-21T05:59:37.451150+0000","last_undegraded":"2024-09-21T05:59:37.451150+0000","last_fullsized":"2024-09-21T05:59:37.451150+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:17.297839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432540+0000","last_change":"2024-09-21T05:59:37.432540+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432540+0000","last_undegraded":"2024-09-21T05:59:37.432540+0000","last_fullsized":"2024-09-21T05:59:37.432540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.444761+0000","last_change":"2024-09-21T05:59:37.443312+0000","last_active":"2024-09-21T05:59:37.444761+0000","last_peered":"2024-09-21T05:59:37.444761+0000","last_clean":"2024-09-21T05:59:37.444761+0000","last_became_active":"2024-09-21T05:59:37.442457+0000","last_became_peered":"2024-09-21T05:59:37.442457+0000","last_unstale":"2024-09-21T05:59:37.444761+0000","last_undegraded":"2024-09-21T05:59:37.444761+0000","last_fullsized":"2024-09-21T05:59:37.444761+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:54.217575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.422517+0000","last_change":"2024-09-21T05:59:37.422517+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.422517+0000","last_undegraded":"2024-09-21T05:59:37.422517+0000","last_fullsized":"2024-09-21T05:59:37.422517+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-21T05:59:36.420517+0000","last_change":"2024-09-21T05:59:36.420517+0000","last_active":"2024-09-21T05:59:36.408615+0000","last_peered":"2024-09-21T05:59:36.408615+0000","last_clean":"2024-09-21T05:59:36.408615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:36.420517+0000","last_undegraded":"2024-09-21T05:59:36.420517+0000","last_fullsized":"2024-09-21T05:59:36.420517+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:59:37.411723+0000","last_change":"2024-09-21T05:59:37.411723+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.411723+0000","last_undegraded":"2024-09-21T05:59:37.411723+0000","last_fullsized":"2024-09-21T05:59:37.411723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.449829+0000","last_change":"2024-09-21T05:59:37.443607+0000","last_active":"2024-09-21T05:59:37.449829+0000","last_peered":"2024-09-21T05:59:37.449829+0000","last_clean":"2024-09-21T05:59:37.449829+0000","last_became_active":"2024-09-21T05:59:37.443262+0000","last_became_peered":"2024-09-21T05:59:37.443262+0000","last_unstale":"2024-09-21T05:59:37.449829+0000","last_undegraded":"2024-09-21T05:59:37.449829+0000","last_fullsized":"2024-09-21T05:59:37.449829+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:21.369884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.422840+0000","last_change":"2024-09-21T05:59:37.422840+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.422840+0000","last_undegraded":"2024-09-21T05:59:37.422840+0000","last_fullsized":"2024-09-21T05:59:37.422840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451244+0000","last_change":"2024-09-21T05:59:37.447920+0000","last_active":"2024-09-21T05:59:37.451244+0000","last_peered":"2024-09-21T05:59:37.451244+0000","last_clean":"2024-09-21T05:59:37.451244+0000","last_became_active":"2024-09-21T05:59:37.447240+0000","last_became_peered":"2024-09-21T05:59:37.447240+0000","last_unstale":"2024-09-21T05:59:37.451244+0000","last_undegraded":"2024-09-21T05:59:37.451244+0000","last_fullsized":"2024-09-21T05:59:37.451244+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:52.027735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432049+0000","last_change":"2024-09-21T05:59:37.432049+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432049+0000","last_undegraded":"2024-09-21T05:59:37.432049+0000","last_fullsized":"2024-09-21T05:59:37.432049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.438146+0000","last_change":"2024-09-21T05:59:37.429073+0000","last_active":"2024-09-21T05:59:37.438146+0000","last_peered":"2024-09-21T05:59:37.438146+0000","last_clean":"2024-09-21T05:59:37.438146+0000","last_became_active":"2024-09-21T05:59:37.428733+0000","last_became_peered":"2024-09-21T05:59:37.428733+0000","last_unstale":"2024-09-21T05:59:37.438146+0000","last_undegraded":"2024-09-21T05:59:37.438146+0000","last_fullsized":"2024-09-21T05:59:37.438146+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:39.275851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433120+0000","last_change":"2024-09-21T05:59:37.433120+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433120+0000","last_undegraded":"2024-09-21T05:59:37.433120+0000","last_fullsized":"2024-09-21T05:59:37.433120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.440195+0000","last_change":"2024-09-21T05:59:37.437489+0000","last_active":"2024-09-21T05:59:37.440195+0000","last_peered":"2024-09-21T05:59:37.440195+0000","last_clean":"2024-09-21T05:59:37.440195+0000","last_became_active":"2024-09-21T05:59:37.437170+0000","last_became_peered":"2024-09-21T05:59:37.437170+0000","last_unstale":"2024-09-21T05:59:37.440195+0000","last_undegraded":"2024-09-21T05:59:37.440195+0000","last_fullsized":"2024-09-21T05:59:37.440195+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:42.959574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.439321+0000","last_change":"2024-09-21T05:59:37.439321+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.439321+0000","last_undegraded":"2024-09-21T05:59:37.439321+0000","last_fullsized":"2024-09-21T05:59:37.439321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451906+0000","last_change":"2024-09-21T05:59:37.449577+0000","last_active":"2024-09-21T05:59:37.451906+0000","last_peered":"2024-09-21T05:59:37.451906+0000","last_clean":"2024-09-21T05:59:37.451906+0000","last_became_active":"2024-09-21T05:59:37.449255+0000","last_became_peered":"2024-09-21T05:59:37.449255+0000","last_unstale":"2024-09-21T05:59:37.451906+0000","last_undegraded":"2024-09-21T05:59:37.451906+0000","last_fullsized":"2024-09-21T05:59:37.451906+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:39.275851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425083+0000","last_change":"2024-09-21T05:59:37.425083+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425083+0000","last_undegraded":"2024-09-21T05:59:37.425083+0000","last_fullsized":"2024-09-21T05:59:37.425083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.444043+0000","last_change":"2024-09-21T05:59:37.440683+0000","last_active":"2024-09-21T05:59:37.444043+0000","last_peered":"2024-09-21T05:59:37.444043+0000","last_clean":"2024-09-21T05:59:37.444043+0000","last_became_active":"2024-09-21T05:59:37.440336+0000","last_became_peered":"2024-09-21T05:59:37.440336+0000","last_unstale":"2024-09-21T05:59:37.444043+0000","last_undegraded":"2024-09-21T05:59:37.444043+0000","last_fullsized":"2024-09-21T05:59:37.444043+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:13.571664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.428994+0000","last_change":"2024-09-21T05:59:37.428994+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.428994+0000","last_undegraded":"2024-09-21T05:59:37.428994+0000","last_fullsized":"2024-09-21T05:59:37.428994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451867+0000","last_change":"2024-09-21T05:59:37.448714+0000","last_active":"2024-09-21T05:59:37.451867+0000","last_peered":"2024-09-21T05:59:37.451867+0000","last_clean":"2024-09-21T05:59:37.451867+0000","last_became_active":"2024-09-21T05:59:37.448310+0000","last_became_peered":"2024-09-21T05:59:37.448310+0000","last_unstale":"2024-09-21T05:59:37.451867+0000","last_undegraded":"2024-09-21T05:59:37.451867+0000","last_fullsized":"2024-09-21T05:59:37.451867+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:21.097323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.426543+0000","last_change":"2024-09-21T05:59:37.426543+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.426543+0000","last_undegraded":"2024-09-21T05:59:37.426543+0000","last_fullsized":"2024-09-21T05:59:37.426543+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450006+0000","last_change":"2024-09-21T05:59:37.445812+0000","last_active":"2024-09-21T05:59:37.450006+0000","last_peered":"2024-09-21T05:59:37.450006+0000","last_clean":"2024-09-21T05:59:37.450006+0000","last_became_active":"2024-09-21T05:59:37.445496+0000","last_became_peered":"2024-09-21T05:59:37.445496+0000","last_unstale":"2024-09-21T05:59:37.450006+0000","last_undegraded":"2024-09-21T05:59:37.450006+0000","last_fullsized":"2024-09-21T05:59:37.450006+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:46.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429754+0000","last_change":"2024-09-21T05:59:37.429754+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429754+0000","last_undegraded":"2024-09-21T05:59:37.429754+0000","last_fullsized":"2024-09-21T05:59:37.429754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.443714+0000","last_change":"2024-09-21T05:59:37.440777+0000","last_active":"2024-09-21T05:59:37.443714+0000","last_peered":"2024-09-21T05:59:37.443714+0000","last_clean":"2024-09-21T05:59:37.443714+0000","last_became_active":"2024-09-21T05:59:37.440460+0000","last_became_peered":"2024-09-21T05:59:37.440460+0000","last_unstale":"2024-09-21T05:59:37.443714+0000","last_undegraded":"2024-09-21T05:59:37.443714+0000","last_fullsized":"2024-09-21T05:59:37.443714+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:03.466004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430964+0000","last_change":"2024-09-21T05:59:37.430964+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430964+0000","last_undegraded":"2024-09-21T05:59:37.430964+0000","last_fullsized":"2024-09-21T05:59:37.430964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451384+0000","last_change":"2024-09-21T05:59:37.447984+0000","last_active":"2024-09-21T05:59:37.451384+0000","last_peered":"2024-09-21T05:59:37.451384+0000","last_clean":"2024-09-21T05:59:37.451384+0000","last_became_active":"2024-09-21T05:59:37.447685+0000","last_became_peered":"2024-09-21T05:59:37.447685+0000","last_unstale":"2024-09-21T05:59:37.451384+0000","last_undegraded":"2024-09-21T05:59:37.451384+0000","last_fullsized":"2024-09-21T05:59:37.451384+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:36.267884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431534+0000","last_change":"2024-09-21T05:59:37.431534+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431534+0000","last_undegraded":"2024-09-21T05:59:37.431534+0000","last_fullsized":"2024-09-21T05:59:37.431534+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.449770+0000","last_change":"2024-09-21T05:59:37.443723+0000","last_active":"2024-09-21T05:59:37.449770+0000","last_peered":"2024-09-21T05:59:37.449770+0000","last_clean":"2024-09-21T05:59:37.449770+0000","last_became_active":"2024-09-21T05:59:37.443383+0000","last_became_peered":"2024-09-21T05:59:37.443383+0000","last_unstale":"2024-09-21T05:59:37.449770+0000","last_undegraded":"2024-09-21T05:59:37.449770+0000","last_fullsized":"2024-09-21T05:59:37.449770+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.689460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.426456+0000","last_change":"2024-09-21T05:59:37.426456+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.426456+0000","last_undegraded":"2024-09-21T05:59:37.426456+0000","last_fullsized":"2024-09-21T05:59:37.426456+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450786+0000","last_change":"2024-09-21T05:59:37.444802+0000","last_active":"2024-09-21T05:59:37.450786+0000","last_peered":"2024-09-21T05:59:37.450786+0000","last_clean":"2024-09-21T05:59:37.450786+0000","last_became_active":"2024-09-21T05:59:37.444457+0000","last_became_peered":"2024-09-21T05:59:37.444457+0000","last_unstale":"2024-09-21T05:59:37.450786+0000","last_undegraded":"2024-09-21T05:59:37.450786+0000","last_fullsized":"2024-09-21T05:59:37.450786+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:46.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.428769+0000","last_change":"2024-09-21T05:59:37.428769+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.428769+0000","last_undegraded":"2024-09-21T05:59:37.428769+0000","last_fullsized":"2024-09-21T05:59:37.428769+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451326+0000","last_change":"2024-09-21T05:59:37.444089+0000","last_active":"2024-09-21T05:59:37.451326+0000","last_peered":"2024-09-21T05:59:37.451326+0000","last_clean":"2024-09-21T05:59:37.451326+0000","last_became_active":"2024-09-21T05:59:37.443438+0000","last_became_peered":"2024-09-21T05:59:37.443438+0000","last_unstale":"2024-09-21T05:59:37.451326+0000","last_undegraded":"2024-09-21T05:59:37.451326+0000","last_fullsized":"2024-09-21T05:59:37.451326+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:34.733846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.435700+0000","last_change":"2024-09-21T05:59:37.435700+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.435700+0000","last_undegraded":"2024-09-21T05:59:37.435700+0000","last_fullsized":"2024-09-21T05:59:37.435700+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.447853+0000","last_change":"2024-09-21T05:59:37.445284+0000","last_active":"2024-09-21T05:59:37.447853+0000","last_peered":"2024-09-21T05:59:37.447853+0000","last_clean":"2024-09-21T05:59:37.447853+0000","last_became_active":"2024-09-21T05:59:37.444912+0000","last_became_peered":"2024-09-21T05:59:37.444912+0000","last_unstale":"2024-09-21T05:59:37.447853+0000","last_undegraded":"2024-09-21T05:59:37.447853+0000","last_fullsized":"2024-09-21T05:59:37.447853+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:20.236897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429835+0000","last_change":"2024-09-21T05:59:37.429835+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429835+0000","last_undegraded":"2024-09-21T05:59:37.429835+0000","last_fullsized":"2024-09-21T05:59:37.429835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.439195+0000","last_change":"2024-09-21T05:59:37.435640+0000","last_active":"2024-09-21T05:59:37.439195+0000","last_peered":"2024-09-21T05:59:37.439195+0000","last_clean":"2024-09-21T05:59:37.439195+0000","last_became_active":"2024-09-21T05:59:37.435202+0000","last_became_peered":"2024-09-21T05:59:37.435202+0000","last_unstale":"2024-09-21T05:59:37.439195+0000","last_undegraded":"2024-09-21T05:59:37.439195+0000","last_fullsized":"2024-09-21T05:59:37.439195+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:49.312472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431533+0000","last_change":"2024-09-21T05:59:37.431533+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431533+0000","last_undegraded":"2024-09-21T05:59:37.431533+0000","last_fullsized":"2024-09-21T05:59:37.431533+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.439961+0000","last_change":"2024-09-21T05:59:37.432827+0000","last_active":"2024-09-21T05:59:37.439961+0000","last_peered":"2024-09-21T05:59:37.439961+0000","last_clean":"2024-09-21T05:59:37.439961+0000","last_became_active":"2024-09-21T05:59:37.432500+0000","last_became_peered":"2024-09-21T05:59:37.432500+0000","last_unstale":"2024-09-21T05:59:37.439961+0000","last_undegraded":"2024-09-21T05:59:37.439961+0000","last_fullsized":"2024-09-21T05:59:37.439961+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.689460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433437+0000","last_change":"2024-09-21T05:59:37.433437+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433437+0000","last_undegraded":"2024-09-21T05:59:37.433437+0000","last_fullsized":"2024-09-21T05:59:37.433437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450224+0000","last_change":"2024-09-21T05:59:37.441674+0000","last_active":"2024-09-21T05:59:37.450224+0000","last_peered":"2024-09-21T05:59:37.450224+0000","last_clean":"2024-09-21T05:59:37.450224+0000","last_became_active":"2024-09-21T05:59:37.441330+0000","last_became_peered":"2024-09-21T05:59:37.441330+0000","last_unstale":"2024-09-21T05:59:37.450224+0000","last_undegraded":"2024-09-21T05:59:37.450224+0000","last_fullsized":"2024-09-21T05:59:37.450224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.968036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.427110+0000","last_change":"2024-09-21T05:59:37.427110+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.427110+0000","last_undegraded":"2024-09-21T05:59:37.427110+0000","last_fullsized":"2024-09-21T05:59:37.427110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450915+0000","last_change":"2024-09-21T05:59:37.447596+0000","last_active":"2024-09-21T05:59:37.450915+0000","last_peered":"2024-09-21T05:59:37.450915+0000","last_clean":"2024-09-21T05:59:37.450915+0000","last_became_active":"2024-09-21T05:59:37.440489+0000","last_became_peered":"2024-09-21T05:59:37.440489+0000","last_unstale":"2024-09-21T05:59:37.450915+0000","last_undegraded":"2024-09-21T05:59:37.450915+0000","last_fullsized":"2024-09-21T05:59:37.450915+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:31.718340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:59:37.411723+0000","last_change":"2024-09-21T05:59:37.411723+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.411723+0000","last_undegraded":"2024-09-21T05:59:37.411723+0000","last_fullsized":"2024-09-21T05:59:37.411723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.440034+0000","last_change":"2024-09-21T05:59:37.434300+0000","last_active":"2024-09-21T05:59:37.440034+0000","last_peered":"2024-09-21T05:59:37.440034+0000","last_clean":"2024-09-21T05:59:37.440034+0000","last_became_active":"2024-09-21T05:59:37.433933+0000","last_became_peered":"2024-09-21T05:59:37.433933+0000","last_unstale":"2024-09-21T05:59:37.440034+0000","last_undegraded":"2024-09-21T05:59:37.440034+0000","last_fullsized":"2024-09-21T05:59:37.440034+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.968036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432848+0000","last_change":"2024-09-21T05:59:37.432848+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432848+0000","last_undegraded":"2024-09-21T05:59:37.432848+0000","last_fullsized":"2024-09-21T05:59:37.432848+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451837+0000","last_change":"2024-09-21T05:59:37.446607+0000","last_active":"2024-09-21T05:59:37.451837+0000","last_peered":"2024-09-21T05:59:37.451837+0000","last_clean":"2024-09-21T05:59:37.451837+0000","last_became_active":"2024-09-21T05:59:37.446318+0000","last_became_peered":"2024-09-21T05:59:37.446318+0000","last_unstale":"2024-09-21T05:59:37.451837+0000","last_undegraded":"2024-09-21T05:59:37.451837+0000","last_fullsized":"2024-09-21T05:59:37.451837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:59.876514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.434548+0000","last_change":"2024-09-21T05:59:37.434548+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.434548+0000","last_undegraded":"2024-09-21T05:59:37.434548+0000","last_fullsized":"2024-09-21T05:59:37.434548+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.449888+0000","last_change":"2024-09-21T05:59:37.445107+0000","last_active":"2024-09-21T05:59:37.449888+0000","last_peered":"2024-09-21T05:59:37.449888+0000","last_clean":"2024-09-21T05:59:37.449888+0000","last_became_active":"2024-09-21T05:59:37.444746+0000","last_became_peered":"2024-09-21T05:59:37.444746+0000","last_unstale":"2024-09-21T05:59:37.449888+0000","last_undegraded":"2024-09-21T05:59:37.449888+0000","last_fullsized":"2024-09-21T05:59:37.449888+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.968036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431977+0000","last_change":"2024-09-21T05:59:37.431977+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431977+0000","last_undegraded":"2024-09-21T05:59:37.431977+0000","last_fullsized":"2024-09-21T05:59:37.431977+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450806+0000","last_change":"2024-09-21T05:59:37.445647+0000","last_active":"2024-09-21T05:59:37.450806+0000","last_peered":"2024-09-21T05:59:37.450806+0000","last_clean":"2024-09-21T05:59:37.450806+0000","last_became_active":"2024-09-21T05:59:37.445165+0000","last_became_peered":"2024-09-21T05:59:37.445165+0000","last_unstale":"2024-09-21T05:59:37.450806+0000","last_undegraded":"2024-09-21T05:59:37.450806+0000","last_fullsized":"2024-09-21T05:59:37.450806+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:42.959574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429475+0000","last_change":"2024-09-21T05:59:37.429475+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429475+0000","last_undegraded":"2024-09-21T05:59:37.429475+0000","last_fullsized":"2024-09-21T05:59:37.429475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.452222+0000","last_change":"2024-09-21T05:59:37.446726+0000","last_active":"2024-09-21T05:59:37.452222+0000","last_peered":"2024-09-21T05:59:37.452222+0000","last_clean":"2024-09-21T05:59:37.452222+0000","last_became_active":"2024-09-21T05:59:37.446299+0000","last_became_peered":"2024-09-21T05:59:37.446299+0000","last_unstale":"2024-09-21T05:59:37.452222+0000","last_undegraded":"2024-09-21T05:59:37.452222+0000","last_fullsized":"2024-09-21T05:59:37.452222+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:31.718340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429164+0000","last_change":"2024-09-21T05:59:37.429164+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429164+0000","last_undegraded":"2024-09-21T05:59:37.429164+0000","last_fullsized":"2024-09-21T05:59:37.429164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-21T05:59:36.420539+0000","last_change":"2024-09-21T05:59:36.420539+0000","last_active":"2024-09-21T05:59:36.408615+0000","last_peered":"2024-09-21T05:59:36.408615+0000","last_clean":"2024-09-21T05:59:36.408615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:36.420539+0000","last_undegraded":"2024-09-21T05:59:36.420539+0000","last_fullsized":"2024-09-21T05:59:36.420539+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425900+0000","last_change":"2024-09-21T05:59:37.425900+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425900+0000","last_undegraded":"2024-09-21T05:59:37.425900+0000","last_fullsized":"2024-09-21T05:59:37.425900+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450041+0000","last_change":"2024-09-21T05:59:37.446310+0000","last_active":"2024-09-21T05:59:37.450041+0000","last_peered":"2024-09-21T05:59:37.450041+0000","last_clean":"2024-09-21T05:59:37.450041+0000","last_became_active":"2024-09-21T05:59:37.445953+0000","last_became_peered":"2024-09-21T05:59:37.445953+0000","last_unstale":"2024-09-21T05:59:37.450041+0000","last_undegraded":"2024-09-21T05:59:37.450041+0000","last_fullsized":"2024-09-21T05:59:37.450041+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:42.959574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425968+0000","last_change":"2024-09-21T05:59:37.425968+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425968+0000","last_undegraded":"2024-09-21T05:59:37.425968+0000","last_fullsized":"2024-09-21T05:59:37.425968+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.439199+0000","last_change":"2024-09-21T05:59:37.435447+0000","last_active":"2024-09-21T05:59:37.439199+0000","last_peered":"2024-09-21T05:59:37.439199+0000","last_clean":"2024-09-21T05:59:37.439199+0000","last_became_active":"2024-09-21T05:59:37.435178+0000","last_became_peered":"2024-09-21T05:59:37.435178+0000","last_unstale":"2024-09-21T05:59:37.439199+0000","last_undegraded":"2024-09-21T05:59:37.439199+0000","last_fullsized":"2024-09-21T05:59:37.439199+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:24.390872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424041+0000","last_change":"2024-09-21T05:59:37.424041+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424041+0000","last_undegraded":"2024-09-21T05:59:37.424041+0000","last_fullsized":"2024-09-21T05:59:37.424041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-21T05:59:36.420486+0000","last_change":"2024-09-21T05:59:36.420486+0000","last_active":"2024-09-21T05:59:36.408615+0000","last_peered":"2024-09-21T05:59:36.408615+0000","last_clean":"2024-09-21T05:59:36.408615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:36.420486+0000","last_undegraded":"2024-09-21T05:59:36.420486+0000","last_fullsized":"2024-09-21T05:59:36.420486+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.422407+0000","last_change":"2024-09-21T05:59:37.422407+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.422407+0000","last_undegraded":"2024-09-21T05:59:37.422407+0000","last_fullsized":"2024-09-21T05:59:37.422407+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451290+0000","last_change":"2024-09-21T05:59:37.442907+0000","last_active":"2024-09-21T05:59:37.451290+0000","last_peered":"2024-09-21T05:59:37.451290+0000","last_clean":"2024-09-21T05:59:37.451290+0000","last_became_active":"2024-09-21T05:59:37.439722+0000","last_became_peered":"2024-09-21T05:59:37.439722+0000","last_unstale":"2024-09-21T05:59:37.451290+0000","last_undegraded":"2024-09-21T05:59:37.451290+0000","last_fullsized":"2024-09-21T05:59:37.451290+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.103990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430505+0000","last_change":"2024-09-21T05:59:37.430505+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430505+0000","last_undegraded":"2024-09-21T05:59:37.430505+0000","last_fullsized":"2024-09-21T05:59:37.430505+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.440071+0000","last_change":"2024-09-21T05:59:37.435112+0000","last_active":"2024-09-21T05:59:37.440071+0000","last_peered":"2024-09-21T05:59:37.440071+0000","last_clean":"2024-09-21T05:59:37.440071+0000","last_became_active":"2024-09-21T05:59:37.434789+0000","last_became_peered":"2024-09-21T05:59:37.434789+0000","last_unstale":"2024-09-21T05:59:37.440071+0000","last_undegraded":"2024-09-21T05:59:37.440071+0000","last_fullsized":"2024-09-21T05:59:37.440071+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:46.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":148,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.436248+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:59:37.436248+0000","last_peered":"2024-09-21T05:59:37.436248+0000","last_clean":"2024-09-21T05:59:37.436248+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:59:37.436248+0000","last_undegraded":"2024-09-21T05:59:37.436248+0000","last_fullsized":"2024-09-21T05:59:37.436248+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":122,"acting":122,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1077248,"data_stored":779961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":471040,"data_stored":183326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31664,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340176,"statfs":{"total":96636764160,"available":96604340224,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31664,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340176,"statfs":{"total":96636764160,"available":96604340224,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35824,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336016,"statfs":{"total":96636764160,"available":96600080384,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36400,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335440,"statfs":{"total":96636764160,"available":96599490560,"internally_reserved":0,"allocated":1077248,"data_stored":779961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39972,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331868,"statfs":{"total":96636764160,"available":96595832832,"internally_reserved":0,"allocated":471040,"data_stored":183326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:59:42.638 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:59:42.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.835+0000 7f17a6800640 1 -- 172.21.15.50:0/4199387424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a015c5d0 msgr2=0x7f17a0073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.835+0000 7f17a6800640 1 --2- 172.21.15.50:0/4199387424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a015c5d0 0x7f17a0073dc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1790009840 tx=0x7f179001c610 comp rx=0 tx=0).stop 2024-09-21T05:59:42.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.835+0000 7f17a6800640 1 -- 172.21.15.50:0/4199387424 shutdown_connections 2024-09-21T05:59:42.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.835+0000 7f17a6800640 1 --2- 172.21.15.50:0/4199387424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a015c5d0 0x7f17a0073dc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.835+0000 7f17a6800640 1 -- 172.21.15.50:0/4199387424 >> 172.21.15.50:0/4199387424 conn(0x7f17a00b2c40 msgr2=0x7f17a00b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:42.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.835+0000 7f17a6800640 1 -- 172.21.15.50:0/4199387424 shutdown_connections 2024-09-21T05:59:42.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.835+0000 7f17a6800640 1 -- 172.21.15.50:0/4199387424 wait complete. 2024-09-21T05:59:42.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f17a6800640 1 Processor -- start 2024-09-21T05:59:42.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f17a6800640 1 -- start start 2024-09-21T05:59:42.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f17a6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17a006fa70 0x7f17a01fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:42.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f17a6800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f17a0074300 0x7f17a0152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:42.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f17a6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a015c5d0 0x7f17a0151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:42.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f17a6800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f17a0169b20 con 0x7f17a015c5d0 2024-09-21T05:59:42.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f17a6800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f17a00b3d50 con 0x7f17a006fa70 2024-09-21T05:59:42.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f17a6800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f17a0169ca0 con 0x7f17a0074300 2024-09-21T05:59:42.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f179fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a015c5d0 0x7f17a0151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:42.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f179fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a015c5d0 0x7f17a0151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46640/0 (socket says 172.21.15.50:46640) 2024-09-21T05:59:42.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f179fe00640 1 -- 172.21.15.50:0/1538408265 learned_addr learned my addr 172.21.15.50:0/1538408265 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:42.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f179f400640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17a006fa70 0x7f17a01fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:42.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f1797e00640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f17a0074300 0x7f17a0152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:42.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f179fe00640 1 -- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f17a0074300 msgr2=0x7f17a0152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f179fe00640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f17a0074300 0x7f17a0152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f179fe00640 1 -- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17a006fa70 msgr2=0x7f17a01fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:42.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f179fe00640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17a006fa70 0x7f17a01fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:42.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f179fe00640 1 -- 172.21.15.50:0/1538408265 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a01f8c40 con 0x7f17a015c5d0 2024-09-21T05:59:42.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f1797e00640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f17a0074300 0x7f17a0152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:42.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f179fe00640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a015c5d0 0x7f17a0151620 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f179800cb40 tx=0x7f179800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:42.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f1797400640 1 -- 172.21.15.50:0/1538408265 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798016020 con 0x7f17a015c5d0 2024-09-21T05:59:42.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f1797400640 1 -- 172.21.15.50:0/1538408265 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f179800aae0 con 0x7f17a015c5d0 2024-09-21T05:59:42.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f1797400640 1 -- 172.21.15.50:0/1538408265 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179800ce20 con 0x7f17a015c5d0 2024-09-21T05:59:42.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f17a6800640 1 -- 172.21.15.50:0/1538408265 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a01f8ec0 con 0x7f17a015c5d0 2024-09-21T05:59:42.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.839+0000 7f17a6800640 1 -- 172.21.15.50:0/1538408265 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a01f9480 con 0x7f17a015c5d0 2024-09-21T05:59:42.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.843+0000 7f17a6800640 1 -- 172.21.15.50:0/1538408265 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a0073dc0 con 0x7f17a015c5d0 2024-09-21T05:59:42.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.843+0000 7f1797400640 1 -- 172.21.15.50:0/1538408265 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f1798019020 con 0x7f17a015c5d0 2024-09-21T05:59:42.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.843+0000 7f1797400640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f177c073590 0x7f177c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:42.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.843+0000 7f1797400640 1 -- 172.21.15.50:0/1538408265 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f1798094ae0 con 0x7f17a015c5d0 2024-09-21T05:59:42.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.843+0000 7f179f400640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f177c073590 0x7f177c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:42.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.843+0000 7f179f400640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f177c073590 0x7f177c075aa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f17900af000 tx=0x7f179001ae50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:42.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:42.847+0000 7f1797400640 1 -- 172.21.15.50:0/1538408265 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1798020020 con 0x7f17a015c5d0 2024-09-21T05:59:43.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.027+0000 7f17a6800640 1 -- 172.21.15.50:0/1538408265 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17a006b540 con 0x7f177c073590 2024-09-21T05:59:43.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.031+0000 7f1797400640 1 -- 172.21.15.50:0/1538408265 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246916 (secure 0 0 0) 0x7f1790005ca0 con 0x7f177c073590 2024-09-21T05:59:43.037 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:43.039 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:59:43.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.035+0000 7f17a6800640 1 -- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f177c073590 msgr2=0x7f177c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.035+0000 7f17a6800640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f177c073590 0x7f177c075aa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f17900af000 tx=0x7f179001ae50 comp rx=0 tx=0).stop 2024-09-21T05:59:43.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.035+0000 7f17a6800640 1 -- 172.21.15.50:0/1538408265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a015c5d0 msgr2=0x7f17a0151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.035+0000 7f17a6800640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a015c5d0 0x7f17a0151620 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f179800cb40 tx=0x7f179800cc30 comp rx=0 tx=0).stop 2024-09-21T05:59:43.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.035+0000 7f17a6800640 1 -- 172.21.15.50:0/1538408265 shutdown_connections 2024-09-21T05:59:43.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.035+0000 7f17a6800640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f177c073590 0x7f177c075aa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.035+0000 7f17a6800640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a015c5d0 0x7f17a0151620 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.035+0000 7f17a6800640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f17a0074300 0x7f17a0152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.035+0000 7f17a6800640 1 --2- 172.21.15.50:0/1538408265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17a006fa70 0x7f17a01fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.035+0000 7f17a6800640 1 -- 172.21.15.50:0/1538408265 >> 172.21.15.50:0/1538408265 conn(0x7f17a00b2c40 msgr2=0x7f17a0075f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:43.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.035+0000 7f17a6800640 1 -- 172.21.15.50:0/1538408265 shutdown_connections 2024-09-21T05:59:43.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.035+0000 7f17a6800640 1 -- 172.21.15.50:0/1538408265 wait complete. 2024-09-21T05:59:43.059 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-21T05:59:42.934565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271128,"kb_used_data":4952,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754703592,"statfs":{"total":773094113280,"available":772816478208,"internally_reserved":0,"allocated":5070848,"data_stored":2704358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001455"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430439+0000","last_change":"2024-09-21T05:59:37.430439+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430439+0000","last_undegraded":"2024-09-21T05:59:37.430439+0000","last_fullsized":"2024-09-21T05:59:37.430439+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424624+0000","last_change":"2024-09-21T05:59:37.424624+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424624+0000","last_undegraded":"2024-09-21T05:59:37.424624+0000","last_fullsized":"2024-09-21T05:59:37.424624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424101+0000","last_change":"2024-09-21T05:59:37.424101+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424101+0000","last_undegraded":"2024-09-21T05:59:37.424101+0000","last_fullsized":"2024-09-21T05:59:37.424101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431709+0000","last_change":"2024-09-21T05:59:37.431709+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431709+0000","last_undegraded":"2024-09-21T05:59:37.431709+0000","last_fullsized":"2024-09-21T05:59:37.431709+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423190+0000","last_change":"2024-09-21T05:59:37.423190+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423190+0000","last_undegraded":"2024-09-21T05:59:37.423190+0000","last_fullsized":"2024-09-21T05:59:37.423190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.428041+0000","last_change":"2024-09-21T05:59:37.428041+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.428041+0000","last_undegraded":"2024-09-21T05:59:37.428041+0000","last_fullsized":"2024-09-21T05:59:37.428041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430128+0000","last_change":"2024-09-21T05:59:37.430128+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430128+0000","last_undegraded":"2024-09-21T05:59:37.430128+0000","last_fullsized":"2024-09-21T05:59:37.430128+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433589+0000","last_change":"2024-09-21T05:59:37.433589+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433589+0000","last_undegraded":"2024-09-21T05:59:37.433589+0000","last_fullsized":"2024-09-21T05:59:37.433589+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430329+0000","last_change":"2024-09-21T05:59:37.430329+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430329+0000","last_undegraded":"2024-09-21T05:59:37.430329+0000","last_fullsized":"2024-09-21T05:59:37.430329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431028+0000","last_change":"2024-09-21T05:59:37.431028+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431028+0000","last_undegraded":"2024-09-21T05:59:37.431028+0000","last_fullsized":"2024-09-21T05:59:37.431028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429854+0000","last_change":"2024-09-21T05:59:37.429854+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429854+0000","last_undegraded":"2024-09-21T05:59:37.429854+0000","last_fullsized":"2024-09-21T05:59:37.429854+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423767+0000","last_change":"2024-09-21T05:59:37.423767+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423767+0000","last_undegraded":"2024-09-21T05:59:37.423767+0000","last_fullsized":"2024-09-21T05:59:37.423767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.435082+0000","last_change":"2024-09-21T05:59:37.435082+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.435082+0000","last_undegraded":"2024-09-21T05:59:37.435082+0000","last_fullsized":"2024-09-21T05:59:37.435082+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432295+0000","last_change":"2024-09-21T05:59:37.432295+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432295+0000","last_undegraded":"2024-09-21T05:59:37.432295+0000","last_fullsized":"2024-09-21T05:59:37.432295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.427649+0000","last_change":"2024-09-21T05:59:37.427649+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.427649+0000","last_undegraded":"2024-09-21T05:59:37.427649+0000","last_fullsized":"2024-09-21T05:59:37.427649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430585+0000","last_change":"2024-09-21T05:59:37.430585+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430585+0000","last_undegraded":"2024-09-21T05:59:37.430585+0000","last_fullsized":"2024-09-21T05:59:37.430585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424708+0000","last_change":"2024-09-21T05:59:37.424708+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424708+0000","last_undegraded":"2024-09-21T05:59:37.424708+0000","last_fullsized":"2024-09-21T05:59:37.424708+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425629+0000","last_change":"2024-09-21T05:59:37.425629+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425629+0000","last_undegraded":"2024-09-21T05:59:37.425629+0000","last_fullsized":"2024-09-21T05:59:37.425629+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432796+0000","last_change":"2024-09-21T05:59:37.432796+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432796+0000","last_undegraded":"2024-09-21T05:59:37.432796+0000","last_fullsized":"2024-09-21T05:59:37.432796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.434597+0000","last_change":"2024-09-21T05:59:37.434597+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.434597+0000","last_undegraded":"2024-09-21T05:59:37.434597+0000","last_fullsized":"2024-09-21T05:59:37.434597+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.427426+0000","last_change":"2024-09-21T05:59:37.427426+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.427426+0000","last_undegraded":"2024-09-21T05:59:37.427426+0000","last_fullsized":"2024-09-21T05:59:37.427426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429310+0000","last_change":"2024-09-21T05:59:37.429310+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429310+0000","last_undegraded":"2024-09-21T05:59:37.429310+0000","last_fullsized":"2024-09-21T05:59:37.429310+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.434559+0000","last_change":"2024-09-21T05:59:37.434559+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.434559+0000","last_undegraded":"2024-09-21T05:59:37.434559+0000","last_fullsized":"2024-09-21T05:59:37.434559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429894+0000","last_change":"2024-09-21T05:59:37.429894+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429894+0000","last_undegraded":"2024-09-21T05:59:37.429894+0000","last_fullsized":"2024-09-21T05:59:37.429894+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433207+0000","last_change":"2024-09-21T05:59:37.433207+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433207+0000","last_undegraded":"2024-09-21T05:59:37.433207+0000","last_fullsized":"2024-09-21T05:59:37.433207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.439276+0000","last_change":"2024-09-21T05:59:37.439276+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.439276+0000","last_undegraded":"2024-09-21T05:59:37.439276+0000","last_fullsized":"2024-09-21T05:59:37.439276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423541+0000","last_change":"2024-09-21T05:59:37.423541+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423541+0000","last_undegraded":"2024-09-21T05:59:37.423541+0000","last_fullsized":"2024-09-21T05:59:37.423541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423216+0000","last_change":"2024-09-21T05:59:37.423216+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423216+0000","last_undegraded":"2024-09-21T05:59:37.423216+0000","last_fullsized":"2024-09-21T05:59:37.423216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431328+0000","last_change":"2024-09-21T05:59:37.431328+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431328+0000","last_undegraded":"2024-09-21T05:59:37.431328+0000","last_fullsized":"2024-09-21T05:59:37.431328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423470+0000","last_change":"2024-09-21T05:59:37.423470+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423470+0000","last_undegraded":"2024-09-21T05:59:37.423470+0000","last_fullsized":"2024-09-21T05:59:37.423470+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.426152+0000","last_change":"2024-09-21T05:59:37.426152+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.426152+0000","last_undegraded":"2024-09-21T05:59:37.426152+0000","last_fullsized":"2024-09-21T05:59:37.426152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.438506+0000","last_change":"2024-09-21T05:59:37.438506+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.438506+0000","last_undegraded":"2024-09-21T05:59:37.438506+0000","last_fullsized":"2024-09-21T05:59:37.438506+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837865+0000","last_change":"2024-09-21T05:59:38.462061+0000","last_active":"2024-09-21T05:59:39.837865+0000","last_peered":"2024-09-21T05:59:39.837865+0000","last_clean":"2024-09-21T05:59:39.837865+0000","last_became_active":"2024-09-21T05:59:38.461740+0000","last_became_peered":"2024-09-21T05:59:38.461740+0000","last_unstale":"2024-09-21T05:59:39.837865+0000","last_undegraded":"2024-09-21T05:59:39.837865+0000","last_fullsized":"2024-09-21T05:59:39.837865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:17.985198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451150+0000","last_change":"2024-09-21T05:59:37.447751+0000","last_active":"2024-09-21T05:59:37.451150+0000","last_peered":"2024-09-21T05:59:37.451150+0000","last_clean":"2024-09-21T05:59:37.451150+0000","last_became_active":"2024-09-21T05:59:37.447392+0000","last_became_peered":"2024-09-21T05:59:37.447392+0000","last_unstale":"2024-09-21T05:59:37.451150+0000","last_undegraded":"2024-09-21T05:59:37.451150+0000","last_fullsized":"2024-09-21T05:59:37.451150+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:17.297839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432540+0000","last_change":"2024-09-21T05:59:37.432540+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432540+0000","last_undegraded":"2024-09-21T05:59:37.432540+0000","last_fullsized":"2024-09-21T05:59:37.432540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.444761+0000","last_change":"2024-09-21T05:59:37.443312+0000","last_active":"2024-09-21T05:59:37.444761+0000","last_peered":"2024-09-21T05:59:37.444761+0000","last_clean":"2024-09-21T05:59:37.444761+0000","last_became_active":"2024-09-21T05:59:37.442457+0000","last_became_peered":"2024-09-21T05:59:37.442457+0000","last_unstale":"2024-09-21T05:59:37.444761+0000","last_undegraded":"2024-09-21T05:59:37.444761+0000","last_fullsized":"2024-09-21T05:59:37.444761+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:54.217575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.422517+0000","last_change":"2024-09-21T05:59:37.422517+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.422517+0000","last_undegraded":"2024-09-21T05:59:37.422517+0000","last_fullsized":"2024-09-21T05:59:37.422517+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837882+0000","last_change":"2024-09-21T05:59:37.440271+0000","last_active":"2024-09-21T05:59:39.837882+0000","last_peered":"2024-09-21T05:59:39.837882+0000","last_clean":"2024-09-21T05:59:39.837882+0000","last_became_active":"2024-09-21T05:59:37.439909+0000","last_became_peered":"2024-09-21T05:59:37.439909+0000","last_unstale":"2024-09-21T05:59:39.837882+0000","last_undegraded":"2024-09-21T05:59:39.837882+0000","last_fullsized":"2024-09-21T05:59:39.837882+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:15.657492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837975+0000","last_change":"2024-09-21T05:59:38.444285+0000","last_active":"2024-09-21T05:59:39.837975+0000","last_peered":"2024-09-21T05:59:39.837975+0000","last_clean":"2024-09-21T05:59:39.837975+0000","last_became_active":"2024-09-21T05:59:38.443952+0000","last_became_peered":"2024-09-21T05:59:38.443952+0000","last_unstale":"2024-09-21T05:59:39.837975+0000","last_undegraded":"2024-09-21T05:59:39.837975+0000","last_fullsized":"2024-09-21T05:59:39.837975+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:31.815943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.449829+0000","last_change":"2024-09-21T05:59:37.443607+0000","last_active":"2024-09-21T05:59:37.449829+0000","last_peered":"2024-09-21T05:59:37.449829+0000","last_clean":"2024-09-21T05:59:37.449829+0000","last_became_active":"2024-09-21T05:59:37.443262+0000","last_became_peered":"2024-09-21T05:59:37.443262+0000","last_unstale":"2024-09-21T05:59:37.449829+0000","last_undegraded":"2024-09-21T05:59:37.449829+0000","last_fullsized":"2024-09-21T05:59:37.449829+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:21.369884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.422840+0000","last_change":"2024-09-21T05:59:37.422840+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.422840+0000","last_undegraded":"2024-09-21T05:59:37.422840+0000","last_fullsized":"2024-09-21T05:59:37.422840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451244+0000","last_change":"2024-09-21T05:59:37.447920+0000","last_active":"2024-09-21T05:59:37.451244+0000","last_peered":"2024-09-21T05:59:37.451244+0000","last_clean":"2024-09-21T05:59:37.451244+0000","last_became_active":"2024-09-21T05:59:37.447240+0000","last_became_peered":"2024-09-21T05:59:37.447240+0000","last_unstale":"2024-09-21T05:59:37.451244+0000","last_undegraded":"2024-09-21T05:59:37.451244+0000","last_fullsized":"2024-09-21T05:59:37.451244+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:52.027735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432049+0000","last_change":"2024-09-21T05:59:37.432049+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432049+0000","last_undegraded":"2024-09-21T05:59:37.432049+0000","last_fullsized":"2024-09-21T05:59:37.432049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.438146+0000","last_change":"2024-09-21T05:59:37.429073+0000","last_active":"2024-09-21T05:59:37.438146+0000","last_peered":"2024-09-21T05:59:37.438146+0000","last_clean":"2024-09-21T05:59:37.438146+0000","last_became_active":"2024-09-21T05:59:37.428733+0000","last_became_peered":"2024-09-21T05:59:37.428733+0000","last_unstale":"2024-09-21T05:59:37.438146+0000","last_undegraded":"2024-09-21T05:59:37.438146+0000","last_fullsized":"2024-09-21T05:59:37.438146+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:39.275851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433120+0000","last_change":"2024-09-21T05:59:37.433120+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433120+0000","last_undegraded":"2024-09-21T05:59:37.433120+0000","last_fullsized":"2024-09-21T05:59:37.433120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.440195+0000","last_change":"2024-09-21T05:59:37.437489+0000","last_active":"2024-09-21T05:59:37.440195+0000","last_peered":"2024-09-21T05:59:37.440195+0000","last_clean":"2024-09-21T05:59:37.440195+0000","last_became_active":"2024-09-21T05:59:37.437170+0000","last_became_peered":"2024-09-21T05:59:37.437170+0000","last_unstale":"2024-09-21T05:59:37.440195+0000","last_undegraded":"2024-09-21T05:59:37.440195+0000","last_fullsized":"2024-09-21T05:59:37.440195+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:42.959574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.439321+0000","last_change":"2024-09-21T05:59:37.439321+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.439321+0000","last_undegraded":"2024-09-21T05:59:37.439321+0000","last_fullsized":"2024-09-21T05:59:37.439321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451906+0000","last_change":"2024-09-21T05:59:37.449577+0000","last_active":"2024-09-21T05:59:37.451906+0000","last_peered":"2024-09-21T05:59:37.451906+0000","last_clean":"2024-09-21T05:59:37.451906+0000","last_became_active":"2024-09-21T05:59:37.449255+0000","last_became_peered":"2024-09-21T05:59:37.449255+0000","last_unstale":"2024-09-21T05:59:37.451906+0000","last_undegraded":"2024-09-21T05:59:37.451906+0000","last_fullsized":"2024-09-21T05:59:37.451906+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:39.275851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425083+0000","last_change":"2024-09-21T05:59:37.425083+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425083+0000","last_undegraded":"2024-09-21T05:59:37.425083+0000","last_fullsized":"2024-09-21T05:59:37.425083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.444043+0000","last_change":"2024-09-21T05:59:37.440683+0000","last_active":"2024-09-21T05:59:37.444043+0000","last_peered":"2024-09-21T05:59:37.444043+0000","last_clean":"2024-09-21T05:59:37.444043+0000","last_became_active":"2024-09-21T05:59:37.440336+0000","last_became_peered":"2024-09-21T05:59:37.440336+0000","last_unstale":"2024-09-21T05:59:37.444043+0000","last_undegraded":"2024-09-21T05:59:37.444043+0000","last_fullsized":"2024-09-21T05:59:37.444043+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:13.571664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.428994+0000","last_change":"2024-09-21T05:59:37.428994+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.428994+0000","last_undegraded":"2024-09-21T05:59:37.428994+0000","last_fullsized":"2024-09-21T05:59:37.428994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451867+0000","last_change":"2024-09-21T05:59:37.448714+0000","last_active":"2024-09-21T05:59:37.451867+0000","last_peered":"2024-09-21T05:59:37.451867+0000","last_clean":"2024-09-21T05:59:37.451867+0000","last_became_active":"2024-09-21T05:59:37.448310+0000","last_became_peered":"2024-09-21T05:59:37.448310+0000","last_unstale":"2024-09-21T05:59:37.451867+0000","last_undegraded":"2024-09-21T05:59:37.451867+0000","last_fullsized":"2024-09-21T05:59:37.451867+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:21.097323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.426543+0000","last_change":"2024-09-21T05:59:37.426543+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.426543+0000","last_undegraded":"2024-09-21T05:59:37.426543+0000","last_fullsized":"2024-09-21T05:59:37.426543+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450006+0000","last_change":"2024-09-21T05:59:37.445812+0000","last_active":"2024-09-21T05:59:37.450006+0000","last_peered":"2024-09-21T05:59:37.450006+0000","last_clean":"2024-09-21T05:59:37.450006+0000","last_became_active":"2024-09-21T05:59:37.445496+0000","last_became_peered":"2024-09-21T05:59:37.445496+0000","last_unstale":"2024-09-21T05:59:37.450006+0000","last_undegraded":"2024-09-21T05:59:37.450006+0000","last_fullsized":"2024-09-21T05:59:37.450006+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:46.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429754+0000","last_change":"2024-09-21T05:59:37.429754+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429754+0000","last_undegraded":"2024-09-21T05:59:37.429754+0000","last_fullsized":"2024-09-21T05:59:37.429754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.443714+0000","last_change":"2024-09-21T05:59:37.440777+0000","last_active":"2024-09-21T05:59:37.443714+0000","last_peered":"2024-09-21T05:59:37.443714+0000","last_clean":"2024-09-21T05:59:37.443714+0000","last_became_active":"2024-09-21T05:59:37.440460+0000","last_became_peered":"2024-09-21T05:59:37.440460+0000","last_unstale":"2024-09-21T05:59:37.443714+0000","last_undegraded":"2024-09-21T05:59:37.443714+0000","last_fullsized":"2024-09-21T05:59:37.443714+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:03.466004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430964+0000","last_change":"2024-09-21T05:59:37.430964+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430964+0000","last_undegraded":"2024-09-21T05:59:37.430964+0000","last_fullsized":"2024-09-21T05:59:37.430964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451384+0000","last_change":"2024-09-21T05:59:37.447984+0000","last_active":"2024-09-21T05:59:37.451384+0000","last_peered":"2024-09-21T05:59:37.451384+0000","last_clean":"2024-09-21T05:59:37.451384+0000","last_became_active":"2024-09-21T05:59:37.447685+0000","last_became_peered":"2024-09-21T05:59:37.447685+0000","last_unstale":"2024-09-21T05:59:37.451384+0000","last_undegraded":"2024-09-21T05:59:37.451384+0000","last_fullsized":"2024-09-21T05:59:37.451384+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:36.267884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431534+0000","last_change":"2024-09-21T05:59:37.431534+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431534+0000","last_undegraded":"2024-09-21T05:59:37.431534+0000","last_fullsized":"2024-09-21T05:59:37.431534+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.449770+0000","last_change":"2024-09-21T05:59:37.443723+0000","last_active":"2024-09-21T05:59:37.449770+0000","last_peered":"2024-09-21T05:59:37.449770+0000","last_clean":"2024-09-21T05:59:37.449770+0000","last_became_active":"2024-09-21T05:59:37.443383+0000","last_became_peered":"2024-09-21T05:59:37.443383+0000","last_unstale":"2024-09-21T05:59:37.449770+0000","last_undegraded":"2024-09-21T05:59:37.449770+0000","last_fullsized":"2024-09-21T05:59:37.449770+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.689460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.426456+0000","last_change":"2024-09-21T05:59:37.426456+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.426456+0000","last_undegraded":"2024-09-21T05:59:37.426456+0000","last_fullsized":"2024-09-21T05:59:37.426456+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450786+0000","last_change":"2024-09-21T05:59:37.444802+0000","last_active":"2024-09-21T05:59:37.450786+0000","last_peered":"2024-09-21T05:59:37.450786+0000","last_clean":"2024-09-21T05:59:37.450786+0000","last_became_active":"2024-09-21T05:59:37.444457+0000","last_became_peered":"2024-09-21T05:59:37.444457+0000","last_unstale":"2024-09-21T05:59:37.450786+0000","last_undegraded":"2024-09-21T05:59:37.450786+0000","last_fullsized":"2024-09-21T05:59:37.450786+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:46.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.428769+0000","last_change":"2024-09-21T05:59:37.428769+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.428769+0000","last_undegraded":"2024-09-21T05:59:37.428769+0000","last_fullsized":"2024-09-21T05:59:37.428769+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451326+0000","last_change":"2024-09-21T05:59:37.444089+0000","last_active":"2024-09-21T05:59:37.451326+0000","last_peered":"2024-09-21T05:59:37.451326+0000","last_clean":"2024-09-21T05:59:37.451326+0000","last_became_active":"2024-09-21T05:59:37.443438+0000","last_became_peered":"2024-09-21T05:59:37.443438+0000","last_unstale":"2024-09-21T05:59:37.451326+0000","last_undegraded":"2024-09-21T05:59:37.451326+0000","last_fullsized":"2024-09-21T05:59:37.451326+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:34.733846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.435700+0000","last_change":"2024-09-21T05:59:37.435700+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.435700+0000","last_undegraded":"2024-09-21T05:59:37.435700+0000","last_fullsized":"2024-09-21T05:59:37.435700+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.447853+0000","last_change":"2024-09-21T05:59:37.445284+0000","last_active":"2024-09-21T05:59:37.447853+0000","last_peered":"2024-09-21T05:59:37.447853+0000","last_clean":"2024-09-21T05:59:37.447853+0000","last_became_active":"2024-09-21T05:59:37.444912+0000","last_became_peered":"2024-09-21T05:59:37.444912+0000","last_unstale":"2024-09-21T05:59:37.447853+0000","last_undegraded":"2024-09-21T05:59:37.447853+0000","last_fullsized":"2024-09-21T05:59:37.447853+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:20.236897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429835+0000","last_change":"2024-09-21T05:59:37.429835+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429835+0000","last_undegraded":"2024-09-21T05:59:37.429835+0000","last_fullsized":"2024-09-21T05:59:37.429835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.439195+0000","last_change":"2024-09-21T05:59:37.435640+0000","last_active":"2024-09-21T05:59:37.439195+0000","last_peered":"2024-09-21T05:59:37.439195+0000","last_clean":"2024-09-21T05:59:37.439195+0000","last_became_active":"2024-09-21T05:59:37.435202+0000","last_became_peered":"2024-09-21T05:59:37.435202+0000","last_unstale":"2024-09-21T05:59:37.439195+0000","last_undegraded":"2024-09-21T05:59:37.439195+0000","last_fullsized":"2024-09-21T05:59:37.439195+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:49.312472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431533+0000","last_change":"2024-09-21T05:59:37.431533+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431533+0000","last_undegraded":"2024-09-21T05:59:37.431533+0000","last_fullsized":"2024-09-21T05:59:37.431533+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.439961+0000","last_change":"2024-09-21T05:59:37.432827+0000","last_active":"2024-09-21T05:59:37.439961+0000","last_peered":"2024-09-21T05:59:37.439961+0000","last_clean":"2024-09-21T05:59:37.439961+0000","last_became_active":"2024-09-21T05:59:37.432500+0000","last_became_peered":"2024-09-21T05:59:37.432500+0000","last_unstale":"2024-09-21T05:59:37.439961+0000","last_undegraded":"2024-09-21T05:59:37.439961+0000","last_fullsized":"2024-09-21T05:59:37.439961+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.689460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433437+0000","last_change":"2024-09-21T05:59:37.433437+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433437+0000","last_undegraded":"2024-09-21T05:59:37.433437+0000","last_fullsized":"2024-09-21T05:59:37.433437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450224+0000","last_change":"2024-09-21T05:59:37.441674+0000","last_active":"2024-09-21T05:59:37.450224+0000","last_peered":"2024-09-21T05:59:37.450224+0000","last_clean":"2024-09-21T05:59:37.450224+0000","last_became_active":"2024-09-21T05:59:37.441330+0000","last_became_peered":"2024-09-21T05:59:37.441330+0000","last_unstale":"2024-09-21T05:59:37.450224+0000","last_undegraded":"2024-09-21T05:59:37.450224+0000","last_fullsized":"2024-09-21T05:59:37.450224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.968036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.427110+0000","last_change":"2024-09-21T05:59:37.427110+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.427110+0000","last_undegraded":"2024-09-21T05:59:37.427110+0000","last_fullsized":"2024-09-21T05:59:37.427110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450915+0000","last_change":"2024-09-21T05:59:37.447596+0000","last_active":"2024-09-21T05:59:37.450915+0000","last_peered":"2024-09-21T05:59:37.450915+0000","last_clean":"2024-09-21T05:59:37.450915+0000","last_became_active":"2024-09-21T05:59:37.440489+0000","last_became_peered":"2024-09-21T05:59:37.440489+0000","last_unstale":"2024-09-21T05:59:37.450915+0000","last_undegraded":"2024-09-21T05:59:37.450915+0000","last_fullsized":"2024-09-21T05:59:37.450915+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:31.718340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837878+0000","last_change":"2024-09-21T05:59:38.435762+0000","last_active":"2024-09-21T05:59:39.837878+0000","last_peered":"2024-09-21T05:59:39.837878+0000","last_clean":"2024-09-21T05:59:39.837878+0000","last_became_active":"2024-09-21T05:59:38.434259+0000","last_became_peered":"2024-09-21T05:59:38.434259+0000","last_unstale":"2024-09-21T05:59:39.837878+0000","last_undegraded":"2024-09-21T05:59:39.837878+0000","last_fullsized":"2024-09-21T05:59:39.837878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:36.174704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.440034+0000","last_change":"2024-09-21T05:59:37.434300+0000","last_active":"2024-09-21T05:59:37.440034+0000","last_peered":"2024-09-21T05:59:37.440034+0000","last_clean":"2024-09-21T05:59:37.440034+0000","last_became_active":"2024-09-21T05:59:37.433933+0000","last_became_peered":"2024-09-21T05:59:37.433933+0000","last_unstale":"2024-09-21T05:59:37.440034+0000","last_undegraded":"2024-09-21T05:59:37.440034+0000","last_fullsized":"2024-09-21T05:59:37.440034+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.968036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432848+0000","last_change":"2024-09-21T05:59:37.432848+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432848+0000","last_undegraded":"2024-09-21T05:59:37.432848+0000","last_fullsized":"2024-09-21T05:59:37.432848+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451837+0000","last_change":"2024-09-21T05:59:37.446607+0000","last_active":"2024-09-21T05:59:37.451837+0000","last_peered":"2024-09-21T05:59:37.451837+0000","last_clean":"2024-09-21T05:59:37.451837+0000","last_became_active":"2024-09-21T05:59:37.446318+0000","last_became_peered":"2024-09-21T05:59:37.446318+0000","last_unstale":"2024-09-21T05:59:37.451837+0000","last_undegraded":"2024-09-21T05:59:37.451837+0000","last_fullsized":"2024-09-21T05:59:37.451837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:59.876514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.434548+0000","last_change":"2024-09-21T05:59:37.434548+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.434548+0000","last_undegraded":"2024-09-21T05:59:37.434548+0000","last_fullsized":"2024-09-21T05:59:37.434548+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.449888+0000","last_change":"2024-09-21T05:59:37.445107+0000","last_active":"2024-09-21T05:59:37.449888+0000","last_peered":"2024-09-21T05:59:37.449888+0000","last_clean":"2024-09-21T05:59:37.449888+0000","last_became_active":"2024-09-21T05:59:37.444746+0000","last_became_peered":"2024-09-21T05:59:37.444746+0000","last_unstale":"2024-09-21T05:59:37.449888+0000","last_undegraded":"2024-09-21T05:59:37.449888+0000","last_fullsized":"2024-09-21T05:59:37.449888+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.968036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431977+0000","last_change":"2024-09-21T05:59:37.431977+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431977+0000","last_undegraded":"2024-09-21T05:59:37.431977+0000","last_fullsized":"2024-09-21T05:59:37.431977+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450806+0000","last_change":"2024-09-21T05:59:37.445647+0000","last_active":"2024-09-21T05:59:37.450806+0000","last_peered":"2024-09-21T05:59:37.450806+0000","last_clean":"2024-09-21T05:59:37.450806+0000","last_became_active":"2024-09-21T05:59:37.445165+0000","last_became_peered":"2024-09-21T05:59:37.445165+0000","last_unstale":"2024-09-21T05:59:37.450806+0000","last_undegraded":"2024-09-21T05:59:37.450806+0000","last_fullsized":"2024-09-21T05:59:37.450806+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:42.959574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429475+0000","last_change":"2024-09-21T05:59:37.429475+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429475+0000","last_undegraded":"2024-09-21T05:59:37.429475+0000","last_fullsized":"2024-09-21T05:59:37.429475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.452222+0000","last_change":"2024-09-21T05:59:37.446726+0000","last_active":"2024-09-21T05:59:37.452222+0000","last_peered":"2024-09-21T05:59:37.452222+0000","last_clean":"2024-09-21T05:59:37.452222+0000","last_became_active":"2024-09-21T05:59:37.446299+0000","last_became_peered":"2024-09-21T05:59:37.446299+0000","last_unstale":"2024-09-21T05:59:37.452222+0000","last_undegraded":"2024-09-21T05:59:37.452222+0000","last_fullsized":"2024-09-21T05:59:37.452222+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:31.718340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429164+0000","last_change":"2024-09-21T05:59:37.429164+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429164+0000","last_undegraded":"2024-09-21T05:59:37.429164+0000","last_fullsized":"2024-09-21T05:59:37.429164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837548+0000","last_change":"2024-09-21T05:59:37.446929+0000","last_active":"2024-09-21T05:59:39.837548+0000","last_peered":"2024-09-21T05:59:39.837548+0000","last_clean":"2024-09-21T05:59:39.837548+0000","last_became_active":"2024-09-21T05:59:37.446552+0000","last_became_peered":"2024-09-21T05:59:37.446552+0000","last_unstale":"2024-09-21T05:59:39.837548+0000","last_undegraded":"2024-09-21T05:59:39.837548+0000","last_fullsized":"2024-09-21T05:59:39.837548+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:54.236652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425900+0000","last_change":"2024-09-21T05:59:37.425900+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425900+0000","last_undegraded":"2024-09-21T05:59:37.425900+0000","last_fullsized":"2024-09-21T05:59:37.425900+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450041+0000","last_change":"2024-09-21T05:59:37.446310+0000","last_active":"2024-09-21T05:59:37.450041+0000","last_peered":"2024-09-21T05:59:37.450041+0000","last_clean":"2024-09-21T05:59:37.450041+0000","last_became_active":"2024-09-21T05:59:37.445953+0000","last_became_peered":"2024-09-21T05:59:37.445953+0000","last_unstale":"2024-09-21T05:59:37.450041+0000","last_undegraded":"2024-09-21T05:59:37.450041+0000","last_fullsized":"2024-09-21T05:59:37.450041+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:42.959574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425968+0000","last_change":"2024-09-21T05:59:37.425968+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425968+0000","last_undegraded":"2024-09-21T05:59:37.425968+0000","last_fullsized":"2024-09-21T05:59:37.425968+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.439199+0000","last_change":"2024-09-21T05:59:37.435447+0000","last_active":"2024-09-21T05:59:37.439199+0000","last_peered":"2024-09-21T05:59:37.439199+0000","last_clean":"2024-09-21T05:59:37.439199+0000","last_became_active":"2024-09-21T05:59:37.435178+0000","last_became_peered":"2024-09-21T05:59:37.435178+0000","last_unstale":"2024-09-21T05:59:37.439199+0000","last_undegraded":"2024-09-21T05:59:37.439199+0000","last_fullsized":"2024-09-21T05:59:37.439199+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:24.390872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424041+0000","last_change":"2024-09-21T05:59:37.424041+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424041+0000","last_undegraded":"2024-09-21T05:59:37.424041+0000","last_fullsized":"2024-09-21T05:59:37.424041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837563+0000","last_change":"2024-09-21T05:59:37.441729+0000","last_active":"2024-09-21T05:59:39.837563+0000","last_peered":"2024-09-21T05:59:39.837563+0000","last_clean":"2024-09-21T05:59:39.837563+0000","last_became_active":"2024-09-21T05:59:37.441370+0000","last_became_peered":"2024-09-21T05:59:37.441370+0000","last_unstale":"2024-09-21T05:59:39.837563+0000","last_undegraded":"2024-09-21T05:59:39.837563+0000","last_fullsized":"2024-09-21T05:59:39.837563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:28.903549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.422407+0000","last_change":"2024-09-21T05:59:37.422407+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.422407+0000","last_undegraded":"2024-09-21T05:59:37.422407+0000","last_fullsized":"2024-09-21T05:59:37.422407+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451290+0000","last_change":"2024-09-21T05:59:37.442907+0000","last_active":"2024-09-21T05:59:37.451290+0000","last_peered":"2024-09-21T05:59:37.451290+0000","last_clean":"2024-09-21T05:59:37.451290+0000","last_became_active":"2024-09-21T05:59:37.439722+0000","last_became_peered":"2024-09-21T05:59:37.439722+0000","last_unstale":"2024-09-21T05:59:37.451290+0000","last_undegraded":"2024-09-21T05:59:37.451290+0000","last_fullsized":"2024-09-21T05:59:37.451290+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.103990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430505+0000","last_change":"2024-09-21T05:59:37.430505+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430505+0000","last_undegraded":"2024-09-21T05:59:37.430505+0000","last_fullsized":"2024-09-21T05:59:37.430505+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.440071+0000","last_change":"2024-09-21T05:59:37.435112+0000","last_active":"2024-09-21T05:59:37.440071+0000","last_peered":"2024-09-21T05:59:37.440071+0000","last_clean":"2024-09-21T05:59:37.440071+0000","last_became_active":"2024-09-21T05:59:37.434789+0000","last_became_peered":"2024-09-21T05:59:37.434789+0000","last_unstale":"2024-09-21T05:59:37.440071+0000","last_undegraded":"2024-09-21T05:59:37.440071+0000","last_fullsized":"2024-09-21T05:59:37.440071+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:46.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":148,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.436248+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:59:37.436248+0000","last_peered":"2024-09-21T05:59:37.436248+0000","last_clean":"2024-09-21T05:59:37.436248+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:59:37.436248+0000","last_undegraded":"2024-09-21T05:59:37.436248+0000","last_fullsized":"2024-09-21T05:59:37.436248+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1077248,"data_stored":779961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":471040,"data_stored":183326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31664,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340176,"statfs":{"total":96636764160,"available":96604340224,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31664,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340176,"statfs":{"total":96636764160,"available":96604340224,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35824,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336016,"statfs":{"total":96636764160,"available":96600080384,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36400,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335440,"statfs":{"total":96636764160,"available":96599490560,"internally_reserved":0,"allocated":1077248,"data_stored":779961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40008,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331832,"statfs":{"total":96636764160,"available":96595795968,"internally_reserved":0,"allocated":512000,"data_stored":202738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:59:43.063 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:59:43.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.259+0000 7f7bcda00640 1 -- 172.21.15.50:0/1123117543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc80749a0 msgr2=0x7f7bc8074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.259+0000 7f7bcda00640 1 --2- 172.21.15.50:0/1123117543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc80749a0 0x7f7bc8074db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80cfc90 tx=0x7f7bb801c8c0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.259+0000 7f7bcda00640 1 -- 172.21.15.50:0/1123117543 shutdown_connections 2024-09-21T05:59:43.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.259+0000 7f7bcda00640 1 --2- 172.21.15.50:0/1123117543 >> v2:172.21.15.88:3300/0 conn(0x7f7bc80b3340 0x7f7bc80b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.259+0000 7f7bcda00640 1 --2- 172.21.15.50:0/1123117543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc80749a0 0x7f7bc8074db0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.259+0000 7f7bcda00640 1 -- 172.21.15.50:0/1123117543 >> 172.21.15.50:0/1123117543 conn(0x7f7bc80b2040 msgr2=0x7f7bc80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:43.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.259+0000 7f7bcda00640 1 -- 172.21.15.50:0/1123117543 shutdown_connections 2024-09-21T05:59:43.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.259+0000 7f7bcda00640 1 -- 172.21.15.50:0/1123117543 wait complete. 2024-09-21T05:59:43.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.259+0000 7f7bcda00640 1 Processor -- start 2024-09-21T05:59:43.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.259+0000 7f7bcda00640 1 -- start start 2024-09-21T05:59:43.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bcda00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bc80749a0 0x7f7bc80ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:43.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bcda00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7bc80b3340 0x7f7bc812d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:43.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bcda00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc80cefa0 0x7f7bc812d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:43.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bcda00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8169d20 con 0x7f7bc80cefa0 2024-09-21T05:59:43.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bcda00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8169ba0 con 0x7f7bc80749a0 2024-09-21T05:59:43.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bcda00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc8169ea0 con 0x7f7bc80b3340 2024-09-21T05:59:43.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bc7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc80cefa0 0x7f7bc812d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:43.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bc7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc80cefa0 0x7f7bc812d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46648/0 (socket says 172.21.15.50:46648) 2024-09-21T05:59:43.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bc7400640 1 -- 172.21.15.50:0/875694778 learned_addr learned my addr 172.21.15.50:0/875694778 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:43.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bc6a00640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bc80749a0 0x7f7bc80ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bc6000640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7bc80b3340 0x7f7bc812d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bc7400640 1 -- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7bc80b3340 msgr2=0x7f7bc812d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bc7400640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7bc80b3340 0x7f7bc812d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bc7400640 1 -- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bc80749a0 msgr2=0x7f7bc80ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bc7400640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bc80749a0 0x7f7bc80ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bc7400640 1 -- 172.21.15.50:0/875694778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc81fd290 con 0x7f7bc80cefa0 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bc6a00640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bc80749a0 0x7f7bc80ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bc7400640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc80cefa0 0x7f7bc812d9d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000cab0 tx=0x7f7bc000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bbea00640 1 -- 172.21.15.50:0/875694778 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc000ae20 con 0x7f7bc80cefa0 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bbea00640 1 -- 172.21.15.50:0/875694778 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bc000f560 con 0x7f7bc80cefa0 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bbea00640 1 -- 172.21.15.50:0/875694778 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc000f9b0 con 0x7f7bc80cefa0 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bcda00640 1 -- 172.21.15.50:0/875694778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc80ccd90 con 0x7f7bc80cefa0 2024-09-21T05:59:43.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bcda00640 1 -- 172.21.15.50:0/875694778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc80cd570 con 0x7f7bc80cefa0 2024-09-21T05:59:43.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.263+0000 7f7bcda00640 1 -- 172.21.15.50:0/875694778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc81f6880 con 0x7f7bc80cefa0 2024-09-21T05:59:43.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.267+0000 7f7bbea00640 1 -- 172.21.15.50:0/875694778 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f7bc000fb50 con 0x7f7bc80cefa0 2024-09-21T05:59:43.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.267+0000 7f7bbea00640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ba8073560 0x7f7ba8075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:43.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.267+0000 7f7bbea00640 1 -- 172.21.15.50:0/875694778 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f7bc00953a0 con 0x7f7bc80cefa0 2024-09-21T05:59:43.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.267+0000 7f7bc6a00640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ba8073560 0x7f7ba8075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.267+0000 7f7bc6a00640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ba8073560 0x7f7ba8075a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80af000 tx=0x7f7bb801b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:43.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.271+0000 7f7bbea00640 1 -- 172.21.15.50:0/875694778 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7bc81f6880 con 0x7f7bc80cefa0 2024-09-21T05:59:43.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.447+0000 7f7bcda00640 1 -- 172.21.15.50:0/875694778 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7bc806b540 con 0x7f7ba8073560 2024-09-21T05:59:43.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.451+0000 7f7bbea00640 1 -- 172.21.15.50:0/875694778 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246916 (secure 0 0 0) 0x7f7bb80040c0 con 0x7f7ba8073560 2024-09-21T05:59:43.457 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:43.458 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:59:43.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.455+0000 7f7bcda00640 1 -- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ba8073560 msgr2=0x7f7ba8075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.455+0000 7f7bcda00640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ba8073560 0x7f7ba8075a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80af000 tx=0x7f7bb801b100 comp rx=0 tx=0).stop 2024-09-21T05:59:43.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.455+0000 7f7bcda00640 1 -- 172.21.15.50:0/875694778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc80cefa0 msgr2=0x7f7bc812d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.455+0000 7f7bcda00640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc80cefa0 0x7f7bc812d9d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000cab0 tx=0x7f7bc000cf70 comp rx=0 tx=0).stop 2024-09-21T05:59:43.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.455+0000 7f7bcda00640 1 -- 172.21.15.50:0/875694778 shutdown_connections 2024-09-21T05:59:43.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.455+0000 7f7bcda00640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ba8073560 0x7f7ba8075a70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.455+0000 7f7bcda00640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc80cefa0 0x7f7bc812d9d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.455+0000 7f7bcda00640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7bc80b3340 0x7f7bc812d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.455+0000 7f7bcda00640 1 --2- 172.21.15.50:0/875694778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bc80749a0 0x7f7bc80ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.455+0000 7f7bcda00640 1 -- 172.21.15.50:0/875694778 >> 172.21.15.50:0/875694778 conn(0x7f7bc80b2040 msgr2=0x7f7bc815b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:43.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.455+0000 7f7bcda00640 1 -- 172.21.15.50:0/875694778 shutdown_connections 2024-09-21T05:59:43.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.455+0000 7f7bcda00640 1 -- 172.21.15.50:0/875694778 wait complete. 2024-09-21T05:59:43.479 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-21T05:59:42.934565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271128,"kb_used_data":4952,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754703592,"statfs":{"total":773094113280,"available":772816478208,"internally_reserved":0,"allocated":5070848,"data_stored":2704358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001455"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430439+0000","last_change":"2024-09-21T05:59:37.430439+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430439+0000","last_undegraded":"2024-09-21T05:59:37.430439+0000","last_fullsized":"2024-09-21T05:59:37.430439+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424624+0000","last_change":"2024-09-21T05:59:37.424624+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424624+0000","last_undegraded":"2024-09-21T05:59:37.424624+0000","last_fullsized":"2024-09-21T05:59:37.424624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424101+0000","last_change":"2024-09-21T05:59:37.424101+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424101+0000","last_undegraded":"2024-09-21T05:59:37.424101+0000","last_fullsized":"2024-09-21T05:59:37.424101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431709+0000","last_change":"2024-09-21T05:59:37.431709+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431709+0000","last_undegraded":"2024-09-21T05:59:37.431709+0000","last_fullsized":"2024-09-21T05:59:37.431709+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423190+0000","last_change":"2024-09-21T05:59:37.423190+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423190+0000","last_undegraded":"2024-09-21T05:59:37.423190+0000","last_fullsized":"2024-09-21T05:59:37.423190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.428041+0000","last_change":"2024-09-21T05:59:37.428041+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.428041+0000","last_undegraded":"2024-09-21T05:59:37.428041+0000","last_fullsized":"2024-09-21T05:59:37.428041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430128+0000","last_change":"2024-09-21T05:59:37.430128+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430128+0000","last_undegraded":"2024-09-21T05:59:37.430128+0000","last_fullsized":"2024-09-21T05:59:37.430128+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433589+0000","last_change":"2024-09-21T05:59:37.433589+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433589+0000","last_undegraded":"2024-09-21T05:59:37.433589+0000","last_fullsized":"2024-09-21T05:59:37.433589+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430329+0000","last_change":"2024-09-21T05:59:37.430329+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430329+0000","last_undegraded":"2024-09-21T05:59:37.430329+0000","last_fullsized":"2024-09-21T05:59:37.430329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431028+0000","last_change":"2024-09-21T05:59:37.431028+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431028+0000","last_undegraded":"2024-09-21T05:59:37.431028+0000","last_fullsized":"2024-09-21T05:59:37.431028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429854+0000","last_change":"2024-09-21T05:59:37.429854+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429854+0000","last_undegraded":"2024-09-21T05:59:37.429854+0000","last_fullsized":"2024-09-21T05:59:37.429854+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423767+0000","last_change":"2024-09-21T05:59:37.423767+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423767+0000","last_undegraded":"2024-09-21T05:59:37.423767+0000","last_fullsized":"2024-09-21T05:59:37.423767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.435082+0000","last_change":"2024-09-21T05:59:37.435082+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.435082+0000","last_undegraded":"2024-09-21T05:59:37.435082+0000","last_fullsized":"2024-09-21T05:59:37.435082+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432295+0000","last_change":"2024-09-21T05:59:37.432295+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432295+0000","last_undegraded":"2024-09-21T05:59:37.432295+0000","last_fullsized":"2024-09-21T05:59:37.432295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.427649+0000","last_change":"2024-09-21T05:59:37.427649+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.427649+0000","last_undegraded":"2024-09-21T05:59:37.427649+0000","last_fullsized":"2024-09-21T05:59:37.427649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430585+0000","last_change":"2024-09-21T05:59:37.430585+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430585+0000","last_undegraded":"2024-09-21T05:59:37.430585+0000","last_fullsized":"2024-09-21T05:59:37.430585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424708+0000","last_change":"2024-09-21T05:59:37.424708+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424708+0000","last_undegraded":"2024-09-21T05:59:37.424708+0000","last_fullsized":"2024-09-21T05:59:37.424708+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425629+0000","last_change":"2024-09-21T05:59:37.425629+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425629+0000","last_undegraded":"2024-09-21T05:59:37.425629+0000","last_fullsized":"2024-09-21T05:59:37.425629+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432796+0000","last_change":"2024-09-21T05:59:37.432796+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432796+0000","last_undegraded":"2024-09-21T05:59:37.432796+0000","last_fullsized":"2024-09-21T05:59:37.432796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.434597+0000","last_change":"2024-09-21T05:59:37.434597+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.434597+0000","last_undegraded":"2024-09-21T05:59:37.434597+0000","last_fullsized":"2024-09-21T05:59:37.434597+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.427426+0000","last_change":"2024-09-21T05:59:37.427426+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.427426+0000","last_undegraded":"2024-09-21T05:59:37.427426+0000","last_fullsized":"2024-09-21T05:59:37.427426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429310+0000","last_change":"2024-09-21T05:59:37.429310+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429310+0000","last_undegraded":"2024-09-21T05:59:37.429310+0000","last_fullsized":"2024-09-21T05:59:37.429310+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.434559+0000","last_change":"2024-09-21T05:59:37.434559+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.434559+0000","last_undegraded":"2024-09-21T05:59:37.434559+0000","last_fullsized":"2024-09-21T05:59:37.434559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429894+0000","last_change":"2024-09-21T05:59:37.429894+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429894+0000","last_undegraded":"2024-09-21T05:59:37.429894+0000","last_fullsized":"2024-09-21T05:59:37.429894+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433207+0000","last_change":"2024-09-21T05:59:37.433207+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433207+0000","last_undegraded":"2024-09-21T05:59:37.433207+0000","last_fullsized":"2024-09-21T05:59:37.433207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.439276+0000","last_change":"2024-09-21T05:59:37.439276+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.439276+0000","last_undegraded":"2024-09-21T05:59:37.439276+0000","last_fullsized":"2024-09-21T05:59:37.439276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423541+0000","last_change":"2024-09-21T05:59:37.423541+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423541+0000","last_undegraded":"2024-09-21T05:59:37.423541+0000","last_fullsized":"2024-09-21T05:59:37.423541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423216+0000","last_change":"2024-09-21T05:59:37.423216+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423216+0000","last_undegraded":"2024-09-21T05:59:37.423216+0000","last_fullsized":"2024-09-21T05:59:37.423216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431328+0000","last_change":"2024-09-21T05:59:37.431328+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431328+0000","last_undegraded":"2024-09-21T05:59:37.431328+0000","last_fullsized":"2024-09-21T05:59:37.431328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423470+0000","last_change":"2024-09-21T05:59:37.423470+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423470+0000","last_undegraded":"2024-09-21T05:59:37.423470+0000","last_fullsized":"2024-09-21T05:59:37.423470+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.426152+0000","last_change":"2024-09-21T05:59:37.426152+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.426152+0000","last_undegraded":"2024-09-21T05:59:37.426152+0000","last_fullsized":"2024-09-21T05:59:37.426152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.438506+0000","last_change":"2024-09-21T05:59:37.438506+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.438506+0000","last_undegraded":"2024-09-21T05:59:37.438506+0000","last_fullsized":"2024-09-21T05:59:37.438506+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837865+0000","last_change":"2024-09-21T05:59:38.462061+0000","last_active":"2024-09-21T05:59:39.837865+0000","last_peered":"2024-09-21T05:59:39.837865+0000","last_clean":"2024-09-21T05:59:39.837865+0000","last_became_active":"2024-09-21T05:59:38.461740+0000","last_became_peered":"2024-09-21T05:59:38.461740+0000","last_unstale":"2024-09-21T05:59:39.837865+0000","last_undegraded":"2024-09-21T05:59:39.837865+0000","last_fullsized":"2024-09-21T05:59:39.837865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:17.985198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451150+0000","last_change":"2024-09-21T05:59:37.447751+0000","last_active":"2024-09-21T05:59:37.451150+0000","last_peered":"2024-09-21T05:59:37.451150+0000","last_clean":"2024-09-21T05:59:37.451150+0000","last_became_active":"2024-09-21T05:59:37.447392+0000","last_became_peered":"2024-09-21T05:59:37.447392+0000","last_unstale":"2024-09-21T05:59:37.451150+0000","last_undegraded":"2024-09-21T05:59:37.451150+0000","last_fullsized":"2024-09-21T05:59:37.451150+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:17.297839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432540+0000","last_change":"2024-09-21T05:59:37.432540+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432540+0000","last_undegraded":"2024-09-21T05:59:37.432540+0000","last_fullsized":"2024-09-21T05:59:37.432540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.444761+0000","last_change":"2024-09-21T05:59:37.443312+0000","last_active":"2024-09-21T05:59:37.444761+0000","last_peered":"2024-09-21T05:59:37.444761+0000","last_clean":"2024-09-21T05:59:37.444761+0000","last_became_active":"2024-09-21T05:59:37.442457+0000","last_became_peered":"2024-09-21T05:59:37.442457+0000","last_unstale":"2024-09-21T05:59:37.444761+0000","last_undegraded":"2024-09-21T05:59:37.444761+0000","last_fullsized":"2024-09-21T05:59:37.444761+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:54.217575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.422517+0000","last_change":"2024-09-21T05:59:37.422517+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.422517+0000","last_undegraded":"2024-09-21T05:59:37.422517+0000","last_fullsized":"2024-09-21T05:59:37.422517+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837882+0000","last_change":"2024-09-21T05:59:37.440271+0000","last_active":"2024-09-21T05:59:39.837882+0000","last_peered":"2024-09-21T05:59:39.837882+0000","last_clean":"2024-09-21T05:59:39.837882+0000","last_became_active":"2024-09-21T05:59:37.439909+0000","last_became_peered":"2024-09-21T05:59:37.439909+0000","last_unstale":"2024-09-21T05:59:39.837882+0000","last_undegraded":"2024-09-21T05:59:39.837882+0000","last_fullsized":"2024-09-21T05:59:39.837882+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:15.657492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837975+0000","last_change":"2024-09-21T05:59:38.444285+0000","last_active":"2024-09-21T05:59:39.837975+0000","last_peered":"2024-09-21T05:59:39.837975+0000","last_clean":"2024-09-21T05:59:39.837975+0000","last_became_active":"2024-09-21T05:59:38.443952+0000","last_became_peered":"2024-09-21T05:59:38.443952+0000","last_unstale":"2024-09-21T05:59:39.837975+0000","last_undegraded":"2024-09-21T05:59:39.837975+0000","last_fullsized":"2024-09-21T05:59:39.837975+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:31.815943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.449829+0000","last_change":"2024-09-21T05:59:37.443607+0000","last_active":"2024-09-21T05:59:37.449829+0000","last_peered":"2024-09-21T05:59:37.449829+0000","last_clean":"2024-09-21T05:59:37.449829+0000","last_became_active":"2024-09-21T05:59:37.443262+0000","last_became_peered":"2024-09-21T05:59:37.443262+0000","last_unstale":"2024-09-21T05:59:37.449829+0000","last_undegraded":"2024-09-21T05:59:37.449829+0000","last_fullsized":"2024-09-21T05:59:37.449829+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:21.369884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.422840+0000","last_change":"2024-09-21T05:59:37.422840+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.422840+0000","last_undegraded":"2024-09-21T05:59:37.422840+0000","last_fullsized":"2024-09-21T05:59:37.422840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451244+0000","last_change":"2024-09-21T05:59:37.447920+0000","last_active":"2024-09-21T05:59:37.451244+0000","last_peered":"2024-09-21T05:59:37.451244+0000","last_clean":"2024-09-21T05:59:37.451244+0000","last_became_active":"2024-09-21T05:59:37.447240+0000","last_became_peered":"2024-09-21T05:59:37.447240+0000","last_unstale":"2024-09-21T05:59:37.451244+0000","last_undegraded":"2024-09-21T05:59:37.451244+0000","last_fullsized":"2024-09-21T05:59:37.451244+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:52.027735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432049+0000","last_change":"2024-09-21T05:59:37.432049+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432049+0000","last_undegraded":"2024-09-21T05:59:37.432049+0000","last_fullsized":"2024-09-21T05:59:37.432049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.438146+0000","last_change":"2024-09-21T05:59:37.429073+0000","last_active":"2024-09-21T05:59:37.438146+0000","last_peered":"2024-09-21T05:59:37.438146+0000","last_clean":"2024-09-21T05:59:37.438146+0000","last_became_active":"2024-09-21T05:59:37.428733+0000","last_became_peered":"2024-09-21T05:59:37.428733+0000","last_unstale":"2024-09-21T05:59:37.438146+0000","last_undegraded":"2024-09-21T05:59:37.438146+0000","last_fullsized":"2024-09-21T05:59:37.438146+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:39.275851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433120+0000","last_change":"2024-09-21T05:59:37.433120+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433120+0000","last_undegraded":"2024-09-21T05:59:37.433120+0000","last_fullsized":"2024-09-21T05:59:37.433120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.440195+0000","last_change":"2024-09-21T05:59:37.437489+0000","last_active":"2024-09-21T05:59:37.440195+0000","last_peered":"2024-09-21T05:59:37.440195+0000","last_clean":"2024-09-21T05:59:37.440195+0000","last_became_active":"2024-09-21T05:59:37.437170+0000","last_became_peered":"2024-09-21T05:59:37.437170+0000","last_unstale":"2024-09-21T05:59:37.440195+0000","last_undegraded":"2024-09-21T05:59:37.440195+0000","last_fullsized":"2024-09-21T05:59:37.440195+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:42.959574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.439321+0000","last_change":"2024-09-21T05:59:37.439321+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.439321+0000","last_undegraded":"2024-09-21T05:59:37.439321+0000","last_fullsized":"2024-09-21T05:59:37.439321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451906+0000","last_change":"2024-09-21T05:59:37.449577+0000","last_active":"2024-09-21T05:59:37.451906+0000","last_peered":"2024-09-21T05:59:37.451906+0000","last_clean":"2024-09-21T05:59:37.451906+0000","last_became_active":"2024-09-21T05:59:37.449255+0000","last_became_peered":"2024-09-21T05:59:37.449255+0000","last_unstale":"2024-09-21T05:59:37.451906+0000","last_undegraded":"2024-09-21T05:59:37.451906+0000","last_fullsized":"2024-09-21T05:59:37.451906+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:39.275851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425083+0000","last_change":"2024-09-21T05:59:37.425083+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425083+0000","last_undegraded":"2024-09-21T05:59:37.425083+0000","last_fullsized":"2024-09-21T05:59:37.425083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.444043+0000","last_change":"2024-09-21T05:59:37.440683+0000","last_active":"2024-09-21T05:59:37.444043+0000","last_peered":"2024-09-21T05:59:37.444043+0000","last_clean":"2024-09-21T05:59:37.444043+0000","last_became_active":"2024-09-21T05:59:37.440336+0000","last_became_peered":"2024-09-21T05:59:37.440336+0000","last_unstale":"2024-09-21T05:59:37.444043+0000","last_undegraded":"2024-09-21T05:59:37.444043+0000","last_fullsized":"2024-09-21T05:59:37.444043+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:13.571664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.428994+0000","last_change":"2024-09-21T05:59:37.428994+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.428994+0000","last_undegraded":"2024-09-21T05:59:37.428994+0000","last_fullsized":"2024-09-21T05:59:37.428994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451867+0000","last_change":"2024-09-21T05:59:37.448714+0000","last_active":"2024-09-21T05:59:37.451867+0000","last_peered":"2024-09-21T05:59:37.451867+0000","last_clean":"2024-09-21T05:59:37.451867+0000","last_became_active":"2024-09-21T05:59:37.448310+0000","last_became_peered":"2024-09-21T05:59:37.448310+0000","last_unstale":"2024-09-21T05:59:37.451867+0000","last_undegraded":"2024-09-21T05:59:37.451867+0000","last_fullsized":"2024-09-21T05:59:37.451867+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:21.097323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.426543+0000","last_change":"2024-09-21T05:59:37.426543+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.426543+0000","last_undegraded":"2024-09-21T05:59:37.426543+0000","last_fullsized":"2024-09-21T05:59:37.426543+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450006+0000","last_change":"2024-09-21T05:59:37.445812+0000","last_active":"2024-09-21T05:59:37.450006+0000","last_peered":"2024-09-21T05:59:37.450006+0000","last_clean":"2024-09-21T05:59:37.450006+0000","last_became_active":"2024-09-21T05:59:37.445496+0000","last_became_peered":"2024-09-21T05:59:37.445496+0000","last_unstale":"2024-09-21T05:59:37.450006+0000","last_undegraded":"2024-09-21T05:59:37.450006+0000","last_fullsized":"2024-09-21T05:59:37.450006+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:46.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429754+0000","last_change":"2024-09-21T05:59:37.429754+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429754+0000","last_undegraded":"2024-09-21T05:59:37.429754+0000","last_fullsized":"2024-09-21T05:59:37.429754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.443714+0000","last_change":"2024-09-21T05:59:37.440777+0000","last_active":"2024-09-21T05:59:37.443714+0000","last_peered":"2024-09-21T05:59:37.443714+0000","last_clean":"2024-09-21T05:59:37.443714+0000","last_became_active":"2024-09-21T05:59:37.440460+0000","last_became_peered":"2024-09-21T05:59:37.440460+0000","last_unstale":"2024-09-21T05:59:37.443714+0000","last_undegraded":"2024-09-21T05:59:37.443714+0000","last_fullsized":"2024-09-21T05:59:37.443714+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:03.466004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430964+0000","last_change":"2024-09-21T05:59:37.430964+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430964+0000","last_undegraded":"2024-09-21T05:59:37.430964+0000","last_fullsized":"2024-09-21T05:59:37.430964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451384+0000","last_change":"2024-09-21T05:59:37.447984+0000","last_active":"2024-09-21T05:59:37.451384+0000","last_peered":"2024-09-21T05:59:37.451384+0000","last_clean":"2024-09-21T05:59:37.451384+0000","last_became_active":"2024-09-21T05:59:37.447685+0000","last_became_peered":"2024-09-21T05:59:37.447685+0000","last_unstale":"2024-09-21T05:59:37.451384+0000","last_undegraded":"2024-09-21T05:59:37.451384+0000","last_fullsized":"2024-09-21T05:59:37.451384+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:36.267884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431534+0000","last_change":"2024-09-21T05:59:37.431534+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431534+0000","last_undegraded":"2024-09-21T05:59:37.431534+0000","last_fullsized":"2024-09-21T05:59:37.431534+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.449770+0000","last_change":"2024-09-21T05:59:37.443723+0000","last_active":"2024-09-21T05:59:37.449770+0000","last_peered":"2024-09-21T05:59:37.449770+0000","last_clean":"2024-09-21T05:59:37.449770+0000","last_became_active":"2024-09-21T05:59:37.443383+0000","last_became_peered":"2024-09-21T05:59:37.443383+0000","last_unstale":"2024-09-21T05:59:37.449770+0000","last_undegraded":"2024-09-21T05:59:37.449770+0000","last_fullsized":"2024-09-21T05:59:37.449770+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.689460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.426456+0000","last_change":"2024-09-21T05:59:37.426456+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.426456+0000","last_undegraded":"2024-09-21T05:59:37.426456+0000","last_fullsized":"2024-09-21T05:59:37.426456+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450786+0000","last_change":"2024-09-21T05:59:37.444802+0000","last_active":"2024-09-21T05:59:37.450786+0000","last_peered":"2024-09-21T05:59:37.450786+0000","last_clean":"2024-09-21T05:59:37.450786+0000","last_became_active":"2024-09-21T05:59:37.444457+0000","last_became_peered":"2024-09-21T05:59:37.444457+0000","last_unstale":"2024-09-21T05:59:37.450786+0000","last_undegraded":"2024-09-21T05:59:37.450786+0000","last_fullsized":"2024-09-21T05:59:37.450786+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:46.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.428769+0000","last_change":"2024-09-21T05:59:37.428769+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.428769+0000","last_undegraded":"2024-09-21T05:59:37.428769+0000","last_fullsized":"2024-09-21T05:59:37.428769+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451326+0000","last_change":"2024-09-21T05:59:37.444089+0000","last_active":"2024-09-21T05:59:37.451326+0000","last_peered":"2024-09-21T05:59:37.451326+0000","last_clean":"2024-09-21T05:59:37.451326+0000","last_became_active":"2024-09-21T05:59:37.443438+0000","last_became_peered":"2024-09-21T05:59:37.443438+0000","last_unstale":"2024-09-21T05:59:37.451326+0000","last_undegraded":"2024-09-21T05:59:37.451326+0000","last_fullsized":"2024-09-21T05:59:37.451326+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:34.733846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.435700+0000","last_change":"2024-09-21T05:59:37.435700+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.435700+0000","last_undegraded":"2024-09-21T05:59:37.435700+0000","last_fullsized":"2024-09-21T05:59:37.435700+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.447853+0000","last_change":"2024-09-21T05:59:37.445284+0000","last_active":"2024-09-21T05:59:37.447853+0000","last_peered":"2024-09-21T05:59:37.447853+0000","last_clean":"2024-09-21T05:59:37.447853+0000","last_became_active":"2024-09-21T05:59:37.444912+0000","last_became_peered":"2024-09-21T05:59:37.444912+0000","last_unstale":"2024-09-21T05:59:37.447853+0000","last_undegraded":"2024-09-21T05:59:37.447853+0000","last_fullsized":"2024-09-21T05:59:37.447853+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:20.236897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429835+0000","last_change":"2024-09-21T05:59:37.429835+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429835+0000","last_undegraded":"2024-09-21T05:59:37.429835+0000","last_fullsized":"2024-09-21T05:59:37.429835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.439195+0000","last_change":"2024-09-21T05:59:37.435640+0000","last_active":"2024-09-21T05:59:37.439195+0000","last_peered":"2024-09-21T05:59:37.439195+0000","last_clean":"2024-09-21T05:59:37.439195+0000","last_became_active":"2024-09-21T05:59:37.435202+0000","last_became_peered":"2024-09-21T05:59:37.435202+0000","last_unstale":"2024-09-21T05:59:37.439195+0000","last_undegraded":"2024-09-21T05:59:37.439195+0000","last_fullsized":"2024-09-21T05:59:37.439195+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:49.312472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431533+0000","last_change":"2024-09-21T05:59:37.431533+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431533+0000","last_undegraded":"2024-09-21T05:59:37.431533+0000","last_fullsized":"2024-09-21T05:59:37.431533+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.439961+0000","last_change":"2024-09-21T05:59:37.432827+0000","last_active":"2024-09-21T05:59:37.439961+0000","last_peered":"2024-09-21T05:59:37.439961+0000","last_clean":"2024-09-21T05:59:37.439961+0000","last_became_active":"2024-09-21T05:59:37.432500+0000","last_became_peered":"2024-09-21T05:59:37.432500+0000","last_unstale":"2024-09-21T05:59:37.439961+0000","last_undegraded":"2024-09-21T05:59:37.439961+0000","last_fullsized":"2024-09-21T05:59:37.439961+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.689460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433437+0000","last_change":"2024-09-21T05:59:37.433437+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433437+0000","last_undegraded":"2024-09-21T05:59:37.433437+0000","last_fullsized":"2024-09-21T05:59:37.433437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450224+0000","last_change":"2024-09-21T05:59:37.441674+0000","last_active":"2024-09-21T05:59:37.450224+0000","last_peered":"2024-09-21T05:59:37.450224+0000","last_clean":"2024-09-21T05:59:37.450224+0000","last_became_active":"2024-09-21T05:59:37.441330+0000","last_became_peered":"2024-09-21T05:59:37.441330+0000","last_unstale":"2024-09-21T05:59:37.450224+0000","last_undegraded":"2024-09-21T05:59:37.450224+0000","last_fullsized":"2024-09-21T05:59:37.450224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.968036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.427110+0000","last_change":"2024-09-21T05:59:37.427110+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.427110+0000","last_undegraded":"2024-09-21T05:59:37.427110+0000","last_fullsized":"2024-09-21T05:59:37.427110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450915+0000","last_change":"2024-09-21T05:59:37.447596+0000","last_active":"2024-09-21T05:59:37.450915+0000","last_peered":"2024-09-21T05:59:37.450915+0000","last_clean":"2024-09-21T05:59:37.450915+0000","last_became_active":"2024-09-21T05:59:37.440489+0000","last_became_peered":"2024-09-21T05:59:37.440489+0000","last_unstale":"2024-09-21T05:59:37.450915+0000","last_undegraded":"2024-09-21T05:59:37.450915+0000","last_fullsized":"2024-09-21T05:59:37.450915+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:31.718340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837878+0000","last_change":"2024-09-21T05:59:38.435762+0000","last_active":"2024-09-21T05:59:39.837878+0000","last_peered":"2024-09-21T05:59:39.837878+0000","last_clean":"2024-09-21T05:59:39.837878+0000","last_became_active":"2024-09-21T05:59:38.434259+0000","last_became_peered":"2024-09-21T05:59:38.434259+0000","last_unstale":"2024-09-21T05:59:39.837878+0000","last_undegraded":"2024-09-21T05:59:39.837878+0000","last_fullsized":"2024-09-21T05:59:39.837878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:36.174704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.440034+0000","last_change":"2024-09-21T05:59:37.434300+0000","last_active":"2024-09-21T05:59:37.440034+0000","last_peered":"2024-09-21T05:59:37.440034+0000","last_clean":"2024-09-21T05:59:37.440034+0000","last_became_active":"2024-09-21T05:59:37.433933+0000","last_became_peered":"2024-09-21T05:59:37.433933+0000","last_unstale":"2024-09-21T05:59:37.440034+0000","last_undegraded":"2024-09-21T05:59:37.440034+0000","last_fullsized":"2024-09-21T05:59:37.440034+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.968036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432848+0000","last_change":"2024-09-21T05:59:37.432848+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432848+0000","last_undegraded":"2024-09-21T05:59:37.432848+0000","last_fullsized":"2024-09-21T05:59:37.432848+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451837+0000","last_change":"2024-09-21T05:59:37.446607+0000","last_active":"2024-09-21T05:59:37.451837+0000","last_peered":"2024-09-21T05:59:37.451837+0000","last_clean":"2024-09-21T05:59:37.451837+0000","last_became_active":"2024-09-21T05:59:37.446318+0000","last_became_peered":"2024-09-21T05:59:37.446318+0000","last_unstale":"2024-09-21T05:59:37.451837+0000","last_undegraded":"2024-09-21T05:59:37.451837+0000","last_fullsized":"2024-09-21T05:59:37.451837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:59.876514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.434548+0000","last_change":"2024-09-21T05:59:37.434548+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.434548+0000","last_undegraded":"2024-09-21T05:59:37.434548+0000","last_fullsized":"2024-09-21T05:59:37.434548+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.449888+0000","last_change":"2024-09-21T05:59:37.445107+0000","last_active":"2024-09-21T05:59:37.449888+0000","last_peered":"2024-09-21T05:59:37.449888+0000","last_clean":"2024-09-21T05:59:37.449888+0000","last_became_active":"2024-09-21T05:59:37.444746+0000","last_became_peered":"2024-09-21T05:59:37.444746+0000","last_unstale":"2024-09-21T05:59:37.449888+0000","last_undegraded":"2024-09-21T05:59:37.449888+0000","last_fullsized":"2024-09-21T05:59:37.449888+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.968036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431977+0000","last_change":"2024-09-21T05:59:37.431977+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431977+0000","last_undegraded":"2024-09-21T05:59:37.431977+0000","last_fullsized":"2024-09-21T05:59:37.431977+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450806+0000","last_change":"2024-09-21T05:59:37.445647+0000","last_active":"2024-09-21T05:59:37.450806+0000","last_peered":"2024-09-21T05:59:37.450806+0000","last_clean":"2024-09-21T05:59:37.450806+0000","last_became_active":"2024-09-21T05:59:37.445165+0000","last_became_peered":"2024-09-21T05:59:37.445165+0000","last_unstale":"2024-09-21T05:59:37.450806+0000","last_undegraded":"2024-09-21T05:59:37.450806+0000","last_fullsized":"2024-09-21T05:59:37.450806+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:42.959574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429475+0000","last_change":"2024-09-21T05:59:37.429475+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429475+0000","last_undegraded":"2024-09-21T05:59:37.429475+0000","last_fullsized":"2024-09-21T05:59:37.429475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.452222+0000","last_change":"2024-09-21T05:59:37.446726+0000","last_active":"2024-09-21T05:59:37.452222+0000","last_peered":"2024-09-21T05:59:37.452222+0000","last_clean":"2024-09-21T05:59:37.452222+0000","last_became_active":"2024-09-21T05:59:37.446299+0000","last_became_peered":"2024-09-21T05:59:37.446299+0000","last_unstale":"2024-09-21T05:59:37.452222+0000","last_undegraded":"2024-09-21T05:59:37.452222+0000","last_fullsized":"2024-09-21T05:59:37.452222+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:31.718340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429164+0000","last_change":"2024-09-21T05:59:37.429164+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429164+0000","last_undegraded":"2024-09-21T05:59:37.429164+0000","last_fullsized":"2024-09-21T05:59:37.429164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837548+0000","last_change":"2024-09-21T05:59:37.446929+0000","last_active":"2024-09-21T05:59:39.837548+0000","last_peered":"2024-09-21T05:59:39.837548+0000","last_clean":"2024-09-21T05:59:39.837548+0000","last_became_active":"2024-09-21T05:59:37.446552+0000","last_became_peered":"2024-09-21T05:59:37.446552+0000","last_unstale":"2024-09-21T05:59:39.837548+0000","last_undegraded":"2024-09-21T05:59:39.837548+0000","last_fullsized":"2024-09-21T05:59:39.837548+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:54.236652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425900+0000","last_change":"2024-09-21T05:59:37.425900+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425900+0000","last_undegraded":"2024-09-21T05:59:37.425900+0000","last_fullsized":"2024-09-21T05:59:37.425900+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450041+0000","last_change":"2024-09-21T05:59:37.446310+0000","last_active":"2024-09-21T05:59:37.450041+0000","last_peered":"2024-09-21T05:59:37.450041+0000","last_clean":"2024-09-21T05:59:37.450041+0000","last_became_active":"2024-09-21T05:59:37.445953+0000","last_became_peered":"2024-09-21T05:59:37.445953+0000","last_unstale":"2024-09-21T05:59:37.450041+0000","last_undegraded":"2024-09-21T05:59:37.450041+0000","last_fullsized":"2024-09-21T05:59:37.450041+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:42.959574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425968+0000","last_change":"2024-09-21T05:59:37.425968+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425968+0000","last_undegraded":"2024-09-21T05:59:37.425968+0000","last_fullsized":"2024-09-21T05:59:37.425968+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.439199+0000","last_change":"2024-09-21T05:59:37.435447+0000","last_active":"2024-09-21T05:59:37.439199+0000","last_peered":"2024-09-21T05:59:37.439199+0000","last_clean":"2024-09-21T05:59:37.439199+0000","last_became_active":"2024-09-21T05:59:37.435178+0000","last_became_peered":"2024-09-21T05:59:37.435178+0000","last_unstale":"2024-09-21T05:59:37.439199+0000","last_undegraded":"2024-09-21T05:59:37.439199+0000","last_fullsized":"2024-09-21T05:59:37.439199+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:24.390872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424041+0000","last_change":"2024-09-21T05:59:37.424041+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424041+0000","last_undegraded":"2024-09-21T05:59:37.424041+0000","last_fullsized":"2024-09-21T05:59:37.424041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837563+0000","last_change":"2024-09-21T05:59:37.441729+0000","last_active":"2024-09-21T05:59:39.837563+0000","last_peered":"2024-09-21T05:59:39.837563+0000","last_clean":"2024-09-21T05:59:39.837563+0000","last_became_active":"2024-09-21T05:59:37.441370+0000","last_became_peered":"2024-09-21T05:59:37.441370+0000","last_unstale":"2024-09-21T05:59:39.837563+0000","last_undegraded":"2024-09-21T05:59:39.837563+0000","last_fullsized":"2024-09-21T05:59:39.837563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:28.903549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.422407+0000","last_change":"2024-09-21T05:59:37.422407+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.422407+0000","last_undegraded":"2024-09-21T05:59:37.422407+0000","last_fullsized":"2024-09-21T05:59:37.422407+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451290+0000","last_change":"2024-09-21T05:59:37.442907+0000","last_active":"2024-09-21T05:59:37.451290+0000","last_peered":"2024-09-21T05:59:37.451290+0000","last_clean":"2024-09-21T05:59:37.451290+0000","last_became_active":"2024-09-21T05:59:37.439722+0000","last_became_peered":"2024-09-21T05:59:37.439722+0000","last_unstale":"2024-09-21T05:59:37.451290+0000","last_undegraded":"2024-09-21T05:59:37.451290+0000","last_fullsized":"2024-09-21T05:59:37.451290+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.103990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430505+0000","last_change":"2024-09-21T05:59:37.430505+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430505+0000","last_undegraded":"2024-09-21T05:59:37.430505+0000","last_fullsized":"2024-09-21T05:59:37.430505+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.440071+0000","last_change":"2024-09-21T05:59:37.435112+0000","last_active":"2024-09-21T05:59:37.440071+0000","last_peered":"2024-09-21T05:59:37.440071+0000","last_clean":"2024-09-21T05:59:37.440071+0000","last_became_active":"2024-09-21T05:59:37.434789+0000","last_became_peered":"2024-09-21T05:59:37.434789+0000","last_unstale":"2024-09-21T05:59:37.440071+0000","last_undegraded":"2024-09-21T05:59:37.440071+0000","last_fullsized":"2024-09-21T05:59:37.440071+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:46.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":148,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.436248+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:59:37.436248+0000","last_peered":"2024-09-21T05:59:37.436248+0000","last_clean":"2024-09-21T05:59:37.436248+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:59:37.436248+0000","last_undegraded":"2024-09-21T05:59:37.436248+0000","last_fullsized":"2024-09-21T05:59:37.436248+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1077248,"data_stored":779961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":471040,"data_stored":183326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31664,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340176,"statfs":{"total":96636764160,"available":96604340224,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31664,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340176,"statfs":{"total":96636764160,"available":96604340224,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35824,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336016,"statfs":{"total":96636764160,"available":96600080384,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36400,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335440,"statfs":{"total":96636764160,"available":96599490560,"internally_reserved":0,"allocated":1077248,"data_stored":779961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40008,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331832,"statfs":{"total":96636764160,"available":96595795968,"internally_reserved":0,"allocated":512000,"data_stored":202738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:59:43.483 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3f is not active+clean 2024-09-21T05:59:43.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.430439+0000', 'last_change': '2024-09-21T05:59:37.430439+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.430439+0000', 'last_undegraded': '2024-09-21T05:59:37.430439+0000', 'last_fullsized': '2024-09-21T05:59:37.430439+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:59:43.483 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3e is not active+clean 2024-09-21T05:59:43.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.424624+0000', 'last_change': '2024-09-21T05:59:37.424624+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.424624+0000', 'last_undegraded': '2024-09-21T05:59:37.424624+0000', 'last_fullsized': '2024-09-21T05:59:37.424624+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T05:59:43.483 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3d is not active+clean 2024-09-21T05:59:43.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.424101+0000', 'last_change': '2024-09-21T05:59:37.424101+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.424101+0000', 'last_undegraded': '2024-09-21T05:59:37.424101+0000', 'last_fullsized': '2024-09-21T05:59:37.424101+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T05:59:43.483 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3c is not active+clean 2024-09-21T05:59:43.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.431709+0000', 'last_change': '2024-09-21T05:59:37.431709+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.431709+0000', 'last_undegraded': '2024-09-21T05:59:37.431709+0000', 'last_fullsized': '2024-09-21T05:59:37.431709+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.484 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3b is not active+clean 2024-09-21T05:59:43.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.423190+0000', 'last_change': '2024-09-21T05:59:37.423190+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.423190+0000', 'last_undegraded': '2024-09-21T05:59:37.423190+0000', 'last_fullsized': '2024-09-21T05:59:37.423190+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T05:59:43.484 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3a is not active+clean 2024-09-21T05:59:43.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.428041+0000', 'last_change': '2024-09-21T05:59:37.428041+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.428041+0000', 'last_undegraded': '2024-09-21T05:59:37.428041+0000', 'last_fullsized': '2024-09-21T05:59:37.428041+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T05:59:43.484 INFO:tasks.ceph.ceph_manager.ceph:PG 7.39 is not active+clean 2024-09-21T05:59:43.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.430128+0000', 'last_change': '2024-09-21T05:59:37.430128+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.430128+0000', 'last_undegraded': '2024-09-21T05:59:37.430128+0000', 'last_fullsized': '2024-09-21T05:59:37.430128+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:59:43.484 INFO:tasks.ceph.ceph_manager.ceph:PG 7.38 is not active+clean 2024-09-21T05:59:43.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.433589+0000', 'last_change': '2024-09-21T05:59:37.433589+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.433589+0000', 'last_undegraded': '2024-09-21T05:59:37.433589+0000', 'last_fullsized': '2024-09-21T05:59:37.433589+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.484 INFO:tasks.ceph.ceph_manager.ceph:PG 7.37 is not active+clean 2024-09-21T05:59:43.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.430329+0000', 'last_change': '2024-09-21T05:59:37.430329+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.430329+0000', 'last_undegraded': '2024-09-21T05:59:37.430329+0000', 'last_fullsized': '2024-09-21T05:59:37.430329+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.484 INFO:tasks.ceph.ceph_manager.ceph:PG 7.36 is not active+clean 2024-09-21T05:59:43.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.431028+0000', 'last_change': '2024-09-21T05:59:37.431028+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.431028+0000', 'last_undegraded': '2024-09-21T05:59:37.431028+0000', 'last_fullsized': '2024-09-21T05:59:37.431028+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.485 INFO:tasks.ceph.ceph_manager.ceph:PG 7.35 is not active+clean 2024-09-21T05:59:43.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.429854+0000', 'last_change': '2024-09-21T05:59:37.429854+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.429854+0000', 'last_undegraded': '2024-09-21T05:59:37.429854+0000', 'last_fullsized': '2024-09-21T05:59:37.429854+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.485 INFO:tasks.ceph.ceph_manager.ceph:PG 7.34 is not active+clean 2024-09-21T05:59:43.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.423767+0000', 'last_change': '2024-09-21T05:59:37.423767+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.423767+0000', 'last_undegraded': '2024-09-21T05:59:37.423767+0000', 'last_fullsized': '2024-09-21T05:59:37.423767+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T05:59:43.485 INFO:tasks.ceph.ceph_manager.ceph:PG 7.33 is not active+clean 2024-09-21T05:59:43.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.435082+0000', 'last_change': '2024-09-21T05:59:37.435082+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.435082+0000', 'last_undegraded': '2024-09-21T05:59:37.435082+0000', 'last_fullsized': '2024-09-21T05:59:37.435082+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T05:59:43.485 INFO:tasks.ceph.ceph_manager.ceph:PG 7.32 is not active+clean 2024-09-21T05:59:43.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.432295+0000', 'last_change': '2024-09-21T05:59:37.432295+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.432295+0000', 'last_undegraded': '2024-09-21T05:59:37.432295+0000', 'last_fullsized': '2024-09-21T05:59:37.432295+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.485 INFO:tasks.ceph.ceph_manager.ceph:PG 7.31 is not active+clean 2024-09-21T05:59:43.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.427649+0000', 'last_change': '2024-09-21T05:59:37.427649+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.427649+0000', 'last_undegraded': '2024-09-21T05:59:37.427649+0000', 'last_fullsized': '2024-09-21T05:59:37.427649+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T05:59:43.485 INFO:tasks.ceph.ceph_manager.ceph:PG 7.30 is not active+clean 2024-09-21T05:59:43.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.430585+0000', 'last_change': '2024-09-21T05:59:37.430585+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.430585+0000', 'last_undegraded': '2024-09-21T05:59:37.430585+0000', 'last_fullsized': '2024-09-21T05:59:37.430585+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:59:43.486 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2f is not active+clean 2024-09-21T05:59:43.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.424708+0000', 'last_change': '2024-09-21T05:59:37.424708+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.424708+0000', 'last_undegraded': '2024-09-21T05:59:37.424708+0000', 'last_fullsized': '2024-09-21T05:59:37.424708+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T05:59:43.486 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2e is not active+clean 2024-09-21T05:59:43.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.425629+0000', 'last_change': '2024-09-21T05:59:37.425629+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.425629+0000', 'last_undegraded': '2024-09-21T05:59:37.425629+0000', 'last_fullsized': '2024-09-21T05:59:37.425629+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T05:59:43.486 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2d is not active+clean 2024-09-21T05:59:43.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.432796+0000', 'last_change': '2024-09-21T05:59:37.432796+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.432796+0000', 'last_undegraded': '2024-09-21T05:59:37.432796+0000', 'last_fullsized': '2024-09-21T05:59:37.432796+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:59:43.486 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2c is not active+clean 2024-09-21T05:59:43.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.434597+0000', 'last_change': '2024-09-21T05:59:37.434597+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.434597+0000', 'last_undegraded': '2024-09-21T05:59:37.434597+0000', 'last_fullsized': '2024-09-21T05:59:37.434597+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T05:59:43.486 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2b is not active+clean 2024-09-21T05:59:43.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.427426+0000', 'last_change': '2024-09-21T05:59:37.427426+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.427426+0000', 'last_undegraded': '2024-09-21T05:59:37.427426+0000', 'last_fullsized': '2024-09-21T05:59:37.427426+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T05:59:43.486 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2a is not active+clean 2024-09-21T05:59:43.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.429310+0000', 'last_change': '2024-09-21T05:59:37.429310+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.429310+0000', 'last_undegraded': '2024-09-21T05:59:37.429310+0000', 'last_fullsized': '2024-09-21T05:59:37.429310+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.487 INFO:tasks.ceph.ceph_manager.ceph:PG 7.29 is not active+clean 2024-09-21T05:59:43.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.434559+0000', 'last_change': '2024-09-21T05:59:37.434559+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.434559+0000', 'last_undegraded': '2024-09-21T05:59:37.434559+0000', 'last_fullsized': '2024-09-21T05:59:37.434559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:59:43.487 INFO:tasks.ceph.ceph_manager.ceph:PG 7.28 is not active+clean 2024-09-21T05:59:43.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.429894+0000', 'last_change': '2024-09-21T05:59:37.429894+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.429894+0000', 'last_undegraded': '2024-09-21T05:59:37.429894+0000', 'last_fullsized': '2024-09-21T05:59:37.429894+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:59:43.487 INFO:tasks.ceph.ceph_manager.ceph:PG 7.27 is not active+clean 2024-09-21T05:59:43.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.433207+0000', 'last_change': '2024-09-21T05:59:37.433207+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.433207+0000', 'last_undegraded': '2024-09-21T05:59:37.433207+0000', 'last_fullsized': '2024-09-21T05:59:37.433207+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:59:43.487 INFO:tasks.ceph.ceph_manager.ceph:PG 7.26 is not active+clean 2024-09-21T05:59:43.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.439276+0000', 'last_change': '2024-09-21T05:59:37.439276+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.439276+0000', 'last_undegraded': '2024-09-21T05:59:37.439276+0000', 'last_fullsized': '2024-09-21T05:59:37.439276+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T05:59:43.487 INFO:tasks.ceph.ceph_manager.ceph:PG 7.25 is not active+clean 2024-09-21T05:59:43.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.423541+0000', 'last_change': '2024-09-21T05:59:37.423541+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.423541+0000', 'last_undegraded': '2024-09-21T05:59:37.423541+0000', 'last_fullsized': '2024-09-21T05:59:37.423541+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T05:59:43.487 INFO:tasks.ceph.ceph_manager.ceph:PG 7.24 is not active+clean 2024-09-21T05:59:43.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.423216+0000', 'last_change': '2024-09-21T05:59:37.423216+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.423216+0000', 'last_undegraded': '2024-09-21T05:59:37.423216+0000', 'last_fullsized': '2024-09-21T05:59:37.423216+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:PG 7.23 is not active+clean 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.431328+0000', 'last_change': '2024-09-21T05:59:37.431328+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.431328+0000', 'last_undegraded': '2024-09-21T05:59:37.431328+0000', 'last_fullsized': '2024-09-21T05:59:37.431328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:PG 7.22 is not active+clean 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.423470+0000', 'last_change': '2024-09-21T05:59:37.423470+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.423470+0000', 'last_undegraded': '2024-09-21T05:59:37.423470+0000', 'last_fullsized': '2024-09-21T05:59:37.423470+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:PG 7.21 is not active+clean 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.426152+0000', 'last_change': '2024-09-21T05:59:37.426152+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.426152+0000', 'last_undegraded': '2024-09-21T05:59:37.426152+0000', 'last_fullsized': '2024-09-21T05:59:37.426152+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:PG 7.20 is not active+clean 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.438506+0000', 'last_change': '2024-09-21T05:59:37.438506+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.438506+0000', 'last_undegraded': '2024-09-21T05:59:37.438506+0000', 'last_fullsized': '2024-09-21T05:59:37.438506+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1f is not active+clean 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.432540+0000', 'last_change': '2024-09-21T05:59:37.432540+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.432540+0000', 'last_undegraded': '2024-09-21T05:59:37.432540+0000', 'last_fullsized': '2024-09-21T05:59:37.432540+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1c is not active+clean 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.422517+0000', 'last_change': '2024-09-21T05:59:37.422517+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.422517+0000', 'last_undegraded': '2024-09-21T05:59:37.422517+0000', 'last_fullsized': '2024-09-21T05:59:37.422517+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1a is not active+clean 2024-09-21T05:59:43.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.422840+0000', 'last_change': '2024-09-21T05:59:37.422840+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.422840+0000', 'last_undegraded': '2024-09-21T05:59:37.422840+0000', 'last_fullsized': '2024-09-21T05:59:37.422840+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T05:59:43.489 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1b is not active+clean 2024-09-21T05:59:43.489 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.432049+0000', 'last_change': '2024-09-21T05:59:37.432049+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.432049+0000', 'last_undegraded': '2024-09-21T05:59:37.432049+0000', 'last_fullsized': '2024-09-21T05:59:37.432049+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.489 INFO:tasks.ceph.ceph_manager.ceph:PG 7.18 is not active+clean 2024-09-21T05:59:43.489 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.433120+0000', 'last_change': '2024-09-21T05:59:37.433120+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.433120+0000', 'last_undegraded': '2024-09-21T05:59:37.433120+0000', 'last_fullsized': '2024-09-21T05:59:37.433120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.489 INFO:tasks.ceph.ceph_manager.ceph:PG 7.19 is not active+clean 2024-09-21T05:59:43.489 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.439321+0000', 'last_change': '2024-09-21T05:59:37.439321+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.439321+0000', 'last_undegraded': '2024-09-21T05:59:37.439321+0000', 'last_fullsized': '2024-09-21T05:59:37.439321+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:59:43.489 INFO:tasks.ceph.ceph_manager.ceph:PG 7.16 is not active+clean 2024-09-21T05:59:43.489 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.425083+0000', 'last_change': '2024-09-21T05:59:37.425083+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.425083+0000', 'last_undegraded': '2024-09-21T05:59:37.425083+0000', 'last_fullsized': '2024-09-21T05:59:37.425083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T05:59:43.489 INFO:tasks.ceph.ceph_manager.ceph:PG 7.17 is not active+clean 2024-09-21T05:59:43.489 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.428994+0000', 'last_change': '2024-09-21T05:59:37.428994+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.428994+0000', 'last_undegraded': '2024-09-21T05:59:37.428994+0000', 'last_fullsized': '2024-09-21T05:59:37.428994+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:59:43.489 INFO:tasks.ceph.ceph_manager.ceph:PG 7.14 is not active+clean 2024-09-21T05:59:43.489 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.426543+0000', 'last_change': '2024-09-21T05:59:37.426543+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.426543+0000', 'last_undegraded': '2024-09-21T05:59:37.426543+0000', 'last_fullsized': '2024-09-21T05:59:37.426543+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:PG 7.15 is not active+clean 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.429754+0000', 'last_change': '2024-09-21T05:59:37.429754+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.429754+0000', 'last_undegraded': '2024-09-21T05:59:37.429754+0000', 'last_fullsized': '2024-09-21T05:59:37.429754+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:PG 7.12 is not active+clean 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.430964+0000', 'last_change': '2024-09-21T05:59:37.430964+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.430964+0000', 'last_undegraded': '2024-09-21T05:59:37.430964+0000', 'last_fullsized': '2024-09-21T05:59:37.430964+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:PG 7.13 is not active+clean 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.431534+0000', 'last_change': '2024-09-21T05:59:37.431534+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.431534+0000', 'last_undegraded': '2024-09-21T05:59:37.431534+0000', 'last_fullsized': '2024-09-21T05:59:37.431534+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:PG 7.10 is not active+clean 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.426456+0000', 'last_change': '2024-09-21T05:59:37.426456+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.426456+0000', 'last_undegraded': '2024-09-21T05:59:37.426456+0000', 'last_fullsized': '2024-09-21T05:59:37.426456+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:PG 7.11 is not active+clean 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.428769+0000', 'last_change': '2024-09-21T05:59:37.428769+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.428769+0000', 'last_undegraded': '2024-09-21T05:59:37.428769+0000', 'last_fullsized': '2024-09-21T05:59:37.428769+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:PG 7.e is not active+clean 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.435700+0000', 'last_change': '2024-09-21T05:59:37.435700+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.435700+0000', 'last_undegraded': '2024-09-21T05:59:37.435700+0000', 'last_fullsized': '2024-09-21T05:59:37.435700+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T05:59:43.490 INFO:tasks.ceph.ceph_manager.ceph:PG 7.f is not active+clean 2024-09-21T05:59:43.491 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.429835+0000', 'last_change': '2024-09-21T05:59:37.429835+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.429835+0000', 'last_undegraded': '2024-09-21T05:59:37.429835+0000', 'last_fullsized': '2024-09-21T05:59:37.429835+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.491 INFO:tasks.ceph.ceph_manager.ceph:PG 7.c is not active+clean 2024-09-21T05:59:43.491 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.431533+0000', 'last_change': '2024-09-21T05:59:37.431533+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.431533+0000', 'last_undegraded': '2024-09-21T05:59:37.431533+0000', 'last_fullsized': '2024-09-21T05:59:37.431533+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:59:43.491 INFO:tasks.ceph.ceph_manager.ceph:PG 7.d is not active+clean 2024-09-21T05:59:43.491 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.433437+0000', 'last_change': '2024-09-21T05:59:37.433437+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.433437+0000', 'last_undegraded': '2024-09-21T05:59:37.433437+0000', 'last_fullsized': '2024-09-21T05:59:37.433437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:59:43.491 INFO:tasks.ceph.ceph_manager.ceph:PG 7.a is not active+clean 2024-09-21T05:59:43.491 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.427110+0000', 'last_change': '2024-09-21T05:59:37.427110+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.427110+0000', 'last_undegraded': '2024-09-21T05:59:37.427110+0000', 'last_fullsized': '2024-09-21T05:59:37.427110+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T05:59:43.491 INFO:tasks.ceph.ceph_manager.ceph:PG 7.8 is not active+clean 2024-09-21T05:59:43.491 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.432848+0000', 'last_change': '2024-09-21T05:59:37.432848+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.432848+0000', 'last_undegraded': '2024-09-21T05:59:37.432848+0000', 'last_fullsized': '2024-09-21T05:59:37.432848+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T05:59:43.491 INFO:tasks.ceph.ceph_manager.ceph:PG 7.9 is not active+clean 2024-09-21T05:59:43.491 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.434548+0000', 'last_change': '2024-09-21T05:59:37.434548+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.434548+0000', 'last_undegraded': '2024-09-21T05:59:37.434548+0000', 'last_fullsized': '2024-09-21T05:59:37.434548+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:PG 7.7 is not active+clean 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.431977+0000', 'last_change': '2024-09-21T05:59:37.431977+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.431977+0000', 'last_undegraded': '2024-09-21T05:59:37.431977+0000', 'last_fullsized': '2024-09-21T05:59:37.431977+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:PG 7.4 is not active+clean 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.429475+0000', 'last_change': '2024-09-21T05:59:37.429475+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.429475+0000', 'last_undegraded': '2024-09-21T05:59:37.429475+0000', 'last_fullsized': '2024-09-21T05:59:37.429475+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:PG 7.5 is not active+clean 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.429164+0000', 'last_change': '2024-09-21T05:59:37.429164+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.429164+0000', 'last_undegraded': '2024-09-21T05:59:37.429164+0000', 'last_fullsized': '2024-09-21T05:59:37.429164+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2 is not active+clean 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.425900+0000', 'last_change': '2024-09-21T05:59:37.425900+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.425900+0000', 'last_undegraded': '2024-09-21T05:59:37.425900+0000', 'last_fullsized': '2024-09-21T05:59:37.425900+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3 is not active+clean 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.425968+0000', 'last_change': '2024-09-21T05:59:37.425968+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.425968+0000', 'last_undegraded': '2024-09-21T05:59:37.425968+0000', 'last_fullsized': '2024-09-21T05:59:37.425968+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:PG 7.0 is not active+clean 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.424041+0000', 'last_change': '2024-09-21T05:59:37.424041+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.424041+0000', 'last_undegraded': '2024-09-21T05:59:37.424041+0000', 'last_fullsized': '2024-09-21T05:59:37.424041+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T05:59:43.492 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1 is not active+clean 2024-09-21T05:59:43.493 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.422407+0000', 'last_change': '2024-09-21T05:59:37.422407+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.422407+0000', 'last_undegraded': '2024-09-21T05:59:37.422407+0000', 'last_fullsized': '2024-09-21T05:59:37.422407+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T05:59:43.493 INFO:tasks.ceph.ceph_manager.ceph:PG 7.6 is not active+clean 2024-09-21T05:59:43.493 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:59:37.430505+0000', 'last_change': '2024-09-21T05:59:37.430505+0000', 'last_active': '2024-09-21T05:59:37.411723+0000', 'last_peered': '2024-09-21T05:59:37.411723+0000', 'last_clean': '2024-09-21T05:59:37.411723+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:59:37.430505+0000', 'last_undegraded': '2024-09-21T05:59:37.430505+0000', 'last_fullsized': '2024-09-21T05:59:37.430505+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'last_clean_scrub_stamp': '2024-09-21T05:59:37.411723+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T05:59:43.493 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T05:59:43.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.675+0000 7f4fc5200640 1 -- 172.21.15.50:0/3122779190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fc01527b0 msgr2=0x7f4fc015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.675+0000 7f4fc5200640 1 --2- 172.21.15.50:0/3122779190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fc01527b0 0x7f4fc015bb60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009900 tx=0x7f4fa801c8b0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.675+0000 7f4fc5200640 1 -- 172.21.15.50:0/3122779190 shutdown_connections 2024-09-21T05:59:43.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.675+0000 7f4fc5200640 1 --2- 172.21.15.50:0/3122779190 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4fc00b4de0 0x7f4fc00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.675+0000 7f4fc5200640 1 --2- 172.21.15.50:0/3122779190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fc01527b0 0x7f4fc015bb60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.675+0000 7f4fc5200640 1 -- 172.21.15.50:0/3122779190 >> 172.21.15.50:0/3122779190 conn(0x7f4fc00b2040 msgr2=0x7f4fc00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:43.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.675+0000 7f4fc5200640 1 -- 172.21.15.50:0/3122779190 shutdown_connections 2024-09-21T05:59:43.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.675+0000 7f4fc5200640 1 -- 172.21.15.50:0/3122779190 wait complete. 2024-09-21T05:59:43.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.675+0000 7f4fc5200640 1 Processor -- start 2024-09-21T05:59:43.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.675+0000 7f4fc5200640 1 -- start start 2024-09-21T05:59:43.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fc5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fc00b4de0 0x7f4fc013c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:43.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fc5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc00b5c70 0x7f4fc00b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:43.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fc5200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4fc01527b0 0x7f4fc00b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:43.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fc5200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc0168950 con 0x7f4fc00b4de0 2024-09-21T05:59:43.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fc5200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc01687d0 con 0x7f4fc00b5c70 2024-09-21T05:59:43.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fc5200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc0168ad0 con 0x7f4fc01527b0 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fbde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fc00b4de0 0x7f4fc013c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fbde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fc00b4de0 0x7f4fc013c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46666/0 (socket says 172.21.15.50:46666) 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fbde00640 1 -- 172.21.15.50:0/3634049875 learned_addr learned my addr 172.21.15.50:0/3634049875 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fb5400640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc00b5c70 0x7f4fc00b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fbe800640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4fc01527b0 0x7f4fc00b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fbde00640 1 -- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4fc01527b0 msgr2=0x7f4fc00b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fbde00640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4fc01527b0 0x7f4fc00b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fbde00640 1 -- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc00b5c70 msgr2=0x7f4fc00b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fbde00640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc00b5c70 0x7f4fc00b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fbde00640 1 -- 172.21.15.50:0/3634049875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc013cfa0 con 0x7f4fc00b4de0 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fbe800640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4fc01527b0 0x7f4fc00b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fbde00640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fc00b4de0 0x7f4fc013c900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c3c0 tx=0x7f4fb000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fb6a00640 1 -- 172.21.15.50:0/3634049875 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0017070 con 0x7f4fc00b4de0 2024-09-21T05:59:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fb6a00640 1 -- 172.21.15.50:0/3634049875 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fb00053c0 con 0x7f4fc00b4de0 2024-09-21T05:59:43.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fc5200640 1 -- 172.21.15.50:0/3634049875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc013d280 con 0x7f4fc00b4de0 2024-09-21T05:59:43.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fb6a00640 1 -- 172.21.15.50:0/3634049875 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0003860 con 0x7f4fc00b4de0 2024-09-21T05:59:43.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fc5200640 1 -- 172.21.15.50:0/3634049875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc014dd40 con 0x7f4fc00b4de0 2024-09-21T05:59:43.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.679+0000 7f4fc5200640 1 -- 172.21.15.50:0/3634049875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc00b51f0 con 0x7f4fc00b4de0 2024-09-21T05:59:43.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.683+0000 7f4fb6a00640 1 -- 172.21.15.50:0/3634049875 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f4fb001a070 con 0x7f4fc00b4de0 2024-09-21T05:59:43.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.683+0000 7f4fb6a00640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4f9c073560 0x7f4f9c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:43.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.683+0000 7f4fb6a00640 1 -- 172.21.15.50:0/3634049875 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f4fb0093b70 con 0x7f4fc00b4de0 2024-09-21T05:59:43.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.683+0000 7f4fb5400640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4f9c073560 0x7f4f9c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:43.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.683+0000 7f4fb5400640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4f9c073560 0x7f4f9c075a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80af000 tx=0x7f4fa801b0f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:43.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.687+0000 7f4fb6a00640 1 -- 172.21.15.50:0/3634049875 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4fc00b5730 con 0x7f4fc00b4de0 2024-09-21T05:59:43.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 -- 172.21.15.50:0/3634049875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4fc01594d0 con 0x7f4fc00b4de0 2024-09-21T05:59:43.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fb6a00640 1 -- 172.21.15.50:0/3634049875 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1137 (secure 0 0 0) 0x7f4fb0065e20 con 0x7f4fc00b4de0 2024-09-21T05:59:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:43.930 INFO:teuthology.orchestra.run.smithi050.stdout:{"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":220,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":8,"num_up_osds":8,"osd_up_since":1726898177,"num_in_osds":8,"osd_in_since":1726898167,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":61},{"state_name":"active+clean","count":36}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590368,"bytes_used":277635072,"bytes_avail":772816478208,"bytes_total":773094113280,"inactive_pgs_ratio":0.62886595726013184,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":27,"btime":"2024-09-21T05:59:41:444356+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5370}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-21T05:58:22.913083+0000","services":{}},"progress_events":{}} 2024-09-21T05:59:43.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 -- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4f9c073560 msgr2=0x7f4f9c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4f9c073560 0x7f4f9c075a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80af000 tx=0x7f4fa801b0f0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 -- 172.21.15.50:0/3634049875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fc00b4de0 msgr2=0x7f4fc013c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fc00b4de0 0x7f4fc013c900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c3c0 tx=0x7f4fb000c880 comp rx=0 tx=0).stop 2024-09-21T05:59:43.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 -- 172.21.15.50:0/3634049875 shutdown_connections 2024-09-21T05:59:43.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4fc01527b0 0x7f4fc00b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc00b5c70 0x7f4fc00b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4f9c073560 0x7f4f9c075a70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 --2- 172.21.15.50:0/3634049875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fc00b4de0 0x7f4fc013c900 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 -- 172.21.15.50:0/3634049875 >> 172.21.15.50:0/3634049875 conn(0x7f4fc00b2040 msgr2=0x7f4fc0154f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:43.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 -- 172.21.15.50:0/3634049875 shutdown_connections 2024-09-21T05:59:43.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:43.927+0000 7f4fc5200640 1 -- 172.21.15.50:0/3634049875 wait complete. 2024-09-21T05:59:43.952 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T05:59:43.952 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:59:44.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.107+0000 7f1704000640 1 -- 172.21.15.50:0/866348695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16fc1527b0 msgr2=0x7f16fc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:44.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.107+0000 7f1704000640 1 --2- 172.21.15.50:0/866348695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16fc1527b0 0x7f16fc15bb60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009840 tx=0x7f16e801c7f0 comp rx=0 tx=0).stop 2024-09-21T05:59:44.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.107+0000 7f1704000640 1 -- 172.21.15.50:0/866348695 shutdown_connections 2024-09-21T05:59:44.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.107+0000 7f1704000640 1 --2- 172.21.15.50:0/866348695 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f16fc0b4de0 0x7f16fc0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:44.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.107+0000 7f1704000640 1 --2- 172.21.15.50:0/866348695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16fc1527b0 0x7f16fc15bb60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:44.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.107+0000 7f1704000640 1 -- 172.21.15.50:0/866348695 >> 172.21.15.50:0/866348695 conn(0x7f16fc0b2040 msgr2=0x7f16fc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:44.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.107+0000 7f1704000640 1 -- 172.21.15.50:0/866348695 shutdown_connections 2024-09-21T05:59:44.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.107+0000 7f1704000640 1 -- 172.21.15.50:0/866348695 wait complete. 2024-09-21T05:59:44.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.107+0000 7f1704000640 1 Processor -- start 2024-09-21T05:59:44.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1704000640 1 -- start start 2024-09-21T05:59:44.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1704000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f16fc0b4de0 0x7f16fc150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:44.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1704000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16fc0b5c70 0x7f16fc14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:44.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1704000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16fc1527b0 0x7f16fc14eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1704000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc168950 con 0x7f16fc1527b0 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1704000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1687d0 con 0x7f16fc0b5c70 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1704000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f16fc168ad0 con 0x7f16fc0b4de0 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1702000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16fc1527b0 0x7f16fc14eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1702000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16fc1527b0 0x7f16fc14eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46696/0 (socket says 172.21.15.50:46696) 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1702000640 1 -- 172.21.15.50:0/2663410284 learned_addr learned my addr 172.21.15.50:0/2663410284 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1700c00640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16fc0b5c70 0x7f16fc14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1701600640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f16fc0b4de0 0x7f16fc150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1702000640 1 -- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f16fc0b4de0 msgr2=0x7f16fc150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1702000640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f16fc0b4de0 0x7f16fc150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1702000640 1 -- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16fc0b5c70 msgr2=0x7f16fc14c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1702000640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16fc0b5c70 0x7f16fc14c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1702000640 1 -- 172.21.15.50:0/2663410284 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc14e0c0 con 0x7f16fc1527b0 2024-09-21T05:59:44.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1702000640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16fc1527b0 0x7f16fc14eca0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f16f400d9d0 tx=0x7f16f400de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f16f9600640 1 -- 172.21.15.50:0/2663410284 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4017070 con 0x7f16fc1527b0 2024-09-21T05:59:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f16f9600640 1 -- 172.21.15.50:0/2663410284 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16f4004b30 con 0x7f16fc1527b0 2024-09-21T05:59:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1704000640 1 -- 172.21.15.50:0/2663410284 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16fc158320 con 0x7f16fc1527b0 2024-09-21T05:59:44.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f16f9600640 1 -- 172.21.15.50:0/2663410284 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f400be60 con 0x7f16fc1527b0 2024-09-21T05:59:44.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1704000640 1 -- 172.21.15.50:0/2663410284 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16fc165f60 con 0x7f16fc1527b0 2024-09-21T05:59:44.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.111+0000 7f1704000640 1 -- 172.21.15.50:0/2663410284 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16fc0b51f0 con 0x7f16fc1527b0 2024-09-21T05:59:44.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.115+0000 7f16f9600640 1 -- 172.21.15.50:0/2663410284 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f16f4003980 con 0x7f16fc1527b0 2024-09-21T05:59:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.115+0000 7f16f9600640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f16dc073590 0x7f16dc075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:44.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.115+0000 7f16f9600640 1 -- 172.21.15.50:0/2663410284 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f16f4094f90 con 0x7f16fc1527b0 2024-09-21T05:59:44.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.115+0000 7f1701600640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f16dc073590 0x7f16dc075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:44.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.115+0000 7f1701600640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f16dc073590 0x7f16dc075aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f16f0001400 tx=0x7f16f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:44.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.119+0000 7f16f9600640 1 -- 172.21.15.50:0/2663410284 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f16f4020020 con 0x7f16fc1527b0 2024-09-21T05:59:44.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.295+0000 7f1704000640 1 -- 172.21.15.50:0/2663410284 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16fc06b540 con 0x7f16dc073590 2024-09-21T05:59:44.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.299+0000 7f16f9600640 1 -- 172.21.15.50:0/2663410284 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246916 (secure 0 0 0) 0x7f16f000e490 con 0x7f16dc073590 2024-09-21T05:59:44.305 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:44.306 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:59:44.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.303+0000 7f1704000640 1 -- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f16dc073590 msgr2=0x7f16dc075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:44.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.303+0000 7f1704000640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f16dc073590 0x7f16dc075aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f16f0001400 tx=0x7f16f0009040 comp rx=0 tx=0).stop 2024-09-21T05:59:44.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.303+0000 7f1704000640 1 -- 172.21.15.50:0/2663410284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16fc1527b0 msgr2=0x7f16fc14eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:44.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.303+0000 7f1704000640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16fc1527b0 0x7f16fc14eca0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f16f400d9d0 tx=0x7f16f400de90 comp rx=0 tx=0).stop 2024-09-21T05:59:44.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.303+0000 7f1704000640 1 -- 172.21.15.50:0/2663410284 shutdown_connections 2024-09-21T05:59:44.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.303+0000 7f1704000640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f16dc073590 0x7f16dc075aa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:44.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.303+0000 7f1704000640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16fc1527b0 0x7f16fc14eca0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:44.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.303+0000 7f1704000640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16fc0b5c70 0x7f16fc14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:44.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.303+0000 7f1704000640 1 --2- 172.21.15.50:0/2663410284 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f16fc0b4de0 0x7f16fc150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:44.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.303+0000 7f1704000640 1 -- 172.21.15.50:0/2663410284 >> 172.21.15.50:0/2663410284 conn(0x7f16fc0b2040 msgr2=0x7f16fc0b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:44.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.303+0000 7f1704000640 1 -- 172.21.15.50:0/2663410284 shutdown_connections 2024-09-21T05:59:44.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:44.303+0000 7f1704000640 1 -- 172.21.15.50:0/2663410284 wait complete. 2024-09-21T05:59:44.326 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-21T05:59:42.934565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271128,"kb_used_data":4952,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754703592,"statfs":{"total":773094113280,"available":772816478208,"internally_reserved":0,"allocated":5070848,"data_stored":2704358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001455"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430439+0000","last_change":"2024-09-21T05:59:37.430439+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430439+0000","last_undegraded":"2024-09-21T05:59:37.430439+0000","last_fullsized":"2024-09-21T05:59:37.430439+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424624+0000","last_change":"2024-09-21T05:59:37.424624+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424624+0000","last_undegraded":"2024-09-21T05:59:37.424624+0000","last_fullsized":"2024-09-21T05:59:37.424624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424101+0000","last_change":"2024-09-21T05:59:37.424101+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424101+0000","last_undegraded":"2024-09-21T05:59:37.424101+0000","last_fullsized":"2024-09-21T05:59:37.424101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431709+0000","last_change":"2024-09-21T05:59:37.431709+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431709+0000","last_undegraded":"2024-09-21T05:59:37.431709+0000","last_fullsized":"2024-09-21T05:59:37.431709+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423190+0000","last_change":"2024-09-21T05:59:37.423190+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423190+0000","last_undegraded":"2024-09-21T05:59:37.423190+0000","last_fullsized":"2024-09-21T05:59:37.423190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.428041+0000","last_change":"2024-09-21T05:59:37.428041+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.428041+0000","last_undegraded":"2024-09-21T05:59:37.428041+0000","last_fullsized":"2024-09-21T05:59:37.428041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430128+0000","last_change":"2024-09-21T05:59:37.430128+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430128+0000","last_undegraded":"2024-09-21T05:59:37.430128+0000","last_fullsized":"2024-09-21T05:59:37.430128+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433589+0000","last_change":"2024-09-21T05:59:37.433589+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433589+0000","last_undegraded":"2024-09-21T05:59:37.433589+0000","last_fullsized":"2024-09-21T05:59:37.433589+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430329+0000","last_change":"2024-09-21T05:59:37.430329+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430329+0000","last_undegraded":"2024-09-21T05:59:37.430329+0000","last_fullsized":"2024-09-21T05:59:37.430329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431028+0000","last_change":"2024-09-21T05:59:37.431028+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431028+0000","last_undegraded":"2024-09-21T05:59:37.431028+0000","last_fullsized":"2024-09-21T05:59:37.431028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429854+0000","last_change":"2024-09-21T05:59:37.429854+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429854+0000","last_undegraded":"2024-09-21T05:59:37.429854+0000","last_fullsized":"2024-09-21T05:59:37.429854+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423767+0000","last_change":"2024-09-21T05:59:37.423767+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423767+0000","last_undegraded":"2024-09-21T05:59:37.423767+0000","last_fullsized":"2024-09-21T05:59:37.423767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.435082+0000","last_change":"2024-09-21T05:59:37.435082+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.435082+0000","last_undegraded":"2024-09-21T05:59:37.435082+0000","last_fullsized":"2024-09-21T05:59:37.435082+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432295+0000","last_change":"2024-09-21T05:59:37.432295+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432295+0000","last_undegraded":"2024-09-21T05:59:37.432295+0000","last_fullsized":"2024-09-21T05:59:37.432295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.427649+0000","last_change":"2024-09-21T05:59:37.427649+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.427649+0000","last_undegraded":"2024-09-21T05:59:37.427649+0000","last_fullsized":"2024-09-21T05:59:37.427649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430585+0000","last_change":"2024-09-21T05:59:37.430585+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430585+0000","last_undegraded":"2024-09-21T05:59:37.430585+0000","last_fullsized":"2024-09-21T05:59:37.430585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424708+0000","last_change":"2024-09-21T05:59:37.424708+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424708+0000","last_undegraded":"2024-09-21T05:59:37.424708+0000","last_fullsized":"2024-09-21T05:59:37.424708+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425629+0000","last_change":"2024-09-21T05:59:37.425629+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425629+0000","last_undegraded":"2024-09-21T05:59:37.425629+0000","last_fullsized":"2024-09-21T05:59:37.425629+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432796+0000","last_change":"2024-09-21T05:59:37.432796+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432796+0000","last_undegraded":"2024-09-21T05:59:37.432796+0000","last_fullsized":"2024-09-21T05:59:37.432796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.434597+0000","last_change":"2024-09-21T05:59:37.434597+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.434597+0000","last_undegraded":"2024-09-21T05:59:37.434597+0000","last_fullsized":"2024-09-21T05:59:37.434597+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.427426+0000","last_change":"2024-09-21T05:59:37.427426+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.427426+0000","last_undegraded":"2024-09-21T05:59:37.427426+0000","last_fullsized":"2024-09-21T05:59:37.427426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429310+0000","last_change":"2024-09-21T05:59:37.429310+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429310+0000","last_undegraded":"2024-09-21T05:59:37.429310+0000","last_fullsized":"2024-09-21T05:59:37.429310+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.434559+0000","last_change":"2024-09-21T05:59:37.434559+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.434559+0000","last_undegraded":"2024-09-21T05:59:37.434559+0000","last_fullsized":"2024-09-21T05:59:37.434559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429894+0000","last_change":"2024-09-21T05:59:37.429894+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429894+0000","last_undegraded":"2024-09-21T05:59:37.429894+0000","last_fullsized":"2024-09-21T05:59:37.429894+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433207+0000","last_change":"2024-09-21T05:59:37.433207+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433207+0000","last_undegraded":"2024-09-21T05:59:37.433207+0000","last_fullsized":"2024-09-21T05:59:37.433207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.439276+0000","last_change":"2024-09-21T05:59:37.439276+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.439276+0000","last_undegraded":"2024-09-21T05:59:37.439276+0000","last_fullsized":"2024-09-21T05:59:37.439276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423541+0000","last_change":"2024-09-21T05:59:37.423541+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423541+0000","last_undegraded":"2024-09-21T05:59:37.423541+0000","last_fullsized":"2024-09-21T05:59:37.423541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423216+0000","last_change":"2024-09-21T05:59:37.423216+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423216+0000","last_undegraded":"2024-09-21T05:59:37.423216+0000","last_fullsized":"2024-09-21T05:59:37.423216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431328+0000","last_change":"2024-09-21T05:59:37.431328+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431328+0000","last_undegraded":"2024-09-21T05:59:37.431328+0000","last_fullsized":"2024-09-21T05:59:37.431328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.423470+0000","last_change":"2024-09-21T05:59:37.423470+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.423470+0000","last_undegraded":"2024-09-21T05:59:37.423470+0000","last_fullsized":"2024-09-21T05:59:37.423470+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.426152+0000","last_change":"2024-09-21T05:59:37.426152+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.426152+0000","last_undegraded":"2024-09-21T05:59:37.426152+0000","last_fullsized":"2024-09-21T05:59:37.426152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.438506+0000","last_change":"2024-09-21T05:59:37.438506+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.438506+0000","last_undegraded":"2024-09-21T05:59:37.438506+0000","last_fullsized":"2024-09-21T05:59:37.438506+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837865+0000","last_change":"2024-09-21T05:59:38.462061+0000","last_active":"2024-09-21T05:59:39.837865+0000","last_peered":"2024-09-21T05:59:39.837865+0000","last_clean":"2024-09-21T05:59:39.837865+0000","last_became_active":"2024-09-21T05:59:38.461740+0000","last_became_peered":"2024-09-21T05:59:38.461740+0000","last_unstale":"2024-09-21T05:59:39.837865+0000","last_undegraded":"2024-09-21T05:59:39.837865+0000","last_fullsized":"2024-09-21T05:59:39.837865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:17.985198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451150+0000","last_change":"2024-09-21T05:59:37.447751+0000","last_active":"2024-09-21T05:59:37.451150+0000","last_peered":"2024-09-21T05:59:37.451150+0000","last_clean":"2024-09-21T05:59:37.451150+0000","last_became_active":"2024-09-21T05:59:37.447392+0000","last_became_peered":"2024-09-21T05:59:37.447392+0000","last_unstale":"2024-09-21T05:59:37.451150+0000","last_undegraded":"2024-09-21T05:59:37.451150+0000","last_fullsized":"2024-09-21T05:59:37.451150+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:17.297839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432540+0000","last_change":"2024-09-21T05:59:37.432540+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432540+0000","last_undegraded":"2024-09-21T05:59:37.432540+0000","last_fullsized":"2024-09-21T05:59:37.432540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.444761+0000","last_change":"2024-09-21T05:59:37.443312+0000","last_active":"2024-09-21T05:59:37.444761+0000","last_peered":"2024-09-21T05:59:37.444761+0000","last_clean":"2024-09-21T05:59:37.444761+0000","last_became_active":"2024-09-21T05:59:37.442457+0000","last_became_peered":"2024-09-21T05:59:37.442457+0000","last_unstale":"2024-09-21T05:59:37.444761+0000","last_undegraded":"2024-09-21T05:59:37.444761+0000","last_fullsized":"2024-09-21T05:59:37.444761+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:54.217575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.422517+0000","last_change":"2024-09-21T05:59:37.422517+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.422517+0000","last_undegraded":"2024-09-21T05:59:37.422517+0000","last_fullsized":"2024-09-21T05:59:37.422517+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837882+0000","last_change":"2024-09-21T05:59:37.440271+0000","last_active":"2024-09-21T05:59:39.837882+0000","last_peered":"2024-09-21T05:59:39.837882+0000","last_clean":"2024-09-21T05:59:39.837882+0000","last_became_active":"2024-09-21T05:59:37.439909+0000","last_became_peered":"2024-09-21T05:59:37.439909+0000","last_unstale":"2024-09-21T05:59:39.837882+0000","last_undegraded":"2024-09-21T05:59:39.837882+0000","last_fullsized":"2024-09-21T05:59:39.837882+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:15.657492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837975+0000","last_change":"2024-09-21T05:59:38.444285+0000","last_active":"2024-09-21T05:59:39.837975+0000","last_peered":"2024-09-21T05:59:39.837975+0000","last_clean":"2024-09-21T05:59:39.837975+0000","last_became_active":"2024-09-21T05:59:38.443952+0000","last_became_peered":"2024-09-21T05:59:38.443952+0000","last_unstale":"2024-09-21T05:59:39.837975+0000","last_undegraded":"2024-09-21T05:59:39.837975+0000","last_fullsized":"2024-09-21T05:59:39.837975+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:31.815943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.449829+0000","last_change":"2024-09-21T05:59:37.443607+0000","last_active":"2024-09-21T05:59:37.449829+0000","last_peered":"2024-09-21T05:59:37.449829+0000","last_clean":"2024-09-21T05:59:37.449829+0000","last_became_active":"2024-09-21T05:59:37.443262+0000","last_became_peered":"2024-09-21T05:59:37.443262+0000","last_unstale":"2024-09-21T05:59:37.449829+0000","last_undegraded":"2024-09-21T05:59:37.449829+0000","last_fullsized":"2024-09-21T05:59:37.449829+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:21.369884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.422840+0000","last_change":"2024-09-21T05:59:37.422840+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.422840+0000","last_undegraded":"2024-09-21T05:59:37.422840+0000","last_fullsized":"2024-09-21T05:59:37.422840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451244+0000","last_change":"2024-09-21T05:59:37.447920+0000","last_active":"2024-09-21T05:59:37.451244+0000","last_peered":"2024-09-21T05:59:37.451244+0000","last_clean":"2024-09-21T05:59:37.451244+0000","last_became_active":"2024-09-21T05:59:37.447240+0000","last_became_peered":"2024-09-21T05:59:37.447240+0000","last_unstale":"2024-09-21T05:59:37.451244+0000","last_undegraded":"2024-09-21T05:59:37.451244+0000","last_fullsized":"2024-09-21T05:59:37.451244+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:52.027735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432049+0000","last_change":"2024-09-21T05:59:37.432049+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432049+0000","last_undegraded":"2024-09-21T05:59:37.432049+0000","last_fullsized":"2024-09-21T05:59:37.432049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.438146+0000","last_change":"2024-09-21T05:59:37.429073+0000","last_active":"2024-09-21T05:59:37.438146+0000","last_peered":"2024-09-21T05:59:37.438146+0000","last_clean":"2024-09-21T05:59:37.438146+0000","last_became_active":"2024-09-21T05:59:37.428733+0000","last_became_peered":"2024-09-21T05:59:37.428733+0000","last_unstale":"2024-09-21T05:59:37.438146+0000","last_undegraded":"2024-09-21T05:59:37.438146+0000","last_fullsized":"2024-09-21T05:59:37.438146+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:39.275851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433120+0000","last_change":"2024-09-21T05:59:37.433120+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433120+0000","last_undegraded":"2024-09-21T05:59:37.433120+0000","last_fullsized":"2024-09-21T05:59:37.433120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.440195+0000","last_change":"2024-09-21T05:59:37.437489+0000","last_active":"2024-09-21T05:59:37.440195+0000","last_peered":"2024-09-21T05:59:37.440195+0000","last_clean":"2024-09-21T05:59:37.440195+0000","last_became_active":"2024-09-21T05:59:37.437170+0000","last_became_peered":"2024-09-21T05:59:37.437170+0000","last_unstale":"2024-09-21T05:59:37.440195+0000","last_undegraded":"2024-09-21T05:59:37.440195+0000","last_fullsized":"2024-09-21T05:59:37.440195+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:42.959574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.439321+0000","last_change":"2024-09-21T05:59:37.439321+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.439321+0000","last_undegraded":"2024-09-21T05:59:37.439321+0000","last_fullsized":"2024-09-21T05:59:37.439321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451906+0000","last_change":"2024-09-21T05:59:37.449577+0000","last_active":"2024-09-21T05:59:37.451906+0000","last_peered":"2024-09-21T05:59:37.451906+0000","last_clean":"2024-09-21T05:59:37.451906+0000","last_became_active":"2024-09-21T05:59:37.449255+0000","last_became_peered":"2024-09-21T05:59:37.449255+0000","last_unstale":"2024-09-21T05:59:37.451906+0000","last_undegraded":"2024-09-21T05:59:37.451906+0000","last_fullsized":"2024-09-21T05:59:37.451906+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:39.275851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425083+0000","last_change":"2024-09-21T05:59:37.425083+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425083+0000","last_undegraded":"2024-09-21T05:59:37.425083+0000","last_fullsized":"2024-09-21T05:59:37.425083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.444043+0000","last_change":"2024-09-21T05:59:37.440683+0000","last_active":"2024-09-21T05:59:37.444043+0000","last_peered":"2024-09-21T05:59:37.444043+0000","last_clean":"2024-09-21T05:59:37.444043+0000","last_became_active":"2024-09-21T05:59:37.440336+0000","last_became_peered":"2024-09-21T05:59:37.440336+0000","last_unstale":"2024-09-21T05:59:37.444043+0000","last_undegraded":"2024-09-21T05:59:37.444043+0000","last_fullsized":"2024-09-21T05:59:37.444043+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:13.571664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.428994+0000","last_change":"2024-09-21T05:59:37.428994+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.428994+0000","last_undegraded":"2024-09-21T05:59:37.428994+0000","last_fullsized":"2024-09-21T05:59:37.428994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451867+0000","last_change":"2024-09-21T05:59:37.448714+0000","last_active":"2024-09-21T05:59:37.451867+0000","last_peered":"2024-09-21T05:59:37.451867+0000","last_clean":"2024-09-21T05:59:37.451867+0000","last_became_active":"2024-09-21T05:59:37.448310+0000","last_became_peered":"2024-09-21T05:59:37.448310+0000","last_unstale":"2024-09-21T05:59:37.451867+0000","last_undegraded":"2024-09-21T05:59:37.451867+0000","last_fullsized":"2024-09-21T05:59:37.451867+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:21.097323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.426543+0000","last_change":"2024-09-21T05:59:37.426543+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.426543+0000","last_undegraded":"2024-09-21T05:59:37.426543+0000","last_fullsized":"2024-09-21T05:59:37.426543+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450006+0000","last_change":"2024-09-21T05:59:37.445812+0000","last_active":"2024-09-21T05:59:37.450006+0000","last_peered":"2024-09-21T05:59:37.450006+0000","last_clean":"2024-09-21T05:59:37.450006+0000","last_became_active":"2024-09-21T05:59:37.445496+0000","last_became_peered":"2024-09-21T05:59:37.445496+0000","last_unstale":"2024-09-21T05:59:37.450006+0000","last_undegraded":"2024-09-21T05:59:37.450006+0000","last_fullsized":"2024-09-21T05:59:37.450006+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:46.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429754+0000","last_change":"2024-09-21T05:59:37.429754+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429754+0000","last_undegraded":"2024-09-21T05:59:37.429754+0000","last_fullsized":"2024-09-21T05:59:37.429754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.443714+0000","last_change":"2024-09-21T05:59:37.440777+0000","last_active":"2024-09-21T05:59:37.443714+0000","last_peered":"2024-09-21T05:59:37.443714+0000","last_clean":"2024-09-21T05:59:37.443714+0000","last_became_active":"2024-09-21T05:59:37.440460+0000","last_became_peered":"2024-09-21T05:59:37.440460+0000","last_unstale":"2024-09-21T05:59:37.443714+0000","last_undegraded":"2024-09-21T05:59:37.443714+0000","last_fullsized":"2024-09-21T05:59:37.443714+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:03.466004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430964+0000","last_change":"2024-09-21T05:59:37.430964+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430964+0000","last_undegraded":"2024-09-21T05:59:37.430964+0000","last_fullsized":"2024-09-21T05:59:37.430964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451384+0000","last_change":"2024-09-21T05:59:37.447984+0000","last_active":"2024-09-21T05:59:37.451384+0000","last_peered":"2024-09-21T05:59:37.451384+0000","last_clean":"2024-09-21T05:59:37.451384+0000","last_became_active":"2024-09-21T05:59:37.447685+0000","last_became_peered":"2024-09-21T05:59:37.447685+0000","last_unstale":"2024-09-21T05:59:37.451384+0000","last_undegraded":"2024-09-21T05:59:37.451384+0000","last_fullsized":"2024-09-21T05:59:37.451384+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:36.267884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431534+0000","last_change":"2024-09-21T05:59:37.431534+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431534+0000","last_undegraded":"2024-09-21T05:59:37.431534+0000","last_fullsized":"2024-09-21T05:59:37.431534+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.449770+0000","last_change":"2024-09-21T05:59:37.443723+0000","last_active":"2024-09-21T05:59:37.449770+0000","last_peered":"2024-09-21T05:59:37.449770+0000","last_clean":"2024-09-21T05:59:37.449770+0000","last_became_active":"2024-09-21T05:59:37.443383+0000","last_became_peered":"2024-09-21T05:59:37.443383+0000","last_unstale":"2024-09-21T05:59:37.449770+0000","last_undegraded":"2024-09-21T05:59:37.449770+0000","last_fullsized":"2024-09-21T05:59:37.449770+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.689460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.426456+0000","last_change":"2024-09-21T05:59:37.426456+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.426456+0000","last_undegraded":"2024-09-21T05:59:37.426456+0000","last_fullsized":"2024-09-21T05:59:37.426456+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450786+0000","last_change":"2024-09-21T05:59:37.444802+0000","last_active":"2024-09-21T05:59:37.450786+0000","last_peered":"2024-09-21T05:59:37.450786+0000","last_clean":"2024-09-21T05:59:37.450786+0000","last_became_active":"2024-09-21T05:59:37.444457+0000","last_became_peered":"2024-09-21T05:59:37.444457+0000","last_unstale":"2024-09-21T05:59:37.450786+0000","last_undegraded":"2024-09-21T05:59:37.450786+0000","last_fullsized":"2024-09-21T05:59:37.450786+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:46.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.428769+0000","last_change":"2024-09-21T05:59:37.428769+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.428769+0000","last_undegraded":"2024-09-21T05:59:37.428769+0000","last_fullsized":"2024-09-21T05:59:37.428769+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451326+0000","last_change":"2024-09-21T05:59:37.444089+0000","last_active":"2024-09-21T05:59:37.451326+0000","last_peered":"2024-09-21T05:59:37.451326+0000","last_clean":"2024-09-21T05:59:37.451326+0000","last_became_active":"2024-09-21T05:59:37.443438+0000","last_became_peered":"2024-09-21T05:59:37.443438+0000","last_unstale":"2024-09-21T05:59:37.451326+0000","last_undegraded":"2024-09-21T05:59:37.451326+0000","last_fullsized":"2024-09-21T05:59:37.451326+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:34.733846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.435700+0000","last_change":"2024-09-21T05:59:37.435700+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.435700+0000","last_undegraded":"2024-09-21T05:59:37.435700+0000","last_fullsized":"2024-09-21T05:59:37.435700+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.447853+0000","last_change":"2024-09-21T05:59:37.445284+0000","last_active":"2024-09-21T05:59:37.447853+0000","last_peered":"2024-09-21T05:59:37.447853+0000","last_clean":"2024-09-21T05:59:37.447853+0000","last_became_active":"2024-09-21T05:59:37.444912+0000","last_became_peered":"2024-09-21T05:59:37.444912+0000","last_unstale":"2024-09-21T05:59:37.447853+0000","last_undegraded":"2024-09-21T05:59:37.447853+0000","last_fullsized":"2024-09-21T05:59:37.447853+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:20.236897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429835+0000","last_change":"2024-09-21T05:59:37.429835+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429835+0000","last_undegraded":"2024-09-21T05:59:37.429835+0000","last_fullsized":"2024-09-21T05:59:37.429835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.439195+0000","last_change":"2024-09-21T05:59:37.435640+0000","last_active":"2024-09-21T05:59:37.439195+0000","last_peered":"2024-09-21T05:59:37.439195+0000","last_clean":"2024-09-21T05:59:37.439195+0000","last_became_active":"2024-09-21T05:59:37.435202+0000","last_became_peered":"2024-09-21T05:59:37.435202+0000","last_unstale":"2024-09-21T05:59:37.439195+0000","last_undegraded":"2024-09-21T05:59:37.439195+0000","last_fullsized":"2024-09-21T05:59:37.439195+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:49.312472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431533+0000","last_change":"2024-09-21T05:59:37.431533+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431533+0000","last_undegraded":"2024-09-21T05:59:37.431533+0000","last_fullsized":"2024-09-21T05:59:37.431533+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.439961+0000","last_change":"2024-09-21T05:59:37.432827+0000","last_active":"2024-09-21T05:59:37.439961+0000","last_peered":"2024-09-21T05:59:37.439961+0000","last_clean":"2024-09-21T05:59:37.439961+0000","last_became_active":"2024-09-21T05:59:37.432500+0000","last_became_peered":"2024-09-21T05:59:37.432500+0000","last_unstale":"2024-09-21T05:59:37.439961+0000","last_undegraded":"2024-09-21T05:59:37.439961+0000","last_fullsized":"2024-09-21T05:59:37.439961+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.689460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.433437+0000","last_change":"2024-09-21T05:59:37.433437+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.433437+0000","last_undegraded":"2024-09-21T05:59:37.433437+0000","last_fullsized":"2024-09-21T05:59:37.433437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450224+0000","last_change":"2024-09-21T05:59:37.441674+0000","last_active":"2024-09-21T05:59:37.450224+0000","last_peered":"2024-09-21T05:59:37.450224+0000","last_clean":"2024-09-21T05:59:37.450224+0000","last_became_active":"2024-09-21T05:59:37.441330+0000","last_became_peered":"2024-09-21T05:59:37.441330+0000","last_unstale":"2024-09-21T05:59:37.450224+0000","last_undegraded":"2024-09-21T05:59:37.450224+0000","last_fullsized":"2024-09-21T05:59:37.450224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.968036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.427110+0000","last_change":"2024-09-21T05:59:37.427110+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.427110+0000","last_undegraded":"2024-09-21T05:59:37.427110+0000","last_fullsized":"2024-09-21T05:59:37.427110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450915+0000","last_change":"2024-09-21T05:59:37.447596+0000","last_active":"2024-09-21T05:59:37.450915+0000","last_peered":"2024-09-21T05:59:37.450915+0000","last_clean":"2024-09-21T05:59:37.450915+0000","last_became_active":"2024-09-21T05:59:37.440489+0000","last_became_peered":"2024-09-21T05:59:37.440489+0000","last_unstale":"2024-09-21T05:59:37.450915+0000","last_undegraded":"2024-09-21T05:59:37.450915+0000","last_fullsized":"2024-09-21T05:59:37.450915+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:31.718340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837878+0000","last_change":"2024-09-21T05:59:38.435762+0000","last_active":"2024-09-21T05:59:39.837878+0000","last_peered":"2024-09-21T05:59:39.837878+0000","last_clean":"2024-09-21T05:59:39.837878+0000","last_became_active":"2024-09-21T05:59:38.434259+0000","last_became_peered":"2024-09-21T05:59:38.434259+0000","last_unstale":"2024-09-21T05:59:39.837878+0000","last_undegraded":"2024-09-21T05:59:39.837878+0000","last_fullsized":"2024-09-21T05:59:39.837878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:36.174704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.440034+0000","last_change":"2024-09-21T05:59:37.434300+0000","last_active":"2024-09-21T05:59:37.440034+0000","last_peered":"2024-09-21T05:59:37.440034+0000","last_clean":"2024-09-21T05:59:37.440034+0000","last_became_active":"2024-09-21T05:59:37.433933+0000","last_became_peered":"2024-09-21T05:59:37.433933+0000","last_unstale":"2024-09-21T05:59:37.440034+0000","last_undegraded":"2024-09-21T05:59:37.440034+0000","last_fullsized":"2024-09-21T05:59:37.440034+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.968036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.432848+0000","last_change":"2024-09-21T05:59:37.432848+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.432848+0000","last_undegraded":"2024-09-21T05:59:37.432848+0000","last_fullsized":"2024-09-21T05:59:37.432848+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451837+0000","last_change":"2024-09-21T05:59:37.446607+0000","last_active":"2024-09-21T05:59:37.451837+0000","last_peered":"2024-09-21T05:59:37.451837+0000","last_clean":"2024-09-21T05:59:37.451837+0000","last_became_active":"2024-09-21T05:59:37.446318+0000","last_became_peered":"2024-09-21T05:59:37.446318+0000","last_unstale":"2024-09-21T05:59:37.451837+0000","last_undegraded":"2024-09-21T05:59:37.451837+0000","last_fullsized":"2024-09-21T05:59:37.451837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:59.876514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.434548+0000","last_change":"2024-09-21T05:59:37.434548+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.434548+0000","last_undegraded":"2024-09-21T05:59:37.434548+0000","last_fullsized":"2024-09-21T05:59:37.434548+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.449888+0000","last_change":"2024-09-21T05:59:37.445107+0000","last_active":"2024-09-21T05:59:37.449888+0000","last_peered":"2024-09-21T05:59:37.449888+0000","last_clean":"2024-09-21T05:59:37.449888+0000","last_became_active":"2024-09-21T05:59:37.444746+0000","last_became_peered":"2024-09-21T05:59:37.444746+0000","last_unstale":"2024-09-21T05:59:37.449888+0000","last_undegraded":"2024-09-21T05:59:37.449888+0000","last_fullsized":"2024-09-21T05:59:37.449888+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.968036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.431977+0000","last_change":"2024-09-21T05:59:37.431977+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.431977+0000","last_undegraded":"2024-09-21T05:59:37.431977+0000","last_fullsized":"2024-09-21T05:59:37.431977+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450806+0000","last_change":"2024-09-21T05:59:37.445647+0000","last_active":"2024-09-21T05:59:37.450806+0000","last_peered":"2024-09-21T05:59:37.450806+0000","last_clean":"2024-09-21T05:59:37.450806+0000","last_became_active":"2024-09-21T05:59:37.445165+0000","last_became_peered":"2024-09-21T05:59:37.445165+0000","last_unstale":"2024-09-21T05:59:37.450806+0000","last_undegraded":"2024-09-21T05:59:37.450806+0000","last_fullsized":"2024-09-21T05:59:37.450806+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:42.959574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429475+0000","last_change":"2024-09-21T05:59:37.429475+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429475+0000","last_undegraded":"2024-09-21T05:59:37.429475+0000","last_fullsized":"2024-09-21T05:59:37.429475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.452222+0000","last_change":"2024-09-21T05:59:37.446726+0000","last_active":"2024-09-21T05:59:37.452222+0000","last_peered":"2024-09-21T05:59:37.452222+0000","last_clean":"2024-09-21T05:59:37.452222+0000","last_became_active":"2024-09-21T05:59:37.446299+0000","last_became_peered":"2024-09-21T05:59:37.446299+0000","last_unstale":"2024-09-21T05:59:37.452222+0000","last_undegraded":"2024-09-21T05:59:37.452222+0000","last_fullsized":"2024-09-21T05:59:37.452222+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:31.718340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.429164+0000","last_change":"2024-09-21T05:59:37.429164+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.429164+0000","last_undegraded":"2024-09-21T05:59:37.429164+0000","last_fullsized":"2024-09-21T05:59:37.429164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837548+0000","last_change":"2024-09-21T05:59:37.446929+0000","last_active":"2024-09-21T05:59:39.837548+0000","last_peered":"2024-09-21T05:59:39.837548+0000","last_clean":"2024-09-21T05:59:39.837548+0000","last_became_active":"2024-09-21T05:59:37.446552+0000","last_became_peered":"2024-09-21T05:59:37.446552+0000","last_unstale":"2024-09-21T05:59:39.837548+0000","last_undegraded":"2024-09-21T05:59:39.837548+0000","last_fullsized":"2024-09-21T05:59:39.837548+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:54.236652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425900+0000","last_change":"2024-09-21T05:59:37.425900+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425900+0000","last_undegraded":"2024-09-21T05:59:37.425900+0000","last_fullsized":"2024-09-21T05:59:37.425900+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.450041+0000","last_change":"2024-09-21T05:59:37.446310+0000","last_active":"2024-09-21T05:59:37.450041+0000","last_peered":"2024-09-21T05:59:37.450041+0000","last_clean":"2024-09-21T05:59:37.450041+0000","last_became_active":"2024-09-21T05:59:37.445953+0000","last_became_peered":"2024-09-21T05:59:37.445953+0000","last_unstale":"2024-09-21T05:59:37.450041+0000","last_undegraded":"2024-09-21T05:59:37.450041+0000","last_fullsized":"2024-09-21T05:59:37.450041+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:42.959574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.425968+0000","last_change":"2024-09-21T05:59:37.425968+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.425968+0000","last_undegraded":"2024-09-21T05:59:37.425968+0000","last_fullsized":"2024-09-21T05:59:37.425968+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.439199+0000","last_change":"2024-09-21T05:59:37.435447+0000","last_active":"2024-09-21T05:59:37.439199+0000","last_peered":"2024-09-21T05:59:37.439199+0000","last_clean":"2024-09-21T05:59:37.439199+0000","last_became_active":"2024-09-21T05:59:37.435178+0000","last_became_peered":"2024-09-21T05:59:37.435178+0000","last_unstale":"2024-09-21T05:59:37.439199+0000","last_undegraded":"2024-09-21T05:59:37.439199+0000","last_fullsized":"2024-09-21T05:59:37.439199+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:24.390872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.424041+0000","last_change":"2024-09-21T05:59:37.424041+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.424041+0000","last_undegraded":"2024-09-21T05:59:37.424041+0000","last_fullsized":"2024-09-21T05:59:37.424041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837563+0000","last_change":"2024-09-21T05:59:37.441729+0000","last_active":"2024-09-21T05:59:39.837563+0000","last_peered":"2024-09-21T05:59:39.837563+0000","last_clean":"2024-09-21T05:59:39.837563+0000","last_became_active":"2024-09-21T05:59:37.441370+0000","last_became_peered":"2024-09-21T05:59:37.441370+0000","last_unstale":"2024-09-21T05:59:39.837563+0000","last_undegraded":"2024-09-21T05:59:39.837563+0000","last_fullsized":"2024-09-21T05:59:39.837563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:28.903549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.422407+0000","last_change":"2024-09-21T05:59:37.422407+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.422407+0000","last_undegraded":"2024-09-21T05:59:37.422407+0000","last_fullsized":"2024-09-21T05:59:37.422407+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.451290+0000","last_change":"2024-09-21T05:59:37.442907+0000","last_active":"2024-09-21T05:59:37.451290+0000","last_peered":"2024-09-21T05:59:37.451290+0000","last_clean":"2024-09-21T05:59:37.451290+0000","last_became_active":"2024-09-21T05:59:37.439722+0000","last_became_peered":"2024-09-21T05:59:37.439722+0000","last_unstale":"2024-09-21T05:59:37.451290+0000","last_undegraded":"2024-09-21T05:59:37.451290+0000","last_fullsized":"2024-09-21T05:59:37.451290+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.103990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-09-21T05:59:37.430505+0000","last_change":"2024-09-21T05:59:37.430505+0000","last_active":"2024-09-21T05:59:37.411723+0000","last_peered":"2024-09-21T05:59:37.411723+0000","last_clean":"2024-09-21T05:59:37.411723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:59:37.430505+0000","last_undegraded":"2024-09-21T05:59:37.430505+0000","last_fullsized":"2024-09-21T05:59:37.430505+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.440071+0000","last_change":"2024-09-21T05:59:37.435112+0000","last_active":"2024-09-21T05:59:37.440071+0000","last_peered":"2024-09-21T05:59:37.440071+0000","last_clean":"2024-09-21T05:59:37.440071+0000","last_became_active":"2024-09-21T05:59:37.434789+0000","last_became_peered":"2024-09-21T05:59:37.434789+0000","last_unstale":"2024-09-21T05:59:37.440071+0000","last_undegraded":"2024-09-21T05:59:37.440071+0000","last_fullsized":"2024-09-21T05:59:37.440071+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:46.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":148,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T05:59:37.436248+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:59:37.436248+0000","last_peered":"2024-09-21T05:59:37.436248+0000","last_clean":"2024-09-21T05:59:37.436248+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:59:37.436248+0000","last_undegraded":"2024-09-21T05:59:37.436248+0000","last_fullsized":"2024-09-21T05:59:37.436248+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1077248,"data_stored":779961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":471040,"data_stored":183326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31664,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340176,"statfs":{"total":96636764160,"available":96604340224,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31664,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340176,"statfs":{"total":96636764160,"available":96604340224,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35824,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336016,"statfs":{"total":96636764160,"available":96600080384,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36400,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335440,"statfs":{"total":96636764160,"available":96599490560,"internally_reserved":0,"allocated":1077248,"data_stored":779961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40008,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331832,"statfs":{"total":96636764160,"available":96595795968,"internally_reserved":0,"allocated":512000,"data_stored":202738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:59:47.040 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:59:47.044 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:59:47.331 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:59:47.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.499+0000 7fe1dda00640 1 -- 172.21.15.50:0/3594729501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1d80b54f0 msgr2=0x7fe1d815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:47.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.499+0000 7fe1dda00640 1 --2- 172.21.15.50:0/3594729501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1d80b54f0 0x7fe1d815bb60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009bc0 tx=0x7fe1d001cae0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.499+0000 7fe1dda00640 1 -- 172.21.15.50:0/3594729501 shutdown_connections 2024-09-21T05:59:47.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.499+0000 7fe1dda00640 1 --2- 172.21.15.50:0/3594729501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1d80b54f0 0x7fe1d815bb60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.499+0000 7fe1dda00640 1 -- 172.21.15.50:0/3594729501 >> 172.21.15.50:0/3594729501 conn(0x7fe1d80b2040 msgr2=0x7fe1d80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:47.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.499+0000 7fe1dda00640 1 -- 172.21.15.50:0/3594729501 shutdown_connections 2024-09-21T05:59:47.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.499+0000 7fe1dda00640 1 -- 172.21.15.50:0/3594729501 wait complete. 2024-09-21T05:59:47.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1dda00640 1 Processor -- start 2024-09-21T05:59:47.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1dda00640 1 -- start start 2024-09-21T05:59:47.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1dda00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1d80b5c70 0x7fe1d813db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:47.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1dda00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1d81527b0 0x7fe1d814f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:47.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1dda00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe1d8151db0 0x7fe1d814f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1dda00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d8169bd0 con 0x7fe1d81527b0 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1dda00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d8169a50 con 0x7fe1d80b5c70 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1dda00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d8169d50 con 0x7fe1d8151db0 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1cfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1d81527b0 0x7fe1d814f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1cfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1d81527b0 0x7fe1d814f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59942/0 (socket says 172.21.15.50:59942) 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1cfe00640 1 -- 172.21.15.50:0/3736261096 learned_addr learned my addr 172.21.15.50:0/3736261096 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1d6a00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1d80b5c70 0x7fe1d813db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1d7400640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe1d8151db0 0x7fe1d814f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1cfe00640 1 -- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe1d8151db0 msgr2=0x7fe1d814f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1cfe00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe1d8151db0 0x7fe1d814f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1cfe00640 1 -- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1d80b5c70 msgr2=0x7fe1d813db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1cfe00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1d80b5c70 0x7fe1d813db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1cfe00640 1 -- 172.21.15.50:0/3736261096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d8157d50 con 0x7fe1d81527b0 2024-09-21T05:59:47.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1d6a00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1d80b5c70 0x7fe1d813db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:47.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1d7400640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe1d8151db0 0x7fe1d814f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:47.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1cfe00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1d81527b0 0x7fe1d814f420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c420 tx=0x7fe1c000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:47.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1cea00640 1 -- 172.21.15.50:0/3736261096 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0011790 con 0x7fe1d81527b0 2024-09-21T05:59:47.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1cea00640 1 -- 172.21.15.50:0/3736261096 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1c0017070 con 0x7fe1d81527b0 2024-09-21T05:59:47.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1cea00640 1 -- 172.21.15.50:0/3736261096 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0011ce0 con 0x7fe1d81527b0 2024-09-21T05:59:47.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1dda00640 1 -- 172.21.15.50:0/3736261096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d8158030 con 0x7fe1d81527b0 2024-09-21T05:59:47.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.503+0000 7fe1dda00640 1 -- 172.21.15.50:0/3736261096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d81585f0 con 0x7fe1d81527b0 2024-09-21T05:59:47.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.507+0000 7fe1dda00640 1 -- 172.21.15.50:0/3736261096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d815bb60 con 0x7fe1d81527b0 2024-09-21T05:59:47.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.507+0000 7fe1cea00640 1 -- 172.21.15.50:0/3736261096 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fe1c001c460 con 0x7fe1d81527b0 2024-09-21T05:59:47.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.507+0000 7fe1cea00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe1b40734d0 0x7fe1b40759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:47.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.507+0000 7fe1cea00640 1 -- 172.21.15.50:0/3736261096 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fe1c00946d0 con 0x7fe1d81527b0 2024-09-21T05:59:47.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.507+0000 7fe1d6a00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe1b40734d0 0x7fe1b40759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:47.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.507+0000 7fe1d6a00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe1b40734d0 0x7fe1b40759e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8001630 tx=0x7fe1c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:47.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.511+0000 7fe1cea00640 1 -- 172.21.15.50:0/3736261096 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe1c005f1b0 con 0x7fe1d81527b0 2024-09-21T05:59:47.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.687+0000 7fe1dda00640 1 -- 172.21.15.50:0/3736261096 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1d806b540 con 0x7fe1b40734d0 2024-09-21T05:59:47.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.691+0000 7fe1cea00640 1 -- 172.21.15.50:0/3736261096 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251949 (secure 0 0 0) 0x7fe1c800d4a0 con 0x7fe1b40734d0 2024-09-21T05:59:47.699 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:47.700 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T05:59:47.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.695+0000 7fe1dda00640 1 -- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe1b40734d0 msgr2=0x7fe1b40759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:47.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.695+0000 7fe1dda00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe1b40734d0 0x7fe1b40759e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8001630 tx=0x7fe1c8009040 comp rx=0 tx=0).stop 2024-09-21T05:59:47.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.695+0000 7fe1dda00640 1 -- 172.21.15.50:0/3736261096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1d81527b0 msgr2=0x7fe1d814f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:47.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.695+0000 7fe1dda00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1d81527b0 0x7fe1d814f420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c420 tx=0x7fe1c000c8e0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.695+0000 7fe1dda00640 1 -- 172.21.15.50:0/3736261096 shutdown_connections 2024-09-21T05:59:47.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.695+0000 7fe1dda00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe1d8151db0 0x7fe1d814f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.695+0000 7fe1dda00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe1b40734d0 0x7fe1b40759e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.695+0000 7fe1dda00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1d81527b0 0x7fe1d814f420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.695+0000 7fe1dda00640 1 --2- 172.21.15.50:0/3736261096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1d80b5c70 0x7fe1d813db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.695+0000 7fe1dda00640 1 -- 172.21.15.50:0/3736261096 >> 172.21.15.50:0/3736261096 conn(0x7fe1d80b2040 msgr2=0x7fe1d8159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:47.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.695+0000 7fe1dda00640 1 -- 172.21.15.50:0/3736261096 shutdown_connections 2024-09-21T05:59:47.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.695+0000 7fe1dda00640 1 -- 172.21.15.50:0/3736261096 wait complete. 2024-09-21T05:59:47.721 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-21T05:59:46.935735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275516,"kb_used_data":5192,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754699204,"statfs":{"total":773094113280,"available":772811984896,"internally_reserved":0,"allocated":5316608,"data_stored":2806900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.513390"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.967173+0000","last_change":"2024-09-21T05:59:38.449048+0000","last_active":"2024-09-21T05:59:39.967173+0000","last_peered":"2024-09-21T05:59:39.967173+0000","last_clean":"2024-09-21T05:59:39.967173+0000","last_became_active":"2024-09-21T05:59:38.448636+0000","last_became_peered":"2024-09-21T05:59:38.448636+0000","last_unstale":"2024-09-21T05:59:39.967173+0000","last_undegraded":"2024-09-21T05:59:39.967173+0000","last_fullsized":"2024-09-21T05:59:39.967173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:42.950593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.442735+0000","last_change":"2024-09-21T05:59:38.465705+0000","last_active":"2024-09-21T05:59:39.442735+0000","last_peered":"2024-09-21T05:59:39.442735+0000","last_clean":"2024-09-21T05:59:39.442735+0000","last_became_active":"2024-09-21T05:59:38.465366+0000","last_became_peered":"2024-09-21T05:59:38.465366+0000","last_unstale":"2024-09-21T05:59:39.442735+0000","last_undegraded":"2024-09-21T05:59:39.442735+0000","last_fullsized":"2024-09-21T05:59:39.442735+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:38.772869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411399+0000","last_change":"2024-09-21T05:59:38.442712+0000","last_active":"2024-09-21T05:59:40.411399+0000","last_peered":"2024-09-21T05:59:40.411399+0000","last_clean":"2024-09-21T05:59:40.411399+0000","last_became_active":"2024-09-21T05:59:38.442090+0000","last_became_peered":"2024-09-21T05:59:38.442090+0000","last_unstale":"2024-09-21T05:59:40.411399+0000","last_undegraded":"2024-09-21T05:59:40.411399+0000","last_fullsized":"2024-09-21T05:59:40.411399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:58.858234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412624+0000","last_change":"2024-09-21T05:59:38.459501+0000","last_active":"2024-09-21T05:59:40.412624+0000","last_peered":"2024-09-21T05:59:40.412624+0000","last_clean":"2024-09-21T05:59:40.412624+0000","last_became_active":"2024-09-21T05:59:38.459133+0000","last_became_peered":"2024-09-21T05:59:38.459133+0000","last_unstale":"2024-09-21T05:59:40.412624+0000","last_undegraded":"2024-09-21T05:59:40.412624+0000","last_fullsized":"2024-09-21T05:59:40.412624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:14.476988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.442608+0000","last_change":"2024-09-21T05:59:38.454668+0000","last_active":"2024-09-21T05:59:39.442608+0000","last_peered":"2024-09-21T05:59:39.442608+0000","last_clean":"2024-09-21T05:59:39.442608+0000","last_became_active":"2024-09-21T05:59:38.454302+0000","last_became_peered":"2024-09-21T05:59:38.454302+0000","last_unstale":"2024-09-21T05:59:39.442608+0000","last_undegraded":"2024-09-21T05:59:39.442608+0000","last_fullsized":"2024-09-21T05:59:39.442608+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:53.757813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411881+0000","last_change":"2024-09-21T05:59:38.459209+0000","last_active":"2024-09-21T05:59:40.411881+0000","last_peered":"2024-09-21T05:59:40.411881+0000","last_clean":"2024-09-21T05:59:40.411881+0000","last_became_active":"2024-09-21T05:59:38.458890+0000","last_became_peered":"2024-09-21T05:59:38.458890+0000","last_unstale":"2024-09-21T05:59:40.411881+0000","last_undegraded":"2024-09-21T05:59:40.411881+0000","last_fullsized":"2024-09-21T05:59:40.411881+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:42.378996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.967543+0000","last_change":"2024-09-21T05:59:38.449392+0000","last_active":"2024-09-21T05:59:39.967543+0000","last_peered":"2024-09-21T05:59:39.967543+0000","last_clean":"2024-09-21T05:59:39.967543+0000","last_became_active":"2024-09-21T05:59:38.448662+0000","last_became_peered":"2024-09-21T05:59:38.448662+0000","last_unstale":"2024-09-21T05:59:39.967543+0000","last_undegraded":"2024-09-21T05:59:39.967543+0000","last_fullsized":"2024-09-21T05:59:39.967543+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:02.309059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411661+0000","last_change":"2024-09-21T05:59:38.459699+0000","last_active":"2024-09-21T05:59:40.411661+0000","last_peered":"2024-09-21T05:59:40.411661+0000","last_clean":"2024-09-21T05:59:40.411661+0000","last_became_active":"2024-09-21T05:59:38.456083+0000","last_became_peered":"2024-09-21T05:59:38.456083+0000","last_unstale":"2024-09-21T05:59:40.411661+0000","last_undegraded":"2024-09-21T05:59:40.411661+0000","last_fullsized":"2024-09-21T05:59:40.411661+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:37.270992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.413308+0000","last_change":"2024-09-21T05:59:38.455004+0000","last_active":"2024-09-21T05:59:40.413308+0000","last_peered":"2024-09-21T05:59:40.413308+0000","last_clean":"2024-09-21T05:59:40.413308+0000","last_became_active":"2024-09-21T05:59:38.454676+0000","last_became_peered":"2024-09-21T05:59:38.454676+0000","last_unstale":"2024-09-21T05:59:40.413308+0000","last_undegraded":"2024-09-21T05:59:40.413308+0000","last_fullsized":"2024-09-21T05:59:40.413308+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:42.950593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412771+0000","last_change":"2024-09-21T05:59:38.458371+0000","last_active":"2024-09-21T05:59:40.412771+0000","last_peered":"2024-09-21T05:59:40.412771+0000","last_clean":"2024-09-21T05:59:40.412771+0000","last_became_active":"2024-09-21T05:59:38.457668+0000","last_became_peered":"2024-09-21T05:59:38.457668+0000","last_unstale":"2024-09-21T05:59:40.412771+0000","last_undegraded":"2024-09-21T05:59:40.412771+0000","last_fullsized":"2024-09-21T05:59:40.412771+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:42.213468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412424+0000","last_change":"2024-09-21T05:59:38.453907+0000","last_active":"2024-09-21T05:59:40.412424+0000","last_peered":"2024-09-21T05:59:40.412424+0000","last_clean":"2024-09-21T05:59:40.412424+0000","last_became_active":"2024-09-21T05:59:38.453580+0000","last_became_peered":"2024-09-21T05:59:38.453580+0000","last_unstale":"2024-09-21T05:59:40.412424+0000","last_undegraded":"2024-09-21T05:59:40.412424+0000","last_fullsized":"2024-09-21T05:59:40.412424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:31.513804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.443902+0000","last_change":"2024-09-21T05:59:38.452268+0000","last_active":"2024-09-21T05:59:39.443902+0000","last_peered":"2024-09-21T05:59:39.443902+0000","last_clean":"2024-09-21T05:59:39.443902+0000","last_became_active":"2024-09-21T05:59:38.451981+0000","last_became_peered":"2024-09-21T05:59:38.451981+0000","last_unstale":"2024-09-21T05:59:39.443902+0000","last_undegraded":"2024-09-21T05:59:39.443902+0000","last_fullsized":"2024-09-21T05:59:39.443902+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:39.224418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.440156+0000","last_change":"2024-09-21T05:59:38.466355+0000","last_active":"2024-09-21T05:59:39.440156+0000","last_peered":"2024-09-21T05:59:39.440156+0000","last_clean":"2024-09-21T05:59:39.440156+0000","last_became_active":"2024-09-21T05:59:38.466032+0000","last_became_peered":"2024-09-21T05:59:38.466032+0000","last_unstale":"2024-09-21T05:59:39.440156+0000","last_undegraded":"2024-09-21T05:59:39.440156+0000","last_fullsized":"2024-09-21T05:59:39.440156+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:50.315580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411983+0000","last_change":"2024-09-21T05:59:38.455460+0000","last_active":"2024-09-21T05:59:40.411983+0000","last_peered":"2024-09-21T05:59:40.411983+0000","last_clean":"2024-09-21T05:59:40.411983+0000","last_became_active":"2024-09-21T05:59:38.454812+0000","last_became_peered":"2024-09-21T05:59:38.454812+0000","last_unstale":"2024-09-21T05:59:40.411983+0000","last_undegraded":"2024-09-21T05:59:40.411983+0000","last_fullsized":"2024-09-21T05:59:40.411983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:05.192078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412155+0000","last_change":"2024-09-21T05:59:38.447312+0000","last_active":"2024-09-21T05:59:40.412155+0000","last_peered":"2024-09-21T05:59:40.412155+0000","last_clean":"2024-09-21T05:59:40.412155+0000","last_became_active":"2024-09-21T05:59:38.444363+0000","last_became_peered":"2024-09-21T05:59:38.444363+0000","last_unstale":"2024-09-21T05:59:40.412155+0000","last_undegraded":"2024-09-21T05:59:40.412155+0000","last_fullsized":"2024-09-21T05:59:40.412155+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:47.514069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.442537+0000","last_change":"2024-09-21T05:59:38.462747+0000","last_active":"2024-09-21T05:59:39.442537+0000","last_peered":"2024-09-21T05:59:39.442537+0000","last_clean":"2024-09-21T05:59:39.442537+0000","last_became_active":"2024-09-21T05:59:38.462408+0000","last_became_peered":"2024-09-21T05:59:38.462408+0000","last_unstale":"2024-09-21T05:59:39.442537+0000","last_undegraded":"2024-09-21T05:59:39.442537+0000","last_fullsized":"2024-09-21T05:59:39.442537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:11.042214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411506+0000","last_change":"2024-09-21T05:59:38.438531+0000","last_active":"2024-09-21T05:59:40.411506+0000","last_peered":"2024-09-21T05:59:40.411506+0000","last_clean":"2024-09-21T05:59:40.411506+0000","last_became_active":"2024-09-21T05:59:38.437635+0000","last_became_peered":"2024-09-21T05:59:38.437635+0000","last_unstale":"2024-09-21T05:59:40.411506+0000","last_undegraded":"2024-09-21T05:59:40.411506+0000","last_fullsized":"2024-09-21T05:59:40.411506+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:31.513804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411443+0000","last_change":"2024-09-21T05:59:38.466387+0000","last_active":"2024-09-21T05:59:40.411443+0000","last_peered":"2024-09-21T05:59:40.411443+0000","last_clean":"2024-09-21T05:59:40.411443+0000","last_became_active":"2024-09-21T05:59:38.465698+0000","last_became_peered":"2024-09-21T05:59:38.465698+0000","last_unstale":"2024-09-21T05:59:40.411443+0000","last_undegraded":"2024-09-21T05:59:40.411443+0000","last_fullsized":"2024-09-21T05:59:40.411443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:05.192078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.443942+0000","last_change":"2024-09-21T05:59:38.455543+0000","last_active":"2024-09-21T05:59:39.443942+0000","last_peered":"2024-09-21T05:59:39.443942+0000","last_clean":"2024-09-21T05:59:39.443942+0000","last_became_active":"2024-09-21T05:59:38.453893+0000","last_became_peered":"2024-09-21T05:59:38.453893+0000","last_unstale":"2024-09-21T05:59:39.443942+0000","last_undegraded":"2024-09-21T05:59:39.443942+0000","last_fullsized":"2024-09-21T05:59:39.443942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:43.962683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.439774+0000","last_change":"2024-09-21T05:59:38.469635+0000","last_active":"2024-09-21T05:59:39.439774+0000","last_peered":"2024-09-21T05:59:39.439774+0000","last_clean":"2024-09-21T05:59:39.439774+0000","last_became_active":"2024-09-21T05:59:38.469103+0000","last_became_peered":"2024-09-21T05:59:38.469103+0000","last_unstale":"2024-09-21T05:59:39.439774+0000","last_undegraded":"2024-09-21T05:59:39.439774+0000","last_fullsized":"2024-09-21T05:59:39.439774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:22.100431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412055+0000","last_change":"2024-09-21T05:59:38.454552+0000","last_active":"2024-09-21T05:59:40.412055+0000","last_peered":"2024-09-21T05:59:40.412055+0000","last_clean":"2024-09-21T05:59:40.412055+0000","last_became_active":"2024-09-21T05:59:38.448444+0000","last_became_peered":"2024-09-21T05:59:38.448444+0000","last_unstale":"2024-09-21T05:59:40.412055+0000","last_undegraded":"2024-09-21T05:59:40.412055+0000","last_fullsized":"2024-09-21T05:59:40.412055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:51.322134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412209+0000","last_change":"2024-09-21T05:59:38.455157+0000","last_active":"2024-09-21T05:59:40.412209+0000","last_peered":"2024-09-21T05:59:40.412209+0000","last_clean":"2024-09-21T05:59:40.412209+0000","last_became_active":"2024-09-21T05:59:38.454495+0000","last_became_peered":"2024-09-21T05:59:38.454495+0000","last_unstale":"2024-09-21T05:59:40.412209+0000","last_undegraded":"2024-09-21T05:59:40.412209+0000","last_fullsized":"2024-09-21T05:59:40.412209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:52.379439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.443115+0000","last_change":"2024-09-21T05:59:38.447773+0000","last_active":"2024-09-21T05:59:39.443115+0000","last_peered":"2024-09-21T05:59:39.443115+0000","last_clean":"2024-09-21T05:59:39.443115+0000","last_became_active":"2024-09-21T05:59:38.447397+0000","last_became_peered":"2024-09-21T05:59:38.447397+0000","last_unstale":"2024-09-21T05:59:39.443115+0000","last_undegraded":"2024-09-21T05:59:39.443115+0000","last_fullsized":"2024-09-21T05:59:39.443115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:43.696347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.442793+0000","last_change":"2024-09-21T05:59:38.451958+0000","last_active":"2024-09-21T05:59:39.442793+0000","last_peered":"2024-09-21T05:59:39.442793+0000","last_clean":"2024-09-21T05:59:39.442793+0000","last_became_active":"2024-09-21T05:59:38.451650+0000","last_became_peered":"2024-09-21T05:59:38.451650+0000","last_unstale":"2024-09-21T05:59:39.442793+0000","last_undegraded":"2024-09-21T05:59:39.442793+0000","last_fullsized":"2024-09-21T05:59:39.442793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:27.933737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.443889+0000","last_change":"2024-09-21T05:59:38.449106+0000","last_active":"2024-09-21T05:59:39.443889+0000","last_peered":"2024-09-21T05:59:39.443889+0000","last_clean":"2024-09-21T05:59:39.443889+0000","last_became_active":"2024-09-21T05:59:38.448565+0000","last_became_peered":"2024-09-21T05:59:38.448565+0000","last_unstale":"2024-09-21T05:59:39.443889+0000","last_undegraded":"2024-09-21T05:59:39.443889+0000","last_fullsized":"2024-09-21T05:59:39.443889+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:47.803152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.440042+0000","last_change":"2024-09-21T05:59:38.467839+0000","last_active":"2024-09-21T05:59:39.440042+0000","last_peered":"2024-09-21T05:59:39.440042+0000","last_clean":"2024-09-21T05:59:39.440042+0000","last_became_active":"2024-09-21T05:59:38.466905+0000","last_became_peered":"2024-09-21T05:59:38.466905+0000","last_unstale":"2024-09-21T05:59:39.440042+0000","last_undegraded":"2024-09-21T05:59:39.440042+0000","last_fullsized":"2024-09-21T05:59:39.440042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:40.278959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411361+0000","last_change":"2024-09-21T05:59:38.437940+0000","last_active":"2024-09-21T05:59:40.411361+0000","last_peered":"2024-09-21T05:59:40.411361+0000","last_clean":"2024-09-21T05:59:40.411361+0000","last_became_active":"2024-09-21T05:59:38.437605+0000","last_became_peered":"2024-09-21T05:59:38.437605+0000","last_unstale":"2024-09-21T05:59:40.411361+0000","last_undegraded":"2024-09-21T05:59:40.411361+0000","last_fullsized":"2024-09-21T05:59:40.411361+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:23.012107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.443951+0000","last_change":"2024-09-21T05:59:38.452152+0000","last_active":"2024-09-21T05:59:39.443951+0000","last_peered":"2024-09-21T05:59:39.443951+0000","last_clean":"2024-09-21T05:59:39.443951+0000","last_became_active":"2024-09-21T05:59:38.451864+0000","last_became_peered":"2024-09-21T05:59:38.451864+0000","last_unstale":"2024-09-21T05:59:39.443951+0000","last_undegraded":"2024-09-21T05:59:39.443951+0000","last_fullsized":"2024-09-21T05:59:39.443951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:14:18.877455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.443222+0000","last_change":"2024-09-21T05:59:38.448702+0000","last_active":"2024-09-21T05:59:39.443222+0000","last_peered":"2024-09-21T05:59:39.443222+0000","last_clean":"2024-09-21T05:59:39.443222+0000","last_became_active":"2024-09-21T05:59:38.446087+0000","last_became_peered":"2024-09-21T05:59:38.446087+0000","last_unstale":"2024-09-21T05:59:39.443222+0000","last_undegraded":"2024-09-21T05:59:39.443222+0000","last_fullsized":"2024-09-21T05:59:39.443222+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:22.372992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411187+0000","last_change":"2024-09-21T05:59:38.444927+0000","last_active":"2024-09-21T05:59:40.411187+0000","last_peered":"2024-09-21T05:59:40.411187+0000","last_clean":"2024-09-21T05:59:40.411187+0000","last_became_active":"2024-09-21T05:59:38.444112+0000","last_became_peered":"2024-09-21T05:59:38.444112+0000","last_unstale":"2024-09-21T05:59:40.411187+0000","last_undegraded":"2024-09-21T05:59:40.411187+0000","last_fullsized":"2024-09-21T05:59:40.411187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:20.968785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412462+0000","last_change":"2024-09-21T05:59:38.441054+0000","last_active":"2024-09-21T05:59:40.412462+0000","last_peered":"2024-09-21T05:59:40.412462+0000","last_clean":"2024-09-21T05:59:40.412462+0000","last_became_active":"2024-09-21T05:59:38.440701+0000","last_became_peered":"2024-09-21T05:59:38.440701+0000","last_unstale":"2024-09-21T05:59:40.412462+0000","last_undegraded":"2024-09-21T05:59:40.412462+0000","last_fullsized":"2024-09-21T05:59:40.412462+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:43.156953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.439631+0000","last_change":"2024-09-21T05:59:38.438532+0000","last_active":"2024-09-21T05:59:39.439631+0000","last_peered":"2024-09-21T05:59:39.439631+0000","last_clean":"2024-09-21T05:59:39.439631+0000","last_became_active":"2024-09-21T05:59:38.438152+0000","last_became_peered":"2024-09-21T05:59:38.438152+0000","last_unstale":"2024-09-21T05:59:39.439631+0000","last_undegraded":"2024-09-21T05:59:39.439631+0000","last_fullsized":"2024-09-21T05:59:39.439631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:00.879622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837865+0000","last_change":"2024-09-21T05:59:38.462061+0000","last_active":"2024-09-21T05:59:39.837865+0000","last_peered":"2024-09-21T05:59:39.837865+0000","last_clean":"2024-09-21T05:59:39.837865+0000","last_became_active":"2024-09-21T05:59:38.461740+0000","last_became_peered":"2024-09-21T05:59:38.461740+0000","last_unstale":"2024-09-21T05:59:39.837865+0000","last_undegraded":"2024-09-21T05:59:39.837865+0000","last_fullsized":"2024-09-21T05:59:39.837865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:17.985198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":19,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.413344+0000","last_change":"2024-09-21T05:59:37.447751+0000","last_active":"2024-09-21T05:59:40.413344+0000","last_peered":"2024-09-21T05:59:40.413344+0000","last_clean":"2024-09-21T05:59:40.413344+0000","last_became_active":"2024-09-21T05:59:37.447392+0000","last_became_peered":"2024-09-21T05:59:37.447392+0000","last_unstale":"2024-09-21T05:59:40.413344+0000","last_undegraded":"2024-09-21T05:59:40.413344+0000","last_fullsized":"2024-09-21T05:59:40.413344+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:31.303606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.444046+0000","last_change":"2024-09-21T05:59:38.443111+0000","last_active":"2024-09-21T05:59:39.444046+0000","last_peered":"2024-09-21T05:59:39.444046+0000","last_clean":"2024-09-21T05:59:39.444046+0000","last_became_active":"2024-09-21T05:59:38.442828+0000","last_became_peered":"2024-09-21T05:59:38.442828+0000","last_unstale":"2024-09-21T05:59:39.444046+0000","last_undegraded":"2024-09-21T05:59:39.444046+0000","last_fullsized":"2024-09-21T05:59:39.444046+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:32.721448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411720+0000","last_change":"2024-09-21T05:59:37.443312+0000","last_active":"2024-09-21T05:59:40.411720+0000","last_peered":"2024-09-21T05:59:40.411720+0000","last_clean":"2024-09-21T05:59:40.411720+0000","last_became_active":"2024-09-21T05:59:37.442457+0000","last_became_peered":"2024-09-21T05:59:37.442457+0000","last_unstale":"2024-09-21T05:59:40.411720+0000","last_undegraded":"2024-09-21T05:59:40.411720+0000","last_fullsized":"2024-09-21T05:59:40.411720+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:12.254762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.444159+0000","last_change":"2024-09-21T05:59:38.445237+0000","last_active":"2024-09-21T05:59:39.444159+0000","last_peered":"2024-09-21T05:59:39.444159+0000","last_clean":"2024-09-21T05:59:39.444159+0000","last_became_active":"2024-09-21T05:59:38.444927+0000","last_became_peered":"2024-09-21T05:59:38.444927+0000","last_unstale":"2024-09-21T05:59:39.444159+0000","last_undegraded":"2024-09-21T05:59:39.444159+0000","last_fullsized":"2024-09-21T05:59:39.444159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:16.488422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837882+0000","last_change":"2024-09-21T05:59:37.440271+0000","last_active":"2024-09-21T05:59:39.837882+0000","last_peered":"2024-09-21T05:59:39.837882+0000","last_clean":"2024-09-21T05:59:39.837882+0000","last_became_active":"2024-09-21T05:59:37.439909+0000","last_became_peered":"2024-09-21T05:59:37.439909+0000","last_unstale":"2024-09-21T05:59:39.837882+0000","last_undegraded":"2024-09-21T05:59:39.837882+0000","last_fullsized":"2024-09-21T05:59:39.837882+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:15.657492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837975+0000","last_change":"2024-09-21T05:59:38.444285+0000","last_active":"2024-09-21T05:59:39.837975+0000","last_peered":"2024-09-21T05:59:39.837975+0000","last_clean":"2024-09-21T05:59:39.837975+0000","last_became_active":"2024-09-21T05:59:38.443952+0000","last_became_peered":"2024-09-21T05:59:38.443952+0000","last_unstale":"2024-09-21T05:59:39.837975+0000","last_undegraded":"2024-09-21T05:59:39.837975+0000","last_fullsized":"2024-09-21T05:59:39.837975+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:31.815943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.966813+0000","last_change":"2024-09-21T05:59:37.443607+0000","last_active":"2024-09-21T05:59:39.966813+0000","last_peered":"2024-09-21T05:59:39.966813+0000","last_clean":"2024-09-21T05:59:39.966813+0000","last_became_active":"2024-09-21T05:59:37.443262+0000","last_became_peered":"2024-09-21T05:59:37.443262+0000","last_unstale":"2024-09-21T05:59:39.966813+0000","last_undegraded":"2024-09-21T05:59:39.966813+0000","last_fullsized":"2024-09-21T05:59:39.966813+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:22.008999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411177+0000","last_change":"2024-09-21T05:59:38.447845+0000","last_active":"2024-09-21T05:59:40.411177+0000","last_peered":"2024-09-21T05:59:40.411177+0000","last_clean":"2024-09-21T05:59:40.411177+0000","last_became_active":"2024-09-21T05:59:38.447498+0000","last_became_peered":"2024-09-21T05:59:38.447498+0000","last_unstale":"2024-09-21T05:59:40.411177+0000","last_undegraded":"2024-09-21T05:59:40.411177+0000","last_fullsized":"2024-09-21T05:59:40.411177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:53.030843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412539+0000","last_change":"2024-09-21T05:59:37.447920+0000","last_active":"2024-09-21T05:59:40.412539+0000","last_peered":"2024-09-21T05:59:40.412539+0000","last_clean":"2024-09-21T05:59:40.412539+0000","last_became_active":"2024-09-21T05:59:37.447240+0000","last_became_peered":"2024-09-21T05:59:37.447240+0000","last_unstale":"2024-09-21T05:59:40.412539+0000","last_undegraded":"2024-09-21T05:59:40.412539+0000","last_fullsized":"2024-09-21T05:59:40.412539+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:37.769761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412529+0000","last_change":"2024-09-21T05:59:38.468510+0000","last_active":"2024-09-21T05:59:40.412529+0000","last_peered":"2024-09-21T05:59:40.412529+0000","last_clean":"2024-09-21T05:59:40.412529+0000","last_became_active":"2024-09-21T05:59:38.465517+0000","last_became_peered":"2024-09-21T05:59:38.465517+0000","last_unstale":"2024-09-21T05:59:40.412529+0000","last_undegraded":"2024-09-21T05:59:40.412529+0000","last_fullsized":"2024-09-21T05:59:40.412529+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:36.347527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411437+0000","last_change":"2024-09-21T05:59:37.429073+0000","last_active":"2024-09-21T05:59:40.411437+0000","last_peered":"2024-09-21T05:59:40.411437+0000","last_clean":"2024-09-21T05:59:40.411437+0000","last_became_active":"2024-09-21T05:59:37.428733+0000","last_became_peered":"2024-09-21T05:59:37.428733+0000","last_unstale":"2024-09-21T05:59:40.411437+0000","last_undegraded":"2024-09-21T05:59:40.411437+0000","last_fullsized":"2024-09-21T05:59:40.411437+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:52.710822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411752+0000","last_change":"2024-09-21T05:59:38.447347+0000","last_active":"2024-09-21T05:59:40.411752+0000","last_peered":"2024-09-21T05:59:40.411752+0000","last_clean":"2024-09-21T05:59:40.411752+0000","last_became_active":"2024-09-21T05:59:38.447016+0000","last_became_peered":"2024-09-21T05:59:38.447016+0000","last_unstale":"2024-09-21T05:59:40.411752+0000","last_undegraded":"2024-09-21T05:59:40.411752+0000","last_fullsized":"2024-09-21T05:59:40.411752+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:58.858234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.442242+0000","last_change":"2024-09-21T05:59:37.437489+0000","last_active":"2024-09-21T05:59:39.442242+0000","last_peered":"2024-09-21T05:59:39.442242+0000","last_clean":"2024-09-21T05:59:39.442242+0000","last_became_active":"2024-09-21T05:59:37.437170+0000","last_became_peered":"2024-09-21T05:59:37.437170+0000","last_unstale":"2024-09-21T05:59:39.442242+0000","last_undegraded":"2024-09-21T05:59:39.442242+0000","last_fullsized":"2024-09-21T05:59:39.442242+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:34.218775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.442816+0000","last_change":"2024-09-21T05:59:38.453467+0000","last_active":"2024-09-21T05:59:39.442816+0000","last_peered":"2024-09-21T05:59:39.442816+0000","last_clean":"2024-09-21T05:59:39.442816+0000","last_became_active":"2024-09-21T05:59:38.453120+0000","last_became_peered":"2024-09-21T05:59:38.453120+0000","last_unstale":"2024-09-21T05:59:39.442816+0000","last_undegraded":"2024-09-21T05:59:39.442816+0000","last_fullsized":"2024-09-21T05:59:39.442816+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:21.931133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411909+0000","last_change":"2024-09-21T05:59:37.449577+0000","last_active":"2024-09-21T05:59:40.411909+0000","last_peered":"2024-09-21T05:59:40.411909+0000","last_clean":"2024-09-21T05:59:40.411909+0000","last_became_active":"2024-09-21T05:59:37.449255+0000","last_became_peered":"2024-09-21T05:59:37.449255+0000","last_unstale":"2024-09-21T05:59:40.411909+0000","last_undegraded":"2024-09-21T05:59:40.411909+0000","last_fullsized":"2024-09-21T05:59:40.411909+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:29.743402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411505+0000","last_change":"2024-09-21T05:59:38.465982+0000","last_active":"2024-09-21T05:59:40.411505+0000","last_peered":"2024-09-21T05:59:40.411505+0000","last_clean":"2024-09-21T05:59:40.411505+0000","last_became_active":"2024-09-21T05:59:38.465685+0000","last_became_peered":"2024-09-21T05:59:38.465685+0000","last_unstale":"2024-09-21T05:59:40.411505+0000","last_undegraded":"2024-09-21T05:59:40.411505+0000","last_fullsized":"2024-09-21T05:59:40.411505+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:52.379439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411729+0000","last_change":"2024-09-21T05:59:37.440683+0000","last_active":"2024-09-21T05:59:40.411729+0000","last_peered":"2024-09-21T05:59:40.411729+0000","last_clean":"2024-09-21T05:59:40.411729+0000","last_became_active":"2024-09-21T05:59:37.440336+0000","last_became_peered":"2024-09-21T05:59:37.440336+0000","last_unstale":"2024-09-21T05:59:40.411729+0000","last_undegraded":"2024-09-21T05:59:40.411729+0000","last_fullsized":"2024-09-21T05:59:40.411729+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:44.526096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.967131+0000","last_change":"2024-09-21T05:59:38.451176+0000","last_active":"2024-09-21T05:59:39.967131+0000","last_peered":"2024-09-21T05:59:39.967131+0000","last_clean":"2024-09-21T05:59:39.967131+0000","last_became_active":"2024-09-21T05:59:38.450879+0000","last_became_peered":"2024-09-21T05:59:38.450879+0000","last_unstale":"2024-09-21T05:59:39.967131+0000","last_undegraded":"2024-09-21T05:59:39.967131+0000","last_fullsized":"2024-09-21T05:59:39.967131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:14:18.877455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.413042+0000","last_change":"2024-09-21T05:59:37.448714+0000","last_active":"2024-09-21T05:59:40.413042+0000","last_peered":"2024-09-21T05:59:40.413042+0000","last_clean":"2024-09-21T05:59:40.413042+0000","last_became_active":"2024-09-21T05:59:37.448310+0000","last_became_peered":"2024-09-21T05:59:37.448310+0000","last_unstale":"2024-09-21T05:59:40.413042+0000","last_undegraded":"2024-09-21T05:59:40.413042+0000","last_fullsized":"2024-09-21T05:59:40.413042+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:44.078542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.413024+0000","last_change":"2024-09-21T05:59:38.464293+0000","last_active":"2024-09-21T05:59:40.413024+0000","last_peered":"2024-09-21T05:59:40.413024+0000","last_clean":"2024-09-21T05:59:40.413024+0000","last_became_active":"2024-09-21T05:59:38.464007+0000","last_became_peered":"2024-09-21T05:59:38.464007+0000","last_unstale":"2024-09-21T05:59:40.413024+0000","last_undegraded":"2024-09-21T05:59:40.413024+0000","last_fullsized":"2024-09-21T05:59:40.413024+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:44.518079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.967394+0000","last_change":"2024-09-21T05:59:37.445812+0000","last_active":"2024-09-21T05:59:39.967394+0000","last_peered":"2024-09-21T05:59:39.967394+0000","last_clean":"2024-09-21T05:59:39.967394+0000","last_became_active":"2024-09-21T05:59:37.445496+0000","last_became_peered":"2024-09-21T05:59:37.445496+0000","last_unstale":"2024-09-21T05:59:39.967394+0000","last_undegraded":"2024-09-21T05:59:39.967394+0000","last_fullsized":"2024-09-21T05:59:39.967394+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:43.258849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.967126+0000","last_change":"2024-09-21T05:59:38.452287+0000","last_active":"2024-09-21T05:59:39.967126+0000","last_peered":"2024-09-21T05:59:39.967126+0000","last_clean":"2024-09-21T05:59:39.967126+0000","last_became_active":"2024-09-21T05:59:38.451960+0000","last_became_peered":"2024-09-21T05:59:38.451960+0000","last_unstale":"2024-09-21T05:59:39.967126+0000","last_undegraded":"2024-09-21T05:59:39.967126+0000","last_fullsized":"2024-09-21T05:59:39.967126+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:21.178717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":21,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.439767+0000","last_change":"2024-09-21T05:59:37.440777+0000","last_active":"2024-09-21T05:59:39.439767+0000","last_peered":"2024-09-21T05:59:39.439767+0000","last_clean":"2024-09-21T05:59:39.439767+0000","last_became_active":"2024-09-21T05:59:37.440460+0000","last_became_peered":"2024-09-21T05:59:37.440460+0000","last_unstale":"2024-09-21T05:59:39.439767+0000","last_undegraded":"2024-09-21T05:59:39.439767+0000","last_fullsized":"2024-09-21T05:59:39.439767+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.580920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.443202+0000","last_change":"2024-09-21T05:59:38.462090+0000","last_active":"2024-09-21T05:59:39.443202+0000","last_peered":"2024-09-21T05:59:39.443202+0000","last_clean":"2024-09-21T05:59:39.443202+0000","last_became_active":"2024-09-21T05:59:38.461726+0000","last_became_peered":"2024-09-21T05:59:38.461726+0000","last_unstale":"2024-09-21T05:59:39.443202+0000","last_undegraded":"2024-09-21T05:59:39.443202+0000","last_fullsized":"2024-09-21T05:59:39.443202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:36.240675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412674+0000","last_change":"2024-09-21T05:59:37.447984+0000","last_active":"2024-09-21T05:59:40.412674+0000","last_peered":"2024-09-21T05:59:40.412674+0000","last_clean":"2024-09-21T05:59:40.412674+0000","last_became_active":"2024-09-21T05:59:37.447685+0000","last_became_peered":"2024-09-21T05:59:37.447685+0000","last_unstale":"2024-09-21T05:59:40.412674+0000","last_undegraded":"2024-09-21T05:59:40.412674+0000","last_fullsized":"2024-09-21T05:59:40.412674+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:38.221309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412812+0000","last_change":"2024-09-21T05:59:38.451149+0000","last_active":"2024-09-21T05:59:40.412812+0000","last_peered":"2024-09-21T05:59:40.412812+0000","last_clean":"2024-09-21T05:59:40.412812+0000","last_became_active":"2024-09-21T05:59:38.450841+0000","last_became_peered":"2024-09-21T05:59:38.450841+0000","last_unstale":"2024-09-21T05:59:40.412812+0000","last_undegraded":"2024-09-21T05:59:40.412812+0000","last_fullsized":"2024-09-21T05:59:40.412812+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:53.757813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.966836+0000","last_change":"2024-09-21T05:59:37.443723+0000","last_active":"2024-09-21T05:59:39.966836+0000","last_peered":"2024-09-21T05:59:39.966836+0000","last_clean":"2024-09-21T05:59:39.966836+0000","last_became_active":"2024-09-21T05:59:37.443383+0000","last_became_peered":"2024-09-21T05:59:37.443383+0000","last_unstale":"2024-09-21T05:59:39.966836+0000","last_undegraded":"2024-09-21T05:59:39.966836+0000","last_fullsized":"2024-09-21T05:59:39.966836+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:14.877836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.413221+0000","last_change":"2024-09-21T05:59:38.446272+0000","last_active":"2024-09-21T05:59:40.413221+0000","last_peered":"2024-09-21T05:59:40.413221+0000","last_clean":"2024-09-21T05:59:40.413221+0000","last_became_active":"2024-09-21T05:59:38.444384+0000","last_became_peered":"2024-09-21T05:59:38.444384+0000","last_unstale":"2024-09-21T05:59:40.413221+0000","last_undegraded":"2024-09-21T05:59:40.413221+0000","last_fullsized":"2024-09-21T05:59:40.413221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:23.580845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412677+0000","last_change":"2024-09-21T05:59:37.444802+0000","last_active":"2024-09-21T05:59:40.412677+0000","last_peered":"2024-09-21T05:59:40.412677+0000","last_clean":"2024-09-21T05:59:40.412677+0000","last_became_active":"2024-09-21T05:59:37.444457+0000","last_became_peered":"2024-09-21T05:59:37.444457+0000","last_unstale":"2024-09-21T05:59:40.412677+0000","last_undegraded":"2024-09-21T05:59:40.412677+0000","last_fullsized":"2024-09-21T05:59:40.412677+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:29.339627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.967549+0000","last_change":"2024-09-21T05:59:38.454597+0000","last_active":"2024-09-21T05:59:39.967549+0000","last_peered":"2024-09-21T05:59:39.967549+0000","last_clean":"2024-09-21T05:59:39.967549+0000","last_became_active":"2024-09-21T05:59:38.454132+0000","last_became_peered":"2024-09-21T05:59:38.454132+0000","last_unstale":"2024-09-21T05:59:39.967549+0000","last_undegraded":"2024-09-21T05:59:39.967549+0000","last_fullsized":"2024-09-21T05:59:39.967549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:59.904753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":23,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411946+0000","last_change":"2024-09-21T05:59:37.444089+0000","last_active":"2024-09-21T05:59:40.411946+0000","last_peered":"2024-09-21T05:59:40.411946+0000","last_clean":"2024-09-21T05:59:40.411946+0000","last_became_active":"2024-09-21T05:59:37.443438+0000","last_became_peered":"2024-09-21T05:59:37.443438+0000","last_unstale":"2024-09-21T05:59:40.411946+0000","last_undegraded":"2024-09-21T05:59:40.411946+0000","last_fullsized":"2024-09-21T05:59:40.411946+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:24.655972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.440132+0000","last_change":"2024-09-21T05:59:38.439982+0000","last_active":"2024-09-21T05:59:39.440132+0000","last_peered":"2024-09-21T05:59:39.440132+0000","last_clean":"2024-09-21T05:59:39.440132+0000","last_became_active":"2024-09-21T05:59:38.439695+0000","last_became_peered":"2024-09-21T05:59:38.439695+0000","last_unstale":"2024-09-21T05:59:39.440132+0000","last_undegraded":"2024-09-21T05:59:39.440132+0000","last_fullsized":"2024-09-21T05:59:39.440132+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:25.393981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.440072+0000","last_change":"2024-09-21T05:59:37.445284+0000","last_active":"2024-09-21T05:59:39.440072+0000","last_peered":"2024-09-21T05:59:39.440072+0000","last_clean":"2024-09-21T05:59:39.440072+0000","last_became_active":"2024-09-21T05:59:37.444912+0000","last_became_peered":"2024-09-21T05:59:37.444912+0000","last_unstale":"2024-09-21T05:59:39.440072+0000","last_undegraded":"2024-09-21T05:59:39.440072+0000","last_fullsized":"2024-09-21T05:59:39.440072+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:56.825563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.413494+0000","last_change":"2024-09-21T05:59:38.469299+0000","last_active":"2024-09-21T05:59:40.413494+0000","last_peered":"2024-09-21T05:59:40.413494+0000","last_clean":"2024-09-21T05:59:40.413494+0000","last_became_active":"2024-09-21T05:59:38.468996+0000","last_became_peered":"2024-09-21T05:59:38.468996+0000","last_unstale":"2024-09-21T05:59:40.413494+0000","last_undegraded":"2024-09-21T05:59:40.413494+0000","last_fullsized":"2024-09-21T05:59:40.413494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:16.687594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411721+0000","last_change":"2024-09-21T05:59:37.435640+0000","last_active":"2024-09-21T05:59:40.411721+0000","last_peered":"2024-09-21T05:59:40.411721+0000","last_clean":"2024-09-21T05:59:40.411721+0000","last_became_active":"2024-09-21T05:59:37.435202+0000","last_became_peered":"2024-09-21T05:59:37.435202+0000","last_unstale":"2024-09-21T05:59:40.411721+0000","last_undegraded":"2024-09-21T05:59:40.411721+0000","last_fullsized":"2024-09-21T05:59:40.411721+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:56.659794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.443528+0000","last_change":"2024-09-21T05:59:38.463095+0000","last_active":"2024-09-21T05:59:39.443528+0000","last_peered":"2024-09-21T05:59:39.443528+0000","last_clean":"2024-09-21T05:59:39.443528+0000","last_became_active":"2024-09-21T05:59:38.462817+0000","last_became_peered":"2024-09-21T05:59:38.462817+0000","last_unstale":"2024-09-21T05:59:39.443528+0000","last_undegraded":"2024-09-21T05:59:39.443528+0000","last_fullsized":"2024-09-21T05:59:39.443528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:16.692568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.442878+0000","last_change":"2024-09-21T05:59:37.432827+0000","last_active":"2024-09-21T05:59:39.442878+0000","last_peered":"2024-09-21T05:59:39.442878+0000","last_clean":"2024-09-21T05:59:39.442878+0000","last_became_active":"2024-09-21T05:59:37.432500+0000","last_became_peered":"2024-09-21T05:59:37.432500+0000","last_unstale":"2024-09-21T05:59:39.442878+0000","last_undegraded":"2024-09-21T05:59:39.442878+0000","last_fullsized":"2024-09-21T05:59:39.442878+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:14.877836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.443770+0000","last_change":"2024-09-21T05:59:38.450728+0000","last_active":"2024-09-21T05:59:39.443770+0000","last_peered":"2024-09-21T05:59:39.443770+0000","last_clean":"2024-09-21T05:59:39.443770+0000","last_became_active":"2024-09-21T05:59:38.450369+0000","last_became_peered":"2024-09-21T05:59:38.450369+0000","last_unstale":"2024-09-21T05:59:39.443770+0000","last_undegraded":"2024-09-21T05:59:39.443770+0000","last_fullsized":"2024-09-21T05:59:39.443770+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:33.971144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.413037+0000","last_change":"2024-09-21T05:59:37.441674+0000","last_active":"2024-09-21T05:59:40.413037+0000","last_peered":"2024-09-21T05:59:40.413037+0000","last_clean":"2024-09-21T05:59:40.413037+0000","last_became_active":"2024-09-21T05:59:37.441330+0000","last_became_peered":"2024-09-21T05:59:37.441330+0000","last_unstale":"2024-09-21T05:59:40.413037+0000","last_undegraded":"2024-09-21T05:59:40.413037+0000","last_fullsized":"2024-09-21T05:59:40.413037+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:14:17.874347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412180+0000","last_change":"2024-09-21T05:59:38.453080+0000","last_active":"2024-09-21T05:59:40.412180+0000","last_peered":"2024-09-21T05:59:40.412180+0000","last_clean":"2024-09-21T05:59:40.412180+0000","last_became_active":"2024-09-21T05:59:38.448593+0000","last_became_peered":"2024-09-21T05:59:38.448593+0000","last_unstale":"2024-09-21T05:59:40.412180+0000","last_undegraded":"2024-09-21T05:59:40.412180+0000","last_fullsized":"2024-09-21T05:59:40.412180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:18.599030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412793+0000","last_change":"2024-09-21T05:59:37.447596+0000","last_active":"2024-09-21T05:59:40.412793+0000","last_peered":"2024-09-21T05:59:40.412793+0000","last_clean":"2024-09-21T05:59:40.412793+0000","last_became_active":"2024-09-21T05:59:37.440489+0000","last_became_peered":"2024-09-21T05:59:37.440489+0000","last_unstale":"2024-09-21T05:59:40.412793+0000","last_undegraded":"2024-09-21T05:59:40.412793+0000","last_fullsized":"2024-09-21T05:59:40.412793+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:15.485314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837878+0000","last_change":"2024-09-21T05:59:38.435762+0000","last_active":"2024-09-21T05:59:39.837878+0000","last_peered":"2024-09-21T05:59:39.837878+0000","last_clean":"2024-09-21T05:59:39.837878+0000","last_became_active":"2024-09-21T05:59:38.434259+0000","last_became_peered":"2024-09-21T05:59:38.434259+0000","last_unstale":"2024-09-21T05:59:39.837878+0000","last_undegraded":"2024-09-21T05:59:39.837878+0000","last_fullsized":"2024-09-21T05:59:39.837878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:36.174704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.442267+0000","last_change":"2024-09-21T05:59:37.434300+0000","last_active":"2024-09-21T05:59:39.442267+0000","last_peered":"2024-09-21T05:59:39.442267+0000","last_clean":"2024-09-21T05:59:39.442267+0000","last_became_active":"2024-09-21T05:59:37.433933+0000","last_became_peered":"2024-09-21T05:59:37.433933+0000","last_unstale":"2024-09-21T05:59:39.442267+0000","last_undegraded":"2024-09-21T05:59:39.442267+0000","last_fullsized":"2024-09-21T05:59:39.442267+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:22.008999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.439631+0000","last_change":"2024-09-21T05:59:38.457706+0000","last_active":"2024-09-21T05:59:39.439631+0000","last_peered":"2024-09-21T05:59:39.439631+0000","last_clean":"2024-09-21T05:59:39.439631+0000","last_became_active":"2024-09-21T05:59:38.457338+0000","last_became_peered":"2024-09-21T05:59:38.457338+0000","last_unstale":"2024-09-21T05:59:39.439631+0000","last_undegraded":"2024-09-21T05:59:39.439631+0000","last_fullsized":"2024-09-21T05:59:39.439631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:35.736954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411797+0000","last_change":"2024-09-21T05:59:37.446607+0000","last_active":"2024-09-21T05:59:40.411797+0000","last_peered":"2024-09-21T05:59:40.411797+0000","last_clean":"2024-09-21T05:59:40.411797+0000","last_became_active":"2024-09-21T05:59:37.446318+0000","last_became_peered":"2024-09-21T05:59:37.446318+0000","last_unstale":"2024-09-21T05:59:40.411797+0000","last_undegraded":"2024-09-21T05:59:40.411797+0000","last_fullsized":"2024-09-21T05:59:40.411797+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:33.988632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411714+0000","last_change":"2024-09-21T05:59:38.459115+0000","last_active":"2024-09-21T05:59:40.411714+0000","last_peered":"2024-09-21T05:59:40.411714+0000","last_clean":"2024-09-21T05:59:40.411714+0000","last_became_active":"2024-09-21T05:59:38.451533+0000","last_became_peered":"2024-09-21T05:59:38.451533+0000","last_unstale":"2024-09-21T05:59:40.411714+0000","last_undegraded":"2024-09-21T05:59:40.411714+0000","last_fullsized":"2024-09-21T05:59:40.411714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:10.631466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.966796+0000","last_change":"2024-09-21T05:59:37.445107+0000","last_active":"2024-09-21T05:59:39.966796+0000","last_peered":"2024-09-21T05:59:39.966796+0000","last_clean":"2024-09-21T05:59:39.966796+0000","last_became_active":"2024-09-21T05:59:37.444746+0000","last_became_peered":"2024-09-21T05:59:37.444746+0000","last_unstale":"2024-09-21T05:59:39.966796+0000","last_undegraded":"2024-09-21T05:59:39.966796+0000","last_fullsized":"2024-09-21T05:59:39.966796+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:43.937811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.444163+0000","last_change":"2024-09-21T05:59:38.462539+0000","last_active":"2024-09-21T05:59:39.444163+0000","last_peered":"2024-09-21T05:59:39.444163+0000","last_clean":"2024-09-21T05:59:39.444163+0000","last_became_active":"2024-09-21T05:59:38.462185+0000","last_became_peered":"2024-09-21T05:59:38.462185+0000","last_unstale":"2024-09-21T05:59:39.444163+0000","last_undegraded":"2024-09-21T05:59:39.444163+0000","last_fullsized":"2024-09-21T05:59:39.444163+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:18.300947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412962+0000","last_change":"2024-09-21T05:59:37.445647+0000","last_active":"2024-09-21T05:59:40.412962+0000","last_peered":"2024-09-21T05:59:40.412962+0000","last_clean":"2024-09-21T05:59:40.412962+0000","last_became_active":"2024-09-21T05:59:37.445165+0000","last_became_peered":"2024-09-21T05:59:37.445165+0000","last_unstale":"2024-09-21T05:59:40.412962+0000","last_undegraded":"2024-09-21T05:59:40.412962+0000","last_fullsized":"2024-09-21T05:59:40.412962+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:22.836709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412896+0000","last_change":"2024-09-21T05:59:38.456247+0000","last_active":"2024-09-21T05:59:40.412896+0000","last_peered":"2024-09-21T05:59:40.412896+0000","last_clean":"2024-09-21T05:59:40.412896+0000","last_became_active":"2024-09-21T05:59:38.455936+0000","last_became_peered":"2024-09-21T05:59:38.455936+0000","last_unstale":"2024-09-21T05:59:40.412896+0000","last_undegraded":"2024-09-21T05:59:40.412896+0000","last_fullsized":"2024-09-21T05:59:40.412896+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:12.326495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.967395+0000","last_change":"2024-09-21T05:59:37.446726+0000","last_active":"2024-09-21T05:59:39.967395+0000","last_peered":"2024-09-21T05:59:39.967395+0000","last_clean":"2024-09-21T05:59:39.967395+0000","last_became_active":"2024-09-21T05:59:37.446299+0000","last_became_peered":"2024-09-21T05:59:37.446299+0000","last_unstale":"2024-09-21T05:59:39.967395+0000","last_undegraded":"2024-09-21T05:59:39.967395+0000","last_fullsized":"2024-09-21T05:59:39.967395+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:48.377140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.967199+0000","last_change":"2024-09-21T05:59:38.447788+0000","last_active":"2024-09-21T05:59:39.967199+0000","last_peered":"2024-09-21T05:59:39.967199+0000","last_clean":"2024-09-21T05:59:39.967199+0000","last_became_active":"2024-09-21T05:59:38.447390+0000","last_became_peered":"2024-09-21T05:59:38.447390+0000","last_unstale":"2024-09-21T05:59:39.967199+0000","last_undegraded":"2024-09-21T05:59:39.967199+0000","last_fullsized":"2024-09-21T05:59:39.967199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:23.839817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837548+0000","last_change":"2024-09-21T05:59:37.446929+0000","last_active":"2024-09-21T05:59:39.837548+0000","last_peered":"2024-09-21T05:59:39.837548+0000","last_clean":"2024-09-21T05:59:39.837548+0000","last_became_active":"2024-09-21T05:59:37.446552+0000","last_became_peered":"2024-09-21T05:59:37.446552+0000","last_unstale":"2024-09-21T05:59:39.837548+0000","last_undegraded":"2024-09-21T05:59:39.837548+0000","last_fullsized":"2024-09-21T05:59:39.837548+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:54.236652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412472+0000","last_change":"2024-09-21T05:59:38.454591+0000","last_active":"2024-09-21T05:59:40.412472+0000","last_peered":"2024-09-21T05:59:40.412472+0000","last_clean":"2024-09-21T05:59:40.412472+0000","last_became_active":"2024-09-21T05:59:38.449004+0000","last_became_peered":"2024-09-21T05:59:38.449004+0000","last_unstale":"2024-09-21T05:59:40.412472+0000","last_undegraded":"2024-09-21T05:59:40.412472+0000","last_fullsized":"2024-09-21T05:59:40.412472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:30.346925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.967049+0000","last_change":"2024-09-21T05:59:37.446310+0000","last_active":"2024-09-21T05:59:39.967049+0000","last_peered":"2024-09-21T05:59:39.967049+0000","last_clean":"2024-09-21T05:59:39.967049+0000","last_became_active":"2024-09-21T05:59:37.445953+0000","last_became_peered":"2024-09-21T05:59:37.445953+0000","last_unstale":"2024-09-21T05:59:39.967049+0000","last_undegraded":"2024-09-21T05:59:39.967049+0000","last_fullsized":"2024-09-21T05:59:39.967049+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:19.965677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412347+0000","last_change":"2024-09-21T05:59:38.448746+0000","last_active":"2024-09-21T05:59:40.412347+0000","last_peered":"2024-09-21T05:59:40.412347+0000","last_clean":"2024-09-21T05:59:40.412347+0000","last_became_active":"2024-09-21T05:59:38.448036+0000","last_became_peered":"2024-09-21T05:59:38.448036+0000","last_unstale":"2024-09-21T05:59:40.412347+0000","last_undegraded":"2024-09-21T05:59:40.412347+0000","last_fullsized":"2024-09-21T05:59:40.412347+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:38.355228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411473+0000","last_change":"2024-09-21T05:59:37.435447+0000","last_active":"2024-09-21T05:59:40.411473+0000","last_peered":"2024-09-21T05:59:40.411473+0000","last_clean":"2024-09-21T05:59:40.411473+0000","last_became_active":"2024-09-21T05:59:37.435178+0000","last_became_peered":"2024-09-21T05:59:37.435178+0000","last_unstale":"2024-09-21T05:59:40.411473+0000","last_undegraded":"2024-09-21T05:59:40.411473+0000","last_fullsized":"2024-09-21T05:59:40.411473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:31.159117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.442502+0000","last_change":"2024-09-21T05:59:38.430000+0000","last_active":"2024-09-21T05:59:39.442502+0000","last_peered":"2024-09-21T05:59:39.442502+0000","last_clean":"2024-09-21T05:59:39.442502+0000","last_became_active":"2024-09-21T05:59:38.429550+0000","last_became_peered":"2024-09-21T05:59:38.429550+0000","last_unstale":"2024-09-21T05:59:39.442502+0000","last_undegraded":"2024-09-21T05:59:39.442502+0000","last_fullsized":"2024-09-21T05:59:39.442502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:21.308796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.837563+0000","last_change":"2024-09-21T05:59:37.441729+0000","last_active":"2024-09-21T05:59:39.837563+0000","last_peered":"2024-09-21T05:59:39.837563+0000","last_clean":"2024-09-21T05:59:39.837563+0000","last_became_active":"2024-09-21T05:59:37.441370+0000","last_became_peered":"2024-09-21T05:59:37.441370+0000","last_unstale":"2024-09-21T05:59:39.837563+0000","last_undegraded":"2024-09-21T05:59:39.837563+0000","last_fullsized":"2024-09-21T05:59:39.837563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:28.903549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.411138+0000","last_change":"2024-09-21T05:59:38.444211+0000","last_active":"2024-09-21T05:59:40.411138+0000","last_peered":"2024-09-21T05:59:40.411138+0000","last_clean":"2024-09-21T05:59:40.411138+0000","last_became_active":"2024-09-21T05:59:38.443897+0000","last_became_peered":"2024-09-21T05:59:38.443897+0000","last_unstale":"2024-09-21T05:59:40.411138+0000","last_undegraded":"2024-09-21T05:59:40.411138+0000","last_fullsized":"2024-09-21T05:59:40.411138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:18.300947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.412051+0000","last_change":"2024-09-21T05:59:37.442907+0000","last_active":"2024-09-21T05:59:40.412051+0000","last_peered":"2024-09-21T05:59:40.412051+0000","last_clean":"2024-09-21T05:59:40.412051+0000","last_became_active":"2024-09-21T05:59:37.439722+0000","last_became_peered":"2024-09-21T05:59:37.439722+0000","last_unstale":"2024-09-21T05:59:40.412051+0000","last_undegraded":"2024-09-21T05:59:40.412051+0000","last_fullsized":"2024-09-21T05:59:40.412051+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:27.667815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:40.413063+0000","last_change":"2024-09-21T05:59:38.455813+0000","last_active":"2024-09-21T05:59:40.413063+0000","last_peered":"2024-09-21T05:59:40.413063+0000","last_clean":"2024-09-21T05:59:40.413063+0000","last_became_active":"2024-09-21T05:59:38.455487+0000","last_became_peered":"2024-09-21T05:59:38.455487+0000","last_unstale":"2024-09-21T05:59:40.413063+0000","last_undegraded":"2024-09-21T05:59:40.413063+0000","last_fullsized":"2024-09-21T05:59:40.413063+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:37.411723+0000","last_clean_scrub_stamp":"2024-09-21T05:59:37.411723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:21.178717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.443046+0000","last_change":"2024-09-21T05:59:37.435112+0000","last_active":"2024-09-21T05:59:39.443046+0000","last_peered":"2024-09-21T05:59:39.443046+0000","last_clean":"2024-09-21T05:59:39.443046+0000","last_became_active":"2024-09-21T05:59:37.434789+0000","last_became_peered":"2024-09-21T05:59:37.434789+0000","last_unstale":"2024-09-21T05:59:39.443046+0000","last_undegraded":"2024-09-21T05:59:39.443046+0000","last_fullsized":"2024-09-21T05:59:39.443046+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:59:36.408615+0000","last_clean_scrub_stamp":"2024-09-21T05:59:36.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:19.965677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":152,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T05:59:39.966929+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T05:59:39.966929+0000","last_peered":"2024-09-21T05:59:39.966929+0000","last_clean":"2024-09-21T05:59:39.966929+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T05:59:39.966929+0000","last_undegraded":"2024-09-21T05:59:39.966929+0000","last_fullsized":"2024-09-21T05:59:39.966929+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509484,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1105920,"data_stored":793126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31696,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":516096,"data_stored":203340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31692,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":512000,"data_stored":202738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509484,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":520192,"data_stored":204280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35856,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335984,"statfs":{"total":96636764160,"available":96600047616,"internally_reserved":0,"allocated":520192,"data_stored":203306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35864,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335976,"statfs":{"total":96636764160,"available":96600039424,"internally_reserved":0,"allocated":524288,"data_stored":204302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509483,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36428,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335412,"statfs":{"total":96636764160,"available":96599461888,"internally_reserved":0,"allocated":1105920,"data_stored":793070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40008,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331832,"statfs":{"total":96636764160,"available":96595795968,"internally_reserved":0,"allocated":512000,"data_stored":202738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:59:47.725 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:59:47.725 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T05:59:47.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.927+0000 7f4adea00640 1 -- 172.21.15.50:0/1980125927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad80b4de0 msgr2=0x7f4ad80b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:47.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.927+0000 7f4adea00640 1 --2- 172.21.15.50:0/1980125927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad80b4de0 0x7f4ad80b51f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80703e0 tx=0x7f4ad001ccc0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.927+0000 7f4adea00640 1 -- 172.21.15.50:0/1980125927 shutdown_connections 2024-09-21T05:59:47.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.927+0000 7f4adea00640 1 --2- 172.21.15.50:0/1980125927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad80b4de0 0x7f4ad80b51f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.927+0000 7f4adea00640 1 --2- 172.21.15.50:0/1980125927 >> v2:172.21.15.88:3300/0 conn(0x7f4ad81527b0 0x7f4ad8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.927+0000 7f4adea00640 1 --2- 172.21.15.50:0/1980125927 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ad80b5c70 0x7f4ad815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.927+0000 7f4adea00640 1 -- 172.21.15.50:0/1980125927 >> 172.21.15.50:0/1980125927 conn(0x7f4ad80b2040 msgr2=0x7f4ad80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:47.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.927+0000 7f4adea00640 1 -- 172.21.15.50:0/1980125927 shutdown_connections 2024-09-21T05:59:47.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.927+0000 7f4adea00640 1 -- 172.21.15.50:0/1980125927 wait complete. 2024-09-21T05:59:47.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.927+0000 7f4adea00640 1 Processor -- start 2024-09-21T05:59:47.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adea00640 1 -- start start 2024-09-21T05:59:47.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ad80b4de0 0x7f4ad80cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:47.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adea00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ad80b5c70 0x7f4ad80ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:47.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad81527b0 0x7f4ad80ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:47.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adea00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad815f6b0 con 0x7f4ad81527b0 2024-09-21T05:59:47.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adea00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad815f530 con 0x7f4ad80b4de0 2024-09-21T05:59:47.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adea00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad815f830 con 0x7f4ad80b5c70 2024-09-21T05:59:47.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad81527b0 0x7f4ad80ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:47.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4ad7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ad80b4de0 0x7f4ad80cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:47.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad81527b0 0x7f4ad80ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59950/0 (socket says 172.21.15.50:59950) 2024-09-21T05:59:47.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4ad7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ad80b4de0 0x7f4ad80cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:52518/0 (socket says 172.21.15.50:52518) 2024-09-21T05:59:47.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adca00640 1 -- 172.21.15.50:0/1028857989 learned_addr learned my addr 172.21.15.50:0/1028857989 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:47.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4acfe00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ad80b5c70 0x7f4ad80ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:47.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adca00640 1 -- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ad80b5c70 msgr2=0x7f4ad80ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:47.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adca00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ad80b5c70 0x7f4ad80ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adca00640 1 -- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ad80b4de0 msgr2=0x7f4ad80cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:47.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adca00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ad80b4de0 0x7f4ad80cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:47.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adca00640 1 -- 172.21.15.50:0/1028857989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad8141720 con 0x7f4ad81527b0 2024-09-21T05:59:47.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adca00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad81527b0 0x7f4ad80ce9c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4ad001b370 tx=0x7f4ad0005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:47.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4ad4c00640 1 -- 172.21.15.50:0/1028857989 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00bc070 con 0x7f4ad81527b0 2024-09-21T05:59:47.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4ad4c00640 1 -- 172.21.15.50:0/1028857989 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ad0004330 con 0x7f4ad81527b0 2024-09-21T05:59:47.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adea00640 1 -- 172.21.15.50:0/1028857989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad8157ba0 con 0x7f4ad81527b0 2024-09-21T05:59:47.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4ad4c00640 1 -- 172.21.15.50:0/1028857989 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00b2040 con 0x7f4ad81527b0 2024-09-21T05:59:47.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.931+0000 7f4adea00640 1 -- 172.21.15.50:0/1028857989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad8157e40 con 0x7f4ad81527b0 2024-09-21T05:59:47.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.935+0000 7f4adea00640 1 -- 172.21.15.50:0/1028857989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad813e470 con 0x7f4ad81527b0 2024-09-21T05:59:47.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.935+0000 7f4ad4c00640 1 -- 172.21.15.50:0/1028857989 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f4ad00bf070 con 0x7f4ad81527b0 2024-09-21T05:59:47.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.935+0000 7f4ad4c00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ab40734d0 0x7f4ab40759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:47.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.935+0000 7f4ad4c00640 1 -- 172.21.15.50:0/1028857989 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f4ad012e740 con 0x7f4ad81527b0 2024-09-21T05:59:47.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.935+0000 7f4ad7e00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ab40734d0 0x7f4ab40759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:47.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.935+0000 7f4ad7e00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ab40734d0 0x7f4ab40759e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8004630 tx=0x7f4ac800b260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:47.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:47.939+0000 7f4ad4c00640 1 -- 172.21.15.50:0/1028857989 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4ad813e470 con 0x7f4ad81527b0 2024-09-21T05:59:48.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.187+0000 7f4adea00640 1 -- 172.21.15.50:0/1028857989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4ad815fd50 con 0x7f4ad81527b0 2024-09-21T05:59:48.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4ad4c00640 1 -- 172.21.15.50:0/1028857989 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v30) ==== 190+0+147 (secure 0 0 0) 0x7f4ad0016970 con 0x7f4ad81527b0 2024-09-21T05:59:48.194 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-09-21T05:59:48.195 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw== 2024-09-21T05:59:48.195 INFO:teuthology.orchestra.run.smithi050.stdout: caps mds = "allow" 2024-09-21T05:59:48.195 INFO:teuthology.orchestra.run.smithi050.stdout: caps mon = "allow r" 2024-09-21T05:59:48.195 INFO:teuthology.orchestra.run.smithi050.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T05:59:48.195 INFO:teuthology.orchestra.run.smithi050.stderr:updated caps for client.0 2024-09-21T05:59:48.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4adea00640 1 -- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ab40734d0 msgr2=0x7f4ab40759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:48.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4adea00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ab40734d0 0x7f4ab40759e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8004630 tx=0x7f4ac800b260 comp rx=0 tx=0).stop 2024-09-21T05:59:48.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4adea00640 1 -- 172.21.15.50:0/1028857989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad81527b0 msgr2=0x7f4ad80ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:48.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4adea00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad81527b0 0x7f4ad80ce9c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4ad001b370 tx=0x7f4ad0005b10 comp rx=0 tx=0).stop 2024-09-21T05:59:48.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4adea00640 1 -- 172.21.15.50:0/1028857989 shutdown_connections 2024-09-21T05:59:48.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4adea00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ab40734d0 0x7f4ab40759e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4adea00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad81527b0 0x7f4ad80ce9c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4adea00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ad80b5c70 0x7f4ad80ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4adea00640 1 --2- 172.21.15.50:0/1028857989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ad80b4de0 0x7f4ad80cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4adea00640 1 -- 172.21.15.50:0/1028857989 >> 172.21.15.50:0/1028857989 conn(0x7f4ad80b2040 msgr2=0x7f4ad80b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:48.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4adea00640 1 -- 172.21.15.50:0/1028857989 shutdown_connections 2024-09-21T05:59:48.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.191+0000 7f4adea00640 1 -- 172.21.15.50:0/1028857989 wait complete. 2024-09-21T05:59:48.217 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:59:48.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.375+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2725257679 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b88004da0 msgr2=0x7f4b88005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:48.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.375+0000 7f4ba3e00640 1 --2- 172.21.15.50:0/2725257679 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b88004da0 0x7f4b88005230 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c001f50 tx=0x7f4b9c01f3b0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.375+0000 7f4ba3400640 1 -- 172.21.15.50:0/2725257679 reap_dead start 2024-09-21T05:59:48.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.375+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2725257679 shutdown_connections 2024-09-21T05:59:48.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.375+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2725257679 >> 172.21.15.50:0/2725257679 conn(0x7f4ba40b41f0 msgr2=0x7f4ba40b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:48.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.375+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2725257679 shutdown_connections 2024-09-21T05:59:48.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.375+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2725257679 wait complete. 2024-09-21T05:59:48.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.375+0000 7f4ba3e00640 1 Processor -- start 2024-09-21T05:59:48.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.375+0000 7f4ba3e00640 1 -- start start 2024-09-21T05:59:48.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4ba3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b88004da0 0x7f4ba40cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:48.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4ba3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ba40c9f10 0x7f4ba40c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:48.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4ba3e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ba40c8ab0 0x7f4ba40c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4ba3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba41600e0 con 0x7f4ba40c9f10 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4ba3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba40d0db0 con 0x7f4b88004da0 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4ba3e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4ba40c6aa0 con 0x7f4ba40c8ab0 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ba40c9f10 0x7f4ba40c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ba40c9f10 0x7f4ba40c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59982/0 (socket says 172.21.15.50:59982) 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9be00640 1 -- 172.21.15.50:0/2874337014 learned_addr learned my addr 172.21.15.50:0/2874337014 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4ba2a00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b88004da0 0x7f4ba40cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4ba3400640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ba40c8ab0 0x7f4ba40c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9be00640 1 -- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ba40c8ab0 msgr2=0x7f4ba40c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9be00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ba40c8ab0 0x7f4ba40c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9be00640 1 -- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b88004da0 msgr2=0x7f4ba40cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9be00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b88004da0 0x7f4ba40cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9be00640 1 -- 172.21.15.50:0/2874337014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba40c8ec0 con 0x7f4ba40c9f10 2024-09-21T05:59:48.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4ba2a00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b88004da0 0x7f4ba40cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:48.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9be00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ba40c9f10 0x7f4ba40c8570 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c006630 tx=0x7f4b8c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:48.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9aa00640 1 -- 172.21.15.50:0/2874337014 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c015070 con 0x7f4ba40c9f10 2024-09-21T05:59:48.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9aa00640 1 -- 172.21.15.50:0/2874337014 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b8c011840 con 0x7f4ba40c9f10 2024-09-21T05:59:48.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9aa00640 1 -- 172.21.15.50:0/2874337014 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c011c90 con 0x7f4ba40c9f10 2024-09-21T05:59:48.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2874337014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba40cb210 con 0x7f4ba40c9f10 2024-09-21T05:59:48.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2874337014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba40c9430 con 0x7f4ba40c9f10 2024-09-21T05:59:48.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2874337014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba40cb3e0 con 0x7f4ba40c9f10 2024-09-21T05:59:48.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.379+0000 7f4b9aa00640 1 -- 172.21.15.50:0/2874337014 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f4b8c018070 con 0x7f4ba40c9f10 2024-09-21T05:59:48.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.383+0000 7f4b9aa00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4b80073590 0x7f4b80075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:48.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.383+0000 7f4b9aa00640 1 -- 172.21.15.50:0/2874337014 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f4b8c095790 con 0x7f4ba40c9f10 2024-09-21T05:59:48.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.383+0000 7f4ba2a00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4b80073590 0x7f4b80075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:48.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.383+0000 7f4ba2a00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4b80073590 0x7f4b80075aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b94009c60 tx=0x7f4b940091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:48.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.387+0000 7f4b9aa00640 1 -- 172.21.15.50:0/2874337014 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4ba40c8ec0 con 0x7f4ba40c9f10 2024-09-21T05:59:48.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.599+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2874337014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ba40b0a20 con 0x7f4ba40c9f10 2024-09-21T05:59:48.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.599+0000 7f4b9aa00640 1 -- 172.21.15.50:0/2874337014 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5217 (secure 0 0 0) 0x7f4b8c060170 con 0x7f4ba40c9f10 2024-09-21T05:59:48.604 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T05:59:48.605 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"btime":"2024-09-21T05:59:41:444356+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/723044219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":723044219},{"type":"v1","addr":"172.21.15.88:6836","nonce":723044219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1966998855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1966998855},{"type":"v1","addr":"172.21.15.88:6837","nonce":1966998855}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2536951841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2536951841},{"type":"v1","addr":"172.21.15.50:6834","nonce":2536951841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:59:37.853304+0000","modified":"2024-09-21T05:59:40.872885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5370":{"gid":5370,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/988935074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":988935074},{"type":"v1","addr":"172.21.15.50:6835","nonce":988935074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5370,"qdb_cluster":[5370]},"id":3}]} 2024-09-21T05:59:48.605 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 27 2024-09-21T05:59:48.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.603+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4b80073590 msgr2=0x7f4b80075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:48.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.603+0000 7f4ba3e00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4b80073590 0x7f4b80075aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b94009c60 tx=0x7f4b940091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.603+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2874337014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ba40c9f10 msgr2=0x7f4ba40c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:48.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.603+0000 7f4ba3e00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ba40c9f10 0x7f4ba40c8570 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c006630 tx=0x7f4b8c0039d0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.603+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2874337014 shutdown_connections 2024-09-21T05:59:48.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.603+0000 7f4ba3e00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ba40c8ab0 0x7f4ba40c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.603+0000 7f4ba3e00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4b80073590 0x7f4b80075aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.603+0000 7f4ba3e00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ba40c9f10 0x7f4ba40c8570 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.603+0000 7f4ba3e00640 1 --2- 172.21.15.50:0/2874337014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b88004da0 0x7f4ba40cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:48.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.603+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2874337014 >> 172.21.15.50:0/2874337014 conn(0x7f4ba40b41f0 msgr2=0x7f4ba40cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:48.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.603+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2874337014 shutdown_connections 2024-09-21T05:59:48.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:48.603+0000 7f4ba3e00640 1 -- 172.21.15.50:0/2874337014 wait complete. 2024-09-21T05:59:49.030 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:59:37.853304+0000', 'modified': '2024-09-21T05:59:40.872885+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5370}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5370': {'gid': 5370, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6835/988935074', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6832', 'nonce': 988935074}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 988935074}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5370, 'qdb_cluster': [5370]} 2024-09-21T05:59:49.030 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:59:49.031 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T05:59:49.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.191+0000 7f2874600640 1 -- 172.21.15.50:0/1190575251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f286c1527b0 msgr2=0x7f286c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.191+0000 7f2874600640 1 --2- 172.21.15.50:0/1190575251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f286c1527b0 0x7f286c15bb60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f286c0cca90 tx=0x7f285801c620 comp rx=0 tx=0).stop 2024-09-21T05:59:49.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.191+0000 7f2874600640 1 -- 172.21.15.50:0/1190575251 shutdown_connections 2024-09-21T05:59:49.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.191+0000 7f2874600640 1 --2- 172.21.15.50:0/1190575251 >> v2:172.21.15.88:3300/0 conn(0x7f286c0b4de0 0x7f286c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.191+0000 7f2874600640 1 --2- 172.21.15.50:0/1190575251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f286c1527b0 0x7f286c15bb60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.191+0000 7f2874600640 1 -- 172.21.15.50:0/1190575251 >> 172.21.15.50:0/1190575251 conn(0x7f286c0b2040 msgr2=0x7f286c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:49.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.191+0000 7f2874600640 1 -- 172.21.15.50:0/1190575251 shutdown_connections 2024-09-21T05:59:49.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.191+0000 7f2874600640 1 -- 172.21.15.50:0/1190575251 wait complete. 2024-09-21T05:59:49.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.191+0000 7f2874600640 1 Processor -- start 2024-09-21T05:59:49.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.191+0000 7f2874600640 1 -- start start 2024-09-21T05:59:49.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2874600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f286c0b4de0 0x7f286c0acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2874600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f286c0b5c70 0x7f286c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2874600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f286c1527b0 0x7f286c0b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2874600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f286c168910 con 0x7f286c0b5c70 2024-09-21T05:59:49.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2874600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f286c168790 con 0x7f286c0b4de0 2024-09-21T05:59:49.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2874600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f286c168a90 con 0x7f286c1527b0 2024-09-21T05:59:49.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2871200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f286c0b5c70 0x7f286c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2871200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f286c0b5c70 0x7f286c0affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59998/0 (socket says 172.21.15.50:59998) 2024-09-21T05:59:49.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2871200640 1 -- 172.21.15.50:0/802476496 learned_addr learned my addr 172.21.15.50:0/802476496 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:49.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2871200640 1 -- 172.21.15.50:0/802476496 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f286c1527b0 msgr2=0x7f286c0b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2871200640 1 --2- 172.21.15.50:0/802476496 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f286c1527b0 0x7f286c0b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2871200640 1 -- 172.21.15.50:0/802476496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f286c0b4de0 msgr2=0x7f286c0acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2871200640 1 --2- 172.21.15.50:0/802476496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f286c0b4de0 0x7f286c0acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2871200640 1 -- 172.21.15.50:0/802476496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c1414b0 con 0x7f286c0b5c70 2024-09-21T05:59:49.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2871200640 1 --2- 172.21.15.50:0/802476496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f286c0b5c70 0x7f286c0affb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f28580b0040 tx=0x7f28580a67d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:49.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2869600640 1 -- 172.21.15.50:0/802476496 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858004300 con 0x7f286c0b5c70 2024-09-21T05:59:49.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2869600640 1 -- 172.21.15.50:0/802476496 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2858005020 con 0x7f286c0b5c70 2024-09-21T05:59:49.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2874600640 1 -- 172.21.15.50:0/802476496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c141790 con 0x7f286c0b5c70 2024-09-21T05:59:49.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2869600640 1 -- 172.21.15.50:0/802476496 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858005470 con 0x7f286c0b5c70 2024-09-21T05:59:49.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.195+0000 7f2874600640 1 -- 172.21.15.50:0/802476496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c141d50 con 0x7f286c0b5c70 2024-09-21T05:59:49.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2869600640 1 -- 172.21.15.50:0/802476496 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f28580be060 con 0x7f286c0b5c70 2024-09-21T05:59:49.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2869600640 1 --2- 172.21.15.50:0/802476496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f284c073590 0x7f284c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2869600640 1 -- 172.21.15.50:0/802476496 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f285812e990 con 0x7f286c0b5c70 2024-09-21T05:59:49.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f287be72000 1 Processor -- start 2024-09-21T05:59:49.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2871c00640 1 --2- 172.21.15.50:0/802476496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f284c073590 0x7f284c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f287be72000 1 -- start start 2024-09-21T05:59:49.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2871c00640 1 --2- 172.21.15.50:0/802476496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f284c073590 0x7f284c075aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2860009670 tx=0x7f2860006bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f287be72000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b06268a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f287be72000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 0x55b06268abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f287be72000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b06268b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f287be72000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b06255f410 con 0x55b062149460 2024-09-21T05:59:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f287be72000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b06249cb40 con 0x55b062522ca0 2024-09-21T05:59:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f287be72000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55b06248bf70 con 0x55b062539950 2024-09-21T05:59:49.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2871c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b06268a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2871c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b06268a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60010/0 (socket says 172.21.15.50:60010) 2024-09-21T05:59:49.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2871c00640 1 -- 172.21.15.50:0/745401150 learned_addr learned my addr 172.21.15.50:0/745401150 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:49.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2871200640 1 --2- 172.21.15.50:0/745401150 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 0x55b06268abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2872600640 1 --2- 172.21.15.50:0/745401150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b06268b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2871c00640 1 -- 172.21.15.50:0/745401150 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 msgr2=0x55b06268abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:49.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2871c00640 1 --2- 172.21.15.50:0/745401150 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 0x55b06268abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2871c00640 1 -- 172.21.15.50:0/745401150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 msgr2=0x55b06268b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:49.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.199+0000 7f2871c00640 1 --2- 172.21.15.50:0/745401150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b06268b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f2871c00640 1 -- 172.21.15.50:0/745401150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0624535b0 con 0x55b062149460 2024-09-21T05:59:49.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f2872600640 1 --2- 172.21.15.50:0/745401150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b06268b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:49.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f2871c00640 1 --2- 172.21.15.50:0/745401150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b06268a6a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2860013890 tx=0x7f286000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4152f519d5682f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:49.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f285e000640 1 -- 172.21.15.50:0/745401150 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860013e10 con 0x55b062149460 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f285e000640 1 -- 172.21.15.50:0/745401150 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f286000baf0 con 0x55b062149460 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f285e000640 1 -- 172.21.15.50:0/745401150 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860004020 con 0x55b062149460 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f287be72000 1 -- 172.21.15.50:0/745401150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 msgr2=0x55b06268a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f287be72000 1 --2- 172.21.15.50:0/745401150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b06268a6a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2860013890 tx=0x7f286000e000 comp rx=0 tx=0).stop 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f287be72000 1 -- 172.21.15.50:0/745401150 shutdown_connections 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f287be72000 1 --2- 172.21.15.50:0/745401150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b06268b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f287be72000 1 --2- 172.21.15.50:0/745401150 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 0x55b06268abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f287be72000 1 --2- 172.21.15.50:0/745401150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b06268a6a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f287be72000 1 -- 172.21.15.50:0/745401150 >> 172.21.15.50:0/745401150 conn(0x55b06260fd90 msgr2=0x55b06263f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f287be72000 1 -- 172.21.15.50:0/745401150 shutdown_connections 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f287be72000 1 -- 172.21.15.50:0/745401150 wait complete. 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f287be72000 1 -- start start 2024-09-21T05:59:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.203+0000 7f287be72000 1 Processor -- start 2024-09-21T05:59:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062149460 0x55b062691920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062522ca0 0x55b062691e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062539950 0x55b0626923a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b0624265f0 con 0x55b062539950 2024-09-21T05:59:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b06248fd80 con 0x55b062149460 2024-09-21T05:59:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55b062432520 con 0x55b062522ca0 2024-09-21T05:59:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 10 client.0 _ll_register_callbacks cb 0x55b062436830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2872600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062539950 0x55b0626923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2871c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062149460 0x55b062691920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2872600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062539950 0x55b0626923a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60014/0 (socket says 172.21.15.50:60014) 2024-09-21T05:59:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2872600640 1 -- 172.21.15.50:0/3910698800 learned_addr learned my addr 172.21.15.50:0/3910698800 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:49.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2871200640 1 --2- 172.21.15.50:0/3910698800 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062522ca0 0x55b062691e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2872600640 1 -- 172.21.15.50:0/3910698800 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062522ca0 msgr2=0x55b062691e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:49.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2872600640 1 --2- 172.21.15.50:0/3910698800 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062522ca0 0x55b062691e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2872600640 1 -- 172.21.15.50:0/3910698800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062149460 msgr2=0x55b062691920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:49.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2872600640 1 --2- 172.21.15.50:0/3910698800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062149460 0x55b062691920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2872600640 1 -- 172.21.15.50:0/3910698800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b06242a190 con 0x55b062539950 2024-09-21T05:59:49.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2872600640 1 --2- 172.21.15.50:0/3910698800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062539950 0x55b0626923a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f286c074950 tx=0x7f2864010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3be9aa31e3d757 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:49.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2852a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T05:59:49.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2852a00640 1 -- 172.21.15.50:0/3910698800 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864016070 con 0x55b062539950 2024-09-21T05:59:49.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 1 -- 172.21.15.50:0/3910698800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b062424110 con 0x55b062539950 2024-09-21T05:59:49.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2852a00640 1 -- 172.21.15.50:0/3910698800 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f2864003d90 con 0x55b062539950 2024-09-21T05:59:49.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2852a00640 1 -- 172.21.15.50:0/3910698800 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2864004050 con 0x55b062539950 2024-09-21T05:59:49.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 10 client.5517 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:49.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 1 -- 172.21.15.50:0/3910698800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55b062510f10 con 0x55b062539950 2024-09-21T05:59:49.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2852a00640 1 -- 172.21.15.50:0/3910698800 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b062424110 con 0x55b062539950 2024-09-21T05:59:49.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2852a00640 1 -- 172.21.15.50:0/3910698800 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f2864026080 con 0x55b062539950 2024-09-21T05:59:49.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 10 client.5517 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:49.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 10 client.5517 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:59:49.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 20 client.5517 populate_metadata read hostname 'smithi050' 2024-09-21T05:59:49.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 1 --2- 172.21.15.50:0/3910698800 >> [v2:172.21.15.50:6832/988935074,v1:172.21.15.50:6835/988935074] conn(0x55b06212e7d0 0x55b06269bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 4 client.5517 mds_command: new command op to 5370 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:59:49.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f287be72000 1 -- 172.21.15.50:0/3910698800 --> [v2:172.21.15.50:6832/988935074,v1:172.21.15.50:6835/988935074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b06249ea90 con 0x55b06212e7d0 2024-09-21T05:59:49.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2871c00640 1 --2- 172.21.15.50:0/3910698800 >> [v2:172.21.15.50:6832/988935074,v1:172.21.15.50:6835/988935074] conn(0x55b06212e7d0 0x55b06269bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2871c00640 1 --2- 172.21.15.50:0/3910698800 >> [v2:172.21.15.50:6832/988935074,v1:172.21.15.50:6835/988935074] conn(0x55b06212e7d0 0x55b06269bd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c7b8da135325d1e server_cookie=4754ae1473901f31 in_seq=0 out_seq=0 2024-09-21T05:59:49.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.207+0000 7f2852a00640 10 client.5517 ms_handle_connect on v2:172.21.15.50:6832/988935074 2024-09-21T05:59:49.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f2852a00640 1 -- 172.21.15.50:0/3910698800 <== mds.0 v2:172.21.15.50:6832/988935074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b06249ea90 con 0x55b06212e7d0 2024-09-21T05:59:49.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f2852a00640 10 client.5517 handle_command_reply: tid=0 2024-09-21T05:59:49.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f287be72000 1 client.5517 shutdown 2024-09-21T05:59:49.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f287be72000 1 -- 172.21.15.50:0/3910698800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062539950 msgr2=0x55b0626923a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f287be72000 1 --2- 172.21.15.50:0/3910698800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062539950 0x55b0626923a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f286c074950 tx=0x7f2864010a40 comp rx=0 tx=0).stop 2024-09-21T05:59:49.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f287be72000 1 -- 172.21.15.50:0/3910698800 shutdown_connections 2024-09-21T05:59:49.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f287be72000 1 --2- 172.21.15.50:0/3910698800 >> [v2:172.21.15.50:6832/988935074,v1:172.21.15.50:6835/988935074] conn(0x55b06212e7d0 0x55b06269bd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f287be72000 1 --2- 172.21.15.50:0/3910698800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062539950 0x55b0626923a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f287be72000 1 --2- 172.21.15.50:0/3910698800 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062522ca0 0x55b062691e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f287be72000 1 --2- 172.21.15.50:0/3910698800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062149460 0x55b062691920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f287be72000 1 -- 172.21.15.50:0/3910698800 >> 172.21.15.50:0/3910698800 conn(0x55b06260fd90 msgr2=0x55b062646440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f2852a00640 0 client.5517 ms_handle_reset on v2:172.21.15.50:6832/988935074 2024-09-21T05:59:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f287be72000 1 -- 172.21.15.50:0/3910698800 shutdown_connections 2024-09-21T05:59:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f287be72000 1 -- 172.21.15.50:0/3910698800 wait complete. 2024-09-21T05:59:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.211+0000 7f287be72000 20 client.5517 trim_cache size 0 max 16384 2024-09-21T05:59:49.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 Processor -- start 2024-09-21T05:59:49.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 -- start start 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b0626fc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b0626fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 0x55b0626fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f286c200eb0 con 0x55b062149460 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f286c0ce2a0 con 0x55b062522ca0 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f286c143050 con 0x55b062539950 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f2871c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b0626fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f2871c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b0626fc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60028/0 (socket says 172.21.15.50:60028) 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f2871c00640 1 -- 172.21.15.50:0/3778086276 learned_addr learned my addr 172.21.15.50:0/3778086276 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f2872600640 1 --2- 172.21.15.50:0/3778086276 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 0x55b0626fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f2871200640 1 --2- 172.21.15.50:0/3778086276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b0626fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f2871c00640 1 -- 172.21.15.50:0/3778086276 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 msgr2=0x55b0626fd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f2871c00640 1 --2- 172.21.15.50:0/3778086276 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 0x55b0626fd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f2871c00640 1 -- 172.21.15.50:0/3778086276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 msgr2=0x55b0626fcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:49.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f2871c00640 1 --2- 172.21.15.50:0/3778086276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b0626fcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f2871c00640 1 -- 172.21.15.50:0/3778086276 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0624e7db0 con 0x55b062149460 2024-09-21T05:59:49.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f2871c00640 1 --2- 172.21.15.50:0/3778086276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b0626fc620 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f286001e000 tx=0x7f2860004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac70e9e5a0928b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:49.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f285d600640 1 -- 172.21.15.50:0/3778086276 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860004040 con 0x55b062149460 2024-09-21T05:59:49.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f285d600640 1 -- 172.21.15.50:0/3778086276 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28600041e0 con 0x55b062149460 2024-09-21T05:59:49.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f285d600640 1 -- 172.21.15.50:0/3778086276 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860004410 con 0x55b062149460 2024-09-21T05:59:49.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 -- 172.21.15.50:0/3778086276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 msgr2=0x55b0626fc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 --2- 172.21.15.50:0/3778086276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b0626fc620 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f286001e000 tx=0x7f2860004770 comp rx=0 tx=0).stop 2024-09-21T05:59:49.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 -- 172.21.15.50:0/3778086276 shutdown_connections 2024-09-21T05:59:49.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 --2- 172.21.15.50:0/3778086276 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 0x55b0626fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 --2- 172.21.15.50:0/3778086276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b0626fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 --2- 172.21.15.50:0/3778086276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b0626fc620 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 -- 172.21.15.50:0/3778086276 >> 172.21.15.50:0/3778086276 conn(0x55b06260fd90 msgr2=0x55b062688620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:49.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 -- 172.21.15.50:0/3778086276 shutdown_connections 2024-09-21T05:59:49.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.231+0000 7f287be72000 1 -- 172.21.15.50:0/3778086276 wait complete. 2024-09-21T05:59:49.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f287be72000 1 -- start start 2024-09-21T05:59:49.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f287be72000 1 Processor -- start 2024-09-21T05:59:49.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f287be72000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b0627005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f287be72000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b062700ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f287be72000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 0x55b062701020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f287be72000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f286c0cdc50 con 0x55b062149460 2024-09-21T05:59:49.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f287be72000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b06254b750 con 0x55b062522ca0 2024-09-21T05:59:49.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f287be72000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55b06249b910 con 0x55b062539950 2024-09-21T05:59:49.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f287be72000 10 client.0 _ll_register_callbacks cb 0x7f28640094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:49.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2871c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b0627005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2871c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b0627005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60044/0 (socket says 172.21.15.50:60044) 2024-09-21T05:59:49.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2871c00640 1 -- 172.21.15.50:0/743893903 learned_addr learned my addr 172.21.15.50:0/743893903 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:49.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2872600640 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 0x55b062701020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2871200640 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b062700ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2871c00640 1 -- 172.21.15.50:0/743893903 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 msgr2=0x55b062701020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:49.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2871c00640 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 0x55b062701020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2871c00640 1 -- 172.21.15.50:0/743893903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 msgr2=0x55b062700ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:49.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2871c00640 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b062700ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2871c00640 1 -- 172.21.15.50:0/743893903 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b062452ab0 con 0x55b062149460 2024-09-21T05:59:49.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2871c00640 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b0627005a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2860015010 tx=0x7f28600094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbc114aab36d038 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:49.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2852a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T05:59:49.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2852a00640 1 -- 172.21.15.50:0/743893903 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286003d030 con 0x55b062149460 2024-09-21T05:59:49.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2852a00640 1 -- 172.21.15.50:0/743893903 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f2860035050 con 0x55b062149460 2024-09-21T05:59:49.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f287be72000 1 -- 172.21.15.50:0/743893903 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28600041e0 con 0x55b062149460 2024-09-21T05:59:49.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f2852a00640 1 -- 172.21.15.50:0/743893903 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2860004390 con 0x55b062149460 2024-09-21T05:59:49.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f287be72000 10 client.5523 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:49.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.235+0000 7f287be72000 1 -- 172.21.15.50:0/743893903 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55b062431070 con 0x55b062149460 2024-09-21T05:59:49.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2852a00640 1 -- 172.21.15.50:0/743893903 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28600041e0 con 0x55b062149460 2024-09-21T05:59:49.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2852a00640 1 -- 172.21.15.50:0/743893903 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f2860030070 con 0x55b062149460 2024-09-21T05:59:49.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 10 client.5523 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:49.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 10 client.5523 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:59:49.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 20 client.5523 populate_metadata read hostname 'smithi050' 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.50:6832/988935074,v1:172.21.15.50:6835/988935074] conn(0x55b06211c830 0x55b0626bc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 4 client.5523 mds_command: new command op to 5370 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 -- 172.21.15.50:0/743893903 --> [v2:172.21.15.50:6832/988935074,v1:172.21.15.50:6835/988935074] -- command(tid 0: {"prefix": "status"}) -- 0x55b06249a540 con 0x55b06211c830 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2871200640 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.50:6832/988935074,v1:172.21.15.50:6835/988935074] conn(0x55b06211c830 0x55b0626bc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2852a00640 10 client.5523 ms_handle_connect on v2:172.21.15.50:6832/988935074 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2871200640 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.50:6832/988935074,v1:172.21.15.50:6835/988935074] conn(0x55b06211c830 0x55b0626bc610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9708284354f8947a server_cookie=73ba0d0428728e1b in_seq=0 out_seq=0 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2852a00640 1 -- 172.21.15.50:0/743893903 <== mds.0 v2:172.21.15.50:6832/988935074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b06249a540 con 0x55b06211c830 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2852a00640 10 client.5523 handle_command_reply: tid=0 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 client.5523 shutdown 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 -- 172.21.15.50:0/743893903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 msgr2=0x55b0627005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b0627005a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2860015010 tx=0x7f28600094e0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 -- 172.21.15.50:0/743893903 shutdown_connections 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.50:6832/988935074,v1:172.21.15.50:6835/988935074] conn(0x55b06211c830 0x55b0626bc610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55b062539950 0x55b062701020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b062522ca0 0x55b062700ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 --2- 172.21.15.50:0/743893903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b062149460 0x55b0627005a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2852a00640 0 client.5523 ms_handle_reset on v2:172.21.15.50:6832/988935074 2024-09-21T05:59:49.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 -- 172.21.15.50:0/743893903 >> 172.21.15.50:0/743893903 conn(0x55b06260fd90 msgr2=0x55b06269b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:49.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 -- 172.21.15.50:0/743893903 shutdown_connections 2024-09-21T05:59:49.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 1 -- 172.21.15.50:0/743893903 wait complete. 2024-09-21T05:59:49.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f287be72000 20 client.5523 trim_cache size 0 max 16384 2024-09-21T05:59:49.246 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T05:59:49.247 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T05:59:49.247 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-21T05:59:49.247 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 5370, 2024-09-21T05:59:49.247 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-21T05:59:49.247 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-21T05:59:49.247 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-21T05:59:49.247 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 10.821308803000001, 2024-09-21T05:59:49.247 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 27, 2024-09-21T05:59:49.247 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 44, 2024-09-21T05:59:49.247 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 44, 2024-09-21T05:59:49.247 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 15.213229197 2024-09-21T05:59:49.247 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T05:59:49.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2874600640 1 -- 172.21.15.50:0/802476496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f284c073590 msgr2=0x7f284c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2874600640 1 --2- 172.21.15.50:0/802476496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f284c073590 0x7f284c075aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2860009670 tx=0x7f2860006bd0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2874600640 1 -- 172.21.15.50:0/802476496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f286c0b5c70 msgr2=0x7f286c0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2874600640 1 --2- 172.21.15.50:0/802476496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f286c0b5c70 0x7f286c0affb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f28580b0040 tx=0x7f28580a67d0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2874600640 1 -- 172.21.15.50:0/802476496 shutdown_connections 2024-09-21T05:59:49.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2874600640 1 --2- 172.21.15.50:0/802476496 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f286c1527b0 0x7f286c0b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2874600640 1 --2- 172.21.15.50:0/802476496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f284c073590 0x7f284c075aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2874600640 1 --2- 172.21.15.50:0/802476496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f286c0b5c70 0x7f286c0affb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2874600640 1 --2- 172.21.15.50:0/802476496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f286c0b4de0 0x7f286c0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2874600640 1 -- 172.21.15.50:0/802476496 >> 172.21.15.50:0/802476496 conn(0x7f286c0b2040 msgr2=0x7f286c154b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:49.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2874600640 1 -- 172.21.15.50:0/802476496 shutdown_connections 2024-09-21T05:59:49.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.239+0000 7f2874600640 1 -- 172.21.15.50:0/802476496 wait complete. 2024-09-21T05:59:49.263 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:59:49.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.415+0000 7f9edee00640 1 -- 172.21.15.50:0/11018325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed80b5a80 msgr2=0x7f9ed8074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.415+0000 7f9edee00640 1 --2- 172.21.15.50:0/11018325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed80b5a80 0x7f9ed8074410 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009840 tx=0x7f9ec001c820 comp rx=0 tx=0).stop 2024-09-21T05:59:49.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.415+0000 7f9edee00640 1 -- 172.21.15.50:0/11018325 shutdown_connections 2024-09-21T05:59:49.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.415+0000 7f9edee00640 1 --2- 172.21.15.50:0/11018325 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ed8074950 0x7f9ed815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.415+0000 7f9edee00640 1 --2- 172.21.15.50:0/11018325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed80b5a80 0x7f9ed8074410 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.415+0000 7f9edee00640 1 -- 172.21.15.50:0/11018325 >> 172.21.15.50:0/11018325 conn(0x7f9ed80708c0 msgr2=0x7f9ed8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:49.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.415+0000 7f9edee00640 1 -- 172.21.15.50:0/11018325 shutdown_connections 2024-09-21T05:59:49.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.415+0000 7f9edee00640 1 -- 172.21.15.50:0/11018325 wait complete. 2024-09-21T05:59:49.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.419+0000 7f9edee00640 1 Processor -- start 2024-09-21T05:59:49.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.419+0000 7f9edee00640 1 -- start start 2024-09-21T05:59:49.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.419+0000 7f9edee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed8074950 0x7f9ed8146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.419+0000 7f9edee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed80b5a80 0x7f9ed8154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.419+0000 7f9edee00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ed8157030 0x7f9ed80d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.419+0000 7f9edee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed816a310 con 0x7f9ed80b5a80 2024-09-21T05:59:49.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.419+0000 7f9edee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed816a190 con 0x7f9ed8074950 2024-09-21T05:59:49.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.419+0000 7f9edee00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed816a490 con 0x7f9ed8157030 2024-09-21T05:59:49.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.419+0000 7f9ecfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed80b5a80 0x7f9ed8154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.419+0000 7f9ecfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed80b5a80 0x7f9ed8154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60052/0 (socket says 172.21.15.50:60052) 2024-09-21T05:59:49.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.419+0000 7f9ecfe00640 1 -- 172.21.15.50:0/3222221792 learned_addr learned my addr 172.21.15.50:0/3222221792 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:49.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.419+0000 7f9edce00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ed8157030 0x7f9ed80d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed8074950 0x7f9ed8146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9ecfe00640 1 -- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ed8157030 msgr2=0x7f9ed80d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9ecfe00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ed8157030 0x7f9ed80d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9ecfe00640 1 -- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed8074950 msgr2=0x7f9ed8146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9ecfe00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed8074950 0x7f9ed8146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9ecfe00640 1 -- 172.21.15.50:0/3222221792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed81fd8c0 con 0x7f9ed80b5a80 2024-09-21T05:59:49.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9ecfe00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed80b5a80 0x7f9ed8154c90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009790 tx=0x7f9ec00a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:49.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9ed4c00640 1 -- 172.21.15.50:0/3222221792 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00bb070 con 0x7f9ed80b5a80 2024-09-21T05:59:49.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9ed4c00640 1 -- 172.21.15.50:0/3222221792 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ec0004470 con 0x7f9ed80b5a80 2024-09-21T05:59:49.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9edee00640 1 -- 172.21.15.50:0/3222221792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed814c0e0 con 0x7f9ed80b5a80 2024-09-21T05:59:49.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9ed4c00640 1 -- 172.21.15.50:0/3222221792 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00b2040 con 0x7f9ed80b5a80 2024-09-21T05:59:49.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9edee00640 1 -- 172.21.15.50:0/3222221792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed814c6a0 con 0x7f9ed80b5a80 2024-09-21T05:59:49.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.423+0000 7f9edee00640 1 -- 172.21.15.50:0/3222221792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed8155730 con 0x7f9ed80b5a80 2024-09-21T05:59:49.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.427+0000 7f9ed4c00640 1 -- 172.21.15.50:0/3222221792 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f9ec0005290 con 0x7f9ed80b5a80 2024-09-21T05:59:49.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.427+0000 7f9ed4c00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9eb4073590 0x7f9eb4075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:49.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.427+0000 7f9ed4c00640 1 -- 172.21.15.50:0/3222221792 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9ec012e640 con 0x7f9ed80b5a80 2024-09-21T05:59:49.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.427+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9eb4073590 0x7f9eb4075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:49.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.427+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9eb4073590 0x7f9eb4075aa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8007fd0 tx=0x7f9ec8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:49.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.431+0000 7f9ed4c00640 1 -- 172.21.15.50:0/3222221792 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9ec00c4020 con 0x7f9ed80b5a80 2024-09-21T05:59:49.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.647+0000 7f9edee00640 1 -- 172.21.15.50:0/3222221792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9ed815cf90 con 0x7f9ed80b5a80 2024-09-21T05:59:49.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.647+0000 7f9ed4c00640 1 -- 172.21.15.50:0/3222221792 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f9ec0016410 con 0x7f9ed80b5a80 2024-09-21T05:59:49.651 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:59:49.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.651+0000 7f9edee00640 1 -- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9eb4073590 msgr2=0x7f9eb4075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.651+0000 7f9edee00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9eb4073590 0x7f9eb4075aa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8007fd0 tx=0x7f9ec8009040 comp rx=0 tx=0).stop 2024-09-21T05:59:49.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.651+0000 7f9edee00640 1 -- 172.21.15.50:0/3222221792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed80b5a80 msgr2=0x7f9ed8154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:49.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.651+0000 7f9edee00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed80b5a80 0x7f9ed8154c90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009790 tx=0x7f9ec00a5650 comp rx=0 tx=0).stop 2024-09-21T05:59:49.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.651+0000 7f9edee00640 1 -- 172.21.15.50:0/3222221792 shutdown_connections 2024-09-21T05:59:49.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.651+0000 7f9edee00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ed8157030 0x7f9ed80d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.651+0000 7f9edee00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9eb4073590 0x7f9eb4075aa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.651+0000 7f9edee00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed80b5a80 0x7f9ed8154c90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.651+0000 7f9edee00640 1 --2- 172.21.15.50:0/3222221792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed8074950 0x7f9ed8146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:49.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.651+0000 7f9edee00640 1 -- 172.21.15.50:0/3222221792 >> 172.21.15.50:0/3222221792 conn(0x7f9ed80708c0 msgr2=0x7f9ed81561a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:49.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.651+0000 7f9edee00640 1 -- 172.21.15.50:0/3222221792 shutdown_connections 2024-09-21T05:59:49.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49.651+0000 7f9edee00640 1 -- 172.21.15.50:0/3222221792 wait complete. 2024-09-21T05:59:49.674 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:59:49.674 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:59:49.674 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-21T05:59:49.674 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:59:49.674 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:59:49.674 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:59:49.674 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:59:49.674 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:59:49.674 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:59:49.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:49.675 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-21T05:59:49.684 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:59:49.684 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:59:49.684 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:59:49.684 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:59:49.684 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T05:59:49.684 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:59:49.684 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:59:49.684 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:59:49.684 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42206sec preferred_lft 42206sec 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T05:59:49.685 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:8b:d0:0a:5d brd ff:ff:ff:ff:ff:ff 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T05:59:49.686 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:59:49.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:49.687 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:59:49.747 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-21T05:59:49.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:49.749 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-21T05:59:49.806 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-21T05:59:49.806 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:59:49.806 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-21T05:59:49.806 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T05:59:49.806 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-21T05:59:49.806 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:59:49.807 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:59:49.807 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T05:59:49.807 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:59:49.807 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:59:49.807 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:59:49.807 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T05:59:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:59:49.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:59:49.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:49.946 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T05:59:49.955 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:59:49.955 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:59:49.955 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:59:49.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:49.955 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-09-21T05:59:50.021 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:59:50.021 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:59:50.073 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:59:50.073 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:59:50.073 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:59:50.124 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:59:50.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:50.124 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:59:50.226 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:59:50.226 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:59:50.226 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T05:59:50.244 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:59:50.244 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:59:50.244 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T05:59:50.244 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T05:59:50.244 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:59:50.255 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:59:50.255 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:59:50.255 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T05:59:50.255 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T05:59:50.256 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:59:50.256 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T05:59:50.256 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T05:59:50.256 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T05:59:50.259 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:59:50.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:50.260 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:59:50.281 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:59:50.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:59:50.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:59:50.392 INFO:teuthology.orchestra.run.smithi050.stdout:client5126 2024-09-21T05:59:50.392 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:59:50.392 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:59:50.401 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T05:59:50.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.603+0000 7fcd84200640 1 -- 172.21.15.50:0/1489120835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd64003a60 msgr2=0x7fcd64003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:50.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.603+0000 7fcd84200640 1 --2- 172.21.15.50:0/1489120835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd64003a60 0x7fcd64003ed0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009b60 tx=0x7fcd7001d220 comp rx=0 tx=0).stop 2024-09-21T05:59:50.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.603+0000 7fcd82200640 1 -- 172.21.15.50:0/1489120835 reap_dead start 2024-09-21T05:59:50.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.603+0000 7fcd84200640 1 -- 172.21.15.50:0/1489120835 shutdown_connections 2024-09-21T05:59:50.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.603+0000 7fcd84200640 1 -- 172.21.15.50:0/1489120835 >> 172.21.15.50:0/1489120835 conn(0x7fcd7c0b2c40 msgr2=0x7fcd7c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:50.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.603+0000 7fcd84200640 1 -- 172.21.15.50:0/1489120835 shutdown_connections 2024-09-21T05:59:50.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.603+0000 7fcd84200640 1 -- 172.21.15.50:0/1489120835 wait complete. 2024-09-21T05:59:50.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd84200640 1 Processor -- start 2024-09-21T05:59:50.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd84200640 1 -- start start 2024-09-21T05:59:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd84200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd64003a60 0x7fcd7c14d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd84200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7c161300 0x7fcd7c161710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd84200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd7c1528c0 0x7fcd7c152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd84200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c06f980 con 0x7fcd64003a60 2024-09-21T05:59:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd84200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c0b3d50 con 0x7fcd7c161300 2024-09-21T05:59:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd84200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7c06fb00 con 0x7fcd7c1528c0 2024-09-21T05:59:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd81800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd64003a60 0x7fcd7c14d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd81800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd64003a60 0x7fcd7c14d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60068/0 (socket says 172.21.15.50:60068) 2024-09-21T05:59:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd81800640 1 -- 172.21.15.50:0/2111904315 learned_addr learned my addr 172.21.15.50:0/2111904315 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd80e00640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7c161300 0x7fcd7c161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd82200640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd7c1528c0 0x7fcd7c152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd81800640 1 -- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd7c1528c0 msgr2=0x7fcd7c152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:50.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd81800640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd7c1528c0 0x7fcd7c152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:50.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd81800640 1 -- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7c161300 msgr2=0x7fcd7c161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:50.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd81800640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7c161300 0x7fcd7c161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:50.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.607+0000 7fcd81800640 1 -- 172.21.15.50:0/2111904315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c161c50 con 0x7fcd64003a60 2024-09-21T05:59:50.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.611+0000 7fcd81800640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd64003a60 0x7fcd7c14d960 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcd700af040 tx=0x7fcd700055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:50.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.611+0000 7fcd79600640 1 -- 172.21.15.50:0/2111904315 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd700b1040 con 0x7fcd64003a60 2024-09-21T05:59:50.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.611+0000 7fcd79600640 1 -- 172.21.15.50:0/2111904315 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd70004030 con 0x7fcd64003a60 2024-09-21T05:59:50.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.611+0000 7fcd84200640 1 -- 172.21.15.50:0/2111904315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7c153210 con 0x7fcd64003a60 2024-09-21T05:59:50.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.611+0000 7fcd79600640 1 -- 172.21.15.50:0/2111904315 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd700a5e60 con 0x7fcd64003a60 2024-09-21T05:59:50.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.611+0000 7fcd84200640 1 -- 172.21.15.50:0/2111904315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7c1fc9a0 con 0x7fcd64003a60 2024-09-21T05:59:50.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.611+0000 7fcd84200640 1 -- 172.21.15.50:0/2111904315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7c073dc0 con 0x7fcd64003a60 2024-09-21T05:59:50.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.611+0000 7fcd79600640 1 -- 172.21.15.50:0/2111904315 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fcd700bd020 con 0x7fcd64003a60 2024-09-21T05:59:50.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.611+0000 7fcd79600640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcd5c073560 0x7fcd5c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:50.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.611+0000 7fcd79600640 1 -- 172.21.15.50:0/2111904315 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fcd7012ef70 con 0x7fcd64003a60 2024-09-21T05:59:50.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.615+0000 7fcd80e00640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcd5c073560 0x7fcd5c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:50.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.615+0000 7fcd80e00640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcd5c073560 0x7fcd5c075a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c153470 tx=0x7fcd680091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:50.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.619+0000 7fcd79600640 1 -- 172.21.15.50:0/2111904315 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcd700c4020 con 0x7fcd64003a60 2024-09-21T05:59:50.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:50.831+0000 7fcd84200640 1 -- 172.21.15.50:0/2111904315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fcd7c0b1b40 con 0x7fcd64003a60 2024-09-21T05:59:51.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.483+0000 7fcd79600640 1 -- 172.21.15.50:0/2111904315 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v28) ==== 136+0+0 (secure 0 0 0) 0x7fcd7c161c50 con 0x7fcd64003a60 2024-09-21T05:59:51.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.487+0000 7fcd84200640 1 -- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcd5c073560 msgr2=0x7fcd5c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:51.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.487+0000 7fcd84200640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcd5c073560 0x7fcd5c075a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c153470 tx=0x7fcd680091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.487+0000 7fcd84200640 1 -- 172.21.15.50:0/2111904315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd64003a60 msgr2=0x7fcd7c14d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:51.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.487+0000 7fcd84200640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd64003a60 0x7fcd7c14d960 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcd700af040 tx=0x7fcd700055b0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.487+0000 7fcd84200640 1 -- 172.21.15.50:0/2111904315 shutdown_connections 2024-09-21T05:59:51.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.487+0000 7fcd84200640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd7c1528c0 0x7fcd7c152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.487+0000 7fcd84200640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7c161300 0x7fcd7c161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.487+0000 7fcd84200640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcd5c073560 0x7fcd5c075a70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.487+0000 7fcd84200640 1 --2- 172.21.15.50:0/2111904315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd64003a60 0x7fcd7c14d960 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.487+0000 7fcd84200640 1 -- 172.21.15.50:0/2111904315 >> 172.21.15.50:0/2111904315 conn(0x7fcd7c0b2c40 msgr2=0x7fcd7c075d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:51.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.487+0000 7fcd84200640 1 -- 172.21.15.50:0/2111904315 shutdown_connections 2024-09-21T05:59:51.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.487+0000 7fcd84200640 1 -- 172.21.15.50:0/2111904315 wait complete. 2024-09-21T05:59:51.513 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T05:59:51.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.671+0000 7f0a27e00640 1 -- 172.21.15.50:0/4210665968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a200b5c70 msgr2=0x7f0a2015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:51.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.671+0000 7f0a27e00640 1 --2- 172.21.15.50:0/4210665968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a200b5c70 0x7f0a2015bb60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0a20074b20 tx=0x7f0a1401c820 comp rx=0 tx=0).stop 2024-09-21T05:59:51.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.671+0000 7f0a27e00640 1 -- 172.21.15.50:0/4210665968 shutdown_connections 2024-09-21T05:59:51.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.671+0000 7f0a27e00640 1 --2- 172.21.15.50:0/4210665968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a200b5c70 0x7f0a2015bb60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.671+0000 7f0a27e00640 1 -- 172.21.15.50:0/4210665968 >> 172.21.15.50:0/4210665968 conn(0x7f0a200b2040 msgr2=0x7f0a200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:51.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.671+0000 7f0a27e00640 1 -- 172.21.15.50:0/4210665968 shutdown_connections 2024-09-21T05:59:51.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.671+0000 7f0a27e00640 1 -- 172.21.15.50:0/4210665968 wait complete. 2024-09-21T05:59:51.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.671+0000 7f0a27e00640 1 Processor -- start 2024-09-21T05:59:51.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.671+0000 7f0a27e00640 1 -- start start 2024-09-21T05:59:51.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a27e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a200b4de0 0x7f0a201fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:51.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a27e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0a200b5c70 0x7f0a20158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:51.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a27e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a201527b0 0x7f0a20158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:51.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a27e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a20169b30 con 0x7f0a200b4de0 2024-09-21T05:59:51.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a27e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0a201699b0 con 0x7f0a201527b0 2024-09-21T05:59:51.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a27e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0a20169cb0 con 0x7f0a200b5c70 2024-09-21T05:59:51.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a25400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a200b4de0 0x7f0a201fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:51.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a25400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a200b4de0 0x7f0a201fa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60072/0 (socket says 172.21.15.50:60072) 2024-09-21T05:59:51.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a25400640 1 -- 172.21.15.50:0/299542849 learned_addr learned my addr 172.21.15.50:0/299542849 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:51.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a24a00640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0a200b5c70 0x7f0a20158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:51.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a25e00640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a201527b0 0x7f0a20158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:51.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a25400640 1 -- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0a200b5c70 msgr2=0x7f0a20158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:51.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a25400640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0a200b5c70 0x7f0a20158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a25400640 1 -- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a201527b0 msgr2=0x7f0a20158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:51.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a25400640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a201527b0 0x7f0a20158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a25400640 1 -- 172.21.15.50:0/299542849 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a201f7820 con 0x7f0a200b4de0 2024-09-21T05:59:51.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a25400640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a200b4de0 0x7f0a201fa7b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0a140b0040 tx=0x7f0a140040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:51.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a1d600640 1 -- 172.21.15.50:0/299542849 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a140b2040 con 0x7f0a200b4de0 2024-09-21T05:59:51.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a1d600640 1 -- 172.21.15.50:0/299542849 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a140bd070 con 0x7f0a200b4de0 2024-09-21T05:59:51.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a1d600640 1 -- 172.21.15.50:0/299542849 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14003860 con 0x7f0a200b4de0 2024-09-21T05:59:51.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a27e00640 1 -- 172.21.15.50:0/299542849 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a201f6ca0 con 0x7f0a200b4de0 2024-09-21T05:59:51.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a27e00640 1 -- 172.21.15.50:0/299542849 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a201f7260 con 0x7f0a200b4de0 2024-09-21T05:59:51.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.675+0000 7f0a27e00640 1 -- 172.21.15.50:0/299542849 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2015bb60 con 0x7f0a200b4de0 2024-09-21T05:59:51.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.679+0000 7f0a1d600640 1 -- 172.21.15.50:0/299542849 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f0a14003a00 con 0x7f0a200b4de0 2024-09-21T05:59:51.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.679+0000 7f0a1d600640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0a00073590 0x7f0a00075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:51.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.679+0000 7f0a1d600640 1 -- 172.21.15.50:0/299542849 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f0a1412eef0 con 0x7f0a200b4de0 2024-09-21T05:59:51.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.679+0000 7f0a24a00640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0a00073590 0x7f0a00075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:51.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.679+0000 7f0a24a00640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0a00073590 0x7f0a00075aa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c000f70 tx=0x7f0a0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.683+0000 7f0a1d600640 1 -- 172.21.15.50:0/299542849 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0a140c6020 con 0x7f0a200b4de0 2024-09-21T05:59:51.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.879+0000 7f0a27e00640 1 -- 172.21.15.50:0/299542849 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f0a201f68b0 con 0x7f0a00073590 2024-09-21T05:59:51.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.883+0000 7f0a1d600640 1 -- 172.21.15.50:0/299542849 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f0a0c001c80 con 0x7f0a00073590 2024-09-21T05:59:51.887 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T05:59:51.887 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T05:59:51.887 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T05:59:51.887 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T05:59:51.887 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T05:59:51.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.887+0000 7f0a27e00640 1 -- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0a00073590 msgr2=0x7f0a00075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:51.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.887+0000 7f0a27e00640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0a00073590 0x7f0a00075aa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c000f70 tx=0x7f0a0c009040 comp rx=0 tx=0).stop 2024-09-21T05:59:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.887+0000 7f0a27e00640 1 -- 172.21.15.50:0/299542849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a200b4de0 msgr2=0x7f0a201fa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.887+0000 7f0a27e00640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a200b4de0 0x7f0a201fa7b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0a140b0040 tx=0x7f0a140040c0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.887+0000 7f0a27e00640 1 -- 172.21.15.50:0/299542849 shutdown_connections 2024-09-21T05:59:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.887+0000 7f0a27e00640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a201527b0 0x7f0a20158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.887+0000 7f0a27e00640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0a200b5c70 0x7f0a20158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.887+0000 7f0a27e00640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0a00073590 0x7f0a00075aa0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.887+0000 7f0a27e00640 1 --2- 172.21.15.50:0/299542849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a200b4de0 0x7f0a201fa7b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.887+0000 7f0a27e00640 1 -- 172.21.15.50:0/299542849 >> 172.21.15.50:0/299542849 conn(0x7f0a200b2040 msgr2=0x7f0a2015a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.887+0000 7f0a27e00640 1 -- 172.21.15.50:0/299542849 shutdown_connections 2024-09-21T05:59:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:51.887+0000 7f0a27e00640 1 -- 172.21.15.50:0/299542849 wait complete. 2024-09-21T05:59:51.910 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T05:59:52.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.067+0000 7fe458c00640 1 -- 172.21.15.50:0/1775493514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe454160470 msgr2=0x7fe454074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.067+0000 7fe458c00640 1 --2- 172.21.15.50:0/1775493514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe454160470 0x7fe454074780 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009c80 tx=0x7fe44c01cd10 comp rx=0 tx=0).stop 2024-09-21T05:59:52.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.067+0000 7fe458c00640 1 -- 172.21.15.50:0/1775493514 shutdown_connections 2024-09-21T05:59:52.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.067+0000 7fe458c00640 1 --2- 172.21.15.50:0/1775493514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe454160470 0x7fe454074780 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.067+0000 7fe458c00640 1 --2- 172.21.15.50:0/1775493514 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe454160060 0x7fe454074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.067+0000 7fe458c00640 1 -- 172.21.15.50:0/1775493514 >> 172.21.15.50:0/1775493514 conn(0x7fe4540b4560 msgr2=0x7fe4540d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:52.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.067+0000 7fe458c00640 1 -- 172.21.15.50:0/1775493514 shutdown_connections 2024-09-21T05:59:52.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.067+0000 7fe458c00640 1 -- 172.21.15.50:0/1775493514 wait complete. 2024-09-21T05:59:52.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe458c00640 1 Processor -- start 2024-09-21T05:59:52.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe458c00640 1 -- start start 2024-09-21T05:59:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe458c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4541536f0 0x7fe45414f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe458c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe454160060 0x7fe4540d1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe458c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe454160470 0x7fe454155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe458c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe454169790 con 0x7fe4541536f0 2024-09-21T05:59:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe458c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe454169610 con 0x7fe454160060 2024-09-21T05:59:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe458c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe454169910 con 0x7fe454160470 2024-09-21T05:59:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe453400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4541536f0 0x7fe45414f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe453400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4541536f0 0x7fe45414f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60088/0 (socket says 172.21.15.50:60088) 2024-09-21T05:59:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe453400640 1 -- 172.21.15.50:0/407916881 learned_addr learned my addr 172.21.15.50:0/407916881 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe453e00640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe454160470 0x7fe454155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe44be00640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe454160060 0x7fe4540d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe453400640 1 -- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe454160470 msgr2=0x7fe454155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe453400640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe454160470 0x7fe454155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe453400640 1 -- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe454160060 msgr2=0x7fe4540d1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe453400640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe454160060 0x7fe4540d1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe453400640 1 -- 172.21.15.50:0/407916881 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4541fb6f0 con 0x7fe4541536f0 2024-09-21T05:59:52.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe453400640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4541536f0 0x7fe45414f1a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe44400c3c0 tx=0x7fe44400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:52.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe44b400640 1 -- 172.21.15.50:0/407916881 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444017070 con 0x7fe4541536f0 2024-09-21T05:59:52.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe44b400640 1 -- 172.21.15.50:0/407916881 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4440053c0 con 0x7fe4541536f0 2024-09-21T05:59:52.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe458c00640 1 -- 172.21.15.50:0/407916881 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4540d24b0 con 0x7fe4541536f0 2024-09-21T05:59:52.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe44b400640 1 -- 172.21.15.50:0/407916881 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444003860 con 0x7fe4541536f0 2024-09-21T05:59:52.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.071+0000 7fe458c00640 1 -- 172.21.15.50:0/407916881 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4541fbc70 con 0x7fe4541536f0 2024-09-21T05:59:52.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.075+0000 7fe458c00640 1 -- 172.21.15.50:0/407916881 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45415e600 con 0x7fe4541536f0 2024-09-21T05:59:52.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.075+0000 7fe44b400640 1 -- 172.21.15.50:0/407916881 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fe44401a070 con 0x7fe4541536f0 2024-09-21T05:59:52.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.075+0000 7fe44b400640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe430073560 0x7fe430075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:52.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.075+0000 7fe44b400640 1 -- 172.21.15.50:0/407916881 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fe444093c10 con 0x7fe4541536f0 2024-09-21T05:59:52.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.075+0000 7fe44be00640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe430073560 0x7fe430075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:52.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.079+0000 7fe44be00640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe430073560 0x7fe430075a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe4540d18d0 tx=0x7fe43c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:52.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.079+0000 7fe44b400640 1 -- 172.21.15.50:0/407916881 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe44405e6d0 con 0x7fe4541536f0 2024-09-21T05:59:52.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 -- 172.21.15.50:0/407916881 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe4540d27f0 con 0x7fe4541536f0 2024-09-21T05:59:52.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe44b400640 1 -- 172.21.15.50:0/407916881 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7fe44405dfd0 con 0x7fe4541536f0 2024-09-21T05:59:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 -- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe430073560 msgr2=0x7fe430075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe430073560 0x7fe430075a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe4540d18d0 tx=0x7fe43c0091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 -- 172.21.15.50:0/407916881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4541536f0 msgr2=0x7fe45414f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4541536f0 0x7fe45414f1a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe44400c3c0 tx=0x7fe44400c880 comp rx=0 tx=0).stop 2024-09-21T05:59:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 -- 172.21.15.50:0/407916881 shutdown_connections 2024-09-21T05:59:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe454160470 0x7fe454155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe454160060 0x7fe4540d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe430073560 0x7fe430075a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 --2- 172.21.15.50:0/407916881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4541536f0 0x7fe45414f1a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 -- 172.21.15.50:0/407916881 >> 172.21.15.50:0/407916881 conn(0x7fe4540b4560 msgr2=0x7fe45406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 -- 172.21.15.50:0/407916881 shutdown_connections 2024-09-21T05:59:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.259+0000 7fe458c00640 1 -- 172.21.15.50:0/407916881 wait complete. 2024-09-21T05:59:52.285 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-21T05:59:52.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.443+0000 7f23e5000640 1 -- 172.21.15.50:0/941717022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23e0074b20 msgr2=0x7f23e015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.443+0000 7f23e5000640 1 --2- 172.21.15.50:0/941717022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23e0074b20 0x7f23e015cf90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009bc0 tx=0x7f23d801cb10 comp rx=0 tx=0).stop 2024-09-21T05:59:52.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.443+0000 7f23e5000640 1 -- 172.21.15.50:0/941717022 shutdown_connections 2024-09-21T05:59:52.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.443+0000 7f23e5000640 1 --2- 172.21.15.50:0/941717022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23e0074b20 0x7f23e015cf90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.443+0000 7f23e5000640 1 -- 172.21.15.50:0/941717022 >> 172.21.15.50:0/941717022 conn(0x7f23e00708c0 msgr2=0x7f23e0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:52.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.443+0000 7f23e5000640 1 -- 172.21.15.50:0/941717022 shutdown_connections 2024-09-21T05:59:52.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.443+0000 7f23e5000640 1 -- 172.21.15.50:0/941717022 wait complete. 2024-09-21T05:59:52.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.443+0000 7f23e5000640 1 Processor -- start 2024-09-21T05:59:52.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.443+0000 7f23e5000640 1 -- start start 2024-09-21T05:59:52.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23e5000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f23e00b5a80 0x7f23e0150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:52.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23e5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23e0157030 0x7f23e013ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:52.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23e5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23e01530e0 0x7f23e01f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:52.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23e5000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f23e016b4e0 con 0x7f23e0157030 2024-09-21T05:59:52.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23e5000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f23e016b360 con 0x7f23e01530e0 2024-09-21T05:59:52.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23e5000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f23e016b660 con 0x7f23e00b5a80 2024-09-21T05:59:52.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23d5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23e0157030 0x7f23e013ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:52.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23d5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23e0157030 0x7f23e013ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60106/0 (socket says 172.21.15.50:60106) 2024-09-21T05:59:52.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23d5400640 1 -- 172.21.15.50:0/3201420007 learned_addr learned my addr 172.21.15.50:0/3201420007 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:52.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23dde00640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f23e00b5a80 0x7f23e0150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:52.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23de800640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23e01530e0 0x7f23e01f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:52.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23d5400640 1 -- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f23e00b5a80 msgr2=0x7f23e0150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23d5400640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f23e00b5a80 0x7f23e0150bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23d5400640 1 -- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23e01530e0 msgr2=0x7f23e01f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23d5400640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23e01530e0 0x7f23e01f75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23d5400640 1 -- 172.21.15.50:0/3201420007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e01516e0 con 0x7f23e0157030 2024-09-21T05:59:52.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23d5400640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23e0157030 0x7f23e013ae70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f23c800b440 tx=0x7f23c800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:52.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23d6a00640 1 -- 172.21.15.50:0/3201420007 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8015020 con 0x7f23e0157030 2024-09-21T05:59:52.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23d6a00640 1 -- 172.21.15.50:0/3201420007 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23c8011980 con 0x7f23e0157030 2024-09-21T05:59:52.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23d6a00640 1 -- 172.21.15.50:0/3201420007 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8011dd0 con 0x7f23e0157030 2024-09-21T05:59:52.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23e5000640 1 -- 172.21.15.50:0/3201420007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e01fae30 con 0x7f23e0157030 2024-09-21T05:59:52.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23e5000640 1 -- 172.21.15.50:0/3201420007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e01f7080 con 0x7f23e0157030 2024-09-21T05:59:52.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.447+0000 7f23e5000640 1 -- 172.21.15.50:0/3201420007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e015cf90 con 0x7f23e0157030 2024-09-21T05:59:52.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.451+0000 7f23d6a00640 1 -- 172.21.15.50:0/3201420007 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f23c8018020 con 0x7f23e0157030 2024-09-21T05:59:52.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.451+0000 7f23d6a00640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f23bc073560 0x7f23bc075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:52.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.451+0000 7f23d6a00640 1 -- 172.21.15.50:0/3201420007 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f23c80945e0 con 0x7f23e0157030 2024-09-21T05:59:52.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.451+0000 7f23dde00640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f23bc073560 0x7f23bc075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:52.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.451+0000 7f23dde00640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f23bc073560 0x7f23bc075a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f23d0007fd0 tx=0x7f23d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:52.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.455+0000 7f23d6a00640 1 -- 172.21.15.50:0/3201420007 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f23c801f030 con 0x7f23e0157030 2024-09-21T05:59:52.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.631+0000 7f23e5000640 1 -- 172.21.15.50:0/3201420007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f23e01518b0 con 0x7f23e0157030 2024-09-21T05:59:52.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.631+0000 7f23d6a00640 1 -- 172.21.15.50:0/3201420007 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v35) ==== 116+0+2 (secure 0 0 0) 0x7f23c800d040 con 0x7f23e0157030 2024-09-21T05:59:52.637 INFO:teuthology.orchestra.run.smithi050.stdout:4 2024-09-21T05:59:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.635+0000 7f23e5000640 1 -- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f23bc073560 msgr2=0x7f23bc075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.635+0000 7f23e5000640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f23bc073560 0x7f23bc075a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f23d0007fd0 tx=0x7f23d0009040 comp rx=0 tx=0).stop 2024-09-21T05:59:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.635+0000 7f23e5000640 1 -- 172.21.15.50:0/3201420007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23e0157030 msgr2=0x7f23e013ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.635+0000 7f23e5000640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23e0157030 0x7f23e013ae70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f23c800b440 tx=0x7f23c800b900 comp rx=0 tx=0).stop 2024-09-21T05:59:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.635+0000 7f23e5000640 1 -- 172.21.15.50:0/3201420007 shutdown_connections 2024-09-21T05:59:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.635+0000 7f23e5000640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23e01530e0 0x7f23e01f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.635+0000 7f23e5000640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f23bc073560 0x7f23bc075a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.635+0000 7f23e5000640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23e0157030 0x7f23e013ae70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.635+0000 7f23e5000640 1 --2- 172.21.15.50:0/3201420007 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f23e00b5a80 0x7f23e0150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.635+0000 7f23e5000640 1 -- 172.21.15.50:0/3201420007 >> 172.21.15.50:0/3201420007 conn(0x7f23e00708c0 msgr2=0x7f23e015eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.635+0000 7f23e5000640 1 -- 172.21.15.50:0/3201420007 shutdown_connections 2024-09-21T05:59:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.635+0000 7f23e5000640 1 -- 172.21.15.50:0/3201420007 wait complete. 2024-09-21T05:59:52.659 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T05:59:52.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.819+0000 7f4331e00640 1 -- 172.21.15.50:0/4172059433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4310005160 msgr2=0x7f431000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.819+0000 7f4331e00640 1 --2- 172.21.15.50:0/4172059433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4310005160 0x7f431000b9d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f432c0b2fd0 tx=0x7f432401d3f0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.819+0000 7f432b400640 1 -- 172.21.15.50:0/4172059433 reap_dead start 2024-09-21T05:59:52.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.819+0000 7f4331e00640 1 -- 172.21.15.50:0/4172059433 shutdown_connections 2024-09-21T05:59:52.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.819+0000 7f4331e00640 1 -- 172.21.15.50:0/4172059433 >> 172.21.15.50:0/4172059433 conn(0x7f432c0b2040 msgr2=0x7f432c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:52.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.819+0000 7f4331e00640 1 -- 172.21.15.50:0/4172059433 shutdown_connections 2024-09-21T05:59:52.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.819+0000 7f4331e00640 1 -- 172.21.15.50:0/4172059433 wait complete. 2024-09-21T05:59:52.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4331e00640 1 Processor -- start 2024-09-21T05:59:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4331e00640 1 -- start start 2024-09-21T05:59:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4331e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4310005160 0x7f432c154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4331e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432c13d420 0x7f432c1e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4331e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c1e4de0 0x7f432c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4331e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0b3a10 con 0x7f432c1e4de0 2024-09-21T05:59:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4331e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0b3890 con 0x7f4310005160 2024-09-21T05:59:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4331e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f432c0cf280 con 0x7f432c13d420 2024-09-21T05:59:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f432b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c1e4de0 0x7f432c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f432b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c1e4de0 0x7f432c0ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60128/0 (socket says 172.21.15.50:60128) 2024-09-21T05:59:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f432b400640 1 -- 172.21.15.50:0/142531400 learned_addr learned my addr 172.21.15.50:0/142531400 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4323e00640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432c13d420 0x7f432c1e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f432aa00640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4310005160 0x7f432c154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f432b400640 1 -- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432c13d420 msgr2=0x7f432c1e48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f432b400640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432c13d420 0x7f432c1e48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f432b400640 1 -- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4310005160 msgr2=0x7f432c154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f432b400640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4310005160 0x7f432c154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f432b400640 1 -- 172.21.15.50:0/142531400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c13c660 con 0x7f432c1e4de0 2024-09-21T05:59:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f432b400640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c1e4de0 0x7f432c0ae8e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f432401d220 tx=0x7f4324005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4323400640 1 -- 172.21.15.50:0/142531400 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324009a50 con 0x7f432c1e4de0 2024-09-21T05:59:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4323400640 1 -- 172.21.15.50:0/142531400 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43240bb070 con 0x7f432c1e4de0 2024-09-21T05:59:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4323400640 1 -- 172.21.15.50:0/142531400 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324005590 con 0x7f432c1e4de0 2024-09-21T05:59:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4331e00640 1 -- 172.21.15.50:0/142531400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c13c940 con 0x7f432c1e4de0 2024-09-21T05:59:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.823+0000 7f4331e00640 1 -- 172.21.15.50:0/142531400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c1e54f0 con 0x7f432c1e4de0 2024-09-21T05:59:52.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.827+0000 7f4331e00640 1 -- 172.21.15.50:0/142531400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432c1e5260 con 0x7f432c1e4de0 2024-09-21T05:59:52.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.827+0000 7f4323400640 1 -- 172.21.15.50:0/142531400 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f4324005730 con 0x7f432c1e4de0 2024-09-21T05:59:52.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.827+0000 7f4323400640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4308073560 0x7f4308075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:52.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.827+0000 7f4323400640 1 -- 172.21.15.50:0/142531400 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f432412fe80 con 0x7f432c1e4de0 2024-09-21T05:59:52.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.827+0000 7f432aa00640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4308073560 0x7f4308075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:52.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.827+0000 7f432aa00640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4308073560 0x7f4308075a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f431c009d80 tx=0x7f431c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:52.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:52.831+0000 7f4323400640 1 -- 172.21.15.50:0/142531400 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f43240c7070 con 0x7f432c1e4de0 2024-09-21T05:59:53.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.011+0000 7f4331e00640 1 -- 172.21.15.50:0/142531400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f432c0cc320 con 0x7f432c1e4de0 2024-09-21T05:59:53.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4323400640 1 -- 172.21.15.50:0/142531400 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v36)=0 v36) ==== 131+0+0 (secure 0 0 0) 0x7f4324103020 con 0x7f432c1e4de0 2024-09-21T05:59:53.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4331e00640 1 -- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4308073560 msgr2=0x7f4308075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:53.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4331e00640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4308073560 0x7f4308075a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f431c009d80 tx=0x7f431c0091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4331e00640 1 -- 172.21.15.50:0/142531400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c1e4de0 msgr2=0x7f432c0ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:53.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4331e00640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c1e4de0 0x7f432c0ae8e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f432401d220 tx=0x7f4324005c00 comp rx=0 tx=0).stop 2024-09-21T05:59:53.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4331e00640 1 -- 172.21.15.50:0/142531400 shutdown_connections 2024-09-21T05:59:53.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4331e00640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4308073560 0x7f4308075a70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4331e00640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c1e4de0 0x7f432c0ae8e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4331e00640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432c13d420 0x7f432c1e48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4331e00640 1 --2- 172.21.15.50:0/142531400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4310005160 0x7f432c154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4331e00640 1 -- 172.21.15.50:0/142531400 >> 172.21.15.50:0/142531400 conn(0x7f432c0b2040 msgr2=0x7f432c13db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:53.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4331e00640 1 -- 172.21.15.50:0/142531400 shutdown_connections 2024-09-21T05:59:53.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.015+0000 7f4331e00640 1 -- 172.21.15.50:0/142531400 wait complete. 2024-09-21T05:59:53.035 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:53.033+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:53.035 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:53.033+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:53.036 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:53.033+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:53.036 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:53.033+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:53.036 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:53.033+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:53.043 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-21T05:59:53.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.203+0000 7f54eba00640 1 -- 172.21.15.50:0/1440987646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54c8004810 msgr2=0x7f54c8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:53.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.203+0000 7f54eba00640 1 --2- 172.21.15.50:0/1440987646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54c8004810 0x7f54c8004c20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f54e4071fd0 tx=0x7f54cc01cef0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.203+0000 7f54e9a00640 1 -- 172.21.15.50:0/1440987646 reap_dead start 2024-09-21T05:59:53.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.203+0000 7f54eba00640 1 -- 172.21.15.50:0/1440987646 shutdown_connections 2024-09-21T05:59:53.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.203+0000 7f54eba00640 1 -- 172.21.15.50:0/1440987646 >> 172.21.15.50:0/1440987646 conn(0x7f54e40b2040 msgr2=0x7f54e40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:53.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.203+0000 7f54eba00640 1 -- 172.21.15.50:0/1440987646 shutdown_connections 2024-09-21T05:59:53.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.203+0000 7f54eba00640 1 -- 172.21.15.50:0/1440987646 wait complete. 2024-09-21T05:59:53.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.203+0000 7f54eba00640 1 Processor -- start 2024-09-21T05:59:53.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.203+0000 7f54eba00640 1 -- start start 2024-09-21T05:59:53.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54eba00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c8004810 0x7f54e415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54eba00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54e41630a0 0x7f54e414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54eba00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54e4156d40 0x7f54e4157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54eba00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f54e4166910 con 0x7f54e4156d40 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54eba00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f54e4166790 con 0x7f54c8004810 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54eba00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f54e4166a90 con 0x7f54e41630a0 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54e4156d40 0x7f54e4157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54e4156d40 0x7f54e4157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60140/0 (socket says 172.21.15.50:60140) 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e9a00640 1 -- 172.21.15.50:0/2053891957 learned_addr learned my addr 172.21.15.50:0/2053891957 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e9000640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c8004810 0x7f54e415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54dbe00640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54e41630a0 0x7f54e414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e9a00640 1 -- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54e41630a0 msgr2=0x7f54e414f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e9a00640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54e41630a0 0x7f54e414f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e9a00640 1 -- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c8004810 msgr2=0x7f54e415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:53.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e9a00640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c8004810 0x7f54e415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e9a00640 1 -- 172.21.15.50:0/2053891957 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e4157690 con 0x7f54e4156d40 2024-09-21T05:59:53.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e9a00640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54e4156d40 0x7f54e4157150 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f54e4156170 tx=0x7f54dc010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:53.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e1600640 1 -- 172.21.15.50:0/2053891957 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc003980 con 0x7f54e4156d40 2024-09-21T05:59:53.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e1600640 1 -- 172.21.15.50:0/2053891957 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54dc015070 con 0x7f54e4156d40 2024-09-21T05:59:53.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54eba00640 1 -- 172.21.15.50:0/2053891957 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e415f760 con 0x7f54e4156d40 2024-09-21T05:59:53.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54e1600640 1 -- 172.21.15.50:0/2053891957 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc004030 con 0x7f54e4156d40 2024-09-21T05:59:53.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54eba00640 1 -- 172.21.15.50:0/2053891957 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e41625c0 con 0x7f54e4156d40 2024-09-21T05:59:53.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.207+0000 7f54eba00640 1 -- 172.21.15.50:0/2053891957 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e415f4d0 con 0x7f54e4156d40 2024-09-21T05:59:53.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.211+0000 7f54e1600640 1 -- 172.21.15.50:0/2053891957 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7f54dc0041d0 con 0x7f54e4156d40 2024-09-21T05:59:53.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.211+0000 7f54e1600640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54c0073560 0x7f54c0075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:53.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.211+0000 7f54e1600640 1 -- 172.21.15.50:0/2053891957 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f54dc095420 con 0x7f54e4156d40 2024-09-21T05:59:53.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.211+0000 7f54e9000640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54c0073560 0x7f54c0075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:53.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.211+0000 7f54e9000640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54c0073560 0x7f54c0075a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009a40 tx=0x7f54d40091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:53.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.215+0000 7f54e1600640 1 -- 172.21.15.50:0/2053891957 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f54dc05f7d0 con 0x7f54e4156d40 2024-09-21T05:59:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.419+0000 7f54eba00640 1 -- 172.21.15.50:0/2053891957 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f54e406b540 con 0x7f54c0073560 2024-09-21T05:59:53.492 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:53.489+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:53.492 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:53.489+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:53.492 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:53.489+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:53.492 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:53.489+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:53.492 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T05:59:53.489+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:59:53.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.531+0000 7f54e1600640 1 -- 172.21.15.50:0/2053891957 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f54d4012b10 con 0x7f54c0073560 2024-09-21T05:59:53.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.535+0000 7f54eba00640 1 -- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54c0073560 msgr2=0x7f54c0075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:53.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.535+0000 7f54eba00640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54c0073560 0x7f54c0075a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009a40 tx=0x7f54d40091e0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.535+0000 7f54eba00640 1 -- 172.21.15.50:0/2053891957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54e4156d40 msgr2=0x7f54e4157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:53.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.535+0000 7f54eba00640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54e4156d40 0x7f54e4157150 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f54e4156170 tx=0x7f54dc010f40 comp rx=0 tx=0).stop 2024-09-21T05:59:53.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.535+0000 7f54eba00640 1 -- 172.21.15.50:0/2053891957 shutdown_connections 2024-09-21T05:59:53.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.535+0000 7f54eba00640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54c0073560 0x7f54c0075a70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.535+0000 7f54eba00640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54e4156d40 0x7f54e4157150 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.535+0000 7f54eba00640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54e41630a0 0x7f54e414f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.535+0000 7f54eba00640 1 --2- 172.21.15.50:0/2053891957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c8004810 0x7f54e415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.535+0000 7f54eba00640 1 -- 172.21.15.50:0/2053891957 >> 172.21.15.50:0/2053891957 conn(0x7f54e40b2040 msgr2=0x7f54e4151c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:53.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.535+0000 7f54eba00640 1 -- 172.21.15.50:0/2053891957 shutdown_connections 2024-09-21T05:59:53.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.535+0000 7f54eba00640 1 -- 172.21.15.50:0/2053891957 wait complete. 2024-09-21T05:59:53.558 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T05:59:53.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.715+0000 7fc533200640 1 -- 172.21.15.50:0/3057186434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc52c0b5c70 msgr2=0x7fc52c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:53.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.715+0000 7fc533200640 1 --2- 172.21.15.50:0/3057186434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc52c0b5c70 0x7fc52c15bb60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc52c074b20 tx=0x7fc51c01c760 comp rx=0 tx=0).stop 2024-09-21T05:59:53.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.715+0000 7fc533200640 1 -- 172.21.15.50:0/3057186434 shutdown_connections 2024-09-21T05:59:53.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.715+0000 7fc533200640 1 --2- 172.21.15.50:0/3057186434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc52c0b5c70 0x7fc52c15bb60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.715+0000 7fc533200640 1 -- 172.21.15.50:0/3057186434 >> 172.21.15.50:0/3057186434 conn(0x7fc52c0b2040 msgr2=0x7fc52c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:53.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.715+0000 7fc533200640 1 -- 172.21.15.50:0/3057186434 shutdown_connections 2024-09-21T05:59:53.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.715+0000 7fc533200640 1 -- 172.21.15.50:0/3057186434 wait complete. 2024-09-21T05:59:53.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc533200640 1 Processor -- start 2024-09-21T05:59:53.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc533200640 1 -- start start 2024-09-21T05:59:53.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc533200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc52c0b4de0 0x7fc52c0acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:53.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc533200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc52c0b5c70 0x7fc52c0aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:53.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc533200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc52c1527b0 0x7fc52c0af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:53.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc533200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c169b30 con 0x7fc52c0b5c70 2024-09-21T05:59:53.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc533200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1699b0 con 0x7fc52c1527b0 2024-09-21T05:59:53.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc533200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc52c169cb0 con 0x7fc52c0b4de0 2024-09-21T05:59:53.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc523e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc52c0b5c70 0x7fc52c0aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:53.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc523e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc52c0b5c70 0x7fc52c0aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60154/0 (socket says 172.21.15.50:60154) 2024-09-21T05:59:53.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc523e00640 1 -- 172.21.15.50:0/442464783 learned_addr learned my addr 172.21.15.50:0/442464783 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T05:59:53.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc531200640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc52c1527b0 0x7fc52c0af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:53.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc52be00640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc52c0b4de0 0x7fc52c0acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:53.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc523e00640 1 -- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc52c0b4de0 msgr2=0x7fc52c0acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:53.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc523e00640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc52c0b4de0 0x7fc52c0acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc523e00640 1 -- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc52c1527b0 msgr2=0x7fc52c0af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:53.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc523e00640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc52c1527b0 0x7fc52c0af3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc523e00640 1 -- 172.21.15.50:0/442464783 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c0ad520 con 0x7fc52c0b5c70 2024-09-21T05:59:53.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc523e00640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc52c0b5c70 0x7fc52c0aee80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc51400c430 tx=0x7fc51400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:53.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc528c00640 1 -- 172.21.15.50:0/442464783 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514011790 con 0x7fc52c0b5c70 2024-09-21T05:59:53.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc528c00640 1 -- 172.21.15.50:0/442464783 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc514017070 con 0x7fc52c0b5c70 2024-09-21T05:59:53.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.719+0000 7fc528c00640 1 -- 172.21.15.50:0/442464783 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514011cc0 con 0x7fc52c0b5c70 2024-09-21T05:59:53.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.723+0000 7fc533200640 1 -- 172.21.15.50:0/442464783 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52c14c2e0 con 0x7fc52c0b5c70 2024-09-21T05:59:53.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.723+0000 7fc533200640 1 -- 172.21.15.50:0/442464783 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52c14c8a0 con 0x7fc52c0b5c70 2024-09-21T05:59:53.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.723+0000 7fc533200640 1 -- 172.21.15.50:0/442464783 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52c15bb60 con 0x7fc52c0b5c70 2024-09-21T05:59:53.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.723+0000 7fc528c00640 1 -- 172.21.15.50:0/442464783 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) ==== 97540+0+0 (secure 0 0 0) 0x7fc51401c460 con 0x7fc52c0b5c70 2024-09-21T05:59:53.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.723+0000 7fc528c00640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc508073560 0x7fc508075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:53.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.723+0000 7fc528c00640 1 -- 172.21.15.50:0/442464783 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fc5140945a0 con 0x7fc52c0b5c70 2024-09-21T05:59:53.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.723+0000 7fc52be00640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc508073560 0x7fc508075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:53.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.727+0000 7fc52be00640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc508073560 0x7fc508075a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0af000 tx=0x7fc51c01afa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:53.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.727+0000 7fc528c00640 1 -- 172.21.15.50:0/442464783 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc51405e990 con 0x7fc52c0b5c70 2024-09-21T05:59:53.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.935+0000 7fc533200640 1 -- 172.21.15.50:0/442464783 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fc52c14c090 con 0x7fc508073560 2024-09-21T05:59:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.939+0000 7fc528c00640 1 -- 172.21.15.50:0/442464783 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fc52c14c090 con 0x7fc508073560 2024-09-21T05:59:53.943 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78 2024-09-21T05:59:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.943+0000 7fc533200640 1 -- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc508073560 msgr2=0x7fc508075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.943+0000 7fc533200640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc508073560 0x7fc508075a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0af000 tx=0x7fc51c01afa0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.943+0000 7fc533200640 1 -- 172.21.15.50:0/442464783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc52c0b5c70 msgr2=0x7fc52c0aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.943+0000 7fc533200640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc52c0b5c70 0x7fc52c0aee80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc51400c430 tx=0x7fc51400c8f0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.943+0000 7fc533200640 1 -- 172.21.15.50:0/442464783 shutdown_connections 2024-09-21T05:59:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.943+0000 7fc533200640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc52c1527b0 0x7fc52c0af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.943+0000 7fc533200640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc508073560 0x7fc508075a70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.943+0000 7fc533200640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc52c0b5c70 0x7fc52c0aee80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.943+0000 7fc533200640 1 --2- 172.21.15.50:0/442464783 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc52c0b4de0 0x7fc52c0acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.943+0000 7fc533200640 1 -- 172.21.15.50:0/442464783 >> 172.21.15.50:0/442464783 conn(0x7fc52c0b2040 msgr2=0x7fc52c0b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.943+0000 7fc533200640 1 -- 172.21.15.50:0/442464783 shutdown_connections 2024-09-21T05:59:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T05:59:53.943+0000 7fc533200640 1 -- 172.21.15.50:0/442464783 wait complete. 2024-09-21T05:59:53.966 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78 2024-09-21T05:59:53.966 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T05:59:53.966 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:59:53.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:59:53.975 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:00:04.175 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:00:04.176 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:00:04.176 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.1438 s, 106 MB/s 2024-09-21T06:00:04.177 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:00:04.177 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:00:04.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:00:04.186 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:00:14.054 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:00:14.057 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:00:14.057 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.81669 s, 109 MB/s 2024-09-21T06:00:14.058 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:00:14.058 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:00:14.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:00:14.067 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:00:17.102 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:00:17.106 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:00:23.864 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:00:23.864 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:00:23.864 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.7464 s, 110 MB/s 2024-09-21T06:00:23.865 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:00:23.865 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:00:23.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:00:23.874 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:00:33.621 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:00:33.622 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:00:33.622 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.69547 s, 111 MB/s 2024-09-21T06:00:33.623 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:00:33.623 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:00:33.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:00:33.632 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:00:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:00:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:00:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.58904 s, 112 MB/s 2024-09-21T06:00:43.272 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:00:43.272 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:00:43.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:00:43.281 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:00:47.165 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:00:47.168 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:00:53.124 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:00:53.124 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:00:53.124 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.78629 s, 110 MB/s 2024-09-21T06:00:53.125 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:00:53.125 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:00:53.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:00:53.134 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:01:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:01:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:01:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.97126 s, 108 MB/s 2024-09-21T06:01:03.162 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:01:03.162 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:01:03.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:01:03.172 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:01:13.151 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:01:13.152 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:01:13.152 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.92603 s, 108 MB/s 2024-09-21T06:01:13.153 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:01:13.153 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:01:13.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:01:13.162 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:01:17.226 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:01:17.230 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:01:23.250 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:01:23.250 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:01:23.250 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.0333 s, 107 MB/s 2024-09-21T06:01:23.251 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:01:23.251 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:01:23.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:01:23.260 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:01:33.109 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:01:33.109 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:01:33.109 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.79245 s, 110 MB/s 2024-09-21T06:01:33.114 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-21T06:01:33.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.277+0000 7f34ac200640 1 -- 172.21.15.50:0/651475009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a40b5c70 msgr2=0x7f34a415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:33.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.277+0000 7f34ac200640 1 --2- 172.21.15.50:0/651475009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a40b5c70 0x7f34a415bb60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f34a4074b20 tx=0x7f349801c820 comp rx=0 tx=0).stop 2024-09-21T06:01:33.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.277+0000 7f34ac200640 1 -- 172.21.15.50:0/651475009 shutdown_connections 2024-09-21T06:01:33.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.277+0000 7f34ac200640 1 --2- 172.21.15.50:0/651475009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a40b5c70 0x7f34a415bb60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.277+0000 7f34ac200640 1 -- 172.21.15.50:0/651475009 >> 172.21.15.50:0/651475009 conn(0x7f34a40b2040 msgr2=0x7f34a40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:33.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.277+0000 7f34ac200640 1 -- 172.21.15.50:0/651475009 shutdown_connections 2024-09-21T06:01:33.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.277+0000 7f34ac200640 1 -- 172.21.15.50:0/651475009 wait complete. 2024-09-21T06:01:33.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.277+0000 7f34ac200640 1 Processor -- start 2024-09-21T06:01:33.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.277+0000 7f34ac200640 1 -- start start 2024-09-21T06:01:33.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34ac200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a40b4de0 0x7f34a4147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:33.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34ac200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a40b5c70 0x7f34a4151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:33.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34ac200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34a41527b0 0x7f34a4151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:33.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34ac200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4169b30 con 0x7f34a40b4de0 2024-09-21T06:01:33.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34ac200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f34a41699b0 con 0x7f34a40b5c70 2024-09-21T06:01:33.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34ac200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f34a4169cb0 con 0x7f34a41527b0 2024-09-21T06:01:33.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a9800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a40b4de0 0x7f34a4147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:33.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a9800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a40b4de0 0x7f34a4147af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40468/0 (socket says 172.21.15.50:40468) 2024-09-21T06:01:33.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a9800640 1 -- 172.21.15.50:0/2742356542 learned_addr learned my addr 172.21.15.50:0/2742356542 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:33.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34aa200640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34a41527b0 0x7f34a4151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:33.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a8e00640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a40b5c70 0x7f34a4151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:33.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a9800640 1 -- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34a41527b0 msgr2=0x7f34a4151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:33.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a9800640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34a41527b0 0x7f34a4151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a9800640 1 -- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a40b5c70 msgr2=0x7f34a4151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:33.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a9800640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a40b5c70 0x7f34a4151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a9800640 1 -- 172.21.15.50:0/2742356542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a40cf270 con 0x7f34a40b4de0 2024-09-21T06:01:33.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a9800640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a40b4de0 0x7f34a4147af0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3498019430 tx=0x7f34980040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:33.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a1600640 1 -- 172.21.15.50:0/2742356542 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980bd070 con 0x7f34a40b4de0 2024-09-21T06:01:33.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a1600640 1 -- 172.21.15.50:0/2742356542 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3498003600 con 0x7f34a40b4de0 2024-09-21T06:01:33.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34a1600640 1 -- 172.21.15.50:0/2742356542 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980b2040 con 0x7f34a40b4de0 2024-09-21T06:01:33.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34ac200640 1 -- 172.21.15.50:0/2742356542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a40cf550 con 0x7f34a40b4de0 2024-09-21T06:01:33.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34ac200640 1 -- 172.21.15.50:0/2742356542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a40cfb10 con 0x7f34a40b4de0 2024-09-21T06:01:33.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.281+0000 7f34ac200640 1 -- 172.21.15.50:0/2742356542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a415bb60 con 0x7f34a40b4de0 2024-09-21T06:01:33.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.285+0000 7f34a1600640 1 -- 172.21.15.50:0/2742356542 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 97540+0+0 (secure 0 0 0) 0x7f3498003a30 con 0x7f34a40b4de0 2024-09-21T06:01:33.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.285+0000 7f34a1600640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3484073560 0x7f3484075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:33.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.285+0000 7f34a1600640 1 -- 172.21.15.50:0/2742356542 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f349812e890 con 0x7f34a40b4de0 2024-09-21T06:01:33.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.285+0000 7f34a8e00640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3484073560 0x7f3484075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:33.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.285+0000 7f34a8e00640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3484073560 0x7f3484075a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f34a40cfe10 tx=0x7f3490009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:33.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.289+0000 7f34a1600640 1 -- 172.21.15.50:0/2742356542 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f34980164d0 con 0x7f34a40b4de0 2024-09-21T06:01:33.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.497+0000 7f34ac200640 1 -- 172.21.15.50:0/2742356542 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f34a4159530 con 0x7f3484073560 2024-09-21T06:01:33.503 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:33.500+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:33.503 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:33.500+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:33.503 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:33.500+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:33.503 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:33.500+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:33.503 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:33.500+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:33.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34a1600640 1 -- 172.21.15.50:0/2742356542 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3490001d20 con 0x7f3484073560 2024-09-21T06:01:33.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34ac200640 1 -- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3484073560 msgr2=0x7f3484075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:33.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34ac200640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3484073560 0x7f3484075a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f34a40cfe10 tx=0x7f3490009040 comp rx=0 tx=0).stop 2024-09-21T06:01:33.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34ac200640 1 -- 172.21.15.50:0/2742356542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a40b4de0 msgr2=0x7f34a4147af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:33.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34ac200640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a40b4de0 0x7f34a4147af0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3498019430 tx=0x7f34980040c0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34ac200640 1 -- 172.21.15.50:0/2742356542 shutdown_connections 2024-09-21T06:01:33.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34ac200640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34a41527b0 0x7f34a4151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34ac200640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a40b5c70 0x7f34a4151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34ac200640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3484073560 0x7f3484075a70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34ac200640 1 --2- 172.21.15.50:0/2742356542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a40b4de0 0x7f34a4147af0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34ac200640 1 -- 172.21.15.50:0/2742356542 >> 172.21.15.50:0/2742356542 conn(0x7f34a40b2040 msgr2=0x7f34a415af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:33.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34ac200640 1 -- 172.21.15.50:0/2742356542 shutdown_connections 2024-09-21T06:01:33.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.529+0000 7f34ac200640 1 -- 172.21.15.50:0/2742356542 wait complete. 2024-09-21T06:01:33.551 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:01:33.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.709+0000 7f7efa600640 1 -- 172.21.15.50:0/1653711289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ef415b2d0 msgr2=0x7f7ef415b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:33.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.709+0000 7f7efa600640 1 --2- 172.21.15.50:0/1653711289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ef415b2d0 0x7f7ef415b6e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40b3040 tx=0x7f7edc01c8c0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 -- 172.21.15.50:0/1653711289 shutdown_connections 2024-09-21T06:01:33.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 --2- 172.21.15.50:0/1653711289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ef415b2d0 0x7f7ef415b6e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 --2- 172.21.15.50:0/1653711289 >> v2:172.21.15.88:3300/0 conn(0x7f7ef415e8c0 0x7f7ef415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 -- 172.21.15.50:0/1653711289 >> 172.21.15.50:0/1653711289 conn(0x7f7ef40b2040 msgr2=0x7f7ef40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:33.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 -- 172.21.15.50:0/1653711289 shutdown_connections 2024-09-21T06:01:33.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 -- 172.21.15.50:0/1653711289 wait complete. 2024-09-21T06:01:33.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 Processor -- start 2024-09-21T06:01:33.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 -- start start 2024-09-21T06:01:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ef415b2d0 0x7f7ef413c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ef415bc20 0x7f7ef413cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ef415e8c0 0x7f7ef41fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef415e040 con 0x7f7ef415b2d0 2024-09-21T06:01:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef415dec0 con 0x7f7ef415e8c0 2024-09-21T06:01:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.713+0000 7f7efa600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef415e1c0 con 0x7f7ef415bc20 2024-09-21T06:01:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7ef3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ef415b2d0 0x7f7ef413c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7ef3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ef415b2d0 0x7f7ef413c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40474/0 (socket says 172.21.15.50:40474) 2024-09-21T06:01:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7ef3400640 1 -- 172.21.15.50:0/2309817833 learned_addr learned my addr 172.21.15.50:0/2309817833 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7ef3e00640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ef415e8c0 0x7f7ef41fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:33.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7eebe00640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ef415bc20 0x7f7ef413cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:33.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7ef3400640 1 -- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ef415bc20 msgr2=0x7f7ef413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:33.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7ef3400640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ef415bc20 0x7f7ef413cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7ef3400640 1 -- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ef415e8c0 msgr2=0x7f7ef41fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:33.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7ef3400640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ef415e8c0 0x7f7ef41fbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7ef3400640 1 -- 172.21.15.50:0/2309817833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef4164ae0 con 0x7f7ef415b2d0 2024-09-21T06:01:33.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7ef3e00640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ef415e8c0 0x7f7ef41fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:01:33.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7ef3400640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ef415b2d0 0x7f7ef413c6b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7ee400a870 tx=0x7f7ee400ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:33.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7eeb400640 1 -- 172.21.15.50:0/2309817833 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4004020 con 0x7f7ef415b2d0 2024-09-21T06:01:33.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7eeb400640 1 -- 172.21.15.50:0/2309817833 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ee40041c0 con 0x7f7ef415b2d0 2024-09-21T06:01:33.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7eeb400640 1 -- 172.21.15.50:0/2309817833 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee40079b0 con 0x7f7ef415b2d0 2024-09-21T06:01:33.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7efa600640 1 -- 172.21.15.50:0/2309817833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef4164dc0 con 0x7f7ef415b2d0 2024-09-21T06:01:33.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7efa600640 1 -- 172.21.15.50:0/2309817833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef4165380 con 0x7f7ef415b2d0 2024-09-21T06:01:33.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7efa600640 1 -- 172.21.15.50:0/2309817833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef415b6e0 con 0x7f7ef415b2d0 2024-09-21T06:01:33.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.717+0000 7f7eeb400640 1 -- 172.21.15.50:0/2309817833 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 97540+0+0 (secure 0 0 0) 0x7f7ee401b020 con 0x7f7ef415b2d0 2024-09-21T06:01:33.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.721+0000 7f7eeb400640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ed0073590 0x7f7ed0075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:33.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.721+0000 7f7eeb400640 1 -- 172.21.15.50:0/2309817833 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f7ee4093fc0 con 0x7f7ef415b2d0 2024-09-21T06:01:33.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.721+0000 7f7eebe00640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ed0073590 0x7f7ed0075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:33.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.721+0000 7f7eebe00640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ed0073590 0x7f7ed0075aa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0af000 tx=0x7f7edc01b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:33.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.725+0000 7f7eeb400640 1 -- 172.21.15.50:0/2309817833 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7ee405eb60 con 0x7f7ef415b2d0 2024-09-21T06:01:33.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.929+0000 7f7efa600640 1 -- 172.21.15.50:0/2309817833 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f7ef40753d0 con 0x7f7ed0073590 2024-09-21T06:01:33.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.937+0000 7f7eeb400640 1 -- 172.21.15.50:0/2309817833 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f7ef40753d0 con 0x7f7ed0073590 2024-09-21T06:01:33.939 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78 2024-09-21T06:01:33.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.941+0000 7f7efa600640 1 -- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ed0073590 msgr2=0x7f7ed0075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:33.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.941+0000 7f7efa600640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ed0073590 0x7f7ed0075aa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0af000 tx=0x7f7edc01b100 comp rx=0 tx=0).stop 2024-09-21T06:01:33.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.941+0000 7f7efa600640 1 -- 172.21.15.50:0/2309817833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ef415b2d0 msgr2=0x7f7ef413c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:33.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.941+0000 7f7efa600640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ef415b2d0 0x7f7ef413c6b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7ee400a870 tx=0x7f7ee400ad30 comp rx=0 tx=0).stop 2024-09-21T06:01:33.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.941+0000 7f7efa600640 1 -- 172.21.15.50:0/2309817833 shutdown_connections 2024-09-21T06:01:33.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.941+0000 7f7efa600640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ef415e8c0 0x7f7ef41fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.941+0000 7f7efa600640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ef415bc20 0x7f7ef413cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.941+0000 7f7efa600640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ed0073590 0x7f7ed0075aa0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.941+0000 7f7efa600640 1 --2- 172.21.15.50:0/2309817833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ef415b2d0 0x7f7ef413c6b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:33.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.941+0000 7f7efa600640 1 -- 172.21.15.50:0/2309817833 >> 172.21.15.50:0/2309817833 conn(0x7f7ef40b2040 msgr2=0x7f7ef415aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:33.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.941+0000 7f7efa600640 1 -- 172.21.15.50:0/2309817833 shutdown_connections 2024-09-21T06:01:33.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:33.941+0000 7f7efa600640 1 -- 172.21.15.50:0/2309817833 wait complete. 2024-09-21T06:01:33.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:01:33.961 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78 2024-09-21T06:01:33.961 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:01:33.970 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78 2024-09-21T06:01:33.971 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="10488365056" 2024-09-21T06:01:33.971 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:01:33.971 INFO:tasks.cephfs.test_volumes:r_size = 10488365056 exp_size = 10737418240 2024-09-21T06:01:35.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:01:35.972 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78 2024-09-21T06:01:35.972 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:01:35.983 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/8f95f21a-4ab9-4641-b768-3e7aaca4ad78 2024-09-21T06:01:35.983 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="10737418240" 2024-09-21T06:01:35.983 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:01:35.984 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-21T06:01:35.984 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone20400 2024-09-21T06:01:36.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.205+0000 7fedfa200640 1 -- 172.21.15.50:0/3457896460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedf40cefa0 msgr2=0x7fedf40cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:36.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.205+0000 7fedfa200640 1 --2- 172.21.15.50:0/3457896460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedf40cefa0 0x7fedf40cf3b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7feddc009900 tx=0x7feddc01c8e0 comp rx=0 tx=0).stop 2024-09-21T06:01:36.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.205+0000 7fedfa200640 1 -- 172.21.15.50:0/3457896460 shutdown_connections 2024-09-21T06:01:36.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.205+0000 7fedfa200640 1 --2- 172.21.15.50:0/3457896460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedf40cefa0 0x7fedf40cf3b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:36.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.205+0000 7fedfa200640 1 -- 172.21.15.50:0/3457896460 >> 172.21.15.50:0/3457896460 conn(0x7fedf40b2040 msgr2=0x7fedf40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:36.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.205+0000 7fedfa200640 1 -- 172.21.15.50:0/3457896460 shutdown_connections 2024-09-21T06:01:36.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.205+0000 7fedfa200640 1 -- 172.21.15.50:0/3457896460 wait complete. 2024-09-21T06:01:36.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.205+0000 7fedfa200640 1 Processor -- start 2024-09-21T06:01:36.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.205+0000 7fedfa200640 1 -- start start 2024-09-21T06:01:36.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedfa200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fedf40749a0 0x7fedf413a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:36.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedfa200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf40b3340 0x7fedf41fc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:36.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedfa200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedf40cefa0 0x7fedf40cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:36.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedfa200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fedf416b350 con 0x7fedf40cefa0 2024-09-21T06:01:36.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedfa200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fedf416b1d0 con 0x7fedf40b3340 2024-09-21T06:01:36.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedfa200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fedf416b4d0 con 0x7fedf40749a0 2024-09-21T06:01:36.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedf3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedf40cefa0 0x7fedf40cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:36.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedf3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedf40cefa0 0x7fedf40cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40478/0 (socket says 172.21.15.50:40478) 2024-09-21T06:01:36.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedf3e00640 1 -- 172.21.15.50:0/947652307 learned_addr learned my addr 172.21.15.50:0/947652307 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:36.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedeaa00640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf40b3340 0x7fedf41fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:36.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedf3400640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fedf40749a0 0x7fedf413a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:36.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedf3e00640 1 -- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fedf40749a0 msgr2=0x7fedf413a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:36.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedf3e00640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fedf40749a0 0x7fedf413a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:36.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedf3e00640 1 -- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf40b3340 msgr2=0x7fedf41fc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:36.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedf3e00640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf40b3340 0x7fedf41fc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:36.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedf3e00640 1 -- 172.21.15.50:0/947652307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf41433f0 con 0x7fedf40cefa0 2024-09-21T06:01:36.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedeaa00640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf40b3340 0x7fedf41fc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:01:36.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedf3e00640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedf40cefa0 0x7fedf40cde50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b4c0 tx=0x7fedec00b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:36.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedebe00640 1 -- 172.21.15.50:0/947652307 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec015020 con 0x7fedf40cefa0 2024-09-21T06:01:36.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedebe00640 1 -- 172.21.15.50:0/947652307 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedec011920 con 0x7fedf40cefa0 2024-09-21T06:01:36.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedfa200640 1 -- 172.21.15.50:0/947652307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf41436d0 con 0x7fedf40cefa0 2024-09-21T06:01:36.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedebe00640 1 -- 172.21.15.50:0/947652307 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec011d70 con 0x7fedf40cefa0 2024-09-21T06:01:36.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedfa200640 1 -- 172.21.15.50:0/947652307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf4138920 con 0x7fedf40cefa0 2024-09-21T06:01:36.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.209+0000 7fedfa200640 1 -- 172.21.15.50:0/947652307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf4143910 con 0x7fedf40cefa0 2024-09-21T06:01:36.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.213+0000 7fedebe00640 1 -- 172.21.15.50:0/947652307 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 14) ==== 97591+0+0 (secure 0 0 0) 0x7fedec018020 con 0x7fedf40cefa0 2024-09-21T06:01:36.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.213+0000 7fedebe00640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fedd00735f0 0x7fedd0075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:36.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.213+0000 7fedebe00640 1 -- 172.21.15.50:0/947652307 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fedec094660 con 0x7fedf40cefa0 2024-09-21T06:01:36.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.213+0000 7fedf3400640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fedd00735f0 0x7fedd0075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:36.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.213+0000 7fedf3400640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fedd00735f0 0x7fedd0075b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fede40012f0 tx=0x7fede4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:36.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.217+0000 7fedebe00640 1 -- 172.21.15.50:0/947652307 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fedec01f030 con 0x7fedf40cefa0 2024-09-21T06:01:36.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.425+0000 7fedfa200640 1 -- 172.21.15.50:0/947652307 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone20400", "target": ["mon-mgr", ""]}) -- 0x7fedf41fd580 con 0x7fedd00735f0 2024-09-21T06:01:36.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.497+0000 7fedebe00640 1 -- 172.21.15.50:0/947652307 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fedf41fd580 con 0x7fedd00735f0 2024-09-21T06:01:36.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.501+0000 7fedfa200640 1 -- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fedd00735f0 msgr2=0x7fedd0075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:36.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.501+0000 7fedfa200640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fedd00735f0 0x7fedd0075b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fede40012f0 tx=0x7fede4009040 comp rx=0 tx=0).stop 2024-09-21T06:01:36.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.501+0000 7fedfa200640 1 -- 172.21.15.50:0/947652307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedf40cefa0 msgr2=0x7fedf40cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:36.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.501+0000 7fedfa200640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedf40cefa0 0x7fedf40cde50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b4c0 tx=0x7fedec00b980 comp rx=0 tx=0).stop 2024-09-21T06:01:36.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.501+0000 7fedfa200640 1 -- 172.21.15.50:0/947652307 shutdown_connections 2024-09-21T06:01:36.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.501+0000 7fedfa200640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fedd00735f0 0x7fedd0075b00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:36.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.501+0000 7fedfa200640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedf40cefa0 0x7fedf40cde50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:36.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.501+0000 7fedfa200640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf40b3340 0x7fedf41fc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:36.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.501+0000 7fedfa200640 1 --2- 172.21.15.50:0/947652307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fedf40749a0 0x7fedf413a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:36.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.501+0000 7fedfa200640 1 -- 172.21.15.50:0/947652307 >> 172.21.15.50:0/947652307 conn(0x7fedf40b2040 msgr2=0x7fedf40b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:36.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.501+0000 7fedfa200640 1 -- 172.21.15.50:0/947652307 shutdown_connections 2024-09-21T06:01:36.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.501+0000 7fedfa200640 1 -- 172.21.15.50:0/947652307 wait complete. 2024-09-21T06:01:36.503 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:36.500+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:36.503 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:36.500+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:36.503 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:36.500+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:36.503 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:36.500+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:36.503 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:36.500+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:36.521 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:36.520+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:36.521 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:36.520+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:36.521 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:36.520+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:36.522 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:36.520+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:36.522 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:36.520+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:36.523 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone29661 2024-09-21T06:01:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.693+0000 7f35b0c00640 1 -- 172.21.15.50:0/2928184531 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f35ac06dbc0 msgr2=0x7f35ac06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.693+0000 7f35b0c00640 1 --2- 172.21.15.50:0/2928184531 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f35ac06dbc0 0x7f35ac06dfd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3594009840 tx=0x7f359401c790 comp rx=0 tx=0).stop 2024-09-21T06:01:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.693+0000 7f35b0c00640 1 -- 172.21.15.50:0/2928184531 shutdown_connections 2024-09-21T06:01:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.693+0000 7f35b0c00640 1 --2- 172.21.15.50:0/2928184531 >> v2:172.21.15.88:3300/0 conn(0x7f35ac15f5c0 0x7f35ac15f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.693+0000 7f35b0c00640 1 --2- 172.21.15.50:0/2928184531 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f35ac06dbc0 0x7f35ac06dfd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.693+0000 7f35b0c00640 1 -- 172.21.15.50:0/2928184531 >> 172.21.15.50:0/2928184531 conn(0x7f35ac0b41f0 msgr2=0x7f35ac0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.693+0000 7f35b0c00640 1 -- 172.21.15.50:0/2928184531 shutdown_connections 2024-09-21T06:01:36.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.693+0000 7f35b0c00640 1 -- 172.21.15.50:0/2928184531 wait complete. 2024-09-21T06:01:36.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35b0c00640 1 Processor -- start 2024-09-21T06:01:36.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35b0c00640 1 -- start start 2024-09-21T06:01:36.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35b0c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f35ac06dbc0 0x7f35ac0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:36.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35b0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35ac0d1140 0x7f35ac0afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:36.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35b0c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35ac15f5c0 0x7f35ac15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:36.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35b0c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac168ff0 con 0x7f35ac0d1140 2024-09-21T06:01:36.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35b0c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac168e70 con 0x7f35ac15f5c0 2024-09-21T06:01:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35b0c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f35ac169170 con 0x7f35ac06dbc0 2024-09-21T06:01:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35a3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35ac0d1140 0x7f35ac0afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35a3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35ac0d1140 0x7f35ac0afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45646/0 (socket says 172.21.15.50:45646) 2024-09-21T06:01:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35a3e00640 1 -- 172.21.15.50:0/3610153481 learned_addr learned my addr 172.21.15.50:0/3610153481 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35ab400640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f35ac06dbc0 0x7f35ac0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35abe00640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35ac15f5c0 0x7f35ac15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35a3e00640 1 -- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f35ac06dbc0 msgr2=0x7f35ac0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35a3e00640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f35ac06dbc0 0x7f35ac0ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35a3e00640 1 -- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35ac15f5c0 msgr2=0x7f35ac15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35a3e00640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35ac15f5c0 0x7f35ac15a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35a3e00640 1 -- 172.21.15.50:0/3610153481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac15e810 con 0x7f35ac0d1140 2024-09-21T06:01:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35a3e00640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35ac0d1140 0x7f35ac0afa60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f359401cc70 tx=0x7f3594005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:36.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35a3400640 1 -- 172.21.15.50:0/3610153481 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35940b2040 con 0x7f35ac0d1140 2024-09-21T06:01:36.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35a3400640 1 -- 172.21.15.50:0/3610153481 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35940bb070 con 0x7f35ac0d1140 2024-09-21T06:01:36.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35b0c00640 1 -- 172.21.15.50:0/3610153481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac15eaf0 con 0x7f35ac0d1140 2024-09-21T06:01:36.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35a3400640 1 -- 172.21.15.50:0/3610153481 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594004330 con 0x7f35ac0d1140 2024-09-21T06:01:36.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.697+0000 7f35b0c00640 1 -- 172.21.15.50:0/3610153481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac0cdd30 con 0x7f35ac0d1140 2024-09-21T06:01:36.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.701+0000 7f35b0c00640 1 -- 172.21.15.50:0/3610153481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac0cd840 con 0x7f35ac0d1140 2024-09-21T06:01:36.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.701+0000 7f35a3400640 1 -- 172.21.15.50:0/3610153481 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 14) ==== 97591+0+0 (secure 0 0 0) 0x7f3594003600 con 0x7f35ac0d1140 2024-09-21T06:01:36.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.701+0000 7f35a3400640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f35880735f0 0x7f3588075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:36.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.701+0000 7f35a3400640 1 -- 172.21.15.50:0/3610153481 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f359412ed10 con 0x7f35ac0d1140 2024-09-21T06:01:36.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.701+0000 7f35ab400640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f35880735f0 0x7f3588075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:36.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.701+0000 7f35ab400640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f35880735f0 0x7f3588075b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f359c007d70 tx=0x7f359c007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:36.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.705+0000 7f35a3400640 1 -- 172.21.15.50:0/3610153481 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3594135080 con 0x7f35ac0d1140 2024-09-21T06:01:36.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:36.917+0000 7f35b0c00640 1 -- 172.21.15.50:0/3610153481 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone29661", "target": ["mon-mgr", ""]}) -- 0x7f35ac15ecf0 con 0x7f35880735f0 2024-09-21T06:01:37.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.133+0000 7f35a3400640 1 -- 172.21.15.50:0/3610153481 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 15) ==== 97693+0+0 (secure 0 0 0) 0x7f35940f51f0 con 0x7f35ac0d1140 2024-09-21T06:01:37.505 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:37.500+0000 7f6595800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:37.505 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:37.500+0000 7f6595800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:37.505 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:37.500+0000 7f6595800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:37.505 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:37.500+0000 7f6595800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:37.505 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:37.500+0000 7f6595800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:37.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.625+0000 7f35a3400640 1 -- 172.21.15.50:0/3610153481 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f35ac15ecf0 con 0x7f35880735f0 2024-09-21T06:01:37.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.637+0000 7f35b0c00640 1 -- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f35880735f0 msgr2=0x7f3588075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:37.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.637+0000 7f35b0c00640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f35880735f0 0x7f3588075b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f359c007d70 tx=0x7f359c007d00 comp rx=0 tx=0).stop 2024-09-21T06:01:37.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.637+0000 7f35b0c00640 1 -- 172.21.15.50:0/3610153481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35ac0d1140 msgr2=0x7f35ac0afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:37.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.637+0000 7f35b0c00640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35ac0d1140 0x7f35ac0afa60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f359401cc70 tx=0x7f3594005760 comp rx=0 tx=0).stop 2024-09-21T06:01:37.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.637+0000 7f35b0c00640 1 -- 172.21.15.50:0/3610153481 shutdown_connections 2024-09-21T06:01:37.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.637+0000 7f35b0c00640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35ac15f5c0 0x7f35ac15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:37.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.637+0000 7f35b0c00640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f35880735f0 0x7f3588075b00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:37.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.637+0000 7f35b0c00640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35ac0d1140 0x7f35ac0afa60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:37.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.637+0000 7f35b0c00640 1 --2- 172.21.15.50:0/3610153481 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f35ac06dbc0 0x7f35ac0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:37.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.637+0000 7f35b0c00640 1 -- 172.21.15.50:0/3610153481 >> 172.21.15.50:0/3610153481 conn(0x7f35ac0b41f0 msgr2=0x7f35ac0c9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:37.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.641+0000 7f35b0c00640 1 -- 172.21.15.50:0/3610153481 shutdown_connections 2024-09-21T06:01:37.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.641+0000 7f35b0c00640 1 -- 172.21.15.50:0/3610153481 wait complete. 2024-09-21T06:01:37.681 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone33912 2024-09-21T06:01:37.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.949+0000 7f80c7a00640 1 -- 172.21.15.50:0/4288632355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c0159130 msgr2=0x7f80c015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:37.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.949+0000 7f80c7a00640 1 --2- 172.21.15.50:0/4288632355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c0159130 0x7f80c015bb90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009840 tx=0x7f80b401c700 comp rx=0 tx=0).stop 2024-09-21T06:01:37.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.949+0000 7f80c7a00640 1 -- 172.21.15.50:0/4288632355 shutdown_connections 2024-09-21T06:01:37.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.949+0000 7f80c7a00640 1 --2- 172.21.15.50:0/4288632355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c0159130 0x7f80c015bb90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:37.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.949+0000 7f80c7a00640 1 -- 172.21.15.50:0/4288632355 >> 172.21.15.50:0/4288632355 conn(0x7f80c00b35f0 msgr2=0x7f80c00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:37.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.949+0000 7f80c7a00640 1 -- 172.21.15.50:0/4288632355 shutdown_connections 2024-09-21T06:01:37.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.949+0000 7f80c7a00640 1 -- 172.21.15.50:0/4288632355 wait complete. 2024-09-21T06:01:37.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.949+0000 7f80c7a00640 1 Processor -- start 2024-09-21T06:01:37.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.949+0000 7f80c7a00640 1 -- start start 2024-09-21T06:01:37.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80c7a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80c006cdf0 0x7f80c0158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:37.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80c7a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c00703a0 0x7f80c0150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:37.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80c7a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80c0159130 0x7f80c0151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:37.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80c7a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f80c0169d40 con 0x7f80c00703a0 2024-09-21T06:01:37.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80c7a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f80c006d580 con 0x7f80c0159130 2024-09-21T06:01:37.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80c7a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f80c0169ec0 con 0x7f80c006cdf0 2024-09-21T06:01:37.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80bfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c00703a0 0x7f80c0150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:37.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80bfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c00703a0 0x7f80c0150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45674/0 (socket says 172.21.15.50:45674) 2024-09-21T06:01:37.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80bfe00640 1 -- 172.21.15.50:0/1521518037 learned_addr learned my addr 172.21.15.50:0/1521518037 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:37.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80bfe00640 1 -- 172.21.15.50:0/1521518037 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80c006cdf0 msgr2=0x7f80c0158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:37.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80bfe00640 1 --2- 172.21.15.50:0/1521518037 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80c006cdf0 0x7f80c0158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:37.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80bfe00640 1 -- 172.21.15.50:0/1521518037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80c0159130 msgr2=0x7f80c0151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:37.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80bfe00640 1 --2- 172.21.15.50:0/1521518037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80c0159130 0x7f80c0151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:37.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80bfe00640 1 -- 172.21.15.50:0/1521518037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c01540d0 con 0x7f80c00703a0 2024-09-21T06:01:37.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80bfe00640 1 --2- 172.21.15.50:0/1521518037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c00703a0 0x7f80c0150f40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00b3e0 tx=0x7f80ac00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:37.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80bcc00640 1 -- 172.21.15.50:0/1521518037 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac015020 con 0x7f80c00703a0 2024-09-21T06:01:37.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80bcc00640 1 -- 172.21.15.50:0/1521518037 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80ac011940 con 0x7f80c00703a0 2024-09-21T06:01:37.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80c7a00640 1 -- 172.21.15.50:0/1521518037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c0156f60 con 0x7f80c00703a0 2024-09-21T06:01:37.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80bcc00640 1 -- 172.21.15.50:0/1521518037 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac011d90 con 0x7f80c00703a0 2024-09-21T06:01:37.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.953+0000 7f80c7a00640 1 -- 172.21.15.50:0/1521518037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c014be60 con 0x7f80c00703a0 2024-09-21T06:01:37.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.957+0000 7f80c7a00640 1 -- 172.21.15.50:0/1521518037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c015bb90 con 0x7f80c00703a0 2024-09-21T06:01:37.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.957+0000 7f80bcc00640 1 -- 172.21.15.50:0/1521518037 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) ==== 97693+0+0 (secure 0 0 0) 0x7f80ac018020 con 0x7f80c00703a0 2024-09-21T06:01:37.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.961+0000 7f80bcc00640 1 --2- 172.21.15.50:0/1521518037 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80a0073740 0x7f80a0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:37.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.961+0000 7f80bcc00640 1 -- 172.21.15.50:0/1521518037 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f80ac094650 con 0x7f80c00703a0 2024-09-21T06:01:37.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.965+0000 7f80bcc00640 1 -- 172.21.15.50:0/1521518037 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f80ac01f070 con 0x7f80c00703a0 2024-09-21T06:01:37.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.973+0000 7f80c5000640 1 --2- 172.21.15.50:0/1521518037 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80a0073740 0x7f80a0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:37.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:37.977+0000 7f80c5000640 1 --2- 172.21.15.50:0/1521518037 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80a0073740 0x7f80a0075c50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f80b40af000 tx=0x7f80b40194a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:38.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.317+0000 7f80c7a00640 1 -- 172.21.15.50:0/1521518037 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone33912", "target": ["mon-mgr", ""]}) -- 0x7f80c006b540 con 0x7f80a0073740 2024-09-21T06:01:38.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.749+0000 7f80bcc00640 1 -- 172.21.15.50:0/1521518037 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f80b40a85f0 con 0x7f80a0073740 2024-09-21T06:01:38.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.753+0000 7f80c7a00640 1 -- 172.21.15.50:0/1521518037 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80a0073740 msgr2=0x7f80a0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:38.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.753+0000 7f80c7a00640 1 --2- 172.21.15.50:0/1521518037 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80a0073740 0x7f80a0075c50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f80b40af000 tx=0x7f80b40194a0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.753+0000 7f80c7a00640 1 -- 172.21.15.50:0/1521518037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c00703a0 msgr2=0x7f80c0150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:38.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.753+0000 7f80c7a00640 1 --2- 172.21.15.50:0/1521518037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c00703a0 0x7f80c0150f40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00b3e0 tx=0x7f80ac00b8a0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.753+0000 7f80c7a00640 1 -- 172.21.15.50:0/1521518037 shutdown_connections 2024-09-21T06:01:38.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.753+0000 7f80c7a00640 1 --2- 172.21.15.50:0/1521518037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80c0159130 0x7f80c0151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.753+0000 7f80c7a00640 1 --2- 172.21.15.50:0/1521518037 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80a0073740 0x7f80a0075c50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.753+0000 7f80c7a00640 1 --2- 172.21.15.50:0/1521518037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c00703a0 0x7f80c0150f40 secure :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00b3e0 tx=0x7f80ac00b8a0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.753+0000 7f80c7a00640 1 --2- 172.21.15.50:0/1521518037 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80c006cdf0 0x7f80c0158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.753+0000 7f80c7a00640 1 -- 172.21.15.50:0/1521518037 >> 172.21.15.50:0/1521518037 conn(0x7f80c00b35f0 msgr2=0x7f80c00b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:38.758 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:38.756+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:38.758 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:38.756+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:38.758 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:38.756+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:38.759 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:38.756+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:38.759 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:38.756+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:38.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.757+0000 7f80c7a00640 1 -- 172.21.15.50:0/1521518037 shutdown_connections 2024-09-21T06:01:38.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:38.757+0000 7f80c7a00640 1 -- 172.21.15.50:0/1521518037 wait complete. 2024-09-21T06:01:38.797 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone30643 2024-09-21T06:01:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.041+0000 7fb43a400640 1 -- 172.21.15.50:0/2360911014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb43406cdf0 msgr2=0x7fb43406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.041+0000 7fb43a400640 1 --2- 172.21.15.50:0/2360911014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb43406cdf0 0x7fb43406d200 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0098a0 tx=0x7fb41c01e8c0 comp rx=0 tx=0).stop 2024-09-21T06:01:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.041+0000 7fb43a400640 1 -- 172.21.15.50:0/2360911014 shutdown_connections 2024-09-21T06:01:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.041+0000 7fb43a400640 1 --2- 172.21.15.50:0/2360911014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb43406cdf0 0x7fb43406d200 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.041+0000 7fb43a400640 1 -- 172.21.15.50:0/2360911014 >> 172.21.15.50:0/2360911014 conn(0x7fb4340b35f0 msgr2=0x7fb4340b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.041+0000 7fb43a400640 1 -- 172.21.15.50:0/2360911014 shutdown_connections 2024-09-21T06:01:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.041+0000 7fb43a400640 1 -- 172.21.15.50:0/2360911014 wait complete. 2024-09-21T06:01:39.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.045+0000 7fb43a400640 1 Processor -- start 2024-09-21T06:01:39.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.045+0000 7fb43a400640 1 -- start start 2024-09-21T06:01:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.045+0000 7fb43a400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb43406cdf0 0x7fb43415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb43a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb43406fc90 0x7fb43415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb43a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb434159130 0x7fb43414cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb43a400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb434169530 con 0x7fb434159130 2024-09-21T06:01:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb43a400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4341696b0 con 0x7fb43406fc90 2024-09-21T06:01:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb43a400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb434169830 con 0x7fb43406cdf0 2024-09-21T06:01:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb433e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb434159130 0x7fb43414cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb433e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb434159130 0x7fb43414cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45694/0 (socket says 172.21.15.50:45694) 2024-09-21T06:01:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb433e00640 1 -- 172.21.15.50:0/4071762052 learned_addr learned my addr 172.21.15.50:0/4071762052 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb433400640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb43406cdf0 0x7fb43415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:39.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb433e00640 1 -- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb43406cdf0 msgr2=0x7fb43415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:39.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb433e00640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb43406cdf0 0x7fb43415cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:39.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb433e00640 1 -- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb43406fc90 msgr2=0x7fb43415a1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:01:39.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb433e00640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb43406fc90 0x7fb43415a1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb433e00640 1 -- 172.21.15.50:0/4071762052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43415ee90 con 0x7fb434159130 2024-09-21T06:01:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb433400640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb43406cdf0 0x7fb43415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:01:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb433e00640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb434159130 0x7fb43414cdd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00c740 tx=0x7fb42c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb42b400640 1 -- 172.21.15.50:0/4071762052 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c003b80 con 0x7fb434159130 2024-09-21T06:01:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb43a400640 1 -- 172.21.15.50:0/4071762052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43415a700 con 0x7fb434159130 2024-09-21T06:01:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb42b400640 1 -- 172.21.15.50:0/4071762052 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb42c00ce50 con 0x7fb434159130 2024-09-21T06:01:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb42b400640 1 -- 172.21.15.50:0/4071762052 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00a850 con 0x7fb434159130 2024-09-21T06:01:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb43a400640 1 -- 172.21.15.50:0/4071762052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4341546d0 con 0x7fb434159130 2024-09-21T06:01:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.049+0000 7fb43a400640 1 -- 172.21.15.50:0/4071762052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4341565c0 con 0x7fb434159130 2024-09-21T06:01:39.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.053+0000 7fb42b400640 1 -- 172.21.15.50:0/4071762052 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) ==== 97693+0+0 (secure 0 0 0) 0x7fb42c00a9f0 con 0x7fb434159130 2024-09-21T06:01:39.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.053+0000 7fb42b400640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb410073710 0x7fb410075c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:39.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.053+0000 7fb42b400640 1 -- 172.21.15.50:0/4071762052 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fb42c094b90 con 0x7fb434159130 2024-09-21T06:01:39.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.057+0000 7fb433400640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb410073710 0x7fb410075c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:39.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.057+0000 7fb433400640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb410073710 0x7fb410075c20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb424001230 tx=0x7fb424009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:39.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.061+0000 7fb42b400640 1 -- 172.21.15.50:0/4071762052 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb42c05f2a0 con 0x7fb434159130 2024-09-21T06:01:39.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.165+0000 7fb42b400640 1 -- 172.21.15.50:0/4071762052 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 16) ==== 97795+0+0 (secure 0 0 0) 0x7fb42c00ace0 con 0x7fb434159130 2024-09-21T06:01:39.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:39.477+0000 7fb43a400640 1 -- 172.21.15.50:0/4071762052 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone30643", "target": ["mon-mgr", ""]}) -- 0x7fb43406b540 con 0x7fb410073710 2024-09-21T06:01:40.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.613+0000 7fb42b400640 1 -- 172.21.15.50:0/4071762052 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb42400d750 con 0x7fb410073710 2024-09-21T06:01:40.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.621+0000 7fb43a400640 1 -- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb410073710 msgr2=0x7fb410075c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:40.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.621+0000 7fb43a400640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb410073710 0x7fb410075c20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb424001230 tx=0x7fb424009040 comp rx=0 tx=0).stop 2024-09-21T06:01:40.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.621+0000 7fb43a400640 1 -- 172.21.15.50:0/4071762052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb434159130 msgr2=0x7fb43414cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:40.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.621+0000 7fb43a400640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb434159130 0x7fb43414cdd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00c740 tx=0x7fb42c00cc00 comp rx=0 tx=0).stop 2024-09-21T06:01:40.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.625+0000 7fb43a400640 1 -- 172.21.15.50:0/4071762052 shutdown_connections 2024-09-21T06:01:40.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.625+0000 7fb43a400640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb410073710 0x7fb410075c20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.625+0000 7fb43a400640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb434159130 0x7fb43414cdd0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.625+0000 7fb43a400640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb43406fc90 0x7fb43415a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.625+0000 7fb43a400640 1 --2- 172.21.15.50:0/4071762052 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb43406cdf0 0x7fb43415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.625+0000 7fb43a400640 1 -- 172.21.15.50:0/4071762052 >> 172.21.15.50:0/4071762052 conn(0x7fb4340b35f0 msgr2=0x7fb4340b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:40.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.625+0000 7fb43a400640 1 -- 172.21.15.50:0/4071762052 shutdown_connections 2024-09-21T06:01:40.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:40.625+0000 7fb43a400640 1 -- 172.21.15.50:0/4071762052 wait complete. 2024-09-21T06:01:40.689 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:01:40.760 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:40.756+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:40.760 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:40.756+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:40.760 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:40.756+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:40.760 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:40.756+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:40.760 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:40.756+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:41.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.065+0000 7f3f7de00640 1 -- 172.21.15.50:0/4160626493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f5c0051d0 msgr2=0x7f3f5c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:41.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.065+0000 7f3f7de00640 1 --2- 172.21.15.50:0/4160626493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f5c0051d0 0x7f3f5c00b9d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000bc00 tx=0x7f3f7001ce00 comp rx=0 tx=0).stop 2024-09-21T06:01:41.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.065+0000 7f3f77400640 1 -- 172.21.15.50:0/4160626493 reap_dead start 2024-09-21T06:01:41.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.065+0000 7f3f7de00640 1 -- 172.21.15.50:0/4160626493 shutdown_connections 2024-09-21T06:01:41.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.065+0000 7f3f7de00640 1 -- 172.21.15.50:0/4160626493 >> 172.21.15.50:0/4160626493 conn(0x7f3f780b35f0 msgr2=0x7f3f780b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:41.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.065+0000 7f3f7de00640 1 -- 172.21.15.50:0/4160626493 shutdown_connections 2024-09-21T06:01:41.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.065+0000 7f3f7de00640 1 -- 172.21.15.50:0/4160626493 wait complete. 2024-09-21T06:01:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.065+0000 7f3f7de00640 1 Processor -- start 2024-09-21T06:01:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.069+0000 7f3f7de00640 1 -- start start 2024-09-21T06:01:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.069+0000 7f3f7de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f5c0051d0 0x7f3f780c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.069+0000 7f3f7de00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f7815a7e0 0x7f3f781551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.069+0000 7f3f7de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f781f4a70 0x7f3f781f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.069+0000 7f3f7de00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7815c7a0 con 0x7f3f781f4a70 2024-09-21T06:01:41.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.069+0000 7f3f7de00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7815c620 con 0x7f3f5c0051d0 2024-09-21T06:01:41.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.069+0000 7f3f7de00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7815c920 con 0x7f3f7815a7e0 2024-09-21T06:01:41.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.069+0000 7f3f77400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f781f4a70 0x7f3f781f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:41.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.069+0000 7f3f77400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f781f4a70 0x7f3f781f4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45712/0 (socket says 172.21.15.50:45712) 2024-09-21T06:01:41.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.069+0000 7f3f77400640 1 -- 172.21.15.50:0/1094443980 learned_addr learned my addr 172.21.15.50:0/1094443980 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:41.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.069+0000 7f3f76000640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f7815a7e0 0x7f3f781551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.073+0000 7f3f77400640 1 -- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f7815a7e0 msgr2=0x7f3f781551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.073+0000 7f3f77400640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f7815a7e0 0x7f3f781551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.073+0000 7f3f77400640 1 -- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f5c0051d0 msgr2=0x7f3f780c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.073+0000 7f3f77400640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f5c0051d0 0x7f3f780c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.073+0000 7f3f77400640 1 -- 172.21.15.50:0/1094443980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f781f53c0 con 0x7f3f781f4a70 2024-09-21T06:01:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.073+0000 7f3f77400640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f781f4a70 0x7f3f781f4e80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3f7001c870 tx=0x7f3f7001b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:41.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.073+0000 7f3f6f400640 1 -- 172.21.15.50:0/1094443980 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f700ab040 con 0x7f3f781f4a70 2024-09-21T06:01:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.073+0000 7f3f7de00640 1 -- 172.21.15.50:0/1094443980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f78155710 con 0x7f3f781f4a70 2024-09-21T06:01:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.073+0000 7f3f7de00640 1 -- 172.21.15.50:0/1094443980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f780c6c50 con 0x7f3f781f4a70 2024-09-21T06:01:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.073+0000 7f3f6f400640 1 -- 172.21.15.50:0/1094443980 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f70005030 con 0x7f3f781f4a70 2024-09-21T06:01:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.073+0000 7f3f6f400640 1 -- 172.21.15.50:0/1094443980 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7000b7d0 con 0x7f3f781f4a70 2024-09-21T06:01:41.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.073+0000 7f3f6cc00640 1 -- 172.21.15.50:0/1094443980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f480052a0 con 0x7f3f781f4a70 2024-09-21T06:01:41.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.077+0000 7f3f6f400640 1 -- 172.21.15.50:0/1094443980 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 97795+0+0 (secure 0 0 0) 0x7f3f700053e0 con 0x7f3f781f4a70 2024-09-21T06:01:41.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.081+0000 7f3f6f400640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f540738b0 0x7f3f54075dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:41.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.081+0000 7f3f6f400640 1 -- 172.21.15.50:0/1094443980 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f3f7012eb20 con 0x7f3f781f4a70 2024-09-21T06:01:41.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.085+0000 7f3f6f400640 1 -- 172.21.15.50:0/1094443980 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3f781f53c0 con 0x7f3f781f4a70 2024-09-21T06:01:41.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.085+0000 7f3f76a00640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f540738b0 0x7f3f54075dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:41.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.093+0000 7f3f76a00640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f540738b0 0x7f3f54075dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f7806df20 tx=0x7f3f68001500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:41.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.245+0000 7f3f6f400640 1 -- 172.21.15.50:0/1094443980 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 17) ==== 97846+0+0 (secure 0 0 0) 0x7f3f700f4fd0 con 0x7f3f781f4a70 2024-09-21T06:01:41.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.537+0000 7f3f6cc00640 1 -- 172.21.15.50:0/1094443980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3f48005860 con 0x7f3f781f4a70 2024-09-21T06:01:41.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.541+0000 7f3f6f400640 1 -- 172.21.15.50:0/1094443980 <== mon.0 v2:172.21.15.50:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2167 (secure 0 0 0) 0x7f3f70016b10 con 0x7f3f781f4a70 2024-09-21T06:01:41.548 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:01:41.548 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:01:41.549 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 338, 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 46, 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:01:41.550 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:01:41.551 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 2677, 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 11099449712, 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 21909835776, 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 751184277504, 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 36687106, 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 119511989, 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 9, 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 33 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 28, 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T05:59:51:480904+0000", 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3, 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:01:41.552 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 3, 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "c", 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 5370 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:01:41.553 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 10, 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:00:46.952552+0000", 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": { 2024-09-21T06:01:41.554 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr-vol-ongoing-clones": { 2024-09-21T06:01:41.555 INFO:teuthology.orchestra.run.smithi050.stdout: "message": "1 ongoing clones - average progress is 0.859% (0s)\n [............................] ", 2024-09-21T06:01:41.555 INFO:teuthology.orchestra.run.smithi050.stdout: "progress": 0.0085899997502565384, 2024-09-21T06:01:41.555 INFO:teuthology.orchestra.run.smithi050.stdout: "add_to_ceph_s": true 2024-09-21T06:01:41.555 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:01:41.555 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:01:41.555 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:01:41.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.549+0000 7f3f6cc00640 1 -- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f540738b0 msgr2=0x7f3f54075dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:41.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.549+0000 7f3f6cc00640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f540738b0 0x7f3f54075dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f7806df20 tx=0x7f3f68001500 comp rx=0 tx=0).stop 2024-09-21T06:01:41.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.549+0000 7f3f6cc00640 1 -- 172.21.15.50:0/1094443980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f781f4a70 msgr2=0x7f3f781f4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:41.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.549+0000 7f3f6cc00640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f781f4a70 0x7f3f781f4e80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3f7001c870 tx=0x7f3f7001b640 comp rx=0 tx=0).stop 2024-09-21T06:01:41.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.549+0000 7f3f6cc00640 1 -- 172.21.15.50:0/1094443980 shutdown_connections 2024-09-21T06:01:41.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.549+0000 7f3f6cc00640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f540738b0 0x7f3f54075dc0 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f7806df20 tx=0x7f3f68001500 comp rx=0 tx=0).stop 2024-09-21T06:01:41.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.549+0000 7f3f6cc00640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f781f4a70 0x7f3f781f4e80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:41.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.549+0000 7f3f6cc00640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f7815a7e0 0x7f3f781551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:41.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.549+0000 7f3f6cc00640 1 --2- 172.21.15.50:0/1094443980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f5c0051d0 0x7f3f780c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:41.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.549+0000 7f3f6cc00640 1 -- 172.21.15.50:0/1094443980 >> 172.21.15.50:0/1094443980 conn(0x7f3f780b35f0 msgr2=0x7f3f7815f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:41.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.553+0000 7f3f6cc00640 1 -- 172.21.15.50:0/1094443980 shutdown_connections 2024-09-21T06:01:41.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.553+0000 7f3f6cc00640 1 -- 172.21.15.50:0/1094443980 wait complete. 2024-09-21T06:01:41.630 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone20400 2024-09-21T06:01:41.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.917+0000 7f19c5a00640 1 -- 172.21.15.50:0/506990739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19c0074d30 msgr2=0x7f19c00b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:41.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.917+0000 7f19c5a00640 1 --2- 172.21.15.50:0/506990739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19c0074d30 0x7f19c00b1dc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009900 tx=0x7f19b001c8b0 comp rx=0 tx=0).stop 2024-09-21T06:01:41.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.917+0000 7f19c5a00640 1 -- 172.21.15.50:0/506990739 shutdown_connections 2024-09-21T06:01:41.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.917+0000 7f19c5a00640 1 --2- 172.21.15.50:0/506990739 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f19c00cf370 0x7f19c00d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:41.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.917+0000 7f19c5a00640 1 --2- 172.21.15.50:0/506990739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19c0074d30 0x7f19c00b1dc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:41.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.917+0000 7f19c5a00640 1 -- 172.21.15.50:0/506990739 >> 172.21.15.50:0/506990739 conn(0x7f19c006cdf0 msgr2=0x7f19c006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:41.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.921+0000 7f19c5a00640 1 -- 172.21.15.50:0/506990739 shutdown_connections 2024-09-21T06:01:41.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.921+0000 7f19c5a00640 1 -- 172.21.15.50:0/506990739 wait complete. 2024-09-21T06:01:41.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.925+0000 7f19c5a00640 1 Processor -- start 2024-09-21T06:01:41.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.925+0000 7f19c5a00640 1 -- start start 2024-09-21T06:01:41.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.925+0000 7f19c5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19c0074d30 0x7f19c013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:41.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.925+0000 7f19c5a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c00b2300 0x7f19c0152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:41.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.925+0000 7f19c5a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f19c00cf370 0x7f19c013ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:41.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.925+0000 7f19c5a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f19c0169a60 con 0x7f19c0074d30 2024-09-21T06:01:41.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.925+0000 7f19c5a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f19c00d0ef0 con 0x7f19c00b2300 2024-09-21T06:01:41.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.925+0000 7f19c5a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f19c0169be0 con 0x7f19c00cf370 2024-09-21T06:01:41.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.925+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19c0074d30 0x7f19c013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:41.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.925+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19c0074d30 0x7f19c013ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45738/0 (socket says 172.21.15.50:45738) 2024-09-21T06:01:41.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.925+0000 7f19bfe00640 1 -- 172.21.15.50:0/1866429179 learned_addr learned my addr 172.21.15.50:0/1866429179 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:41.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.929+0000 7f19bfe00640 1 -- 172.21.15.50:0/1866429179 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f19c00cf370 msgr2=0x7f19c013ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:41.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.929+0000 7f19bfe00640 1 --2- 172.21.15.50:0/1866429179 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f19c00cf370 0x7f19c013ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:41.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.929+0000 7f19bfe00640 1 -- 172.21.15.50:0/1866429179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c00b2300 msgr2=0x7f19c0152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:41.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.929+0000 7f19bfe00640 1 --2- 172.21.15.50:0/1866429179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c00b2300 0x7f19c0152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:41.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.929+0000 7f19bfe00640 1 -- 172.21.15.50:0/1866429179 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c013b390 con 0x7f19c0074d30 2024-09-21T06:01:41.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.929+0000 7f19bfe00640 1 --2- 172.21.15.50:0/1866429179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19c0074d30 0x7f19c013ae50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f19b001a4e0 tx=0x7f19b00056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:41.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.929+0000 7f19bcc00640 1 -- 172.21.15.50:0/1866429179 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b00bb070 con 0x7f19c0074d30 2024-09-21T06:01:41.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.929+0000 7f19bcc00640 1 -- 172.21.15.50:0/1866429179 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19b00038f0 con 0x7f19c0074d30 2024-09-21T06:01:41.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.929+0000 7f19bcc00640 1 -- 172.21.15.50:0/1866429179 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b00b2040 con 0x7f19c0074d30 2024-09-21T06:01:41.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.929+0000 7f19c5a00640 1 -- 172.21.15.50:0/1866429179 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c01fd140 con 0x7f19c0074d30 2024-09-21T06:01:41.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.929+0000 7f19c5a00640 1 -- 172.21.15.50:0/1866429179 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c01fd400 con 0x7f19c0074d30 2024-09-21T06:01:41.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.933+0000 7f19bcc00640 1 -- 172.21.15.50:0/1866429179 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 97846+0+0 (secure 0 0 0) 0x7f19b0003d00 con 0x7f19c0074d30 2024-09-21T06:01:41.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.933+0000 7f19bcc00640 1 --2- 172.21.15.50:0/1866429179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f199c073990 0x7f199c075ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:41.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.933+0000 7f19bcc00640 1 -- 172.21.15.50:0/1866429179 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f19b012ee10 con 0x7f19c0074d30 2024-09-21T06:01:41.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.937+0000 7f19c5a00640 1 -- 172.21.15.50:0/1866429179 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c00b1dc0 con 0x7f19c0074d30 2024-09-21T06:01:41.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.945+0000 7f19bcc00640 1 -- 172.21.15.50:0/1866429179 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f19b00164d0 con 0x7f19c0074d30 2024-09-21T06:01:41.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.965+0000 7f19b7400640 1 --2- 172.21.15.50:0/1866429179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f199c073990 0x7f199c075ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:41.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:41.973+0000 7f19b7400640 1 --2- 172.21.15.50:0/1866429179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f199c073990 0x7f199c075ea0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f19a8000f70 tx=0x7f19a80079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:42.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:42.377+0000 7f19c5a00640 1 -- 172.21.15.50:0/1866429179 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone20400", "target": ["mon-mgr", ""]}) -- 0x7f19c006b540 con 0x7f199c073990 2024-09-21T06:01:42.679 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:42.676+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:42.679 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:42.676+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:42.679 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:42.676+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:42.679 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:42.676+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:42.679 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:42.676+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:44.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.761+0000 7f19bcc00640 1 -- 172.21.15.50:0/1866429179 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19a80050d0 con 0x7f199c073990 2024-09-21T06:01:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.765+0000 7f19c5a00640 1 -- 172.21.15.50:0/1866429179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f199c073990 msgr2=0x7f199c075ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.765+0000 7f19c5a00640 1 --2- 172.21.15.50:0/1866429179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f199c073990 0x7f199c075ea0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f19a8000f70 tx=0x7f19a80079c0 comp rx=0 tx=0).stop 2024-09-21T06:01:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.765+0000 7f19c5a00640 1 -- 172.21.15.50:0/1866429179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19c0074d30 msgr2=0x7f19c013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.765+0000 7f19c5a00640 1 --2- 172.21.15.50:0/1866429179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19c0074d30 0x7f19c013ae50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f19b001a4e0 tx=0x7f19b00056e0 comp rx=0 tx=0).stop 2024-09-21T06:01:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.765+0000 7f19c5a00640 1 -- 172.21.15.50:0/1866429179 shutdown_connections 2024-09-21T06:01:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.765+0000 7f19c5a00640 1 --2- 172.21.15.50:0/1866429179 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f19c00cf370 0x7f19c013ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.765+0000 7f19c5a00640 1 --2- 172.21.15.50:0/1866429179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c00b2300 0x7f19c0152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.765+0000 7f19c5a00640 1 --2- 172.21.15.50:0/1866429179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f199c073990 0x7f199c075ea0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.765+0000 7f19c5a00640 1 --2- 172.21.15.50:0/1866429179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19c0074d30 0x7f19c013ae50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.765+0000 7f19c5a00640 1 -- 172.21.15.50:0/1866429179 >> 172.21.15.50:0/1866429179 conn(0x7f19c006cdf0 msgr2=0x7f19c006ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.765+0000 7f19c5a00640 1 -- 172.21.15.50:0/1866429179 shutdown_connections 2024-09-21T06:01:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:44.765+0000 7f19c5a00640 1 -- 172.21.15.50:0/1866429179 wait complete. 2024-09-21T06:01:44.811 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone29661 2024-09-21T06:01:45.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.221+0000 7f5295a00640 1 -- 172.21.15.50:0/1649610257 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52900cf210 msgr2=0x7f52900ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:45.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.221+0000 7f5295a00640 1 --2- 172.21.15.50:0/1649610257 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52900cf210 0x7f52900ae8e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f527c009840 tx=0x7f527c01c740 comp rx=0 tx=0).stop 2024-09-21T06:01:45.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.221+0000 7f5295a00640 1 -- 172.21.15.50:0/1649610257 shutdown_connections 2024-09-21T06:01:45.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.221+0000 7f5295a00640 1 --2- 172.21.15.50:0/1649610257 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52900cf210 0x7f52900ae8e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:45.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.221+0000 7f5295a00640 1 --2- 172.21.15.50:0/1649610257 >> v2:172.21.15.88:3300/0 conn(0x7f529015f8f0 0x7f52901557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:45.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.221+0000 7f5295a00640 1 -- 172.21.15.50:0/1649610257 >> 172.21.15.50:0/1649610257 conn(0x7f529006e1d0 msgr2=0x7f529006e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:45.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.221+0000 7f5295a00640 1 -- 172.21.15.50:0/1649610257 shutdown_connections 2024-09-21T06:01:45.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.221+0000 7f5295a00640 1 -- 172.21.15.50:0/1649610257 wait complete. 2024-09-21T06:01:45.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.225+0000 7f5295a00640 1 Processor -- start 2024-09-21T06:01:45.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.225+0000 7f5295a00640 1 -- start start 2024-09-21T06:01:45.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.229+0000 7f5295a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52900cf210 0x7f52901494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:45.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.229+0000 7f5295a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52900cf620 0x7f5290146c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:45.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.233+0000 7f5295a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f529015f8f0 0x7f529014bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:45.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.233+0000 7f5295a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f529016c740 con 0x7f52900cf210 2024-09-21T06:01:45.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.233+0000 7f5295a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f529016c5c0 con 0x7f52900cf620 2024-09-21T06:01:45.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.233+0000 7f5295a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f529016c8c0 con 0x7f529015f8f0 2024-09-21T06:01:45.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.233+0000 7f528ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52900cf210 0x7f52901494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:45.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.233+0000 7f528ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52900cf210 0x7f52901494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45744/0 (socket says 172.21.15.50:45744) 2024-09-21T06:01:45.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.233+0000 7f528ea00640 1 -- 172.21.15.50:0/339822236 learned_addr learned my addr 172.21.15.50:0/339822236 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:45.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.233+0000 7f528ea00640 1 -- 172.21.15.50:0/339822236 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f529015f8f0 msgr2=0x7f529014bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:45.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.233+0000 7f528ea00640 1 --2- 172.21.15.50:0/339822236 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f529015f8f0 0x7f529014bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:45.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.233+0000 7f528ea00640 1 -- 172.21.15.50:0/339822236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52900cf620 msgr2=0x7f5290146c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:45.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.233+0000 7f528ea00640 1 --2- 172.21.15.50:0/339822236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52900cf620 0x7f5290146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:45.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.237+0000 7f528ea00640 1 -- 172.21.15.50:0/339822236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5290147630 con 0x7f52900cf210 2024-09-21T06:01:45.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.237+0000 7f528ea00640 1 --2- 172.21.15.50:0/339822236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52900cf210 0x7f52901494a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f528000d950 tx=0x7f528000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:45.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.237+0000 7f5286a00640 1 -- 172.21.15.50:0/339822236 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280017070 con 0x7f52900cf210 2024-09-21T06:01:45.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.237+0000 7f5286a00640 1 -- 172.21.15.50:0/339822236 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5280009cc0 con 0x7f52900cf210 2024-09-21T06:01:45.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.237+0000 7f5286a00640 1 -- 172.21.15.50:0/339822236 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52800106c0 con 0x7f52900cf210 2024-09-21T06:01:45.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.237+0000 7f5295a00640 1 -- 172.21.15.50:0/339822236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5290147910 con 0x7f52900cf210 2024-09-21T06:01:45.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.241+0000 7f5295a00640 1 -- 172.21.15.50:0/339822236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5290147ed0 con 0x7f52900cf210 2024-09-21T06:01:45.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.241+0000 7f5295a00640 1 -- 172.21.15.50:0/339822236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52900ae8e0 con 0x7f52900cf210 2024-09-21T06:01:45.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.249+0000 7f5286a00640 1 -- 172.21.15.50:0/339822236 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 97846+0+0 (secure 0 0 0) 0x7f528001a070 con 0x7f52900cf210 2024-09-21T06:01:45.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.253+0000 7f5286a00640 1 --2- 172.21.15.50:0/339822236 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f52700738f0 0x7f5270075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:45.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.253+0000 7f5286a00640 1 -- 172.21.15.50:0/339822236 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f5280095400 con 0x7f52900cf210 2024-09-21T06:01:45.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.257+0000 7f5286a00640 1 -- 172.21.15.50:0/339822236 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f528005fa40 con 0x7f52900cf210 2024-09-21T06:01:45.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.273+0000 7f528e000640 1 --2- 172.21.15.50:0/339822236 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f52700738f0 0x7f5270075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:45.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.277+0000 7f528e000640 1 --2- 172.21.15.50:0/339822236 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f52700738f0 0x7f5270075e00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f527c0af000 tx=0x7f527c0194e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:45.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.357+0000 7f5286a00640 1 -- 172.21.15.50:0/339822236 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 18) ==== 97897+0+0 (secure 0 0 0) 0x7f528005b5b0 con 0x7f52900cf210 2024-09-21T06:01:45.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:45.617+0000 7f5295a00640 1 -- 172.21.15.50:0/339822236 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone29661", "target": ["mon-mgr", ""]}) -- 0x7f529006b540 con 0x7f52700738f0 2024-09-21T06:01:46.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.669+0000 7f5286a00640 1 -- 172.21.15.50:0/339822236 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f527c0a8530 con 0x7f52700738f0 2024-09-21T06:01:46.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.673+0000 7f5295a00640 1 -- 172.21.15.50:0/339822236 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f52700738f0 msgr2=0x7f5270075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:46.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.673+0000 7f5295a00640 1 --2- 172.21.15.50:0/339822236 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f52700738f0 0x7f5270075e00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f527c0af000 tx=0x7f527c0194e0 comp rx=0 tx=0).stop 2024-09-21T06:01:46.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.673+0000 7f5295a00640 1 -- 172.21.15.50:0/339822236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52900cf210 msgr2=0x7f52901494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:46.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.673+0000 7f5295a00640 1 --2- 172.21.15.50:0/339822236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52900cf210 0x7f52901494a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f528000d950 tx=0x7f528000de10 comp rx=0 tx=0).stop 2024-09-21T06:01:46.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.673+0000 7f5295a00640 1 -- 172.21.15.50:0/339822236 shutdown_connections 2024-09-21T06:01:46.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.673+0000 7f5295a00640 1 --2- 172.21.15.50:0/339822236 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f529015f8f0 0x7f529014bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:46.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.673+0000 7f5295a00640 1 --2- 172.21.15.50:0/339822236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52900cf620 0x7f5290146c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:46.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.673+0000 7f5295a00640 1 --2- 172.21.15.50:0/339822236 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f52700738f0 0x7f5270075e00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:46.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.673+0000 7f5295a00640 1 --2- 172.21.15.50:0/339822236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52900cf210 0x7f52901494a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:46.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.673+0000 7f5295a00640 1 -- 172.21.15.50:0/339822236 >> 172.21.15.50:0/339822236 conn(0x7f529006e1d0 msgr2=0x7f529014e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:46.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.673+0000 7f5295a00640 1 -- 172.21.15.50:0/339822236 shutdown_connections 2024-09-21T06:01:46.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:46.677+0000 7f5295a00640 1 -- 172.21.15.50:0/339822236 wait complete. 2024-09-21T06:01:46.732 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone33912 2024-09-21T06:01:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.037+0000 7f8bf9a00640 1 -- 172.21.15.50:0/1772477564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd80099c0 msgr2=0x7f8bd800bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.037+0000 7f8bf9a00640 1 --2- 172.21.15.50:0/1772477564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd80099c0 0x7f8bd800bde0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8be4006ef0 tx=0x7f8be401f5b0 comp rx=0 tx=0).stop 2024-09-21T06:01:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.037+0000 7f8bf9a00640 1 -- 172.21.15.50:0/1772477564 shutdown_connections 2024-09-21T06:01:47.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.037+0000 7f8bf9a00640 1 --2- 172.21.15.50:0/1772477564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd80099c0 0x7f8bd800bde0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:47.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.037+0000 7f8bf9a00640 1 -- 172.21.15.50:0/1772477564 >> 172.21.15.50:0/1772477564 conn(0x7f8bf406e4a0 msgr2=0x7f8bf406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:47.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.037+0000 7f8bf9a00640 1 -- 172.21.15.50:0/1772477564 shutdown_connections 2024-09-21T06:01:47.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.037+0000 7f8bf9a00640 1 -- 172.21.15.50:0/1772477564 wait complete. 2024-09-21T06:01:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.041+0000 7f8bf9a00640 1 Processor -- start 2024-09-21T06:01:47.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.041+0000 7f8bf9a00640 1 -- start start 2024-09-21T06:01:47.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd80099c0 0x7f8bf414ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:47.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bf40b2200 0x7f8bf40b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:47.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf9a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8bf4161590 0x7f8bf41619a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:47.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf9a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4070300 con 0x7f8bd80099c0 2024-09-21T06:01:47.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf9a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4070180 con 0x7f8bf40b2200 2024-09-21T06:01:47.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf9a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf4070480 con 0x7f8bf4161590 2024-09-21T06:01:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd80099c0 0x7f8bf414ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd80099c0 0x7f8bf414ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51000/0 (socket says 172.21.15.50:51000) 2024-09-21T06:01:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf3e00640 1 -- 172.21.15.50:0/2735109072 learned_addr learned my addr 172.21.15.50:0/2735109072 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf9000640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8bf4161590 0x7f8bf41619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:47.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf3e00640 1 -- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8bf4161590 msgr2=0x7f8bf41619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:47.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf3e00640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8bf4161590 0x7f8bf41619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:47.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf3e00640 1 -- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bf40b2200 msgr2=0x7f8bf40b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:47.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf3e00640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bf40b2200 0x7f8bf40b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:47.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf3e00640 1 -- 172.21.15.50:0/2735109072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf415a9e0 con 0x7f8bd80099c0 2024-09-21T06:01:47.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf3e00640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd80099c0 0x7f8bf414ade0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00ca80 tx=0x7f8bec00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:47.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bebe00640 1 -- 172.21.15.50:0/2735109072 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec007d90 con 0x7f8bd80099c0 2024-09-21T06:01:47.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bebe00640 1 -- 172.21.15.50:0/2735109072 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bec003a90 con 0x7f8bd80099c0 2024-09-21T06:01:47.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf9a00640 1 -- 172.21.15.50:0/2735109072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf4161ee0 con 0x7f8bd80099c0 2024-09-21T06:01:47.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bf9a00640 1 -- 172.21.15.50:0/2735109072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf41598a0 con 0x7f8bd80099c0 2024-09-21T06:01:47.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.045+0000 7f8bebe00640 1 -- 172.21.15.50:0/2735109072 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec004020 con 0x7f8bd80099c0 2024-09-21T06:01:47.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.049+0000 7f8bf9a00640 1 -- 172.21.15.50:0/2735109072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc0052a0 con 0x7f8bd80099c0 2024-09-21T06:01:47.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.049+0000 7f8bebe00640 1 -- 172.21.15.50:0/2735109072 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 97897+0+0 (secure 0 0 0) 0x7f8bec0041c0 con 0x7f8bd80099c0 2024-09-21T06:01:47.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.053+0000 7f8bebe00640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8bd0073980 0x7f8bd0075e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:47.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.053+0000 7f8bebe00640 1 -- 172.21.15.50:0/2735109072 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f8bec096320 con 0x7f8bd80099c0 2024-09-21T06:01:47.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.057+0000 7f8bf3400640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8bd0073980 0x7f8bd0075e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:47.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.061+0000 7f8bebe00640 1 -- 172.21.15.50:0/2735109072 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8bf4161ee0 con 0x7f8bd80099c0 2024-09-21T06:01:47.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.061+0000 7f8bf3400640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8bd0073980 0x7f8bd0075e90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc000f70 tx=0x7f8bdc007a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:47.288 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:01:47.293 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:01:47.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.481+0000 7f8bf9a00640 1 -- 172.21.15.50:0/2735109072 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone33912", "target": ["mon-mgr", ""]}) -- 0x7f8bcc000bd0 con 0x7f8bd0073980 2024-09-21T06:01:47.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.933+0000 7f8bebe00640 1 -- 172.21.15.50:0/2735109072 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8bcc000bd0 con 0x7f8bd0073980 2024-09-21T06:01:47.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.937+0000 7f8bf9a00640 1 -- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8bd0073980 msgr2=0x7f8bd0075e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:47.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.937+0000 7f8bf9a00640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8bd0073980 0x7f8bd0075e90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc000f70 tx=0x7f8bdc007a60 comp rx=0 tx=0).stop 2024-09-21T06:01:47.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.937+0000 7f8bf9a00640 1 -- 172.21.15.50:0/2735109072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd80099c0 msgr2=0x7f8bf414ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:47.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.937+0000 7f8bf9a00640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd80099c0 0x7f8bf414ade0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00ca80 tx=0x7f8bec00cf40 comp rx=0 tx=0).stop 2024-09-21T06:01:47.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.937+0000 7f8bf9a00640 1 -- 172.21.15.50:0/2735109072 shutdown_connections 2024-09-21T06:01:47.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.937+0000 7f8bf9a00640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8bf4161590 0x7f8bf41619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:47.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.937+0000 7f8bf9a00640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bf40b2200 0x7f8bf40b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:47.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.937+0000 7f8bf9a00640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8bd0073980 0x7f8bd0075e90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:47.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.937+0000 7f8bf9a00640 1 --2- 172.21.15.50:0/2735109072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd80099c0 0x7f8bf414ade0 secure :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00ca80 tx=0x7f8bec00cf40 comp rx=0 tx=0).stop 2024-09-21T06:01:47.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.937+0000 7f8bf9a00640 1 -- 172.21.15.50:0/2735109072 >> 172.21.15.50:0/2735109072 conn(0x7f8bf406e4a0 msgr2=0x7f8bf40cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:47.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.937+0000 7f8bf9a00640 1 -- 172.21.15.50:0/2735109072 shutdown_connections 2024-09-21T06:01:47.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:47.937+0000 7f8bf9a00640 1 -- 172.21.15.50:0/2735109072 wait complete. 2024-09-21T06:01:47.986 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone30643 2024-09-21T06:01:48.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.253+0000 7f4adf200640 1 -- 172.21.15.50:0/4028670122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae015f5c0 msgr2=0x7f4ae015f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:48.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.253+0000 7f4adf200640 1 --2- 172.21.15.50:0/4028670122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae015f5c0 0x7f4ae015f9d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00ced20 tx=0x7f4ad801ccf0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.253+0000 7f4adf200640 1 -- 172.21.15.50:0/4028670122 shutdown_connections 2024-09-21T06:01:48.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.253+0000 7f4adf200640 1 --2- 172.21.15.50:0/4028670122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae015f5c0 0x7f4ae015f9d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.253+0000 7f4adf200640 1 --2- 172.21.15.50:0/4028670122 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ae006dbc0 0x7f4ae006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.253+0000 7f4adf200640 1 --2- 172.21.15.50:0/4028670122 >> v2:172.21.15.88:3300/0 conn(0x7f4ae00d1140 0x7f4ae00b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.253+0000 7f4adf200640 1 -- 172.21.15.50:0/4028670122 >> 172.21.15.50:0/4028670122 conn(0x7f4ae00b41f0 msgr2=0x7f4ae00b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:48.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.253+0000 7f4adf200640 1 -- 172.21.15.50:0/4028670122 shutdown_connections 2024-09-21T06:01:48.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.253+0000 7f4adf200640 1 -- 172.21.15.50:0/4028670122 wait complete. 2024-09-21T06:01:48.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.253+0000 7f4adf200640 1 Processor -- start 2024-09-21T06:01:48.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.253+0000 7f4adf200640 1 -- start start 2024-09-21T06:01:48.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4adf200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae006dbc0 0x7f4ae01f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:48.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4adf200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ae00d1140 0x7f4ae015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:48.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4adf200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ae015f5c0 0x7f4ae015d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:48.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4adf200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae0074dc0 con 0x7f4ae006dbc0 2024-09-21T06:01:48.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4adf200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae0074c40 con 0x7f4ae015f5c0 2024-09-21T06:01:48.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4adf200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae0074f40 con 0x7f4ae00d1140 2024-09-21T06:01:48.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4adde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae006dbc0 0x7f4ae01f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:48.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4add400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ae00d1140 0x7f4ae015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:48.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4adde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae006dbc0 0x7f4ae01f92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51014/0 (socket says 172.21.15.50:51014) 2024-09-21T06:01:48.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4adde00640 1 -- 172.21.15.50:0/2613990082 learned_addr learned my addr 172.21.15.50:0/2613990082 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:48.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4ade800640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ae015f5c0 0x7f4ae015d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:48.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4add400640 1 -- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ae015f5c0 msgr2=0x7f4ae015d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:48.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4add400640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ae015f5c0 0x7f4ae015d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4add400640 1 -- 172.21.15.50:0/2613990082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae006dbc0 msgr2=0x7f4ae01f92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:48.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4add400640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae006dbc0 0x7f4ae01f92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4add400640 1 -- 172.21.15.50:0/2613990082 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae015be80 con 0x7f4ae00d1140 2024-09-21T06:01:48.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4adde00640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae006dbc0 0x7f4ae01f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:01:48.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.257+0000 7f4add400640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ae00d1140 0x7f4ae015d3e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0072250 tx=0x7f4acc00bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:48.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.261+0000 7f4ad6000640 1 -- 172.21.15.50:0/2613990082 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc017070 con 0x7f4ae00d1140 2024-09-21T06:01:48.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.261+0000 7f4ad6000640 1 -- 172.21.15.50:0/2613990082 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4acc005330 con 0x7f4ae00d1140 2024-09-21T06:01:48.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.261+0000 7f4ad6000640 1 -- 172.21.15.50:0/2613990082 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc0047b0 con 0x7f4ae00d1140 2024-09-21T06:01:48.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.261+0000 7f4adf200640 1 -- 172.21.15.50:0/2613990082 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae015c160 con 0x7f4ae00d1140 2024-09-21T06:01:48.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.261+0000 7f4adf200640 1 -- 172.21.15.50:0/2613990082 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae01f97f0 con 0x7f4ae00d1140 2024-09-21T06:01:48.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.265+0000 7f4adf200640 1 -- 172.21.15.50:0/2613990082 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae00b0730 con 0x7f4ae00d1140 2024-09-21T06:01:48.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.265+0000 7f4ad6000640 1 -- 172.21.15.50:0/2613990082 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 18) ==== 97897+0+0 (secure 0 0 0) 0x7f4acc004950 con 0x7f4ae00d1140 2024-09-21T06:01:48.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.265+0000 7f4ad6000640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ac0073980 0x7f4ac0075e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:48.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.265+0000 7f4ad6000640 1 -- 172.21.15.50:0/2613990082 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f4acc095d30 con 0x7f4ae00d1140 2024-09-21T06:01:48.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.269+0000 7f4adde00640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ac0073980 0x7f4ac0075e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:48.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.269+0000 7f4adde00640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ac0073980 0x7f4ac0075e90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0011440 tx=0x7f4ad00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:48.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.273+0000 7f4ad6000640 1 -- 172.21.15.50:0/2613990082 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4acc05f780 con 0x7f4ae00d1140 2024-09-21T06:01:48.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.601+0000 7f4adf200640 1 -- 172.21.15.50:0/2613990082 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone30643", "target": ["mon-mgr", ""]}) -- 0x7f4ae006b540 con 0x7f4ac0073980 2024-09-21T06:01:48.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4ad6000640 1 -- 172.21.15.50:0/2613990082 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4ad0009c40 con 0x7f4ac0073980 2024-09-21T06:01:48.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4adf200640 1 -- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ac0073980 msgr2=0x7f4ac0075e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:48.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4adf200640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ac0073980 0x7f4ac0075e90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0011440 tx=0x7f4ad00091e0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4adf200640 1 -- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ae00d1140 msgr2=0x7f4ae015d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:48.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4adf200640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ae00d1140 0x7f4ae015d3e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0072250 tx=0x7f4acc00bf90 comp rx=0 tx=0).stop 2024-09-21T06:01:48.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4adf200640 1 -- 172.21.15.50:0/2613990082 shutdown_connections 2024-09-21T06:01:48.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4adf200640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ae015f5c0 0x7f4ae015d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4adf200640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ae00d1140 0x7f4ae015d3e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4adf200640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ac0073980 0x7f4ac0075e90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4adf200640 1 --2- 172.21.15.50:0/2613990082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae006dbc0 0x7f4ae01f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4adf200640 1 -- 172.21.15.50:0/2613990082 >> 172.21.15.50:0/2613990082 conn(0x7f4ae00b41f0 msgr2=0x7f4ae01563c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:48.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4adf200640 1 -- 172.21.15.50:0/2613990082 shutdown_connections 2024-09-21T06:01:48.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.785+0000 7f4adf200640 1 -- 172.21.15.50:0/2613990082 wait complete. 2024-09-21T06:01:48.808 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:01:48.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.969+0000 7f2027c00640 1 -- 172.21.15.50:0/1216985252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20201527b0 msgr2=0x7f2020074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:48.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.969+0000 7f2027c00640 1 --2- 172.21.15.50:0/1216985252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20201527b0 0x7f2020074e30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f200c009900 tx=0x7f200c01c820 comp rx=0 tx=0).stop 2024-09-21T06:01:48.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.969+0000 7f2027c00640 1 -- 172.21.15.50:0/1216985252 shutdown_connections 2024-09-21T06:01:48.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.969+0000 7f2027c00640 1 --2- 172.21.15.50:0/1216985252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20201527b0 0x7f2020074e30 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.969+0000 7f2027c00640 1 --2- 172.21.15.50:0/1216985252 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20200b5c70 0x7f202015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.969+0000 7f2027c00640 1 -- 172.21.15.50:0/1216985252 >> 172.21.15.50:0/1216985252 conn(0x7f20200b2040 msgr2=0x7f20200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:48.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.969+0000 7f2027c00640 1 -- 172.21.15.50:0/1216985252 shutdown_connections 2024-09-21T06:01:48.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.969+0000 7f2027c00640 1 -- 172.21.15.50:0/1216985252 wait complete. 2024-09-21T06:01:48.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.969+0000 7f2027c00640 1 Processor -- start 2024-09-21T06:01:48.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.969+0000 7f2027c00640 1 -- start start 2024-09-21T06:01:48.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f2027c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20200b4de0 0x7f20200ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:48.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f2027c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20200b5c70 0x7f20201413a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:48.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f2027c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20201527b0 0x7f20201418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:48.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f2027c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2020168bb0 con 0x7f20200b5c70 2024-09-21T06:01:48.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f2027c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2020168a30 con 0x7f20200b4de0 2024-09-21T06:01:48.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f2027c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2020168d30 con 0x7f20201527b0 2024-09-21T06:01:48.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20200b5c70 0x7f20201413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:48.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20200b5c70 0x7f20201413a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51030/0 (socket says 172.21.15.50:51030) 2024-09-21T06:01:48.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201fe00640 1 -- 172.21.15.50:0/380636634 learned_addr learned my addr 172.21.15.50:0/380636634 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:48.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f2025200640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20200b4de0 0x7f20200ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:48.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f2025c00640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20201527b0 0x7f20201418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:48.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201fe00640 1 -- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20201527b0 msgr2=0x7f20201418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:48.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201fe00640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20201527b0 0x7f20201418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201fe00640 1 -- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20200b4de0 msgr2=0x7f20200ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:48.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201fe00640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20200b4de0 0x7f20200ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:48.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201fe00640 1 -- 172.21.15.50:0/380636634 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20200ccaf0 con 0x7f20200b5c70 2024-09-21T06:01:48.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f2025200640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20200b4de0 0x7f20200ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:01:48.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201fe00640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20200b5c70 0x7f20201413a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f200c009850 tx=0x7f200c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:48.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201cc00640 1 -- 172.21.15.50:0/380636634 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0b2040 con 0x7f20200b5c70 2024-09-21T06:01:48.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201cc00640 1 -- 172.21.15.50:0/380636634 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f200c005030 con 0x7f20200b5c70 2024-09-21T06:01:48.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201cc00640 1 -- 172.21.15.50:0/380636634 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c005440 con 0x7f20200b5c70 2024-09-21T06:01:48.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f2027c00640 1 -- 172.21.15.50:0/380636634 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20200cdac0 con 0x7f20200b5c70 2024-09-21T06:01:48.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f2027c00640 1 -- 172.21.15.50:0/380636634 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2020141ee0 con 0x7f20200b5c70 2024-09-21T06:01:48.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f2027c00640 1 -- 172.21.15.50:0/380636634 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202015bb60 con 0x7f20200b5c70 2024-09-21T06:01:48.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.973+0000 7f201cc00640 1 -- 172.21.15.50:0/380636634 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 97897+0+0 (secure 0 0 0) 0x7f200c0046d0 con 0x7f20200b5c70 2024-09-21T06:01:48.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.977+0000 7f201cc00640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2000073950 0x7f2000075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:48.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.977+0000 7f201cc00640 1 -- 172.21.15.50:0/380636634 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f200c12f370 con 0x7f20200b5c70 2024-09-21T06:01:48.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.977+0000 7f2025200640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2000073950 0x7f2000075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:48.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.977+0000 7f2025200640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2000073950 0x7f2000075e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2014005fd0 tx=0x7f2014005f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:48.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:48.981+0000 7f201cc00640 1 -- 172.21.15.50:0/380636634 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2020075580 con 0x7f20200b5c70 2024-09-21T06:01:49.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.185+0000 7f2027c00640 1 -- 172.21.15.50:0/380636634 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f202006b540 con 0x7f2000073950 2024-09-21T06:01:49.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f201cc00640 1 -- 172.21.15.50:0/380636634 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f201400fb30 con 0x7f2000073950 2024-09-21T06:01:49.188 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:01:49.188 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:01:49.189 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:01:49.189 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:01:49.189 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:01:49.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f2027c00640 1 -- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2000073950 msgr2=0x7f2000075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:49.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f2027c00640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2000073950 0x7f2000075e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2014005fd0 tx=0x7f2014005f00 comp rx=0 tx=0).stop 2024-09-21T06:01:49.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f2027c00640 1 -- 172.21.15.50:0/380636634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20200b5c70 msgr2=0x7f20201413a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:49.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f2027c00640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20200b5c70 0x7f20201413a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f200c009850 tx=0x7f200c004040 comp rx=0 tx=0).stop 2024-09-21T06:01:49.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f2027c00640 1 -- 172.21.15.50:0/380636634 shutdown_connections 2024-09-21T06:01:49.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f2027c00640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20201527b0 0x7f20201418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f2027c00640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2000073950 0x7f2000075e60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f2027c00640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20200b5c70 0x7f20201413a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f2027c00640 1 --2- 172.21.15.50:0/380636634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20200b4de0 0x7f20200ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f2027c00640 1 -- 172.21.15.50:0/380636634 >> 172.21.15.50:0/380636634 conn(0x7f20200b2040 msgr2=0x7f2020151a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:49.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f2027c00640 1 -- 172.21.15.50:0/380636634 shutdown_connections 2024-09-21T06:01:49.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.189+0000 7f2027c00640 1 -- 172.21.15.50:0/380636634 wait complete. 2024-09-21T06:01:49.211 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-21T06:01:49.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.369+0000 7fca10c00640 1 -- 172.21.15.50:0/4210253981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca0c15d630 msgr2=0x7fca0c0cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:49.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.369+0000 7fca10c00640 1 --2- 172.21.15.50:0/4210253981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca0c15d630 0x7fca0c0cfea0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fca04009bc0 tx=0x7fca0401ca50 comp rx=0 tx=0).stop 2024-09-21T06:01:49.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.369+0000 7fca10c00640 1 -- 172.21.15.50:0/4210253981 shutdown_connections 2024-09-21T06:01:49.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.369+0000 7fca10c00640 1 --2- 172.21.15.50:0/4210253981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca0c15d630 0x7fca0c0cfea0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.369+0000 7fca10c00640 1 --2- 172.21.15.50:0/4210253981 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fca0c071940 0x7fca0c068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.369+0000 7fca10c00640 1 -- 172.21.15.50:0/4210253981 >> 172.21.15.50:0/4210253981 conn(0x7fca0c0b35f0 msgr2=0x7fca0c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:49.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.369+0000 7fca10c00640 1 -- 172.21.15.50:0/4210253981 shutdown_connections 2024-09-21T06:01:49.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.369+0000 7fca10c00640 1 -- 172.21.15.50:0/4210253981 wait complete. 2024-09-21T06:01:49.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca10c00640 1 Processor -- start 2024-09-21T06:01:49.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca10c00640 1 -- start start 2024-09-21T06:01:49.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca10c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca0c069330 0x7fca0c14ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:49.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca10c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fca0c071940 0x7fca0c156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:49.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca10c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca0c15d630 0x7fca0c1f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:49.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca10c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c169b60 con 0x7fca0c15d630 2024-09-21T06:01:49.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca10c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1699e0 con 0x7fca0c069330 2024-09-21T06:01:49.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca10c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fca0c169ce0 con 0x7fca0c071940 2024-09-21T06:01:49.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca0a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca0c15d630 0x7fca0c1f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:49.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca0a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca0c15d630 0x7fca0c1f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51046/0 (socket says 172.21.15.50:51046) 2024-09-21T06:01:49.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca0a800640 1 -- 172.21.15.50:0/3895546118 learned_addr learned my addr 172.21.15.50:0/3895546118 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:49.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca03e00640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fca0c071940 0x7fca0c156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:49.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca09e00640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca0c069330 0x7fca0c14ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:49.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca0a800640 1 -- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fca0c071940 msgr2=0x7fca0c156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:49.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca0a800640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fca0c071940 0x7fca0c156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca0a800640 1 -- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca0c069330 msgr2=0x7fca0c14ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:49.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca0a800640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca0c069330 0x7fca0c14ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca0a800640 1 -- 172.21.15.50:0/3895546118 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0c14e960 con 0x7fca0c15d630 2024-09-21T06:01:49.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca0a800640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca0c15d630 0x7fca0c1f6880 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fca0401a680 tx=0x7fca04003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:49.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca02000640 1 -- 172.21.15.50:0/3895546118 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca040bb070 con 0x7fca0c15d630 2024-09-21T06:01:49.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca02000640 1 -- 172.21.15.50:0/3895546118 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca04004e00 con 0x7fca0c15d630 2024-09-21T06:01:49.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca02000640 1 -- 172.21.15.50:0/3895546118 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca040b2040 con 0x7fca0c15d630 2024-09-21T06:01:49.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.373+0000 7fca10c00640 1 -- 172.21.15.50:0/3895546118 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0c14ec40 con 0x7fca0c15d630 2024-09-21T06:01:49.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.377+0000 7fca10c00640 1 -- 172.21.15.50:0/3895546118 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca0c14f200 con 0x7fca0c15d630 2024-09-21T06:01:49.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.377+0000 7fca10c00640 1 -- 172.21.15.50:0/3895546118 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca0c068df0 con 0x7fca0c15d630 2024-09-21T06:01:49.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.377+0000 7fca02000640 1 -- 172.21.15.50:0/3895546118 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 97897+0+0 (secure 0 0 0) 0x7fca04004230 con 0x7fca0c15d630 2024-09-21T06:01:49.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.377+0000 7fca02000640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc9e8073950 0x7fc9e8075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:49.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.377+0000 7fca02000640 1 -- 172.21.15.50:0/3895546118 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fca0412f520 con 0x7fca0c15d630 2024-09-21T06:01:49.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.377+0000 7fca09e00640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc9e8073950 0x7fc9e8075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:49.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.381+0000 7fca09e00640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc9e8073950 0x7fc9e8075e60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc004630 tx=0x7fc9fc00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:49.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.381+0000 7fca02000640 1 -- 172.21.15.50:0/3895546118 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fca040c4020 con 0x7fca0c15d630 2024-09-21T06:01:49.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.589+0000 7fca10c00640 1 -- 172.21.15.50:0/3895546118 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fca0c14e300 con 0x7fc9e8073950 2024-09-21T06:01:49.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.593+0000 7fca02000640 1 -- 172.21.15.50:0/3895546118 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+234 (secure 0 0 0) 0x7fca0c14e300 con 0x7fc9e8073950 2024-09-21T06:01:49.596 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:01:49.596 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:01:49.596 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:01:49.596 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sv1" 2024-09-21T06:01:49.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:01:49.596 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:01:49.596 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone20400" 2024-09-21T06:01:49.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:01:49.596 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:01:49.597 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone29661" 2024-09-21T06:01:49.597 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:01:49.597 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:01:49.597 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone33912" 2024-09-21T06:01:49.597 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:01:49.597 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:01:49.597 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone30643" 2024-09-21T06:01:49.597 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:01:49.597 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:01:49.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.597+0000 7fca10c00640 1 -- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc9e8073950 msgr2=0x7fc9e8075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:49.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.597+0000 7fca10c00640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc9e8073950 0x7fc9e8075e60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc004630 tx=0x7fc9fc00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.597+0000 7fca10c00640 1 -- 172.21.15.50:0/3895546118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca0c15d630 msgr2=0x7fca0c1f6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:49.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.597+0000 7fca10c00640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca0c15d630 0x7fca0c1f6880 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fca0401a680 tx=0x7fca04003f60 comp rx=0 tx=0).stop 2024-09-21T06:01:49.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.597+0000 7fca10c00640 1 -- 172.21.15.50:0/3895546118 shutdown_connections 2024-09-21T06:01:49.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.597+0000 7fca10c00640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc9e8073950 0x7fc9e8075e60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.597+0000 7fca10c00640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca0c15d630 0x7fca0c1f6880 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.597+0000 7fca10c00640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fca0c071940 0x7fca0c156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.597+0000 7fca10c00640 1 --2- 172.21.15.50:0/3895546118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca0c069330 0x7fca0c14ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.597+0000 7fca10c00640 1 -- 172.21.15.50:0/3895546118 >> 172.21.15.50:0/3895546118 conn(0x7fca0c0b35f0 msgr2=0x7fca0c0760d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:49.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.597+0000 7fca10c00640 1 -- 172.21.15.50:0/3895546118 shutdown_connections 2024-09-21T06:01:49.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.597+0000 7fca10c00640 1 -- 172.21.15.50:0/3895546118 wait complete. 2024-09-21T06:01:49.619 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-21T06:01:49.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.781+0000 7f7a32400640 1 -- 172.21.15.50:0/4236288358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a2c0b4de0 msgr2=0x7f7a2c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:49.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.781+0000 7f7a32400640 1 --2- 172.21.15.50:0/4236288358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a2c0b4de0 0x7f7a2c0b51f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009870 tx=0x7f7a1401c710 comp rx=0 tx=0).stop 2024-09-21T06:01:49.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.781+0000 7f7a32400640 1 -- 172.21.15.50:0/4236288358 shutdown_connections 2024-09-21T06:01:49.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.781+0000 7f7a32400640 1 --2- 172.21.15.50:0/4236288358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a2c0b4de0 0x7f7a2c0b51f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.781+0000 7f7a32400640 1 --2- 172.21.15.50:0/4236288358 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a2c0738b0 0x7f7a2c075010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.781+0000 7f7a32400640 1 -- 172.21.15.50:0/4236288358 >> 172.21.15.50:0/4236288358 conn(0x7f7a2c0b2040 msgr2=0x7f7a2c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:49.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.781+0000 7f7a32400640 1 -- 172.21.15.50:0/4236288358 shutdown_connections 2024-09-21T06:01:49.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.781+0000 7f7a32400640 1 -- 172.21.15.50:0/4236288358 wait complete. 2024-09-21T06:01:49.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a32400640 1 Processor -- start 2024-09-21T06:01:49.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a32400640 1 -- start start 2024-09-21T06:01:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a32400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a2c0738b0 0x7f7a2c139c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a32400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a2c0b4de0 0x7f7a2c156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a32400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c15cc60 0x7f7a2c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a32400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c168b20 con 0x7f7a2c15cc60 2024-09-21T06:01:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a32400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c152a50 con 0x7f7a2c0b4de0 2024-09-21T06:01:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a32400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7a2c168ca0 con 0x7f7a2c0738b0 2024-09-21T06:01:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c15cc60 0x7f7a2c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c15cc60 0x7f7a2c1568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51052/0 (socket says 172.21.15.50:51052) 2024-09-21T06:01:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a2be00640 1 -- 172.21.15.50:0/2890512406 learned_addr learned my addr 172.21.15.50:0/2890512406 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a2b400640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a2c0738b0 0x7f7a2c139c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a23e00640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a2c0b4de0 0x7f7a2c156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a2be00640 1 -- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a2c0738b0 msgr2=0x7f7a2c139c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:49.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a2be00640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a2c0738b0 0x7f7a2c139c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a2be00640 1 -- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a2c0b4de0 msgr2=0x7f7a2c156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:49.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a2be00640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a2c0b4de0 0x7f7a2c156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:49.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a2be00640 1 -- 172.21.15.50:0/2890512406 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c0ced50 con 0x7f7a2c15cc60 2024-09-21T06:01:49.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a2be00640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c15cc60 0x7f7a2c1568c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400cbc0 tx=0x7f7a2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:49.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a23400640 1 -- 172.21.15.50:0/2890512406 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24016020 con 0x7f7a2c15cc60 2024-09-21T06:01:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a23400640 1 -- 172.21.15.50:0/2890512406 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a2400ab00 con 0x7f7a2c15cc60 2024-09-21T06:01:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a32400640 1 -- 172.21.15.50:0/2890512406 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c0cf030 con 0x7f7a2c15cc60 2024-09-21T06:01:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a23400640 1 -- 172.21.15.50:0/2890512406 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24012420 con 0x7f7a2c15cc60 2024-09-21T06:01:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.785+0000 7f7a32400640 1 -- 172.21.15.50:0/2890512406 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c0cf5f0 con 0x7f7a2c15cc60 2024-09-21T06:01:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.789+0000 7f7a32400640 1 -- 172.21.15.50:0/2890512406 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c0ceaa0 con 0x7f7a2c15cc60 2024-09-21T06:01:49.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.789+0000 7f7a23400640 1 -- 172.21.15.50:0/2890512406 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 97897+0+0 (secure 0 0 0) 0x7f7a24019020 con 0x7f7a2c15cc60 2024-09-21T06:01:49.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.789+0000 7f7a23400640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a08073980 0x7f7a08075e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:49.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.789+0000 7f7a23400640 1 -- 172.21.15.50:0/2890512406 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f7a24096190 con 0x7f7a2c15cc60 2024-09-21T06:01:49.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.789+0000 7f7a2b400640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a08073980 0x7f7a08075e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.789+0000 7f7a2b400640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a08073980 0x7f7a08075e90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0b6600 tx=0x7f7a1c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:49.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.793+0000 7f7a23400640 1 -- 172.21.15.50:0/2890512406 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7a24020020 con 0x7f7a2c15cc60 2024-09-21T06:01:50.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:49.997+0000 7f7a32400640 1 -- 172.21.15.50:0/2890512406 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a2c06b540 con 0x7f7a08073980 2024-09-21T06:01:50.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.013+0000 7f7a23400640 1 -- 172.21.15.50:0/2890512406 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7a1c011bb0 con 0x7f7a08073980 2024-09-21T06:01:50.018 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:01:50.018 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:01:50.018 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:01:50.018 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ss1" 2024-09-21T06:01:50.018 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:01:50.018 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:01:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.013+0000 7f7a32400640 1 -- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a08073980 msgr2=0x7f7a08075e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.013+0000 7f7a32400640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a08073980 0x7f7a08075e90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0b6600 tx=0x7f7a1c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.013+0000 7f7a32400640 1 -- 172.21.15.50:0/2890512406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c15cc60 msgr2=0x7f7a2c1568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.013+0000 7f7a32400640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c15cc60 0x7f7a2c1568c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400cbc0 tx=0x7f7a2400ccb0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.017+0000 7f7a32400640 1 -- 172.21.15.50:0/2890512406 shutdown_connections 2024-09-21T06:01:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.017+0000 7f7a32400640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a08073980 0x7f7a08075e90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.017+0000 7f7a32400640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c15cc60 0x7f7a2c1568c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.017+0000 7f7a32400640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a2c0b4de0 0x7f7a2c156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.017+0000 7f7a32400640 1 --2- 172.21.15.50:0/2890512406 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a2c0738b0 0x7f7a2c139c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.017+0000 7f7a32400640 1 -- 172.21.15.50:0/2890512406 >> 172.21.15.50:0/2890512406 conn(0x7f7a2c0b2040 msgr2=0x7f7a2c15d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.017+0000 7f7a32400640 1 -- 172.21.15.50:0/2890512406 shutdown_connections 2024-09-21T06:01:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.017+0000 7f7a32400640 1 -- 172.21.15.50:0/2890512406 wait complete. 2024-09-21T06:01:50.037 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-21T06:01:50.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.201+0000 7f262a400640 1 -- 172.21.15.50:0/764187104 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2624073f90 msgr2=0x7f26240743a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.201+0000 7f262a400640 1 --2- 172.21.15.50:0/764187104 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2624073f90 0x7f26240743a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f26240720a0 tx=0x7f260c01c800 comp rx=0 tx=0).stop 2024-09-21T06:01:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.201+0000 7f262a400640 1 -- 172.21.15.50:0/764187104 shutdown_connections 2024-09-21T06:01:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.201+0000 7f262a400640 1 --2- 172.21.15.50:0/764187104 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2624073f90 0x7f26240743a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.201+0000 7f262a400640 1 --2- 172.21.15.50:0/764187104 >> v2:172.21.15.88:3300/0 conn(0x7f262415c730 0x7f26240b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.201+0000 7f262a400640 1 -- 172.21.15.50:0/764187104 >> 172.21.15.50:0/764187104 conn(0x7f26240d06e0 msgr2=0x7f26240d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.201+0000 7f262a400640 1 -- 172.21.15.50:0/764187104 shutdown_connections 2024-09-21T06:01:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.201+0000 7f262a400640 1 -- 172.21.15.50:0/764187104 wait complete. 2024-09-21T06:01:50.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.201+0000 7f262a400640 1 Processor -- start 2024-09-21T06:01:50.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.201+0000 7f262a400640 1 -- start start 2024-09-21T06:01:50.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f262a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073f90 0x7f2624151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:50.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f262a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26240af520 0x7f26240ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:50.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f262a400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f262415c730 0x7f26241f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f262a400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f262406f680 con 0x7f26240af520 2024-09-21T06:01:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f262a400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f262406f500 con 0x7f2624073f90 2024-09-21T06:01:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f262a400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f262406f800 con 0x7f262415c730 2024-09-21T06:01:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f261be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26240af520 0x7f26240ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f261be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26240af520 0x7f26240ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51070/0 (socket says 172.21.15.50:51070) 2024-09-21T06:01:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f261be00640 1 -- 172.21.15.50:0/2533180394 learned_addr learned my addr 172.21.15.50:0/2533180394 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f2623e00640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f262415c730 0x7f26241f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f2623400640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073f90 0x7f2624151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:50.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f261be00640 1 -- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f262415c730 msgr2=0x7f26241f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:50.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f261be00640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f262415c730 0x7f26241f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f261be00640 1 -- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073f90 msgr2=0x7f2624151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:50.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f261be00640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073f90 0x7f2624151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f261be00640 1 -- 172.21.15.50:0/2533180394 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26240d1030 con 0x7f26240af520 2024-09-21T06:01:50.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f2623400640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073f90 0x7f2624151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:01:50.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f261be00640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26240af520 0x7f26240ad5d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f260c000c10 tx=0x7f260c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:50.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f261b400640 1 -- 172.21.15.50:0/2533180394 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c0042f0 con 0x7f26240af520 2024-09-21T06:01:50.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f261b400640 1 -- 172.21.15.50:0/2533180394 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f260c0bd070 con 0x7f26240af520 2024-09-21T06:01:50.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f262a400640 1 -- 172.21.15.50:0/2533180394 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2624150770 con 0x7f26240af520 2024-09-21T06:01:50.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f261b400640 1 -- 172.21.15.50:0/2533180394 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c005270 con 0x7f26240af520 2024-09-21T06:01:50.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f262a400640 1 -- 172.21.15.50:0/2533180394 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2624150d30 con 0x7f26240af520 2024-09-21T06:01:50.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.205+0000 7f262a400640 1 -- 172.21.15.50:0/2533180394 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262406fcd0 con 0x7f26240af520 2024-09-21T06:01:50.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.209+0000 7f261b400640 1 -- 172.21.15.50:0/2533180394 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 97897+0+0 (secure 0 0 0) 0x7f260c005410 con 0x7f26240af520 2024-09-21T06:01:50.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.209+0000 7f261b400640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2600073980 0x7f2600075e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:50.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.209+0000 7f261b400640 1 -- 172.21.15.50:0/2533180394 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f260c12f630 con 0x7f26240af520 2024-09-21T06:01:50.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.209+0000 7f2623400640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2600073980 0x7f2600075e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:50.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.209+0000 7f2623400640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2600073980 0x7f2600075e90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2614005fd0 tx=0x7f261400a540 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.213+0000 7f261b400640 1 -- 172.21.15.50:0/2533180394 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f260c0c6020 con 0x7f26240af520 2024-09-21T06:01:50.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.425+0000 7f262a400640 1 -- 172.21.15.50:0/2533180394 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f262406b540 con 0x7f2600073980 2024-09-21T06:01:50.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.457+0000 7f261b400640 1 -- 172.21.15.50:0/2533180394 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f261400eb50 con 0x7f2600073980 2024-09-21T06:01:50.458 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:01:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.457+0000 7f262a400640 1 -- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2600073980 msgr2=0x7f2600075e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.457+0000 7f262a400640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2600073980 0x7f2600075e90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2614005fd0 tx=0x7f261400a540 comp rx=0 tx=0).stop 2024-09-21T06:01:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.457+0000 7f262a400640 1 -- 172.21.15.50:0/2533180394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26240af520 msgr2=0x7f26240ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.457+0000 7f262a400640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26240af520 0x7f26240ad5d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f260c000c10 tx=0x7f260c004040 comp rx=0 tx=0).stop 2024-09-21T06:01:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.457+0000 7f262a400640 1 -- 172.21.15.50:0/2533180394 shutdown_connections 2024-09-21T06:01:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.457+0000 7f262a400640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f262415c730 0x7f26241f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.457+0000 7f262a400640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2600073980 0x7f2600075e90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.457+0000 7f262a400640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26240af520 0x7f26240ad5d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.457+0000 7f262a400640 1 --2- 172.21.15.50:0/2533180394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073f90 0x7f2624151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.457+0000 7f262a400640 1 -- 172.21.15.50:0/2533180394 >> 172.21.15.50:0/2533180394 conn(0x7f26240d06e0 msgr2=0x7f262415cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.457+0000 7f262a400640 1 -- 172.21.15.50:0/2533180394 shutdown_connections 2024-09-21T06:01:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.461+0000 7f262a400640 1 -- 172.21.15.50:0/2533180394 wait complete. 2024-09-21T06:01:50.481 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-21T06:01:50.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.645+0000 7f386e800640 1 -- 172.21.15.50:0/812401995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38680749a0 msgr2=0x7f3868074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:50.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.645+0000 7f386e800640 1 --2- 172.21.15.50:0/812401995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38680749a0 0x7f3868074db0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f38680726f0 tx=0x7f385801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.645+0000 7f386e800640 1 -- 172.21.15.50:0/812401995 shutdown_connections 2024-09-21T06:01:50.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.645+0000 7f386e800640 1 --2- 172.21.15.50:0/812401995 >> v2:172.21.15.88:3300/0 conn(0x7f38680cefa0 0x7f38680cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.645+0000 7f386e800640 1 --2- 172.21.15.50:0/812401995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38680749a0 0x7f3868074db0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.645+0000 7f386e800640 1 -- 172.21.15.50:0/812401995 >> 172.21.15.50:0/812401995 conn(0x7f38680b2040 msgr2=0x7f38680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:50.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.645+0000 7f386e800640 1 -- 172.21.15.50:0/812401995 shutdown_connections 2024-09-21T06:01:50.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.645+0000 7f386e800640 1 -- 172.21.15.50:0/812401995 wait complete. 2024-09-21T06:01:50.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.645+0000 7f386e800640 1 Processor -- start 2024-09-21T06:01:50.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.645+0000 7f386e800640 1 -- start start 2024-09-21T06:01:50.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f386e800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f38680749a0 0x7f3868159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f386e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38680cefa0 0x7f38681413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f386e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3868160330 0x7f3868158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f386e800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f386816a060 con 0x7f38680cefa0 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f386e800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3868169ee0 con 0x7f3868160330 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f386e800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f386816a1e0 con 0x7f38680749a0 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f385fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38680cefa0 0x7f38681413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f385fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38680cefa0 0x7f38681413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51090/0 (socket says 172.21.15.50:51090) 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f385fe00640 1 -- 172.21.15.50:0/919675886 learned_addr learned my addr 172.21.15.50:0/919675886 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f3867e00640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3868160330 0x7f3868158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f3867400640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f38680749a0 0x7f3868159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f385fe00640 1 -- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f38680749a0 msgr2=0x7f3868159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f385fe00640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f38680749a0 0x7f3868159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f385fe00640 1 -- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3868160330 msgr2=0x7f3868158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:50.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f385fe00640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3868160330 0x7f3868158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f385fe00640 1 -- 172.21.15.50:0/919675886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3868158f70 con 0x7f38680cefa0 2024-09-21T06:01:50.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f385fe00640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38680cefa0 0x7f38681413c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f385000e7f0 tx=0x7f385000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:50.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f385f400640 1 -- 172.21.15.50:0/919675886 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385000cc60 con 0x7f38680cefa0 2024-09-21T06:01:50.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f385f400640 1 -- 172.21.15.50:0/919675886 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f385000ce00 con 0x7f38680cefa0 2024-09-21T06:01:50.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f385f400640 1 -- 172.21.15.50:0/919675886 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850005280 con 0x7f38680cefa0 2024-09-21T06:01:50.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f386e800640 1 -- 172.21.15.50:0/919675886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386815a350 con 0x7f38680cefa0 2024-09-21T06:01:50.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f386e800640 1 -- 172.21.15.50:0/919675886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f386813d340 con 0x7f38680cefa0 2024-09-21T06:01:50.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.649+0000 7f386e800640 1 -- 172.21.15.50:0/919675886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3868074db0 con 0x7f38680cefa0 2024-09-21T06:01:50.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.653+0000 7f385f400640 1 -- 172.21.15.50:0/919675886 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 97897+0+0 (secure 0 0 0) 0x7f3850005420 con 0x7f38680cefa0 2024-09-21T06:01:50.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.653+0000 7f385f400640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3844073950 0x7f3844075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:50.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.653+0000 7f385f400640 1 -- 172.21.15.50:0/919675886 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f3850095ca0 con 0x7f38680cefa0 2024-09-21T06:01:50.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.653+0000 7f3867400640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3844073950 0x7f3844075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:50.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.653+0000 7f3867400640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3844073950 0x7f3844075e60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f38580af000 tx=0x7f385801b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:50.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.657+0000 7f385f400640 1 -- 172.21.15.50:0/919675886 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f385005f670 con 0x7f38680cefa0 2024-09-21T06:01:50.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.857+0000 7f386e800640 1 -- 172.21.15.50:0/919675886 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f386806b540 con 0x7f3844073950 2024-09-21T06:01:50.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f385f400640 1 -- 172.21.15.50:0/919675886 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3858005de0 con 0x7f3844073950 2024-09-21T06:01:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f386e800640 1 -- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3844073950 msgr2=0x7f3844075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f386e800640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3844073950 0x7f3844075e60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f38580af000 tx=0x7f385801b100 comp rx=0 tx=0).stop 2024-09-21T06:01:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f386e800640 1 -- 172.21.15.50:0/919675886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38680cefa0 msgr2=0x7f38681413c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f386e800640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38680cefa0 0x7f38681413c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f385000e7f0 tx=0x7f385000ecb0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f386e800640 1 -- 172.21.15.50:0/919675886 shutdown_connections 2024-09-21T06:01:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f386e800640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3868160330 0x7f3868158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f386e800640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3844073950 0x7f3844075e60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f386e800640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38680cefa0 0x7f38681413c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f386e800640 1 --2- 172.21.15.50:0/919675886 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f38680749a0 0x7f3868159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:50.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f386e800640 1 -- 172.21.15.50:0/919675886 >> 172.21.15.50:0/919675886 conn(0x7f38680b2040 msgr2=0x7f386813fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:50.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f386e800640 1 -- 172.21.15.50:0/919675886 shutdown_connections 2024-09-21T06:01:50.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:50.889+0000 7f386e800640 1 -- 172.21.15.50:0/919675886 wait complete. 2024-09-21T06:01:50.893 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:50.892+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:50.893 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:50.892+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:50.894 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:50.892+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:50.894 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:50.892+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:50.894 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:50.892+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:50.914 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone20400 2024-09-21T06:01:50.914 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:50.912+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:50.914 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:50.912+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:50.914 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:50.912+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:50.915 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:50.912+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:50.915 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:50.912+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.081+0000 7f0799800640 1 -- 172.21.15.50:0/1860326296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07940703a0 msgr2=0x7f079406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.081+0000 7f0799800640 1 --2- 172.21.15.50:0/1860326296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07940703a0 0x7f079406d200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f078c009bc0 tx=0x7f078c01ca30 comp rx=0 tx=0).stop 2024-09-21T06:01:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.081+0000 7f0799800640 1 -- 172.21.15.50:0/1860326296 shutdown_connections 2024-09-21T06:01:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.081+0000 7f0799800640 1 --2- 172.21.15.50:0/1860326296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07940703a0 0x7f079406d200 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.081+0000 7f0799800640 1 -- 172.21.15.50:0/1860326296 >> 172.21.15.50:0/1860326296 conn(0x7f07940b35f0 msgr2=0x7f07940b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.081+0000 7f0799800640 1 -- 172.21.15.50:0/1860326296 shutdown_connections 2024-09-21T06:01:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.081+0000 7f0799800640 1 -- 172.21.15.50:0/1860326296 wait complete. 2024-09-21T06:01:51.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.081+0000 7f0799800640 1 Processor -- start 2024-09-21T06:01:51.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.081+0000 7f0799800640 1 -- start start 2024-09-21T06:01:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0799800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f079406cdf0 0x7f0794143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0799800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07940703a0 0x7f079415cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0799800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0794159130 0x7f079415d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0799800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f079416a4a0 con 0x7f0794159130 2024-09-21T06:01:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0799800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f079416a320 con 0x7f07940703a0 2024-09-21T06:01:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0799800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f079416a620 con 0x7f079406cdf0 2024-09-21T06:01:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0793400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0794159130 0x7f079415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0793400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0794159130 0x7f079415d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51100/0 (socket says 172.21.15.50:51100) 2024-09-21T06:01:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0792a00640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f079406cdf0 0x7f0794143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0793400640 1 -- 172.21.15.50:0/2072996481 learned_addr learned my addr 172.21.15.50:0/2072996481 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f078be00640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07940703a0 0x7f079415cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0793400640 1 -- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f079406cdf0 msgr2=0x7f0794143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0793400640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f079406cdf0 0x7f0794143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0793400640 1 -- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07940703a0 msgr2=0x7f079415cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0793400640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07940703a0 0x7f079415cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0793400640 1 -- 172.21.15.50:0/2072996481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07941441d0 con 0x7f0794159130 2024-09-21T06:01:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0792a00640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f079406cdf0 0x7f0794143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:01:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0793400640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0794159130 0x7f079415d270 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f078c019640 tx=0x7f078c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f078aa00640 1 -- 172.21.15.50:0/2072996481 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0bb070 con 0x7f0794159130 2024-09-21T06:01:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0799800640 1 -- 172.21.15.50:0/2072996481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0794156b30 con 0x7f0794159130 2024-09-21T06:01:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f078aa00640 1 -- 172.21.15.50:0/2072996481 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f078c004d40 con 0x7f0794159130 2024-09-21T06:01:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f078aa00640 1 -- 172.21.15.50:0/2072996481 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0b2040 con 0x7f0794159130 2024-09-21T06:01:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0799800640 1 -- 172.21.15.50:0/2072996481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f079415e0c0 con 0x7f0794159130 2024-09-21T06:01:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.085+0000 7f0799800640 1 -- 172.21.15.50:0/2072996481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07941568a0 con 0x7f0794159130 2024-09-21T06:01:51.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.089+0000 7f078aa00640 1 -- 172.21.15.50:0/2072996481 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 97897+0+0 (secure 0 0 0) 0x7f078c0a5b60 con 0x7f0794159130 2024-09-21T06:01:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.089+0000 7f078aa00640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0770073950 0x7f0770075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.089+0000 7f078aa00640 1 -- 172.21.15.50:0/2072996481 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f078c12ea40 con 0x7f0794159130 2024-09-21T06:01:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.089+0000 7f0792a00640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0770073950 0x7f0770075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.089+0000 7f0792a00640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0770073950 0x7f0770075e60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0784007fd0 tx=0x7f0784009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.093+0000 7f078aa00640 1 -- 172.21.15.50:0/2072996481 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f078c0166b0 con 0x7f0794159130 2024-09-21T06:01:51.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.297+0000 7f0799800640 1 -- 172.21.15.50:0/2072996481 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone20400", "target": ["mon-mgr", ""]}) -- 0x7f079406b540 con 0x7f0770073950 2024-09-21T06:01:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f078aa00640 1 -- 172.21.15.50:0/2072996481 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0784001cf0 con 0x7f0770073950 2024-09-21T06:01:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f0799800640 1 -- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0770073950 msgr2=0x7f0770075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f0799800640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0770073950 0x7f0770075e60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0784007fd0 tx=0x7f0784009040 comp rx=0 tx=0).stop 2024-09-21T06:01:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f0799800640 1 -- 172.21.15.50:0/2072996481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0794159130 msgr2=0x7f079415d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f0799800640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0794159130 0x7f079415d270 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f078c019640 tx=0x7f078c003f60 comp rx=0 tx=0).stop 2024-09-21T06:01:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f0799800640 1 -- 172.21.15.50:0/2072996481 shutdown_connections 2024-09-21T06:01:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f0799800640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0770073950 0x7f0770075e60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f0799800640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0794159130 0x7f079415d270 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f0799800640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07940703a0 0x7f079415cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f0799800640 1 --2- 172.21.15.50:0/2072996481 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f079406cdf0 0x7f0794143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f0799800640 1 -- 172.21.15.50:0/2072996481 >> 172.21.15.50:0/2072996481 conn(0x7f07940b35f0 msgr2=0x7f0794155a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f0799800640 1 -- 172.21.15.50:0/2072996481 shutdown_connections 2024-09-21T06:01:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.313+0000 7f0799800640 1 -- 172.21.15.50:0/2072996481 wait complete. 2024-09-21T06:01:51.325 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:51.324+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:51.325 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:51.324+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:51.325 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:51.324+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:51.325 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:51.324+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:51.325 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:01:51.324+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:01:51.339 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone29661 2024-09-21T06:01:51.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.505+0000 7fe9e8c00640 1 -- 172.21.15.50:0/97712838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9e406cdf0 msgr2=0x7fe9e406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.505+0000 7fe9e8c00640 1 --2- 172.21.15.50:0/97712838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9e406cdf0 0x7fe9e406d200 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40ce170 tx=0x7fe9cc01c830 comp rx=0 tx=0).stop 2024-09-21T06:01:51.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.505+0000 7fe9e8c00640 1 -- 172.21.15.50:0/97712838 shutdown_connections 2024-09-21T06:01:51.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.505+0000 7fe9e8c00640 1 --2- 172.21.15.50:0/97712838 >> v2:172.21.15.88:3300/0 conn(0x7fe9e406fc90 0x7fe9e40700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.505+0000 7fe9e8c00640 1 --2- 172.21.15.50:0/97712838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9e406cdf0 0x7fe9e406d200 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.505+0000 7fe9e8c00640 1 -- 172.21.15.50:0/97712838 >> 172.21.15.50:0/97712838 conn(0x7fe9e40b35f0 msgr2=0x7fe9e40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:51.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.509+0000 7fe9e8c00640 1 -- 172.21.15.50:0/97712838 shutdown_connections 2024-09-21T06:01:51.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.509+0000 7fe9e8c00640 1 -- 172.21.15.50:0/97712838 wait complete. 2024-09-21T06:01:51.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.509+0000 7fe9e8c00640 1 Processor -- start 2024-09-21T06:01:51.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.509+0000 7fe9e8c00640 1 -- start start 2024-09-21T06:01:51.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.509+0000 7fe9e8c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe9e406cdf0 0x7fe9e41f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:51.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.509+0000 7fe9e8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9e406fc90 0x7fe9e415cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:51.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.509+0000 7fe9e8c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9e4159130 0x7fe9e40cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:51.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.509+0000 7fe9e8c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e4168d00 con 0x7fe9e406fc90 2024-09-21T06:01:51.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.509+0000 7fe9e8c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e4168b80 con 0x7fe9e4159130 2024-09-21T06:01:51.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.509+0000 7fe9e8c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e4168e80 con 0x7fe9e406cdf0 2024-09-21T06:01:51.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9dbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9e406fc90 0x7fe9e415cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:51.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9dbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9e406fc90 0x7fe9e415cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51116/0 (socket says 172.21.15.50:51116) 2024-09-21T06:01:51.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9dbe00640 1 -- 172.21.15.50:0/1324378718 learned_addr learned my addr 172.21.15.50:0/1324378718 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:51.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9e1e00640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe9e406cdf0 0x7fe9e41f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:51.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9e2800640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9e4159130 0x7fe9e40cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:51.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9dbe00640 1 -- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe9e406cdf0 msgr2=0x7fe9e41f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9dbe00640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe9e406cdf0 0x7fe9e41f9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9dbe00640 1 -- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9e4159130 msgr2=0x7fe9e40cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9dbe00640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9e4159130 0x7fe9e40cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9dbe00640 1 -- 172.21.15.50:0/1324378718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e40ce040 con 0x7fe9e406fc90 2024-09-21T06:01:51.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9dbe00640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9e406fc90 0x7fe9e415cd30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009230 tx=0x7fe9cc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:51.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9da000640 1 -- 172.21.15.50:0/1324378718 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc0044b0 con 0x7fe9e406fc90 2024-09-21T06:01:51.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9da000640 1 -- 172.21.15.50:0/1324378718 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9cc0bd070 con 0x7fe9e406fc90 2024-09-21T06:01:51.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9da000640 1 -- 172.21.15.50:0/1324378718 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc005360 con 0x7fe9e406fc90 2024-09-21T06:01:51.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9e8c00640 1 -- 172.21.15.50:0/1324378718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e40cdd60 con 0x7fe9e406fc90 2024-09-21T06:01:51.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9e8c00640 1 -- 172.21.15.50:0/1324378718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e40cea70 con 0x7fe9e406fc90 2024-09-21T06:01:51.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.513+0000 7fe9e8c00640 1 -- 172.21.15.50:0/1324378718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e406d200 con 0x7fe9e406fc90 2024-09-21T06:01:51.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.517+0000 7fe9da000640 1 -- 172.21.15.50:0/1324378718 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 97999+0+0 (secure 0 0 0) 0x7fe9cc005500 con 0x7fe9e406fc90 2024-09-21T06:01:51.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.517+0000 7fe9da000640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe9c0073a70 0x7fe9c0075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:51.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.517+0000 7fe9da000640 1 -- 172.21.15.50:0/1324378718 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fe9cc12ded0 con 0x7fe9e406fc90 2024-09-21T06:01:51.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.517+0000 7fe9e1e00640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe9c0073a70 0x7fe9c0075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:51.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.517+0000 7fe9e1e00640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe9c0073a70 0x7fe9c0075f80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4007fd0 tx=0x7fe9d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:51.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.521+0000 7fe9da000640 1 -- 172.21.15.50:0/1324378718 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe9cc0c6020 con 0x7fe9e406fc90 2024-09-21T06:01:51.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.725+0000 7fe9e8c00640 1 -- 172.21.15.50:0/1324378718 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone29661", "target": ["mon-mgr", ""]}) -- 0x7fe9e40cdb10 con 0x7fe9c0073a70 2024-09-21T06:01:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.749+0000 7fe9da000640 1 -- 172.21.15.50:0/1324378718 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe9e40cdb10 con 0x7fe9c0073a70 2024-09-21T06:01:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.753+0000 7fe9e8c00640 1 -- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe9c0073a70 msgr2=0x7fe9c0075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.753+0000 7fe9e8c00640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe9c0073a70 0x7fe9c0075f80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4007fd0 tx=0x7fe9d4009040 comp rx=0 tx=0).stop 2024-09-21T06:01:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.753+0000 7fe9e8c00640 1 -- 172.21.15.50:0/1324378718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9e406fc90 msgr2=0x7fe9e415cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.753+0000 7fe9e8c00640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9e406fc90 0x7fe9e415cd30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009230 tx=0x7fe9cc0040c0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.753+0000 7fe9e8c00640 1 -- 172.21.15.50:0/1324378718 shutdown_connections 2024-09-21T06:01:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.753+0000 7fe9e8c00640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9e4159130 0x7fe9e40cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.753+0000 7fe9e8c00640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe9c0073a70 0x7fe9c0075f80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.753+0000 7fe9e8c00640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9e406fc90 0x7fe9e415cd30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.753+0000 7fe9e8c00640 1 --2- 172.21.15.50:0/1324378718 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe9e406cdf0 0x7fe9e41f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.753+0000 7fe9e8c00640 1 -- 172.21.15.50:0/1324378718 >> 172.21.15.50:0/1324378718 conn(0x7fe9e40b35f0 msgr2=0x7fe9e40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.753+0000 7fe9e8c00640 1 -- 172.21.15.50:0/1324378718 shutdown_connections 2024-09-21T06:01:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.753+0000 7fe9e8c00640 1 -- 172.21.15.50:0/1324378718 wait complete. 2024-09-21T06:01:51.776 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone33912 2024-09-21T06:01:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.941+0000 7f2147800640 1 -- 172.21.15.50:0/2710220915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21400b5c70 msgr2=0x7f214015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.941+0000 7f2147800640 1 --2- 172.21.15.50:0/2710220915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21400b5c70 0x7f214015bb60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2134009900 tx=0x7f213401c820 comp rx=0 tx=0).stop 2024-09-21T06:01:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.941+0000 7f2147800640 1 -- 172.21.15.50:0/2710220915 shutdown_connections 2024-09-21T06:01:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.941+0000 7f2147800640 1 --2- 172.21.15.50:0/2710220915 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21401527b0 0x7f2140074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.941+0000 7f2147800640 1 --2- 172.21.15.50:0/2710220915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21400b5c70 0x7f214015bb60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.941+0000 7f2147800640 1 -- 172.21.15.50:0/2710220915 >> 172.21.15.50:0/2710220915 conn(0x7f21400b2040 msgr2=0x7f21400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.941+0000 7f2147800640 1 -- 172.21.15.50:0/2710220915 shutdown_connections 2024-09-21T06:01:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.941+0000 7f2147800640 1 -- 172.21.15.50:0/2710220915 wait complete. 2024-09-21T06:01:51.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.941+0000 7f2147800640 1 Processor -- start 2024-09-21T06:01:51.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.941+0000 7f2147800640 1 -- start start 2024-09-21T06:01:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2147800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21400b4de0 0x7f2140157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2147800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21400b5c70 0x7f214014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2147800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21401527b0 0x7f21401501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2147800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2140168ca0 con 0x7f21400b4de0 2024-09-21T06:01:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2147800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2140168b20 con 0x7f21401527b0 2024-09-21T06:01:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2147800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2140168e20 con 0x7f21400b5c70 2024-09-21T06:01:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2144e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21400b4de0 0x7f2140157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2144e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21400b4de0 0x7f2140157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51130/0 (socket says 172.21.15.50:51130) 2024-09-21T06:01:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2144e00640 1 -- 172.21.15.50:0/2929624970 learned_addr learned my addr 172.21.15.50:0/2929624970 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f213fe00640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21400b5c70 0x7f214014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2145800640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21401527b0 0x7f21401501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2144e00640 1 -- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21400b5c70 msgr2=0x7f214014fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2144e00640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21400b5c70 0x7f214014fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2144e00640 1 -- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21401527b0 msgr2=0x7f21401501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2144e00640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21401527b0 0x7f21401501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2144e00640 1 -- 172.21.15.50:0/2929624970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21401f8dc0 con 0x7f21400b4de0 2024-09-21T06:01:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2144e00640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21400b4de0 0x7f2140157c00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2134009850 tx=0x7f21340a57e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f213cc00640 1 -- 172.21.15.50:0/2929624970 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340b2040 con 0x7f21400b4de0 2024-09-21T06:01:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f213cc00640 1 -- 172.21.15.50:0/2929624970 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21340044b0 con 0x7f21400b4de0 2024-09-21T06:01:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2147800640 1 -- 172.21.15.50:0/2929624970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21401f90a0 con 0x7f21400b4de0 2024-09-21T06:01:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f213cc00640 1 -- 172.21.15.50:0/2929624970 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134005260 con 0x7f21400b4de0 2024-09-21T06:01:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2147800640 1 -- 172.21.15.50:0/2929624970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21400cd9e0 con 0x7f21400b4de0 2024-09-21T06:01:51.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.945+0000 7f2147800640 1 -- 172.21.15.50:0/2929624970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214015bb60 con 0x7f21400b4de0 2024-09-21T06:01:51.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.949+0000 7f213cc00640 1 -- 172.21.15.50:0/2929624970 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 97999+0+0 (secure 0 0 0) 0x7f2134005400 con 0x7f21400b4de0 2024-09-21T06:01:51.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.949+0000 7f213cc00640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2120073aa0 0x7f2120075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:51.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.949+0000 7f213cc00640 1 -- 172.21.15.50:0/2929624970 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f213412e9d0 con 0x7f21400b4de0 2024-09-21T06:01:51.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.949+0000 7f213fe00640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2120073aa0 0x7f2120075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.949+0000 7f213fe00640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2120073aa0 0x7f2120075fb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2140157950 tx=0x7f212c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:51.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:51.953+0000 7f213cc00640 1 -- 172.21.15.50:0/2929624970 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2140075580 con 0x7f21400b4de0 2024-09-21T06:01:52.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.157+0000 7f2147800640 1 -- 172.21.15.50:0/2929624970 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone33912", "target": ["mon-mgr", ""]}) -- 0x7f21401f8b60 con 0x7f2120073aa0 2024-09-21T06:01:52.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.373+0000 7f213cc00640 1 -- 172.21.15.50:0/2929624970 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f212c005020 con 0x7f2120073aa0 2024-09-21T06:01:52.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.377+0000 7f2147800640 1 -- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2120073aa0 msgr2=0x7f2120075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:52.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.377+0000 7f2147800640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2120073aa0 0x7f2120075fb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2140157950 tx=0x7f212c00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.377+0000 7f2147800640 1 -- 172.21.15.50:0/2929624970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21400b4de0 msgr2=0x7f2140157c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:52.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.377+0000 7f2147800640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21400b4de0 0x7f2140157c00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2134009850 tx=0x7f21340a57e0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.377+0000 7f2147800640 1 -- 172.21.15.50:0/2929624970 shutdown_connections 2024-09-21T06:01:52.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.377+0000 7f2147800640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21401527b0 0x7f21401501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.377+0000 7f2147800640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f21400b5c70 0x7f214014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.377+0000 7f2147800640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2120073aa0 0x7f2120075fb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.377+0000 7f2147800640 1 --2- 172.21.15.50:0/2929624970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21400b4de0 0x7f2140157c00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.377+0000 7f2147800640 1 -- 172.21.15.50:0/2929624970 >> 172.21.15.50:0/2929624970 conn(0x7f21400b2040 msgr2=0x7f2140151b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:52.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.377+0000 7f2147800640 1 -- 172.21.15.50:0/2929624970 shutdown_connections 2024-09-21T06:01:52.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.377+0000 7f2147800640 1 -- 172.21.15.50:0/2929624970 wait complete. 2024-09-21T06:01:52.400 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone30643 2024-09-21T06:01:52.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.565+0000 7f50eb200640 1 -- 172.21.15.50:0/3481833329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec0d1140 msgr2=0x7f50ec0b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:52.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.565+0000 7f50eb200640 1 --2- 172.21.15.50:0/3481833329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec0d1140 0x7f50ec0b0730 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009870 tx=0x7f50dc01c790 comp rx=0 tx=0).stop 2024-09-21T06:01:52.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.565+0000 7f50eb200640 1 -- 172.21.15.50:0/3481833329 shutdown_connections 2024-09-21T06:01:52.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.565+0000 7f50eb200640 1 --2- 172.21.15.50:0/3481833329 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50ec06dbc0 0x7f50ec06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.565+0000 7f50eb200640 1 --2- 172.21.15.50:0/3481833329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec0d1140 0x7f50ec0b0730 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.565+0000 7f50eb200640 1 -- 172.21.15.50:0/3481833329 >> 172.21.15.50:0/3481833329 conn(0x7f50ec0b41f0 msgr2=0x7f50ec0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:52.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.565+0000 7f50eb200640 1 -- 172.21.15.50:0/3481833329 shutdown_connections 2024-09-21T06:01:52.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.565+0000 7f50eb200640 1 -- 172.21.15.50:0/3481833329 wait complete. 2024-09-21T06:01:52.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50eb200640 1 Processor -- start 2024-09-21T06:01:52.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50eb200640 1 -- start start 2024-09-21T06:01:52.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50eb200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50ec06dbc0 0x7f50ec1fa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:52.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50eb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50ec0d1140 0x7f50ec0af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:52.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50eb200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec15f5c0 0x7f50ec0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:52.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50eb200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec068f70 con 0x7f50ec15f5c0 2024-09-21T06:01:52.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50eb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec068df0 con 0x7f50ec0d1140 2024-09-21T06:01:52.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50eb200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f50ec0690f0 con 0x7f50ec06dbc0 2024-09-21T06:01:52.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50ea800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec15f5c0 0x7f50ec0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:52.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50e9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50ec0d1140 0x7f50ec0af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:52.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50e9e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50ec06dbc0 0x7f50ec1fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:52.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50ea800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec15f5c0 0x7f50ec0ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51140/0 (socket says 172.21.15.50:51140) 2024-09-21T06:01:52.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.569+0000 7f50ea800640 1 -- 172.21.15.50:0/213058029 learned_addr learned my addr 172.21.15.50:0/213058029 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50e9400640 1 -- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50ec06dbc0 msgr2=0x7f50ec1fa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50e9400640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50ec06dbc0 0x7f50ec1fa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50e9400640 1 -- 172.21.15.50:0/213058029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec15f5c0 msgr2=0x7f50ec0ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50e9400640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec15f5c0 0x7f50ec0ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50e9400640 1 -- 172.21.15.50:0/213058029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec0c70b0 con 0x7f50ec0d1140 2024-09-21T06:01:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50e9e00640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50ec06dbc0 0x7f50ec1fa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:01:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50ea800640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec15f5c0 0x7f50ec0ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:01:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50e9400640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50ec0d1140 0x7f50ec0af2c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f50d800e9f0 tx=0x7f50d800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:52.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50e2000640 1 -- 172.21.15.50:0/213058029 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d800ccc0 con 0x7f50ec0d1140 2024-09-21T06:01:52.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50e2000640 1 -- 172.21.15.50:0/213058029 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50d800ce60 con 0x7f50ec0d1140 2024-09-21T06:01:52.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50eb200640 1 -- 172.21.15.50:0/213058029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ec0c5cc0 con 0x7f50ec0d1140 2024-09-21T06:01:52.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50e2000640 1 -- 172.21.15.50:0/213058029 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8004280 con 0x7f50ec0d1140 2024-09-21T06:01:52.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50eb200640 1 -- 172.21.15.50:0/213058029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ec1fa550 con 0x7f50ec0d1140 2024-09-21T06:01:52.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.573+0000 7f50eb200640 1 -- 172.21.15.50:0/213058029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ec06dfd0 con 0x7f50ec0d1140 2024-09-21T06:01:52.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.577+0000 7f50e2000640 1 -- 172.21.15.50:0/213058029 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) ==== 97999+0+0 (secure 0 0 0) 0x7f50d8003600 con 0x7f50ec0d1140 2024-09-21T06:01:52.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.577+0000 7f50e2000640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50cc073a70 0x7f50cc075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:52.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.577+0000 7f50e2000640 1 -- 172.21.15.50:0/213058029 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f50d80933e0 con 0x7f50ec0d1140 2024-09-21T06:01:52.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.577+0000 7f50e9e00640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50cc073a70 0x7f50cc075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:52.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.577+0000 7f50e9e00640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50cc073a70 0x7f50cc075f80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0af000 tx=0x7f50dc01afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:52.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.581+0000 7f50e2000640 1 -- 172.21.15.50:0/213058029 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f50d805f8d0 con 0x7f50ec0d1140 2024-09-21T06:01:52.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.781+0000 7f50eb200640 1 -- 172.21.15.50:0/213058029 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone30643", "target": ["mon-mgr", ""]}) -- 0x7f50ec06b540 con 0x7f50cc073a70 2024-09-21T06:01:52.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.853+0000 7f50e2000640 1 -- 172.21.15.50:0/213058029 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f50dc005ca0 con 0x7f50cc073a70 2024-09-21T06:01:52.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.857+0000 7f50eb200640 1 -- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50cc073a70 msgr2=0x7f50cc075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:52.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.857+0000 7f50eb200640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50cc073a70 0x7f50cc075f80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0af000 tx=0x7f50dc01afd0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.857+0000 7f50eb200640 1 -- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50ec0d1140 msgr2=0x7f50ec0af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:52.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.857+0000 7f50eb200640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50ec0d1140 0x7f50ec0af2c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f50d800e9f0 tx=0x7f50d800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.857+0000 7f50eb200640 1 -- 172.21.15.50:0/213058029 shutdown_connections 2024-09-21T06:01:52.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.857+0000 7f50eb200640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50cc073a70 0x7f50cc075f80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.857+0000 7f50eb200640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec15f5c0 0x7f50ec0ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.857+0000 7f50eb200640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50ec0d1140 0x7f50ec0af2c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.857+0000 7f50eb200640 1 --2- 172.21.15.50:0/213058029 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50ec06dbc0 0x7f50ec1fa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:52.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.857+0000 7f50eb200640 1 -- 172.21.15.50:0/213058029 >> 172.21.15.50:0/213058029 conn(0x7f50ec0b41f0 msgr2=0x7f50ec0d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:52.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.857+0000 7f50eb200640 1 -- 172.21.15.50:0/213058029 shutdown_connections 2024-09-21T06:01:52.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52.857+0000 7f50eb200640 1 -- 172.21.15.50:0/213058029 wait complete. 2024-09-21T06:01:52.879 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T06:01:52.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:01:52.936 INFO:teuthology.orchestra.run.smithi050.stdout:2 2024-09-21T06:01:52.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:01:52.943 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:01:53.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.105+0000 7f5464200640 1 -- 172.21.15.50:0/3742744230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c15e8c0 msgr2=0x7f545c15ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.105+0000 7f5464200640 1 --2- 172.21.15.50:0/3742744230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c15e8c0 0x7f545c15ecd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f545000a950 tx=0x7f545001db70 comp rx=0 tx=0).stop 2024-09-21T06:01:53.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.105+0000 7f5464200640 1 -- 172.21.15.50:0/3742744230 shutdown_connections 2024-09-21T06:01:53.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.105+0000 7f5464200640 1 --2- 172.21.15.50:0/3742744230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c15e8c0 0x7f545c15ecd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.105+0000 7f5464200640 1 -- 172.21.15.50:0/3742744230 >> 172.21.15.50:0/3742744230 conn(0x7f545c0b2040 msgr2=0x7f545c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:53.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.105+0000 7f5464200640 1 -- 172.21.15.50:0/3742744230 shutdown_connections 2024-09-21T06:01:53.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.105+0000 7f5464200640 1 -- 172.21.15.50:0/3742744230 wait complete. 2024-09-21T06:01:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.105+0000 7f5464200640 1 Processor -- start 2024-09-21T06:01:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.105+0000 7f5464200640 1 -- start start 2024-09-21T06:01:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5464200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f545c15b2d0 0x7f545c157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5464200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c15e290 0x7f545c13d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5464200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f545c15e8c0 0x7f545c154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5464200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f545c169180 con 0x7f545c15e290 2024-09-21T06:01:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5464200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f545c169000 con 0x7f545c15b2d0 2024-09-21T06:01:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5464200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f545c169300 con 0x7f545c15e8c0 2024-09-21T06:01:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5460e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c15e290 0x7f545c13d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5460e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c15e290 0x7f545c13d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51156/0 (socket says 172.21.15.50:51156) 2024-09-21T06:01:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5460e00640 1 -- 172.21.15.50:0/1006464841 learned_addr learned my addr 172.21.15.50:0/1006464841 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5461800640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f545c15b2d0 0x7f545c157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5462200640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f545c15e8c0 0x7f545c154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5460e00640 1 -- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f545c15e8c0 msgr2=0x7f545c154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5460e00640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f545c15e8c0 0x7f545c154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5460e00640 1 -- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f545c15b2d0 msgr2=0x7f545c157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5460e00640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f545c15b2d0 0x7f545c157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5460e00640 1 -- 172.21.15.50:0/1006464841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c13d9d0 con 0x7f545c15e290 2024-09-21T06:01:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5462200640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f545c15e8c0 0x7f545c154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:01:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5460e00640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c15e290 0x7f545c13d490 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f544800c400 tx=0x7f544800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:53.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5459600640 1 -- 172.21.15.50:0/1006464841 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448011790 con 0x7f545c15e290 2024-09-21T06:01:53.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5459600640 1 -- 172.21.15.50:0/1006464841 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5448017070 con 0x7f545c15e290 2024-09-21T06:01:53.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5459600640 1 -- 172.21.15.50:0/1006464841 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448011cc0 con 0x7f545c15e290 2024-09-21T06:01:53.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5464200640 1 -- 172.21.15.50:0/1006464841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c156da0 con 0x7f545c15e290 2024-09-21T06:01:53.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5464200640 1 -- 172.21.15.50:0/1006464841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c157360 con 0x7f545c15e290 2024-09-21T06:01:53.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.109+0000 7f5464200640 1 -- 172.21.15.50:0/1006464841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545c15ecd0 con 0x7f545c15e290 2024-09-21T06:01:53.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.113+0000 7f5459600640 1 -- 172.21.15.50:0/1006464841 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 97999+0+0 (secure 0 0 0) 0x7f5448005020 con 0x7f545c15e290 2024-09-21T06:01:53.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.113+0000 7f5459600640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f543c073aa0 0x7f543c075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.113+0000 7f5459600640 1 -- 172.21.15.50:0/1006464841 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f54480957d0 con 0x7f545c15e290 2024-09-21T06:01:53.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.113+0000 7f5461800640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f543c073aa0 0x7f543c075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.113+0000 7f5461800640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f543c073aa0 0x7f543c075fb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f54500b0000 tx=0x7f545001c3b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:53.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.117+0000 7f5459600640 1 -- 172.21.15.50:0/1006464841 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f544805fd30 con 0x7f545c15e290 2024-09-21T06:01:53.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 -- 172.21.15.50:0/1006464841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f545c15af70 con 0x7f545c15e290 2024-09-21T06:01:53.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5459600640 1 -- 172.21.15.50:0/1006464841 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v36)=0 v36) ==== 131+0+0 (secure 0 0 0) 0x7f544805f630 con 0x7f545c15e290 2024-09-21T06:01:53.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 -- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f543c073aa0 msgr2=0x7f543c075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f543c073aa0 0x7f543c075fb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f54500b0000 tx=0x7f545001c3b0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 -- 172.21.15.50:0/1006464841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c15e290 msgr2=0x7f545c13d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c15e290 0x7f545c13d490 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f544800c400 tx=0x7f544800c8c0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 -- 172.21.15.50:0/1006464841 shutdown_connections 2024-09-21T06:01:53.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f545c15e8c0 0x7f545c154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f543c073aa0 0x7f543c075fb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c15e290 0x7f545c13d490 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 --2- 172.21.15.50:0/1006464841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f545c15b2d0 0x7f545c157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 -- 172.21.15.50:0/1006464841 >> 172.21.15.50:0/1006464841 conn(0x7f545c0b2040 msgr2=0x7f545c0cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:53.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 -- 172.21.15.50:0/1006464841 shutdown_connections 2024-09-21T06:01:53.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.293+0000 7f5464200640 1 -- 172.21.15.50:0/1006464841 wait complete. 2024-09-21T06:01:53.316 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:01:53.316 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:01:53.407 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:01:53.523 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:01:53.524 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:01:53.524 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:01:53.524 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:01:53.533 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:01:53.533 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:01:53.589 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:01:53.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.909+0000 7f523a800640 1 -- 172.21.15.50:0/2842571461 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5234155ad0 msgr2=0x7f523406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.909+0000 7f523a800640 1 --2- 172.21.15.50:0/2842571461 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5234155ad0 0x7f523406e520 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5224009840 tx=0x7f522401c820 comp rx=0 tx=0).stop 2024-09-21T06:01:53.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.909+0000 7f523a800640 1 -- 172.21.15.50:0/2842571461 shutdown_connections 2024-09-21T06:01:53.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.909+0000 7f523a800640 1 --2- 172.21.15.50:0/2842571461 >> v2:172.21.15.50:3300/0 conn(0x7f523415f180 0x7f523415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.909+0000 7f523a800640 1 --2- 172.21.15.50:0/2842571461 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5234155ad0 0x7f523406e520 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.909+0000 7f523a800640 1 -- 172.21.15.50:0/2842571461 >> 172.21.15.50:0/2842571461 conn(0x7f52340708c0 msgr2=0x7f5234070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:53.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.909+0000 7f523a800640 1 -- 172.21.15.50:0/2842571461 shutdown_connections 2024-09-21T06:01:53.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.909+0000 7f523a800640 1 -- 172.21.15.50:0/2842571461 wait complete. 2024-09-21T06:01:53.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.913+0000 7f523a800640 1 Processor -- start 2024-09-21T06:01:53.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.913+0000 7f523a800640 1 -- start start 2024-09-21T06:01:53.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f523a800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52340d29c0 0x7f52340b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f523a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5234155ad0 0x7f5234145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f523a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f523415f180 0x7f523414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f523a800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f523416c4a0 con 0x7f523415f180 2024-09-21T06:01:53.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f523a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f523416c320 con 0x7f5234155ad0 2024-09-21T06:01:53.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f523a800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f523416c620 con 0x7f52340d29c0 2024-09-21T06:01:53.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f5233e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f523415f180 0x7f523414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f5233e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f523415f180 0x7f523414df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51184/0 (socket says 172.21.15.50:51184) 2024-09-21T06:01:53.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f5233e00640 1 -- 172.21.15.50:0/2353580551 learned_addr learned my addr 172.21.15.50:0/2353580551 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:53.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f5232a00640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5234155ad0 0x7f5234145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f5233400640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52340d29c0 0x7f52340b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f5233e00640 1 -- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52340d29c0 msgr2=0x7f52340b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f5233e00640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52340d29c0 0x7f52340b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f5233e00640 1 -- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5234155ad0 msgr2=0x7f5234145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f5233e00640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5234155ad0 0x7f5234145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f5233e00640 1 -- 172.21.15.50:0/2353580551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5234161d90 con 0x7f523415f180 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.917+0000 7f5233e00640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f523415f180 0x7f523414df20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f522c00edd0 tx=0x7f522c00c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.921+0000 7f522b400640 1 -- 172.21.15.50:0/2353580551 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c004d30 con 0x7f523415f180 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.921+0000 7f522b400640 1 -- 172.21.15.50:0/2353580551 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f522c005020 con 0x7f523415f180 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.921+0000 7f522b400640 1 -- 172.21.15.50:0/2353580551 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c005450 con 0x7f523415f180 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.921+0000 7f523a800640 1 -- 172.21.15.50:0/2353580551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52340b49e0 con 0x7f523415f180 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.921+0000 7f523a800640 1 -- 172.21.15.50:0/2353580551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52341f9220 con 0x7f523415f180 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.921+0000 7f523a800640 1 -- 172.21.15.50:0/2353580551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523406e520 con 0x7f523415f180 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.925+0000 7f522b400640 1 -- 172.21.15.50:0/2353580551 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f522c0055f0 con 0x7f523415f180 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.925+0000 7f522b400640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5214073b00 0x7f5214076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.925+0000 7f522b400640 1 -- 172.21.15.50:0/2353580551 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f522c093330 con 0x7f523415f180 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.929+0000 7f5233400640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5214073b00 0x7f5214076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.929+0000 7f5233400640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5214073b00 0x7f5214076010 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f52240af000 tx=0x7f522401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:53.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:53.937+0000 7f522b400640 1 -- 172.21.15.50:0/2353580551 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f522c05f750 con 0x7f523415f180 2024-09-21T06:01:54.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.349+0000 7f523a800640 1 -- 172.21.15.50:0/2353580551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f523406b250 con 0x7f523415f180 2024-09-21T06:01:54.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.353+0000 7f522b400640 1 -- 172.21.15.50:0/2353580551 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7f522c064180 con 0x7f523415f180 2024-09-21T06:01:54.353 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T06:01:54.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.357+0000 7f523a800640 1 -- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5214073b00 msgr2=0x7f5214076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:54.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.357+0000 7f523a800640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5214073b00 0x7f5214076010 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f52240af000 tx=0x7f522401b060 comp rx=0 tx=0).stop 2024-09-21T06:01:54.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.357+0000 7f523a800640 1 -- 172.21.15.50:0/2353580551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f523415f180 msgr2=0x7f523414df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:54.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.357+0000 7f523a800640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f523415f180 0x7f523414df20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f522c00edd0 tx=0x7f522c00c6e0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.357+0000 7f523a800640 1 -- 172.21.15.50:0/2353580551 shutdown_connections 2024-09-21T06:01:54.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.357+0000 7f523a800640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5214073b00 0x7f5214076010 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.357+0000 7f523a800640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f523415f180 0x7f523414df20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.357+0000 7f523a800640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5234155ad0 0x7f5234145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.357+0000 7f523a800640 1 --2- 172.21.15.50:0/2353580551 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52340d29c0 0x7f52340b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.357+0000 7f523a800640 1 -- 172.21.15.50:0/2353580551 >> 172.21.15.50:0/2353580551 conn(0x7f52340708c0 msgr2=0x7f5234148ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:54.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.361+0000 7f523a800640 1 -- 172.21.15.50:0/2353580551 shutdown_connections 2024-09-21T06:01:54.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.361+0000 7f523a800640 1 -- 172.21.15.50:0/2353580551 wait complete. 2024-09-21T06:01:54.408 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:01:54.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:01:54.408 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:01:54.499 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:01:54.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:01:54.500 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:01:54.525 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:01:54.526 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:01:54.526 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:01:54.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:01:54.535 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:01:54.536 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:01:54.536 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:01:54.585 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:01:54.586 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:01:54.586 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:01:54.586 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:01:54.601 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:01:54.601 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:01:54.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.925+0000 7f9e17c00640 1 -- 172.21.15.50:0/3509056477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e100b3890 msgr2=0x7f9e100b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:54.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.929+0000 7f9e17c00640 1 --2- 172.21.15.50:0/3509056477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e100b3890 0x7f9e100b3ca0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9e100ce790 tx=0x7f9e0801ccc0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.929+0000 7f9e17c00640 1 -- 172.21.15.50:0/3509056477 shutdown_connections 2024-09-21T06:01:54.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.929+0000 7f9e17c00640 1 --2- 172.21.15.50:0/3509056477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e100b3890 0x7f9e100b3ca0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.929+0000 7f9e17c00640 1 --2- 172.21.15.50:0/3509056477 >> v2:172.21.15.50:3300/0 conn(0x7f9e100cf210 0x7f9e100cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.929+0000 7f9e17c00640 1 -- 172.21.15.50:0/3509056477 >> 172.21.15.50:0/3509056477 conn(0x7f9e1006e1d0 msgr2=0x7f9e1006e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:54.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.929+0000 7f9e17c00640 1 -- 172.21.15.50:0/3509056477 shutdown_connections 2024-09-21T06:01:54.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.929+0000 7f9e17c00640 1 -- 172.21.15.50:0/3509056477 wait complete. 2024-09-21T06:01:54.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.929+0000 7f9e17c00640 1 Processor -- start 2024-09-21T06:01:54.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e17c00640 1 -- start start 2024-09-21T06:01:54.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e17c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9e100b3890 0x7f9e1014a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:54.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e17c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e100cd7a0 0x7f9e1014ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:54.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e17c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e100cf210 0x7f9e1014f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:54.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e17c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e100ccec0 con 0x7f9e100cf210 2024-09-21T06:01:54.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e17c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9e100ccd40 con 0x7f9e100cd7a0 2024-09-21T06:01:54.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e17c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9e100cd040 con 0x7f9e100b3890 2024-09-21T06:01:54.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e15200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9e100b3890 0x7f9e1014a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:54.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e15200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9e100b3890 0x7f9e1014a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:60726/0 (socket says 172.21.15.50:60726) 2024-09-21T06:01:54.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e15200640 1 -- 172.21.15.50:0/3992839166 learned_addr learned my addr 172.21.15.50:0/3992839166 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:54.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e15c00640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e100cf210 0x7f9e1014f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:54.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e07e00640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e100cd7a0 0x7f9e1014ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:54.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e15200640 1 -- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e100cd7a0 msgr2=0x7f9e1014ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:54.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e15200640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e100cd7a0 0x7f9e1014ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e15200640 1 -- 172.21.15.50:0/3992839166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e100cf210 msgr2=0x7f9e1014f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:54.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e15200640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e100cf210 0x7f9e1014f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e15200640 1 -- 172.21.15.50:0/3992839166 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1014eb00 con 0x7f9e100b3890 2024-09-21T06:01:54.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.933+0000 7f9e07e00640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e100cd7a0 0x7f9e1014ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:01:54.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.937+0000 7f9e15200640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9e100b3890 0x7f9e1014a470 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9df800c3c0 tx=0x7f9df800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:54.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.937+0000 7f9e0d600640 1 -- 172.21.15.50:0/3992839166 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8017070 con 0x7f9e100b3890 2024-09-21T06:01:54.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.937+0000 7f9e0d600640 1 -- 172.21.15.50:0/3992839166 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9df80053c0 con 0x7f9e100b3890 2024-09-21T06:01:54.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.937+0000 7f9e0d600640 1 -- 172.21.15.50:0/3992839166 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8003860 con 0x7f9e100b3890 2024-09-21T06:01:54.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.937+0000 7f9e17c00640 1 -- 172.21.15.50:0/3992839166 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1014e820 con 0x7f9e100b3890 2024-09-21T06:01:54.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.937+0000 7f9e17c00640 1 -- 172.21.15.50:0/3992839166 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1014fd10 con 0x7f9e100b3890 2024-09-21T06:01:54.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.937+0000 7f9e17c00640 1 -- 172.21.15.50:0/3992839166 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e100b3ca0 con 0x7f9e100b3890 2024-09-21T06:01:54.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.941+0000 7f9e0d600640 1 -- 172.21.15.50:0/3992839166 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f9df801a070 con 0x7f9e100b3890 2024-09-21T06:01:54.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.941+0000 7f9e0d600640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9dec073b00 0x7f9dec076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:54.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.941+0000 7f9e0d600640 1 -- 172.21.15.50:0/3992839166 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f9df8091d30 con 0x7f9e100b3890 2024-09-21T06:01:54.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.941+0000 7f9e07e00640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9dec073b00 0x7f9dec076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:54.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.941+0000 7f9e07e00640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9dec073b00 0x7f9dec076010 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9e00005fd0 tx=0x7f9e00005f20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:54.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:54.945+0000 7f9e0d600640 1 -- 172.21.15.50:0/3992839166 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9df805e220 con 0x7f9e100b3890 2024-09-21T06:01:55.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.429+0000 7f9e17c00640 1 -- 172.21.15.50:0/3992839166 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e10075280 con 0x7f9e100b3890 2024-09-21T06:01:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.429+0000 7f9e0d600640 1 -- 172.21.15.50:0/3992839166 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5217 (secure 0 0 0) 0x7f9df8062c50 con 0x7f9e100b3890 2024-09-21T06:01:55.434 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:01:55.434 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":28,"btime":"2024-09-21T05:59:51:480904+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/723044219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":723044219},{"type":"v1","addr":"172.21.15.88:6836","nonce":723044219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1966998855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1966998855},{"type":"v1","addr":"172.21.15.88:6837","nonce":1966998855}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2536951841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2536951841},{"type":"v1","addr":"172.21.15.50:6834","nonce":2536951841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:59:37.853304+0000","modified":"2024-09-21T05:59:40.872885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5370":{"gid":5370,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/988935074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":988935074},{"type":"v1","addr":"172.21.15.50:6835","nonce":988935074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5370,"qdb_cluster":[5370]},"id":3}]} 2024-09-21T06:01:55.434 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 28 2024-09-21T06:01:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.433+0000 7f9e17c00640 1 -- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9dec073b00 msgr2=0x7f9dec076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.433+0000 7f9e17c00640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9dec073b00 0x7f9dec076010 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9e00005fd0 tx=0x7f9e00005f20 comp rx=0 tx=0).stop 2024-09-21T06:01:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.433+0000 7f9e17c00640 1 -- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9e100b3890 msgr2=0x7f9e1014a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.433+0000 7f9e17c00640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9e100b3890 0x7f9e1014a470 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9df800c3c0 tx=0x7f9df800c880 comp rx=0 tx=0).stop 2024-09-21T06:01:55.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.433+0000 7f9e17c00640 1 -- 172.21.15.50:0/3992839166 shutdown_connections 2024-09-21T06:01:55.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.433+0000 7f9e17c00640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9dec073b00 0x7f9dec076010 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:55.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.433+0000 7f9e17c00640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e100cf210 0x7f9e1014f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:55.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.433+0000 7f9e17c00640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e100cd7a0 0x7f9e1014ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:55.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.433+0000 7f9e17c00640 1 --2- 172.21.15.50:0/3992839166 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9e100b3890 0x7f9e1014a470 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:55.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.433+0000 7f9e17c00640 1 -- 172.21.15.50:0/3992839166 >> 172.21.15.50:0/3992839166 conn(0x7f9e1006e1d0 msgr2=0x7f9e10073e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:55.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.433+0000 7f9e17c00640 1 -- 172.21.15.50:0/3992839166 shutdown_connections 2024-09-21T06:01:55.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.433+0000 7f9e17c00640 1 -- 172.21.15.50:0/3992839166 wait complete. 2024-09-21T06:01:55.475 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:01:55.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.741+0000 7f650da00640 1 -- 172.21.15.50:0/628338062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f650815aa50 msgr2=0x7f65080cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:55.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.741+0000 7f650da00640 1 --2- 172.21.15.50:0/628338062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f650815aa50 0x7f65080cd970 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f65080ce8d0 tx=0x7f64f801c820 comp rx=0 tx=0).stop 2024-09-21T06:01:55.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.745+0000 7f650da00640 1 -- 172.21.15.50:0/628338062 shutdown_connections 2024-09-21T06:01:55.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.745+0000 7f650da00640 1 --2- 172.21.15.50:0/628338062 >> v2:172.21.15.88:3300/0 conn(0x7f65080b3340 0x7f65080b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:55.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.745+0000 7f650da00640 1 --2- 172.21.15.50:0/628338062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f650815aa50 0x7f65080cd970 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:55.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.745+0000 7f650da00640 1 -- 172.21.15.50:0/628338062 >> 172.21.15.50:0/628338062 conn(0x7f65080b2040 msgr2=0x7f65080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:55.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.745+0000 7f650da00640 1 -- 172.21.15.50:0/628338062 shutdown_connections 2024-09-21T06:01:55.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.745+0000 7f650da00640 1 -- 172.21.15.50:0/628338062 wait complete. 2024-09-21T06:01:55.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.749+0000 7f650da00640 1 Processor -- start 2024-09-21T06:01:55.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.753+0000 7f650da00640 1 -- start start 2024-09-21T06:01:55.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f650da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65080b3340 0x7f6508150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:55.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f650da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65080ce4c0 0x7f6508146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:55.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f650da00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f650815aa50 0x7f65081fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:55.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f650da00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6508169f70 con 0x7f65080ce4c0 2024-09-21T06:01:55.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f650da00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6508169df0 con 0x7f65080b3340 2024-09-21T06:01:55.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f650da00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f650816a0f0 con 0x7f650815aa50 2024-09-21T06:01:55.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f6506a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65080b3340 0x7f6508150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:55.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f6507400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f650815aa50 0x7f65081fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:55.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f6507400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f650815aa50 0x7f65081fcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:60730/0 (socket says 172.21.15.50:60730) 2024-09-21T06:01:55.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f6507400640 1 -- 172.21.15.50:0/62937262 learned_addr learned my addr 172.21.15.50:0/62937262 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:55.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f6506000640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65080ce4c0 0x7f6508146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:55.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f6506a00640 1 -- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f650815aa50 msgr2=0x7f65081fcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:55.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f6506a00640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f650815aa50 0x7f65081fcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:55.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f6506a00640 1 -- 172.21.15.50:0/62937262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65080ce4c0 msgr2=0x7f6508146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:55.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f6506a00640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65080ce4c0 0x7f6508146e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:55.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f6506a00640 1 -- 172.21.15.50:0/62937262 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65081473c0 con 0x7f65080b3340 2024-09-21T06:01:55.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.757+0000 7f6506a00640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65080b3340 0x7f6508150630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f650000cab0 tx=0x7f650000cba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:55.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.761+0000 7f64fea00640 1 -- 172.21.15.50:0/62937262 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500016020 con 0x7f65080b3340 2024-09-21T06:01:55.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.761+0000 7f64fea00640 1 -- 172.21.15.50:0/62937262 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f650000a950 con 0x7f65080b3340 2024-09-21T06:01:55.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.761+0000 7f64fea00640 1 -- 172.21.15.50:0/62937262 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650000ada0 con 0x7f65080b3340 2024-09-21T06:01:55.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.761+0000 7f650da00640 1 -- 172.21.15.50:0/62937262 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6508154940 con 0x7f65080b3340 2024-09-21T06:01:55.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.761+0000 7f650da00640 1 -- 172.21.15.50:0/62937262 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6508154f00 con 0x7f65080b3340 2024-09-21T06:01:55.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.765+0000 7f64fea00640 1 -- 172.21.15.50:0/62937262 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f6500019020 con 0x7f65080b3340 2024-09-21T06:01:55.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.765+0000 7f64fea00640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f64e8073b50 0x7f64e8076060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:55.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.765+0000 7f64fea00640 1 -- 172.21.15.50:0/62937262 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7f65000956e0 con 0x7f65080b3340 2024-09-21T06:01:55.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.765+0000 7f6506000640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f64e8073b50 0x7f64e8076060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:55.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.769+0000 7f6506000640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f64e8073b50 0x7f64e8076060 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f64f80af000 tx=0x7f64f801b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:55.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.769+0000 7f650da00640 1 -- 172.21.15.50:0/62937262 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65080b3750 con 0x7f65080b3340 2024-09-21T06:01:55.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:55.781+0000 7f64fea00640 1 -- 172.21.15.50:0/62937262 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6500020020 con 0x7f65080b3340 2024-09-21T06:01:56.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.205+0000 7f650da00640 1 -- 172.21.15.50:0/62937262 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f650806b250 con 0x7f65080b3340 2024-09-21T06:01:56.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.205+0000 7f64fea00640 1 -- 172.21.15.50:0/62937262 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5217 (secure 0 0 0) 0x7f650005f6f0 con 0x7f65080b3340 2024-09-21T06:01:56.207 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:01:56.207 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":28,"btime":"2024-09-21T05:59:51:480904+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/723044219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":723044219},{"type":"v1","addr":"172.21.15.88:6836","nonce":723044219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1966998855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1966998855},{"type":"v1","addr":"172.21.15.88:6837","nonce":1966998855}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2536951841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2536951841},{"type":"v1","addr":"172.21.15.50:6834","nonce":2536951841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:59:37.853304+0000","modified":"2024-09-21T05:59:40.872885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5370":{"gid":5370,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/988935074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":988935074},{"type":"v1","addr":"172.21.15.50:6835","nonce":988935074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5370,"qdb_cluster":[5370]},"id":3}]} 2024-09-21T06:01:56.210 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 28 2024-09-21T06:01:56.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.209+0000 7f650da00640 1 -- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f64e8073b50 msgr2=0x7f64e8076060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:56.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.209+0000 7f650da00640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f64e8073b50 0x7f64e8076060 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f64f80af000 tx=0x7f64f801b060 comp rx=0 tx=0).stop 2024-09-21T06:01:56.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.209+0000 7f650da00640 1 -- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65080b3340 msgr2=0x7f6508150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:56.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.209+0000 7f650da00640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65080b3340 0x7f6508150630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f650000cab0 tx=0x7f650000cba0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.209+0000 7f650da00640 1 -- 172.21.15.50:0/62937262 shutdown_connections 2024-09-21T06:01:56.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.209+0000 7f650da00640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f650815aa50 0x7f65081fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.209+0000 7f650da00640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f64e8073b50 0x7f64e8076060 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.209+0000 7f650da00640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65080ce4c0 0x7f6508146e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.209+0000 7f650da00640 1 --2- 172.21.15.50:0/62937262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65080b3340 0x7f6508150630 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.209+0000 7f650da00640 1 -- 172.21.15.50:0/62937262 >> 172.21.15.50:0/62937262 conn(0x7f65080b2040 msgr2=0x7f65080b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:56.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.209+0000 7f650da00640 1 -- 172.21.15.50:0/62937262 shutdown_connections 2024-09-21T06:01:56.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.209+0000 7f650da00640 1 -- 172.21.15.50:0/62937262 wait complete. 2024-09-21T06:01:56.230 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:01:56.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.389+0000 7fa2b3e00640 1 -- 172.21.15.50:0/1428259849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2ac0b5c70 msgr2=0x7fa294003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:56.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.389+0000 7fa2b3e00640 1 --2- 172.21.15.50:0/1428259849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2ac0b5c70 0x7fa294003c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009b40 tx=0x7fa2a001d290 comp rx=0 tx=0).stop 2024-09-21T06:01:56.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.389+0000 7fa2b1e00640 1 -- 172.21.15.50:0/1428259849 reap_dead start 2024-09-21T06:01:56.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.389+0000 7fa2b3e00640 1 -- 172.21.15.50:0/1428259849 shutdown_connections 2024-09-21T06:01:56.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.389+0000 7fa2b3e00640 1 -- 172.21.15.50:0/1428259849 >> 172.21.15.50:0/1428259849 conn(0x7fa2ac0b2040 msgr2=0x7fa2ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:56.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.389+0000 7fa2b3e00640 1 -- 172.21.15.50:0/1428259849 shutdown_connections 2024-09-21T06:01:56.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.389+0000 7fa2b3e00640 1 -- 172.21.15.50:0/1428259849 wait complete. 2024-09-21T06:01:56.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.389+0000 7fa2b3e00640 1 Processor -- start 2024-09-21T06:01:56.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.389+0000 7fa2b3e00640 1 -- start start 2024-09-21T06:01:56.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b3e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2ac0b5c70 0x7fa2ac148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:56.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2ac143250 0x7fa2ac158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:56.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2ac1587c0 0x7fa2ac1f9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:56.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac15cfe0 con 0x7fa2ac143250 2024-09-21T06:01:56.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac15ce60 con 0x7fa2ac1587c0 2024-09-21T06:01:56.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b3e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa2ac15d160 con 0x7fa2ac0b5c70 2024-09-21T06:01:56.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2ac143250 0x7fa2ac158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2ac143250 0x7fa2ac158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51224/0 (socket says 172.21.15.50:51224) 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b0a00640 1 -- 172.21.15.50:0/3435352450 learned_addr learned my addr 172.21.15.50:0/3435352450 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b1e00640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2ac1587c0 0x7fa2ac1f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b1400640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2ac0b5c70 0x7fa2ac148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b0a00640 1 -- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2ac0b5c70 msgr2=0x7fa2ac148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b0a00640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2ac0b5c70 0x7fa2ac148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b0a00640 1 -- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2ac1587c0 msgr2=0x7fa2ac1f9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b0a00640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2ac1587c0 0x7fa2ac1f9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b0a00640 1 -- 172.21.15.50:0/3435352450 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ac152bc0 con 0x7fa2ac143250 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b1e00640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2ac1587c0 0x7fa2ac1f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b1400640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2ac0b5c70 0x7fa2ac148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b0a00640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2ac143250 0x7fa2ac158280 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa298005ef0 tx=0x7fa298011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:56.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2a9600640 1 -- 172.21.15.50:0/3435352450 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298005320 con 0x7fa2ac143250 2024-09-21T06:01:56.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2a9600640 1 -- 172.21.15.50:0/3435352450 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa298018020 con 0x7fa2ac143250 2024-09-21T06:01:56.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b3e00640 1 -- 172.21.15.50:0/3435352450 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ac1574c0 con 0x7fa2ac143250 2024-09-21T06:01:56.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2a9600640 1 -- 172.21.15.50:0/3435352450 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298003860 con 0x7fa2ac143250 2024-09-21T06:01:56.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b3e00640 1 -- 172.21.15.50:0/3435352450 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ac157a80 con 0x7fa2ac143250 2024-09-21T06:01:56.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.393+0000 7fa2b3e00640 1 -- 172.21.15.50:0/3435352450 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2ac074e30 con 0x7fa2ac143250 2024-09-21T06:01:56.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.397+0000 7fa2a9600640 1 -- 172.21.15.50:0/3435352450 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fa298003a00 con 0x7fa2ac143250 2024-09-21T06:01:56.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.397+0000 7fa2a9600640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa28c073b00 0x7fa28c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:56.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.397+0000 7fa2a9600640 1 -- 172.21.15.50:0/3435352450 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7fa298096590 con 0x7fa2ac143250 2024-09-21T06:01:56.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.397+0000 7fa2b1400640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa28c073b00 0x7fa28c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:56.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.397+0000 7fa2b1400640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa28c073b00 0x7fa28c076010 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00ad000 tx=0x7fa2a001a030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:56.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.401+0000 7fa2a9600640 1 -- 172.21.15.50:0/3435352450 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa298022020 con 0x7fa2ac143250 2024-09-21T06:01:56.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 -- 172.21.15.50:0/3435352450 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2ac075300 con 0x7fa2ac143250 2024-09-21T06:01:56.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2a9600640 1 -- 172.21.15.50:0/3435352450 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16825 (secure 0 0 0) 0x7fa298062e10 con 0x7fa2ac143250 2024-09-21T06:01:56.582 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:01:56.582 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":47,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:01:54.940117+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:59:35.596192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T05:59:36.818395+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":4}]},{"pgid":"7.35","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:01:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 -- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa28c073b00 msgr2=0x7fa28c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa28c073b00 0x7fa28c076010 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00ad000 tx=0x7fa2a001a030 comp rx=0 tx=0).stop 2024-09-21T06:01:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 -- 172.21.15.50:0/3435352450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2ac143250 msgr2=0x7fa2ac158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2ac143250 0x7fa2ac158280 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa298005ef0 tx=0x7fa298011740 comp rx=0 tx=0).stop 2024-09-21T06:01:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 -- 172.21.15.50:0/3435352450 shutdown_connections 2024-09-21T06:01:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2ac1587c0 0x7fa2ac1f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa28c073b00 0x7fa28c076010 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2ac143250 0x7fa2ac158280 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 --2- 172.21.15.50:0/3435352450 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2ac0b5c70 0x7fa2ac148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 -- 172.21.15.50:0/3435352450 >> 172.21.15.50:0/3435352450 conn(0x7fa2ac0b2040 msgr2=0x7fa2ac1e98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:56.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 -- 172.21.15.50:0/3435352450 shutdown_connections 2024-09-21T06:01:56.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.581+0000 7fa2b3e00640 1 -- 172.21.15.50:0/3435352450 wait complete. 2024-09-21T06:01:56.605 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:01:56.605 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:01:56.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.761+0000 7efd79a00640 1 -- 172.21.15.50:0/3612038675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd74074c30 msgr2=0x7efd74075040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:56.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.761+0000 7efd79a00640 1 --2- 172.21.15.50:0/3612038675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd74074c30 0x7efd74075040 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efd74075760 tx=0x7efd5c01c780 comp rx=0 tx=0).stop 2024-09-21T06:01:56.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.761+0000 7efd79a00640 1 -- 172.21.15.50:0/3612038675 shutdown_connections 2024-09-21T06:01:56.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.761+0000 7efd79a00640 1 --2- 172.21.15.50:0/3612038675 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efd740d1190 0x7efd740d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.761+0000 7efd79a00640 1 --2- 172.21.15.50:0/3612038675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd74074c30 0x7efd74075040 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.761+0000 7efd79a00640 1 --2- 172.21.15.50:0/3612038675 >> v2:172.21.15.88:3300/0 conn(0x7efd740746e0 0x7efd7415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.761+0000 7efd79a00640 1 -- 172.21.15.50:0/3612038675 >> 172.21.15.50:0/3612038675 conn(0x7efd740b41f0 msgr2=0x7efd740b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:56.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.761+0000 7efd79a00640 1 -- 172.21.15.50:0/3612038675 shutdown_connections 2024-09-21T06:01:56.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.761+0000 7efd79a00640 1 -- 172.21.15.50:0/3612038675 wait complete. 2024-09-21T06:01:56.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd79a00640 1 Processor -- start 2024-09-21T06:01:56.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd79a00640 1 -- start start 2024-09-21T06:01:56.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd79a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efd740746e0 0x7efd74156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:56.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd79a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd74074c30 0x7efd740703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:56.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd79a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd740d1190 0x7efd74070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:56.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd79a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efd740ce620 con 0x7efd740d1190 2024-09-21T06:01:56.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd79a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efd740ce4a0 con 0x7efd74074c30 2024-09-21T06:01:56.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd79a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7efd740ce7a0 con 0x7efd740746e0 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd79000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd740d1190 0x7efd74070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd6be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd74074c30 0x7efd740703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd73e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efd740746e0 0x7efd74156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd6be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd74074c30 0x7efd740703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:36778/0 (socket says 172.21.15.50:36778) 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd6be00640 1 -- 172.21.15.50:0/1154229401 learned_addr learned my addr 172.21.15.50:0/1154229401 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd6be00640 1 -- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efd740746e0 msgr2=0x7efd74156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd6be00640 1 --2- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efd740746e0 0x7efd74156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd6be00640 1 -- 172.21.15.50:0/1154229401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd740d1190 msgr2=0x7efd74070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd6be00640 1 --2- 172.21.15.50:0/1154229401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd740d1190 0x7efd74070920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd6be00640 1 -- 172.21.15.50:0/1154229401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd74099400 con 0x7efd74074c30 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.765+0000 7efd6be00640 1 --2- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd74074c30 0x7efd740703e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0b0040 tx=0x7efd5c005d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.769+0000 7efd70c00640 1 -- 172.21.15.50:0/1154229401 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0b2040 con 0x7efd74074c30 2024-09-21T06:01:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.769+0000 7efd70c00640 1 -- 172.21.15.50:0/1154229401 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd5c0a5c80 con 0x7efd74074c30 2024-09-21T06:01:56.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.769+0000 7efd70c00640 1 -- 172.21.15.50:0/1154229401 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c005290 con 0x7efd74074c30 2024-09-21T06:01:56.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.769+0000 7efd79a00640 1 -- 172.21.15.50:0/1154229401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd740996e0 con 0x7efd74074c30 2024-09-21T06:01:56.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.769+0000 7efd79a00640 1 -- 172.21.15.50:0/1154229401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd74099ca0 con 0x7efd74074c30 2024-09-21T06:01:56.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.769+0000 7efd79a00640 1 -- 172.21.15.50:0/1154229401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd74075040 con 0x7efd74074c30 2024-09-21T06:01:56.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.769+0000 7efd70c00640 1 -- 172.21.15.50:0/1154229401 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7efd5c005430 con 0x7efd74074c30 2024-09-21T06:01:56.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.769+0000 7efd70c00640 1 --2- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efd50073b00 0x7efd50076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:56.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.769+0000 7efd70c00640 1 -- 172.21.15.50:0/1154229401 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7efd5c12cf80 con 0x7efd74074c30 2024-09-21T06:01:56.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.769+0000 7efd73e00640 1 --2- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efd50073b00 0x7efd50076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:56.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.773+0000 7efd73e00640 1 --2- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efd50073b00 0x7efd50076010 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd64009ac0 tx=0x7efd640091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:56.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.773+0000 7efd70c00640 1 -- 172.21.15.50:0/1154229401 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efd5c016490 con 0x7efd74074c30 2024-09-21T06:01:56.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.985+0000 7efd79a00640 1 -- 172.21.15.50:0/1154229401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd740b4f30 con 0x7efd74074c30 2024-09-21T06:01:56.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.985+0000 7efd70c00640 1 -- 172.21.15.50:0/1154229401 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5217 (secure 0 0 0) 0x7efd5c0f9470 con 0x7efd74074c30 2024-09-21T06:01:56.988 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:01:56.988 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":28,"btime":"2024-09-21T05:59:51:480904+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/723044219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":723044219},{"type":"v1","addr":"172.21.15.88:6836","nonce":723044219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1966998855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1966998855},{"type":"v1","addr":"172.21.15.88:6837","nonce":1966998855}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2536951841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2536951841},{"type":"v1","addr":"172.21.15.50:6834","nonce":2536951841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:59:37.853304+0000","modified":"2024-09-21T05:59:40.872885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5370":{"gid":5370,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/988935074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":988935074},{"type":"v1","addr":"172.21.15.50:6835","nonce":988935074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5370,"qdb_cluster":[5370]},"id":3}]} 2024-09-21T06:01:56.989 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 28 2024-09-21T06:01:56.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.989+0000 7efd79a00640 1 -- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efd50073b00 msgr2=0x7efd50076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:56.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.989+0000 7efd79a00640 1 --2- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efd50073b00 0x7efd50076010 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd64009ac0 tx=0x7efd640091e0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.989+0000 7efd79a00640 1 -- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd74074c30 msgr2=0x7efd740703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:56.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.989+0000 7efd79a00640 1 --2- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd74074c30 0x7efd740703e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0b0040 tx=0x7efd5c005d60 comp rx=0 tx=0).stop 2024-09-21T06:01:56.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.989+0000 7efd79a00640 1 -- 172.21.15.50:0/1154229401 shutdown_connections 2024-09-21T06:01:56.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.989+0000 7efd79a00640 1 --2- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efd50073b00 0x7efd50076010 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.989+0000 7efd79a00640 1 --2- 172.21.15.50:0/1154229401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd740d1190 0x7efd74070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.989+0000 7efd79a00640 1 --2- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd74074c30 0x7efd740703e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.989+0000 7efd79a00640 1 --2- 172.21.15.50:0/1154229401 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efd740746e0 0x7efd74156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:56.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.989+0000 7efd79a00640 1 -- 172.21.15.50:0/1154229401 >> 172.21.15.50:0/1154229401 conn(0x7efd740b41f0 msgr2=0x7efd74158470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:56.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.989+0000 7efd79a00640 1 -- 172.21.15.50:0/1154229401 shutdown_connections 2024-09-21T06:01:56.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:56.989+0000 7efd79a00640 1 -- 172.21.15.50:0/1154229401 wait complete. 2024-09-21T06:01:57.011 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:01:57.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.169+0000 7f5547c00640 1 -- 172.21.15.50:0/1017706126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55400b4de0 msgr2=0x7f55400b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.169+0000 7f5547c00640 1 --2- 172.21.15.50:0/1017706126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55400b4de0 0x7f55400b51f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5540071cb0 tx=0x7f553801cac0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.169+0000 7f5547c00640 1 -- 172.21.15.50:0/1017706126 shutdown_connections 2024-09-21T06:01:57.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.169+0000 7f5547c00640 1 --2- 172.21.15.50:0/1017706126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55400b4de0 0x7f55400b51f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.169+0000 7f5547c00640 1 --2- 172.21.15.50:0/1017706126 >> v2:172.21.15.88:3300/0 conn(0x7f55400b5c70 0x7f554015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.169+0000 7f5547c00640 1 -- 172.21.15.50:0/1017706126 >> 172.21.15.50:0/1017706126 conn(0x7f55400b2040 msgr2=0x7f55400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:57.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.169+0000 7f5547c00640 1 -- 172.21.15.50:0/1017706126 shutdown_connections 2024-09-21T06:01:57.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.169+0000 7f5547c00640 1 -- 172.21.15.50:0/1017706126 wait complete. 2024-09-21T06:01:57.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.169+0000 7f5547c00640 1 Processor -- start 2024-09-21T06:01:57.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.169+0000 7f5547c00640 1 -- start start 2024-09-21T06:01:57.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5547c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55400b4de0 0x7f55401498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:57.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5547c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55400b5c70 0x7f55400cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:57.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5547c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55401527b0 0x7f55400ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:57.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5547c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f55401688b0 con 0x7f55401527b0 2024-09-21T06:01:57.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5547c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5540168730 con 0x7f55400b5c70 2024-09-21T06:01:57.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5547c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5540168a30 con 0x7f55400b4de0 2024-09-21T06:01:57.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5545c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55401527b0 0x7f55400ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:57.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5545c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55401527b0 0x7f55400ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44608/0 (socket says 172.21.15.50:44608) 2024-09-21T06:01:57.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5545200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55400b4de0 0x7f55401498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:57.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5545c00640 1 -- 172.21.15.50:0/3398660918 learned_addr learned my addr 172.21.15.50:0/3398660918 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:57.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f553fe00640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55400b5c70 0x7f55400cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:57.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5545c00640 1 -- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55400b4de0 msgr2=0x7f55401498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5545c00640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55400b4de0 0x7f55401498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5545c00640 1 -- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55400b5c70 msgr2=0x7f55400cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5545c00640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55400b5c70 0x7f55400cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5545c00640 1 -- 172.21.15.50:0/3398660918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55400764c0 con 0x7f55401527b0 2024-09-21T06:01:57.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f553fe00640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55400b5c70 0x7f55400cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:01:57.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5545c00640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55401527b0 0x7f55400ce170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f55380986b0 tx=0x7f5538003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:57.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f553cc00640 1 -- 172.21.15.50:0/3398660918 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55380b2040 con 0x7f55401527b0 2024-09-21T06:01:57.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f553cc00640 1 -- 172.21.15.50:0/3398660918 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55380bb070 con 0x7f55401527b0 2024-09-21T06:01:57.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5547c00640 1 -- 172.21.15.50:0/3398660918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554014b840 con 0x7f55401527b0 2024-09-21T06:01:57.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f553cc00640 1 -- 172.21.15.50:0/3398660918 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538004330 con 0x7f55401527b0 2024-09-21T06:01:57.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5547c00640 1 -- 172.21.15.50:0/3398660918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554014c020 con 0x7f55401527b0 2024-09-21T06:01:57.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.173+0000 7f5547c00640 1 -- 172.21.15.50:0/3398660918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55400b51f0 con 0x7f55401527b0 2024-09-21T06:01:57.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.177+0000 7f553cc00640 1 -- 172.21.15.50:0/3398660918 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f55380044d0 con 0x7f55401527b0 2024-09-21T06:01:57.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.177+0000 7f553cc00640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5520073b30 0x7f5520076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:57.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.177+0000 7f553cc00640 1 -- 172.21.15.50:0/3398660918 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7f553812d2f0 con 0x7f55401527b0 2024-09-21T06:01:57.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.177+0000 7f5545200640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5520073b30 0x7f5520076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:57.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.177+0000 7f5545200640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5520073b30 0x7f5520076040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5534010e60 tx=0x7f5534012040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:57.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.181+0000 7f553cc00640 1 -- 172.21.15.50:0/3398660918 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5538016770 con 0x7f55401527b0 2024-09-21T06:01:57.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.393+0000 7f5547c00640 1 -- 172.21.15.50:0/3398660918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f554015c0f0 con 0x7f55401527b0 2024-09-21T06:01:57.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.393+0000 7f553cc00640 1 -- 172.21.15.50:0/3398660918 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5217 (secure 0 0 0) 0x7f55380f9710 con 0x7f55401527b0 2024-09-21T06:01:57.396 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:01:57.396 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":28,"btime":"2024-09-21T05:59:51:480904+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/723044219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":723044219},{"type":"v1","addr":"172.21.15.88:6836","nonce":723044219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1966998855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1966998855},{"type":"v1","addr":"172.21.15.88:6837","nonce":1966998855}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2536951841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2536951841},{"type":"v1","addr":"172.21.15.50:6834","nonce":2536951841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:59:37.853304+0000","modified":"2024-09-21T05:59:40.872885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5370":{"gid":5370,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/988935074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":988935074},{"type":"v1","addr":"172.21.15.50:6835","nonce":988935074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5370,"qdb_cluster":[5370]},"id":3}]} 2024-09-21T06:01:57.396 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 28 2024-09-21T06:01:57.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.397+0000 7f5547c00640 1 -- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5520073b30 msgr2=0x7f5520076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.397+0000 7f5547c00640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5520073b30 0x7f5520076040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5534010e60 tx=0x7f5534012040 comp rx=0 tx=0).stop 2024-09-21T06:01:57.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.397+0000 7f5547c00640 1 -- 172.21.15.50:0/3398660918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55401527b0 msgr2=0x7f55400ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.397+0000 7f5547c00640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55401527b0 0x7f55400ce170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f55380986b0 tx=0x7f5538003f40 comp rx=0 tx=0).stop 2024-09-21T06:01:57.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.397+0000 7f5547c00640 1 -- 172.21.15.50:0/3398660918 shutdown_connections 2024-09-21T06:01:57.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.397+0000 7f5547c00640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5520073b30 0x7f5520076040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.397+0000 7f5547c00640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55401527b0 0x7f55400ce170 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.397+0000 7f5547c00640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55400b5c70 0x7f55400cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.397+0000 7f5547c00640 1 --2- 172.21.15.50:0/3398660918 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55400b4de0 0x7f55401498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.397+0000 7f5547c00640 1 -- 172.21.15.50:0/3398660918 >> 172.21.15.50:0/3398660918 conn(0x7f55400b2040 msgr2=0x7f554014d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:57.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.397+0000 7f5547c00640 1 -- 172.21.15.50:0/3398660918 shutdown_connections 2024-09-21T06:01:57.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.397+0000 7f5547c00640 1 -- 172.21.15.50:0/3398660918 wait complete. 2024-09-21T06:01:57.418 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:01:57.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.577+0000 7fef7aa00640 1 -- 172.21.15.50:0/1242243551 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef740b5c70 msgr2=0x7fef7415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.577+0000 7fef7aa00640 1 --2- 172.21.15.50:0/1242243551 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef740b5c70 0x7fef7415bb60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef64009870 tx=0x7fef6401c7a0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.577+0000 7fef7aa00640 1 -- 172.21.15.50:0/1242243551 shutdown_connections 2024-09-21T06:01:57.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.577+0000 7fef7aa00640 1 --2- 172.21.15.50:0/1242243551 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef740b5c70 0x7fef7415bb60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.577+0000 7fef7aa00640 1 -- 172.21.15.50:0/1242243551 >> 172.21.15.50:0/1242243551 conn(0x7fef740b2040 msgr2=0x7fef740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:57.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.577+0000 7fef7aa00640 1 -- 172.21.15.50:0/1242243551 shutdown_connections 2024-09-21T06:01:57.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.577+0000 7fef7aa00640 1 -- 172.21.15.50:0/1242243551 wait complete. 2024-09-21T06:01:57.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.577+0000 7fef7aa00640 1 Processor -- start 2024-09-21T06:01:57.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.577+0000 7fef7aa00640 1 -- start start 2024-09-21T06:01:57.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef7aa00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef740b4de0 0x7fef741466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef7aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef740b5c70 0x7fef74156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef7aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef741527b0 0x7fef7414b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef7aa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef74169c60 con 0x7fef741527b0 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef7aa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fef74169ae0 con 0x7fef740b5c70 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef7aa00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fef74169de0 con 0x7fef740b4de0 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef73e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef740b4de0 0x7fef741466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef78a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef741527b0 0x7fef7414b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef6b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef740b5c70 0x7fef74156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef73e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef740b4de0 0x7fef741466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:58566/0 (socket says 172.21.15.50:58566) 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef73e00640 1 -- 172.21.15.50:0/1761002275 learned_addr learned my addr 172.21.15.50:0/1761002275 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef78a00640 1 -- 172.21.15.50:0/1761002275 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef740b4de0 msgr2=0x7fef741466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef78a00640 1 --2- 172.21.15.50:0/1761002275 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef740b4de0 0x7fef741466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef78a00640 1 -- 172.21.15.50:0/1761002275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef740b5c70 msgr2=0x7fef74156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef78a00640 1 --2- 172.21.15.50:0/1761002275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef740b5c70 0x7fef74156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef78a00640 1 -- 172.21.15.50:0/1761002275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef74146c30 con 0x7fef741527b0 2024-09-21T06:01:57.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef78a00640 1 --2- 172.21.15.50:0/1761002275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef741527b0 0x7fef7414b540 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c390 tx=0x7fef6c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:57.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef70c00640 1 -- 172.21.15.50:0/1761002275 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c011790 con 0x7fef741527b0 2024-09-21T06:01:57.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef70c00640 1 -- 172.21.15.50:0/1761002275 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef6c017070 con 0x7fef741527b0 2024-09-21T06:01:57.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef70c00640 1 -- 172.21.15.50:0/1761002275 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c011ce0 con 0x7fef741527b0 2024-09-21T06:01:57.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef7aa00640 1 -- 172.21.15.50:0/1761002275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef74156a50 con 0x7fef741527b0 2024-09-21T06:01:57.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef7aa00640 1 -- 172.21.15.50:0/1761002275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef74153c90 con 0x7fef741527b0 2024-09-21T06:01:57.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.581+0000 7fef7aa00640 1 -- 172.21.15.50:0/1761002275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7415bb60 con 0x7fef741527b0 2024-09-21T06:01:57.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.585+0000 7fef70c00640 1 -- 172.21.15.50:0/1761002275 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fef6c01c460 con 0x7fef741527b0 2024-09-21T06:01:57.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.585+0000 7fef70c00640 1 --2- 172.21.15.50:0/1761002275 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef50073b30 0x7fef50076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:57.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.585+0000 7fef70c00640 1 -- 172.21.15.50:0/1761002275 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7fef6c092740 con 0x7fef741527b0 2024-09-21T06:01:57.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.585+0000 7fef73e00640 1 --2- 172.21.15.50:0/1761002275 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef50073b30 0x7fef50076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:57.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.585+0000 7fef73e00640 1 --2- 172.21.15.50:0/1761002275 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef50073b30 0x7fef50076040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef64009870 tx=0x7fef6401afe0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:57.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.589+0000 7fef70c00640 1 -- 172.21.15.50:0/1761002275 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fef6c05eb60 con 0x7fef741527b0 2024-09-21T06:01:57.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.769+0000 7fef7aa00640 1 -- 172.21.15.50:0/1761002275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef740b4a10 con 0x7fef741527b0 2024-09-21T06:01:57.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.769+0000 7fef70c00640 1 -- 172.21.15.50:0/1761002275 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16825 (secure 0 0 0) 0x7fef6c063590 con 0x7fef741527b0 2024-09-21T06:01:57.771 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:01:57.771 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":47,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:01:54.940117+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:59:35.596192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T05:59:36.818395+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":4}]},{"pgid":"7.35","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:01:57.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.773+0000 7fef7aa00640 1 -- 172.21.15.50:0/1761002275 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef50073b30 msgr2=0x7fef50076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.773+0000 7fef7aa00640 1 --2- 172.21.15.50:0/1761002275 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef50073b30 0x7fef50076040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef64009870 tx=0x7fef6401afe0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.773+0000 7fef7aa00640 1 -- 172.21.15.50:0/1761002275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef741527b0 msgr2=0x7fef7414b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.773+0000 7fef7aa00640 1 --2- 172.21.15.50:0/1761002275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef741527b0 0x7fef7414b540 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c390 tx=0x7fef6c00c850 comp rx=0 tx=0).stop 2024-09-21T06:01:57.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.773+0000 7fef7aa00640 1 -- 172.21.15.50:0/1761002275 shutdown_connections 2024-09-21T06:01:57.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.773+0000 7fef7aa00640 1 --2- 172.21.15.50:0/1761002275 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef50073b30 0x7fef50076040 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.773+0000 7fef7aa00640 1 --2- 172.21.15.50:0/1761002275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef741527b0 0x7fef7414b540 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.773+0000 7fef7aa00640 1 --2- 172.21.15.50:0/1761002275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef740b5c70 0x7fef74156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.773+0000 7fef7aa00640 1 --2- 172.21.15.50:0/1761002275 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef740b4de0 0x7fef741466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.773+0000 7fef7aa00640 1 -- 172.21.15.50:0/1761002275 >> 172.21.15.50:0/1761002275 conn(0x7fef740b2040 msgr2=0x7fef74159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:57.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.773+0000 7fef7aa00640 1 -- 172.21.15.50:0/1761002275 shutdown_connections 2024-09-21T06:01:57.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.773+0000 7fef7aa00640 1 -- 172.21.15.50:0/1761002275 wait complete. 2024-09-21T06:01:57.793 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:01:57.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.953+0000 7fa82b000640 1 -- 172.21.15.50:0/2091211822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8240749a0 msgr2=0x7fa824074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.953+0000 7fa82b000640 1 --2- 172.21.15.50:0/2091211822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8240749a0 0x7fa824074db0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa8240726f0 tx=0x7fa81401c800 comp rx=0 tx=0).stop 2024-09-21T06:01:57.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.953+0000 7fa82b000640 1 -- 172.21.15.50:0/2091211822 shutdown_connections 2024-09-21T06:01:57.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.953+0000 7fa82b000640 1 --2- 172.21.15.50:0/2091211822 >> v2:172.21.15.88:3300/0 conn(0x7fa8240cefa0 0x7fa8240cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.953+0000 7fa82b000640 1 --2- 172.21.15.50:0/2091211822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8240749a0 0x7fa824074db0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.953+0000 7fa82b000640 1 -- 172.21.15.50:0/2091211822 >> 172.21.15.50:0/2091211822 conn(0x7fa8240b2040 msgr2=0x7fa8240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:57.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.953+0000 7fa82b000640 1 -- 172.21.15.50:0/2091211822 shutdown_connections 2024-09-21T06:01:57.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.953+0000 7fa82b000640 1 -- 172.21.15.50:0/2091211822 wait complete. 2024-09-21T06:01:57.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.953+0000 7fa82b000640 1 Processor -- start 2024-09-21T06:01:57.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.953+0000 7fa82b000640 1 -- start start 2024-09-21T06:01:57.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa82b000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8240749a0 0x7fa8241fce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:57.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa82b000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8240cefa0 0x7fa824157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:57.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa82b000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa824160330 0x7fa82413ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:57.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa82b000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa82416a060 con 0x7fa824160330 2024-09-21T06:01:57.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa82b000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa824169ee0 con 0x7fa8240749a0 2024-09-21T06:01:57.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa82b000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa82416a1e0 con 0x7fa8240cefa0 2024-09-21T06:01:57.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa829000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa824160330 0x7fa82413ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:57.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa823e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8240749a0 0x7fa8241fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:57.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa829000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa824160330 0x7fa82413ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44642/0 (socket says 172.21.15.50:44642) 2024-09-21T06:01:57.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa829000640 1 -- 172.21.15.50:0/2703897528 learned_addr learned my addr 172.21.15.50:0/2703897528 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:57.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa823400640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8240cefa0 0x7fa824157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:57.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa829000640 1 -- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8240cefa0 msgr2=0x7fa824157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa829000640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8240cefa0 0x7fa824157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa829000640 1 -- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8240749a0 msgr2=0x7fa8241fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:57.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa829000640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8240749a0 0x7fa8241fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:57.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa829000640 1 -- 172.21.15.50:0/2703897528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8240b1e50 con 0x7fa824160330 2024-09-21T06:01:57.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa829000640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa824160330 0x7fa82413ba20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00cae0 tx=0x7fa81c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:57.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa81be00640 1 -- 172.21.15.50:0/2703897528 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c016020 con 0x7fa824160330 2024-09-21T06:01:57.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa81be00640 1 -- 172.21.15.50:0/2703897528 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa81c00ab40 con 0x7fa824160330 2024-09-21T06:01:57.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa82b000640 1 -- 172.21.15.50:0/2703897528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8240b1670 con 0x7fa824160330 2024-09-21T06:01:57.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa81be00640 1 -- 172.21.15.50:0/2703897528 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00ce20 con 0x7fa824160330 2024-09-21T06:01:57.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa82b000640 1 -- 172.21.15.50:0/2703897528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8240b1b70 con 0x7fa824160330 2024-09-21T06:01:57.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.957+0000 7fa82b000640 1 -- 172.21.15.50:0/2703897528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8240b18e0 con 0x7fa824160330 2024-09-21T06:01:57.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.961+0000 7fa81be00640 1 -- 172.21.15.50:0/2703897528 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fa81c019020 con 0x7fa824160330 2024-09-21T06:01:57.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.961+0000 7fa81be00640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa804073b00 0x7fa804076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:57.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.961+0000 7fa81be00640 1 -- 172.21.15.50:0/2703897528 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7fa81c0956e0 con 0x7fa824160330 2024-09-21T06:01:57.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.961+0000 7fa823e00640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa804073b00 0x7fa804076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:57.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.961+0000 7fa823e00640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa804073b00 0x7fa804076010 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa8140af000 tx=0x7fa81401b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:57.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:57.965+0000 7fa81be00640 1 -- 172.21.15.50:0/2703897528 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa81c020020 con 0x7fa824160330 2024-09-21T06:01:58.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.181+0000 7fa82b000640 1 -- 172.21.15.50:0/2703897528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa824074db0 con 0x7fa824160330 2024-09-21T06:01:58.181 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-09-21T06:01:58.181+0000 7f0d45000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:01:58.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.181+0000 7fa81be00640 1 -- 172.21.15.50:0/2703897528 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7fa81c05f0b0 con 0x7fa824160330 2024-09-21T06:01:58.183 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:01:58.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.185+0000 7fa82b000640 1 -- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa804073b00 msgr2=0x7fa804076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:58.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.185+0000 7fa82b000640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa804073b00 0x7fa804076010 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa8140af000 tx=0x7fa81401b040 comp rx=0 tx=0).stop 2024-09-21T06:01:58.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.185+0000 7fa82b000640 1 -- 172.21.15.50:0/2703897528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa824160330 msgr2=0x7fa82413ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:58.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.185+0000 7fa82b000640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa824160330 0x7fa82413ba20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00cae0 tx=0x7fa81c00cbd0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.185+0000 7fa82b000640 1 -- 172.21.15.50:0/2703897528 shutdown_connections 2024-09-21T06:01:58.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.185+0000 7fa82b000640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa804073b00 0x7fa804076010 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa8140af000 tx=0x7fa81401b040 comp rx=0 tx=0).stop 2024-09-21T06:01:58.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.185+0000 7fa82b000640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa824160330 0x7fa82413ba20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.185+0000 7fa82b000640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8240cefa0 0x7fa824157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.185+0000 7fa82b000640 1 --2- 172.21.15.50:0/2703897528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8240749a0 0x7fa8241fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.185+0000 7fa82b000640 1 -- 172.21.15.50:0/2703897528 >> 172.21.15.50:0/2703897528 conn(0x7fa8240b2040 msgr2=0x7fa824159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:58.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.189+0000 7fa82b000640 1 -- 172.21.15.50:0/2703897528 shutdown_connections 2024-09-21T06:01:58.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.189+0000 7fa82b000640 1 -- 172.21.15.50:0/2703897528 wait complete. 2024-09-21T06:01:58.210 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:01:58.327 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T06:01:58.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.373+0000 7f471f800640 1 -- 172.21.15.50:0/1455492683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4700004810 msgr2=0x7f4700004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:58.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.373+0000 7f471f800640 1 --2- 172.21.15.50:0/1455492683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4700004810 0x7f4700004c20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4704001860 tx=0x7f470401f050 comp rx=0 tx=0).stop 2024-09-21T06:01:58.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.373+0000 7f471d800640 1 -- 172.21.15.50:0/1455492683 reap_dead start 2024-09-21T06:01:58.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.373+0000 7f471f800640 1 -- 172.21.15.50:0/1455492683 shutdown_connections 2024-09-21T06:01:58.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.373+0000 7f471f800640 1 -- 172.21.15.50:0/1455492683 >> 172.21.15.50:0/1455492683 conn(0x7f47180b2040 msgr2=0x7f47180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:58.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.373+0000 7f471f800640 1 -- 172.21.15.50:0/1455492683 shutdown_connections 2024-09-21T06:01:58.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.373+0000 7f471f800640 1 -- 172.21.15.50:0/1455492683 wait complete. 2024-09-21T06:01:58.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471f800640 1 Processor -- start 2024-09-21T06:01:58.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471f800640 1 -- start start 2024-09-21T06:01:58.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471f800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4700004810 0x7f4718144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:58.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471f800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4718155b60 0x7f4718155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:58.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471f800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47181564b0 0x7f47181f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:58.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471f800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f47180b5eb0 con 0x7f47181564b0 2024-09-21T06:01:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471f800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f47180b5d30 con 0x7f4700004810 2024-09-21T06:01:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471f800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f47180702f0 con 0x7f4718155b60 2024-09-21T06:01:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47181564b0 0x7f47181f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47181564b0 0x7f47181f8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44680/0 (socket says 172.21.15.50:44680) 2024-09-21T06:01:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471d800640 1 -- 172.21.15.50:0/3433681904 learned_addr learned my addr 172.21.15.50:0/3433681904 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471ce00640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4700004810 0x7f4718144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f4717e00640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4718155b60 0x7f4718155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:58.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471d800640 1 -- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4718155b60 msgr2=0x7f4718155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:58.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471d800640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4718155b60 0x7f4718155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471d800640 1 -- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4700004810 msgr2=0x7f4718144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:58.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471d800640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4700004810 0x7f4718144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471d800640 1 -- 172.21.15.50:0/3433681904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47181e89b0 con 0x7f47181564b0 2024-09-21T06:01:58.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.377+0000 7f471d800640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47181564b0 0x7f47181f8fc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f47180cc770 tx=0x7f4710006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:58.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.381+0000 7f4714c00640 1 -- 172.21.15.50:0/3433681904 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471000fa80 con 0x7f47181564b0 2024-09-21T06:01:58.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.381+0000 7f4714c00640 1 -- 172.21.15.50:0/3433681904 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4710018070 con 0x7f47181564b0 2024-09-21T06:01:58.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.381+0000 7f4714c00640 1 -- 172.21.15.50:0/3433681904 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710011430 con 0x7f47181564b0 2024-09-21T06:01:58.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.381+0000 7f471f800640 1 -- 172.21.15.50:0/3433681904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4718155660 con 0x7f47181564b0 2024-09-21T06:01:58.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.381+0000 7f471f800640 1 -- 172.21.15.50:0/3433681904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47181eba80 con 0x7f47181564b0 2024-09-21T06:01:58.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.381+0000 7f471f800640 1 -- 172.21.15.50:0/3433681904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47181e8b80 con 0x7f47181564b0 2024-09-21T06:01:58.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.381+0000 7f4714c00640 1 -- 172.21.15.50:0/3433681904 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f47100115d0 con 0x7f47181564b0 2024-09-21T06:01:58.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.381+0000 7f4714c00640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f46f8073b00 0x7f46f8076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:58.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.381+0000 7f4714c00640 1 -- 172.21.15.50:0/3433681904 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5881+0+0 (secure 0 0 0) 0x7f47100932c0 con 0x7f47181564b0 2024-09-21T06:01:58.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.381+0000 7f471ce00640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f46f8073b00 0x7f46f8076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:58.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.385+0000 7f471ce00640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f46f8073b00 0x7f46f8076010 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f471806f210 tx=0x7f470c00a2f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:58.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.385+0000 7f4714c00640 1 -- 172.21.15.50:0/3433681904 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f47181e8b80 con 0x7f47181564b0 2024-09-21T06:01:58.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:58.596+0000 7f471f800640 1 -- 172.21.15.50:0/3433681904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f47180b27b0 con 0x7f47181564b0 2024-09-21T06:01:59.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.184+0000 7f4714c00640 1 -- 172.21.15.50:0/3433681904 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f471005f6e0 con 0x7f47181564b0 2024-09-21T06:01:59.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.184+0000 7f471f800640 1 -- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f46f8073b00 msgr2=0x7f46f8076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:59.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.184+0000 7f471f800640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f46f8073b00 0x7f46f8076010 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f471806f210 tx=0x7f470c00a2f0 comp rx=0 tx=0).stop 2024-09-21T06:01:59.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.184+0000 7f471f800640 1 -- 172.21.15.50:0/3433681904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47181564b0 msgr2=0x7f47181f8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:59.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.184+0000 7f471f800640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47181564b0 0x7f47181f8fc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f47180cc770 tx=0x7f4710006720 comp rx=0 tx=0).stop 2024-09-21T06:01:59.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.188+0000 7f471f800640 1 -- 172.21.15.50:0/3433681904 shutdown_connections 2024-09-21T06:01:59.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.188+0000 7f471f800640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f46f8073b00 0x7f46f8076010 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:59.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.188+0000 7f471f800640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47181564b0 0x7f47181f8fc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:59.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.188+0000 7f471f800640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4718155b60 0x7f4718155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:59.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.188+0000 7f471f800640 1 --2- 172.21.15.50:0/3433681904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4700004810 0x7f4718144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:59.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.188+0000 7f471f800640 1 -- 172.21.15.50:0/3433681904 >> 172.21.15.50:0/3433681904 conn(0x7f47180b2040 msgr2=0x7f4718159fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:59.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.188+0000 7f471f800640 1 -- 172.21.15.50:0/3433681904 shutdown_connections 2024-09-21T06:01:59.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.188+0000 7f471f800640 1 -- 172.21.15.50:0/3433681904 wait complete. 2024-09-21T06:01:59.213 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:01:59.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.380+0000 7fa6f3e00640 1 -- 172.21.15.50:0/1464706568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d8005360 msgr2=0x7fa6d800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:59.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.380+0000 7fa6f3e00640 1 --2- 172.21.15.50:0/1464706568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d8005360 0x7fa6d800b9d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00c630 tx=0x7fa6ec01fa20 comp rx=0 tx=0).stop 2024-09-21T06:01:59.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.380+0000 7fa6f3e00640 1 -- 172.21.15.50:0/1464706568 shutdown_connections 2024-09-21T06:01:59.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.380+0000 7fa6f3e00640 1 --2- 172.21.15.50:0/1464706568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d8005360 0x7fa6d800b9d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:59.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.380+0000 7fa6f3e00640 1 -- 172.21.15.50:0/1464706568 >> 172.21.15.50:0/1464706568 conn(0x7fa6f40b41f0 msgr2=0x7fa6f40b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:59.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.380+0000 7fa6f3e00640 1 -- 172.21.15.50:0/1464706568 shutdown_connections 2024-09-21T06:01:59.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.380+0000 7fa6f3e00640 1 -- 172.21.15.50:0/1464706568 wait complete. 2024-09-21T06:01:59.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f3e00640 1 Processor -- start 2024-09-21T06:01:59.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f3e00640 1 -- start start 2024-09-21T06:01:59.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d8005360 0x7fa6f40cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:59.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f3e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f4154c30 0x7fa6f415e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:59.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f415bde0 0x7fa6f40ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:59.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f40b1db0 con 0x7fa6d8005360 2024-09-21T06:01:59.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f40b1c30 con 0x7fa6f415bde0 2024-09-21T06:01:59.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f3e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f40b0730 con 0x7fa6f4154c30 2024-09-21T06:01:59.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d8005360 0x7fa6f40cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:59.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d8005360 0x7fa6f40cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44706/0 (socket says 172.21.15.50:44706) 2024-09-21T06:01:59.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f2a00640 1 -- 172.21.15.50:0/546572961 learned_addr learned my addr 172.21.15.50:0/546572961 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:01:59.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f3400640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f415bde0 0x7fa6f40ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:59.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6ebe00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f4154c30 0x7fa6f415e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:59.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f2a00640 1 -- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f4154c30 msgr2=0x7fa6f415e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:59.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f2a00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f4154c30 0x7fa6f415e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:59.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f2a00640 1 -- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f415bde0 msgr2=0x7fa6f40ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:59.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f2a00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f415bde0 0x7fa6f40ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:59.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f2a00640 1 -- 172.21.15.50:0/546572961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f40708a0 con 0x7fa6d8005360 2024-09-21T06:01:59.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f2a00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d8005360 0x7fa6f40cc7b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4011ae0 tx=0x7fa6e4011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:59.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6eaa00640 1 -- 172.21.15.50:0/546572961 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400f9a0 con 0x7fa6d8005360 2024-09-21T06:01:59.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6eaa00640 1 -- 172.21.15.50:0/546572961 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6e400fb40 con 0x7fa6d8005360 2024-09-21T06:01:59.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6eaa00640 1 -- 172.21.15.50:0/546572961 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4003600 con 0x7fa6d8005360 2024-09-21T06:01:59.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f3e00640 1 -- 172.21.15.50:0/546572961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f41553d0 con 0x7fa6d8005360 2024-09-21T06:01:59.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.384+0000 7fa6f3e00640 1 -- 172.21.15.50:0/546572961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f40ccd80 con 0x7fa6d8005360 2024-09-21T06:01:59.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.388+0000 7fa6f3e00640 1 -- 172.21.15.50:0/546572961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f4070d80 con 0x7fa6d8005360 2024-09-21T06:01:59.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.388+0000 7fa6eaa00640 1 -- 172.21.15.50:0/546572961 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fa6e40037a0 con 0x7fa6d8005360 2024-09-21T06:01:59.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.388+0000 7fa6eaa00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6d0073b30 0x7fa6d0076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:59.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.388+0000 7fa6eaa00640 1 -- 172.21.15.50:0/546572961 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5881+0+0 (secure 0 0 0) 0x7fa6e4093ad0 con 0x7fa6d8005360 2024-09-21T06:01:59.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.388+0000 7fa6ebe00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6d0073b30 0x7fa6d0076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:59.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.388+0000 7fa6ebe00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6d0073b30 0x7fa6d0076040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40d0bc0 tx=0x7fa6dc006b80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:59.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.392+0000 7fa6eaa00640 1 -- 172.21.15.50:0/546572961 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa6f4070d80 con 0x7fa6d8005360 2024-09-21T06:01:59.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:01:59.572+0000 7fa6f3e00640 1 -- 172.21.15.50:0/546572961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa6f40ccfe0 con 0x7fa6d8005360 2024-09-21T06:02:00.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.184+0000 7fa6eaa00640 1 -- 172.21.15.50:0/546572961 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) ==== 181+0+0 (secure 0 0 0) 0x7fa6e40605d0 con 0x7fa6d8005360 2024-09-21T06:02:00.188 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:02:00.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.188+0000 7fa6f3e00640 1 -- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6d0073b30 msgr2=0x7fa6d0076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:00.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.188+0000 7fa6f3e00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6d0073b30 0x7fa6d0076040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40d0bc0 tx=0x7fa6dc006b80 comp rx=0 tx=0).stop 2024-09-21T06:02:00.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.188+0000 7fa6f3e00640 1 -- 172.21.15.50:0/546572961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d8005360 msgr2=0x7fa6f40cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:00.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.188+0000 7fa6f3e00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d8005360 0x7fa6f40cc7b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4011ae0 tx=0x7fa6e4011ee0 comp rx=0 tx=0).stop 2024-09-21T06:02:00.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.188+0000 7fa6f3e00640 1 -- 172.21.15.50:0/546572961 shutdown_connections 2024-09-21T06:02:00.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.188+0000 7fa6f3e00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f415bde0 0x7fa6f40ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:00.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.188+0000 7fa6f3e00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f4154c30 0x7fa6f415e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:00.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.188+0000 7fa6f3e00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6d0073b30 0x7fa6d0076040 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:00.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.188+0000 7fa6f3e00640 1 --2- 172.21.15.50:0/546572961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d8005360 0x7fa6f40cc7b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:00.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.188+0000 7fa6f3e00640 1 -- 172.21.15.50:0/546572961 >> 172.21.15.50:0/546572961 conn(0x7fa6f40b41f0 msgr2=0x7fa6f40af430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:00.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.188+0000 7fa6f3e00640 1 -- 172.21.15.50:0/546572961 shutdown_connections 2024-09-21T06:02:00.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.188+0000 7fa6f3e00640 1 -- 172.21.15.50:0/546572961 wait complete. 2024-09-21T06:02:00.211 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:02:00.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.372+0000 7fa51d800640 1 -- 172.21.15.50:0/56105776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4fc003a10 msgr2=0x7fa4fc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:00.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.372+0000 7fa51d800640 1 --2- 172.21.15.50:0/56105776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4fc003a10 0x7fa4fc003e80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa508009b40 tx=0x7fa50801d270 comp rx=0 tx=0).stop 2024-09-21T06:02:00.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.372+0000 7fa517400640 1 -- 172.21.15.50:0/56105776 reap_dead start 2024-09-21T06:02:00.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.372+0000 7fa51d800640 1 -- 172.21.15.50:0/56105776 shutdown_connections 2024-09-21T06:02:00.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.372+0000 7fa51d800640 1 -- 172.21.15.50:0/56105776 >> 172.21.15.50:0/56105776 conn(0x7fa5180b2040 msgr2=0x7fa5180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:00.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.372+0000 7fa51d800640 1 -- 172.21.15.50:0/56105776 shutdown_connections 2024-09-21T06:02:00.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.372+0000 7fa51d800640 1 -- 172.21.15.50:0/56105776 wait complete. 2024-09-21T06:02:00.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa51d800640 1 Processor -- start 2024-09-21T06:02:00.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa51d800640 1 -- start start 2024-09-21T06:02:00.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa51d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4fc003a10 0x7fa518159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:00.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa51d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa518158c30 0x7fa518151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:00.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa51d800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa5181f9f90 0x7fa5181f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:00.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa51d800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa51815cf30 con 0x7fa518158c30 2024-09-21T06:02:00.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa51d800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa51815cdb0 con 0x7fa4fc003a10 2024-09-21T06:02:00.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa51d800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa51815d0b0 con 0x7fa5181f9f90 2024-09-21T06:02:00.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa516000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa518158c30 0x7fa518151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:00.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa516000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa518158c30 0x7fa518151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44724/0 (socket says 172.21.15.50:44724) 2024-09-21T06:02:00.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa516000640 1 -- 172.21.15.50:0/801028271 learned_addr learned my addr 172.21.15.50:0/801028271 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:00.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa517400640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa5181f9f90 0x7fa5181f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:00.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa516a00640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4fc003a10 0x7fa518159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:00.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa516000640 1 -- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa5181f9f90 msgr2=0x7fa5181f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:00.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa516000640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa5181f9f90 0x7fa5181f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:00.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa516000640 1 -- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4fc003a10 msgr2=0x7fa518159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:00.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa516000640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4fc003a10 0x7fa518159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:00.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.376+0000 7fa516000640 1 -- 172.21.15.50:0/801028271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5181e2850 con 0x7fa518158c30 2024-09-21T06:02:00.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.380+0000 7fa516000640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa518158c30 0x7fa518151e90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa504004610 tx=0x7fa5040127b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:00.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.380+0000 7fa50ea00640 1 -- 172.21.15.50:0/801028271 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504013860 con 0x7fa518158c30 2024-09-21T06:02:00.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.380+0000 7fa50ea00640 1 -- 172.21.15.50:0/801028271 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa504013a00 con 0x7fa518158c30 2024-09-21T06:02:00.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.380+0000 7fa51d800640 1 -- 172.21.15.50:0/801028271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa518158150 con 0x7fa518158c30 2024-09-21T06:02:00.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.380+0000 7fa50ea00640 1 -- 172.21.15.50:0/801028271 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504013e50 con 0x7fa518158c30 2024-09-21T06:02:00.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.380+0000 7fa51d800640 1 -- 172.21.15.50:0/801028271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa518158710 con 0x7fa518158c30 2024-09-21T06:02:00.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.380+0000 7fa51d800640 1 -- 172.21.15.50:0/801028271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5181e25c0 con 0x7fa518158c30 2024-09-21T06:02:00.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.380+0000 7fa50ea00640 1 -- 172.21.15.50:0/801028271 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fa504027440 con 0x7fa518158c30 2024-09-21T06:02:00.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.380+0000 7fa50ea00640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4f8073cd0 0x7fa4f80761e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:00.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.380+0000 7fa50ea00640 1 -- 172.21.15.50:0/801028271 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5443+0+0 (secure 0 0 0) 0x7fa504096df0 con 0x7fa518158c30 2024-09-21T06:02:00.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.384+0000 7fa516a00640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4f8073cd0 0x7fa4f80761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:00.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.384+0000 7fa516a00640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4f8073cd0 0x7fa4f80761e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa5080ad000 tx=0x7fa50801a010 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:00.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.384+0000 7fa50ea00640 1 -- 172.21.15.50:0/801028271 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa5180b5730 con 0x7fa518158c30 2024-09-21T06:02:00.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:00.564+0000 7fa51d800640 1 -- 172.21.15.50:0/801028271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa5181523d0 con 0x7fa518158c30 2024-09-21T06:02:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.188+0000 7fa50ea00640 1 -- 172.21.15.50:0/801028271 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) ==== 169+0+0 (secure 0 0 0) 0x7fa5181523d0 con 0x7fa518158c30 2024-09-21T06:02:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-09-21T06:02:01.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.192+0000 7fa51d800640 1 -- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4f8073cd0 msgr2=0x7fa4f80761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.192+0000 7fa51d800640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4f8073cd0 0x7fa4f80761e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa5080ad000 tx=0x7fa50801a010 comp rx=0 tx=0).stop 2024-09-21T06:02:01.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.192+0000 7fa51d800640 1 -- 172.21.15.50:0/801028271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa518158c30 msgr2=0x7fa518151e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.192+0000 7fa51d800640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa518158c30 0x7fa518151e90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa504004610 tx=0x7fa5040127b0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.192+0000 7fa51d800640 1 -- 172.21.15.50:0/801028271 shutdown_connections 2024-09-21T06:02:01.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.192+0000 7fa51d800640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa5181f9f90 0x7fa5181f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.192+0000 7fa51d800640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4f8073cd0 0x7fa4f80761e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.192+0000 7fa51d800640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa518158c30 0x7fa518151e90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.192+0000 7fa51d800640 1 --2- 172.21.15.50:0/801028271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4fc003a10 0x7fa518159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.192+0000 7fa51d800640 1 -- 172.21.15.50:0/801028271 >> 172.21.15.50:0/801028271 conn(0x7fa5180b2040 msgr2=0x7fa518154bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:01.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.192+0000 7fa51d800640 1 -- 172.21.15.50:0/801028271 shutdown_connections 2024-09-21T06:02:01.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.192+0000 7fa51d800640 1 -- 172.21.15.50:0/801028271 wait complete. 2024-09-21T06:02:01.225 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:02:01.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.380+0000 7f45e4c00640 1 -- 172.21.15.50:0/4232479618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45e015eeb0 msgr2=0x7f45e00cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.380+0000 7f45e4c00640 1 --2- 172.21.15.50:0/4232479618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45e015eeb0 0x7f45e00cf0e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f45e0075770 tx=0x7f45d001c8e0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.380+0000 7f45e4c00640 1 -- 172.21.15.50:0/4232479618 shutdown_connections 2024-09-21T06:02:01.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.380+0000 7f45e4c00640 1 --2- 172.21.15.50:0/4232479618 >> v2:172.21.15.88:3300/0 conn(0x7f45e015b810 0x7f45e015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.380+0000 7f45e4c00640 1 --2- 172.21.15.50:0/4232479618 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45e00cf620 0x7f45e015b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.380+0000 7f45e4c00640 1 --2- 172.21.15.50:0/4232479618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45e015eeb0 0x7f45e00cf0e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.380+0000 7f45e4c00640 1 -- 172.21.15.50:0/4232479618 >> 172.21.15.50:0/4232479618 conn(0x7f45e00b2040 msgr2=0x7f45e00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:01.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.380+0000 7f45e4c00640 1 -- 172.21.15.50:0/4232479618 shutdown_connections 2024-09-21T06:02:01.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.380+0000 7f45e4c00640 1 -- 172.21.15.50:0/4232479618 wait complete. 2024-09-21T06:02:01.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.384+0000 7f45e4c00640 1 Processor -- start 2024-09-21T06:02:01.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.384+0000 7f45e4c00640 1 -- start start 2024-09-21T06:02:01.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.384+0000 7f45e4c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45e00cf620 0x7f45e01533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.384+0000 7f45e4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45e015b810 0x7f45e00ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.384+0000 7f45e4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45e015eeb0 0x7f45e00aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.384+0000 7f45e4c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f45e00b3880 con 0x7f45e015b810 2024-09-21T06:02:01.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.384+0000 7f45e4c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f45e00b3700 con 0x7f45e015eeb0 2024-09-21T06:02:01.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.384+0000 7f45e4c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f45e00b3a00 con 0x7f45e00cf620 2024-09-21T06:02:01.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.384+0000 7f45d7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45e015b810 0x7f45e00ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45d7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45e015b810 0x7f45e00ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44732/0 (socket says 172.21.15.50:44732) 2024-09-21T06:02:01.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45d7e00640 1 -- 172.21.15.50:0/281626626 learned_addr learned my addr 172.21.15.50:0/281626626 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:01.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45dde00640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45e00cf620 0x7f45e01533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45de800640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45e015eeb0 0x7f45e00aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45d7e00640 1 -- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45e00cf620 msgr2=0x7f45e01533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45d7e00640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45e00cf620 0x7f45e01533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45d7e00640 1 -- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45e015eeb0 msgr2=0x7f45e00aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45d7e00640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45e015eeb0 0x7f45e00aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45d7e00640 1 -- 172.21.15.50:0/281626626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e0162460 con 0x7f45e015b810 2024-09-21T06:02:01.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45d7e00640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45e015b810 0x7f45e00ae8e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f45c800b410 tx=0x7f45c800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:01.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45d6000640 1 -- 172.21.15.50:0/281626626 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8015020 con 0x7f45e015b810 2024-09-21T06:02:01.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45d6000640 1 -- 172.21.15.50:0/281626626 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45c8011920 con 0x7f45e015b810 2024-09-21T06:02:01.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45d6000640 1 -- 172.21.15.50:0/281626626 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8011d70 con 0x7f45e015b810 2024-09-21T06:02:01.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45e4c00640 1 -- 172.21.15.50:0/281626626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e0162740 con 0x7f45e015b810 2024-09-21T06:02:01.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45e4c00640 1 -- 172.21.15.50:0/281626626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e01641a0 con 0x7f45e015b810 2024-09-21T06:02:01.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45e4c00640 1 -- 172.21.15.50:0/281626626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e013abc0 con 0x7f45e015b810 2024-09-21T06:02:01.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.388+0000 7f45d6000640 1 -- 172.21.15.50:0/281626626 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f45c8018020 con 0x7f45e015b810 2024-09-21T06:02:01.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.392+0000 7f45d6000640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45bc073b00 0x7f45bc076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.392+0000 7f45d6000640 1 -- 172.21.15.50:0/281626626 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4971+0+0 (secure 0 0 0) 0x7f45c80928f0 con 0x7f45e015b810 2024-09-21T06:02:01.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.392+0000 7f45dde00640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45bc073b00 0x7f45bc076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.392+0000 7f45dde00640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45bc073b00 0x7f45bc076010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f45d00af000 tx=0x7f45d001b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:01.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.396+0000 7f45d6000640 1 -- 172.21.15.50:0/281626626 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f45c801f020 con 0x7f45e015b810 2024-09-21T06:02:01.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.572+0000 7f45e4c00640 1 -- 172.21.15.50:0/281626626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f45e015bc20 con 0x7f45e015b810 2024-09-21T06:02:01.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.576+0000 7f45d6000640 1 -- 172.21.15.50:0/281626626 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7f45c805eda0 con 0x7f45e015b810 2024-09-21T06:02:01.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.580+0000 7f45e4c00640 1 -- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45bc073b00 msgr2=0x7f45bc076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.580+0000 7f45e4c00640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45bc073b00 0x7f45bc076010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f45d00af000 tx=0x7f45d001b120 comp rx=0 tx=0).stop 2024-09-21T06:02:01.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.580+0000 7f45e4c00640 1 -- 172.21.15.50:0/281626626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45e015b810 msgr2=0x7f45e00ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.580+0000 7f45e4c00640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45e015b810 0x7f45e00ae8e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f45c800b410 tx=0x7f45c800b8d0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.580+0000 7f45e4c00640 1 -- 172.21.15.50:0/281626626 shutdown_connections 2024-09-21T06:02:01.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.580+0000 7f45e4c00640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45e015eeb0 0x7f45e00aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.580+0000 7f45e4c00640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45bc073b00 0x7f45bc076010 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.580+0000 7f45e4c00640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45e015b810 0x7f45e00ae8e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.580+0000 7f45e4c00640 1 --2- 172.21.15.50:0/281626626 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45e00cf620 0x7f45e01533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.580+0000 7f45e4c00640 1 -- 172.21.15.50:0/281626626 >> 172.21.15.50:0/281626626 conn(0x7f45e00b2040 msgr2=0x7f45e00cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:01.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.580+0000 7f45e4c00640 1 -- 172.21.15.50:0/281626626 shutdown_connections 2024-09-21T06:02:01.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.580+0000 7f45e4c00640 1 -- 172.21.15.50:0/281626626 wait complete. 2024-09-21T06:02:01.601 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-21T06:02:01.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.760+0000 7f2a69e00640 1 -- 172.21.15.50:0/3309629598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a64155a00 msgr2=0x7f2a6415f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.760+0000 7f2a69e00640 1 --2- 172.21.15.50:0/3309629598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a64155a00 0x7f2a6415f070 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009870 tx=0x7f2a5401c760 comp rx=0 tx=0).stop 2024-09-21T06:02:01.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.760+0000 7f2a69e00640 1 -- 172.21.15.50:0/3309629598 shutdown_connections 2024-09-21T06:02:01.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.760+0000 7f2a69e00640 1 --2- 172.21.15.50:0/3309629598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a64155a00 0x7f2a6415f070 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.760+0000 7f2a69e00640 1 -- 172.21.15.50:0/3309629598 >> 172.21.15.50:0/3309629598 conn(0x7f2a6406cdf0 msgr2=0x7f2a6406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:01.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.760+0000 7f2a69e00640 1 -- 172.21.15.50:0/3309629598 shutdown_connections 2024-09-21T06:02:01.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.760+0000 7f2a69e00640 1 -- 172.21.15.50:0/3309629598 wait complete. 2024-09-21T06:02:01.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.760+0000 7f2a69e00640 1 Processor -- start 2024-09-21T06:02:01.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.760+0000 7f2a69e00640 1 -- start start 2024-09-21T06:02:01.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a69e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a64155a00 0x7f2a6413b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a69e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6415f5b0 0x7f2a64151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a69e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a641f80b0 0x7f2a640aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a69e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64169a70 con 0x7f2a6415f5b0 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a69e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641698f0 con 0x7f2a64155a00 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a69e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2a64169bf0 con 0x7f2a641f80b0 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6415f5b0 0x7f2a64151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6415f5b0 0x7f2a64151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44748/0 (socket says 172.21.15.50:44748) 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a5be00640 1 -- 172.21.15.50:0/878167196 learned_addr learned my addr 172.21.15.50:0/878167196 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a68a00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a64155a00 0x7f2a6413b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a69400640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a641f80b0 0x7f2a640aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a5be00640 1 -- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a641f80b0 msgr2=0x7f2a640aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a5be00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a641f80b0 0x7f2a640aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a5be00640 1 -- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a64155a00 msgr2=0x7f2a6413b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a5be00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a64155a00 0x7f2a6413b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a5be00640 1 -- 172.21.15.50:0/878167196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6413bf50 con 0x7f2a6415f5b0 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a69400640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a641f80b0 0x7f2a640aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:02:01.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a68a00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a64155a00 0x7f2a6413b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:01.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a5be00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6415f5b0 0x7f2a64151b70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00c3c0 tx=0x7f2a4c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:01.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a61600640 1 -- 172.21.15.50:0/878167196 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c011790 con 0x7f2a6415f5b0 2024-09-21T06:02:01.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a61600640 1 -- 172.21.15.50:0/878167196 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a4c017070 con 0x7f2a6415f5b0 2024-09-21T06:02:01.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a69e00640 1 -- 172.21.15.50:0/878167196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6413c230 con 0x7f2a6415f5b0 2024-09-21T06:02:01.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a61600640 1 -- 172.21.15.50:0/878167196 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c011ca0 con 0x7f2a6415f5b0 2024-09-21T06:02:01.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a69e00640 1 -- 172.21.15.50:0/878167196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a641fb900 con 0x7f2a6415f5b0 2024-09-21T06:02:01.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.764+0000 7f2a69e00640 1 -- 172.21.15.50:0/878167196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a641fb6f0 con 0x7f2a6415f5b0 2024-09-21T06:02:01.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.768+0000 7f2a61600640 1 -- 172.21.15.50:0/878167196 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f2a4c01c460 con 0x7f2a6415f5b0 2024-09-21T06:02:01.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.768+0000 7f2a61600640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a40073b00 0x7f2a40076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.768+0000 7f2a61600640 1 -- 172.21.15.50:0/878167196 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4971+0+0 (secure 0 0 0) 0x7f2a4c0928d0 con 0x7f2a6415f5b0 2024-09-21T06:02:01.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.768+0000 7f2a68a00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a40073b00 0x7f2a40076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.768+0000 7f2a68a00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a40073b00 0x7f2a40076010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2a540af000 tx=0x7f2a5401afa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:01.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:01.772+0000 7f2a61600640 1 -- 172.21.15.50:0/878167196 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2a4c05f2c0 con 0x7f2a6415f5b0 2024-09-21T06:02:02.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.012+0000 7f2a69e00640 1 -- 172.21.15.50:0/878167196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7f2a6413bce0 con 0x7f2a6415f5b0 2024-09-21T06:02:02.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.200+0000 7f2a61600640 1 -- 172.21.15.50:0/878167196 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v250) ==== 166+0+0 (secure 0 0 0) 0x7f2a4c05ebc0 con 0x7f2a6415f5b0 2024-09-21T06:02:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.204+0000 7f2a69e00640 1 -- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a40073b00 msgr2=0x7f2a40076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.204+0000 7f2a69e00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a40073b00 0x7f2a40076010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2a540af000 tx=0x7f2a5401afa0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.204+0000 7f2a69e00640 1 -- 172.21.15.50:0/878167196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6415f5b0 msgr2=0x7f2a64151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.204+0000 7f2a69e00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6415f5b0 0x7f2a64151b70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00c3c0 tx=0x7f2a4c00c880 comp rx=0 tx=0).stop 2024-09-21T06:02:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.204+0000 7f2a69e00640 1 -- 172.21.15.50:0/878167196 shutdown_connections 2024-09-21T06:02:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.204+0000 7f2a69e00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a641f80b0 0x7f2a640aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.204+0000 7f2a69e00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a40073b00 0x7f2a40076010 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.204+0000 7f2a69e00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a6415f5b0 0x7f2a64151b70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.204+0000 7f2a69e00640 1 --2- 172.21.15.50:0/878167196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a64155a00 0x7f2a6413b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.204+0000 7f2a69e00640 1 -- 172.21.15.50:0/878167196 >> 172.21.15.50:0/878167196 conn(0x7f2a6406cdf0 msgr2=0x7f2a6415d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.204+0000 7f2a69e00640 1 -- 172.21.15.50:0/878167196 shutdown_connections 2024-09-21T06:02:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.204+0000 7f2a69e00640 1 -- 172.21.15.50:0/878167196 wait complete. 2024-09-21T06:02:02.226 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-21T06:02:02.226 INFO:tasks.cephfs_test_runner:Starting test: test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-21T06:02:02.226 INFO:tasks.cephfs_test_runner:test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-21T06:02:02.227 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:02:02.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.396+0000 7f071dc00640 1 -- 172.21.15.50:0/3791590138 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07180b4de0 msgr2=0x7f07180b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:02.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.396+0000 7f071dc00640 1 --2- 172.21.15.50:0/3791590138 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07180b4de0 0x7f07180b51f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0710009bc0 tx=0x7f071001cb70 comp rx=0 tx=0).stop 2024-09-21T06:02:02.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.396+0000 7f071dc00640 1 -- 172.21.15.50:0/3791590138 shutdown_connections 2024-09-21T06:02:02.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.396+0000 7f071dc00640 1 --2- 172.21.15.50:0/3791590138 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07180b4de0 0x7f07180b51f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.396+0000 7f071dc00640 1 --2- 172.21.15.50:0/3791590138 >> v2:172.21.15.88:3300/0 conn(0x7f07180b5c70 0x7f071815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.396+0000 7f071dc00640 1 -- 172.21.15.50:0/3791590138 >> 172.21.15.50:0/3791590138 conn(0x7f07180b2040 msgr2=0x7f07180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:02.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.400+0000 7f071dc00640 1 -- 172.21.15.50:0/3791590138 shutdown_connections 2024-09-21T06:02:02.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.400+0000 7f071dc00640 1 -- 172.21.15.50:0/3791590138 wait complete. 2024-09-21T06:02:02.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.400+0000 7f071dc00640 1 Processor -- start 2024-09-21T06:02:02.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.400+0000 7f071dc00640 1 -- start start 2024-09-21T06:02:02.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.400+0000 7f071dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07180b4de0 0x7f07181f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:02.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.400+0000 7f071dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07180b5c70 0x7f07181fbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:02.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.400+0000 7f071dc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07181527b0 0x7f07180ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:02.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.400+0000 7f071dc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f07181689d0 con 0x7f07180b4de0 2024-09-21T06:02:02.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.400+0000 7f071dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0718168850 con 0x7f07180b5c70 2024-09-21T06:02:02.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.400+0000 7f071dc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0718168b50 con 0x7f07181527b0 2024-09-21T06:02:02.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f0716a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07180b4de0 0x7f07181f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:02.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f0716a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07180b4de0 0x7f07181f8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44754/0 (socket says 172.21.15.50:44754) 2024-09-21T06:02:02.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f0716a00640 1 -- 172.21.15.50:0/1691651547 learned_addr learned my addr 172.21.15.50:0/1691651547 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:02.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f0717400640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07181527b0 0x7f07180ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:02.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f070fe00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07180b5c70 0x7f07181fbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:02.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f0716a00640 1 -- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07181527b0 msgr2=0x7f07180ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:02.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f0716a00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07181527b0 0x7f07180ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f0716a00640 1 -- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07180b5c70 msgr2=0x7f07181fbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:02.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f0716a00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07180b5c70 0x7f07181fbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f0716a00640 1 -- 172.21.15.50:0/1691651547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0718153800 con 0x7f07180b4de0 2024-09-21T06:02:02.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f0716a00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07180b4de0 0x7f07181f8ba0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f070800e970 tx=0x7f070800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:02.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f070ea00640 1 -- 172.21.15.50:0/1691651547 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070800cd20 con 0x7f07180b4de0 2024-09-21T06:02:02.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f071dc00640 1 -- 172.21.15.50:0/1691651547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07181f94b0 con 0x7f07180b4de0 2024-09-21T06:02:02.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f070ea00640 1 -- 172.21.15.50:0/1691651547 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0708004020 con 0x7f07180b4de0 2024-09-21T06:02:02.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f070ea00640 1 -- 172.21.15.50:0/1691651547 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708004470 con 0x7f07180b4de0 2024-09-21T06:02:02.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f071dc00640 1 -- 172.21.15.50:0/1691651547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07181f9a70 con 0x7f07180b4de0 2024-09-21T06:02:02.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f071dc00640 1 -- 172.21.15.50:0/1691651547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07180b51f0 con 0x7f07180b4de0 2024-09-21T06:02:02.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.404+0000 7f070ea00640 1 -- 172.21.15.50:0/1691651547 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f0708003600 con 0x7f07180b4de0 2024-09-21T06:02:02.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.408+0000 7f070ea00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f06f4073b30 0x7f06f4076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:02.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.408+0000 7f070ea00640 1 -- 172.21.15.50:0/1691651547 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4971+0+0 (secure 0 0 0) 0x7f07080934d0 con 0x7f07180b4de0 2024-09-21T06:02:02.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.408+0000 7f070fe00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f06f4073b30 0x7f06f4076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:02.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.408+0000 7f070fe00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f06f4073b30 0x7f06f4076040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0718153310 tx=0x7f0700009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:02.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.412+0000 7f070ea00640 1 -- 172.21.15.50:0/1691651547 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f070805fec0 con 0x7f07180b4de0 2024-09-21T06:02:02.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.588+0000 7f071dc00640 1 -- 172.21.15.50:0/1691651547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f071814d430 con 0x7f07180b4de0 2024-09-21T06:02:02.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f070ea00640 1 -- 172.21.15.50:0/1691651547 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13025 (secure 0 0 0) 0x7f070805f7c0 con 0x7f07180b4de0 2024-09-21T06:02:02.593 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:02.593 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":50,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:02:01.188289+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6835/988935074":"2024-09-22T06:01:58.182264+0000","172.21.15.50:6832/988935074":"2024-09-22T06:01:58.182264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:02:02.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f071dc00640 1 -- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f06f4073b30 msgr2=0x7f06f4076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:02.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f071dc00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f06f4073b30 0x7f06f4076040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0718153310 tx=0x7f0700009040 comp rx=0 tx=0).stop 2024-09-21T06:02:02.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f071dc00640 1 -- 172.21.15.50:0/1691651547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07180b4de0 msgr2=0x7f07181f8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f071dc00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07180b4de0 0x7f07181f8ba0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f070800e970 tx=0x7f070800ee30 comp rx=0 tx=0).stop 2024-09-21T06:02:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f071dc00640 1 -- 172.21.15.50:0/1691651547 shutdown_connections 2024-09-21T06:02:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f071dc00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07181527b0 0x7f07180ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f071dc00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07180b5c70 0x7f07181fbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f071dc00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f06f4073b30 0x7f06f4076040 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f071dc00640 1 --2- 172.21.15.50:0/1691651547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07180b4de0 0x7f07181f8ba0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f071dc00640 1 -- 172.21.15.50:0/1691651547 >> 172.21.15.50:0/1691651547 conn(0x7f07180b2040 msgr2=0x7f0718154e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f071dc00640 1 -- 172.21.15.50:0/1691651547 shutdown_connections 2024-09-21T06:02:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.592+0000 7f071dc00640 1 -- 172.21.15.50:0/1691651547 wait complete. 2024-09-21T06:02:02.616 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:56:18.921197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:02:02.617 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:02:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.776+0000 7fd6bbc00640 1 -- 172.21.15.50:0/15693778 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b41527b0 msgr2=0x7fd6b415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.776+0000 7fd6bbc00640 1 --2- 172.21.15.50:0/15693778 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b41527b0 0x7fd6b415bb60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009870 tx=0x7fd6a001c7a0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.776+0000 7fd6bbc00640 1 -- 172.21.15.50:0/15693778 shutdown_connections 2024-09-21T06:02:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.776+0000 7fd6bbc00640 1 --2- 172.21.15.50:0/15693778 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b41527b0 0x7fd6b415bb60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.776+0000 7fd6bbc00640 1 -- 172.21.15.50:0/15693778 >> 172.21.15.50:0/15693778 conn(0x7fd6b40b2040 msgr2=0x7fd6b40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.776+0000 7fd6bbc00640 1 -- 172.21.15.50:0/15693778 shutdown_connections 2024-09-21T06:02:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.776+0000 7fd6bbc00640 1 -- 172.21.15.50:0/15693778 wait complete. 2024-09-21T06:02:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6bbc00640 1 Processor -- start 2024-09-21T06:02:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6bbc00640 1 -- start start 2024-09-21T06:02:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6bbc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b40b4de0 0x7fd6b4154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6bbc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b40b5c70 0x7fd6b41575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6bbc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b41527b0 0x7fd6b41f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6bbc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b4169d00 con 0x7fd6b40b4de0 2024-09-21T06:02:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6bbc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b4169b80 con 0x7fd6b41527b0 2024-09-21T06:02:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6bbc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b4169e80 con 0x7fd6b40b5c70 2024-09-21T06:02:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b40b4de0 0x7fd6b4154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b40b4de0 0x7fd6b4154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44770/0 (socket says 172.21.15.50:44770) 2024-09-21T06:02:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b9200640 1 -- 172.21.15.50:0/2967176930 learned_addr learned my addr 172.21.15.50:0/2967176930 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b3e00640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b40b5c70 0x7fd6b41575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b9c00640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b41527b0 0x7fd6b41f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b9200640 1 -- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b40b5c70 msgr2=0x7fd6b41575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b9200640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b40b5c70 0x7fd6b41575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b9200640 1 -- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b41527b0 msgr2=0x7fd6b41f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b9200640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b41527b0 0x7fd6b41f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b9200640 1 -- 172.21.15.50:0/2967176930 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b4156530 con 0x7fd6b40b4de0 2024-09-21T06:02:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b9200640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b40b4de0 0x7fd6b4154a30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800c490 tx=0x7fd6a800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b0c00640 1 -- 172.21.15.50:0/2967176930 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8016020 con 0x7fd6b40b4de0 2024-09-21T06:02:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b0c00640 1 -- 172.21.15.50:0/2967176930 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6a8012a00 con 0x7fd6b40b4de0 2024-09-21T06:02:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6bbc00640 1 -- 172.21.15.50:0/2967176930 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b4156810 con 0x7fd6b40b4de0 2024-09-21T06:02:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6b0c00640 1 -- 172.21.15.50:0/2967176930 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8012e50 con 0x7fd6b40b4de0 2024-09-21T06:02:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.780+0000 7fd6bbc00640 1 -- 172.21.15.50:0/2967176930 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b4156dd0 con 0x7fd6b40b4de0 2024-09-21T06:02:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.784+0000 7fd6bbc00640 1 -- 172.21.15.50:0/2967176930 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b415bb60 con 0x7fd6b40b4de0 2024-09-21T06:02:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.784+0000 7fd6b0c00640 1 -- 172.21.15.50:0/2967176930 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fd6a8019020 con 0x7fd6b40b4de0 2024-09-21T06:02:02.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.784+0000 7fd6b0c00640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd694073cd0 0x7fd6940761e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:02.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.784+0000 7fd6b0c00640 1 -- 172.21.15.50:0/2967176930 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4971+0+0 (secure 0 0 0) 0x7fd6a8092a80 con 0x7fd6b40b4de0 2024-09-21T06:02:02.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.784+0000 7fd6b3e00640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd694073cd0 0x7fd6940761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:02.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.784+0000 7fd6b3e00640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd694073cd0 0x7fd6940761e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00af000 tx=0x7fd6a001afe0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.788+0000 7fd6b0c00640 1 -- 172.21.15.50:0/2967176930 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd6a8020020 con 0x7fd6b40b4de0 2024-09-21T06:02:02.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.964+0000 7fd6bbc00640 1 -- 172.21.15.50:0/2967176930 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6b4155cc0 con 0x7fd6b40b4de0 2024-09-21T06:02:02.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.964+0000 7fd6b0c00640 1 -- 172.21.15.50:0/2967176930 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7fd6a805f550 con 0x7fd6b40b4de0 2024-09-21T06:02:02.968 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-09-21T06:02:02.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.968+0000 7fd6bbc00640 1 -- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd694073cd0 msgr2=0x7fd6940761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:02.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.968+0000 7fd6bbc00640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd694073cd0 0x7fd6940761e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00af000 tx=0x7fd6a001afe0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.968+0000 7fd6bbc00640 1 -- 172.21.15.50:0/2967176930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b40b4de0 msgr2=0x7fd6b4154a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:02.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.968+0000 7fd6bbc00640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b40b4de0 0x7fd6b4154a30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800c490 tx=0x7fd6a800c950 comp rx=0 tx=0).stop 2024-09-21T06:02:02.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.968+0000 7fd6bbc00640 1 -- 172.21.15.50:0/2967176930 shutdown_connections 2024-09-21T06:02:02.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.968+0000 7fd6bbc00640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b41527b0 0x7fd6b41f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.968+0000 7fd6bbc00640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6b40b5c70 0x7fd6b41575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.968+0000 7fd6bbc00640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd694073cd0 0x7fd6940761e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.968+0000 7fd6bbc00640 1 --2- 172.21.15.50:0/2967176930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b40b4de0 0x7fd6b4154a30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:02.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.968+0000 7fd6bbc00640 1 -- 172.21.15.50:0/2967176930 >> 172.21.15.50:0/2967176930 conn(0x7fd6b40b2040 msgr2=0x7fd6b40cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:02.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.968+0000 7fd6bbc00640 1 -- 172.21.15.50:0/2967176930 shutdown_connections 2024-09-21T06:02:02.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:02.968+0000 7fd6bbc00640 1 -- 172.21.15.50:0/2967176930 wait complete. 2024-09-21T06:02:02.991 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-21T06:02:03.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.148+0000 7f4245800640 1 -- 172.21.15.50:0/1882896371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42401527b0 msgr2=0x7f424015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:03.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.148+0000 7f4245800640 1 --2- 172.21.15.50:0/1882896371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42401527b0 0x7f424015bb60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4228009840 tx=0x7f422801c7f0 comp rx=0 tx=0).stop 2024-09-21T06:02:03.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.148+0000 7f4245800640 1 -- 172.21.15.50:0/1882896371 shutdown_connections 2024-09-21T06:02:03.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.148+0000 7f4245800640 1 --2- 172.21.15.50:0/1882896371 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f42400b4de0 0x7f42400b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:03.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.148+0000 7f4245800640 1 --2- 172.21.15.50:0/1882896371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42401527b0 0x7f424015bb60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:03.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.148+0000 7f4245800640 1 -- 172.21.15.50:0/1882896371 >> 172.21.15.50:0/1882896371 conn(0x7f42400b2040 msgr2=0x7f42400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:03.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.148+0000 7f4245800640 1 -- 172.21.15.50:0/1882896371 shutdown_connections 2024-09-21T06:02:03.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.148+0000 7f4245800640 1 -- 172.21.15.50:0/1882896371 wait complete. 2024-09-21T06:02:03.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.148+0000 7f4245800640 1 Processor -- start 2024-09-21T06:02:03.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.148+0000 7f4245800640 1 -- start start 2024-09-21T06:02:03.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4245800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42400b4de0 0x7f42401f8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:03.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4245800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42400b5c70 0x7f4240157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:03.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4245800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f42401527b0 0x7f42400ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4245800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4240168950 con 0x7f42400b5c70 2024-09-21T06:02:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4245800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f42401687d0 con 0x7f42400b4de0 2024-09-21T06:02:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4245800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4240168ad0 con 0x7f42401527b0 2024-09-21T06:02:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4237e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42400b5c70 0x7f4240157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4237e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42400b5c70 0x7f4240157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44780/0 (socket says 172.21.15.50:44780) 2024-09-21T06:02:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4237e00640 1 -- 172.21.15.50:0/2821167517 learned_addr learned my addr 172.21.15.50:0/2821167517 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f423ea00640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42400b4de0 0x7f42401f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f423f400640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f42401527b0 0x7f42400ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4237e00640 1 -- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f42401527b0 msgr2=0x7f42400ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4237e00640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f42401527b0 0x7f42400ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4237e00640 1 -- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42400b4de0 msgr2=0x7f42401f8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4237e00640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42400b4de0 0x7f42401f8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4237e00640 1 -- 172.21.15.50:0/2821167517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4240154240 con 0x7f42400b5c70 2024-09-21T06:02:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4237e00640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42400b5c70 0x7f4240157a00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4228009790 tx=0x7f42280a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4236a00640 1 -- 172.21.15.50:0/2821167517 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42280b2040 con 0x7f42400b5c70 2024-09-21T06:02:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4236a00640 1 -- 172.21.15.50:0/2821167517 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4228004430 con 0x7f42400b5c70 2024-09-21T06:02:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4236a00640 1 -- 172.21.15.50:0/2821167517 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228003820 con 0x7f42400b5c70 2024-09-21T06:02:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4245800640 1 -- 172.21.15.50:0/2821167517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42401531a0 con 0x7f42400b5c70 2024-09-21T06:02:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4245800640 1 -- 172.21.15.50:0/2821167517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4240153760 con 0x7f42400b5c70 2024-09-21T06:02:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.152+0000 7f4245800640 1 -- 172.21.15.50:0/2821167517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4240153e80 con 0x7f42400b5c70 2024-09-21T06:02:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.156+0000 7f4236a00640 1 -- 172.21.15.50:0/2821167517 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f42280bd060 con 0x7f42400b5c70 2024-09-21T06:02:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.156+0000 7f4236a00640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f421c073b30 0x7f421c076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.156+0000 7f4236a00640 1 -- 172.21.15.50:0/2821167517 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4971+0+0 (secure 0 0 0) 0x7f422812bed0 con 0x7f42400b5c70 2024-09-21T06:02:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.156+0000 7f423ea00640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f421c073b30 0x7f421c076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.156+0000 7f423ea00640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f421c073b30 0x7f421c076040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4230007fd0 tx=0x7f4230009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.160+0000 7f4236a00640 1 -- 172.21.15.50:0/2821167517 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4228016410 con 0x7f42400b5c70 2024-09-21T06:02:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:03.400+0000 7f4245800640 1 -- 172.21.15.50:0/2821167517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7f42400b51f0 con 0x7f42400b5c70 2024-09-21T06:02:04.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.208+0000 7f4236a00640 1 -- 172.21.15.50:0/2821167517 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v252) ==== 170+0+0 (secure 0 0 0) 0x7f42280f8380 con 0x7f42400b5c70 2024-09-21T06:02:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.212+0000 7f4245800640 1 -- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f421c073b30 msgr2=0x7f421c076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.212+0000 7f4245800640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f421c073b30 0x7f421c076040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4230007fd0 tx=0x7f4230009040 comp rx=0 tx=0).stop 2024-09-21T06:02:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.212+0000 7f4245800640 1 -- 172.21.15.50:0/2821167517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42400b5c70 msgr2=0x7f4240157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.212+0000 7f4245800640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42400b5c70 0x7f4240157a00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4228009790 tx=0x7f42280a57c0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.212+0000 7f4245800640 1 -- 172.21.15.50:0/2821167517 shutdown_connections 2024-09-21T06:02:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.212+0000 7f4245800640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f42401527b0 0x7f42400ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.212+0000 7f4245800640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f421c073b30 0x7f421c076040 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.212+0000 7f4245800640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42400b5c70 0x7f4240157a00 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.212+0000 7f4245800640 1 --2- 172.21.15.50:0/2821167517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42400b4de0 0x7f42401f8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.212+0000 7f4245800640 1 -- 172.21.15.50:0/2821167517 >> 172.21.15.50:0/2821167517 conn(0x7f42400b2040 msgr2=0x7f4240159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.212+0000 7f4245800640 1 -- 172.21.15.50:0/2821167517 shutdown_connections 2024-09-21T06:02:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.212+0000 7f4245800640 1 -- 172.21.15.50:0/2821167517 wait complete. 2024-09-21T06:02:04.238 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:02:04.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.400+0000 7f3d2ca00640 1 -- 172.21.15.50:0/436584630 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d280cefa0 msgr2=0x7f3d280cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:04.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.400+0000 7f3d2ca00640 1 --2- 172.21.15.50:0/436584630 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d280cefa0 0x7f3d280cf3b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009840 tx=0x7f3d1001c820 comp rx=0 tx=0).stop 2024-09-21T06:02:04.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.400+0000 7f3d2ca00640 1 -- 172.21.15.50:0/436584630 shutdown_connections 2024-09-21T06:02:04.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.400+0000 7f3d2ca00640 1 --2- 172.21.15.50:0/436584630 >> v2:172.21.15.88:3300/0 conn(0x7f3d280b3340 0x7f3d280b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.400+0000 7f3d2ca00640 1 --2- 172.21.15.50:0/436584630 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d280cefa0 0x7f3d280cf3b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.400+0000 7f3d2ca00640 1 -- 172.21.15.50:0/436584630 >> 172.21.15.50:0/436584630 conn(0x7f3d280b2040 msgr2=0x7f3d280b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:04.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.400+0000 7f3d2ca00640 1 -- 172.21.15.50:0/436584630 shutdown_connections 2024-09-21T06:02:04.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.400+0000 7f3d2ca00640 1 -- 172.21.15.50:0/436584630 wait complete. 2024-09-21T06:02:04.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d2ca00640 1 Processor -- start 2024-09-21T06:02:04.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d2ca00640 1 -- start start 2024-09-21T06:02:04.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d2ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d280749a0 0x7f3d28145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d2ca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d280b3340 0x7f3d2815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d2ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d280cefa0 0x7f3d2813f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d2ca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28169e00 con 0x7f3d280749a0 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d2ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28169c80 con 0x7f3d280cefa0 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d2ca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3d28169f80 con 0x7f3d280b3340 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d25e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d280749a0 0x7f3d28145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d25e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d280749a0 0x7f3d28145090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44796/0 (socket says 172.21.15.50:44796) 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d25e00640 1 -- 172.21.15.50:0/3232690303 learned_addr learned my addr 172.21.15.50:0/3232690303 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d26800640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d280cefa0 0x7f3d2813f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d1fe00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d280b3340 0x7f3d2815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d25e00640 1 -- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d280b3340 msgr2=0x7f3d2815aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d25e00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d280b3340 0x7f3d2815aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d25e00640 1 -- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d280cefa0 msgr2=0x7f3d2813f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:04.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d25e00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d280cefa0 0x7f3d2813f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d25e00640 1 -- 172.21.15.50:0/3232690303 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d28146070 con 0x7f3d280749a0 2024-09-21T06:02:04.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d1fe00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d280b3340 0x7f3d2815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:04.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d26800640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d280cefa0 0x7f3d2813f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:04.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.404+0000 7f3d25e00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d280749a0 0x7f3d28145090 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800c430 tx=0x7f3d1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:04.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.408+0000 7f3d1e000640 1 -- 172.21.15.50:0/3232690303 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18011790 con 0x7f3d280749a0 2024-09-21T06:02:04.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.408+0000 7f3d1e000640 1 -- 172.21.15.50:0/3232690303 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d18017070 con 0x7f3d280749a0 2024-09-21T06:02:04.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.408+0000 7f3d2ca00640 1 -- 172.21.15.50:0/3232690303 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d281f9d40 con 0x7f3d280749a0 2024-09-21T06:02:04.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.408+0000 7f3d1e000640 1 -- 172.21.15.50:0/3232690303 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18011cc0 con 0x7f3d280749a0 2024-09-21T06:02:04.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.408+0000 7f3d2ca00640 1 -- 172.21.15.50:0/3232690303 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281fa300 con 0x7f3d280749a0 2024-09-21T06:02:04.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.408+0000 7f3d2ca00640 1 -- 172.21.15.50:0/3232690303 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d280b3750 con 0x7f3d280749a0 2024-09-21T06:02:04.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.408+0000 7f3d1e000640 1 -- 172.21.15.50:0/3232690303 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f3d1801c460 con 0x7f3d280749a0 2024-09-21T06:02:04.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.408+0000 7f3d1e000640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d04073b00 0x7f3d04076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:04.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.408+0000 7f3d1e000640 1 -- 172.21.15.50:0/3232690303 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4971+0+0 (secure 0 0 0) 0x7f3d180928c0 con 0x7f3d280749a0 2024-09-21T06:02:04.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.408+0000 7f3d1fe00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d04073b00 0x7f3d04076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:04.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.412+0000 7f3d1fe00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d04073b00 0x7f3d04076010 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3d100af000 tx=0x7f3d1001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:04.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.412+0000 7f3d1e000640 1 -- 172.21.15.50:0/3232690303 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3d1805f390 con 0x7f3d280749a0 2024-09-21T06:02:04.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 -- 172.21.15.50:0/3232690303 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f3d28160f20 con 0x7f3d280749a0 2024-09-21T06:02:04.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d1e000640 1 -- 172.21.15.50:0/3232690303 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+2294 (secure 0 0 0) 0x7f3d1805ec90 con 0x7f3d280749a0 2024-09-21T06:02:04.593 INFO:teuthology.orchestra.run.smithi050.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:02:04.593 INFO:teuthology.orchestra.run.smithi050.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:02:04.593 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:02:04.593 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_client 20/20 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3735.149299 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4471.400731 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3640.131623 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3663.645087 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3762.055672 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3661.558528 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3661.456695 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3796.187852 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds 20/20 2024-09-21T06:02:04.594 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_ms 2/2 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_frag true 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_verify_scatter true 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced client_mount_timeout 600 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_client 20/20 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_ms 1/1 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:02:04.595 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:02:04.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 -- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d04073b00 msgr2=0x7f3d04076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:04.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d04073b00 0x7f3d04076010 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3d100af000 tx=0x7f3d1001b060 comp rx=0 tx=0).stop 2024-09-21T06:02:04.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 -- 172.21.15.50:0/3232690303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d280749a0 msgr2=0x7f3d28145090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:04.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d280749a0 0x7f3d28145090 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800c430 tx=0x7f3d1800c8f0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 -- 172.21.15.50:0/3232690303 shutdown_connections 2024-09-21T06:02:04.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d280cefa0 0x7f3d2813f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d280b3340 0x7f3d2815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d04073b00 0x7f3d04076010 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 --2- 172.21.15.50:0/3232690303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d280749a0 0x7f3d28145090 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 -- 172.21.15.50:0/3232690303 >> 172.21.15.50:0/3232690303 conn(0x7f3d280b2040 msgr2=0x7f3d280b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:04.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 -- 172.21.15.50:0/3232690303 shutdown_connections 2024-09-21T06:02:04.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.592+0000 7f3d2ca00640 1 -- 172.21.15.50:0/3232690303 wait complete. 2024-09-21T06:02:04.616 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:02:04.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.776+0000 7f693f800640 1 -- 172.21.15.50:0/3064833707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6920005220 msgr2=0x7f692000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:04.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.776+0000 7f693f800640 1 --2- 172.21.15.50:0/3064833707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6920005220 0x7f692000b9d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6938072720 tx=0x7f693001cbe0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.776+0000 7f693d800640 1 -- 172.21.15.50:0/3064833707 reap_dead start 2024-09-21T06:02:04.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.776+0000 7f693f800640 1 -- 172.21.15.50:0/3064833707 shutdown_connections 2024-09-21T06:02:04.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.776+0000 7f693f800640 1 -- 172.21.15.50:0/3064833707 >> 172.21.15.50:0/3064833707 conn(0x7f69380b2040 msgr2=0x7f69380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:04.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.776+0000 7f693f800640 1 -- 172.21.15.50:0/3064833707 shutdown_connections 2024-09-21T06:02:04.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.776+0000 7f693f800640 1 -- 172.21.15.50:0/3064833707 wait complete. 2024-09-21T06:02:04.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693f800640 1 Processor -- start 2024-09-21T06:02:04.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693f800640 1 -- start start 2024-09-21T06:02:04.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693f800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6920005220 0x7f6938145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:04.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693f800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f693815a140 0x7f693815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:04.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693f800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693815c230 0x7f6938144310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:04.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693f800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f69380b4260 con 0x7f693815c230 2024-09-21T06:02:04.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693f800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f69380cf3b0 con 0x7f693815a140 2024-09-21T06:02:04.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693f800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f69381e7a90 con 0x7f6920005220 2024-09-21T06:02:04.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693815c230 0x7f6938144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:04.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693815c230 0x7f6938144310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44806/0 (socket says 172.21.15.50:44806) 2024-09-21T06:02:04.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693d800640 1 -- 172.21.15.50:0/2830068270 learned_addr learned my addr 172.21.15.50:0/2830068270 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:04.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693ce00640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6920005220 0x7f6938145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f6937e00640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f693815a140 0x7f693815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693d800640 1 -- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6920005220 msgr2=0x7f6938145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693d800640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6920005220 0x7f6938145e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693d800640 1 -- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f693815a140 msgr2=0x7f693815bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693d800640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f693815a140 0x7f693815bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693d800640 1 -- 172.21.15.50:0/2830068270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69380afcb0 con 0x7f693815c230 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f6937e00640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f693815a140 0x7f693815bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693d800640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693815c230 0x7f6938144310 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6930005040 tx=0x7f6930006e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f6934c00640 1 -- 172.21.15.50:0/2830068270 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69300a5e60 con 0x7f693815c230 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f6934c00640 1 -- 172.21.15.50:0/2830068270 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6930006150 con 0x7f693815c230 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f693f800640 1 -- 172.21.15.50:0/2830068270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693815c640 con 0x7f693815c230 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.780+0000 7f6934c00640 1 -- 172.21.15.50:0/2830068270 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930006c50 con 0x7f693815c230 2024-09-21T06:02:04.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.784+0000 7f693f800640 1 -- 172.21.15.50:0/2830068270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69381ea990 con 0x7f693815c230 2024-09-21T06:02:04.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.784+0000 7f693f800640 1 -- 172.21.15.50:0/2830068270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69380cf910 con 0x7f693815c230 2024-09-21T06:02:04.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.784+0000 7f6934c00640 1 -- 172.21.15.50:0/2830068270 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f6930006540 con 0x7f693815c230 2024-09-21T06:02:04.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.784+0000 7f6934c00640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6918073b00 0x7f6918076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:04.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.784+0000 7f6934c00640 1 -- 172.21.15.50:0/2830068270 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4971+0+0 (secure 0 0 0) 0x7f693012d270 con 0x7f693815c230 2024-09-21T06:02:04.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.784+0000 7f693ce00640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6918073b00 0x7f6918076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:04.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.788+0000 7f693ce00640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6918073b00 0x7f6918076010 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f692c003ff0 tx=0x7f692c001580 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:04.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.788+0000 7f6934c00640 1 -- 172.21.15.50:0/2830068270 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f693815c640 con 0x7f693815c230 2024-09-21T06:02:04.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.964+0000 7f693f800640 1 -- 172.21.15.50:0/2830068270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f69380b5ce0 con 0x7f693815c230 2024-09-21T06:02:04.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.968+0000 7f6934c00640 1 -- 172.21.15.50:0/2830068270 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f6930016990 con 0x7f693815c230 2024-09-21T06:02:04.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.972+0000 7f693f800640 1 -- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6918073b00 msgr2=0x7f6918076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:04.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.972+0000 7f693f800640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6918073b00 0x7f6918076010 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f692c003ff0 tx=0x7f692c001580 comp rx=0 tx=0).stop 2024-09-21T06:02:04.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.972+0000 7f693f800640 1 -- 172.21.15.50:0/2830068270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693815c230 msgr2=0x7f6938144310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:04.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.972+0000 7f693f800640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693815c230 0x7f6938144310 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6930005040 tx=0x7f6930006e20 comp rx=0 tx=0).stop 2024-09-21T06:02:04.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.972+0000 7f693f800640 1 -- 172.21.15.50:0/2830068270 shutdown_connections 2024-09-21T06:02:04.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.972+0000 7f693f800640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6918073b00 0x7f6918076010 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.972+0000 7f693f800640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693815c230 0x7f6938144310 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.972+0000 7f693f800640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f693815a140 0x7f693815bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.972+0000 7f693f800640 1 --2- 172.21.15.50:0/2830068270 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6920005220 0x7f6938145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:04.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.972+0000 7f693f800640 1 -- 172.21.15.50:0/2830068270 >> 172.21.15.50:0/2830068270 conn(0x7f69380b2040 msgr2=0x7f6938072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:04.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.972+0000 7f693f800640 1 -- 172.21.15.50:0/2830068270 shutdown_connections 2024-09-21T06:02:04.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:04.972+0000 7f693f800640 1 -- 172.21.15.50:0/2830068270 wait complete. 2024-09-21T06:02:04.993 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:02:04.993 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:02:05.062 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:02:05.143 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:02:05.144 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:02:05.144 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:02:05.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:02:05.153 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:02:05.153 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:02:05.162 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:02:05.162 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:02:05.162 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:02:05.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:02:05.204 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:02:05.204 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:02:05.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:02:05.214 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:02:05.214 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:02:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.404+0000 7f6ae4800640 1 -- 172.21.15.50:0/125922777 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6adc0b4de0 msgr2=0x7f6adc0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.404+0000 7f6ae4800640 1 --2- 172.21.15.50:0/125922777 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6adc0b4de0 0x7f6adc0b51f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6adc071cb0 tx=0x7f6ad801cac0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.404+0000 7f6ae4800640 1 -- 172.21.15.50:0/125922777 shutdown_connections 2024-09-21T06:02:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.404+0000 7f6ae4800640 1 --2- 172.21.15.50:0/125922777 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6adc0b4de0 0x7f6adc0b51f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.404+0000 7f6ae4800640 1 --2- 172.21.15.50:0/125922777 >> v2:172.21.15.88:3300/0 conn(0x7f6adc0b5c70 0x7f6adc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.404+0000 7f6ae4800640 1 -- 172.21.15.50:0/125922777 >> 172.21.15.50:0/125922777 conn(0x7f6adc0b2040 msgr2=0x7f6adc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.404+0000 7f6ae4800640 1 -- 172.21.15.50:0/125922777 shutdown_connections 2024-09-21T06:02:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.404+0000 7f6ae4800640 1 -- 172.21.15.50:0/125922777 wait complete. 2024-09-21T06:02:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.404+0000 7f6ae4800640 1 Processor -- start 2024-09-21T06:02:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.404+0000 7f6ae4800640 1 -- start start 2024-09-21T06:02:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae4800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6adc0b4de0 0x7f6adc0aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae4800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6adc0b5c70 0x7f6adc1436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae4800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6adc1527b0 0x7f6adc15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae4800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc1688b0 con 0x7f6adc0b5c70 2024-09-21T06:02:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae4800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc168730 con 0x7f6adc1527b0 2024-09-21T06:02:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae4800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6adc168a30 con 0x7f6adc0b4de0 2024-09-21T06:02:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6adc0b5c70 0x7f6adc1436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6adc0b5c70 0x7f6adc1436c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44826/0 (socket says 172.21.15.50:44826) 2024-09-21T06:02:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae1400640 1 -- 172.21.15.50:0/1425734011 learned_addr learned my addr 172.21.15.50:0/1425734011 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae1e00640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6adc0b4de0 0x7f6adc0aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae2800640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6adc1527b0 0x7f6adc15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae1400640 1 -- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6adc0b4de0 msgr2=0x7f6adc0aaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae1400640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6adc0b4de0 0x7f6adc0aaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae1400640 1 -- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6adc1527b0 msgr2=0x7f6adc15a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae1400640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6adc1527b0 0x7f6adc15a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae1400640 1 -- 172.21.15.50:0/1425734011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc1f6550 con 0x7f6adc0b5c70 2024-09-21T06:02:05.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae1400640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6adc0b5c70 0x7f6adc1436c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6acc001ec0 tx=0x7f6acc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:05.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ad6000640 1 -- 172.21.15.50:0/1425734011 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc017070 con 0x7f6adc0b5c70 2024-09-21T06:02:05.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ad6000640 1 -- 172.21.15.50:0/1425734011 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6acc003600 con 0x7f6adc0b5c70 2024-09-21T06:02:05.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae4800640 1 -- 172.21.15.50:0/1425734011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6adc143c00 con 0x7f6adc0b5c70 2024-09-21T06:02:05.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ad6000640 1 -- 172.21.15.50:0/1425734011 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc003a50 con 0x7f6adc0b5c70 2024-09-21T06:02:05.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae4800640 1 -- 172.21.15.50:0/1425734011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6adc1f6b20 con 0x7f6adc0b5c70 2024-09-21T06:02:05.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.408+0000 7f6ae4800640 1 -- 172.21.15.50:0/1425734011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6adc0b51f0 con 0x7f6adc0b5c70 2024-09-21T06:02:05.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.412+0000 7f6ad6000640 1 -- 172.21.15.50:0/1425734011 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f6acc01a070 con 0x7f6adc0b5c70 2024-09-21T06:02:05.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.412+0000 7f6ad6000640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ac0073b00 0x7f6ac0076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:05.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.412+0000 7f6ad6000640 1 -- 172.21.15.50:0/1425734011 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4971+0+0 (secure 0 0 0) 0x7f6acc091d60 con 0x7f6adc0b5c70 2024-09-21T06:02:05.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.412+0000 7f6ae1e00640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ac0073b00 0x7f6ac0076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:05.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.412+0000 7f6ae1e00640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ac0073b00 0x7f6ac0076010 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0004630 tx=0x7f6ad000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:05.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.416+0000 7f6ad6000640 1 -- 172.21.15.50:0/1425734011 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6adc0b5730 con 0x7f6adc0b5c70 2024-09-21T06:02:05.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.628+0000 7f6ae4800640 1 -- 172.21.15.50:0/1425734011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6adc0cf950 con 0x7f6adc0b5c70 2024-09-21T06:02:05.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ad6000640 1 -- 172.21.15.50:0/1425734011 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+3908 (secure 0 0 0) 0x7f6acc05e210 con 0x7f6adc0b5c70 2024-09-21T06:02:05.632 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:05.632 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":30,"btime":"2024-09-21T06:01:59:184642+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/723044219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":723044219},{"type":"v1","addr":"172.21.15.88:6836","nonce":723044219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1966998855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1966998855},{"type":"v1","addr":"172.21.15.88:6837","nonce":1966998855}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2536951841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2536951841},{"type":"v1","addr":"172.21.15.50:6834","nonce":2536951841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5853,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1588705097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1588705097},{"type":"v1","addr":"172.21.15.50:6835","nonce":1588705097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[]} 2024-09-21T06:02:05.633 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 30 2024-09-21T06:02:05.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ae4800640 1 -- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ac0073b00 msgr2=0x7f6ac0076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:05.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ae4800640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ac0073b00 0x7f6ac0076010 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0004630 tx=0x7f6ad000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ae4800640 1 -- 172.21.15.50:0/1425734011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6adc0b5c70 msgr2=0x7f6adc1436c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:05.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ae4800640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6adc0b5c70 0x7f6adc1436c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6acc001ec0 tx=0x7f6acc00c890 comp rx=0 tx=0).stop 2024-09-21T06:02:05.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ae4800640 1 -- 172.21.15.50:0/1425734011 shutdown_connections 2024-09-21T06:02:05.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ae4800640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6adc1527b0 0x7f6adc15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ae4800640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ac0073b00 0x7f6ac0076010 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ae4800640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6adc0b5c70 0x7f6adc1436c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ae4800640 1 --2- 172.21.15.50:0/1425734011 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6adc0b4de0 0x7f6adc0aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ae4800640 1 -- 172.21.15.50:0/1425734011 >> 172.21.15.50:0/1425734011 conn(0x7f6adc0b2040 msgr2=0x7f6adc153ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:05.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ae4800640 1 -- 172.21.15.50:0/1425734011 shutdown_connections 2024-09-21T06:02:05.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.632+0000 7f6ae4800640 1 -- 172.21.15.50:0/1425734011 wait complete. 2024-09-21T06:02:05.656 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:02:05.656 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:02:05.656 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:02:05.656 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:02:05.656 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:02:05.656 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:02:05.657 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:02:05.657 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:02:05.657 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:02:05.657 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:02:05.657 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:02:05.657 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:02:05.658 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:02:05.658 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:02:05.658 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:02:05.658 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:02:05.671 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:02:05.671 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:02:05.675 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:02:05.675 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:02:05.675 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:02:05.676 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:02:05.680 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:02:05.680 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:02:05.724 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:02:05.724 DEBUG:teuthology.parallel:result is None 2024-09-21T06:02:05.725 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:02:05.726 DEBUG:teuthology.parallel:result is None 2024-09-21T06:02:05.726 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:02:05.727 DEBUG:teuthology.parallel:result is None 2024-09-21T06:02:05.729 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:02:05.729 DEBUG:teuthology.parallel:result is None 2024-09-21T06:02:05.729 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:02:05.827 INFO:tasks.ceph.mds.d.smithi088.stdout:starting mds.d at 2024-09-21T06:02:05.828 INFO:tasks.ceph.mds.b.smithi088.stdout:starting mds.b at 2024-09-21T06:02:05.829 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T06:02:05.829 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-09-21T06:02:05.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.892+0000 7fdc31400640 1 -- 172.21.15.50:0/3391297641 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdc2c074950 msgr2=0x7fdc2c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:05.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.892+0000 7fdc31400640 1 --2- 172.21.15.50:0/3391297641 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdc2c074950 0x7fdc2c15cf90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009bc0 tx=0x7fdc2401cba0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.892+0000 7fdc31400640 1 -- 172.21.15.50:0/3391297641 shutdown_connections 2024-09-21T06:02:05.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.892+0000 7fdc31400640 1 --2- 172.21.15.50:0/3391297641 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdc2c074950 0x7fdc2c15cf90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.892+0000 7fdc31400640 1 --2- 172.21.15.50:0/3391297641 >> v2:172.21.15.50:3300/0 conn(0x7fdc2c0b5a80 0x7fdc2c074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.892+0000 7fdc31400640 1 -- 172.21.15.50:0/3391297641 >> 172.21.15.50:0/3391297641 conn(0x7fdc2c0708c0 msgr2=0x7fdc2c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:05.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.892+0000 7fdc31400640 1 -- 172.21.15.50:0/3391297641 shutdown_connections 2024-09-21T06:02:05.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.892+0000 7fdc31400640 1 -- 172.21.15.50:0/3391297641 wait complete. 2024-09-21T06:02:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc31400640 1 Processor -- start 2024-09-21T06:02:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc31400640 1 -- start start 2024-09-21T06:02:05.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc31400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdc2c074950 0x7fdc2c148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:05.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc31400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc2c0b5a80 0x7fdc2c14ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc31400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc2c157030 0x7fdc2c14d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc31400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c16a310 con 0x7fdc2c0b5a80 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc31400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c16a190 con 0x7fdc2c157030 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc31400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fdc2c16a490 con 0x7fdc2c074950 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc2c0b5a80 0x7fdc2c14ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc2c0b5a80 0x7fdc2c14ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44862/0 (socket says 172.21.15.50:44862) 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc23e00640 1 -- 172.21.15.50:0/1282855134 learned_addr learned my addr 172.21.15.50:0/1282855134 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc2b200640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc2c157030 0x7fdc2c14d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc2a800640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdc2c074950 0x7fdc2c148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc23e00640 1 -- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdc2c074950 msgr2=0x7fdc2c148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc23e00640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdc2c074950 0x7fdc2c148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc23e00640 1 -- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc2c157030 msgr2=0x7fdc2c14d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc23e00640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc2c157030 0x7fdc2c14d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:05.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc23e00640 1 -- 172.21.15.50:0/1282855134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c1522f0 con 0x7fdc2c0b5a80 2024-09-21T06:02:05.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.896+0000 7fdc23e00640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc2c0b5a80 0x7fdc2c14ce00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400e9f0 tx=0x7fdc1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:05.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.900+0000 7fdc22a00640 1 -- 172.21.15.50:0/1282855134 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400cd80 con 0x7fdc2c0b5a80 2024-09-21T06:02:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.900+0000 7fdc22a00640 1 -- 172.21.15.50:0/1282855134 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc14004020 con 0x7fdc2c0b5a80 2024-09-21T06:02:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.900+0000 7fdc31400640 1 -- 172.21.15.50:0/1282855134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2c15bfc0 con 0x7fdc2c0b5a80 2024-09-21T06:02:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.900+0000 7fdc22a00640 1 -- 172.21.15.50:0/1282855134 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14004470 con 0x7fdc2c0b5a80 2024-09-21T06:02:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.900+0000 7fdc31400640 1 -- 172.21.15.50:0/1282855134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2c152560 con 0x7fdc2c0b5a80 2024-09-21T06:02:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.900+0000 7fdc31400640 1 -- 172.21.15.50:0/1282855134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2c149680 con 0x7fdc2c0b5a80 2024-09-21T06:02:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.900+0000 7fdc22a00640 1 -- 172.21.15.50:0/1282855134 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fdc14003600 con 0x7fdc2c0b5a80 2024-09-21T06:02:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.900+0000 7fdc22a00640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdc08073b00 0x7fdc08076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.900+0000 7fdc22a00640 1 -- 172.21.15.50:0/1282855134 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4971+0+0 (secure 0 0 0) 0x7fdc140933a0 con 0x7fdc2c0b5a80 2024-09-21T06:02:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.900+0000 7fdc2a800640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdc08073b00 0x7fdc08076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.904+0000 7fdc2a800640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdc08073b00 0x7fdc08076010 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c007fd0 tx=0x7fdc1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:05.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:05.904+0000 7fdc22a00640 1 -- 172.21.15.50:0/1282855134 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdc1405f740 con 0x7fdc2c0b5a80 2024-09-21T06:02:06.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.100+0000 7fdc31400640 1 -- 172.21.15.50:0/1282855134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fdc2c15cf90 con 0x7fdc2c0b5a80 2024-09-21T06:02:06.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.984+0000 7fdc22a00640 1 -- 172.21.15.50:0/1282855134 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v55) ==== 97+0+0 (secure 0 0 0) 0x7fdc14064170 con 0x7fdc2c0b5a80 2024-09-21T06:02:06.985 INFO:teuthology.orchestra.run.smithi050.stderr: removed all blocklist entries 2024-09-21T06:02:06.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.988+0000 7fdc31400640 1 -- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdc08073b00 msgr2=0x7fdc08076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:06.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.988+0000 7fdc31400640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdc08073b00 0x7fdc08076010 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c007fd0 tx=0x7fdc1c009040 comp rx=0 tx=0).stop 2024-09-21T06:02:06.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.988+0000 7fdc31400640 1 -- 172.21.15.50:0/1282855134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc2c0b5a80 msgr2=0x7fdc2c14ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.988+0000 7fdc31400640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc2c0b5a80 0x7fdc2c14ce00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400e9f0 tx=0x7fdc1400edf0 comp rx=0 tx=0).stop 2024-09-21T06:02:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.988+0000 7fdc31400640 1 -- 172.21.15.50:0/1282855134 shutdown_connections 2024-09-21T06:02:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.988+0000 7fdc31400640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc2c157030 0x7fdc2c14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.988+0000 7fdc31400640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdc08073b00 0x7fdc08076010 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.988+0000 7fdc31400640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc2c0b5a80 0x7fdc2c14ce00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.988+0000 7fdc31400640 1 --2- 172.21.15.50:0/1282855134 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdc2c074950 0x7fdc2c148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.988+0000 7fdc31400640 1 -- 172.21.15.50:0/1282855134 >> 172.21.15.50:0/1282855134 conn(0x7fdc2c0708c0 msgr2=0x7fdc2c150110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.988+0000 7fdc31400640 1 -- 172.21.15.50:0/1282855134 shutdown_connections 2024-09-21T06:02:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:06.988+0000 7fdc31400640 1 -- 172.21.15.50:0/1282855134 wait complete. 2024-09-21T06:02:07.014 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:02:07.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.176+0000 7fab21600640 1 -- 172.21.15.50:0/1439295373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1c0b4de0 msgr2=0x7fab1c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.176+0000 7fab21600640 1 --2- 172.21.15.50:0/1439295373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1c0b4de0 0x7fab1c0b51f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0703e0 tx=0x7fab1401ccc0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.176+0000 7fab21600640 1 -- 172.21.15.50:0/1439295373 shutdown_connections 2024-09-21T06:02:07.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.176+0000 7fab21600640 1 --2- 172.21.15.50:0/1439295373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1c0b4de0 0x7fab1c0b51f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.176+0000 7fab21600640 1 --2- 172.21.15.50:0/1439295373 >> v2:172.21.15.88:3300/0 conn(0x7fab1c1527b0 0x7fab1c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.176+0000 7fab21600640 1 --2- 172.21.15.50:0/1439295373 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab1c0b5c70 0x7fab1c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.176+0000 7fab21600640 1 -- 172.21.15.50:0/1439295373 >> 172.21.15.50:0/1439295373 conn(0x7fab1c0b2040 msgr2=0x7fab1c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:07.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.176+0000 7fab21600640 1 -- 172.21.15.50:0/1439295373 shutdown_connections 2024-09-21T06:02:07.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.176+0000 7fab21600640 1 -- 172.21.15.50:0/1439295373 wait complete. 2024-09-21T06:02:07.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.176+0000 7fab21600640 1 Processor -- start 2024-09-21T06:02:07.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab21600640 1 -- start start 2024-09-21T06:02:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab21600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab1c0b4de0 0x7fab1c148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab21600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1c0b5c70 0x7fab1c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab21600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab1c1527b0 0x7fab1c157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab21600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c15f730 con 0x7fab1c0b5c70 2024-09-21T06:02:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab21600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c15f5b0 con 0x7fab1c1527b0 2024-09-21T06:02:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab21600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fab1c15f8b0 con 0x7fab1c0b4de0 2024-09-21T06:02:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab13e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1c0b5c70 0x7fab1c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab13e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1c0b5c70 0x7fab1c0cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48030/0 (socket says 172.21.15.50:48030) 2024-09-21T06:02:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab13e00640 1 -- 172.21.15.50:0/2730437238 learned_addr learned my addr 172.21.15.50:0/2730437238 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab1b400640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab1c1527b0 0x7fab1c157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab1aa00640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab1c0b4de0 0x7fab1c148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab13e00640 1 -- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab1c0b4de0 msgr2=0x7fab1c148b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab13e00640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab1c0b4de0 0x7fab1c148b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab13e00640 1 -- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab1c1527b0 msgr2=0x7fab1c157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab13e00640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab1c1527b0 0x7fab1c157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab13e00640 1 -- 172.21.15.50:0/2730437238 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c0cce90 con 0x7fab1c0b5c70 2024-09-21T06:02:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab13e00640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1c0b5c70 0x7fab1c0cc950 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fab0400e7a0 tx=0x7fab0400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab12a00640 1 -- 172.21.15.50:0/2730437238 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0400ccc0 con 0x7fab1c0b5c70 2024-09-21T06:02:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab12a00640 1 -- 172.21.15.50:0/2730437238 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab04010430 con 0x7fab1c0b5c70 2024-09-21T06:02:07.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab21600640 1 -- 172.21.15.50:0/2730437238 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1c0cd3a0 con 0x7fab1c0b5c70 2024-09-21T06:02:07.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab12a00640 1 -- 172.21.15.50:0/2730437238 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab04010860 con 0x7fab1c0b5c70 2024-09-21T06:02:07.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab21600640 1 -- 172.21.15.50:0/2730437238 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1c0cd960 con 0x7fab1c0b5c70 2024-09-21T06:02:07.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.180+0000 7fab21600640 1 -- 172.21.15.50:0/2730437238 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1c15fdd0 con 0x7fab1c0b5c70 2024-09-21T06:02:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.184+0000 7fab12a00640 1 -- 172.21.15.50:0/2730437238 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fab04010a00 con 0x7fab1c0b5c70 2024-09-21T06:02:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.184+0000 7fab12a00640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faaf8073b00 0x7faaf8076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.184+0000 7fab12a00640 1 -- 172.21.15.50:0/2730437238 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4885+0+0 (secure 0 0 0) 0x7fab04093350 con 0x7fab1c0b5c70 2024-09-21T06:02:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.184+0000 7fab1aa00640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faaf8073b00 0x7faaf8076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.188+0000 7fab1aa00640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faaf8073b00 0x7faaf8076010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009ac0 tx=0x7fab0c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:07.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.188+0000 7fab12a00640 1 -- 172.21.15.50:0/2730437238 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fab0405f840 con 0x7fab1c0b5c70 2024-09-21T06:02:07.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 -- 172.21.15.50:0/2730437238 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fab1c0cd130 con 0x7fab1c0b5c70 2024-09-21T06:02:07.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab12a00640 1 -- 172.21.15.50:0/2730437238 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) ==== 80+0+5304 (secure 0 0 0) 0x7fab04064270 con 0x7fab1c0b5c70 2024-09-21T06:02:07.433 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:07.433 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:02:07.433 INFO:teuthology.orchestra.run.smithi050.stdout: "auth_dump": [ 2024-09-21T06:02:07.433 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.433 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.a", 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mSKq0ARAA2egaHGYWhRUEq6oRl1CIMg==", 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.b", 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mwv0+DxAAY+eZPxv5uY9PnLl0asdr6w==", 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.434 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.c", 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5me6maCRAA4jvHgTLTKpb/mxzzG71SfQ==", 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:02:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.d", 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mzYnuFhAA/hEigjluXspzs/hyeWxBjQ==", 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.436 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.0", 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDIX+5mrZF/NhAA33M6ummG4r4yXU8/YOL5WA==", 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.1", 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDMX+5myVz5DxAAMYfTPep6ubD7Cxnfa9dMCg==", 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.437 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:02:07.438 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:02:07.438 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.438 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.438 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.438 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.438 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.2", 2024-09-21T06:02:07.438 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDPX+5mA7ahIRAAf/osnY5l83p8ZWna0ErD2Q==", 2024-09-21T06:02:07.438 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.438 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:02:07.438 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:02:07.439 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.439 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.439 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.439 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.439 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.3", 2024-09-21T06:02:07.439 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDSX+5m+rDpNRAApGPZJIlAaA0DyjtPXPgoaQ==", 2024-09-21T06:02:07.439 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.439 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:02:07.439 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:02:07.439 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.439 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.439 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.4", 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDYX+5mKL9SORAA21ITQUwUSoDUjXaD0Cwb3g==", 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.5", 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDcX+5minRqFBAAMJOBQNvJFD4TNdKbPMLBig==", 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.440 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.6", 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDfX+5m6HadJhAAAv670Tvyi4IKMByFPiBlAw==", 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.441 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.7", 2024-09-21T06:02:07.442 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDjX+5muPrrAhAAuRB7Muamiht+f7LvSmI6UQ==", 2024-09-21T06:02:07.442 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.442 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:02:07.442 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:02:07.442 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.442 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.442 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.442 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.442 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.0", 2024-09-21T06:02:07.442 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw==", 2024-09-21T06:02:07.442 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow r", 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.1", 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mo94GJBAAMBSb1NPb2L93vzvnAh59kw==", 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:02:07.443 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:02:07.444 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.444 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.444 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.444 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.2", 2024-09-21T06:02:07.444 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5m4IrnHxAAFjER3ywuS+FlbuA7bPZVqw==", 2024-09-21T06:02:07.444 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.444 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:02:07.444 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:02:07.444 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:02:07.444 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:02:07.444 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.444 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.3", 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mysQeKBAAq5c4bD3hbsU1H7JabaMjwQ==", 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.admin", 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mcUXKHhAAV1Rs0OMNVPeAsRiAVt67jA==", 2024-09-21T06:02:07.445 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.446 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:02:07.446 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:02:07.446 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:02:07.446 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.446 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.446 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.446 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.446 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.x", 2024-09-21T06:02:07.446 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5m2F9WOBAAZuQTUyR7oTT01CRD1jo+/Q==", 2024-09-21T06:02:07.446 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.446 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.y", 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mEjC/MxAAHgwIu4CLDgFajwD2rL5WKA==", 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.447 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.448 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-21T06:02:07.448 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:02:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 -- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faaf8073b00 msgr2=0x7faaf8076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faaf8073b00 0x7faaf8076010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009ac0 tx=0x7fab0c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 -- 172.21.15.50:0/2730437238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1c0b5c70 msgr2=0x7fab1c0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1c0b5c70 0x7fab1c0cc950 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fab0400e7a0 tx=0x7fab0400ec60 comp rx=0 tx=0).stop 2024-09-21T06:02:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 -- 172.21.15.50:0/2730437238 shutdown_connections 2024-09-21T06:02:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab1c1527b0 0x7fab1c157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faaf8073b00 0x7faaf8076010 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1c0b5c70 0x7fab1c0cc950 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 --2- 172.21.15.50:0/2730437238 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab1c0b4de0 0x7fab1c148b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 -- 172.21.15.50:0/2730437238 >> 172.21.15.50:0/2730437238 conn(0x7fab1c0b2040 msgr2=0x7fab1c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 -- 172.21.15.50:0/2730437238 shutdown_connections 2024-09-21T06:02:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.432+0000 7fab21600640 1 -- 172.21.15.50:0/2730437238 wait complete. 2024-09-21T06:02:07.455 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:02:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.612+0000 7f75c1800640 1 -- 172.21.15.50:0/268589700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75a0003c40 msgr2=0x7f75a00040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.612+0000 7f75c1800640 1 --2- 172.21.15.50:0/268589700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75a0003c40 0x7f75a00040b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f75ac001860 tx=0x7f75ac01c890 comp rx=0 tx=0).stop 2024-09-21T06:02:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.612+0000 7f75c1800640 1 -- 172.21.15.50:0/268589700 shutdown_connections 2024-09-21T06:02:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.612+0000 7f75c1800640 1 --2- 172.21.15.50:0/268589700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75a0003c40 0x7f75a00040b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.612+0000 7f75c1800640 1 -- 172.21.15.50:0/268589700 >> 172.21.15.50:0/268589700 conn(0x7f75bc0b2040 msgr2=0x7f75bc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.612+0000 7f75c1800640 1 -- 172.21.15.50:0/268589700 shutdown_connections 2024-09-21T06:02:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.612+0000 7f75c1800640 1 -- 172.21.15.50:0/268589700 wait complete. 2024-09-21T06:02:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75c1800640 1 Processor -- start 2024-09-21T06:02:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75c1800640 1 -- start start 2024-09-21T06:02:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75c1800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75a0003c40 0x7f75bc13cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75c1800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75bc159130 0x7f75bc0b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75c1800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75bc0ce400 0x7f75bc0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75c1800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc0b55d0 con 0x7f75a0003c40 2024-09-21T06:02:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75c1800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc0b5450 con 0x7f75bc0ce400 2024-09-21T06:02:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75c1800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f75bc0b5750 con 0x7f75bc159130 2024-09-21T06:02:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75baa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75a0003c40 0x7f75bc13cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75baa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75a0003c40 0x7f75bc13cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48046/0 (socket says 172.21.15.50:48046) 2024-09-21T06:02:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75baa00640 1 -- 172.21.15.50:0/538404427 learned_addr learned my addr 172.21.15.50:0/538404427 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75bb400640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75bc0ce400 0x7f75bc0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75b2000640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75bc159130 0x7f75bc0b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75baa00640 1 -- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75bc159130 msgr2=0x7f75bc0b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75baa00640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75bc159130 0x7f75bc0b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75baa00640 1 -- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75bc0ce400 msgr2=0x7f75bc0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75baa00640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75bc0ce400 0x7f75bc0ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75baa00640 1 -- 172.21.15.50:0/538404427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc0b0980 con 0x7f75a0003c40 2024-09-21T06:02:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75baa00640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75a0003c40 0x7f75bc13cc10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f75ac005040 tx=0x7f75ac0a5c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75b3400640 1 -- 172.21.15.50:0/538404427 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac001480 con 0x7f75a0003c40 2024-09-21T06:02:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75b3400640 1 -- 172.21.15.50:0/538404427 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75ac0038f0 con 0x7f75a0003c40 2024-09-21T06:02:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75c1800640 1 -- 172.21.15.50:0/538404427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75bc1566a0 con 0x7f75a0003c40 2024-09-21T06:02:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75b3400640 1 -- 172.21.15.50:0/538404427 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac0068d0 con 0x7f75a0003c40 2024-09-21T06:02:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.616+0000 7f75c1800640 1 -- 172.21.15.50:0/538404427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75bc156c20 con 0x7f75a0003c40 2024-09-21T06:02:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.620+0000 7f75c1800640 1 -- 172.21.15.50:0/538404427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75bc0b1950 con 0x7f75a0003c40 2024-09-21T06:02:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.620+0000 7f75b3400640 1 -- 172.21.15.50:0/538404427 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f75ac006150 con 0x7f75a0003c40 2024-09-21T06:02:07.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.620+0000 7f75b3400640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7598073b00 0x7f7598076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.620+0000 7f75b3400640 1 -- 172.21.15.50:0/538404427 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4885+0+0 (secure 0 0 0) 0x7f75ac12cfc0 con 0x7f75a0003c40 2024-09-21T06:02:07.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.620+0000 7f75b2000640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7598073b00 0x7f7598076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.620+0000 7f75b2000640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7598073b00 0x7f7598076010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0cca90 tx=0x7f75a40091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:07.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.624+0000 7f75b3400640 1 -- 172.21.15.50:0/538404427 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f75ac016670 con 0x7f75a0003c40 2024-09-21T06:02:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.800+0000 7f75c1800640 1 -- 172.21.15.50:0/538404427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f75bc15f560 con 0x7f75a0003c40 2024-09-21T06:02:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75b3400640 1 -- 172.21.15.50:0/538404427 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) ==== 84+0+65 (secure 0 0 0) 0x7f75ac0f9580 con 0x7f75a0003c40 2024-09-21T06:02:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:07.805 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:02:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-09-21T06:02:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-09-21T06:02:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:07.805 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:02:07.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75c1800640 1 -- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7598073b00 msgr2=0x7f7598076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75c1800640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7598073b00 0x7f7598076010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0cca90 tx=0x7f75a40091e0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75c1800640 1 -- 172.21.15.50:0/538404427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75a0003c40 msgr2=0x7f75bc13cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75c1800640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75a0003c40 0x7f75bc13cc10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f75ac005040 tx=0x7f75ac0a5c80 comp rx=0 tx=0).stop 2024-09-21T06:02:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75c1800640 1 -- 172.21.15.50:0/538404427 shutdown_connections 2024-09-21T06:02:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75c1800640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75bc0ce400 0x7f75bc0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75c1800640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75bc159130 0x7f75bc0b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75c1800640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7598073b00 0x7f7598076010 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75c1800640 1 --2- 172.21.15.50:0/538404427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75a0003c40 0x7f75bc13cc10 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75c1800640 1 -- 172.21.15.50:0/538404427 >> 172.21.15.50:0/538404427 conn(0x7f75bc0b2040 msgr2=0x7f75bc151c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75c1800640 1 -- 172.21.15.50:0/538404427 shutdown_connections 2024-09-21T06:02:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.804+0000 7f75c1800640 1 -- 172.21.15.50:0/538404427 wait complete. 2024-09-21T06:02:07.827 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:02:07.827 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:02:07.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.988+0000 7f6ee1000640 1 -- 172.21.15.50:0/3342577123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6edc15f5b0 msgr2=0x7f6edc0b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.988+0000 7f6ee1000640 1 --2- 172.21.15.50:0/3342577123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6edc15f5b0 0x7f6edc0b1dc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009840 tx=0x7f6ec401c7f0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.988+0000 7f6ee1000640 1 -- 172.21.15.50:0/3342577123 shutdown_connections 2024-09-21T06:02:07.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.988+0000 7f6ee1000640 1 --2- 172.21.15.50:0/3342577123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6edc15f5b0 0x7f6edc0b1dc0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.988+0000 7f6ee1000640 1 --2- 172.21.15.50:0/3342577123 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6edc155a00 0x7f6edc15f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.988+0000 7f6ee1000640 1 -- 172.21.15.50:0/3342577123 >> 172.21.15.50:0/3342577123 conn(0x7f6edc06cdf0 msgr2=0x7f6edc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:07.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.988+0000 7f6ee1000640 1 -- 172.21.15.50:0/3342577123 shutdown_connections 2024-09-21T06:02:07.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.988+0000 7f6ee1000640 1 -- 172.21.15.50:0/3342577123 wait complete. 2024-09-21T06:02:07.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ee1000640 1 Processor -- start 2024-09-21T06:02:07.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ee1000640 1 -- start start 2024-09-21T06:02:07.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ee1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6edc0b2300 0x7f6edc1f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ee1000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6edc155a00 0x7f6edc138980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ee1000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6edc15f5b0 0x7f6edc1fb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ee1000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc168870 con 0x7f6edc0b2300 2024-09-21T06:02:07.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ee1000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc1686f0 con 0x7f6edc155a00 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ee1000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6edc1689f0 con 0x7f6edc15f5b0 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6edb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6edc0b2300 0x7f6edc1f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6edb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6edc0b2300 0x7f6edc1f8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48068/0 (socket says 172.21.15.50:48068) 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6edb400640 1 -- 172.21.15.50:0/1544784307 learned_addr learned my addr 172.21.15.50:0/1544784307 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ed3e00640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6edc155a00 0x7f6edc138980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6edbe00640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6edc15f5b0 0x7f6edc1fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6edb400640 1 -- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6edc15f5b0 msgr2=0x7f6edc1fb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6edb400640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6edc15f5b0 0x7f6edc1fb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6edb400640 1 -- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6edc155a00 msgr2=0x7f6edc138980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6edb400640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6edc155a00 0x7f6edc138980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6edb400640 1 -- 172.21.15.50:0/1544784307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc14b680 con 0x7f6edc0b2300 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6edbe00640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6edc15f5b0 0x7f6edc1fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6edb400640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6edc0b2300 0x7f6edc1f8f70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00d720 tx=0x7f6ecc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ed3400640 1 -- 172.21.15.50:0/1544784307 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc017070 con 0x7f6edc0b2300 2024-09-21T06:02:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ed3400640 1 -- 172.21.15.50:0/1544784307 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ecc009d20 con 0x7f6edc0b2300 2024-09-21T06:02:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ee1000640 1 -- 172.21.15.50:0/1544784307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6edc1fc550 con 0x7f6edc0b2300 2024-09-21T06:02:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ed3400640 1 -- 172.21.15.50:0/1544784307 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc005280 con 0x7f6edc0b2300 2024-09-21T06:02:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ee1000640 1 -- 172.21.15.50:0/1544784307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6edc1f8490 con 0x7f6edc0b2300 2024-09-21T06:02:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.992+0000 7f6ee1000640 1 -- 172.21.15.50:0/1544784307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6edc15f070 con 0x7f6edc0b2300 2024-09-21T06:02:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.996+0000 7f6ed3400640 1 -- 172.21.15.50:0/1544784307 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f6ecc005420 con 0x7f6edc0b2300 2024-09-21T06:02:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.996+0000 7f6ed3400640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6eb8073b30 0x7f6eb8076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.996+0000 7f6ed3400640 1 -- 172.21.15.50:0/1544784307 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4885+0+0 (secure 0 0 0) 0x7f6ecc095370 con 0x7f6edc0b2300 2024-09-21T06:02:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.996+0000 7f6ed3e00640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6eb8073b30 0x7f6eb8076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:07.996+0000 7f6ed3e00640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6eb8073b30 0x7f6eb8076040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40af000 tx=0x7f6ec401b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:08.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.000+0000 7f6ed3400640 1 -- 172.21.15.50:0/1544784307 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6edc14b680 con 0x7f6edc0b2300 2024-09-21T06:02:08.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.180+0000 7f6ee1000640 1 -- 172.21.15.50:0/1544784307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6edc0b34b0 con 0x7f6edc0b2300 2024-09-21T06:02:08.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.232+0000 7f6ed3400640 1 -- 172.21.15.50:0/1544784307 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) ==== 138+0+0 (secure 0 0 0) 0x7f6ecc05f6a0 con 0x7f6edc0b2300 2024-09-21T06:02:08.235 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-09-21T06:02:08.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.236+0000 7f6ee1000640 1 -- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6eb8073b30 msgr2=0x7f6eb8076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:08.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.236+0000 7f6ee1000640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6eb8073b30 0x7f6eb8076040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40af000 tx=0x7f6ec401b030 comp rx=0 tx=0).stop 2024-09-21T06:02:08.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.236+0000 7f6ee1000640 1 -- 172.21.15.50:0/1544784307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6edc0b2300 msgr2=0x7f6edc1f8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:08.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.236+0000 7f6ee1000640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6edc0b2300 0x7f6edc1f8f70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00d720 tx=0x7f6ecc00dbe0 comp rx=0 tx=0).stop 2024-09-21T06:02:08.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.236+0000 7f6ee1000640 1 -- 172.21.15.50:0/1544784307 shutdown_connections 2024-09-21T06:02:08.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.236+0000 7f6ee1000640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6edc15f5b0 0x7f6edc1fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:08.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.236+0000 7f6ee1000640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6edc155a00 0x7f6edc138980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:08.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.236+0000 7f6ee1000640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6eb8073b30 0x7f6eb8076040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:08.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.236+0000 7f6ee1000640 1 --2- 172.21.15.50:0/1544784307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6edc0b2300 0x7f6edc1f8f70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:08.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.236+0000 7f6ee1000640 1 -- 172.21.15.50:0/1544784307 >> 172.21.15.50:0/1544784307 conn(0x7f6edc06cdf0 msgr2=0x7f6edc073190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:08.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.236+0000 7f6ee1000640 1 -- 172.21.15.50:0/1544784307 shutdown_connections 2024-09-21T06:02:08.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.236+0000 7f6ee1000640 1 -- 172.21.15.50:0/1544784307 wait complete. 2024-09-21T06:02:08.263 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:02:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.428+0000 7f56baa00640 1 -- 172.21.15.50:0/1189359873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698004810 msgr2=0x7f5698004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.428+0000 7f56baa00640 1 --2- 172.21.15.50:0/1189359873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698004810 0x7f5698004c20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f56b4071fd0 tx=0x7f569c01d220 comp rx=0 tx=0).stop 2024-09-21T06:02:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.428+0000 7f56b8a00640 1 -- 172.21.15.50:0/1189359873 reap_dead start 2024-09-21T06:02:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.428+0000 7f56baa00640 1 -- 172.21.15.50:0/1189359873 shutdown_connections 2024-09-21T06:02:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.428+0000 7f56baa00640 1 -- 172.21.15.50:0/1189359873 >> 172.21.15.50:0/1189359873 conn(0x7f56b40b2040 msgr2=0x7f56b40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.428+0000 7f56baa00640 1 -- 172.21.15.50:0/1189359873 shutdown_connections 2024-09-21T06:02:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.428+0000 7f56baa00640 1 -- 172.21.15.50:0/1189359873 wait complete. 2024-09-21T06:02:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.428+0000 7f56baa00640 1 Processor -- start 2024-09-21T06:02:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.428+0000 7f56baa00640 1 -- start start 2024-09-21T06:02:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56baa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698004810 0x7f56b414ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56baa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56b41554f0 0x7f56b4155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56baa00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f56b40affb0 0x7f56b40b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56baa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4166940 con 0x7f5698004810 2024-09-21T06:02:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56baa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41667c0 con 0x7f56b41554f0 2024-09-21T06:02:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56baa00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f56b4166ac0 con 0x7f56b40affb0 2024-09-21T06:02:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698004810 0x7f56b414ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698004810 0x7f56b414ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48080/0 (socket says 172.21.15.50:48080) 2024-09-21T06:02:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b3e00640 1 -- 172.21.15.50:0/1524560855 learned_addr learned my addr 172.21.15.50:0/1524560855 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b8a00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f56b40affb0 0x7f56b40b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56abe00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56b41554f0 0x7f56b4155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b3e00640 1 -- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f56b40affb0 msgr2=0x7f56b40b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b3e00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f56b40affb0 0x7f56b40b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b3e00640 1 -- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56b41554f0 msgr2=0x7f56b4155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b3e00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56b41554f0 0x7f56b4155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b3e00640 1 -- 172.21.15.50:0/1524560855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b40b0900 con 0x7f5698004810 2024-09-21T06:02:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b3e00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698004810 0x7f56b414ed50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f56a4011ab0 tx=0x7f56a4011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:08.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b0c00640 1 -- 172.21.15.50:0/1524560855 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a400fe10 con 0x7f5698004810 2024-09-21T06:02:08.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b0c00640 1 -- 172.21.15.50:0/1524560855 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56a4016070 con 0x7f5698004810 2024-09-21T06:02:08.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56b0c00640 1 -- 172.21.15.50:0/1524560855 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a40126b0 con 0x7f5698004810 2024-09-21T06:02:08.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56baa00640 1 -- 172.21.15.50:0/1524560855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b40aebc0 con 0x7f5698004810 2024-09-21T06:02:08.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56baa00640 1 -- 172.21.15.50:0/1524560855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b40af180 con 0x7f5698004810 2024-09-21T06:02:08.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.432+0000 7f56baa00640 1 -- 172.21.15.50:0/1524560855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b40af8a0 con 0x7f5698004810 2024-09-21T06:02:08.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.436+0000 7f56b0c00640 1 -- 172.21.15.50:0/1524560855 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f56a4018030 con 0x7f5698004810 2024-09-21T06:02:08.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.436+0000 7f56b0c00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5690073b00 0x7f5690076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:08.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.436+0000 7f56b0c00640 1 -- 172.21.15.50:0/1524560855 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5255+0+0 (secure 0 0 0) 0x7f56a4093bf0 con 0x7f5698004810 2024-09-21T06:02:08.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.436+0000 7f56abe00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5690073b00 0x7f5690076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.436+0000 7f56abe00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5690073b00 0x7f5690076010 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f56b4071fd0 tx=0x7f569c019fc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:08.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.440+0000 7f56b0c00640 1 -- 172.21.15.50:0/1524560855 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f56b40b0900 con 0x7f5698004810 2024-09-21T06:02:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:08.616+0000 7f56baa00640 1 -- 172.21.15.50:0/1524560855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f56b4074e30 con 0x7f5698004810 2024-09-21T06:02:09.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.240+0000 7f56b0c00640 1 -- 172.21.15.50:0/1524560855 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) ==== 170+0+0 (secure 0 0 0) 0x7f56a40600a0 con 0x7f5698004810 2024-09-21T06:02:09.243 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-09-21T06:02:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.244+0000 7f56baa00640 1 -- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5690073b00 msgr2=0x7f5690076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.244+0000 7f56baa00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5690073b00 0x7f5690076010 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f56b4071fd0 tx=0x7f569c019fc0 comp rx=0 tx=0).stop 2024-09-21T06:02:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.244+0000 7f56baa00640 1 -- 172.21.15.50:0/1524560855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698004810 msgr2=0x7f56b414ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.244+0000 7f56baa00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698004810 0x7f56b414ed50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f56a4011ab0 tx=0x7f56a4011f70 comp rx=0 tx=0).stop 2024-09-21T06:02:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.244+0000 7f56baa00640 1 -- 172.21.15.50:0/1524560855 shutdown_connections 2024-09-21T06:02:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.244+0000 7f56baa00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f56b40affb0 0x7f56b40b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.244+0000 7f56baa00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56b41554f0 0x7f56b4155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.244+0000 7f56baa00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5690073b00 0x7f5690076010 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:09.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.244+0000 7f56baa00640 1 --2- 172.21.15.50:0/1524560855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698004810 0x7f56b414ed50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:09.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.244+0000 7f56baa00640 1 -- 172.21.15.50:0/1524560855 >> 172.21.15.50:0/1524560855 conn(0x7f56b40b2040 msgr2=0x7f56b4150e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:09.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.244+0000 7f56baa00640 1 -- 172.21.15.50:0/1524560855 shutdown_connections 2024-09-21T06:02:09.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.244+0000 7f56baa00640 1 -- 172.21.15.50:0/1524560855 wait complete. 2024-09-21T06:02:09.293 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:02:09.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.456+0000 7f5151200640 1 -- 172.21.15.50:0/4005423294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f514c1527b0 msgr2=0x7f514c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:09.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.456+0000 7f5151200640 1 --2- 172.21.15.50:0/4005423294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f514c1527b0 0x7f514c074e30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f514c071cb0 tx=0x7f513401c890 comp rx=0 tx=0).stop 2024-09-21T06:02:09.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.456+0000 7f5151200640 1 -- 172.21.15.50:0/4005423294 shutdown_connections 2024-09-21T06:02:09.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.456+0000 7f5151200640 1 --2- 172.21.15.50:0/4005423294 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f514c0b4de0 0x7f514c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:09.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.456+0000 7f5151200640 1 --2- 172.21.15.50:0/4005423294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f514c1527b0 0x7f514c074e30 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:09.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.456+0000 7f5151200640 1 --2- 172.21.15.50:0/4005423294 >> v2:172.21.15.88:3300/0 conn(0x7f514c0b5c70 0x7f514c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:09.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.456+0000 7f5151200640 1 -- 172.21.15.50:0/4005423294 >> 172.21.15.50:0/4005423294 conn(0x7f514c0b2040 msgr2=0x7f514c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:09.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.456+0000 7f5151200640 1 -- 172.21.15.50:0/4005423294 shutdown_connections 2024-09-21T06:02:09.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.456+0000 7f5151200640 1 -- 172.21.15.50:0/4005423294 wait complete. 2024-09-21T06:02:09.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.456+0000 7f5151200640 1 Processor -- start 2024-09-21T06:02:09.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.456+0000 7f5151200640 1 -- start start 2024-09-21T06:02:09.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5151200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f514c0b4de0 0x7f514c0cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:09.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5151200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f514c0b5c70 0x7f514c0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:09.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5151200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f514c1527b0 0x7f514c0cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:09.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5151200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f514c15f6b0 con 0x7f514c1527b0 2024-09-21T06:02:09.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5151200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f514c15f530 con 0x7f514c0b4de0 2024-09-21T06:02:09.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5151200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f514c15f830 con 0x7f514c0b5c70 2024-09-21T06:02:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f514a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f514c1527b0 0x7f514c0cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5143e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f514c0b5c70 0x7f514c0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5149e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f514c0b4de0 0x7f514c0cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f514a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f514c1527b0 0x7f514c0cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48106/0 (socket says 172.21.15.50:48106) 2024-09-21T06:02:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f514a800640 1 -- 172.21.15.50:0/3222677490 learned_addr learned my addr 172.21.15.50:0/3222677490 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f514a800640 1 -- 172.21.15.50:0/3222677490 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f514c0b5c70 msgr2=0x7f514c0b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f514a800640 1 --2- 172.21.15.50:0/3222677490 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f514c0b5c70 0x7f514c0b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f514a800640 1 -- 172.21.15.50:0/3222677490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f514c0b4de0 msgr2=0x7f514c0cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f514a800640 1 --2- 172.21.15.50:0/3222677490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f514c0b4de0 0x7f514c0cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f514a800640 1 -- 172.21.15.50:0/3222677490 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c0b4210 con 0x7f514c1527b0 2024-09-21T06:02:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f514a800640 1 --2- 172.21.15.50:0/3222677490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f514c1527b0 0x7f514c0cd480 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f514400e9d0 tx=0x7f514400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5142000640 1 -- 172.21.15.50:0/3222677490 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514400cde0 con 0x7f514c1527b0 2024-09-21T06:02:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5142000640 1 -- 172.21.15.50:0/3222677490 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5144004c20 con 0x7f514c1527b0 2024-09-21T06:02:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5142000640 1 -- 172.21.15.50:0/3222677490 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144005020 con 0x7f514c1527b0 2024-09-21T06:02:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5151200640 1 -- 172.21.15.50:0/3222677490 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514c0b4720 con 0x7f514c1527b0 2024-09-21T06:02:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5151200640 1 -- 172.21.15.50:0/3222677490 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f514c1f6e70 con 0x7f514c1527b0 2024-09-21T06:02:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.460+0000 7f5151200640 1 -- 172.21.15.50:0/3222677490 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514c15fd70 con 0x7f514c1527b0 2024-09-21T06:02:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.464+0000 7f5142000640 1 -- 172.21.15.50:0/3222677490 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f51440051c0 con 0x7f514c1527b0 2024-09-21T06:02:09.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.464+0000 7f5142000640 1 --2- 172.21.15.50:0/3222677490 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5128073b00 0x7f5128076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:09.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.464+0000 7f5142000640 1 -- 172.21.15.50:0/3222677490 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5637+0+0 (secure 0 0 0) 0x7f5144093ae0 con 0x7f514c1527b0 2024-09-21T06:02:09.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.464+0000 7f5149e00640 1 --2- 172.21.15.50:0/3222677490 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5128073b00 0x7f5128076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:09.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.464+0000 7f5149e00640 1 --2- 172.21.15.50:0/3222677490 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5128073b00 0x7f5128076010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f513c009b40 tx=0x7f513c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:09.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.468+0000 7f5142000640 1 -- 172.21.15.50:0/3222677490 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5144027080 con 0x7f514c1527b0 2024-09-21T06:02:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:09.684+0000 7f5151200640 1 -- 172.21.15.50:0/3222677490 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f514c0b4470 con 0x7f514c1527b0 2024-09-21T06:02:10.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.244+0000 7f5142000640 1 -- 172.21.15.50:0/3222677490 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v32) ==== 172+0+0 (secure 0 0 0) 0x7f514405ff90 con 0x7f514c1527b0 2024-09-21T06:02:10.245 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 8 and data pool 9 2024-09-21T06:02:10.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.244+0000 7f5151200640 1 -- 172.21.15.50:0/3222677490 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5128073b00 msgr2=0x7f5128076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:10.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.244+0000 7f5151200640 1 --2- 172.21.15.50:0/3222677490 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5128073b00 0x7f5128076010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f513c009b40 tx=0x7f513c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.244+0000 7f5151200640 1 -- 172.21.15.50:0/3222677490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f514c1527b0 msgr2=0x7f514c0cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:10.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.244+0000 7f5151200640 1 --2- 172.21.15.50:0/3222677490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f514c1527b0 0x7f514c0cd480 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f514400e9d0 tx=0x7f514400ee90 comp rx=0 tx=0).stop 2024-09-21T06:02:10.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.244+0000 7f5151200640 1 -- 172.21.15.50:0/3222677490 shutdown_connections 2024-09-21T06:02:10.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.244+0000 7f5151200640 1 --2- 172.21.15.50:0/3222677490 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5128073b00 0x7f5128076010 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.244+0000 7f5151200640 1 --2- 172.21.15.50:0/3222677490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f514c1527b0 0x7f514c0cd480 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.244+0000 7f5151200640 1 --2- 172.21.15.50:0/3222677490 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f514c0b5c70 0x7f514c0b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.244+0000 7f5151200640 1 --2- 172.21.15.50:0/3222677490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f514c0b4de0 0x7f514c0cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.244+0000 7f5151200640 1 -- 172.21.15.50:0/3222677490 >> 172.21.15.50:0/3222677490 conn(0x7f514c0b2040 msgr2=0x7f514c0b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:10.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.248+0000 7f5151200640 1 -- 172.21.15.50:0/3222677490 shutdown_connections 2024-09-21T06:02:10.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.248+0000 7f5151200640 1 -- 172.21.15.50:0/3222677490 wait complete. 2024-09-21T06:02:10.295 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:02:10.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.452+0000 7fd4cd000640 1 -- 172.21.15.50:0/4252256531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4c80703a0 msgr2=0x7fd4c806d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:10.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.452+0000 7fd4cd000640 1 --2- 172.21.15.50:0/4252256531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4c80703a0 0x7fd4c806d200 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009bc0 tx=0x7fd4c001cb10 comp rx=0 tx=0).stop 2024-09-21T06:02:10.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.452+0000 7fd4cd000640 1 -- 172.21.15.50:0/4252256531 shutdown_connections 2024-09-21T06:02:10.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.452+0000 7fd4cd000640 1 --2- 172.21.15.50:0/4252256531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4c80703a0 0x7fd4c806d200 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.452+0000 7fd4cd000640 1 -- 172.21.15.50:0/4252256531 >> 172.21.15.50:0/4252256531 conn(0x7fd4c80b35f0 msgr2=0x7fd4c80b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:10.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.452+0000 7fd4cd000640 1 -- 172.21.15.50:0/4252256531 shutdown_connections 2024-09-21T06:02:10.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.452+0000 7fd4cd000640 1 -- 172.21.15.50:0/4252256531 wait complete. 2024-09-21T06:02:10.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4cd000640 1 Processor -- start 2024-09-21T06:02:10.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4cd000640 1 -- start start 2024-09-21T06:02:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4cd000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd4c806cdf0 0x7fd4c81542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4cd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c80703a0 0x7fd4c80aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4cd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4c8159130 0x7fd4c8151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4cd000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c816a470 con 0x7fd4c8159130 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4cd000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c816a2f0 con 0x7fd4c80703a0 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4cd000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c816a5f0 con 0x7fd4c806cdf0 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4c6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4c8159130 0x7fd4c8151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4c6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4c8159130 0x7fd4c8151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48112/0 (socket says 172.21.15.50:48112) 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4c6800640 1 -- 172.21.15.50:0/2351334457 learned_addr learned my addr 172.21.15.50:0/2351334457 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4c5e00640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd4c806cdf0 0x7fd4c81542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4c5400640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c80703a0 0x7fd4c80aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4c6800640 1 -- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd4c806cdf0 msgr2=0x7fd4c81542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4c6800640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd4c806cdf0 0x7fd4c81542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4c6800640 1 -- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c80703a0 msgr2=0x7fd4c80aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4c6800640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c80703a0 0x7fd4c80aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4c6800640 1 -- 172.21.15.50:0/2351334457 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c81522a0 con 0x7fd4c8159130 2024-09-21T06:02:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4c6800640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4c8159130 0x7fd4c8151a40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00b0040 tx=0x7fd4c0005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4be000640 1 -- 172.21.15.50:0/2351334457 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c00ba020 con 0x7fd4c8159130 2024-09-21T06:02:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4be000640 1 -- 172.21.15.50:0/2351334457 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4c0004330 con 0x7fd4c8159130 2024-09-21T06:02:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4cd000640 1 -- 172.21.15.50:0/2351334457 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c8152580 con 0x7fd4c8159130 2024-09-21T06:02:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4be000640 1 -- 172.21.15.50:0/2351334457 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c00b2040 con 0x7fd4c8159130 2024-09-21T06:02:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.456+0000 7fd4cd000640 1 -- 172.21.15.50:0/2351334457 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c8152b40 con 0x7fd4c8159130 2024-09-21T06:02:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.460+0000 7fd4cd000640 1 -- 172.21.15.50:0/2351334457 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c806d200 con 0x7fd4c8159130 2024-09-21T06:02:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.460+0000 7fd4be000640 1 -- 172.21.15.50:0/2351334457 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fd4c00bd020 con 0x7fd4c8159130 2024-09-21T06:02:10.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.460+0000 7fd4be000640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd4a8073b30 0x7fd4a8076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:10.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.460+0000 7fd4be000640 1 -- 172.21.15.50:0/2351334457 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5737+0+0 (secure 0 0 0) 0x7fd4c012db60 con 0x7fd4c8159130 2024-09-21T06:02:10.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.460+0000 7fd4c5e00640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd4a8073b30 0x7fd4a8076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:10.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.460+0000 7fd4c5e00640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd4a8073b30 0x7fd4a8076040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8000f70 tx=0x7fd4b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:10.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.464+0000 7fd4be000640 1 -- 172.21.15.50:0/2351334457 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd4c00c4030 con 0x7fd4c8159130 2024-09-21T06:02:10.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.640+0000 7fd4cd000640 1 -- 172.21.15.50:0/2351334457 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4c8070fd0 con 0x7fd4c8159130 2024-09-21T06:02:10.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4be000640 1 -- 172.21.15.50:0/2351334457 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16733 (secure 0 0 0) 0x7fd4c0016790 con 0x7fd4c8159130 2024-09-21T06:02:10.645 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:10.645 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":58,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:02:09.689448+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:02:08.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T06:02:08.630388+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:02:10.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4cd000640 1 -- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd4a8073b30 msgr2=0x7fd4a8076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:10.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4cd000640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd4a8073b30 0x7fd4a8076040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8000f70 tx=0x7fd4b8009040 comp rx=0 tx=0).stop 2024-09-21T06:02:10.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4cd000640 1 -- 172.21.15.50:0/2351334457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4c8159130 msgr2=0x7fd4c8151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:10.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4cd000640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4c8159130 0x7fd4c8151a40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00b0040 tx=0x7fd4c0005ae0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4cd000640 1 -- 172.21.15.50:0/2351334457 shutdown_connections 2024-09-21T06:02:10.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4cd000640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd4a8073b30 0x7fd4a8076040 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4cd000640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4c8159130 0x7fd4c8151a40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4cd000640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c80703a0 0x7fd4c80aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4cd000640 1 --2- 172.21.15.50:0/2351334457 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd4c806cdf0 0x7fd4c81542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4cd000640 1 -- 172.21.15.50:0/2351334457 >> 172.21.15.50:0/2351334457 conn(0x7fd4c80b35f0 msgr2=0x7fd4c80b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:10.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4cd000640 1 -- 172.21.15.50:0/2351334457 shutdown_connections 2024-09-21T06:02:10.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.644+0000 7fd4cd000640 1 -- 172.21.15.50:0/2351334457 wait complete. 2024-09-21T06:02:10.668 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:02:10.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.824+0000 7fa966600640 1 -- 172.21.15.50:0/3429298378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa96006cdf0 msgr2=0x7fa96006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:10.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.824+0000 7fa966600640 1 --2- 172.21.15.50:0/3429298378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa96006cdf0 0x7fa96006d200 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa9600b6bf0 tx=0x7fa94801c830 comp rx=0 tx=0).stop 2024-09-21T06:02:10.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.824+0000 7fa966600640 1 -- 172.21.15.50:0/3429298378 shutdown_connections 2024-09-21T06:02:10.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.824+0000 7fa966600640 1 --2- 172.21.15.50:0/3429298378 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa96006fc90 0x7fa9600700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.824+0000 7fa966600640 1 --2- 172.21.15.50:0/3429298378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa96006cdf0 0x7fa96006d200 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.824+0000 7fa966600640 1 --2- 172.21.15.50:0/3429298378 >> v2:172.21.15.88:3300/0 conn(0x7fa960159130 0x7fa96015bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.824+0000 7fa966600640 1 -- 172.21.15.50:0/3429298378 >> 172.21.15.50:0/3429298378 conn(0x7fa9600b35f0 msgr2=0x7fa9600b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:10.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.824+0000 7fa966600640 1 -- 172.21.15.50:0/3429298378 shutdown_connections 2024-09-21T06:02:10.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.824+0000 7fa966600640 1 -- 172.21.15.50:0/3429298378 wait complete. 2024-09-21T06:02:10.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa966600640 1 Processor -- start 2024-09-21T06:02:10.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa966600640 1 -- start start 2024-09-21T06:02:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa966600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa96006cdf0 0x7fa9601628f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa966600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa96006fc90 0x7fa96015d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa966600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa960159130 0x7fa96015d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa966600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa96015fa10 con 0x7fa96006fc90 2024-09-21T06:02:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa966600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa96015f890 con 0x7fa960159130 2024-09-21T06:02:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa966600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa96015fb90 con 0x7fa96006cdf0 2024-09-21T06:02:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa957e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa96006fc90 0x7fa96015d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa95f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa96006cdf0 0x7fa9601628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa95f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa96006cdf0 0x7fa9601628f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:40246/0 (socket says 172.21.15.50:40246) 2024-09-21T06:02:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa95f400640 1 -- 172.21.15.50:0/1121754389 learned_addr learned my addr 172.21.15.50:0/1121754389 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa95fe00640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa960159130 0x7fa96015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:10.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa957e00640 1 -- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa96006cdf0 msgr2=0x7fa9601628f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:10.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa957e00640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa96006cdf0 0x7fa9601628f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa957e00640 1 -- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa960159130 msgr2=0x7fa96015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:10.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa957e00640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa960159130 0x7fa96015d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:10.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa957e00640 1 -- 172.21.15.50:0/1121754389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96015eeb0 con 0x7fa96006fc90 2024-09-21T06:02:10.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa957e00640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa96006fc90 0x7fa96015d3f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa9480ac000 tx=0x7fa9480a66a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:10.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa957400640 1 -- 172.21.15.50:0/1121754389 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9480040c0 con 0x7fa96006fc90 2024-09-21T06:02:10.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa957400640 1 -- 172.21.15.50:0/1121754389 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa948004370 con 0x7fa96006fc90 2024-09-21T06:02:10.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa957400640 1 -- 172.21.15.50:0/1121754389 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948005280 con 0x7fa96006fc90 2024-09-21T06:02:10.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa966600640 1 -- 172.21.15.50:0/1121754389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa960164fb0 con 0x7fa96006fc90 2024-09-21T06:02:10.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.828+0000 7fa966600640 1 -- 172.21.15.50:0/1121754389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96015e470 con 0x7fa96006fc90 2024-09-21T06:02:10.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.832+0000 7fa966600640 1 -- 172.21.15.50:0/1121754389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9600700a0 con 0x7fa96006fc90 2024-09-21T06:02:10.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.832+0000 7fa957400640 1 -- 172.21.15.50:0/1121754389 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fa9480c0020 con 0x7fa96006fc90 2024-09-21T06:02:10.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.832+0000 7fa957400640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa93c073b00 0x7fa93c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:10.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.832+0000 7fa957400640 1 -- 172.21.15.50:0/1121754389 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5737+0+0 (secure 0 0 0) 0x7fa94812db30 con 0x7fa96006fc90 2024-09-21T06:02:10.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.836+0000 7fa95f400640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa93c073b00 0x7fa93c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:10.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.836+0000 7fa95f400640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa93c073b00 0x7fa93c076010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa950005fd0 tx=0x7fa95000b410 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:10.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:10.836+0000 7fa957400640 1 -- 172.21.15.50:0/1121754389 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa9480164b0 con 0x7fa96006fc90 2024-09-21T06:02:11.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.016+0000 7fa966600640 1 -- 172.21.15.50:0/1121754389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa96015ec40 con 0x7fa96006fc90 2024-09-21T06:02:11.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.016+0000 7fa957400640 1 -- 172.21.15.50:0/1121754389 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16733 (secure 0 0 0) 0x7fa9480f9f50 con 0x7fa96006fc90 2024-09-21T06:02:11.018 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:11.018 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":58,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:02:09.689448+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:02:08.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T06:02:08.630388+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:02:11.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.016+0000 7fa966600640 1 -- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa93c073b00 msgr2=0x7fa93c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:11.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.016+0000 7fa966600640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa93c073b00 0x7fa93c076010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa950005fd0 tx=0x7fa95000b410 comp rx=0 tx=0).stop 2024-09-21T06:02:11.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.016+0000 7fa966600640 1 -- 172.21.15.50:0/1121754389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa96006fc90 msgr2=0x7fa96015d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:11.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.016+0000 7fa966600640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa96006fc90 0x7fa96015d3f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa9480ac000 tx=0x7fa9480a66a0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.016+0000 7fa966600640 1 -- 172.21.15.50:0/1121754389 shutdown_connections 2024-09-21T06:02:11.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.016+0000 7fa966600640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa960159130 0x7fa96015d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.016+0000 7fa966600640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa93c073b00 0x7fa93c076010 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.016+0000 7fa966600640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa96006fc90 0x7fa96015d3f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.016+0000 7fa966600640 1 --2- 172.21.15.50:0/1121754389 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa96006cdf0 0x7fa9601628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.016+0000 7fa966600640 1 -- 172.21.15.50:0/1121754389 >> 172.21.15.50:0/1121754389 conn(0x7fa9600b35f0 msgr2=0x7fa9600b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:11.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.020+0000 7fa966600640 1 -- 172.21.15.50:0/1121754389 shutdown_connections 2024-09-21T06:02:11.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.020+0000 7fa966600640 1 -- 172.21.15.50:0/1121754389 wait complete. 2024-09-21T06:02:11.041 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:02:11.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.200+0000 7fd492e00640 1 -- 172.21.15.50:0/1568736763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd48c0b3340 msgr2=0x7fd48c0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:11.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.200+0000 7fd492e00640 1 --2- 172.21.15.50:0/1568736763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd48c0b3340 0x7fd48c0b3750 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0b3040 tx=0x7fd48401c8c0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.200+0000 7fd492e00640 1 -- 172.21.15.50:0/1568736763 shutdown_connections 2024-09-21T06:02:11.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.200+0000 7fd492e00640 1 --2- 172.21.15.50:0/1568736763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd48c0b3340 0x7fd48c0b3750 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.200+0000 7fd492e00640 1 --2- 172.21.15.50:0/1568736763 >> v2:172.21.15.88:3300/0 conn(0x7fd48c0749a0 0x7fd48c074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.200+0000 7fd492e00640 1 -- 172.21.15.50:0/1568736763 >> 172.21.15.50:0/1568736763 conn(0x7fd48c0b2040 msgr2=0x7fd48c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:11.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.200+0000 7fd492e00640 1 -- 172.21.15.50:0/1568736763 shutdown_connections 2024-09-21T06:02:11.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.200+0000 7fd492e00640 1 -- 172.21.15.50:0/1568736763 wait complete. 2024-09-21T06:02:11.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd492e00640 1 Processor -- start 2024-09-21T06:02:11.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd492e00640 1 -- start start 2024-09-21T06:02:11.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd492e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd48c0749a0 0x7fd48c145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:11.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd492e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd48c0b3340 0x7fd48c13a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:11.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd492e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd48c0cefa0 0x7fd48c13ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd492e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c169d50 con 0x7fd48c0cefa0 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd492e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c169bd0 con 0x7fd48c0749a0 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd492e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd48c169ed0 con 0x7fd48c0b3340 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd490e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd48c0cefa0 0x7fd48c13ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd490e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd48c0cefa0 0x7fd48c13ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48144/0 (socket says 172.21.15.50:48144) 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd490e00640 1 -- 172.21.15.50:0/1529387489 learned_addr learned my addr 172.21.15.50:0/1529387489 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd48be00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd48c0749a0 0x7fd48c145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd483400640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd48c0b3340 0x7fd48c13a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd490e00640 1 -- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd48c0b3340 msgr2=0x7fd48c13a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd490e00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd48c0b3340 0x7fd48c13a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd490e00640 1 -- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd48c0749a0 msgr2=0x7fd48c145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd490e00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd48c0749a0 0x7fd48c145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd490e00640 1 -- 172.21.15.50:0/1529387489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c1440b0 con 0x7fd48c0cefa0 2024-09-21T06:02:11.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd490e00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd48c0cefa0 0x7fd48c13ac80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd4840b0040 tx=0x7fd4840a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:11.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd488c00640 1 -- 172.21.15.50:0/1529387489 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4840b2040 con 0x7fd48c0cefa0 2024-09-21T06:02:11.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.204+0000 7fd488c00640 1 -- 172.21.15.50:0/1529387489 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4840044d0 con 0x7fd48c0cefa0 2024-09-21T06:02:11.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.208+0000 7fd492e00640 1 -- 172.21.15.50:0/1529387489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c143510 con 0x7fd48c0cefa0 2024-09-21T06:02:11.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.208+0000 7fd488c00640 1 -- 172.21.15.50:0/1529387489 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484003840 con 0x7fd48c0cefa0 2024-09-21T06:02:11.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.208+0000 7fd492e00640 1 -- 172.21.15.50:0/1529387489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c147970 con 0x7fd48c0cefa0 2024-09-21T06:02:11.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.208+0000 7fd492e00640 1 -- 172.21.15.50:0/1529387489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c074db0 con 0x7fd48c0cefa0 2024-09-21T06:02:11.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.208+0000 7fd488c00640 1 -- 172.21.15.50:0/1529387489 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fd4840bf060 con 0x7fd48c0cefa0 2024-09-21T06:02:11.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.208+0000 7fd488c00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd468073b30 0x7fd468076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:11.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.208+0000 7fd488c00640 1 -- 172.21.15.50:0/1529387489 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5737+0+0 (secure 0 0 0) 0x7fd48412c680 con 0x7fd48c0cefa0 2024-09-21T06:02:11.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.212+0000 7fd48be00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd468073b30 0x7fd468076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:11.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.212+0000 7fd48be00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd468073b30 0x7fd468076040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009f20 tx=0x7fd47c0094f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:11.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.216+0000 7fd488c00640 1 -- 172.21.15.50:0/1529387489 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd4840164b0 con 0x7fd48c0cefa0 2024-09-21T06:02:11.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:11.428+0000 7fd492e00640 1 -- 172.21.15.50:0/1529387489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd48c143e40 con 0x7fd48c0cefa0 2024-09-21T06:02:12.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd488c00640 1 -- 172.21.15.50:0/1529387489 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7fd4840f90f0 con 0x7fd48c0cefa0 2024-09-21T06:02:12.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd492e00640 1 -- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd468073b30 msgr2=0x7fd468076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:12.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd492e00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd468073b30 0x7fd468076040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009f20 tx=0x7fd47c0094f0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd492e00640 1 -- 172.21.15.50:0/1529387489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd48c0cefa0 msgr2=0x7fd48c13ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:12.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd492e00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd48c0cefa0 0x7fd48c13ac80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd4840b0040 tx=0x7fd4840a57b0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd492e00640 1 -- 172.21.15.50:0/1529387489 shutdown_connections 2024-09-21T06:02:12.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd492e00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd468073b30 0x7fd468076040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd492e00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd48c0cefa0 0x7fd48c13ac80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd492e00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd48c0b3340 0x7fd48c13a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd492e00640 1 --2- 172.21.15.50:0/1529387489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd48c0749a0 0x7fd48c145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd492e00640 1 -- 172.21.15.50:0/1529387489 >> 172.21.15.50:0/1529387489 conn(0x7fd48c0b2040 msgr2=0x7fd48c15a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:12.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd492e00640 1 -- 172.21.15.50:0/1529387489 shutdown_connections 2024-09-21T06:02:12.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.272+0000 7fd492e00640 1 -- 172.21.15.50:0/1529387489 wait complete. 2024-09-21T06:02:12.301 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:02:12.301 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:02:12.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.464+0000 7f4516400640 1 -- 172.21.15.50:0/774760482 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45100b4de0 msgr2=0x7f45100b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:12.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.464+0000 7f4516400640 1 --2- 172.21.15.50:0/774760482 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45100b4de0 0x7f45100b51f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4508009c80 tx=0x7f450801cd70 comp rx=0 tx=0).stop 2024-09-21T06:02:12.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.464+0000 7f4516400640 1 -- 172.21.15.50:0/774760482 shutdown_connections 2024-09-21T06:02:12.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.464+0000 7f4516400640 1 --2- 172.21.15.50:0/774760482 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45100b4de0 0x7f45100b51f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.464+0000 7f4516400640 1 --2- 172.21.15.50:0/774760482 >> v2:172.21.15.88:3300/0 conn(0x7f45100b5c70 0x7f451015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.464+0000 7f4516400640 1 -- 172.21.15.50:0/774760482 >> 172.21.15.50:0/774760482 conn(0x7f45100b2040 msgr2=0x7f45100b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:12.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.464+0000 7f4516400640 1 -- 172.21.15.50:0/774760482 shutdown_connections 2024-09-21T06:02:12.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.464+0000 7f4516400640 1 -- 172.21.15.50:0/774760482 wait complete. 2024-09-21T06:02:12.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f4516400640 1 Processor -- start 2024-09-21T06:02:12.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f4516400640 1 -- start start 2024-09-21T06:02:12.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f4516400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45100b4de0 0x7f4510149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f4516400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45100b5c70 0x7f451015ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f4516400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45101527b0 0x7f451014b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f4516400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4510168a00 con 0x7f45100b4de0 2024-09-21T06:02:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f4516400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4510168880 con 0x7f45101527b0 2024-09-21T06:02:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f4516400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4510168b80 con 0x7f45100b5c70 2024-09-21T06:02:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f450f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45100b4de0 0x7f4510149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f450f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45100b4de0 0x7f4510149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48160/0 (socket says 172.21.15.50:48160) 2024-09-21T06:02:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f450f400640 1 -- 172.21.15.50:0/149327469 learned_addr learned my addr 172.21.15.50:0/149327469 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f450fe00640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45101527b0 0x7f451014b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.468+0000 7f450ea00640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45100b5c70 0x7f451015ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f450f400640 1 -- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45100b5c70 msgr2=0x7f451015ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f450f400640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45100b5c70 0x7f451015ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f450f400640 1 -- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45101527b0 msgr2=0x7f451014b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f450f400640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45101527b0 0x7f451014b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f450f400640 1 -- 172.21.15.50:0/149327469 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45101f6a00 con 0x7f45100b4de0 2024-09-21T06:02:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f450f400640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45100b4de0 0x7f4510149be0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f450000e9a0 tx=0x7f450000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f4507400640 1 -- 172.21.15.50:0/149327469 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450000cd80 con 0x7f45100b4de0 2024-09-21T06:02:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f4516400640 1 -- 172.21.15.50:0/149327469 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45101f6cc0 con 0x7f45100b4de0 2024-09-21T06:02:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f4507400640 1 -- 172.21.15.50:0/149327469 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4500004020 con 0x7f45100b4de0 2024-09-21T06:02:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f4507400640 1 -- 172.21.15.50:0/149327469 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500004450 con 0x7f45100b4de0 2024-09-21T06:02:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f4516400640 1 -- 172.21.15.50:0/149327469 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45101f7280 con 0x7f45100b4de0 2024-09-21T06:02:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f4516400640 1 -- 172.21.15.50:0/149327469 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45100b51f0 con 0x7f45100b4de0 2024-09-21T06:02:12.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.472+0000 7f4507400640 1 -- 172.21.15.50:0/149327469 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f4500003600 con 0x7f45100b4de0 2024-09-21T06:02:12.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.476+0000 7f4507400640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44f0073b30 0x7f44f0076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.476+0000 7f4507400640 1 -- 172.21.15.50:0/149327469 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f45000933f0 con 0x7f45100b4de0 2024-09-21T06:02:12.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.476+0000 7f450ea00640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44f0073b30 0x7f44f0076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.476+0000 7f450ea00640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44f0073b30 0x7f44f0076040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f44fc007fd0 tx=0x7f44fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:12.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.480+0000 7f4507400640 1 -- 172.21.15.50:0/149327469 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f450005f790 con 0x7f45100b4de0 2024-09-21T06:02:12.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:12.696+0000 7f4516400640 1 -- 172.21.15.50:0/149327469 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f451014a210 con 0x7f45100b4de0 2024-09-21T06:02:13.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.276+0000 7f4507400640 1 -- 172.21.15.50:0/149327469 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f45000641c0 con 0x7f45100b4de0 2024-09-21T06:02:13.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.280+0000 7f4516400640 1 -- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44f0073b30 msgr2=0x7f44f0076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:13.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.280+0000 7f4516400640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44f0073b30 0x7f44f0076040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f44fc007fd0 tx=0x7f44fc009040 comp rx=0 tx=0).stop 2024-09-21T06:02:13.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.280+0000 7f4516400640 1 -- 172.21.15.50:0/149327469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45100b4de0 msgr2=0x7f4510149be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:13.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.280+0000 7f4516400640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45100b4de0 0x7f4510149be0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f450000e9a0 tx=0x7f450000ee60 comp rx=0 tx=0).stop 2024-09-21T06:02:13.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.280+0000 7f4516400640 1 -- 172.21.15.50:0/149327469 shutdown_connections 2024-09-21T06:02:13.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.280+0000 7f4516400640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45101527b0 0x7f451014b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.280+0000 7f4516400640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45100b5c70 0x7f451015ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.280+0000 7f4516400640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44f0073b30 0x7f44f0076040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.280+0000 7f4516400640 1 --2- 172.21.15.50:0/149327469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45100b4de0 0x7f4510149be0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.280+0000 7f4516400640 1 -- 172.21.15.50:0/149327469 >> 172.21.15.50:0/149327469 conn(0x7f45100b2040 msgr2=0x7f4510154e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.280+0000 7f4516400640 1 -- 172.21.15.50:0/149327469 shutdown_connections 2024-09-21T06:02:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.280+0000 7f4516400640 1 -- 172.21.15.50:0/149327469 wait complete. 2024-09-21T06:02:13.306 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:02:13.306 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:02:13.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.468+0000 7fdbeca00640 1 -- 172.21.15.50:0/3186606263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbe80b5a80 msgr2=0x7fdbe8074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:13.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.468+0000 7fdbeca00640 1 --2- 172.21.15.50:0/3186606263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbe80b5a80 0x7fdbe8074410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009840 tx=0x7fdbd001c6b0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.468+0000 7fdbeca00640 1 -- 172.21.15.50:0/3186606263 shutdown_connections 2024-09-21T06:02:13.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.468+0000 7fdbeca00640 1 --2- 172.21.15.50:0/3186606263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbe80b5a80 0x7fdbe8074410 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.468+0000 7fdbeca00640 1 -- 172.21.15.50:0/3186606263 >> 172.21.15.50:0/3186606263 conn(0x7fdbe80708c0 msgr2=0x7fdbe8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:13.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.468+0000 7fdbeca00640 1 -- 172.21.15.50:0/3186606263 shutdown_connections 2024-09-21T06:02:13.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.468+0000 7fdbeca00640 1 -- 172.21.15.50:0/3186606263 wait complete. 2024-09-21T06:02:13.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.468+0000 7fdbeca00640 1 Processor -- start 2024-09-21T06:02:13.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.468+0000 7fdbeca00640 1 -- start start 2024-09-21T06:02:13.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbeca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdbe8074950 0x7fdbe814e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:13.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbeca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe80b5a80 0x7fdbe8146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:13.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbeca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbe8157030 0x7fdbe815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:13.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbeca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe816b640 con 0x7fdbe80b5a80 2024-09-21T06:02:13.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbeca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe816b4c0 con 0x7fdbe8157030 2024-09-21T06:02:13.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbeca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe816b7c0 con 0x7fdbe8074950 2024-09-21T06:02:13.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbdfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe80b5a80 0x7fdbe8146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:13.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbdfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe80b5a80 0x7fdbe8146c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48162/0 (socket says 172.21.15.50:48162) 2024-09-21T06:02:13.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbdfe00640 1 -- 172.21.15.50:0/3712453683 learned_addr learned my addr 172.21.15.50:0/3712453683 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:13.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbe6800640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbe8157030 0x7fdbe815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:13.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbe5e00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdbe8074950 0x7fdbe814e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:13.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbdfe00640 1 -- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdbe8074950 msgr2=0x7fdbe814e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:13.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbdfe00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdbe8074950 0x7fdbe814e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbdfe00640 1 -- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbe8157030 msgr2=0x7fdbe815ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:13.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbdfe00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbe8157030 0x7fdbe815ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbdfe00640 1 -- 172.21.15.50:0/3712453683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe8150880 con 0x7fdbe80b5a80 2024-09-21T06:02:13.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbe5e00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdbe8074950 0x7fdbe814e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:13.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbdfe00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe80b5a80 0x7fdbe8146c70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdbd001ad60 tx=0x7fdbd00a96a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:13.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbde000640 1 -- 172.21.15.50:0/3712453683 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd00ad070 con 0x7fdbe80b5a80 2024-09-21T06:02:13.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbde000640 1 -- 172.21.15.50:0/3712453683 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbd00a9c60 con 0x7fdbe80b5a80 2024-09-21T06:02:13.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbeca00640 1 -- 172.21.15.50:0/3712453683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe8156430 con 0x7fdbe80b5a80 2024-09-21T06:02:13.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbde000640 1 -- 172.21.15.50:0/3712453683 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0003860 con 0x7fdbe80b5a80 2024-09-21T06:02:13.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbeca00640 1 -- 172.21.15.50:0/3712453683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe81569f0 con 0x7fdbe80b5a80 2024-09-21T06:02:13.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.472+0000 7fdbeca00640 1 -- 172.21.15.50:0/3712453683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe8074410 con 0x7fdbe80b5a80 2024-09-21T06:02:13.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.476+0000 7fdbde000640 1 -- 172.21.15.50:0/3712453683 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fdbd00b0070 con 0x7fdbe80b5a80 2024-09-21T06:02:13.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.476+0000 7fdbde000640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdbc4073b00 0x7fdbc4076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:13.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.476+0000 7fdbde000640 1 -- 172.21.15.50:0/3712453683 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fdbd012cb70 con 0x7fdbe80b5a80 2024-09-21T06:02:13.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.476+0000 7fdbe5e00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdbc4073b00 0x7fdbc4076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:13.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.476+0000 7fdbe5e00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdbc4073b00 0x7fdbc4076010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8001130 tx=0x7fdbd8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:13.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.480+0000 7fdbde000640 1 -- 172.21.15.50:0/3712453683 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdbd00b7070 con 0x7fdbe80b5a80 2024-09-21T06:02:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.696+0000 7fdbeca00640 1 -- 172.21.15.50:0/3712453683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbe8150610 con 0x7fdbe80b5a80 2024-09-21T06:02:13.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.696+0000 7fdbde000640 1 -- 172.21.15.50:0/3712453683 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5217 (secure 0 0 0) 0x7fdbd0016330 con 0x7fdbe80b5a80 2024-09-21T06:02:13.698 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:13.698 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":36,"btime":"2024-09-21T06:02:13:274179+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/288562749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":288562749},{"type":"v1","addr":"172.21.15.50:6834","nonce":288562749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5686,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2920049831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2920049831},{"type":"v1","addr":"172.21.15.88:6836","nonce":2920049831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1072517376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1072517376},{"type":"v1","addr":"172.21.15.88:6837","nonce":1072517376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:02:09.689390+0000","modified":"2024-09-21T06:02:12.699140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/451843324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":451843324},{"type":"v1","addr":"172.21.15.50:6835","nonce":451843324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5695,"qdb_cluster":[5695]},"id":4}]} 2024-09-21T06:02:13.698 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 36 2024-09-21T06:02:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.700+0000 7fdbeca00640 1 -- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdbc4073b00 msgr2=0x7fdbc4076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.700+0000 7fdbeca00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdbc4073b00 0x7fdbc4076010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8001130 tx=0x7fdbd8009040 comp rx=0 tx=0).stop 2024-09-21T06:02:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.700+0000 7fdbeca00640 1 -- 172.21.15.50:0/3712453683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe80b5a80 msgr2=0x7fdbe8146c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.700+0000 7fdbeca00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe80b5a80 0x7fdbe8146c70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdbd001ad60 tx=0x7fdbd00a96a0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.700+0000 7fdbeca00640 1 -- 172.21.15.50:0/3712453683 shutdown_connections 2024-09-21T06:02:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.700+0000 7fdbeca00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbe8157030 0x7fdbe815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.700+0000 7fdbeca00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdbc4073b00 0x7fdbc4076010 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.700+0000 7fdbeca00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe80b5a80 0x7fdbe8146c70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.700+0000 7fdbeca00640 1 --2- 172.21.15.50:0/3712453683 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdbe8074950 0x7fdbe814e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.700+0000 7fdbeca00640 1 -- 172.21.15.50:0/3712453683 >> 172.21.15.50:0/3712453683 conn(0x7fdbe80708c0 msgr2=0x7fdbe815e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.700+0000 7fdbeca00640 1 -- 172.21.15.50:0/3712453683 shutdown_connections 2024-09-21T06:02:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.700+0000 7fdbeca00640 1 -- 172.21.15.50:0/3712453683 wait complete. 2024-09-21T06:02:13.721 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:02:13.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 -- 172.21.15.50:0/3521399238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4157030 msgr2=0x7f5ba40b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:13.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 --2- 172.21.15.50:0/3521399238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4157030 0x7f5ba40b5a80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009840 tx=0x7f5b9401c820 comp rx=0 tx=0).stop 2024-09-21T06:02:13.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 -- 172.21.15.50:0/3521399238 shutdown_connections 2024-09-21T06:02:13.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 --2- 172.21.15.50:0/3521399238 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5ba4160d50 0x7f5ba4074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 --2- 172.21.15.50:0/3521399238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4157030 0x7f5ba40b5a80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 -- 172.21.15.50:0/3521399238 >> 172.21.15.50:0/3521399238 conn(0x7f5ba40708c0 msgr2=0x7f5ba4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:13.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 -- 172.21.15.50:0/3521399238 shutdown_connections 2024-09-21T06:02:13.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 -- 172.21.15.50:0/3521399238 wait complete. 2024-09-21T06:02:13.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 Processor -- start 2024-09-21T06:02:13.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 -- start start 2024-09-21T06:02:13.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5ba4157030 0x7f5ba41fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:13.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4160d50 0x7f5ba4144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:13.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.880+0000 7f5ba8a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4161160 0x7f5ba41f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:13.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba8a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba416a820 con 0x7f5ba4161160 2024-09-21T06:02:13.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba8a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba416a6a0 con 0x7f5ba4160d50 2024-09-21T06:02:13.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba8a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba416a9a0 con 0x7f5ba4157030 2024-09-21T06:02:13.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4161160 0x7f5ba41f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:13.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4161160 0x7f5ba41f7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48176/0 (socket says 172.21.15.50:48176) 2024-09-21T06:02:13.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba2800640 1 -- 172.21.15.50:0/1014159167 learned_addr learned my addr 172.21.15.50:0/1014159167 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:13.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5b9be00640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4160d50 0x7f5ba4144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:13.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba1e00640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5ba4157030 0x7f5ba41fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba2800640 1 -- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5ba4157030 msgr2=0x7f5ba41fb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba2800640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5ba4157030 0x7f5ba41fb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba2800640 1 -- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4160d50 msgr2=0x7f5ba4144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba2800640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4160d50 0x7f5ba4144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba2800640 1 -- 172.21.15.50:0/1014159167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba41f70a0 con 0x7f5ba4161160 2024-09-21T06:02:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba1e00640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5ba4157030 0x7f5ba41fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba2800640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4161160 0x7f5ba41f7b60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00c730 tx=0x7f5b9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5b9a000640 1 -- 172.21.15.50:0/1014159167 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c003b80 con 0x7f5ba4161160 2024-09-21T06:02:13.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5b9a000640 1 -- 172.21.15.50:0/1014159167 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b9c003d40 con 0x7f5ba4161160 2024-09-21T06:02:13.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba8a00640 1 -- 172.21.15.50:0/1014159167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba414f860 con 0x7f5ba4161160 2024-09-21T06:02:13.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5b9a000640 1 -- 172.21.15.50:0/1014159167 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0052d0 con 0x7f5ba4161160 2024-09-21T06:02:13.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba8a00640 1 -- 172.21.15.50:0/1014159167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba41f7360 con 0x7f5ba4161160 2024-09-21T06:02:13.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5ba8a00640 1 -- 172.21.15.50:0/1014159167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba40d2260 con 0x7f5ba4161160 2024-09-21T06:02:13.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.884+0000 7f5b9a000640 1 -- 172.21.15.50:0/1014159167 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f5b9c005680 con 0x7f5ba4161160 2024-09-21T06:02:13.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.888+0000 7f5b9a000640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b80073b00 0x7f5b80076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:13.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.888+0000 7f5b9a000640 1 -- 172.21.15.50:0/1014159167 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f5b9c092a60 con 0x7f5ba4161160 2024-09-21T06:02:13.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.888+0000 7f5ba1e00640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b80073b00 0x7f5b80076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:13.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.888+0000 7f5ba1e00640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b80073b00 0x7f5b80076010 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5b940af000 tx=0x7f5b9401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:13.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:13.892+0000 7f5b9a000640 1 -- 172.21.15.50:0/1014159167 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5b9c05f560 con 0x7f5ba4161160 2024-09-21T06:02:14.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.068+0000 7f5ba8a00640 1 -- 172.21.15.50:0/1014159167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ba40b1b30 con 0x7f5ba4161160 2024-09-21T06:02:14.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5b9a000640 1 -- 172.21.15.50:0/1014159167 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16720 (secure 0 0 0) 0x7f5b9c093e20 con 0x7f5ba4161160 2024-09-21T06:02:14.073 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:14.073 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":59,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:02:11.246140+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:02:08.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T06:02:08.630388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:02:14.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5ba8a00640 1 -- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b80073b00 msgr2=0x7f5b80076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5ba8a00640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b80073b00 0x7f5b80076010 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5b940af000 tx=0x7f5b9401b060 comp rx=0 tx=0).stop 2024-09-21T06:02:14.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5ba8a00640 1 -- 172.21.15.50:0/1014159167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4161160 msgr2=0x7f5ba41f7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5ba8a00640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4161160 0x7f5ba41f7b60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00c730 tx=0x7f5b9c00cbf0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5ba8a00640 1 -- 172.21.15.50:0/1014159167 shutdown_connections 2024-09-21T06:02:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5ba8a00640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b80073b00 0x7f5b80076010 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5ba8a00640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4161160 0x7f5ba41f7b60 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5ba8a00640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4160d50 0x7f5ba4144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5ba8a00640 1 --2- 172.21.15.50:0/1014159167 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5ba4157030 0x7f5ba41fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5ba8a00640 1 -- 172.21.15.50:0/1014159167 >> 172.21.15.50:0/1014159167 conn(0x7f5ba40708c0 msgr2=0x7f5ba40b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5ba8a00640 1 -- 172.21.15.50:0/1014159167 shutdown_connections 2024-09-21T06:02:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.072+0000 7f5ba8a00640 1 -- 172.21.15.50:0/1014159167 wait complete. 2024-09-21T06:02:14.096 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:02:14.096 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:02:14.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.252+0000 7ff6b6a00640 1 -- 172.21.15.50:0/1653633413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b00b4de0 msgr2=0x7ff6b00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.252+0000 7ff6b6a00640 1 --2- 172.21.15.50:0/1653633413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b00b4de0 0x7ff6b00b51f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8009bc0 tx=0x7ff6a801cae0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.252+0000 7ff6b6a00640 1 -- 172.21.15.50:0/1653633413 shutdown_connections 2024-09-21T06:02:14.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.252+0000 7ff6b6a00640 1 --2- 172.21.15.50:0/1653633413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b00b4de0 0x7ff6b00b51f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.252+0000 7ff6b6a00640 1 --2- 172.21.15.50:0/1653633413 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6b01527b0 0x7ff6b015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.252+0000 7ff6b6a00640 1 -- 172.21.15.50:0/1653633413 >> 172.21.15.50:0/1653633413 conn(0x7ff6b00b2040 msgr2=0x7ff6b00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:14.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.252+0000 7ff6b6a00640 1 -- 172.21.15.50:0/1653633413 shutdown_connections 2024-09-21T06:02:14.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.252+0000 7ff6b6a00640 1 -- 172.21.15.50:0/1653633413 wait complete. 2024-09-21T06:02:14.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6b6a00640 1 Processor -- start 2024-09-21T06:02:14.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6b6a00640 1 -- start start 2024-09-21T06:02:14.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6b6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6b00b4de0 0x7ff6b01498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:14.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6b6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b00b5c70 0x7ff6b0154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:14.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6b6a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6b01527b0 0x7ff6b015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:14.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6b6a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b0168ab0 con 0x7ff6b00b4de0 2024-09-21T06:02:14.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6b6a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b0168930 con 0x7ff6b00b5c70 2024-09-21T06:02:14.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6b6a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b0168c30 con 0x7ff6b01527b0 2024-09-21T06:02:14.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6afe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6b00b4de0 0x7ff6b01498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:14.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6afe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6b00b4de0 0x7ff6b01498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48188/0 (socket says 172.21.15.50:48188) 2024-09-21T06:02:14.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6afe00640 1 -- 172.21.15.50:0/571841033 learned_addr learned my addr 172.21.15.50:0/571841033 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:14.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6a7e00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b00b5c70 0x7ff6b0154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:14.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6b4a00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6b01527b0 0x7ff6b015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:14.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6afe00640 1 -- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6b01527b0 msgr2=0x7ff6b015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6afe00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6b01527b0 0x7ff6b015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6afe00640 1 -- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b00b5c70 msgr2=0x7ff6b0154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6afe00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b00b5c70 0x7ff6b0154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6afe00640 1 -- 172.21.15.50:0/571841033 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b0149df0 con 0x7ff6b00b4de0 2024-09-21T06:02:14.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6afe00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6b00b4de0 0x7ff6b01498b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000c3c0 tx=0x7ff6a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:14.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6acc00640 1 -- 172.21.15.50:0/571841033 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0017070 con 0x7ff6b00b4de0 2024-09-21T06:02:14.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6acc00640 1 -- 172.21.15.50:0/571841033 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6a00053c0 con 0x7ff6b00b4de0 2024-09-21T06:02:14.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6acc00640 1 -- 172.21.15.50:0/571841033 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0003860 con 0x7ff6b00b4de0 2024-09-21T06:02:14.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6b6a00640 1 -- 172.21.15.50:0/571841033 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b01fbda0 con 0x7ff6b00b4de0 2024-09-21T06:02:14.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.256+0000 7ff6b6a00640 1 -- 172.21.15.50:0/571841033 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b01fc360 con 0x7ff6b00b4de0 2024-09-21T06:02:14.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.260+0000 7ff6b6a00640 1 -- 172.21.15.50:0/571841033 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6b00b51f0 con 0x7ff6b00b4de0 2024-09-21T06:02:14.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.260+0000 7ff6acc00640 1 -- 172.21.15.50:0/571841033 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7ff6a001a070 con 0x7ff6b00b4de0 2024-09-21T06:02:14.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.260+0000 7ff6acc00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff68c073b30 0x7ff68c076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:14.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.260+0000 7ff6acc00640 1 -- 172.21.15.50:0/571841033 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7ff6a00946c0 con 0x7ff6b00b4de0 2024-09-21T06:02:14.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.260+0000 7ff6a7e00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff68c073b30 0x7ff68c076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:14.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.260+0000 7ff6a7e00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff68c073b30 0x7ff68c076040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0153ea0 tx=0x7ff698009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.264+0000 7ff6acc00640 1 -- 172.21.15.50:0/571841033 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff6b00b5730 con 0x7ff6b00b4de0 2024-09-21T06:02:14.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.444+0000 7ff6b6a00640 1 -- 172.21.15.50:0/571841033 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6b00b2910 con 0x7ff68c073b30 2024-09-21T06:02:14.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.448+0000 7ff6acc00640 1 -- 172.21.15.50:0/571841033 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241098 (secure 0 0 0) 0x7ff6b00b2910 con 0x7ff68c073b30 2024-09-21T06:02:14.452 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:14.454 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:02:14.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.452+0000 7ff6b6a00640 1 -- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff68c073b30 msgr2=0x7ff68c076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.452+0000 7ff6b6a00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff68c073b30 0x7ff68c076040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0153ea0 tx=0x7ff698009260 comp rx=0 tx=0).stop 2024-09-21T06:02:14.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.452+0000 7ff6b6a00640 1 -- 172.21.15.50:0/571841033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6b00b4de0 msgr2=0x7ff6b01498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.452+0000 7ff6b6a00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6b00b4de0 0x7ff6b01498b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000c3c0 tx=0x7ff6a000c880 comp rx=0 tx=0).stop 2024-09-21T06:02:14.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.452+0000 7ff6b6a00640 1 -- 172.21.15.50:0/571841033 shutdown_connections 2024-09-21T06:02:14.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.452+0000 7ff6b6a00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6b01527b0 0x7ff6b015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.452+0000 7ff6b6a00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b00b5c70 0x7ff6b0154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.452+0000 7ff6b6a00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff68c073b30 0x7ff68c076040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.452+0000 7ff6b6a00640 1 --2- 172.21.15.50:0/571841033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6b00b4de0 0x7ff6b01498b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.452+0000 7ff6b6a00640 1 -- 172.21.15.50:0/571841033 >> 172.21.15.50:0/571841033 conn(0x7ff6b00b2040 msgr2=0x7ff6b0154dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:14.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.452+0000 7ff6b6a00640 1 -- 172.21.15.50:0/571841033 shutdown_connections 2024-09-21T06:02:14.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.452+0000 7ff6b6a00640 1 -- 172.21.15.50:0/571841033 wait complete. 2024-09-21T06:02:14.474 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-21T06:02:12.983730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":358568,"kb_used_data":6312,"kb_used_omap":12,"kb_used_meta":352051,"kb_avail":754616152,"statfs":{"total":773094113280,"available":772726939648,"internally_reserved":0,"allocated":6463488,"data_stored":3330662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":360500784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.746307"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790900+0000","last_change":"2024-09-21T06:02:10.262539+0000","last_active":"2024-09-21T06:02:11.790900+0000","last_peered":"2024-09-21T06:02:11.790900+0000","last_clean":"2024-09-21T06:02:11.790900+0000","last_became_active":"2024-09-21T06:02:10.261557+0000","last_became_peered":"2024-09-21T06:02:10.261557+0000","last_unstale":"2024-09-21T06:02:11.790900+0000","last_undegraded":"2024-09-21T06:02:11.790900+0000","last_fullsized":"2024-09-21T06:02:11.790900+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:46.770394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790427+0000","last_change":"2024-09-21T06:02:10.280853+0000","last_active":"2024-09-21T06:02:11.790427+0000","last_peered":"2024-09-21T06:02:11.790427+0000","last_clean":"2024-09-21T06:02:11.790427+0000","last_became_active":"2024-09-21T06:02:10.280106+0000","last_became_peered":"2024-09-21T06:02:10.280106+0000","last_unstale":"2024-09-21T06:02:11.790427+0000","last_undegraded":"2024-09-21T06:02:11.790427+0000","last_fullsized":"2024-09-21T06:02:11.790427+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:54.589979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.242772+0000","last_change":"2024-09-21T06:02:08.242772+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.242772+0000","last_undegraded":"2024-09-21T06:02:08.242772+0000","last_fullsized":"2024-09-21T06:02:08.242772+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790843+0000","last_change":"2024-09-21T06:02:10.271022+0000","last_active":"2024-09-21T06:02:11.790843+0000","last_peered":"2024-09-21T06:02:11.790843+0000","last_clean":"2024-09-21T06:02:11.790843+0000","last_became_active":"2024-09-21T06:02:10.270717+0000","last_became_peered":"2024-09-21T06:02:10.270717+0000","last_unstale":"2024-09-21T06:02:11.790843+0000","last_undegraded":"2024-09-21T06:02:11.790843+0000","last_fullsized":"2024-09-21T06:02:11.790843+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:25.972520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240634+0000","last_change":"2024-09-21T06:02:08.240634+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240634+0000","last_undegraded":"2024-09-21T06:02:08.240634+0000","last_fullsized":"2024-09-21T06:02:08.240634+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.242585+0000","last_change":"2024-09-21T06:02:08.242585+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.242585+0000","last_undegraded":"2024-09-21T06:02:08.242585+0000","last_fullsized":"2024-09-21T06:02:08.242585+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.242880+0000","last_change":"2024-09-21T06:02:08.242880+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.242880+0000","last_undegraded":"2024-09-21T06:02:08.242880+0000","last_fullsized":"2024-09-21T06:02:08.242880+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240546+0000","last_change":"2024-09-21T06:02:08.240546+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240546+0000","last_undegraded":"2024-09-21T06:02:08.240546+0000","last_fullsized":"2024-09-21T06:02:08.240546+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.242604+0000","last_change":"2024-09-21T06:02:08.242604+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.242604+0000","last_undegraded":"2024-09-21T06:02:08.242604+0000","last_fullsized":"2024-09-21T06:02:08.242604+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.243036+0000","last_change":"2024-09-21T06:02:08.243036+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.243036+0000","last_undegraded":"2024-09-21T06:02:08.243036+0000","last_fullsized":"2024-09-21T06:02:08.243036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240351+0000","last_change":"2024-09-21T06:02:08.240351+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240351+0000","last_undegraded":"2024-09-21T06:02:08.240351+0000","last_fullsized":"2024-09-21T06:02:08.240351+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.246856+0000","last_change":"2024-09-21T06:02:08.246856+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.246856+0000","last_undegraded":"2024-09-21T06:02:08.246856+0000","last_fullsized":"2024-09-21T06:02:08.246856+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.239994+0000","last_change":"2024-09-21T06:02:08.239994+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.239994+0000","last_undegraded":"2024-09-21T06:02:08.239994+0000","last_fullsized":"2024-09-21T06:02:08.239994+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790136+0000","last_change":"2024-09-21T06:02:10.272781+0000","last_active":"2024-09-21T06:02:11.790136+0000","last_peered":"2024-09-21T06:02:11.790136+0000","last_clean":"2024-09-21T06:02:11.790136+0000","last_became_active":"2024-09-21T06:02:10.272474+0000","last_became_peered":"2024-09-21T06:02:10.272474+0000","last_unstale":"2024-09-21T06:02:11.790136+0000","last_undegraded":"2024-09-21T06:02:11.790136+0000","last_fullsized":"2024-09-21T06:02:11.790136+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:19.569443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.246113+0000","last_change":"2024-09-21T06:02:08.246113+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.246113+0000","last_undegraded":"2024-09-21T06:02:08.246113+0000","last_fullsized":"2024-09-21T06:02:08.246113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240352+0000","last_change":"2024-09-21T06:02:08.240352+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240352+0000","last_undegraded":"2024-09-21T06:02:08.240352+0000","last_fullsized":"2024-09-21T06:02:08.240352+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240647+0000","last_change":"2024-09-21T06:02:08.240647+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240647+0000","last_undegraded":"2024-09-21T06:02:08.240647+0000","last_fullsized":"2024-09-21T06:02:08.240647+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.246817+0000","last_change":"2024-09-21T06:02:08.246817+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.246817+0000","last_undegraded":"2024-09-21T06:02:08.246817+0000","last_fullsized":"2024-09-21T06:02:08.246817+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.241370+0000","last_change":"2024-09-21T06:02:08.241370+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.241370+0000","last_undegraded":"2024-09-21T06:02:08.241370+0000","last_fullsized":"2024-09-21T06:02:08.241370+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.244769+0000","last_change":"2024-09-21T06:02:08.244769+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.244769+0000","last_undegraded":"2024-09-21T06:02:08.244769+0000","last_fullsized":"2024-09-21T06:02:08.244769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240228+0000","last_change":"2024-09-21T06:02:08.240228+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240228+0000","last_undegraded":"2024-09-21T06:02:08.240228+0000","last_fullsized":"2024-09-21T06:02:08.240228+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.243659+0000","last_change":"2024-09-21T06:02:08.243659+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.243659+0000","last_undegraded":"2024-09-21T06:02:08.243659+0000","last_fullsized":"2024-09-21T06:02:08.243659+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.241351+0000","last_change":"2024-09-21T06:02:08.241351+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.241351+0000","last_undegraded":"2024-09-21T06:02:08.241351+0000","last_fullsized":"2024-09-21T06:02:08.241351+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.243155+0000","last_change":"2024-09-21T06:02:08.243155+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.243155+0000","last_undegraded":"2024-09-21T06:02:08.243155+0000","last_fullsized":"2024-09-21T06:02:08.243155+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.239998+0000","last_change":"2024-09-21T06:02:08.239998+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.239998+0000","last_undegraded":"2024-09-21T06:02:08.239998+0000","last_fullsized":"2024-09-21T06:02:08.239998+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789710+0000","last_change":"2024-09-21T06:02:09.271164+0000","last_active":"2024-09-21T06:02:11.789710+0000","last_peered":"2024-09-21T06:02:11.789710+0000","last_clean":"2024-09-21T06:02:11.789710+0000","last_became_active":"2024-09-21T06:02:09.270848+0000","last_became_peered":"2024-09-21T06:02:09.270848+0000","last_unstale":"2024-09-21T06:02:11.789710+0000","last_undegraded":"2024-09-21T06:02:11.789710+0000","last_fullsized":"2024-09-21T06:02:11.789710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:58.592636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240562+0000","last_change":"2024-09-21T06:02:08.240562+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240562+0000","last_undegraded":"2024-09-21T06:02:08.240562+0000","last_fullsized":"2024-09-21T06:02:08.240562+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790736+0000","last_change":"2024-09-21T06:02:09.275643+0000","last_active":"2024-09-21T06:02:11.790736+0000","last_peered":"2024-09-21T06:02:11.790736+0000","last_clean":"2024-09-21T06:02:11.790736+0000","last_became_active":"2024-09-21T06:02:09.275319+0000","last_became_peered":"2024-09-21T06:02:09.275319+0000","last_unstale":"2024-09-21T06:02:11.790736+0000","last_undegraded":"2024-09-21T06:02:11.790736+0000","last_fullsized":"2024-09-21T06:02:11.790736+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:46.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.242652+0000","last_change":"2024-09-21T06:02:08.242652+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.242652+0000","last_undegraded":"2024-09-21T06:02:08.242652+0000","last_fullsized":"2024-09-21T06:02:08.242652+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790403+0000","last_change":"2024-09-21T06:02:10.262950+0000","last_active":"2024-09-21T06:02:11.790403+0000","last_peered":"2024-09-21T06:02:11.790403+0000","last_clean":"2024-09-21T06:02:11.790403+0000","last_became_active":"2024-09-21T06:02:10.262636+0000","last_became_peered":"2024-09-21T06:02:10.262636+0000","last_unstale":"2024-09-21T06:02:11.790403+0000","last_undegraded":"2024-09-21T06:02:11.790403+0000","last_fullsized":"2024-09-21T06:02:11.790403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:32.505486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.246801+0000","last_change":"2024-09-21T06:02:08.246801+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.246801+0000","last_undegraded":"2024-09-21T06:02:08.246801+0000","last_fullsized":"2024-09-21T06:02:08.246801+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.241785+0000","last_change":"2024-09-21T06:02:08.241785+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.241785+0000","last_undegraded":"2024-09-21T06:02:08.241785+0000","last_fullsized":"2024-09-21T06:02:08.241785+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790058+0000","last_change":"2024-09-21T06:02:09.263348+0000","last_active":"2024-09-21T06:02:11.790058+0000","last_peered":"2024-09-21T06:02:11.790058+0000","last_clean":"2024-09-21T06:02:11.790058+0000","last_became_active":"2024-09-21T06:02:09.262942+0000","last_became_peered":"2024-09-21T06:02:09.262942+0000","last_unstale":"2024-09-21T06:02:11.790058+0000","last_undegraded":"2024-09-21T06:02:11.790058+0000","last_fullsized":"2024-09-21T06:02:11.790058+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:13.084051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240981+0000","last_change":"2024-09-21T06:02:08.240981+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240981+0000","last_undegraded":"2024-09-21T06:02:08.240981+0000","last_fullsized":"2024-09-21T06:02:08.240981+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790834+0000","last_change":"2024-09-21T06:02:10.261037+0000","last_active":"2024-09-21T06:02:11.790834+0000","last_peered":"2024-09-21T06:02:11.790834+0000","last_clean":"2024-09-21T06:02:11.790834+0000","last_became_active":"2024-09-21T06:02:10.260729+0000","last_became_peered":"2024-09-21T06:02:10.260729+0000","last_unstale":"2024-09-21T06:02:11.790834+0000","last_undegraded":"2024-09-21T06:02:11.790834+0000","last_fullsized":"2024-09-21T06:02:11.790834+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:00.468017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789596+0000","last_change":"2024-09-21T06:02:09.264048+0000","last_active":"2024-09-21T06:02:11.789596+0000","last_peered":"2024-09-21T06:02:11.789596+0000","last_clean":"2024-09-21T06:02:11.789596+0000","last_became_active":"2024-09-21T06:02:09.263708+0000","last_became_peered":"2024-09-21T06:02:09.263708+0000","last_unstale":"2024-09-21T06:02:11.789596+0000","last_undegraded":"2024-09-21T06:02:11.789596+0000","last_fullsized":"2024-09-21T06:02:11.789596+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:25.531925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.239998+0000","last_change":"2024-09-21T06:02:08.239998+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.239998+0000","last_undegraded":"2024-09-21T06:02:08.239998+0000","last_fullsized":"2024-09-21T06:02:08.239998+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.246809+0000","last_change":"2024-09-21T06:02:08.246809+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.246809+0000","last_undegraded":"2024-09-21T06:02:08.246809+0000","last_fullsized":"2024-09-21T06:02:08.246809+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":174,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T06:02:08.247360+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:02:08.247360+0000","last_peered":"2024-09-21T06:02:08.247360+0000","last_clean":"2024-09-21T06:02:08.247360+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:02:08.247360+0000","last_undegraded":"2024-09-21T06:02:08.247360+0000","last_fullsized":"2024-09-21T06:02:08.247360+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40728,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331112,"statfs":{"total":96636764160,"available":96595058688,"internally_reserved":0,"allocated":1245184,"data_stored":855208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509513,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45728,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326112,"statfs":{"total":96636764160,"available":96589938688,"internally_reserved":0,"allocated":663552,"data_stored":270685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40160,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331680,"statfs":{"total":96636764160,"available":96595640320,"internally_reserved":0,"allocated":651264,"data_stored":264840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509513,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62352,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309488,"statfs":{"total":96636764160,"available":96572915712,"internally_reserved":0,"allocated":651264,"data_stored":264840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45712,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326128,"statfs":{"total":96636764160,"available":96589955072,"internally_reserved":0,"allocated":651264,"data_stored":264840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509514,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":264840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46296,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325544,"statfs":{"total":96636764160,"available":96589357056,"internally_reserved":0,"allocated":1245184,"data_stored":855208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542218,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49928,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321912,"statfs":{"total":96636764160,"available":96585637888,"internally_reserved":0,"allocated":704512,"data_stored":290201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:02:14.517 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:02:14.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.676+0000 7f82ac400640 1 -- 172.21.15.50:0/3895442117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f828c003c20 msgr2=0x7f828c004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.676+0000 7f82ac400640 1 --2- 172.21.15.50:0/3895442117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f828c003c20 0x7f828c004070 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8298009b60 tx=0x7f829801d290 comp rx=0 tx=0).stop 2024-09-21T06:02:14.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.676+0000 7f82aa400640 1 -- 172.21.15.50:0/3895442117 reap_dead start 2024-09-21T06:02:14.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.676+0000 7f82ac400640 1 -- 172.21.15.50:0/3895442117 shutdown_connections 2024-09-21T06:02:14.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.676+0000 7f82ac400640 1 -- 172.21.15.50:0/3895442117 >> 172.21.15.50:0/3895442117 conn(0x7f82a40708c0 msgr2=0x7f82a4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:14.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.676+0000 7f82ac400640 1 -- 172.21.15.50:0/3895442117 shutdown_connections 2024-09-21T06:02:14.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.676+0000 7f82ac400640 1 -- 172.21.15.50:0/3895442117 wait complete. 2024-09-21T06:02:14.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82ac400640 1 Processor -- start 2024-09-21T06:02:14.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82ac400640 1 -- start start 2024-09-21T06:02:14.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82ac400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f828c003c20 0x7f82a41f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82ac400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a4150510 0x7f82a414e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82ac400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82a414e5f0 0x7f82a40b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82ac400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41693d0 con 0x7f82a4150510 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82ac400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4169250 con 0x7f82a414e5f0 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82ac400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f82a4169550 con 0x7f828c003c20 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82a9000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a4150510 0x7f82a414e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82a9000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a4150510 0x7f82a414e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48214/0 (socket says 172.21.15.50:48214) 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82a9000640 1 -- 172.21.15.50:0/1328540911 learned_addr learned my addr 172.21.15.50:0/1328540911 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82aa400640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82a414e5f0 0x7f82a40b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82a9a00640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f828c003c20 0x7f82a41f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82a9000640 1 -- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f828c003c20 msgr2=0x7f82a41f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82a9000640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f828c003c20 0x7f82a41f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82a9000640 1 -- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82a414e5f0 msgr2=0x7f82a40b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82a9000640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82a414e5f0 0x7f82a40b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.680+0000 7f82a9000640 1 -- 172.21.15.50:0/1328540911 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a414ea00 con 0x7f82a4150510 2024-09-21T06:02:14.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.684+0000 7f82a9000640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a4150510 0x7f82a414e0b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8290003ec0 tx=0x7f8290001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:14.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.684+0000 7f82a1600640 1 -- 172.21.15.50:0/1328540911 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290015070 con 0x7f82a4150510 2024-09-21T06:02:14.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.684+0000 7f82a1600640 1 -- 172.21.15.50:0/1328540911 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8290011900 con 0x7f82a4150510 2024-09-21T06:02:14.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.684+0000 7f82ac400640 1 -- 172.21.15.50:0/1328540911 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a4150920 con 0x7f82a4150510 2024-09-21T06:02:14.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.684+0000 7f82a1600640 1 -- 172.21.15.50:0/1328540911 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290011d50 con 0x7f82a4150510 2024-09-21T06:02:14.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.684+0000 7f82ac400640 1 -- 172.21.15.50:0/1328540911 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a414bcc0 con 0x7f82a4150510 2024-09-21T06:02:14.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.684+0000 7f82ac400640 1 -- 172.21.15.50:0/1328540911 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a415cf90 con 0x7f82a4150510 2024-09-21T06:02:14.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.684+0000 7f82a1600640 1 -- 172.21.15.50:0/1328540911 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f8290010420 con 0x7f82a4150510 2024-09-21T06:02:14.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.684+0000 7f82a1600640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8284073b00 0x7f8284076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:14.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.684+0000 7f82a1600640 1 -- 172.21.15.50:0/1328540911 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f8290093780 con 0x7f82a4150510 2024-09-21T06:02:14.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.684+0000 7f82a9a00640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8284073b00 0x7f8284076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:14.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.688+0000 7f82a9a00640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8284073b00 0x7f8284076010 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f82980ae000 tx=0x7f829801a030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:14.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.688+0000 7f82a1600640 1 -- 172.21.15.50:0/1328540911 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f82a414ea00 con 0x7f82a4150510 2024-09-21T06:02:14.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.860+0000 7f82ac400640 1 -- 172.21.15.50:0/1328540911 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82a406b540 con 0x7f8284073b00 2024-09-21T06:02:14.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.868+0000 7f82a1600640 1 -- 172.21.15.50:0/1328540911 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241098 (secure 0 0 0) 0x7f82980051e0 con 0x7f8284073b00 2024-09-21T06:02:14.872 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:14.873 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:02:14.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.872+0000 7f82ac400640 1 -- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8284073b00 msgr2=0x7f8284076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.872+0000 7f82ac400640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8284073b00 0x7f8284076010 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f82980ae000 tx=0x7f829801a030 comp rx=0 tx=0).stop 2024-09-21T06:02:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.872+0000 7f82ac400640 1 -- 172.21.15.50:0/1328540911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a4150510 msgr2=0x7f82a414e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.872+0000 7f82ac400640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a4150510 0x7f82a414e0b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8290003ec0 tx=0x7f8290001810 comp rx=0 tx=0).stop 2024-09-21T06:02:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.872+0000 7f82ac400640 1 -- 172.21.15.50:0/1328540911 shutdown_connections 2024-09-21T06:02:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.872+0000 7f82ac400640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82a414e5f0 0x7f82a40b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.872+0000 7f82ac400640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8284073b00 0x7f8284076010 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.872+0000 7f82ac400640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a4150510 0x7f82a414e0b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.872+0000 7f82ac400640 1 --2- 172.21.15.50:0/1328540911 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f828c003c20 0x7f82a41f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.872+0000 7f82ac400640 1 -- 172.21.15.50:0/1328540911 >> 172.21.15.50:0/1328540911 conn(0x7f82a40708c0 msgr2=0x7f82a40d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.872+0000 7f82ac400640 1 -- 172.21.15.50:0/1328540911 shutdown_connections 2024-09-21T06:02:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:14.872+0000 7f82ac400640 1 -- 172.21.15.50:0/1328540911 wait complete. 2024-09-21T06:02:14.893 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-21T06:02:12.983730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":358568,"kb_used_data":6312,"kb_used_omap":12,"kb_used_meta":352051,"kb_avail":754616152,"statfs":{"total":773094113280,"available":772726939648,"internally_reserved":0,"allocated":6463488,"data_stored":3330662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":360500784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.746307"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790900+0000","last_change":"2024-09-21T06:02:10.262539+0000","last_active":"2024-09-21T06:02:11.790900+0000","last_peered":"2024-09-21T06:02:11.790900+0000","last_clean":"2024-09-21T06:02:11.790900+0000","last_became_active":"2024-09-21T06:02:10.261557+0000","last_became_peered":"2024-09-21T06:02:10.261557+0000","last_unstale":"2024-09-21T06:02:11.790900+0000","last_undegraded":"2024-09-21T06:02:11.790900+0000","last_fullsized":"2024-09-21T06:02:11.790900+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:46.770394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790427+0000","last_change":"2024-09-21T06:02:10.280853+0000","last_active":"2024-09-21T06:02:11.790427+0000","last_peered":"2024-09-21T06:02:11.790427+0000","last_clean":"2024-09-21T06:02:11.790427+0000","last_became_active":"2024-09-21T06:02:10.280106+0000","last_became_peered":"2024-09-21T06:02:10.280106+0000","last_unstale":"2024-09-21T06:02:11.790427+0000","last_undegraded":"2024-09-21T06:02:11.790427+0000","last_fullsized":"2024-09-21T06:02:11.790427+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:54.589979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.242772+0000","last_change":"2024-09-21T06:02:08.242772+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.242772+0000","last_undegraded":"2024-09-21T06:02:08.242772+0000","last_fullsized":"2024-09-21T06:02:08.242772+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790843+0000","last_change":"2024-09-21T06:02:10.271022+0000","last_active":"2024-09-21T06:02:11.790843+0000","last_peered":"2024-09-21T06:02:11.790843+0000","last_clean":"2024-09-21T06:02:11.790843+0000","last_became_active":"2024-09-21T06:02:10.270717+0000","last_became_peered":"2024-09-21T06:02:10.270717+0000","last_unstale":"2024-09-21T06:02:11.790843+0000","last_undegraded":"2024-09-21T06:02:11.790843+0000","last_fullsized":"2024-09-21T06:02:11.790843+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:25.972520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240634+0000","last_change":"2024-09-21T06:02:08.240634+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240634+0000","last_undegraded":"2024-09-21T06:02:08.240634+0000","last_fullsized":"2024-09-21T06:02:08.240634+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.242585+0000","last_change":"2024-09-21T06:02:08.242585+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.242585+0000","last_undegraded":"2024-09-21T06:02:08.242585+0000","last_fullsized":"2024-09-21T06:02:08.242585+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.242880+0000","last_change":"2024-09-21T06:02:08.242880+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.242880+0000","last_undegraded":"2024-09-21T06:02:08.242880+0000","last_fullsized":"2024-09-21T06:02:08.242880+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240546+0000","last_change":"2024-09-21T06:02:08.240546+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240546+0000","last_undegraded":"2024-09-21T06:02:08.240546+0000","last_fullsized":"2024-09-21T06:02:08.240546+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.242604+0000","last_change":"2024-09-21T06:02:08.242604+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.242604+0000","last_undegraded":"2024-09-21T06:02:08.242604+0000","last_fullsized":"2024-09-21T06:02:08.242604+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.243036+0000","last_change":"2024-09-21T06:02:08.243036+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.243036+0000","last_undegraded":"2024-09-21T06:02:08.243036+0000","last_fullsized":"2024-09-21T06:02:08.243036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240351+0000","last_change":"2024-09-21T06:02:08.240351+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240351+0000","last_undegraded":"2024-09-21T06:02:08.240351+0000","last_fullsized":"2024-09-21T06:02:08.240351+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.246856+0000","last_change":"2024-09-21T06:02:08.246856+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.246856+0000","last_undegraded":"2024-09-21T06:02:08.246856+0000","last_fullsized":"2024-09-21T06:02:08.246856+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.239994+0000","last_change":"2024-09-21T06:02:08.239994+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.239994+0000","last_undegraded":"2024-09-21T06:02:08.239994+0000","last_fullsized":"2024-09-21T06:02:08.239994+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790136+0000","last_change":"2024-09-21T06:02:10.272781+0000","last_active":"2024-09-21T06:02:11.790136+0000","last_peered":"2024-09-21T06:02:11.790136+0000","last_clean":"2024-09-21T06:02:11.790136+0000","last_became_active":"2024-09-21T06:02:10.272474+0000","last_became_peered":"2024-09-21T06:02:10.272474+0000","last_unstale":"2024-09-21T06:02:11.790136+0000","last_undegraded":"2024-09-21T06:02:11.790136+0000","last_fullsized":"2024-09-21T06:02:11.790136+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:19.569443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.246113+0000","last_change":"2024-09-21T06:02:08.246113+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.246113+0000","last_undegraded":"2024-09-21T06:02:08.246113+0000","last_fullsized":"2024-09-21T06:02:08.246113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240352+0000","last_change":"2024-09-21T06:02:08.240352+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240352+0000","last_undegraded":"2024-09-21T06:02:08.240352+0000","last_fullsized":"2024-09-21T06:02:08.240352+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240647+0000","last_change":"2024-09-21T06:02:08.240647+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240647+0000","last_undegraded":"2024-09-21T06:02:08.240647+0000","last_fullsized":"2024-09-21T06:02:08.240647+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.246817+0000","last_change":"2024-09-21T06:02:08.246817+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.246817+0000","last_undegraded":"2024-09-21T06:02:08.246817+0000","last_fullsized":"2024-09-21T06:02:08.246817+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.241370+0000","last_change":"2024-09-21T06:02:08.241370+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.241370+0000","last_undegraded":"2024-09-21T06:02:08.241370+0000","last_fullsized":"2024-09-21T06:02:08.241370+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.244769+0000","last_change":"2024-09-21T06:02:08.244769+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.244769+0000","last_undegraded":"2024-09-21T06:02:08.244769+0000","last_fullsized":"2024-09-21T06:02:08.244769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240228+0000","last_change":"2024-09-21T06:02:08.240228+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240228+0000","last_undegraded":"2024-09-21T06:02:08.240228+0000","last_fullsized":"2024-09-21T06:02:08.240228+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.243659+0000","last_change":"2024-09-21T06:02:08.243659+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.243659+0000","last_undegraded":"2024-09-21T06:02:08.243659+0000","last_fullsized":"2024-09-21T06:02:08.243659+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.241351+0000","last_change":"2024-09-21T06:02:08.241351+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.241351+0000","last_undegraded":"2024-09-21T06:02:08.241351+0000","last_fullsized":"2024-09-21T06:02:08.241351+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.243155+0000","last_change":"2024-09-21T06:02:08.243155+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.243155+0000","last_undegraded":"2024-09-21T06:02:08.243155+0000","last_fullsized":"2024-09-21T06:02:08.243155+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.239998+0000","last_change":"2024-09-21T06:02:08.239998+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.239998+0000","last_undegraded":"2024-09-21T06:02:08.239998+0000","last_fullsized":"2024-09-21T06:02:08.239998+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789710+0000","last_change":"2024-09-21T06:02:09.271164+0000","last_active":"2024-09-21T06:02:11.789710+0000","last_peered":"2024-09-21T06:02:11.789710+0000","last_clean":"2024-09-21T06:02:11.789710+0000","last_became_active":"2024-09-21T06:02:09.270848+0000","last_became_peered":"2024-09-21T06:02:09.270848+0000","last_unstale":"2024-09-21T06:02:11.789710+0000","last_undegraded":"2024-09-21T06:02:11.789710+0000","last_fullsized":"2024-09-21T06:02:11.789710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:58.592636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240562+0000","last_change":"2024-09-21T06:02:08.240562+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240562+0000","last_undegraded":"2024-09-21T06:02:08.240562+0000","last_fullsized":"2024-09-21T06:02:08.240562+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790736+0000","last_change":"2024-09-21T06:02:09.275643+0000","last_active":"2024-09-21T06:02:11.790736+0000","last_peered":"2024-09-21T06:02:11.790736+0000","last_clean":"2024-09-21T06:02:11.790736+0000","last_became_active":"2024-09-21T06:02:09.275319+0000","last_became_peered":"2024-09-21T06:02:09.275319+0000","last_unstale":"2024-09-21T06:02:11.790736+0000","last_undegraded":"2024-09-21T06:02:11.790736+0000","last_fullsized":"2024-09-21T06:02:11.790736+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:46.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.242652+0000","last_change":"2024-09-21T06:02:08.242652+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.242652+0000","last_undegraded":"2024-09-21T06:02:08.242652+0000","last_fullsized":"2024-09-21T06:02:08.242652+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790403+0000","last_change":"2024-09-21T06:02:10.262950+0000","last_active":"2024-09-21T06:02:11.790403+0000","last_peered":"2024-09-21T06:02:11.790403+0000","last_clean":"2024-09-21T06:02:11.790403+0000","last_became_active":"2024-09-21T06:02:10.262636+0000","last_became_peered":"2024-09-21T06:02:10.262636+0000","last_unstale":"2024-09-21T06:02:11.790403+0000","last_undegraded":"2024-09-21T06:02:11.790403+0000","last_fullsized":"2024-09-21T06:02:11.790403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:32.505486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.246801+0000","last_change":"2024-09-21T06:02:08.246801+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.246801+0000","last_undegraded":"2024-09-21T06:02:08.246801+0000","last_fullsized":"2024-09-21T06:02:08.246801+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.241785+0000","last_change":"2024-09-21T06:02:08.241785+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.241785+0000","last_undegraded":"2024-09-21T06:02:08.241785+0000","last_fullsized":"2024-09-21T06:02:08.241785+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790058+0000","last_change":"2024-09-21T06:02:09.263348+0000","last_active":"2024-09-21T06:02:11.790058+0000","last_peered":"2024-09-21T06:02:11.790058+0000","last_clean":"2024-09-21T06:02:11.790058+0000","last_became_active":"2024-09-21T06:02:09.262942+0000","last_became_peered":"2024-09-21T06:02:09.262942+0000","last_unstale":"2024-09-21T06:02:11.790058+0000","last_undegraded":"2024-09-21T06:02:11.790058+0000","last_fullsized":"2024-09-21T06:02:11.790058+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:13.084051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.240981+0000","last_change":"2024-09-21T06:02:08.240981+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.240981+0000","last_undegraded":"2024-09-21T06:02:08.240981+0000","last_fullsized":"2024-09-21T06:02:08.240981+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790834+0000","last_change":"2024-09-21T06:02:10.261037+0000","last_active":"2024-09-21T06:02:11.790834+0000","last_peered":"2024-09-21T06:02:11.790834+0000","last_clean":"2024-09-21T06:02:11.790834+0000","last_became_active":"2024-09-21T06:02:10.260729+0000","last_became_peered":"2024-09-21T06:02:10.260729+0000","last_unstale":"2024-09-21T06:02:11.790834+0000","last_undegraded":"2024-09-21T06:02:11.790834+0000","last_fullsized":"2024-09-21T06:02:11.790834+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:00.468017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789596+0000","last_change":"2024-09-21T06:02:09.264048+0000","last_active":"2024-09-21T06:02:11.789596+0000","last_peered":"2024-09-21T06:02:11.789596+0000","last_clean":"2024-09-21T06:02:11.789596+0000","last_became_active":"2024-09-21T06:02:09.263708+0000","last_became_peered":"2024-09-21T06:02:09.263708+0000","last_unstale":"2024-09-21T06:02:11.789596+0000","last_undegraded":"2024-09-21T06:02:11.789596+0000","last_fullsized":"2024-09-21T06:02:11.789596+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:25.531925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.239998+0000","last_change":"2024-09-21T06:02:08.239998+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.239998+0000","last_undegraded":"2024-09-21T06:02:08.239998+0000","last_fullsized":"2024-09-21T06:02:08.239998+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:02:09.236506+0000","last_change":"2024-09-21T06:02:09.236506+0000","last_active":"2024-09-21T06:02:09.236506+0000","last_peered":"2024-09-21T06:02:09.236506+0000","last_clean":"2024-09-21T06:02:09.236506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:09.236506+0000","last_undegraded":"2024-09-21T06:02:09.236506+0000","last_fullsized":"2024-09-21T06:02:09.236506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-21T06:02:08.246809+0000","last_change":"2024-09-21T06:02:08.246809+0000","last_active":"2024-09-21T06:02:08.228627+0000","last_peered":"2024-09-21T06:02:08.228627+0000","last_clean":"2024-09-21T06:02:08.228627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:02:08.246809+0000","last_undegraded":"2024-09-21T06:02:08.246809+0000","last_fullsized":"2024-09-21T06:02:08.246809+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":174,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T06:02:08.247360+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:02:08.247360+0000","last_peered":"2024-09-21T06:02:08.247360+0000","last_clean":"2024-09-21T06:02:08.247360+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:02:08.247360+0000","last_undegraded":"2024-09-21T06:02:08.247360+0000","last_fullsized":"2024-09-21T06:02:08.247360+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40728,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331112,"statfs":{"total":96636764160,"available":96595058688,"internally_reserved":0,"allocated":1245184,"data_stored":855208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509513,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45728,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326112,"statfs":{"total":96636764160,"available":96589938688,"internally_reserved":0,"allocated":663552,"data_stored":270685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40160,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331680,"statfs":{"total":96636764160,"available":96595640320,"internally_reserved":0,"allocated":651264,"data_stored":264840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509513,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62352,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309488,"statfs":{"total":96636764160,"available":96572915712,"internally_reserved":0,"allocated":651264,"data_stored":264840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45712,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326128,"statfs":{"total":96636764160,"available":96589955072,"internally_reserved":0,"allocated":651264,"data_stored":264840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509514,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":264840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46296,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325544,"statfs":{"total":96636764160,"available":96589357056,"internally_reserved":0,"allocated":1245184,"data_stored":855208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542218,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49928,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321912,"statfs":{"total":96636764160,"available":96585637888,"internally_reserved":0,"allocated":704512,"data_stored":290201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:02:14.898 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:02:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.092+0000 7fa88f400640 1 -- 172.21.15.50:0/4191016262 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8880b4ce0 msgr2=0x7fa88815d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.092+0000 7fa88f400640 1 --2- 172.21.15.50:0/4191016262 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8880b4ce0 0x7fa88815d890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa880009bf0 tx=0x7fa88815e5e0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.092+0000 7fa88f400640 1 -- 172.21.15.50:0/4191016262 shutdown_connections 2024-09-21T06:02:15.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.092+0000 7fa88f400640 1 --2- 172.21.15.50:0/4191016262 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8880b4ce0 0x7fa88815d890 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.092+0000 7fa88f400640 1 -- 172.21.15.50:0/4191016262 >> 172.21.15.50:0/4191016262 conn(0x7fa8880b2040 msgr2=0x7fa8880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:15.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.092+0000 7fa88f400640 1 -- 172.21.15.50:0/4191016262 shutdown_connections 2024-09-21T06:02:15.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.092+0000 7fa88f400640 1 -- 172.21.15.50:0/4191016262 wait complete. 2024-09-21T06:02:15.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88f400640 1 Processor -- start 2024-09-21T06:02:15.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88f400640 1 -- start start 2024-09-21T06:02:15.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8880b4ce0 0x7fa8881fc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880cf900 0x7fa888154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8880ce240 0x7fa8880ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88f400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa888168c70 con 0x7fa8880b4ce0 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88f400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa8880b3b50 con 0x7fa8880cf900 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88f400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa888168df0 con 0x7fa8880ce240 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8880b4ce0 0x7fa8881fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8880b4ce0 0x7fa8881fc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48224/0 (socket says 172.21.15.50:48224) 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88ca00640 1 -- 172.21.15.50:0/1945340852 learned_addr learned my addr 172.21.15.50:0/1945340852 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88d400640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8880ce240 0x7fa8880ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa887e00640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880cf900 0x7fa888154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88ca00640 1 -- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8880ce240 msgr2=0x7fa8880ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88ca00640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8880ce240 0x7fa8880ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88ca00640 1 -- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880cf900 msgr2=0x7fa888154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88ca00640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880cf900 0x7fa888154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88ca00640 1 -- 172.21.15.50:0/1945340852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88815f220 con 0x7fa8880b4ce0 2024-09-21T06:02:15.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88ca00640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8880b4ce0 0x7fa8881fc580 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00c400 tx=0x7fa87c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:15.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa884c00640 1 -- 172.21.15.50:0/1945340852 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c017070 con 0x7fa8880b4ce0 2024-09-21T06:02:15.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa884c00640 1 -- 172.21.15.50:0/1945340852 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa87c003600 con 0x7fa8880b4ce0 2024-09-21T06:02:15.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa884c00640 1 -- 172.21.15.50:0/1945340852 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c003a50 con 0x7fa8880b4ce0 2024-09-21T06:02:15.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88f400640 1 -- 172.21.15.50:0/1945340852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8880ceb90 con 0x7fa8880b4ce0 2024-09-21T06:02:15.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.096+0000 7fa88f400640 1 -- 172.21.15.50:0/1945340852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8881fcac0 con 0x7fa8880b4ce0 2024-09-21T06:02:15.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.100+0000 7fa88f400640 1 -- 172.21.15.50:0/1945340852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa88813bbb0 con 0x7fa8880b4ce0 2024-09-21T06:02:15.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.100+0000 7fa884c00640 1 -- 172.21.15.50:0/1945340852 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fa87c01a070 con 0x7fa8880b4ce0 2024-09-21T06:02:15.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.100+0000 7fa884c00640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa868073b00 0x7fa868076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:15.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.100+0000 7fa884c00640 1 -- 172.21.15.50:0/1945340852 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fa87c091e50 con 0x7fa8880b4ce0 2024-09-21T06:02:15.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.100+0000 7fa887e00640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa868073b00 0x7fa868076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:15.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.104+0000 7fa887e00640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa868073b00 0x7fa868076010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa888139c60 tx=0x7fa874009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:15.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.104+0000 7fa884c00640 1 -- 172.21.15.50:0/1945340852 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa88813bbb0 con 0x7fa8880b4ce0 2024-09-21T06:02:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.304+0000 7fa88f400640 1 -- 172.21.15.50:0/1945340852 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa88806b540 con 0x7fa868073b00 2024-09-21T06:02:15.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.308+0000 7fa884c00640 1 -- 172.21.15.50:0/1945340852 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251943 (secure 0 0 0) 0x7fa874015430 con 0x7fa868073b00 2024-09-21T06:02:15.312 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:15.313 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:02:15.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.312+0000 7fa88f400640 1 -- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa868073b00 msgr2=0x7fa868076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.312+0000 7fa88f400640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa868073b00 0x7fa868076010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa888139c60 tx=0x7fa874009040 comp rx=0 tx=0).stop 2024-09-21T06:02:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.312+0000 7fa88f400640 1 -- 172.21.15.50:0/1945340852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8880b4ce0 msgr2=0x7fa8881fc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.312+0000 7fa88f400640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8880b4ce0 0x7fa8881fc580 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00c400 tx=0x7fa87c00c8c0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.312+0000 7fa88f400640 1 -- 172.21.15.50:0/1945340852 shutdown_connections 2024-09-21T06:02:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.312+0000 7fa88f400640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8880ce240 0x7fa8880ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.312+0000 7fa88f400640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880cf900 0x7fa888154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.312+0000 7fa88f400640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa868073b00 0x7fa868076010 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.312+0000 7fa88f400640 1 --2- 172.21.15.50:0/1945340852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8880b4ce0 0x7fa8881fc580 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.312+0000 7fa88f400640 1 -- 172.21.15.50:0/1945340852 >> 172.21.15.50:0/1945340852 conn(0x7fa8880b2040 msgr2=0x7fa88806e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.312+0000 7fa88f400640 1 -- 172.21.15.50:0/1945340852 shutdown_connections 2024-09-21T06:02:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.312+0000 7fa88f400640 1 -- 172.21.15.50:0/1945340852 wait complete. 2024-09-21T06:02:15.335 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-21T06:02:14.984316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":358960,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":352051,"kb_avail":754615760,"statfs":{"total":773094113280,"available":772726538240,"internally_reserved":0,"allocated":6856704,"data_stored":3506444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":360500784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001742"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250964+0000","last_change":"2024-09-21T06:02:10.283497+0000","last_active":"2024-09-21T06:02:11.250964+0000","last_peered":"2024-09-21T06:02:11.250964+0000","last_clean":"2024-09-21T06:02:11.250964+0000","last_became_active":"2024-09-21T06:02:10.283183+0000","last_became_peered":"2024-09-21T06:02:10.283183+0000","last_unstale":"2024-09-21T06:02:11.250964+0000","last_undegraded":"2024-09-21T06:02:11.250964+0000","last_fullsized":"2024-09-21T06:02:11.250964+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:50.423813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792195+0000","last_change":"2024-09-21T06:02:10.285494+0000","last_active":"2024-09-21T06:02:11.792195+0000","last_peered":"2024-09-21T06:02:11.792195+0000","last_clean":"2024-09-21T06:02:11.792195+0000","last_became_active":"2024-09-21T06:02:10.285121+0000","last_became_peered":"2024-09-21T06:02:10.285121+0000","last_unstale":"2024-09-21T06:02:11.792195+0000","last_undegraded":"2024-09-21T06:02:11.792195+0000","last_fullsized":"2024-09-21T06:02:11.792195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:32.261338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289739+0000","last_change":"2024-09-21T06:02:10.277961+0000","last_active":"2024-09-21T06:02:11.289739+0000","last_peered":"2024-09-21T06:02:11.289739+0000","last_clean":"2024-09-21T06:02:11.289739+0000","last_became_active":"2024-09-21T06:02:10.277639+0000","last_became_peered":"2024-09-21T06:02:10.277639+0000","last_unstale":"2024-09-21T06:02:11.289739+0000","last_undegraded":"2024-09-21T06:02:11.289739+0000","last_fullsized":"2024-09-21T06:02:11.289739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:52.411453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254579+0000","last_change":"2024-09-21T06:02:10.267778+0000","last_active":"2024-09-21T06:02:11.254579+0000","last_peered":"2024-09-21T06:02:11.254579+0000","last_clean":"2024-09-21T06:02:11.254579+0000","last_became_active":"2024-09-21T06:02:10.263807+0000","last_became_peered":"2024-09-21T06:02:10.263807+0000","last_unstale":"2024-09-21T06:02:11.254579+0000","last_undegraded":"2024-09-21T06:02:11.254579+0000","last_fullsized":"2024-09-21T06:02:11.254579+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:00.353554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250565+0000","last_change":"2024-09-21T06:02:10.268540+0000","last_active":"2024-09-21T06:02:11.250565+0000","last_peered":"2024-09-21T06:02:11.250565+0000","last_clean":"2024-09-21T06:02:11.250565+0000","last_became_active":"2024-09-21T06:02:10.267456+0000","last_became_peered":"2024-09-21T06:02:10.267456+0000","last_unstale":"2024-09-21T06:02:11.250565+0000","last_undegraded":"2024-09-21T06:02:11.250565+0000","last_fullsized":"2024-09-21T06:02:11.250565+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:14.203778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791022+0000","last_change":"2024-09-21T06:02:10.279996+0000","last_active":"2024-09-21T06:02:11.791022+0000","last_peered":"2024-09-21T06:02:11.791022+0000","last_clean":"2024-09-21T06:02:11.791022+0000","last_became_active":"2024-09-21T06:02:10.279672+0000","last_became_peered":"2024-09-21T06:02:10.279672+0000","last_unstale":"2024-09-21T06:02:11.791022+0000","last_undegraded":"2024-09-21T06:02:11.791022+0000","last_fullsized":"2024-09-21T06:02:11.791022+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:51.112377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253169+0000","last_change":"2024-09-21T06:02:10.267014+0000","last_active":"2024-09-21T06:02:11.253169+0000","last_peered":"2024-09-21T06:02:11.253169+0000","last_clean":"2024-09-21T06:02:11.253169+0000","last_became_active":"2024-09-21T06:02:10.266091+0000","last_became_peered":"2024-09-21T06:02:10.266091+0000","last_unstale":"2024-09-21T06:02:11.253169+0000","last_undegraded":"2024-09-21T06:02:11.253169+0000","last_fullsized":"2024-09-21T06:02:11.253169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:44.072609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791264+0000","last_change":"2024-09-21T06:02:10.275841+0000","last_active":"2024-09-21T06:02:11.791264+0000","last_peered":"2024-09-21T06:02:11.791264+0000","last_clean":"2024-09-21T06:02:11.791264+0000","last_became_active":"2024-09-21T06:02:10.275526+0000","last_became_peered":"2024-09-21T06:02:10.275526+0000","last_unstale":"2024-09-21T06:02:11.791264+0000","last_undegraded":"2024-09-21T06:02:11.791264+0000","last_fullsized":"2024-09-21T06:02:11.791264+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:42.101108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253587+0000","last_change":"2024-09-21T06:02:10.293470+0000","last_active":"2024-09-21T06:02:11.253587+0000","last_peered":"2024-09-21T06:02:11.253587+0000","last_clean":"2024-09-21T06:02:11.253587+0000","last_became_active":"2024-09-21T06:02:10.292993+0000","last_became_peered":"2024-09-21T06:02:10.292993+0000","last_unstale":"2024-09-21T06:02:11.253587+0000","last_undegraded":"2024-09-21T06:02:11.253587+0000","last_fullsized":"2024-09-21T06:02:11.253587+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:16.117957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790900+0000","last_change":"2024-09-21T06:02:10.262539+0000","last_active":"2024-09-21T06:02:11.790900+0000","last_peered":"2024-09-21T06:02:11.790900+0000","last_clean":"2024-09-21T06:02:11.790900+0000","last_became_active":"2024-09-21T06:02:10.261557+0000","last_became_peered":"2024-09-21T06:02:10.261557+0000","last_unstale":"2024-09-21T06:02:11.790900+0000","last_undegraded":"2024-09-21T06:02:11.790900+0000","last_fullsized":"2024-09-21T06:02:11.790900+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:46.770394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790427+0000","last_change":"2024-09-21T06:02:10.280853+0000","last_active":"2024-09-21T06:02:11.790427+0000","last_peered":"2024-09-21T06:02:11.790427+0000","last_clean":"2024-09-21T06:02:11.790427+0000","last_became_active":"2024-09-21T06:02:10.280106+0000","last_became_peered":"2024-09-21T06:02:10.280106+0000","last_unstale":"2024-09-21T06:02:11.790427+0000","last_undegraded":"2024-09-21T06:02:11.790427+0000","last_fullsized":"2024-09-21T06:02:11.790427+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:54.589979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254706+0000","last_change":"2024-09-21T06:02:10.272692+0000","last_active":"2024-09-21T06:02:11.254706+0000","last_peered":"2024-09-21T06:02:11.254706+0000","last_clean":"2024-09-21T06:02:11.254706+0000","last_became_active":"2024-09-21T06:02:10.272304+0000","last_became_peered":"2024-09-21T06:02:10.272304+0000","last_unstale":"2024-09-21T06:02:11.254706+0000","last_undegraded":"2024-09-21T06:02:11.254706+0000","last_fullsized":"2024-09-21T06:02:11.254706+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:19.605248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790044+0000","last_change":"2024-09-21T06:02:10.282612+0000","last_active":"2024-09-21T06:02:11.790044+0000","last_peered":"2024-09-21T06:02:11.790044+0000","last_clean":"2024-09-21T06:02:11.790044+0000","last_became_active":"2024-09-21T06:02:10.282114+0000","last_became_peered":"2024-09-21T06:02:10.282114+0000","last_unstale":"2024-09-21T06:02:11.790044+0000","last_undegraded":"2024-09-21T06:02:11.790044+0000","last_fullsized":"2024-09-21T06:02:11.790044+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:43.415017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253561+0000","last_change":"2024-09-21T06:02:10.275529+0000","last_active":"2024-09-21T06:02:11.253561+0000","last_peered":"2024-09-21T06:02:11.253561+0000","last_clean":"2024-09-21T06:02:11.253561+0000","last_became_active":"2024-09-21T06:02:10.275228+0000","last_became_peered":"2024-09-21T06:02:10.275228+0000","last_unstale":"2024-09-21T06:02:11.253561+0000","last_undegraded":"2024-09-21T06:02:11.253561+0000","last_fullsized":"2024-09-21T06:02:11.253561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:13.670308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253850+0000","last_change":"2024-09-21T06:02:10.276288+0000","last_active":"2024-09-21T06:02:11.253850+0000","last_peered":"2024-09-21T06:02:11.253850+0000","last_clean":"2024-09-21T06:02:11.253850+0000","last_became_active":"2024-09-21T06:02:10.274618+0000","last_became_peered":"2024-09-21T06:02:10.274618+0000","last_unstale":"2024-09-21T06:02:11.253850+0000","last_undegraded":"2024-09-21T06:02:11.253850+0000","last_fullsized":"2024-09-21T06:02:11.253850+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:23.366129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789824+0000","last_change":"2024-09-21T06:02:10.279231+0000","last_active":"2024-09-21T06:02:11.789824+0000","last_peered":"2024-09-21T06:02:11.789824+0000","last_clean":"2024-09-21T06:02:11.789824+0000","last_became_active":"2024-09-21T06:02:10.278439+0000","last_became_peered":"2024-09-21T06:02:10.278439+0000","last_unstale":"2024-09-21T06:02:11.789824+0000","last_undegraded":"2024-09-21T06:02:11.789824+0000","last_fullsized":"2024-09-21T06:02:11.789824+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:00.331491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792149+0000","last_change":"2024-09-21T06:02:10.287270+0000","last_active":"2024-09-21T06:02:11.792149+0000","last_peered":"2024-09-21T06:02:11.792149+0000","last_clean":"2024-09-21T06:02:11.792149+0000","last_became_active":"2024-09-21T06:02:10.285162+0000","last_became_peered":"2024-09-21T06:02:10.285162+0000","last_unstale":"2024-09-21T06:02:11.792149+0000","last_undegraded":"2024-09-21T06:02:11.792149+0000","last_fullsized":"2024-09-21T06:02:11.792149+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.910114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290107+0000","last_change":"2024-09-21T06:02:10.288834+0000","last_active":"2024-09-21T06:02:11.290107+0000","last_peered":"2024-09-21T06:02:11.290107+0000","last_clean":"2024-09-21T06:02:11.290107+0000","last_became_active":"2024-09-21T06:02:10.288513+0000","last_became_peered":"2024-09-21T06:02:10.288513+0000","last_unstale":"2024-09-21T06:02:11.290107+0000","last_undegraded":"2024-09-21T06:02:11.290107+0000","last_fullsized":"2024-09-21T06:02:11.290107+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:10.144897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250654+0000","last_change":"2024-09-21T06:02:10.275125+0000","last_active":"2024-09-21T06:02:11.250654+0000","last_peered":"2024-09-21T06:02:11.250654+0000","last_clean":"2024-09-21T06:02:11.250654+0000","last_became_active":"2024-09-21T06:02:10.274544+0000","last_became_peered":"2024-09-21T06:02:10.274544+0000","last_unstale":"2024-09-21T06:02:11.250654+0000","last_undegraded":"2024-09-21T06:02:11.250654+0000","last_fullsized":"2024-09-21T06:02:11.250654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:19.338852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791447+0000","last_change":"2024-09-21T06:02:10.282190+0000","last_active":"2024-09-21T06:02:11.791447+0000","last_peered":"2024-09-21T06:02:11.791447+0000","last_clean":"2024-09-21T06:02:11.791447+0000","last_became_active":"2024-09-21T06:02:10.281877+0000","last_became_peered":"2024-09-21T06:02:10.281877+0000","last_unstale":"2024-09-21T06:02:11.791447+0000","last_undegraded":"2024-09-21T06:02:11.791447+0000","last_fullsized":"2024-09-21T06:02:11.791447+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:43.790427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289797+0000","last_change":"2024-09-21T06:02:10.283752+0000","last_active":"2024-09-21T06:02:11.289797+0000","last_peered":"2024-09-21T06:02:11.289797+0000","last_clean":"2024-09-21T06:02:11.289797+0000","last_became_active":"2024-09-21T06:02:10.283417+0000","last_became_peered":"2024-09-21T06:02:10.283417+0000","last_unstale":"2024-09-21T06:02:11.289797+0000","last_undegraded":"2024-09-21T06:02:11.289797+0000","last_fullsized":"2024-09-21T06:02:11.289797+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:32.351750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791244+0000","last_change":"2024-09-21T06:02:10.282281+0000","last_active":"2024-09-21T06:02:11.791244+0000","last_peered":"2024-09-21T06:02:11.791244+0000","last_clean":"2024-09-21T06:02:11.791244+0000","last_became_active":"2024-09-21T06:02:10.281585+0000","last_became_peered":"2024-09-21T06:02:10.281585+0000","last_unstale":"2024-09-21T06:02:11.791244+0000","last_undegraded":"2024-09-21T06:02:11.791244+0000","last_fullsized":"2024-09-21T06:02:11.791244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:32.507248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290027+0000","last_change":"2024-09-21T06:02:10.287973+0000","last_active":"2024-09-21T06:02:11.290027+0000","last_peered":"2024-09-21T06:02:11.290027+0000","last_clean":"2024-09-21T06:02:11.290027+0000","last_became_active":"2024-09-21T06:02:10.287123+0000","last_became_peered":"2024-09-21T06:02:10.287123+0000","last_unstale":"2024-09-21T06:02:11.290027+0000","last_undegraded":"2024-09-21T06:02:11.290027+0000","last_fullsized":"2024-09-21T06:02:11.290027+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:10.646439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790293+0000","last_change":"2024-09-21T06:02:10.282167+0000","last_active":"2024-09-21T06:02:11.790293+0000","last_peered":"2024-09-21T06:02:11.790293+0000","last_clean":"2024-09-21T06:02:11.790293+0000","last_became_active":"2024-09-21T06:02:10.281870+0000","last_became_peered":"2024-09-21T06:02:10.281870+0000","last_unstale":"2024-09-21T06:02:11.790293+0000","last_undegraded":"2024-09-21T06:02:11.790293+0000","last_fullsized":"2024-09-21T06:02:11.790293+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:40.075266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.251246+0000","last_change":"2024-09-21T06:02:10.275662+0000","last_active":"2024-09-21T06:02:11.251246+0000","last_peered":"2024-09-21T06:02:11.251246+0000","last_clean":"2024-09-21T06:02:11.251246+0000","last_became_active":"2024-09-21T06:02:10.274366+0000","last_became_peered":"2024-09-21T06:02:10.274366+0000","last_unstale":"2024-09-21T06:02:11.251246+0000","last_undegraded":"2024-09-21T06:02:11.251246+0000","last_fullsized":"2024-09-21T06:02:11.251246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:10.180010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254453+0000","last_change":"2024-09-21T06:02:10.263425+0000","last_active":"2024-09-21T06:02:11.254453+0000","last_peered":"2024-09-21T06:02:11.254453+0000","last_clean":"2024-09-21T06:02:11.254453+0000","last_became_active":"2024-09-21T06:02:10.263066+0000","last_became_peered":"2024-09-21T06:02:10.263066+0000","last_unstale":"2024-09-21T06:02:11.254453+0000","last_undegraded":"2024-09-21T06:02:11.254453+0000","last_fullsized":"2024-09-21T06:02:11.254453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:03.263466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790375+0000","last_change":"2024-09-21T06:02:10.291344+0000","last_active":"2024-09-21T06:02:11.790375+0000","last_peered":"2024-09-21T06:02:11.790375+0000","last_clean":"2024-09-21T06:02:11.790375+0000","last_became_active":"2024-09-21T06:02:10.286643+0000","last_became_peered":"2024-09-21T06:02:10.286643+0000","last_unstale":"2024-09-21T06:02:11.790375+0000","last_undegraded":"2024-09-21T06:02:11.790375+0000","last_fullsized":"2024-09-21T06:02:11.790375+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:23.219415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250636+0000","last_change":"2024-09-21T06:02:10.273465+0000","last_active":"2024-09-21T06:02:11.250636+0000","last_peered":"2024-09-21T06:02:11.250636+0000","last_clean":"2024-09-21T06:02:11.250636+0000","last_became_active":"2024-09-21T06:02:10.272894+0000","last_became_peered":"2024-09-21T06:02:10.272894+0000","last_unstale":"2024-09-21T06:02:11.250636+0000","last_undegraded":"2024-09-21T06:02:11.250636+0000","last_fullsized":"2024-09-21T06:02:11.250636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:23.146917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791745+0000","last_change":"2024-09-21T06:02:10.286555+0000","last_active":"2024-09-21T06:02:11.791745+0000","last_peered":"2024-09-21T06:02:11.791745+0000","last_clean":"2024-09-21T06:02:11.791745+0000","last_became_active":"2024-09-21T06:02:10.275977+0000","last_became_peered":"2024-09-21T06:02:10.275977+0000","last_unstale":"2024-09-21T06:02:11.791745+0000","last_undegraded":"2024-09-21T06:02:11.791745+0000","last_fullsized":"2024-09-21T06:02:11.791745+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:49.043147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289689+0000","last_change":"2024-09-21T06:02:10.276271+0000","last_active":"2024-09-21T06:02:11.289689+0000","last_peered":"2024-09-21T06:02:11.289689+0000","last_clean":"2024-09-21T06:02:11.289689+0000","last_became_active":"2024-09-21T06:02:10.275948+0000","last_became_peered":"2024-09-21T06:02:10.275948+0000","last_unstale":"2024-09-21T06:02:11.289689+0000","last_undegraded":"2024-09-21T06:02:11.289689+0000","last_fullsized":"2024-09-21T06:02:11.289689+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:54.824679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250230+0000","last_change":"2024-09-21T06:02:10.276173+0000","last_active":"2024-09-21T06:02:11.250230+0000","last_peered":"2024-09-21T06:02:11.250230+0000","last_clean":"2024-09-21T06:02:11.250230+0000","last_became_active":"2024-09-21T06:02:10.275712+0000","last_became_peered":"2024-09-21T06:02:10.275712+0000","last_unstale":"2024-09-21T06:02:11.250230+0000","last_undegraded":"2024-09-21T06:02:11.250230+0000","last_fullsized":"2024-09-21T06:02:11.250230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:39.258731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790950+0000","last_change":"2024-09-21T06:02:10.291460+0000","last_active":"2024-09-21T06:02:11.790950+0000","last_peered":"2024-09-21T06:02:11.790950+0000","last_clean":"2024-09-21T06:02:11.790950+0000","last_became_active":"2024-09-21T06:02:10.287310+0000","last_became_peered":"2024-09-21T06:02:10.287310+0000","last_unstale":"2024-09-21T06:02:11.790950+0000","last_undegraded":"2024-09-21T06:02:11.790950+0000","last_fullsized":"2024-09-21T06:02:11.790950+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:44.150713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790426+0000","last_change":"2024-09-21T06:02:10.278806+0000","last_active":"2024-09-21T06:02:11.790426+0000","last_peered":"2024-09-21T06:02:11.790426+0000","last_clean":"2024-09-21T06:02:11.790426+0000","last_became_active":"2024-09-21T06:02:10.278388+0000","last_became_peered":"2024-09-21T06:02:10.278388+0000","last_unstale":"2024-09-21T06:02:11.790426+0000","last_undegraded":"2024-09-21T06:02:11.790426+0000","last_fullsized":"2024-09-21T06:02:11.790426+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:15.541144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790128+0000","last_change":"2024-09-21T06:02:09.259400+0000","last_active":"2024-09-21T06:02:11.790128+0000","last_peered":"2024-09-21T06:02:11.790128+0000","last_clean":"2024-09-21T06:02:11.790128+0000","last_became_active":"2024-09-21T06:02:09.259059+0000","last_became_peered":"2024-09-21T06:02:09.259059+0000","last_unstale":"2024-09-21T06:02:11.790128+0000","last_undegraded":"2024-09-21T06:02:11.790128+0000","last_fullsized":"2024-09-21T06:02:11.790128+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:32.593732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790843+0000","last_change":"2024-09-21T06:02:10.271022+0000","last_active":"2024-09-21T06:02:11.790843+0000","last_peered":"2024-09-21T06:02:11.790843+0000","last_clean":"2024-09-21T06:02:11.790843+0000","last_became_active":"2024-09-21T06:02:10.270717+0000","last_became_peered":"2024-09-21T06:02:10.270717+0000","last_unstale":"2024-09-21T06:02:11.790843+0000","last_undegraded":"2024-09-21T06:02:11.790843+0000","last_fullsized":"2024-09-21T06:02:11.790843+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:25.972520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792150+0000","last_change":"2024-09-21T06:02:09.265832+0000","last_active":"2024-09-21T06:02:11.792150+0000","last_peered":"2024-09-21T06:02:11.792150+0000","last_clean":"2024-09-21T06:02:11.792150+0000","last_became_active":"2024-09-21T06:02:09.265458+0000","last_became_peered":"2024-09-21T06:02:09.265458+0000","last_unstale":"2024-09-21T06:02:11.792150+0000","last_undegraded":"2024-09-21T06:02:11.792150+0000","last_fullsized":"2024-09-21T06:02:11.792150+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:18.597369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790086+0000","last_change":"2024-09-21T06:02:10.259160+0000","last_active":"2024-09-21T06:02:11.790086+0000","last_peered":"2024-09-21T06:02:11.790086+0000","last_clean":"2024-09-21T06:02:11.790086+0000","last_became_active":"2024-09-21T06:02:10.258851+0000","last_became_peered":"2024-09-21T06:02:10.258851+0000","last_unstale":"2024-09-21T06:02:11.790086+0000","last_undegraded":"2024-09-21T06:02:11.790086+0000","last_fullsized":"2024-09-21T06:02:11.790086+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:53.190221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790742+0000","last_change":"2024-09-21T06:02:09.260660+0000","last_active":"2024-09-21T06:02:11.790742+0000","last_peered":"2024-09-21T06:02:11.790742+0000","last_clean":"2024-09-21T06:02:11.790742+0000","last_became_active":"2024-09-21T06:02:09.260356+0000","last_became_peered":"2024-09-21T06:02:09.260356+0000","last_unstale":"2024-09-21T06:02:11.790742+0000","last_undegraded":"2024-09-21T06:02:11.790742+0000","last_fullsized":"2024-09-21T06:02:11.790742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:17.375769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790661+0000","last_change":"2024-09-21T06:02:10.291888+0000","last_active":"2024-09-21T06:02:11.790661+0000","last_peered":"2024-09-21T06:02:11.790661+0000","last_clean":"2024-09-21T06:02:11.790661+0000","last_became_active":"2024-09-21T06:02:10.279623+0000","last_became_peered":"2024-09-21T06:02:10.279623+0000","last_unstale":"2024-09-21T06:02:11.790661+0000","last_undegraded":"2024-09-21T06:02:11.790661+0000","last_fullsized":"2024-09-21T06:02:11.790661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:55.391528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254634+0000","last_change":"2024-09-21T06:02:09.262422+0000","last_active":"2024-09-21T06:02:11.254634+0000","last_peered":"2024-09-21T06:02:11.254634+0000","last_clean":"2024-09-21T06:02:11.254634+0000","last_became_active":"2024-09-21T06:02:09.262078+0000","last_became_peered":"2024-09-21T06:02:09.262078+0000","last_unstale":"2024-09-21T06:02:11.254634+0000","last_undegraded":"2024-09-21T06:02:11.254634+0000","last_fullsized":"2024-09-21T06:02:11.254634+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:12.662429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791513+0000","last_change":"2024-09-21T06:02:10.282522+0000","last_active":"2024-09-21T06:02:11.791513+0000","last_peered":"2024-09-21T06:02:11.791513+0000","last_clean":"2024-09-21T06:02:11.791513+0000","last_became_active":"2024-09-21T06:02:10.282036+0000","last_became_peered":"2024-09-21T06:02:10.282036+0000","last_unstale":"2024-09-21T06:02:11.791513+0000","last_undegraded":"2024-09-21T06:02:11.791513+0000","last_fullsized":"2024-09-21T06:02:11.791513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:53.258944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253029+0000","last_change":"2024-09-21T06:02:09.275773+0000","last_active":"2024-09-21T06:02:11.253029+0000","last_peered":"2024-09-21T06:02:11.253029+0000","last_clean":"2024-09-21T06:02:11.253029+0000","last_became_active":"2024-09-21T06:02:09.275434+0000","last_became_peered":"2024-09-21T06:02:09.275434+0000","last_unstale":"2024-09-21T06:02:11.253029+0000","last_undegraded":"2024-09-21T06:02:11.253029+0000","last_fullsized":"2024-09-21T06:02:11.253029+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:39.067387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791336+0000","last_change":"2024-09-21T06:02:10.286736+0000","last_active":"2024-09-21T06:02:11.791336+0000","last_peered":"2024-09-21T06:02:11.791336+0000","last_clean":"2024-09-21T06:02:11.791336+0000","last_became_active":"2024-09-21T06:02:10.286341+0000","last_became_peered":"2024-09-21T06:02:10.286341+0000","last_unstale":"2024-09-21T06:02:11.791336+0000","last_undegraded":"2024-09-21T06:02:11.791336+0000","last_fullsized":"2024-09-21T06:02:11.791336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:49.283469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789816+0000","last_change":"2024-09-21T06:02:09.270902+0000","last_active":"2024-09-21T06:02:11.789816+0000","last_peered":"2024-09-21T06:02:11.789816+0000","last_clean":"2024-09-21T06:02:11.789816+0000","last_became_active":"2024-09-21T06:02:09.270573+0000","last_became_peered":"2024-09-21T06:02:09.270573+0000","last_unstale":"2024-09-21T06:02:11.789816+0000","last_undegraded":"2024-09-21T06:02:11.789816+0000","last_fullsized":"2024-09-21T06:02:11.789816+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:15.110078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790233+0000","last_change":"2024-09-21T06:02:10.277306+0000","last_active":"2024-09-21T06:02:11.790233+0000","last_peered":"2024-09-21T06:02:11.790233+0000","last_clean":"2024-09-21T06:02:11.790233+0000","last_became_active":"2024-09-21T06:02:10.270375+0000","last_became_peered":"2024-09-21T06:02:10.270375+0000","last_unstale":"2024-09-21T06:02:11.790233+0000","last_undegraded":"2024-09-21T06:02:11.790233+0000","last_fullsized":"2024-09-21T06:02:11.790233+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:26.447373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253638+0000","last_change":"2024-09-21T06:02:09.266199+0000","last_active":"2024-09-21T06:02:11.253638+0000","last_peered":"2024-09-21T06:02:11.253638+0000","last_clean":"2024-09-21T06:02:11.253638+0000","last_became_active":"2024-09-21T06:02:09.265875+0000","last_became_peered":"2024-09-21T06:02:09.265875+0000","last_unstale":"2024-09-21T06:02:11.253638+0000","last_undegraded":"2024-09-21T06:02:11.253638+0000","last_fullsized":"2024-09-21T06:02:11.253638+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:39.929859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791608+0000","last_change":"2024-09-21T06:02:10.269696+0000","last_active":"2024-09-21T06:02:11.791608+0000","last_peered":"2024-09-21T06:02:11.791608+0000","last_clean":"2024-09-21T06:02:11.791608+0000","last_became_active":"2024-09-21T06:02:10.269223+0000","last_became_peered":"2024-09-21T06:02:10.269223+0000","last_unstale":"2024-09-21T06:02:11.791608+0000","last_undegraded":"2024-09-21T06:02:11.791608+0000","last_fullsized":"2024-09-21T06:02:11.791608+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:57.344419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.252862+0000","last_change":"2024-09-21T06:02:09.266846+0000","last_active":"2024-09-21T06:02:11.252862+0000","last_peered":"2024-09-21T06:02:11.252862+0000","last_clean":"2024-09-21T06:02:11.252862+0000","last_became_active":"2024-09-21T06:02:09.266468+0000","last_became_peered":"2024-09-21T06:02:09.266468+0000","last_unstale":"2024-09-21T06:02:11.252862+0000","last_undegraded":"2024-09-21T06:02:11.252862+0000","last_fullsized":"2024-09-21T06:02:11.252862+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:17.833244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253718+0000","last_change":"2024-09-21T06:02:10.273869+0000","last_active":"2024-09-21T06:02:11.253718+0000","last_peered":"2024-09-21T06:02:11.253718+0000","last_clean":"2024-09-21T06:02:11.253718+0000","last_became_active":"2024-09-21T06:02:10.273564+0000","last_became_peered":"2024-09-21T06:02:10.273564+0000","last_unstale":"2024-09-21T06:02:11.253718+0000","last_undegraded":"2024-09-21T06:02:11.253718+0000","last_fullsized":"2024-09-21T06:02:11.253718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:18.383647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290332+0000","last_change":"2024-09-21T06:02:09.270820+0000","last_active":"2024-09-21T06:02:11.290332+0000","last_peered":"2024-09-21T06:02:11.290332+0000","last_clean":"2024-09-21T06:02:11.290332+0000","last_became_active":"2024-09-21T06:02:09.270500+0000","last_became_peered":"2024-09-21T06:02:09.270500+0000","last_unstale":"2024-09-21T06:02:11.290332+0000","last_undegraded":"2024-09-21T06:02:11.290332+0000","last_fullsized":"2024-09-21T06:02:11.290332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:05.116966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290438+0000","last_change":"2024-09-21T06:02:10.278998+0000","last_active":"2024-09-21T06:02:11.290438+0000","last_peered":"2024-09-21T06:02:11.290438+0000","last_clean":"2024-09-21T06:02:11.290438+0000","last_became_active":"2024-09-21T06:02:10.278700+0000","last_became_peered":"2024-09-21T06:02:10.278700+0000","last_unstale":"2024-09-21T06:02:11.290438+0000","last_undegraded":"2024-09-21T06:02:11.290438+0000","last_fullsized":"2024-09-21T06:02:11.290438+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:44.150713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792239+0000","last_change":"2024-09-21T06:02:09.271907+0000","last_active":"2024-09-21T06:02:11.792239+0000","last_peered":"2024-09-21T06:02:11.792239+0000","last_clean":"2024-09-21T06:02:11.792239+0000","last_became_active":"2024-09-21T06:02:09.271563+0000","last_became_peered":"2024-09-21T06:02:09.271563+0000","last_unstale":"2024-09-21T06:02:11.792239+0000","last_undegraded":"2024-09-21T06:02:11.792239+0000","last_fullsized":"2024-09-21T06:02:11.792239+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:59.323612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790136+0000","last_change":"2024-09-21T06:02:10.272781+0000","last_active":"2024-09-21T06:02:11.790136+0000","last_peered":"2024-09-21T06:02:11.790136+0000","last_clean":"2024-09-21T06:02:11.790136+0000","last_became_active":"2024-09-21T06:02:10.272474+0000","last_became_peered":"2024-09-21T06:02:10.272474+0000","last_unstale":"2024-09-21T06:02:11.790136+0000","last_undegraded":"2024-09-21T06:02:11.790136+0000","last_fullsized":"2024-09-21T06:02:11.790136+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:19.569443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290056+0000","last_change":"2024-09-21T06:02:09.278081+0000","last_active":"2024-09-21T06:02:11.290056+0000","last_peered":"2024-09-21T06:02:11.290056+0000","last_clean":"2024-09-21T06:02:11.290056+0000","last_became_active":"2024-09-21T06:02:09.277726+0000","last_became_peered":"2024-09-21T06:02:09.277726+0000","last_unstale":"2024-09-21T06:02:11.290056+0000","last_undegraded":"2024-09-21T06:02:11.290056+0000","last_fullsized":"2024-09-21T06:02:11.290056+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:09.862737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289912+0000","last_change":"2024-09-21T06:02:10.280615+0000","last_active":"2024-09-21T06:02:11.289912+0000","last_peered":"2024-09-21T06:02:11.289912+0000","last_clean":"2024-09-21T06:02:11.289912+0000","last_became_active":"2024-09-21T06:02:10.280351+0000","last_became_peered":"2024-09-21T06:02:10.280351+0000","last_unstale":"2024-09-21T06:02:11.289912+0000","last_undegraded":"2024-09-21T06:02:11.289912+0000","last_fullsized":"2024-09-21T06:02:11.289912+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:55.704062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":21,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253261+0000","last_change":"2024-09-21T06:02:09.259775+0000","last_active":"2024-09-21T06:02:11.253261+0000","last_peered":"2024-09-21T06:02:11.253261+0000","last_clean":"2024-09-21T06:02:11.253261+0000","last_became_active":"2024-09-21T06:02:09.258709+0000","last_became_peered":"2024-09-21T06:02:09.258709+0000","last_unstale":"2024-09-21T06:02:11.253261+0000","last_undegraded":"2024-09-21T06:02:11.253261+0000","last_fullsized":"2024-09-21T06:02:11.253261+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:25.439494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253660+0000","last_change":"2024-09-21T06:02:10.275691+0000","last_active":"2024-09-21T06:02:11.253660+0000","last_peered":"2024-09-21T06:02:11.253660+0000","last_clean":"2024-09-21T06:02:11.253660+0000","last_became_active":"2024-09-21T06:02:10.275339+0000","last_became_peered":"2024-09-21T06:02:10.275339+0000","last_unstale":"2024-09-21T06:02:11.253660+0000","last_undegraded":"2024-09-21T06:02:11.253660+0000","last_fullsized":"2024-09-21T06:02:11.253660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:43.120115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791428+0000","last_change":"2024-09-21T06:02:09.276297+0000","last_active":"2024-09-21T06:02:11.791428+0000","last_peered":"2024-09-21T06:02:11.791428+0000","last_clean":"2024-09-21T06:02:11.791428+0000","last_became_active":"2024-09-21T06:02:09.275994+0000","last_became_peered":"2024-09-21T06:02:09.275994+0000","last_unstale":"2024-09-21T06:02:11.791428+0000","last_undegraded":"2024-09-21T06:02:11.791428+0000","last_fullsized":"2024-09-21T06:02:11.791428+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:54.696183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289767+0000","last_change":"2024-09-21T06:02:10.276275+0000","last_active":"2024-09-21T06:02:11.289767+0000","last_peered":"2024-09-21T06:02:11.289767+0000","last_clean":"2024-09-21T06:02:11.289767+0000","last_became_active":"2024-09-21T06:02:10.276049+0000","last_became_peered":"2024-09-21T06:02:10.276049+0000","last_unstale":"2024-09-21T06:02:11.289767+0000","last_undegraded":"2024-09-21T06:02:11.289767+0000","last_fullsized":"2024-09-21T06:02:11.289767+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:00.003986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789805+0000","last_change":"2024-09-21T06:02:09.267418+0000","last_active":"2024-09-21T06:02:11.789805+0000","last_peered":"2024-09-21T06:02:11.789805+0000","last_clean":"2024-09-21T06:02:11.789805+0000","last_became_active":"2024-09-21T06:02:09.267111+0000","last_became_peered":"2024-09-21T06:02:09.267111+0000","last_unstale":"2024-09-21T06:02:11.789805+0000","last_undegraded":"2024-09-21T06:02:11.789805+0000","last_fullsized":"2024-09-21T06:02:11.789805+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:10.270750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253090+0000","last_change":"2024-09-21T06:02:10.278157+0000","last_active":"2024-09-21T06:02:11.253090+0000","last_peered":"2024-09-21T06:02:11.253090+0000","last_clean":"2024-09-21T06:02:11.253090+0000","last_became_active":"2024-09-21T06:02:10.271151+0000","last_became_peered":"2024-09-21T06:02:10.271151+0000","last_unstale":"2024-09-21T06:02:11.253090+0000","last_undegraded":"2024-09-21T06:02:11.253090+0000","last_fullsized":"2024-09-21T06:02:11.253090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:09.465555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253777+0000","last_change":"2024-09-21T06:02:09.263864+0000","last_active":"2024-09-21T06:02:11.253777+0000","last_peered":"2024-09-21T06:02:11.253777+0000","last_clean":"2024-09-21T06:02:11.253777+0000","last_became_active":"2024-09-21T06:02:09.263556+0000","last_became_peered":"2024-09-21T06:02:09.263556+0000","last_unstale":"2024-09-21T06:02:11.253777+0000","last_undegraded":"2024-09-21T06:02:11.253777+0000","last_fullsized":"2024-09-21T06:02:11.253777+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:08.457676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289746+0000","last_change":"2024-09-21T06:02:10.287377+0000","last_active":"2024-09-21T06:02:11.289746+0000","last_peered":"2024-09-21T06:02:11.289746+0000","last_clean":"2024-09-21T06:02:11.289746+0000","last_became_active":"2024-09-21T06:02:10.287127+0000","last_became_peered":"2024-09-21T06:02:10.287127+0000","last_unstale":"2024-09-21T06:02:11.289746+0000","last_undegraded":"2024-09-21T06:02:11.289746+0000","last_fullsized":"2024-09-21T06:02:11.289746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:54.355475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289357+0000","last_change":"2024-09-21T06:02:09.267367+0000","last_active":"2024-09-21T06:02:11.289357+0000","last_peered":"2024-09-21T06:02:11.289357+0000","last_clean":"2024-09-21T06:02:11.289357+0000","last_became_active":"2024-09-21T06:02:09.267032+0000","last_became_peered":"2024-09-21T06:02:09.267032+0000","last_unstale":"2024-09-21T06:02:11.289357+0000","last_undegraded":"2024-09-21T06:02:11.289357+0000","last_fullsized":"2024-09-21T06:02:11.289357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:06.608982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290157+0000","last_change":"2024-09-21T06:02:10.275595+0000","last_active":"2024-09-21T06:02:11.290157+0000","last_peered":"2024-09-21T06:02:11.290157+0000","last_clean":"2024-09-21T06:02:11.290157+0000","last_became_active":"2024-09-21T06:02:10.275277+0000","last_became_peered":"2024-09-21T06:02:10.275277+0000","last_unstale":"2024-09-21T06:02:11.290157+0000","last_undegraded":"2024-09-21T06:02:11.290157+0000","last_fullsized":"2024-09-21T06:02:11.290157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:00.331491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792192+0000","last_change":"2024-09-21T06:02:09.274332+0000","last_active":"2024-09-21T06:02:11.792192+0000","last_peered":"2024-09-21T06:02:11.792192+0000","last_clean":"2024-09-21T06:02:11.792192+0000","last_became_active":"2024-09-21T06:02:09.274007+0000","last_became_peered":"2024-09-21T06:02:09.274007+0000","last_unstale":"2024-09-21T06:02:11.792192+0000","last_undegraded":"2024-09-21T06:02:11.792192+0000","last_fullsized":"2024-09-21T06:02:11.792192+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:43.142834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254114+0000","last_change":"2024-09-21T06:02:10.276861+0000","last_active":"2024-09-21T06:02:11.254114+0000","last_peered":"2024-09-21T06:02:11.254114+0000","last_clean":"2024-09-21T06:02:11.254114+0000","last_became_active":"2024-09-21T06:02:10.276566+0000","last_became_peered":"2024-09-21T06:02:10.276566+0000","last_unstale":"2024-09-21T06:02:11.254114+0000","last_undegraded":"2024-09-21T06:02:11.254114+0000","last_fullsized":"2024-09-21T06:02:11.254114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:32.351750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790419+0000","last_change":"2024-09-21T06:02:09.272265+0000","last_active":"2024-09-21T06:02:11.790419+0000","last_peered":"2024-09-21T06:02:11.790419+0000","last_clean":"2024-09-21T06:02:11.790419+0000","last_became_active":"2024-09-21T06:02:09.268092+0000","last_became_peered":"2024-09-21T06:02:09.268092+0000","last_unstale":"2024-09-21T06:02:11.790419+0000","last_undegraded":"2024-09-21T06:02:11.790419+0000","last_fullsized":"2024-09-21T06:02:11.790419+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:00.089511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791877+0000","last_change":"2024-09-21T06:02:10.275046+0000","last_active":"2024-09-21T06:02:11.791877+0000","last_peered":"2024-09-21T06:02:11.791877+0000","last_clean":"2024-09-21T06:02:11.791877+0000","last_became_active":"2024-09-21T06:02:10.274759+0000","last_became_peered":"2024-09-21T06:02:10.274759+0000","last_unstale":"2024-09-21T06:02:11.791877+0000","last_undegraded":"2024-09-21T06:02:11.791877+0000","last_fullsized":"2024-09-21T06:02:11.791877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:17.491964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253777+0000","last_change":"2024-09-21T06:02:09.268844+0000","last_active":"2024-09-21T06:02:11.253777+0000","last_peered":"2024-09-21T06:02:11.253777+0000","last_clean":"2024-09-21T06:02:11.253777+0000","last_became_active":"2024-09-21T06:02:09.268279+0000","last_became_peered":"2024-09-21T06:02:09.268279+0000","last_unstale":"2024-09-21T06:02:11.253777+0000","last_undegraded":"2024-09-21T06:02:11.253777+0000","last_fullsized":"2024-09-21T06:02:11.253777+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:43.064730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290070+0000","last_change":"2024-09-21T06:02:10.283907+0000","last_active":"2024-09-21T06:02:11.290070+0000","last_peered":"2024-09-21T06:02:11.290070+0000","last_clean":"2024-09-21T06:02:11.290070+0000","last_became_active":"2024-09-21T06:02:10.282734+0000","last_became_peered":"2024-09-21T06:02:10.282734+0000","last_unstale":"2024-09-21T06:02:11.290070+0000","last_undegraded":"2024-09-21T06:02:11.290070+0000","last_fullsized":"2024-09-21T06:02:11.290070+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:03.263466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790269+0000","last_change":"2024-09-21T06:02:09.269326+0000","last_active":"2024-09-21T06:02:11.790269+0000","last_peered":"2024-09-21T06:02:11.790269+0000","last_clean":"2024-09-21T06:02:11.790269+0000","last_became_active":"2024-09-21T06:02:09.267680+0000","last_became_peered":"2024-09-21T06:02:09.267680+0000","last_unstale":"2024-09-21T06:02:11.790269+0000","last_undegraded":"2024-09-21T06:02:11.790269+0000","last_fullsized":"2024-09-21T06:02:11.790269+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:22.358250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253933+0000","last_change":"2024-09-21T06:02:10.275681+0000","last_active":"2024-09-21T06:02:11.253933+0000","last_peered":"2024-09-21T06:02:11.253933+0000","last_clean":"2024-09-21T06:02:11.253933+0000","last_became_active":"2024-09-21T06:02:10.275228+0000","last_became_peered":"2024-09-21T06:02:10.275228+0000","last_unstale":"2024-09-21T06:02:11.253933+0000","last_undegraded":"2024-09-21T06:02:11.253933+0000","last_fullsized":"2024-09-21T06:02:11.253933+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:00.003986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791802+0000","last_change":"2024-09-21T06:02:09.266422+0000","last_active":"2024-09-21T06:02:11.791802+0000","last_peered":"2024-09-21T06:02:11.791802+0000","last_clean":"2024-09-21T06:02:11.791802+0000","last_became_active":"2024-09-21T06:02:09.266121+0000","last_became_peered":"2024-09-21T06:02:09.266121+0000","last_unstale":"2024-09-21T06:02:11.791802+0000","last_undegraded":"2024-09-21T06:02:11.791802+0000","last_fullsized":"2024-09-21T06:02:11.791802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:09.638560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790006+0000","last_change":"2024-09-21T06:02:10.265400+0000","last_active":"2024-09-21T06:02:11.790006+0000","last_peered":"2024-09-21T06:02:11.790006+0000","last_clean":"2024-09-21T06:02:11.790006+0000","last_became_active":"2024-09-21T06:02:10.263526+0000","last_became_peered":"2024-09-21T06:02:10.263526+0000","last_unstale":"2024-09-21T06:02:11.790006+0000","last_undegraded":"2024-09-21T06:02:11.790006+0000","last_fullsized":"2024-09-21T06:02:11.790006+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:48.448199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789710+0000","last_change":"2024-09-21T06:02:09.271164+0000","last_active":"2024-09-21T06:02:11.789710+0000","last_peered":"2024-09-21T06:02:11.789710+0000","last_clean":"2024-09-21T06:02:11.789710+0000","last_became_active":"2024-09-21T06:02:09.270848+0000","last_became_peered":"2024-09-21T06:02:09.270848+0000","last_unstale":"2024-09-21T06:02:11.789710+0000","last_undegraded":"2024-09-21T06:02:11.789710+0000","last_fullsized":"2024-09-21T06:02:11.789710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:58.592636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789752+0000","last_change":"2024-09-21T06:02:10.265812+0000","last_active":"2024-09-21T06:02:11.789752+0000","last_peered":"2024-09-21T06:02:11.789752+0000","last_clean":"2024-09-21T06:02:11.789752+0000","last_became_active":"2024-09-21T06:02:10.265514+0000","last_became_peered":"2024-09-21T06:02:10.265514+0000","last_unstale":"2024-09-21T06:02:11.789752+0000","last_undegraded":"2024-09-21T06:02:11.789752+0000","last_fullsized":"2024-09-21T06:02:11.789752+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:19.605248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.252801+0000","last_change":"2024-09-21T06:02:09.267046+0000","last_active":"2024-09-21T06:02:11.252801+0000","last_peered":"2024-09-21T06:02:11.252801+0000","last_clean":"2024-09-21T06:02:11.252801+0000","last_became_active":"2024-09-21T06:02:09.266750+0000","last_became_peered":"2024-09-21T06:02:09.266750+0000","last_unstale":"2024-09-21T06:02:11.252801+0000","last_undegraded":"2024-09-21T06:02:11.252801+0000","last_fullsized":"2024-09-21T06:02:11.252801+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:48.018613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254310+0000","last_change":"2024-09-21T06:02:10.261217+0000","last_active":"2024-09-21T06:02:11.254310+0000","last_peered":"2024-09-21T06:02:11.254310+0000","last_clean":"2024-09-21T06:02:11.254310+0000","last_became_active":"2024-09-21T06:02:10.260828+0000","last_became_peered":"2024-09-21T06:02:10.260828+0000","last_unstale":"2024-09-21T06:02:11.254310+0000","last_undegraded":"2024-09-21T06:02:11.254310+0000","last_fullsized":"2024-09-21T06:02:11.254310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:55.704062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790736+0000","last_change":"2024-09-21T06:02:09.275643+0000","last_active":"2024-09-21T06:02:11.790736+0000","last_peered":"2024-09-21T06:02:11.790736+0000","last_clean":"2024-09-21T06:02:11.790736+0000","last_became_active":"2024-09-21T06:02:09.275319+0000","last_became_peered":"2024-09-21T06:02:09.275319+0000","last_unstale":"2024-09-21T06:02:11.790736+0000","last_undegraded":"2024-09-21T06:02:11.790736+0000","last_fullsized":"2024-09-21T06:02:11.790736+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:46.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254321+0000","last_change":"2024-09-21T06:02:10.270206+0000","last_active":"2024-09-21T06:02:11.254321+0000","last_peered":"2024-09-21T06:02:11.254321+0000","last_clean":"2024-09-21T06:02:11.254321+0000","last_became_active":"2024-09-21T06:02:10.269531+0000","last_became_peered":"2024-09-21T06:02:10.269531+0000","last_unstale":"2024-09-21T06:02:11.254321+0000","last_undegraded":"2024-09-21T06:02:11.254321+0000","last_fullsized":"2024-09-21T06:02:11.254321+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:10.646439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790569+0000","last_change":"2024-09-21T06:02:09.272606+0000","last_active":"2024-09-21T06:02:11.790569+0000","last_peered":"2024-09-21T06:02:11.790569+0000","last_clean":"2024-09-21T06:02:11.790569+0000","last_became_active":"2024-09-21T06:02:09.271935+0000","last_became_peered":"2024-09-21T06:02:09.271935+0000","last_unstale":"2024-09-21T06:02:11.790569+0000","last_undegraded":"2024-09-21T06:02:11.790569+0000","last_fullsized":"2024-09-21T06:02:11.790569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:53.780735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790403+0000","last_change":"2024-09-21T06:02:10.262950+0000","last_active":"2024-09-21T06:02:11.790403+0000","last_peered":"2024-09-21T06:02:11.790403+0000","last_clean":"2024-09-21T06:02:11.790403+0000","last_became_active":"2024-09-21T06:02:10.262636+0000","last_became_peered":"2024-09-21T06:02:10.262636+0000","last_unstale":"2024-09-21T06:02:11.790403+0000","last_undegraded":"2024-09-21T06:02:11.790403+0000","last_fullsized":"2024-09-21T06:02:11.790403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:32.505486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789909+0000","last_change":"2024-09-21T06:02:09.267841+0000","last_active":"2024-09-21T06:02:11.789909+0000","last_peered":"2024-09-21T06:02:11.789909+0000","last_clean":"2024-09-21T06:02:11.789909+0000","last_became_active":"2024-09-21T06:02:09.267444+0000","last_became_peered":"2024-09-21T06:02:09.267444+0000","last_unstale":"2024-09-21T06:02:11.789909+0000","last_undegraded":"2024-09-21T06:02:11.789909+0000","last_fullsized":"2024-09-21T06:02:11.789909+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:33.721188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790709+0000","last_change":"2024-09-21T06:02:10.263121+0000","last_active":"2024-09-21T06:02:11.790709+0000","last_peered":"2024-09-21T06:02:11.790709+0000","last_clean":"2024-09-21T06:02:11.790709+0000","last_became_active":"2024-09-21T06:02:10.262809+0000","last_became_peered":"2024-09-21T06:02:10.262809+0000","last_unstale":"2024-09-21T06:02:11.790709+0000","last_undegraded":"2024-09-21T06:02:11.790709+0000","last_fullsized":"2024-09-21T06:02:11.790709+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:53.926538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254710+0000","last_change":"2024-09-21T06:02:09.263354+0000","last_active":"2024-09-21T06:02:11.254710+0000","last_peered":"2024-09-21T06:02:11.254710+0000","last_clean":"2024-09-21T06:02:11.254710+0000","last_became_active":"2024-09-21T06:02:09.263051+0000","last_became_peered":"2024-09-21T06:02:09.263051+0000","last_unstale":"2024-09-21T06:02:11.254710+0000","last_undegraded":"2024-09-21T06:02:11.254710+0000","last_fullsized":"2024-09-21T06:02:11.254710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:15.110078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791787+0000","last_change":"2024-09-21T06:02:10.280540+0000","last_active":"2024-09-21T06:02:11.791787+0000","last_peered":"2024-09-21T06:02:11.791787+0000","last_clean":"2024-09-21T06:02:11.791787+0000","last_became_active":"2024-09-21T06:02:10.274182+0000","last_became_peered":"2024-09-21T06:02:10.274182+0000","last_unstale":"2024-09-21T06:02:11.791787+0000","last_undegraded":"2024-09-21T06:02:11.791787+0000","last_fullsized":"2024-09-21T06:02:11.791787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:38.369387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790058+0000","last_change":"2024-09-21T06:02:09.263348+0000","last_active":"2024-09-21T06:02:11.790058+0000","last_peered":"2024-09-21T06:02:11.790058+0000","last_clean":"2024-09-21T06:02:11.790058+0000","last_became_active":"2024-09-21T06:02:09.262942+0000","last_became_peered":"2024-09-21T06:02:09.262942+0000","last_unstale":"2024-09-21T06:02:11.790058+0000","last_undegraded":"2024-09-21T06:02:11.790058+0000","last_fullsized":"2024-09-21T06:02:11.790058+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:13.084051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290121+0000","last_change":"2024-09-21T06:02:10.288104+0000","last_active":"2024-09-21T06:02:11.290121+0000","last_peered":"2024-09-21T06:02:11.290121+0000","last_clean":"2024-09-21T06:02:11.290121+0000","last_became_active":"2024-09-21T06:02:10.287460+0000","last_became_peered":"2024-09-21T06:02:10.287460+0000","last_unstale":"2024-09-21T06:02:11.290121+0000","last_undegraded":"2024-09-21T06:02:11.290121+0000","last_fullsized":"2024-09-21T06:02:11.290121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:00.353554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253392+0000","last_change":"2024-09-21T06:02:09.259123+0000","last_active":"2024-09-21T06:02:11.253392+0000","last_peered":"2024-09-21T06:02:11.253392+0000","last_clean":"2024-09-21T06:02:11.253392+0000","last_became_active":"2024-09-21T06:02:09.258760+0000","last_became_peered":"2024-09-21T06:02:09.258760+0000","last_unstale":"2024-09-21T06:02:11.253392+0000","last_undegraded":"2024-09-21T06:02:11.253392+0000","last_fullsized":"2024-09-21T06:02:11.253392+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:42.407138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790834+0000","last_change":"2024-09-21T06:02:10.261037+0000","last_active":"2024-09-21T06:02:11.790834+0000","last_peered":"2024-09-21T06:02:11.790834+0000","last_clean":"2024-09-21T06:02:11.790834+0000","last_became_active":"2024-09-21T06:02:10.260729+0000","last_became_peered":"2024-09-21T06:02:10.260729+0000","last_unstale":"2024-09-21T06:02:11.790834+0000","last_undegraded":"2024-09-21T06:02:11.790834+0000","last_fullsized":"2024-09-21T06:02:11.790834+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:00.468017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789596+0000","last_change":"2024-09-21T06:02:09.264048+0000","last_active":"2024-09-21T06:02:11.789596+0000","last_peered":"2024-09-21T06:02:11.789596+0000","last_clean":"2024-09-21T06:02:11.789596+0000","last_became_active":"2024-09-21T06:02:09.263708+0000","last_became_peered":"2024-09-21T06:02:09.263708+0000","last_unstale":"2024-09-21T06:02:11.789596+0000","last_undegraded":"2024-09-21T06:02:11.789596+0000","last_fullsized":"2024-09-21T06:02:11.789596+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:25.531925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791392+0000","last_change":"2024-09-21T06:02:10.262498+0000","last_active":"2024-09-21T06:02:11.791392+0000","last_peered":"2024-09-21T06:02:11.791392+0000","last_clean":"2024-09-21T06:02:11.791392+0000","last_became_active":"2024-09-21T06:02:10.262000+0000","last_became_peered":"2024-09-21T06:02:10.262000+0000","last_unstale":"2024-09-21T06:02:11.791392+0000","last_undegraded":"2024-09-21T06:02:11.791392+0000","last_fullsized":"2024-09-21T06:02:11.791392+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:39.062283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791682+0000","last_change":"2024-09-21T06:02:09.265629+0000","last_active":"2024-09-21T06:02:11.791682+0000","last_peered":"2024-09-21T06:02:11.791682+0000","last_clean":"2024-09-21T06:02:11.791682+0000","last_became_active":"2024-09-21T06:02:09.265255+0000","last_became_peered":"2024-09-21T06:02:09.265255+0000","last_unstale":"2024-09-21T06:02:11.791682+0000","last_undegraded":"2024-09-21T06:02:11.791682+0000","last_fullsized":"2024-09-21T06:02:11.791682+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:02.255587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791518+0000","last_change":"2024-09-21T06:02:10.283113+0000","last_active":"2024-09-21T06:02:11.791518+0000","last_peered":"2024-09-21T06:02:11.791518+0000","last_clean":"2024-09-21T06:02:11.791518+0000","last_became_active":"2024-09-21T06:02:10.282826+0000","last_became_peered":"2024-09-21T06:02:10.282826+0000","last_unstale":"2024-09-21T06:02:11.791518+0000","last_undegraded":"2024-09-21T06:02:11.791518+0000","last_fullsized":"2024-09-21T06:02:11.791518+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:57.773356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789812+0000","last_change":"2024-09-21T06:02:09.272885+0000","last_active":"2024-09-21T06:02:11.789812+0000","last_peered":"2024-09-21T06:02:11.789812+0000","last_clean":"2024-09-21T06:02:11.789812+0000","last_became_active":"2024-09-21T06:02:09.272520+0000","last_became_peered":"2024-09-21T06:02:09.272520+0000","last_unstale":"2024-09-21T06:02:11.789812+0000","last_undegraded":"2024-09-21T06:02:11.789812+0000","last_fullsized":"2024-09-21T06:02:11.789812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:37.738718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":180,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289489+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:02:11.289489+0000","last_peered":"2024-09-21T06:02:11.289489+0000","last_clean":"2024-09-21T06:02:11.289489+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:02:11.289489+0000","last_undegraded":"2024-09-21T06:02:11.289489+0000","last_fullsized":"2024-09-21T06:02:11.289489+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509514,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40788,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331052,"statfs":{"total":96636764160,"available":96594997248,"internally_reserved":0,"allocated":1302528,"data_stored":880639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509514,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45764,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326076,"statfs":{"total":96636764160,"available":96589901824,"internally_reserved":0,"allocated":700416,"data_stored":290111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509513,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":708608,"data_stored":290763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62416,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309424,"statfs":{"total":96636764160,"available":96572850176,"internally_reserved":0,"allocated":712704,"data_stored":291723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45772,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326068,"statfs":{"total":96636764160,"available":96589893632,"internally_reserved":0,"allocated":708608,"data_stored":290167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509515,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":708608,"data_stored":290167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":1310720,"data_stored":882673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542218,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49928,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321912,"statfs":{"total":96636764160,"available":96585637888,"internally_reserved":0,"allocated":704512,"data_stored":290201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:02:15.340 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:02:15.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.536+0000 7fab47400640 1 -- 172.21.15.50:0/128540320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28003c20 msgr2=0x7fab28004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.536+0000 7fab47400640 1 --2- 172.21.15.50:0/128540320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28003c20 0x7fab28004070 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fab34001860 tx=0x7fab3401c8b0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.536+0000 7fab45400640 1 -- 172.21.15.50:0/128540320 reap_dead start 2024-09-21T06:02:15.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.536+0000 7fab47400640 1 -- 172.21.15.50:0/128540320 shutdown_connections 2024-09-21T06:02:15.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.536+0000 7fab47400640 1 -- 172.21.15.50:0/128540320 >> 172.21.15.50:0/128540320 conn(0x7fab400b2040 msgr2=0x7fab400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:15.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.536+0000 7fab47400640 1 -- 172.21.15.50:0/128540320 shutdown_connections 2024-09-21T06:02:15.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.536+0000 7fab47400640 1 -- 172.21.15.50:0/128540320 wait complete. 2024-09-21T06:02:15.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.536+0000 7fab47400640 1 Processor -- start 2024-09-21T06:02:15.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.536+0000 7fab47400640 1 -- start start 2024-09-21T06:02:15.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab47400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab28003c20 0x7fab40144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:15.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab47400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab401571d0 0x7fab400b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:15.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab47400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab4014f600 0x7fab400b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:15.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab47400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fab40167bc0 con 0x7fab4014f600 2024-09-21T06:02:15.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab47400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fab40167a40 con 0x7fab28003c20 2024-09-21T06:02:15.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab47400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fab40167d40 con 0x7fab401571d0 2024-09-21T06:02:15.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab45400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab4014f600 0x7fab400b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:15.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab45400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab4014f600 0x7fab400b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48238/0 (socket says 172.21.15.50:48238) 2024-09-21T06:02:15.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab45400640 1 -- 172.21.15.50:0/3000983912 learned_addr learned my addr 172.21.15.50:0/3000983912 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:15.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab44a00640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab28003c20 0x7fab40144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:15.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab3fe00640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab401571d0 0x7fab400b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:15.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab45400640 1 -- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab401571d0 msgr2=0x7fab400b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab45400640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab401571d0 0x7fab400b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab45400640 1 -- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab28003c20 msgr2=0x7fab40144cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab45400640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab28003c20 0x7fab40144cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab45400640 1 -- 172.21.15.50:0/3000983912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab40151630 con 0x7fab4014f600 2024-09-21T06:02:15.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab45400640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab4014f600 0x7fab400b08d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fab40073060 tx=0x7fab3800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:15.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab3cc00640 1 -- 172.21.15.50:0/3000983912 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38010890 con 0x7fab4014f600 2024-09-21T06:02:15.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab3cc00640 1 -- 172.21.15.50:0/3000983912 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab38018070 con 0x7fab4014f600 2024-09-21T06:02:15.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab3cc00640 1 -- 172.21.15.50:0/3000983912 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38010de0 con 0x7fab4014f600 2024-09-21T06:02:15.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab47400640 1 -- 172.21.15.50:0/3000983912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab40151aa0 con 0x7fab4014f600 2024-09-21T06:02:15.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab47400640 1 -- 172.21.15.50:0/3000983912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab40155800 con 0x7fab4014f600 2024-09-21T06:02:15.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.540+0000 7fab47400640 1 -- 172.21.15.50:0/3000983912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab40074e30 con 0x7fab4014f600 2024-09-21T06:02:15.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.544+0000 7fab3cc00640 1 -- 172.21.15.50:0/3000983912 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fab3801d460 con 0x7fab4014f600 2024-09-21T06:02:15.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.544+0000 7fab3cc00640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fab20073b00 0x7fab20076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:15.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.544+0000 7fab3cc00640 1 -- 172.21.15.50:0/3000983912 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fab38092dd0 con 0x7fab4014f600 2024-09-21T06:02:15.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.544+0000 7fab44a00640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fab20073b00 0x7fab20076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:15.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.544+0000 7fab44a00640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fab20073b00 0x7fab20076010 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fab34001830 tx=0x7fab340196f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:15.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.548+0000 7fab3cc00640 1 -- 172.21.15.50:0/3000983912 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fab3805f8d0 con 0x7fab4014f600 2024-09-21T06:02:15.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.792+0000 7fab47400640 1 -- 172.21.15.50:0/3000983912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fab40155a00 con 0x7fab4014f600 2024-09-21T06:02:15.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.792+0000 7fab3cc00640 1 -- 172.21.15.50:0/3000983912 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1056 (secure 0 0 0) 0x7fab3805f1d0 con 0x7fab4014f600 2024-09-21T06:02:15.795 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:15.796 INFO:teuthology.orchestra.run.smithi050.stdout:{"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":372,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":8,"num_up_osds":8,"osd_up_since":1726898177,"num_in_osds":8,"osd_in_since":1726898167,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":367575040,"bytes_avail":772726538240,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":36,"btime":"2024-09-21T06:02:13:274179+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":5695}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-21T06:00:46.952552+0000","services":{}},"progress_events":{}} 2024-09-21T06:02:15.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.796+0000 7fab47400640 1 -- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fab20073b00 msgr2=0x7fab20076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.796+0000 7fab47400640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fab20073b00 0x7fab20076010 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fab34001830 tx=0x7fab340196f0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.796+0000 7fab47400640 1 -- 172.21.15.50:0/3000983912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab4014f600 msgr2=0x7fab400b08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.796+0000 7fab47400640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab4014f600 0x7fab400b08d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fab40073060 tx=0x7fab3800e970 comp rx=0 tx=0).stop 2024-09-21T06:02:15.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.796+0000 7fab47400640 1 -- 172.21.15.50:0/3000983912 shutdown_connections 2024-09-21T06:02:15.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.796+0000 7fab47400640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fab20073b00 0x7fab20076010 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.796+0000 7fab47400640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab4014f600 0x7fab400b08d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.796+0000 7fab47400640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab401571d0 0x7fab400b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.796+0000 7fab47400640 1 --2- 172.21.15.50:0/3000983912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab28003c20 0x7fab40144cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.796+0000 7fab47400640 1 -- 172.21.15.50:0/3000983912 >> 172.21.15.50:0/3000983912 conn(0x7fab400b2040 msgr2=0x7fab400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:15.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.796+0000 7fab47400640 1 -- 172.21.15.50:0/3000983912 shutdown_connections 2024-09-21T06:02:15.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.796+0000 7fab47400640 1 -- 172.21.15.50:0/3000983912 wait complete. 2024-09-21T06:02:15.818 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T06:02:15.819 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:02:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.976+0000 7fe72be00640 1 -- 172.21.15.50:0/146177496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7240703a0 msgr2=0x7fe72406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.976+0000 7fe72be00640 1 --2- 172.21.15.50:0/146177496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7240703a0 0x7fe72406d200 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009bc0 tx=0x7fe71c01cb10 comp rx=0 tx=0).stop 2024-09-21T06:02:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.976+0000 7fe72be00640 1 -- 172.21.15.50:0/146177496 shutdown_connections 2024-09-21T06:02:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.976+0000 7fe72be00640 1 --2- 172.21.15.50:0/146177496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7240703a0 0x7fe72406d200 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.976+0000 7fe72be00640 1 -- 172.21.15.50:0/146177496 >> 172.21.15.50:0/146177496 conn(0x7fe7240b35f0 msgr2=0x7fe7240b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.976+0000 7fe72be00640 1 -- 172.21.15.50:0/146177496 shutdown_connections 2024-09-21T06:02:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.976+0000 7fe72be00640 1 -- 172.21.15.50:0/146177496 wait complete. 2024-09-21T06:02:15.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.976+0000 7fe72be00640 1 Processor -- start 2024-09-21T06:02:15.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.976+0000 7fe72be00640 1 -- start start 2024-09-21T06:02:15.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe72be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe72406cdf0 0x7fe72415aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:15.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe72be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7240703a0 0x7fe724157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:15.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe72be00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe724159130 0x7fe7241fa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:15.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe72be00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe72416a470 con 0x7fe7240703a0 2024-09-21T06:02:15.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe72be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe72416a2f0 con 0x7fe72406cdf0 2024-09-21T06:02:15.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe72be00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe72416a5f0 con 0x7fe724159130 2024-09-21T06:02:15.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe728a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7240703a0 0x7fe724157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:15.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe728a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7240703a0 0x7fe724157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48244/0 (socket says 172.21.15.50:48244) 2024-09-21T06:02:15.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe728a00640 1 -- 172.21.15.50:0/2972694343 learned_addr learned my addr 172.21.15.50:0/2972694343 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:15.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe729400640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe72406cdf0 0x7fe72415aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:15.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe729e00640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe724159130 0x7fe7241fa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:15.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe728a00640 1 -- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe724159130 msgr2=0x7fe7241fa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe728a00640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe724159130 0x7fe7241fa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe728a00640 1 -- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe72406cdf0 msgr2=0x7fe72415aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:15.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe728a00640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe72406cdf0 0x7fe72415aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:15.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe728a00640 1 -- 172.21.15.50:0/2972694343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7241fa0b0 con 0x7fe7240703a0 2024-09-21T06:02:15.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe728a00640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7240703a0 0x7fe724157110 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe71000c430 tx=0x7fe71000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:15.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe721600640 1 -- 172.21.15.50:0/2972694343 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710011790 con 0x7fe7240703a0 2024-09-21T06:02:15.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe721600640 1 -- 172.21.15.50:0/2972694343 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe710017070 con 0x7fe7240703a0 2024-09-21T06:02:15.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe72be00640 1 -- 172.21.15.50:0/2972694343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72415a3d0 con 0x7fe7240703a0 2024-09-21T06:02:15.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe721600640 1 -- 172.21.15.50:0/2972694343 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710011ce0 con 0x7fe7240703a0 2024-09-21T06:02:15.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe72be00640 1 -- 172.21.15.50:0/2972694343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72415a990 con 0x7fe7240703a0 2024-09-21T06:02:15.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.980+0000 7fe72be00640 1 -- 172.21.15.50:0/2972694343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72415a1c0 con 0x7fe7240703a0 2024-09-21T06:02:15.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.984+0000 7fe721600640 1 -- 172.21.15.50:0/2972694343 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fe71001c460 con 0x7fe7240703a0 2024-09-21T06:02:15.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.984+0000 7fe721600640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe704073b00 0x7fe704076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:15.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.984+0000 7fe721600640 1 -- 172.21.15.50:0/2972694343 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fe710093360 con 0x7fe7240703a0 2024-09-21T06:02:15.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.984+0000 7fe729400640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe704073b00 0x7fe704076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:15.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.984+0000 7fe729400640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe704073b00 0x7fe704076010 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe718007fd0 tx=0x7fe718009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:15.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:15.988+0000 7fe721600640 1 -- 172.21.15.50:0/2972694343 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe71005f840 con 0x7fe7240703a0 2024-09-21T06:02:16.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.160+0000 7fe72be00640 1 -- 172.21.15.50:0/2972694343 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe72406b540 con 0x7fe704073b00 2024-09-21T06:02:16.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.168+0000 7fe721600640 1 -- 172.21.15.50:0/2972694343 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251943 (secure 0 0 0) 0x7fe718001d20 con 0x7fe704073b00 2024-09-21T06:02:16.172 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:16.173 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:02:16.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.172+0000 7fe72be00640 1 -- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe704073b00 msgr2=0x7fe704076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:16.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.172+0000 7fe72be00640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe704073b00 0x7fe704076010 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe718007fd0 tx=0x7fe718009040 comp rx=0 tx=0).stop 2024-09-21T06:02:16.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.172+0000 7fe72be00640 1 -- 172.21.15.50:0/2972694343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7240703a0 msgr2=0x7fe724157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:16.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.172+0000 7fe72be00640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7240703a0 0x7fe724157110 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe71000c430 tx=0x7fe71000c8f0 comp rx=0 tx=0).stop 2024-09-21T06:02:16.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.172+0000 7fe72be00640 1 -- 172.21.15.50:0/2972694343 shutdown_connections 2024-09-21T06:02:16.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.172+0000 7fe72be00640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe724159130 0x7fe7241fa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:16.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.172+0000 7fe72be00640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe704073b00 0x7fe704076010 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:16.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.172+0000 7fe72be00640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7240703a0 0x7fe724157110 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:16.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.172+0000 7fe72be00640 1 --2- 172.21.15.50:0/2972694343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe72406cdf0 0x7fe72415aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:16.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.172+0000 7fe72be00640 1 -- 172.21.15.50:0/2972694343 >> 172.21.15.50:0/2972694343 conn(0x7fe7240b35f0 msgr2=0x7fe7240b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:16.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.172+0000 7fe72be00640 1 -- 172.21.15.50:0/2972694343 shutdown_connections 2024-09-21T06:02:16.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:16.172+0000 7fe72be00640 1 -- 172.21.15.50:0/2972694343 wait complete. 2024-09-21T06:02:16.194 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-21T06:02:14.984316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":358960,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":352051,"kb_avail":754615760,"statfs":{"total":773094113280,"available":772726538240,"internally_reserved":0,"allocated":6856704,"data_stored":3506444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":360500784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001742"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250964+0000","last_change":"2024-09-21T06:02:10.283497+0000","last_active":"2024-09-21T06:02:11.250964+0000","last_peered":"2024-09-21T06:02:11.250964+0000","last_clean":"2024-09-21T06:02:11.250964+0000","last_became_active":"2024-09-21T06:02:10.283183+0000","last_became_peered":"2024-09-21T06:02:10.283183+0000","last_unstale":"2024-09-21T06:02:11.250964+0000","last_undegraded":"2024-09-21T06:02:11.250964+0000","last_fullsized":"2024-09-21T06:02:11.250964+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:50.423813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792195+0000","last_change":"2024-09-21T06:02:10.285494+0000","last_active":"2024-09-21T06:02:11.792195+0000","last_peered":"2024-09-21T06:02:11.792195+0000","last_clean":"2024-09-21T06:02:11.792195+0000","last_became_active":"2024-09-21T06:02:10.285121+0000","last_became_peered":"2024-09-21T06:02:10.285121+0000","last_unstale":"2024-09-21T06:02:11.792195+0000","last_undegraded":"2024-09-21T06:02:11.792195+0000","last_fullsized":"2024-09-21T06:02:11.792195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:32.261338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289739+0000","last_change":"2024-09-21T06:02:10.277961+0000","last_active":"2024-09-21T06:02:11.289739+0000","last_peered":"2024-09-21T06:02:11.289739+0000","last_clean":"2024-09-21T06:02:11.289739+0000","last_became_active":"2024-09-21T06:02:10.277639+0000","last_became_peered":"2024-09-21T06:02:10.277639+0000","last_unstale":"2024-09-21T06:02:11.289739+0000","last_undegraded":"2024-09-21T06:02:11.289739+0000","last_fullsized":"2024-09-21T06:02:11.289739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:52.411453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254579+0000","last_change":"2024-09-21T06:02:10.267778+0000","last_active":"2024-09-21T06:02:11.254579+0000","last_peered":"2024-09-21T06:02:11.254579+0000","last_clean":"2024-09-21T06:02:11.254579+0000","last_became_active":"2024-09-21T06:02:10.263807+0000","last_became_peered":"2024-09-21T06:02:10.263807+0000","last_unstale":"2024-09-21T06:02:11.254579+0000","last_undegraded":"2024-09-21T06:02:11.254579+0000","last_fullsized":"2024-09-21T06:02:11.254579+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:00.353554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250565+0000","last_change":"2024-09-21T06:02:10.268540+0000","last_active":"2024-09-21T06:02:11.250565+0000","last_peered":"2024-09-21T06:02:11.250565+0000","last_clean":"2024-09-21T06:02:11.250565+0000","last_became_active":"2024-09-21T06:02:10.267456+0000","last_became_peered":"2024-09-21T06:02:10.267456+0000","last_unstale":"2024-09-21T06:02:11.250565+0000","last_undegraded":"2024-09-21T06:02:11.250565+0000","last_fullsized":"2024-09-21T06:02:11.250565+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:14.203778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791022+0000","last_change":"2024-09-21T06:02:10.279996+0000","last_active":"2024-09-21T06:02:11.791022+0000","last_peered":"2024-09-21T06:02:11.791022+0000","last_clean":"2024-09-21T06:02:11.791022+0000","last_became_active":"2024-09-21T06:02:10.279672+0000","last_became_peered":"2024-09-21T06:02:10.279672+0000","last_unstale":"2024-09-21T06:02:11.791022+0000","last_undegraded":"2024-09-21T06:02:11.791022+0000","last_fullsized":"2024-09-21T06:02:11.791022+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:51.112377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253169+0000","last_change":"2024-09-21T06:02:10.267014+0000","last_active":"2024-09-21T06:02:11.253169+0000","last_peered":"2024-09-21T06:02:11.253169+0000","last_clean":"2024-09-21T06:02:11.253169+0000","last_became_active":"2024-09-21T06:02:10.266091+0000","last_became_peered":"2024-09-21T06:02:10.266091+0000","last_unstale":"2024-09-21T06:02:11.253169+0000","last_undegraded":"2024-09-21T06:02:11.253169+0000","last_fullsized":"2024-09-21T06:02:11.253169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:44.072609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791264+0000","last_change":"2024-09-21T06:02:10.275841+0000","last_active":"2024-09-21T06:02:11.791264+0000","last_peered":"2024-09-21T06:02:11.791264+0000","last_clean":"2024-09-21T06:02:11.791264+0000","last_became_active":"2024-09-21T06:02:10.275526+0000","last_became_peered":"2024-09-21T06:02:10.275526+0000","last_unstale":"2024-09-21T06:02:11.791264+0000","last_undegraded":"2024-09-21T06:02:11.791264+0000","last_fullsized":"2024-09-21T06:02:11.791264+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:42.101108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253587+0000","last_change":"2024-09-21T06:02:10.293470+0000","last_active":"2024-09-21T06:02:11.253587+0000","last_peered":"2024-09-21T06:02:11.253587+0000","last_clean":"2024-09-21T06:02:11.253587+0000","last_became_active":"2024-09-21T06:02:10.292993+0000","last_became_peered":"2024-09-21T06:02:10.292993+0000","last_unstale":"2024-09-21T06:02:11.253587+0000","last_undegraded":"2024-09-21T06:02:11.253587+0000","last_fullsized":"2024-09-21T06:02:11.253587+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:16.117957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790900+0000","last_change":"2024-09-21T06:02:10.262539+0000","last_active":"2024-09-21T06:02:11.790900+0000","last_peered":"2024-09-21T06:02:11.790900+0000","last_clean":"2024-09-21T06:02:11.790900+0000","last_became_active":"2024-09-21T06:02:10.261557+0000","last_became_peered":"2024-09-21T06:02:10.261557+0000","last_unstale":"2024-09-21T06:02:11.790900+0000","last_undegraded":"2024-09-21T06:02:11.790900+0000","last_fullsized":"2024-09-21T06:02:11.790900+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:46.770394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790427+0000","last_change":"2024-09-21T06:02:10.280853+0000","last_active":"2024-09-21T06:02:11.790427+0000","last_peered":"2024-09-21T06:02:11.790427+0000","last_clean":"2024-09-21T06:02:11.790427+0000","last_became_active":"2024-09-21T06:02:10.280106+0000","last_became_peered":"2024-09-21T06:02:10.280106+0000","last_unstale":"2024-09-21T06:02:11.790427+0000","last_undegraded":"2024-09-21T06:02:11.790427+0000","last_fullsized":"2024-09-21T06:02:11.790427+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:54.589979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254706+0000","last_change":"2024-09-21T06:02:10.272692+0000","last_active":"2024-09-21T06:02:11.254706+0000","last_peered":"2024-09-21T06:02:11.254706+0000","last_clean":"2024-09-21T06:02:11.254706+0000","last_became_active":"2024-09-21T06:02:10.272304+0000","last_became_peered":"2024-09-21T06:02:10.272304+0000","last_unstale":"2024-09-21T06:02:11.254706+0000","last_undegraded":"2024-09-21T06:02:11.254706+0000","last_fullsized":"2024-09-21T06:02:11.254706+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:19.605248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790044+0000","last_change":"2024-09-21T06:02:10.282612+0000","last_active":"2024-09-21T06:02:11.790044+0000","last_peered":"2024-09-21T06:02:11.790044+0000","last_clean":"2024-09-21T06:02:11.790044+0000","last_became_active":"2024-09-21T06:02:10.282114+0000","last_became_peered":"2024-09-21T06:02:10.282114+0000","last_unstale":"2024-09-21T06:02:11.790044+0000","last_undegraded":"2024-09-21T06:02:11.790044+0000","last_fullsized":"2024-09-21T06:02:11.790044+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:43.415017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253561+0000","last_change":"2024-09-21T06:02:10.275529+0000","last_active":"2024-09-21T06:02:11.253561+0000","last_peered":"2024-09-21T06:02:11.253561+0000","last_clean":"2024-09-21T06:02:11.253561+0000","last_became_active":"2024-09-21T06:02:10.275228+0000","last_became_peered":"2024-09-21T06:02:10.275228+0000","last_unstale":"2024-09-21T06:02:11.253561+0000","last_undegraded":"2024-09-21T06:02:11.253561+0000","last_fullsized":"2024-09-21T06:02:11.253561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:13.670308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253850+0000","last_change":"2024-09-21T06:02:10.276288+0000","last_active":"2024-09-21T06:02:11.253850+0000","last_peered":"2024-09-21T06:02:11.253850+0000","last_clean":"2024-09-21T06:02:11.253850+0000","last_became_active":"2024-09-21T06:02:10.274618+0000","last_became_peered":"2024-09-21T06:02:10.274618+0000","last_unstale":"2024-09-21T06:02:11.253850+0000","last_undegraded":"2024-09-21T06:02:11.253850+0000","last_fullsized":"2024-09-21T06:02:11.253850+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:23.366129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789824+0000","last_change":"2024-09-21T06:02:10.279231+0000","last_active":"2024-09-21T06:02:11.789824+0000","last_peered":"2024-09-21T06:02:11.789824+0000","last_clean":"2024-09-21T06:02:11.789824+0000","last_became_active":"2024-09-21T06:02:10.278439+0000","last_became_peered":"2024-09-21T06:02:10.278439+0000","last_unstale":"2024-09-21T06:02:11.789824+0000","last_undegraded":"2024-09-21T06:02:11.789824+0000","last_fullsized":"2024-09-21T06:02:11.789824+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:00.331491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792149+0000","last_change":"2024-09-21T06:02:10.287270+0000","last_active":"2024-09-21T06:02:11.792149+0000","last_peered":"2024-09-21T06:02:11.792149+0000","last_clean":"2024-09-21T06:02:11.792149+0000","last_became_active":"2024-09-21T06:02:10.285162+0000","last_became_peered":"2024-09-21T06:02:10.285162+0000","last_unstale":"2024-09-21T06:02:11.792149+0000","last_undegraded":"2024-09-21T06:02:11.792149+0000","last_fullsized":"2024-09-21T06:02:11.792149+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.910114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290107+0000","last_change":"2024-09-21T06:02:10.288834+0000","last_active":"2024-09-21T06:02:11.290107+0000","last_peered":"2024-09-21T06:02:11.290107+0000","last_clean":"2024-09-21T06:02:11.290107+0000","last_became_active":"2024-09-21T06:02:10.288513+0000","last_became_peered":"2024-09-21T06:02:10.288513+0000","last_unstale":"2024-09-21T06:02:11.290107+0000","last_undegraded":"2024-09-21T06:02:11.290107+0000","last_fullsized":"2024-09-21T06:02:11.290107+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:10.144897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250654+0000","last_change":"2024-09-21T06:02:10.275125+0000","last_active":"2024-09-21T06:02:11.250654+0000","last_peered":"2024-09-21T06:02:11.250654+0000","last_clean":"2024-09-21T06:02:11.250654+0000","last_became_active":"2024-09-21T06:02:10.274544+0000","last_became_peered":"2024-09-21T06:02:10.274544+0000","last_unstale":"2024-09-21T06:02:11.250654+0000","last_undegraded":"2024-09-21T06:02:11.250654+0000","last_fullsized":"2024-09-21T06:02:11.250654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:19.338852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791447+0000","last_change":"2024-09-21T06:02:10.282190+0000","last_active":"2024-09-21T06:02:11.791447+0000","last_peered":"2024-09-21T06:02:11.791447+0000","last_clean":"2024-09-21T06:02:11.791447+0000","last_became_active":"2024-09-21T06:02:10.281877+0000","last_became_peered":"2024-09-21T06:02:10.281877+0000","last_unstale":"2024-09-21T06:02:11.791447+0000","last_undegraded":"2024-09-21T06:02:11.791447+0000","last_fullsized":"2024-09-21T06:02:11.791447+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:43.790427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289797+0000","last_change":"2024-09-21T06:02:10.283752+0000","last_active":"2024-09-21T06:02:11.289797+0000","last_peered":"2024-09-21T06:02:11.289797+0000","last_clean":"2024-09-21T06:02:11.289797+0000","last_became_active":"2024-09-21T06:02:10.283417+0000","last_became_peered":"2024-09-21T06:02:10.283417+0000","last_unstale":"2024-09-21T06:02:11.289797+0000","last_undegraded":"2024-09-21T06:02:11.289797+0000","last_fullsized":"2024-09-21T06:02:11.289797+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:32.351750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791244+0000","last_change":"2024-09-21T06:02:10.282281+0000","last_active":"2024-09-21T06:02:11.791244+0000","last_peered":"2024-09-21T06:02:11.791244+0000","last_clean":"2024-09-21T06:02:11.791244+0000","last_became_active":"2024-09-21T06:02:10.281585+0000","last_became_peered":"2024-09-21T06:02:10.281585+0000","last_unstale":"2024-09-21T06:02:11.791244+0000","last_undegraded":"2024-09-21T06:02:11.791244+0000","last_fullsized":"2024-09-21T06:02:11.791244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:32.507248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290027+0000","last_change":"2024-09-21T06:02:10.287973+0000","last_active":"2024-09-21T06:02:11.290027+0000","last_peered":"2024-09-21T06:02:11.290027+0000","last_clean":"2024-09-21T06:02:11.290027+0000","last_became_active":"2024-09-21T06:02:10.287123+0000","last_became_peered":"2024-09-21T06:02:10.287123+0000","last_unstale":"2024-09-21T06:02:11.290027+0000","last_undegraded":"2024-09-21T06:02:11.290027+0000","last_fullsized":"2024-09-21T06:02:11.290027+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:10.646439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790293+0000","last_change":"2024-09-21T06:02:10.282167+0000","last_active":"2024-09-21T06:02:11.790293+0000","last_peered":"2024-09-21T06:02:11.790293+0000","last_clean":"2024-09-21T06:02:11.790293+0000","last_became_active":"2024-09-21T06:02:10.281870+0000","last_became_peered":"2024-09-21T06:02:10.281870+0000","last_unstale":"2024-09-21T06:02:11.790293+0000","last_undegraded":"2024-09-21T06:02:11.790293+0000","last_fullsized":"2024-09-21T06:02:11.790293+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:40.075266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.251246+0000","last_change":"2024-09-21T06:02:10.275662+0000","last_active":"2024-09-21T06:02:11.251246+0000","last_peered":"2024-09-21T06:02:11.251246+0000","last_clean":"2024-09-21T06:02:11.251246+0000","last_became_active":"2024-09-21T06:02:10.274366+0000","last_became_peered":"2024-09-21T06:02:10.274366+0000","last_unstale":"2024-09-21T06:02:11.251246+0000","last_undegraded":"2024-09-21T06:02:11.251246+0000","last_fullsized":"2024-09-21T06:02:11.251246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:10.180010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254453+0000","last_change":"2024-09-21T06:02:10.263425+0000","last_active":"2024-09-21T06:02:11.254453+0000","last_peered":"2024-09-21T06:02:11.254453+0000","last_clean":"2024-09-21T06:02:11.254453+0000","last_became_active":"2024-09-21T06:02:10.263066+0000","last_became_peered":"2024-09-21T06:02:10.263066+0000","last_unstale":"2024-09-21T06:02:11.254453+0000","last_undegraded":"2024-09-21T06:02:11.254453+0000","last_fullsized":"2024-09-21T06:02:11.254453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:03.263466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790375+0000","last_change":"2024-09-21T06:02:10.291344+0000","last_active":"2024-09-21T06:02:11.790375+0000","last_peered":"2024-09-21T06:02:11.790375+0000","last_clean":"2024-09-21T06:02:11.790375+0000","last_became_active":"2024-09-21T06:02:10.286643+0000","last_became_peered":"2024-09-21T06:02:10.286643+0000","last_unstale":"2024-09-21T06:02:11.790375+0000","last_undegraded":"2024-09-21T06:02:11.790375+0000","last_fullsized":"2024-09-21T06:02:11.790375+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:23.219415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250636+0000","last_change":"2024-09-21T06:02:10.273465+0000","last_active":"2024-09-21T06:02:11.250636+0000","last_peered":"2024-09-21T06:02:11.250636+0000","last_clean":"2024-09-21T06:02:11.250636+0000","last_became_active":"2024-09-21T06:02:10.272894+0000","last_became_peered":"2024-09-21T06:02:10.272894+0000","last_unstale":"2024-09-21T06:02:11.250636+0000","last_undegraded":"2024-09-21T06:02:11.250636+0000","last_fullsized":"2024-09-21T06:02:11.250636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:23.146917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791745+0000","last_change":"2024-09-21T06:02:10.286555+0000","last_active":"2024-09-21T06:02:11.791745+0000","last_peered":"2024-09-21T06:02:11.791745+0000","last_clean":"2024-09-21T06:02:11.791745+0000","last_became_active":"2024-09-21T06:02:10.275977+0000","last_became_peered":"2024-09-21T06:02:10.275977+0000","last_unstale":"2024-09-21T06:02:11.791745+0000","last_undegraded":"2024-09-21T06:02:11.791745+0000","last_fullsized":"2024-09-21T06:02:11.791745+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:49.043147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289689+0000","last_change":"2024-09-21T06:02:10.276271+0000","last_active":"2024-09-21T06:02:11.289689+0000","last_peered":"2024-09-21T06:02:11.289689+0000","last_clean":"2024-09-21T06:02:11.289689+0000","last_became_active":"2024-09-21T06:02:10.275948+0000","last_became_peered":"2024-09-21T06:02:10.275948+0000","last_unstale":"2024-09-21T06:02:11.289689+0000","last_undegraded":"2024-09-21T06:02:11.289689+0000","last_fullsized":"2024-09-21T06:02:11.289689+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:54.824679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250230+0000","last_change":"2024-09-21T06:02:10.276173+0000","last_active":"2024-09-21T06:02:11.250230+0000","last_peered":"2024-09-21T06:02:11.250230+0000","last_clean":"2024-09-21T06:02:11.250230+0000","last_became_active":"2024-09-21T06:02:10.275712+0000","last_became_peered":"2024-09-21T06:02:10.275712+0000","last_unstale":"2024-09-21T06:02:11.250230+0000","last_undegraded":"2024-09-21T06:02:11.250230+0000","last_fullsized":"2024-09-21T06:02:11.250230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:39.258731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790950+0000","last_change":"2024-09-21T06:02:10.291460+0000","last_active":"2024-09-21T06:02:11.790950+0000","last_peered":"2024-09-21T06:02:11.790950+0000","last_clean":"2024-09-21T06:02:11.790950+0000","last_became_active":"2024-09-21T06:02:10.287310+0000","last_became_peered":"2024-09-21T06:02:10.287310+0000","last_unstale":"2024-09-21T06:02:11.790950+0000","last_undegraded":"2024-09-21T06:02:11.790950+0000","last_fullsized":"2024-09-21T06:02:11.790950+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:44.150713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790426+0000","last_change":"2024-09-21T06:02:10.278806+0000","last_active":"2024-09-21T06:02:11.790426+0000","last_peered":"2024-09-21T06:02:11.790426+0000","last_clean":"2024-09-21T06:02:11.790426+0000","last_became_active":"2024-09-21T06:02:10.278388+0000","last_became_peered":"2024-09-21T06:02:10.278388+0000","last_unstale":"2024-09-21T06:02:11.790426+0000","last_undegraded":"2024-09-21T06:02:11.790426+0000","last_fullsized":"2024-09-21T06:02:11.790426+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:15.541144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790128+0000","last_change":"2024-09-21T06:02:09.259400+0000","last_active":"2024-09-21T06:02:11.790128+0000","last_peered":"2024-09-21T06:02:11.790128+0000","last_clean":"2024-09-21T06:02:11.790128+0000","last_became_active":"2024-09-21T06:02:09.259059+0000","last_became_peered":"2024-09-21T06:02:09.259059+0000","last_unstale":"2024-09-21T06:02:11.790128+0000","last_undegraded":"2024-09-21T06:02:11.790128+0000","last_fullsized":"2024-09-21T06:02:11.790128+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:32.593732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790843+0000","last_change":"2024-09-21T06:02:10.271022+0000","last_active":"2024-09-21T06:02:11.790843+0000","last_peered":"2024-09-21T06:02:11.790843+0000","last_clean":"2024-09-21T06:02:11.790843+0000","last_became_active":"2024-09-21T06:02:10.270717+0000","last_became_peered":"2024-09-21T06:02:10.270717+0000","last_unstale":"2024-09-21T06:02:11.790843+0000","last_undegraded":"2024-09-21T06:02:11.790843+0000","last_fullsized":"2024-09-21T06:02:11.790843+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:25.972520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792150+0000","last_change":"2024-09-21T06:02:09.265832+0000","last_active":"2024-09-21T06:02:11.792150+0000","last_peered":"2024-09-21T06:02:11.792150+0000","last_clean":"2024-09-21T06:02:11.792150+0000","last_became_active":"2024-09-21T06:02:09.265458+0000","last_became_peered":"2024-09-21T06:02:09.265458+0000","last_unstale":"2024-09-21T06:02:11.792150+0000","last_undegraded":"2024-09-21T06:02:11.792150+0000","last_fullsized":"2024-09-21T06:02:11.792150+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:18.597369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790086+0000","last_change":"2024-09-21T06:02:10.259160+0000","last_active":"2024-09-21T06:02:11.790086+0000","last_peered":"2024-09-21T06:02:11.790086+0000","last_clean":"2024-09-21T06:02:11.790086+0000","last_became_active":"2024-09-21T06:02:10.258851+0000","last_became_peered":"2024-09-21T06:02:10.258851+0000","last_unstale":"2024-09-21T06:02:11.790086+0000","last_undegraded":"2024-09-21T06:02:11.790086+0000","last_fullsized":"2024-09-21T06:02:11.790086+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:53.190221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790742+0000","last_change":"2024-09-21T06:02:09.260660+0000","last_active":"2024-09-21T06:02:11.790742+0000","last_peered":"2024-09-21T06:02:11.790742+0000","last_clean":"2024-09-21T06:02:11.790742+0000","last_became_active":"2024-09-21T06:02:09.260356+0000","last_became_peered":"2024-09-21T06:02:09.260356+0000","last_unstale":"2024-09-21T06:02:11.790742+0000","last_undegraded":"2024-09-21T06:02:11.790742+0000","last_fullsized":"2024-09-21T06:02:11.790742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:17.375769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790661+0000","last_change":"2024-09-21T06:02:10.291888+0000","last_active":"2024-09-21T06:02:11.790661+0000","last_peered":"2024-09-21T06:02:11.790661+0000","last_clean":"2024-09-21T06:02:11.790661+0000","last_became_active":"2024-09-21T06:02:10.279623+0000","last_became_peered":"2024-09-21T06:02:10.279623+0000","last_unstale":"2024-09-21T06:02:11.790661+0000","last_undegraded":"2024-09-21T06:02:11.790661+0000","last_fullsized":"2024-09-21T06:02:11.790661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:55.391528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254634+0000","last_change":"2024-09-21T06:02:09.262422+0000","last_active":"2024-09-21T06:02:11.254634+0000","last_peered":"2024-09-21T06:02:11.254634+0000","last_clean":"2024-09-21T06:02:11.254634+0000","last_became_active":"2024-09-21T06:02:09.262078+0000","last_became_peered":"2024-09-21T06:02:09.262078+0000","last_unstale":"2024-09-21T06:02:11.254634+0000","last_undegraded":"2024-09-21T06:02:11.254634+0000","last_fullsized":"2024-09-21T06:02:11.254634+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:12.662429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791513+0000","last_change":"2024-09-21T06:02:10.282522+0000","last_active":"2024-09-21T06:02:11.791513+0000","last_peered":"2024-09-21T06:02:11.791513+0000","last_clean":"2024-09-21T06:02:11.791513+0000","last_became_active":"2024-09-21T06:02:10.282036+0000","last_became_peered":"2024-09-21T06:02:10.282036+0000","last_unstale":"2024-09-21T06:02:11.791513+0000","last_undegraded":"2024-09-21T06:02:11.791513+0000","last_fullsized":"2024-09-21T06:02:11.791513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:53.258944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253029+0000","last_change":"2024-09-21T06:02:09.275773+0000","last_active":"2024-09-21T06:02:11.253029+0000","last_peered":"2024-09-21T06:02:11.253029+0000","last_clean":"2024-09-21T06:02:11.253029+0000","last_became_active":"2024-09-21T06:02:09.275434+0000","last_became_peered":"2024-09-21T06:02:09.275434+0000","last_unstale":"2024-09-21T06:02:11.253029+0000","last_undegraded":"2024-09-21T06:02:11.253029+0000","last_fullsized":"2024-09-21T06:02:11.253029+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:39.067387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791336+0000","last_change":"2024-09-21T06:02:10.286736+0000","last_active":"2024-09-21T06:02:11.791336+0000","last_peered":"2024-09-21T06:02:11.791336+0000","last_clean":"2024-09-21T06:02:11.791336+0000","last_became_active":"2024-09-21T06:02:10.286341+0000","last_became_peered":"2024-09-21T06:02:10.286341+0000","last_unstale":"2024-09-21T06:02:11.791336+0000","last_undegraded":"2024-09-21T06:02:11.791336+0000","last_fullsized":"2024-09-21T06:02:11.791336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:49.283469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789816+0000","last_change":"2024-09-21T06:02:09.270902+0000","last_active":"2024-09-21T06:02:11.789816+0000","last_peered":"2024-09-21T06:02:11.789816+0000","last_clean":"2024-09-21T06:02:11.789816+0000","last_became_active":"2024-09-21T06:02:09.270573+0000","last_became_peered":"2024-09-21T06:02:09.270573+0000","last_unstale":"2024-09-21T06:02:11.789816+0000","last_undegraded":"2024-09-21T06:02:11.789816+0000","last_fullsized":"2024-09-21T06:02:11.789816+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:15.110078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790233+0000","last_change":"2024-09-21T06:02:10.277306+0000","last_active":"2024-09-21T06:02:11.790233+0000","last_peered":"2024-09-21T06:02:11.790233+0000","last_clean":"2024-09-21T06:02:11.790233+0000","last_became_active":"2024-09-21T06:02:10.270375+0000","last_became_peered":"2024-09-21T06:02:10.270375+0000","last_unstale":"2024-09-21T06:02:11.790233+0000","last_undegraded":"2024-09-21T06:02:11.790233+0000","last_fullsized":"2024-09-21T06:02:11.790233+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:26.447373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253638+0000","last_change":"2024-09-21T06:02:09.266199+0000","last_active":"2024-09-21T06:02:11.253638+0000","last_peered":"2024-09-21T06:02:11.253638+0000","last_clean":"2024-09-21T06:02:11.253638+0000","last_became_active":"2024-09-21T06:02:09.265875+0000","last_became_peered":"2024-09-21T06:02:09.265875+0000","last_unstale":"2024-09-21T06:02:11.253638+0000","last_undegraded":"2024-09-21T06:02:11.253638+0000","last_fullsized":"2024-09-21T06:02:11.253638+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:39.929859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791608+0000","last_change":"2024-09-21T06:02:10.269696+0000","last_active":"2024-09-21T06:02:11.791608+0000","last_peered":"2024-09-21T06:02:11.791608+0000","last_clean":"2024-09-21T06:02:11.791608+0000","last_became_active":"2024-09-21T06:02:10.269223+0000","last_became_peered":"2024-09-21T06:02:10.269223+0000","last_unstale":"2024-09-21T06:02:11.791608+0000","last_undegraded":"2024-09-21T06:02:11.791608+0000","last_fullsized":"2024-09-21T06:02:11.791608+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:57.344419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.252862+0000","last_change":"2024-09-21T06:02:09.266846+0000","last_active":"2024-09-21T06:02:11.252862+0000","last_peered":"2024-09-21T06:02:11.252862+0000","last_clean":"2024-09-21T06:02:11.252862+0000","last_became_active":"2024-09-21T06:02:09.266468+0000","last_became_peered":"2024-09-21T06:02:09.266468+0000","last_unstale":"2024-09-21T06:02:11.252862+0000","last_undegraded":"2024-09-21T06:02:11.252862+0000","last_fullsized":"2024-09-21T06:02:11.252862+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:17.833244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253718+0000","last_change":"2024-09-21T06:02:10.273869+0000","last_active":"2024-09-21T06:02:11.253718+0000","last_peered":"2024-09-21T06:02:11.253718+0000","last_clean":"2024-09-21T06:02:11.253718+0000","last_became_active":"2024-09-21T06:02:10.273564+0000","last_became_peered":"2024-09-21T06:02:10.273564+0000","last_unstale":"2024-09-21T06:02:11.253718+0000","last_undegraded":"2024-09-21T06:02:11.253718+0000","last_fullsized":"2024-09-21T06:02:11.253718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:18.383647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290332+0000","last_change":"2024-09-21T06:02:09.270820+0000","last_active":"2024-09-21T06:02:11.290332+0000","last_peered":"2024-09-21T06:02:11.290332+0000","last_clean":"2024-09-21T06:02:11.290332+0000","last_became_active":"2024-09-21T06:02:09.270500+0000","last_became_peered":"2024-09-21T06:02:09.270500+0000","last_unstale":"2024-09-21T06:02:11.290332+0000","last_undegraded":"2024-09-21T06:02:11.290332+0000","last_fullsized":"2024-09-21T06:02:11.290332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:05.116966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290438+0000","last_change":"2024-09-21T06:02:10.278998+0000","last_active":"2024-09-21T06:02:11.290438+0000","last_peered":"2024-09-21T06:02:11.290438+0000","last_clean":"2024-09-21T06:02:11.290438+0000","last_became_active":"2024-09-21T06:02:10.278700+0000","last_became_peered":"2024-09-21T06:02:10.278700+0000","last_unstale":"2024-09-21T06:02:11.290438+0000","last_undegraded":"2024-09-21T06:02:11.290438+0000","last_fullsized":"2024-09-21T06:02:11.290438+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:44.150713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792239+0000","last_change":"2024-09-21T06:02:09.271907+0000","last_active":"2024-09-21T06:02:11.792239+0000","last_peered":"2024-09-21T06:02:11.792239+0000","last_clean":"2024-09-21T06:02:11.792239+0000","last_became_active":"2024-09-21T06:02:09.271563+0000","last_became_peered":"2024-09-21T06:02:09.271563+0000","last_unstale":"2024-09-21T06:02:11.792239+0000","last_undegraded":"2024-09-21T06:02:11.792239+0000","last_fullsized":"2024-09-21T06:02:11.792239+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:59.323612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790136+0000","last_change":"2024-09-21T06:02:10.272781+0000","last_active":"2024-09-21T06:02:11.790136+0000","last_peered":"2024-09-21T06:02:11.790136+0000","last_clean":"2024-09-21T06:02:11.790136+0000","last_became_active":"2024-09-21T06:02:10.272474+0000","last_became_peered":"2024-09-21T06:02:10.272474+0000","last_unstale":"2024-09-21T06:02:11.790136+0000","last_undegraded":"2024-09-21T06:02:11.790136+0000","last_fullsized":"2024-09-21T06:02:11.790136+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:19.569443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290056+0000","last_change":"2024-09-21T06:02:09.278081+0000","last_active":"2024-09-21T06:02:11.290056+0000","last_peered":"2024-09-21T06:02:11.290056+0000","last_clean":"2024-09-21T06:02:11.290056+0000","last_became_active":"2024-09-21T06:02:09.277726+0000","last_became_peered":"2024-09-21T06:02:09.277726+0000","last_unstale":"2024-09-21T06:02:11.290056+0000","last_undegraded":"2024-09-21T06:02:11.290056+0000","last_fullsized":"2024-09-21T06:02:11.290056+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:09.862737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289912+0000","last_change":"2024-09-21T06:02:10.280615+0000","last_active":"2024-09-21T06:02:11.289912+0000","last_peered":"2024-09-21T06:02:11.289912+0000","last_clean":"2024-09-21T06:02:11.289912+0000","last_became_active":"2024-09-21T06:02:10.280351+0000","last_became_peered":"2024-09-21T06:02:10.280351+0000","last_unstale":"2024-09-21T06:02:11.289912+0000","last_undegraded":"2024-09-21T06:02:11.289912+0000","last_fullsized":"2024-09-21T06:02:11.289912+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:55.704062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":21,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253261+0000","last_change":"2024-09-21T06:02:09.259775+0000","last_active":"2024-09-21T06:02:11.253261+0000","last_peered":"2024-09-21T06:02:11.253261+0000","last_clean":"2024-09-21T06:02:11.253261+0000","last_became_active":"2024-09-21T06:02:09.258709+0000","last_became_peered":"2024-09-21T06:02:09.258709+0000","last_unstale":"2024-09-21T06:02:11.253261+0000","last_undegraded":"2024-09-21T06:02:11.253261+0000","last_fullsized":"2024-09-21T06:02:11.253261+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:25.439494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253660+0000","last_change":"2024-09-21T06:02:10.275691+0000","last_active":"2024-09-21T06:02:11.253660+0000","last_peered":"2024-09-21T06:02:11.253660+0000","last_clean":"2024-09-21T06:02:11.253660+0000","last_became_active":"2024-09-21T06:02:10.275339+0000","last_became_peered":"2024-09-21T06:02:10.275339+0000","last_unstale":"2024-09-21T06:02:11.253660+0000","last_undegraded":"2024-09-21T06:02:11.253660+0000","last_fullsized":"2024-09-21T06:02:11.253660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:43.120115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791428+0000","last_change":"2024-09-21T06:02:09.276297+0000","last_active":"2024-09-21T06:02:11.791428+0000","last_peered":"2024-09-21T06:02:11.791428+0000","last_clean":"2024-09-21T06:02:11.791428+0000","last_became_active":"2024-09-21T06:02:09.275994+0000","last_became_peered":"2024-09-21T06:02:09.275994+0000","last_unstale":"2024-09-21T06:02:11.791428+0000","last_undegraded":"2024-09-21T06:02:11.791428+0000","last_fullsized":"2024-09-21T06:02:11.791428+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:54.696183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289767+0000","last_change":"2024-09-21T06:02:10.276275+0000","last_active":"2024-09-21T06:02:11.289767+0000","last_peered":"2024-09-21T06:02:11.289767+0000","last_clean":"2024-09-21T06:02:11.289767+0000","last_became_active":"2024-09-21T06:02:10.276049+0000","last_became_peered":"2024-09-21T06:02:10.276049+0000","last_unstale":"2024-09-21T06:02:11.289767+0000","last_undegraded":"2024-09-21T06:02:11.289767+0000","last_fullsized":"2024-09-21T06:02:11.289767+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:00.003986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789805+0000","last_change":"2024-09-21T06:02:09.267418+0000","last_active":"2024-09-21T06:02:11.789805+0000","last_peered":"2024-09-21T06:02:11.789805+0000","last_clean":"2024-09-21T06:02:11.789805+0000","last_became_active":"2024-09-21T06:02:09.267111+0000","last_became_peered":"2024-09-21T06:02:09.267111+0000","last_unstale":"2024-09-21T06:02:11.789805+0000","last_undegraded":"2024-09-21T06:02:11.789805+0000","last_fullsized":"2024-09-21T06:02:11.789805+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:10.270750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253090+0000","last_change":"2024-09-21T06:02:10.278157+0000","last_active":"2024-09-21T06:02:11.253090+0000","last_peered":"2024-09-21T06:02:11.253090+0000","last_clean":"2024-09-21T06:02:11.253090+0000","last_became_active":"2024-09-21T06:02:10.271151+0000","last_became_peered":"2024-09-21T06:02:10.271151+0000","last_unstale":"2024-09-21T06:02:11.253090+0000","last_undegraded":"2024-09-21T06:02:11.253090+0000","last_fullsized":"2024-09-21T06:02:11.253090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:09.465555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253777+0000","last_change":"2024-09-21T06:02:09.263864+0000","last_active":"2024-09-21T06:02:11.253777+0000","last_peered":"2024-09-21T06:02:11.253777+0000","last_clean":"2024-09-21T06:02:11.253777+0000","last_became_active":"2024-09-21T06:02:09.263556+0000","last_became_peered":"2024-09-21T06:02:09.263556+0000","last_unstale":"2024-09-21T06:02:11.253777+0000","last_undegraded":"2024-09-21T06:02:11.253777+0000","last_fullsized":"2024-09-21T06:02:11.253777+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:08.457676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289746+0000","last_change":"2024-09-21T06:02:10.287377+0000","last_active":"2024-09-21T06:02:11.289746+0000","last_peered":"2024-09-21T06:02:11.289746+0000","last_clean":"2024-09-21T06:02:11.289746+0000","last_became_active":"2024-09-21T06:02:10.287127+0000","last_became_peered":"2024-09-21T06:02:10.287127+0000","last_unstale":"2024-09-21T06:02:11.289746+0000","last_undegraded":"2024-09-21T06:02:11.289746+0000","last_fullsized":"2024-09-21T06:02:11.289746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:54.355475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289357+0000","last_change":"2024-09-21T06:02:09.267367+0000","last_active":"2024-09-21T06:02:11.289357+0000","last_peered":"2024-09-21T06:02:11.289357+0000","last_clean":"2024-09-21T06:02:11.289357+0000","last_became_active":"2024-09-21T06:02:09.267032+0000","last_became_peered":"2024-09-21T06:02:09.267032+0000","last_unstale":"2024-09-21T06:02:11.289357+0000","last_undegraded":"2024-09-21T06:02:11.289357+0000","last_fullsized":"2024-09-21T06:02:11.289357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:06.608982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290157+0000","last_change":"2024-09-21T06:02:10.275595+0000","last_active":"2024-09-21T06:02:11.290157+0000","last_peered":"2024-09-21T06:02:11.290157+0000","last_clean":"2024-09-21T06:02:11.290157+0000","last_became_active":"2024-09-21T06:02:10.275277+0000","last_became_peered":"2024-09-21T06:02:10.275277+0000","last_unstale":"2024-09-21T06:02:11.290157+0000","last_undegraded":"2024-09-21T06:02:11.290157+0000","last_fullsized":"2024-09-21T06:02:11.290157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:00.331491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792192+0000","last_change":"2024-09-21T06:02:09.274332+0000","last_active":"2024-09-21T06:02:11.792192+0000","last_peered":"2024-09-21T06:02:11.792192+0000","last_clean":"2024-09-21T06:02:11.792192+0000","last_became_active":"2024-09-21T06:02:09.274007+0000","last_became_peered":"2024-09-21T06:02:09.274007+0000","last_unstale":"2024-09-21T06:02:11.792192+0000","last_undegraded":"2024-09-21T06:02:11.792192+0000","last_fullsized":"2024-09-21T06:02:11.792192+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:43.142834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254114+0000","last_change":"2024-09-21T06:02:10.276861+0000","last_active":"2024-09-21T06:02:11.254114+0000","last_peered":"2024-09-21T06:02:11.254114+0000","last_clean":"2024-09-21T06:02:11.254114+0000","last_became_active":"2024-09-21T06:02:10.276566+0000","last_became_peered":"2024-09-21T06:02:10.276566+0000","last_unstale":"2024-09-21T06:02:11.254114+0000","last_undegraded":"2024-09-21T06:02:11.254114+0000","last_fullsized":"2024-09-21T06:02:11.254114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:32.351750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790419+0000","last_change":"2024-09-21T06:02:09.272265+0000","last_active":"2024-09-21T06:02:11.790419+0000","last_peered":"2024-09-21T06:02:11.790419+0000","last_clean":"2024-09-21T06:02:11.790419+0000","last_became_active":"2024-09-21T06:02:09.268092+0000","last_became_peered":"2024-09-21T06:02:09.268092+0000","last_unstale":"2024-09-21T06:02:11.790419+0000","last_undegraded":"2024-09-21T06:02:11.790419+0000","last_fullsized":"2024-09-21T06:02:11.790419+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:00.089511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791877+0000","last_change":"2024-09-21T06:02:10.275046+0000","last_active":"2024-09-21T06:02:11.791877+0000","last_peered":"2024-09-21T06:02:11.791877+0000","last_clean":"2024-09-21T06:02:11.791877+0000","last_became_active":"2024-09-21T06:02:10.274759+0000","last_became_peered":"2024-09-21T06:02:10.274759+0000","last_unstale":"2024-09-21T06:02:11.791877+0000","last_undegraded":"2024-09-21T06:02:11.791877+0000","last_fullsized":"2024-09-21T06:02:11.791877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:17.491964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253777+0000","last_change":"2024-09-21T06:02:09.268844+0000","last_active":"2024-09-21T06:02:11.253777+0000","last_peered":"2024-09-21T06:02:11.253777+0000","last_clean":"2024-09-21T06:02:11.253777+0000","last_became_active":"2024-09-21T06:02:09.268279+0000","last_became_peered":"2024-09-21T06:02:09.268279+0000","last_unstale":"2024-09-21T06:02:11.253777+0000","last_undegraded":"2024-09-21T06:02:11.253777+0000","last_fullsized":"2024-09-21T06:02:11.253777+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:43.064730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290070+0000","last_change":"2024-09-21T06:02:10.283907+0000","last_active":"2024-09-21T06:02:11.290070+0000","last_peered":"2024-09-21T06:02:11.290070+0000","last_clean":"2024-09-21T06:02:11.290070+0000","last_became_active":"2024-09-21T06:02:10.282734+0000","last_became_peered":"2024-09-21T06:02:10.282734+0000","last_unstale":"2024-09-21T06:02:11.290070+0000","last_undegraded":"2024-09-21T06:02:11.290070+0000","last_fullsized":"2024-09-21T06:02:11.290070+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:03.263466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790269+0000","last_change":"2024-09-21T06:02:09.269326+0000","last_active":"2024-09-21T06:02:11.790269+0000","last_peered":"2024-09-21T06:02:11.790269+0000","last_clean":"2024-09-21T06:02:11.790269+0000","last_became_active":"2024-09-21T06:02:09.267680+0000","last_became_peered":"2024-09-21T06:02:09.267680+0000","last_unstale":"2024-09-21T06:02:11.790269+0000","last_undegraded":"2024-09-21T06:02:11.790269+0000","last_fullsized":"2024-09-21T06:02:11.790269+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:22.358250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253933+0000","last_change":"2024-09-21T06:02:10.275681+0000","last_active":"2024-09-21T06:02:11.253933+0000","last_peered":"2024-09-21T06:02:11.253933+0000","last_clean":"2024-09-21T06:02:11.253933+0000","last_became_active":"2024-09-21T06:02:10.275228+0000","last_became_peered":"2024-09-21T06:02:10.275228+0000","last_unstale":"2024-09-21T06:02:11.253933+0000","last_undegraded":"2024-09-21T06:02:11.253933+0000","last_fullsized":"2024-09-21T06:02:11.253933+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:00.003986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791802+0000","last_change":"2024-09-21T06:02:09.266422+0000","last_active":"2024-09-21T06:02:11.791802+0000","last_peered":"2024-09-21T06:02:11.791802+0000","last_clean":"2024-09-21T06:02:11.791802+0000","last_became_active":"2024-09-21T06:02:09.266121+0000","last_became_peered":"2024-09-21T06:02:09.266121+0000","last_unstale":"2024-09-21T06:02:11.791802+0000","last_undegraded":"2024-09-21T06:02:11.791802+0000","last_fullsized":"2024-09-21T06:02:11.791802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:09.638560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790006+0000","last_change":"2024-09-21T06:02:10.265400+0000","last_active":"2024-09-21T06:02:11.790006+0000","last_peered":"2024-09-21T06:02:11.790006+0000","last_clean":"2024-09-21T06:02:11.790006+0000","last_became_active":"2024-09-21T06:02:10.263526+0000","last_became_peered":"2024-09-21T06:02:10.263526+0000","last_unstale":"2024-09-21T06:02:11.790006+0000","last_undegraded":"2024-09-21T06:02:11.790006+0000","last_fullsized":"2024-09-21T06:02:11.790006+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:48.448199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789710+0000","last_change":"2024-09-21T06:02:09.271164+0000","last_active":"2024-09-21T06:02:11.789710+0000","last_peered":"2024-09-21T06:02:11.789710+0000","last_clean":"2024-09-21T06:02:11.789710+0000","last_became_active":"2024-09-21T06:02:09.270848+0000","last_became_peered":"2024-09-21T06:02:09.270848+0000","last_unstale":"2024-09-21T06:02:11.789710+0000","last_undegraded":"2024-09-21T06:02:11.789710+0000","last_fullsized":"2024-09-21T06:02:11.789710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:58.592636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789752+0000","last_change":"2024-09-21T06:02:10.265812+0000","last_active":"2024-09-21T06:02:11.789752+0000","last_peered":"2024-09-21T06:02:11.789752+0000","last_clean":"2024-09-21T06:02:11.789752+0000","last_became_active":"2024-09-21T06:02:10.265514+0000","last_became_peered":"2024-09-21T06:02:10.265514+0000","last_unstale":"2024-09-21T06:02:11.789752+0000","last_undegraded":"2024-09-21T06:02:11.789752+0000","last_fullsized":"2024-09-21T06:02:11.789752+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:19.605248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.252801+0000","last_change":"2024-09-21T06:02:09.267046+0000","last_active":"2024-09-21T06:02:11.252801+0000","last_peered":"2024-09-21T06:02:11.252801+0000","last_clean":"2024-09-21T06:02:11.252801+0000","last_became_active":"2024-09-21T06:02:09.266750+0000","last_became_peered":"2024-09-21T06:02:09.266750+0000","last_unstale":"2024-09-21T06:02:11.252801+0000","last_undegraded":"2024-09-21T06:02:11.252801+0000","last_fullsized":"2024-09-21T06:02:11.252801+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:48.018613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254310+0000","last_change":"2024-09-21T06:02:10.261217+0000","last_active":"2024-09-21T06:02:11.254310+0000","last_peered":"2024-09-21T06:02:11.254310+0000","last_clean":"2024-09-21T06:02:11.254310+0000","last_became_active":"2024-09-21T06:02:10.260828+0000","last_became_peered":"2024-09-21T06:02:10.260828+0000","last_unstale":"2024-09-21T06:02:11.254310+0000","last_undegraded":"2024-09-21T06:02:11.254310+0000","last_fullsized":"2024-09-21T06:02:11.254310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:55.704062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790736+0000","last_change":"2024-09-21T06:02:09.275643+0000","last_active":"2024-09-21T06:02:11.790736+0000","last_peered":"2024-09-21T06:02:11.790736+0000","last_clean":"2024-09-21T06:02:11.790736+0000","last_became_active":"2024-09-21T06:02:09.275319+0000","last_became_peered":"2024-09-21T06:02:09.275319+0000","last_unstale":"2024-09-21T06:02:11.790736+0000","last_undegraded":"2024-09-21T06:02:11.790736+0000","last_fullsized":"2024-09-21T06:02:11.790736+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:46.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254321+0000","last_change":"2024-09-21T06:02:10.270206+0000","last_active":"2024-09-21T06:02:11.254321+0000","last_peered":"2024-09-21T06:02:11.254321+0000","last_clean":"2024-09-21T06:02:11.254321+0000","last_became_active":"2024-09-21T06:02:10.269531+0000","last_became_peered":"2024-09-21T06:02:10.269531+0000","last_unstale":"2024-09-21T06:02:11.254321+0000","last_undegraded":"2024-09-21T06:02:11.254321+0000","last_fullsized":"2024-09-21T06:02:11.254321+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:10.646439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790569+0000","last_change":"2024-09-21T06:02:09.272606+0000","last_active":"2024-09-21T06:02:11.790569+0000","last_peered":"2024-09-21T06:02:11.790569+0000","last_clean":"2024-09-21T06:02:11.790569+0000","last_became_active":"2024-09-21T06:02:09.271935+0000","last_became_peered":"2024-09-21T06:02:09.271935+0000","last_unstale":"2024-09-21T06:02:11.790569+0000","last_undegraded":"2024-09-21T06:02:11.790569+0000","last_fullsized":"2024-09-21T06:02:11.790569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:53.780735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790403+0000","last_change":"2024-09-21T06:02:10.262950+0000","last_active":"2024-09-21T06:02:11.790403+0000","last_peered":"2024-09-21T06:02:11.790403+0000","last_clean":"2024-09-21T06:02:11.790403+0000","last_became_active":"2024-09-21T06:02:10.262636+0000","last_became_peered":"2024-09-21T06:02:10.262636+0000","last_unstale":"2024-09-21T06:02:11.790403+0000","last_undegraded":"2024-09-21T06:02:11.790403+0000","last_fullsized":"2024-09-21T06:02:11.790403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:32.505486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789909+0000","last_change":"2024-09-21T06:02:09.267841+0000","last_active":"2024-09-21T06:02:11.789909+0000","last_peered":"2024-09-21T06:02:11.789909+0000","last_clean":"2024-09-21T06:02:11.789909+0000","last_became_active":"2024-09-21T06:02:09.267444+0000","last_became_peered":"2024-09-21T06:02:09.267444+0000","last_unstale":"2024-09-21T06:02:11.789909+0000","last_undegraded":"2024-09-21T06:02:11.789909+0000","last_fullsized":"2024-09-21T06:02:11.789909+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:33.721188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790709+0000","last_change":"2024-09-21T06:02:10.263121+0000","last_active":"2024-09-21T06:02:11.790709+0000","last_peered":"2024-09-21T06:02:11.790709+0000","last_clean":"2024-09-21T06:02:11.790709+0000","last_became_active":"2024-09-21T06:02:10.262809+0000","last_became_peered":"2024-09-21T06:02:10.262809+0000","last_unstale":"2024-09-21T06:02:11.790709+0000","last_undegraded":"2024-09-21T06:02:11.790709+0000","last_fullsized":"2024-09-21T06:02:11.790709+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:53.926538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254710+0000","last_change":"2024-09-21T06:02:09.263354+0000","last_active":"2024-09-21T06:02:11.254710+0000","last_peered":"2024-09-21T06:02:11.254710+0000","last_clean":"2024-09-21T06:02:11.254710+0000","last_became_active":"2024-09-21T06:02:09.263051+0000","last_became_peered":"2024-09-21T06:02:09.263051+0000","last_unstale":"2024-09-21T06:02:11.254710+0000","last_undegraded":"2024-09-21T06:02:11.254710+0000","last_fullsized":"2024-09-21T06:02:11.254710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:15.110078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791787+0000","last_change":"2024-09-21T06:02:10.280540+0000","last_active":"2024-09-21T06:02:11.791787+0000","last_peered":"2024-09-21T06:02:11.791787+0000","last_clean":"2024-09-21T06:02:11.791787+0000","last_became_active":"2024-09-21T06:02:10.274182+0000","last_became_peered":"2024-09-21T06:02:10.274182+0000","last_unstale":"2024-09-21T06:02:11.791787+0000","last_undegraded":"2024-09-21T06:02:11.791787+0000","last_fullsized":"2024-09-21T06:02:11.791787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:38.369387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790058+0000","last_change":"2024-09-21T06:02:09.263348+0000","last_active":"2024-09-21T06:02:11.790058+0000","last_peered":"2024-09-21T06:02:11.790058+0000","last_clean":"2024-09-21T06:02:11.790058+0000","last_became_active":"2024-09-21T06:02:09.262942+0000","last_became_peered":"2024-09-21T06:02:09.262942+0000","last_unstale":"2024-09-21T06:02:11.790058+0000","last_undegraded":"2024-09-21T06:02:11.790058+0000","last_fullsized":"2024-09-21T06:02:11.790058+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:13.084051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290121+0000","last_change":"2024-09-21T06:02:10.288104+0000","last_active":"2024-09-21T06:02:11.290121+0000","last_peered":"2024-09-21T06:02:11.290121+0000","last_clean":"2024-09-21T06:02:11.290121+0000","last_became_active":"2024-09-21T06:02:10.287460+0000","last_became_peered":"2024-09-21T06:02:10.287460+0000","last_unstale":"2024-09-21T06:02:11.290121+0000","last_undegraded":"2024-09-21T06:02:11.290121+0000","last_fullsized":"2024-09-21T06:02:11.290121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:00.353554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253392+0000","last_change":"2024-09-21T06:02:09.259123+0000","last_active":"2024-09-21T06:02:11.253392+0000","last_peered":"2024-09-21T06:02:11.253392+0000","last_clean":"2024-09-21T06:02:11.253392+0000","last_became_active":"2024-09-21T06:02:09.258760+0000","last_became_peered":"2024-09-21T06:02:09.258760+0000","last_unstale":"2024-09-21T06:02:11.253392+0000","last_undegraded":"2024-09-21T06:02:11.253392+0000","last_fullsized":"2024-09-21T06:02:11.253392+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:42.407138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790834+0000","last_change":"2024-09-21T06:02:10.261037+0000","last_active":"2024-09-21T06:02:11.790834+0000","last_peered":"2024-09-21T06:02:11.790834+0000","last_clean":"2024-09-21T06:02:11.790834+0000","last_became_active":"2024-09-21T06:02:10.260729+0000","last_became_peered":"2024-09-21T06:02:10.260729+0000","last_unstale":"2024-09-21T06:02:11.790834+0000","last_undegraded":"2024-09-21T06:02:11.790834+0000","last_fullsized":"2024-09-21T06:02:11.790834+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:00.468017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789596+0000","last_change":"2024-09-21T06:02:09.264048+0000","last_active":"2024-09-21T06:02:11.789596+0000","last_peered":"2024-09-21T06:02:11.789596+0000","last_clean":"2024-09-21T06:02:11.789596+0000","last_became_active":"2024-09-21T06:02:09.263708+0000","last_became_peered":"2024-09-21T06:02:09.263708+0000","last_unstale":"2024-09-21T06:02:11.789596+0000","last_undegraded":"2024-09-21T06:02:11.789596+0000","last_fullsized":"2024-09-21T06:02:11.789596+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:25.531925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791392+0000","last_change":"2024-09-21T06:02:10.262498+0000","last_active":"2024-09-21T06:02:11.791392+0000","last_peered":"2024-09-21T06:02:11.791392+0000","last_clean":"2024-09-21T06:02:11.791392+0000","last_became_active":"2024-09-21T06:02:10.262000+0000","last_became_peered":"2024-09-21T06:02:10.262000+0000","last_unstale":"2024-09-21T06:02:11.791392+0000","last_undegraded":"2024-09-21T06:02:11.791392+0000","last_fullsized":"2024-09-21T06:02:11.791392+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:39.062283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791682+0000","last_change":"2024-09-21T06:02:09.265629+0000","last_active":"2024-09-21T06:02:11.791682+0000","last_peered":"2024-09-21T06:02:11.791682+0000","last_clean":"2024-09-21T06:02:11.791682+0000","last_became_active":"2024-09-21T06:02:09.265255+0000","last_became_peered":"2024-09-21T06:02:09.265255+0000","last_unstale":"2024-09-21T06:02:11.791682+0000","last_undegraded":"2024-09-21T06:02:11.791682+0000","last_fullsized":"2024-09-21T06:02:11.791682+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:02.255587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791518+0000","last_change":"2024-09-21T06:02:10.283113+0000","last_active":"2024-09-21T06:02:11.791518+0000","last_peered":"2024-09-21T06:02:11.791518+0000","last_clean":"2024-09-21T06:02:11.791518+0000","last_became_active":"2024-09-21T06:02:10.282826+0000","last_became_peered":"2024-09-21T06:02:10.282826+0000","last_unstale":"2024-09-21T06:02:11.791518+0000","last_undegraded":"2024-09-21T06:02:11.791518+0000","last_fullsized":"2024-09-21T06:02:11.791518+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:57.773356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789812+0000","last_change":"2024-09-21T06:02:09.272885+0000","last_active":"2024-09-21T06:02:11.789812+0000","last_peered":"2024-09-21T06:02:11.789812+0000","last_clean":"2024-09-21T06:02:11.789812+0000","last_became_active":"2024-09-21T06:02:09.272520+0000","last_became_peered":"2024-09-21T06:02:09.272520+0000","last_unstale":"2024-09-21T06:02:11.789812+0000","last_undegraded":"2024-09-21T06:02:11.789812+0000","last_fullsized":"2024-09-21T06:02:11.789812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:37.738718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":180,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289489+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:02:11.289489+0000","last_peered":"2024-09-21T06:02:11.289489+0000","last_clean":"2024-09-21T06:02:11.289489+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:02:11.289489+0000","last_undegraded":"2024-09-21T06:02:11.289489+0000","last_fullsized":"2024-09-21T06:02:11.289489+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509514,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40788,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331052,"statfs":{"total":96636764160,"available":96594997248,"internally_reserved":0,"allocated":1302528,"data_stored":880639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509514,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45764,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326076,"statfs":{"total":96636764160,"available":96589901824,"internally_reserved":0,"allocated":700416,"data_stored":290111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509513,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":708608,"data_stored":290763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62416,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309424,"statfs":{"total":96636764160,"available":96572850176,"internally_reserved":0,"allocated":712704,"data_stored":291723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45772,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326068,"statfs":{"total":96636764160,"available":96589893632,"internally_reserved":0,"allocated":708608,"data_stored":290167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509515,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":708608,"data_stored":290167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":1310720,"data_stored":882673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542218,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49928,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321912,"statfs":{"total":96636764160,"available":96585637888,"internally_reserved":0,"allocated":704512,"data_stored":290201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:02:16.226 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-09-21T06:02:16.224+0000 7f1a68e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:02:17.378 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:02:17.381 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:02:19.198 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:02:19.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.368+0000 7fcc78c00640 1 -- 172.21.15.50:0/1083619479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc740b54f0 msgr2=0x7fcc7415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:19.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.368+0000 7fcc78c00640 1 --2- 172.21.15.50:0/1083619479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc740b54f0 0x7fcc7415bb60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009bc0 tx=0x7fcc6c01cae0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.368+0000 7fcc78c00640 1 -- 172.21.15.50:0/1083619479 shutdown_connections 2024-09-21T06:02:19.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.368+0000 7fcc78c00640 1 --2- 172.21.15.50:0/1083619479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc740b54f0 0x7fcc7415bb60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.368+0000 7fcc78c00640 1 -- 172.21.15.50:0/1083619479 >> 172.21.15.50:0/1083619479 conn(0x7fcc740b2040 msgr2=0x7fcc740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:19.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.368+0000 7fcc78c00640 1 -- 172.21.15.50:0/1083619479 shutdown_connections 2024-09-21T06:02:19.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.368+0000 7fcc78c00640 1 -- 172.21.15.50:0/1083619479 wait complete. 2024-09-21T06:02:19.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc78c00640 1 Processor -- start 2024-09-21T06:02:19.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc78c00640 1 -- start start 2024-09-21T06:02:19.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc78c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc740b5c70 0x7fcc74144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:19.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc78c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc741527b0 0x7fcc740ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:19.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc78c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74154960 0x7fcc740aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:19.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc78c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc74169b30 con 0x7fcc74154960 2024-09-21T06:02:19.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc78c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcc741699b0 con 0x7fcc741527b0 2024-09-21T06:02:19.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc78c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fcc74169cb0 con 0x7fcc740b5c70 2024-09-21T06:02:19.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc72800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74154960 0x7fcc740aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:19.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc72800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74154960 0x7fcc740aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58166/0 (socket says 172.21.15.50:58166) 2024-09-21T06:02:19.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc72800640 1 -- 172.21.15.50:0/3980300351 learned_addr learned my addr 172.21.15.50:0/3980300351 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:19.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc6be00640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc741527b0 0x7fcc740ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:19.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc71e00640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc740b5c70 0x7fcc74144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:19.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc72800640 1 -- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc740b5c70 msgr2=0x7fcc74144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:19.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc72800640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc740b5c70 0x7fcc74144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc72800640 1 -- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc741527b0 msgr2=0x7fcc740ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:19.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc72800640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc741527b0 0x7fcc740ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc72800640 1 -- 172.21.15.50:0/3980300351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc74157710 con 0x7fcc74154960 2024-09-21T06:02:19.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc72800640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74154960 0x7fcc740aee20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c01c550 tx=0x7fcc6c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:19.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc6a000640 1 -- 172.21.15.50:0/3980300351 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0b2040 con 0x7fcc74154960 2024-09-21T06:02:19.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc6a000640 1 -- 172.21.15.50:0/3980300351 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc6c0bb070 con 0x7fcc74154960 2024-09-21T06:02:19.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc78c00640 1 -- 172.21.15.50:0/3980300351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc741579f0 con 0x7fcc74154960 2024-09-21T06:02:19.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc6a000640 1 -- 172.21.15.50:0/3980300351 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c004630 con 0x7fcc74154960 2024-09-21T06:02:19.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.372+0000 7fcc78c00640 1 -- 172.21.15.50:0/3980300351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc74145060 con 0x7fcc74154960 2024-09-21T06:02:19.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.376+0000 7fcc78c00640 1 -- 172.21.15.50:0/3980300351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc740c5400 con 0x7fcc74154960 2024-09-21T06:02:19.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.376+0000 7fcc6a000640 1 -- 172.21.15.50:0/3980300351 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fcc6c0ad440 con 0x7fcc74154960 2024-09-21T06:02:19.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.376+0000 7fcc6a000640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc50073b00 0x7fcc50076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:19.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.376+0000 7fcc6a000640 1 -- 172.21.15.50:0/3980300351 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fcc6c12e140 con 0x7fcc74154960 2024-09-21T06:02:19.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.376+0000 7fcc71e00640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc50073b00 0x7fcc50076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:19.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.380+0000 7fcc71e00640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc50073b00 0x7fcc50076010 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400b810 tx=0x7fcc64008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:19.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.380+0000 7fcc6a000640 1 -- 172.21.15.50:0/3980300351 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcc6c016790 con 0x7fcc74154960 2024-09-21T06:02:19.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.556+0000 7fcc78c00640 1 -- 172.21.15.50:0/3980300351 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc740c5690 con 0x7fcc50073b00 2024-09-21T06:02:19.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.560+0000 7fcc6a000640 1 -- 172.21.15.50:0/3980300351 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251856 (secure 0 0 0) 0x7fcc740c5690 con 0x7fcc50073b00 2024-09-21T06:02:19.565 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:19.566 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:02:19.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.564+0000 7fcc78c00640 1 -- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc50073b00 msgr2=0x7fcc50076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:19.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.564+0000 7fcc78c00640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc50073b00 0x7fcc50076010 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400b810 tx=0x7fcc64008040 comp rx=0 tx=0).stop 2024-09-21T06:02:19.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.564+0000 7fcc78c00640 1 -- 172.21.15.50:0/3980300351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74154960 msgr2=0x7fcc740aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:19.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.564+0000 7fcc78c00640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74154960 0x7fcc740aee20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c01c550 tx=0x7fcc6c005ae0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.564+0000 7fcc78c00640 1 -- 172.21.15.50:0/3980300351 shutdown_connections 2024-09-21T06:02:19.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.564+0000 7fcc78c00640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc50073b00 0x7fcc50076010 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.564+0000 7fcc78c00640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74154960 0x7fcc740aee20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.564+0000 7fcc78c00640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc741527b0 0x7fcc740ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.564+0000 7fcc78c00640 1 --2- 172.21.15.50:0/3980300351 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc740b5c70 0x7fcc74144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.564+0000 7fcc78c00640 1 -- 172.21.15.50:0/3980300351 >> 172.21.15.50:0/3980300351 conn(0x7fcc740b2040 msgr2=0x7fcc740cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:19.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.564+0000 7fcc78c00640 1 -- 172.21.15.50:0/3980300351 shutdown_connections 2024-09-21T06:02:19.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.564+0000 7fcc78c00640 1 -- 172.21.15.50:0/3980300351 wait complete. 2024-09-21T06:02:19.587 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-21T06:02:18.985740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":358960,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":352051,"kb_avail":754615760,"statfs":{"total":773094113280,"available":772726538240,"internally_reserved":0,"allocated":6856704,"data_stored":3506444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":360500784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.734564"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250964+0000","last_change":"2024-09-21T06:02:10.283497+0000","last_active":"2024-09-21T06:02:11.250964+0000","last_peered":"2024-09-21T06:02:11.250964+0000","last_clean":"2024-09-21T06:02:11.250964+0000","last_became_active":"2024-09-21T06:02:10.283183+0000","last_became_peered":"2024-09-21T06:02:10.283183+0000","last_unstale":"2024-09-21T06:02:11.250964+0000","last_undegraded":"2024-09-21T06:02:11.250964+0000","last_fullsized":"2024-09-21T06:02:11.250964+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:50.423813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792195+0000","last_change":"2024-09-21T06:02:10.285494+0000","last_active":"2024-09-21T06:02:11.792195+0000","last_peered":"2024-09-21T06:02:11.792195+0000","last_clean":"2024-09-21T06:02:11.792195+0000","last_became_active":"2024-09-21T06:02:10.285121+0000","last_became_peered":"2024-09-21T06:02:10.285121+0000","last_unstale":"2024-09-21T06:02:11.792195+0000","last_undegraded":"2024-09-21T06:02:11.792195+0000","last_fullsized":"2024-09-21T06:02:11.792195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:32.261338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289739+0000","last_change":"2024-09-21T06:02:10.277961+0000","last_active":"2024-09-21T06:02:11.289739+0000","last_peered":"2024-09-21T06:02:11.289739+0000","last_clean":"2024-09-21T06:02:11.289739+0000","last_became_active":"2024-09-21T06:02:10.277639+0000","last_became_peered":"2024-09-21T06:02:10.277639+0000","last_unstale":"2024-09-21T06:02:11.289739+0000","last_undegraded":"2024-09-21T06:02:11.289739+0000","last_fullsized":"2024-09-21T06:02:11.289739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:52.411453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254579+0000","last_change":"2024-09-21T06:02:10.267778+0000","last_active":"2024-09-21T06:02:11.254579+0000","last_peered":"2024-09-21T06:02:11.254579+0000","last_clean":"2024-09-21T06:02:11.254579+0000","last_became_active":"2024-09-21T06:02:10.263807+0000","last_became_peered":"2024-09-21T06:02:10.263807+0000","last_unstale":"2024-09-21T06:02:11.254579+0000","last_undegraded":"2024-09-21T06:02:11.254579+0000","last_fullsized":"2024-09-21T06:02:11.254579+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:00.353554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250565+0000","last_change":"2024-09-21T06:02:10.268540+0000","last_active":"2024-09-21T06:02:11.250565+0000","last_peered":"2024-09-21T06:02:11.250565+0000","last_clean":"2024-09-21T06:02:11.250565+0000","last_became_active":"2024-09-21T06:02:10.267456+0000","last_became_peered":"2024-09-21T06:02:10.267456+0000","last_unstale":"2024-09-21T06:02:11.250565+0000","last_undegraded":"2024-09-21T06:02:11.250565+0000","last_fullsized":"2024-09-21T06:02:11.250565+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:14.203778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791022+0000","last_change":"2024-09-21T06:02:10.279996+0000","last_active":"2024-09-21T06:02:11.791022+0000","last_peered":"2024-09-21T06:02:11.791022+0000","last_clean":"2024-09-21T06:02:11.791022+0000","last_became_active":"2024-09-21T06:02:10.279672+0000","last_became_peered":"2024-09-21T06:02:10.279672+0000","last_unstale":"2024-09-21T06:02:11.791022+0000","last_undegraded":"2024-09-21T06:02:11.791022+0000","last_fullsized":"2024-09-21T06:02:11.791022+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:51.112377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253169+0000","last_change":"2024-09-21T06:02:10.267014+0000","last_active":"2024-09-21T06:02:11.253169+0000","last_peered":"2024-09-21T06:02:11.253169+0000","last_clean":"2024-09-21T06:02:11.253169+0000","last_became_active":"2024-09-21T06:02:10.266091+0000","last_became_peered":"2024-09-21T06:02:10.266091+0000","last_unstale":"2024-09-21T06:02:11.253169+0000","last_undegraded":"2024-09-21T06:02:11.253169+0000","last_fullsized":"2024-09-21T06:02:11.253169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:44.072609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791264+0000","last_change":"2024-09-21T06:02:10.275841+0000","last_active":"2024-09-21T06:02:11.791264+0000","last_peered":"2024-09-21T06:02:11.791264+0000","last_clean":"2024-09-21T06:02:11.791264+0000","last_became_active":"2024-09-21T06:02:10.275526+0000","last_became_peered":"2024-09-21T06:02:10.275526+0000","last_unstale":"2024-09-21T06:02:11.791264+0000","last_undegraded":"2024-09-21T06:02:11.791264+0000","last_fullsized":"2024-09-21T06:02:11.791264+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:42.101108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253587+0000","last_change":"2024-09-21T06:02:10.293470+0000","last_active":"2024-09-21T06:02:11.253587+0000","last_peered":"2024-09-21T06:02:11.253587+0000","last_clean":"2024-09-21T06:02:11.253587+0000","last_became_active":"2024-09-21T06:02:10.292993+0000","last_became_peered":"2024-09-21T06:02:10.292993+0000","last_unstale":"2024-09-21T06:02:11.253587+0000","last_undegraded":"2024-09-21T06:02:11.253587+0000","last_fullsized":"2024-09-21T06:02:11.253587+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:16.117957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790900+0000","last_change":"2024-09-21T06:02:10.262539+0000","last_active":"2024-09-21T06:02:11.790900+0000","last_peered":"2024-09-21T06:02:11.790900+0000","last_clean":"2024-09-21T06:02:11.790900+0000","last_became_active":"2024-09-21T06:02:10.261557+0000","last_became_peered":"2024-09-21T06:02:10.261557+0000","last_unstale":"2024-09-21T06:02:11.790900+0000","last_undegraded":"2024-09-21T06:02:11.790900+0000","last_fullsized":"2024-09-21T06:02:11.790900+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:46.770394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790427+0000","last_change":"2024-09-21T06:02:10.280853+0000","last_active":"2024-09-21T06:02:11.790427+0000","last_peered":"2024-09-21T06:02:11.790427+0000","last_clean":"2024-09-21T06:02:11.790427+0000","last_became_active":"2024-09-21T06:02:10.280106+0000","last_became_peered":"2024-09-21T06:02:10.280106+0000","last_unstale":"2024-09-21T06:02:11.790427+0000","last_undegraded":"2024-09-21T06:02:11.790427+0000","last_fullsized":"2024-09-21T06:02:11.790427+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:54.589979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254706+0000","last_change":"2024-09-21T06:02:10.272692+0000","last_active":"2024-09-21T06:02:11.254706+0000","last_peered":"2024-09-21T06:02:11.254706+0000","last_clean":"2024-09-21T06:02:11.254706+0000","last_became_active":"2024-09-21T06:02:10.272304+0000","last_became_peered":"2024-09-21T06:02:10.272304+0000","last_unstale":"2024-09-21T06:02:11.254706+0000","last_undegraded":"2024-09-21T06:02:11.254706+0000","last_fullsized":"2024-09-21T06:02:11.254706+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:19.605248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790044+0000","last_change":"2024-09-21T06:02:10.282612+0000","last_active":"2024-09-21T06:02:11.790044+0000","last_peered":"2024-09-21T06:02:11.790044+0000","last_clean":"2024-09-21T06:02:11.790044+0000","last_became_active":"2024-09-21T06:02:10.282114+0000","last_became_peered":"2024-09-21T06:02:10.282114+0000","last_unstale":"2024-09-21T06:02:11.790044+0000","last_undegraded":"2024-09-21T06:02:11.790044+0000","last_fullsized":"2024-09-21T06:02:11.790044+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:43.415017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253561+0000","last_change":"2024-09-21T06:02:10.275529+0000","last_active":"2024-09-21T06:02:11.253561+0000","last_peered":"2024-09-21T06:02:11.253561+0000","last_clean":"2024-09-21T06:02:11.253561+0000","last_became_active":"2024-09-21T06:02:10.275228+0000","last_became_peered":"2024-09-21T06:02:10.275228+0000","last_unstale":"2024-09-21T06:02:11.253561+0000","last_undegraded":"2024-09-21T06:02:11.253561+0000","last_fullsized":"2024-09-21T06:02:11.253561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:13.670308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253850+0000","last_change":"2024-09-21T06:02:10.276288+0000","last_active":"2024-09-21T06:02:11.253850+0000","last_peered":"2024-09-21T06:02:11.253850+0000","last_clean":"2024-09-21T06:02:11.253850+0000","last_became_active":"2024-09-21T06:02:10.274618+0000","last_became_peered":"2024-09-21T06:02:10.274618+0000","last_unstale":"2024-09-21T06:02:11.253850+0000","last_undegraded":"2024-09-21T06:02:11.253850+0000","last_fullsized":"2024-09-21T06:02:11.253850+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:23.366129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789824+0000","last_change":"2024-09-21T06:02:10.279231+0000","last_active":"2024-09-21T06:02:11.789824+0000","last_peered":"2024-09-21T06:02:11.789824+0000","last_clean":"2024-09-21T06:02:11.789824+0000","last_became_active":"2024-09-21T06:02:10.278439+0000","last_became_peered":"2024-09-21T06:02:10.278439+0000","last_unstale":"2024-09-21T06:02:11.789824+0000","last_undegraded":"2024-09-21T06:02:11.789824+0000","last_fullsized":"2024-09-21T06:02:11.789824+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:00.331491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792149+0000","last_change":"2024-09-21T06:02:10.287270+0000","last_active":"2024-09-21T06:02:11.792149+0000","last_peered":"2024-09-21T06:02:11.792149+0000","last_clean":"2024-09-21T06:02:11.792149+0000","last_became_active":"2024-09-21T06:02:10.285162+0000","last_became_peered":"2024-09-21T06:02:10.285162+0000","last_unstale":"2024-09-21T06:02:11.792149+0000","last_undegraded":"2024-09-21T06:02:11.792149+0000","last_fullsized":"2024-09-21T06:02:11.792149+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.910114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290107+0000","last_change":"2024-09-21T06:02:10.288834+0000","last_active":"2024-09-21T06:02:11.290107+0000","last_peered":"2024-09-21T06:02:11.290107+0000","last_clean":"2024-09-21T06:02:11.290107+0000","last_became_active":"2024-09-21T06:02:10.288513+0000","last_became_peered":"2024-09-21T06:02:10.288513+0000","last_unstale":"2024-09-21T06:02:11.290107+0000","last_undegraded":"2024-09-21T06:02:11.290107+0000","last_fullsized":"2024-09-21T06:02:11.290107+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:10.144897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250654+0000","last_change":"2024-09-21T06:02:10.275125+0000","last_active":"2024-09-21T06:02:11.250654+0000","last_peered":"2024-09-21T06:02:11.250654+0000","last_clean":"2024-09-21T06:02:11.250654+0000","last_became_active":"2024-09-21T06:02:10.274544+0000","last_became_peered":"2024-09-21T06:02:10.274544+0000","last_unstale":"2024-09-21T06:02:11.250654+0000","last_undegraded":"2024-09-21T06:02:11.250654+0000","last_fullsized":"2024-09-21T06:02:11.250654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:19.338852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791447+0000","last_change":"2024-09-21T06:02:10.282190+0000","last_active":"2024-09-21T06:02:11.791447+0000","last_peered":"2024-09-21T06:02:11.791447+0000","last_clean":"2024-09-21T06:02:11.791447+0000","last_became_active":"2024-09-21T06:02:10.281877+0000","last_became_peered":"2024-09-21T06:02:10.281877+0000","last_unstale":"2024-09-21T06:02:11.791447+0000","last_undegraded":"2024-09-21T06:02:11.791447+0000","last_fullsized":"2024-09-21T06:02:11.791447+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:43.790427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289797+0000","last_change":"2024-09-21T06:02:10.283752+0000","last_active":"2024-09-21T06:02:11.289797+0000","last_peered":"2024-09-21T06:02:11.289797+0000","last_clean":"2024-09-21T06:02:11.289797+0000","last_became_active":"2024-09-21T06:02:10.283417+0000","last_became_peered":"2024-09-21T06:02:10.283417+0000","last_unstale":"2024-09-21T06:02:11.289797+0000","last_undegraded":"2024-09-21T06:02:11.289797+0000","last_fullsized":"2024-09-21T06:02:11.289797+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:32.351750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791244+0000","last_change":"2024-09-21T06:02:10.282281+0000","last_active":"2024-09-21T06:02:11.791244+0000","last_peered":"2024-09-21T06:02:11.791244+0000","last_clean":"2024-09-21T06:02:11.791244+0000","last_became_active":"2024-09-21T06:02:10.281585+0000","last_became_peered":"2024-09-21T06:02:10.281585+0000","last_unstale":"2024-09-21T06:02:11.791244+0000","last_undegraded":"2024-09-21T06:02:11.791244+0000","last_fullsized":"2024-09-21T06:02:11.791244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:32.507248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290027+0000","last_change":"2024-09-21T06:02:10.287973+0000","last_active":"2024-09-21T06:02:11.290027+0000","last_peered":"2024-09-21T06:02:11.290027+0000","last_clean":"2024-09-21T06:02:11.290027+0000","last_became_active":"2024-09-21T06:02:10.287123+0000","last_became_peered":"2024-09-21T06:02:10.287123+0000","last_unstale":"2024-09-21T06:02:11.290027+0000","last_undegraded":"2024-09-21T06:02:11.290027+0000","last_fullsized":"2024-09-21T06:02:11.290027+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:10.646439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790293+0000","last_change":"2024-09-21T06:02:10.282167+0000","last_active":"2024-09-21T06:02:11.790293+0000","last_peered":"2024-09-21T06:02:11.790293+0000","last_clean":"2024-09-21T06:02:11.790293+0000","last_became_active":"2024-09-21T06:02:10.281870+0000","last_became_peered":"2024-09-21T06:02:10.281870+0000","last_unstale":"2024-09-21T06:02:11.790293+0000","last_undegraded":"2024-09-21T06:02:11.790293+0000","last_fullsized":"2024-09-21T06:02:11.790293+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:40.075266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.251246+0000","last_change":"2024-09-21T06:02:10.275662+0000","last_active":"2024-09-21T06:02:11.251246+0000","last_peered":"2024-09-21T06:02:11.251246+0000","last_clean":"2024-09-21T06:02:11.251246+0000","last_became_active":"2024-09-21T06:02:10.274366+0000","last_became_peered":"2024-09-21T06:02:10.274366+0000","last_unstale":"2024-09-21T06:02:11.251246+0000","last_undegraded":"2024-09-21T06:02:11.251246+0000","last_fullsized":"2024-09-21T06:02:11.251246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:10.180010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254453+0000","last_change":"2024-09-21T06:02:10.263425+0000","last_active":"2024-09-21T06:02:11.254453+0000","last_peered":"2024-09-21T06:02:11.254453+0000","last_clean":"2024-09-21T06:02:11.254453+0000","last_became_active":"2024-09-21T06:02:10.263066+0000","last_became_peered":"2024-09-21T06:02:10.263066+0000","last_unstale":"2024-09-21T06:02:11.254453+0000","last_undegraded":"2024-09-21T06:02:11.254453+0000","last_fullsized":"2024-09-21T06:02:11.254453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:03.263466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790375+0000","last_change":"2024-09-21T06:02:10.291344+0000","last_active":"2024-09-21T06:02:11.790375+0000","last_peered":"2024-09-21T06:02:11.790375+0000","last_clean":"2024-09-21T06:02:11.790375+0000","last_became_active":"2024-09-21T06:02:10.286643+0000","last_became_peered":"2024-09-21T06:02:10.286643+0000","last_unstale":"2024-09-21T06:02:11.790375+0000","last_undegraded":"2024-09-21T06:02:11.790375+0000","last_fullsized":"2024-09-21T06:02:11.790375+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:23.219415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250636+0000","last_change":"2024-09-21T06:02:10.273465+0000","last_active":"2024-09-21T06:02:11.250636+0000","last_peered":"2024-09-21T06:02:11.250636+0000","last_clean":"2024-09-21T06:02:11.250636+0000","last_became_active":"2024-09-21T06:02:10.272894+0000","last_became_peered":"2024-09-21T06:02:10.272894+0000","last_unstale":"2024-09-21T06:02:11.250636+0000","last_undegraded":"2024-09-21T06:02:11.250636+0000","last_fullsized":"2024-09-21T06:02:11.250636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:23.146917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791745+0000","last_change":"2024-09-21T06:02:10.286555+0000","last_active":"2024-09-21T06:02:11.791745+0000","last_peered":"2024-09-21T06:02:11.791745+0000","last_clean":"2024-09-21T06:02:11.791745+0000","last_became_active":"2024-09-21T06:02:10.275977+0000","last_became_peered":"2024-09-21T06:02:10.275977+0000","last_unstale":"2024-09-21T06:02:11.791745+0000","last_undegraded":"2024-09-21T06:02:11.791745+0000","last_fullsized":"2024-09-21T06:02:11.791745+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:49.043147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289689+0000","last_change":"2024-09-21T06:02:10.276271+0000","last_active":"2024-09-21T06:02:11.289689+0000","last_peered":"2024-09-21T06:02:11.289689+0000","last_clean":"2024-09-21T06:02:11.289689+0000","last_became_active":"2024-09-21T06:02:10.275948+0000","last_became_peered":"2024-09-21T06:02:10.275948+0000","last_unstale":"2024-09-21T06:02:11.289689+0000","last_undegraded":"2024-09-21T06:02:11.289689+0000","last_fullsized":"2024-09-21T06:02:11.289689+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:54.824679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.250230+0000","last_change":"2024-09-21T06:02:10.276173+0000","last_active":"2024-09-21T06:02:11.250230+0000","last_peered":"2024-09-21T06:02:11.250230+0000","last_clean":"2024-09-21T06:02:11.250230+0000","last_became_active":"2024-09-21T06:02:10.275712+0000","last_became_peered":"2024-09-21T06:02:10.275712+0000","last_unstale":"2024-09-21T06:02:11.250230+0000","last_undegraded":"2024-09-21T06:02:11.250230+0000","last_fullsized":"2024-09-21T06:02:11.250230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:39.258731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790950+0000","last_change":"2024-09-21T06:02:10.291460+0000","last_active":"2024-09-21T06:02:11.790950+0000","last_peered":"2024-09-21T06:02:11.790950+0000","last_clean":"2024-09-21T06:02:11.790950+0000","last_became_active":"2024-09-21T06:02:10.287310+0000","last_became_peered":"2024-09-21T06:02:10.287310+0000","last_unstale":"2024-09-21T06:02:11.790950+0000","last_undegraded":"2024-09-21T06:02:11.790950+0000","last_fullsized":"2024-09-21T06:02:11.790950+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:44.150713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790426+0000","last_change":"2024-09-21T06:02:10.278806+0000","last_active":"2024-09-21T06:02:11.790426+0000","last_peered":"2024-09-21T06:02:11.790426+0000","last_clean":"2024-09-21T06:02:11.790426+0000","last_became_active":"2024-09-21T06:02:10.278388+0000","last_became_peered":"2024-09-21T06:02:10.278388+0000","last_unstale":"2024-09-21T06:02:11.790426+0000","last_undegraded":"2024-09-21T06:02:11.790426+0000","last_fullsized":"2024-09-21T06:02:11.790426+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:15.541144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790128+0000","last_change":"2024-09-21T06:02:09.259400+0000","last_active":"2024-09-21T06:02:11.790128+0000","last_peered":"2024-09-21T06:02:11.790128+0000","last_clean":"2024-09-21T06:02:11.790128+0000","last_became_active":"2024-09-21T06:02:09.259059+0000","last_became_peered":"2024-09-21T06:02:09.259059+0000","last_unstale":"2024-09-21T06:02:11.790128+0000","last_undegraded":"2024-09-21T06:02:11.790128+0000","last_fullsized":"2024-09-21T06:02:11.790128+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:32.593732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790843+0000","last_change":"2024-09-21T06:02:10.271022+0000","last_active":"2024-09-21T06:02:11.790843+0000","last_peered":"2024-09-21T06:02:11.790843+0000","last_clean":"2024-09-21T06:02:11.790843+0000","last_became_active":"2024-09-21T06:02:10.270717+0000","last_became_peered":"2024-09-21T06:02:10.270717+0000","last_unstale":"2024-09-21T06:02:11.790843+0000","last_undegraded":"2024-09-21T06:02:11.790843+0000","last_fullsized":"2024-09-21T06:02:11.790843+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:25.972520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792150+0000","last_change":"2024-09-21T06:02:09.265832+0000","last_active":"2024-09-21T06:02:11.792150+0000","last_peered":"2024-09-21T06:02:11.792150+0000","last_clean":"2024-09-21T06:02:11.792150+0000","last_became_active":"2024-09-21T06:02:09.265458+0000","last_became_peered":"2024-09-21T06:02:09.265458+0000","last_unstale":"2024-09-21T06:02:11.792150+0000","last_undegraded":"2024-09-21T06:02:11.792150+0000","last_fullsized":"2024-09-21T06:02:11.792150+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:18.597369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790086+0000","last_change":"2024-09-21T06:02:10.259160+0000","last_active":"2024-09-21T06:02:11.790086+0000","last_peered":"2024-09-21T06:02:11.790086+0000","last_clean":"2024-09-21T06:02:11.790086+0000","last_became_active":"2024-09-21T06:02:10.258851+0000","last_became_peered":"2024-09-21T06:02:10.258851+0000","last_unstale":"2024-09-21T06:02:11.790086+0000","last_undegraded":"2024-09-21T06:02:11.790086+0000","last_fullsized":"2024-09-21T06:02:11.790086+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:53.190221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790742+0000","last_change":"2024-09-21T06:02:09.260660+0000","last_active":"2024-09-21T06:02:11.790742+0000","last_peered":"2024-09-21T06:02:11.790742+0000","last_clean":"2024-09-21T06:02:11.790742+0000","last_became_active":"2024-09-21T06:02:09.260356+0000","last_became_peered":"2024-09-21T06:02:09.260356+0000","last_unstale":"2024-09-21T06:02:11.790742+0000","last_undegraded":"2024-09-21T06:02:11.790742+0000","last_fullsized":"2024-09-21T06:02:11.790742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:17.375769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790661+0000","last_change":"2024-09-21T06:02:10.291888+0000","last_active":"2024-09-21T06:02:11.790661+0000","last_peered":"2024-09-21T06:02:11.790661+0000","last_clean":"2024-09-21T06:02:11.790661+0000","last_became_active":"2024-09-21T06:02:10.279623+0000","last_became_peered":"2024-09-21T06:02:10.279623+0000","last_unstale":"2024-09-21T06:02:11.790661+0000","last_undegraded":"2024-09-21T06:02:11.790661+0000","last_fullsized":"2024-09-21T06:02:11.790661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:55.391528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254634+0000","last_change":"2024-09-21T06:02:09.262422+0000","last_active":"2024-09-21T06:02:11.254634+0000","last_peered":"2024-09-21T06:02:11.254634+0000","last_clean":"2024-09-21T06:02:11.254634+0000","last_became_active":"2024-09-21T06:02:09.262078+0000","last_became_peered":"2024-09-21T06:02:09.262078+0000","last_unstale":"2024-09-21T06:02:11.254634+0000","last_undegraded":"2024-09-21T06:02:11.254634+0000","last_fullsized":"2024-09-21T06:02:11.254634+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:12.662429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791513+0000","last_change":"2024-09-21T06:02:10.282522+0000","last_active":"2024-09-21T06:02:11.791513+0000","last_peered":"2024-09-21T06:02:11.791513+0000","last_clean":"2024-09-21T06:02:11.791513+0000","last_became_active":"2024-09-21T06:02:10.282036+0000","last_became_peered":"2024-09-21T06:02:10.282036+0000","last_unstale":"2024-09-21T06:02:11.791513+0000","last_undegraded":"2024-09-21T06:02:11.791513+0000","last_fullsized":"2024-09-21T06:02:11.791513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:53.258944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253029+0000","last_change":"2024-09-21T06:02:09.275773+0000","last_active":"2024-09-21T06:02:11.253029+0000","last_peered":"2024-09-21T06:02:11.253029+0000","last_clean":"2024-09-21T06:02:11.253029+0000","last_became_active":"2024-09-21T06:02:09.275434+0000","last_became_peered":"2024-09-21T06:02:09.275434+0000","last_unstale":"2024-09-21T06:02:11.253029+0000","last_undegraded":"2024-09-21T06:02:11.253029+0000","last_fullsized":"2024-09-21T06:02:11.253029+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:39.067387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791336+0000","last_change":"2024-09-21T06:02:10.286736+0000","last_active":"2024-09-21T06:02:11.791336+0000","last_peered":"2024-09-21T06:02:11.791336+0000","last_clean":"2024-09-21T06:02:11.791336+0000","last_became_active":"2024-09-21T06:02:10.286341+0000","last_became_peered":"2024-09-21T06:02:10.286341+0000","last_unstale":"2024-09-21T06:02:11.791336+0000","last_undegraded":"2024-09-21T06:02:11.791336+0000","last_fullsized":"2024-09-21T06:02:11.791336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:49.283469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789816+0000","last_change":"2024-09-21T06:02:09.270902+0000","last_active":"2024-09-21T06:02:11.789816+0000","last_peered":"2024-09-21T06:02:11.789816+0000","last_clean":"2024-09-21T06:02:11.789816+0000","last_became_active":"2024-09-21T06:02:09.270573+0000","last_became_peered":"2024-09-21T06:02:09.270573+0000","last_unstale":"2024-09-21T06:02:11.789816+0000","last_undegraded":"2024-09-21T06:02:11.789816+0000","last_fullsized":"2024-09-21T06:02:11.789816+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:15.110078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790233+0000","last_change":"2024-09-21T06:02:10.277306+0000","last_active":"2024-09-21T06:02:11.790233+0000","last_peered":"2024-09-21T06:02:11.790233+0000","last_clean":"2024-09-21T06:02:11.790233+0000","last_became_active":"2024-09-21T06:02:10.270375+0000","last_became_peered":"2024-09-21T06:02:10.270375+0000","last_unstale":"2024-09-21T06:02:11.790233+0000","last_undegraded":"2024-09-21T06:02:11.790233+0000","last_fullsized":"2024-09-21T06:02:11.790233+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:26.447373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253638+0000","last_change":"2024-09-21T06:02:09.266199+0000","last_active":"2024-09-21T06:02:11.253638+0000","last_peered":"2024-09-21T06:02:11.253638+0000","last_clean":"2024-09-21T06:02:11.253638+0000","last_became_active":"2024-09-21T06:02:09.265875+0000","last_became_peered":"2024-09-21T06:02:09.265875+0000","last_unstale":"2024-09-21T06:02:11.253638+0000","last_undegraded":"2024-09-21T06:02:11.253638+0000","last_fullsized":"2024-09-21T06:02:11.253638+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:39.929859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791608+0000","last_change":"2024-09-21T06:02:10.269696+0000","last_active":"2024-09-21T06:02:11.791608+0000","last_peered":"2024-09-21T06:02:11.791608+0000","last_clean":"2024-09-21T06:02:11.791608+0000","last_became_active":"2024-09-21T06:02:10.269223+0000","last_became_peered":"2024-09-21T06:02:10.269223+0000","last_unstale":"2024-09-21T06:02:11.791608+0000","last_undegraded":"2024-09-21T06:02:11.791608+0000","last_fullsized":"2024-09-21T06:02:11.791608+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:57.344419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.252862+0000","last_change":"2024-09-21T06:02:09.266846+0000","last_active":"2024-09-21T06:02:11.252862+0000","last_peered":"2024-09-21T06:02:11.252862+0000","last_clean":"2024-09-21T06:02:11.252862+0000","last_became_active":"2024-09-21T06:02:09.266468+0000","last_became_peered":"2024-09-21T06:02:09.266468+0000","last_unstale":"2024-09-21T06:02:11.252862+0000","last_undegraded":"2024-09-21T06:02:11.252862+0000","last_fullsized":"2024-09-21T06:02:11.252862+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:17.833244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253718+0000","last_change":"2024-09-21T06:02:10.273869+0000","last_active":"2024-09-21T06:02:11.253718+0000","last_peered":"2024-09-21T06:02:11.253718+0000","last_clean":"2024-09-21T06:02:11.253718+0000","last_became_active":"2024-09-21T06:02:10.273564+0000","last_became_peered":"2024-09-21T06:02:10.273564+0000","last_unstale":"2024-09-21T06:02:11.253718+0000","last_undegraded":"2024-09-21T06:02:11.253718+0000","last_fullsized":"2024-09-21T06:02:11.253718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:18.383647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290332+0000","last_change":"2024-09-21T06:02:09.270820+0000","last_active":"2024-09-21T06:02:11.290332+0000","last_peered":"2024-09-21T06:02:11.290332+0000","last_clean":"2024-09-21T06:02:11.290332+0000","last_became_active":"2024-09-21T06:02:09.270500+0000","last_became_peered":"2024-09-21T06:02:09.270500+0000","last_unstale":"2024-09-21T06:02:11.290332+0000","last_undegraded":"2024-09-21T06:02:11.290332+0000","last_fullsized":"2024-09-21T06:02:11.290332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:05.116966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290438+0000","last_change":"2024-09-21T06:02:10.278998+0000","last_active":"2024-09-21T06:02:11.290438+0000","last_peered":"2024-09-21T06:02:11.290438+0000","last_clean":"2024-09-21T06:02:11.290438+0000","last_became_active":"2024-09-21T06:02:10.278700+0000","last_became_peered":"2024-09-21T06:02:10.278700+0000","last_unstale":"2024-09-21T06:02:11.290438+0000","last_undegraded":"2024-09-21T06:02:11.290438+0000","last_fullsized":"2024-09-21T06:02:11.290438+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:44.150713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792239+0000","last_change":"2024-09-21T06:02:09.271907+0000","last_active":"2024-09-21T06:02:11.792239+0000","last_peered":"2024-09-21T06:02:11.792239+0000","last_clean":"2024-09-21T06:02:11.792239+0000","last_became_active":"2024-09-21T06:02:09.271563+0000","last_became_peered":"2024-09-21T06:02:09.271563+0000","last_unstale":"2024-09-21T06:02:11.792239+0000","last_undegraded":"2024-09-21T06:02:11.792239+0000","last_fullsized":"2024-09-21T06:02:11.792239+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:59.323612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790136+0000","last_change":"2024-09-21T06:02:10.272781+0000","last_active":"2024-09-21T06:02:11.790136+0000","last_peered":"2024-09-21T06:02:11.790136+0000","last_clean":"2024-09-21T06:02:11.790136+0000","last_became_active":"2024-09-21T06:02:10.272474+0000","last_became_peered":"2024-09-21T06:02:10.272474+0000","last_unstale":"2024-09-21T06:02:11.790136+0000","last_undegraded":"2024-09-21T06:02:11.790136+0000","last_fullsized":"2024-09-21T06:02:11.790136+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:19.569443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290056+0000","last_change":"2024-09-21T06:02:09.278081+0000","last_active":"2024-09-21T06:02:11.290056+0000","last_peered":"2024-09-21T06:02:11.290056+0000","last_clean":"2024-09-21T06:02:11.290056+0000","last_became_active":"2024-09-21T06:02:09.277726+0000","last_became_peered":"2024-09-21T06:02:09.277726+0000","last_unstale":"2024-09-21T06:02:11.290056+0000","last_undegraded":"2024-09-21T06:02:11.290056+0000","last_fullsized":"2024-09-21T06:02:11.290056+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:09.862737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289912+0000","last_change":"2024-09-21T06:02:10.280615+0000","last_active":"2024-09-21T06:02:11.289912+0000","last_peered":"2024-09-21T06:02:11.289912+0000","last_clean":"2024-09-21T06:02:11.289912+0000","last_became_active":"2024-09-21T06:02:10.280351+0000","last_became_peered":"2024-09-21T06:02:10.280351+0000","last_unstale":"2024-09-21T06:02:11.289912+0000","last_undegraded":"2024-09-21T06:02:11.289912+0000","last_fullsized":"2024-09-21T06:02:11.289912+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:55.704062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":21,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253261+0000","last_change":"2024-09-21T06:02:09.259775+0000","last_active":"2024-09-21T06:02:11.253261+0000","last_peered":"2024-09-21T06:02:11.253261+0000","last_clean":"2024-09-21T06:02:11.253261+0000","last_became_active":"2024-09-21T06:02:09.258709+0000","last_became_peered":"2024-09-21T06:02:09.258709+0000","last_unstale":"2024-09-21T06:02:11.253261+0000","last_undegraded":"2024-09-21T06:02:11.253261+0000","last_fullsized":"2024-09-21T06:02:11.253261+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:25.439494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253660+0000","last_change":"2024-09-21T06:02:10.275691+0000","last_active":"2024-09-21T06:02:11.253660+0000","last_peered":"2024-09-21T06:02:11.253660+0000","last_clean":"2024-09-21T06:02:11.253660+0000","last_became_active":"2024-09-21T06:02:10.275339+0000","last_became_peered":"2024-09-21T06:02:10.275339+0000","last_unstale":"2024-09-21T06:02:11.253660+0000","last_undegraded":"2024-09-21T06:02:11.253660+0000","last_fullsized":"2024-09-21T06:02:11.253660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:43.120115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791428+0000","last_change":"2024-09-21T06:02:09.276297+0000","last_active":"2024-09-21T06:02:11.791428+0000","last_peered":"2024-09-21T06:02:11.791428+0000","last_clean":"2024-09-21T06:02:11.791428+0000","last_became_active":"2024-09-21T06:02:09.275994+0000","last_became_peered":"2024-09-21T06:02:09.275994+0000","last_unstale":"2024-09-21T06:02:11.791428+0000","last_undegraded":"2024-09-21T06:02:11.791428+0000","last_fullsized":"2024-09-21T06:02:11.791428+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:54.696183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289767+0000","last_change":"2024-09-21T06:02:10.276275+0000","last_active":"2024-09-21T06:02:11.289767+0000","last_peered":"2024-09-21T06:02:11.289767+0000","last_clean":"2024-09-21T06:02:11.289767+0000","last_became_active":"2024-09-21T06:02:10.276049+0000","last_became_peered":"2024-09-21T06:02:10.276049+0000","last_unstale":"2024-09-21T06:02:11.289767+0000","last_undegraded":"2024-09-21T06:02:11.289767+0000","last_fullsized":"2024-09-21T06:02:11.289767+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:00.003986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789805+0000","last_change":"2024-09-21T06:02:09.267418+0000","last_active":"2024-09-21T06:02:11.789805+0000","last_peered":"2024-09-21T06:02:11.789805+0000","last_clean":"2024-09-21T06:02:11.789805+0000","last_became_active":"2024-09-21T06:02:09.267111+0000","last_became_peered":"2024-09-21T06:02:09.267111+0000","last_unstale":"2024-09-21T06:02:11.789805+0000","last_undegraded":"2024-09-21T06:02:11.789805+0000","last_fullsized":"2024-09-21T06:02:11.789805+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:10.270750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253090+0000","last_change":"2024-09-21T06:02:10.278157+0000","last_active":"2024-09-21T06:02:11.253090+0000","last_peered":"2024-09-21T06:02:11.253090+0000","last_clean":"2024-09-21T06:02:11.253090+0000","last_became_active":"2024-09-21T06:02:10.271151+0000","last_became_peered":"2024-09-21T06:02:10.271151+0000","last_unstale":"2024-09-21T06:02:11.253090+0000","last_undegraded":"2024-09-21T06:02:11.253090+0000","last_fullsized":"2024-09-21T06:02:11.253090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:09.465555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253777+0000","last_change":"2024-09-21T06:02:09.263864+0000","last_active":"2024-09-21T06:02:11.253777+0000","last_peered":"2024-09-21T06:02:11.253777+0000","last_clean":"2024-09-21T06:02:11.253777+0000","last_became_active":"2024-09-21T06:02:09.263556+0000","last_became_peered":"2024-09-21T06:02:09.263556+0000","last_unstale":"2024-09-21T06:02:11.253777+0000","last_undegraded":"2024-09-21T06:02:11.253777+0000","last_fullsized":"2024-09-21T06:02:11.253777+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:08.457676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289746+0000","last_change":"2024-09-21T06:02:10.287377+0000","last_active":"2024-09-21T06:02:11.289746+0000","last_peered":"2024-09-21T06:02:11.289746+0000","last_clean":"2024-09-21T06:02:11.289746+0000","last_became_active":"2024-09-21T06:02:10.287127+0000","last_became_peered":"2024-09-21T06:02:10.287127+0000","last_unstale":"2024-09-21T06:02:11.289746+0000","last_undegraded":"2024-09-21T06:02:11.289746+0000","last_fullsized":"2024-09-21T06:02:11.289746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:54.355475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289357+0000","last_change":"2024-09-21T06:02:09.267367+0000","last_active":"2024-09-21T06:02:11.289357+0000","last_peered":"2024-09-21T06:02:11.289357+0000","last_clean":"2024-09-21T06:02:11.289357+0000","last_became_active":"2024-09-21T06:02:09.267032+0000","last_became_peered":"2024-09-21T06:02:09.267032+0000","last_unstale":"2024-09-21T06:02:11.289357+0000","last_undegraded":"2024-09-21T06:02:11.289357+0000","last_fullsized":"2024-09-21T06:02:11.289357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:06.608982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290157+0000","last_change":"2024-09-21T06:02:10.275595+0000","last_active":"2024-09-21T06:02:11.290157+0000","last_peered":"2024-09-21T06:02:11.290157+0000","last_clean":"2024-09-21T06:02:11.290157+0000","last_became_active":"2024-09-21T06:02:10.275277+0000","last_became_peered":"2024-09-21T06:02:10.275277+0000","last_unstale":"2024-09-21T06:02:11.290157+0000","last_undegraded":"2024-09-21T06:02:11.290157+0000","last_fullsized":"2024-09-21T06:02:11.290157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:00.331491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.792192+0000","last_change":"2024-09-21T06:02:09.274332+0000","last_active":"2024-09-21T06:02:11.792192+0000","last_peered":"2024-09-21T06:02:11.792192+0000","last_clean":"2024-09-21T06:02:11.792192+0000","last_became_active":"2024-09-21T06:02:09.274007+0000","last_became_peered":"2024-09-21T06:02:09.274007+0000","last_unstale":"2024-09-21T06:02:11.792192+0000","last_undegraded":"2024-09-21T06:02:11.792192+0000","last_fullsized":"2024-09-21T06:02:11.792192+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:43.142834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254114+0000","last_change":"2024-09-21T06:02:10.276861+0000","last_active":"2024-09-21T06:02:11.254114+0000","last_peered":"2024-09-21T06:02:11.254114+0000","last_clean":"2024-09-21T06:02:11.254114+0000","last_became_active":"2024-09-21T06:02:10.276566+0000","last_became_peered":"2024-09-21T06:02:10.276566+0000","last_unstale":"2024-09-21T06:02:11.254114+0000","last_undegraded":"2024-09-21T06:02:11.254114+0000","last_fullsized":"2024-09-21T06:02:11.254114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:32.351750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790419+0000","last_change":"2024-09-21T06:02:09.272265+0000","last_active":"2024-09-21T06:02:11.790419+0000","last_peered":"2024-09-21T06:02:11.790419+0000","last_clean":"2024-09-21T06:02:11.790419+0000","last_became_active":"2024-09-21T06:02:09.268092+0000","last_became_peered":"2024-09-21T06:02:09.268092+0000","last_unstale":"2024-09-21T06:02:11.790419+0000","last_undegraded":"2024-09-21T06:02:11.790419+0000","last_fullsized":"2024-09-21T06:02:11.790419+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:00.089511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791877+0000","last_change":"2024-09-21T06:02:10.275046+0000","last_active":"2024-09-21T06:02:11.791877+0000","last_peered":"2024-09-21T06:02:11.791877+0000","last_clean":"2024-09-21T06:02:11.791877+0000","last_became_active":"2024-09-21T06:02:10.274759+0000","last_became_peered":"2024-09-21T06:02:10.274759+0000","last_unstale":"2024-09-21T06:02:11.791877+0000","last_undegraded":"2024-09-21T06:02:11.791877+0000","last_fullsized":"2024-09-21T06:02:11.791877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:17.491964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253777+0000","last_change":"2024-09-21T06:02:09.268844+0000","last_active":"2024-09-21T06:02:11.253777+0000","last_peered":"2024-09-21T06:02:11.253777+0000","last_clean":"2024-09-21T06:02:11.253777+0000","last_became_active":"2024-09-21T06:02:09.268279+0000","last_became_peered":"2024-09-21T06:02:09.268279+0000","last_unstale":"2024-09-21T06:02:11.253777+0000","last_undegraded":"2024-09-21T06:02:11.253777+0000","last_fullsized":"2024-09-21T06:02:11.253777+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:43.064730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290070+0000","last_change":"2024-09-21T06:02:10.283907+0000","last_active":"2024-09-21T06:02:11.290070+0000","last_peered":"2024-09-21T06:02:11.290070+0000","last_clean":"2024-09-21T06:02:11.290070+0000","last_became_active":"2024-09-21T06:02:10.282734+0000","last_became_peered":"2024-09-21T06:02:10.282734+0000","last_unstale":"2024-09-21T06:02:11.290070+0000","last_undegraded":"2024-09-21T06:02:11.290070+0000","last_fullsized":"2024-09-21T06:02:11.290070+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:03.263466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790269+0000","last_change":"2024-09-21T06:02:09.269326+0000","last_active":"2024-09-21T06:02:11.790269+0000","last_peered":"2024-09-21T06:02:11.790269+0000","last_clean":"2024-09-21T06:02:11.790269+0000","last_became_active":"2024-09-21T06:02:09.267680+0000","last_became_peered":"2024-09-21T06:02:09.267680+0000","last_unstale":"2024-09-21T06:02:11.790269+0000","last_undegraded":"2024-09-21T06:02:11.790269+0000","last_fullsized":"2024-09-21T06:02:11.790269+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:22.358250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253933+0000","last_change":"2024-09-21T06:02:10.275681+0000","last_active":"2024-09-21T06:02:11.253933+0000","last_peered":"2024-09-21T06:02:11.253933+0000","last_clean":"2024-09-21T06:02:11.253933+0000","last_became_active":"2024-09-21T06:02:10.275228+0000","last_became_peered":"2024-09-21T06:02:10.275228+0000","last_unstale":"2024-09-21T06:02:11.253933+0000","last_undegraded":"2024-09-21T06:02:11.253933+0000","last_fullsized":"2024-09-21T06:02:11.253933+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:00.003986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791802+0000","last_change":"2024-09-21T06:02:09.266422+0000","last_active":"2024-09-21T06:02:11.791802+0000","last_peered":"2024-09-21T06:02:11.791802+0000","last_clean":"2024-09-21T06:02:11.791802+0000","last_became_active":"2024-09-21T06:02:09.266121+0000","last_became_peered":"2024-09-21T06:02:09.266121+0000","last_unstale":"2024-09-21T06:02:11.791802+0000","last_undegraded":"2024-09-21T06:02:11.791802+0000","last_fullsized":"2024-09-21T06:02:11.791802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:09.638560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790006+0000","last_change":"2024-09-21T06:02:10.265400+0000","last_active":"2024-09-21T06:02:11.790006+0000","last_peered":"2024-09-21T06:02:11.790006+0000","last_clean":"2024-09-21T06:02:11.790006+0000","last_became_active":"2024-09-21T06:02:10.263526+0000","last_became_peered":"2024-09-21T06:02:10.263526+0000","last_unstale":"2024-09-21T06:02:11.790006+0000","last_undegraded":"2024-09-21T06:02:11.790006+0000","last_fullsized":"2024-09-21T06:02:11.790006+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:48.448199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789710+0000","last_change":"2024-09-21T06:02:09.271164+0000","last_active":"2024-09-21T06:02:11.789710+0000","last_peered":"2024-09-21T06:02:11.789710+0000","last_clean":"2024-09-21T06:02:11.789710+0000","last_became_active":"2024-09-21T06:02:09.270848+0000","last_became_peered":"2024-09-21T06:02:09.270848+0000","last_unstale":"2024-09-21T06:02:11.789710+0000","last_undegraded":"2024-09-21T06:02:11.789710+0000","last_fullsized":"2024-09-21T06:02:11.789710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:58.592636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789752+0000","last_change":"2024-09-21T06:02:10.265812+0000","last_active":"2024-09-21T06:02:11.789752+0000","last_peered":"2024-09-21T06:02:11.789752+0000","last_clean":"2024-09-21T06:02:11.789752+0000","last_became_active":"2024-09-21T06:02:10.265514+0000","last_became_peered":"2024-09-21T06:02:10.265514+0000","last_unstale":"2024-09-21T06:02:11.789752+0000","last_undegraded":"2024-09-21T06:02:11.789752+0000","last_fullsized":"2024-09-21T06:02:11.789752+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:19.605248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.252801+0000","last_change":"2024-09-21T06:02:09.267046+0000","last_active":"2024-09-21T06:02:11.252801+0000","last_peered":"2024-09-21T06:02:11.252801+0000","last_clean":"2024-09-21T06:02:11.252801+0000","last_became_active":"2024-09-21T06:02:09.266750+0000","last_became_peered":"2024-09-21T06:02:09.266750+0000","last_unstale":"2024-09-21T06:02:11.252801+0000","last_undegraded":"2024-09-21T06:02:11.252801+0000","last_fullsized":"2024-09-21T06:02:11.252801+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:48.018613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254310+0000","last_change":"2024-09-21T06:02:10.261217+0000","last_active":"2024-09-21T06:02:11.254310+0000","last_peered":"2024-09-21T06:02:11.254310+0000","last_clean":"2024-09-21T06:02:11.254310+0000","last_became_active":"2024-09-21T06:02:10.260828+0000","last_became_peered":"2024-09-21T06:02:10.260828+0000","last_unstale":"2024-09-21T06:02:11.254310+0000","last_undegraded":"2024-09-21T06:02:11.254310+0000","last_fullsized":"2024-09-21T06:02:11.254310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:55.704062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790736+0000","last_change":"2024-09-21T06:02:09.275643+0000","last_active":"2024-09-21T06:02:11.790736+0000","last_peered":"2024-09-21T06:02:11.790736+0000","last_clean":"2024-09-21T06:02:11.790736+0000","last_became_active":"2024-09-21T06:02:09.275319+0000","last_became_peered":"2024-09-21T06:02:09.275319+0000","last_unstale":"2024-09-21T06:02:11.790736+0000","last_undegraded":"2024-09-21T06:02:11.790736+0000","last_fullsized":"2024-09-21T06:02:11.790736+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:46.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254321+0000","last_change":"2024-09-21T06:02:10.270206+0000","last_active":"2024-09-21T06:02:11.254321+0000","last_peered":"2024-09-21T06:02:11.254321+0000","last_clean":"2024-09-21T06:02:11.254321+0000","last_became_active":"2024-09-21T06:02:10.269531+0000","last_became_peered":"2024-09-21T06:02:10.269531+0000","last_unstale":"2024-09-21T06:02:11.254321+0000","last_undegraded":"2024-09-21T06:02:11.254321+0000","last_fullsized":"2024-09-21T06:02:11.254321+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:10.646439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790569+0000","last_change":"2024-09-21T06:02:09.272606+0000","last_active":"2024-09-21T06:02:11.790569+0000","last_peered":"2024-09-21T06:02:11.790569+0000","last_clean":"2024-09-21T06:02:11.790569+0000","last_became_active":"2024-09-21T06:02:09.271935+0000","last_became_peered":"2024-09-21T06:02:09.271935+0000","last_unstale":"2024-09-21T06:02:11.790569+0000","last_undegraded":"2024-09-21T06:02:11.790569+0000","last_fullsized":"2024-09-21T06:02:11.790569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:53.780735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790403+0000","last_change":"2024-09-21T06:02:10.262950+0000","last_active":"2024-09-21T06:02:11.790403+0000","last_peered":"2024-09-21T06:02:11.790403+0000","last_clean":"2024-09-21T06:02:11.790403+0000","last_became_active":"2024-09-21T06:02:10.262636+0000","last_became_peered":"2024-09-21T06:02:10.262636+0000","last_unstale":"2024-09-21T06:02:11.790403+0000","last_undegraded":"2024-09-21T06:02:11.790403+0000","last_fullsized":"2024-09-21T06:02:11.790403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:32.505486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789909+0000","last_change":"2024-09-21T06:02:09.267841+0000","last_active":"2024-09-21T06:02:11.789909+0000","last_peered":"2024-09-21T06:02:11.789909+0000","last_clean":"2024-09-21T06:02:11.789909+0000","last_became_active":"2024-09-21T06:02:09.267444+0000","last_became_peered":"2024-09-21T06:02:09.267444+0000","last_unstale":"2024-09-21T06:02:11.789909+0000","last_undegraded":"2024-09-21T06:02:11.789909+0000","last_fullsized":"2024-09-21T06:02:11.789909+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:33.721188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790709+0000","last_change":"2024-09-21T06:02:10.263121+0000","last_active":"2024-09-21T06:02:11.790709+0000","last_peered":"2024-09-21T06:02:11.790709+0000","last_clean":"2024-09-21T06:02:11.790709+0000","last_became_active":"2024-09-21T06:02:10.262809+0000","last_became_peered":"2024-09-21T06:02:10.262809+0000","last_unstale":"2024-09-21T06:02:11.790709+0000","last_undegraded":"2024-09-21T06:02:11.790709+0000","last_fullsized":"2024-09-21T06:02:11.790709+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:53.926538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.254710+0000","last_change":"2024-09-21T06:02:09.263354+0000","last_active":"2024-09-21T06:02:11.254710+0000","last_peered":"2024-09-21T06:02:11.254710+0000","last_clean":"2024-09-21T06:02:11.254710+0000","last_became_active":"2024-09-21T06:02:09.263051+0000","last_became_peered":"2024-09-21T06:02:09.263051+0000","last_unstale":"2024-09-21T06:02:11.254710+0000","last_undegraded":"2024-09-21T06:02:11.254710+0000","last_fullsized":"2024-09-21T06:02:11.254710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:15.110078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791787+0000","last_change":"2024-09-21T06:02:10.280540+0000","last_active":"2024-09-21T06:02:11.791787+0000","last_peered":"2024-09-21T06:02:11.791787+0000","last_clean":"2024-09-21T06:02:11.791787+0000","last_became_active":"2024-09-21T06:02:10.274182+0000","last_became_peered":"2024-09-21T06:02:10.274182+0000","last_unstale":"2024-09-21T06:02:11.791787+0000","last_undegraded":"2024-09-21T06:02:11.791787+0000","last_fullsized":"2024-09-21T06:02:11.791787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:38.369387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790058+0000","last_change":"2024-09-21T06:02:09.263348+0000","last_active":"2024-09-21T06:02:11.790058+0000","last_peered":"2024-09-21T06:02:11.790058+0000","last_clean":"2024-09-21T06:02:11.790058+0000","last_became_active":"2024-09-21T06:02:09.262942+0000","last_became_peered":"2024-09-21T06:02:09.262942+0000","last_unstale":"2024-09-21T06:02:11.790058+0000","last_undegraded":"2024-09-21T06:02:11.790058+0000","last_fullsized":"2024-09-21T06:02:11.790058+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:13.084051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.290121+0000","last_change":"2024-09-21T06:02:10.288104+0000","last_active":"2024-09-21T06:02:11.290121+0000","last_peered":"2024-09-21T06:02:11.290121+0000","last_clean":"2024-09-21T06:02:11.290121+0000","last_became_active":"2024-09-21T06:02:10.287460+0000","last_became_peered":"2024-09-21T06:02:10.287460+0000","last_unstale":"2024-09-21T06:02:11.290121+0000","last_undegraded":"2024-09-21T06:02:11.290121+0000","last_fullsized":"2024-09-21T06:02:11.290121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:00.353554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.253392+0000","last_change":"2024-09-21T06:02:09.259123+0000","last_active":"2024-09-21T06:02:11.253392+0000","last_peered":"2024-09-21T06:02:11.253392+0000","last_clean":"2024-09-21T06:02:11.253392+0000","last_became_active":"2024-09-21T06:02:09.258760+0000","last_became_peered":"2024-09-21T06:02:09.258760+0000","last_unstale":"2024-09-21T06:02:11.253392+0000","last_undegraded":"2024-09-21T06:02:11.253392+0000","last_fullsized":"2024-09-21T06:02:11.253392+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:42.407138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.790834+0000","last_change":"2024-09-21T06:02:10.261037+0000","last_active":"2024-09-21T06:02:11.790834+0000","last_peered":"2024-09-21T06:02:11.790834+0000","last_clean":"2024-09-21T06:02:11.790834+0000","last_became_active":"2024-09-21T06:02:10.260729+0000","last_became_peered":"2024-09-21T06:02:10.260729+0000","last_unstale":"2024-09-21T06:02:11.790834+0000","last_undegraded":"2024-09-21T06:02:11.790834+0000","last_fullsized":"2024-09-21T06:02:11.790834+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:00.468017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789596+0000","last_change":"2024-09-21T06:02:09.264048+0000","last_active":"2024-09-21T06:02:11.789596+0000","last_peered":"2024-09-21T06:02:11.789596+0000","last_clean":"2024-09-21T06:02:11.789596+0000","last_became_active":"2024-09-21T06:02:09.263708+0000","last_became_peered":"2024-09-21T06:02:09.263708+0000","last_unstale":"2024-09-21T06:02:11.789596+0000","last_undegraded":"2024-09-21T06:02:11.789596+0000","last_fullsized":"2024-09-21T06:02:11.789596+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:25.531925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791392+0000","last_change":"2024-09-21T06:02:10.262498+0000","last_active":"2024-09-21T06:02:11.791392+0000","last_peered":"2024-09-21T06:02:11.791392+0000","last_clean":"2024-09-21T06:02:11.791392+0000","last_became_active":"2024-09-21T06:02:10.262000+0000","last_became_peered":"2024-09-21T06:02:10.262000+0000","last_unstale":"2024-09-21T06:02:11.791392+0000","last_undegraded":"2024-09-21T06:02:11.791392+0000","last_fullsized":"2024-09-21T06:02:11.791392+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:39.062283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791682+0000","last_change":"2024-09-21T06:02:09.265629+0000","last_active":"2024-09-21T06:02:11.791682+0000","last_peered":"2024-09-21T06:02:11.791682+0000","last_clean":"2024-09-21T06:02:11.791682+0000","last_became_active":"2024-09-21T06:02:09.265255+0000","last_became_peered":"2024-09-21T06:02:09.265255+0000","last_unstale":"2024-09-21T06:02:11.791682+0000","last_undegraded":"2024-09-21T06:02:11.791682+0000","last_fullsized":"2024-09-21T06:02:11.791682+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:02.255587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.791518+0000","last_change":"2024-09-21T06:02:10.283113+0000","last_active":"2024-09-21T06:02:11.791518+0000","last_peered":"2024-09-21T06:02:11.791518+0000","last_clean":"2024-09-21T06:02:11.791518+0000","last_became_active":"2024-09-21T06:02:10.282826+0000","last_became_peered":"2024-09-21T06:02:10.282826+0000","last_unstale":"2024-09-21T06:02:11.791518+0000","last_undegraded":"2024-09-21T06:02:11.791518+0000","last_fullsized":"2024-09-21T06:02:11.791518+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:09.236506+0000","last_clean_scrub_stamp":"2024-09-21T06:02:09.236506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:57.773356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.789812+0000","last_change":"2024-09-21T06:02:09.272885+0000","last_active":"2024-09-21T06:02:11.789812+0000","last_peered":"2024-09-21T06:02:11.789812+0000","last_clean":"2024-09-21T06:02:11.789812+0000","last_became_active":"2024-09-21T06:02:09.272520+0000","last_became_peered":"2024-09-21T06:02:09.272520+0000","last_unstale":"2024-09-21T06:02:11.789812+0000","last_undegraded":"2024-09-21T06:02:11.789812+0000","last_fullsized":"2024-09-21T06:02:11.789812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:02:08.228627+0000","last_clean_scrub_stamp":"2024-09-21T06:02:08.228627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:37.738718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":180,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T06:02:11.289489+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:02:11.289489+0000","last_peered":"2024-09-21T06:02:11.289489+0000","last_clean":"2024-09-21T06:02:11.289489+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:02:11.289489+0000","last_undegraded":"2024-09-21T06:02:11.289489+0000","last_fullsized":"2024-09-21T06:02:11.289489+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509515,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40788,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331052,"statfs":{"total":96636764160,"available":96594997248,"internally_reserved":0,"allocated":1302528,"data_stored":880639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509514,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45764,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326076,"statfs":{"total":96636764160,"available":96589901824,"internally_reserved":0,"allocated":700416,"data_stored":290111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509514,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":708608,"data_stored":290763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62416,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309424,"statfs":{"total":96636764160,"available":96572850176,"internally_reserved":0,"allocated":712704,"data_stored":291723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509514,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45772,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326068,"statfs":{"total":96636764160,"available":96589893632,"internally_reserved":0,"allocated":708608,"data_stored":290167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":708608,"data_stored":290167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509514,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":1310720,"data_stored":882673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542219,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49928,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321912,"statfs":{"total":96636764160,"available":96585637888,"internally_reserved":0,"allocated":704512,"data_stored":290201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:02:19.591 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:02:19.591 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:02:19.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.796+0000 7f1a63800640 1 -- 172.21.15.50:0/1105201916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a5c0cd2e0 msgr2=0x7f1a5c070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:19.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.796+0000 7f1a63800640 1 --2- 172.21.15.50:0/1105201916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a5c0cd2e0 0x7f1a5c070e80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009900 tx=0x7f1a4c01c880 comp rx=0 tx=0).stop 2024-09-21T06:02:19.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.796+0000 7f1a63800640 1 -- 172.21.15.50:0/1105201916 shutdown_connections 2024-09-21T06:02:19.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.796+0000 7f1a63800640 1 --2- 172.21.15.50:0/1105201916 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a5c1602a0 0x7f1a5c1606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.796+0000 7f1a63800640 1 --2- 172.21.15.50:0/1105201916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a5c0cd2e0 0x7f1a5c070e80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.796+0000 7f1a63800640 1 -- 172.21.15.50:0/1105201916 >> 172.21.15.50:0/1105201916 conn(0x7f1a5c0b2c40 msgr2=0x7f1a5c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:19.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.796+0000 7f1a63800640 1 -- 172.21.15.50:0/1105201916 shutdown_connections 2024-09-21T06:02:19.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.796+0000 7f1a63800640 1 -- 172.21.15.50:0/1105201916 wait complete. 2024-09-21T06:02:19.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a63800640 1 Processor -- start 2024-09-21T06:02:19.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a63800640 1 -- start start 2024-09-21T06:02:19.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a63800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a5c0cd2e0 0x7f1a5c137670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:19.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a63800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a5c15cd20 0x7f1a5c165360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:19.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a63800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a5c1602a0 0x7f1a5c1658a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:19.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a63800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c16a330 con 0x7f1a5c1602a0 2024-09-21T06:02:19.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a63800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c16a1b0 con 0x7f1a5c15cd20 2024-09-21T06:02:19.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a63800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1a5c16a4b0 con 0x7f1a5c0cd2e0 2024-09-21T06:02:19.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a61800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a5c1602a0 0x7f1a5c1658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:19.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a61800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a5c1602a0 0x7f1a5c1658a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58194/0 (socket says 172.21.15.50:58194) 2024-09-21T06:02:19.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a61800640 1 -- 172.21.15.50:0/539605823 learned_addr learned my addr 172.21.15.50:0/539605823 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a60e00640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a5c0cd2e0 0x7f1a5c137670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a5be00640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a5c15cd20 0x7f1a5c165360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a61800640 1 -- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a5c0cd2e0 msgr2=0x7f1a5c137670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a61800640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a5c0cd2e0 0x7f1a5c137670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a61800640 1 -- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a5c15cd20 msgr2=0x7f1a5c165360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a61800640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a5c15cd20 0x7f1a5c165360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a61800640 1 -- 172.21.15.50:0/539605823 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5c15d3c0 con 0x7f1a5c1602a0 2024-09-21T06:02:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a61800640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a5c1602a0 0x7f1a5c1658a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400c6e0 tx=0x7f1a5400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a58c00640 1 -- 172.21.15.50:0/539605823 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54003b80 con 0x7f1a5c1602a0 2024-09-21T06:02:19.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a58c00640 1 -- 172.21.15.50:0/539605823 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a54003d20 con 0x7f1a5c1602a0 2024-09-21T06:02:19.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a63800640 1 -- 172.21.15.50:0/539605823 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5c166d00 con 0x7f1a5c1602a0 2024-09-21T06:02:19.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a58c00640 1 -- 172.21.15.50:0/539605823 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54005330 con 0x7f1a5c1602a0 2024-09-21T06:02:19.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.800+0000 7f1a63800640 1 -- 172.21.15.50:0/539605823 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a5c166030 con 0x7f1a5c1602a0 2024-09-21T06:02:19.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.804+0000 7f1a63800640 1 -- 172.21.15.50:0/539605823 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a5c13d2b0 con 0x7f1a5c1602a0 2024-09-21T06:02:19.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.804+0000 7f1a58c00640 1 -- 172.21.15.50:0/539605823 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f1a540054d0 con 0x7f1a5c1602a0 2024-09-21T06:02:19.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.804+0000 7f1a58c00640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1a3c073b00 0x7f1a3c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:19.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.804+0000 7f1a58c00640 1 -- 172.21.15.50:0/539605823 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f1a54092830 con 0x7f1a5c1602a0 2024-09-21T06:02:19.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.804+0000 7f1a60e00640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1a3c073b00 0x7f1a3c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:19.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.808+0000 7f1a60e00640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1a3c073b00 0x7f1a3c076010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0af000 tx=0x7f1a4c01b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:19.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:19.808+0000 7f1a58c00640 1 -- 172.21.15.50:0/539605823 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1a5405f330 con 0x7f1a5c1602a0 2024-09-21T06:02:20.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.052+0000 7f1a63800640 1 -- 172.21.15.50:0/539605823 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1a5c070e80 con 0x7f1a5c1602a0 2024-09-21T06:02:20.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.056+0000 7f1a58c00640 1 -- 172.21.15.50:0/539605823 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v31) ==== 190+0+147 (secure 0 0 0) 0x7f1a5405ec30 con 0x7f1a5c1602a0 2024-09-21T06:02:20.058 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-09-21T06:02:20.058 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw== 2024-09-21T06:02:20.058 INFO:teuthology.orchestra.run.smithi050.stdout: caps mds = "allow" 2024-09-21T06:02:20.058 INFO:teuthology.orchestra.run.smithi050.stdout: caps mon = "allow r" 2024-09-21T06:02:20.058 INFO:teuthology.orchestra.run.smithi050.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:02:20.059 INFO:teuthology.orchestra.run.smithi050.stderr:updated caps for client.0 2024-09-21T06:02:20.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.060+0000 7f1a63800640 1 -- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1a3c073b00 msgr2=0x7f1a3c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:20.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.060+0000 7f1a63800640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1a3c073b00 0x7f1a3c076010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0af000 tx=0x7f1a4c01b120 comp rx=0 tx=0).stop 2024-09-21T06:02:20.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.060+0000 7f1a63800640 1 -- 172.21.15.50:0/539605823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a5c1602a0 msgr2=0x7f1a5c1658a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:20.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.060+0000 7f1a63800640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a5c1602a0 0x7f1a5c1658a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400c6e0 tx=0x7f1a5400cba0 comp rx=0 tx=0).stop 2024-09-21T06:02:20.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.060+0000 7f1a63800640 1 -- 172.21.15.50:0/539605823 shutdown_connections 2024-09-21T06:02:20.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.060+0000 7f1a63800640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1a3c073b00 0x7f1a3c076010 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:20.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.060+0000 7f1a63800640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a5c1602a0 0x7f1a5c1658a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:20.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.060+0000 7f1a63800640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a5c15cd20 0x7f1a5c165360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:20.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.060+0000 7f1a63800640 1 --2- 172.21.15.50:0/539605823 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a5c0cd2e0 0x7f1a5c137670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:20.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.060+0000 7f1a63800640 1 -- 172.21.15.50:0/539605823 >> 172.21.15.50:0/539605823 conn(0x7f1a5c0b2c40 msgr2=0x7f1a5c15c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:20.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.060+0000 7f1a63800640 1 -- 172.21.15.50:0/539605823 shutdown_connections 2024-09-21T06:02:20.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.060+0000 7f1a63800640 1 -- 172.21.15.50:0/539605823 wait complete. 2024-09-21T06:02:20.081 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:02:20.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.240+0000 7fd532400640 1 -- 172.21.15.50:0/873527452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd510004810 msgr2=0x7fd510004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:20.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.240+0000 7fd532400640 1 --2- 172.21.15.50:0/873527452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd510004810 0x7fd510004c20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd514009b40 tx=0x7fd51401d220 comp rx=0 tx=0).stop 2024-09-21T06:02:20.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.240+0000 7fd532400640 1 -- 172.21.15.50:0/873527452 shutdown_connections 2024-09-21T06:02:20.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.240+0000 7fd532400640 1 --2- 172.21.15.50:0/873527452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd510004810 0x7fd510004c20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:20.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.240+0000 7fd532400640 1 -- 172.21.15.50:0/873527452 >> 172.21.15.50:0/873527452 conn(0x7fd52c0b2040 msgr2=0x7fd52c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:20.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.240+0000 7fd532400640 1 -- 172.21.15.50:0/873527452 shutdown_connections 2024-09-21T06:02:20.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.240+0000 7fd532400640 1 -- 172.21.15.50:0/873527452 wait complete. 2024-09-21T06:02:20.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.240+0000 7fd532400640 1 Processor -- start 2024-09-21T06:02:20.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.240+0000 7fd532400640 1 -- start start 2024-09-21T06:02:20.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd532400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd510004810 0x7fd52c153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:20.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd532400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c154da0 0x7fd52c155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:20.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd532400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd52c14e770 0x7fd52c1f9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:20.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd532400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c0b5580 con 0x7fd52c154da0 2024-09-21T06:02:20.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd532400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c0b5400 con 0x7fd52c14e770 2024-09-21T06:02:20.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd532400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd52c0b5700 con 0x7fd510004810 2024-09-21T06:02:20.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd522a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c154da0 0x7fd52c155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:20.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd522a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c154da0 0x7fd52c155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58216/0 (socket says 172.21.15.50:58216) 2024-09-21T06:02:20.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd522a00640 1 -- 172.21.15.50:0/4277351995 learned_addr learned my addr 172.21.15.50:0/4277351995 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:20.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd52b400640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd510004810 0x7fd52c153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:20.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd52be00640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd52c14e770 0x7fd52c1f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:20.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd522a00640 1 -- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd510004810 msgr2=0x7fd52c153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:20.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd522a00640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd510004810 0x7fd52c153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:20.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd522a00640 1 -- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd52c14e770 msgr2=0x7fd52c1f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:20.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd522a00640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd52c14e770 0x7fd52c1f9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:20.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd522a00640 1 -- 172.21.15.50:0/4277351995 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c0cfa40 con 0x7fd52c154da0 2024-09-21T06:02:20.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd52be00640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd52c14e770 0x7fd52c1f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:20.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd522a00640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c154da0 0x7fd52c155780 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd51401d050 tx=0x7fd514005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:20.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd523e00640 1 -- 172.21.15.50:0/4277351995 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140bb030 con 0x7fd52c154da0 2024-09-21T06:02:20.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd523e00640 1 -- 172.21.15.50:0/4277351995 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5140046d0 con 0x7fd52c154da0 2024-09-21T06:02:20.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd532400640 1 -- 172.21.15.50:0/4277351995 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52c0cfd20 con 0x7fd52c154da0 2024-09-21T06:02:20.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd523e00640 1 -- 172.21.15.50:0/4277351995 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514004b20 con 0x7fd52c154da0 2024-09-21T06:02:20.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd532400640 1 -- 172.21.15.50:0/4277351995 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52c1f8f50 con 0x7fd52c154da0 2024-09-21T06:02:20.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.244+0000 7fd532400640 1 -- 172.21.15.50:0/4277351995 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52c15f500 con 0x7fd52c154da0 2024-09-21T06:02:20.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.248+0000 7fd523e00640 1 -- 172.21.15.50:0/4277351995 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fd5140be030 con 0x7fd52c154da0 2024-09-21T06:02:20.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.248+0000 7fd523e00640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd508073b00 0x7fd508076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:20.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.248+0000 7fd523e00640 1 -- 172.21.15.50:0/4277351995 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fd51412e2b0 con 0x7fd52c154da0 2024-09-21T06:02:20.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.248+0000 7fd52b400640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd508073b00 0x7fd508076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:20.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.248+0000 7fd52b400640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd508073b00 0x7fd508076010 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd51c001ec0 tx=0x7fd51c006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:20.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.252+0000 7fd523e00640 1 -- 172.21.15.50:0/4277351995 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd514016f60 con 0x7fd52c154da0 2024-09-21T06:02:20.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.468+0000 7fd532400640 1 -- 172.21.15.50:0/4277351995 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd52c15dea0 con 0x7fd52c154da0 2024-09-21T06:02:20.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.468+0000 7fd523e00640 1 -- 172.21.15.50:0/4277351995 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5217 (secure 0 0 0) 0x7fd5140fa880 con 0x7fd52c154da0 2024-09-21T06:02:20.470 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:02:20.470 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":36,"btime":"2024-09-21T06:02:13:274179+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/288562749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":288562749},{"type":"v1","addr":"172.21.15.50:6834","nonce":288562749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5686,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2920049831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2920049831},{"type":"v1","addr":"172.21.15.88:6836","nonce":2920049831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1072517376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1072517376},{"type":"v1","addr":"172.21.15.88:6837","nonce":1072517376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:02:09.689390+0000","modified":"2024-09-21T06:02:12.699140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/451843324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":451843324},{"type":"v1","addr":"172.21.15.50:6835","nonce":451843324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5695,"qdb_cluster":[5695]},"id":4}]} 2024-09-21T06:02:20.470 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 36 2024-09-21T06:02:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.468+0000 7fd532400640 1 -- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd508073b00 msgr2=0x7fd508076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.468+0000 7fd532400640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd508073b00 0x7fd508076010 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd51c001ec0 tx=0x7fd51c006040 comp rx=0 tx=0).stop 2024-09-21T06:02:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.468+0000 7fd532400640 1 -- 172.21.15.50:0/4277351995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c154da0 msgr2=0x7fd52c155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.468+0000 7fd532400640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c154da0 0x7fd52c155780 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd51401d050 tx=0x7fd514005850 comp rx=0 tx=0).stop 2024-09-21T06:02:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.468+0000 7fd532400640 1 -- 172.21.15.50:0/4277351995 shutdown_connections 2024-09-21T06:02:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.468+0000 7fd532400640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd52c14e770 0x7fd52c1f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:20.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.468+0000 7fd532400640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd508073b00 0x7fd508076010 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:20.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.468+0000 7fd532400640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c154da0 0x7fd52c155780 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:20.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.468+0000 7fd532400640 1 --2- 172.21.15.50:0/4277351995 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd510004810 0x7fd52c153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:20.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.472+0000 7fd532400640 1 -- 172.21.15.50:0/4277351995 >> 172.21.15.50:0/4277351995 conn(0x7fd52c0b2040 msgr2=0x7fd52c151cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:20.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.472+0000 7fd532400640 1 -- 172.21.15.50:0/4277351995 shutdown_connections 2024-09-21T06:02:20.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:20.472+0000 7fd532400640 1 -- 172.21.15.50:0/4277351995 wait complete. 2024-09-21T06:02:20.895 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:02:09.689390+0000', 'modified': '2024-09-21T06:02:12.699140+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5695}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5695': {'gid': 5695, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6835/451843324', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6833', 'nonce': 451843324}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 451843324}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5695, 'qdb_cluster': [5695]} 2024-09-21T06:02:20.895 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:02:20.895 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T06:02:21.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 -- 172.21.15.50:0/300946280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93c8074950 msgr2=0x7f93c80b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 --2- 172.21.15.50:0/300946280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93c8074950 0x7f93c80b5260 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009bc0 tx=0x7f93c001cb10 comp rx=0 tx=0).stop 2024-09-21T06:02:21.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 -- 172.21.15.50:0/300946280 shutdown_connections 2024-09-21T06:02:21.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 --2- 172.21.15.50:0/300946280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93c8074950 0x7f93c80b5260 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 --2- 172.21.15.50:0/300946280 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f93c81536f0 0x7f93c815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 -- 172.21.15.50:0/300946280 >> 172.21.15.50:0/300946280 conn(0x7f93c80b4560 msgr2=0x7f93c80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 -- 172.21.15.50:0/300946280 shutdown_connections 2024-09-21T06:02:21.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 -- 172.21.15.50:0/300946280 wait complete. 2024-09-21T06:02:21.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 Processor -- start 2024-09-21T06:02:21.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 -- start start 2024-09-21T06:02:21.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f93c81536f0 0x7f93c81fc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93c8160060 0x7f93c815b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c81496e0 0x7f93c815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8169e60 con 0x7f93c8160060 2024-09-21T06:02:21.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8169ce0 con 0x7f93c81496e0 2024-09-21T06:02:21.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.056+0000 7f93cda00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f93c8169fe0 con 0x7f93c81536f0 2024-09-21T06:02:21.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93bf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93c8160060 0x7f93c815b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93bf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93c8160060 0x7f93c815b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58232/0 (socket says 172.21.15.50:58232) 2024-09-21T06:02:21.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93bf400640 1 -- 172.21.15.50:0/1256605767 learned_addr learned my addr 172.21.15.50:0/1256605767 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:21.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93c7e00640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f93c81536f0 0x7f93c81fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93cd000640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c81496e0 0x7f93c815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93bf400640 1 -- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f93c81536f0 msgr2=0x7f93c81fc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93bf400640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f93c81536f0 0x7f93c81fc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93bf400640 1 -- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c81496e0 msgr2=0x7f93c815ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93bf400640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c81496e0 0x7f93c815ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93bf400640 1 -- 172.21.15.50:0/1256605767 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c8138c30 con 0x7f93c8160060 2024-09-21T06:02:21.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93cd000640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c81496e0 0x7f93c815ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:21.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93c7e00640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f93c81536f0 0x7f93c81fc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:21.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93bf400640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93c8160060 0x7f93c815b4e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f93b0001ec0 tx=0x7f93b000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93c4c00640 1 -- 172.21.15.50:0/1256605767 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0017070 con 0x7f93c8160060 2024-09-21T06:02:21.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93c4c00640 1 -- 172.21.15.50:0/1256605767 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93b0003600 con 0x7f93c8160060 2024-09-21T06:02:21.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93c4c00640 1 -- 172.21.15.50:0/1256605767 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0003a50 con 0x7f93c8160060 2024-09-21T06:02:21.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93cda00640 1 -- 172.21.15.50:0/1256605767 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c81fcb70 con 0x7f93c8160060 2024-09-21T06:02:21.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93cda00640 1 -- 172.21.15.50:0/1256605767 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c8139180 con 0x7f93c8160060 2024-09-21T06:02:21.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.060+0000 7f93c4c00640 1 -- 172.21.15.50:0/1256605767 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f93b001a070 con 0x7f93c8160060 2024-09-21T06:02:21.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93c4c00640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f93a4073b30 0x7f93a4076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93c4c00640 1 -- 172.21.15.50:0/1256605767 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f93b0091dd0 con 0x7f93c8160060 2024-09-21T06:02:21.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93c7e00640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f93a4073b30 0x7f93a4076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93d6589000 1 Processor -- start 2024-09-21T06:02:21.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93d6589000 1 -- start start 2024-09-21T06:02:21.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93c7e00640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f93a4073b30 0x7f93a4076040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009b90 tx=0x7f93b80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93d6589000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7be26460 0x558c7c3676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93d6589000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c216950 0x558c7c367be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93d6589000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 0x558c7c368120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93d6589000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558c7c23c410 con 0x558c7c1ffca0 2024-09-21T06:02:21.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93d6589000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558c7c179b40 con 0x558c7c216950 2024-09-21T06:02:21.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93d6589000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x558c7c168f70 con 0x558c7be26460 2024-09-21T06:02:21.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93cd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 0x558c7c368120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93cd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 0x558c7c368120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58244/0 (socket says 172.21.15.50:58244) 2024-09-21T06:02:21.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93cd000640 1 -- 172.21.15.50:0/1057771824 learned_addr learned my addr 172.21.15.50:0/1057771824 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:21.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93bf400640 1 --2- 172.21.15.50:0/1057771824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c216950 0x558c7c367be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.064+0000 7f93c7e00640 1 --2- 172.21.15.50:0/1057771824 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7be26460 0x558c7c3676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93cd000640 1 -- 172.21.15.50:0/1057771824 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7be26460 msgr2=0x558c7c3676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93cd000640 1 --2- 172.21.15.50:0/1057771824 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7be26460 0x558c7c3676a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93cd000640 1 -- 172.21.15.50:0/1057771824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c216950 msgr2=0x558c7c367be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93cd000640 1 --2- 172.21.15.50:0/1057771824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c216950 0x558c7c367be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93cd000640 1 -- 172.21.15.50:0/1057771824 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c7c1305b0 con 0x558c7c1ffca0 2024-09-21T06:02:21.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93c7e00640 1 --2- 172.21.15.50:0/1057771824 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7be26460 0x558c7c3676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:21.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93cd000640 1 --2- 172.21.15.50:0/1057771824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 0x558c7c368120 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009b10 tx=0x7f93c00198b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889ed77ec6325dbf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93bcc00640 1 -- 172.21.15.50:0/1057771824 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c00b2040 con 0x558c7c1ffca0 2024-09-21T06:02:21.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93bcc00640 1 -- 172.21.15.50:0/1057771824 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93c0004a50 con 0x558c7c1ffca0 2024-09-21T06:02:21.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93bcc00640 1 -- 172.21.15.50:0/1057771824 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0003980 con 0x558c7c1ffca0 2024-09-21T06:02:21.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93d6589000 1 -- 172.21.15.50:0/1057771824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 msgr2=0x558c7c368120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93d6589000 1 --2- 172.21.15.50:0/1057771824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 0x558c7c368120 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009b10 tx=0x7f93c00198b0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93d6589000 1 -- 172.21.15.50:0/1057771824 shutdown_connections 2024-09-21T06:02:21.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93d6589000 1 --2- 172.21.15.50:0/1057771824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 0x558c7c368120 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93d6589000 1 --2- 172.21.15.50:0/1057771824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c216950 0x558c7c367be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93d6589000 1 --2- 172.21.15.50:0/1057771824 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7be26460 0x558c7c3676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93d6589000 1 -- 172.21.15.50:0/1057771824 >> 172.21.15.50:0/1057771824 conn(0x558c7c2ecd90 msgr2=0x558c7c31c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93d6589000 1 -- 172.21.15.50:0/1057771824 shutdown_connections 2024-09-21T06:02:21.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93d6589000 1 -- 172.21.15.50:0/1057771824 wait complete. 2024-09-21T06:02:21.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93d6589000 1 -- start start 2024-09-21T06:02:21.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.068+0000 7f93d6589000 1 Processor -- start 2024-09-21T06:02:21.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 0x558c7c36e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c1ffca0 0x558c7c36ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c216950 0x558c7c36f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558c7c1035f0 con 0x558c7be26460 2024-09-21T06:02:21.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558c7c16cd80 con 0x558c7c1ffca0 2024-09-21T06:02:21.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x558c7c10f520 con 0x558c7c216950 2024-09-21T06:02:21.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 10 client.0 _ll_register_callbacks cb 0x558c7c113830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:21.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93c7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 0x558c7c36e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93c7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 0x558c7c36e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58254/0 (socket says 172.21.15.50:58254) 2024-09-21T06:02:21.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93c7e00640 1 -- 172.21.15.50:0/1177396895 learned_addr learned my addr 172.21.15.50:0/1177396895 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:21.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93cd000640 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c216950 0x558c7c36f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93bf400640 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c1ffca0 0x558c7c36ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93c7e00640 1 -- 172.21.15.50:0/1177396895 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c216950 msgr2=0x558c7c36f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93c7e00640 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c216950 0x558c7c36f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93c7e00640 1 -- 172.21.15.50:0/1177396895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c1ffca0 msgr2=0x558c7c36ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93c7e00640 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c1ffca0 0x558c7c36ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93c7e00640 1 -- 172.21.15.50:0/1177396895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c7c107190 con 0x558c7be26460 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93cd000640 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c216950 0x558c7c36f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93c7e00640 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 0x558c7c36e920 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f93b80044d0 tx=0x7f93b8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97874840074a9b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93b5600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93b5600640 1 -- 172.21.15.50:0/1177396895 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8025070 con 0x558c7be26460 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93b5600640 1 -- 172.21.15.50:0/1177396895 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f93b8012920 con 0x558c7be26460 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 1 -- 172.21.15.50:0/1177396895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558c7c101110 con 0x558c7be26460 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93b5600640 1 -- 172.21.15.50:0/1177396895 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f93b8030050 con 0x558c7be26460 2024-09-21T06:02:21.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 10 client.6084 fetch_fsmap learned FSMap version 36 2024-09-21T06:02:21.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 1 -- 172.21.15.50:0/1177396895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x558c7c1edf10 con 0x558c7be26460 2024-09-21T06:02:21.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93b5600640 1 -- 172.21.15.50:0/1177396895 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558c7c101110 con 0x558c7be26460 2024-09-21T06:02:21.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93b5600640 1 -- 172.21.15.50:0/1177396895 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f93b8033020 con 0x558c7be26460 2024-09-21T06:02:21.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 10 client.6084 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T06:02:21.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 10 client.6084 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:02:21.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 20 client.6084 populate_metadata read hostname 'smithi050' 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.50:6833/451843324,v1:172.21.15.50:6835/451843324] conn(0x558c7be0b7d0 0x558c7c378d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 4 client.6084 mds_command: new command op to 5695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93d6589000 1 -- 172.21.15.50:0/1177396895 --> [v2:172.21.15.50:6833/451843324,v1:172.21.15.50:6835/451843324] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c7c17ba90 con 0x558c7be0b7d0 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93bf400640 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.50:6833/451843324,v1:172.21.15.50:6835/451843324] conn(0x558c7be0b7d0 0x558c7c378d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93bf400640 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.50:6833/451843324,v1:172.21.15.50:6835/451843324] conn(0x558c7be0b7d0 0x558c7c378d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1674c48bbe54cd28 server_cookie=60d2168bfe45d407 in_seq=0 out_seq=0 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.072+0000 7f93b5600640 10 client.6084 ms_handle_connect on v2:172.21.15.50:6833/451843324 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93b5600640 1 -- 172.21.15.50:0/1177396895 <== mds.0 v2:172.21.15.50:6833/451843324 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c7c17ba90 con 0x558c7be0b7d0 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93b5600640 10 client.6084 handle_command_reply: tid=0 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93d6589000 1 client.6084 shutdown 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93d6589000 1 -- 172.21.15.50:0/1177396895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 msgr2=0x558c7c36e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93d6589000 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 0x558c7c36e920 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f93b80044d0 tx=0x7f93b8004740 comp rx=0 tx=0).stop 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93d6589000 1 -- 172.21.15.50:0/1177396895 shutdown_connections 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93d6589000 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.50:6833/451843324,v1:172.21.15.50:6835/451843324] conn(0x558c7be0b7d0 0x558c7c378d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93d6589000 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c216950 0x558c7c36f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93d6589000 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c1ffca0 0x558c7c36ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93d6589000 1 --2- 172.21.15.50:0/1177396895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 0x558c7c36e920 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93b5600640 0 client.6084 ms_handle_reset on v2:172.21.15.50:6833/451843324 2024-09-21T06:02:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93d6589000 1 -- 172.21.15.50:0/1177396895 >> 172.21.15.50:0/1177396895 conn(0x558c7c2ecd90 msgr2=0x558c7c323440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93d6589000 1 -- 172.21.15.50:0/1177396895 shutdown_connections 2024-09-21T06:02:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93d6589000 1 -- 172.21.15.50:0/1177396895 wait complete. 2024-09-21T06:02:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.076+0000 7f93d6589000 20 client.6084 trim_cache size 0 max 16384 2024-09-21T06:02:21.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.092+0000 7f93d6589000 1 Processor -- start 2024-09-21T06:02:21.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 -- start start 2024-09-21T06:02:21.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c2ecd90 0x558c7c3d9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7be26460 0x558c7c3d9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 0x558c7c3da0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8205640 con 0x558c7c1ffca0 2024-09-21T06:02:21.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8204620 con 0x558c7be26460 2024-09-21T06:02:21.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f93c806d7d0 con 0x558c7c2ecd90 2024-09-21T06:02:21.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93cd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 0x558c7c3da0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93cd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 0x558c7c3da0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58270/0 (socket says 172.21.15.50:58270) 2024-09-21T06:02:21.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93cd000640 1 -- 172.21.15.50:0/1330220512 learned_addr learned my addr 172.21.15.50:0/1330220512 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:21.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93cd000640 1 -- 172.21.15.50:0/1330220512 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c2ecd90 msgr2=0x558c7c3d9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93cd000640 1 --2- 172.21.15.50:0/1330220512 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c2ecd90 0x558c7c3d9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93cd000640 1 -- 172.21.15.50:0/1330220512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7be26460 msgr2=0x558c7c3d9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93cd000640 1 --2- 172.21.15.50:0/1330220512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7be26460 0x558c7c3d9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93cd000640 1 -- 172.21.15.50:0/1330220512 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c7c1c4db0 con 0x558c7c1ffca0 2024-09-21T06:02:21.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93cd000640 1 --2- 172.21.15.50:0/1330220512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 0x558c7c3da0b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f93c001c580 tx=0x7f93c0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33694983bd4c932 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93b7e00640 1 -- 172.21.15.50:0/1330220512 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c00b2040 con 0x558c7c1ffca0 2024-09-21T06:02:21.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93b7e00640 1 -- 172.21.15.50:0/1330220512 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93c0004a50 con 0x558c7c1ffca0 2024-09-21T06:02:21.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93b7e00640 1 -- 172.21.15.50:0/1330220512 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0003980 con 0x558c7c1ffca0 2024-09-21T06:02:21.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 -- 172.21.15.50:0/1330220512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 msgr2=0x558c7c3da0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 --2- 172.21.15.50:0/1330220512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 0x558c7c3da0b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f93c001c580 tx=0x7f93c0009500 comp rx=0 tx=0).stop 2024-09-21T06:02:21.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 -- 172.21.15.50:0/1330220512 shutdown_connections 2024-09-21T06:02:21.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 --2- 172.21.15.50:0/1330220512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7c1ffca0 0x558c7c3da0b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 --2- 172.21.15.50:0/1330220512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7be26460 0x558c7c3d9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 --2- 172.21.15.50:0/1330220512 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c2ecd90 0x558c7c3d9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 -- 172.21.15.50:0/1330220512 >> 172.21.15.50:0/1330220512 conn(0x558c7c323740 msgr2=0x558c7c31bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 -- 172.21.15.50:0/1330220512 shutdown_connections 2024-09-21T06:02:21.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.096+0000 7f93d6589000 1 -- 172.21.15.50:0/1330220512 wait complete. 2024-09-21T06:02:21.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93d6589000 1 -- start start 2024-09-21T06:02:21.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93d6589000 1 Processor -- start 2024-09-21T06:02:21.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93d6589000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 0x558c7c3dd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93d6589000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c1ffca0 0x558c7c3dd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93d6589000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c2ecd90 0x558c7c3ddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93d6589000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558c7c1035f0 con 0x558c7be26460 2024-09-21T06:02:21.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93d6589000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8204030 con 0x558c7c1ffca0 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93d6589000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x558c7c16c090 con 0x558c7c2ecd90 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93d6589000 10 client.0 _ll_register_callbacks cb 0x7f93b8009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93c7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 0x558c7c3dd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93c7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 0x558c7c3dd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58276/0 (socket says 172.21.15.50:58276) 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93c7e00640 1 -- 172.21.15.50:0/2252691924 learned_addr learned my addr 172.21.15.50:0/2252691924 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93bf400640 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c1ffca0 0x558c7c3dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93cd000640 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c2ecd90 0x558c7c3ddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93c7e00640 1 -- 172.21.15.50:0/2252691924 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c2ecd90 msgr2=0x558c7c3ddcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93c7e00640 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c2ecd90 0x558c7c3ddcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93c7e00640 1 -- 172.21.15.50:0/2252691924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c1ffca0 msgr2=0x558c7c3dd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93c7e00640 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c1ffca0 0x558c7c3dd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93c7e00640 1 -- 172.21.15.50:0/2252691924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c7c12fab0 con 0x558c7be26460 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93cd000640 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c2ecd90 0x558c7c3ddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93c7e00640 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 0x558c7c3dd240 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f93b8000c10 tx=0x7f93b8004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02bb3d9a9ae0173 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93b5600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:02:21.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93b5600640 1 -- 172.21.15.50:0/2252691924 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8039060 con 0x558c7be26460 2024-09-21T06:02:21.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93d6589000 1 -- 172.21.15.50:0/2252691924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c0004a50 con 0x558c7be26460 2024-09-21T06:02:21.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93b5600640 1 -- 172.21.15.50:0/2252691924 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f93b802a850 con 0x558c7be26460 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93b5600640 1 -- 172.21.15.50:0/2252691924 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f93b803f050 con 0x558c7be26460 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93d6589000 10 client.6090 fetch_fsmap learned FSMap version 36 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.100+0000 7f93d6589000 1 -- 172.21.15.50:0/2252691924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x558c7c10e070 con 0x558c7be26460 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93b5600640 1 -- 172.21.15.50:0/2252691924 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93c0004a50 con 0x558c7be26460 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93b5600640 1 -- 172.21.15.50:0/2252691924 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f93b8042020 con 0x558c7be26460 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 10 client.6090 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 10 client.6090 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 20 client.6090 populate_metadata read hostname 'smithi050' 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.50:6833/451843324,v1:172.21.15.50:6835/451843324] conn(0x558c7be24e80 0x558c7c399610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 4 client.6090 mds_command: new command op to 5695 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 -- 172.21.15.50:0/2252691924 --> [v2:172.21.15.50:6833/451843324,v1:172.21.15.50:6835/451843324] -- command(tid 0: {"prefix": "status"}) -- 0x558c7c177540 con 0x558c7be24e80 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93bf400640 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.50:6833/451843324,v1:172.21.15.50:6835/451843324] conn(0x558c7be24e80 0x558c7c399610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93bf400640 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.50:6833/451843324,v1:172.21.15.50:6835/451843324] conn(0x558c7be24e80 0x558c7c399610 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8cef1985175aa57 server_cookie=5242dccdfe529886 in_seq=0 out_seq=0 2024-09-21T06:02:21.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93b5600640 10 client.6090 ms_handle_connect on v2:172.21.15.50:6833/451843324 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93b5600640 1 -- 172.21.15.50:0/2252691924 <== mds.0 v2:172.21.15.50:6833/451843324 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x558c7c177540 con 0x558c7be24e80 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93b5600640 10 client.6090 handle_command_reply: tid=0 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 client.6090 shutdown 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 -- 172.21.15.50:0/2252691924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 msgr2=0x558c7c3dd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 0x558c7c3dd240 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f93b8000c10 tx=0x7f93b8004050 comp rx=0 tx=0).stop 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 -- 172.21.15.50:0/2252691924 shutdown_connections 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.50:6833/451843324,v1:172.21.15.50:6835/451843324] conn(0x558c7be24e80 0x558c7c399610 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x558c7c2ecd90 0x558c7c3ddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c7c1ffca0 0x558c7c3dd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 --2- 172.21.15.50:0/2252691924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c7be26460 0x558c7c3dd240 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 -- 172.21.15.50:0/2252691924 >> 172.21.15.50:0/2252691924 conn(0x558c7c323740 msgr2=0x558c7c3786a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93b5600640 0 client.6090 ms_handle_reset on v2:172.21.15.50:6833/451843324 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 -- 172.21.15.50:0/2252691924 shutdown_connections 2024-09-21T06:02:21.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 1 -- 172.21.15.50:0/2252691924 wait complete. 2024-09-21T06:02:21.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93d6589000 20 client.6090 trim_cache size 0 max 16384 2024-09-21T06:02:21.109 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:02:21.109 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:02:21.109 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-21T06:02:21.109 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 5695, 2024-09-21T06:02:21.109 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-21T06:02:21.109 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-21T06:02:21.109 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-21T06:02:21.109 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 10.855469651, 2024-09-21T06:02:21.109 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 36, 2024-09-21T06:02:21.109 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 59, 2024-09-21T06:02:21.110 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 59, 2024-09-21T06:02:21.110 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 15.272727561 2024-09-21T06:02:21.110 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:02:21.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93cda00640 1 -- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f93a4073b30 msgr2=0x7f93a4076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93cda00640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f93a4073b30 0x7f93a4076040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009b90 tx=0x7f93b80091e0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93cda00640 1 -- 172.21.15.50:0/1256605767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93c8160060 msgr2=0x7f93c815b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93cda00640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93c8160060 0x7f93c815b4e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f93b0001ec0 tx=0x7f93b000c890 comp rx=0 tx=0).stop 2024-09-21T06:02:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93cda00640 1 -- 172.21.15.50:0/1256605767 shutdown_connections 2024-09-21T06:02:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93cda00640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c81496e0 0x7f93c815ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93cda00640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f93a4073b30 0x7f93a4076040 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93cda00640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93c8160060 0x7f93c815b4e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93cda00640 1 --2- 172.21.15.50:0/1256605767 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f93c81536f0 0x7f93c81fc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93cda00640 1 -- 172.21.15.50:0/1256605767 >> 172.21.15.50:0/1256605767 conn(0x7f93c80b4560 msgr2=0x7f93c8071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93cda00640 1 -- 172.21.15.50:0/1256605767 shutdown_connections 2024-09-21T06:02:21.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.104+0000 7f93cda00640 1 -- 172.21.15.50:0/1256605767 wait complete. 2024-09-21T06:02:21.126 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:02:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.280+0000 7f218b000640 1 -- 172.21.15.50:0/2109818480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21840703a0 msgr2=0x7f218406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.280+0000 7f218b000640 1 --2- 172.21.15.50:0/2109818480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21840703a0 0x7f218406d200 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f217c009bc0 tx=0x7f217c01cb10 comp rx=0 tx=0).stop 2024-09-21T06:02:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.280+0000 7f218b000640 1 -- 172.21.15.50:0/2109818480 shutdown_connections 2024-09-21T06:02:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.280+0000 7f218b000640 1 --2- 172.21.15.50:0/2109818480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21840703a0 0x7f218406d200 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.280+0000 7f218b000640 1 -- 172.21.15.50:0/2109818480 >> 172.21.15.50:0/2109818480 conn(0x7f21840b35f0 msgr2=0x7f21840b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.280+0000 7f218b000640 1 -- 172.21.15.50:0/2109818480 shutdown_connections 2024-09-21T06:02:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.280+0000 7f218b000640 1 -- 172.21.15.50:0/2109818480 wait complete. 2024-09-21T06:02:21.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.284+0000 7f218b000640 1 Processor -- start 2024-09-21T06:02:21.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.284+0000 7f218b000640 1 -- start start 2024-09-21T06:02:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.284+0000 7f218b000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f218406cdf0 0x7f21841fd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.284+0000 7f218b000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21840703a0 0x7f2184155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.284+0000 7f218b000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2184159130 0x7f2184155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.284+0000 7f218b000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f218416a510 con 0x7f218406cdf0 2024-09-21T06:02:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.284+0000 7f218b000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f218416a390 con 0x7f21840703a0 2024-09-21T06:02:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.284+0000 7f218b000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f218416a690 con 0x7f2184159130 2024-09-21T06:02:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.284+0000 7f2183e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f218406cdf0 0x7f21841fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.284+0000 7f2183e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f218406cdf0 0x7f21841fd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58304/0 (socket says 172.21.15.50:58304) 2024-09-21T06:02:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.284+0000 7f2183e00640 1 -- 172.21.15.50:0/2072767383 learned_addr learned my addr 172.21.15.50:0/2072767383 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f2189000640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2184159130 0x7f2184155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f217b400640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21840703a0 0x7f2184155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f2183e00640 1 -- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2184159130 msgr2=0x7f2184155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f2183e00640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2184159130 0x7f2184155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f2183e00640 1 -- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21840703a0 msgr2=0x7f2184155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f2183e00640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21840703a0 0x7f2184155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f2183e00640 1 -- 172.21.15.50:0/2072767383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21840d1430 con 0x7f218406cdf0 2024-09-21T06:02:21.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f2183e00640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f218406cdf0 0x7f21841fd650 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f217400c450 tx=0x7f217400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f2180c00640 1 -- 172.21.15.50:0/2072767383 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174016020 con 0x7f218406cdf0 2024-09-21T06:02:21.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f2180c00640 1 -- 172.21.15.50:0/2072767383 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2174012920 con 0x7f218406cdf0 2024-09-21T06:02:21.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f2180c00640 1 -- 172.21.15.50:0/2072767383 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174012d50 con 0x7f218406cdf0 2024-09-21T06:02:21.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f218b000640 1 -- 172.21.15.50:0/2072767383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218415cdd0 con 0x7f218406cdf0 2024-09-21T06:02:21.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f218b000640 1 -- 172.21.15.50:0/2072767383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f218415d070 con 0x7f218406cdf0 2024-09-21T06:02:21.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f218b000640 1 -- 172.21.15.50:0/2072767383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f218406d200 con 0x7f218406cdf0 2024-09-21T06:02:21.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.288+0000 7f2180c00640 1 -- 172.21.15.50:0/2072767383 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f2174019020 con 0x7f218406cdf0 2024-09-21T06:02:21.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.292+0000 7f2180c00640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2160073b00 0x7f2160076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.292+0000 7f2180c00640 1 -- 172.21.15.50:0/2072767383 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f2174092500 con 0x7f218406cdf0 2024-09-21T06:02:21.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.292+0000 7f217b400640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2160073b00 0x7f2160076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.292+0000 7f217b400640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2160073b00 0x7f2160076010 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f21841f3d10 tx=0x7f216c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.296+0000 7f2180c00640 1 -- 172.21.15.50:0/2072767383 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2174020070 con 0x7f218406cdf0 2024-09-21T06:02:21.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.512+0000 7f218b000640 1 -- 172.21.15.50:0/2072767383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f218415d240 con 0x7f218406cdf0 2024-09-21T06:02:21.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f2180c00640 1 -- 172.21.15.50:0/2072767383 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f218415cdd0 con 0x7f218406cdf0 2024-09-21T06:02:21.516 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:02:21.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f218b000640 1 -- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2160073b00 msgr2=0x7f2160076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f218b000640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2160073b00 0x7f2160076010 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f21841f3d10 tx=0x7f216c009040 comp rx=0 tx=0).stop 2024-09-21T06:02:21.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f218b000640 1 -- 172.21.15.50:0/2072767383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f218406cdf0 msgr2=0x7f21841fd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f218b000640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f218406cdf0 0x7f21841fd650 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f217400c450 tx=0x7f217400c910 comp rx=0 tx=0).stop 2024-09-21T06:02:21.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f218b000640 1 -- 172.21.15.50:0/2072767383 shutdown_connections 2024-09-21T06:02:21.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f218b000640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2184159130 0x7f2184155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f218b000640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21840703a0 0x7f2184155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f218b000640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2160073b00 0x7f2160076010 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f218b000640 1 --2- 172.21.15.50:0/2072767383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f218406cdf0 0x7f21841fd650 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f218b000640 1 -- 172.21.15.50:0/2072767383 >> 172.21.15.50:0/2072767383 conn(0x7f21840b35f0 msgr2=0x7f21840b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f218b000640 1 -- 172.21.15.50:0/2072767383 shutdown_connections 2024-09-21T06:02:21.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21.516+0000 7f218b000640 1 -- 172.21.15.50:0/2072767383 wait complete. 2024-09-21T06:02:21.539 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:02:21.539 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:02:21.539 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-21T06:02:21.539 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:02:21.539 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:02:21.539 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:02:21.539 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:02:21.539 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:02:21.540 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:02:21.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:02:21.540 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42054sec preferred_lft 42054sec 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:8b:d0:0a:5d brd ff:ff:ff:ff:ff:ff 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:02:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:02:21.552 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T06:02:21.552 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:02:21.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:02:21.553 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:02:21.613 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-21T06:02:21.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:02:21.615 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-21T06:02:21.665 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-21T06:02:21.665 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:02:21.665 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-21T06:02:21.665 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T06:02:21.665 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-21T06:02:21.665 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:02:21.665 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:02:21.665 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T06:02:21.665 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:02:21.666 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:02:21.666 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:02:21.666 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T06:02:21.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:02:21.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:02:21.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:02:21.804 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T06:02:21.812 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:02:21.813 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:02:21.813 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:02:21.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:02:21.814 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-09-21T06:02:21.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:02:21.875 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:02:21.927 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:02:21.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:02:21.928 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:02:21.983 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:02:21.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:02:21.983 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:02:22.084 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:02:22.084 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:02:22.084 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:02:22.103 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:02:22.103 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:02:22.103 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:02:22.103 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:02:22.103 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:02:22.114 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:02:22.114 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:02:22.114 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:02:22.114 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:02:22.114 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:02:22.114 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T06:02:22.115 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T06:02:22.115 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T06:02:22.118 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:02:22.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:02:22.118 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:02:22.138 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:02:22.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:02:22.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:02:22.248 INFO:teuthology.orchestra.run.smithi050.stdout:client5839 2024-09-21T06:02:22.248 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:02:22.248 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:02:22.257 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T06:02:22.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.460+0000 7f4464a00640 1 -- 172.21.15.50:0/3498989015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44600b5040 msgr2=0x7f44600cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:22.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.460+0000 7f4464a00640 1 --2- 172.21.15.50:0/3498989015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44600b5040 0x7f44600cfe40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4450009900 tx=0x7f445001c8b0 comp rx=0 tx=0).stop 2024-09-21T06:02:22.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.460+0000 7f4464a00640 1 -- 172.21.15.50:0/3498989015 shutdown_connections 2024-09-21T06:02:22.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.460+0000 7f4464a00640 1 --2- 172.21.15.50:0/3498989015 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44600b3840 0x7f4460072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:22.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.460+0000 7f4464a00640 1 --2- 172.21.15.50:0/3498989015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44600b5040 0x7f44600cfe40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:22.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.460+0000 7f4464a00640 1 -- 172.21.15.50:0/3498989015 >> 172.21.15.50:0/3498989015 conn(0x7f446006cdf0 msgr2=0x7f446006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:22.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.464+0000 7f4464a00640 1 -- 172.21.15.50:0/3498989015 shutdown_connections 2024-09-21T06:02:22.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.464+0000 7f4464a00640 1 -- 172.21.15.50:0/3498989015 wait complete. 2024-09-21T06:02:22.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.464+0000 7f4464a00640 1 Processor -- start 2024-09-21T06:02:22.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.464+0000 7f4464a00640 1 -- start start 2024-09-21T06:02:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.464+0000 7f4464a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44600b3840 0x7f44601632c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.464+0000 7f4464a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44600b5040 0x7f44601f9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.464+0000 7f4464a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44600d0380 0x7f44601fc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.464+0000 7f4464a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4460167bd0 con 0x7f44600b3840 2024-09-21T06:02:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.464+0000 7f4464a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4460167d50 con 0x7f44600b5040 2024-09-21T06:02:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.464+0000 7f4464a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4460167ed0 con 0x7f44600d0380 2024-09-21T06:02:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f445e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44600b3840 0x7f44601632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f445e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44600b3840 0x7f44601632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58306/0 (socket says 172.21.15.50:58306) 2024-09-21T06:02:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f445e800640 1 -- 172.21.15.50:0/3654753289 learned_addr learned my addr 172.21.15.50:0/3654753289 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f4455e00640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44600b5040 0x7f44601f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f445f200640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44600d0380 0x7f44601fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f445e800640 1 -- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44600d0380 msgr2=0x7f44601fc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f445e800640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44600d0380 0x7f44601fc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f445e800640 1 -- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44600b5040 msgr2=0x7f44601f9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f445e800640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44600b5040 0x7f44601f9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f445e800640 1 -- 172.21.15.50:0/3654753289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446015dc30 con 0x7f44600b3840 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f4455e00640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44600b5040 0x7f44601f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f445e800640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44600b3840 0x7f44601632c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4450009850 tx=0x7f44500a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f4457400640 1 -- 172.21.15.50:0/3654753289 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44500b2040 con 0x7f44600b3840 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f4457400640 1 -- 172.21.15.50:0/3654753289 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44500043b0 con 0x7f44600b3840 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f4464a00640 1 -- 172.21.15.50:0/3654753289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446015def0 con 0x7f44600b3840 2024-09-21T06:02:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f4457400640 1 -- 172.21.15.50:0/3654753289 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450005280 con 0x7f44600b3840 2024-09-21T06:02:22.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f4464a00640 1 -- 172.21.15.50:0/3654753289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446014a2a0 con 0x7f44600b3840 2024-09-21T06:02:22.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f4464a00640 1 -- 172.21.15.50:0/3654753289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4460072030 con 0x7f44600b3840 2024-09-21T06:02:22.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.468+0000 7f4457400640 1 -- 172.21.15.50:0/3654753289 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f4450005420 con 0x7f44600b3840 2024-09-21T06:02:22.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.472+0000 7f4457400640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f443c073b00 0x7f443c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:22.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.472+0000 7f4457400640 1 -- 172.21.15.50:0/3654753289 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f445012c600 con 0x7f44600b3840 2024-09-21T06:02:22.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.472+0000 7f4455e00640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f443c073b00 0x7f443c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:22.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.472+0000 7f4455e00640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f443c073b00 0x7f443c076010 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f446015cf60 tx=0x7f4448009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:22.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.476+0000 7f4457400640 1 -- 172.21.15.50:0/3654753289 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f44500164d0 con 0x7f44600b3840 2024-09-21T06:02:22.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:22.696+0000 7f4464a00640 1 -- 172.21.15.50:0/3654753289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f44600b5690 con 0x7f44600b3840 2024-09-21T06:02:23.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.316+0000 7f4457400640 1 -- 172.21.15.50:0/3654753289 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) ==== 136+0+0 (secure 0 0 0) 0x7f44600b5690 con 0x7f44600b3840 2024-09-21T06:02:23.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.320+0000 7f4464a00640 1 -- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f443c073b00 msgr2=0x7f443c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:23.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.320+0000 7f4464a00640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f443c073b00 0x7f443c076010 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f446015cf60 tx=0x7f4448009040 comp rx=0 tx=0).stop 2024-09-21T06:02:23.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.320+0000 7f4464a00640 1 -- 172.21.15.50:0/3654753289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44600b3840 msgr2=0x7f44601632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:23.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.320+0000 7f4464a00640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44600b3840 0x7f44601632c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4450009850 tx=0x7f44500a5770 comp rx=0 tx=0).stop 2024-09-21T06:02:23.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.320+0000 7f4464a00640 1 -- 172.21.15.50:0/3654753289 shutdown_connections 2024-09-21T06:02:23.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.320+0000 7f4464a00640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44600d0380 0x7f44601fc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.320+0000 7f4464a00640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44600b5040 0x7f44601f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.320+0000 7f4464a00640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f443c073b00 0x7f443c076010 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.320+0000 7f4464a00640 1 --2- 172.21.15.50:0/3654753289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44600b3840 0x7f44601632c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.320+0000 7f4464a00640 1 -- 172.21.15.50:0/3654753289 >> 172.21.15.50:0/3654753289 conn(0x7f446006cdf0 msgr2=0x7f4460140ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:23.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.320+0000 7f4464a00640 1 -- 172.21.15.50:0/3654753289 shutdown_connections 2024-09-21T06:02:23.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.320+0000 7f4464a00640 1 -- 172.21.15.50:0/3654753289 wait complete. 2024-09-21T06:02:23.347 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:02:23.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.508+0000 7f0b93a00640 1 -- 172.21.15.50:0/160000809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8c06f170 msgr2=0x7f0b8c06f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:23.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.508+0000 7f0b93a00640 1 --2- 172.21.15.50:0/160000809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8c06f170 0x7f0b8c06f580 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009bc0 tx=0x7f0b8401cb70 comp rx=0 tx=0).stop 2024-09-21T06:02:23.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.508+0000 7f0b93a00640 1 -- 172.21.15.50:0/160000809 shutdown_connections 2024-09-21T06:02:23.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.508+0000 7f0b93a00640 1 --2- 172.21.15.50:0/160000809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8c06f170 0x7f0b8c06f580 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.508+0000 7f0b93a00640 1 -- 172.21.15.50:0/160000809 >> 172.21.15.50:0/160000809 conn(0x7f0b8c0b2c40 msgr2=0x7f0b8c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:23.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.508+0000 7f0b93a00640 1 -- 172.21.15.50:0/160000809 shutdown_connections 2024-09-21T06:02:23.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.508+0000 7f0b93a00640 1 -- 172.21.15.50:0/160000809 wait complete. 2024-09-21T06:02:23.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.512+0000 7f0b93a00640 1 Processor -- start 2024-09-21T06:02:23.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.512+0000 7f0b93a00640 1 -- start start 2024-09-21T06:02:23.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.512+0000 7f0b93a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b8c06f170 0x7f0b8c139b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:23.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.512+0000 7f0b93a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b8c15c5d0 0x7f0b8c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:23.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.512+0000 7f0b93a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8c154640 0x7f0b8c154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:23.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.512+0000 7f0b93a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c169f00 con 0x7f0b8c154640 2024-09-21T06:02:23.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.512+0000 7f0b93a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c169d80 con 0x7f0b8c06f170 2024-09-21T06:02:23.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.512+0000 7f0b93a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8c16a080 con 0x7f0b8c15c5d0 2024-09-21T06:02:23.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.512+0000 7f0b91a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8c154640 0x7f0b8c154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:23.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.512+0000 7f0b91a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8c154640 0x7f0b8c154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58328/0 (socket says 172.21.15.50:58328) 2024-09-21T06:02:23.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.512+0000 7f0b91a00640 1 -- 172.21.15.50:0/2844772301 learned_addr learned my addr 172.21.15.50:0/2844772301 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:23.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b91a00640 1 -- 172.21.15.50:0/2844772301 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b8c15c5d0 msgr2=0x7f0b8c159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:23.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b91a00640 1 --2- 172.21.15.50:0/2844772301 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b8c15c5d0 0x7f0b8c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b91a00640 1 -- 172.21.15.50:0/2844772301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b8c06f170 msgr2=0x7f0b8c139b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:23.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b91a00640 1 --2- 172.21.15.50:0/2844772301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b8c06f170 0x7f0b8c139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b91a00640 1 -- 172.21.15.50:0/2844772301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c154f90 con 0x7f0b8c154640 2024-09-21T06:02:23.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b91a00640 1 --2- 172.21.15.50:0/2844772301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8c154640 0x7f0b8c154a50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0b8401a7a0 tx=0x7f0b84005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:23.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b88c00640 1 -- 172.21.15.50:0/2844772301 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b840bc070 con 0x7f0b8c154640 2024-09-21T06:02:23.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b88c00640 1 -- 172.21.15.50:0/2844772301 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b840042f0 con 0x7f0b8c154640 2024-09-21T06:02:23.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b88c00640 1 -- 172.21.15.50:0/2844772301 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b840b2040 con 0x7f0b8c154640 2024-09-21T06:02:23.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b93a00640 1 -- 172.21.15.50:0/2844772301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c15a1d0 con 0x7f0b8c154640 2024-09-21T06:02:23.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b93a00640 1 -- 172.21.15.50:0/2844772301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c13c770 con 0x7f0b8c154640 2024-09-21T06:02:23.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b93a00640 1 -- 172.21.15.50:0/2844772301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b8c06f580 con 0x7f0b8c154640 2024-09-21T06:02:23.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b88c00640 1 -- 172.21.15.50:0/2844772301 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f0b84004700 con 0x7f0b8c154640 2024-09-21T06:02:23.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.516+0000 7f0b88c00640 1 --2- 172.21.15.50:0/2844772301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b6c073b00 0x7f0b6c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:23.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.520+0000 7f0b88c00640 1 -- 172.21.15.50:0/2844772301 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f0b8412ca00 con 0x7f0b8c154640 2024-09-21T06:02:23.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.520+0000 7f0b91000640 1 --2- 172.21.15.50:0/2844772301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b6c073b00 0x7f0b6c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:23.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.520+0000 7f0b91000640 1 --2- 172.21.15.50:0/2844772301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b6c073b00 0x7f0b6c076010 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009670 tx=0x7f0b80006bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:23.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.524+0000 7f0b88c00640 1 -- 172.21.15.50:0/2844772301 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0b840f3290 con 0x7f0b8c154640 2024-09-21T06:02:23.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.720+0000 7f0b93a00640 1 -- 172.21.15.50:0/2844772301 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f0b8c13a0b0 con 0x7f0b6c073b00 2024-09-21T06:02:23.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b88c00640 1 -- 172.21.15.50:0/2844772301 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f0b8c13a0b0 con 0x7f0b6c073b00 2024-09-21T06:02:23.726 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:02:23.726 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:02:23.726 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:02:23.726 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:02:23.726 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:02:23.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b93a00640 1 -- 172.21.15.50:0/2844772301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b6c073b00 msgr2=0x7f0b6c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:23.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b93a00640 1 --2- 172.21.15.50:0/2844772301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b6c073b00 0x7f0b6c076010 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009670 tx=0x7f0b80006bd0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b93a00640 1 -- 172.21.15.50:0/2844772301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8c154640 msgr2=0x7f0b8c154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:23.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b93a00640 1 --2- 172.21.15.50:0/2844772301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8c154640 0x7f0b8c154a50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0b8401a7a0 tx=0x7f0b84005b10 comp rx=0 tx=0).stop 2024-09-21T06:02:23.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b93a00640 1 -- 172.21.15.50:0/2844772301 shutdown_connections 2024-09-21T06:02:23.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b93a00640 1 --2- 172.21.15.50:0/2844772301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b6c073b00 0x7f0b6c076010 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b93a00640 1 --2- 172.21.15.50:0/2844772301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8c154640 0x7f0b8c154a50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b93a00640 1 --2- 172.21.15.50:0/2844772301 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b8c15c5d0 0x7f0b8c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b93a00640 1 --2- 172.21.15.50:0/2844772301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b8c06f170 0x7f0b8c139b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b93a00640 1 -- 172.21.15.50:0/2844772301 >> 172.21.15.50:0/2844772301 conn(0x7f0b8c0b2c40 msgr2=0x7f0b8c070050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:23.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b93a00640 1 -- 172.21.15.50:0/2844772301 shutdown_connections 2024-09-21T06:02:23.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.724+0000 7f0b93a00640 1 -- 172.21.15.50:0/2844772301 wait complete. 2024-09-21T06:02:23.748 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:02:23.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.908+0000 7f55be200640 1 -- 172.21.15.50:0/2432480841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55b80ce780 msgr2=0x7f55b80ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:23.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.908+0000 7f55be200640 1 --2- 172.21.15.50:0/2432480841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55b80ce780 0x7f55b80ceb90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009bc0 tx=0x7f55b001cb10 comp rx=0 tx=0).stop 2024-09-21T06:02:23.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.908+0000 7f55be200640 1 -- 172.21.15.50:0/2432480841 shutdown_connections 2024-09-21T06:02:23.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.908+0000 7f55be200640 1 --2- 172.21.15.50:0/2432480841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55b80ce780 0x7f55b80ceb90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.908+0000 7f55be200640 1 --2- 172.21.15.50:0/2432480841 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55b8070a30 0x7f55b8160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.908+0000 7f55be200640 1 -- 172.21.15.50:0/2432480841 >> 172.21.15.50:0/2432480841 conn(0x7f55b80d06e0 msgr2=0x7f55b80d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:23.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.908+0000 7f55be200640 1 -- 172.21.15.50:0/2432480841 shutdown_connections 2024-09-21T06:02:23.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.908+0000 7f55be200640 1 -- 172.21.15.50:0/2432480841 wait complete. 2024-09-21T06:02:23.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55be200640 1 Processor -- start 2024-09-21T06:02:23.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55be200640 1 -- start start 2024-09-21T06:02:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55be200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55b8070a30 0x7f55b8166c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55be200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55b80ce780 0x7f55b80abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55be200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b81607b0 0x7f55b8141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55be200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8169f80 con 0x7f55b81607b0 2024-09-21T06:02:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55be200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8169e00 con 0x7f55b8070a30 2024-09-21T06:02:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55be200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f55b816a100 con 0x7f55b80ce780 2024-09-21T06:02:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55b7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b81607b0 0x7f55b8141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55b7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b81607b0 0x7f55b8141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58338/0 (socket says 172.21.15.50:58338) 2024-09-21T06:02:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55b7e00640 1 -- 172.21.15.50:0/136190113 learned_addr learned my addr 172.21.15.50:0/136190113 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:23.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55b7400640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55b8070a30 0x7f55b8166c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55afe00640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55b80ce780 0x7f55b80abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55b7e00640 1 -- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55b80ce780 msgr2=0x7f55b80abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55b7e00640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55b80ce780 0x7f55b80abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55b7e00640 1 -- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55b8070a30 msgr2=0x7f55b8166c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55b7e00640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55b8070a30 0x7f55b8166c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55b7e00640 1 -- 172.21.15.50:0/136190113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b813fd40 con 0x7f55b81607b0 2024-09-21T06:02:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55b7e00640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b81607b0 0x7f55b8141930 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f55b001c580 tx=0x7f55b0003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55af400640 1 -- 172.21.15.50:0/136190113 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b00b2040 con 0x7f55b81607b0 2024-09-21T06:02:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55af400640 1 -- 172.21.15.50:0/136190113 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55b00bb070 con 0x7f55b81607b0 2024-09-21T06:02:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55af400640 1 -- 172.21.15.50:0/136190113 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0004370 con 0x7f55b81607b0 2024-09-21T06:02:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55be200640 1 -- 172.21.15.50:0/136190113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b8140020 con 0x7f55b81607b0 2024-09-21T06:02:23.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.912+0000 7f55be200640 1 -- 172.21.15.50:0/136190113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b81405e0 con 0x7f55b81607b0 2024-09-21T06:02:23.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.916+0000 7f55be200640 1 -- 172.21.15.50:0/136190113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b8160270 con 0x7f55b81607b0 2024-09-21T06:02:23.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.916+0000 7f55af400640 1 -- 172.21.15.50:0/136190113 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f55b0004510 con 0x7f55b81607b0 2024-09-21T06:02:23.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.916+0000 7f55af400640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5594073b00 0x7f5594076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:23.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.916+0000 7f55af400640 1 -- 172.21.15.50:0/136190113 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f55b012d3b0 con 0x7f55b81607b0 2024-09-21T06:02:23.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.916+0000 7f55b7400640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5594073b00 0x7f5594076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:23.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.916+0000 7f55b7400640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5594073b00 0x7f5594076010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f55b80709d0 tx=0x7f55a80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:23.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:23.920+0000 7f55af400640 1 -- 172.21.15.50:0/136190113 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f55b0016790 con 0x7f55b81607b0 2024-09-21T06:02:24.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 -- 172.21.15.50:0/136190113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f55b81671b0 con 0x7f55b81607b0 2024-09-21T06:02:24.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55af400640 1 -- 172.21.15.50:0/136190113 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f55b00f9e30 con 0x7f55b81607b0 2024-09-21T06:02:24.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 -- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5594073b00 msgr2=0x7f5594076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:24.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5594073b00 0x7f5594076010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f55b80709d0 tx=0x7f55a80091e0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 -- 172.21.15.50:0/136190113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b81607b0 msgr2=0x7f55b8141930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:24.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b81607b0 0x7f55b8141930 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f55b001c580 tx=0x7f55b0003f40 comp rx=0 tx=0).stop 2024-09-21T06:02:24.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 -- 172.21.15.50:0/136190113 shutdown_connections 2024-09-21T06:02:24.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5594073b00 0x7f5594076010 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b81607b0 0x7f55b8141930 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55b80ce780 0x7f55b80abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 --2- 172.21.15.50:0/136190113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55b8070a30 0x7f55b8166c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 -- 172.21.15.50:0/136190113 >> 172.21.15.50:0/136190113 conn(0x7f55b80d06e0 msgr2=0x7f55b806fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:24.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 -- 172.21.15.50:0/136190113 shutdown_connections 2024-09-21T06:02:24.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.100+0000 7f55be200640 1 -- 172.21.15.50:0/136190113 wait complete. 2024-09-21T06:02:24.124 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-21T06:02:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.284+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2571508668 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3c00b4de0 msgr2=0x7fc3c00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.284+0000 7fc3c5e00640 1 --2- 172.21.15.50:0/2571508668 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3c00b4de0 0x7fc3c00b51f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009c80 tx=0x7fc3b801cd70 comp rx=0 tx=0).stop 2024-09-21T06:02:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.284+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2571508668 shutdown_connections 2024-09-21T06:02:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.284+0000 7fc3c5e00640 1 --2- 172.21.15.50:0/2571508668 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3c00b4de0 0x7fc3c00b51f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.284+0000 7fc3c5e00640 1 --2- 172.21.15.50:0/2571508668 >> v2:172.21.15.88:3300/0 conn(0x7fc3c01527b0 0x7fc3c015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.284+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2571508668 >> 172.21.15.50:0/2571508668 conn(0x7fc3c00b2040 msgr2=0x7fc3c00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.284+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2571508668 shutdown_connections 2024-09-21T06:02:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.284+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2571508668 wait complete. 2024-09-21T06:02:24.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3c5e00640 1 Processor -- start 2024-09-21T06:02:24.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3c5e00640 1 -- start start 2024-09-21T06:02:24.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3c5e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3c00b4de0 0x7fc3c0156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:24.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3c5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3c00b5c70 0x7fc3c00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:24.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3c5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c01527b0 0x7fc3c00cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:24.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3c5e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c01689d0 con 0x7fc3c01527b0 2024-09-21T06:02:24.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3c5e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c0168850 con 0x7fc3c00b5c70 2024-09-21T06:02:24.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3c5e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c0168b50 con 0x7fc3c00b4de0 2024-09-21T06:02:24.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3bf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c01527b0 0x7fc3c00cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:24.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3bf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c01527b0 0x7fc3c00cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58354/0 (socket says 172.21.15.50:58354) 2024-09-21T06:02:24.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3bf400640 1 -- 172.21.15.50:0/2314619719 learned_addr learned my addr 172.21.15.50:0/2314619719 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:24.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3b7e00640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3c00b5c70 0x7fc3c00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:24.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3bea00640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3c00b4de0 0x7fc3c0156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:24.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3bf400640 1 -- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3c00b4de0 msgr2=0x7fc3c0156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:24.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3bf400640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3c00b4de0 0x7fc3c0156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3bf400640 1 -- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3c00b5c70 msgr2=0x7fc3c00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:24.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3bf400640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3c00b5c70 0x7fc3c00cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3bf400640 1 -- 172.21.15.50:0/2314619719 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c01f8710 con 0x7fc3c01527b0 2024-09-21T06:02:24.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3b7e00640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3c00b5c70 0x7fc3c00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:24.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3bf400640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c01527b0 0x7fc3c00cf9e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc3b801a9a0 tx=0x7fc3b8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:24.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3b7400640 1 -- 172.21.15.50:0/2314619719 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b80bb070 con 0x7fc3c01527b0 2024-09-21T06:02:24.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3b7400640 1 -- 172.21.15.50:0/2314619719 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3b8004b40 con 0x7fc3c01527b0 2024-09-21T06:02:24.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.288+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2314619719 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c01566c0 con 0x7fc3c01527b0 2024-09-21T06:02:24.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.292+0000 7fc3b7400640 1 -- 172.21.15.50:0/2314619719 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b80b2040 con 0x7fc3c01527b0 2024-09-21T06:02:24.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.292+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2314619719 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c00cdda0 con 0x7fc3c01527b0 2024-09-21T06:02:24.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.292+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2314619719 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c00b51f0 con 0x7fc3c01527b0 2024-09-21T06:02:24.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.292+0000 7fc3b7400640 1 -- 172.21.15.50:0/2314619719 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fc3b80be070 con 0x7fc3c01527b0 2024-09-21T06:02:24.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.292+0000 7fc3b7400640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc39c073b30 0x7fc39c076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:24.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.292+0000 7fc3b7400640 1 -- 172.21.15.50:0/2314619719 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fc3b812c8f0 con 0x7fc3c01527b0 2024-09-21T06:02:24.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.292+0000 7fc3bea00640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc39c073b30 0x7fc39c076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:24.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.296+0000 7fc3bea00640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc39c073b30 0x7fc39c076040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0001650 tx=0x7fc3b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:24.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.296+0000 7fc3b7400640 1 -- 172.21.15.50:0/2314619719 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc3b8016990 con 0x7fc3c01527b0 2024-09-21T06:02:24.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.480+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2314619719 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fc3c00cd850 con 0x7fc3c01527b0 2024-09-21T06:02:24.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.480+0000 7fc3b7400640 1 -- 172.21.15.50:0/2314619719 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v38) ==== 116+0+2 (secure 0 0 0) 0x7fc3b80f9360 con 0x7fc3c01527b0 2024-09-21T06:02:24.482 INFO:teuthology.orchestra.run.smithi050.stdout:4 2024-09-21T06:02:24.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.484+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc39c073b30 msgr2=0x7fc39c076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:24.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.484+0000 7fc3c5e00640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc39c073b30 0x7fc39c076040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0001650 tx=0x7fc3b0009040 comp rx=0 tx=0).stop 2024-09-21T06:02:24.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.484+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2314619719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c01527b0 msgr2=0x7fc3c00cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:24.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.484+0000 7fc3c5e00640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c01527b0 0x7fc3c00cf9e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc3b801a9a0 tx=0x7fc3b8005b00 comp rx=0 tx=0).stop 2024-09-21T06:02:24.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.484+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2314619719 shutdown_connections 2024-09-21T06:02:24.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.484+0000 7fc3c5e00640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc39c073b30 0x7fc39c076040 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.484+0000 7fc3c5e00640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c01527b0 0x7fc3c00cf9e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.484+0000 7fc3c5e00640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3c00b5c70 0x7fc3c00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.484+0000 7fc3c5e00640 1 --2- 172.21.15.50:0/2314619719 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3c00b4de0 0x7fc3c0156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.484+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2314619719 >> 172.21.15.50:0/2314619719 conn(0x7fc3c00b2040 msgr2=0x7fc3c00b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:24.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.484+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2314619719 shutdown_connections 2024-09-21T06:02:24.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.484+0000 7fc3c5e00640 1 -- 172.21.15.50:0/2314619719 wait complete. 2024-09-21T06:02:24.505 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:02:24.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.664+0000 7f4595000640 1 -- 172.21.15.50:0/1894501729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b4de0 msgr2=0x7f45900b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:24.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.664+0000 7f4595000640 1 --2- 172.21.15.50:0/1894501729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b4de0 0x7f45900b51f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4588009c80 tx=0x7f458801cce0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.664+0000 7f4595000640 1 -- 172.21.15.50:0/1894501729 shutdown_connections 2024-09-21T06:02:24.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.664+0000 7f4595000640 1 --2- 172.21.15.50:0/1894501729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b4de0 0x7f45900b51f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.664+0000 7f4595000640 1 --2- 172.21.15.50:0/1894501729 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45900b5c70 0x7f459015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.664+0000 7f4595000640 1 -- 172.21.15.50:0/1894501729 >> 172.21.15.50:0/1894501729 conn(0x7f45900b2040 msgr2=0x7f45900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:24.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.664+0000 7f4595000640 1 -- 172.21.15.50:0/1894501729 shutdown_connections 2024-09-21T06:02:24.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.664+0000 7f4595000640 1 -- 172.21.15.50:0/1894501729 wait complete. 2024-09-21T06:02:24.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4595000640 1 Processor -- start 2024-09-21T06:02:24.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4595000640 1 -- start start 2024-09-21T06:02:24.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4595000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45900b4de0 0x7f4590157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:24.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4595000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b5c70 0x7f4590144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:24.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4595000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45901527b0 0x7f45901fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:24.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4595000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4590168ab0 con 0x7f45900b5c70 2024-09-21T06:02:24.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4595000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4590168930 con 0x7f45900b4de0 2024-09-21T06:02:24.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4595000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4590168c30 con 0x7f45901527b0 2024-09-21T06:02:24.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4587e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b5c70 0x7f4590144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:24.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4587e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b5c70 0x7f4590144b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58372/0 (socket says 172.21.15.50:58372) 2024-09-21T06:02:24.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4587e00640 1 -- 172.21.15.50:0/2652796541 learned_addr learned my addr 172.21.15.50:0/2652796541 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:24.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f458de00640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45900b4de0 0x7f4590157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:24.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f458e800640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45901527b0 0x7f45901fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:24.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4587e00640 1 -- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45901527b0 msgr2=0x7f45901fbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:24.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4587e00640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45901527b0 0x7f45901fbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4587e00640 1 -- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45900b4de0 msgr2=0x7f4590157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:24.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4587e00640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45900b4de0 0x7f4590157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4587e00640 1 -- 172.21.15.50:0/2652796541 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4590157b10 con 0x7f45900b5c70 2024-09-21T06:02:24.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4587e00640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b5c70 0x7f4590144b60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f457800c390 tx=0x7f457800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:24.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4586000640 1 -- 172.21.15.50:0/2652796541 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578017070 con 0x7f45900b5c70 2024-09-21T06:02:24.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4586000640 1 -- 172.21.15.50:0/2652796541 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45780053c0 con 0x7f45900b5c70 2024-09-21T06:02:24.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4595000640 1 -- 172.21.15.50:0/2652796541 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4590157df0 con 0x7f45900b5c70 2024-09-21T06:02:24.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4586000640 1 -- 172.21.15.50:0/2652796541 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578003840 con 0x7f45900b5c70 2024-09-21T06:02:24.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.668+0000 7f4595000640 1 -- 172.21.15.50:0/2652796541 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45901583b0 con 0x7f45900b5c70 2024-09-21T06:02:24.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.672+0000 7f4595000640 1 -- 172.21.15.50:0/2652796541 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45900b51f0 con 0x7f45900b5c70 2024-09-21T06:02:24.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.672+0000 7f4586000640 1 -- 172.21.15.50:0/2652796541 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f457801a070 con 0x7f45900b5c70 2024-09-21T06:02:24.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.672+0000 7f4586000640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f456c073b00 0x7f456c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:24.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.672+0000 7f4586000640 1 -- 172.21.15.50:0/2652796541 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f4578091da0 con 0x7f45900b5c70 2024-09-21T06:02:24.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.672+0000 7f458de00640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f456c073b00 0x7f456c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:24.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.672+0000 7f458de00640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f456c073b00 0x7f456c076010 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4580004630 tx=0x7f458000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:24.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.676+0000 7f4586000640 1 -- 172.21.15.50:0/2652796541 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f45900b5730 con 0x7f45900b5c70 2024-09-21T06:02:24.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.852+0000 7f4595000640 1 -- 172.21.15.50:0/2652796541 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f45901578a0 con 0x7f45900b5c70 2024-09-21T06:02:24.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.856+0000 7f4586000640 1 -- 172.21.15.50:0/2652796541 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v39)=0 v39) ==== 131+0+0 (secure 0 0 0) 0x7f457805e200 con 0x7f45900b5c70 2024-09-21T06:02:24.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.860+0000 7f4595000640 1 -- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f456c073b00 msgr2=0x7f456c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:24.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.860+0000 7f4595000640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f456c073b00 0x7f456c076010 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4580004630 tx=0x7f458000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.860+0000 7f4595000640 1 -- 172.21.15.50:0/2652796541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b5c70 msgr2=0x7f4590144b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:24.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.860+0000 7f4595000640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b5c70 0x7f4590144b60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f457800c390 tx=0x7f457800c850 comp rx=0 tx=0).stop 2024-09-21T06:02:24.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.860+0000 7f4595000640 1 -- 172.21.15.50:0/2652796541 shutdown_connections 2024-09-21T06:02:24.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.860+0000 7f4595000640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45901527b0 0x7f45901fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.860+0000 7f4595000640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f456c073b00 0x7f456c076010 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.860+0000 7f4595000640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45900b5c70 0x7f4590144b60 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.860+0000 7f4595000640 1 --2- 172.21.15.50:0/2652796541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45900b4de0 0x7f4590157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:24.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.860+0000 7f4595000640 1 -- 172.21.15.50:0/2652796541 >> 172.21.15.50:0/2652796541 conn(0x7f45900b2040 msgr2=0x7f4590151c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:24.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.860+0000 7f4595000640 1 -- 172.21.15.50:0/2652796541 shutdown_connections 2024-09-21T06:02:24.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:24.860+0000 7f4595000640 1 -- 172.21.15.50:0/2652796541 wait complete. 2024-09-21T06:02:24.877 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:02:24.875+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:02:24.878 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:02:24.875+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:02:24.878 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:02:24.875+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:02:24.878 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:02:24.875+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:02:24.878 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:02:24.875+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:02:24.883 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-21T06:02:25.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.100+0000 7f2c40400640 1 -- 172.21.15.50:0/3293104775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c20004d50 msgr2=0x7f2c20005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:25.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.100+0000 7f2c40400640 1 --2- 172.21.15.50:0/3293104775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c20004d50 0x7f2c20005160 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009e90 tx=0x7f2c3001d590 comp rx=0 tx=0).stop 2024-09-21T06:02:25.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.100+0000 7f2c3e400640 1 -- 172.21.15.50:0/3293104775 reap_dead start 2024-09-21T06:02:25.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.100+0000 7f2c40400640 1 -- 172.21.15.50:0/3293104775 shutdown_connections 2024-09-21T06:02:25.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.100+0000 7f2c40400640 1 -- 172.21.15.50:0/3293104775 >> 172.21.15.50:0/3293104775 conn(0x7f2c380b2040 msgr2=0x7f2c380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:25.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.100+0000 7f2c40400640 1 -- 172.21.15.50:0/3293104775 shutdown_connections 2024-09-21T06:02:25.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.100+0000 7f2c40400640 1 -- 172.21.15.50:0/3293104775 wait complete. 2024-09-21T06:02:25.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.100+0000 7f2c40400640 1 Processor -- start 2024-09-21T06:02:25.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c40400640 1 -- start start 2024-09-21T06:02:25.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c40400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2c20004d50 0x7f2c380af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:25.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c40400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3813f340 0x7f2c381f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:25.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c40400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c381f5660 0x7f2c3813c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:25.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c40400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3815cd20 con 0x7f2c3813f340 2024-09-21T06:02:25.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c40400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3815cba0 con 0x7f2c381f5660 2024-09-21T06:02:25.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c40400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3815cea0 con 0x7f2c20004d50 2024-09-21T06:02:25.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c3d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3813f340 0x7f2c381f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:25.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c3d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3813f340 0x7f2c381f5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58402/0 (socket says 172.21.15.50:58402) 2024-09-21T06:02:25.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c3da00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2c20004d50 0x7f2c380af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:25.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c3d000640 1 -- 172.21.15.50:0/577805045 learned_addr learned my addr 172.21.15.50:0/577805045 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:25.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c3e400640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c381f5660 0x7f2c3813c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:25.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c3d000640 1 -- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2c20004d50 msgr2=0x7f2c380af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:25.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c3d000640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2c20004d50 0x7f2c380af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c3d000640 1 -- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c381f5660 msgr2=0x7f2c3813c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:25.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c3d000640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c381f5660 0x7f2c3813c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c3d000640 1 -- 172.21.15.50:0/577805045 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c380af710 con 0x7f2c3813f340 2024-09-21T06:02:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c3d000640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3813f340 0x7f2c381f5120 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2c24011ae0 tx=0x7f2c24009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c35600640 1 -- 172.21.15.50:0/577805045 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2400fc50 con 0x7f2c3813f340 2024-09-21T06:02:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c35600640 1 -- 172.21.15.50:0/577805045 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c2400fdf0 con 0x7f2c3813f340 2024-09-21T06:02:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c35600640 1 -- 172.21.15.50:0/577805045 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24004280 con 0x7f2c3813f340 2024-09-21T06:02:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c40400640 1 -- 172.21.15.50:0/577805045 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c380ce5e0 con 0x7f2c3813f340 2024-09-21T06:02:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c40400640 1 -- 172.21.15.50:0/577805045 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c380ceba0 con 0x7f2c3813f340 2024-09-21T06:02:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.104+0000 7f2c40400640 1 -- 172.21.15.50:0/577805045 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3813d3a0 con 0x7f2c3813f340 2024-09-21T06:02:25.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.108+0000 7f2c35600640 1 -- 172.21.15.50:0/577805045 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f2c24003600 con 0x7f2c3813f340 2024-09-21T06:02:25.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.108+0000 7f2c35600640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2c18073b60 0x7f2c18076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:25.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.108+0000 7f2c35600640 1 -- 172.21.15.50:0/577805045 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f2c24093500 con 0x7f2c3813f340 2024-09-21T06:02:25.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.108+0000 7f2c3da00640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2c18073b60 0x7f2c18076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:25.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.108+0000 7f2c3da00640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2c18073b60 0x7f2c18076070 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c001de0 tx=0x7f2c2c001d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:25.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.112+0000 7f2c35600640 1 -- 172.21.15.50:0/577805045 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2c3813d3a0 con 0x7f2c3813f340 2024-09-21T06:02:25.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.316+0000 7f2c40400640 1 -- 172.21.15.50:0/577805045 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f2c3806b540 con 0x7f2c18073b60 2024-09-21T06:02:25.332 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:02:25.327+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:02:25.332 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:02:25.327+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:02:25.332 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:02:25.327+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:02:25.332 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:02:25.327+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:02:25.332 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:02:25.327+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:02:25.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c35600640 1 -- 172.21.15.50:0/577805045 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2c2c001790 con 0x7f2c18073b60 2024-09-21T06:02:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c40400640 1 -- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2c18073b60 msgr2=0x7f2c18076070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c40400640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2c18073b60 0x7f2c18076070 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c001de0 tx=0x7f2c2c001d70 comp rx=0 tx=0).stop 2024-09-21T06:02:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c40400640 1 -- 172.21.15.50:0/577805045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3813f340 msgr2=0x7f2c381f5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c40400640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3813f340 0x7f2c381f5120 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2c24011ae0 tx=0x7f2c24009f20 comp rx=0 tx=0).stop 2024-09-21T06:02:25.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c40400640 1 -- 172.21.15.50:0/577805045 shutdown_connections 2024-09-21T06:02:25.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c40400640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c381f5660 0x7f2c3813c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c40400640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2c18073b60 0x7f2c18076070 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c40400640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3813f340 0x7f2c381f5120 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c40400640 1 --2- 172.21.15.50:0/577805045 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2c20004d50 0x7f2c380af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c40400640 1 -- 172.21.15.50:0/577805045 >> 172.21.15.50:0/577805045 conn(0x7f2c380b2040 msgr2=0x7f2c380b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:25.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c40400640 1 -- 172.21.15.50:0/577805045 shutdown_connections 2024-09-21T06:02:25.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.376+0000 7f2c40400640 1 -- 172.21.15.50:0/577805045 wait complete. 2024-09-21T06:02:25.400 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:02:25.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.560+0000 7f92a4e00640 1 -- 172.21.15.50:0/2622992215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a00749a0 msgr2=0x7f92a0074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:25.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.560+0000 7f92a4e00640 1 --2- 172.21.15.50:0/2622992215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a00749a0 0x7f92a0074db0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f92a00cfc90 tx=0x7f929001c8c0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.560+0000 7f92a4e00640 1 -- 172.21.15.50:0/2622992215 shutdown_connections 2024-09-21T06:02:25.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.560+0000 7f92a4e00640 1 --2- 172.21.15.50:0/2622992215 >> v2:172.21.15.88:3300/0 conn(0x7f92a00b3340 0x7f92a00b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.560+0000 7f92a4e00640 1 --2- 172.21.15.50:0/2622992215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a00749a0 0x7f92a0074db0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.560+0000 7f92a4e00640 1 -- 172.21.15.50:0/2622992215 >> 172.21.15.50:0/2622992215 conn(0x7f92a00b2040 msgr2=0x7f92a00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:25.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.560+0000 7f92a4e00640 1 -- 172.21.15.50:0/2622992215 shutdown_connections 2024-09-21T06:02:25.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.560+0000 7f92a4e00640 1 -- 172.21.15.50:0/2622992215 wait complete. 2024-09-21T06:02:25.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.560+0000 7f92a4e00640 1 Processor -- start 2024-09-21T06:02:25.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.560+0000 7f92a4e00640 1 -- start start 2024-09-21T06:02:25.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f92a4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92a00749a0 0x7f92a015fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:25.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f92a4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a00b3340 0x7f92a013dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:25.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f92a4e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f92a00cefa0 0x7f92a013f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:25.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f92a4e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0169d20 con 0x7f92a00b3340 2024-09-21T06:02:25.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f92a4e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0169ba0 con 0x7f92a00749a0 2024-09-21T06:02:25.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f92a4e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f92a0169ea0 con 0x7f92a00cefa0 2024-09-21T06:02:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f9297e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a00b3340 0x7f92a013dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f9297e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a00b3340 0x7f92a013dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58418/0 (socket says 172.21.15.50:58418) 2024-09-21T06:02:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f9297e00640 1 -- 172.21.15.50:0/3094312653 learned_addr learned my addr 172.21.15.50:0/3094312653 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:02:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f929de00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92a00749a0 0x7f92a015fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f929e800640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f92a00cefa0 0x7f92a013f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f9297e00640 1 -- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f92a00cefa0 msgr2=0x7f92a013f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f9297e00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f92a00cefa0 0x7f92a013f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f9297e00640 1 -- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92a00749a0 msgr2=0x7f92a015fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f9297e00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92a00749a0 0x7f92a015fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f9297e00640 1 -- 172.21.15.50:0/3094312653 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a0145920 con 0x7f92a00b3340 2024-09-21T06:02:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f929de00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92a00749a0 0x7f92a015fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:02:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f9297e00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a00b3340 0x7f92a013dab0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f928800c3f0 tx=0x7f928800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:25.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f9296000640 1 -- 172.21.15.50:0/3094312653 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288017070 con 0x7f92a00b3340 2024-09-21T06:02:25.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f9296000640 1 -- 172.21.15.50:0/3094312653 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92880053c0 con 0x7f92a00b3340 2024-09-21T06:02:25.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f92a4e00640 1 -- 172.21.15.50:0/3094312653 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a0145c00 con 0x7f92a00b3340 2024-09-21T06:02:25.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f9296000640 1 -- 172.21.15.50:0/3094312653 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288003860 con 0x7f92a00b3340 2024-09-21T06:02:25.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f92a4e00640 1 -- 172.21.15.50:0/3094312653 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a01461c0 con 0x7f92a00b3340 2024-09-21T06:02:25.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.564+0000 7f92a4e00640 1 -- 172.21.15.50:0/3094312653 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a0074db0 con 0x7f92a00b3340 2024-09-21T06:02:25.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.568+0000 7f9296000640 1 -- 172.21.15.50:0/3094312653 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f928801a070 con 0x7f92a00b3340 2024-09-21T06:02:25.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.568+0000 7f9296000640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f927c073b00 0x7f927c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:25.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.568+0000 7f9296000640 1 -- 172.21.15.50:0/3094312653 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f9288091f20 con 0x7f92a00b3340 2024-09-21T06:02:25.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.568+0000 7f929de00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f927c073b00 0x7f927c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:25.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.568+0000 7f929de00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f927c073b00 0x7f927c076010 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f92900af000 tx=0x7f929001b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:25.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.572+0000 7f9296000640 1 -- 172.21.15.50:0/3094312653 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9288021080 con 0x7f92a00b3340 2024-09-21T06:02:25.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.776+0000 7f92a4e00640 1 -- 172.21.15.50:0/3094312653 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f92a006b540 con 0x7f927c073b00 2024-09-21T06:02:25.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f9296000640 1 -- 172.21.15.50:0/3094312653 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f92900b3040 con 0x7f927c073b00 2024-09-21T06:02:25.785 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03 2024-09-21T06:02:25.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f92a4e00640 1 -- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f927c073b00 msgr2=0x7f927c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:25.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f92a4e00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f927c073b00 0x7f927c076010 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f92900af000 tx=0x7f929001b100 comp rx=0 tx=0).stop 2024-09-21T06:02:25.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f92a4e00640 1 -- 172.21.15.50:0/3094312653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a00b3340 msgr2=0x7f92a013dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:25.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f92a4e00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a00b3340 0x7f92a013dab0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f928800c3f0 tx=0x7f928800c8b0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f92a4e00640 1 -- 172.21.15.50:0/3094312653 shutdown_connections 2024-09-21T06:02:25.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f92a4e00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f92a00cefa0 0x7f92a013f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f92a4e00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f927c073b00 0x7f927c076010 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f92a4e00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a00b3340 0x7f92a013dab0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f92a4e00640 1 --2- 172.21.15.50:0/3094312653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92a00749a0 0x7f92a015fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:25.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f92a4e00640 1 -- 172.21.15.50:0/3094312653 >> 172.21.15.50:0/3094312653 conn(0x7f92a00b2040 msgr2=0x7f92a00cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:25.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f92a4e00640 1 -- 172.21.15.50:0/3094312653 shutdown_connections 2024-09-21T06:02:25.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:02:25.784+0000 7f92a4e00640 1 -- 172.21.15.50:0/3094312653 wait complete. 2024-09-21T06:02:25.808 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03 2024-09-21T06:02:25.808 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:02:25.808 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:02:25.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:02:25.819 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:02:35.516 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:02:35.516 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:02:35.516 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.63995 s, 111 MB/s 2024-09-21T06:02:35.517 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:02:35.518 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:02:35.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:02:35.527 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:02:44.970 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:02:44.970 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:02:44.970 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.39331 s, 114 MB/s 2024-09-21T06:02:44.972 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:02:44.972 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:02:44.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:02:44.981 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:02:47.436 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:02:47.439 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:02:54.908 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:02:54.909 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:02:54.909 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.87203 s, 109 MB/s 2024-09-21T06:02:54.910 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:02:54.910 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:02:54.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:02:54.918 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:03:04.544 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:03:04.544 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:03:04.544 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.5719 s, 112 MB/s 2024-09-21T06:03:04.545 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:03:04.545 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:03:04.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:03:04.554 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:03:14.065 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:03:14.065 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:03:14.065 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.46035 s, 113 MB/s 2024-09-21T06:03:14.066 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:03:14.066 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:03:14.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:03:14.075 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:03:17.498 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:03:17.502 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:03:23.625 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:03:23.625 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:03:23.625 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.4974 s, 113 MB/s 2024-09-21T06:03:23.626 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:03:23.626 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:03:23.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:03:23.635 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:03:33.608 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:03:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:03:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.92097 s, 108 MB/s 2024-09-21T06:03:33.610 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:03:33.610 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:03:33.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:03:33.619 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:03:43.490 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:03:43.491 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:03:43.491 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.81902 s, 109 MB/s 2024-09-21T06:03:43.492 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:03:43.492 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:03:43.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:03:43.501 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:03:47.594 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:03:47.597 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:03:53.754 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:03:53.754 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:03:53.754 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.1982 s, 105 MB/s 2024-09-21T06:03:53.755 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:03:53.756 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:03:53.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:03:53.765 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:04:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:04:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:04:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.73133 s, 110 MB/s 2024-09-21T06:04:03.552 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-21T06:04:03.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.715+0000 7f5b47600640 1 -- 172.21.15.50:0/308731830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b400b4de0 msgr2=0x7f5b400b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:03.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.715+0000 7f5b47600640 1 --2- 172.21.15.50:0/308731830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b400b4de0 0x7f5b400b51f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5b40071cb0 tx=0x7f5b3801ccc0 comp rx=0 tx=0).stop 2024-09-21T06:04:03.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.715+0000 7f5b47600640 1 -- 172.21.15.50:0/308731830 shutdown_connections 2024-09-21T06:04:03.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.715+0000 7f5b47600640 1 --2- 172.21.15.50:0/308731830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b400b4de0 0x7f5b400b51f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:03.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.715+0000 7f5b47600640 1 --2- 172.21.15.50:0/308731830 >> v2:172.21.15.88:3300/0 conn(0x7f5b400b5c70 0x7f5b4015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:03.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.715+0000 7f5b47600640 1 -- 172.21.15.50:0/308731830 >> 172.21.15.50:0/308731830 conn(0x7f5b400b2040 msgr2=0x7f5b400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:03.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.715+0000 7f5b47600640 1 -- 172.21.15.50:0/308731830 shutdown_connections 2024-09-21T06:04:03.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.715+0000 7f5b47600640 1 -- 172.21.15.50:0/308731830 wait complete. 2024-09-21T06:04:03.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.715+0000 7f5b47600640 1 Processor -- start 2024-09-21T06:04:03.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.715+0000 7f5b47600640 1 -- start start 2024-09-21T06:04:03.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b47600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b400b4de0 0x7f5b401f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:03.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b47600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b400b5c70 0x7f5b4014e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:03.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b47600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b401527b0 0x7f5b4015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b47600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40168920 con 0x7f5b400b4de0 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b47600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401687a0 con 0x7f5b400b5c70 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b47600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5b40168aa0 con 0x7f5b401527b0 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b44c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b400b4de0 0x7f5b401f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b44c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b400b4de0 0x7f5b401f6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47098/0 (socket says 172.21.15.50:47098) 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b44c00640 1 -- 172.21.15.50:0/1369098644 learned_addr learned my addr 172.21.15.50:0/1369098644 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b37e00640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b400b5c70 0x7f5b4014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b45600640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b401527b0 0x7f5b4015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b44c00640 1 -- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b401527b0 msgr2=0x7f5b4015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b44c00640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b401527b0 0x7f5b4015a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b44c00640 1 -- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b400b5c70 msgr2=0x7f5b4014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b44c00640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b400b5c70 0x7f5b4014e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:03.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b44c00640 1 -- 172.21.15.50:0/1369098644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b400cdf00 con 0x7f5b400b4de0 2024-09-21T06:04:03.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b45600640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b401527b0 0x7f5b4015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:03.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b44c00640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b400b4de0 0x7f5b401f6760 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000e7c0 tx=0x7f5b3000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:03.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b3d600640 1 -- 172.21.15.50:0/1369098644 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3000ccc0 con 0x7f5b400b4de0 2024-09-21T06:04:03.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b3d600640 1 -- 172.21.15.50:0/1369098644 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b3000ce80 con 0x7f5b400b4de0 2024-09-21T06:04:03.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b3d600640 1 -- 172.21.15.50:0/1369098644 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30005280 con 0x7f5b400b4de0 2024-09-21T06:04:03.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b47600640 1 -- 172.21.15.50:0/1369098644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b400ce130 con 0x7f5b400b4de0 2024-09-21T06:04:03.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b47600640 1 -- 172.21.15.50:0/1369098644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b400cd2c0 con 0x7f5b400b4de0 2024-09-21T06:04:03.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.719+0000 7f5b47600640 1 -- 172.21.15.50:0/1369098644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b400b51f0 con 0x7f5b400b4de0 2024-09-21T06:04:03.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.723+0000 7f5b3d600640 1 -- 172.21.15.50:0/1369098644 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 24) ==== 98050+0+0 (secure 0 0 0) 0x7f5b30005420 con 0x7f5b400b4de0 2024-09-21T06:04:03.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.723+0000 7f5b3d600640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b1c073b00 0x7f5b1c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:03.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.723+0000 7f5b3d600640 1 -- 172.21.15.50:0/1369098644 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f5b300934f0 con 0x7f5b400b4de0 2024-09-21T06:04:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.723+0000 7f5b37e00640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b1c073b00 0x7f5b1c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.723+0000 7f5b37e00640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b1c073b00 0x7f5b1c076010 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5b400afa20 tx=0x7f5b28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:03.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.727+0000 7f5b3d600640 1 -- 172.21.15.50:0/1369098644 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5b3005fff0 con 0x7f5b400b4de0 2024-09-21T06:04:03.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.935+0000 7f5b47600640 1 -- 172.21.15.50:0/1369098644 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f5b401516e0 con 0x7f5b1c073b00 2024-09-21T06:04:03.942 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:03.938+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:03.942 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:03.938+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:03.942 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:03.938+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:03.942 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:03.938+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:03.942 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:03.938+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.963+0000 7f5b3d600640 1 -- 172.21.15.50:0/1369098644 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5b401516e0 con 0x7f5b1c073b00 2024-09-21T06:04:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.967+0000 7f5b47600640 1 -- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b1c073b00 msgr2=0x7f5b1c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.967+0000 7f5b47600640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b1c073b00 0x7f5b1c076010 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5b400afa20 tx=0x7f5b28009040 comp rx=0 tx=0).stop 2024-09-21T06:04:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.967+0000 7f5b47600640 1 -- 172.21.15.50:0/1369098644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b400b4de0 msgr2=0x7f5b401f6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.967+0000 7f5b47600640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b400b4de0 0x7f5b401f6760 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000e7c0 tx=0x7f5b3000ec80 comp rx=0 tx=0).stop 2024-09-21T06:04:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.967+0000 7f5b47600640 1 -- 172.21.15.50:0/1369098644 shutdown_connections 2024-09-21T06:04:03.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.967+0000 7f5b47600640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b401527b0 0x7f5b4015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:03.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.967+0000 7f5b47600640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b400b5c70 0x7f5b4014e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:03.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.967+0000 7f5b47600640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b1c073b00 0x7f5b1c076010 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:03.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.967+0000 7f5b47600640 1 --2- 172.21.15.50:0/1369098644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b400b4de0 0x7f5b401f6760 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:03.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.967+0000 7f5b47600640 1 -- 172.21.15.50:0/1369098644 >> 172.21.15.50:0/1369098644 conn(0x7f5b400b2040 msgr2=0x7f5b40151c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:03.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.967+0000 7f5b47600640 1 -- 172.21.15.50:0/1369098644 shutdown_connections 2024-09-21T06:04:03.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:03.967+0000 7f5b47600640 1 -- 172.21.15.50:0/1369098644 wait complete. 2024-09-21T06:04:03.988 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:04:04.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.147+0000 7f4376c00640 1 -- 172.21.15.50:0/355471358 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43700b5a80 msgr2=0x7f4370074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.147+0000 7f4376c00640 1 --2- 172.21.15.50:0/355471358 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43700b5a80 0x7f4370074410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4358009870 tx=0x7f435801c860 comp rx=0 tx=0).stop 2024-09-21T06:04:04.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.147+0000 7f4376c00640 1 -- 172.21.15.50:0/355471358 shutdown_connections 2024-09-21T06:04:04.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.147+0000 7f4376c00640 1 --2- 172.21.15.50:0/355471358 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43700b5a80 0x7f4370074410 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.147+0000 7f4376c00640 1 -- 172.21.15.50:0/355471358 >> 172.21.15.50:0/355471358 conn(0x7f43700708c0 msgr2=0x7f4370070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:04.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.147+0000 7f4376c00640 1 -- 172.21.15.50:0/355471358 shutdown_connections 2024-09-21T06:04:04.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.147+0000 7f4376c00640 1 -- 172.21.15.50:0/355471358 wait complete. 2024-09-21T06:04:04.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f4376c00640 1 Processor -- start 2024-09-21T06:04:04.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f4376c00640 1 -- start start 2024-09-21T06:04:04.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f4376c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4370074950 0x7f4370139ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f4376c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43700b5a80 0x7f437015c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f4376c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4370157030 0x7f4370159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f4376c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f437016b6f0 con 0x7f4370157030 2024-09-21T06:04:04.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f4376c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f437016b570 con 0x7f4370074950 2024-09-21T06:04:04.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f4376c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f437016b870 con 0x7f43700b5a80 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f4374c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4370157030 0x7f4370159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f4374c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4370157030 0x7f4370159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47102/0 (socket says 172.21.15.50:47102) 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f4374c00640 1 -- 172.21.15.50:0/937683244 learned_addr learned my addr 172.21.15.50:0/937683244 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f4367e00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43700b5a80 0x7f437015c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.151+0000 7f436fe00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4370074950 0x7f4370139ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f4374c00640 1 -- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43700b5a80 msgr2=0x7f437015c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f4374c00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43700b5a80 0x7f437015c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f4374c00640 1 -- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4370074950 msgr2=0x7f4370139ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f4374c00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4370074950 0x7f4370139ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f4374c00640 1 -- 172.21.15.50:0/937683244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437013ca50 con 0x7f4370157030 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f4374c00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4370157030 0x7f4370159ee0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4368001f20 tx=0x7f4368002010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f436cc00640 1 -- 172.21.15.50:0/937683244 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368016020 con 0x7f4370157030 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f436cc00640 1 -- 172.21.15.50:0/937683244 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f436800ab60 con 0x7f4370157030 2024-09-21T06:04:04.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f436cc00640 1 -- 172.21.15.50:0/937683244 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368012420 con 0x7f4370157030 2024-09-21T06:04:04.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f4376c00640 1 -- 172.21.15.50:0/937683244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437013be40 con 0x7f4370157030 2024-09-21T06:04:04.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f4376c00640 1 -- 172.21.15.50:0/937683244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437013c400 con 0x7f4370157030 2024-09-21T06:04:04.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f4376c00640 1 -- 172.21.15.50:0/937683244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437013bb90 con 0x7f4370157030 2024-09-21T06:04:04.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.155+0000 7f436cc00640 1 -- 172.21.15.50:0/937683244 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 24) ==== 98050+0+0 (secure 0 0 0) 0x7f4368019020 con 0x7f4370157030 2024-09-21T06:04:04.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.159+0000 7f436cc00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f434c073b00 0x7f434c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.159+0000 7f436cc00640 1 -- 172.21.15.50:0/937683244 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f4368092e00 con 0x7f4370157030 2024-09-21T06:04:04.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.159+0000 7f436fe00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f434c073b00 0x7f434c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.159+0000 7f436fe00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f434c073b00 0x7f434c076010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4360000c10 tx=0x7f4360009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:04.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.163+0000 7f436cc00640 1 -- 172.21.15.50:0/937683244 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4368020020 con 0x7f4370157030 2024-09-21T06:04:04.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.363+0000 7f4376c00640 1 -- 172.21.15.50:0/937683244 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f437006b540 con 0x7f434c073b00 2024-09-21T06:04:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.371+0000 7f436cc00640 1 -- 172.21.15.50:0/937683244 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f4360001440 con 0x7f434c073b00 2024-09-21T06:04:04.372 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03 2024-09-21T06:04:04.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.375+0000 7f4376c00640 1 -- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f434c073b00 msgr2=0x7f434c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.375+0000 7f4376c00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f434c073b00 0x7f434c076010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4360000c10 tx=0x7f4360009040 comp rx=0 tx=0).stop 2024-09-21T06:04:04.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.375+0000 7f4376c00640 1 -- 172.21.15.50:0/937683244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4370157030 msgr2=0x7f4370159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.375+0000 7f4376c00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4370157030 0x7f4370159ee0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4368001f20 tx=0x7f4368002010 comp rx=0 tx=0).stop 2024-09-21T06:04:04.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.375+0000 7f4376c00640 1 -- 172.21.15.50:0/937683244 shutdown_connections 2024-09-21T06:04:04.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.375+0000 7f4376c00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f434c073b00 0x7f434c076010 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.375+0000 7f4376c00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4370157030 0x7f4370159ee0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.375+0000 7f4376c00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43700b5a80 0x7f437015c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.375+0000 7f4376c00640 1 --2- 172.21.15.50:0/937683244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4370074950 0x7f4370139ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.375+0000 7f4376c00640 1 -- 172.21.15.50:0/937683244 >> 172.21.15.50:0/937683244 conn(0x7f43700708c0 msgr2=0x7f437015f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:04.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.375+0000 7f4376c00640 1 -- 172.21.15.50:0/937683244 shutdown_connections 2024-09-21T06:04:04.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:04.375+0000 7f4376c00640 1 -- 172.21.15.50:0/937683244 wait complete. 2024-09-21T06:04:04.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:04.395 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03 2024-09-21T06:04:04.395 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:04:04.405 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03 2024-09-21T06:04:04.405 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="10488365056" 2024-09-21T06:04:04.405 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:04.406 INFO:tasks.cephfs.test_volumes:r_size = 10488365056 exp_size = 10737418240 2024-09-21T06:04:06.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:06.407 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03 2024-09-21T06:04:06.407 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:04:06.418 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/6823f102-ddc3-4da0-ac4a-bee6f9940f03 2024-09-21T06:04:06.418 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="10737418240" 2024-09-21T06:04:06.418 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:06.419 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-21T06:04:06.419 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-21T06:04:06.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.639+0000 7f0b74600640 1 -- 172.21.15.50:0/439407895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b6c0b5c70 msgr2=0x7f0b6c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.639+0000 7f0b74600640 1 --2- 172.21.15.50:0/439407895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b6c0b5c70 0x7f0b6c15bb60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c070710 tx=0x7f0b6001c740 comp rx=0 tx=0).stop 2024-09-21T06:04:06.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.639+0000 7f0b74600640 1 -- 172.21.15.50:0/439407895 shutdown_connections 2024-09-21T06:04:06.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.639+0000 7f0b74600640 1 --2- 172.21.15.50:0/439407895 >> v2:172.21.15.88:3300/0 conn(0x7f0b6c1527b0 0x7f0b6c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.639+0000 7f0b74600640 1 --2- 172.21.15.50:0/439407895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b6c0b5c70 0x7f0b6c15bb60 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.639+0000 7f0b74600640 1 -- 172.21.15.50:0/439407895 >> 172.21.15.50:0/439407895 conn(0x7f0b6c0b2040 msgr2=0x7f0b6c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:06.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.639+0000 7f0b74600640 1 -- 172.21.15.50:0/439407895 shutdown_connections 2024-09-21T06:04:06.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.639+0000 7f0b74600640 1 -- 172.21.15.50:0/439407895 wait complete. 2024-09-21T06:04:06.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.639+0000 7f0b74600640 1 Processor -- start 2024-09-21T06:04:06.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.639+0000 7f0b74600640 1 -- start start 2024-09-21T06:04:06.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b74600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b6c0b4de0 0x7f0b6c0cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b74600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b6c0b5c70 0x7f0b6c0cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b74600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b6c1527b0 0x7f0b6c1fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b74600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c168a80 con 0x7f0b6c0b4de0 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b74600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c168900 con 0x7f0b6c1527b0 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b74600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6c168c00 con 0x7f0b6c0b5c70 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b71c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b6c0b4de0 0x7f0b6c0cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b71c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b6c0b4de0 0x7f0b6c0cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47122/0 (socket says 172.21.15.50:47122) 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b71c00640 1 -- 172.21.15.50:0/1690668944 learned_addr learned my addr 172.21.15.50:0/1690668944 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b71c00640 1 -- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b6c0b5c70 msgr2=0x7f0b6c0cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b71200640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b6c0b5c70 0x7f0b6c0cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b72600640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b6c1527b0 0x7f0b6c1fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b71c00640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b6c0b5c70 0x7f0b6c0cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b71c00640 1 -- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b6c1527b0 msgr2=0x7f0b6c1fbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b71c00640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b6c1527b0 0x7f0b6c1fbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b71c00640 1 -- 172.21.15.50:0/1690668944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c157ff0 con 0x7f0b6c0b4de0 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b71200640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b6c0b5c70 0x7f0b6c0cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b72600640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b6c1527b0 0x7f0b6c1fbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b71c00640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b6c0b4de0 0x7f0b6c0cbd20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0b6001cc20 tx=0x7f0b60005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b69600640 1 -- 172.21.15.50:0/1690668944 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b600b2040 con 0x7f0b6c0b4de0 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b69600640 1 -- 172.21.15.50:0/1690668944 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b60004130 con 0x7f0b6c0b4de0 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b69600640 1 -- 172.21.15.50:0/1690668944 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60003600 con 0x7f0b6c0b4de0 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b74600640 1 -- 172.21.15.50:0/1690668944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c1582d0 con 0x7f0b6c0b4de0 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b74600640 1 -- 172.21.15.50:0/1690668944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c158890 con 0x7f0b6c0b4de0 2024-09-21T06:04:06.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.643+0000 7f0b74600640 1 -- 172.21.15.50:0/1690668944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6c15bb60 con 0x7f0b6c0b4de0 2024-09-21T06:04:06.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.647+0000 7f0b69600640 1 -- 172.21.15.50:0/1690668944 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 98101+0+0 (secure 0 0 0) 0x7f0b600bd030 con 0x7f0b6c0b4de0 2024-09-21T06:04:06.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.647+0000 7f0b69600640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b4c073bc0 0x7f0b4c0760d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.647+0000 7f0b69600640 1 -- 172.21.15.50:0/1690668944 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f0b6012c500 con 0x7f0b6c0b4de0 2024-09-21T06:04:06.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.647+0000 7f0b71200640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b4c073bc0 0x7f0b4c0760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.647+0000 7f0b71200640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b4c073bc0 0x7f0b4c0760d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c1458c0 tx=0x7f0b580091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:06.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.651+0000 7f0b69600640 1 -- 172.21.15.50:0/1690668944 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0b600c4060 con 0x7f0b6c0b4de0 2024-09-21T06:04:06.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.859+0000 7f0b74600640 1 -- 172.21.15.50:0/1690668944 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f0b6c159e10 con 0x7f0b4c073bc0 2024-09-21T06:04:06.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b69600640 1 -- 172.21.15.50:0/1690668944 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0b6c159e10 con 0x7f0b4c073bc0 2024-09-21T06:04:06.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b74600640 1 -- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b4c073bc0 msgr2=0x7f0b4c0760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b74600640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b4c073bc0 0x7f0b4c0760d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c1458c0 tx=0x7f0b580091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b74600640 1 -- 172.21.15.50:0/1690668944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b6c0b4de0 msgr2=0x7f0b6c0cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b74600640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b6c0b4de0 0x7f0b6c0cbd20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0b6001cc20 tx=0x7f0b60005db0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b74600640 1 -- 172.21.15.50:0/1690668944 shutdown_connections 2024-09-21T06:04:06.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b74600640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b6c1527b0 0x7f0b6c1fbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b74600640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b6c0b5c70 0x7f0b6c0cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b74600640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b4c073bc0 0x7f0b4c0760d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b74600640 1 --2- 172.21.15.50:0/1690668944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b6c0b4de0 0x7f0b6c0cbd20 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b74600640 1 -- 172.21.15.50:0/1690668944 >> 172.21.15.50:0/1690668944 conn(0x7f0b6c0b2040 msgr2=0x7f0b6c15a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:06.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b74600640 1 -- 172.21.15.50:0/1690668944 shutdown_connections 2024-09-21T06:04:06.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:06.935+0000 7f0b74600640 1 -- 172.21.15.50:0/1690668944 wait complete. 2024-09-21T06:04:06.939 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:06.934+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:06.939 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:06.934+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:06.939 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:06.934+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:06.939 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:06.934+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:06.939 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:06.934+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:06.958 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:06.954+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:06.958 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:06.954+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:06.958 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:06.954+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:06.958 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:06.954+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:06.958 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:06.954+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:06.959 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:04:07.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.115+0000 7f54b0200640 1 -- 172.21.15.50:0/1501899790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a80cefa0 msgr2=0x7f54a80cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.115+0000 7f54b0200640 1 --2- 172.21.15.50:0/1501899790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a80cefa0 0x7f54a80cf3b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f54a80b3040 tx=0x7f549401c800 comp rx=0 tx=0).stop 2024-09-21T06:04:07.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.115+0000 7f54b0200640 1 -- 172.21.15.50:0/1501899790 shutdown_connections 2024-09-21T06:04:07.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.115+0000 7f54b0200640 1 --2- 172.21.15.50:0/1501899790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a80cefa0 0x7f54a80cf3b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.115+0000 7f54b0200640 1 --2- 172.21.15.50:0/1501899790 >> v2:172.21.15.88:3300/0 conn(0x7f54a80749a0 0x7f54a8074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.115+0000 7f54b0200640 1 -- 172.21.15.50:0/1501899790 >> 172.21.15.50:0/1501899790 conn(0x7f54a80b2040 msgr2=0x7f54a80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:07.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.115+0000 7f54b0200640 1 -- 172.21.15.50:0/1501899790 shutdown_connections 2024-09-21T06:04:07.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.119+0000 7f54b0200640 1 -- 172.21.15.50:0/1501899790 wait complete. 2024-09-21T06:04:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.119+0000 7f54b0200640 1 Processor -- start 2024-09-21T06:04:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.119+0000 7f54b0200640 1 -- start start 2024-09-21T06:04:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.119+0000 7f54b0200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a80749a0 0x7f54a8140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.119+0000 7f54b0200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54a80cefa0 0x7f54a80af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.119+0000 7f54b0200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a8160330 0x7f54a8141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.119+0000 7f54b0200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f54a816a060 con 0x7f54a8160330 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.119+0000 7f54b0200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8169ee0 con 0x7f54a80749a0 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.119+0000 7f54b0200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f54a816a1e0 con 0x7f54a80cefa0 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ad800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a80749a0 0x7f54a8140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ace00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54a80cefa0 0x7f54a80af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ad800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a80749a0 0x7f54a8140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:34102/0 (socket says 172.21.15.50:34102) 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ad800640 1 -- 172.21.15.50:0/2564032795 learned_addr learned my addr 172.21.15.50:0/2564032795 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ae200640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a8160330 0x7f54a8141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ace00640 1 -- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a80749a0 msgr2=0x7f54a8140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ace00640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a80749a0 0x7f54a8140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ace00640 1 -- 172.21.15.50:0/2564032795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a8160330 msgr2=0x7f54a8141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ace00640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a8160330 0x7f54a8141900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ace00640 1 -- 172.21.15.50:0/2564032795 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a80cd490 con 0x7f54a80cefa0 2024-09-21T06:04:07.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ad800640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a80749a0 0x7f54a8140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:07.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ae200640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a8160330 0x7f54a8141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:07.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54ace00640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54a80cefa0 0x7f54a80af740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f54940ac000 tx=0x7f54940a56c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54a5600640 1 -- 172.21.15.50:0/2564032795 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54940b2040 con 0x7f54a80cefa0 2024-09-21T06:04:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54a5600640 1 -- 172.21.15.50:0/2564032795 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54940043b0 con 0x7f54a80cefa0 2024-09-21T06:04:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54a5600640 1 -- 172.21.15.50:0/2564032795 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494003860 con 0x7f54a80cefa0 2024-09-21T06:04:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54b0200640 1 -- 172.21.15.50:0/2564032795 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a80ae980 con 0x7f54a80cefa0 2024-09-21T06:04:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54b0200640 1 -- 172.21.15.50:0/2564032795 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a80aef40 con 0x7f54a80cefa0 2024-09-21T06:04:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54b0200640 1 -- 172.21.15.50:0/2564032795 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a8074db0 con 0x7f54a80cefa0 2024-09-21T06:04:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.123+0000 7f54a5600640 1 -- 172.21.15.50:0/2564032795 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 25) ==== 98101+0+0 (secure 0 0 0) 0x7f54940bd060 con 0x7f54a80cefa0 2024-09-21T06:04:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.127+0000 7f54a5600640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5488073b90 0x7f54880760a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.127+0000 7f54a5600640 1 -- 172.21.15.50:0/2564032795 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f549412bd10 con 0x7f54a80cefa0 2024-09-21T06:04:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.127+0000 7f54ad800640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5488073b90 0x7f54880760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.127+0000 7f54ad800640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5488073b90 0x7f54880760a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f549c011490 tx=0x7f549c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:07.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.131+0000 7f54a5600640 1 -- 172.21.15.50:0/2564032795 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f54940163f0 con 0x7f54a80cefa0 2024-09-21T06:04:07.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.367+0000 7f54b0200640 1 -- 172.21.15.50:0/2564032795 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f54a80cd220 con 0x7f54a80cefa0 2024-09-21T06:04:07.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54a5600640 1 -- 172.21.15.50:0/2564032795 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1883 (secure 0 0 0) 0x7f54940f8370 con 0x7f54a80cefa0 2024-09-21T06:04:07.371 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:07.371 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:04:07.371 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:04:07.371 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:04:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 483, 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 59, 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:04:07.373 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 2568, 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 10667251011, 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 21520924672, 2024-09-21T06:04:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 751573188608, 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 167038121, 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 0, 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 41 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 37, 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T06:02:23:310529+0000", 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4, 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 4, 2024-09-21T06:04:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "c", 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 5695 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:04:07.376 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:07.377 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:04:07.377 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:07.377 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:04:07.377 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 12, 2024-09-21T06:04:07.377 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:03:17.002246+0000", 2024-09-21T06:04:07.377 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:04:07.377 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:07.377 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": {} 2024-09-21T06:04:07.377 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:04:07.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54b0200640 1 -- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5488073b90 msgr2=0x7f54880760a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54b0200640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5488073b90 0x7f54880760a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f549c011490 tx=0x7f549c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54b0200640 1 -- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54a80cefa0 msgr2=0x7f54a80af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54b0200640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54a80cefa0 0x7f54a80af740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f54940ac000 tx=0x7f54940a56c0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54b0200640 1 -- 172.21.15.50:0/2564032795 shutdown_connections 2024-09-21T06:04:07.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54b0200640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5488073b90 0x7f54880760a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54b0200640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a8160330 0x7f54a8141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54b0200640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54a80cefa0 0x7f54a80af740 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54b0200640 1 --2- 172.21.15.50:0/2564032795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a80749a0 0x7f54a8140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54b0200640 1 -- 172.21.15.50:0/2564032795 >> 172.21.15.50:0/2564032795 conn(0x7f54a80b2040 msgr2=0x7f54a80b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:07.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54b0200640 1 -- 172.21.15.50:0/2564032795 shutdown_connections 2024-09-21T06:04:07.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:07.371+0000 7f54b0200640 1 -- 172.21.15.50:0/2564032795 wait complete. 2024-09-21T06:04:07.945 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:07.942+0000 7f6594e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:07.945 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:07.942+0000 7f6594e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:07.945 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:07.942+0000 7f6594e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:07.945 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:07.942+0000 7f6594e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:07.945 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:07.942+0000 7f6594e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:08.394 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:04:08.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.667+0000 7f6822800640 1 -- 172.21.15.50:0/3622923828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f681c160d50 msgr2=0x7f681c074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:08.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.667+0000 7f6822800640 1 --2- 172.21.15.50:0/3622923828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f681c160d50 0x7f681c074410 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f681c0b1740 tx=0x7f680801c770 comp rx=0 tx=0).stop 2024-09-21T06:04:08.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.667+0000 7f6822800640 1 -- 172.21.15.50:0/3622923828 shutdown_connections 2024-09-21T06:04:08.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.667+0000 7f6822800640 1 --2- 172.21.15.50:0/3622923828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f681c160d50 0x7f681c074410 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.667+0000 7f6822800640 1 --2- 172.21.15.50:0/3622923828 >> v2:172.21.15.50:3300/0 conn(0x7f681c157030 0x7f681c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.667+0000 7f6822800640 1 -- 172.21.15.50:0/3622923828 >> 172.21.15.50:0/3622923828 conn(0x7f681c0708c0 msgr2=0x7f681c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:08.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.667+0000 7f6822800640 1 -- 172.21.15.50:0/3622923828 shutdown_connections 2024-09-21T06:04:08.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.667+0000 7f6822800640 1 -- 172.21.15.50:0/3622923828 wait complete. 2024-09-21T06:04:08.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.671+0000 7f6822800640 1 Processor -- start 2024-09-21T06:04:08.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.671+0000 7f6822800640 1 -- start start 2024-09-21T06:04:08.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.675+0000 7f6822800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f681c157030 0x7f681c154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.675+0000 7f6822800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f681c160d50 0x7f681c13bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.675+0000 7f6822800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f681c161160 0x7f681c0b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.675+0000 7f6822800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f681c06d660 con 0x7f681c157030 2024-09-21T06:04:08.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.675+0000 7f6822800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f681c06d4e0 con 0x7f681c160d50 2024-09-21T06:04:08.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.675+0000 7f6822800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f681c06d7e0 con 0x7f681c161160 2024-09-21T06:04:08.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f681b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f681c157030 0x7f681c154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f681b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f681c157030 0x7f681c154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39834/0 (socket says 172.21.15.50:39834) 2024-09-21T06:04:08.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f681b400640 1 -- 172.21.15.50:0/546143729 learned_addr learned my addr 172.21.15.50:0/546143729 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:08.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f681be00640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f681c161160 0x7f681c0b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f681b400640 1 -- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f681c161160 msgr2=0x7f681c0b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:08.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f681b400640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f681c161160 0x7f681c0b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f681b400640 1 -- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f681c160d50 msgr2=0x7f681c13bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:08.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f681b400640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f681c160d50 0x7f681c13bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f681b400640 1 -- 172.21.15.50:0/546143729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681c1f9480 con 0x7f681c157030 2024-09-21T06:04:08.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f681b400640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f681c157030 0x7f681c154000 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f680c00e7a0 tx=0x7f680c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:08.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f6813400640 1 -- 172.21.15.50:0/546143729 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c00ccc0 con 0x7f681c157030 2024-09-21T06:04:08.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f6813400640 1 -- 172.21.15.50:0/546143729 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f680c010430 con 0x7f681c157030 2024-09-21T06:04:08.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.679+0000 7f6813400640 1 -- 172.21.15.50:0/546143729 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c010880 con 0x7f681c157030 2024-09-21T06:04:08.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.683+0000 7f6822800640 1 -- 172.21.15.50:0/546143729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681c153520 con 0x7f681c157030 2024-09-21T06:04:08.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.683+0000 7f6822800640 1 -- 172.21.15.50:0/546143729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681c153ae0 con 0x7f681c157030 2024-09-21T06:04:08.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.683+0000 7f6822800640 1 -- 172.21.15.50:0/546143729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f681c0d2260 con 0x7f681c157030 2024-09-21T06:04:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.691+0000 7f6813400640 1 -- 172.21.15.50:0/546143729 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 98203+0+0 (secure 0 0 0) 0x7f680c010a20 con 0x7f681c157030 2024-09-21T06:04:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.691+0000 7f6813400640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67fc073ce0 0x7f67fc0761f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.691+0000 7f6813400640 1 -- 172.21.15.50:0/546143729 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f680c093a30 con 0x7f681c157030 2024-09-21T06:04:08.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.695+0000 7f6813400640 1 -- 172.21.15.50:0/546143729 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f680c026020 con 0x7f681c157030 2024-09-21T06:04:08.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.695+0000 7f681aa00640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67fc073ce0 0x7f67fc0761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:08.699+0000 7f681aa00640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67fc073ce0 0x7f67fc0761f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f68080af000 tx=0x7f6808019510 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.007+0000 7f6822800640 1 -- 172.21.15.50:0/546143729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f681c153000 con 0x7f681c157030 2024-09-21T06:04:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.007+0000 7f6813400640 1 -- 172.21.15.50:0/546143729 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1882 (secure 0 0 0) 0x7f680c060570 con 0x7f681c157030 2024-09-21T06:04:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:09.011 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:04:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:04:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:04:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:04:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:04:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:04:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:04:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:04:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:04:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:04:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:04:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:04:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:04:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:04:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 485, 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 59, 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:04:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 2568, 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 10667251011, 2024-09-21T06:04:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 21520924672, 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 751573188608, 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 92955279, 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 0, 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 23 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 37, 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T06:02:23:310529+0000", 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4, 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:04:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 4, 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "c", 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 5695 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:04:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:04:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:04:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:04:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:04:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 12, 2024-09-21T06:04:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:03:17.002246+0000", 2024-09-21T06:04:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:04:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": {} 2024-09-21T06:04:09.017 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:04:09.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.011+0000 7f6822800640 1 -- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67fc073ce0 msgr2=0x7f67fc0761f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:09.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.011+0000 7f6822800640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67fc073ce0 0x7f67fc0761f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f68080af000 tx=0x7f6808019510 comp rx=0 tx=0).stop 2024-09-21T06:04:09.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.011+0000 7f6822800640 1 -- 172.21.15.50:0/546143729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f681c157030 msgr2=0x7f681c154000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:09.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.011+0000 7f6822800640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f681c157030 0x7f681c154000 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f680c00e7a0 tx=0x7f680c00ec60 comp rx=0 tx=0).stop 2024-09-21T06:04:09.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.011+0000 7f6822800640 1 -- 172.21.15.50:0/546143729 shutdown_connections 2024-09-21T06:04:09.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.011+0000 7f6822800640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f681c161160 0x7f681c0b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:09.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.011+0000 7f6822800640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f681c160d50 0x7f681c13bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:09.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.011+0000 7f6822800640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f67fc073ce0 0x7f67fc0761f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:09.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.011+0000 7f6822800640 1 --2- 172.21.15.50:0/546143729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f681c157030 0x7f681c154000 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:09.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.011+0000 7f6822800640 1 -- 172.21.15.50:0/546143729 >> 172.21.15.50:0/546143729 conn(0x7f681c0708c0 msgr2=0x7f681c152230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:09.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.011+0000 7f6822800640 1 -- 172.21.15.50:0/546143729 shutdown_connections 2024-09-21T06:04:09.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:09.011+0000 7f6822800640 1 -- 172.21.15.50:0/546143729 wait complete. 2024-09-21T06:04:10.034 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:04:10.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.323+0000 7f5b88e00640 1 -- 172.21.15.50:0/1157772982 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b84072a50 msgr2=0x7f5b84072e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:10.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.323+0000 7f5b88e00640 1 --2- 172.21.15.50:0/1157772982 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b84072a50 0x7f5b84072e60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009c80 tx=0x7f5b7c01cc50 comp rx=0 tx=0).stop 2024-09-21T06:04:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.323+0000 7f5b88e00640 1 -- 172.21.15.50:0/1157772982 shutdown_connections 2024-09-21T06:04:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.323+0000 7f5b88e00640 1 --2- 172.21.15.50:0/1157772982 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b84072a50 0x7f5b84072e60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.323+0000 7f5b88e00640 1 --2- 172.21.15.50:0/1157772982 >> v2:172.21.15.88:3300/0 conn(0x7f5b840d1150 0x7f5b840cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.323+0000 7f5b88e00640 1 -- 172.21.15.50:0/1157772982 >> 172.21.15.50:0/1157772982 conn(0x7f5b8406d1f0 msgr2=0x7f5b8406d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.323+0000 7f5b88e00640 1 -- 172.21.15.50:0/1157772982 shutdown_connections 2024-09-21T06:04:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.323+0000 7f5b88e00640 1 -- 172.21.15.50:0/1157772982 wait complete. 2024-09-21T06:04:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.327+0000 7f5b88e00640 1 Processor -- start 2024-09-21T06:04:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.327+0000 7f5b88e00640 1 -- start start 2024-09-21T06:04:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b88e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b84072a50 0x7f5b84150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b88e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b840cf0d0 0x7f5b84141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b88e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b840b1fc0 0x7f5b84142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b88e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84169dc0 con 0x7f5b84072a50 2024-09-21T06:04:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b88e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84169c40 con 0x7f5b840cf0d0 2024-09-21T06:04:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b88e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5b84169f40 con 0x7f5b840b1fc0 2024-09-21T06:04:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b83400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b84072a50 0x7f5b84150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b83400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b84072a50 0x7f5b84150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39842/0 (socket says 172.21.15.50:39842) 2024-09-21T06:04:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b83400640 1 -- 172.21.15.50:0/1725668795 learned_addr learned my addr 172.21.15.50:0/1725668795 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b82a00640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b840cf0d0 0x7f5b84141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b83e00640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b840b1fc0 0x7f5b84142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b83400640 1 -- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b840b1fc0 msgr2=0x7f5b84142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b83400640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b840b1fc0 0x7f5b84142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b83400640 1 -- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b840cf0d0 msgr2=0x7f5b84141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b83400640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b840cf0d0 0x7f5b84141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b83400640 1 -- 172.21.15.50:0/1725668795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b840b2640 con 0x7f5b84072a50 2024-09-21T06:04:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b83400640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b84072a50 0x7f5b84150cb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c370 tx=0x7f5b7400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b7b400640 1 -- 172.21.15.50:0/1725668795 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74017070 con 0x7f5b84072a50 2024-09-21T06:04:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b7b400640 1 -- 172.21.15.50:0/1725668795 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b740053c0 con 0x7f5b84072a50 2024-09-21T06:04:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b7b400640 1 -- 172.21.15.50:0/1725668795 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74003890 con 0x7f5b84072a50 2024-09-21T06:04:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.331+0000 7f5b88e00640 1 -- 172.21.15.50:0/1725668795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b841511f0 con 0x7f5b84072a50 2024-09-21T06:04:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.335+0000 7f5b88e00640 1 -- 172.21.15.50:0/1725668795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b840afbc0 con 0x7f5b84072a50 2024-09-21T06:04:10.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.335+0000 7f5b88e00640 1 -- 172.21.15.50:0/1725668795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b840cebf0 con 0x7f5b84072a50 2024-09-21T06:04:10.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.339+0000 7f5b7b400640 1 -- 172.21.15.50:0/1725668795 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 27) ==== 98254+0+0 (secure 0 0 0) 0x7f5b7401a070 con 0x7f5b84072a50 2024-09-21T06:04:10.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.339+0000 7f5b7b400640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b64073d70 0x7f5b64076280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:10.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.339+0000 7f5b7b400640 1 -- 172.21.15.50:0/1725668795 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f5b74092090 con 0x7f5b84072a50 2024-09-21T06:04:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.347+0000 7f5b7b400640 1 -- 172.21.15.50:0/1725668795 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5b7409b080 con 0x7f5b84072a50 2024-09-21T06:04:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.347+0000 7f5b82a00640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b64073d70 0x7f5b64076280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.355+0000 7f5b82a00640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b64073d70 0x7f5b64076280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5b8406a750 tx=0x7f5b70006f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:10.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.691+0000 7f5b88e00640 1 -- 172.21.15.50:0/1725668795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5b840cdb50 con 0x7f5b84072a50 2024-09-21T06:04:10.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.691+0000 7f5b7b400640 1 -- 172.21.15.50:0/1725668795 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2145 (secure 0 0 0) 0x7f5b840b2640 con 0x7f5b84072a50 2024-09-21T06:04:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:10.694 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:04:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:04:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:04:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:04:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:04:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:04:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:04:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 487, 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:04:10.695 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 59, 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:04:10.696 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 2619, 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 10868689764, 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 21957369856, 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 751136743424, 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 14327616, 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 109741091, 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 3, 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 31 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:10.697 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 37, 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T06:02:23:310529+0000", 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4, 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 4, 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "c", 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 5695 2024-09-21T06:04:10.698 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:04:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 12, 2024-09-21T06:04:10.700 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:03:17.002246+0000", 2024-09-21T06:04:10.700 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:04:10.700 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:10.700 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": { 2024-09-21T06:04:10.700 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr-vol-ongoing-clones": { 2024-09-21T06:04:10.700 INFO:teuthology.orchestra.run.smithi050.stdout: "message": "1 ongoing clones - average progress is 0.0% (0s)\n [............................] ", 2024-09-21T06:04:10.700 INFO:teuthology.orchestra.run.smithi050.stdout: "progress": 0, 2024-09-21T06:04:10.700 INFO:teuthology.orchestra.run.smithi050.stdout: "add_to_ceph_s": true 2024-09-21T06:04:10.700 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:10.700 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:10.700 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:04:10.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.695+0000 7f5b88e00640 1 -- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b64073d70 msgr2=0x7f5b64076280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:10.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.695+0000 7f5b88e00640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b64073d70 0x7f5b64076280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5b8406a750 tx=0x7f5b70006f90 comp rx=0 tx=0).stop 2024-09-21T06:04:10.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.695+0000 7f5b88e00640 1 -- 172.21.15.50:0/1725668795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b84072a50 msgr2=0x7f5b84150cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:10.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.695+0000 7f5b88e00640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b84072a50 0x7f5b84150cb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c370 tx=0x7f5b7400c830 comp rx=0 tx=0).stop 2024-09-21T06:04:10.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.695+0000 7f5b88e00640 1 -- 172.21.15.50:0/1725668795 shutdown_connections 2024-09-21T06:04:10.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.695+0000 7f5b88e00640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5b840b1fc0 0x7f5b84142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:10.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.695+0000 7f5b88e00640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b840cf0d0 0x7f5b84141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:10.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.695+0000 7f5b88e00640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5b64073d70 0x7f5b64076280 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:10.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.695+0000 7f5b88e00640 1 --2- 172.21.15.50:0/1725668795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b84072a50 0x7f5b84150cb0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:10.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.695+0000 7f5b88e00640 1 -- 172.21.15.50:0/1725668795 >> 172.21.15.50:0/1725668795 conn(0x7f5b8406d1f0 msgr2=0x7f5b8406fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:10.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.695+0000 7f5b88e00640 1 -- 172.21.15.50:0/1725668795 shutdown_connections 2024-09-21T06:04:10.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:10.695+0000 7f5b88e00640 1 -- 172.21.15.50:0/1725668795 wait complete. 2024-09-21T06:04:10.728 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-21T06:04:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.003+0000 7fe6daa00640 1 -- 172.21.15.50:0/1759802297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6b80040d0 msgr2=0x7fe6b8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.003+0000 7fe6daa00640 1 --2- 172.21.15.50:0/1759802297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6b80040d0 0x7fe6b8004810 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000c610 tx=0x7fe6c001d0e0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.003+0000 7fe6d8a00640 1 -- 172.21.15.50:0/1759802297 reap_dead start 2024-09-21T06:04:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.003+0000 7fe6daa00640 1 -- 172.21.15.50:0/1759802297 shutdown_connections 2024-09-21T06:04:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.003+0000 7fe6daa00640 1 -- 172.21.15.50:0/1759802297 >> 172.21.15.50:0/1759802297 conn(0x7fe6d40b35f0 msgr2=0x7fe6d40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:11.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.003+0000 7fe6daa00640 1 -- 172.21.15.50:0/1759802297 shutdown_connections 2024-09-21T06:04:11.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.003+0000 7fe6daa00640 1 -- 172.21.15.50:0/1759802297 wait complete. 2024-09-21T06:04:11.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6daa00640 1 Processor -- start 2024-09-21T06:04:11.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6daa00640 1 -- start start 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6b80040d0 0x7fe6d4201110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6daa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6d41fd110 0x7fe6d4153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6daa00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe6d41f1080 0x7fe6d415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6daa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d40696d0 con 0x7fe6b80040d0 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6daa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d4069550 con 0x7fe6d41fd110 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6daa00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d4069850 con 0x7fe6d41f1080 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6d3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6b80040d0 0x7fe6d4201110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6d3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6b80040d0 0x7fe6d4201110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39868/0 (socket says 172.21.15.50:39868) 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6d3e00640 1 -- 172.21.15.50:0/2866906352 learned_addr learned my addr 172.21.15.50:0/2866906352 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6d3400640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6d41fd110 0x7fe6d4153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6d8a00640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe6d41f1080 0x7fe6d415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6d3e00640 1 -- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe6d41f1080 msgr2=0x7fe6d415dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6d3e00640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe6d41f1080 0x7fe6d415dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6d3e00640 1 -- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6d41fd110 msgr2=0x7fe6d4153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6d3e00640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6d41fd110 0x7fe6d4153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6d3e00640 1 -- 172.21.15.50:0/2866906352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d4201650 con 0x7fe6b80040d0 2024-09-21T06:04:11.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6d3e00640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6b80040d0 0x7fe6d4201110 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400b520 tx=0x7fe6c400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:11.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6cbe00640 1 -- 172.21.15.50:0/2866906352 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4014070 con 0x7fe6b80040d0 2024-09-21T06:04:11.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6cbe00640 1 -- 172.21.15.50:0/2866906352 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6c4010980 con 0x7fe6b80040d0 2024-09-21T06:04:11.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6daa00640 1 -- 172.21.15.50:0/2866906352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d4154ac0 con 0x7fe6b80040d0 2024-09-21T06:04:11.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6cbe00640 1 -- 172.21.15.50:0/2866906352 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4010dd0 con 0x7fe6b80040d0 2024-09-21T06:04:11.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.007+0000 7fe6daa00640 1 -- 172.21.15.50:0/2866906352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d41f0880 con 0x7fe6b80040d0 2024-09-21T06:04:11.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.011+0000 7fe6daa00640 1 -- 172.21.15.50:0/2866906352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d415bb90 con 0x7fe6b80040d0 2024-09-21T06:04:11.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.011+0000 7fe6cbe00640 1 -- 172.21.15.50:0/2866906352 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 27) ==== 98254+0+0 (secure 0 0 0) 0x7fe6c4017070 con 0x7fe6b80040d0 2024-09-21T06:04:11.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.015+0000 7fe6cbe00640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe6b4073d40 0x7fe6b4076250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.015+0000 7fe6cbe00640 1 -- 172.21.15.50:0/2866906352 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fe6c4092db0 con 0x7fe6b80040d0 2024-09-21T06:04:11.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.019+0000 7fe6cbe00640 1 -- 172.21.15.50:0/2866906352 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe6d4201650 con 0x7fe6b80040d0 2024-09-21T06:04:11.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.019+0000 7fe6d3400640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe6b4073d40 0x7fe6b4076250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.023+0000 7fe6d3400640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe6b4073d40 0x7fe6b4076250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00ac000 tx=0x7fe6c0019e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.363+0000 7fe6daa00640 1 -- 172.21.15.50:0/2866906352 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fe6d4154600 con 0x7fe6b4073d40 2024-09-21T06:04:11.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.515+0000 7fe6cbe00640 1 -- 172.21.15.50:0/2866906352 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe6c00016d0 con 0x7fe6b4073d40 2024-09-21T06:04:11.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.519+0000 7fe6daa00640 1 -- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe6b4073d40 msgr2=0x7fe6b4076250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.519+0000 7fe6daa00640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe6b4073d40 0x7fe6b4076250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00ac000 tx=0x7fe6c0019e80 comp rx=0 tx=0).stop 2024-09-21T06:04:11.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.519+0000 7fe6daa00640 1 -- 172.21.15.50:0/2866906352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6b80040d0 msgr2=0x7fe6d4201110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.519+0000 7fe6daa00640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6b80040d0 0x7fe6d4201110 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400b520 tx=0x7fe6c400b9e0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.519+0000 7fe6daa00640 1 -- 172.21.15.50:0/2866906352 shutdown_connections 2024-09-21T06:04:11.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.519+0000 7fe6daa00640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe6d41f1080 0x7fe6d415dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.519+0000 7fe6daa00640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6d41fd110 0x7fe6d4153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.519+0000 7fe6daa00640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe6b4073d40 0x7fe6b4076250 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.519+0000 7fe6daa00640 1 --2- 172.21.15.50:0/2866906352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6b80040d0 0x7fe6d4201110 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.519+0000 7fe6daa00640 1 -- 172.21.15.50:0/2866906352 >> 172.21.15.50:0/2866906352 conn(0x7fe6d40b35f0 msgr2=0x7fe6d4075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:11.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.519+0000 7fe6daa00640 1 -- 172.21.15.50:0/2866906352 shutdown_connections 2024-09-21T06:04:11.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.519+0000 7fe6daa00640 1 -- 172.21.15.50:0/2866906352 wait complete. 2024-09-21T06:04:11.541 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:04:11.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.699+0000 7f6da1200640 1 -- 172.21.15.50:0/1768832778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d9c0d07a0 msgr2=0x7f6d9c15cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.699+0000 7f6da1200640 1 --2- 172.21.15.50:0/1768832778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d9c0d07a0 0x7f6d9c15cd30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009bc0 tx=0x7f6d9401cae0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.699+0000 7f6da1200640 1 -- 172.21.15.50:0/1768832778 shutdown_connections 2024-09-21T06:04:11.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.699+0000 7f6da1200640 1 --2- 172.21.15.50:0/1768832778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d9c0d07a0 0x7f6d9c15cd30 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.699+0000 7f6da1200640 1 -- 172.21.15.50:0/1768832778 >> 172.21.15.50:0/1768832778 conn(0x7f6d9c0b35f0 msgr2=0x7f6d9c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:11.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.699+0000 7f6da1200640 1 -- 172.21.15.50:0/1768832778 shutdown_connections 2024-09-21T06:04:11.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.699+0000 7f6da1200640 1 -- 172.21.15.50:0/1768832778 wait complete. 2024-09-21T06:04:11.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6da1200640 1 Processor -- start 2024-09-21T06:04:11.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6da1200640 1 -- start start 2024-09-21T06:04:11.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6da1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d9c071940 0x7f6d9c1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6da1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d9c0d07a0 0x7f6d9c14d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6da1200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d9c150420 0x7f6d9c0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6da1200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c16aee0 con 0x7f6d9c0d07a0 2024-09-21T06:04:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6da1200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c16ad60 con 0x7f6d9c071940 2024-09-21T06:04:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6da1200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6d9c16b060 con 0x7f6d9c150420 2024-09-21T06:04:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d93e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d9c0d07a0 0x7f6d9c14d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d93e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d9c0d07a0 0x7f6d9c14d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39874/0 (socket says 172.21.15.50:39874) 2024-09-21T06:04:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d93e00640 1 -- 172.21.15.50:0/2806667706 learned_addr learned my addr 172.21.15.50:0/2806667706 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d9a800640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d9c150420 0x7f6d9c0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d99e00640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d9c071940 0x7f6d9c1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d93e00640 1 -- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d9c150420 msgr2=0x7f6d9c0b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d93e00640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d9c150420 0x7f6d9c0b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d93e00640 1 -- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d9c071940 msgr2=0x7f6d9c1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d93e00640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d9c071940 0x7f6d9c1566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d93e00640 1 -- 172.21.15.50:0/2806667706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9c14dc00 con 0x7f6d9c0d07a0 2024-09-21T06:04:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d93e00640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d9c0d07a0 0x7f6d9c14d6c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400c700 tx=0x7f6d8400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d92000640 1 -- 172.21.15.50:0/2806667706 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84016020 con 0x7f6d9c0d07a0 2024-09-21T06:04:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d92000640 1 -- 172.21.15.50:0/2806667706 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d8400a8d0 con 0x7f6d9c0d07a0 2024-09-21T06:04:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6d92000640 1 -- 172.21.15.50:0/2806667706 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400ada0 con 0x7f6d9c0d07a0 2024-09-21T06:04:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6da1200640 1 -- 172.21.15.50:0/2806667706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9c1f9bc0 con 0x7f6d9c0d07a0 2024-09-21T06:04:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.703+0000 7f6da1200640 1 -- 172.21.15.50:0/2806667706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d9c1fa180 con 0x7f6d9c0d07a0 2024-09-21T06:04:11.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.707+0000 7f6da1200640 1 -- 172.21.15.50:0/2806667706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d9c068df0 con 0x7f6d9c0d07a0 2024-09-21T06:04:11.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.707+0000 7f6d92000640 1 -- 172.21.15.50:0/2806667706 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 27) ==== 98254+0+0 (secure 0 0 0) 0x7f6d84019020 con 0x7f6d9c0d07a0 2024-09-21T06:04:11.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.707+0000 7f6d92000640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d78073d70 0x7f6d78076280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.707+0000 7f6d92000640 1 -- 172.21.15.50:0/2806667706 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f6d84092530 con 0x7f6d9c0d07a0 2024-09-21T06:04:11.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.707+0000 7f6d99e00640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d78073d70 0x7f6d78076280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.711+0000 7f6d99e00640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d78073d70 0x7f6d78076280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c000c10 tx=0x7f6d8c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:11.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.711+0000 7f6d92000640 1 -- 172.21.15.50:0/2806667706 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6d84020070 con 0x7f6d9c0d07a0 2024-09-21T06:04:11.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.911+0000 7f6da1200640 1 -- 172.21.15.50:0/2806667706 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f6d9c06b540 con 0x7f6d78073d70 2024-09-21T06:04:11.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.911+0000 7f6d92000640 1 -- 172.21.15.50:0/2806667706 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6d8c001440 con 0x7f6d78073d70 2024-09-21T06:04:11.913 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:04:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:04:11.913 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:11.913 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:04:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.911+0000 7f6da1200640 1 -- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d78073d70 msgr2=0x7f6d78076280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.911+0000 7f6da1200640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d78073d70 0x7f6d78076280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c000c10 tx=0x7f6d8c009040 comp rx=0 tx=0).stop 2024-09-21T06:04:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.911+0000 7f6da1200640 1 -- 172.21.15.50:0/2806667706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d9c0d07a0 msgr2=0x7f6d9c14d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.911+0000 7f6da1200640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d9c0d07a0 0x7f6d9c14d6c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400c700 tx=0x7f6d8400cbc0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.915+0000 7f6da1200640 1 -- 172.21.15.50:0/2806667706 shutdown_connections 2024-09-21T06:04:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.915+0000 7f6da1200640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d9c150420 0x7f6d9c0b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.915+0000 7f6da1200640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d78073d70 0x7f6d78076280 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.915+0000 7f6da1200640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d9c0d07a0 0x7f6d9c14d6c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.915+0000 7f6da1200640 1 --2- 172.21.15.50:0/2806667706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d9c071940 0x7f6d9c1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.915+0000 7f6da1200640 1 -- 172.21.15.50:0/2806667706 >> 172.21.15.50:0/2806667706 conn(0x7f6d9c0b35f0 msgr2=0x7f6d9c15ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.915+0000 7f6da1200640 1 -- 172.21.15.50:0/2806667706 shutdown_connections 2024-09-21T06:04:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:11.915+0000 7f6da1200640 1 -- 172.21.15.50:0/2806667706 wait complete. 2024-09-21T06:04:11.935 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-21T06:04:12.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.095+0000 7f2745a00640 1 -- 172.21.15.50:0/1628931892 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2740155a00 msgr2=0x7f274015f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.095+0000 7f2745a00640 1 --2- 172.21.15.50:0/1628931892 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2740155a00 0x7f274015f070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2730009900 tx=0x7f273001c8b0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.095+0000 7f2745a00640 1 -- 172.21.15.50:0/1628931892 shutdown_connections 2024-09-21T06:04:12.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.095+0000 7f2745a00640 1 --2- 172.21.15.50:0/1628931892 >> v2:172.21.15.88:3300/0 conn(0x7f274015f5b0 0x7f27400b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.095+0000 7f2745a00640 1 --2- 172.21.15.50:0/1628931892 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2740155a00 0x7f274015f070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.095+0000 7f2745a00640 1 -- 172.21.15.50:0/1628931892 >> 172.21.15.50:0/1628931892 conn(0x7f274006cdf0 msgr2=0x7f274006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:12.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.095+0000 7f2745a00640 1 -- 172.21.15.50:0/1628931892 shutdown_connections 2024-09-21T06:04:12.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.095+0000 7f2745a00640 1 -- 172.21.15.50:0/1628931892 wait complete. 2024-09-21T06:04:12.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.095+0000 7f2745a00640 1 Processor -- start 2024-09-21T06:04:12.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.095+0000 7f2745a00640 1 -- start start 2024-09-21T06:04:12.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f2745a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27400b2300 0x7f274014c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f2745a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2740155a00 0x7f27400aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f2745a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f274015f5b0 0x7f27400af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f2745a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f27401687b0 con 0x7f2740155a00 2024-09-21T06:04:12.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f2745a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2740168630 con 0x7f27400b2300 2024-09-21T06:04:12.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f2745a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2740168930 con 0x7f274015f5b0 2024-09-21T06:04:12.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f2737400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2740155a00 0x7f27400aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f273fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27400b2300 0x7f274014c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f273fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27400b2300 0x7f274014c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:34220/0 (socket says 172.21.15.50:34220) 2024-09-21T06:04:12.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f273fe00640 1 -- 172.21.15.50:0/1668093410 learned_addr learned my addr 172.21.15.50:0/1668093410 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:12.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f2745000640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f274015f5b0 0x7f27400af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f273fe00640 1 -- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f274015f5b0 msgr2=0x7f27400af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f273fe00640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f274015f5b0 0x7f27400af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f273fe00640 1 -- 172.21.15.50:0/1668093410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2740155a00 msgr2=0x7f27400aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f273fe00640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2740155a00 0x7f27400aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f273fe00640 1 -- 172.21.15.50:0/1668093410 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27401f85a0 con 0x7f27400b2300 2024-09-21T06:04:12.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f273fe00640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27400b2300 0x7f274014c6e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2730009850 tx=0x7f2730005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:12.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f273cc00640 1 -- 172.21.15.50:0/1668093410 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27300b2040 con 0x7f27400b2300 2024-09-21T06:04:12.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f273cc00640 1 -- 172.21.15.50:0/1668093410 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2730005020 con 0x7f27400b2300 2024-09-21T06:04:12.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f2745a00640 1 -- 172.21.15.50:0/1668093410 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27401f8810 con 0x7f27400b2300 2024-09-21T06:04:12.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f273cc00640 1 -- 172.21.15.50:0/1668093410 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730005430 con 0x7f27400b2300 2024-09-21T06:04:12.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f2745a00640 1 -- 172.21.15.50:0/1668093410 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27401f8dd0 con 0x7f27400b2300 2024-09-21T06:04:12.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.099+0000 7f2745a00640 1 -- 172.21.15.50:0/1668093410 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274015f070 con 0x7f27400b2300 2024-09-21T06:04:12.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.103+0000 7f273cc00640 1 -- 172.21.15.50:0/1668093410 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 27) ==== 98254+0+0 (secure 0 0 0) 0x7f27300bf030 con 0x7f27400b2300 2024-09-21T06:04:12.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.103+0000 7f273cc00640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f271c073d40 0x7f271c076250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.103+0000 7f273cc00640 1 -- 172.21.15.50:0/1668093410 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f273012d650 con 0x7f27400b2300 2024-09-21T06:04:12.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.103+0000 7f2737400640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f271c073d40 0x7f271c076250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.103+0000 7f2737400640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f271c073d40 0x7f271c076250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f27400afe70 tx=0x7f27280091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:12.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.107+0000 7f273cc00640 1 -- 172.21.15.50:0/1668093410 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f27300164d0 con 0x7f27400b2300 2024-09-21T06:04:12.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.311+0000 7f2745a00640 1 -- 172.21.15.50:0/1668093410 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f274006b540 con 0x7f271c073d40 2024-09-21T06:04:12.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.315+0000 7f273cc00640 1 -- 172.21.15.50:0/1668093410 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f2728011bb0 con 0x7f271c073d40 2024-09-21T06:04:12.317 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:12.317 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:04:12.317 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:12.317 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sv1" 2024-09-21T06:04:12.317 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:12.317 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:12.317 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ss1clone1" 2024-09-21T06:04:12.317 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:12.318 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:04:12.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.319+0000 7f2745a00640 1 -- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f271c073d40 msgr2=0x7f271c076250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.319+0000 7f2745a00640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f271c073d40 0x7f271c076250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f27400afe70 tx=0x7f27280091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.319+0000 7f2745a00640 1 -- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27400b2300 msgr2=0x7f274014c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.319+0000 7f2745a00640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27400b2300 0x7f274014c6e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2730009850 tx=0x7f2730005de0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.319+0000 7f2745a00640 1 -- 172.21.15.50:0/1668093410 shutdown_connections 2024-09-21T06:04:12.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.319+0000 7f2745a00640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f274015f5b0 0x7f27400af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.319+0000 7f2745a00640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f271c073d40 0x7f271c076250 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.319+0000 7f2745a00640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2740155a00 0x7f27400aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.319+0000 7f2745a00640 1 --2- 172.21.15.50:0/1668093410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27400b2300 0x7f274014c6e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.319+0000 7f2745a00640 1 -- 172.21.15.50:0/1668093410 >> 172.21.15.50:0/1668093410 conn(0x7f274006cdf0 msgr2=0x7f2740151bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:12.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.319+0000 7f2745a00640 1 -- 172.21.15.50:0/1668093410 shutdown_connections 2024-09-21T06:04:12.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.319+0000 7f2745a00640 1 -- 172.21.15.50:0/1668093410 wait complete. 2024-09-21T06:04:12.340 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-21T06:04:12.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.503+0000 7fe7d5600640 1 -- 172.21.15.50:0/4067268006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7b4004da0 msgr2=0x7fe7b4005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.503+0000 7fe7d5600640 1 --2- 172.21.15.50:0/4067268006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7b4004da0 0x7fe7b4005250 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800c560 tx=0x7fe7c801d260 comp rx=0 tx=0).stop 2024-09-21T06:04:12.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.503+0000 7fe7cf400640 1 -- 172.21.15.50:0/4067268006 reap_dead start 2024-09-21T06:04:12.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.503+0000 7fe7d5600640 1 -- 172.21.15.50:0/4067268006 shutdown_connections 2024-09-21T06:04:12.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.503+0000 7fe7d5600640 1 -- 172.21.15.50:0/4067268006 >> 172.21.15.50:0/4067268006 conn(0x7fe7d00b2040 msgr2=0x7fe7d00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:12.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.503+0000 7fe7d5600640 1 -- 172.21.15.50:0/4067268006 shutdown_connections 2024-09-21T06:04:12.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.503+0000 7fe7d5600640 1 -- 172.21.15.50:0/4067268006 wait complete. 2024-09-21T06:04:12.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7d5600640 1 Processor -- start 2024-09-21T06:04:12.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7d5600640 1 -- start start 2024-09-21T06:04:12.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7d5600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7b4004da0 0x7fe7d00ac980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7d5600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe7d0151d40 0x7fe7d00c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7d5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7d01f8b80 0x7fe7d01f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7d5600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d015d0d0 con 0x7fe7d01f8b80 2024-09-21T06:04:12.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7d5600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d015cf50 con 0x7fe7b4004da0 2024-09-21T06:04:12.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7d5600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d015d250 con 0x7fe7d0151d40 2024-09-21T06:04:12.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7cf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7d01f8b80 0x7fe7d01f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7cf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7d01f8b80 0x7fe7d01f8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39906/0 (socket says 172.21.15.50:39906) 2024-09-21T06:04:12.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7cf400640 1 -- 172.21.15.50:0/224968348 learned_addr learned my addr 172.21.15.50:0/224968348 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:12.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7cea00640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7b4004da0 0x7fe7d00ac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7c7e00640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe7d0151d40 0x7fe7d00c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7cf400640 1 -- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe7d0151d40 msgr2=0x7fe7d00c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7cf400640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe7d0151d40 0x7fe7d00c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7cf400640 1 -- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7b4004da0 msgr2=0x7fe7d00ac980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7cf400640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7b4004da0 0x7fe7d00ac980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7cf400640 1 -- 172.21.15.50:0/224968348 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d01f94d0 con 0x7fe7d01f8b80 2024-09-21T06:04:12.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7cf400640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7d01f8b80 0x7fe7d01f8f90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000f90 tx=0x7fe7c8004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:12.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7c6a00640 1 -- 172.21.15.50:0/224968348 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80011d0 con 0x7fe7d01f8b80 2024-09-21T06:04:12.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7c6a00640 1 -- 172.21.15.50:0/224968348 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7c8003980 con 0x7fe7d01f8b80 2024-09-21T06:04:12.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7d5600640 1 -- 172.21.15.50:0/224968348 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d00c5500 con 0x7fe7d01f8b80 2024-09-21T06:04:12.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7c6a00640 1 -- 172.21.15.50:0/224968348 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8003d30 con 0x7fe7d01f8b80 2024-09-21T06:04:12.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.507+0000 7fe7d5600640 1 -- 172.21.15.50:0/224968348 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d00ac180 con 0x7fe7d01f8b80 2024-09-21T06:04:12.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.511+0000 7fe7d5600640 1 -- 172.21.15.50:0/224968348 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d0074e30 con 0x7fe7d01f8b80 2024-09-21T06:04:12.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.511+0000 7fe7c6a00640 1 -- 172.21.15.50:0/224968348 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 27) ==== 98254+0+0 (secure 0 0 0) 0x7fe7c80ad430 con 0x7fe7d01f8b80 2024-09-21T06:04:12.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.511+0000 7fe7c6a00640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe7ac073d70 0x7fe7ac076280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.511+0000 7fe7c6a00640 1 -- 172.21.15.50:0/224968348 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fe7c812d2c0 con 0x7fe7d01f8b80 2024-09-21T06:04:12.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.511+0000 7fe7cea00640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe7ac073d70 0x7fe7ac076280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.511+0000 7fe7cea00640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe7ac073d70 0x7fe7ac076280 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0000c10 tx=0x7fe7c0006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:12.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.515+0000 7fe7c6a00640 1 -- 172.21.15.50:0/224968348 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe7c80be060 con 0x7fe7d01f8b80 2024-09-21T06:04:12.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.727+0000 7fe7d5600640 1 -- 172.21.15.50:0/224968348 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7d00c52b0 con 0x7fe7ac073d70 2024-09-21T06:04:12.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7c6a00640 1 -- 172.21.15.50:0/224968348 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe7d00c52b0 con 0x7fe7ac073d70 2024-09-21T06:04:12.744 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:12.744 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:04:12.744 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:12.744 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ss1" 2024-09-21T06:04:12.744 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:12.744 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:04:12.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7d5600640 1 -- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe7ac073d70 msgr2=0x7fe7ac076280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7d5600640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe7ac073d70 0x7fe7ac076280 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0000c10 tx=0x7fe7c0006b20 comp rx=0 tx=0).stop 2024-09-21T06:04:12.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7d5600640 1 -- 172.21.15.50:0/224968348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7d01f8b80 msgr2=0x7fe7d01f8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7d5600640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7d01f8b80 0x7fe7d01f8f90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000f90 tx=0x7fe7c8004ef0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7d5600640 1 -- 172.21.15.50:0/224968348 shutdown_connections 2024-09-21T06:04:12.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7d5600640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe7ac073d70 0x7fe7ac076280 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7d5600640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7d01f8b80 0x7fe7d01f8f90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7d5600640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe7d0151d40 0x7fe7d00c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7d5600640 1 --2- 172.21.15.50:0/224968348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7b4004da0 0x7fe7d00ac980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7d5600640 1 -- 172.21.15.50:0/224968348 >> 172.21.15.50:0/224968348 conn(0x7fe7d00b2040 msgr2=0x7fe7d00cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:12.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7d5600640 1 -- 172.21.15.50:0/224968348 shutdown_connections 2024-09-21T06:04:12.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.743+0000 7fe7d5600640 1 -- 172.21.15.50:0/224968348 wait complete. 2024-09-21T06:04:12.767 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-21T06:04:12.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.931+0000 7fd934400640 1 -- 172.21.15.50:0/3323623700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd92c160d50 msgr2=0x7fd92c074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.931+0000 7fd934400640 1 --2- 172.21.15.50:0/3323623700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd92c160d50 0x7fd92c074410 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0713e0 tx=0x7fd91801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.931+0000 7fd934400640 1 -- 172.21.15.50:0/3323623700 shutdown_connections 2024-09-21T06:04:12.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.931+0000 7fd934400640 1 --2- 172.21.15.50:0/3323623700 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd92c161160 0x7fd92c074950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.931+0000 7fd934400640 1 --2- 172.21.15.50:0/3323623700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd92c160d50 0x7fd92c074410 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.931+0000 7fd934400640 1 --2- 172.21.15.50:0/3323623700 >> v2:172.21.15.88:3300/0 conn(0x7fd92c157030 0x7fd92c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.931+0000 7fd934400640 1 -- 172.21.15.50:0/3323623700 >> 172.21.15.50:0/3323623700 conn(0x7fd92c0708c0 msgr2=0x7fd92c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:12.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.931+0000 7fd934400640 1 -- 172.21.15.50:0/3323623700 shutdown_connections 2024-09-21T06:04:12.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.931+0000 7fd934400640 1 -- 172.21.15.50:0/3323623700 wait complete. 2024-09-21T06:04:12.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.931+0000 7fd934400640 1 Processor -- start 2024-09-21T06:04:12.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.931+0000 7fd934400640 1 -- start start 2024-09-21T06:04:12.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd934400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd92c157030 0x7fd92c14f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd934400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd92c160d50 0x7fd92c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd934400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd92c161160 0x7fd92c13ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd934400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c072180 con 0x7fd92c160d50 2024-09-21T06:04:12.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd934400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c072000 con 0x7fd92c161160 2024-09-21T06:04:12.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd934400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd92c072300 con 0x7fd92c157030 2024-09-21T06:04:12.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd931000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd92c160d50 0x7fd92c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd931000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd92c160d50 0x7fd92c06d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39922/0 (socket says 172.21.15.50:39922) 2024-09-21T06:04:12.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd931000640 1 -- 172.21.15.50:0/3540568920 learned_addr learned my addr 172.21.15.50:0/3540568920 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:12.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd931a00640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd92c157030 0x7fd92c14f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd931000640 1 -- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd92c157030 msgr2=0x7fd92c14f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd931000640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd92c157030 0x7fd92c14f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd931000640 1 -- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd92c161160 msgr2=0x7fd92c13ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd932400640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd92c161160 0x7fd92c13ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd931000640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd92c161160 0x7fd92c13ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd931000640 1 -- 172.21.15.50:0/3540568920 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92c06d6f0 con 0x7fd92c160d50 2024-09-21T06:04:12.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd931a00640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd92c157030 0x7fd92c14f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:12.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd932400640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd92c161160 0x7fd92c13ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:12.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd931000640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd92c160d50 0x7fd92c06d1b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd9180b0040 tx=0x7fd918005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:12.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd929600640 1 -- 172.21.15.50:0/3540568920 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9180b2040 con 0x7fd92c160d50 2024-09-21T06:04:12.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd929600640 1 -- 172.21.15.50:0/3540568920 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd918004330 con 0x7fd92c160d50 2024-09-21T06:04:12.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd929600640 1 -- 172.21.15.50:0/3540568920 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918005290 con 0x7fd92c160d50 2024-09-21T06:04:12.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd934400640 1 -- 172.21.15.50:0/3540568920 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92c1fcce0 con 0x7fd92c160d50 2024-09-21T06:04:12.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd934400640 1 -- 172.21.15.50:0/3540568920 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd92c1fd2a0 con 0x7fd92c160d50 2024-09-21T06:04:12.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd934400640 1 -- 172.21.15.50:0/3540568920 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd92c0727b0 con 0x7fd92c160d50 2024-09-21T06:04:12.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.935+0000 7fd929600640 1 -- 172.21.15.50:0/3540568920 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 27) ==== 98254+0+0 (secure 0 0 0) 0x7fd918005430 con 0x7fd92c160d50 2024-09-21T06:04:12.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.939+0000 7fd929600640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd90c073d40 0x7fd90c076250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.939+0000 7fd929600640 1 -- 172.21.15.50:0/3540568920 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fd91812cd90 con 0x7fd92c160d50 2024-09-21T06:04:12.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.939+0000 7fd931a00640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd90c073d40 0x7fd90c076250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.939+0000 7fd931a00640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd90c073d40 0x7fd90c076250 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd920009a40 tx=0x7fd9200091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:12.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:12.943+0000 7fd929600640 1 -- 172.21.15.50:0/3540568920 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd9180c0030 con 0x7fd92c160d50 2024-09-21T06:04:13.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.151+0000 7fd934400640 1 -- 172.21.15.50:0/3540568920 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd92c06b540 con 0x7fd90c073d40 2024-09-21T06:04:13.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.179+0000 7fd929600640 1 -- 172.21.15.50:0/3540568920 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd920010ab0 con 0x7fd90c073d40 2024-09-21T06:04:13.181 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:13.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.183+0000 7fd934400640 1 -- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd90c073d40 msgr2=0x7fd90c076250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.183+0000 7fd934400640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd90c073d40 0x7fd90c076250 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd920009a40 tx=0x7fd9200091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.183+0000 7fd934400640 1 -- 172.21.15.50:0/3540568920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd92c160d50 msgr2=0x7fd92c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.183+0000 7fd934400640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd92c160d50 0x7fd92c06d1b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd9180b0040 tx=0x7fd918005d60 comp rx=0 tx=0).stop 2024-09-21T06:04:13.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.183+0000 7fd934400640 1 -- 172.21.15.50:0/3540568920 shutdown_connections 2024-09-21T06:04:13.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.183+0000 7fd934400640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd92c161160 0x7fd92c13ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.183+0000 7fd934400640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd90c073d40 0x7fd90c076250 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.183+0000 7fd934400640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd92c160d50 0x7fd92c06d1b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.183+0000 7fd934400640 1 --2- 172.21.15.50:0/3540568920 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd92c157030 0x7fd92c14f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.183+0000 7fd934400640 1 -- 172.21.15.50:0/3540568920 >> 172.21.15.50:0/3540568920 conn(0x7fd92c0708c0 msgr2=0x7fd92c0b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:13.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.183+0000 7fd934400640 1 -- 172.21.15.50:0/3540568920 shutdown_connections 2024-09-21T06:04:13.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.183+0000 7fd934400640 1 -- 172.21.15.50:0/3540568920 wait complete. 2024-09-21T06:04:13.204 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-21T06:04:13.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.363+0000 7fe8d0200640 1 -- 172.21.15.50:0/531390960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c80749a0 msgr2=0x7fe8c8074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.363+0000 7fe8d0200640 1 --2- 172.21.15.50:0/531390960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c80749a0 0x7fe8c8074db0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009840 tx=0x7fe8bc01c820 comp rx=0 tx=0).stop 2024-09-21T06:04:13.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.363+0000 7fe8d0200640 1 -- 172.21.15.50:0/531390960 shutdown_connections 2024-09-21T06:04:13.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.363+0000 7fe8d0200640 1 --2- 172.21.15.50:0/531390960 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8c80cefa0 0x7fe8c80cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.363+0000 7fe8d0200640 1 --2- 172.21.15.50:0/531390960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c80749a0 0x7fe8c8074db0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.363+0000 7fe8d0200640 1 -- 172.21.15.50:0/531390960 >> 172.21.15.50:0/531390960 conn(0x7fe8c80b2040 msgr2=0x7fe8c80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:13.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.363+0000 7fe8d0200640 1 -- 172.21.15.50:0/531390960 shutdown_connections 2024-09-21T06:04:13.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.367+0000 7fe8d0200640 1 -- 172.21.15.50:0/531390960 wait complete. 2024-09-21T06:04:13.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.367+0000 7fe8d0200640 1 Processor -- start 2024-09-21T06:04:13.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.367+0000 7fe8d0200640 1 -- start start 2024-09-21T06:04:13.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.367+0000 7fe8d0200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c80749a0 0x7fe8c81441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.367+0000 7fe8d0200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8c80cefa0 0x7fe8c8144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.367+0000 7fe8d0200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8c8160330 0x7fe8c81414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.367+0000 7fe8d0200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c816a0b0 con 0x7fe8c80cefa0 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.367+0000 7fe8d0200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8169f30 con 0x7fe8c80749a0 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.367+0000 7fe8d0200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c816a230 con 0x7fe8c8160330 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8cce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8c80cefa0 0x7fe8c8144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8cd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c80749a0 0x7fe8c81441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8cd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c80749a0 0x7fe8c81441b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:34260/0 (socket says 172.21.15.50:34260) 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8cce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8c80cefa0 0x7fe8c8144a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39936/0 (socket says 172.21.15.50:39936) 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8cd800640 1 -- 172.21.15.50:0/3836957773 learned_addr learned my addr 172.21.15.50:0/3836957773 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8ce200640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8c8160330 0x7fe8c81414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8cce00640 1 -- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8c8160330 msgr2=0x7fe8c81414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8cce00640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8c8160330 0x7fe8c81414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8cce00640 1 -- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c80749a0 msgr2=0x7fe8c81441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8cce00640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c80749a0 0x7fe8c81441b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8cce00640 1 -- 172.21.15.50:0/3836957773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c81433f0 con 0x7fe8c80cefa0 2024-09-21T06:04:13.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8cd800640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c80749a0 0x7fe8c81441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:13.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8ce200640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8c8160330 0x7fe8c81414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:13.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8cce00640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8c80cefa0 0x7fe8c8144a80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400e6c0 tx=0x7fe8b400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:13.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8c5600640 1 -- 172.21.15.50:0/3836957773 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4004220 con 0x7fe8c80cefa0 2024-09-21T06:04:13.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8c5600640 1 -- 172.21.15.50:0/3836957773 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8b400c850 con 0x7fe8c80cefa0 2024-09-21T06:04:13.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8d0200640 1 -- 172.21.15.50:0/3836957773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c81436d0 con 0x7fe8c80cefa0 2024-09-21T06:04:13.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8c5600640 1 -- 172.21.15.50:0/3836957773 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b400cca0 con 0x7fe8c80cefa0 2024-09-21T06:04:13.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8d0200640 1 -- 172.21.15.50:0/3836957773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c8143c90 con 0x7fe8c80cefa0 2024-09-21T06:04:13.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8d0200640 1 -- 172.21.15.50:0/3836957773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c8074db0 con 0x7fe8c80cefa0 2024-09-21T06:04:13.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.371+0000 7fe8c5600640 1 -- 172.21.15.50:0/3836957773 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 27) ==== 98254+0+0 (secure 0 0 0) 0x7fe8b4003600 con 0x7fe8c80cefa0 2024-09-21T06:04:13.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.375+0000 7fe8c5600640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8a8073d70 0x7fe8a8076280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:13.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.375+0000 7fe8c5600640 1 -- 172.21.15.50:0/3836957773 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fe8b4093280 con 0x7fe8c80cefa0 2024-09-21T06:04:13.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.375+0000 7fe8cd800640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8a8073d70 0x7fe8a8076280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:13.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.375+0000 7fe8cd800640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8a8073d70 0x7fe8a8076280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc01a450 tx=0x7fe8bc01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:13.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.379+0000 7fe8c5600640 1 -- 172.21.15.50:0/3836957773 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe8b4099030 con 0x7fe8c80cefa0 2024-09-21T06:04:13.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.583+0000 7fe8d0200640 1 -- 172.21.15.50:0/3836957773 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fe8c80cdd60 con 0x7fe8a8073d70 2024-09-21T06:04:13.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.607+0000 7fe8c5600640 1 -- 172.21.15.50:0/3836957773 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe8c80cdd60 con 0x7fe8a8073d70 2024-09-21T06:04:13.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.611+0000 7fe8d0200640 1 -- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8a8073d70 msgr2=0x7fe8a8076280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.611+0000 7fe8d0200640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8a8073d70 0x7fe8a8076280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc01a450 tx=0x7fe8bc01b060 comp rx=0 tx=0).stop 2024-09-21T06:04:13.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.611+0000 7fe8d0200640 1 -- 172.21.15.50:0/3836957773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8c80cefa0 msgr2=0x7fe8c8144a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.611+0000 7fe8d0200640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8c80cefa0 0x7fe8c8144a80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400e6c0 tx=0x7fe8b400eb80 comp rx=0 tx=0).stop 2024-09-21T06:04:13.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.611+0000 7fe8d0200640 1 -- 172.21.15.50:0/3836957773 shutdown_connections 2024-09-21T06:04:13.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.611+0000 7fe8d0200640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8c8160330 0x7fe8c81414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.611+0000 7fe8d0200640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8a8073d70 0x7fe8a8076280 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.611+0000 7fe8d0200640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8c80cefa0 0x7fe8c8144a80 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.611+0000 7fe8d0200640 1 --2- 172.21.15.50:0/3836957773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c80749a0 0x7fe8c81441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.611+0000 7fe8d0200640 1 -- 172.21.15.50:0/3836957773 >> 172.21.15.50:0/3836957773 conn(0x7fe8c80b2040 msgr2=0x7fe8c80af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:13.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.611+0000 7fe8d0200640 1 -- 172.21.15.50:0/3836957773 shutdown_connections 2024-09-21T06:04:13.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.611+0000 7fe8d0200640 1 -- 172.21.15.50:0/3836957773 wait complete. 2024-09-21T06:04:13.619 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:13.614+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:13.619 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:13.614+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:13.619 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:13.614+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:13.619 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:13.614+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:13.619 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:13.614+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:13.633 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-21T06:04:13.640 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:13.638+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:13.640 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:13.638+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:13.640 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:13.638+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:13.640 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:13.638+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:13.640 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:13.638+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:13.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.799+0000 7ff296600640 1 -- 172.21.15.50:0/976888509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff290157030 msgr2=0x7ff2900b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.799+0000 7ff296600640 1 --2- 172.21.15.50:0/976888509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff290157030 0x7ff2900b5a80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff2900b1740 tx=0x7ff28001c770 comp rx=0 tx=0).stop 2024-09-21T06:04:13.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.799+0000 7ff296600640 1 -- 172.21.15.50:0/976888509 shutdown_connections 2024-09-21T06:04:13.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.799+0000 7ff296600640 1 --2- 172.21.15.50:0/976888509 >> v2:172.21.15.88:3300/0 conn(0x7ff290160d50 0x7ff290074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.799+0000 7ff296600640 1 --2- 172.21.15.50:0/976888509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff290157030 0x7ff2900b5a80 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.799+0000 7ff296600640 1 -- 172.21.15.50:0/976888509 >> 172.21.15.50:0/976888509 conn(0x7ff2900708c0 msgr2=0x7ff290070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:13.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.799+0000 7ff296600640 1 -- 172.21.15.50:0/976888509 shutdown_connections 2024-09-21T06:04:13.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.799+0000 7ff296600640 1 -- 172.21.15.50:0/976888509 wait complete. 2024-09-21T06:04:13.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.799+0000 7ff296600640 1 Processor -- start 2024-09-21T06:04:13.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.799+0000 7ff296600640 1 -- start start 2024-09-21T06:04:13.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff296600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff290157030 0x7ff2901f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:13.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff296600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff290160d50 0x7ff2901509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:13.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff296600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff290161160 0x7ff290159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:13.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff296600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff29016a6f0 con 0x7ff290161160 2024-09-21T06:04:13.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff296600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff29016a570 con 0x7ff290157030 2024-09-21T06:04:13.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff296600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff29016a870 con 0x7ff290160d50 2024-09-21T06:04:13.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff28f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff290157030 0x7ff2901f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:13.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff28fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff290161160 0x7ff290159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:13.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff28f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff290157030 0x7ff2901f7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:34264/0 (socket says 172.21.15.50:34264) 2024-09-21T06:04:13.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff28f400640 1 -- 172.21.15.50:0/2735140235 learned_addr learned my addr 172.21.15.50:0/2735140235 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:13.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff287e00640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff290160d50 0x7ff2901509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:13.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff28f400640 1 -- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff290160d50 msgr2=0x7ff2901509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff28f400640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff290160d50 0x7ff2901509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff28f400640 1 -- 172.21.15.50:0/2735140235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff290161160 msgr2=0x7ff290159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff28f400640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff290161160 0x7ff290159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff28f400640 1 -- 172.21.15.50:0/2735140235 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff290150f00 con 0x7ff290157030 2024-09-21T06:04:13.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff28f400640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff290157030 0x7ff2901f7d40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff28001a3a0 tx=0x7ff2800a6700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:13.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff287400640 1 -- 172.21.15.50:0/2735140235 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2800be070 con 0x7ff290157030 2024-09-21T06:04:13.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff287400640 1 -- 172.21.15.50:0/2735140235 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2800a6c40 con 0x7ff290157030 2024-09-21T06:04:13.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff287400640 1 -- 172.21.15.50:0/2735140235 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280005040 con 0x7ff290157030 2024-09-21T06:04:13.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff296600640 1 -- 172.21.15.50:0/2735140235 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29014e0d0 con 0x7ff290157030 2024-09-21T06:04:13.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff296600640 1 -- 172.21.15.50:0/2735140235 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29014e690 con 0x7ff290157030 2024-09-21T06:04:13.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.803+0000 7ff296600640 1 -- 172.21.15.50:0/2735140235 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2900d1ff0 con 0x7ff290157030 2024-09-21T06:04:13.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.807+0000 7ff287400640 1 -- 172.21.15.50:0/2735140235 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 27) ==== 98254+0+0 (secure 0 0 0) 0x7ff2800051e0 con 0x7ff290157030 2024-09-21T06:04:13.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.807+0000 7ff287400640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff26c073d70 0x7ff26c076280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:13.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.807+0000 7ff287400640 1 -- 172.21.15.50:0/2735140235 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7ff28012c820 con 0x7ff290157030 2024-09-21T06:04:13.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.807+0000 7ff287e00640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff26c073d70 0x7ff26c076280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:13.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.807+0000 7ff287e00640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff26c073d70 0x7ff26c076280 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff2901f8430 tx=0x7ff2780091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:13.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:13.811+0000 7ff287400640 1 -- 172.21.15.50:0/2735140235 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff2800ac030 con 0x7ff290157030 2024-09-21T06:04:14.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:14.027+0000 7ff296600640 1 -- 172.21.15.50:0/2735140235 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7ff29006b540 con 0x7ff26c073d70 2024-09-21T06:04:15.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.543+0000 7ff287400640 1 -- 172.21.15.50:0/2735140235 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff278011bb0 con 0x7ff26c073d70 2024-09-21T06:04:15.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.547+0000 7ff296600640 1 -- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff26c073d70 msgr2=0x7ff26c076280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:15.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.547+0000 7ff296600640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff26c073d70 0x7ff26c076280 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff2901f8430 tx=0x7ff2780091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.547+0000 7ff296600640 1 -- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff290157030 msgr2=0x7ff2901f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:15.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.547+0000 7ff296600640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff290157030 0x7ff2901f7d40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff28001a3a0 tx=0x7ff2800a6700 comp rx=0 tx=0).stop 2024-09-21T06:04:15.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.547+0000 7ff296600640 1 -- 172.21.15.50:0/2735140235 shutdown_connections 2024-09-21T06:04:15.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.547+0000 7ff296600640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff26c073d70 0x7ff26c076280 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.547+0000 7ff296600640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff290161160 0x7ff290159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.547+0000 7ff296600640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff290160d50 0x7ff2901509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.547+0000 7ff296600640 1 --2- 172.21.15.50:0/2735140235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff290157030 0x7ff2901f7d40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.547+0000 7ff296600640 1 -- 172.21.15.50:0/2735140235 >> 172.21.15.50:0/2735140235 conn(0x7ff2900708c0 msgr2=0x7ff2900b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:15.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.547+0000 7ff296600640 1 -- 172.21.15.50:0/2735140235 shutdown_connections 2024-09-21T06:04:15.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.547+0000 7ff296600640 1 -- 172.21.15.50:0/2735140235 wait complete. 2024-09-21T06:04:15.568 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T06:04:15.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:04:15.628 INFO:teuthology.orchestra.run.smithi050.stdout:2 2024-09-21T06:04:15.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:04:15.637 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:04:15.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.799+0000 7f0ebd000640 1 -- 172.21.15.50:0/3972178468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb81527b0 msgr2=0x7f0eb815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:15.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.799+0000 7f0ebd000640 1 --2- 172.21.15.50:0/3972178468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb81527b0 0x7f0eb815bb60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8074b20 tx=0x7f0ea001c760 comp rx=0 tx=0).stop 2024-09-21T06:04:15.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.799+0000 7f0ebd000640 1 -- 172.21.15.50:0/3972178468 shutdown_connections 2024-09-21T06:04:15.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.799+0000 7f0ebd000640 1 --2- 172.21.15.50:0/3972178468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb81527b0 0x7f0eb815bb60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.799+0000 7f0ebd000640 1 -- 172.21.15.50:0/3972178468 >> 172.21.15.50:0/3972178468 conn(0x7f0eb80b2040 msgr2=0x7f0eb80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:15.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.799+0000 7f0ebd000640 1 -- 172.21.15.50:0/3972178468 shutdown_connections 2024-09-21T06:04:15.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.799+0000 7f0ebd000640 1 -- 172.21.15.50:0/3972178468 wait complete. 2024-09-21T06:04:15.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.799+0000 7f0ebd000640 1 Processor -- start 2024-09-21T06:04:15.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.799+0000 7f0ebd000640 1 -- start start 2024-09-21T06:04:15.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0ebd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0eb80b4de0 0x7f0eb80cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:15.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0ebd000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0eb80b5c70 0x7f0eb81f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:15.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0ebd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb81527b0 0x7f0eb80cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:15.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0ebd000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8169bd0 con 0x7f0eb81527b0 2024-09-21T06:04:15.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0ebd000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8169a50 con 0x7f0eb80b4de0 2024-09-21T06:04:15.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0ebd000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb8169d50 con 0x7f0eb80b5c70 2024-09-21T06:04:15.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eb6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb81527b0 0x7f0eb80cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:15.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eb6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb81527b0 0x7f0eb80cbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39960/0 (socket says 172.21.15.50:39960) 2024-09-21T06:04:15.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eb6800640 1 -- 172.21.15.50:0/2280734183 learned_addr learned my addr 172.21.15.50:0/2280734183 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:15.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eafe00640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0eb80b5c70 0x7f0eb81f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:15.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eb5e00640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0eb80b4de0 0x7f0eb80cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:15.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eb6800640 1 -- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0eb80b5c70 msgr2=0x7f0eb81f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:15.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eb6800640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0eb80b5c70 0x7f0eb81f8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eb6800640 1 -- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0eb80b4de0 msgr2=0x7f0eb80cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:15.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eb6800640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0eb80b4de0 0x7f0eb80cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eb6800640 1 -- 172.21.15.50:0/2280734183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb80cf110 con 0x7f0eb81527b0 2024-09-21T06:04:15.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eb6800640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb81527b0 0x7f0eb80cbd10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000c730 tx=0x7f0eb000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:15.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eae000640 1 -- 172.21.15.50:0/2280734183 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0003b80 con 0x7f0eb81527b0 2024-09-21T06:04:15.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eae000640 1 -- 172.21.15.50:0/2280734183 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0eb0003d40 con 0x7f0eb81527b0 2024-09-21T06:04:15.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0ebd000640 1 -- 172.21.15.50:0/2280734183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb81f9730 con 0x7f0eb81527b0 2024-09-21T06:04:15.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0eae000640 1 -- 172.21.15.50:0/2280734183 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00052d0 con 0x7f0eb81527b0 2024-09-21T06:04:15.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0ebd000640 1 -- 172.21.15.50:0/2280734183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb80ce3d0 con 0x7f0eb81527b0 2024-09-21T06:04:15.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.803+0000 7f0ebd000640 1 -- 172.21.15.50:0/2280734183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb80ce1c0 con 0x7f0eb81527b0 2024-09-21T06:04:15.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.807+0000 7f0eae000640 1 -- 172.21.15.50:0/2280734183 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f0eb0005680 con 0x7f0eb81527b0 2024-09-21T06:04:15.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.807+0000 7f0eae000640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e94073e60 0x7f0e94076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:15.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.807+0000 7f0eae000640 1 -- 172.21.15.50:0/2280734183 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f0eb00929f0 con 0x7f0eb81527b0 2024-09-21T06:04:15.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.807+0000 7f0eb5e00640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e94073e60 0x7f0e94076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:15.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.807+0000 7f0eb5e00640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e94073e60 0x7f0e94076370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80098e0 tx=0x7f0ea8009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:15.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.811+0000 7f0eae000640 1 -- 172.21.15.50:0/2280734183 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0eb005eea0 con 0x7f0eb81527b0 2024-09-21T06:04:15.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.991+0000 7f0ebd000640 1 -- 172.21.15.50:0/2280734183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f0eb815bb60 con 0x7f0eb81527b0 2024-09-21T06:04:15.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.991+0000 7f0eae000640 1 -- 172.21.15.50:0/2280734183 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v39)=0 v39) ==== 131+0+0 (secure 0 0 0) 0x7f0eb00638d0 con 0x7f0eb81527b0 2024-09-21T06:04:15.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.995+0000 7f0ebd000640 1 -- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e94073e60 msgr2=0x7f0e94076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:15.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.995+0000 7f0ebd000640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e94073e60 0x7f0e94076370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80098e0 tx=0x7f0ea8009060 comp rx=0 tx=0).stop 2024-09-21T06:04:15.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.995+0000 7f0ebd000640 1 -- 172.21.15.50:0/2280734183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb81527b0 msgr2=0x7f0eb80cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:15.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.995+0000 7f0ebd000640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb81527b0 0x7f0eb80cbd10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000c730 tx=0x7f0eb000cbf0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.995+0000 7f0ebd000640 1 -- 172.21.15.50:0/2280734183 shutdown_connections 2024-09-21T06:04:15.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.995+0000 7f0ebd000640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e94073e60 0x7f0e94076370 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.995+0000 7f0ebd000640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb81527b0 0x7f0eb80cbd10 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.995+0000 7f0ebd000640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0eb80b5c70 0x7f0eb81f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.995+0000 7f0ebd000640 1 --2- 172.21.15.50:0/2280734183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0eb80b4de0 0x7f0eb80cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:15.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.995+0000 7f0ebd000640 1 -- 172.21.15.50:0/2280734183 >> 172.21.15.50:0/2280734183 conn(0x7f0eb80b2040 msgr2=0x7f0eb8151a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:15.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.995+0000 7f0ebd000640 1 -- 172.21.15.50:0/2280734183 shutdown_connections 2024-09-21T06:04:15.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:15.995+0000 7f0ebd000640 1 -- 172.21.15.50:0/2280734183 wait complete. 2024-09-21T06:04:16.016 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:04:16.016 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:04:16.105 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:04:16.206 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:04:16.206 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:04:16.206 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:04:16.206 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:04:16.216 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:04:16.216 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:04:16.266 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:04:16.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.531+0000 7f575b600640 1 -- 172.21.15.50:0/2494364541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57540d29e0 msgr2=0x7f57540d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:16.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.531+0000 7f575b600640 1 --2- 172.21.15.50:0/2494364541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57540d29e0 0x7f57540d3990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f574c009bc0 tx=0x7f574c01cb30 comp rx=0 tx=0).stop 2024-09-21T06:04:16.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.531+0000 7f575b600640 1 -- 172.21.15.50:0/2494364541 shutdown_connections 2024-09-21T06:04:16.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.531+0000 7f575b600640 1 --2- 172.21.15.50:0/2494364541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57540d29e0 0x7f57540d3990 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:16.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.531+0000 7f575b600640 1 -- 172.21.15.50:0/2494364541 >> 172.21.15.50:0/2494364541 conn(0x7f5754073e00 msgr2=0x7f575406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:16.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.531+0000 7f575b600640 1 -- 172.21.15.50:0/2494364541 shutdown_connections 2024-09-21T06:04:16.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.535+0000 7f575b600640 1 -- 172.21.15.50:0/2494364541 wait complete. 2024-09-21T06:04:16.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.539+0000 7f575b600640 1 Processor -- start 2024-09-21T06:04:16.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.539+0000 7f575b600640 1 -- start start 2024-09-21T06:04:16.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.539+0000 7f575b600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57540d16b0 0x7f57541566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:16.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.539+0000 7f575b600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57540d25d0 0x7f5754160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:16.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.539+0000 7f575b600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57540d29e0 0x7f57540b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:16.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.539+0000 7f575b600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f575416d720 con 0x7f57540d29e0 2024-09-21T06:04:16.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.539+0000 7f575b600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f575416d5a0 con 0x7f57540d16b0 2024-09-21T06:04:16.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.539+0000 7f575b600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f575416d8a0 con 0x7f57540d25d0 2024-09-21T06:04:16.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5758c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57540d16b0 0x7f57541566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:16.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5758c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57540d16b0 0x7f57541566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:34282/0 (socket says 172.21.15.50:34282) 2024-09-21T06:04:16.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5758c00640 1 -- 172.21.15.50:0/3022272283 learned_addr learned my addr 172.21.15.50:0/3022272283 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:16.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5753e00640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57540d25d0 0x7f5754160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:16.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5759600640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57540d29e0 0x7f57540b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:16.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5753e00640 1 -- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57540d16b0 msgr2=0x7f57541566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:16.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5753e00640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57540d16b0 0x7f57541566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:16.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5753e00640 1 -- 172.21.15.50:0/3022272283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57540d29e0 msgr2=0x7f57540b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:16.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5753e00640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57540d29e0 0x7f57540b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:16.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5753e00640 1 -- 172.21.15.50:0/3022272283 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5754051020 con 0x7f57540d25d0 2024-09-21T06:04:16.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5753e00640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57540d25d0 0x7f5754160e60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5754162420 tx=0x7f574000c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:16.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5750c00640 1 -- 172.21.15.50:0/3022272283 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740011790 con 0x7f57540d25d0 2024-09-21T06:04:16.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5750c00640 1 -- 172.21.15.50:0/3022272283 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5740017070 con 0x7f57540d25d0 2024-09-21T06:04:16.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f5750c00640 1 -- 172.21.15.50:0/3022272283 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740011c80 con 0x7f57540d25d0 2024-09-21T06:04:16.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f575b600640 1 -- 172.21.15.50:0/3022272283 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57540cf950 con 0x7f57540d25d0 2024-09-21T06:04:16.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.543+0000 7f575b600640 1 -- 172.21.15.50:0/3022272283 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f57540b8a50 con 0x7f57540d25d0 2024-09-21T06:04:16.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.547+0000 7f575b600640 1 -- 172.21.15.50:0/3022272283 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57540b87c0 con 0x7f57540d25d0 2024-09-21T06:04:16.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.547+0000 7f5750c00640 1 -- 172.21.15.50:0/3022272283 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f574001c460 con 0x7f57540d25d0 2024-09-21T06:04:16.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.551+0000 7f5750c00640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5734073ee0 0x7f57340763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:16.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.551+0000 7f5750c00640 1 -- 172.21.15.50:0/3022272283 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f57400931b0 con 0x7f57540d25d0 2024-09-21T06:04:16.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.551+0000 7f5758c00640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5734073ee0 0x7f57340763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:16.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.551+0000 7f5758c00640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5734073ee0 0x7f57340763f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f57480017e0 tx=0x7f5748009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:16.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.555+0000 7f5750c00640 1 -- 172.21.15.50:0/3022272283 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5740039070 con 0x7f57540d25d0 2024-09-21T06:04:16.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.979+0000 7f575b600640 1 -- 172.21.15.50:0/3022272283 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f57540d0fa0 con 0x7f57540d25d0 2024-09-21T06:04:16.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.979+0000 7f5750c00640 1 -- 172.21.15.50:0/3022272283 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7f574005fc60 con 0x7f57540d25d0 2024-09-21T06:04:16.983 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T06:04:16.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.987+0000 7f575b600640 1 -- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5734073ee0 msgr2=0x7f57340763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:16.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.987+0000 7f575b600640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5734073ee0 0x7f57340763f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f57480017e0 tx=0x7f5748009040 comp rx=0 tx=0).stop 2024-09-21T06:04:16.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.987+0000 7f575b600640 1 -- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57540d25d0 msgr2=0x7f5754160e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:16.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.987+0000 7f575b600640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57540d25d0 0x7f5754160e60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5754162420 tx=0x7f574000c820 comp rx=0 tx=0).stop 2024-09-21T06:04:16.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.987+0000 7f575b600640 1 -- 172.21.15.50:0/3022272283 shutdown_connections 2024-09-21T06:04:16.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.987+0000 7f575b600640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5734073ee0 0x7f57340763f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:16.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.987+0000 7f575b600640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57540d29e0 0x7f57540b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:16.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.987+0000 7f575b600640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57540d25d0 0x7f5754160e60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:16.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.987+0000 7f575b600640 1 --2- 172.21.15.50:0/3022272283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57540d16b0 0x7f57541566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:16.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.987+0000 7f575b600640 1 -- 172.21.15.50:0/3022272283 >> 172.21.15.50:0/3022272283 conn(0x7f5754073e00 msgr2=0x7f575406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:16.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.987+0000 7f575b600640 1 -- 172.21.15.50:0/3022272283 shutdown_connections 2024-09-21T06:04:16.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:16.987+0000 7f575b600640 1 -- 172.21.15.50:0/3022272283 wait complete. 2024-09-21T06:04:17.033 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:04:17.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:17.033 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:04:17.127 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:04:17.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:17.127 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:04:17.161 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:04:17.161 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:04:17.161 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:04:17.174 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:04:17.174 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:04:17.174 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:04:17.174 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:04:17.216 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:04:17.216 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:04:17.216 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:04:17.217 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:04:17.229 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:04:17.230 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:04:17.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.559+0000 7f813de00640 1 -- 172.21.15.50:0/1323184492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81380d24f0 msgr2=0x7f81380d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:17.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.559+0000 7f813de00640 1 --2- 172.21.15.50:0/1323184492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81380d24f0 0x7f81380d2900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8130009bc0 tx=0x7f813001ca30 comp rx=0 tx=0).stop 2024-09-21T06:04:17.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.559+0000 7f813de00640 1 -- 172.21.15.50:0/1323184492 shutdown_connections 2024-09-21T06:04:17.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.559+0000 7f813de00640 1 --2- 172.21.15.50:0/1323184492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81380d24f0 0x7f81380d2900 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:17.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.559+0000 7f813de00640 1 -- 172.21.15.50:0/1323184492 >> 172.21.15.50:0/1323184492 conn(0x7f8138073a10 msgr2=0x7f813806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:17.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.559+0000 7f813de00640 1 -- 172.21.15.50:0/1323184492 shutdown_connections 2024-09-21T06:04:17.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.559+0000 7f813de00640 1 -- 172.21.15.50:0/1323184492 wait complete. 2024-09-21T06:04:17.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.567+0000 7f813de00640 1 Processor -- start 2024-09-21T06:04:17.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.567+0000 7f813de00640 1 -- start start 2024-09-21T06:04:17.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.571+0000 7f813de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81380d0ce0 0x7f813814e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:17.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.571+0000 7f813de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81380d24f0 0x7f8138157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:17.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.571+0000 7f813de00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81381625f0 0x7f81380c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:17.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813de00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f813816d5a0 con 0x7f81380d24f0 2024-09-21T06:04:17.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813de00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f813816d420 con 0x7f81380d0ce0 2024-09-21T06:04:17.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813d400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81381625f0 0x7f81380c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:17.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813d400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81381625f0 0x7f81380c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:50590/0 (socket says 172.21.15.50:50590) 2024-09-21T06:04:17.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813d400640 1 -- 172.21.15.50:0/1911090210 learned_addr learned my addr 172.21.15.50:0/1911090210 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:17.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813ca00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81380d0ce0 0x7f813814e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:17.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813de00640 1 -- 172.21.15.50:0/1911090210 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f813816d720 con 0x7f81381625f0 2024-09-21T06:04:17.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f8137e00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81380d24f0 0x7f8138157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:17.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813ca00640 1 -- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81381625f0 msgr2=0x7f81380c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:17.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813ca00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81381625f0 0x7f81380c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:17.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813ca00640 1 -- 172.21.15.50:0/1911090210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81380d24f0 msgr2=0x7f8138157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:17.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813ca00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81380d24f0 0x7f8138157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:17.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813ca00640 1 -- 172.21.15.50:0/1911090210 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81380b3fa0 con 0x7f81380d0ce0 2024-09-21T06:04:17.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813d400640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81381625f0 0x7f81380c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:17.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f8137e00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81380d24f0 0x7f8138157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:17.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813ca00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81380d0ce0 0x7f813814e320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f812c001390 tx=0x7f812c00c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:17.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f8134c00640 1 -- 172.21.15.50:0/1911090210 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c011790 con 0x7f81380d0ce0 2024-09-21T06:04:17.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813de00640 1 -- 172.21.15.50:0/1911090210 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81380b4280 con 0x7f81380d0ce0 2024-09-21T06:04:17.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f8134c00640 1 -- 172.21.15.50:0/1911090210 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f812c017070 con 0x7f81380d0ce0 2024-09-21T06:04:17.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f8134c00640 1 -- 172.21.15.50:0/1911090210 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c011790 con 0x7f81380d0ce0 2024-09-21T06:04:17.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.575+0000 7f813de00640 1 -- 172.21.15.50:0/1911090210 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81380b4840 con 0x7f81380d0ce0 2024-09-21T06:04:17.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.579+0000 7f813de00640 1 -- 172.21.15.50:0/1911090210 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81380d2900 con 0x7f81380d0ce0 2024-09-21T06:04:17.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.579+0000 7f8134c00640 1 -- 172.21.15.50:0/1911090210 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f812c011a80 con 0x7f81380d0ce0 2024-09-21T06:04:17.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.583+0000 7f8134c00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8118074060 0x7f8118076570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:17.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.583+0000 7f8134c00640 1 -- 172.21.15.50:0/1911090210 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5793+0+0 (secure 0 0 0) 0x7f812c092580 con 0x7f81380d0ce0 2024-09-21T06:04:17.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.583+0000 7f8137e00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8118074060 0x7f8118076570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:17.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.583+0000 7f8137e00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8118074060 0x7f8118076570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8124000c10 tx=0x7f8124009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:17.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:17.587+0000 7f8134c00640 1 -- 172.21.15.50:0/1911090210 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f812c05ea00 con 0x7f81380d0ce0 2024-09-21T06:04:17.672 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:04:17.678 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:04:18.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.039+0000 7f813de00640 1 -- 172.21.15.50:0/1911090210 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f813815eb40 con 0x7f81380d0ce0 2024-09-21T06:04:18.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.039+0000 7f8134c00640 1 -- 172.21.15.50:0/1911090210 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5217 (secure 0 0 0) 0x7f812c063430 con 0x7f81380d0ce0 2024-09-21T06:04:18.158 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:18.158 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":37,"btime":"2024-09-21T06:02:23:310529+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/288562749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":288562749},{"type":"v1","addr":"172.21.15.50:6834","nonce":288562749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5686,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2920049831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2920049831},{"type":"v1","addr":"172.21.15.88:6836","nonce":2920049831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1072517376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1072517376},{"type":"v1","addr":"172.21.15.88:6837","nonce":1072517376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:02:09.689390+0000","modified":"2024-09-21T06:02:12.699140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/451843324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":451843324},{"type":"v1","addr":"172.21.15.50:6835","nonce":451843324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5695,"qdb_cluster":[5695]},"id":4}]} 2024-09-21T06:04:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 37 2024-09-21T06:04:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.043+0000 7f813de00640 1 -- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8118074060 msgr2=0x7f8118076570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.043+0000 7f813de00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8118074060 0x7f8118076570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8124000c10 tx=0x7f8124009040 comp rx=0 tx=0).stop 2024-09-21T06:04:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.043+0000 7f813de00640 1 -- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81380d0ce0 msgr2=0x7f813814e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.043+0000 7f813de00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81380d0ce0 0x7f813814e320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f812c001390 tx=0x7f812c00c8e0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.047+0000 7f813de00640 1 -- 172.21.15.50:0/1911090210 shutdown_connections 2024-09-21T06:04:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.047+0000 7f813de00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81381625f0 0x7f81380c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.047+0000 7f813de00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8118074060 0x7f8118076570 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.047+0000 7f813de00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81380d24f0 0x7f8138157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.047+0000 7f813de00640 1 --2- 172.21.15.50:0/1911090210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81380d0ce0 0x7f813814e320 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.047+0000 7f813de00640 1 -- 172.21.15.50:0/1911090210 >> 172.21.15.50:0/1911090210 conn(0x7f8138073a10 msgr2=0x7f8138160150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:18.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.047+0000 7f813de00640 1 -- 172.21.15.50:0/1911090210 shutdown_connections 2024-09-21T06:04:18.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.047+0000 7f813de00640 1 -- 172.21.15.50:0/1911090210 wait complete. 2024-09-21T06:04:18.161 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:04:18.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.323+0000 7f5f3b800640 1 -- 172.21.15.50:0/2839172508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f341527b0 msgr2=0x7f5f3415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:18.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.323+0000 7f5f3b800640 1 --2- 172.21.15.50:0/2839172508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f341527b0 0x7f5f3415bb60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5f34074b20 tx=0x7f5f1c01c760 comp rx=0 tx=0).stop 2024-09-21T06:04:18.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.323+0000 7f5f3b800640 1 -- 172.21.15.50:0/2839172508 shutdown_connections 2024-09-21T06:04:18.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.323+0000 7f5f3b800640 1 --2- 172.21.15.50:0/2839172508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f341527b0 0x7f5f3415bb60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.323+0000 7f5f3b800640 1 -- 172.21.15.50:0/2839172508 >> 172.21.15.50:0/2839172508 conn(0x7f5f340b2040 msgr2=0x7f5f340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:18.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.323+0000 7f5f3b800640 1 -- 172.21.15.50:0/2839172508 shutdown_connections 2024-09-21T06:04:18.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.323+0000 7f5f3b800640 1 -- 172.21.15.50:0/2839172508 wait complete. 2024-09-21T06:04:18.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f3b800640 1 Processor -- start 2024-09-21T06:04:18.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f3b800640 1 -- start start 2024-09-21T06:04:18.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f3b800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f340b4de0 0x7f5f3413dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:18.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f3b800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5f340b5c70 0x7f5f34153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:18.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f3b800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f341527b0 0x7f5f341541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:18.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f3b800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34169b30 con 0x7f5f340b4de0 2024-09-21T06:04:18.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f3b800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341699b0 con 0x7f5f341527b0 2024-09-21T06:04:18.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f3b800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5f34169cb0 con 0x7f5f340b5c70 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f38e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f340b4de0 0x7f5f3413dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f38e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f340b4de0 0x7f5f3413dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58134/0 (socket says 172.21.15.50:58134) 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f38e00640 1 -- 172.21.15.50:0/1352437917 learned_addr learned my addr 172.21.15.50:0/1352437917 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f39800640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f341527b0 0x7f5f341541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f2be00640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5f340b5c70 0x7f5f34153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f38e00640 1 -- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5f340b5c70 msgr2=0x7f5f34153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f38e00640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5f340b5c70 0x7f5f34153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f38e00640 1 -- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f341527b0 msgr2=0x7f5f341541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f38e00640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f341527b0 0x7f5f341541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f38e00640 1 -- 172.21.15.50:0/1352437917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f34154980 con 0x7f5f340b4de0 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f38e00640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f340b4de0 0x7f5f3413dcd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400c420 tx=0x7f5f2400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f31600640 1 -- 172.21.15.50:0/1352437917 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24017070 con 0x7f5f340b4de0 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f31600640 1 -- 172.21.15.50:0/1352437917 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f240053c0 con 0x7f5f340b4de0 2024-09-21T06:04:18.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f3b800640 1 -- 172.21.15.50:0/1352437917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f34154c60 con 0x7f5f340b4de0 2024-09-21T06:04:18.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f31600640 1 -- 172.21.15.50:0/1352437917 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24003860 con 0x7f5f340b4de0 2024-09-21T06:04:18.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.327+0000 7f5f3b800640 1 -- 172.21.15.50:0/1352437917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f34155220 con 0x7f5f340b4de0 2024-09-21T06:04:18.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.331+0000 7f5f3b800640 1 -- 172.21.15.50:0/1352437917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3415bb60 con 0x7f5f340b4de0 2024-09-21T06:04:18.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.331+0000 7f5f31600640 1 -- 172.21.15.50:0/1352437917 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f5f2401a070 con 0x7f5f340b4de0 2024-09-21T06:04:18.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.331+0000 7f5f31600640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5f10073e60 0x7f5f10076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:18.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.331+0000 7f5f31600640 1 -- 172.21.15.50:0/1352437917 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5793+0+0 (secure 0 0 0) 0x7f5f24092160 con 0x7f5f340b4de0 2024-09-21T06:04:18.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.335+0000 7f5f2be00640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5f10073e60 0x7f5f10076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:18.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.335+0000 7f5f2be00640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5f10073e60 0x7f5f10076370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0af000 tx=0x7f5f1c01afa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:18.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.339+0000 7f5f31600640 1 -- 172.21.15.50:0/1352437917 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5f2405e390 con 0x7f5f340b4de0 2024-09-21T06:04:18.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 -- 172.21.15.50:0/1352437917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f340afc90 con 0x7f5f340b4de0 2024-09-21T06:04:18.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f31600640 1 -- 172.21.15.50:0/1352437917 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5217 (secure 0 0 0) 0x7f5f24062dc0 con 0x7f5f340b4de0 2024-09-21T06:04:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:18.552 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":37,"btime":"2024-09-21T06:02:23:310529+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/288562749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":288562749},{"type":"v1","addr":"172.21.15.50:6834","nonce":288562749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5686,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2920049831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2920049831},{"type":"v1","addr":"172.21.15.88:6836","nonce":2920049831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1072517376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1072517376},{"type":"v1","addr":"172.21.15.88:6837","nonce":1072517376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:02:09.689390+0000","modified":"2024-09-21T06:02:12.699140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/451843324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":451843324},{"type":"v1","addr":"172.21.15.50:6835","nonce":451843324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5695,"qdb_cluster":[5695]},"id":4}]} 2024-09-21T06:04:18.552 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 37 2024-09-21T06:04:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 -- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5f10073e60 msgr2=0x7f5f10076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5f10073e60 0x7f5f10076370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0af000 tx=0x7f5f1c01afa0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 -- 172.21.15.50:0/1352437917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f340b4de0 msgr2=0x7f5f3413dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f340b4de0 0x7f5f3413dcd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400c420 tx=0x7f5f2400c8e0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 -- 172.21.15.50:0/1352437917 shutdown_connections 2024-09-21T06:04:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f341527b0 0x7f5f341541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5f340b5c70 0x7f5f34153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5f10073e60 0x7f5f10076370 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 --2- 172.21.15.50:0/1352437917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f340b4de0 0x7f5f3413dcd0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 -- 172.21.15.50:0/1352437917 >> 172.21.15.50:0/1352437917 conn(0x7f5f340b2040 msgr2=0x7f5f340b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:18.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 -- 172.21.15.50:0/1352437917 shutdown_connections 2024-09-21T06:04:18.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.551+0000 7f5f3b800640 1 -- 172.21.15.50:0/1352437917 wait complete. 2024-09-21T06:04:18.575 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:04:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.735+0000 7f137ee00640 1 -- 172.21.15.50:0/3387074603 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f137815aa50 msgr2=0x7f1378166670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.735+0000 7f137ee00640 1 --2- 172.21.15.50:0/3387074603 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f137815aa50 0x7f1378166670 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1370009bf0 tx=0x7f137001cbe0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.735+0000 7f137ee00640 1 -- 172.21.15.50:0/3387074603 shutdown_connections 2024-09-21T06:04:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.735+0000 7f137ee00640 1 --2- 172.21.15.50:0/3387074603 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f137815aa50 0x7f1378166670 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.735+0000 7f137ee00640 1 -- 172.21.15.50:0/3387074603 >> 172.21.15.50:0/3387074603 conn(0x7f13780b2040 msgr2=0x7f13780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.735+0000 7f137ee00640 1 -- 172.21.15.50:0/3387074603 shutdown_connections 2024-09-21T06:04:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.735+0000 7f137ee00640 1 -- 172.21.15.50:0/3387074603 wait complete. 2024-09-21T06:04:18.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.735+0000 7f137ee00640 1 Processor -- start 2024-09-21T06:04:18.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.735+0000 7f137ee00640 1 -- start start 2024-09-21T06:04:18.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f137ee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13780cdaf0 0x7f13781516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:18.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f137ee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13780ce4c0 0x7f137814fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:18.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f137ee00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f137815aa50 0x7f1378150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:18.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f137ee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1378168dd0 con 0x7f13780cdaf0 2024-09-21T06:04:18.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f137ee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f13780b5f70 con 0x7f13780ce4c0 2024-09-21T06:04:18.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f137ee00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1378168f50 con 0x7f137815aa50 2024-09-21T06:04:18.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f1377e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13780cdaf0 0x7f13781516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:18.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f1377e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13780cdaf0 0x7f13781516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58146/0 (socket says 172.21.15.50:58146) 2024-09-21T06:04:18.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f1377e00640 1 -- 172.21.15.50:0/1623546955 learned_addr learned my addr 172.21.15.50:0/1623546955 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f137ce00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f137815aa50 0x7f1378150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f136fe00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13780ce4c0 0x7f137814fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f1377e00640 1 -- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f137815aa50 msgr2=0x7f1378150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f1377e00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f137815aa50 0x7f1378150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f1377e00640 1 -- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13780ce4c0 msgr2=0x7f137814fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f1377e00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13780ce4c0 0x7f137814fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f1377e00640 1 -- 172.21.15.50:0/1623546955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1378157070 con 0x7f13780cdaf0 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f136fe00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13780ce4c0 0x7f137814fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f1377e00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13780cdaf0 0x7f13781516c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f136800c3f0 tx=0x7f136800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f1374c00640 1 -- 172.21.15.50:0/1623546955 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368017070 con 0x7f13780cdaf0 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f1374c00640 1 -- 172.21.15.50:0/1623546955 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1368003600 con 0x7f13780cdaf0 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f1374c00640 1 -- 172.21.15.50:0/1623546955 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368003a50 con 0x7f13780cdaf0 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f137ee00640 1 -- 172.21.15.50:0/1623546955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1378157350 con 0x7f13780cdaf0 2024-09-21T06:04:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f137ee00640 1 -- 172.21.15.50:0/1623546955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1378157910 con 0x7f13780cdaf0 2024-09-21T06:04:18.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.739+0000 7f137ee00640 1 -- 172.21.15.50:0/1623546955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f137815e4e0 con 0x7f13780cdaf0 2024-09-21T06:04:18.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.743+0000 7f1374c00640 1 -- 172.21.15.50:0/1623546955 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f136801a070 con 0x7f13780cdaf0 2024-09-21T06:04:18.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.743+0000 7f1374c00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1354073e90 0x7f13540763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:18.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.743+0000 7f1374c00640 1 -- 172.21.15.50:0/1623546955 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5793+0+0 (secure 0 0 0) 0x7f1368091fb0 con 0x7f13780cdaf0 2024-09-21T06:04:18.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.743+0000 7f136fe00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1354073e90 0x7f13540763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:18.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.743+0000 7f136fe00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1354073e90 0x7f13540763a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f13781582a0 tx=0x7f1360009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:18.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.747+0000 7f1374c00640 1 -- 172.21.15.50:0/1623546955 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f136805e1e0 con 0x7f13780cdaf0 2024-09-21T06:04:18.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.923+0000 7f137ee00640 1 -- 172.21.15.50:0/1623546955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13780008e0 con 0x7f13780cdaf0 2024-09-21T06:04:18.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.923+0000 7f1374c00640 1 -- 172.21.15.50:0/1623546955 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16829 (secure 0 0 0) 0x7f1368062c10 con 0x7f13780cdaf0 2024-09-21T06:04:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:18.926 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":60,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:04:16.626967+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:02:08.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T06:02:08.630388+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:04:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.927+0000 7f137ee00640 1 -- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1354073e90 msgr2=0x7f13540763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.927+0000 7f137ee00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1354073e90 0x7f13540763a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f13781582a0 tx=0x7f1360009040 comp rx=0 tx=0).stop 2024-09-21T06:04:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.927+0000 7f137ee00640 1 -- 172.21.15.50:0/1623546955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13780cdaf0 msgr2=0x7f13781516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.927+0000 7f137ee00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13780cdaf0 0x7f13781516c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f136800c3f0 tx=0x7f136800c8b0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.927+0000 7f137ee00640 1 -- 172.21.15.50:0/1623546955 shutdown_connections 2024-09-21T06:04:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.927+0000 7f137ee00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f137815aa50 0x7f1378150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.927+0000 7f137ee00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13780ce4c0 0x7f137814fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.927+0000 7f137ee00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1354073e90 0x7f13540763a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.927+0000 7f137ee00640 1 --2- 172.21.15.50:0/1623546955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13780cdaf0 0x7f13781516c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:18.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.927+0000 7f137ee00640 1 -- 172.21.15.50:0/1623546955 >> 172.21.15.50:0/1623546955 conn(0x7f13780b2040 msgr2=0x7f13780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:18.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.927+0000 7f137ee00640 1 -- 172.21.15.50:0/1623546955 shutdown_connections 2024-09-21T06:04:18.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:18.927+0000 7f137ee00640 1 -- 172.21.15.50:0/1623546955 wait complete. 2024-09-21T06:04:18.948 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:04:18.949 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:04:19.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.107+0000 7f4b24e00640 1 -- 172.21.15.50:0/2688504204 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b201527b0 msgr2=0x7f4b2015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.107+0000 7f4b24e00640 1 --2- 172.21.15.50:0/2688504204 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b201527b0 0x7f4b2015bb60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009870 tx=0x7f4b0801c7a0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.107+0000 7f4b24e00640 1 -- 172.21.15.50:0/2688504204 shutdown_connections 2024-09-21T06:04:19.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.107+0000 7f4b24e00640 1 --2- 172.21.15.50:0/2688504204 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b201527b0 0x7f4b2015bb60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.107+0000 7f4b24e00640 1 -- 172.21.15.50:0/2688504204 >> 172.21.15.50:0/2688504204 conn(0x7f4b200b2040 msgr2=0x7f4b200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:19.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.107+0000 7f4b24e00640 1 -- 172.21.15.50:0/2688504204 shutdown_connections 2024-09-21T06:04:19.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.107+0000 7f4b24e00640 1 -- 172.21.15.50:0/2688504204 wait complete. 2024-09-21T06:04:19.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.107+0000 7f4b24e00640 1 Processor -- start 2024-09-21T06:04:19.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.107+0000 7f4b24e00640 1 -- start start 2024-09-21T06:04:19.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b24e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b200b4de0 0x7f4b201f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:19.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b24e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b200b5c70 0x7f4b2013d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:19.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b24e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b201527b0 0x7f4b201fc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:19.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b24e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20169c60 con 0x7f4b200b5c70 2024-09-21T06:04:19.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b24e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20169ae0 con 0x7f4b201527b0 2024-09-21T06:04:19.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b24e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4b20169de0 con 0x7f4b200b4de0 2024-09-21T06:04:19.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b17e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b200b5c70 0x7f4b2013d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:19.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b17e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b200b5c70 0x7f4b2013d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58152/0 (socket says 172.21.15.50:58152) 2024-09-21T06:04:19.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b17e00640 1 -- 172.21.15.50:0/2188941626 learned_addr learned my addr 172.21.15.50:0/2188941626 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:19.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b1de00640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b200b4de0 0x7f4b201f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:19.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b1e800640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b201527b0 0x7f4b201fc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:19.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b17e00640 1 -- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b200b4de0 msgr2=0x7f4b201f8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b17e00640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b200b4de0 0x7f4b201f8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b17e00640 1 -- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b201527b0 msgr2=0x7f4b201fc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b17e00640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b201527b0 0x7f4b201fc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b17e00640 1 -- 172.21.15.50:0/2188941626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b201f94b0 con 0x7f4b200b5c70 2024-09-21T06:04:19.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b1e800640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b201527b0 0x7f4b201fc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:19.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b17e00640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b200b5c70 0x7f4b2013d880 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4b08000c10 tx=0x7f4b08005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:19.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b16000640 1 -- 172.21.15.50:0/2188941626 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b080bb070 con 0x7f4b200b5c70 2024-09-21T06:04:19.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b24e00640 1 -- 172.21.15.50:0/2188941626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b201f9790 con 0x7f4b200b5c70 2024-09-21T06:04:19.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b16000640 1 -- 172.21.15.50:0/2188941626 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b080043f0 con 0x7f4b200b5c70 2024-09-21T06:04:19.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b16000640 1 -- 172.21.15.50:0/2188941626 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b080b2040 con 0x7f4b200b5c70 2024-09-21T06:04:19.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b24e00640 1 -- 172.21.15.50:0/2188941626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b201f8fc0 con 0x7f4b200b5c70 2024-09-21T06:04:19.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.111+0000 7f4b24e00640 1 -- 172.21.15.50:0/2188941626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2015bb60 con 0x7f4b200b5c70 2024-09-21T06:04:19.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.115+0000 7f4b16000640 1 -- 172.21.15.50:0/2188941626 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f4b08005290 con 0x7f4b200b5c70 2024-09-21T06:04:19.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.115+0000 7f4b16000640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4afc073e60 0x7f4afc076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:19.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.115+0000 7f4b16000640 1 -- 172.21.15.50:0/2188941626 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5793+0+0 (secure 0 0 0) 0x7f4b0812c4f0 con 0x7f4b200b5c70 2024-09-21T06:04:19.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.115+0000 7f4b1de00640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4afc073e60 0x7f4afc076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:19.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.115+0000 7f4b1de00640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4afc073e60 0x7f4afc076370 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4b10007fd0 tx=0x7f4b100013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.119+0000 7f4b16000640 1 -- 172.21.15.50:0/2188941626 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4b08016450 con 0x7f4b200b5c70 2024-09-21T06:04:19.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.331+0000 7f4b24e00640 1 -- 172.21.15.50:0/2188941626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b201f9220 con 0x7f4b200b5c70 2024-09-21T06:04:19.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.331+0000 7f4b16000640 1 -- 172.21.15.50:0/2188941626 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5217 (secure 0 0 0) 0x7f4b080f8920 con 0x7f4b200b5c70 2024-09-21T06:04:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:19.333 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":37,"btime":"2024-09-21T06:02:23:310529+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/288562749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":288562749},{"type":"v1","addr":"172.21.15.50:6834","nonce":288562749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5686,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2920049831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2920049831},{"type":"v1","addr":"172.21.15.88:6836","nonce":2920049831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1072517376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1072517376},{"type":"v1","addr":"172.21.15.88:6837","nonce":1072517376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:02:09.689390+0000","modified":"2024-09-21T06:02:12.699140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/451843324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":451843324},{"type":"v1","addr":"172.21.15.50:6835","nonce":451843324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5695,"qdb_cluster":[5695]},"id":4}]} 2024-09-21T06:04:19.333 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 37 2024-09-21T06:04:19.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.335+0000 7f4b24e00640 1 -- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4afc073e60 msgr2=0x7f4afc076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.335+0000 7f4b24e00640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4afc073e60 0x7f4afc076370 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4b10007fd0 tx=0x7f4b100013a0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.335+0000 7f4b24e00640 1 -- 172.21.15.50:0/2188941626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b200b5c70 msgr2=0x7f4b2013d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.335+0000 7f4b24e00640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b200b5c70 0x7f4b2013d880 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4b08000c10 tx=0x7f4b08005db0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.335+0000 7f4b24e00640 1 -- 172.21.15.50:0/2188941626 shutdown_connections 2024-09-21T06:04:19.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.335+0000 7f4b24e00640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b201527b0 0x7f4b201fc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.335+0000 7f4b24e00640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4afc073e60 0x7f4afc076370 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.335+0000 7f4b24e00640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b200b5c70 0x7f4b2013d880 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.335+0000 7f4b24e00640 1 --2- 172.21.15.50:0/2188941626 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4b200b4de0 0x7f4b201f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.335+0000 7f4b24e00640 1 -- 172.21.15.50:0/2188941626 >> 172.21.15.50:0/2188941626 conn(0x7f4b200b2040 msgr2=0x7f4b20154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:19.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.335+0000 7f4b24e00640 1 -- 172.21.15.50:0/2188941626 shutdown_connections 2024-09-21T06:04:19.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.335+0000 7f4b24e00640 1 -- 172.21.15.50:0/2188941626 wait complete. 2024-09-21T06:04:19.356 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:04:19.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.515+0000 7f9f1f400640 1 -- 172.21.15.50:0/2681420203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f18160d50 msgr2=0x7f9f18074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.515+0000 7f9f1f400640 1 --2- 172.21.15.50:0/2681420203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f18160d50 0x7f9f18074410 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9f18160830 tx=0x7f9f0001c8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.515+0000 7f9f1f400640 1 -- 172.21.15.50:0/2681420203 shutdown_connections 2024-09-21T06:04:19.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.515+0000 7f9f1f400640 1 --2- 172.21.15.50:0/2681420203 >> v2:172.21.15.88:3300/0 conn(0x7f9f18161160 0x7f9f18074950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.515+0000 7f9f1f400640 1 --2- 172.21.15.50:0/2681420203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f18160d50 0x7f9f18074410 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.515+0000 7f9f1f400640 1 --2- 172.21.15.50:0/2681420203 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9f18157030 0x7f9f180b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.515+0000 7f9f1f400640 1 -- 172.21.15.50:0/2681420203 >> 172.21.15.50:0/2681420203 conn(0x7f9f180708c0 msgr2=0x7f9f18070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:19.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.515+0000 7f9f1f400640 1 -- 172.21.15.50:0/2681420203 shutdown_connections 2024-09-21T06:04:19.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.515+0000 7f9f1f400640 1 -- 172.21.15.50:0/2681420203 wait complete. 2024-09-21T06:04:19.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.515+0000 7f9f1f400640 1 Processor -- start 2024-09-21T06:04:19.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.515+0000 7f9f1f400640 1 -- start start 2024-09-21T06:04:19.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f18157030 0x7f9f1806e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:19.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9f18160d50 0x7f9f18150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:19.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f18161160 0x7f9f1814c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:19.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1f400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18072290 con 0x7f9f18161160 2024-09-21T06:04:19.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1f400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18072110 con 0x7f9f18157030 2024-09-21T06:04:19.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1f400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9f18072410 con 0x7f9f18160d50 2024-09-21T06:04:19.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f18157030 0x7f9f1806e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:19.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f18161160 0x7f9f1814c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:19.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f18157030 0x7f9f1806e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:40230/0 (socket says 172.21.15.50:40230) 2024-09-21T06:04:19.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1ca00640 1 -- 172.21.15.50:0/2499922931 learned_addr learned my addr 172.21.15.50:0/2499922931 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:19.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f0fe00640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9f18160d50 0x7f9f18150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:19.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1d400640 1 -- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9f18160d50 msgr2=0x7f9f18150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1d400640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9f18160d50 0x7f9f18150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1d400640 1 -- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f18157030 msgr2=0x7f9f1806e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1d400640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f18157030 0x7f9f1806e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1d400640 1 -- 172.21.15.50:0/2499922931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f180d2070 con 0x7f9f18161160 2024-09-21T06:04:19.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1d400640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f18161160 0x7f9f1814c600 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000e9f0 tx=0x7f9f1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:19.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f15600640 1 -- 172.21.15.50:0/2499922931 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1000cde0 con 0x7f9f18161160 2024-09-21T06:04:19.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f15600640 1 -- 172.21.15.50:0/2499922931 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f10004c20 con 0x7f9f18161160 2024-09-21T06:04:19.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f15600640 1 -- 172.21.15.50:0/2499922931 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10005020 con 0x7f9f18161160 2024-09-21T06:04:19.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1f400640 1 -- 172.21.15.50:0/2499922931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1813f300 con 0x7f9f18161160 2024-09-21T06:04:19.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1f400640 1 -- 172.21.15.50:0/2499922931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1814fa60 con 0x7f9f18161160 2024-09-21T06:04:19.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.519+0000 7f9f1f400640 1 -- 172.21.15.50:0/2499922931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f180728c0 con 0x7f9f18161160 2024-09-21T06:04:19.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.523+0000 7f9f15600640 1 -- 172.21.15.50:0/2499922931 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f9f100051c0 con 0x7f9f18161160 2024-09-21T06:04:19.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.523+0000 7f9f15600640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ef4073e90 0x7f9ef40763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:19.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.523+0000 7f9f15600640 1 -- 172.21.15.50:0/2499922931 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5793+0+0 (secure 0 0 0) 0x7f9f100938e0 con 0x7f9f18161160 2024-09-21T06:04:19.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.523+0000 7f9f1ca00640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ef4073e90 0x7f9ef40763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:19.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.523+0000 7f9f1ca00640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ef4073e90 0x7f9ef40763a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009b80 tx=0x7f9f080091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:19.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.527+0000 7f9f15600640 1 -- 172.21.15.50:0/2499922931 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9f1005fc90 con 0x7f9f18161160 2024-09-21T06:04:19.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.743+0000 7f9f1f400640 1 -- 172.21.15.50:0/2499922931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f1806e180 con 0x7f9f18161160 2024-09-21T06:04:19.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.743+0000 7f9f15600640 1 -- 172.21.15.50:0/2499922931 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5217 (secure 0 0 0) 0x7f9f100646c0 con 0x7f9f18161160 2024-09-21T06:04:19.745 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:19.746 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":37,"btime":"2024-09-21T06:02:23:310529+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/288562749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":288562749},{"type":"v1","addr":"172.21.15.50:6834","nonce":288562749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5686,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2920049831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2920049831},{"type":"v1","addr":"172.21.15.88:6836","nonce":2920049831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1072517376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1072517376},{"type":"v1","addr":"172.21.15.88:6837","nonce":1072517376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:02:09.689390+0000","modified":"2024-09-21T06:02:12.699140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/451843324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":451843324},{"type":"v1","addr":"172.21.15.50:6835","nonce":451843324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5695,"qdb_cluster":[5695]},"id":4}]} 2024-09-21T06:04:19.746 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 37 2024-09-21T06:04:19.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.747+0000 7f9f1f400640 1 -- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ef4073e90 msgr2=0x7f9ef40763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.747+0000 7f9f1f400640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ef4073e90 0x7f9ef40763a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009b80 tx=0x7f9f080091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.747+0000 7f9f1f400640 1 -- 172.21.15.50:0/2499922931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f18161160 msgr2=0x7f9f1814c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.747+0000 7f9f1f400640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f18161160 0x7f9f1814c600 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000e9f0 tx=0x7f9f1000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.747+0000 7f9f1f400640 1 -- 172.21.15.50:0/2499922931 shutdown_connections 2024-09-21T06:04:19.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.747+0000 7f9f1f400640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ef4073e90 0x7f9ef40763a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.747+0000 7f9f1f400640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f18161160 0x7f9f1814c600 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.747+0000 7f9f1f400640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9f18160d50 0x7f9f18150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.747+0000 7f9f1f400640 1 --2- 172.21.15.50:0/2499922931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f18157030 0x7f9f1806e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.747+0000 7f9f1f400640 1 -- 172.21.15.50:0/2499922931 >> 172.21.15.50:0/2499922931 conn(0x7f9f180708c0 msgr2=0x7f9f180d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:19.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.747+0000 7f9f1f400640 1 -- 172.21.15.50:0/2499922931 shutdown_connections 2024-09-21T06:04:19.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.747+0000 7f9f1f400640 1 -- 172.21.15.50:0/2499922931 wait complete. 2024-09-21T06:04:19.768 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:04:19.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.923+0000 7f4cd9400640 1 -- 172.21.15.50:0/2168434624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cb8005320 msgr2=0x7f4cb800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.923+0000 7f4cd9400640 1 --2- 172.21.15.50:0/2168434624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cb8005320 0x7f4cb800b9d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00c630 tx=0x7f4ccc01d200 comp rx=0 tx=0).stop 2024-09-21T06:04:19.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.923+0000 7f4cd3200640 1 -- 172.21.15.50:0/2168434624 reap_dead start 2024-09-21T06:04:19.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.923+0000 7f4cd9400640 1 -- 172.21.15.50:0/2168434624 shutdown_connections 2024-09-21T06:04:19.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.923+0000 7f4cd9400640 1 -- 172.21.15.50:0/2168434624 >> 172.21.15.50:0/2168434624 conn(0x7f4cd40b35f0 msgr2=0x7f4cd40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:19.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.923+0000 7f4cd9400640 1 -- 172.21.15.50:0/2168434624 shutdown_connections 2024-09-21T06:04:19.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.923+0000 7f4cd9400640 1 -- 172.21.15.50:0/2168434624 wait complete. 2024-09-21T06:04:19.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.927+0000 7f4cd9400640 1 Processor -- start 2024-09-21T06:04:19.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.927+0000 7f4cd9400640 1 -- start start 2024-09-21T06:04:19.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.927+0000 7f4cd9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cb8005320 0x7f4cd4140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:19.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.927+0000 7f4cd9400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4cd41f7190 0x7f4cd41f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:19.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.927+0000 7f4cd9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd415e100 0x7f4cd415e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:19.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.927+0000 7f4cd9400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd40b5ef0 con 0x7f4cd415e100 2024-09-21T06:04:19.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.927+0000 7f4cd9400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd40b5d70 con 0x7f4cb8005320 2024-09-21T06:04:19.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.927+0000 7f4cd9400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd40b6070 con 0x7f4cd41f7190 2024-09-21T06:04:19.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.927+0000 7f4cd3200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd415e100 0x7f4cd415e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:19.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.927+0000 7f4cd3200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd415e100 0x7f4cd415e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58194/0 (socket says 172.21.15.50:58194) 2024-09-21T06:04:19.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.927+0000 7f4cd3200640 1 -- 172.21.15.50:0/890619864 learned_addr learned my addr 172.21.15.50:0/890619864 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:19.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4cd2800640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cb8005320 0x7f4cd4140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:19.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4ccbe00640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4cd41f7190 0x7f4cd41f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:19.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4cd3200640 1 -- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4cd41f7190 msgr2=0x7f4cd41f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4cd3200640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4cd41f7190 0x7f4cd41f75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4cd3200640 1 -- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cb8005320 msgr2=0x7f4cd4140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:19.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4cd3200640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cb8005320 0x7f4cd4140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:19.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4cd3200640 1 -- 172.21.15.50:0/890619864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd415ea50 con 0x7f4cd415e100 2024-09-21T06:04:19.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4cd3200640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd415e100 0x7f4cd415e510 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc01daa0 tx=0x7f4ccc006620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:19.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4ccaa00640 1 -- 172.21.15.50:0/890619864 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc006d30 con 0x7f4cd415e100 2024-09-21T06:04:19.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4ccaa00640 1 -- 172.21.15.50:0/890619864 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ccc004e40 con 0x7f4cd415e100 2024-09-21T06:04:19.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4cd9400640 1 -- 172.21.15.50:0/890619864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd4144060 con 0x7f4cd415e100 2024-09-21T06:04:19.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4ccaa00640 1 -- 172.21.15.50:0/890619864 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc006710 con 0x7f4cd415e100 2024-09-21T06:04:19.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4cd9400640 1 -- 172.21.15.50:0/890619864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd4143580 con 0x7f4cd415e100 2024-09-21T06:04:19.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4cd9400640 1 -- 172.21.15.50:0/890619864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd41f7ca0 con 0x7f4cd415e100 2024-09-21T06:04:19.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.931+0000 7f4ccaa00640 1 -- 172.21.15.50:0/890619864 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f4ccc0c0060 con 0x7f4cd415e100 2024-09-21T06:04:19.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.935+0000 7f4ccaa00640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4cb0073e60 0x7f4cb0076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:19.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.935+0000 7f4ccaa00640 1 -- 172.21.15.50:0/890619864 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5793+0+0 (secure 0 0 0) 0x7f4ccc12d400 con 0x7f4cd415e100 2024-09-21T06:04:19.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.935+0000 7f4cd2800640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4cb0073e60 0x7f4cb0076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:19.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.935+0000 7f4cd2800640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4cb0073e60 0x7f4cb0076370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4000c10 tx=0x7f4cc4006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:19.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:19.939+0000 7f4ccaa00640 1 -- 172.21.15.50:0/890619864 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4cd415ea50 con 0x7f4cd415e100 2024-09-21T06:04:20.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.115+0000 7f4cd9400640 1 -- 172.21.15.50:0/890619864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cd40700a0 con 0x7f4cd415e100 2024-09-21T06:04:20.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.115+0000 7f4ccaa00640 1 -- 172.21.15.50:0/890619864 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16829 (secure 0 0 0) 0x7f4ccc016fb0 con 0x7f4cd415e100 2024-09-21T06:04:20.117 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:20.117 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":60,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:04:16.626967+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:02:08.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T06:02:08.630388+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:04:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.119+0000 7f4cd9400640 1 -- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4cb0073e60 msgr2=0x7f4cb0076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.119+0000 7f4cd9400640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4cb0073e60 0x7f4cb0076370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4000c10 tx=0x7f4cc4006b20 comp rx=0 tx=0).stop 2024-09-21T06:04:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.119+0000 7f4cd9400640 1 -- 172.21.15.50:0/890619864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd415e100 msgr2=0x7f4cd415e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.119+0000 7f4cd9400640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd415e100 0x7f4cd415e510 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc01daa0 tx=0x7f4ccc006620 comp rx=0 tx=0).stop 2024-09-21T06:04:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.119+0000 7f4cd9400640 1 -- 172.21.15.50:0/890619864 shutdown_connections 2024-09-21T06:04:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.119+0000 7f4cd9400640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4cb0073e60 0x7f4cb0076370 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.119+0000 7f4cd9400640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd415e100 0x7f4cd415e510 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.119+0000 7f4cd9400640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4cd41f7190 0x7f4cd41f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.119+0000 7f4cd9400640 1 --2- 172.21.15.50:0/890619864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cb8005320 0x7f4cd4140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.119+0000 7f4cd9400640 1 -- 172.21.15.50:0/890619864 >> 172.21.15.50:0/890619864 conn(0x7f4cd40b35f0 msgr2=0x7f4cd4157c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.119+0000 7f4cd9400640 1 -- 172.21.15.50:0/890619864 shutdown_connections 2024-09-21T06:04:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.119+0000 7f4cd9400640 1 -- 172.21.15.50:0/890619864 wait complete. 2024-09-21T06:04:20.141 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:04:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.299+0000 7fac09e00640 1 -- 172.21.15.50:0/788810139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac04070a30 msgr2=0x7fac04160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.299+0000 7fac09e00640 1 --2- 172.21.15.50:0/788810139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac04070a30 0x7fac04160270 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fac04074700 tx=0x7fabf401c8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.299+0000 7fac09e00640 1 -- 172.21.15.50:0/788810139 shutdown_connections 2024-09-21T06:04:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.299+0000 7fac09e00640 1 --2- 172.21.15.50:0/788810139 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac0415cc90 0x7fac0415d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.299+0000 7fac09e00640 1 --2- 172.21.15.50:0/788810139 >> v2:172.21.15.88:3300/0 conn(0x7fac041607b0 0x7fac04160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.299+0000 7fac09e00640 1 --2- 172.21.15.50:0/788810139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac04070a30 0x7fac04160270 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.299+0000 7fac09e00640 1 -- 172.21.15.50:0/788810139 >> 172.21.15.50:0/788810139 conn(0x7fac040d06e0 msgr2=0x7fac040d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:20.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.299+0000 7fac09e00640 1 -- 172.21.15.50:0/788810139 shutdown_connections 2024-09-21T06:04:20.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.299+0000 7fac09e00640 1 -- 172.21.15.50:0/788810139 wait complete. 2024-09-21T06:04:20.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.299+0000 7fac09e00640 1 Processor -- start 2024-09-21T06:04:20.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.299+0000 7fac09e00640 1 -- start start 2024-09-21T06:04:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac09e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac04070a30 0x7fac04165cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac09e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac0415cc90 0x7fac040aa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac09e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac041607b0 0x7fac040aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac09e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fac040cf5c0 con 0x7fac0415cc90 2024-09-21T06:04:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac09e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fac040cf440 con 0x7fac04070a30 2024-09-21T06:04:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac09e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fac04169450 con 0x7fac041607b0 2024-09-21T06:04:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fabfa000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac0415cc90 0x7fac040aa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fabfa000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac0415cc90 0x7fac040aa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58214/0 (socket says 172.21.15.50:58214) 2024-09-21T06:04:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fabfa000640 1 -- 172.21.15.50:0/1543167368 learned_addr learned my addr 172.21.15.50:0/1543167368 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac03400640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac041607b0 0x7fac040aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac02a00640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac04070a30 0x7fac04165cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fabfa000640 1 -- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac041607b0 msgr2=0x7fac040aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fabfa000640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac041607b0 0x7fac040aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fabfa000640 1 -- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac04070a30 msgr2=0x7fac04165cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fabfa000640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac04070a30 0x7fac04165cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fabfa000640 1 -- 172.21.15.50:0/1543167368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac040ab470 con 0x7fac0415cc90 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac03400640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac041607b0 0x7fac040aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac02a00640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac04070a30 0x7fac04165cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fabfa000640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac0415cc90 0x7fac040aa6a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fabec0058d0 tx=0x7fabec00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fabfbe00640 1 -- 172.21.15.50:0/1543167368 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec017070 con 0x7fac0415cc90 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fabfbe00640 1 -- 172.21.15.50:0/1543167368 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabec009d20 con 0x7fac0415cc90 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac09e00640 1 -- 172.21.15.50:0/1543167368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac040ac110 con 0x7fac0415cc90 2024-09-21T06:04:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fabfbe00640 1 -- 172.21.15.50:0/1543167368 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec0106c0 con 0x7fac0415cc90 2024-09-21T06:04:20.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac09e00640 1 -- 172.21.15.50:0/1543167368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac040ac6d0 con 0x7fac0415cc90 2024-09-21T06:04:20.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.303+0000 7fac09e00640 1 -- 172.21.15.50:0/1543167368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac040ab1e0 con 0x7fac0415cc90 2024-09-21T06:04:20.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.307+0000 7fabfbe00640 1 -- 172.21.15.50:0/1543167368 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7fabec01a070 con 0x7fac0415cc90 2024-09-21T06:04:20.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.307+0000 7fabfbe00640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabe0073e60 0x7fabe0076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:20.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.307+0000 7fabfbe00640 1 -- 172.21.15.50:0/1543167368 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5793+0+0 (secure 0 0 0) 0x7fabec0934f0 con 0x7fac0415cc90 2024-09-21T06:04:20.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.307+0000 7fac02a00640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabe0073e60 0x7fabe0076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:20.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.307+0000 7fac02a00640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabe0073e60 0x7fabe0076370 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fabf40af000 tx=0x7fabf401b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:20.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.311+0000 7fabfbe00640 1 -- 172.21.15.50:0/1543167368 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fabec05fce0 con 0x7fac0415cc90 2024-09-21T06:04:20.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.519+0000 7fac09e00640 1 -- 172.21.15.50:0/1543167368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fac0415d0a0 con 0x7fac0415cc90 2024-09-21T06:04:20.523 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-09-21T06:04:20.523+0000 7f0d45000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:04:20.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.523+0000 7fabfbe00640 1 -- 172.21.15.50:0/1543167368 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7fabec05f5e0 con 0x7fac0415cc90 2024-09-21T06:04:20.525 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:04:20.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.527+0000 7fac09e00640 1 -- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabe0073e60 msgr2=0x7fabe0076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:20.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.527+0000 7fac09e00640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabe0073e60 0x7fabe0076370 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fabf40af000 tx=0x7fabf401b100 comp rx=0 tx=0).stop 2024-09-21T06:04:20.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.527+0000 7fac09e00640 1 -- 172.21.15.50:0/1543167368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac0415cc90 msgr2=0x7fac040aa6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:20.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.527+0000 7fac09e00640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac0415cc90 0x7fac040aa6a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fabec0058d0 tx=0x7fabec00dde0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.527+0000 7fac09e00640 1 -- 172.21.15.50:0/1543167368 shutdown_connections 2024-09-21T06:04:20.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.527+0000 7fac09e00640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac041607b0 0x7fac040aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.527+0000 7fac09e00640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabe0073e60 0x7fabe0076370 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.527+0000 7fac09e00640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac0415cc90 0x7fac040aa6a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.527+0000 7fac09e00640 1 --2- 172.21.15.50:0/1543167368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac04070a30 0x7fac04165cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.527+0000 7fac09e00640 1 -- 172.21.15.50:0/1543167368 >> 172.21.15.50:0/1543167368 conn(0x7fac040d06e0 msgr2=0x7fac041445a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:20.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.527+0000 7fac09e00640 1 -- 172.21.15.50:0/1543167368 shutdown_connections 2024-09-21T06:04:20.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.527+0000 7fac09e00640 1 -- 172.21.15.50:0/1543167368 wait complete. 2024-09-21T06:04:20.557 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:04:20.681 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T06:04:20.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.723+0000 7f96b5600640 1 -- 172.21.15.50:0/603233397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b00d1250 msgr2=0x7f96b0068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:20.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.723+0000 7f96b5600640 1 --2- 172.21.15.50:0/603233397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b00d1250 0x7f96b0068df0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f96b00b0860 tx=0x7f969801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.723+0000 7f96b5600640 1 -- 172.21.15.50:0/603233397 shutdown_connections 2024-09-21T06:04:20.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.723+0000 7f96b5600640 1 --2- 172.21.15.50:0/603233397 >> v2:172.21.15.88:3300/0 conn(0x7f96b0069330 0x7f96b015dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.723+0000 7f96b5600640 1 --2- 172.21.15.50:0/603233397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b00d1250 0x7f96b0068df0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.723+0000 7f96b5600640 1 -- 172.21.15.50:0/603233397 >> 172.21.15.50:0/603233397 conn(0x7f96b00b35f0 msgr2=0x7f96b00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:20.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.723+0000 7f96b5600640 1 -- 172.21.15.50:0/603233397 shutdown_connections 2024-09-21T06:04:20.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.723+0000 7f96b5600640 1 -- 172.21.15.50:0/603233397 wait complete. 2024-09-21T06:04:20.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96b5600640 1 Processor -- start 2024-09-21T06:04:20.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96b5600640 1 -- start start 2024-09-21T06:04:20.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96b5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b0069330 0x7f96b013f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:20.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96b5600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f96b00d0e40 0x7f96b0151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:20.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96b5600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96b0070950 0x7f96b0070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96b5600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96b016a110 con 0x7f96b0069330 2024-09-21T06:04:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96b5600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0169f90 con 0x7f96b0070950 2024-09-21T06:04:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96b5600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f96b016a290 con 0x7f96b00d0e40 2024-09-21T06:04:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96aea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b0069330 0x7f96b013f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96aea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b0069330 0x7f96b013f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58240/0 (socket says 172.21.15.50:58240) 2024-09-21T06:04:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96aea00640 1 -- 172.21.15.50:0/3694777583 learned_addr learned my addr 172.21.15.50:0/3694777583 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96a7e00640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f96b00d0e40 0x7f96b0151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96af400640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96b0070950 0x7f96b0070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96aea00640 1 -- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f96b00d0e40 msgr2=0x7f96b0151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96aea00640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f96b00d0e40 0x7f96b0151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96aea00640 1 -- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96b0070950 msgr2=0x7f96b0070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96aea00640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96b0070950 0x7f96b0070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:20.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96aea00640 1 -- 172.21.15.50:0/3694777583 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b00712a0 con 0x7f96b0069330 2024-09-21T06:04:20.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96aea00640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b0069330 0x7f96b013f090 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f96a000c3c0 tx=0x7f96a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:20.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96a6a00640 1 -- 172.21.15.50:0/3694777583 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0017070 con 0x7f96b0069330 2024-09-21T06:04:20.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96a6a00640 1 -- 172.21.15.50:0/3694777583 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96a00053c0 con 0x7f96b0069330 2024-09-21T06:04:20.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96b5600640 1 -- 172.21.15.50:0/3694777583 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b0153300 con 0x7f96b0069330 2024-09-21T06:04:20.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.727+0000 7f96a6a00640 1 -- 172.21.15.50:0/3694777583 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0003860 con 0x7f96b0069330 2024-09-21T06:04:20.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.731+0000 7f96b5600640 1 -- 172.21.15.50:0/3694777583 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b00b4950 con 0x7f96b0069330 2024-09-21T06:04:20.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.731+0000 7f96b5600640 1 -- 172.21.15.50:0/3694777583 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b00d1250 con 0x7f96b0069330 2024-09-21T06:04:20.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.731+0000 7f96a6a00640 1 -- 172.21.15.50:0/3694777583 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f96a001a070 con 0x7f96b0069330 2024-09-21T06:04:20.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.731+0000 7f96a6a00640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f968c073e60 0x7f968c076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:20.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.731+0000 7f96a6a00640 1 -- 172.21.15.50:0/3694777583 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5851+0+0 (secure 0 0 0) 0x7f96a0092150 con 0x7f96b0069330 2024-09-21T06:04:20.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.731+0000 7f96a7e00640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f968c073e60 0x7f968c076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:20.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.735+0000 7f96a7e00640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f968c073e60 0x7f968c076370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f96980af000 tx=0x7f969801b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:20.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.735+0000 7f96a6a00640 1 -- 172.21.15.50:0/3694777583 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f96b00712a0 con 0x7f96b0069330 2024-09-21T06:04:20.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:20.951+0000 7f96b5600640 1 -- 172.21.15.50:0/3694777583 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f96b00b1070 con 0x7f96b0069330 2024-09-21T06:04:21.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.663+0000 7f96a6a00640 1 -- 172.21.15.50:0/3694777583 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f96b0069740 con 0x7f96b0069330 2024-09-21T06:04:21.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.667+0000 7f96b5600640 1 -- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f968c073e60 msgr2=0x7f968c076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.667+0000 7f96b5600640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f968c073e60 0x7f968c076370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f96980af000 tx=0x7f969801b100 comp rx=0 tx=0).stop 2024-09-21T06:04:21.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.667+0000 7f96b5600640 1 -- 172.21.15.50:0/3694777583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b0069330 msgr2=0x7f96b013f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.667+0000 7f96b5600640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b0069330 0x7f96b013f090 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f96a000c3c0 tx=0x7f96a000c880 comp rx=0 tx=0).stop 2024-09-21T06:04:21.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.667+0000 7f96b5600640 1 -- 172.21.15.50:0/3694777583 shutdown_connections 2024-09-21T06:04:21.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.667+0000 7f96b5600640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96b0070950 0x7f96b0070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.667+0000 7f96b5600640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f96b00d0e40 0x7f96b0151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.667+0000 7f96b5600640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f968c073e60 0x7f968c076370 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.667+0000 7f96b5600640 1 --2- 172.21.15.50:0/3694777583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b0069330 0x7f96b013f090 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.667+0000 7f96b5600640 1 -- 172.21.15.50:0/3694777583 >> 172.21.15.50:0/3694777583 conn(0x7f96b00b35f0 msgr2=0x7f96b0075d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:21.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.667+0000 7f96b5600640 1 -- 172.21.15.50:0/3694777583 shutdown_connections 2024-09-21T06:04:21.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.667+0000 7f96b5600640 1 -- 172.21.15.50:0/3694777583 wait complete. 2024-09-21T06:04:21.693 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:04:21.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.859+0000 7fbe00800640 1 -- 172.21.15.50:0/2684550033 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdf8074300 msgr2=0x7fbdf815f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.859+0000 7fbe00800640 1 --2- 172.21.15.50:0/2684550033 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdf8074300 0x7fbdf815f170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbde8009840 tx=0x7fbde801c790 comp rx=0 tx=0).stop 2024-09-21T06:04:21.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.859+0000 7fbe00800640 1 -- 172.21.15.50:0/2684550033 shutdown_connections 2024-09-21T06:04:21.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.859+0000 7fbe00800640 1 --2- 172.21.15.50:0/2684550033 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdf8074300 0x7fbdf815f170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.859+0000 7fbe00800640 1 --2- 172.21.15.50:0/2684550033 >> v2:172.21.15.88:3300/0 conn(0x7fbdf815c5d0 0x7fbdf8073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.859+0000 7fbe00800640 1 -- 172.21.15.50:0/2684550033 >> 172.21.15.50:0/2684550033 conn(0x7fbdf80b2c40 msgr2=0x7fbdf80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:21.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.859+0000 7fbe00800640 1 -- 172.21.15.50:0/2684550033 shutdown_connections 2024-09-21T06:04:21.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.859+0000 7fbe00800640 1 -- 172.21.15.50:0/2684550033 wait complete. 2024-09-21T06:04:21.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbe00800640 1 Processor -- start 2024-09-21T06:04:21.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbe00800640 1 -- start start 2024-09-21T06:04:21.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbe00800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdf8074300 0x7fbdf8157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbe00800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf815c5d0 0x7fbdf814f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbe00800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdf815f6b0 0x7fbdf80b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbe00800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf806f980 con 0x7fbdf815f6b0 2024-09-21T06:04:21.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbe00800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf80b3d50 con 0x7fbdf815c5d0 2024-09-21T06:04:21.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbe00800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf806fb00 con 0x7fbdf8074300 2024-09-21T06:04:21.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdfe800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdf815f6b0 0x7fbdf80b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdfe800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdf815f6b0 0x7fbdf80b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58246/0 (socket says 172.21.15.50:58246) 2024-09-21T06:04:21.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdfe800640 1 -- 172.21.15.50:0/2887108860 learned_addr learned my addr 172.21.15.50:0/2887108860 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:21.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdfde00640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdf8074300 0x7fbdf8157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdfd400640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf815c5d0 0x7fbdf814f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdfe800640 1 -- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdf8074300 msgr2=0x7fbdf8157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdfe800640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdf8074300 0x7fbdf8157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdfe800640 1 -- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf815c5d0 msgr2=0x7fbdf814f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdfe800640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf815c5d0 0x7fbdf814f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdfe800640 1 -- 172.21.15.50:0/2887108860 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf8150060 con 0x7fbdf815f6b0 2024-09-21T06:04:21.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdfe800640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdf815f6b0 0x7fbdf80b0ee0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400cb10 tx=0x7fbdf400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:21.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdf2000640 1 -- 172.21.15.50:0/2887108860 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4016020 con 0x7fbdf815f6b0 2024-09-21T06:04:21.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdf2000640 1 -- 172.21.15.50:0/2887108860 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdf400ce50 con 0x7fbdf815f6b0 2024-09-21T06:04:21.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbe00800640 1 -- 172.21.15.50:0/2887108860 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf81fa050 con 0x7fbdf815f6b0 2024-09-21T06:04:21.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbdf2000640 1 -- 172.21.15.50:0/2887108860 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400ae00 con 0x7fbdf815f6b0 2024-09-21T06:04:21.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.863+0000 7fbe00800640 1 -- 172.21.15.50:0/2887108860 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf81fa610 con 0x7fbdf815f6b0 2024-09-21T06:04:21.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.867+0000 7fbe00800640 1 -- 172.21.15.50:0/2887108860 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf815f170 con 0x7fbdf815f6b0 2024-09-21T06:04:21.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.867+0000 7fbdf2000640 1 -- 172.21.15.50:0/2887108860 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7fbdf4019020 con 0x7fbdf815f6b0 2024-09-21T06:04:21.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.867+0000 7fbdf2000640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbddc073e90 0x7fbddc0763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.867+0000 7fbdf2000640 1 -- 172.21.15.50:0/2887108860 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5851+0+0 (secure 0 0 0) 0x7fbdf4092f70 con 0x7fbdf815f6b0 2024-09-21T06:04:21.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.867+0000 7fbdfde00640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbddc073e90 0x7fbddc0763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.871+0000 7fbdfde00640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbddc073e90 0x7fbddc0763a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009b40 tx=0x7fbdec0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:21.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:21.871+0000 7fbdf2000640 1 -- 172.21.15.50:0/2887108860 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbdf409b020 con 0x7fbdf815f6b0 2024-09-21T06:04:22.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.051+0000 7fbe00800640 1 -- 172.21.15.50:0/2887108860 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbdf80701d0 con 0x7fbdf815f6b0 2024-09-21T06:04:22.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.667+0000 7fbdf2000640 1 -- 172.21.15.50:0/2887108860 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v62) ==== 181+0+0 (secure 0 0 0) 0x7fbdf4020020 con 0x7fbdf815f6b0 2024-09-21T06:04:22.669 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:04:22.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.671+0000 7fbe00800640 1 -- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbddc073e90 msgr2=0x7fbddc0763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:22.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.671+0000 7fbe00800640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbddc073e90 0x7fbddc0763a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009b40 tx=0x7fbdec0091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:22.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.671+0000 7fbe00800640 1 -- 172.21.15.50:0/2887108860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdf815f6b0 msgr2=0x7fbdf80b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:22.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.671+0000 7fbe00800640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdf815f6b0 0x7fbdf80b0ee0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400cb10 tx=0x7fbdf400cc00 comp rx=0 tx=0).stop 2024-09-21T06:04:22.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.671+0000 7fbe00800640 1 -- 172.21.15.50:0/2887108860 shutdown_connections 2024-09-21T06:04:22.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.671+0000 7fbe00800640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbddc073e90 0x7fbddc0763a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:22.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.671+0000 7fbe00800640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdf815f6b0 0x7fbdf80b0ee0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:22.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.671+0000 7fbe00800640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf815c5d0 0x7fbdf814f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:22.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.671+0000 7fbe00800640 1 --2- 172.21.15.50:0/2887108860 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdf8074300 0x7fbdf8157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:22.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.671+0000 7fbe00800640 1 -- 172.21.15.50:0/2887108860 >> 172.21.15.50:0/2887108860 conn(0x7fbdf80b2c40 msgr2=0x7fbdf814d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:22.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.671+0000 7fbe00800640 1 -- 172.21.15.50:0/2887108860 shutdown_connections 2024-09-21T06:04:22.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.671+0000 7fbe00800640 1 -- 172.21.15.50:0/2887108860 wait complete. 2024-09-21T06:04:22.698 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:04:22.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.863+0000 7ff84d000640 1 -- 172.21.15.50:0/462928628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8480b4de0 msgr2=0x7ff8480b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:22.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.863+0000 7ff84d000640 1 --2- 172.21.15.50:0/462928628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8480b4de0 0x7ff8480b51f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff840009c80 tx=0x7ff84001cce0 comp rx=0 tx=0).stop 2024-09-21T06:04:22.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.863+0000 7ff84d000640 1 -- 172.21.15.50:0/462928628 shutdown_connections 2024-09-21T06:04:22.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.863+0000 7ff84d000640 1 --2- 172.21.15.50:0/462928628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8480b4de0 0x7ff8480b51f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:22.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.863+0000 7ff84d000640 1 --2- 172.21.15.50:0/462928628 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8480b5c70 0x7ff84815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:22.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.863+0000 7ff84d000640 1 -- 172.21.15.50:0/462928628 >> 172.21.15.50:0/462928628 conn(0x7ff8480b2040 msgr2=0x7ff8480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:22.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.863+0000 7ff84d000640 1 -- 172.21.15.50:0/462928628 shutdown_connections 2024-09-21T06:04:22.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.863+0000 7ff84d000640 1 -- 172.21.15.50:0/462928628 wait complete. 2024-09-21T06:04:22.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff84d000640 1 Processor -- start 2024-09-21T06:04:22.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff84d000640 1 -- start start 2024-09-21T06:04:22.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff84d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8480b4de0 0x7ff848157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff84d000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8480b5c70 0x7ff848144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff84d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8481527b0 0x7ff8481fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff84d000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff848168a10 con 0x7ff8480b4de0 2024-09-21T06:04:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff84d000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff848168890 con 0x7ff8481527b0 2024-09-21T06:04:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff84d000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff848168b90 con 0x7ff8480b5c70 2024-09-21T06:04:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff845e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8480b4de0 0x7ff848157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff845e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8480b4de0 0x7ff848157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58254/0 (socket says 172.21.15.50:58254) 2024-09-21T06:04:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff845e00640 1 -- 172.21.15.50:0/1673243769 learned_addr learned my addr 172.21.15.50:0/1673243769 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff83fe00640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8480b5c70 0x7ff848144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff846800640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8481527b0 0x7ff8481fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff845e00640 1 -- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8480b5c70 msgr2=0x7ff848144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff845e00640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8480b5c70 0x7ff848144b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:22.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff845e00640 1 -- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8481527b0 msgr2=0x7ff8481fbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:22.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff845e00640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8481527b0 0x7ff8481fbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:22.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff845e00640 1 -- 172.21.15.50:0/1673243769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff848157b10 con 0x7ff8480b4de0 2024-09-21T06:04:22.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff846800640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8481527b0 0x7ff8481fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:22.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff845e00640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8480b4de0 0x7ff848157210 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff83800d720 tx=0x7ff83800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:22.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff83e000640 1 -- 172.21.15.50:0/1673243769 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838017070 con 0x7ff8480b4de0 2024-09-21T06:04:22.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff84d000640 1 -- 172.21.15.50:0/1673243769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff848157df0 con 0x7ff8480b4de0 2024-09-21T06:04:22.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff83e000640 1 -- 172.21.15.50:0/1673243769 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff838009d20 con 0x7ff8480b4de0 2024-09-21T06:04:22.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff83e000640 1 -- 172.21.15.50:0/1673243769 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838005280 con 0x7ff8480b4de0 2024-09-21T06:04:22.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.867+0000 7ff84d000640 1 -- 172.21.15.50:0/1673243769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8481583b0 con 0x7ff8480b4de0 2024-09-21T06:04:22.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.871+0000 7ff84d000640 1 -- 172.21.15.50:0/1673243769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8480b51f0 con 0x7ff8480b4de0 2024-09-21T06:04:22.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.871+0000 7ff83e000640 1 -- 172.21.15.50:0/1673243769 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7ff838005420 con 0x7ff8480b4de0 2024-09-21T06:04:22.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.871+0000 7ff83e000640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff824073e60 0x7ff824076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:22.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.871+0000 7ff83e000640 1 -- 172.21.15.50:0/1673243769 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5413+0+0 (secure 0 0 0) 0x7ff838093030 con 0x7ff8480b4de0 2024-09-21T06:04:22.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.871+0000 7ff83fe00640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff824073e60 0x7ff824076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:22.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.875+0000 7ff83fe00640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff824073e60 0x7ff824076370 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff848145870 tx=0x7ff830009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:22.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:22.875+0000 7ff83e000640 1 -- 172.21.15.50:0/1673243769 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff83805f820 con 0x7ff8480b4de0 2024-09-21T06:04:23.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.055+0000 7ff84d000640 1 -- 172.21.15.50:0/1673243769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff84815bc00 con 0x7ff8480b4de0 2024-09-21T06:04:23.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.395+0000 7ff83e000640 1 -- 172.21.15.50:0/1673243769 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v63) ==== 169+0+0 (secure 0 0 0) 0x7ff83805f120 con 0x7ff8480b4de0 2024-09-21T06:04:23.395 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-09-21T06:04:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.399+0000 7ff84d000640 1 -- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff824073e60 msgr2=0x7ff824076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:23.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.399+0000 7ff84d000640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff824073e60 0x7ff824076370 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff848145870 tx=0x7ff830009040 comp rx=0 tx=0).stop 2024-09-21T06:04:23.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.399+0000 7ff84d000640 1 -- 172.21.15.50:0/1673243769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8480b4de0 msgr2=0x7ff848157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:23.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.399+0000 7ff84d000640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8480b4de0 0x7ff848157210 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff83800d720 tx=0x7ff83800dbe0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.399+0000 7ff84d000640 1 -- 172.21.15.50:0/1673243769 shutdown_connections 2024-09-21T06:04:23.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.399+0000 7ff84d000640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8481527b0 0x7ff8481fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.399+0000 7ff84d000640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8480b5c70 0x7ff848144b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.399+0000 7ff84d000640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff824073e60 0x7ff824076370 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.399+0000 7ff84d000640 1 --2- 172.21.15.50:0/1673243769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8480b4de0 0x7ff848157210 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.399+0000 7ff84d000640 1 -- 172.21.15.50:0/1673243769 >> 172.21.15.50:0/1673243769 conn(0x7ff8480b2040 msgr2=0x7ff848151c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:23.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.399+0000 7ff84d000640 1 -- 172.21.15.50:0/1673243769 shutdown_connections 2024-09-21T06:04:23.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.399+0000 7ff84d000640 1 -- 172.21.15.50:0/1673243769 wait complete. 2024-09-21T06:04:23.427 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:04:23.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.599+0000 7f579ba00640 1 -- 172.21.15.50:0/4159222788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57940b5c70 msgr2=0x7f579415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:23.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.599+0000 7f579ba00640 1 --2- 172.21.15.50:0/4159222788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57940b5c70 0x7f579415bb60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5788009840 tx=0x7f578801c7f0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.599+0000 7f579ba00640 1 -- 172.21.15.50:0/4159222788 shutdown_connections 2024-09-21T06:04:23.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.599+0000 7f579ba00640 1 --2- 172.21.15.50:0/4159222788 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57940b4de0 0x7f57940b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.599+0000 7f579ba00640 1 --2- 172.21.15.50:0/4159222788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57940b5c70 0x7f579415bb60 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.599+0000 7f579ba00640 1 -- 172.21.15.50:0/4159222788 >> 172.21.15.50:0/4159222788 conn(0x7f57940b2040 msgr2=0x7f57940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:23.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.599+0000 7f579ba00640 1 -- 172.21.15.50:0/4159222788 shutdown_connections 2024-09-21T06:04:23.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.599+0000 7f579ba00640 1 -- 172.21.15.50:0/4159222788 wait complete. 2024-09-21T06:04:23.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f579ba00640 1 Processor -- start 2024-09-21T06:04:23.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f579ba00640 1 -- start start 2024-09-21T06:04:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f579ba00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57940b4de0 0x7f57940c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f579ba00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57940b5c70 0x7f5794146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f579ba00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57941527b0 0x7f57940c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f579ba00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57940b4410 con 0x7f57940b4de0 2024-09-21T06:04:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f579ba00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57940b4290 con 0x7f57941527b0 2024-09-21T06:04:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f579ba00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f57940b4590 con 0x7f57940b5c70 2024-09-21T06:04:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f5799000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57940b4de0 0x7f57940c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f5799000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57940b4de0 0x7f57940c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58278/0 (socket says 172.21.15.50:58278) 2024-09-21T06:04:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f5799000640 1 -- 172.21.15.50:0/606368795 learned_addr learned my addr 172.21.15.50:0/606368795 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f5793e00640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57940b5c70 0x7f5794146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f5799a00640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57941527b0 0x7f57940c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f5799000640 1 -- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57940b5c70 msgr2=0x7f5794146cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f5799000640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57940b5c70 0x7f5794146cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f5799000640 1 -- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57941527b0 msgr2=0x7f57940c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f5799000640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57941527b0 0x7f57940c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f5799000640 1 -- 172.21.15.50:0/606368795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57941fcd90 con 0x7f57940b4de0 2024-09-21T06:04:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.603+0000 7f5799a00640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57941527b0 0x7f57940c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.607+0000 7f5799000640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57940b4de0 0x7f57940c52b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5788000c10 tx=0x7f57880a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.607+0000 7f5790c00640 1 -- 172.21.15.50:0/606368795 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57880b2040 con 0x7f57940b4de0 2024-09-21T06:04:23.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.607+0000 7f5790c00640 1 -- 172.21.15.50:0/606368795 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57880043b0 con 0x7f57940b4de0 2024-09-21T06:04:23.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.607+0000 7f5790c00640 1 -- 172.21.15.50:0/606368795 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788003800 con 0x7f57940b4de0 2024-09-21T06:04:23.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.607+0000 7f579ba00640 1 -- 172.21.15.50:0/606368795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5794147210 con 0x7f57940b4de0 2024-09-21T06:04:23.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.607+0000 7f579ba00640 1 -- 172.21.15.50:0/606368795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57941fc4f0 con 0x7f57940b4de0 2024-09-21T06:04:23.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.607+0000 7f579ba00640 1 -- 172.21.15.50:0/606368795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57940b4b30 con 0x7f57940b4de0 2024-09-21T06:04:23.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.607+0000 7f5790c00640 1 -- 172.21.15.50:0/606368795 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f57880bd060 con 0x7f57940b4de0 2024-09-21T06:04:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.607+0000 7f5790c00640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5774073e60 0x7f5774076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.607+0000 7f5790c00640 1 -- 172.21.15.50:0/606368795 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4999+0+0 (secure 0 0 0) 0x7f578812c5d0 con 0x7f57940b4de0 2024-09-21T06:04:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.611+0000 7f5793e00640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5774073e60 0x7f5774076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.611+0000 7f5793e00640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5774073e60 0x7f5774076370 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5794155ef0 tx=0x7f5780009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:23.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.615+0000 7f5790c00640 1 -- 172.21.15.50:0/606368795 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5788016410 con 0x7f57940b4de0 2024-09-21T06:04:23.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.795+0000 7f579ba00640 1 -- 172.21.15.50:0/606368795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f5794154820 con 0x7f57940b4de0 2024-09-21T06:04:23.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f5790c00640 1 -- 172.21.15.50:0/606368795 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v40) ==== 71+0+0 (secure 0 0 0) 0x7f57880f90d0 con 0x7f57940b4de0 2024-09-21T06:04:23.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f579ba00640 1 -- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5774073e60 msgr2=0x7f5774076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:23.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f579ba00640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5774073e60 0x7f5774076370 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5794155ef0 tx=0x7f5780009040 comp rx=0 tx=0).stop 2024-09-21T06:04:23.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f579ba00640 1 -- 172.21.15.50:0/606368795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57940b4de0 msgr2=0x7f57940c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:23.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f579ba00640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57940b4de0 0x7f57940c52b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5788000c10 tx=0x7f57880a56b0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f579ba00640 1 -- 172.21.15.50:0/606368795 shutdown_connections 2024-09-21T06:04:23.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f579ba00640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57941527b0 0x7f57940c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f579ba00640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57940b5c70 0x7f5794146cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f579ba00640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5774073e60 0x7f5774076370 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f579ba00640 1 --2- 172.21.15.50:0/606368795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57940b4de0 0x7f57940c52b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f579ba00640 1 -- 172.21.15.50:0/606368795 >> 172.21.15.50:0/606368795 conn(0x7f57940b2040 msgr2=0x7f57941537b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:23.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f579ba00640 1 -- 172.21.15.50:0/606368795 shutdown_connections 2024-09-21T06:04:23.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.799+0000 7f579ba00640 1 -- 172.21.15.50:0/606368795 wait complete. 2024-09-21T06:04:23.822 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-21T06:04:23.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.979+0000 7fef3c800640 1 -- 172.21.15.50:0/233388297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3415e8c0 msgr2=0x7fef3415ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:23.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.979+0000 7fef3c800640 1 --2- 172.21.15.50:0/233388297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3415e8c0 0x7fef3415ecd0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fef28009900 tx=0x7fef2801c8e0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.979+0000 7fef3c800640 1 -- 172.21.15.50:0/233388297 shutdown_connections 2024-09-21T06:04:23.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.979+0000 7fef3c800640 1 --2- 172.21.15.50:0/233388297 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef3415e290 0x7fef3415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.979+0000 7fef3c800640 1 --2- 172.21.15.50:0/233388297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3415e8c0 0x7fef3415ecd0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.979+0000 7fef3c800640 1 -- 172.21.15.50:0/233388297 >> 172.21.15.50:0/233388297 conn(0x7fef340b2040 msgr2=0x7fef340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:23.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.979+0000 7fef3c800640 1 -- 172.21.15.50:0/233388297 shutdown_connections 2024-09-21T06:04:23.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.979+0000 7fef3c800640 1 -- 172.21.15.50:0/233388297 wait complete. 2024-09-21T06:04:23.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef3c800640 1 Processor -- start 2024-09-21T06:04:23.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef3c800640 1 -- start start 2024-09-21T06:04:23.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef3c800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3415b2d0 0x7fef34164510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:23.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef3c800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef3415e290 0x7fef340cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:23.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef3c800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef3415e8c0 0x7fef34161bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:23.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef3c800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef34168230 con 0x7fef3415b2d0 2024-09-21T06:04:23.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef3c800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fef341680b0 con 0x7fef3415e290 2024-09-21T06:04:23.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef3c800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fef341683b0 con 0x7fef3415e8c0 2024-09-21T06:04:23.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef39e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3415b2d0 0x7fef34164510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:23.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef39e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3415b2d0 0x7fef34164510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58294/0 (socket says 172.21.15.50:58294) 2024-09-21T06:04:23.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef39e00640 1 -- 172.21.15.50:0/4100294657 learned_addr learned my addr 172.21.15.50:0/4100294657 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:23.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef39400640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef3415e290 0x7fef340cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:23.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef3a800640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef3415e8c0 0x7fef34161bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:23.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef39e00640 1 -- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef3415e8c0 msgr2=0x7fef34161bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:23.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef39e00640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef3415e8c0 0x7fef34161bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef39e00640 1 -- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef3415e290 msgr2=0x7fef340cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:23.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef39e00640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef3415e290 0x7fef340cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:23.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef39e00640 1 -- 172.21.15.50:0/4100294657 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef34164cd0 con 0x7fef3415b2d0 2024-09-21T06:04:23.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef39e00640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3415b2d0 0x7fef34164510 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fef280b0040 tx=0x7fef28005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:23.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef2e000640 1 -- 172.21.15.50:0/4100294657 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef280b2040 con 0x7fef3415b2d0 2024-09-21T06:04:23.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef2e000640 1 -- 172.21.15.50:0/4100294657 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef28004320 con 0x7fef3415b2d0 2024-09-21T06:04:23.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.983+0000 7fef3c800640 1 -- 172.21.15.50:0/4100294657 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef34164fb0 con 0x7fef3415b2d0 2024-09-21T06:04:23.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.987+0000 7fef2e000640 1 -- 172.21.15.50:0/4100294657 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28005290 con 0x7fef3415b2d0 2024-09-21T06:04:23.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.987+0000 7fef3c800640 1 -- 172.21.15.50:0/4100294657 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef34165570 con 0x7fef3415b2d0 2024-09-21T06:04:23.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.987+0000 7fef3c800640 1 -- 172.21.15.50:0/4100294657 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef3415ecd0 con 0x7fef3415b2d0 2024-09-21T06:04:23.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.987+0000 7fef2e000640 1 -- 172.21.15.50:0/4100294657 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7fef28005430 con 0x7fef3415b2d0 2024-09-21T06:04:23.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.987+0000 7fef2e000640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef18073e60 0x7fef18076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:23.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.987+0000 7fef2e000640 1 -- 172.21.15.50:0/4100294657 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4999+0+0 (secure 0 0 0) 0x7fef2812c8f0 con 0x7fef3415b2d0 2024-09-21T06:04:23.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.987+0000 7fef39400640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef18073e60 0x7fef18076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:23.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.991+0000 7fef39400640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef18073e60 0x7fef18076370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fef341582a0 tx=0x7fef24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:23.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:23.991+0000 7fef2e000640 1 -- 172.21.15.50:0/4100294657 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fef280bc060 con 0x7fef3415b2d0 2024-09-21T06:04:24.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.231+0000 7fef3c800640 1 -- 172.21.15.50:0/4100294657 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7fef34155730 con 0x7fef3415b2d0 2024-09-21T06:04:24.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.687+0000 7fef2e000640 1 -- 172.21.15.50:0/4100294657 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v343) ==== 167+0+0 (secure 0 0 0) 0x7fef280164d0 con 0x7fef3415b2d0 2024-09-21T06:04:24.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.687+0000 7fef3c800640 1 -- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef18073e60 msgr2=0x7fef18076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:24.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.687+0000 7fef3c800640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef18073e60 0x7fef18076370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fef341582a0 tx=0x7fef24009040 comp rx=0 tx=0).stop 2024-09-21T06:04:24.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.687+0000 7fef3c800640 1 -- 172.21.15.50:0/4100294657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3415b2d0 msgr2=0x7fef34164510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:24.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.687+0000 7fef3c800640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3415b2d0 0x7fef34164510 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fef280b0040 tx=0x7fef28005760 comp rx=0 tx=0).stop 2024-09-21T06:04:24.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.691+0000 7fef3c800640 1 -- 172.21.15.50:0/4100294657 shutdown_connections 2024-09-21T06:04:24.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.691+0000 7fef3c800640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef3415e8c0 0x7fef34161bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:24.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.691+0000 7fef3c800640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef3415e290 0x7fef340cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:24.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.691+0000 7fef3c800640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef18073e60 0x7fef18076370 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:24.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.691+0000 7fef3c800640 1 --2- 172.21.15.50:0/4100294657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3415b2d0 0x7fef34164510 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:24.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.691+0000 7fef3c800640 1 -- 172.21.15.50:0/4100294657 >> 172.21.15.50:0/4100294657 conn(0x7fef340b2040 msgr2=0x7fef34155020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:24.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.691+0000 7fef3c800640 1 -- 172.21.15.50:0/4100294657 shutdown_connections 2024-09-21T06:04:24.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.691+0000 7fef3c800640 1 -- 172.21.15.50:0/4100294657 wait complete. 2024-09-21T06:04:24.713 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-21T06:04:24.714 INFO:tasks.cephfs_test_runner:Starting test: test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that 2 progress bars are printed in output of "ceph status" 2024-09-21T06:04:24.714 INFO:tasks.cephfs_test_runner:test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-21T06:04:24.714 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:04:24.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.875+0000 7fb48b600640 1 -- 172.21.15.50:0/78070899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4840b3340 msgr2=0x7fb4840b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:24.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.875+0000 7fb48b600640 1 --2- 172.21.15.50:0/78070899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4840b3340 0x7fb4840b3750 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009c80 tx=0x7fb47c01cda0 comp rx=0 tx=0).stop 2024-09-21T06:04:24.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.875+0000 7fb48b600640 1 -- 172.21.15.50:0/78070899 shutdown_connections 2024-09-21T06:04:24.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.875+0000 7fb48b600640 1 --2- 172.21.15.50:0/78070899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4840b3340 0x7fb4840b3750 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:24.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.875+0000 7fb48b600640 1 --2- 172.21.15.50:0/78070899 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4840749a0 0x7fb484074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:24.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.875+0000 7fb48b600640 1 -- 172.21.15.50:0/78070899 >> 172.21.15.50:0/78070899 conn(0x7fb4840b2040 msgr2=0x7fb4840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:24.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.875+0000 7fb48b600640 1 -- 172.21.15.50:0/78070899 shutdown_connections 2024-09-21T06:04:24.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.875+0000 7fb48b600640 1 -- 172.21.15.50:0/78070899 wait complete. 2024-09-21T06:04:24.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.875+0000 7fb48b600640 1 Processor -- start 2024-09-21T06:04:24.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.875+0000 7fb48b600640 1 -- start start 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb48b600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4840749a0 0x7fb48413fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb48b600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4840b3340 0x7fb4841f95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb48b600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4840cefa0 0x7fb4841606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb48b600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb484169dc0 con 0x7fb4840749a0 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb48b600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb484169c40 con 0x7fb4840cefa0 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb48b600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb484169f40 con 0x7fb4840b3340 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb488c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4840749a0 0x7fb48413fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb488c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4840749a0 0x7fb48413fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58310/0 (socket says 172.21.15.50:58310) 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb488c00640 1 -- 172.21.15.50:0/605032592 learned_addr learned my addr 172.21.15.50:0/605032592 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb489600640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4840cefa0 0x7fb4841606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb483e00640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4840b3340 0x7fb4841f95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb488c00640 1 -- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4840b3340 msgr2=0x7fb4841f95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:24.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb488c00640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4840b3340 0x7fb4841f95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:24.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb488c00640 1 -- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4840cefa0 msgr2=0x7fb4841606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:24.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb488c00640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4840cefa0 0x7fb4841606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:24.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb488c00640 1 -- 172.21.15.50:0/605032592 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb484145bd0 con 0x7fb4840749a0 2024-09-21T06:04:24.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb489600640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4840cefa0 0x7fb4841606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:24.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb488c00640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4840749a0 0x7fb48413fa50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb47800d750 tx=0x7fb47800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:24.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb480c00640 1 -- 172.21.15.50:0/605032592 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478017070 con 0x7fb4840749a0 2024-09-21T06:04:24.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb480c00640 1 -- 172.21.15.50:0/605032592 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb478009d20 con 0x7fb4840749a0 2024-09-21T06:04:24.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb48b600640 1 -- 172.21.15.50:0/605032592 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb484145eb0 con 0x7fb4840749a0 2024-09-21T06:04:24.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb480c00640 1 -- 172.21.15.50:0/605032592 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47800de60 con 0x7fb4840749a0 2024-09-21T06:04:24.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb48b600640 1 -- 172.21.15.50:0/605032592 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48413cd80 con 0x7fb4840749a0 2024-09-21T06:04:24.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.879+0000 7fb48b600640 1 -- 172.21.15.50:0/605032592 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb484074db0 con 0x7fb4840749a0 2024-09-21T06:04:24.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.883+0000 7fb480c00640 1 -- 172.21.15.50:0/605032592 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7fb478004280 con 0x7fb4840749a0 2024-09-21T06:04:24.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.883+0000 7fb480c00640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb464073e90 0x7fb4640763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:24.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.883+0000 7fb480c00640 1 -- 172.21.15.50:0/605032592 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4999+0+0 (secure 0 0 0) 0x7fb4780932d0 con 0x7fb4840749a0 2024-09-21T06:04:24.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.883+0000 7fb483e00640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb464073e90 0x7fb4640763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:24.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.883+0000 7fb483e00640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb464073e90 0x7fb4640763a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb48413ca00 tx=0x7fb470009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:24.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:24.887+0000 7fb480c00640 1 -- 172.21.15.50:0/605032592 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb478020020 con 0x7fb4840749a0 2024-09-21T06:04:25.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.067+0000 7fb48b600640 1 -- 172.21.15.50:0/605032592 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4840afc10 con 0x7fb4840749a0 2024-09-21T06:04:25.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.067+0000 7fb480c00640 1 -- 172.21.15.50:0/605032592 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+13068 (secure 0 0 0) 0x7fb47805fad0 con 0x7fb4840749a0 2024-09-21T06:04:25.069 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:25.069 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":63,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:04:23.396231+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6835/451843324":"2024-09-22T06:04:20.525664+0000","172.21.15.50:6833/451843324":"2024-09-22T06:04:20.525664+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:04:25.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.071+0000 7fb48b600640 1 -- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb464073e90 msgr2=0x7fb4640763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:25.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.071+0000 7fb48b600640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb464073e90 0x7fb4640763a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb48413ca00 tx=0x7fb470009040 comp rx=0 tx=0).stop 2024-09-21T06:04:25.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.071+0000 7fb48b600640 1 -- 172.21.15.50:0/605032592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4840749a0 msgr2=0x7fb48413fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:25.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.071+0000 7fb48b600640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4840749a0 0x7fb48413fa50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb47800d750 tx=0x7fb47800dc10 comp rx=0 tx=0).stop 2024-09-21T06:04:25.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.071+0000 7fb48b600640 1 -- 172.21.15.50:0/605032592 shutdown_connections 2024-09-21T06:04:25.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.071+0000 7fb48b600640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4840cefa0 0x7fb4841606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.071+0000 7fb48b600640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4840b3340 0x7fb4841f95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.071+0000 7fb48b600640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb464073e90 0x7fb4640763a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.071+0000 7fb48b600640 1 --2- 172.21.15.50:0/605032592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4840749a0 0x7fb48413fa50 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.071+0000 7fb48b600640 1 -- 172.21.15.50:0/605032592 >> 172.21.15.50:0/605032592 conn(0x7fb4840b2040 msgr2=0x7fb4840cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:25.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.071+0000 7fb48b600640 1 -- 172.21.15.50:0/605032592 shutdown_connections 2024-09-21T06:04:25.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.071+0000 7fb48b600640 1 -- 172.21.15.50:0/605032592 wait complete. 2024-09-21T06:04:25.092 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:56:18.921197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:04:25.093 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:04:25.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.255+0000 7f33f6600640 1 -- 172.21.15.50:0/1591087855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33d4004da0 msgr2=0x7f33d4005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:25.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.255+0000 7f33f6600640 1 --2- 172.21.15.50:0/1591087855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33d4004da0 0x7f33d4005230 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009ec0 tx=0x7f33e801d5f0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.255+0000 7f33efe00640 1 -- 172.21.15.50:0/1591087855 reap_dead start 2024-09-21T06:04:25.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.255+0000 7f33f6600640 1 -- 172.21.15.50:0/1591087855 shutdown_connections 2024-09-21T06:04:25.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.255+0000 7f33f6600640 1 -- 172.21.15.50:0/1591087855 >> 172.21.15.50:0/1591087855 conn(0x7f33f00b35f0 msgr2=0x7f33f00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:25.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.255+0000 7f33f6600640 1 -- 172.21.15.50:0/1591087855 shutdown_connections 2024-09-21T06:04:25.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.255+0000 7f33f6600640 1 -- 172.21.15.50:0/1591087855 wait complete. 2024-09-21T06:04:25.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33f6600640 1 Processor -- start 2024-09-21T06:04:25.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33f6600640 1 -- start start 2024-09-21T06:04:25.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33f6600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33d4004da0 0x7f33f014d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:25.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33f6600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f00afeb0 0x7f33f01f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:25.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33f6600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f0150660 0x7f33f01f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:25.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33f6600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f33f015d420 con 0x7f33f0150660 2024-09-21T06:04:25.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33f6600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f33f015d2a0 con 0x7f33d4004da0 2024-09-21T06:04:25.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33f6600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f33f015d5a0 con 0x7f33f00afeb0 2024-09-21T06:04:25.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f0150660 0x7f33f01f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:25.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f0150660 0x7f33f01f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58316/0 (socket says 172.21.15.50:58316) 2024-09-21T06:04:25.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33efe00640 1 -- 172.21.15.50:0/2257141351 learned_addr learned my addr 172.21.15.50:0/2257141351 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:25.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33d4004da0 0x7f33f014d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:25.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33e7e00640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f00afeb0 0x7f33f01f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33efe00640 1 -- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f00afeb0 msgr2=0x7f33f01f9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33efe00640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f00afeb0 0x7f33f01f9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33efe00640 1 -- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33d4004da0 msgr2=0x7f33f014d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33efe00640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33d4004da0 0x7f33f014d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33efe00640 1 -- 172.21.15.50:0/2257141351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f014ca30 con 0x7f33f0150660 2024-09-21T06:04:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33efe00640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f0150660 0x7f33f01f5cc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f33e801d060 tx=0x7f33e8004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33e7400640 1 -- 172.21.15.50:0/2257141351 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8009a90 con 0x7f33f0150660 2024-09-21T06:04:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33e7400640 1 -- 172.21.15.50:0/2257141351 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33e80a94f0 con 0x7f33f0150660 2024-09-21T06:04:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33e7400640 1 -- 172.21.15.50:0/2257141351 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e80a9cb0 con 0x7f33f0150660 2024-09-21T06:04:25.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33f6600640 1 -- 172.21.15.50:0/2257141351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f00c68e0 con 0x7f33f0150660 2024-09-21T06:04:25.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.259+0000 7f33f6600640 1 -- 172.21.15.50:0/2257141351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f014cf80 con 0x7f33f0150660 2024-09-21T06:04:25.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.263+0000 7f33f6600640 1 -- 172.21.15.50:0/2257141351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f015c620 con 0x7f33f0150660 2024-09-21T06:04:25.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.263+0000 7f33e7400640 1 -- 172.21.15.50:0/2257141351 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f33e80b9030 con 0x7f33f0150660 2024-09-21T06:04:25.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.263+0000 7f33e7400640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33cc073e60 0x7f33cc076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:25.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.263+0000 7f33e7400640 1 -- 172.21.15.50:0/2257141351 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4999+0+0 (secure 0 0 0) 0x7f33e812daf0 con 0x7f33f0150660 2024-09-21T06:04:25.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.263+0000 7f33ef400640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33cc073e60 0x7f33cc076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:25.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.263+0000 7f33ef400640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33cc073e60 0x7f33cc076370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f33e0006fd0 tx=0x7f33e0006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:25.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.267+0000 7f33e7400640 1 -- 172.21.15.50:0/2257141351 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f33e80ad070 con 0x7f33f0150660 2024-09-21T06:04:25.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.443+0000 7f33f6600640 1 -- 172.21.15.50:0/2257141351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f33f0070560 con 0x7f33f0150660 2024-09-21T06:04:25.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.443+0000 7f33e7400640 1 -- 172.21.15.50:0/2257141351 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7f33e8017300 con 0x7f33f0150660 2024-09-21T06:04:25.444 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-09-21T06:04:25.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.447+0000 7f33f6600640 1 -- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33cc073e60 msgr2=0x7f33cc076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:25.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.447+0000 7f33f6600640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33cc073e60 0x7f33cc076370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f33e0006fd0 tx=0x7f33e0006b20 comp rx=0 tx=0).stop 2024-09-21T06:04:25.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.447+0000 7f33f6600640 1 -- 172.21.15.50:0/2257141351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f0150660 msgr2=0x7f33f01f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:25.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.447+0000 7f33f6600640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f0150660 0x7f33f01f5cc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f33e801d060 tx=0x7f33e8004f50 comp rx=0 tx=0).stop 2024-09-21T06:04:25.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.447+0000 7f33f6600640 1 -- 172.21.15.50:0/2257141351 shutdown_connections 2024-09-21T06:04:25.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.447+0000 7f33f6600640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33cc073e60 0x7f33cc076370 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.447+0000 7f33f6600640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f0150660 0x7f33f01f5cc0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.447+0000 7f33f6600640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f00afeb0 0x7f33f01f9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.447+0000 7f33f6600640 1 --2- 172.21.15.50:0/2257141351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33d4004da0 0x7f33f014d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.447+0000 7f33f6600640 1 -- 172.21.15.50:0/2257141351 >> 172.21.15.50:0/2257141351 conn(0x7f33f00b35f0 msgr2=0x7f33f006f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:25.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.447+0000 7f33f6600640 1 -- 172.21.15.50:0/2257141351 shutdown_connections 2024-09-21T06:04:25.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.447+0000 7f33f6600640 1 -- 172.21.15.50:0/2257141351 wait complete. 2024-09-21T06:04:25.467 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-21T06:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.623+0000 7f699e200640 1 -- 172.21.15.50:0/1736220338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69980b5c70 msgr2=0x7f699815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.623+0000 7f699e200640 1 --2- 172.21.15.50:0/1736220338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69980b5c70 0x7f699815bb60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6998070710 tx=0x7f698801c800 comp rx=0 tx=0).stop 2024-09-21T06:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 -- 172.21.15.50:0/1736220338 shutdown_connections 2024-09-21T06:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 --2- 172.21.15.50:0/1736220338 >> v2:172.21.15.88:3300/0 conn(0x7f69981527b0 0x7f6998074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 --2- 172.21.15.50:0/1736220338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69980b5c70 0x7f699815bb60 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 -- 172.21.15.50:0/1736220338 >> 172.21.15.50:0/1736220338 conn(0x7f69980b2040 msgr2=0x7f69980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 -- 172.21.15.50:0/1736220338 shutdown_connections 2024-09-21T06:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 -- 172.21.15.50:0/1736220338 wait complete. 2024-09-21T06:04:25.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 Processor -- start 2024-09-21T06:04:25.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 -- start start 2024-09-21T06:04:25.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69980b4de0 0x7f6998149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:25.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69980b5c70 0x7f69980ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69981527b0 0x7f69980aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6998168b20 con 0x7f69980b4de0 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f69981689a0 con 0x7f69981527b0 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.627+0000 7f699e200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6998168ca0 con 0x7f69980b5c70 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f6997400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69980b4de0 0x7f6998149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f6997400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69980b4de0 0x7f6998149840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58328/0 (socket says 172.21.15.50:58328) 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f6997400640 1 -- 172.21.15.50:0/1303140843 learned_addr learned my addr 172.21.15.50:0/1303140843 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f6997e00640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69981527b0 0x7f69980aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f698fe00640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69980b5c70 0x7f69980ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f6997400640 1 -- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69980b5c70 msgr2=0x7f69980ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f6997400640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69980b5c70 0x7f69980ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f6997400640 1 -- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69981527b0 msgr2=0x7f69980aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f6997400640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69981527b0 0x7f69980aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:25.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f6997400640 1 -- 172.21.15.50:0/1303140843 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6998152f90 con 0x7f69980b4de0 2024-09-21T06:04:25.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f6997e00640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69981527b0 0x7f69980aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:25.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f698fe00640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69980b5c70 0x7f69980ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:25.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f6997400640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69980b4de0 0x7f6998149840 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f698801cce0 tx=0x7f69880a5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:25.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f698f400640 1 -- 172.21.15.50:0/1303140843 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880bc070 con 0x7f69980b4de0 2024-09-21T06:04:25.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f698f400640 1 -- 172.21.15.50:0/1303140843 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69880044b0 con 0x7f69980b4de0 2024-09-21T06:04:25.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f699e200640 1 -- 172.21.15.50:0/1303140843 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699814dd90 con 0x7f69980b4de0 2024-09-21T06:04:25.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f698f400640 1 -- 172.21.15.50:0/1303140843 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880b2040 con 0x7f69980b4de0 2024-09-21T06:04:25.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f699e200640 1 -- 172.21.15.50:0/1303140843 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699814e330 con 0x7f69980b4de0 2024-09-21T06:04:25.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f699e200640 1 -- 172.21.15.50:0/1303140843 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69980ccd10 con 0x7f69980b4de0 2024-09-21T06:04:25.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.631+0000 7f698f400640 1 -- 172.21.15.50:0/1303140843 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f6988005290 con 0x7f69980b4de0 2024-09-21T06:04:25.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.635+0000 7f698f400640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6974073e60 0x7f6974076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:25.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.635+0000 7f698f400640 1 -- 172.21.15.50:0/1303140843 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4999+0+0 (secure 0 0 0) 0x7f698812c8f0 con 0x7f69980b4de0 2024-09-21T06:04:25.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.635+0000 7f698fe00640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6974073e60 0x7f6974076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:25.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.635+0000 7f698fe00640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6974073e60 0x7f6974076370 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6980005fd0 tx=0x7f6980005dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:25.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.639+0000 7f698f400640 1 -- 172.21.15.50:0/1303140843 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f69980ccd10 con 0x7f69980b4de0 2024-09-21T06:04:25.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:25.879+0000 7f699e200640 1 -- 172.21.15.50:0/1303140843 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7f699815bb60 con 0x7f69980b4de0 2024-09-21T06:04:26.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.695+0000 7f698f400640 1 -- 172.21.15.50:0/1303140843 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v345) ==== 170+0+0 (secure 0 0 0) 0x7f6998075580 con 0x7f69980b4de0 2024-09-21T06:04:26.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.699+0000 7f699e200640 1 -- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6974073e60 msgr2=0x7f6974076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:26.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.699+0000 7f699e200640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6974073e60 0x7f6974076370 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6980005fd0 tx=0x7f6980005dc0 comp rx=0 tx=0).stop 2024-09-21T06:04:26.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.699+0000 7f699e200640 1 -- 172.21.15.50:0/1303140843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69980b4de0 msgr2=0x7f6998149840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:26.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.699+0000 7f699e200640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69980b4de0 0x7f6998149840 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f698801cce0 tx=0x7f69880a5840 comp rx=0 tx=0).stop 2024-09-21T06:04:26.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.699+0000 7f699e200640 1 -- 172.21.15.50:0/1303140843 shutdown_connections 2024-09-21T06:04:26.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.699+0000 7f699e200640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69981527b0 0x7f69980aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:26.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.699+0000 7f699e200640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69980b5c70 0x7f69980ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:26.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.699+0000 7f699e200640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6974073e60 0x7f6974076370 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:26.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.699+0000 7f699e200640 1 --2- 172.21.15.50:0/1303140843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69980b4de0 0x7f6998149840 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:26.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.699+0000 7f699e200640 1 -- 172.21.15.50:0/1303140843 >> 172.21.15.50:0/1303140843 conn(0x7f69980b2040 msgr2=0x7f69981509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:26.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.699+0000 7f699e200640 1 -- 172.21.15.50:0/1303140843 shutdown_connections 2024-09-21T06:04:26.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.699+0000 7f699e200640 1 -- 172.21.15.50:0/1303140843 wait complete. 2024-09-21T06:04:26.724 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:04:26.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.887+0000 7f5834200640 1 -- 172.21.15.50:0/1529708679 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f582c0b54f0 msgr2=0x7f582c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:26.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.887+0000 7f5834200640 1 --2- 172.21.15.50:0/1529708679 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f582c0b54f0 0x7f582c15bb60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5824009bc0 tx=0x7f582401cae0 comp rx=0 tx=0).stop 2024-09-21T06:04:26.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.887+0000 7f5834200640 1 -- 172.21.15.50:0/1529708679 shutdown_connections 2024-09-21T06:04:26.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.887+0000 7f5834200640 1 --2- 172.21.15.50:0/1529708679 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f582c0b54f0 0x7f582c15bb60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:26.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.887+0000 7f5834200640 1 -- 172.21.15.50:0/1529708679 >> 172.21.15.50:0/1529708679 conn(0x7f582c0b2040 msgr2=0x7f582c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:26.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.887+0000 7f5834200640 1 -- 172.21.15.50:0/1529708679 shutdown_connections 2024-09-21T06:04:26.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.887+0000 7f5834200640 1 -- 172.21.15.50:0/1529708679 wait complete. 2024-09-21T06:04:26.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5834200640 1 Processor -- start 2024-09-21T06:04:26.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5834200640 1 -- start start 2024-09-21T06:04:26.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5834200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f582c0b5c70 0x7f582c159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:26.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5834200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f582c1527b0 0x7f582c142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5834200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f582c151700 0x7f582c151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5834200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f582c169bd0 con 0x7f582c0b5c70 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5834200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f582c169a50 con 0x7f582c1527b0 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5834200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f582c169d50 con 0x7f582c151700 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5831800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f582c0b5c70 0x7f582c159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5831800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f582c0b5c70 0x7f582c159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40322/0 (socket says 172.21.15.50:40322) 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5831800640 1 -- 172.21.15.50:0/3756213076 learned_addr learned my addr 172.21.15.50:0/3756213076 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5832200640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f582c151700 0x7f582c151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5830e00640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f582c1527b0 0x7f582c142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5831800640 1 -- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f582c151700 msgr2=0x7f582c151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5831800640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f582c151700 0x7f582c151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5831800640 1 -- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f582c1527b0 msgr2=0x7f582c142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:26.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5831800640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f582c1527b0 0x7f582c142db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:26.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5831800640 1 -- 172.21.15.50:0/3756213076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582c1432f0 con 0x7f582c0b5c70 2024-09-21T06:04:26.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5831800640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f582c0b5c70 0x7f582c159000 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f582000c450 tx=0x7f582000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:26.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5829600640 1 -- 172.21.15.50:0/3756213076 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820016020 con 0x7f582c0b5c70 2024-09-21T06:04:26.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5829600640 1 -- 172.21.15.50:0/3756213076 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5820012920 con 0x7f582c0b5c70 2024-09-21T06:04:26.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.891+0000 7f5834200640 1 -- 172.21.15.50:0/3756213076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582c152050 con 0x7f582c0b5c70 2024-09-21T06:04:26.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.895+0000 7f5829600640 1 -- 172.21.15.50:0/3756213076 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820012d50 con 0x7f582c0b5c70 2024-09-21T06:04:26.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.895+0000 7f5834200640 1 -- 172.21.15.50:0/3756213076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f582c1fc070 con 0x7f582c0b5c70 2024-09-21T06:04:26.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.895+0000 7f5834200640 1 -- 172.21.15.50:0/3756213076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f582c15bb60 con 0x7f582c0b5c70 2024-09-21T06:04:26.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.895+0000 7f5829600640 1 -- 172.21.15.50:0/3756213076 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f5820019020 con 0x7f582c0b5c70 2024-09-21T06:04:26.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.895+0000 7f5829600640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f580c073e60 0x7f580c076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:26.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.895+0000 7f5829600640 1 -- 172.21.15.50:0/3756213076 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4999+0+0 (secure 0 0 0) 0x7f5820092b80 con 0x7f582c0b5c70 2024-09-21T06:04:26.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.895+0000 7f5830e00640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f580c073e60 0x7f580c076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:26.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.899+0000 7f5830e00640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f580c073e60 0x7f580c076370 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f582c166b30 tx=0x7f5818009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:26.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:26.899+0000 7f5829600640 1 -- 172.21.15.50:0/3756213076 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5820020030 con 0x7f582c0b5c70 2024-09-21T06:04:27.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.079+0000 7f5834200640 1 -- 172.21.15.50:0/3756213076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f582c0b25f0 con 0x7f582c0b5c70 2024-09-21T06:04:27.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.079+0000 7f5829600640 1 -- 172.21.15.50:0/3756213076 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v40) ==== 59+0+2294 (secure 0 0 0) 0x7f582005f370 con 0x7f582c0b5c70 2024-09-21T06:04:27.081 INFO:teuthology.orchestra.run.smithi050.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:04:27.081 INFO:teuthology.orchestra.run.smithi050.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:04:27.081 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:04:27.081 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_client 20/20 2024-09-21T06:04:27.081 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:04:27.081 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:04:27.081 INFO:teuthology.orchestra.run.smithi050.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:04:27.081 INFO:teuthology.orchestra.run.smithi050.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3735.149299 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4471.400731 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3640.131623 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3663.645087 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3762.055672 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3661.558528 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3661.456695 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3796.187852 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds 20/20 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_ms 2/2 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_frag true 2024-09-21T06:04:27.082 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:04:27.083 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:04:27.083 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_verify_scatter true 2024-09-21T06:04:27.083 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:04:27.083 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:04:27.083 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:04:27.083 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced client_mount_timeout 600 2024-09-21T06:04:27.083 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_client 20/20 2024-09-21T06:04:27.083 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_ms 1/1 2024-09-21T06:04:27.083 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:04:27.083 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:04:27.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.083+0000 7f5834200640 1 -- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f580c073e60 msgr2=0x7f580c076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:27.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.083+0000 7f5834200640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f580c073e60 0x7f580c076370 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f582c166b30 tx=0x7f5818009040 comp rx=0 tx=0).stop 2024-09-21T06:04:27.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.083+0000 7f5834200640 1 -- 172.21.15.50:0/3756213076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f582c0b5c70 msgr2=0x7f582c159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:27.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.083+0000 7f5834200640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f582c0b5c70 0x7f582c159000 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f582000c450 tx=0x7f582000c910 comp rx=0 tx=0).stop 2024-09-21T06:04:27.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.083+0000 7f5834200640 1 -- 172.21.15.50:0/3756213076 shutdown_connections 2024-09-21T06:04:27.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.083+0000 7f5834200640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f582c151700 0x7f582c151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.083+0000 7f5834200640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f582c1527b0 0x7f582c142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.083+0000 7f5834200640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f580c073e60 0x7f580c076370 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.083+0000 7f5834200640 1 --2- 172.21.15.50:0/3756213076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f582c0b5c70 0x7f582c159000 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.083+0000 7f5834200640 1 -- 172.21.15.50:0/3756213076 >> 172.21.15.50:0/3756213076 conn(0x7f582c0b2040 msgr2=0x7f582c15a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:27.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.083+0000 7f5834200640 1 -- 172.21.15.50:0/3756213076 shutdown_connections 2024-09-21T06:04:27.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.083+0000 7f5834200640 1 -- 172.21.15.50:0/3756213076 wait complete. 2024-09-21T06:04:27.103 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:04:27.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.263+0000 7f33fac00640 1 -- 172.21.15.50:0/3949979949 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f40b3130 msgr2=0x7f33f40b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:27.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.263+0000 7f33fac00640 1 --2- 172.21.15.50:0/3949979949 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f40b3130 0x7f33f40b3540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009c20 tx=0x7f33ec01cb00 comp rx=0 tx=0).stop 2024-09-21T06:04:27.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.263+0000 7f33fac00640 1 -- 172.21.15.50:0/3949979949 shutdown_connections 2024-09-21T06:04:27.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.263+0000 7f33fac00640 1 --2- 172.21.15.50:0/3949979949 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f40b3130 0x7f33f40b3540 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.263+0000 7f33fac00640 1 -- 172.21.15.50:0/3949979949 >> 172.21.15.50:0/3949979949 conn(0x7f33f406cdf0 msgr2=0x7f33f406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:27.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.263+0000 7f33fac00640 1 -- 172.21.15.50:0/3949979949 shutdown_connections 2024-09-21T06:04:27.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.263+0000 7f33fac00640 1 -- 172.21.15.50:0/3949979949 wait complete. 2024-09-21T06:04:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33fac00640 1 Processor -- start 2024-09-21T06:04:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33fac00640 1 -- start start 2024-09-21T06:04:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33fac00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33f40b1dc0 0x7f33f41379c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33fac00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f40b3130 0x7f33f40ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33fac00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f4155a00 0x7f33f40ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33fac00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f33f416a020 con 0x7f33f40b3130 2024-09-21T06:04:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33fac00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f33f4169ea0 con 0x7f33f40b1dc0 2024-09-21T06:04:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33fac00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f33f416a1a0 con 0x7f33f4155a00 2024-09-21T06:04:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33f8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f40b3130 0x7f33f40ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33f8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f40b3130 0x7f33f40ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40334/0 (socket says 172.21.15.50:40334) 2024-09-21T06:04:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33f8e00640 1 -- 172.21.15.50:0/140848409 learned_addr learned my addr 172.21.15.50:0/140848409 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33f9800640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33f40b1dc0 0x7f33f41379c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33fa200640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f4155a00 0x7f33f40ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33f8e00640 1 -- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f4155a00 msgr2=0x7f33f40ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33f8e00640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f4155a00 0x7f33f40ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33f8e00640 1 -- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33f40b1dc0 msgr2=0x7f33f41379c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33f8e00640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33f40b1dc0 0x7f33f41379c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33f8e00640 1 -- 172.21.15.50:0/140848409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f41383c0 con 0x7f33f40b3130 2024-09-21T06:04:27.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.267+0000 7f33f8e00640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f40b3130 0x7f33f40ce620 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f33e000c410 tx=0x7f33e000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:27.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.271+0000 7f33f1600640 1 -- 172.21.15.50:0/140848409 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0011790 con 0x7f33f40b3130 2024-09-21T06:04:27.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.271+0000 7f33f1600640 1 -- 172.21.15.50:0/140848409 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33e0017070 con 0x7f33f40b3130 2024-09-21T06:04:27.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.271+0000 7f33fac00640 1 -- 172.21.15.50:0/140848409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f415cc50 con 0x7f33f40b3130 2024-09-21T06:04:27.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.271+0000 7f33f1600640 1 -- 172.21.15.50:0/140848409 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0011d40 con 0x7f33f40b3130 2024-09-21T06:04:27.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.271+0000 7f33fac00640 1 -- 172.21.15.50:0/140848409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f415d210 con 0x7f33f40b3130 2024-09-21T06:04:27.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.271+0000 7f33fac00640 1 -- 172.21.15.50:0/140848409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f40cef00 con 0x7f33f40b3130 2024-09-21T06:04:27.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.271+0000 7f33f1600640 1 -- 172.21.15.50:0/140848409 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f33e001c460 con 0x7f33f40b3130 2024-09-21T06:04:27.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.271+0000 7f33f1600640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33d4073e60 0x7f33d4076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:27.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.271+0000 7f33f1600640 1 -- 172.21.15.50:0/140848409 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4999+0+0 (secure 0 0 0) 0x7f33e0092970 con 0x7f33f40b3130 2024-09-21T06:04:27.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.275+0000 7f33f9800640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33d4073e60 0x7f33d4076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:27.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.275+0000 7f33f9800640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33d4073e60 0x7f33d4076370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f33e8000c10 tx=0x7f33e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:27.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.279+0000 7f33f1600640 1 -- 172.21.15.50:0/140848409 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f33e005ed50 con 0x7f33f40b3130 2024-09-21T06:04:27.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.455+0000 7f33fac00640 1 -- 172.21.15.50:0/140848409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f33f40b3540 con 0x7f33f40b3130 2024-09-21T06:04:27.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.459+0000 7f33f1600640 1 -- 172.21.15.50:0/140848409 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7f33e0063780 con 0x7f33f40b3130 2024-09-21T06:04:27.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.463+0000 7f33fac00640 1 -- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33d4073e60 msgr2=0x7f33d4076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:27.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.463+0000 7f33fac00640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33d4073e60 0x7f33d4076370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f33e8000c10 tx=0x7f33e8009040 comp rx=0 tx=0).stop 2024-09-21T06:04:27.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.463+0000 7f33fac00640 1 -- 172.21.15.50:0/140848409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f40b3130 msgr2=0x7f33f40ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:27.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.463+0000 7f33fac00640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f40b3130 0x7f33f40ce620 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f33e000c410 tx=0x7f33e000c8d0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.463+0000 7f33fac00640 1 -- 172.21.15.50:0/140848409 shutdown_connections 2024-09-21T06:04:27.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.463+0000 7f33fac00640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f33f4155a00 0x7f33f40ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.463+0000 7f33fac00640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33d4073e60 0x7f33d4076370 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.463+0000 7f33fac00640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f40b3130 0x7f33f40ce620 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.463+0000 7f33fac00640 1 --2- 172.21.15.50:0/140848409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33f40b1dc0 0x7f33f41379c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.463+0000 7f33fac00640 1 -- 172.21.15.50:0/140848409 >> 172.21.15.50:0/140848409 conn(0x7f33f406cdf0 msgr2=0x7f33f40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:27.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.463+0000 7f33fac00640 1 -- 172.21.15.50:0/140848409 shutdown_connections 2024-09-21T06:04:27.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.463+0000 7f33fac00640 1 -- 172.21.15.50:0/140848409 wait complete. 2024-09-21T06:04:27.484 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:04:27.484 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:04:27.561 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:04:27.646 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:04:27.646 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:04:27.646 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:04:27.655 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:04:27.655 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:04:27.655 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:04:27.664 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:04:27.664 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:04:27.664 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:04:27.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:04:27.707 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:04:27.707 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:04:27.716 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:04:27.716 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:04:27.716 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:04:27.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.911+0000 7fbfe0600640 1 -- 172.21.15.50:0/2910809479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfc0004d50 msgr2=0x7fbfc00051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:27.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.911+0000 7fbfe0600640 1 --2- 172.21.15.50:0/2910809479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfc0004d50 0x7fbfc00051a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000c640 tx=0x7fbfd001d240 comp rx=0 tx=0).stop 2024-09-21T06:04:27.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.911+0000 7fbfde600640 1 -- 172.21.15.50:0/2910809479 reap_dead start 2024-09-21T06:04:27.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.911+0000 7fbfe0600640 1 -- 172.21.15.50:0/2910809479 shutdown_connections 2024-09-21T06:04:27.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.911+0000 7fbfe0600640 1 -- 172.21.15.50:0/2910809479 >> 172.21.15.50:0/2910809479 conn(0x7fbfd80b2040 msgr2=0x7fbfd80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:27.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.911+0000 7fbfe0600640 1 -- 172.21.15.50:0/2910809479 shutdown_connections 2024-09-21T06:04:27.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.911+0000 7fbfe0600640 1 -- 172.21.15.50:0/2910809479 wait complete. 2024-09-21T06:04:27.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfe0600640 1 Processor -- start 2024-09-21T06:04:27.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfe0600640 1 -- start start 2024-09-21T06:04:27.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfe0600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbfc0004d50 0x7fbfd8141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:27.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfe0600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfd81e2380 0x7fbfd813e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:27.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfe0600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd813e7b0 0x7fbfd813ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:27.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfe0600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd815cfe0 con 0x7fbfd813e7b0 2024-09-21T06:04:27.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfe0600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd815ce60 con 0x7fbfd81e2380 2024-09-21T06:04:27.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfe0600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd815d160 con 0x7fbfc0004d50 2024-09-21T06:04:27.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfde600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd813e7b0 0x7fbfd813ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:27.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfde600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd813e7b0 0x7fbfd813ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40354/0 (socket says 172.21.15.50:40354) 2024-09-21T06:04:27.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfde600640 1 -- 172.21.15.50:0/2589286572 learned_addr learned my addr 172.21.15.50:0/2589286572 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:27.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfdd200640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfd81e2380 0x7fbfd813e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:27.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfddc00640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbfc0004d50 0x7fbfd8141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:27.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfde600640 1 -- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbfc0004d50 msgr2=0x7fbfd8141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:27.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfde600640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbfc0004d50 0x7fbfd8141960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfde600640 1 -- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfd81e2380 msgr2=0x7fbfd813e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:27.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfde600640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfd81e2380 0x7fbfd813e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:27.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfde600640 1 -- 172.21.15.50:0/2589286572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd80ab210 con 0x7fbfd813e7b0 2024-09-21T06:04:27.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.915+0000 7fbfde600640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd813e7b0 0x7fbfd813ed30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0098e10 tx=0x7fbfd0004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:27.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.919+0000 7fbfd5600640 1 -- 172.21.15.50:0/2589286572 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0003b80 con 0x7fbfd813e7b0 2024-09-21T06:04:27.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.919+0000 7fbfd5600640 1 -- 172.21.15.50:0/2589286572 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfd0003e10 con 0x7fbfd813e7b0 2024-09-21T06:04:27.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.919+0000 7fbfd5600640 1 -- 172.21.15.50:0/2589286572 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0006270 con 0x7fbfd813e7b0 2024-09-21T06:04:27.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.919+0000 7fbfe0600640 1 -- 172.21.15.50:0/2589286572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd80ab4f0 con 0x7fbfd813e7b0 2024-09-21T06:04:27.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.919+0000 7fbfe0600640 1 -- 172.21.15.50:0/2589286572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd80abab0 con 0x7fbfd813e7b0 2024-09-21T06:04:27.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.919+0000 7fbfe0600640 1 -- 172.21.15.50:0/2589286572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd8073cf0 con 0x7fbfd813e7b0 2024-09-21T06:04:27.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.919+0000 7fbfd5600640 1 -- 172.21.15.50:0/2589286572 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7fbfd0006410 con 0x7fbfd813e7b0 2024-09-21T06:04:27.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.919+0000 7fbfd5600640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbfb8073e90 0x7fbfb80763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:27.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.919+0000 7fbfd5600640 1 -- 172.21.15.50:0/2589286572 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4999+0+0 (secure 0 0 0) 0x7fbfd012d6a0 con 0x7fbfd813e7b0 2024-09-21T06:04:27.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.919+0000 7fbfddc00640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbfb8073e90 0x7fbfb80763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:27.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.923+0000 7fbfddc00640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbfb8073e90 0x7fbfb80763a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0066c0 tx=0x7fbfcc0113b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:27.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:27.923+0000 7fbfd5600640 1 -- 172.21.15.50:0/2589286572 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbfd00bf060 con 0x7fbfd813e7b0 2024-09-21T06:04:28.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.139+0000 7fbfe0600640 1 -- 172.21.15.50:0/2589286572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfd80b26a0 con 0x7fbfd813e7b0 2024-09-21T06:04:28.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.139+0000 7fbfd5600640 1 -- 172.21.15.50:0/2589286572 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3908 (secure 0 0 0) 0x7fbfd0016ff0 con 0x7fbfd813e7b0 2024-09-21T06:04:28.142 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:28.142 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":39,"btime":"2024-09-21T06:04:21:662866+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/288562749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":288562749},{"type":"v1","addr":"172.21.15.50:6834","nonce":288562749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5686,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2920049831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2920049831},{"type":"v1","addr":"172.21.15.88:6836","nonce":2920049831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1072517376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1072517376},{"type":"v1","addr":"172.21.15.88:6837","nonce":1072517376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4139227584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":4139227584},{"type":"v1","addr":"172.21.15.50:6835","nonce":4139227584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-09-21T06:04:28.142 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 39 2024-09-21T06:04:28.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.143+0000 7fbfe0600640 1 -- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbfb8073e90 msgr2=0x7fbfb80763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:28.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.143+0000 7fbfe0600640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbfb8073e90 0x7fbfb80763a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0066c0 tx=0x7fbfcc0113b0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.143+0000 7fbfe0600640 1 -- 172.21.15.50:0/2589286572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd813e7b0 msgr2=0x7fbfd813ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:28.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.143+0000 7fbfe0600640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd813e7b0 0x7fbfd813ed30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0098e10 tx=0x7fbfd0004ed0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.143+0000 7fbfe0600640 1 -- 172.21.15.50:0/2589286572 shutdown_connections 2024-09-21T06:04:28.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.143+0000 7fbfe0600640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbfb8073e90 0x7fbfb80763a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.143+0000 7fbfe0600640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd813e7b0 0x7fbfd813ed30 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.143+0000 7fbfe0600640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfd81e2380 0x7fbfd813e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.143+0000 7fbfe0600640 1 --2- 172.21.15.50:0/2589286572 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbfc0004d50 0x7fbfd8141960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.143+0000 7fbfe0600640 1 -- 172.21.15.50:0/2589286572 >> 172.21.15.50:0/2589286572 conn(0x7fbfd80b2040 msgr2=0x7fbfd8152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:28.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.143+0000 7fbfe0600640 1 -- 172.21.15.50:0/2589286572 shutdown_connections 2024-09-21T06:04:28.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.143+0000 7fbfe0600640 1 -- 172.21.15.50:0/2589286572 wait complete. 2024-09-21T06:04:28.165 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:04:28.165 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:04:28.165 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:04:28.165 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:04:28.166 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:04:28.166 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:04:28.166 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:04:28.166 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:04:28.166 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:04:28.166 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:04:28.166 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:04:28.167 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:04:28.167 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:04:28.167 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:04:28.167 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:04:28.167 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:04:28.181 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:04:28.181 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:04:28.182 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:04:28.182 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:04:28.183 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:04:28.183 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:04:28.188 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:04:28.188 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:04:28.226 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:04:28.226 DEBUG:teuthology.parallel:result is None 2024-09-21T06:04:28.228 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:04:28.228 DEBUG:teuthology.parallel:result is None 2024-09-21T06:04:28.236 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:04:28.236 DEBUG:teuthology.parallel:result is None 2024-09-21T06:04:28.238 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:04:28.238 DEBUG:teuthology.parallel:result is None 2024-09-21T06:04:28.239 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:04:28.332 INFO:tasks.ceph.mds.b.smithi088.stdout:starting mds.b at 2024-09-21T06:04:28.333 INFO:tasks.ceph.mds.d.smithi088.stdout:starting mds.d at 2024-09-21T06:04:28.336 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T06:04:28.340 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-09-21T06:04:28.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.403+0000 7f7b99e00640 1 -- 172.21.15.50:0/3601067297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b940b4de0 msgr2=0x7f7b940b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:28.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.403+0000 7f7b99e00640 1 --2- 172.21.15.50:0/3601067297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b940b4de0 0x7f7b940b51f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7b94070710 tx=0x7f7b8c01ccc0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.403+0000 7f7b99e00640 1 -- 172.21.15.50:0/3601067297 shutdown_connections 2024-09-21T06:04:28.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.403+0000 7f7b99e00640 1 --2- 172.21.15.50:0/3601067297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b940b4de0 0x7f7b940b51f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.403+0000 7f7b99e00640 1 --2- 172.21.15.50:0/3601067297 >> v2:172.21.15.88:3300/0 conn(0x7f7b941527b0 0x7f7b9415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.403+0000 7f7b99e00640 1 -- 172.21.15.50:0/3601067297 >> 172.21.15.50:0/3601067297 conn(0x7f7b940b2040 msgr2=0x7f7b940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:28.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.403+0000 7f7b99e00640 1 -- 172.21.15.50:0/3601067297 shutdown_connections 2024-09-21T06:04:28.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.403+0000 7f7b99e00640 1 -- 172.21.15.50:0/3601067297 wait complete. 2024-09-21T06:04:28.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.407+0000 7f7b99e00640 1 Processor -- start 2024-09-21T06:04:28.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.407+0000 7f7b99e00640 1 -- start start 2024-09-21T06:04:28.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.407+0000 7f7b99e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b940b4de0 0x7f7b9414d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:28.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.407+0000 7f7b99e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7b940b5c70 0x7f7b9414d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:28.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.407+0000 7f7b99e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b941527b0 0x7f7b941568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:28.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.407+0000 7f7b99e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7b94168920 con 0x7f7b941527b0 2024-09-21T06:04:28.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.407+0000 7f7b99e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7b941687a0 con 0x7f7b940b4de0 2024-09-21T06:04:28.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.407+0000 7f7b99e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7b94168aa0 con 0x7f7b940b5c70 2024-09-21T06:04:28.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b93400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b941527b0 0x7f7b941568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:28.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b92a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b940b4de0 0x7f7b9414d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:28.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b8be00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7b940b5c70 0x7f7b9414d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:28.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b93400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b941527b0 0x7f7b941568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40400/0 (socket says 172.21.15.50:40400) 2024-09-21T06:04:28.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b93400640 1 -- 172.21.15.50:0/1994725509 learned_addr learned my addr 172.21.15.50:0/1994725509 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b93400640 1 -- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7b940b5c70 msgr2=0x7f7b9414d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b93400640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7b940b5c70 0x7f7b9414d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b93400640 1 -- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b940b4de0 msgr2=0x7f7b9414d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b93400640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b940b4de0 0x7f7b9414d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b93400640 1 -- 172.21.15.50:0/1994725509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b940cf750 con 0x7f7b941527b0 2024-09-21T06:04:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b8be00640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7b940b5c70 0x7f7b9414d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b92a00640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b940b4de0 0x7f7b9414d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b93400640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b941527b0 0x7f7b941568b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0198d0 tx=0x7f7b8c0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b8b400640 1 -- 172.21.15.50:0/1994725509 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0bb070 con 0x7f7b941527b0 2024-09-21T06:04:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b8b400640 1 -- 172.21.15.50:0/1994725509 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b8c004d40 con 0x7f7b941527b0 2024-09-21T06:04:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b99e00640 1 -- 172.21.15.50:0/1994725509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b941f77f0 con 0x7f7b941527b0 2024-09-21T06:04:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b8b400640 1 -- 172.21.15.50:0/1994725509 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0b2040 con 0x7f7b941527b0 2024-09-21T06:04:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b99e00640 1 -- 172.21.15.50:0/1994725509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b940cded0 con 0x7f7b941527b0 2024-09-21T06:04:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b99e00640 1 -- 172.21.15.50:0/1994725509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b941f7c30 con 0x7f7b941527b0 2024-09-21T06:04:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.411+0000 7f7b8b400640 1 -- 172.21.15.50:0/1994725509 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f7b8c0be070 con 0x7f7b941527b0 2024-09-21T06:04:28.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.415+0000 7f7b8b400640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7b70073e60 0x7f7b70076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:28.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.415+0000 7f7b8b400640 1 -- 172.21.15.50:0/1994725509 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4999+0+0 (secure 0 0 0) 0x7f7b8c12c970 con 0x7f7b941527b0 2024-09-21T06:04:28.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.415+0000 7f7b92a00640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7b70073e60 0x7f7b70076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:28.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.415+0000 7f7b92a00640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7b70073e60 0x7f7b70076370 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7b84007fd0 tx=0x7f7b84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:28.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.419+0000 7f7b8b400640 1 -- 172.21.15.50:0/1994725509 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7b8c016970 con 0x7f7b941527b0 2024-09-21T06:04:28.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.607+0000 7f7b99e00640 1 -- 172.21.15.50:0/1994725509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7b940b51f0 con 0x7f7b941527b0 2024-09-21T06:04:28.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.715+0000 7f7b8b400640 1 -- 172.21.15.50:0/1994725509 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v65) ==== 97+0+0 (secure 0 0 0) 0x7f7b8c0f8d90 con 0x7f7b941527b0 2024-09-21T06:04:28.715 INFO:teuthology.orchestra.run.smithi050.stderr: removed all blocklist entries 2024-09-21T06:04:28.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.715+0000 7f7b99e00640 1 -- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7b70073e60 msgr2=0x7f7b70076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:28.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.715+0000 7f7b99e00640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7b70073e60 0x7f7b70076370 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7b84007fd0 tx=0x7f7b84009040 comp rx=0 tx=0).stop 2024-09-21T06:04:28.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.715+0000 7f7b99e00640 1 -- 172.21.15.50:0/1994725509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b941527b0 msgr2=0x7f7b941568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:28.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.715+0000 7f7b99e00640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b941527b0 0x7f7b941568b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0198d0 tx=0x7f7b8c0092c0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.715+0000 7f7b99e00640 1 -- 172.21.15.50:0/1994725509 shutdown_connections 2024-09-21T06:04:28.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.715+0000 7f7b99e00640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7b70073e60 0x7f7b70076370 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.715+0000 7f7b99e00640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b941527b0 0x7f7b941568b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.715+0000 7f7b99e00640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7b940b5c70 0x7f7b9414d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.715+0000 7f7b99e00640 1 --2- 172.21.15.50:0/1994725509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b940b4de0 0x7f7b9414d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.715+0000 7f7b99e00640 1 -- 172.21.15.50:0/1994725509 >> 172.21.15.50:0/1994725509 conn(0x7f7b940b2040 msgr2=0x7f7b941511b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:28.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.719+0000 7f7b99e00640 1 -- 172.21.15.50:0/1994725509 shutdown_connections 2024-09-21T06:04:28.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.719+0000 7f7b99e00640 1 -- 172.21.15.50:0/1994725509 wait complete. 2024-09-21T06:04:28.742 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:04:28.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.899+0000 7f7e86e00640 1 -- 172.21.15.50:0/152085422 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e80159130 msgr2=0x7f7e8015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:28.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.899+0000 7f7e86e00640 1 --2- 172.21.15.50:0/152085422 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e80159130 0x7f7e8015bb90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009870 tx=0x7f7e7001c7d0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.899+0000 7f7e86e00640 1 -- 172.21.15.50:0/152085422 shutdown_connections 2024-09-21T06:04:28.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.899+0000 7f7e86e00640 1 --2- 172.21.15.50:0/152085422 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e80159130 0x7f7e8015bb90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.899+0000 7f7e86e00640 1 -- 172.21.15.50:0/152085422 >> 172.21.15.50:0/152085422 conn(0x7f7e800b35f0 msgr2=0x7f7e800b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:28.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.899+0000 7f7e86e00640 1 -- 172.21.15.50:0/152085422 shutdown_connections 2024-09-21T06:04:28.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.899+0000 7f7e86e00640 1 -- 172.21.15.50:0/152085422 wait complete. 2024-09-21T06:04:28.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e86e00640 1 Processor -- start 2024-09-21T06:04:28.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e86e00640 1 -- start start 2024-09-21T06:04:28.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e86e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e8006cdf0 0x7f7e801604c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:28.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e86e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e800703a0 0x7f7e80152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:28.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e86e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e80159130 0x7f7e8015deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:28.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e86e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80169d40 con 0x7f7e80159130 2024-09-21T06:04:28.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e86e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8006d5a0 con 0x7f7e800703a0 2024-09-21T06:04:28.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e86e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7e80169ec0 con 0x7f7e8006cdf0 2024-09-21T06:04:28.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e84e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e80159130 0x7f7e8015deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:28.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e7fe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e8006cdf0 0x7f7e801604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:28.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e7fe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e8006cdf0 0x7f7e801604c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:38992/0 (socket says 172.21.15.50:38992) 2024-09-21T06:04:28.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e7fe00640 1 -- 172.21.15.50:0/3153812539 learned_addr learned my addr 172.21.15.50:0/3153812539 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:28.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e7f400640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e800703a0 0x7f7e80152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:28.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e7fe00640 1 -- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e800703a0 msgr2=0x7f7e80152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:28.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e7fe00640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e800703a0 0x7f7e80152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e7fe00640 1 -- 172.21.15.50:0/3153812539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e80159130 msgr2=0x7f7e8015deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:28.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e7fe00640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e80159130 0x7f7e8015deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:28.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.903+0000 7f7e7fe00640 1 -- 172.21.15.50:0/3153812539 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e801526d0 con 0x7f7e8006cdf0 2024-09-21T06:04:28.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.907+0000 7f7e7fe00640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e8006cdf0 0x7f7e801604c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7e700193e0 tx=0x7f7e700a6530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:28.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.907+0000 7f7e77e00640 1 -- 172.21.15.50:0/3153812539 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70005040 con 0x7f7e8006cdf0 2024-09-21T06:04:28.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.907+0000 7f7e77e00640 1 -- 172.21.15.50:0/3153812539 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e700051e0 con 0x7f7e8006cdf0 2024-09-21T06:04:28.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.907+0000 7f7e86e00640 1 -- 172.21.15.50:0/3153812539 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e801529b0 con 0x7f7e8006cdf0 2024-09-21T06:04:28.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.907+0000 7f7e77e00640 1 -- 172.21.15.50:0/3153812539 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70005630 con 0x7f7e8006cdf0 2024-09-21T06:04:28.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.907+0000 7f7e86e00640 1 -- 172.21.15.50:0/3153812539 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e80152f70 con 0x7f7e8006cdf0 2024-09-21T06:04:28.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.907+0000 7f7e86e00640 1 -- 172.21.15.50:0/3153812539 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e8015bb90 con 0x7f7e8006cdf0 2024-09-21T06:04:28.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.907+0000 7f7e77e00640 1 -- 172.21.15.50:0/3153812539 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f7e700046d0 con 0x7f7e8006cdf0 2024-09-21T06:04:28.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.907+0000 7f7e77e00640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7e60073e60 0x7f7e60076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:28.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.907+0000 7f7e77e00640 1 -- 172.21.15.50:0/3153812539 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(67..67 src has 1..67) ==== 4885+0+0 (secure 0 0 0) 0x7f7e7012d090 con 0x7f7e8006cdf0 2024-09-21T06:04:28.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.907+0000 7f7e7f400640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7e60073e60 0x7f7e60076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.911+0000 7f7e7f400640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7e60073e60 0x7f7e60076370 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7e80165c80 tx=0x7f7e6c001050 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:28.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:28.915+0000 7f7e77e00640 1 -- 172.21.15.50:0/3153812539 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7e700bf060 con 0x7f7e8006cdf0 2024-09-21T06:04:29.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.163+0000 7f7e86e00640 1 -- 172.21.15.50:0/3153812539 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7e800b2420 con 0x7f7e8006cdf0 2024-09-21T06:04:29.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.163+0000 7f7e77e00640 1 -- 172.21.15.50:0/3153812539 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v32) ==== 80+0+5304 (secure 0 0 0) 0x7f7e700ac030 con 0x7f7e8006cdf0 2024-09-21T06:04:29.166 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:29.166 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:04:29.166 INFO:teuthology.orchestra.run.smithi050.stdout: "auth_dump": [ 2024-09-21T06:04:29.166 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.166 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.a", 2024-09-21T06:04:29.166 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mSKq0ARAA2egaHGYWhRUEq6oRl1CIMg==", 2024-09-21T06:04:29.166 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.166 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:04:29.166 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:04:29.166 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.b", 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mwv0+DxAAY+eZPxv5uY9PnLl0asdr6w==", 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.167 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.c", 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5me6maCRAA4jvHgTLTKpb/mxzzG71SfQ==", 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.d", 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mzYnuFhAA/hEigjluXspzs/hyeWxBjQ==", 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.168 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.0", 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDIX+5mrZF/NhAA33M6ummG4r4yXU8/YOL5WA==", 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.169 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.1", 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDMX+5myVz5DxAAMYfTPep6ubD7Cxnfa9dMCg==", 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.2", 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDPX+5mA7ahIRAAf/osnY5l83p8ZWna0ErD2Q==", 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:04:29.170 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.3", 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDSX+5m+rDpNRAApGPZJIlAaA0DyjtPXPgoaQ==", 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.4", 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDYX+5mKL9SORAA21ITQUwUSoDUjXaD0Cwb3g==", 2024-09-21T06:04:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.5", 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDcX+5minRqFBAAMJOBQNvJFD4TNdKbPMLBig==", 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.172 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.6", 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDfX+5m6HadJhAAAv670Tvyi4IKMByFPiBlAw==", 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.7", 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDjX+5muPrrAhAAuRB7Muamiht+f7LvSmI6UQ==", 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.173 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.0", 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw==", 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow r", 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.174 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.1", 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mo94GJBAAMBSb1NPb2L93vzvnAh59kw==", 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.2", 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5m4IrnHxAAFjER3ywuS+FlbuA7bPZVqw==", 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:04:29.175 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.3", 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mysQeKBAAq5c4bD3hbsU1H7JabaMjwQ==", 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.admin", 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mcUXKHhAAV1Rs0OMNVPeAsRiAVt67jA==", 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.x", 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5m2F9WOBAAZuQTUyR7oTT01CRD1jo+/Q==", 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.y", 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mEjC/MxAAHgwIu4CLDgFajwD2rL5WKA==", 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.178 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-21T06:04:29.179 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:04:29.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.167+0000 7f7e86e00640 1 -- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7e60073e60 msgr2=0x7f7e60076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:29.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.167+0000 7f7e86e00640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7e60073e60 0x7f7e60076370 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7e80165c80 tx=0x7f7e6c001050 comp rx=0 tx=0).stop 2024-09-21T06:04:29.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.167+0000 7f7e86e00640 1 -- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e8006cdf0 msgr2=0x7f7e801604c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:29.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.167+0000 7f7e86e00640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e8006cdf0 0x7f7e801604c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7e700193e0 tx=0x7f7e700a6530 comp rx=0 tx=0).stop 2024-09-21T06:04:29.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.167+0000 7f7e86e00640 1 -- 172.21.15.50:0/3153812539 shutdown_connections 2024-09-21T06:04:29.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.167+0000 7f7e86e00640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7e60073e60 0x7f7e60076370 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.167+0000 7f7e86e00640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e80159130 0x7f7e8015deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.167+0000 7f7e86e00640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e800703a0 0x7f7e80152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.167+0000 7f7e86e00640 1 --2- 172.21.15.50:0/3153812539 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e8006cdf0 0x7f7e801604c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.167+0000 7f7e86e00640 1 -- 172.21.15.50:0/3153812539 >> 172.21.15.50:0/3153812539 conn(0x7f7e800b35f0 msgr2=0x7f7e800b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:29.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.167+0000 7f7e86e00640 1 -- 172.21.15.50:0/3153812539 shutdown_connections 2024-09-21T06:04:29.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.167+0000 7f7e86e00640 1 -- 172.21.15.50:0/3153812539 wait complete. 2024-09-21T06:04:29.189 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:04:29.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.347+0000 7f26f0400640 1 -- 172.21.15.50:0/4120391320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e815e8c0 msgr2=0x7f26e815ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:29.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.347+0000 7f26f0400640 1 --2- 172.21.15.50:0/4120391320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e815e8c0 0x7f26e815ecd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009840 tx=0x7f26dc01c820 comp rx=0 tx=0).stop 2024-09-21T06:04:29.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.347+0000 7f26f0400640 1 -- 172.21.15.50:0/4120391320 shutdown_connections 2024-09-21T06:04:29.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.347+0000 7f26f0400640 1 --2- 172.21.15.50:0/4120391320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e815e8c0 0x7f26e815ecd0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.347+0000 7f26f0400640 1 -- 172.21.15.50:0/4120391320 >> 172.21.15.50:0/4120391320 conn(0x7f26e80b2040 msgr2=0x7f26e80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:29.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.347+0000 7f26f0400640 1 -- 172.21.15.50:0/4120391320 shutdown_connections 2024-09-21T06:04:29.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.347+0000 7f26f0400640 1 -- 172.21.15.50:0/4120391320 wait complete. 2024-09-21T06:04:29.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26f0400640 1 Processor -- start 2024-09-21T06:04:29.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26f0400640 1 -- start start 2024-09-21T06:04:29.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26f0400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f26e815b2d0 0x7f26e813d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:29.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26f0400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e815e290 0x7f26e80cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:29.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26f0400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e815e8c0 0x7f26e80ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:29.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26f0400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8169df0 con 0x7f26e815e8c0 2024-09-21T06:04:29.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26f0400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8169c70 con 0x7f26e815e290 2024-09-21T06:04:29.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26f0400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f26e8169f70 con 0x7f26e815b2d0 2024-09-21T06:04:29.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26ee400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e815e8c0 0x7f26e80ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:29.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26eda00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f26e815b2d0 0x7f26e813d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:29.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26ed000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e815e290 0x7f26e80cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:29.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26ee400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e815e8c0 0x7f26e80ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40418/0 (socket says 172.21.15.50:40418) 2024-09-21T06:04:29.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26eda00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f26e815b2d0 0x7f26e813d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:38998/0 (socket says 172.21.15.50:38998) 2024-09-21T06:04:29.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26ee400640 1 -- 172.21.15.50:0/1779261301 learned_addr learned my addr 172.21.15.50:0/1779261301 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:29.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26ee400640 1 -- 172.21.15.50:0/1779261301 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f26e815b2d0 msgr2=0x7f26e813d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:29.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26ee400640 1 --2- 172.21.15.50:0/1779261301 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f26e815b2d0 0x7f26e813d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26ee400640 1 -- 172.21.15.50:0/1779261301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e815e290 msgr2=0x7f26e80cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:29.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26ee400640 1 --2- 172.21.15.50:0/1779261301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e815e290 0x7f26e80cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26ee400640 1 -- 172.21.15.50:0/1779261301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e80ceaa0 con 0x7f26e815e8c0 2024-09-21T06:04:29.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26ee400640 1 --2- 172.21.15.50:0/1779261301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e815e8c0 0x7f26e80ce040 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f26e000c730 tx=0x7f26e000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:29.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26e5600640 1 -- 172.21.15.50:0/1779261301 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0003b80 con 0x7f26e815e8c0 2024-09-21T06:04:29.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26e5600640 1 -- 172.21.15.50:0/1779261301 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26e0003d20 con 0x7f26e815e8c0 2024-09-21T06:04:29.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26f0400640 1 -- 172.21.15.50:0/1779261301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e80ced80 con 0x7f26e815e8c0 2024-09-21T06:04:29.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26e5600640 1 -- 172.21.15.50:0/1779261301 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e000aab0 con 0x7f26e815e8c0 2024-09-21T06:04:29.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.351+0000 7f26f0400640 1 -- 172.21.15.50:0/1779261301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e815fa90 con 0x7f26e815e8c0 2024-09-21T06:04:29.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.355+0000 7f26f0400640 1 -- 172.21.15.50:0/1779261301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e815ecd0 con 0x7f26e815e8c0 2024-09-21T06:04:29.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.355+0000 7f26e5600640 1 -- 172.21.15.50:0/1779261301 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f26e000ac50 con 0x7f26e815e8c0 2024-09-21T06:04:29.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.355+0000 7f26e5600640 1 --2- 172.21.15.50:0/1779261301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f26c8073e60 0x7f26c8076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:29.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.355+0000 7f26e5600640 1 -- 172.21.15.50:0/1779261301 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4885+0+0 (secure 0 0 0) 0x7f26e0092980 con 0x7f26e815e8c0 2024-09-21T06:04:29.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.355+0000 7f26eda00640 1 --2- 172.21.15.50:0/1779261301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f26c8073e60 0x7f26c8076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:29.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.359+0000 7f26eda00640 1 --2- 172.21.15.50:0/1779261301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f26c8073e60 0x7f26c8076370 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f26dc0af000 tx=0x7f26dc01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:29.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.359+0000 7f26e5600640 1 -- 172.21.15.50:0/1779261301 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f26e815f210 con 0x7f26e815e8c0 2024-09-21T06:04:29.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.539+0000 7f26f0400640 1 -- 172.21.15.50:0/1779261301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f26e815dd60 con 0x7f26e815e8c0 2024-09-21T06:04:29.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.539+0000 7f26e5600640 1 -- 172.21.15.50:0/1779261301 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) ==== 84+0+65 (secure 0 0 0) 0x7f26e005eda0 con 0x7f26e815e8c0 2024-09-21T06:04:29.541 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:29.541 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:04:29.541 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:29.541 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-09-21T06:04:29.541 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-09-21T06:04:29.541 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:29.542 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:04:29.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.543+0000 7f26f0400640 1 -- 172.21.15.50:0/1779261301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f26c8073e60 msgr2=0x7f26c8076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:29.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.543+0000 7f26f0400640 1 --2- 172.21.15.50:0/1779261301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f26c8073e60 0x7f26c8076370 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f26dc0af000 tx=0x7f26dc01b060 comp rx=0 tx=0).stop 2024-09-21T06:04:29.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.543+0000 7f26f0400640 1 -- 172.21.15.50:0/1779261301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e815e8c0 msgr2=0x7f26e80ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:29.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.543+0000 7f26f0400640 1 --2- 172.21.15.50:0/1779261301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e815e8c0 0x7f26e80ce040 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f26e000c730 tx=0x7f26e000cbf0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.543+0000 7f26f0400640 1 -- 172.21.15.50:0/1779261301 shutdown_connections 2024-09-21T06:04:29.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.543+0000 7f26f0400640 1 --2- 172.21.15.50:0/1779261301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f26c8073e60 0x7f26c8076370 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.543+0000 7f26f0400640 1 --2- 172.21.15.50:0/1779261301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e815e8c0 0x7f26e80ce040 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.543+0000 7f26f0400640 1 --2- 172.21.15.50:0/1779261301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e815e290 0x7f26e80cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.543+0000 7f26f0400640 1 --2- 172.21.15.50:0/1779261301 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f26e815b2d0 0x7f26e813d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.543+0000 7f26f0400640 1 -- 172.21.15.50:0/1779261301 >> 172.21.15.50:0/1779261301 conn(0x7f26e80b2040 msgr2=0x7f26e80744f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:29.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.543+0000 7f26f0400640 1 -- 172.21.15.50:0/1779261301 shutdown_connections 2024-09-21T06:04:29.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.543+0000 7f26f0400640 1 -- 172.21.15.50:0/1779261301 wait complete. 2024-09-21T06:04:29.564 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:04:29.564 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:04:29.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.727+0000 7f9169e00640 1 -- 172.21.15.50:0/960433225 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f91640746e0 msgr2=0x7f916415c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:29.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.727+0000 7f9169e00640 1 --2- 172.21.15.50:0/960433225 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f91640746e0 0x7f916415c920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9158009840 tx=0x7f915801c730 comp rx=0 tx=0).stop 2024-09-21T06:04:29.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.727+0000 7f9169e00640 1 -- 172.21.15.50:0/960433225 shutdown_connections 2024-09-21T06:04:29.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.727+0000 7f9169e00640 1 --2- 172.21.15.50:0/960433225 >> v2:172.21.15.88:3300/0 conn(0x7f91640cda80 0x7f91640cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.727+0000 7f9169e00640 1 --2- 172.21.15.50:0/960433225 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f91640746e0 0x7f916415c920 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.727+0000 7f9169e00640 1 -- 172.21.15.50:0/960433225 >> 172.21.15.50:0/960433225 conn(0x7f91640b41f0 msgr2=0x7f91640b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:29.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.731+0000 7f9169e00640 1 -- 172.21.15.50:0/960433225 shutdown_connections 2024-09-21T06:04:29.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.731+0000 7f9169e00640 1 -- 172.21.15.50:0/960433225 wait complete. 2024-09-21T06:04:29.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.731+0000 7f9169e00640 1 Processor -- start 2024-09-21T06:04:29.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.731+0000 7f9169e00640 1 -- start start 2024-09-21T06:04:29.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.731+0000 7f9169e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91640746e0 0x7f916415d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:29.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9169e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9164074c30 0x7f9164166540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:29.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9169e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91640cda80 0x7f916415f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:29.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9169e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9164169ee0 con 0x7f91640cda80 2024-09-21T06:04:29.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9169e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9164169d60 con 0x7f91640746e0 2024-09-21T06:04:29.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9169e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f916416a060 con 0x7f9164074c30 2024-09-21T06:04:29.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9168a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91640746e0 0x7f916415d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:29.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9168a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91640746e0 0x7f916415d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:51866/0 (socket says 172.21.15.50:51866) 2024-09-21T06:04:29.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9169400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91640cda80 0x7f916415f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:29.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9168a00640 1 -- 172.21.15.50:0/1288641919 learned_addr learned my addr 172.21.15.50:0/1288641919 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:29.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9163e00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9164074c30 0x7f9164166540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:29.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9168a00640 1 -- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9164074c30 msgr2=0x7f9164166540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:29.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9168a00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9164074c30 0x7f9164166540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9168a00640 1 -- 172.21.15.50:0/1288641919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91640cda80 msgr2=0x7f916415f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:29.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9168a00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91640cda80 0x7f916415f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:29.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9168a00640 1 -- 172.21.15.50:0/1288641919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916415d360 con 0x7f91640746e0 2024-09-21T06:04:29.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9169400640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91640cda80 0x7f916415f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:29.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9168a00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91640746e0 0x7f916415d8e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9158009790 tx=0x7f91580a5f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:29.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9160c00640 1 -- 172.21.15.50:0/1288641919 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91580b2040 con 0x7f91640746e0 2024-09-21T06:04:29.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9160c00640 1 -- 172.21.15.50:0/1288641919 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9158004130 con 0x7f91640746e0 2024-09-21T06:04:29.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9169e00640 1 -- 172.21.15.50:0/1288641919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9164165a60 con 0x7f91640746e0 2024-09-21T06:04:29.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9160c00640 1 -- 172.21.15.50:0/1288641919 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158003600 con 0x7f91640746e0 2024-09-21T06:04:29.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9169e00640 1 -- 172.21.15.50:0/1288641919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9164166020 con 0x7f91640746e0 2024-09-21T06:04:29.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9169e00640 1 -- 172.21.15.50:0/1288641919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f916415c920 con 0x7f91640746e0 2024-09-21T06:04:29.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.735+0000 7f9160c00640 1 -- 172.21.15.50:0/1288641919 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f91580bd060 con 0x7f91640746e0 2024-09-21T06:04:29.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.739+0000 7f9160c00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9144073e90 0x7f91440763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:29.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.739+0000 7f9160c00640 1 -- 172.21.15.50:0/1288641919 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4885+0+0 (secure 0 0 0) 0x7f915812c1a0 con 0x7f91640746e0 2024-09-21T06:04:29.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.739+0000 7f9163e00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9144073e90 0x7f91440763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:29.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.739+0000 7f9163e00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9144073e90 0x7f91440763a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f916409f3d0 tx=0x7f91500013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:29.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.743+0000 7f9160c00640 1 -- 172.21.15.50:0/1288641919 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9158016410 con 0x7f91640746e0 2024-09-21T06:04:29.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:29.919+0000 7f9169e00640 1 -- 172.21.15.50:0/1288641919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f91640b4f30 con 0x7f91640746e0 2024-09-21T06:04:30.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.739+0000 7f9160c00640 1 -- 172.21.15.50:0/1288641919 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v68) ==== 138+0+0 (secure 0 0 0) 0x7f91580f85c0 con 0x7f91640746e0 2024-09-21T06:04:30.741 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-09-21T06:04:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.747+0000 7f9169e00640 1 -- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9144073e90 msgr2=0x7f91440763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.747+0000 7f9169e00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9144073e90 0x7f91440763a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f916409f3d0 tx=0x7f91500013a0 comp rx=0 tx=0).stop 2024-09-21T06:04:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.747+0000 7f9169e00640 1 -- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91640746e0 msgr2=0x7f916415d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.747+0000 7f9169e00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91640746e0 0x7f916415d8e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9158009790 tx=0x7f91580a5f90 comp rx=0 tx=0).stop 2024-09-21T06:04:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.747+0000 7f9169e00640 1 -- 172.21.15.50:0/1288641919 shutdown_connections 2024-09-21T06:04:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.747+0000 7f9169e00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9144073e90 0x7f91440763a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.747+0000 7f9169e00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91640cda80 0x7f916415f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.747+0000 7f9169e00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9164074c30 0x7f9164166540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.747+0000 7f9169e00640 1 --2- 172.21.15.50:0/1288641919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91640746e0 0x7f916415d8e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.747+0000 7f9169e00640 1 -- 172.21.15.50:0/1288641919 >> 172.21.15.50:0/1288641919 conn(0x7f91640b41f0 msgr2=0x7f916406f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.747+0000 7f9169e00640 1 -- 172.21.15.50:0/1288641919 shutdown_connections 2024-09-21T06:04:30.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.747+0000 7f9169e00640 1 -- 172.21.15.50:0/1288641919 wait complete. 2024-09-21T06:04:30.771 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:04:30.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.939+0000 7f432ec00640 1 -- 172.21.15.50:0/1401161183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f432815f5b0 msgr2=0x7f43280b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:30.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.939+0000 7f432ec00640 1 --2- 172.21.15.50:0/1401161183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f432815f5b0 0x7f43280b1dc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4314009840 tx=0x7f431401c7f0 comp rx=0 tx=0).stop 2024-09-21T06:04:30.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.939+0000 7f432ec00640 1 -- 172.21.15.50:0/1401161183 shutdown_connections 2024-09-21T06:04:30.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.939+0000 7f432ec00640 1 --2- 172.21.15.50:0/1401161183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f432815f5b0 0x7f43280b1dc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:30.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.939+0000 7f432ec00640 1 --2- 172.21.15.50:0/1401161183 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4328155a00 0x7f432815f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:30.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.939+0000 7f432ec00640 1 -- 172.21.15.50:0/1401161183 >> 172.21.15.50:0/1401161183 conn(0x7f432806cdf0 msgr2=0x7f432806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:30.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.939+0000 7f432ec00640 1 -- 172.21.15.50:0/1401161183 shutdown_connections 2024-09-21T06:04:30.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.939+0000 7f432ec00640 1 -- 172.21.15.50:0/1401161183 wait complete. 2024-09-21T06:04:30.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ec00640 1 Processor -- start 2024-09-21T06:04:30.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ec00640 1 -- start start 2024-09-21T06:04:30.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43280b2300 0x7f432814ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:30.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ec00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4328155a00 0x7f432815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:30.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ec00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432815f5b0 0x7f4328149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ec00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4328168820 con 0x7f4328155a00 2024-09-21T06:04:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43281686a0 con 0x7f43280b2300 2024-09-21T06:04:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ec00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f43281689a0 con 0x7f432815f5b0 2024-09-21T06:04:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4328155a00 0x7f432815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4328155a00 0x7f432815e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40436/0 (socket says 172.21.15.50:40436) 2024-09-21T06:04:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ce00640 1 -- 172.21.15.50:0/1806402574 learned_addr learned my addr 172.21.15.50:0/1806402574 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432d800640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43280b2300 0x7f432814ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:30.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432e200640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432815f5b0 0x7f4328149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:30.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ce00640 1 -- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432815f5b0 msgr2=0x7f4328149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:30.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ce00640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432815f5b0 0x7f4328149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:30.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ce00640 1 -- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43280b2300 msgr2=0x7f432814ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:30.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ce00640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43280b2300 0x7f432814ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:30.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.943+0000 7f432ce00640 1 -- 172.21.15.50:0/1806402574 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432806d690 con 0x7f4328155a00 2024-09-21T06:04:30.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.947+0000 7f432ce00640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4328155a00 0x7f432815e0b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4314009790 tx=0x7f43140a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.947+0000 7f4325600640 1 -- 172.21.15.50:0/1806402574 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43140b2040 con 0x7f4328155a00 2024-09-21T06:04:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.947+0000 7f4325600640 1 -- 172.21.15.50:0/1806402574 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4314005020 con 0x7f4328155a00 2024-09-21T06:04:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.947+0000 7f432ec00640 1 -- 172.21.15.50:0/1806402574 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432806e7c0 con 0x7f4328155a00 2024-09-21T06:04:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.947+0000 7f4325600640 1 -- 172.21.15.50:0/1806402574 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43140044f0 con 0x7f4328155a00 2024-09-21T06:04:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.947+0000 7f432ec00640 1 -- 172.21.15.50:0/1806402574 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43281f9730 con 0x7f4328155a00 2024-09-21T06:04:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.947+0000 7f432ec00640 1 -- 172.21.15.50:0/1806402574 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432815f070 con 0x7f4328155a00 2024-09-21T06:04:30.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.947+0000 7f4325600640 1 -- 172.21.15.50:0/1806402574 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f4314005450 con 0x7f4328155a00 2024-09-21T06:04:30.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.951+0000 7f4325600640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4308073e90 0x7f43080763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:30.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.951+0000 7f4325600640 1 -- 172.21.15.50:0/1806402574 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5255+0+0 (secure 0 0 0) 0x7f431412c6c0 con 0x7f4328155a00 2024-09-21T06:04:30.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.951+0000 7f432d800640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4308073e90 0x7f43080763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:30.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.951+0000 7f432d800640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4308073e90 0x7f43080763a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f431c009600 tx=0x7f431c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:30.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:30.955+0000 7f4325600640 1 -- 172.21.15.50:0/1806402574 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f43140bc060 con 0x7f4328155a00 2024-09-21T06:04:31.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.131+0000 7f432ec00640 1 -- 172.21.15.50:0/1806402574 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f43280cc730 con 0x7f4328155a00 2024-09-21T06:04:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.743+0000 7f4325600640 1 -- 172.21.15.50:0/1806402574 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v69) ==== 170+0+0 (secure 0 0 0) 0x7f4314016410 con 0x7f4328155a00 2024-09-21T06:04:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-09-21T06:04:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.743+0000 7f432ec00640 1 -- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4308073e90 msgr2=0x7f43080763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.743+0000 7f432ec00640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4308073e90 0x7f43080763a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f431c009600 tx=0x7f431c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.743+0000 7f432ec00640 1 -- 172.21.15.50:0/1806402574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4328155a00 msgr2=0x7f432815e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.743+0000 7f432ec00640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4328155a00 0x7f432815e0b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4314009790 tx=0x7f43140a57a0 comp rx=0 tx=0).stop 2024-09-21T06:04:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.747+0000 7f432ec00640 1 -- 172.21.15.50:0/1806402574 shutdown_connections 2024-09-21T06:04:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.747+0000 7f432ec00640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432815f5b0 0x7f4328149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.747+0000 7f432ec00640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4308073e90 0x7f43080763a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.747+0000 7f432ec00640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4328155a00 0x7f432815e0b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.747+0000 7f432ec00640 1 --2- 172.21.15.50:0/1806402574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43280b2300 0x7f432814ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.747+0000 7f432ec00640 1 -- 172.21.15.50:0/1806402574 >> 172.21.15.50:0/1806402574 conn(0x7f432806cdf0 msgr2=0x7f432814bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.747+0000 7f432ec00640 1 -- 172.21.15.50:0/1806402574 shutdown_connections 2024-09-21T06:04:31.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.747+0000 7f432ec00640 1 -- 172.21.15.50:0/1806402574 wait complete. 2024-09-21T06:04:31.788 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:04:31.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.951+0000 7fe037c00640 1 -- 172.21.15.50:0/261530409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0300b4de0 msgr2=0x7fe0300b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:31.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.951+0000 7fe037c00640 1 --2- 172.21.15.50:0/261530409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0300b4de0 0x7fe0300b51f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe028009c80 tx=0x7fe02801cce0 comp rx=0 tx=0).stop 2024-09-21T06:04:31.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.951+0000 7fe037c00640 1 -- 172.21.15.50:0/261530409 shutdown_connections 2024-09-21T06:04:31.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.951+0000 7fe037c00640 1 --2- 172.21.15.50:0/261530409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0300b4de0 0x7fe0300b51f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:31.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.951+0000 7fe037c00640 1 --2- 172.21.15.50:0/261530409 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe0300b5c70 0x7fe03015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:31.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.951+0000 7fe037c00640 1 -- 172.21.15.50:0/261530409 >> 172.21.15.50:0/261530409 conn(0x7fe0300b2040 msgr2=0x7fe0300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:31.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.951+0000 7fe037c00640 1 -- 172.21.15.50:0/261530409 shutdown_connections 2024-09-21T06:04:31.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.951+0000 7fe037c00640 1 -- 172.21.15.50:0/261530409 wait complete. 2024-09-21T06:04:31.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.951+0000 7fe037c00640 1 Processor -- start 2024-09-21T06:04:31.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.951+0000 7fe037c00640 1 -- start start 2024-09-21T06:04:31.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe037c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0300b4de0 0x7fe03013d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:31.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe037c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe0300b5c70 0x7fe03013a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:31.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe037c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0301527b0 0x7fe0301f9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:31.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe037c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe030168a00 con 0x7fe0300b4de0 2024-09-21T06:04:31.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe037c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe030168880 con 0x7fe0301527b0 2024-09-21T06:04:31.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe037c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe030168b80 con 0x7fe0300b5c70 2024-09-21T06:04:31.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe035200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0300b4de0 0x7fe03013d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:31.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe035200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0300b4de0 0x7fe03013d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40450/0 (socket says 172.21.15.50:40450) 2024-09-21T06:04:31.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe035200640 1 -- 172.21.15.50:0/955701477 learned_addr learned my addr 172.21.15.50:0/955701477 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:31.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe02fe00640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe0300b5c70 0x7fe03013a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:31.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe035c00640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0301527b0 0x7fe0301f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:31.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe035200640 1 -- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe0300b5c70 msgr2=0x7fe03013a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:31.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe035200640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe0300b5c70 0x7fe03013a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:31.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe035200640 1 -- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0301527b0 msgr2=0x7fe0301f9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:31.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe035200640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0301527b0 0x7fe0301f9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:31.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe035200640 1 -- 172.21.15.50:0/955701477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03013a940 con 0x7fe0300b4de0 2024-09-21T06:04:31.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe035c00640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0301527b0 0x7fe0301f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:31.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe035200640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0300b4de0 0x7fe03013d720 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe02400d750 tx=0x7fe02400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:31.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe02cc00640 1 -- 172.21.15.50:0/955701477 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024017070 con 0x7fe0300b4de0 2024-09-21T06:04:31.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe02cc00640 1 -- 172.21.15.50:0/955701477 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe024009d20 con 0x7fe0300b4de0 2024-09-21T06:04:31.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe02cc00640 1 -- 172.21.15.50:0/955701477 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024005280 con 0x7fe0300b4de0 2024-09-21T06:04:31.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe037c00640 1 -- 172.21.15.50:0/955701477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03013cc70 con 0x7fe0300b4de0 2024-09-21T06:04:31.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe037c00640 1 -- 172.21.15.50:0/955701477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0301fa600 con 0x7fe0300b4de0 2024-09-21T06:04:31.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.955+0000 7fe037c00640 1 -- 172.21.15.50:0/955701477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0300b51f0 con 0x7fe0300b4de0 2024-09-21T06:04:31.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.959+0000 7fe02cc00640 1 -- 172.21.15.50:0/955701477 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7fe024005420 con 0x7fe0300b4de0 2024-09-21T06:04:31.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.959+0000 7fe02cc00640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe010073e90 0x7fe0100763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:31.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.959+0000 7fe02cc00640 1 -- 172.21.15.50:0/955701477 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5637+0+0 (secure 0 0 0) 0x7fe024093390 con 0x7fe0300b4de0 2024-09-21T06:04:31.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.959+0000 7fe02fe00640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe010073e90 0x7fe0100763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:31.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.959+0000 7fe02fe00640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe010073e90 0x7fe0100763a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe03015b830 tx=0x7fe01c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:31.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:31.963+0000 7fe02cc00640 1 -- 172.21.15.50:0/955701477 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe02400de60 con 0x7fe0300b4de0 2024-09-21T06:04:32.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.175+0000 7fe037c00640 1 -- 172.21.15.50:0/955701477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe0301f9da0 con 0x7fe0300b4de0 2024-09-21T06:04:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.743+0000 7fe02cc00640 1 -- 172.21.15.50:0/955701477 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) ==== 174+0+0 (secure 0 0 0) 0x7fe02405f5c0 con 0x7fe0300b4de0 2024-09-21T06:04:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 10 and data pool 11 2024-09-21T06:04:32.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.747+0000 7fe037c00640 1 -- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe010073e90 msgr2=0x7fe0100763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:32.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.747+0000 7fe037c00640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe010073e90 0x7fe0100763a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe03015b830 tx=0x7fe01c009040 comp rx=0 tx=0).stop 2024-09-21T06:04:32.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.747+0000 7fe037c00640 1 -- 172.21.15.50:0/955701477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0300b4de0 msgr2=0x7fe03013d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:32.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.747+0000 7fe037c00640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0300b4de0 0x7fe03013d720 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe02400d750 tx=0x7fe02400dc10 comp rx=0 tx=0).stop 2024-09-21T06:04:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.747+0000 7fe037c00640 1 -- 172.21.15.50:0/955701477 shutdown_connections 2024-09-21T06:04:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.747+0000 7fe037c00640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0301527b0 0x7fe0301f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.747+0000 7fe037c00640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe0300b5c70 0x7fe03013a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.747+0000 7fe037c00640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe010073e90 0x7fe0100763a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.747+0000 7fe037c00640 1 --2- 172.21.15.50:0/955701477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0300b4de0 0x7fe03013d720 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.747+0000 7fe037c00640 1 -- 172.21.15.50:0/955701477 >> 172.21.15.50:0/955701477 conn(0x7fe0300b2040 msgr2=0x7fe030154590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:32.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.747+0000 7fe037c00640 1 -- 172.21.15.50:0/955701477 shutdown_connections 2024-09-21T06:04:32.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.747+0000 7fe037c00640 1 -- 172.21.15.50:0/955701477 wait complete. 2024-09-21T06:04:32.789 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:04:32.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.947+0000 7fd9c2c00640 1 -- 172.21.15.50:0/89024044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9bc1527b0 msgr2=0x7fd9bc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:32.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.947+0000 7fd9c2c00640 1 --2- 172.21.15.50:0/89024044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9bc1527b0 0x7fd9bc15bb60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009900 tx=0x7fd9a401c8b0 comp rx=0 tx=0).stop 2024-09-21T06:04:32.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.947+0000 7fd9c2c00640 1 -- 172.21.15.50:0/89024044 shutdown_connections 2024-09-21T06:04:32.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.947+0000 7fd9c2c00640 1 --2- 172.21.15.50:0/89024044 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd9bc0b4de0 0x7fd9bc0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:32.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.947+0000 7fd9c2c00640 1 --2- 172.21.15.50:0/89024044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9bc1527b0 0x7fd9bc15bb60 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:32.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.947+0000 7fd9c2c00640 1 -- 172.21.15.50:0/89024044 >> 172.21.15.50:0/89024044 conn(0x7fd9bc0b2040 msgr2=0x7fd9bc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:32.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.947+0000 7fd9c2c00640 1 -- 172.21.15.50:0/89024044 shutdown_connections 2024-09-21T06:04:32.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.947+0000 7fd9c2c00640 1 -- 172.21.15.50:0/89024044 wait complete. 2024-09-21T06:04:32.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.947+0000 7fd9c2c00640 1 Processor -- start 2024-09-21T06:04:32.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.947+0000 7fd9c2c00640 1 -- start start 2024-09-21T06:04:32.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c2c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9bc0b4de0 0x7fd9bc153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:32.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c2c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd9bc0b5c70 0x7fd9bc14c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:32.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c2c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9bc1527b0 0x7fd9bc0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:32.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c2c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc168a00 con 0x7fd9bc1527b0 2024-09-21T06:04:32.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c2c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc168880 con 0x7fd9bc0b4de0 2024-09-21T06:04:32.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c2c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd9bc168b80 con 0x7fd9bc0b5c70 2024-09-21T06:04:32.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9bc1527b0 0x7fd9bc0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:32.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9bbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9bc0b4de0 0x7fd9bc153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:32.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9bbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9bc0b4de0 0x7fd9bc153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:51898/0 (socket says 172.21.15.50:51898) 2024-09-21T06:04:32.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9bbe00640 1 -- 172.21.15.50:0/3638093917 learned_addr learned my addr 172.21.15.50:0/3638093917 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:32.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9b3e00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd9bc0b5c70 0x7fd9bc14c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:32.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c0c00640 1 -- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd9bc0b5c70 msgr2=0x7fd9bc14c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:32.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c0c00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd9bc0b5c70 0x7fd9bc14c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:32.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c0c00640 1 -- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9bc0b4de0 msgr2=0x7fd9bc153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:32.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c0c00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9bc0b4de0 0x7fd9bc153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:32.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c0c00640 1 -- 172.21.15.50:0/3638093917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc1fc630 con 0x7fd9bc1527b0 2024-09-21T06:04:32.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c0c00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9bc1527b0 0x7fd9bc0ae8e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0ccab0 tx=0x7fd9b400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:32.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9b8c00640 1 -- 172.21.15.50:0/3638093917 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4017070 con 0x7fd9bc1527b0 2024-09-21T06:04:32.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9b8c00640 1 -- 172.21.15.50:0/3638093917 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9b4004e20 con 0x7fd9bc1527b0 2024-09-21T06:04:32.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c2c00640 1 -- 172.21.15.50:0/3638093917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9bc1fc910 con 0x7fd9bc1527b0 2024-09-21T06:04:32.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9b8c00640 1 -- 172.21.15.50:0/3638093917 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400bcb0 con 0x7fd9bc1527b0 2024-09-21T06:04:32.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c2c00640 1 -- 172.21.15.50:0/3638093917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9bc156390 con 0x7fd9bc1527b0 2024-09-21T06:04:32.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.951+0000 7fd9c2c00640 1 -- 172.21.15.50:0/3638093917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9bc1fc3a0 con 0x7fd9bc1527b0 2024-09-21T06:04:32.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.955+0000 7fd9b8c00640 1 -- 172.21.15.50:0/3638093917 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7fd9b4003980 con 0x7fd9bc1527b0 2024-09-21T06:04:32.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.955+0000 7fd9b8c00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd998073e60 0x7fd998076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:32.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.955+0000 7fd9b8c00640 1 -- 172.21.15.50:0/3638093917 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5737+0+0 (secure 0 0 0) 0x7fd9b4093be0 con 0x7fd9bc1527b0 2024-09-21T06:04:32.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.955+0000 7fd9bbe00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd998073e60 0x7fd998076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:32.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.955+0000 7fd9bbe00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd998073e60 0x7fd998076370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac001870 tx=0x7fd9ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:32.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:32.959+0000 7fd9b8c00640 1 -- 172.21.15.50:0/3638093917 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd9b405ffc0 con 0x7fd9bc1527b0 2024-09-21T06:04:33.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.135+0000 7fd9c2c00640 1 -- 172.21.15.50:0/3638093917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9bc0b51f0 con 0x7fd9bc1527b0 2024-09-21T06:04:33.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9b8c00640 1 -- 172.21.15.50:0/3638093917 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16736 (secure 0 0 0) 0x7fd9b40649f0 con 0x7fd9bc1527b0 2024-09-21T06:04:33.139 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:33.139 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":70,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:04:32.178496+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:04:29.936073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T06:04:31.146344+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:04:33.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9c2c00640 1 -- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd998073e60 msgr2=0x7fd998076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:33.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9c2c00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd998073e60 0x7fd998076370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac001870 tx=0x7fd9ac009040 comp rx=0 tx=0).stop 2024-09-21T06:04:33.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9c2c00640 1 -- 172.21.15.50:0/3638093917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9bc1527b0 msgr2=0x7fd9bc0ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:33.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9c2c00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9bc1527b0 0x7fd9bc0ae8e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0ccab0 tx=0x7fd9b400dd90 comp rx=0 tx=0).stop 2024-09-21T06:04:33.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9c2c00640 1 -- 172.21.15.50:0/3638093917 shutdown_connections 2024-09-21T06:04:33.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9c2c00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd998073e60 0x7fd998076370 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9c2c00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9bc1527b0 0x7fd9bc0ae8e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9c2c00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd9bc0b5c70 0x7fd9bc14c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9c2c00640 1 --2- 172.21.15.50:0/3638093917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9bc0b4de0 0x7fd9bc153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9c2c00640 1 -- 172.21.15.50:0/3638093917 >> 172.21.15.50:0/3638093917 conn(0x7fd9bc0b2040 msgr2=0x7fd9bc15ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:33.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9c2c00640 1 -- 172.21.15.50:0/3638093917 shutdown_connections 2024-09-21T06:04:33.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.139+0000 7fd9c2c00640 1 -- 172.21.15.50:0/3638093917 wait complete. 2024-09-21T06:04:33.162 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:04:33.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.319+0000 7f2ee3000640 1 -- 172.21.15.50:0/289140534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ec0003a10 msgr2=0x7f2ec0003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:33.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.319+0000 7f2ee3000640 1 --2- 172.21.15.50:0/289140534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ec0003a10 0x7f2ec0003e80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009b90 tx=0x7f2ecc01d260 comp rx=0 tx=0).stop 2024-09-21T06:04:33.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.319+0000 7f2ee1000640 1 -- 172.21.15.50:0/289140534 reap_dead start 2024-09-21T06:04:33.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.319+0000 7f2ee3000640 1 -- 172.21.15.50:0/289140534 shutdown_connections 2024-09-21T06:04:33.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.319+0000 7f2ee3000640 1 -- 172.21.15.50:0/289140534 >> 172.21.15.50:0/289140534 conn(0x7f2edc0b2040 msgr2=0x7f2edc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:33.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.319+0000 7f2ee3000640 1 -- 172.21.15.50:0/289140534 shutdown_connections 2024-09-21T06:04:33.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.319+0000 7f2ee3000640 1 -- 172.21.15.50:0/289140534 wait complete. 2024-09-21T06:04:33.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.323+0000 7f2ee3000640 1 Processor -- start 2024-09-21T06:04:33.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.323+0000 7f2ee3000640 1 -- start start 2024-09-21T06:04:33.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.323+0000 7f2ee3000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ec0003a10 0x7f2edc14fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:33.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.323+0000 7f2ee3000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2edc1f6ed0 0x7f2edc0b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:33.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.323+0000 7f2ee3000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2edc0b41f0 0x7f2edc163cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:33.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.323+0000 7f2ee3000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc166990 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.323+0000 7f2ee3000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc166810 con 0x7f2edc0b41f0 2024-09-21T06:04:33.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.323+0000 7f2ee3000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2edc166b10 con 0x7f2ec0003a10 2024-09-21T06:04:33.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.323+0000 7f2ed3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2edc1f6ed0 0x7f2edc0b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:33.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.323+0000 7f2ed3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2edc1f6ed0 0x7f2edc0b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40496/0 (socket says 172.21.15.50:40496) 2024-09-21T06:04:33.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.323+0000 7f2ed3e00640 1 -- 172.21.15.50:0/3333465994 learned_addr learned my addr 172.21.15.50:0/3333465994 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:33.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ee1000640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2edc0b41f0 0x7f2edc163cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:33.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2edbe00640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ec0003a10 0x7f2edc14fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:33.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ed3e00640 1 -- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ec0003a10 msgr2=0x7f2edc14fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:33.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ed3e00640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ec0003a10 0x7f2edc14fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ed3e00640 1 -- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2edc0b41f0 msgr2=0x7f2edc163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:33.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ed3e00640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2edc0b41f0 0x7f2edc163cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ed3e00640 1 -- 172.21.15.50:0/3333465994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc1503c0 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ed3e00640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2edc1f6ed0 0x7f2edc0b3cb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4011c00 tx=0x7f2ec400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:33.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ed8c00640 1 -- 172.21.15.50:0/3333465994 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4004460 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ed8c00640 1 -- 172.21.15.50:0/3333465994 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ec4005020 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ed8c00640 1 -- 172.21.15.50:0/3333465994 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4005430 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ee3000640 1 -- 172.21.15.50:0/3333465994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc0b4600 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ee3000640 1 -- 172.21.15.50:0/3333465994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc158fb0 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ee3000640 1 -- 172.21.15.50:0/3333465994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2edc074e30 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.327+0000 7f2ed8c00640 1 -- 172.21.15.50:0/3333465994 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f2ec40047b0 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.331+0000 7f2ed8c00640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2eb8073e60 0x7f2eb8076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:33.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.331+0000 7f2ed8c00640 1 -- 172.21.15.50:0/3333465994 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5737+0+0 (secure 0 0 0) 0x7f2ec4093f00 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.331+0000 7f2edbe00640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2eb8073e60 0x7f2eb8076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:33.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.331+0000 7f2edbe00640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2eb8073e60 0x7f2eb8076370 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0ae000 tx=0x7f2ecc01a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:33.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.335+0000 7f2ed8c00640 1 -- 172.21.15.50:0/3333465994 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2edc0b4600 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.515+0000 7f2ee3000640 1 -- 172.21.15.50:0/3333465994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2edc158d40 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.515+0000 7f2ed8c00640 1 -- 172.21.15.50:0/3333465994 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16736 (secure 0 0 0) 0x7f2ec4060a00 con 0x7f2edc1f6ed0 2024-09-21T06:04:33.517 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:33.517 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":70,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:04:32.178496+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:04:29.936073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T06:04:31.146344+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:04:33.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.519+0000 7f2ee3000640 1 -- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2eb8073e60 msgr2=0x7f2eb8076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:33.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.519+0000 7f2ee3000640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2eb8073e60 0x7f2eb8076370 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0ae000 tx=0x7f2ecc01a000 comp rx=0 tx=0).stop 2024-09-21T06:04:33.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.519+0000 7f2ee3000640 1 -- 172.21.15.50:0/3333465994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2edc1f6ed0 msgr2=0x7f2edc0b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:33.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.519+0000 7f2ee3000640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2edc1f6ed0 0x7f2edc0b3cb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4011c00 tx=0x7f2ec400f790 comp rx=0 tx=0).stop 2024-09-21T06:04:33.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.519+0000 7f2ee3000640 1 -- 172.21.15.50:0/3333465994 shutdown_connections 2024-09-21T06:04:33.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.519+0000 7f2ee3000640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2edc0b41f0 0x7f2edc163cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.519+0000 7f2ee3000640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2eb8073e60 0x7f2eb8076370 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.519+0000 7f2ee3000640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2edc1f6ed0 0x7f2edc0b3cb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.519+0000 7f2ee3000640 1 --2- 172.21.15.50:0/3333465994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ec0003a10 0x7f2edc14fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.519+0000 7f2ee3000640 1 -- 172.21.15.50:0/3333465994 >> 172.21.15.50:0/3333465994 conn(0x7f2edc0b2040 msgr2=0x7f2edc15fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:33.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.519+0000 7f2ee3000640 1 -- 172.21.15.50:0/3333465994 shutdown_connections 2024-09-21T06:04:33.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.519+0000 7f2ee3000640 1 -- 172.21.15.50:0/3333465994 wait complete. 2024-09-21T06:04:33.540 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:04:33.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.703+0000 7f3d59000640 1 -- 172.21.15.50:0/3223037920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d540b5c70 msgr2=0x7f3d5415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:33.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.703+0000 7f3d59000640 1 --2- 172.21.15.50:0/3223037920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d540b5c70 0x7f3d5415bb60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009840 tx=0x7f3d4401c7f0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.703+0000 7f3d59000640 1 -- 172.21.15.50:0/3223037920 shutdown_connections 2024-09-21T06:04:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.703+0000 7f3d59000640 1 --2- 172.21.15.50:0/3223037920 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d540b4de0 0x7f3d540b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.703+0000 7f3d59000640 1 --2- 172.21.15.50:0/3223037920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d540b5c70 0x7f3d5415bb60 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.703+0000 7f3d59000640 1 -- 172.21.15.50:0/3223037920 >> 172.21.15.50:0/3223037920 conn(0x7f3d540b2040 msgr2=0x7f3d540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.703+0000 7f3d59000640 1 -- 172.21.15.50:0/3223037920 shutdown_connections 2024-09-21T06:04:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.703+0000 7f3d59000640 1 -- 172.21.15.50:0/3223037920 wait complete. 2024-09-21T06:04:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.703+0000 7f3d59000640 1 Processor -- start 2024-09-21T06:04:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.703+0000 7f3d59000640 1 -- start start 2024-09-21T06:04:33.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d59000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d540b4de0 0x7f3d54149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:33.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d59000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d540b5c70 0x7f3d541fc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:33.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d59000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d541527b0 0x7f3d540ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:33.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d59000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3d54168970 con 0x7f3d541527b0 2024-09-21T06:04:33.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d59000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3d541687f0 con 0x7f3d540b5c70 2024-09-21T06:04:33.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d59000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3d54168af0 con 0x7f3d540b4de0 2024-09-21T06:04:33.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d52800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d541527b0 0x7f3d540ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:33.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d52800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d541527b0 0x7f3d540ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40506/0 (socket says 172.21.15.50:40506) 2024-09-21T06:04:33.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d52800640 1 -- 172.21.15.50:0/2693637594 learned_addr learned my addr 172.21.15.50:0/2693637594 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:33.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d51e00640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d540b4de0 0x7f3d54149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:33.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d4be00640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d540b5c70 0x7f3d541fc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:33.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d52800640 1 -- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d540b4de0 msgr2=0x7f3d54149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:33.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d52800640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d540b4de0 0x7f3d54149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d52800640 1 -- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d540b5c70 msgr2=0x7f3d541fc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:33.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d52800640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d540b5c70 0x7f3d541fc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:33.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d52800640 1 -- 172.21.15.50:0/2693637594 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d540cd4c0 con 0x7f3d541527b0 2024-09-21T06:04:33.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d52800640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d541527b0 0x7f3d540ae8e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00e940 tx=0x7f3d4c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:33.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d4a000640 1 -- 172.21.15.50:0/2693637594 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c004d30 con 0x7f3d541527b0 2024-09-21T06:04:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d4a000640 1 -- 172.21.15.50:0/2693637594 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d4c00cb20 con 0x7f3d541527b0 2024-09-21T06:04:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d4a000640 1 -- 172.21.15.50:0/2693637594 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c005020 con 0x7f3d541527b0 2024-09-21T06:04:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d59000640 1 -- 172.21.15.50:0/2693637594 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d54152da0 con 0x7f3d541527b0 2024-09-21T06:04:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d59000640 1 -- 172.21.15.50:0/2693637594 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d54153360 con 0x7f3d541527b0 2024-09-21T06:04:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.707+0000 7f3d59000640 1 -- 172.21.15.50:0/2693637594 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d540b51f0 con 0x7f3d541527b0 2024-09-21T06:04:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.711+0000 7f3d4a000640 1 -- 172.21.15.50:0/2693637594 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f3d4c0051c0 con 0x7f3d541527b0 2024-09-21T06:04:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.711+0000 7f3d4a000640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d30073e60 0x7f3d30076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.711+0000 7f3d4a000640 1 -- 172.21.15.50:0/2693637594 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5737+0+0 (secure 0 0 0) 0x7f3d4c093400 con 0x7f3d541527b0 2024-09-21T06:04:33.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.711+0000 7f3d51e00640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d30073e60 0x7f3d30076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:33.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.711+0000 7f3d51e00640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d30073e60 0x7f3d30076370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3d440af000 tx=0x7f3d4401b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:33.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.715+0000 7f3d4a000640 1 -- 172.21.15.50:0/2693637594 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3d4c016030 con 0x7f3d541527b0 2024-09-21T06:04:33.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:33.931+0000 7f3d59000640 1 -- 172.21.15.50:0/2693637594 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3d5415c080 con 0x7f3d541527b0 2024-09-21T06:04:34.720 INFO:tasks.ceph.mds.d.smithi088.stderr:2024-09-21T06:04:34.718+0000 7fe656200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:04:34.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.771+0000 7f3d4a000640 1 -- 172.21.15.50:0/2693637594 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) ==== 118+0+0 (secure 0 0 0) 0x7f3d4c05f880 con 0x7f3d541527b0 2024-09-21T06:04:34.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.775+0000 7f3d59000640 1 -- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d30073e60 msgr2=0x7f3d30076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.775+0000 7f3d59000640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d30073e60 0x7f3d30076370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3d440af000 tx=0x7f3d4401b030 comp rx=0 tx=0).stop 2024-09-21T06:04:34.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.775+0000 7f3d59000640 1 -- 172.21.15.50:0/2693637594 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d541527b0 msgr2=0x7f3d540ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.775+0000 7f3d59000640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d541527b0 0x7f3d540ae8e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00e940 tx=0x7f3d4c00ee00 comp rx=0 tx=0).stop 2024-09-21T06:04:34.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.775+0000 7f3d59000640 1 -- 172.21.15.50:0/2693637594 shutdown_connections 2024-09-21T06:04:34.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.775+0000 7f3d59000640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d30073e60 0x7f3d30076370 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.775+0000 7f3d59000640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d541527b0 0x7f3d540ae8e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.775+0000 7f3d59000640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d540b5c70 0x7f3d541fc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.775+0000 7f3d59000640 1 --2- 172.21.15.50:0/2693637594 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3d540b4de0 0x7f3d54149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.775+0000 7f3d59000640 1 -- 172.21.15.50:0/2693637594 >> 172.21.15.50:0/2693637594 conn(0x7f3d540b2040 msgr2=0x7f3d54151860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:34.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.775+0000 7f3d59000640 1 -- 172.21.15.50:0/2693637594 shutdown_connections 2024-09-21T06:04:34.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.775+0000 7f3d59000640 1 -- 172.21.15.50:0/2693637594 wait complete. 2024-09-21T06:04:34.803 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:04:34.803 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:04:34.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.967+0000 7f1068400640 1 -- 172.21.15.50:0/1053833542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f106006f170 msgr2=0x7f106006f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.967+0000 7f1068400640 1 --2- 172.21.15.50:0/1053833542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f106006f170 0x7f106006f580 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1058009bc0 tx=0x7f105801cb70 comp rx=0 tx=0).stop 2024-09-21T06:04:34.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.967+0000 7f1068400640 1 -- 172.21.15.50:0/1053833542 shutdown_connections 2024-09-21T06:04:34.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.967+0000 7f1068400640 1 --2- 172.21.15.50:0/1053833542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f106006f170 0x7f106006f580 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.967+0000 7f1068400640 1 -- 172.21.15.50:0/1053833542 >> 172.21.15.50:0/1053833542 conn(0x7f10600b2c40 msgr2=0x7f10600b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:34.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.967+0000 7f1068400640 1 -- 172.21.15.50:0/1053833542 shutdown_connections 2024-09-21T06:04:34.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.967+0000 7f1068400640 1 -- 172.21.15.50:0/1053833542 wait complete. 2024-09-21T06:04:34.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1068400640 1 Processor -- start 2024-09-21T06:04:34.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1068400640 1 -- start start 2024-09-21T06:04:34.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1068400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f106006f170 0x7f106013dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1068400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f106015c5d0 0x7f1060153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1068400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1060151630 0x7f10600ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1068400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1060169f00 con 0x7f106006f170 2024-09-21T06:04:34.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1068400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1060169d80 con 0x7f106015c5d0 2024-09-21T06:04:34.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1068400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f106016a080 con 0x7f1060151630 2024-09-21T06:04:34.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1065a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f106006f170 0x7f106013dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1065a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f106006f170 0x7f106013dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40518/0 (socket says 172.21.15.50:40518) 2024-09-21T06:04:34.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1065a00640 1 -- 172.21.15.50:0/616838402 learned_addr learned my addr 172.21.15.50:0/616838402 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:34.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1065000640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f106015c5d0 0x7f1060153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1066400640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1060151630 0x7f10600ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1065a00640 1 -- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1060151630 msgr2=0x7f10600ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1065a00640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1060151630 0x7f10600ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1065a00640 1 -- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f106015c5d0 msgr2=0x7f1060153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1065a00640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f106015c5d0 0x7f1060153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1065a00640 1 -- 172.21.15.50:0/616838402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1060152a00 con 0x7f106006f170 2024-09-21T06:04:34.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.971+0000 7f1065a00640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f106006f170 0x7f106013dc00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f105400c420 tx=0x7f105400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:34.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.975+0000 7f105d600640 1 -- 172.21.15.50:0/616838402 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054017070 con 0x7f106006f170 2024-09-21T06:04:34.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.975+0000 7f105d600640 1 -- 172.21.15.50:0/616838402 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10540053c0 con 0x7f106006f170 2024-09-21T06:04:34.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.975+0000 7f1068400640 1 -- 172.21.15.50:0/616838402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1060152ce0 con 0x7f106006f170 2024-09-21T06:04:34.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.975+0000 7f105d600640 1 -- 172.21.15.50:0/616838402 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054003860 con 0x7f106006f170 2024-09-21T06:04:34.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.975+0000 7f1068400640 1 -- 172.21.15.50:0/616838402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10601532a0 con 0x7f106006f170 2024-09-21T06:04:34.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.975+0000 7f1068400640 1 -- 172.21.15.50:0/616838402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10600706b0 con 0x7f106006f170 2024-09-21T06:04:34.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.975+0000 7f105d600640 1 -- 172.21.15.50:0/616838402 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f105401a070 con 0x7f106006f170 2024-09-21T06:04:34.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.975+0000 7f105d600640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1040073e60 0x7f1040076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.975+0000 7f105d600640 1 -- 172.21.15.50:0/616838402 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f1054092210 con 0x7f106006f170 2024-09-21T06:04:34.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.975+0000 7f1065000640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1040073e60 0x7f1040076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.979+0000 7f1065000640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1040073e60 0x7f1040076370 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f10601fbbe0 tx=0x7f104c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:34.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:34.979+0000 7f105d600640 1 -- 172.21.15.50:0/616838402 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f105405e3b0 con 0x7f106006f170 2024-09-21T06:04:35.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.191+0000 7f1068400640 1 -- 172.21.15.50:0/616838402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f106006f580 con 0x7f106006f170 2024-09-21T06:04:35.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.779+0000 7f105d600640 1 -- 172.21.15.50:0/616838402 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f1054062de0 con 0x7f106006f170 2024-09-21T06:04:35.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.783+0000 7f1068400640 1 -- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1040073e60 msgr2=0x7f1040076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:35.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.783+0000 7f1068400640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1040073e60 0x7f1040076370 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f10601fbbe0 tx=0x7f104c009040 comp rx=0 tx=0).stop 2024-09-21T06:04:35.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.783+0000 7f1068400640 1 -- 172.21.15.50:0/616838402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f106006f170 msgr2=0x7f106013dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:35.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.783+0000 7f1068400640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f106006f170 0x7f106013dc00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f105400c420 tx=0x7f105400c8e0 comp rx=0 tx=0).stop 2024-09-21T06:04:35.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.783+0000 7f1068400640 1 -- 172.21.15.50:0/616838402 shutdown_connections 2024-09-21T06:04:35.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.783+0000 7f1068400640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1060151630 0x7f10600ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:35.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.783+0000 7f1068400640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f106015c5d0 0x7f1060153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:35.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.783+0000 7f1068400640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1040073e60 0x7f1040076370 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:35.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.783+0000 7f1068400640 1 --2- 172.21.15.50:0/616838402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f106006f170 0x7f106013dc00 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:35.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.783+0000 7f1068400640 1 -- 172.21.15.50:0/616838402 >> 172.21.15.50:0/616838402 conn(0x7f10600b2c40 msgr2=0x7f10600b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:35.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.783+0000 7f1068400640 1 -- 172.21.15.50:0/616838402 shutdown_connections 2024-09-21T06:04:35.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.783+0000 7f1068400640 1 -- 172.21.15.50:0/616838402 wait complete. 2024-09-21T06:04:35.808 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:04:35.809 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:04:35.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.967+0000 7f3e32600640 1 -- 172.21.15.50:0/1008238798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e10004d50 msgr2=0x7f3e10005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:35.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.967+0000 7f3e32600640 1 --2- 172.21.15.50:0/1008238798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e10004d50 0x7f3e10005160 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400c640 tx=0x7f3e2401fa60 comp rx=0 tx=0).stop 2024-09-21T06:04:35.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.967+0000 7f3e2be00640 1 -- 172.21.15.50:0/1008238798 reap_dead start 2024-09-21T06:04:35.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.967+0000 7f3e32600640 1 -- 172.21.15.50:0/1008238798 shutdown_connections 2024-09-21T06:04:35.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.967+0000 7f3e32600640 1 -- 172.21.15.50:0/1008238798 >> 172.21.15.50:0/1008238798 conn(0x7f3e2c0b2040 msgr2=0x7f3e2c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:35.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.967+0000 7f3e32600640 1 -- 172.21.15.50:0/1008238798 shutdown_connections 2024-09-21T06:04:35.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.967+0000 7f3e32600640 1 -- 172.21.15.50:0/1008238798 wait complete. 2024-09-21T06:04:35.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e32600640 1 Processor -- start 2024-09-21T06:04:35.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e32600640 1 -- start start 2024-09-21T06:04:35.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e32600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e10004d50 0x7f3e2c1535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:35.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e32600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e2c151c60 0x7f3e2c0ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:35.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e32600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c0cf420 0x7f3e2c1f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:35.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e32600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c15cf30 con 0x7f3e2c0cf420 2024-09-21T06:04:35.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e32600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c15cdb0 con 0x7f3e2c151c60 2024-09-21T06:04:35.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e32600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2c15d0b0 con 0x7f3e10004d50 2024-09-21T06:04:35.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c0cf420 0x7f3e2c1f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:35.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c0cf420 0x7f3e2c1f9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40536/0 (socket says 172.21.15.50:40536) 2024-09-21T06:04:35.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e2be00640 1 -- 172.21.15.50:0/2195151576 learned_addr learned my addr 172.21.15.50:0/2195151576 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:35.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e2b400640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e10004d50 0x7f3e2c1535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:35.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e23e00640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e2c151c60 0x7f3e2c0ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:35.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e2be00640 1 -- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e10004d50 msgr2=0x7f3e2c1535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:35.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e2be00640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e10004d50 0x7f3e2c1535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:35.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e2be00640 1 -- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e2c151c60 msgr2=0x7f3e2c0ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:35.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e2be00640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e2c151c60 0x7f3e2c0ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:35.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.971+0000 7f3e2be00640 1 -- 172.21.15.50:0/2195151576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c0cf830 con 0x7f3e2c0cf420 2024-09-21T06:04:35.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.975+0000 7f3e2be00640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c0cf420 0x7f3e2c1f9100 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3e24021140 tx=0x7f3e24004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:35.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.975+0000 7f3e23400640 1 -- 172.21.15.50:0/2195151576 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240011d0 con 0x7f3e2c0cf420 2024-09-21T06:04:35.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.975+0000 7f3e23400640 1 -- 172.21.15.50:0/2195151576 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e24001a40 con 0x7f3e2c0cf420 2024-09-21T06:04:35.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.975+0000 7f3e23400640 1 -- 172.21.15.50:0/2195151576 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24002040 con 0x7f3e2c0cf420 2024-09-21T06:04:35.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.975+0000 7f3e32600640 1 -- 172.21.15.50:0/2195151576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2c153fb0 con 0x7f3e2c0cf420 2024-09-21T06:04:35.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.975+0000 7f3e32600640 1 -- 172.21.15.50:0/2195151576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2c154570 con 0x7f3e2c0cf420 2024-09-21T06:04:35.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.975+0000 7f3e32600640 1 -- 172.21.15.50:0/2195151576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e2c153d20 con 0x7f3e2c0cf420 2024-09-21T06:04:35.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.975+0000 7f3e23400640 1 -- 172.21.15.50:0/2195151576 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f3e24003bc0 con 0x7f3e2c0cf420 2024-09-21T06:04:35.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.975+0000 7f3e23400640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3e08073e90 0x7f3e080763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:35.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.975+0000 7f3e23400640 1 -- 172.21.15.50:0/2195151576 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f3e240b9020 con 0x7f3e2c0cf420 2024-09-21T06:04:35.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.979+0000 7f3e2b400640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3e08073e90 0x7f3e080763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:35.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.979+0000 7f3e2b400640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3e08073e90 0x7f3e080763a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009ce0 tx=0x7f3e1c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:35.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:35.979+0000 7f3e23400640 1 -- 172.21.15.50:0/2195151576 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3e2c0cf830 con 0x7f3e2c0cf420 2024-09-21T06:04:36.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 -- 172.21.15.50:0/2195151576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e2c0768c0 con 0x7f3e2c0cf420 2024-09-21T06:04:36.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e23400640 1 -- 172.21.15.50:0/2195151576 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5222 (secure 0 0 0) 0x7f3e24019810 con 0x7f3e2c0cf420 2024-09-21T06:04:36.195 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:36.195 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":45,"btime":"2024-09-21T06:04:35:777183+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/2233580616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2233580616},{"type":"v1","addr":"172.21.15.50:6833","nonce":2233580616}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3749582023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3749582023},{"type":"v1","addr":"172.21.15.50:6835","nonce":3749582023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6139,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/382971646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":382971646},{"type":"v1","addr":"172.21.15.88:6836","nonce":382971646}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:04:32.178456+0000","modified":"2024-09-21T06:04:35.197824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6423":{"gid":6423,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/2131576448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2131576448},{"type":"v1","addr":"172.21.15.88:6837","nonce":2131576448}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6423,"qdb_cluster":[6423]},"id":5}]} 2024-09-21T06:04:36.196 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 45 2024-09-21T06:04:36.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 -- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3e08073e90 msgr2=0x7f3e080763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:36.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3e08073e90 0x7f3e080763a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009ce0 tx=0x7f3e1c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 -- 172.21.15.50:0/2195151576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c0cf420 msgr2=0x7f3e2c1f9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:36.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c0cf420 0x7f3e2c1f9100 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3e24021140 tx=0x7f3e24004e60 comp rx=0 tx=0).stop 2024-09-21T06:04:36.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 -- 172.21.15.50:0/2195151576 shutdown_connections 2024-09-21T06:04:36.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3e08073e90 0x7f3e080763a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c0cf420 0x7f3e2c1f9100 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e2c151c60 0x7f3e2c0ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 --2- 172.21.15.50:0/2195151576 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3e10004d50 0x7f3e2c1535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 -- 172.21.15.50:0/2195151576 >> 172.21.15.50:0/2195151576 conn(0x7f3e2c0b2040 msgr2=0x7f3e2c0b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:36.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 -- 172.21.15.50:0/2195151576 shutdown_connections 2024-09-21T06:04:36.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.195+0000 7f3e32600640 1 -- 172.21.15.50:0/2195151576 wait complete. 2024-09-21T06:04:36.218 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:04:36.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.375+0000 7fc544400640 1 -- 172.21.15.50:0/828044219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53c1527b0 msgr2=0x7fc53c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:36.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.375+0000 7fc544400640 1 --2- 172.21.15.50:0/828044219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53c1527b0 0x7fc53c074e30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc53c071cb0 tx=0x7fc52801c890 comp rx=0 tx=0).stop 2024-09-21T06:04:36.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.375+0000 7fc544400640 1 -- 172.21.15.50:0/828044219 shutdown_connections 2024-09-21T06:04:36.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.375+0000 7fc544400640 1 --2- 172.21.15.50:0/828044219 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc53c0b4de0 0x7fc53c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.375+0000 7fc544400640 1 --2- 172.21.15.50:0/828044219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53c1527b0 0x7fc53c074e30 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.375+0000 7fc544400640 1 --2- 172.21.15.50:0/828044219 >> v2:172.21.15.88:3300/0 conn(0x7fc53c0b5c70 0x7fc53c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.375+0000 7fc544400640 1 -- 172.21.15.50:0/828044219 >> 172.21.15.50:0/828044219 conn(0x7fc53c0b2040 msgr2=0x7fc53c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:36.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.375+0000 7fc544400640 1 -- 172.21.15.50:0/828044219 shutdown_connections 2024-09-21T06:04:36.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.375+0000 7fc544400640 1 -- 172.21.15.50:0/828044219 wait complete. 2024-09-21T06:04:36.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.375+0000 7fc544400640 1 Processor -- start 2024-09-21T06:04:36.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.375+0000 7fc544400640 1 -- start start 2024-09-21T06:04:36.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc544400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53c0b4de0 0x7fc53c144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:36.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc544400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc53c0b5c70 0x7fc53c0cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:36.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc544400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc53c1527b0 0x7fc53c1fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:36.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc544400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c15f6b0 con 0x7fc53c0b4de0 2024-09-21T06:04:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc544400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c15f530 con 0x7fc53c0b5c70 2024-09-21T06:04:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc544400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc53c15f830 con 0x7fc53c1527b0 2024-09-21T06:04:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc541a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53c0b4de0 0x7fc53c144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc541a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53c0b4de0 0x7fc53c144c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40562/0 (socket says 172.21.15.50:40562) 2024-09-21T06:04:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc541a00640 1 -- 172.21.15.50:0/2814825238 learned_addr learned my addr 172.21.15.50:0/2814825238 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc541000640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc53c0b5c70 0x7fc53c0cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc542400640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc53c1527b0 0x7fc53c1fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc541a00640 1 -- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc53c1527b0 msgr2=0x7fc53c1fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc541a00640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc53c1527b0 0x7fc53c1fbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc541a00640 1 -- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc53c0b5c70 msgr2=0x7fc53c0cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc541a00640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc53c0b5c70 0x7fc53c0cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc541a00640 1 -- 172.21.15.50:0/2814825238 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c1fcc50 con 0x7fc53c0b4de0 2024-09-21T06:04:36.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc541a00640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53c0b4de0 0x7fc53c144c20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc53000e7a0 tx=0x7fc53000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:36.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc539600640 1 -- 172.21.15.50:0/2814825238 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53000ccc0 con 0x7fc53c0b4de0 2024-09-21T06:04:36.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc539600640 1 -- 172.21.15.50:0/2814825238 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc530010430 con 0x7fc53c0b4de0 2024-09-21T06:04:36.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc544400640 1 -- 172.21.15.50:0/2814825238 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c0cfc50 con 0x7fc53c0b4de0 2024-09-21T06:04:36.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc539600640 1 -- 172.21.15.50:0/2814825238 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530010860 con 0x7fc53c0b4de0 2024-09-21T06:04:36.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc544400640 1 -- 172.21.15.50:0/2814825238 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c154460 con 0x7fc53c0b4de0 2024-09-21T06:04:36.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.379+0000 7fc544400640 1 -- 172.21.15.50:0/2814825238 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc53c15fd50 con 0x7fc53c0b4de0 2024-09-21T06:04:36.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.383+0000 7fc539600640 1 -- 172.21.15.50:0/2814825238 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7fc530010a00 con 0x7fc53c0b4de0 2024-09-21T06:04:36.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.383+0000 7fc539600640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc51c073e90 0x7fc51c0763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:36.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.383+0000 7fc539600640 1 -- 172.21.15.50:0/2814825238 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7fc5300934c0 con 0x7fc53c0b4de0 2024-09-21T06:04:36.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.383+0000 7fc541000640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc51c073e90 0x7fc51c0763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:36.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.383+0000 7fc541000640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc51c073e90 0x7fc51c0763a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc5280af000 tx=0x7fc5280a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:36.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.387+0000 7fc539600640 1 -- 172.21.15.50:0/2814825238 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc53005f9d0 con 0x7fc53c0b4de0 2024-09-21T06:04:36.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.563+0000 7fc544400640 1 -- 172.21.15.50:0/2814825238 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc53c0cd810 con 0x7fc53c0b4de0 2024-09-21T06:04:36.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.563+0000 7fc539600640 1 -- 172.21.15.50:0/2814825238 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16723 (secure 0 0 0) 0x7fc530064400 con 0x7fc53c0b4de0 2024-09-21T06:04:36.566 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:36.566 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":71,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:04:33.761833+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:04:29.936073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T06:04:31.146344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:04:36.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.567+0000 7fc544400640 1 -- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc51c073e90 msgr2=0x7fc51c0763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:36.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.567+0000 7fc544400640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc51c073e90 0x7fc51c0763a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc5280af000 tx=0x7fc5280a7040 comp rx=0 tx=0).stop 2024-09-21T06:04:36.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.567+0000 7fc544400640 1 -- 172.21.15.50:0/2814825238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53c0b4de0 msgr2=0x7fc53c144c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:36.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.567+0000 7fc544400640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53c0b4de0 0x7fc53c144c20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc53000e7a0 tx=0x7fc53000ec60 comp rx=0 tx=0).stop 2024-09-21T06:04:36.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.567+0000 7fc544400640 1 -- 172.21.15.50:0/2814825238 shutdown_connections 2024-09-21T06:04:36.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.567+0000 7fc544400640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc53c1527b0 0x7fc53c1fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.567+0000 7fc544400640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc53c0b5c70 0x7fc53c0cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.567+0000 7fc544400640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc51c073e90 0x7fc51c0763a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.567+0000 7fc544400640 1 --2- 172.21.15.50:0/2814825238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53c0b4de0 0x7fc53c144c20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.567+0000 7fc544400640 1 -- 172.21.15.50:0/2814825238 >> 172.21.15.50:0/2814825238 conn(0x7fc53c0b2040 msgr2=0x7fc53c159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:36.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.567+0000 7fc544400640 1 -- 172.21.15.50:0/2814825238 shutdown_connections 2024-09-21T06:04:36.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.567+0000 7fc544400640 1 -- 172.21.15.50:0/2814825238 wait complete. 2024-09-21T06:04:36.588 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:04:36.588 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:04:36.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.747+0000 7f3628800640 1 -- 172.21.15.50:0/486888319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36200b4de0 msgr2=0x7f36200b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:36.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.747+0000 7f3628800640 1 --2- 172.21.15.50:0/486888319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36200b4de0 0x7f36200b51f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3620071cb0 tx=0x7f361c01ccc0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.747+0000 7f3628800640 1 -- 172.21.15.50:0/486888319 shutdown_connections 2024-09-21T06:04:36.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.747+0000 7f3628800640 1 --2- 172.21.15.50:0/486888319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36200b4de0 0x7f36200b51f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.747+0000 7f3628800640 1 --2- 172.21.15.50:0/486888319 >> v2:172.21.15.88:3300/0 conn(0x7f36200b5c70 0x7f362015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.747+0000 7f3628800640 1 -- 172.21.15.50:0/486888319 >> 172.21.15.50:0/486888319 conn(0x7f36200b2040 msgr2=0x7f36200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:36.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.747+0000 7f3628800640 1 -- 172.21.15.50:0/486888319 shutdown_connections 2024-09-21T06:04:36.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.747+0000 7f3628800640 1 -- 172.21.15.50:0/486888319 wait complete. 2024-09-21T06:04:36.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.747+0000 7f3628800640 1 Processor -- start 2024-09-21T06:04:36.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.747+0000 7f3628800640 1 -- start start 2024-09-21T06:04:36.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3628800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36200b4de0 0x7f36201fc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:36.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3628800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f36200b5c70 0x7f36201543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:36.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3628800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36201527b0 0x7f36200ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:36.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3628800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3620168920 con 0x7f36201527b0 2024-09-21T06:04:36.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3628800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f36201687a0 con 0x7f36200b4de0 2024-09-21T06:04:36.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3628800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3620168aa0 con 0x7f36200b5c70 2024-09-21T06:04:36.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3626800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36201527b0 0x7f36200ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:36.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3626800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36201527b0 0x7f36200ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36624/0 (socket says 172.21.15.50:36624) 2024-09-21T06:04:36.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3626800640 1 -- 172.21.15.50:0/2852124431 learned_addr learned my addr 172.21.15.50:0/2852124431 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:36.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3625e00640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36200b4de0 0x7f36201fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:36.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3625400640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f36200b5c70 0x7f36201543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:36.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3626800640 1 -- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f36200b5c70 msgr2=0x7f36201543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:36.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3626800640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f36200b5c70 0x7f36201543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3626800640 1 -- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36200b4de0 msgr2=0x7f36201fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:36.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3626800640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36200b4de0 0x7f36201fc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3626800640 1 -- 172.21.15.50:0/2852124431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362015b690 con 0x7f36201527b0 2024-09-21T06:04:36.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3625e00640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36200b4de0 0x7f36201fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:36.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3625400640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f36200b5c70 0x7f36201543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:36.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3626800640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36201527b0 0x7f36200ae8e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f361c009bd0 tx=0x7f361c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:36.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f361a000640 1 -- 172.21.15.50:0/2852124431 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c0bb070 con 0x7f36201527b0 2024-09-21T06:04:36.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f361a000640 1 -- 172.21.15.50:0/2852124431 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f361c004d40 con 0x7f36201527b0 2024-09-21T06:04:36.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3628800640 1 -- 172.21.15.50:0/2852124431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36201f8d20 con 0x7f36201527b0 2024-09-21T06:04:36.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f361a000640 1 -- 172.21.15.50:0/2852124431 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c0b2040 con 0x7f36201527b0 2024-09-21T06:04:36.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3628800640 1 -- 172.21.15.50:0/2852124431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36201f92e0 con 0x7f36201527b0 2024-09-21T06:04:36.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.751+0000 7f3628800640 1 -- 172.21.15.50:0/2852124431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36200b51f0 con 0x7f36201527b0 2024-09-21T06:04:36.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.755+0000 7f361a000640 1 -- 172.21.15.50:0/2852124431 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f361c0be070 con 0x7f36201527b0 2024-09-21T06:04:36.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.755+0000 7f361a000640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3604073e60 0x7f3604076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:36.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.755+0000 7f361a000640 1 -- 172.21.15.50:0/2852124431 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f361c12cd20 con 0x7f36201527b0 2024-09-21T06:04:36.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.755+0000 7f3625e00640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3604073e60 0x7f3604076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:36.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.755+0000 7f3625e00640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3604073e60 0x7f3604076370 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3614005fd0 tx=0x7f3614005dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:36.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.759+0000 7f361a000640 1 -- 172.21.15.50:0/2852124431 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f361c016970 con 0x7f36201527b0 2024-09-21T06:04:36.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.935+0000 7f3628800640 1 -- 172.21.15.50:0/2852124431 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f362006b540 con 0x7f3604073e60 2024-09-21T06:04:36.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.939+0000 7f361a000640 1 -- 172.21.15.50:0/2852124431 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251632 (secure 0 0 0) 0x7f361400ea30 con 0x7f3604073e60 2024-09-21T06:04:36.943 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:36.944 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:04:36.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.943+0000 7f3628800640 1 -- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3604073e60 msgr2=0x7f3604076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:36.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.943+0000 7f3628800640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3604073e60 0x7f3604076370 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3614005fd0 tx=0x7f3614005dc0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.943+0000 7f3628800640 1 -- 172.21.15.50:0/2852124431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36201527b0 msgr2=0x7f36200ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:36.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.943+0000 7f3628800640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36201527b0 0x7f36200ae8e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f361c009bd0 tx=0x7f361c005b00 comp rx=0 tx=0).stop 2024-09-21T06:04:36.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.943+0000 7f3628800640 1 -- 172.21.15.50:0/2852124431 shutdown_connections 2024-09-21T06:04:36.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.943+0000 7f3628800640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3604073e60 0x7f3604076370 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.943+0000 7f3628800640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36201527b0 0x7f36200ae8e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.943+0000 7f3628800640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f36200b5c70 0x7f36201543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.943+0000 7f3628800640 1 --2- 172.21.15.50:0/2852124431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36200b4de0 0x7f36201fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:36.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.943+0000 7f3628800640 1 -- 172.21.15.50:0/2852124431 >> 172.21.15.50:0/2852124431 conn(0x7f36200b2040 msgr2=0x7f36200b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:36.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.943+0000 7f3628800640 1 -- 172.21.15.50:0/2852124431 shutdown_connections 2024-09-21T06:04:36.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:36.943+0000 7f3628800640 1 -- 172.21.15.50:0/2852124431 wait complete. 2024-09-21T06:04:36.965 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-21T06:04:35.022516+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":136,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":418300,"kb_used_data":7748,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754556420,"statfs":{"total":773094113280,"available":772665774080,"internally_reserved":0,"allocated":7933952,"data_stored":3991827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.283317"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793750+0000","last_change":"2024-09-21T06:04:32.787310+0000","last_active":"2024-09-21T06:04:32.793750+0000","last_peered":"2024-09-21T06:04:32.793750+0000","last_clean":"2024-09-21T06:04:32.793750+0000","last_became_active":"2024-09-21T06:04:32.786715+0000","last_became_peered":"2024-09-21T06:04:32.786715+0000","last_unstale":"2024-09-21T06:04:32.793750+0000","last_undegraded":"2024-09-21T06:04:32.793750+0000","last_fullsized":"2024-09-21T06:04:32.793750+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:59.802808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.784907+0000","last_change":"2024-09-21T06:04:32.778255+0000","last_active":"2024-09-21T06:04:32.784907+0000","last_peered":"2024-09-21T06:04:32.784907+0000","last_clean":"2024-09-21T06:04:32.784907+0000","last_became_active":"2024-09-21T06:04:32.777803+0000","last_became_peered":"2024-09-21T06:04:32.777803+0000","last_unstale":"2024-09-21T06:04:32.784907+0000","last_undegraded":"2024-09-21T06:04:32.784907+0000","last_fullsized":"2024-09-21T06:04:32.784907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:54.763471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781150+0000","last_change":"2024-09-21T06:04:32.772652+0000","last_active":"2024-09-21T06:04:32.781150+0000","last_peered":"2024-09-21T06:04:32.781150+0000","last_clean":"2024-09-21T06:04:32.781150+0000","last_became_active":"2024-09-21T06:04:32.772322+0000","last_became_peered":"2024-09-21T06:04:32.772322+0000","last_unstale":"2024-09-21T06:04:32.781150+0000","last_undegraded":"2024-09-21T06:04:32.781150+0000","last_fullsized":"2024-09-21T06:04:32.781150+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:36.624434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.762434+0000","last_change":"2024-09-21T06:04:31.762434+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.762434+0000","last_undegraded":"2024-09-21T06:04:31.762434+0000","last_fullsized":"2024-09-21T06:04:31.762434+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783599+0000","last_change":"2024-09-21T06:04:32.777265+0000","last_active":"2024-09-21T06:04:32.783599+0000","last_peered":"2024-09-21T06:04:32.783599+0000","last_clean":"2024-09-21T06:04:32.783599+0000","last_became_active":"2024-09-21T06:04:32.776943+0000","last_became_peered":"2024-09-21T06:04:32.776943+0000","last_unstale":"2024-09-21T06:04:32.783599+0000","last_undegraded":"2024-09-21T06:04:32.783599+0000","last_fullsized":"2024-09-21T06:04:32.783599+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:05.622248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.796017+0000","last_change":"2024-09-21T06:04:32.794992+0000","last_active":"2024-09-21T06:04:32.796017+0000","last_peered":"2024-09-21T06:04:32.796017+0000","last_clean":"2024-09-21T06:04:32.796017+0000","last_became_active":"2024-09-21T06:04:32.794580+0000","last_became_peered":"2024-09-21T06:04:32.794580+0000","last_unstale":"2024-09-21T06:04:32.796017+0000","last_undegraded":"2024-09-21T06:04:32.796017+0000","last_fullsized":"2024-09-21T06:04:32.796017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:57.984993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.768680+0000","last_change":"2024-09-21T06:04:32.765123+0000","last_active":"2024-09-21T06:04:32.768680+0000","last_peered":"2024-09-21T06:04:32.768680+0000","last_clean":"2024-09-21T06:04:32.768680+0000","last_became_active":"2024-09-21T06:04:32.764446+0000","last_became_peered":"2024-09-21T06:04:32.764446+0000","last_unstale":"2024-09-21T06:04:32.768680+0000","last_undegraded":"2024-09-21T06:04:32.768680+0000","last_fullsized":"2024-09-21T06:04:32.768680+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:00.689363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783317+0000","last_change":"2024-09-21T06:04:32.773644+0000","last_active":"2024-09-21T06:04:32.783317+0000","last_peered":"2024-09-21T06:04:32.783317+0000","last_clean":"2024-09-21T06:04:32.783317+0000","last_became_active":"2024-09-21T06:04:32.771506+0000","last_became_peered":"2024-09-21T06:04:32.771506+0000","last_unstale":"2024-09-21T06:04:32.783317+0000","last_undegraded":"2024-09-21T06:04:32.783317+0000","last_fullsized":"2024-09-21T06:04:32.783317+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:39.994097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.778492+0000","last_change":"2024-09-21T06:04:32.775939+0000","last_active":"2024-09-21T06:04:32.778492+0000","last_peered":"2024-09-21T06:04:32.778492+0000","last_clean":"2024-09-21T06:04:32.778492+0000","last_became_active":"2024-09-21T06:04:32.775526+0000","last_became_peered":"2024-09-21T06:04:32.775526+0000","last_unstale":"2024-09-21T06:04:32.778492+0000","last_undegraded":"2024-09-21T06:04:32.778492+0000","last_fullsized":"2024-09-21T06:04:32.778492+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:17.326812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.769320+0000","last_change":"2024-09-21T06:04:32.768237+0000","last_active":"2024-09-21T06:04:32.769320+0000","last_peered":"2024-09-21T06:04:32.769320+0000","last_clean":"2024-09-21T06:04:32.769320+0000","last_became_active":"2024-09-21T06:04:32.767480+0000","last_became_peered":"2024-09-21T06:04:32.767480+0000","last_unstale":"2024-09-21T06:04:32.769320+0000","last_undegraded":"2024-09-21T06:04:32.769320+0000","last_fullsized":"2024-09-21T06:04:32.769320+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:37.256481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793616+0000","last_change":"2024-09-21T06:04:32.786243+0000","last_active":"2024-09-21T06:04:32.793616+0000","last_peered":"2024-09-21T06:04:32.793616+0000","last_clean":"2024-09-21T06:04:32.793616+0000","last_became_active":"2024-09-21T06:04:32.785278+0000","last_became_peered":"2024-09-21T06:04:32.785278+0000","last_unstale":"2024-09-21T06:04:32.793616+0000","last_undegraded":"2024-09-21T06:04:32.793616+0000","last_fullsized":"2024-09-21T06:04:32.793616+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:41.742767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.761947+0000","last_change":"2024-09-21T06:04:31.761947+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.761947+0000","last_undegraded":"2024-09-21T06:04:31.761947+0000","last_fullsized":"2024-09-21T06:04:31.761947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.775956+0000","last_change":"2024-09-21T06:04:32.774414+0000","last_active":"2024-09-21T06:04:32.775956+0000","last_peered":"2024-09-21T06:04:32.775956+0000","last_clean":"2024-09-21T06:04:32.775956+0000","last_became_active":"2024-09-21T06:04:32.774110+0000","last_became_peered":"2024-09-21T06:04:32.774110+0000","last_unstale":"2024-09-21T06:04:32.775956+0000","last_undegraded":"2024-09-21T06:04:32.775956+0000","last_fullsized":"2024-09-21T06:04:32.775956+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:11.805943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.766598+0000","last_change":"2024-09-21T06:04:32.760679+0000","last_active":"2024-09-21T06:04:32.766598+0000","last_peered":"2024-09-21T06:04:32.766598+0000","last_clean":"2024-09-21T06:04:32.766598+0000","last_became_active":"2024-09-21T06:04:32.760378+0000","last_became_peered":"2024-09-21T06:04:32.760378+0000","last_unstale":"2024-09-21T06:04:32.766598+0000","last_undegraded":"2024-09-21T06:04:32.766598+0000","last_fullsized":"2024-09-21T06:04:32.766598+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:40.885781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.761919+0000","last_change":"2024-09-21T06:04:32.760660+0000","last_active":"2024-09-21T06:04:32.761919+0000","last_peered":"2024-09-21T06:04:32.761919+0000","last_clean":"2024-09-21T06:04:32.761919+0000","last_became_active":"2024-09-21T06:04:32.760303+0000","last_became_peered":"2024-09-21T06:04:32.760303+0000","last_unstale":"2024-09-21T06:04:32.761919+0000","last_undegraded":"2024-09-21T06:04:32.761919+0000","last_fullsized":"2024-09-21T06:04:32.761919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:04.139988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793365+0000","last_change":"2024-09-21T06:04:32.783570+0000","last_active":"2024-09-21T06:04:32.793365+0000","last_peered":"2024-09-21T06:04:32.793365+0000","last_clean":"2024-09-21T06:04:32.793365+0000","last_became_active":"2024-09-21T06:04:32.783265+0000","last_became_peered":"2024-09-21T06:04:32.783265+0000","last_unstale":"2024-09-21T06:04:32.793365+0000","last_undegraded":"2024-09-21T06:04:32.793365+0000","last_fullsized":"2024-09-21T06:04:32.793365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:49.490698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.778520+0000","last_change":"2024-09-21T06:04:32.776322+0000","last_active":"2024-09-21T06:04:32.778520+0000","last_peered":"2024-09-21T06:04:32.778520+0000","last_clean":"2024-09-21T06:04:32.778520+0000","last_became_active":"2024-09-21T06:04:32.776027+0000","last_became_peered":"2024-09-21T06:04:32.776027+0000","last_unstale":"2024-09-21T06:04:32.778520+0000","last_undegraded":"2024-09-21T06:04:32.778520+0000","last_fullsized":"2024-09-21T06:04:32.778520+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:14.913586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.760440+0000","last_change":"2024-09-21T06:04:31.760440+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.760440+0000","last_undegraded":"2024-09-21T06:04:31.760440+0000","last_fullsized":"2024-09-21T06:04:31.760440+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.773202+0000","last_change":"2024-09-21T06:04:32.771994+0000","last_active":"2024-09-21T06:04:32.773202+0000","last_peered":"2024-09-21T06:04:32.773202+0000","last_clean":"2024-09-21T06:04:32.773202+0000","last_became_active":"2024-09-21T06:04:32.771597+0000","last_became_peered":"2024-09-21T06:04:32.771597+0000","last_unstale":"2024-09-21T06:04:32.773202+0000","last_undegraded":"2024-09-21T06:04:32.773202+0000","last_fullsized":"2024-09-21T06:04:32.773202+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:00.871520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.769001+0000","last_change":"2024-09-21T06:04:32.766745+0000","last_active":"2024-09-21T06:04:32.769001+0000","last_peered":"2024-09-21T06:04:32.769001+0000","last_clean":"2024-09-21T06:04:32.769001+0000","last_became_active":"2024-09-21T06:04:32.766362+0000","last_became_peered":"2024-09-21T06:04:32.766362+0000","last_unstale":"2024-09-21T06:04:32.769001+0000","last_undegraded":"2024-09-21T06:04:32.769001+0000","last_fullsized":"2024-09-21T06:04:32.769001+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:12.666560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.765784+0000","last_change":"2024-09-21T06:04:32.760980+0000","last_active":"2024-09-21T06:04:32.765784+0000","last_peered":"2024-09-21T06:04:32.765784+0000","last_clean":"2024-09-21T06:04:32.765784+0000","last_became_active":"2024-09-21T06:04:32.760648+0000","last_became_peered":"2024-09-21T06:04:32.760648+0000","last_unstale":"2024-09-21T06:04:32.765784+0000","last_undegraded":"2024-09-21T06:04:32.765784+0000","last_fullsized":"2024-09-21T06:04:32.765784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:58.412247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.767175+0000","last_change":"2024-09-21T06:04:32.762713+0000","last_active":"2024-09-21T06:04:32.767175+0000","last_peered":"2024-09-21T06:04:32.767175+0000","last_clean":"2024-09-21T06:04:32.767175+0000","last_became_active":"2024-09-21T06:04:32.762345+0000","last_became_peered":"2024-09-21T06:04:32.762345+0000","last_unstale":"2024-09-21T06:04:32.767175+0000","last_undegraded":"2024-09-21T06:04:32.767175+0000","last_fullsized":"2024-09-21T06:04:32.767175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:54.763471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793898+0000","last_change":"2024-09-21T06:04:32.788061+0000","last_active":"2024-09-21T06:04:32.793898+0000","last_peered":"2024-09-21T06:04:32.793898+0000","last_clean":"2024-09-21T06:04:32.793898+0000","last_became_active":"2024-09-21T06:04:32.787777+0000","last_became_peered":"2024-09-21T06:04:32.787777+0000","last_unstale":"2024-09-21T06:04:32.793898+0000","last_undegraded":"2024-09-21T06:04:32.793898+0000","last_fullsized":"2024-09-21T06:04:32.793898+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:53.442251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794388+0000","last_change":"2024-09-21T06:04:32.790938+0000","last_active":"2024-09-21T06:04:32.794388+0000","last_peered":"2024-09-21T06:04:32.794388+0000","last_clean":"2024-09-21T06:04:32.794388+0000","last_became_active":"2024-09-21T06:04:32.790267+0000","last_became_peered":"2024-09-21T06:04:32.790267+0000","last_unstale":"2024-09-21T06:04:32.794388+0000","last_undegraded":"2024-09-21T06:04:32.794388+0000","last_fullsized":"2024-09-21T06:04:32.794388+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:19.004848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.789947+0000","last_change":"2024-09-21T06:04:32.779963+0000","last_active":"2024-09-21T06:04:32.789947+0000","last_peered":"2024-09-21T06:04:32.789947+0000","last_clean":"2024-09-21T06:04:32.789947+0000","last_became_active":"2024-09-21T06:04:32.779659+0000","last_became_peered":"2024-09-21T06:04:32.779659+0000","last_unstale":"2024-09-21T06:04:32.789947+0000","last_undegraded":"2024-09-21T06:04:32.789947+0000","last_fullsized":"2024-09-21T06:04:32.789947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:08.847286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783071+0000","last_change":"2024-09-21T06:04:32.778117+0000","last_active":"2024-09-21T06:04:32.783071+0000","last_peered":"2024-09-21T06:04:32.783071+0000","last_clean":"2024-09-21T06:04:32.783071+0000","last_became_active":"2024-09-21T06:04:32.777105+0000","last_became_peered":"2024-09-21T06:04:32.777105+0000","last_unstale":"2024-09-21T06:04:32.783071+0000","last_undegraded":"2024-09-21T06:04:32.783071+0000","last_fullsized":"2024-09-21T06:04:32.783071+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:37.388005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781124+0000","last_change":"2024-09-21T06:04:32.771946+0000","last_active":"2024-09-21T06:04:32.781124+0000","last_peered":"2024-09-21T06:04:32.781124+0000","last_clean":"2024-09-21T06:04:32.781124+0000","last_became_active":"2024-09-21T06:04:32.771652+0000","last_became_peered":"2024-09-21T06:04:32.771652+0000","last_unstale":"2024-09-21T06:04:32.781124+0000","last_undegraded":"2024-09-21T06:04:32.781124+0000","last_fullsized":"2024-09-21T06:04:32.781124+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:02.037656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792043+0000","last_change":"2024-09-21T06:04:32.780368+0000","last_active":"2024-09-21T06:04:32.792043+0000","last_peered":"2024-09-21T06:04:32.792043+0000","last_clean":"2024-09-21T06:04:32.792043+0000","last_became_active":"2024-09-21T06:04:32.780021+0000","last_became_peered":"2024-09-21T06:04:32.780021+0000","last_unstale":"2024-09-21T06:04:32.792043+0000","last_undegraded":"2024-09-21T06:04:32.792043+0000","last_fullsized":"2024-09-21T06:04:32.792043+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:57.984993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.779808+0000","last_change":"2024-09-21T06:04:32.773537+0000","last_active":"2024-09-21T06:04:32.779808+0000","last_peered":"2024-09-21T06:04:32.779808+0000","last_clean":"2024-09-21T06:04:32.779808+0000","last_became_active":"2024-09-21T06:04:32.773240+0000","last_became_peered":"2024-09-21T06:04:32.773240+0000","last_unstale":"2024-09-21T06:04:32.779808+0000","last_undegraded":"2024-09-21T06:04:32.779808+0000","last_fullsized":"2024-09-21T06:04:32.779808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:04.990837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.789169+0000","last_change":"2024-09-21T06:04:32.782666+0000","last_active":"2024-09-21T06:04:32.789169+0000","last_peered":"2024-09-21T06:04:32.789169+0000","last_clean":"2024-09-21T06:04:32.789169+0000","last_became_active":"2024-09-21T06:04:32.777455+0000","last_became_peered":"2024-09-21T06:04:32.777455+0000","last_unstale":"2024-09-21T06:04:32.789169+0000","last_undegraded":"2024-09-21T06:04:32.789169+0000","last_fullsized":"2024-09-21T06:04:32.789169+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:53.442251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.768646+0000","last_change":"2024-09-21T06:04:32.764381+0000","last_active":"2024-09-21T06:04:32.768646+0000","last_peered":"2024-09-21T06:04:32.768646+0000","last_clean":"2024-09-21T06:04:32.768646+0000","last_became_active":"2024-09-21T06:04:32.764040+0000","last_became_peered":"2024-09-21T06:04:32.764040+0000","last_unstale":"2024-09-21T06:04:32.768646+0000","last_undegraded":"2024-09-21T06:04:32.768646+0000","last_fullsized":"2024-09-21T06:04:32.768646+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:33.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783350+0000","last_change":"2024-09-21T06:04:32.774466+0000","last_active":"2024-09-21T06:04:32.783350+0000","last_peered":"2024-09-21T06:04:32.783350+0000","last_clean":"2024-09-21T06:04:32.783350+0000","last_became_active":"2024-09-21T06:04:32.774133+0000","last_became_peered":"2024-09-21T06:04:32.774133+0000","last_unstale":"2024-09-21T06:04:32.783350+0000","last_undegraded":"2024-09-21T06:04:32.783350+0000","last_fullsized":"2024-09-21T06:04:32.783350+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:58.412247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794461+0000","last_change":"2024-09-21T06:04:32.792389+0000","last_active":"2024-09-21T06:04:32.794461+0000","last_peered":"2024-09-21T06:04:32.794461+0000","last_clean":"2024-09-21T06:04:32.794461+0000","last_became_active":"2024-09-21T06:04:32.792097+0000","last_became_peered":"2024-09-21T06:04:32.792097+0000","last_unstale":"2024-09-21T06:04:32.794461+0000","last_undegraded":"2024-09-21T06:04:32.794461+0000","last_fullsized":"2024-09-21T06:04:32.794461+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:46.730158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793238+0000","last_change":"2024-09-21T06:04:31.777083+0000","last_active":"2024-09-21T06:04:32.793238+0000","last_peered":"2024-09-21T06:04:32.793238+0000","last_clean":"2024-09-21T06:04:32.793238+0000","last_became_active":"2024-09-21T06:04:31.776739+0000","last_became_peered":"2024-09-21T06:04:31.776739+0000","last_unstale":"2024-09-21T06:04:32.793238+0000","last_undegraded":"2024-09-21T06:04:32.793238+0000","last_fullsized":"2024-09-21T06:04:32.793238+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:59.686723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783664+0000","last_change":"2024-09-21T06:04:32.776263+0000","last_active":"2024-09-21T06:04:32.783664+0000","last_peered":"2024-09-21T06:04:32.783664+0000","last_clean":"2024-09-21T06:04:32.783664+0000","last_became_active":"2024-09-21T06:04:32.775659+0000","last_became_peered":"2024-09-21T06:04:32.775659+0000","last_unstale":"2024-09-21T06:04:32.783664+0000","last_undegraded":"2024-09-21T06:04:32.783664+0000","last_fullsized":"2024-09-21T06:04:32.783664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:22.506119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.784109+0000","last_change":"2024-09-21T06:04:31.769588+0000","last_active":"2024-09-21T06:04:32.784109+0000","last_peered":"2024-09-21T06:04:32.784109+0000","last_clean":"2024-09-21T06:04:32.784109+0000","last_became_active":"2024-09-21T06:04:31.768905+0000","last_became_peered":"2024-09-21T06:04:31.768905+0000","last_unstale":"2024-09-21T06:04:32.784109+0000","last_undegraded":"2024-09-21T06:04:32.784109+0000","last_fullsized":"2024-09-21T06:04:32.784109+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:27.975438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.789767+0000","last_change":"2024-09-21T06:04:32.788286+0000","last_active":"2024-09-21T06:04:32.789767+0000","last_peered":"2024-09-21T06:04:32.789767+0000","last_clean":"2024-09-21T06:04:32.789767+0000","last_became_active":"2024-09-21T06:04:32.787926+0000","last_became_peered":"2024-09-21T06:04:32.787926+0000","last_unstale":"2024-09-21T06:04:32.789767+0000","last_undegraded":"2024-09-21T06:04:32.789767+0000","last_fullsized":"2024-09-21T06:04:32.789767+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:08.847286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794349+0000","last_change":"2024-09-21T06:04:31.772963+0000","last_active":"2024-09-21T06:04:32.794349+0000","last_peered":"2024-09-21T06:04:32.794349+0000","last_clean":"2024-09-21T06:04:32.794349+0000","last_became_active":"2024-09-21T06:04:31.772653+0000","last_became_peered":"2024-09-21T06:04:31.772653+0000","last_unstale":"2024-09-21T06:04:32.794349+0000","last_undegraded":"2024-09-21T06:04:32.794349+0000","last_fullsized":"2024-09-21T06:04:32.794349+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:59.868880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793482+0000","last_change":"2024-09-21T06:04:32.785502+0000","last_active":"2024-09-21T06:04:32.793482+0000","last_peered":"2024-09-21T06:04:32.793482+0000","last_clean":"2024-09-21T06:04:32.793482+0000","last_became_active":"2024-09-21T06:04:32.785091+0000","last_became_peered":"2024-09-21T06:04:32.785091+0000","last_unstale":"2024-09-21T06:04:32.793482+0000","last_undegraded":"2024-09-21T06:04:32.793482+0000","last_fullsized":"2024-09-21T06:04:32.793482+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:11.518104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.788989+0000","last_change":"2024-09-21T06:04:31.775276+0000","last_active":"2024-09-21T06:04:32.788989+0000","last_peered":"2024-09-21T06:04:32.788989+0000","last_clean":"2024-09-21T06:04:32.788989+0000","last_became_active":"2024-09-21T06:04:31.774028+0000","last_became_peered":"2024-09-21T06:04:31.774028+0000","last_unstale":"2024-09-21T06:04:32.788989+0000","last_undegraded":"2024-09-21T06:04:32.788989+0000","last_fullsized":"2024-09-21T06:04:32.788989+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:18.843912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783561+0000","last_change":"2024-09-21T06:04:32.777770+0000","last_active":"2024-09-21T06:04:32.783561+0000","last_peered":"2024-09-21T06:04:32.783561+0000","last_clean":"2024-09-21T06:04:32.783561+0000","last_became_active":"2024-09-21T06:04:32.777019+0000","last_became_peered":"2024-09-21T06:04:32.777019+0000","last_unstale":"2024-09-21T06:04:32.783561+0000","last_undegraded":"2024-09-21T06:04:32.783561+0000","last_fullsized":"2024-09-21T06:04:32.783561+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:59.802808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793522+0000","last_change":"2024-09-21T06:04:31.766081+0000","last_active":"2024-09-21T06:04:32.793522+0000","last_peered":"2024-09-21T06:04:32.793522+0000","last_clean":"2024-09-21T06:04:32.793522+0000","last_became_active":"2024-09-21T06:04:31.765541+0000","last_became_peered":"2024-09-21T06:04:31.765541+0000","last_unstale":"2024-09-21T06:04:32.793522+0000","last_undegraded":"2024-09-21T06:04:32.793522+0000","last_fullsized":"2024-09-21T06:04:32.793522+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:40.183984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792327+0000","last_change":"2024-09-21T06:04:32.780478+0000","last_active":"2024-09-21T06:04:32.792327+0000","last_peered":"2024-09-21T06:04:32.792327+0000","last_clean":"2024-09-21T06:04:32.792327+0000","last_became_active":"2024-09-21T06:04:32.778124+0000","last_became_peered":"2024-09-21T06:04:32.778124+0000","last_unstale":"2024-09-21T06:04:32.792327+0000","last_undegraded":"2024-09-21T06:04:32.792327+0000","last_fullsized":"2024-09-21T06:04:32.792327+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:26.746417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.787610+0000","last_change":"2024-09-21T06:04:31.772094+0000","last_active":"2024-09-21T06:04:32.787610+0000","last_peered":"2024-09-21T06:04:32.787610+0000","last_clean":"2024-09-21T06:04:32.787610+0000","last_became_active":"2024-09-21T06:04:31.771544+0000","last_became_peered":"2024-09-21T06:04:31.771544+0000","last_unstale":"2024-09-21T06:04:32.787610+0000","last_undegraded":"2024-09-21T06:04:32.787610+0000","last_fullsized":"2024-09-21T06:04:32.787610+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:03.600601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.767075+0000","last_change":"2024-09-21T06:04:32.761010+0000","last_active":"2024-09-21T06:04:32.767075+0000","last_peered":"2024-09-21T06:04:32.767075+0000","last_clean":"2024-09-21T06:04:32.767075+0000","last_became_active":"2024-09-21T06:04:32.760708+0000","last_became_peered":"2024-09-21T06:04:32.760708+0000","last_unstale":"2024-09-21T06:04:32.767075+0000","last_undegraded":"2024-09-21T06:04:32.767075+0000","last_fullsized":"2024-09-21T06:04:32.767075+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:45.868263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.769744+0000","last_change":"2024-09-21T06:04:31.768175+0000","last_active":"2024-09-21T06:04:31.769744+0000","last_peered":"2024-09-21T06:04:31.769744+0000","last_clean":"2024-09-21T06:04:31.769744+0000","last_became_active":"2024-09-21T06:04:31.767820+0000","last_became_peered":"2024-09-21T06:04:31.767820+0000","last_unstale":"2024-09-21T06:04:31.769744+0000","last_undegraded":"2024-09-21T06:04:31.769744+0000","last_fullsized":"2024-09-21T06:04:31.769744+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:24.459378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.775720+0000","last_change":"2024-09-21T06:04:32.772784+0000","last_active":"2024-09-21T06:04:32.775720+0000","last_peered":"2024-09-21T06:04:32.775720+0000","last_clean":"2024-09-21T06:04:32.775720+0000","last_became_active":"2024-09-21T06:04:32.772246+0000","last_became_peered":"2024-09-21T06:04:32.772246+0000","last_unstale":"2024-09-21T06:04:32.775720+0000","last_undegraded":"2024-09-21T06:04:32.775720+0000","last_fullsized":"2024-09-21T06:04:32.775720+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:39.036138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.752202+0000","last_change":"2024-09-21T06:04:31.759185+0000","last_active":"2024-09-21T06:04:32.752202+0000","last_peered":"2024-09-21T06:04:32.752202+0000","last_clean":"2024-09-21T06:04:32.752202+0000","last_became_active":"2024-09-21T06:04:31.758873+0000","last_became_peered":"2024-09-21T06:04:31.758873+0000","last_unstale":"2024-09-21T06:04:32.752202+0000","last_undegraded":"2024-09-21T06:04:32.752202+0000","last_fullsized":"2024-09-21T06:04:32.752202+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:37.617451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.770924+0000","last_change":"2024-09-21T06:04:32.769066+0000","last_active":"2024-09-21T06:04:32.770924+0000","last_peered":"2024-09-21T06:04:32.770924+0000","last_clean":"2024-09-21T06:04:32.770924+0000","last_became_active":"2024-09-21T06:04:32.768695+0000","last_became_peered":"2024-09-21T06:04:32.768695+0000","last_unstale":"2024-09-21T06:04:32.770924+0000","last_undegraded":"2024-09-21T06:04:32.770924+0000","last_fullsized":"2024-09-21T06:04:32.770924+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:41.742767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.755039+0000","last_change":"2024-09-21T06:04:31.770884+0000","last_active":"2024-09-21T06:04:32.755039+0000","last_peered":"2024-09-21T06:04:32.755039+0000","last_clean":"2024-09-21T06:04:32.755039+0000","last_became_active":"2024-09-21T06:04:31.768869+0000","last_became_peered":"2024-09-21T06:04:31.768869+0000","last_unstale":"2024-09-21T06:04:32.755039+0000","last_undegraded":"2024-09-21T06:04:32.755039+0000","last_fullsized":"2024-09-21T06:04:32.755039+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:18.843912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794408+0000","last_change":"2024-09-21T06:04:32.791807+0000","last_active":"2024-09-21T06:04:32.794408+0000","last_peered":"2024-09-21T06:04:32.794408+0000","last_clean":"2024-09-21T06:04:32.794408+0000","last_became_active":"2024-09-21T06:04:32.791461+0000","last_became_peered":"2024-09-21T06:04:32.791461+0000","last_unstale":"2024-09-21T06:04:32.794408+0000","last_undegraded":"2024-09-21T06:04:32.794408+0000","last_fullsized":"2024-09-21T06:04:32.794408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:47.832984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.766601+0000","last_change":"2024-09-21T06:04:31.770081+0000","last_active":"2024-09-21T06:04:32.766601+0000","last_peered":"2024-09-21T06:04:32.766601+0000","last_clean":"2024-09-21T06:04:32.766601+0000","last_became_active":"2024-09-21T06:04:31.768462+0000","last_became_peered":"2024-09-21T06:04:31.768462+0000","last_unstale":"2024-09-21T06:04:32.766601+0000","last_undegraded":"2024-09-21T06:04:32.766601+0000","last_fullsized":"2024-09-21T06:04:32.766601+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:10.542640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.796309+0000","last_change":"2024-09-21T06:04:32.795515+0000","last_active":"2024-09-21T06:04:32.796309+0000","last_peered":"2024-09-21T06:04:32.796309+0000","last_clean":"2024-09-21T06:04:32.796309+0000","last_became_active":"2024-09-21T06:04:32.795019+0000","last_became_peered":"2024-09-21T06:04:32.795019+0000","last_unstale":"2024-09-21T06:04:32.796309+0000","last_undegraded":"2024-09-21T06:04:32.796309+0000","last_fullsized":"2024-09-21T06:04:32.796309+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:26.746417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792146+0000","last_change":"2024-09-21T06:04:31.769723+0000","last_active":"2024-09-21T06:04:32.792146+0000","last_peered":"2024-09-21T06:04:32.792146+0000","last_clean":"2024-09-21T06:04:32.792146+0000","last_became_active":"2024-09-21T06:04:31.769344+0000","last_became_peered":"2024-09-21T06:04:31.769344+0000","last_unstale":"2024-09-21T06:04:32.792146+0000","last_undegraded":"2024-09-21T06:04:32.792146+0000","last_fullsized":"2024-09-21T06:04:32.792146+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:00.566146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.768354+0000","last_change":"2024-09-21T06:04:32.767569+0000","last_active":"2024-09-21T06:04:32.768354+0000","last_peered":"2024-09-21T06:04:32.768354+0000","last_clean":"2024-09-21T06:04:32.768354+0000","last_became_active":"2024-09-21T06:04:32.766934+0000","last_became_peered":"2024-09-21T06:04:32.766934+0000","last_unstale":"2024-09-21T06:04:32.768354+0000","last_undegraded":"2024-09-21T06:04:32.768354+0000","last_fullsized":"2024-09-21T06:04:32.768354+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:56.103744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.770166+0000","last_change":"2024-09-21T06:04:31.768773+0000","last_active":"2024-09-21T06:04:31.770166+0000","last_peered":"2024-09-21T06:04:31.770166+0000","last_clean":"2024-09-21T06:04:31.770166+0000","last_became_active":"2024-09-21T06:04:31.767292+0000","last_became_peered":"2024-09-21T06:04:31.767292+0000","last_unstale":"2024-09-21T06:04:31.770166+0000","last_undegraded":"2024-09-21T06:04:31.770166+0000","last_fullsized":"2024-09-21T06:04:31.770166+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:32.705430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792225+0000","last_change":"2024-09-21T06:04:32.784585+0000","last_active":"2024-09-21T06:04:32.792225+0000","last_peered":"2024-09-21T06:04:32.792225+0000","last_clean":"2024-09-21T06:04:32.792225+0000","last_became_active":"2024-09-21T06:04:32.784259+0000","last_became_peered":"2024-09-21T06:04:32.784259+0000","last_unstale":"2024-09-21T06:04:32.792225+0000","last_undegraded":"2024-09-21T06:04:32.792225+0000","last_fullsized":"2024-09-21T06:04:32.792225+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:11.446707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.773742+0000","last_change":"2024-09-21T06:04:31.772181+0000","last_active":"2024-09-21T06:04:31.773742+0000","last_peered":"2024-09-21T06:04:31.773742+0000","last_clean":"2024-09-21T06:04:31.773742+0000","last_became_active":"2024-09-21T06:04:31.771816+0000","last_became_peered":"2024-09-21T06:04:31.771816+0000","last_unstale":"2024-09-21T06:04:31.773742+0000","last_undegraded":"2024-09-21T06:04:31.773742+0000","last_fullsized":"2024-09-21T06:04:31.773742+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:47.864470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.770503+0000","last_change":"2024-09-21T06:04:32.768202+0000","last_active":"2024-09-21T06:04:32.770503+0000","last_peered":"2024-09-21T06:04:32.770503+0000","last_clean":"2024-09-21T06:04:32.770503+0000","last_became_active":"2024-09-21T06:04:32.767810+0000","last_became_peered":"2024-09-21T06:04:32.767810+0000","last_unstale":"2024-09-21T06:04:32.770503+0000","last_undegraded":"2024-09-21T06:04:32.770503+0000","last_fullsized":"2024-09-21T06:04:32.770503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:11.518104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.763381+0000","last_change":"2024-09-21T06:04:31.775115+0000","last_active":"2024-09-21T06:04:32.763381+0000","last_peered":"2024-09-21T06:04:32.763381+0000","last_clean":"2024-09-21T06:04:32.763381+0000","last_became_active":"2024-09-21T06:04:31.773039+0000","last_became_peered":"2024-09-21T06:04:31.773039+0000","last_unstale":"2024-09-21T06:04:32.763381+0000","last_undegraded":"2024-09-21T06:04:32.763381+0000","last_fullsized":"2024-09-21T06:04:32.763381+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:54.006741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781404+0000","last_change":"2024-09-21T06:04:32.780256+0000","last_active":"2024-09-21T06:04:32.781404+0000","last_peered":"2024-09-21T06:04:32.781404+0000","last_clean":"2024-09-21T06:04:32.781404+0000","last_became_active":"2024-09-21T06:04:32.779924+0000","last_became_peered":"2024-09-21T06:04:32.779924+0000","last_unstale":"2024-09-21T06:04:32.781404+0000","last_undegraded":"2024-09-21T06:04:32.781404+0000","last_fullsized":"2024-09-21T06:04:32.781404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:52.831924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.756793+0000","last_change":"2024-09-21T06:04:31.775645+0000","last_active":"2024-09-21T06:04:32.756793+0000","last_peered":"2024-09-21T06:04:32.756793+0000","last_clean":"2024-09-21T06:04:32.756793+0000","last_became_active":"2024-09-21T06:04:31.775129+0000","last_became_peered":"2024-09-21T06:04:31.775129+0000","last_unstale":"2024-09-21T06:04:32.756793+0000","last_undegraded":"2024-09-21T06:04:32.756793+0000","last_fullsized":"2024-09-21T06:04:32.756793+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:55.569554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.771789+0000","last_change":"2024-09-21T06:04:32.770814+0000","last_active":"2024-09-21T06:04:32.771789+0000","last_peered":"2024-09-21T06:04:32.771789+0000","last_clean":"2024-09-21T06:04:32.771789+0000","last_became_active":"2024-09-21T06:04:32.770411+0000","last_became_peered":"2024-09-21T06:04:32.770411+0000","last_unstale":"2024-09-21T06:04:32.771789+0000","last_undegraded":"2024-09-21T06:04:32.771789+0000","last_fullsized":"2024-09-21T06:04:32.771789+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:17.290747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793295+0000","last_change":"2024-09-21T06:04:31.772137+0000","last_active":"2024-09-21T06:04:32.793295+0000","last_peered":"2024-09-21T06:04:32.793295+0000","last_clean":"2024-09-21T06:04:32.793295+0000","last_became_active":"2024-09-21T06:04:31.771622+0000","last_became_peered":"2024-09-21T06:04:31.771622+0000","last_unstale":"2024-09-21T06:04:32.793295+0000","last_undegraded":"2024-09-21T06:04:32.793295+0000","last_fullsized":"2024-09-21T06:04:32.793295+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:38.991457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.769708+0000","last_change":"2024-09-21T06:04:32.768582+0000","last_active":"2024-09-21T06:04:32.769708+0000","last_peered":"2024-09-21T06:04:32.769708+0000","last_clean":"2024-09-21T06:04:32.769708+0000","last_became_active":"2024-09-21T06:04:32.767918+0000","last_became_peered":"2024-09-21T06:04:32.767918+0000","last_unstale":"2024-09-21T06:04:32.769708+0000","last_undegraded":"2024-09-21T06:04:32.769708+0000","last_fullsized":"2024-09-21T06:04:32.769708+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:49.109988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.786154+0000","last_change":"2024-09-21T06:04:31.775324+0000","last_active":"2024-09-21T06:04:32.786154+0000","last_peered":"2024-09-21T06:04:32.786154+0000","last_clean":"2024-09-21T06:04:32.786154+0000","last_became_active":"2024-09-21T06:04:31.774989+0000","last_became_peered":"2024-09-21T06:04:31.774989+0000","last_unstale":"2024-09-21T06:04:32.786154+0000","last_undegraded":"2024-09-21T06:04:32.786154+0000","last_fullsized":"2024-09-21T06:04:32.786154+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:10.782962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793956+0000","last_change":"2024-09-21T06:04:32.787340+0000","last_active":"2024-09-21T06:04:32.793956+0000","last_peered":"2024-09-21T06:04:32.793956+0000","last_clean":"2024-09-21T06:04:32.793956+0000","last_became_active":"2024-09-21T06:04:32.787027+0000","last_became_peered":"2024-09-21T06:04:32.787027+0000","last_unstale":"2024-09-21T06:04:32.793956+0000","last_undegraded":"2024-09-21T06:04:32.793956+0000","last_fullsized":"2024-09-21T06:04:32.793956+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:37.388005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.791893+0000","last_change":"2024-09-21T06:04:31.774380+0000","last_active":"2024-09-21T06:04:32.791893+0000","last_peered":"2024-09-21T06:04:32.791893+0000","last_clean":"2024-09-21T06:04:32.791893+0000","last_became_active":"2024-09-21T06:04:31.773835+0000","last_became_peered":"2024-09-21T06:04:31.773835+0000","last_unstale":"2024-09-21T06:04:32.791893+0000","last_undegraded":"2024-09-21T06:04:32.791893+0000","last_fullsized":"2024-09-21T06:04:32.791893+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:53.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.759154+0000","last_change":"2024-09-21T06:04:31.759154+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.759154+0000","last_undegraded":"2024-09-21T06:04:31.759154+0000","last_fullsized":"2024-09-21T06:04:31.759154+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.795017+0000","last_change":"2024-09-21T06:04:31.768595+0000","last_active":"2024-09-21T06:04:32.795017+0000","last_peered":"2024-09-21T06:04:32.795017+0000","last_clean":"2024-09-21T06:04:32.795017+0000","last_became_active":"2024-09-21T06:04:31.767603+0000","last_became_peered":"2024-09-21T06:04:31.767603+0000","last_unstale":"2024-09-21T06:04:32.795017+0000","last_undegraded":"2024-09-21T06:04:32.795017+0000","last_fullsized":"2024-09-21T06:04:32.795017+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:53.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.774981+0000","last_change":"2024-09-21T06:04:32.774290+0000","last_active":"2024-09-21T06:04:32.774981+0000","last_peered":"2024-09-21T06:04:32.774981+0000","last_clean":"2024-09-21T06:04:32.774981+0000","last_became_active":"2024-09-21T06:04:32.774023+0000","last_became_peered":"2024-09-21T06:04:32.774023+0000","last_unstale":"2024-09-21T06:04:32.774981+0000","last_undegraded":"2024-09-21T06:04:32.774981+0000","last_fullsized":"2024-09-21T06:04:32.774981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.944676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.774811+0000","last_change":"2024-09-21T06:04:31.774500+0000","last_active":"2024-09-21T06:04:32.774811+0000","last_peered":"2024-09-21T06:04:32.774811+0000","last_clean":"2024-09-21T06:04:32.774811+0000","last_became_active":"2024-09-21T06:04:31.774161+0000","last_became_peered":"2024-09-21T06:04:31.774161+0000","last_unstale":"2024-09-21T06:04:32.774811+0000","last_undegraded":"2024-09-21T06:04:32.774811+0000","last_fullsized":"2024-09-21T06:04:32.774811+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:32.288314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.773796+0000","last_change":"2024-09-21T06:04:32.769815+0000","last_active":"2024-09-21T06:04:32.773796+0000","last_peered":"2024-09-21T06:04:32.773796+0000","last_clean":"2024-09-21T06:04:32.773796+0000","last_became_active":"2024-09-21T06:04:32.769467+0000","last_became_peered":"2024-09-21T06:04:32.769467+0000","last_unstale":"2024-09-21T06:04:32.773796+0000","last_undegraded":"2024-09-21T06:04:32.773796+0000","last_fullsized":"2024-09-21T06:04:32.773796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:44.954801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.763651+0000","last_change":"2024-09-21T06:04:31.772131+0000","last_active":"2024-09-21T06:04:32.763651+0000","last_peered":"2024-09-21T06:04:32.763651+0000","last_clean":"2024-09-21T06:04:32.763651+0000","last_became_active":"2024-09-21T06:04:31.771792+0000","last_became_peered":"2024-09-21T06:04:31.771792+0000","last_unstale":"2024-09-21T06:04:32.763651+0000","last_undegraded":"2024-09-21T06:04:32.763651+0000","last_fullsized":"2024-09-21T06:04:32.763651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.561776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783512+0000","last_change":"2024-09-21T06:04:32.782337+0000","last_active":"2024-09-21T06:04:32.783512+0000","last_peered":"2024-09-21T06:04:32.783512+0000","last_clean":"2024-09-21T06:04:32.783512+0000","last_became_active":"2024-09-21T06:04:32.782011+0000","last_became_peered":"2024-09-21T06:04:32.782011+0000","last_unstale":"2024-09-21T06:04:32.783512+0000","last_undegraded":"2024-09-21T06:04:32.783512+0000","last_fullsized":"2024-09-21T06:04:32.783512+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:45.735018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.757070+0000","last_change":"2024-09-21T06:04:31.766119+0000","last_active":"2024-09-21T06:04:32.757070+0000","last_peered":"2024-09-21T06:04:32.757070+0000","last_clean":"2024-09-21T06:04:32.757070+0000","last_became_active":"2024-09-21T06:04:31.765398+0000","last_became_peered":"2024-09-21T06:04:31.765398+0000","last_unstale":"2024-09-21T06:04:32.757070+0000","last_undegraded":"2024-09-21T06:04:32.757070+0000","last_fullsized":"2024-09-21T06:04:32.757070+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:23.042000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781363+0000","last_change":"2024-09-21T06:04:32.777061+0000","last_active":"2024-09-21T06:04:32.781363+0000","last_peered":"2024-09-21T06:04:32.781363+0000","last_clean":"2024-09-21T06:04:32.781363+0000","last_became_active":"2024-09-21T06:04:32.776703+0000","last_became_peered":"2024-09-21T06:04:32.776703+0000","last_unstale":"2024-09-21T06:04:32.781363+0000","last_undegraded":"2024-09-21T06:04:32.781363+0000","last_fullsized":"2024-09-21T06:04:32.781363+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:12.555174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.756499+0000","last_change":"2024-09-21T06:04:31.770508+0000","last_active":"2024-09-21T06:04:32.756499+0000","last_peered":"2024-09-21T06:04:32.756499+0000","last_clean":"2024-09-21T06:04:32.756499+0000","last_became_active":"2024-09-21T06:04:31.769020+0000","last_became_peered":"2024-09-21T06:04:31.769020+0000","last_unstale":"2024-09-21T06:04:32.756499+0000","last_undegraded":"2024-09-21T06:04:32.756499+0000","last_fullsized":"2024-09-21T06:04:32.756499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:40.893581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.765322+0000","last_change":"2024-09-21T06:04:32.762522+0000","last_active":"2024-09-21T06:04:32.765322+0000","last_peered":"2024-09-21T06:04:32.765322+0000","last_clean":"2024-09-21T06:04:32.765322+0000","last_became_active":"2024-09-21T06:04:32.762144+0000","last_became_peered":"2024-09-21T06:04:32.762144+0000","last_unstale":"2024-09-21T06:04:32.765322+0000","last_undegraded":"2024-09-21T06:04:32.765322+0000","last_fullsized":"2024-09-21T06:04:32.765322+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:36.679619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.786027+0000","last_change":"2024-09-21T06:04:31.774958+0000","last_active":"2024-09-21T06:04:32.786027+0000","last_peered":"2024-09-21T06:04:32.786027+0000","last_clean":"2024-09-21T06:04:32.786027+0000","last_became_active":"2024-09-21T06:04:31.774595+0000","last_became_peered":"2024-09-21T06:04:31.774595+0000","last_unstale":"2024-09-21T06:04:32.786027+0000","last_undegraded":"2024-09-21T06:04:32.786027+0000","last_fullsized":"2024-09-21T06:04:32.786027+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:05.289920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.770815+0000","last_change":"2024-09-21T06:04:32.766158+0000","last_active":"2024-09-21T06:04:32.770815+0000","last_peered":"2024-09-21T06:04:32.770815+0000","last_clean":"2024-09-21T06:04:32.770815+0000","last_became_active":"2024-09-21T06:04:32.765811+0000","last_became_peered":"2024-09-21T06:04:32.765811+0000","last_unstale":"2024-09-21T06:04:32.770815+0000","last_undegraded":"2024-09-21T06:04:32.770815+0000","last_fullsized":"2024-09-21T06:04:32.770815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:11.446707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.757924+0000","last_change":"2024-09-21T06:04:31.775745+0000","last_active":"2024-09-21T06:04:32.757924+0000","last_peered":"2024-09-21T06:04:32.757924+0000","last_clean":"2024-09-21T06:04:32.757924+0000","last_became_active":"2024-09-21T06:04:31.775037+0000","last_became_peered":"2024-09-21T06:04:31.775037+0000","last_unstale":"2024-09-21T06:04:32.757924+0000","last_undegraded":"2024-09-21T06:04:32.757924+0000","last_fullsized":"2024-09-21T06:04:32.757924+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:40.310390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794553+0000","last_change":"2024-09-21T06:04:32.791087+0000","last_active":"2024-09-21T06:04:32.794553+0000","last_peered":"2024-09-21T06:04:32.794553+0000","last_clean":"2024-09-21T06:04:32.794553+0000","last_became_active":"2024-09-21T06:04:32.790735+0000","last_became_peered":"2024-09-21T06:04:32.790735+0000","last_unstale":"2024-09-21T06:04:32.794553+0000","last_undegraded":"2024-09-21T06:04:32.794553+0000","last_fullsized":"2024-09-21T06:04:32.794553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:30.386322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.751374+0000","last_change":"2024-09-21T06:04:31.761815+0000","last_active":"2024-09-21T06:04:32.751374+0000","last_peered":"2024-09-21T06:04:32.751374+0000","last_clean":"2024-09-21T06:04:32.751374+0000","last_became_active":"2024-09-21T06:04:31.761487+0000","last_became_peered":"2024-09-21T06:04:31.761487+0000","last_unstale":"2024-09-21T06:04:32.751374+0000","last_undegraded":"2024-09-21T06:04:32.751374+0000","last_fullsized":"2024-09-21T06:04:32.751374+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:02:35.169801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794037+0000","last_change":"2024-09-21T06:04:32.788888+0000","last_active":"2024-09-21T06:04:32.794037+0000","last_peered":"2024-09-21T06:04:32.794037+0000","last_clean":"2024-09-21T06:04:32.794037+0000","last_became_active":"2024-09-21T06:04:32.788597+0000","last_became_peered":"2024-09-21T06:04:32.788597+0000","last_unstale":"2024-09-21T06:04:32.794037+0000","last_undegraded":"2024-09-21T06:04:32.794037+0000","last_fullsized":"2024-09-21T06:04:32.794037+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:12.183155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.761494+0000","last_change":"2024-09-21T06:04:31.762529+0000","last_active":"2024-09-21T06:04:32.761494+0000","last_peered":"2024-09-21T06:04:32.761494+0000","last_clean":"2024-09-21T06:04:32.761494+0000","last_became_active":"2024-09-21T06:04:31.762189+0000","last_became_peered":"2024-09-21T06:04:31.762189+0000","last_unstale":"2024-09-21T06:04:32.761494+0000","last_undegraded":"2024-09-21T06:04:32.761494+0000","last_fullsized":"2024-09-21T06:04:32.761494+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:51.822586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.773682+0000","last_change":"2024-09-21T06:04:32.770727+0000","last_active":"2024-09-21T06:04:32.773682+0000","last_peered":"2024-09-21T06:04:32.773682+0000","last_clean":"2024-09-21T06:04:32.773682+0000","last_became_active":"2024-09-21T06:04:32.770393+0000","last_became_peered":"2024-09-21T06:04:32.770393+0000","last_unstale":"2024-09-21T06:04:32.773682+0000","last_undegraded":"2024-09-21T06:04:32.773682+0000","last_fullsized":"2024-09-21T06:04:32.773682+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:23.599523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.751186+0000","last_change":"2024-09-21T06:04:31.768859+0000","last_active":"2024-09-21T06:04:32.751186+0000","last_peered":"2024-09-21T06:04:32.751186+0000","last_clean":"2024-09-21T06:04:32.751186+0000","last_became_active":"2024-09-21T06:04:31.768535+0000","last_became_peered":"2024-09-21T06:04:31.768535+0000","last_unstale":"2024-09-21T06:04:32.751186+0000","last_undegraded":"2024-09-21T06:04:32.751186+0000","last_fullsized":"2024-09-21T06:04:32.751186+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:05.572103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.786505+0000","last_change":"2024-09-21T06:04:32.782856+0000","last_active":"2024-09-21T06:04:32.786505+0000","last_peered":"2024-09-21T06:04:32.786505+0000","last_clean":"2024-09-21T06:04:32.786505+0000","last_became_active":"2024-09-21T06:04:32.782532+0000","last_became_peered":"2024-09-21T06:04:32.782532+0000","last_unstale":"2024-09-21T06:04:32.786505+0000","last_undegraded":"2024-09-21T06:04:32.786505+0000","last_fullsized":"2024-09-21T06:04:32.786505+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:49.109988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.770750+0000","last_change":"2024-09-21T06:04:31.768251+0000","last_active":"2024-09-21T06:04:31.770750+0000","last_peered":"2024-09-21T06:04:31.770750+0000","last_clean":"2024-09-21T06:04:31.770750+0000","last_became_active":"2024-09-21T06:04:31.767918+0000","last_became_peered":"2024-09-21T06:04:31.767918+0000","last_unstale":"2024-09-21T06:04:31.770750+0000","last_undegraded":"2024-09-21T06:04:31.770750+0000","last_fullsized":"2024-09-21T06:04:31.770750+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:28.832034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.760288+0000","last_change":"2024-09-21T06:04:31.760288+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.760288+0000","last_undegraded":"2024-09-21T06:04:31.760288+0000","last_fullsized":"2024-09-21T06:04:31.760288+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.755302+0000","last_change":"2024-09-21T06:04:31.769840+0000","last_active":"2024-09-21T06:04:32.755302+0000","last_peered":"2024-09-21T06:04:32.755302+0000","last_clean":"2024-09-21T06:04:32.755302+0000","last_became_active":"2024-09-21T06:04:31.769534+0000","last_became_peered":"2024-09-21T06:04:31.769534+0000","last_unstale":"2024-09-21T06:04:32.755302+0000","last_undegraded":"2024-09-21T06:04:32.755302+0000","last_fullsized":"2024-09-21T06:04:32.755302+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:10.542640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.784911+0000","last_change":"2024-09-21T06:04:32.779505+0000","last_active":"2024-09-21T06:04:32.784911+0000","last_peered":"2024-09-21T06:04:32.784911+0000","last_clean":"2024-09-21T06:04:32.784911+0000","last_became_active":"2024-09-21T06:04:32.779079+0000","last_became_peered":"2024-09-21T06:04:32.779079+0000","last_unstale":"2024-09-21T06:04:32.784911+0000","last_undegraded":"2024-09-21T06:04:32.784911+0000","last_fullsized":"2024-09-21T06:04:32.784911+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:37.256481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.765120+0000","last_change":"2024-09-21T06:04:31.768166+0000","last_active":"2024-09-21T06:04:32.765120+0000","last_peered":"2024-09-21T06:04:32.765120+0000","last_clean":"2024-09-21T06:04:32.765120+0000","last_became_active":"2024-09-21T06:04:31.767622+0000","last_became_peered":"2024-09-21T06:04:31.767622+0000","last_unstale":"2024-09-21T06:04:32.765120+0000","last_undegraded":"2024-09-21T06:04:32.765120+0000","last_fullsized":"2024-09-21T06:04:32.765120+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:14.758437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.780297+0000","last_change":"2024-09-21T06:04:32.776172+0000","last_active":"2024-09-21T06:04:32.780297+0000","last_peered":"2024-09-21T06:04:32.780297+0000","last_clean":"2024-09-21T06:04:32.780297+0000","last_became_active":"2024-09-21T06:04:32.775877+0000","last_became_peered":"2024-09-21T06:04:32.775877+0000","last_unstale":"2024-09-21T06:04:32.780297+0000","last_undegraded":"2024-09-21T06:04:32.780297+0000","last_fullsized":"2024-09-21T06:04:32.780297+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:54.870810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.788947+0000","last_change":"2024-09-21T06:04:31.772282+0000","last_active":"2024-09-21T06:04:32.788947+0000","last_peered":"2024-09-21T06:04:32.788947+0000","last_clean":"2024-09-21T06:04:32.788947+0000","last_became_active":"2024-09-21T06:04:31.770583+0000","last_became_peered":"2024-09-21T06:04:31.770583+0000","last_unstale":"2024-09-21T06:04:32.788947+0000","last_undegraded":"2024-09-21T06:04:32.788947+0000","last_fullsized":"2024-09-21T06:04:32.788947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:35.915894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":201,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.775042+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:04:32.775042+0000","last_peered":"2024-09-21T06:04:32.775042+0000","last_clean":"2024-09-21T06:04:32.775042+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:04:32.775042+0000","last_undegraded":"2024-09-21T06:04:32.775042+0000","last_fullsized":"2024-09-21T06:04:32.775042+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50656,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321184,"statfs":{"total":96636764160,"available":96584892416,"internally_reserved":0,"allocated":1445888,"data_stored":944799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509542,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":851968,"data_stored":353907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509541,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":847872,"data_stored":353905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50064,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321776,"statfs":{"total":96636764160,"available":96585498624,"internally_reserved":0,"allocated":847872,"data_stored":353885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54232,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317608,"statfs":{"total":96636764160,"available":96581230592,"internally_reserved":0,"allocated":851968,"data_stored":353927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50048,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321792,"statfs":{"total":96636764160,"available":96585515008,"internally_reserved":0,"allocated":831488,"data_stored":347012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509541,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54788,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317052,"statfs":{"total":96636764160,"available":96580661248,"internally_reserved":0,"allocated":1425408,"data_stored":937380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58368,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313472,"statfs":{"total":96636764160,"available":96576995328,"internally_reserved":0,"allocated":831488,"data_stored":347012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:04:36.969 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:04:37.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.163+0000 7f0f22400640 1 -- 172.21.15.50:0/2575385195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f1c1527b0 msgr2=0x7f0f1c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:37.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.163+0000 7f0f22400640 1 --2- 172.21.15.50:0/2575385195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f1c1527b0 0x7f0f1c074e30 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c071cb0 tx=0x7f0f0401c890 comp rx=0 tx=0).stop 2024-09-21T06:04:37.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.163+0000 7f0f22400640 1 -- 172.21.15.50:0/2575385195 shutdown_connections 2024-09-21T06:04:37.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.163+0000 7f0f22400640 1 --2- 172.21.15.50:0/2575385195 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f1c0b4de0 0x7f0f1c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.163+0000 7f0f22400640 1 --2- 172.21.15.50:0/2575385195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f1c1527b0 0x7f0f1c074e30 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.163+0000 7f0f22400640 1 --2- 172.21.15.50:0/2575385195 >> v2:172.21.15.88:3300/0 conn(0x7f0f1c0b5c70 0x7f0f1c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.163+0000 7f0f22400640 1 -- 172.21.15.50:0/2575385195 >> 172.21.15.50:0/2575385195 conn(0x7f0f1c0b2040 msgr2=0x7f0f1c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:37.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.163+0000 7f0f22400640 1 -- 172.21.15.50:0/2575385195 shutdown_connections 2024-09-21T06:04:37.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.163+0000 7f0f22400640 1 -- 172.21.15.50:0/2575385195 wait complete. 2024-09-21T06:04:37.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.163+0000 7f0f22400640 1 Processor -- start 2024-09-21T06:04:37.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.163+0000 7f0f22400640 1 -- start start 2024-09-21T06:04:37.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f22400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f1c0b4de0 0x7f0f1c1590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:37.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f22400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f1c0b5c70 0x7f0f1c144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:37.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f22400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f1c1527b0 0x7f0f1c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:37.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f22400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c15f6b0 con 0x7f0f1c1527b0 2024-09-21T06:04:37.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f22400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c15f530 con 0x7f0f1c0b4de0 2024-09-21T06:04:37.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f22400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0f1c15f830 con 0x7f0f1c0b5c70 2024-09-21T06:04:37.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f1be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f1c1527b0 0x7f0f1c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:37.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f1be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f1c1527b0 0x7f0f1c0affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36646/0 (socket says 172.21.15.50:36646) 2024-09-21T06:04:37.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f1be00640 1 -- 172.21.15.50:0/1621470896 learned_addr learned my addr 172.21.15.50:0/1621470896 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:37.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f13e00640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f1c0b5c70 0x7f0f1c144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:37.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f1b400640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f1c0b4de0 0x7f0f1c1590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:37.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f1be00640 1 -- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f1c0b5c70 msgr2=0x7f0f1c144c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:37.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f1be00640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f1c0b5c70 0x7f0f1c144c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f1be00640 1 -- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f1c0b4de0 msgr2=0x7f0f1c1590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:37.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f1be00640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f1c0b4de0 0x7f0f1c1590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f1be00640 1 -- 172.21.15.50:0/1621470896 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1c145160 con 0x7f0f1c1527b0 2024-09-21T06:04:37.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f1be00640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f1c1527b0 0x7f0f1c0affb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400ea00 tx=0x7f0f1400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:37.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f13400640 1 -- 172.21.15.50:0/1621470896 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1400cde0 con 0x7f0f1c1527b0 2024-09-21T06:04:37.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f13400640 1 -- 172.21.15.50:0/1621470896 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f14004c20 con 0x7f0f1c1527b0 2024-09-21T06:04:37.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f13400640 1 -- 172.21.15.50:0/1621470896 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14005020 con 0x7f0f1c1527b0 2024-09-21T06:04:37.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f22400640 1 -- 172.21.15.50:0/1621470896 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1c1fc350 con 0x7f0f1c1527b0 2024-09-21T06:04:37.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f22400640 1 -- 172.21.15.50:0/1621470896 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f1c1fc910 con 0x7f0f1c1527b0 2024-09-21T06:04:37.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.167+0000 7f0f22400640 1 -- 172.21.15.50:0/1621470896 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f1c15fd50 con 0x7f0f1c1527b0 2024-09-21T06:04:37.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.171+0000 7f0f13400640 1 -- 172.21.15.50:0/1621470896 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f0f140051c0 con 0x7f0f1c1527b0 2024-09-21T06:04:37.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.171+0000 7f0f13400640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ef8073e60 0x7f0ef8076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:37.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.171+0000 7f0f13400640 1 -- 172.21.15.50:0/1621470896 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f0f14093840 con 0x7f0f1c1527b0 2024-09-21T06:04:37.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.171+0000 7f0f1b400640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ef8073e60 0x7f0ef8076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:37.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.171+0000 7f0f1b400640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ef8073e60 0x7f0ef8076370 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009b40 tx=0x7f0f0c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:37.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.175+0000 7f0f13400640 1 -- 172.21.15.50:0/1621470896 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0f1405fc90 con 0x7f0f1c1527b0 2024-09-21T06:04:37.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.347+0000 7f0f22400640 1 -- 172.21.15.50:0/1621470896 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f1c15e380 con 0x7f0ef8073e60 2024-09-21T06:04:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.355+0000 7f0f13400640 1 -- 172.21.15.50:0/1621470896 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251632 (secure 0 0 0) 0x7f0f1c15e380 con 0x7f0ef8073e60 2024-09-21T06:04:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:37.358 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:04:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.359+0000 7f0f22400640 1 -- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ef8073e60 msgr2=0x7f0ef8076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.359+0000 7f0f22400640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ef8073e60 0x7f0ef8076370 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009b40 tx=0x7f0f0c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.359+0000 7f0f22400640 1 -- 172.21.15.50:0/1621470896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f1c1527b0 msgr2=0x7f0f1c0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.359+0000 7f0f22400640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f1c1527b0 0x7f0f1c0affb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400ea00 tx=0x7f0f1400eec0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.359+0000 7f0f22400640 1 -- 172.21.15.50:0/1621470896 shutdown_connections 2024-09-21T06:04:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.359+0000 7f0f22400640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ef8073e60 0x7f0ef8076370 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.359+0000 7f0f22400640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f1c1527b0 0x7f0f1c0affb0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.359+0000 7f0f22400640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f1c0b5c70 0x7f0f1c144c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.359+0000 7f0f22400640 1 --2- 172.21.15.50:0/1621470896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f1c0b4de0 0x7f0f1c1590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.359+0000 7f0f22400640 1 -- 172.21.15.50:0/1621470896 >> 172.21.15.50:0/1621470896 conn(0x7f0f1c0b2040 msgr2=0x7f0f1c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.359+0000 7f0f22400640 1 -- 172.21.15.50:0/1621470896 shutdown_connections 2024-09-21T06:04:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.359+0000 7f0f22400640 1 -- 172.21.15.50:0/1621470896 wait complete. 2024-09-21T06:04:37.379 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-21T06:04:37.023099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":136,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":418300,"kb_used_data":7748,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754556420,"statfs":{"total":773094113280,"available":772665774080,"internally_reserved":0,"allocated":7933952,"data_stored":3991827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001380"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793750+0000","last_change":"2024-09-21T06:04:32.787310+0000","last_active":"2024-09-21T06:04:32.793750+0000","last_peered":"2024-09-21T06:04:32.793750+0000","last_clean":"2024-09-21T06:04:32.793750+0000","last_became_active":"2024-09-21T06:04:32.786715+0000","last_became_peered":"2024-09-21T06:04:32.786715+0000","last_unstale":"2024-09-21T06:04:32.793750+0000","last_undegraded":"2024-09-21T06:04:32.793750+0000","last_fullsized":"2024-09-21T06:04:32.793750+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:59.802808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.784907+0000","last_change":"2024-09-21T06:04:32.778255+0000","last_active":"2024-09-21T06:04:32.784907+0000","last_peered":"2024-09-21T06:04:32.784907+0000","last_clean":"2024-09-21T06:04:32.784907+0000","last_became_active":"2024-09-21T06:04:32.777803+0000","last_became_peered":"2024-09-21T06:04:32.777803+0000","last_unstale":"2024-09-21T06:04:32.784907+0000","last_undegraded":"2024-09-21T06:04:32.784907+0000","last_fullsized":"2024-09-21T06:04:32.784907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:54.763471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781150+0000","last_change":"2024-09-21T06:04:32.772652+0000","last_active":"2024-09-21T06:04:32.781150+0000","last_peered":"2024-09-21T06:04:32.781150+0000","last_clean":"2024-09-21T06:04:32.781150+0000","last_became_active":"2024-09-21T06:04:32.772322+0000","last_became_peered":"2024-09-21T06:04:32.772322+0000","last_unstale":"2024-09-21T06:04:32.781150+0000","last_undegraded":"2024-09-21T06:04:32.781150+0000","last_fullsized":"2024-09-21T06:04:32.781150+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:36.624434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.762434+0000","last_change":"2024-09-21T06:04:31.762434+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.762434+0000","last_undegraded":"2024-09-21T06:04:31.762434+0000","last_fullsized":"2024-09-21T06:04:31.762434+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783599+0000","last_change":"2024-09-21T06:04:32.777265+0000","last_active":"2024-09-21T06:04:32.783599+0000","last_peered":"2024-09-21T06:04:32.783599+0000","last_clean":"2024-09-21T06:04:32.783599+0000","last_became_active":"2024-09-21T06:04:32.776943+0000","last_became_peered":"2024-09-21T06:04:32.776943+0000","last_unstale":"2024-09-21T06:04:32.783599+0000","last_undegraded":"2024-09-21T06:04:32.783599+0000","last_fullsized":"2024-09-21T06:04:32.783599+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:05.622248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.796017+0000","last_change":"2024-09-21T06:04:32.794992+0000","last_active":"2024-09-21T06:04:32.796017+0000","last_peered":"2024-09-21T06:04:32.796017+0000","last_clean":"2024-09-21T06:04:32.796017+0000","last_became_active":"2024-09-21T06:04:32.794580+0000","last_became_peered":"2024-09-21T06:04:32.794580+0000","last_unstale":"2024-09-21T06:04:32.796017+0000","last_undegraded":"2024-09-21T06:04:32.796017+0000","last_fullsized":"2024-09-21T06:04:32.796017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:57.984993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.768680+0000","last_change":"2024-09-21T06:04:32.765123+0000","last_active":"2024-09-21T06:04:32.768680+0000","last_peered":"2024-09-21T06:04:32.768680+0000","last_clean":"2024-09-21T06:04:32.768680+0000","last_became_active":"2024-09-21T06:04:32.764446+0000","last_became_peered":"2024-09-21T06:04:32.764446+0000","last_unstale":"2024-09-21T06:04:32.768680+0000","last_undegraded":"2024-09-21T06:04:32.768680+0000","last_fullsized":"2024-09-21T06:04:32.768680+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:00.689363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783317+0000","last_change":"2024-09-21T06:04:32.773644+0000","last_active":"2024-09-21T06:04:32.783317+0000","last_peered":"2024-09-21T06:04:32.783317+0000","last_clean":"2024-09-21T06:04:32.783317+0000","last_became_active":"2024-09-21T06:04:32.771506+0000","last_became_peered":"2024-09-21T06:04:32.771506+0000","last_unstale":"2024-09-21T06:04:32.783317+0000","last_undegraded":"2024-09-21T06:04:32.783317+0000","last_fullsized":"2024-09-21T06:04:32.783317+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:39.994097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.778492+0000","last_change":"2024-09-21T06:04:32.775939+0000","last_active":"2024-09-21T06:04:32.778492+0000","last_peered":"2024-09-21T06:04:32.778492+0000","last_clean":"2024-09-21T06:04:32.778492+0000","last_became_active":"2024-09-21T06:04:32.775526+0000","last_became_peered":"2024-09-21T06:04:32.775526+0000","last_unstale":"2024-09-21T06:04:32.778492+0000","last_undegraded":"2024-09-21T06:04:32.778492+0000","last_fullsized":"2024-09-21T06:04:32.778492+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:17.326812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.769320+0000","last_change":"2024-09-21T06:04:32.768237+0000","last_active":"2024-09-21T06:04:32.769320+0000","last_peered":"2024-09-21T06:04:32.769320+0000","last_clean":"2024-09-21T06:04:32.769320+0000","last_became_active":"2024-09-21T06:04:32.767480+0000","last_became_peered":"2024-09-21T06:04:32.767480+0000","last_unstale":"2024-09-21T06:04:32.769320+0000","last_undegraded":"2024-09-21T06:04:32.769320+0000","last_fullsized":"2024-09-21T06:04:32.769320+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:37.256481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793616+0000","last_change":"2024-09-21T06:04:32.786243+0000","last_active":"2024-09-21T06:04:32.793616+0000","last_peered":"2024-09-21T06:04:32.793616+0000","last_clean":"2024-09-21T06:04:32.793616+0000","last_became_active":"2024-09-21T06:04:32.785278+0000","last_became_peered":"2024-09-21T06:04:32.785278+0000","last_unstale":"2024-09-21T06:04:32.793616+0000","last_undegraded":"2024-09-21T06:04:32.793616+0000","last_fullsized":"2024-09-21T06:04:32.793616+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:41.742767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.761947+0000","last_change":"2024-09-21T06:04:31.761947+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.761947+0000","last_undegraded":"2024-09-21T06:04:31.761947+0000","last_fullsized":"2024-09-21T06:04:31.761947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.775956+0000","last_change":"2024-09-21T06:04:32.774414+0000","last_active":"2024-09-21T06:04:32.775956+0000","last_peered":"2024-09-21T06:04:32.775956+0000","last_clean":"2024-09-21T06:04:32.775956+0000","last_became_active":"2024-09-21T06:04:32.774110+0000","last_became_peered":"2024-09-21T06:04:32.774110+0000","last_unstale":"2024-09-21T06:04:32.775956+0000","last_undegraded":"2024-09-21T06:04:32.775956+0000","last_fullsized":"2024-09-21T06:04:32.775956+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:11.805943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.766598+0000","last_change":"2024-09-21T06:04:32.760679+0000","last_active":"2024-09-21T06:04:32.766598+0000","last_peered":"2024-09-21T06:04:32.766598+0000","last_clean":"2024-09-21T06:04:32.766598+0000","last_became_active":"2024-09-21T06:04:32.760378+0000","last_became_peered":"2024-09-21T06:04:32.760378+0000","last_unstale":"2024-09-21T06:04:32.766598+0000","last_undegraded":"2024-09-21T06:04:32.766598+0000","last_fullsized":"2024-09-21T06:04:32.766598+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:40.885781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.761919+0000","last_change":"2024-09-21T06:04:32.760660+0000","last_active":"2024-09-21T06:04:32.761919+0000","last_peered":"2024-09-21T06:04:32.761919+0000","last_clean":"2024-09-21T06:04:32.761919+0000","last_became_active":"2024-09-21T06:04:32.760303+0000","last_became_peered":"2024-09-21T06:04:32.760303+0000","last_unstale":"2024-09-21T06:04:32.761919+0000","last_undegraded":"2024-09-21T06:04:32.761919+0000","last_fullsized":"2024-09-21T06:04:32.761919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:04.139988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793365+0000","last_change":"2024-09-21T06:04:32.783570+0000","last_active":"2024-09-21T06:04:32.793365+0000","last_peered":"2024-09-21T06:04:32.793365+0000","last_clean":"2024-09-21T06:04:32.793365+0000","last_became_active":"2024-09-21T06:04:32.783265+0000","last_became_peered":"2024-09-21T06:04:32.783265+0000","last_unstale":"2024-09-21T06:04:32.793365+0000","last_undegraded":"2024-09-21T06:04:32.793365+0000","last_fullsized":"2024-09-21T06:04:32.793365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:49.490698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.778520+0000","last_change":"2024-09-21T06:04:32.776322+0000","last_active":"2024-09-21T06:04:32.778520+0000","last_peered":"2024-09-21T06:04:32.778520+0000","last_clean":"2024-09-21T06:04:32.778520+0000","last_became_active":"2024-09-21T06:04:32.776027+0000","last_became_peered":"2024-09-21T06:04:32.776027+0000","last_unstale":"2024-09-21T06:04:32.778520+0000","last_undegraded":"2024-09-21T06:04:32.778520+0000","last_fullsized":"2024-09-21T06:04:32.778520+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:14.913586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.760440+0000","last_change":"2024-09-21T06:04:31.760440+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.760440+0000","last_undegraded":"2024-09-21T06:04:31.760440+0000","last_fullsized":"2024-09-21T06:04:31.760440+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.773202+0000","last_change":"2024-09-21T06:04:32.771994+0000","last_active":"2024-09-21T06:04:32.773202+0000","last_peered":"2024-09-21T06:04:32.773202+0000","last_clean":"2024-09-21T06:04:32.773202+0000","last_became_active":"2024-09-21T06:04:32.771597+0000","last_became_peered":"2024-09-21T06:04:32.771597+0000","last_unstale":"2024-09-21T06:04:32.773202+0000","last_undegraded":"2024-09-21T06:04:32.773202+0000","last_fullsized":"2024-09-21T06:04:32.773202+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:00.871520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.769001+0000","last_change":"2024-09-21T06:04:32.766745+0000","last_active":"2024-09-21T06:04:32.769001+0000","last_peered":"2024-09-21T06:04:32.769001+0000","last_clean":"2024-09-21T06:04:32.769001+0000","last_became_active":"2024-09-21T06:04:32.766362+0000","last_became_peered":"2024-09-21T06:04:32.766362+0000","last_unstale":"2024-09-21T06:04:32.769001+0000","last_undegraded":"2024-09-21T06:04:32.769001+0000","last_fullsized":"2024-09-21T06:04:32.769001+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:12.666560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.765784+0000","last_change":"2024-09-21T06:04:32.760980+0000","last_active":"2024-09-21T06:04:32.765784+0000","last_peered":"2024-09-21T06:04:32.765784+0000","last_clean":"2024-09-21T06:04:32.765784+0000","last_became_active":"2024-09-21T06:04:32.760648+0000","last_became_peered":"2024-09-21T06:04:32.760648+0000","last_unstale":"2024-09-21T06:04:32.765784+0000","last_undegraded":"2024-09-21T06:04:32.765784+0000","last_fullsized":"2024-09-21T06:04:32.765784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:58.412247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.767175+0000","last_change":"2024-09-21T06:04:32.762713+0000","last_active":"2024-09-21T06:04:32.767175+0000","last_peered":"2024-09-21T06:04:32.767175+0000","last_clean":"2024-09-21T06:04:32.767175+0000","last_became_active":"2024-09-21T06:04:32.762345+0000","last_became_peered":"2024-09-21T06:04:32.762345+0000","last_unstale":"2024-09-21T06:04:32.767175+0000","last_undegraded":"2024-09-21T06:04:32.767175+0000","last_fullsized":"2024-09-21T06:04:32.767175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:54.763471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793898+0000","last_change":"2024-09-21T06:04:32.788061+0000","last_active":"2024-09-21T06:04:32.793898+0000","last_peered":"2024-09-21T06:04:32.793898+0000","last_clean":"2024-09-21T06:04:32.793898+0000","last_became_active":"2024-09-21T06:04:32.787777+0000","last_became_peered":"2024-09-21T06:04:32.787777+0000","last_unstale":"2024-09-21T06:04:32.793898+0000","last_undegraded":"2024-09-21T06:04:32.793898+0000","last_fullsized":"2024-09-21T06:04:32.793898+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:53.442251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794388+0000","last_change":"2024-09-21T06:04:32.790938+0000","last_active":"2024-09-21T06:04:32.794388+0000","last_peered":"2024-09-21T06:04:32.794388+0000","last_clean":"2024-09-21T06:04:32.794388+0000","last_became_active":"2024-09-21T06:04:32.790267+0000","last_became_peered":"2024-09-21T06:04:32.790267+0000","last_unstale":"2024-09-21T06:04:32.794388+0000","last_undegraded":"2024-09-21T06:04:32.794388+0000","last_fullsized":"2024-09-21T06:04:32.794388+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:19.004848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.789947+0000","last_change":"2024-09-21T06:04:32.779963+0000","last_active":"2024-09-21T06:04:32.789947+0000","last_peered":"2024-09-21T06:04:32.789947+0000","last_clean":"2024-09-21T06:04:32.789947+0000","last_became_active":"2024-09-21T06:04:32.779659+0000","last_became_peered":"2024-09-21T06:04:32.779659+0000","last_unstale":"2024-09-21T06:04:32.789947+0000","last_undegraded":"2024-09-21T06:04:32.789947+0000","last_fullsized":"2024-09-21T06:04:32.789947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:08.847286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783071+0000","last_change":"2024-09-21T06:04:32.778117+0000","last_active":"2024-09-21T06:04:32.783071+0000","last_peered":"2024-09-21T06:04:32.783071+0000","last_clean":"2024-09-21T06:04:32.783071+0000","last_became_active":"2024-09-21T06:04:32.777105+0000","last_became_peered":"2024-09-21T06:04:32.777105+0000","last_unstale":"2024-09-21T06:04:32.783071+0000","last_undegraded":"2024-09-21T06:04:32.783071+0000","last_fullsized":"2024-09-21T06:04:32.783071+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:37.388005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781124+0000","last_change":"2024-09-21T06:04:32.771946+0000","last_active":"2024-09-21T06:04:32.781124+0000","last_peered":"2024-09-21T06:04:32.781124+0000","last_clean":"2024-09-21T06:04:32.781124+0000","last_became_active":"2024-09-21T06:04:32.771652+0000","last_became_peered":"2024-09-21T06:04:32.771652+0000","last_unstale":"2024-09-21T06:04:32.781124+0000","last_undegraded":"2024-09-21T06:04:32.781124+0000","last_fullsized":"2024-09-21T06:04:32.781124+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:02.037656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792043+0000","last_change":"2024-09-21T06:04:32.780368+0000","last_active":"2024-09-21T06:04:32.792043+0000","last_peered":"2024-09-21T06:04:32.792043+0000","last_clean":"2024-09-21T06:04:32.792043+0000","last_became_active":"2024-09-21T06:04:32.780021+0000","last_became_peered":"2024-09-21T06:04:32.780021+0000","last_unstale":"2024-09-21T06:04:32.792043+0000","last_undegraded":"2024-09-21T06:04:32.792043+0000","last_fullsized":"2024-09-21T06:04:32.792043+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:57.984993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.779808+0000","last_change":"2024-09-21T06:04:32.773537+0000","last_active":"2024-09-21T06:04:32.779808+0000","last_peered":"2024-09-21T06:04:32.779808+0000","last_clean":"2024-09-21T06:04:32.779808+0000","last_became_active":"2024-09-21T06:04:32.773240+0000","last_became_peered":"2024-09-21T06:04:32.773240+0000","last_unstale":"2024-09-21T06:04:32.779808+0000","last_undegraded":"2024-09-21T06:04:32.779808+0000","last_fullsized":"2024-09-21T06:04:32.779808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:04.990837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.789169+0000","last_change":"2024-09-21T06:04:32.782666+0000","last_active":"2024-09-21T06:04:32.789169+0000","last_peered":"2024-09-21T06:04:32.789169+0000","last_clean":"2024-09-21T06:04:32.789169+0000","last_became_active":"2024-09-21T06:04:32.777455+0000","last_became_peered":"2024-09-21T06:04:32.777455+0000","last_unstale":"2024-09-21T06:04:32.789169+0000","last_undegraded":"2024-09-21T06:04:32.789169+0000","last_fullsized":"2024-09-21T06:04:32.789169+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:53.442251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.768646+0000","last_change":"2024-09-21T06:04:32.764381+0000","last_active":"2024-09-21T06:04:32.768646+0000","last_peered":"2024-09-21T06:04:32.768646+0000","last_clean":"2024-09-21T06:04:32.768646+0000","last_became_active":"2024-09-21T06:04:32.764040+0000","last_became_peered":"2024-09-21T06:04:32.764040+0000","last_unstale":"2024-09-21T06:04:32.768646+0000","last_undegraded":"2024-09-21T06:04:32.768646+0000","last_fullsized":"2024-09-21T06:04:32.768646+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:33.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783350+0000","last_change":"2024-09-21T06:04:32.774466+0000","last_active":"2024-09-21T06:04:32.783350+0000","last_peered":"2024-09-21T06:04:32.783350+0000","last_clean":"2024-09-21T06:04:32.783350+0000","last_became_active":"2024-09-21T06:04:32.774133+0000","last_became_peered":"2024-09-21T06:04:32.774133+0000","last_unstale":"2024-09-21T06:04:32.783350+0000","last_undegraded":"2024-09-21T06:04:32.783350+0000","last_fullsized":"2024-09-21T06:04:32.783350+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:58.412247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794461+0000","last_change":"2024-09-21T06:04:32.792389+0000","last_active":"2024-09-21T06:04:32.794461+0000","last_peered":"2024-09-21T06:04:32.794461+0000","last_clean":"2024-09-21T06:04:32.794461+0000","last_became_active":"2024-09-21T06:04:32.792097+0000","last_became_peered":"2024-09-21T06:04:32.792097+0000","last_unstale":"2024-09-21T06:04:32.794461+0000","last_undegraded":"2024-09-21T06:04:32.794461+0000","last_fullsized":"2024-09-21T06:04:32.794461+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:46.730158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793238+0000","last_change":"2024-09-21T06:04:31.777083+0000","last_active":"2024-09-21T06:04:32.793238+0000","last_peered":"2024-09-21T06:04:32.793238+0000","last_clean":"2024-09-21T06:04:32.793238+0000","last_became_active":"2024-09-21T06:04:31.776739+0000","last_became_peered":"2024-09-21T06:04:31.776739+0000","last_unstale":"2024-09-21T06:04:32.793238+0000","last_undegraded":"2024-09-21T06:04:32.793238+0000","last_fullsized":"2024-09-21T06:04:32.793238+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:59.686723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783664+0000","last_change":"2024-09-21T06:04:32.776263+0000","last_active":"2024-09-21T06:04:32.783664+0000","last_peered":"2024-09-21T06:04:32.783664+0000","last_clean":"2024-09-21T06:04:32.783664+0000","last_became_active":"2024-09-21T06:04:32.775659+0000","last_became_peered":"2024-09-21T06:04:32.775659+0000","last_unstale":"2024-09-21T06:04:32.783664+0000","last_undegraded":"2024-09-21T06:04:32.783664+0000","last_fullsized":"2024-09-21T06:04:32.783664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:22.506119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.784109+0000","last_change":"2024-09-21T06:04:31.769588+0000","last_active":"2024-09-21T06:04:32.784109+0000","last_peered":"2024-09-21T06:04:32.784109+0000","last_clean":"2024-09-21T06:04:32.784109+0000","last_became_active":"2024-09-21T06:04:31.768905+0000","last_became_peered":"2024-09-21T06:04:31.768905+0000","last_unstale":"2024-09-21T06:04:32.784109+0000","last_undegraded":"2024-09-21T06:04:32.784109+0000","last_fullsized":"2024-09-21T06:04:32.784109+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:27.975438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.789767+0000","last_change":"2024-09-21T06:04:32.788286+0000","last_active":"2024-09-21T06:04:32.789767+0000","last_peered":"2024-09-21T06:04:32.789767+0000","last_clean":"2024-09-21T06:04:32.789767+0000","last_became_active":"2024-09-21T06:04:32.787926+0000","last_became_peered":"2024-09-21T06:04:32.787926+0000","last_unstale":"2024-09-21T06:04:32.789767+0000","last_undegraded":"2024-09-21T06:04:32.789767+0000","last_fullsized":"2024-09-21T06:04:32.789767+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:08.847286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794349+0000","last_change":"2024-09-21T06:04:31.772963+0000","last_active":"2024-09-21T06:04:32.794349+0000","last_peered":"2024-09-21T06:04:32.794349+0000","last_clean":"2024-09-21T06:04:32.794349+0000","last_became_active":"2024-09-21T06:04:31.772653+0000","last_became_peered":"2024-09-21T06:04:31.772653+0000","last_unstale":"2024-09-21T06:04:32.794349+0000","last_undegraded":"2024-09-21T06:04:32.794349+0000","last_fullsized":"2024-09-21T06:04:32.794349+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:59.868880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793482+0000","last_change":"2024-09-21T06:04:32.785502+0000","last_active":"2024-09-21T06:04:32.793482+0000","last_peered":"2024-09-21T06:04:32.793482+0000","last_clean":"2024-09-21T06:04:32.793482+0000","last_became_active":"2024-09-21T06:04:32.785091+0000","last_became_peered":"2024-09-21T06:04:32.785091+0000","last_unstale":"2024-09-21T06:04:32.793482+0000","last_undegraded":"2024-09-21T06:04:32.793482+0000","last_fullsized":"2024-09-21T06:04:32.793482+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:11.518104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.788989+0000","last_change":"2024-09-21T06:04:31.775276+0000","last_active":"2024-09-21T06:04:32.788989+0000","last_peered":"2024-09-21T06:04:32.788989+0000","last_clean":"2024-09-21T06:04:32.788989+0000","last_became_active":"2024-09-21T06:04:31.774028+0000","last_became_peered":"2024-09-21T06:04:31.774028+0000","last_unstale":"2024-09-21T06:04:32.788989+0000","last_undegraded":"2024-09-21T06:04:32.788989+0000","last_fullsized":"2024-09-21T06:04:32.788989+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:18.843912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783561+0000","last_change":"2024-09-21T06:04:32.777770+0000","last_active":"2024-09-21T06:04:32.783561+0000","last_peered":"2024-09-21T06:04:32.783561+0000","last_clean":"2024-09-21T06:04:32.783561+0000","last_became_active":"2024-09-21T06:04:32.777019+0000","last_became_peered":"2024-09-21T06:04:32.777019+0000","last_unstale":"2024-09-21T06:04:32.783561+0000","last_undegraded":"2024-09-21T06:04:32.783561+0000","last_fullsized":"2024-09-21T06:04:32.783561+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:59.802808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793522+0000","last_change":"2024-09-21T06:04:31.766081+0000","last_active":"2024-09-21T06:04:32.793522+0000","last_peered":"2024-09-21T06:04:32.793522+0000","last_clean":"2024-09-21T06:04:32.793522+0000","last_became_active":"2024-09-21T06:04:31.765541+0000","last_became_peered":"2024-09-21T06:04:31.765541+0000","last_unstale":"2024-09-21T06:04:32.793522+0000","last_undegraded":"2024-09-21T06:04:32.793522+0000","last_fullsized":"2024-09-21T06:04:32.793522+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:40.183984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792327+0000","last_change":"2024-09-21T06:04:32.780478+0000","last_active":"2024-09-21T06:04:32.792327+0000","last_peered":"2024-09-21T06:04:32.792327+0000","last_clean":"2024-09-21T06:04:32.792327+0000","last_became_active":"2024-09-21T06:04:32.778124+0000","last_became_peered":"2024-09-21T06:04:32.778124+0000","last_unstale":"2024-09-21T06:04:32.792327+0000","last_undegraded":"2024-09-21T06:04:32.792327+0000","last_fullsized":"2024-09-21T06:04:32.792327+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:26.746417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.787610+0000","last_change":"2024-09-21T06:04:31.772094+0000","last_active":"2024-09-21T06:04:32.787610+0000","last_peered":"2024-09-21T06:04:32.787610+0000","last_clean":"2024-09-21T06:04:32.787610+0000","last_became_active":"2024-09-21T06:04:31.771544+0000","last_became_peered":"2024-09-21T06:04:31.771544+0000","last_unstale":"2024-09-21T06:04:32.787610+0000","last_undegraded":"2024-09-21T06:04:32.787610+0000","last_fullsized":"2024-09-21T06:04:32.787610+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:03.600601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.767075+0000","last_change":"2024-09-21T06:04:32.761010+0000","last_active":"2024-09-21T06:04:32.767075+0000","last_peered":"2024-09-21T06:04:32.767075+0000","last_clean":"2024-09-21T06:04:32.767075+0000","last_became_active":"2024-09-21T06:04:32.760708+0000","last_became_peered":"2024-09-21T06:04:32.760708+0000","last_unstale":"2024-09-21T06:04:32.767075+0000","last_undegraded":"2024-09-21T06:04:32.767075+0000","last_fullsized":"2024-09-21T06:04:32.767075+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:45.868263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.769744+0000","last_change":"2024-09-21T06:04:31.768175+0000","last_active":"2024-09-21T06:04:31.769744+0000","last_peered":"2024-09-21T06:04:31.769744+0000","last_clean":"2024-09-21T06:04:31.769744+0000","last_became_active":"2024-09-21T06:04:31.767820+0000","last_became_peered":"2024-09-21T06:04:31.767820+0000","last_unstale":"2024-09-21T06:04:31.769744+0000","last_undegraded":"2024-09-21T06:04:31.769744+0000","last_fullsized":"2024-09-21T06:04:31.769744+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:24.459378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.775720+0000","last_change":"2024-09-21T06:04:32.772784+0000","last_active":"2024-09-21T06:04:32.775720+0000","last_peered":"2024-09-21T06:04:32.775720+0000","last_clean":"2024-09-21T06:04:32.775720+0000","last_became_active":"2024-09-21T06:04:32.772246+0000","last_became_peered":"2024-09-21T06:04:32.772246+0000","last_unstale":"2024-09-21T06:04:32.775720+0000","last_undegraded":"2024-09-21T06:04:32.775720+0000","last_fullsized":"2024-09-21T06:04:32.775720+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:39.036138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.752202+0000","last_change":"2024-09-21T06:04:31.759185+0000","last_active":"2024-09-21T06:04:32.752202+0000","last_peered":"2024-09-21T06:04:32.752202+0000","last_clean":"2024-09-21T06:04:32.752202+0000","last_became_active":"2024-09-21T06:04:31.758873+0000","last_became_peered":"2024-09-21T06:04:31.758873+0000","last_unstale":"2024-09-21T06:04:32.752202+0000","last_undegraded":"2024-09-21T06:04:32.752202+0000","last_fullsized":"2024-09-21T06:04:32.752202+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:37.617451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.770924+0000","last_change":"2024-09-21T06:04:32.769066+0000","last_active":"2024-09-21T06:04:32.770924+0000","last_peered":"2024-09-21T06:04:32.770924+0000","last_clean":"2024-09-21T06:04:32.770924+0000","last_became_active":"2024-09-21T06:04:32.768695+0000","last_became_peered":"2024-09-21T06:04:32.768695+0000","last_unstale":"2024-09-21T06:04:32.770924+0000","last_undegraded":"2024-09-21T06:04:32.770924+0000","last_fullsized":"2024-09-21T06:04:32.770924+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:41.742767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.755039+0000","last_change":"2024-09-21T06:04:31.770884+0000","last_active":"2024-09-21T06:04:32.755039+0000","last_peered":"2024-09-21T06:04:32.755039+0000","last_clean":"2024-09-21T06:04:32.755039+0000","last_became_active":"2024-09-21T06:04:31.768869+0000","last_became_peered":"2024-09-21T06:04:31.768869+0000","last_unstale":"2024-09-21T06:04:32.755039+0000","last_undegraded":"2024-09-21T06:04:32.755039+0000","last_fullsized":"2024-09-21T06:04:32.755039+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:18.843912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794408+0000","last_change":"2024-09-21T06:04:32.791807+0000","last_active":"2024-09-21T06:04:32.794408+0000","last_peered":"2024-09-21T06:04:32.794408+0000","last_clean":"2024-09-21T06:04:32.794408+0000","last_became_active":"2024-09-21T06:04:32.791461+0000","last_became_peered":"2024-09-21T06:04:32.791461+0000","last_unstale":"2024-09-21T06:04:32.794408+0000","last_undegraded":"2024-09-21T06:04:32.794408+0000","last_fullsized":"2024-09-21T06:04:32.794408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:47.832984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.766601+0000","last_change":"2024-09-21T06:04:31.770081+0000","last_active":"2024-09-21T06:04:32.766601+0000","last_peered":"2024-09-21T06:04:32.766601+0000","last_clean":"2024-09-21T06:04:32.766601+0000","last_became_active":"2024-09-21T06:04:31.768462+0000","last_became_peered":"2024-09-21T06:04:31.768462+0000","last_unstale":"2024-09-21T06:04:32.766601+0000","last_undegraded":"2024-09-21T06:04:32.766601+0000","last_fullsized":"2024-09-21T06:04:32.766601+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:10.542640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.796309+0000","last_change":"2024-09-21T06:04:32.795515+0000","last_active":"2024-09-21T06:04:32.796309+0000","last_peered":"2024-09-21T06:04:32.796309+0000","last_clean":"2024-09-21T06:04:32.796309+0000","last_became_active":"2024-09-21T06:04:32.795019+0000","last_became_peered":"2024-09-21T06:04:32.795019+0000","last_unstale":"2024-09-21T06:04:32.796309+0000","last_undegraded":"2024-09-21T06:04:32.796309+0000","last_fullsized":"2024-09-21T06:04:32.796309+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:26.746417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792146+0000","last_change":"2024-09-21T06:04:31.769723+0000","last_active":"2024-09-21T06:04:32.792146+0000","last_peered":"2024-09-21T06:04:32.792146+0000","last_clean":"2024-09-21T06:04:32.792146+0000","last_became_active":"2024-09-21T06:04:31.769344+0000","last_became_peered":"2024-09-21T06:04:31.769344+0000","last_unstale":"2024-09-21T06:04:32.792146+0000","last_undegraded":"2024-09-21T06:04:32.792146+0000","last_fullsized":"2024-09-21T06:04:32.792146+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:00.566146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.768354+0000","last_change":"2024-09-21T06:04:32.767569+0000","last_active":"2024-09-21T06:04:32.768354+0000","last_peered":"2024-09-21T06:04:32.768354+0000","last_clean":"2024-09-21T06:04:32.768354+0000","last_became_active":"2024-09-21T06:04:32.766934+0000","last_became_peered":"2024-09-21T06:04:32.766934+0000","last_unstale":"2024-09-21T06:04:32.768354+0000","last_undegraded":"2024-09-21T06:04:32.768354+0000","last_fullsized":"2024-09-21T06:04:32.768354+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:56.103744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.770166+0000","last_change":"2024-09-21T06:04:31.768773+0000","last_active":"2024-09-21T06:04:31.770166+0000","last_peered":"2024-09-21T06:04:31.770166+0000","last_clean":"2024-09-21T06:04:31.770166+0000","last_became_active":"2024-09-21T06:04:31.767292+0000","last_became_peered":"2024-09-21T06:04:31.767292+0000","last_unstale":"2024-09-21T06:04:31.770166+0000","last_undegraded":"2024-09-21T06:04:31.770166+0000","last_fullsized":"2024-09-21T06:04:31.770166+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:32.705430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792225+0000","last_change":"2024-09-21T06:04:32.784585+0000","last_active":"2024-09-21T06:04:32.792225+0000","last_peered":"2024-09-21T06:04:32.792225+0000","last_clean":"2024-09-21T06:04:32.792225+0000","last_became_active":"2024-09-21T06:04:32.784259+0000","last_became_peered":"2024-09-21T06:04:32.784259+0000","last_unstale":"2024-09-21T06:04:32.792225+0000","last_undegraded":"2024-09-21T06:04:32.792225+0000","last_fullsized":"2024-09-21T06:04:32.792225+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:11.446707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.773742+0000","last_change":"2024-09-21T06:04:31.772181+0000","last_active":"2024-09-21T06:04:31.773742+0000","last_peered":"2024-09-21T06:04:31.773742+0000","last_clean":"2024-09-21T06:04:31.773742+0000","last_became_active":"2024-09-21T06:04:31.771816+0000","last_became_peered":"2024-09-21T06:04:31.771816+0000","last_unstale":"2024-09-21T06:04:31.773742+0000","last_undegraded":"2024-09-21T06:04:31.773742+0000","last_fullsized":"2024-09-21T06:04:31.773742+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:47.864470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.770503+0000","last_change":"2024-09-21T06:04:32.768202+0000","last_active":"2024-09-21T06:04:32.770503+0000","last_peered":"2024-09-21T06:04:32.770503+0000","last_clean":"2024-09-21T06:04:32.770503+0000","last_became_active":"2024-09-21T06:04:32.767810+0000","last_became_peered":"2024-09-21T06:04:32.767810+0000","last_unstale":"2024-09-21T06:04:32.770503+0000","last_undegraded":"2024-09-21T06:04:32.770503+0000","last_fullsized":"2024-09-21T06:04:32.770503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:11.518104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.763381+0000","last_change":"2024-09-21T06:04:31.775115+0000","last_active":"2024-09-21T06:04:32.763381+0000","last_peered":"2024-09-21T06:04:32.763381+0000","last_clean":"2024-09-21T06:04:32.763381+0000","last_became_active":"2024-09-21T06:04:31.773039+0000","last_became_peered":"2024-09-21T06:04:31.773039+0000","last_unstale":"2024-09-21T06:04:32.763381+0000","last_undegraded":"2024-09-21T06:04:32.763381+0000","last_fullsized":"2024-09-21T06:04:32.763381+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:54.006741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781404+0000","last_change":"2024-09-21T06:04:32.780256+0000","last_active":"2024-09-21T06:04:32.781404+0000","last_peered":"2024-09-21T06:04:32.781404+0000","last_clean":"2024-09-21T06:04:32.781404+0000","last_became_active":"2024-09-21T06:04:32.779924+0000","last_became_peered":"2024-09-21T06:04:32.779924+0000","last_unstale":"2024-09-21T06:04:32.781404+0000","last_undegraded":"2024-09-21T06:04:32.781404+0000","last_fullsized":"2024-09-21T06:04:32.781404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:52.831924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.756793+0000","last_change":"2024-09-21T06:04:31.775645+0000","last_active":"2024-09-21T06:04:32.756793+0000","last_peered":"2024-09-21T06:04:32.756793+0000","last_clean":"2024-09-21T06:04:32.756793+0000","last_became_active":"2024-09-21T06:04:31.775129+0000","last_became_peered":"2024-09-21T06:04:31.775129+0000","last_unstale":"2024-09-21T06:04:32.756793+0000","last_undegraded":"2024-09-21T06:04:32.756793+0000","last_fullsized":"2024-09-21T06:04:32.756793+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:55.569554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.771789+0000","last_change":"2024-09-21T06:04:32.770814+0000","last_active":"2024-09-21T06:04:32.771789+0000","last_peered":"2024-09-21T06:04:32.771789+0000","last_clean":"2024-09-21T06:04:32.771789+0000","last_became_active":"2024-09-21T06:04:32.770411+0000","last_became_peered":"2024-09-21T06:04:32.770411+0000","last_unstale":"2024-09-21T06:04:32.771789+0000","last_undegraded":"2024-09-21T06:04:32.771789+0000","last_fullsized":"2024-09-21T06:04:32.771789+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:17.290747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793295+0000","last_change":"2024-09-21T06:04:31.772137+0000","last_active":"2024-09-21T06:04:32.793295+0000","last_peered":"2024-09-21T06:04:32.793295+0000","last_clean":"2024-09-21T06:04:32.793295+0000","last_became_active":"2024-09-21T06:04:31.771622+0000","last_became_peered":"2024-09-21T06:04:31.771622+0000","last_unstale":"2024-09-21T06:04:32.793295+0000","last_undegraded":"2024-09-21T06:04:32.793295+0000","last_fullsized":"2024-09-21T06:04:32.793295+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:38.991457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.769708+0000","last_change":"2024-09-21T06:04:32.768582+0000","last_active":"2024-09-21T06:04:32.769708+0000","last_peered":"2024-09-21T06:04:32.769708+0000","last_clean":"2024-09-21T06:04:32.769708+0000","last_became_active":"2024-09-21T06:04:32.767918+0000","last_became_peered":"2024-09-21T06:04:32.767918+0000","last_unstale":"2024-09-21T06:04:32.769708+0000","last_undegraded":"2024-09-21T06:04:32.769708+0000","last_fullsized":"2024-09-21T06:04:32.769708+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:49.109988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.786154+0000","last_change":"2024-09-21T06:04:31.775324+0000","last_active":"2024-09-21T06:04:32.786154+0000","last_peered":"2024-09-21T06:04:32.786154+0000","last_clean":"2024-09-21T06:04:32.786154+0000","last_became_active":"2024-09-21T06:04:31.774989+0000","last_became_peered":"2024-09-21T06:04:31.774989+0000","last_unstale":"2024-09-21T06:04:32.786154+0000","last_undegraded":"2024-09-21T06:04:32.786154+0000","last_fullsized":"2024-09-21T06:04:32.786154+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:10.782962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793956+0000","last_change":"2024-09-21T06:04:32.787340+0000","last_active":"2024-09-21T06:04:32.793956+0000","last_peered":"2024-09-21T06:04:32.793956+0000","last_clean":"2024-09-21T06:04:32.793956+0000","last_became_active":"2024-09-21T06:04:32.787027+0000","last_became_peered":"2024-09-21T06:04:32.787027+0000","last_unstale":"2024-09-21T06:04:32.793956+0000","last_undegraded":"2024-09-21T06:04:32.793956+0000","last_fullsized":"2024-09-21T06:04:32.793956+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:37.388005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.791893+0000","last_change":"2024-09-21T06:04:31.774380+0000","last_active":"2024-09-21T06:04:32.791893+0000","last_peered":"2024-09-21T06:04:32.791893+0000","last_clean":"2024-09-21T06:04:32.791893+0000","last_became_active":"2024-09-21T06:04:31.773835+0000","last_became_peered":"2024-09-21T06:04:31.773835+0000","last_unstale":"2024-09-21T06:04:32.791893+0000","last_undegraded":"2024-09-21T06:04:32.791893+0000","last_fullsized":"2024-09-21T06:04:32.791893+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:53.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.759154+0000","last_change":"2024-09-21T06:04:31.759154+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.759154+0000","last_undegraded":"2024-09-21T06:04:31.759154+0000","last_fullsized":"2024-09-21T06:04:31.759154+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.795017+0000","last_change":"2024-09-21T06:04:31.768595+0000","last_active":"2024-09-21T06:04:32.795017+0000","last_peered":"2024-09-21T06:04:32.795017+0000","last_clean":"2024-09-21T06:04:32.795017+0000","last_became_active":"2024-09-21T06:04:31.767603+0000","last_became_peered":"2024-09-21T06:04:31.767603+0000","last_unstale":"2024-09-21T06:04:32.795017+0000","last_undegraded":"2024-09-21T06:04:32.795017+0000","last_fullsized":"2024-09-21T06:04:32.795017+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:53.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.774981+0000","last_change":"2024-09-21T06:04:32.774290+0000","last_active":"2024-09-21T06:04:32.774981+0000","last_peered":"2024-09-21T06:04:32.774981+0000","last_clean":"2024-09-21T06:04:32.774981+0000","last_became_active":"2024-09-21T06:04:32.774023+0000","last_became_peered":"2024-09-21T06:04:32.774023+0000","last_unstale":"2024-09-21T06:04:32.774981+0000","last_undegraded":"2024-09-21T06:04:32.774981+0000","last_fullsized":"2024-09-21T06:04:32.774981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.944676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.774811+0000","last_change":"2024-09-21T06:04:31.774500+0000","last_active":"2024-09-21T06:04:32.774811+0000","last_peered":"2024-09-21T06:04:32.774811+0000","last_clean":"2024-09-21T06:04:32.774811+0000","last_became_active":"2024-09-21T06:04:31.774161+0000","last_became_peered":"2024-09-21T06:04:31.774161+0000","last_unstale":"2024-09-21T06:04:32.774811+0000","last_undegraded":"2024-09-21T06:04:32.774811+0000","last_fullsized":"2024-09-21T06:04:32.774811+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:32.288314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.773796+0000","last_change":"2024-09-21T06:04:32.769815+0000","last_active":"2024-09-21T06:04:32.773796+0000","last_peered":"2024-09-21T06:04:32.773796+0000","last_clean":"2024-09-21T06:04:32.773796+0000","last_became_active":"2024-09-21T06:04:32.769467+0000","last_became_peered":"2024-09-21T06:04:32.769467+0000","last_unstale":"2024-09-21T06:04:32.773796+0000","last_undegraded":"2024-09-21T06:04:32.773796+0000","last_fullsized":"2024-09-21T06:04:32.773796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:44.954801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.763651+0000","last_change":"2024-09-21T06:04:31.772131+0000","last_active":"2024-09-21T06:04:32.763651+0000","last_peered":"2024-09-21T06:04:32.763651+0000","last_clean":"2024-09-21T06:04:32.763651+0000","last_became_active":"2024-09-21T06:04:31.771792+0000","last_became_peered":"2024-09-21T06:04:31.771792+0000","last_unstale":"2024-09-21T06:04:32.763651+0000","last_undegraded":"2024-09-21T06:04:32.763651+0000","last_fullsized":"2024-09-21T06:04:32.763651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.561776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783512+0000","last_change":"2024-09-21T06:04:32.782337+0000","last_active":"2024-09-21T06:04:32.783512+0000","last_peered":"2024-09-21T06:04:32.783512+0000","last_clean":"2024-09-21T06:04:32.783512+0000","last_became_active":"2024-09-21T06:04:32.782011+0000","last_became_peered":"2024-09-21T06:04:32.782011+0000","last_unstale":"2024-09-21T06:04:32.783512+0000","last_undegraded":"2024-09-21T06:04:32.783512+0000","last_fullsized":"2024-09-21T06:04:32.783512+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:45.735018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.757070+0000","last_change":"2024-09-21T06:04:31.766119+0000","last_active":"2024-09-21T06:04:32.757070+0000","last_peered":"2024-09-21T06:04:32.757070+0000","last_clean":"2024-09-21T06:04:32.757070+0000","last_became_active":"2024-09-21T06:04:31.765398+0000","last_became_peered":"2024-09-21T06:04:31.765398+0000","last_unstale":"2024-09-21T06:04:32.757070+0000","last_undegraded":"2024-09-21T06:04:32.757070+0000","last_fullsized":"2024-09-21T06:04:32.757070+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:23.042000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781363+0000","last_change":"2024-09-21T06:04:32.777061+0000","last_active":"2024-09-21T06:04:32.781363+0000","last_peered":"2024-09-21T06:04:32.781363+0000","last_clean":"2024-09-21T06:04:32.781363+0000","last_became_active":"2024-09-21T06:04:32.776703+0000","last_became_peered":"2024-09-21T06:04:32.776703+0000","last_unstale":"2024-09-21T06:04:32.781363+0000","last_undegraded":"2024-09-21T06:04:32.781363+0000","last_fullsized":"2024-09-21T06:04:32.781363+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:12.555174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.756499+0000","last_change":"2024-09-21T06:04:31.770508+0000","last_active":"2024-09-21T06:04:32.756499+0000","last_peered":"2024-09-21T06:04:32.756499+0000","last_clean":"2024-09-21T06:04:32.756499+0000","last_became_active":"2024-09-21T06:04:31.769020+0000","last_became_peered":"2024-09-21T06:04:31.769020+0000","last_unstale":"2024-09-21T06:04:32.756499+0000","last_undegraded":"2024-09-21T06:04:32.756499+0000","last_fullsized":"2024-09-21T06:04:32.756499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:40.893581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.765322+0000","last_change":"2024-09-21T06:04:32.762522+0000","last_active":"2024-09-21T06:04:32.765322+0000","last_peered":"2024-09-21T06:04:32.765322+0000","last_clean":"2024-09-21T06:04:32.765322+0000","last_became_active":"2024-09-21T06:04:32.762144+0000","last_became_peered":"2024-09-21T06:04:32.762144+0000","last_unstale":"2024-09-21T06:04:32.765322+0000","last_undegraded":"2024-09-21T06:04:32.765322+0000","last_fullsized":"2024-09-21T06:04:32.765322+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:36.679619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.786027+0000","last_change":"2024-09-21T06:04:31.774958+0000","last_active":"2024-09-21T06:04:32.786027+0000","last_peered":"2024-09-21T06:04:32.786027+0000","last_clean":"2024-09-21T06:04:32.786027+0000","last_became_active":"2024-09-21T06:04:31.774595+0000","last_became_peered":"2024-09-21T06:04:31.774595+0000","last_unstale":"2024-09-21T06:04:32.786027+0000","last_undegraded":"2024-09-21T06:04:32.786027+0000","last_fullsized":"2024-09-21T06:04:32.786027+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:05.289920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.770815+0000","last_change":"2024-09-21T06:04:32.766158+0000","last_active":"2024-09-21T06:04:32.770815+0000","last_peered":"2024-09-21T06:04:32.770815+0000","last_clean":"2024-09-21T06:04:32.770815+0000","last_became_active":"2024-09-21T06:04:32.765811+0000","last_became_peered":"2024-09-21T06:04:32.765811+0000","last_unstale":"2024-09-21T06:04:32.770815+0000","last_undegraded":"2024-09-21T06:04:32.770815+0000","last_fullsized":"2024-09-21T06:04:32.770815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:11.446707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.757924+0000","last_change":"2024-09-21T06:04:31.775745+0000","last_active":"2024-09-21T06:04:32.757924+0000","last_peered":"2024-09-21T06:04:32.757924+0000","last_clean":"2024-09-21T06:04:32.757924+0000","last_became_active":"2024-09-21T06:04:31.775037+0000","last_became_peered":"2024-09-21T06:04:31.775037+0000","last_unstale":"2024-09-21T06:04:32.757924+0000","last_undegraded":"2024-09-21T06:04:32.757924+0000","last_fullsized":"2024-09-21T06:04:32.757924+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:40.310390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794553+0000","last_change":"2024-09-21T06:04:32.791087+0000","last_active":"2024-09-21T06:04:32.794553+0000","last_peered":"2024-09-21T06:04:32.794553+0000","last_clean":"2024-09-21T06:04:32.794553+0000","last_became_active":"2024-09-21T06:04:32.790735+0000","last_became_peered":"2024-09-21T06:04:32.790735+0000","last_unstale":"2024-09-21T06:04:32.794553+0000","last_undegraded":"2024-09-21T06:04:32.794553+0000","last_fullsized":"2024-09-21T06:04:32.794553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:30.386322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.751374+0000","last_change":"2024-09-21T06:04:31.761815+0000","last_active":"2024-09-21T06:04:32.751374+0000","last_peered":"2024-09-21T06:04:32.751374+0000","last_clean":"2024-09-21T06:04:32.751374+0000","last_became_active":"2024-09-21T06:04:31.761487+0000","last_became_peered":"2024-09-21T06:04:31.761487+0000","last_unstale":"2024-09-21T06:04:32.751374+0000","last_undegraded":"2024-09-21T06:04:32.751374+0000","last_fullsized":"2024-09-21T06:04:32.751374+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:02:35.169801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794037+0000","last_change":"2024-09-21T06:04:32.788888+0000","last_active":"2024-09-21T06:04:32.794037+0000","last_peered":"2024-09-21T06:04:32.794037+0000","last_clean":"2024-09-21T06:04:32.794037+0000","last_became_active":"2024-09-21T06:04:32.788597+0000","last_became_peered":"2024-09-21T06:04:32.788597+0000","last_unstale":"2024-09-21T06:04:32.794037+0000","last_undegraded":"2024-09-21T06:04:32.794037+0000","last_fullsized":"2024-09-21T06:04:32.794037+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:12.183155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.761494+0000","last_change":"2024-09-21T06:04:31.762529+0000","last_active":"2024-09-21T06:04:32.761494+0000","last_peered":"2024-09-21T06:04:32.761494+0000","last_clean":"2024-09-21T06:04:32.761494+0000","last_became_active":"2024-09-21T06:04:31.762189+0000","last_became_peered":"2024-09-21T06:04:31.762189+0000","last_unstale":"2024-09-21T06:04:32.761494+0000","last_undegraded":"2024-09-21T06:04:32.761494+0000","last_fullsized":"2024-09-21T06:04:32.761494+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:51.822586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.773682+0000","last_change":"2024-09-21T06:04:32.770727+0000","last_active":"2024-09-21T06:04:32.773682+0000","last_peered":"2024-09-21T06:04:32.773682+0000","last_clean":"2024-09-21T06:04:32.773682+0000","last_became_active":"2024-09-21T06:04:32.770393+0000","last_became_peered":"2024-09-21T06:04:32.770393+0000","last_unstale":"2024-09-21T06:04:32.773682+0000","last_undegraded":"2024-09-21T06:04:32.773682+0000","last_fullsized":"2024-09-21T06:04:32.773682+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:23.599523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.751186+0000","last_change":"2024-09-21T06:04:31.768859+0000","last_active":"2024-09-21T06:04:32.751186+0000","last_peered":"2024-09-21T06:04:32.751186+0000","last_clean":"2024-09-21T06:04:32.751186+0000","last_became_active":"2024-09-21T06:04:31.768535+0000","last_became_peered":"2024-09-21T06:04:31.768535+0000","last_unstale":"2024-09-21T06:04:32.751186+0000","last_undegraded":"2024-09-21T06:04:32.751186+0000","last_fullsized":"2024-09-21T06:04:32.751186+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:05.572103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.786505+0000","last_change":"2024-09-21T06:04:32.782856+0000","last_active":"2024-09-21T06:04:32.786505+0000","last_peered":"2024-09-21T06:04:32.786505+0000","last_clean":"2024-09-21T06:04:32.786505+0000","last_became_active":"2024-09-21T06:04:32.782532+0000","last_became_peered":"2024-09-21T06:04:32.782532+0000","last_unstale":"2024-09-21T06:04:32.786505+0000","last_undegraded":"2024-09-21T06:04:32.786505+0000","last_fullsized":"2024-09-21T06:04:32.786505+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:49.109988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.770750+0000","last_change":"2024-09-21T06:04:31.768251+0000","last_active":"2024-09-21T06:04:31.770750+0000","last_peered":"2024-09-21T06:04:31.770750+0000","last_clean":"2024-09-21T06:04:31.770750+0000","last_became_active":"2024-09-21T06:04:31.767918+0000","last_became_peered":"2024-09-21T06:04:31.767918+0000","last_unstale":"2024-09-21T06:04:31.770750+0000","last_undegraded":"2024-09-21T06:04:31.770750+0000","last_fullsized":"2024-09-21T06:04:31.770750+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:28.832034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.760288+0000","last_change":"2024-09-21T06:04:31.760288+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.760288+0000","last_undegraded":"2024-09-21T06:04:31.760288+0000","last_fullsized":"2024-09-21T06:04:31.760288+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.755302+0000","last_change":"2024-09-21T06:04:31.769840+0000","last_active":"2024-09-21T06:04:32.755302+0000","last_peered":"2024-09-21T06:04:32.755302+0000","last_clean":"2024-09-21T06:04:32.755302+0000","last_became_active":"2024-09-21T06:04:31.769534+0000","last_became_peered":"2024-09-21T06:04:31.769534+0000","last_unstale":"2024-09-21T06:04:32.755302+0000","last_undegraded":"2024-09-21T06:04:32.755302+0000","last_fullsized":"2024-09-21T06:04:32.755302+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:10.542640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.784911+0000","last_change":"2024-09-21T06:04:32.779505+0000","last_active":"2024-09-21T06:04:32.784911+0000","last_peered":"2024-09-21T06:04:32.784911+0000","last_clean":"2024-09-21T06:04:32.784911+0000","last_became_active":"2024-09-21T06:04:32.779079+0000","last_became_peered":"2024-09-21T06:04:32.779079+0000","last_unstale":"2024-09-21T06:04:32.784911+0000","last_undegraded":"2024-09-21T06:04:32.784911+0000","last_fullsized":"2024-09-21T06:04:32.784911+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:37.256481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.765120+0000","last_change":"2024-09-21T06:04:31.768166+0000","last_active":"2024-09-21T06:04:32.765120+0000","last_peered":"2024-09-21T06:04:32.765120+0000","last_clean":"2024-09-21T06:04:32.765120+0000","last_became_active":"2024-09-21T06:04:31.767622+0000","last_became_peered":"2024-09-21T06:04:31.767622+0000","last_unstale":"2024-09-21T06:04:32.765120+0000","last_undegraded":"2024-09-21T06:04:32.765120+0000","last_fullsized":"2024-09-21T06:04:32.765120+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:14.758437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.780297+0000","last_change":"2024-09-21T06:04:32.776172+0000","last_active":"2024-09-21T06:04:32.780297+0000","last_peered":"2024-09-21T06:04:32.780297+0000","last_clean":"2024-09-21T06:04:32.780297+0000","last_became_active":"2024-09-21T06:04:32.775877+0000","last_became_peered":"2024-09-21T06:04:32.775877+0000","last_unstale":"2024-09-21T06:04:32.780297+0000","last_undegraded":"2024-09-21T06:04:32.780297+0000","last_fullsized":"2024-09-21T06:04:32.780297+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:54.870810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.788947+0000","last_change":"2024-09-21T06:04:31.772282+0000","last_active":"2024-09-21T06:04:32.788947+0000","last_peered":"2024-09-21T06:04:32.788947+0000","last_clean":"2024-09-21T06:04:32.788947+0000","last_became_active":"2024-09-21T06:04:31.770583+0000","last_became_peered":"2024-09-21T06:04:31.770583+0000","last_unstale":"2024-09-21T06:04:32.788947+0000","last_undegraded":"2024-09-21T06:04:32.788947+0000","last_fullsized":"2024-09-21T06:04:32.788947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:35.915894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":201,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.775042+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:04:32.775042+0000","last_peered":"2024-09-21T06:04:32.775042+0000","last_clean":"2024-09-21T06:04:32.775042+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:04:32.775042+0000","last_undegraded":"2024-09-21T06:04:32.775042+0000","last_fullsized":"2024-09-21T06:04:32.775042+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50656,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321184,"statfs":{"total":96636764160,"available":96584892416,"internally_reserved":0,"allocated":1445888,"data_stored":944799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509542,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":851968,"data_stored":353907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509541,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":847872,"data_stored":353905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50064,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321776,"statfs":{"total":96636764160,"available":96585498624,"internally_reserved":0,"allocated":847872,"data_stored":353885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54232,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317608,"statfs":{"total":96636764160,"available":96581230592,"internally_reserved":0,"allocated":851968,"data_stored":353927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50048,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321792,"statfs":{"total":96636764160,"available":96585515008,"internally_reserved":0,"allocated":831488,"data_stored":347012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509541,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54788,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317052,"statfs":{"total":96636764160,"available":96580661248,"internally_reserved":0,"allocated":1425408,"data_stored":937380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58368,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313472,"statfs":{"total":96636764160,"available":96576995328,"internally_reserved":0,"allocated":831488,"data_stored":347012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:04:37.383 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:04:37.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.583+0000 7f186ba00640 1 -- 172.21.15.50:0/2344775883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640b4de0 msgr2=0x7f18640b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:37.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.583+0000 7f186ba00640 1 --2- 172.21.15.50:0/2344775883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640b4de0 0x7f18640b51f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f185c009c80 tx=0x7f185c01cce0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.583+0000 7f186ba00640 1 -- 172.21.15.50:0/2344775883 shutdown_connections 2024-09-21T06:04:37.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.583+0000 7f186ba00640 1 --2- 172.21.15.50:0/2344775883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640b4de0 0x7f18640b51f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.583+0000 7f186ba00640 1 --2- 172.21.15.50:0/2344775883 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18640b5c70 0x7f186415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.583+0000 7f186ba00640 1 -- 172.21.15.50:0/2344775883 >> 172.21.15.50:0/2344775883 conn(0x7f18640b2040 msgr2=0x7f18640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:37.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.583+0000 7f186ba00640 1 -- 172.21.15.50:0/2344775883 shutdown_connections 2024-09-21T06:04:37.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.583+0000 7f186ba00640 1 -- 172.21.15.50:0/2344775883 wait complete. 2024-09-21T06:04:37.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.583+0000 7f186ba00640 1 Processor -- start 2024-09-21T06:04:37.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.583+0000 7f186ba00640 1 -- start start 2024-09-21T06:04:37.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f186ba00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18640b4de0 0x7f186413fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:37.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f186ba00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640b5c70 0x7f18641561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:37.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f186ba00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18641527b0 0x7f18640cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:37.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f186ba00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1864168a00 con 0x7f18640b5c70 2024-09-21T06:04:37.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f186ba00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1864168880 con 0x7f18641527b0 2024-09-21T06:04:37.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f186ba00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1864168b80 con 0x7f18640b4de0 2024-09-21T06:04:37.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1863e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640b5c70 0x7f18641561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:37.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1863e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640b5c70 0x7f18641561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36662/0 (socket says 172.21.15.50:36662) 2024-09-21T06:04:37.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1863e00640 1 -- 172.21.15.50:0/2657940489 learned_addr learned my addr 172.21.15.50:0/2657940489 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:37.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1869000640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18640b4de0 0x7f186413fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:37.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1869a00640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18641527b0 0x7f18640cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:37.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1863e00640 1 -- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18640b4de0 msgr2=0x7f186413fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:37.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1863e00640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18640b4de0 0x7f186413fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1863e00640 1 -- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18641527b0 msgr2=0x7f18640cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:37.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1863e00640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18641527b0 0x7f18640cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1863e00640 1 -- 172.21.15.50:0/2657940489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18641fc8e0 con 0x7f18640b5c70 2024-09-21T06:04:37.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1869000640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18640b4de0 0x7f186413fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:37.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1869a00640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18641527b0 0x7f18640cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:37.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1863e00640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640b5c70 0x7f18641561c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f185000c390 tx=0x7f185000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:37.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1860c00640 1 -- 172.21.15.50:0/2657940489 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850017070 con 0x7f18640b5c70 2024-09-21T06:04:37.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1860c00640 1 -- 172.21.15.50:0/2657940489 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18500053c0 con 0x7f18640b5c70 2024-09-21T06:04:37.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f186ba00640 1 -- 172.21.15.50:0/2657940489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18640ced20 con 0x7f18640b5c70 2024-09-21T06:04:37.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f1860c00640 1 -- 172.21.15.50:0/2657940489 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850003840 con 0x7f18640b5c70 2024-09-21T06:04:37.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f186ba00640 1 -- 172.21.15.50:0/2657940489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18640cefc0 con 0x7f18640b5c70 2024-09-21T06:04:37.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.587+0000 7f186ba00640 1 -- 172.21.15.50:0/2657940489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18640b51f0 con 0x7f18640b5c70 2024-09-21T06:04:37.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.591+0000 7f1860c00640 1 -- 172.21.15.50:0/2657940489 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f185001a070 con 0x7f18640b5c70 2024-09-21T06:04:37.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.591+0000 7f1860c00640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1844073e90 0x7f18440763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:37.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.591+0000 7f1860c00640 1 -- 172.21.15.50:0/2657940489 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f18500921e0 con 0x7f18640b5c70 2024-09-21T06:04:37.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.591+0000 7f1869000640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1844073e90 0x7f18440763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:37.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.591+0000 7f1869000640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1844073e90 0x7f18440763a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1858009ac0 tx=0x7f18580091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:37.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.595+0000 7f1860c00640 1 -- 172.21.15.50:0/2657940489 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f18640b5730 con 0x7f18640b5c70 2024-09-21T06:04:37.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.767+0000 7f186ba00640 1 -- 172.21.15.50:0/2657940489 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f186406b540 con 0x7f1844073e90 2024-09-21T06:04:37.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.775+0000 7f1860c00640 1 -- 172.21.15.50:0/2657940489 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251632 (secure 0 0 0) 0x7f1858010a90 con 0x7f1844073e90 2024-09-21T06:04:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:37.778 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:04:37.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.779+0000 7f186ba00640 1 -- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1844073e90 msgr2=0x7f18440763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:37.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.779+0000 7f186ba00640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1844073e90 0x7f18440763a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1858009ac0 tx=0x7f18580091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.779+0000 7f186ba00640 1 -- 172.21.15.50:0/2657940489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640b5c70 msgr2=0x7f18641561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:37.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.779+0000 7f186ba00640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640b5c70 0x7f18641561c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f185000c390 tx=0x7f185000c850 comp rx=0 tx=0).stop 2024-09-21T06:04:37.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.779+0000 7f186ba00640 1 -- 172.21.15.50:0/2657940489 shutdown_connections 2024-09-21T06:04:37.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.779+0000 7f186ba00640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18641527b0 0x7f18640cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.779+0000 7f186ba00640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1844073e90 0x7f18440763a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.779+0000 7f186ba00640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640b5c70 0x7f18641561c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.779+0000 7f186ba00640 1 --2- 172.21.15.50:0/2657940489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18640b4de0 0x7f186413fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:37.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.779+0000 7f186ba00640 1 -- 172.21.15.50:0/2657940489 >> 172.21.15.50:0/2657940489 conn(0x7f18640b2040 msgr2=0x7f1864154590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:37.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.779+0000 7f186ba00640 1 -- 172.21.15.50:0/2657940489 shutdown_connections 2024-09-21T06:04:37.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.779+0000 7f186ba00640 1 -- 172.21.15.50:0/2657940489 wait complete. 2024-09-21T06:04:37.800 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-21T06:04:37.023099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":136,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":418300,"kb_used_data":7748,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754556420,"statfs":{"total":773094113280,"available":772665774080,"internally_reserved":0,"allocated":7933952,"data_stored":3991827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001380"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793750+0000","last_change":"2024-09-21T06:04:32.787310+0000","last_active":"2024-09-21T06:04:32.793750+0000","last_peered":"2024-09-21T06:04:32.793750+0000","last_clean":"2024-09-21T06:04:32.793750+0000","last_became_active":"2024-09-21T06:04:32.786715+0000","last_became_peered":"2024-09-21T06:04:32.786715+0000","last_unstale":"2024-09-21T06:04:32.793750+0000","last_undegraded":"2024-09-21T06:04:32.793750+0000","last_fullsized":"2024-09-21T06:04:32.793750+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:59.802808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.784907+0000","last_change":"2024-09-21T06:04:32.778255+0000","last_active":"2024-09-21T06:04:32.784907+0000","last_peered":"2024-09-21T06:04:32.784907+0000","last_clean":"2024-09-21T06:04:32.784907+0000","last_became_active":"2024-09-21T06:04:32.777803+0000","last_became_peered":"2024-09-21T06:04:32.777803+0000","last_unstale":"2024-09-21T06:04:32.784907+0000","last_undegraded":"2024-09-21T06:04:32.784907+0000","last_fullsized":"2024-09-21T06:04:32.784907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:54.763471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781150+0000","last_change":"2024-09-21T06:04:32.772652+0000","last_active":"2024-09-21T06:04:32.781150+0000","last_peered":"2024-09-21T06:04:32.781150+0000","last_clean":"2024-09-21T06:04:32.781150+0000","last_became_active":"2024-09-21T06:04:32.772322+0000","last_became_peered":"2024-09-21T06:04:32.772322+0000","last_unstale":"2024-09-21T06:04:32.781150+0000","last_undegraded":"2024-09-21T06:04:32.781150+0000","last_fullsized":"2024-09-21T06:04:32.781150+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:36.624434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.762434+0000","last_change":"2024-09-21T06:04:31.762434+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.762434+0000","last_undegraded":"2024-09-21T06:04:31.762434+0000","last_fullsized":"2024-09-21T06:04:31.762434+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783599+0000","last_change":"2024-09-21T06:04:32.777265+0000","last_active":"2024-09-21T06:04:32.783599+0000","last_peered":"2024-09-21T06:04:32.783599+0000","last_clean":"2024-09-21T06:04:32.783599+0000","last_became_active":"2024-09-21T06:04:32.776943+0000","last_became_peered":"2024-09-21T06:04:32.776943+0000","last_unstale":"2024-09-21T06:04:32.783599+0000","last_undegraded":"2024-09-21T06:04:32.783599+0000","last_fullsized":"2024-09-21T06:04:32.783599+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:05.622248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.796017+0000","last_change":"2024-09-21T06:04:32.794992+0000","last_active":"2024-09-21T06:04:32.796017+0000","last_peered":"2024-09-21T06:04:32.796017+0000","last_clean":"2024-09-21T06:04:32.796017+0000","last_became_active":"2024-09-21T06:04:32.794580+0000","last_became_peered":"2024-09-21T06:04:32.794580+0000","last_unstale":"2024-09-21T06:04:32.796017+0000","last_undegraded":"2024-09-21T06:04:32.796017+0000","last_fullsized":"2024-09-21T06:04:32.796017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:57.984993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.768680+0000","last_change":"2024-09-21T06:04:32.765123+0000","last_active":"2024-09-21T06:04:32.768680+0000","last_peered":"2024-09-21T06:04:32.768680+0000","last_clean":"2024-09-21T06:04:32.768680+0000","last_became_active":"2024-09-21T06:04:32.764446+0000","last_became_peered":"2024-09-21T06:04:32.764446+0000","last_unstale":"2024-09-21T06:04:32.768680+0000","last_undegraded":"2024-09-21T06:04:32.768680+0000","last_fullsized":"2024-09-21T06:04:32.768680+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:00.689363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783317+0000","last_change":"2024-09-21T06:04:32.773644+0000","last_active":"2024-09-21T06:04:32.783317+0000","last_peered":"2024-09-21T06:04:32.783317+0000","last_clean":"2024-09-21T06:04:32.783317+0000","last_became_active":"2024-09-21T06:04:32.771506+0000","last_became_peered":"2024-09-21T06:04:32.771506+0000","last_unstale":"2024-09-21T06:04:32.783317+0000","last_undegraded":"2024-09-21T06:04:32.783317+0000","last_fullsized":"2024-09-21T06:04:32.783317+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:39.994097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.778492+0000","last_change":"2024-09-21T06:04:32.775939+0000","last_active":"2024-09-21T06:04:32.778492+0000","last_peered":"2024-09-21T06:04:32.778492+0000","last_clean":"2024-09-21T06:04:32.778492+0000","last_became_active":"2024-09-21T06:04:32.775526+0000","last_became_peered":"2024-09-21T06:04:32.775526+0000","last_unstale":"2024-09-21T06:04:32.778492+0000","last_undegraded":"2024-09-21T06:04:32.778492+0000","last_fullsized":"2024-09-21T06:04:32.778492+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:17.326812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.769320+0000","last_change":"2024-09-21T06:04:32.768237+0000","last_active":"2024-09-21T06:04:32.769320+0000","last_peered":"2024-09-21T06:04:32.769320+0000","last_clean":"2024-09-21T06:04:32.769320+0000","last_became_active":"2024-09-21T06:04:32.767480+0000","last_became_peered":"2024-09-21T06:04:32.767480+0000","last_unstale":"2024-09-21T06:04:32.769320+0000","last_undegraded":"2024-09-21T06:04:32.769320+0000","last_fullsized":"2024-09-21T06:04:32.769320+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:37.256481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793616+0000","last_change":"2024-09-21T06:04:32.786243+0000","last_active":"2024-09-21T06:04:32.793616+0000","last_peered":"2024-09-21T06:04:32.793616+0000","last_clean":"2024-09-21T06:04:32.793616+0000","last_became_active":"2024-09-21T06:04:32.785278+0000","last_became_peered":"2024-09-21T06:04:32.785278+0000","last_unstale":"2024-09-21T06:04:32.793616+0000","last_undegraded":"2024-09-21T06:04:32.793616+0000","last_fullsized":"2024-09-21T06:04:32.793616+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:41.742767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.761947+0000","last_change":"2024-09-21T06:04:31.761947+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.761947+0000","last_undegraded":"2024-09-21T06:04:31.761947+0000","last_fullsized":"2024-09-21T06:04:31.761947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.775956+0000","last_change":"2024-09-21T06:04:32.774414+0000","last_active":"2024-09-21T06:04:32.775956+0000","last_peered":"2024-09-21T06:04:32.775956+0000","last_clean":"2024-09-21T06:04:32.775956+0000","last_became_active":"2024-09-21T06:04:32.774110+0000","last_became_peered":"2024-09-21T06:04:32.774110+0000","last_unstale":"2024-09-21T06:04:32.775956+0000","last_undegraded":"2024-09-21T06:04:32.775956+0000","last_fullsized":"2024-09-21T06:04:32.775956+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:11.805943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.766598+0000","last_change":"2024-09-21T06:04:32.760679+0000","last_active":"2024-09-21T06:04:32.766598+0000","last_peered":"2024-09-21T06:04:32.766598+0000","last_clean":"2024-09-21T06:04:32.766598+0000","last_became_active":"2024-09-21T06:04:32.760378+0000","last_became_peered":"2024-09-21T06:04:32.760378+0000","last_unstale":"2024-09-21T06:04:32.766598+0000","last_undegraded":"2024-09-21T06:04:32.766598+0000","last_fullsized":"2024-09-21T06:04:32.766598+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:40.885781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.761919+0000","last_change":"2024-09-21T06:04:32.760660+0000","last_active":"2024-09-21T06:04:32.761919+0000","last_peered":"2024-09-21T06:04:32.761919+0000","last_clean":"2024-09-21T06:04:32.761919+0000","last_became_active":"2024-09-21T06:04:32.760303+0000","last_became_peered":"2024-09-21T06:04:32.760303+0000","last_unstale":"2024-09-21T06:04:32.761919+0000","last_undegraded":"2024-09-21T06:04:32.761919+0000","last_fullsized":"2024-09-21T06:04:32.761919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:04.139988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793365+0000","last_change":"2024-09-21T06:04:32.783570+0000","last_active":"2024-09-21T06:04:32.793365+0000","last_peered":"2024-09-21T06:04:32.793365+0000","last_clean":"2024-09-21T06:04:32.793365+0000","last_became_active":"2024-09-21T06:04:32.783265+0000","last_became_peered":"2024-09-21T06:04:32.783265+0000","last_unstale":"2024-09-21T06:04:32.793365+0000","last_undegraded":"2024-09-21T06:04:32.793365+0000","last_fullsized":"2024-09-21T06:04:32.793365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:49.490698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.778520+0000","last_change":"2024-09-21T06:04:32.776322+0000","last_active":"2024-09-21T06:04:32.778520+0000","last_peered":"2024-09-21T06:04:32.778520+0000","last_clean":"2024-09-21T06:04:32.778520+0000","last_became_active":"2024-09-21T06:04:32.776027+0000","last_became_peered":"2024-09-21T06:04:32.776027+0000","last_unstale":"2024-09-21T06:04:32.778520+0000","last_undegraded":"2024-09-21T06:04:32.778520+0000","last_fullsized":"2024-09-21T06:04:32.778520+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:14.913586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.760440+0000","last_change":"2024-09-21T06:04:31.760440+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.760440+0000","last_undegraded":"2024-09-21T06:04:31.760440+0000","last_fullsized":"2024-09-21T06:04:31.760440+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.773202+0000","last_change":"2024-09-21T06:04:32.771994+0000","last_active":"2024-09-21T06:04:32.773202+0000","last_peered":"2024-09-21T06:04:32.773202+0000","last_clean":"2024-09-21T06:04:32.773202+0000","last_became_active":"2024-09-21T06:04:32.771597+0000","last_became_peered":"2024-09-21T06:04:32.771597+0000","last_unstale":"2024-09-21T06:04:32.773202+0000","last_undegraded":"2024-09-21T06:04:32.773202+0000","last_fullsized":"2024-09-21T06:04:32.773202+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:00.871520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.769001+0000","last_change":"2024-09-21T06:04:32.766745+0000","last_active":"2024-09-21T06:04:32.769001+0000","last_peered":"2024-09-21T06:04:32.769001+0000","last_clean":"2024-09-21T06:04:32.769001+0000","last_became_active":"2024-09-21T06:04:32.766362+0000","last_became_peered":"2024-09-21T06:04:32.766362+0000","last_unstale":"2024-09-21T06:04:32.769001+0000","last_undegraded":"2024-09-21T06:04:32.769001+0000","last_fullsized":"2024-09-21T06:04:32.769001+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:12.666560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.765784+0000","last_change":"2024-09-21T06:04:32.760980+0000","last_active":"2024-09-21T06:04:32.765784+0000","last_peered":"2024-09-21T06:04:32.765784+0000","last_clean":"2024-09-21T06:04:32.765784+0000","last_became_active":"2024-09-21T06:04:32.760648+0000","last_became_peered":"2024-09-21T06:04:32.760648+0000","last_unstale":"2024-09-21T06:04:32.765784+0000","last_undegraded":"2024-09-21T06:04:32.765784+0000","last_fullsized":"2024-09-21T06:04:32.765784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:58.412247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.767175+0000","last_change":"2024-09-21T06:04:32.762713+0000","last_active":"2024-09-21T06:04:32.767175+0000","last_peered":"2024-09-21T06:04:32.767175+0000","last_clean":"2024-09-21T06:04:32.767175+0000","last_became_active":"2024-09-21T06:04:32.762345+0000","last_became_peered":"2024-09-21T06:04:32.762345+0000","last_unstale":"2024-09-21T06:04:32.767175+0000","last_undegraded":"2024-09-21T06:04:32.767175+0000","last_fullsized":"2024-09-21T06:04:32.767175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:54.763471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793898+0000","last_change":"2024-09-21T06:04:32.788061+0000","last_active":"2024-09-21T06:04:32.793898+0000","last_peered":"2024-09-21T06:04:32.793898+0000","last_clean":"2024-09-21T06:04:32.793898+0000","last_became_active":"2024-09-21T06:04:32.787777+0000","last_became_peered":"2024-09-21T06:04:32.787777+0000","last_unstale":"2024-09-21T06:04:32.793898+0000","last_undegraded":"2024-09-21T06:04:32.793898+0000","last_fullsized":"2024-09-21T06:04:32.793898+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:53.442251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794388+0000","last_change":"2024-09-21T06:04:32.790938+0000","last_active":"2024-09-21T06:04:32.794388+0000","last_peered":"2024-09-21T06:04:32.794388+0000","last_clean":"2024-09-21T06:04:32.794388+0000","last_became_active":"2024-09-21T06:04:32.790267+0000","last_became_peered":"2024-09-21T06:04:32.790267+0000","last_unstale":"2024-09-21T06:04:32.794388+0000","last_undegraded":"2024-09-21T06:04:32.794388+0000","last_fullsized":"2024-09-21T06:04:32.794388+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:19.004848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.789947+0000","last_change":"2024-09-21T06:04:32.779963+0000","last_active":"2024-09-21T06:04:32.789947+0000","last_peered":"2024-09-21T06:04:32.789947+0000","last_clean":"2024-09-21T06:04:32.789947+0000","last_became_active":"2024-09-21T06:04:32.779659+0000","last_became_peered":"2024-09-21T06:04:32.779659+0000","last_unstale":"2024-09-21T06:04:32.789947+0000","last_undegraded":"2024-09-21T06:04:32.789947+0000","last_fullsized":"2024-09-21T06:04:32.789947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:08.847286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783071+0000","last_change":"2024-09-21T06:04:32.778117+0000","last_active":"2024-09-21T06:04:32.783071+0000","last_peered":"2024-09-21T06:04:32.783071+0000","last_clean":"2024-09-21T06:04:32.783071+0000","last_became_active":"2024-09-21T06:04:32.777105+0000","last_became_peered":"2024-09-21T06:04:32.777105+0000","last_unstale":"2024-09-21T06:04:32.783071+0000","last_undegraded":"2024-09-21T06:04:32.783071+0000","last_fullsized":"2024-09-21T06:04:32.783071+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:37.388005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781124+0000","last_change":"2024-09-21T06:04:32.771946+0000","last_active":"2024-09-21T06:04:32.781124+0000","last_peered":"2024-09-21T06:04:32.781124+0000","last_clean":"2024-09-21T06:04:32.781124+0000","last_became_active":"2024-09-21T06:04:32.771652+0000","last_became_peered":"2024-09-21T06:04:32.771652+0000","last_unstale":"2024-09-21T06:04:32.781124+0000","last_undegraded":"2024-09-21T06:04:32.781124+0000","last_fullsized":"2024-09-21T06:04:32.781124+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:02.037656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792043+0000","last_change":"2024-09-21T06:04:32.780368+0000","last_active":"2024-09-21T06:04:32.792043+0000","last_peered":"2024-09-21T06:04:32.792043+0000","last_clean":"2024-09-21T06:04:32.792043+0000","last_became_active":"2024-09-21T06:04:32.780021+0000","last_became_peered":"2024-09-21T06:04:32.780021+0000","last_unstale":"2024-09-21T06:04:32.792043+0000","last_undegraded":"2024-09-21T06:04:32.792043+0000","last_fullsized":"2024-09-21T06:04:32.792043+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:57.984993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.779808+0000","last_change":"2024-09-21T06:04:32.773537+0000","last_active":"2024-09-21T06:04:32.779808+0000","last_peered":"2024-09-21T06:04:32.779808+0000","last_clean":"2024-09-21T06:04:32.779808+0000","last_became_active":"2024-09-21T06:04:32.773240+0000","last_became_peered":"2024-09-21T06:04:32.773240+0000","last_unstale":"2024-09-21T06:04:32.779808+0000","last_undegraded":"2024-09-21T06:04:32.779808+0000","last_fullsized":"2024-09-21T06:04:32.779808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:04.990837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.789169+0000","last_change":"2024-09-21T06:04:32.782666+0000","last_active":"2024-09-21T06:04:32.789169+0000","last_peered":"2024-09-21T06:04:32.789169+0000","last_clean":"2024-09-21T06:04:32.789169+0000","last_became_active":"2024-09-21T06:04:32.777455+0000","last_became_peered":"2024-09-21T06:04:32.777455+0000","last_unstale":"2024-09-21T06:04:32.789169+0000","last_undegraded":"2024-09-21T06:04:32.789169+0000","last_fullsized":"2024-09-21T06:04:32.789169+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:53.442251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.768646+0000","last_change":"2024-09-21T06:04:32.764381+0000","last_active":"2024-09-21T06:04:32.768646+0000","last_peered":"2024-09-21T06:04:32.768646+0000","last_clean":"2024-09-21T06:04:32.768646+0000","last_became_active":"2024-09-21T06:04:32.764040+0000","last_became_peered":"2024-09-21T06:04:32.764040+0000","last_unstale":"2024-09-21T06:04:32.768646+0000","last_undegraded":"2024-09-21T06:04:32.768646+0000","last_fullsized":"2024-09-21T06:04:32.768646+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:33.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783350+0000","last_change":"2024-09-21T06:04:32.774466+0000","last_active":"2024-09-21T06:04:32.783350+0000","last_peered":"2024-09-21T06:04:32.783350+0000","last_clean":"2024-09-21T06:04:32.783350+0000","last_became_active":"2024-09-21T06:04:32.774133+0000","last_became_peered":"2024-09-21T06:04:32.774133+0000","last_unstale":"2024-09-21T06:04:32.783350+0000","last_undegraded":"2024-09-21T06:04:32.783350+0000","last_fullsized":"2024-09-21T06:04:32.783350+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:58.412247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794461+0000","last_change":"2024-09-21T06:04:32.792389+0000","last_active":"2024-09-21T06:04:32.794461+0000","last_peered":"2024-09-21T06:04:32.794461+0000","last_clean":"2024-09-21T06:04:32.794461+0000","last_became_active":"2024-09-21T06:04:32.792097+0000","last_became_peered":"2024-09-21T06:04:32.792097+0000","last_unstale":"2024-09-21T06:04:32.794461+0000","last_undegraded":"2024-09-21T06:04:32.794461+0000","last_fullsized":"2024-09-21T06:04:32.794461+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:46.730158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793238+0000","last_change":"2024-09-21T06:04:31.777083+0000","last_active":"2024-09-21T06:04:32.793238+0000","last_peered":"2024-09-21T06:04:32.793238+0000","last_clean":"2024-09-21T06:04:32.793238+0000","last_became_active":"2024-09-21T06:04:31.776739+0000","last_became_peered":"2024-09-21T06:04:31.776739+0000","last_unstale":"2024-09-21T06:04:32.793238+0000","last_undegraded":"2024-09-21T06:04:32.793238+0000","last_fullsized":"2024-09-21T06:04:32.793238+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:59.686723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783664+0000","last_change":"2024-09-21T06:04:32.776263+0000","last_active":"2024-09-21T06:04:32.783664+0000","last_peered":"2024-09-21T06:04:32.783664+0000","last_clean":"2024-09-21T06:04:32.783664+0000","last_became_active":"2024-09-21T06:04:32.775659+0000","last_became_peered":"2024-09-21T06:04:32.775659+0000","last_unstale":"2024-09-21T06:04:32.783664+0000","last_undegraded":"2024-09-21T06:04:32.783664+0000","last_fullsized":"2024-09-21T06:04:32.783664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:22.506119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.784109+0000","last_change":"2024-09-21T06:04:31.769588+0000","last_active":"2024-09-21T06:04:32.784109+0000","last_peered":"2024-09-21T06:04:32.784109+0000","last_clean":"2024-09-21T06:04:32.784109+0000","last_became_active":"2024-09-21T06:04:31.768905+0000","last_became_peered":"2024-09-21T06:04:31.768905+0000","last_unstale":"2024-09-21T06:04:32.784109+0000","last_undegraded":"2024-09-21T06:04:32.784109+0000","last_fullsized":"2024-09-21T06:04:32.784109+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:27.975438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.789767+0000","last_change":"2024-09-21T06:04:32.788286+0000","last_active":"2024-09-21T06:04:32.789767+0000","last_peered":"2024-09-21T06:04:32.789767+0000","last_clean":"2024-09-21T06:04:32.789767+0000","last_became_active":"2024-09-21T06:04:32.787926+0000","last_became_peered":"2024-09-21T06:04:32.787926+0000","last_unstale":"2024-09-21T06:04:32.789767+0000","last_undegraded":"2024-09-21T06:04:32.789767+0000","last_fullsized":"2024-09-21T06:04:32.789767+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:08.847286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794349+0000","last_change":"2024-09-21T06:04:31.772963+0000","last_active":"2024-09-21T06:04:32.794349+0000","last_peered":"2024-09-21T06:04:32.794349+0000","last_clean":"2024-09-21T06:04:32.794349+0000","last_became_active":"2024-09-21T06:04:31.772653+0000","last_became_peered":"2024-09-21T06:04:31.772653+0000","last_unstale":"2024-09-21T06:04:32.794349+0000","last_undegraded":"2024-09-21T06:04:32.794349+0000","last_fullsized":"2024-09-21T06:04:32.794349+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:59.868880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793482+0000","last_change":"2024-09-21T06:04:32.785502+0000","last_active":"2024-09-21T06:04:32.793482+0000","last_peered":"2024-09-21T06:04:32.793482+0000","last_clean":"2024-09-21T06:04:32.793482+0000","last_became_active":"2024-09-21T06:04:32.785091+0000","last_became_peered":"2024-09-21T06:04:32.785091+0000","last_unstale":"2024-09-21T06:04:32.793482+0000","last_undegraded":"2024-09-21T06:04:32.793482+0000","last_fullsized":"2024-09-21T06:04:32.793482+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:11.518104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.788989+0000","last_change":"2024-09-21T06:04:31.775276+0000","last_active":"2024-09-21T06:04:32.788989+0000","last_peered":"2024-09-21T06:04:32.788989+0000","last_clean":"2024-09-21T06:04:32.788989+0000","last_became_active":"2024-09-21T06:04:31.774028+0000","last_became_peered":"2024-09-21T06:04:31.774028+0000","last_unstale":"2024-09-21T06:04:32.788989+0000","last_undegraded":"2024-09-21T06:04:32.788989+0000","last_fullsized":"2024-09-21T06:04:32.788989+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:18.843912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783561+0000","last_change":"2024-09-21T06:04:32.777770+0000","last_active":"2024-09-21T06:04:32.783561+0000","last_peered":"2024-09-21T06:04:32.783561+0000","last_clean":"2024-09-21T06:04:32.783561+0000","last_became_active":"2024-09-21T06:04:32.777019+0000","last_became_peered":"2024-09-21T06:04:32.777019+0000","last_unstale":"2024-09-21T06:04:32.783561+0000","last_undegraded":"2024-09-21T06:04:32.783561+0000","last_fullsized":"2024-09-21T06:04:32.783561+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:59.802808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793522+0000","last_change":"2024-09-21T06:04:31.766081+0000","last_active":"2024-09-21T06:04:32.793522+0000","last_peered":"2024-09-21T06:04:32.793522+0000","last_clean":"2024-09-21T06:04:32.793522+0000","last_became_active":"2024-09-21T06:04:31.765541+0000","last_became_peered":"2024-09-21T06:04:31.765541+0000","last_unstale":"2024-09-21T06:04:32.793522+0000","last_undegraded":"2024-09-21T06:04:32.793522+0000","last_fullsized":"2024-09-21T06:04:32.793522+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:40.183984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792327+0000","last_change":"2024-09-21T06:04:32.780478+0000","last_active":"2024-09-21T06:04:32.792327+0000","last_peered":"2024-09-21T06:04:32.792327+0000","last_clean":"2024-09-21T06:04:32.792327+0000","last_became_active":"2024-09-21T06:04:32.778124+0000","last_became_peered":"2024-09-21T06:04:32.778124+0000","last_unstale":"2024-09-21T06:04:32.792327+0000","last_undegraded":"2024-09-21T06:04:32.792327+0000","last_fullsized":"2024-09-21T06:04:32.792327+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:26.746417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.787610+0000","last_change":"2024-09-21T06:04:31.772094+0000","last_active":"2024-09-21T06:04:32.787610+0000","last_peered":"2024-09-21T06:04:32.787610+0000","last_clean":"2024-09-21T06:04:32.787610+0000","last_became_active":"2024-09-21T06:04:31.771544+0000","last_became_peered":"2024-09-21T06:04:31.771544+0000","last_unstale":"2024-09-21T06:04:32.787610+0000","last_undegraded":"2024-09-21T06:04:32.787610+0000","last_fullsized":"2024-09-21T06:04:32.787610+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:03.600601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.767075+0000","last_change":"2024-09-21T06:04:32.761010+0000","last_active":"2024-09-21T06:04:32.767075+0000","last_peered":"2024-09-21T06:04:32.767075+0000","last_clean":"2024-09-21T06:04:32.767075+0000","last_became_active":"2024-09-21T06:04:32.760708+0000","last_became_peered":"2024-09-21T06:04:32.760708+0000","last_unstale":"2024-09-21T06:04:32.767075+0000","last_undegraded":"2024-09-21T06:04:32.767075+0000","last_fullsized":"2024-09-21T06:04:32.767075+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:45.868263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.769744+0000","last_change":"2024-09-21T06:04:31.768175+0000","last_active":"2024-09-21T06:04:31.769744+0000","last_peered":"2024-09-21T06:04:31.769744+0000","last_clean":"2024-09-21T06:04:31.769744+0000","last_became_active":"2024-09-21T06:04:31.767820+0000","last_became_peered":"2024-09-21T06:04:31.767820+0000","last_unstale":"2024-09-21T06:04:31.769744+0000","last_undegraded":"2024-09-21T06:04:31.769744+0000","last_fullsized":"2024-09-21T06:04:31.769744+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:24.459378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.775720+0000","last_change":"2024-09-21T06:04:32.772784+0000","last_active":"2024-09-21T06:04:32.775720+0000","last_peered":"2024-09-21T06:04:32.775720+0000","last_clean":"2024-09-21T06:04:32.775720+0000","last_became_active":"2024-09-21T06:04:32.772246+0000","last_became_peered":"2024-09-21T06:04:32.772246+0000","last_unstale":"2024-09-21T06:04:32.775720+0000","last_undegraded":"2024-09-21T06:04:32.775720+0000","last_fullsized":"2024-09-21T06:04:32.775720+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:39.036138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.752202+0000","last_change":"2024-09-21T06:04:31.759185+0000","last_active":"2024-09-21T06:04:32.752202+0000","last_peered":"2024-09-21T06:04:32.752202+0000","last_clean":"2024-09-21T06:04:32.752202+0000","last_became_active":"2024-09-21T06:04:31.758873+0000","last_became_peered":"2024-09-21T06:04:31.758873+0000","last_unstale":"2024-09-21T06:04:32.752202+0000","last_undegraded":"2024-09-21T06:04:32.752202+0000","last_fullsized":"2024-09-21T06:04:32.752202+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:37.617451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.770924+0000","last_change":"2024-09-21T06:04:32.769066+0000","last_active":"2024-09-21T06:04:32.770924+0000","last_peered":"2024-09-21T06:04:32.770924+0000","last_clean":"2024-09-21T06:04:32.770924+0000","last_became_active":"2024-09-21T06:04:32.768695+0000","last_became_peered":"2024-09-21T06:04:32.768695+0000","last_unstale":"2024-09-21T06:04:32.770924+0000","last_undegraded":"2024-09-21T06:04:32.770924+0000","last_fullsized":"2024-09-21T06:04:32.770924+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:41.742767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.755039+0000","last_change":"2024-09-21T06:04:31.770884+0000","last_active":"2024-09-21T06:04:32.755039+0000","last_peered":"2024-09-21T06:04:32.755039+0000","last_clean":"2024-09-21T06:04:32.755039+0000","last_became_active":"2024-09-21T06:04:31.768869+0000","last_became_peered":"2024-09-21T06:04:31.768869+0000","last_unstale":"2024-09-21T06:04:32.755039+0000","last_undegraded":"2024-09-21T06:04:32.755039+0000","last_fullsized":"2024-09-21T06:04:32.755039+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:18.843912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794408+0000","last_change":"2024-09-21T06:04:32.791807+0000","last_active":"2024-09-21T06:04:32.794408+0000","last_peered":"2024-09-21T06:04:32.794408+0000","last_clean":"2024-09-21T06:04:32.794408+0000","last_became_active":"2024-09-21T06:04:32.791461+0000","last_became_peered":"2024-09-21T06:04:32.791461+0000","last_unstale":"2024-09-21T06:04:32.794408+0000","last_undegraded":"2024-09-21T06:04:32.794408+0000","last_fullsized":"2024-09-21T06:04:32.794408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:47.832984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.766601+0000","last_change":"2024-09-21T06:04:31.770081+0000","last_active":"2024-09-21T06:04:32.766601+0000","last_peered":"2024-09-21T06:04:32.766601+0000","last_clean":"2024-09-21T06:04:32.766601+0000","last_became_active":"2024-09-21T06:04:31.768462+0000","last_became_peered":"2024-09-21T06:04:31.768462+0000","last_unstale":"2024-09-21T06:04:32.766601+0000","last_undegraded":"2024-09-21T06:04:32.766601+0000","last_fullsized":"2024-09-21T06:04:32.766601+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:10.542640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.796309+0000","last_change":"2024-09-21T06:04:32.795515+0000","last_active":"2024-09-21T06:04:32.796309+0000","last_peered":"2024-09-21T06:04:32.796309+0000","last_clean":"2024-09-21T06:04:32.796309+0000","last_became_active":"2024-09-21T06:04:32.795019+0000","last_became_peered":"2024-09-21T06:04:32.795019+0000","last_unstale":"2024-09-21T06:04:32.796309+0000","last_undegraded":"2024-09-21T06:04:32.796309+0000","last_fullsized":"2024-09-21T06:04:32.796309+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:26.746417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792146+0000","last_change":"2024-09-21T06:04:31.769723+0000","last_active":"2024-09-21T06:04:32.792146+0000","last_peered":"2024-09-21T06:04:32.792146+0000","last_clean":"2024-09-21T06:04:32.792146+0000","last_became_active":"2024-09-21T06:04:31.769344+0000","last_became_peered":"2024-09-21T06:04:31.769344+0000","last_unstale":"2024-09-21T06:04:32.792146+0000","last_undegraded":"2024-09-21T06:04:32.792146+0000","last_fullsized":"2024-09-21T06:04:32.792146+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:00.566146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.768354+0000","last_change":"2024-09-21T06:04:32.767569+0000","last_active":"2024-09-21T06:04:32.768354+0000","last_peered":"2024-09-21T06:04:32.768354+0000","last_clean":"2024-09-21T06:04:32.768354+0000","last_became_active":"2024-09-21T06:04:32.766934+0000","last_became_peered":"2024-09-21T06:04:32.766934+0000","last_unstale":"2024-09-21T06:04:32.768354+0000","last_undegraded":"2024-09-21T06:04:32.768354+0000","last_fullsized":"2024-09-21T06:04:32.768354+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:56.103744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.770166+0000","last_change":"2024-09-21T06:04:31.768773+0000","last_active":"2024-09-21T06:04:31.770166+0000","last_peered":"2024-09-21T06:04:31.770166+0000","last_clean":"2024-09-21T06:04:31.770166+0000","last_became_active":"2024-09-21T06:04:31.767292+0000","last_became_peered":"2024-09-21T06:04:31.767292+0000","last_unstale":"2024-09-21T06:04:31.770166+0000","last_undegraded":"2024-09-21T06:04:31.770166+0000","last_fullsized":"2024-09-21T06:04:31.770166+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:32.705430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792225+0000","last_change":"2024-09-21T06:04:32.784585+0000","last_active":"2024-09-21T06:04:32.792225+0000","last_peered":"2024-09-21T06:04:32.792225+0000","last_clean":"2024-09-21T06:04:32.792225+0000","last_became_active":"2024-09-21T06:04:32.784259+0000","last_became_peered":"2024-09-21T06:04:32.784259+0000","last_unstale":"2024-09-21T06:04:32.792225+0000","last_undegraded":"2024-09-21T06:04:32.792225+0000","last_fullsized":"2024-09-21T06:04:32.792225+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:11.446707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.773742+0000","last_change":"2024-09-21T06:04:31.772181+0000","last_active":"2024-09-21T06:04:31.773742+0000","last_peered":"2024-09-21T06:04:31.773742+0000","last_clean":"2024-09-21T06:04:31.773742+0000","last_became_active":"2024-09-21T06:04:31.771816+0000","last_became_peered":"2024-09-21T06:04:31.771816+0000","last_unstale":"2024-09-21T06:04:31.773742+0000","last_undegraded":"2024-09-21T06:04:31.773742+0000","last_fullsized":"2024-09-21T06:04:31.773742+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:47.864470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.770503+0000","last_change":"2024-09-21T06:04:32.768202+0000","last_active":"2024-09-21T06:04:32.770503+0000","last_peered":"2024-09-21T06:04:32.770503+0000","last_clean":"2024-09-21T06:04:32.770503+0000","last_became_active":"2024-09-21T06:04:32.767810+0000","last_became_peered":"2024-09-21T06:04:32.767810+0000","last_unstale":"2024-09-21T06:04:32.770503+0000","last_undegraded":"2024-09-21T06:04:32.770503+0000","last_fullsized":"2024-09-21T06:04:32.770503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:11.518104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.763381+0000","last_change":"2024-09-21T06:04:31.775115+0000","last_active":"2024-09-21T06:04:32.763381+0000","last_peered":"2024-09-21T06:04:32.763381+0000","last_clean":"2024-09-21T06:04:32.763381+0000","last_became_active":"2024-09-21T06:04:31.773039+0000","last_became_peered":"2024-09-21T06:04:31.773039+0000","last_unstale":"2024-09-21T06:04:32.763381+0000","last_undegraded":"2024-09-21T06:04:32.763381+0000","last_fullsized":"2024-09-21T06:04:32.763381+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:54.006741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781404+0000","last_change":"2024-09-21T06:04:32.780256+0000","last_active":"2024-09-21T06:04:32.781404+0000","last_peered":"2024-09-21T06:04:32.781404+0000","last_clean":"2024-09-21T06:04:32.781404+0000","last_became_active":"2024-09-21T06:04:32.779924+0000","last_became_peered":"2024-09-21T06:04:32.779924+0000","last_unstale":"2024-09-21T06:04:32.781404+0000","last_undegraded":"2024-09-21T06:04:32.781404+0000","last_fullsized":"2024-09-21T06:04:32.781404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:52.831924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.756793+0000","last_change":"2024-09-21T06:04:31.775645+0000","last_active":"2024-09-21T06:04:32.756793+0000","last_peered":"2024-09-21T06:04:32.756793+0000","last_clean":"2024-09-21T06:04:32.756793+0000","last_became_active":"2024-09-21T06:04:31.775129+0000","last_became_peered":"2024-09-21T06:04:31.775129+0000","last_unstale":"2024-09-21T06:04:32.756793+0000","last_undegraded":"2024-09-21T06:04:32.756793+0000","last_fullsized":"2024-09-21T06:04:32.756793+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:55.569554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.771789+0000","last_change":"2024-09-21T06:04:32.770814+0000","last_active":"2024-09-21T06:04:32.771789+0000","last_peered":"2024-09-21T06:04:32.771789+0000","last_clean":"2024-09-21T06:04:32.771789+0000","last_became_active":"2024-09-21T06:04:32.770411+0000","last_became_peered":"2024-09-21T06:04:32.770411+0000","last_unstale":"2024-09-21T06:04:32.771789+0000","last_undegraded":"2024-09-21T06:04:32.771789+0000","last_fullsized":"2024-09-21T06:04:32.771789+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:17.290747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793295+0000","last_change":"2024-09-21T06:04:31.772137+0000","last_active":"2024-09-21T06:04:32.793295+0000","last_peered":"2024-09-21T06:04:32.793295+0000","last_clean":"2024-09-21T06:04:32.793295+0000","last_became_active":"2024-09-21T06:04:31.771622+0000","last_became_peered":"2024-09-21T06:04:31.771622+0000","last_unstale":"2024-09-21T06:04:32.793295+0000","last_undegraded":"2024-09-21T06:04:32.793295+0000","last_fullsized":"2024-09-21T06:04:32.793295+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:38.991457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.769708+0000","last_change":"2024-09-21T06:04:32.768582+0000","last_active":"2024-09-21T06:04:32.769708+0000","last_peered":"2024-09-21T06:04:32.769708+0000","last_clean":"2024-09-21T06:04:32.769708+0000","last_became_active":"2024-09-21T06:04:32.767918+0000","last_became_peered":"2024-09-21T06:04:32.767918+0000","last_unstale":"2024-09-21T06:04:32.769708+0000","last_undegraded":"2024-09-21T06:04:32.769708+0000","last_fullsized":"2024-09-21T06:04:32.769708+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:49.109988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.786154+0000","last_change":"2024-09-21T06:04:31.775324+0000","last_active":"2024-09-21T06:04:32.786154+0000","last_peered":"2024-09-21T06:04:32.786154+0000","last_clean":"2024-09-21T06:04:32.786154+0000","last_became_active":"2024-09-21T06:04:31.774989+0000","last_became_peered":"2024-09-21T06:04:31.774989+0000","last_unstale":"2024-09-21T06:04:32.786154+0000","last_undegraded":"2024-09-21T06:04:32.786154+0000","last_fullsized":"2024-09-21T06:04:32.786154+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:10.782962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793956+0000","last_change":"2024-09-21T06:04:32.787340+0000","last_active":"2024-09-21T06:04:32.793956+0000","last_peered":"2024-09-21T06:04:32.793956+0000","last_clean":"2024-09-21T06:04:32.793956+0000","last_became_active":"2024-09-21T06:04:32.787027+0000","last_became_peered":"2024-09-21T06:04:32.787027+0000","last_unstale":"2024-09-21T06:04:32.793956+0000","last_undegraded":"2024-09-21T06:04:32.793956+0000","last_fullsized":"2024-09-21T06:04:32.793956+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:37.388005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.791893+0000","last_change":"2024-09-21T06:04:31.774380+0000","last_active":"2024-09-21T06:04:32.791893+0000","last_peered":"2024-09-21T06:04:32.791893+0000","last_clean":"2024-09-21T06:04:32.791893+0000","last_became_active":"2024-09-21T06:04:31.773835+0000","last_became_peered":"2024-09-21T06:04:31.773835+0000","last_unstale":"2024-09-21T06:04:32.791893+0000","last_undegraded":"2024-09-21T06:04:32.791893+0000","last_fullsized":"2024-09-21T06:04:32.791893+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:53.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.759154+0000","last_change":"2024-09-21T06:04:31.759154+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.759154+0000","last_undegraded":"2024-09-21T06:04:31.759154+0000","last_fullsized":"2024-09-21T06:04:31.759154+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.795017+0000","last_change":"2024-09-21T06:04:31.768595+0000","last_active":"2024-09-21T06:04:32.795017+0000","last_peered":"2024-09-21T06:04:32.795017+0000","last_clean":"2024-09-21T06:04:32.795017+0000","last_became_active":"2024-09-21T06:04:31.767603+0000","last_became_peered":"2024-09-21T06:04:31.767603+0000","last_unstale":"2024-09-21T06:04:32.795017+0000","last_undegraded":"2024-09-21T06:04:32.795017+0000","last_fullsized":"2024-09-21T06:04:32.795017+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:53.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.774981+0000","last_change":"2024-09-21T06:04:32.774290+0000","last_active":"2024-09-21T06:04:32.774981+0000","last_peered":"2024-09-21T06:04:32.774981+0000","last_clean":"2024-09-21T06:04:32.774981+0000","last_became_active":"2024-09-21T06:04:32.774023+0000","last_became_peered":"2024-09-21T06:04:32.774023+0000","last_unstale":"2024-09-21T06:04:32.774981+0000","last_undegraded":"2024-09-21T06:04:32.774981+0000","last_fullsized":"2024-09-21T06:04:32.774981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.944676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.774811+0000","last_change":"2024-09-21T06:04:31.774500+0000","last_active":"2024-09-21T06:04:32.774811+0000","last_peered":"2024-09-21T06:04:32.774811+0000","last_clean":"2024-09-21T06:04:32.774811+0000","last_became_active":"2024-09-21T06:04:31.774161+0000","last_became_peered":"2024-09-21T06:04:31.774161+0000","last_unstale":"2024-09-21T06:04:32.774811+0000","last_undegraded":"2024-09-21T06:04:32.774811+0000","last_fullsized":"2024-09-21T06:04:32.774811+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:32.288314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.773796+0000","last_change":"2024-09-21T06:04:32.769815+0000","last_active":"2024-09-21T06:04:32.773796+0000","last_peered":"2024-09-21T06:04:32.773796+0000","last_clean":"2024-09-21T06:04:32.773796+0000","last_became_active":"2024-09-21T06:04:32.769467+0000","last_became_peered":"2024-09-21T06:04:32.769467+0000","last_unstale":"2024-09-21T06:04:32.773796+0000","last_undegraded":"2024-09-21T06:04:32.773796+0000","last_fullsized":"2024-09-21T06:04:32.773796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:44.954801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.763651+0000","last_change":"2024-09-21T06:04:31.772131+0000","last_active":"2024-09-21T06:04:32.763651+0000","last_peered":"2024-09-21T06:04:32.763651+0000","last_clean":"2024-09-21T06:04:32.763651+0000","last_became_active":"2024-09-21T06:04:31.771792+0000","last_became_peered":"2024-09-21T06:04:31.771792+0000","last_unstale":"2024-09-21T06:04:32.763651+0000","last_undegraded":"2024-09-21T06:04:32.763651+0000","last_fullsized":"2024-09-21T06:04:32.763651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.561776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783512+0000","last_change":"2024-09-21T06:04:32.782337+0000","last_active":"2024-09-21T06:04:32.783512+0000","last_peered":"2024-09-21T06:04:32.783512+0000","last_clean":"2024-09-21T06:04:32.783512+0000","last_became_active":"2024-09-21T06:04:32.782011+0000","last_became_peered":"2024-09-21T06:04:32.782011+0000","last_unstale":"2024-09-21T06:04:32.783512+0000","last_undegraded":"2024-09-21T06:04:32.783512+0000","last_fullsized":"2024-09-21T06:04:32.783512+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:45.735018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.757070+0000","last_change":"2024-09-21T06:04:31.766119+0000","last_active":"2024-09-21T06:04:32.757070+0000","last_peered":"2024-09-21T06:04:32.757070+0000","last_clean":"2024-09-21T06:04:32.757070+0000","last_became_active":"2024-09-21T06:04:31.765398+0000","last_became_peered":"2024-09-21T06:04:31.765398+0000","last_unstale":"2024-09-21T06:04:32.757070+0000","last_undegraded":"2024-09-21T06:04:32.757070+0000","last_fullsized":"2024-09-21T06:04:32.757070+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:23.042000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781363+0000","last_change":"2024-09-21T06:04:32.777061+0000","last_active":"2024-09-21T06:04:32.781363+0000","last_peered":"2024-09-21T06:04:32.781363+0000","last_clean":"2024-09-21T06:04:32.781363+0000","last_became_active":"2024-09-21T06:04:32.776703+0000","last_became_peered":"2024-09-21T06:04:32.776703+0000","last_unstale":"2024-09-21T06:04:32.781363+0000","last_undegraded":"2024-09-21T06:04:32.781363+0000","last_fullsized":"2024-09-21T06:04:32.781363+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:12.555174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.756499+0000","last_change":"2024-09-21T06:04:31.770508+0000","last_active":"2024-09-21T06:04:32.756499+0000","last_peered":"2024-09-21T06:04:32.756499+0000","last_clean":"2024-09-21T06:04:32.756499+0000","last_became_active":"2024-09-21T06:04:31.769020+0000","last_became_peered":"2024-09-21T06:04:31.769020+0000","last_unstale":"2024-09-21T06:04:32.756499+0000","last_undegraded":"2024-09-21T06:04:32.756499+0000","last_fullsized":"2024-09-21T06:04:32.756499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:40.893581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.765322+0000","last_change":"2024-09-21T06:04:32.762522+0000","last_active":"2024-09-21T06:04:32.765322+0000","last_peered":"2024-09-21T06:04:32.765322+0000","last_clean":"2024-09-21T06:04:32.765322+0000","last_became_active":"2024-09-21T06:04:32.762144+0000","last_became_peered":"2024-09-21T06:04:32.762144+0000","last_unstale":"2024-09-21T06:04:32.765322+0000","last_undegraded":"2024-09-21T06:04:32.765322+0000","last_fullsized":"2024-09-21T06:04:32.765322+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:36.679619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.786027+0000","last_change":"2024-09-21T06:04:31.774958+0000","last_active":"2024-09-21T06:04:32.786027+0000","last_peered":"2024-09-21T06:04:32.786027+0000","last_clean":"2024-09-21T06:04:32.786027+0000","last_became_active":"2024-09-21T06:04:31.774595+0000","last_became_peered":"2024-09-21T06:04:31.774595+0000","last_unstale":"2024-09-21T06:04:32.786027+0000","last_undegraded":"2024-09-21T06:04:32.786027+0000","last_fullsized":"2024-09-21T06:04:32.786027+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:05.289920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.770815+0000","last_change":"2024-09-21T06:04:32.766158+0000","last_active":"2024-09-21T06:04:32.770815+0000","last_peered":"2024-09-21T06:04:32.770815+0000","last_clean":"2024-09-21T06:04:32.770815+0000","last_became_active":"2024-09-21T06:04:32.765811+0000","last_became_peered":"2024-09-21T06:04:32.765811+0000","last_unstale":"2024-09-21T06:04:32.770815+0000","last_undegraded":"2024-09-21T06:04:32.770815+0000","last_fullsized":"2024-09-21T06:04:32.770815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:11.446707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.757924+0000","last_change":"2024-09-21T06:04:31.775745+0000","last_active":"2024-09-21T06:04:32.757924+0000","last_peered":"2024-09-21T06:04:32.757924+0000","last_clean":"2024-09-21T06:04:32.757924+0000","last_became_active":"2024-09-21T06:04:31.775037+0000","last_became_peered":"2024-09-21T06:04:31.775037+0000","last_unstale":"2024-09-21T06:04:32.757924+0000","last_undegraded":"2024-09-21T06:04:32.757924+0000","last_fullsized":"2024-09-21T06:04:32.757924+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:40.310390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794553+0000","last_change":"2024-09-21T06:04:32.791087+0000","last_active":"2024-09-21T06:04:32.794553+0000","last_peered":"2024-09-21T06:04:32.794553+0000","last_clean":"2024-09-21T06:04:32.794553+0000","last_became_active":"2024-09-21T06:04:32.790735+0000","last_became_peered":"2024-09-21T06:04:32.790735+0000","last_unstale":"2024-09-21T06:04:32.794553+0000","last_undegraded":"2024-09-21T06:04:32.794553+0000","last_fullsized":"2024-09-21T06:04:32.794553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:30.386322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.751374+0000","last_change":"2024-09-21T06:04:31.761815+0000","last_active":"2024-09-21T06:04:32.751374+0000","last_peered":"2024-09-21T06:04:32.751374+0000","last_clean":"2024-09-21T06:04:32.751374+0000","last_became_active":"2024-09-21T06:04:31.761487+0000","last_became_peered":"2024-09-21T06:04:31.761487+0000","last_unstale":"2024-09-21T06:04:32.751374+0000","last_undegraded":"2024-09-21T06:04:32.751374+0000","last_fullsized":"2024-09-21T06:04:32.751374+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:02:35.169801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794037+0000","last_change":"2024-09-21T06:04:32.788888+0000","last_active":"2024-09-21T06:04:32.794037+0000","last_peered":"2024-09-21T06:04:32.794037+0000","last_clean":"2024-09-21T06:04:32.794037+0000","last_became_active":"2024-09-21T06:04:32.788597+0000","last_became_peered":"2024-09-21T06:04:32.788597+0000","last_unstale":"2024-09-21T06:04:32.794037+0000","last_undegraded":"2024-09-21T06:04:32.794037+0000","last_fullsized":"2024-09-21T06:04:32.794037+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:12.183155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.761494+0000","last_change":"2024-09-21T06:04:31.762529+0000","last_active":"2024-09-21T06:04:32.761494+0000","last_peered":"2024-09-21T06:04:32.761494+0000","last_clean":"2024-09-21T06:04:32.761494+0000","last_became_active":"2024-09-21T06:04:31.762189+0000","last_became_peered":"2024-09-21T06:04:31.762189+0000","last_unstale":"2024-09-21T06:04:32.761494+0000","last_undegraded":"2024-09-21T06:04:32.761494+0000","last_fullsized":"2024-09-21T06:04:32.761494+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:51.822586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.773682+0000","last_change":"2024-09-21T06:04:32.770727+0000","last_active":"2024-09-21T06:04:32.773682+0000","last_peered":"2024-09-21T06:04:32.773682+0000","last_clean":"2024-09-21T06:04:32.773682+0000","last_became_active":"2024-09-21T06:04:32.770393+0000","last_became_peered":"2024-09-21T06:04:32.770393+0000","last_unstale":"2024-09-21T06:04:32.773682+0000","last_undegraded":"2024-09-21T06:04:32.773682+0000","last_fullsized":"2024-09-21T06:04:32.773682+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:23.599523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.751186+0000","last_change":"2024-09-21T06:04:31.768859+0000","last_active":"2024-09-21T06:04:32.751186+0000","last_peered":"2024-09-21T06:04:32.751186+0000","last_clean":"2024-09-21T06:04:32.751186+0000","last_became_active":"2024-09-21T06:04:31.768535+0000","last_became_peered":"2024-09-21T06:04:31.768535+0000","last_unstale":"2024-09-21T06:04:32.751186+0000","last_undegraded":"2024-09-21T06:04:32.751186+0000","last_fullsized":"2024-09-21T06:04:32.751186+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:05.572103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.786505+0000","last_change":"2024-09-21T06:04:32.782856+0000","last_active":"2024-09-21T06:04:32.786505+0000","last_peered":"2024-09-21T06:04:32.786505+0000","last_clean":"2024-09-21T06:04:32.786505+0000","last_became_active":"2024-09-21T06:04:32.782532+0000","last_became_peered":"2024-09-21T06:04:32.782532+0000","last_unstale":"2024-09-21T06:04:32.786505+0000","last_undegraded":"2024-09-21T06:04:32.786505+0000","last_fullsized":"2024-09-21T06:04:32.786505+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:49.109988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.770750+0000","last_change":"2024-09-21T06:04:31.768251+0000","last_active":"2024-09-21T06:04:31.770750+0000","last_peered":"2024-09-21T06:04:31.770750+0000","last_clean":"2024-09-21T06:04:31.770750+0000","last_became_active":"2024-09-21T06:04:31.767918+0000","last_became_peered":"2024-09-21T06:04:31.767918+0000","last_unstale":"2024-09-21T06:04:31.770750+0000","last_undegraded":"2024-09-21T06:04:31.770750+0000","last_fullsized":"2024-09-21T06:04:31.770750+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:28.832034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.760288+0000","last_change":"2024-09-21T06:04:31.760288+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.760288+0000","last_undegraded":"2024-09-21T06:04:31.760288+0000","last_fullsized":"2024-09-21T06:04:31.760288+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.755302+0000","last_change":"2024-09-21T06:04:31.769840+0000","last_active":"2024-09-21T06:04:32.755302+0000","last_peered":"2024-09-21T06:04:32.755302+0000","last_clean":"2024-09-21T06:04:32.755302+0000","last_became_active":"2024-09-21T06:04:31.769534+0000","last_became_peered":"2024-09-21T06:04:31.769534+0000","last_unstale":"2024-09-21T06:04:32.755302+0000","last_undegraded":"2024-09-21T06:04:32.755302+0000","last_fullsized":"2024-09-21T06:04:32.755302+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:10.542640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.784911+0000","last_change":"2024-09-21T06:04:32.779505+0000","last_active":"2024-09-21T06:04:32.784911+0000","last_peered":"2024-09-21T06:04:32.784911+0000","last_clean":"2024-09-21T06:04:32.784911+0000","last_became_active":"2024-09-21T06:04:32.779079+0000","last_became_peered":"2024-09-21T06:04:32.779079+0000","last_unstale":"2024-09-21T06:04:32.784911+0000","last_undegraded":"2024-09-21T06:04:32.784911+0000","last_fullsized":"2024-09-21T06:04:32.784911+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:37.256481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.765120+0000","last_change":"2024-09-21T06:04:31.768166+0000","last_active":"2024-09-21T06:04:32.765120+0000","last_peered":"2024-09-21T06:04:32.765120+0000","last_clean":"2024-09-21T06:04:32.765120+0000","last_became_active":"2024-09-21T06:04:31.767622+0000","last_became_peered":"2024-09-21T06:04:31.767622+0000","last_unstale":"2024-09-21T06:04:32.765120+0000","last_undegraded":"2024-09-21T06:04:32.765120+0000","last_fullsized":"2024-09-21T06:04:32.765120+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:14.758437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.780297+0000","last_change":"2024-09-21T06:04:32.776172+0000","last_active":"2024-09-21T06:04:32.780297+0000","last_peered":"2024-09-21T06:04:32.780297+0000","last_clean":"2024-09-21T06:04:32.780297+0000","last_became_active":"2024-09-21T06:04:32.775877+0000","last_became_peered":"2024-09-21T06:04:32.775877+0000","last_unstale":"2024-09-21T06:04:32.780297+0000","last_undegraded":"2024-09-21T06:04:32.780297+0000","last_fullsized":"2024-09-21T06:04:32.780297+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:54.870810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.788947+0000","last_change":"2024-09-21T06:04:31.772282+0000","last_active":"2024-09-21T06:04:32.788947+0000","last_peered":"2024-09-21T06:04:32.788947+0000","last_clean":"2024-09-21T06:04:32.788947+0000","last_became_active":"2024-09-21T06:04:31.770583+0000","last_became_peered":"2024-09-21T06:04:31.770583+0000","last_unstale":"2024-09-21T06:04:32.788947+0000","last_undegraded":"2024-09-21T06:04:32.788947+0000","last_fullsized":"2024-09-21T06:04:32.788947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:35.915894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":201,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.775042+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:04:32.775042+0000","last_peered":"2024-09-21T06:04:32.775042+0000","last_clean":"2024-09-21T06:04:32.775042+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:04:32.775042+0000","last_undegraded":"2024-09-21T06:04:32.775042+0000","last_fullsized":"2024-09-21T06:04:32.775042+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50656,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321184,"statfs":{"total":96636764160,"available":96584892416,"internally_reserved":0,"allocated":1445888,"data_stored":944799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509542,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":851968,"data_stored":353907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509541,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":847872,"data_stored":353905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50064,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321776,"statfs":{"total":96636764160,"available":96585498624,"internally_reserved":0,"allocated":847872,"data_stored":353885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54232,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317608,"statfs":{"total":96636764160,"available":96581230592,"internally_reserved":0,"allocated":851968,"data_stored":353927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50048,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321792,"statfs":{"total":96636764160,"available":96585515008,"internally_reserved":0,"allocated":831488,"data_stored":347012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509541,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54788,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317052,"statfs":{"total":96636764160,"available":96580661248,"internally_reserved":0,"allocated":1425408,"data_stored":937380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58368,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313472,"statfs":{"total":96636764160,"available":96576995328,"internally_reserved":0,"allocated":831488,"data_stored":347012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:04:37.804 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3c is not active+clean 2024-09-21T06:04:37.804 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:04:31.762434+0000', 'last_change': '2024-09-21T06:04:31.762434+0000', 'last_active': '2024-09-21T06:04:31.738639+0000', 'last_peered': '2024-09-21T06:04:31.738639+0000', 'last_clean': '2024-09-21T06:04:31.738639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:04:31.762434+0000', 'last_undegraded': '2024-09-21T06:04:31.762434+0000', 'last_fullsized': '2024-09-21T06:04:31.762434+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'last_clean_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T06:04:37.804 INFO:tasks.ceph.ceph_manager.ceph:PG 11.34 is not active+clean 2024-09-21T06:04:37.804 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:04:31.761947+0000', 'last_change': '2024-09-21T06:04:31.761947+0000', 'last_active': '2024-09-21T06:04:31.738639+0000', 'last_peered': '2024-09-21T06:04:31.738639+0000', 'last_clean': '2024-09-21T06:04:31.738639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:04:31.761947+0000', 'last_undegraded': '2024-09-21T06:04:31.761947+0000', 'last_fullsized': '2024-09-21T06:04:31.761947+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'last_clean_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T06:04:37.804 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2e is not active+clean 2024-09-21T06:04:37.804 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:04:31.760440+0000', 'last_change': '2024-09-21T06:04:31.760440+0000', 'last_active': '2024-09-21T06:04:31.738639+0000', 'last_peered': '2024-09-21T06:04:31.738639+0000', 'last_clean': '2024-09-21T06:04:31.738639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:04:31.760440+0000', 'last_undegraded': '2024-09-21T06:04:31.760440+0000', 'last_fullsized': '2024-09-21T06:04:31.760440+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'last_clean_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T06:04:37.805 INFO:tasks.ceph.ceph_manager.ceph:PG 11.c is not active+clean 2024-09-21T06:04:37.805 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:04:31.759154+0000', 'last_change': '2024-09-21T06:04:31.759154+0000', 'last_active': '2024-09-21T06:04:31.738639+0000', 'last_peered': '2024-09-21T06:04:31.738639+0000', 'last_clean': '2024-09-21T06:04:31.738639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:04:31.759154+0000', 'last_undegraded': '2024-09-21T06:04:31.759154+0000', 'last_fullsized': '2024-09-21T06:04:31.759154+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'last_clean_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T06:04:37.805 INFO:tasks.ceph.ceph_manager.ceph:PG 11.0 is not active+clean 2024-09-21T06:04:37.805 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:04:31.760288+0000', 'last_change': '2024-09-21T06:04:31.760288+0000', 'last_active': '2024-09-21T06:04:31.738639+0000', 'last_peered': '2024-09-21T06:04:31.738639+0000', 'last_clean': '2024-09-21T06:04:31.738639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:04:31.760288+0000', 'last_undegraded': '2024-09-21T06:04:31.760288+0000', 'last_fullsized': '2024-09-21T06:04:31.760288+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'last_clean_scrub_stamp': '2024-09-21T06:04:31.738639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T06:04:37.805 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:04:38.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.999+0000 7fa092600640 1 -- 172.21.15.50:0/1804562930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa08c0b5c70 msgr2=0x7fa08c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.999+0000 7fa092600640 1 --2- 172.21.15.50:0/1804562930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa08c0b5c70 0x7fa08c15bb60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa08c074b20 tx=0x7fa07c01c760 comp rx=0 tx=0).stop 2024-09-21T06:04:38.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.999+0000 7fa092600640 1 -- 172.21.15.50:0/1804562930 shutdown_connections 2024-09-21T06:04:38.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.999+0000 7fa092600640 1 --2- 172.21.15.50:0/1804562930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa08c0b5c70 0x7fa08c15bb60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.999+0000 7fa092600640 1 -- 172.21.15.50:0/1804562930 >> 172.21.15.50:0/1804562930 conn(0x7fa08c0b2040 msgr2=0x7fa08c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:38.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.999+0000 7fa092600640 1 -- 172.21.15.50:0/1804562930 shutdown_connections 2024-09-21T06:04:38.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.999+0000 7fa092600640 1 -- 172.21.15.50:0/1804562930 wait complete. 2024-09-21T06:04:38.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:37.999+0000 7fa092600640 1 Processor -- start 2024-09-21T06:04:38.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa092600640 1 -- start start 2024-09-21T06:04:38.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa092600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa08c0b4de0 0x7fa08c15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:38.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa092600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c0b5c70 0x7fa08c162100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:38.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa092600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa08c1527b0 0x7fa08c1418b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa092600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c0b4650 con 0x7fa08c0b5c70 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa092600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c0b44d0 con 0x7fa08c1527b0 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa092600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c0b47d0 con 0x7fa08c0b4de0 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa083e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c0b5c70 0x7fa08c162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa083e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c0b5c70 0x7fa08c162100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36676/0 (socket says 172.21.15.50:36676) 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa083e00640 1 -- 172.21.15.50:0/3506094662 learned_addr learned my addr 172.21.15.50:0/3506094662 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa08b400640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa08c0b4de0 0x7fa08c15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa08be00640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa08c1527b0 0x7fa08c1418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa083e00640 1 -- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa08c0b4de0 msgr2=0x7fa08c15e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa083e00640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa08c0b4de0 0x7fa08c15e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa083e00640 1 -- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa08c1527b0 msgr2=0x7fa08c1418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa083e00640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa08c1527b0 0x7fa08c1418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa083e00640 1 -- 172.21.15.50:0/3506094662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c142620 con 0x7fa08c0b5c70 2024-09-21T06:04:38.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa08b400640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa08c0b4de0 0x7fa08c15e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:38.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa083e00640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c0b5c70 0x7fa08c162100 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa07400c400 tx=0x7fa07400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:38.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa083400640 1 -- 172.21.15.50:0/3506094662 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074011790 con 0x7fa08c0b5c70 2024-09-21T06:04:38.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa083400640 1 -- 172.21.15.50:0/3506094662 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa074017070 con 0x7fa08c0b5c70 2024-09-21T06:04:38.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa092600640 1 -- 172.21.15.50:0/3506094662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c15da00 con 0x7fa08c0b5c70 2024-09-21T06:04:38.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa083400640 1 -- 172.21.15.50:0/3506094662 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074011ca0 con 0x7fa08c0b5c70 2024-09-21T06:04:38.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa092600640 1 -- 172.21.15.50:0/3506094662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c15dfc0 con 0x7fa08c0b5c70 2024-09-21T06:04:38.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.003+0000 7fa092600640 1 -- 172.21.15.50:0/3506094662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c15bb60 con 0x7fa08c0b5c70 2024-09-21T06:04:38.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.007+0000 7fa083400640 1 -- 172.21.15.50:0/3506094662 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7fa07401c460 con 0x7fa08c0b5c70 2024-09-21T06:04:38.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.007+0000 7fa083400640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa068073e90 0x7fa0680763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:38.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.007+0000 7fa083400640 1 -- 172.21.15.50:0/3506094662 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7fa074092930 con 0x7fa08c0b5c70 2024-09-21T06:04:38.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.007+0000 7fa08b400640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa068073e90 0x7fa0680763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:38.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.007+0000 7fa08b400640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa068073e90 0x7fa0680763a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0af000 tx=0x7fa07c01afa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:38.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.011+0000 7fa083400640 1 -- 172.21.15.50:0/3506094662 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa07405f430 con 0x7fa08c0b5c70 2024-09-21T06:04:38.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.247+0000 7fa092600640 1 -- 172.21.15.50:0/3506094662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa08c06b250 con 0x7fa08c0b5c70 2024-09-21T06:04:38.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa083400640 1 -- 172.21.15.50:0/3506094662 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1141 (secure 0 0 0) 0x7fa07405ed30 con 0x7fa08c0b5c70 2024-09-21T06:04:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:38.251 INFO:teuthology.orchestra.run.smithi050.stdout:{"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":514,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":71,"num_osds":8,"num_up_osds":8,"osd_up_since":1726898177,"num_in_osds":8,"osd_in_since":1726898167,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":92},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":17,"data_bytes":591132,"bytes_used":428339200,"bytes_avail":772665774080,"bytes_total":773094113280,"inactive_pgs_ratio":0.051546391099691391,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":45,"btime":"2024-09-21T06:04:35:777183+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"d","status":"up:active","gid":6423}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-21T06:03:17.002246+0000","services":{}},"progress_events":{}} 2024-09-21T06:04:38.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa092600640 1 -- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa068073e90 msgr2=0x7fa0680763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa092600640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa068073e90 0x7fa0680763a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0af000 tx=0x7fa07c01afa0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa092600640 1 -- 172.21.15.50:0/3506094662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c0b5c70 msgr2=0x7fa08c162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa092600640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c0b5c70 0x7fa08c162100 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa07400c400 tx=0x7fa07400c8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa092600640 1 -- 172.21.15.50:0/3506094662 shutdown_connections 2024-09-21T06:04:38.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa092600640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa08c1527b0 0x7fa08c1418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa092600640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa068073e90 0x7fa0680763a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa092600640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c0b5c70 0x7fa08c162100 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa092600640 1 --2- 172.21.15.50:0/3506094662 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa08c0b4de0 0x7fa08c15e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa092600640 1 -- 172.21.15.50:0/3506094662 >> 172.21.15.50:0/3506094662 conn(0x7fa08c0b2040 msgr2=0x7fa08c0b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:38.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa092600640 1 -- 172.21.15.50:0/3506094662 shutdown_connections 2024-09-21T06:04:38.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.251+0000 7fa092600640 1 -- 172.21.15.50:0/3506094662 wait complete. 2024-09-21T06:04:38.273 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T06:04:38.274 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:04:38.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.431+0000 7febf7a00640 1 -- 172.21.15.50:0/353791268 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7febf015c5d0 msgr2=0x7febf0073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.431+0000 7febf7a00640 1 --2- 172.21.15.50:0/353791268 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7febf015c5d0 0x7febf0073dc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7febe4009840 tx=0x7febe401c790 comp rx=0 tx=0).stop 2024-09-21T06:04:38.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.431+0000 7febf7a00640 1 -- 172.21.15.50:0/353791268 shutdown_connections 2024-09-21T06:04:38.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.431+0000 7febf7a00640 1 --2- 172.21.15.50:0/353791268 >> v2:172.21.15.88:3300/0 conn(0x7febf0074300 0x7febf015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.431+0000 7febf7a00640 1 --2- 172.21.15.50:0/353791268 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7febf015c5d0 0x7febf0073dc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.431+0000 7febf7a00640 1 -- 172.21.15.50:0/353791268 >> 172.21.15.50:0/353791268 conn(0x7febf00b2c40 msgr2=0x7febf00b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:38.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.431+0000 7febf7a00640 1 -- 172.21.15.50:0/353791268 shutdown_connections 2024-09-21T06:04:38.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.431+0000 7febf7a00640 1 -- 172.21.15.50:0/353791268 wait complete. 2024-09-21T06:04:38.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf7a00640 1 Processor -- start 2024-09-21T06:04:38.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf7a00640 1 -- start start 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf7a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febf0074300 0x7febf01511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf7a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7febf015c5d0 0x7febf014c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf7a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf015f6b0 0x7febf01fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf7a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7febf006f980 con 0x7febf015f6b0 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf7a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7febf00b3d50 con 0x7febf0074300 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf7a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7febf006fb00 con 0x7febf015c5d0 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf015f6b0 0x7febf01fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf015f6b0 0x7febf01fcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36686/0 (socket says 172.21.15.50:36686) 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf5a00640 1 -- 172.21.15.50:0/3153482761 learned_addr learned my addr 172.21.15.50:0/3153482761 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febefe00640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7febf015c5d0 0x7febf014c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf5000640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febf0074300 0x7febf01511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf5a00640 1 -- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7febf015c5d0 msgr2=0x7febf014c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf5a00640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7febf015c5d0 0x7febf014c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf5a00640 1 -- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febf0074300 msgr2=0x7febf01511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf5a00640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febf0074300 0x7febf01511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf5a00640 1 -- 172.21.15.50:0/3153482761 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf00ce720 con 0x7febf015f6b0 2024-09-21T06:04:38.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febefe00640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7febf015c5d0 0x7febf014c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:38.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.435+0000 7febf5a00640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf015f6b0 0x7febf01fcba0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7febe800cb40 tx=0x7febe800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:38.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.439+0000 7febecc00640 1 -- 172.21.15.50:0/3153482761 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8016020 con 0x7febf015f6b0 2024-09-21T06:04:38.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.439+0000 7febecc00640 1 -- 172.21.15.50:0/3153482761 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febe800ab60 con 0x7febf015f6b0 2024-09-21T06:04:38.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.439+0000 7febf7a00640 1 -- 172.21.15.50:0/3153482761 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf00b00c0 con 0x7febf015f6b0 2024-09-21T06:04:38.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.439+0000 7febecc00640 1 -- 172.21.15.50:0/3153482761 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8012420 con 0x7febf015f6b0 2024-09-21T06:04:38.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.439+0000 7febf7a00640 1 -- 172.21.15.50:0/3153482761 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf01f8ce0 con 0x7febf015f6b0 2024-09-21T06:04:38.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.439+0000 7febf7a00640 1 -- 172.21.15.50:0/3153482761 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf015f170 con 0x7febf015f6b0 2024-09-21T06:04:38.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.439+0000 7febecc00640 1 -- 172.21.15.50:0/3153482761 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7febe8019020 con 0x7febf015f6b0 2024-09-21T06:04:38.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.439+0000 7febecc00640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7febd0073e60 0x7febd0076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:38.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.439+0000 7febecc00640 1 -- 172.21.15.50:0/3153482761 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7febe80935c0 con 0x7febf015f6b0 2024-09-21T06:04:38.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.439+0000 7febf5000640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7febd0073e60 0x7febd0076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:38.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.442+0000 7febf5000640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7febd0073e60 0x7febd0076370 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7febe40af000 tx=0x7febe401b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:38.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.442+0000 7febecc00640 1 -- 172.21.15.50:0/3153482761 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7febe8020020 con 0x7febf015f6b0 2024-09-21T06:04:38.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.614+0000 7febf7a00640 1 -- 172.21.15.50:0/3153482761 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7febf006b540 con 0x7febd0073e60 2024-09-21T06:04:38.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.622+0000 7febecc00640 1 -- 172.21.15.50:0/3153482761 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251632 (secure 0 0 0) 0x7febe4004040 con 0x7febd0073e60 2024-09-21T06:04:38.626 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:38.627 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:04:38.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.626+0000 7febf7a00640 1 -- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7febd0073e60 msgr2=0x7febd0076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.626+0000 7febf7a00640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7febd0073e60 0x7febd0076370 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7febe40af000 tx=0x7febe401b030 comp rx=0 tx=0).stop 2024-09-21T06:04:38.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.626+0000 7febf7a00640 1 -- 172.21.15.50:0/3153482761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf015f6b0 msgr2=0x7febf01fcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.626+0000 7febf7a00640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf015f6b0 0x7febf01fcba0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7febe800cb40 tx=0x7febe800cc30 comp rx=0 tx=0).stop 2024-09-21T06:04:38.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.626+0000 7febf7a00640 1 -- 172.21.15.50:0/3153482761 shutdown_connections 2024-09-21T06:04:38.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.626+0000 7febf7a00640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7febd0073e60 0x7febd0076370 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.626+0000 7febf7a00640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf015f6b0 0x7febf01fcba0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.626+0000 7febf7a00640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7febf015c5d0 0x7febf014c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.626+0000 7febf7a00640 1 --2- 172.21.15.50:0/3153482761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febf0074300 0x7febf01511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.626+0000 7febf7a00640 1 -- 172.21.15.50:0/3153482761 >> 172.21.15.50:0/3153482761 conn(0x7febf00b2c40 msgr2=0x7febf006ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:38.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.626+0000 7febf7a00640 1 -- 172.21.15.50:0/3153482761 shutdown_connections 2024-09-21T06:04:38.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:38.626+0000 7febf7a00640 1 -- 172.21.15.50:0/3153482761 wait complete. 2024-09-21T06:04:38.647 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-21T06:04:37.023099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":136,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":418300,"kb_used_data":7748,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754556420,"statfs":{"total":773094113280,"available":772665774080,"internally_reserved":0,"allocated":7933952,"data_stored":3991827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001380"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793750+0000","last_change":"2024-09-21T06:04:32.787310+0000","last_active":"2024-09-21T06:04:32.793750+0000","last_peered":"2024-09-21T06:04:32.793750+0000","last_clean":"2024-09-21T06:04:32.793750+0000","last_became_active":"2024-09-21T06:04:32.786715+0000","last_became_peered":"2024-09-21T06:04:32.786715+0000","last_unstale":"2024-09-21T06:04:32.793750+0000","last_undegraded":"2024-09-21T06:04:32.793750+0000","last_fullsized":"2024-09-21T06:04:32.793750+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:59.802808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.784907+0000","last_change":"2024-09-21T06:04:32.778255+0000","last_active":"2024-09-21T06:04:32.784907+0000","last_peered":"2024-09-21T06:04:32.784907+0000","last_clean":"2024-09-21T06:04:32.784907+0000","last_became_active":"2024-09-21T06:04:32.777803+0000","last_became_peered":"2024-09-21T06:04:32.777803+0000","last_unstale":"2024-09-21T06:04:32.784907+0000","last_undegraded":"2024-09-21T06:04:32.784907+0000","last_fullsized":"2024-09-21T06:04:32.784907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:54.763471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781150+0000","last_change":"2024-09-21T06:04:32.772652+0000","last_active":"2024-09-21T06:04:32.781150+0000","last_peered":"2024-09-21T06:04:32.781150+0000","last_clean":"2024-09-21T06:04:32.781150+0000","last_became_active":"2024-09-21T06:04:32.772322+0000","last_became_peered":"2024-09-21T06:04:32.772322+0000","last_unstale":"2024-09-21T06:04:32.781150+0000","last_undegraded":"2024-09-21T06:04:32.781150+0000","last_fullsized":"2024-09-21T06:04:32.781150+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:36.624434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.762434+0000","last_change":"2024-09-21T06:04:31.762434+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.762434+0000","last_undegraded":"2024-09-21T06:04:31.762434+0000","last_fullsized":"2024-09-21T06:04:31.762434+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783599+0000","last_change":"2024-09-21T06:04:32.777265+0000","last_active":"2024-09-21T06:04:32.783599+0000","last_peered":"2024-09-21T06:04:32.783599+0000","last_clean":"2024-09-21T06:04:32.783599+0000","last_became_active":"2024-09-21T06:04:32.776943+0000","last_became_peered":"2024-09-21T06:04:32.776943+0000","last_unstale":"2024-09-21T06:04:32.783599+0000","last_undegraded":"2024-09-21T06:04:32.783599+0000","last_fullsized":"2024-09-21T06:04:32.783599+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:05.622248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.796017+0000","last_change":"2024-09-21T06:04:32.794992+0000","last_active":"2024-09-21T06:04:32.796017+0000","last_peered":"2024-09-21T06:04:32.796017+0000","last_clean":"2024-09-21T06:04:32.796017+0000","last_became_active":"2024-09-21T06:04:32.794580+0000","last_became_peered":"2024-09-21T06:04:32.794580+0000","last_unstale":"2024-09-21T06:04:32.796017+0000","last_undegraded":"2024-09-21T06:04:32.796017+0000","last_fullsized":"2024-09-21T06:04:32.796017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:57.984993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.768680+0000","last_change":"2024-09-21T06:04:32.765123+0000","last_active":"2024-09-21T06:04:32.768680+0000","last_peered":"2024-09-21T06:04:32.768680+0000","last_clean":"2024-09-21T06:04:32.768680+0000","last_became_active":"2024-09-21T06:04:32.764446+0000","last_became_peered":"2024-09-21T06:04:32.764446+0000","last_unstale":"2024-09-21T06:04:32.768680+0000","last_undegraded":"2024-09-21T06:04:32.768680+0000","last_fullsized":"2024-09-21T06:04:32.768680+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:00.689363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783317+0000","last_change":"2024-09-21T06:04:32.773644+0000","last_active":"2024-09-21T06:04:32.783317+0000","last_peered":"2024-09-21T06:04:32.783317+0000","last_clean":"2024-09-21T06:04:32.783317+0000","last_became_active":"2024-09-21T06:04:32.771506+0000","last_became_peered":"2024-09-21T06:04:32.771506+0000","last_unstale":"2024-09-21T06:04:32.783317+0000","last_undegraded":"2024-09-21T06:04:32.783317+0000","last_fullsized":"2024-09-21T06:04:32.783317+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:39.994097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.778492+0000","last_change":"2024-09-21T06:04:32.775939+0000","last_active":"2024-09-21T06:04:32.778492+0000","last_peered":"2024-09-21T06:04:32.778492+0000","last_clean":"2024-09-21T06:04:32.778492+0000","last_became_active":"2024-09-21T06:04:32.775526+0000","last_became_peered":"2024-09-21T06:04:32.775526+0000","last_unstale":"2024-09-21T06:04:32.778492+0000","last_undegraded":"2024-09-21T06:04:32.778492+0000","last_fullsized":"2024-09-21T06:04:32.778492+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:17.326812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.769320+0000","last_change":"2024-09-21T06:04:32.768237+0000","last_active":"2024-09-21T06:04:32.769320+0000","last_peered":"2024-09-21T06:04:32.769320+0000","last_clean":"2024-09-21T06:04:32.769320+0000","last_became_active":"2024-09-21T06:04:32.767480+0000","last_became_peered":"2024-09-21T06:04:32.767480+0000","last_unstale":"2024-09-21T06:04:32.769320+0000","last_undegraded":"2024-09-21T06:04:32.769320+0000","last_fullsized":"2024-09-21T06:04:32.769320+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:37.256481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793616+0000","last_change":"2024-09-21T06:04:32.786243+0000","last_active":"2024-09-21T06:04:32.793616+0000","last_peered":"2024-09-21T06:04:32.793616+0000","last_clean":"2024-09-21T06:04:32.793616+0000","last_became_active":"2024-09-21T06:04:32.785278+0000","last_became_peered":"2024-09-21T06:04:32.785278+0000","last_unstale":"2024-09-21T06:04:32.793616+0000","last_undegraded":"2024-09-21T06:04:32.793616+0000","last_fullsized":"2024-09-21T06:04:32.793616+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:41.742767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.761947+0000","last_change":"2024-09-21T06:04:31.761947+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.761947+0000","last_undegraded":"2024-09-21T06:04:31.761947+0000","last_fullsized":"2024-09-21T06:04:31.761947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.775956+0000","last_change":"2024-09-21T06:04:32.774414+0000","last_active":"2024-09-21T06:04:32.775956+0000","last_peered":"2024-09-21T06:04:32.775956+0000","last_clean":"2024-09-21T06:04:32.775956+0000","last_became_active":"2024-09-21T06:04:32.774110+0000","last_became_peered":"2024-09-21T06:04:32.774110+0000","last_unstale":"2024-09-21T06:04:32.775956+0000","last_undegraded":"2024-09-21T06:04:32.775956+0000","last_fullsized":"2024-09-21T06:04:32.775956+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:11.805943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.766598+0000","last_change":"2024-09-21T06:04:32.760679+0000","last_active":"2024-09-21T06:04:32.766598+0000","last_peered":"2024-09-21T06:04:32.766598+0000","last_clean":"2024-09-21T06:04:32.766598+0000","last_became_active":"2024-09-21T06:04:32.760378+0000","last_became_peered":"2024-09-21T06:04:32.760378+0000","last_unstale":"2024-09-21T06:04:32.766598+0000","last_undegraded":"2024-09-21T06:04:32.766598+0000","last_fullsized":"2024-09-21T06:04:32.766598+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:40.885781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.761919+0000","last_change":"2024-09-21T06:04:32.760660+0000","last_active":"2024-09-21T06:04:32.761919+0000","last_peered":"2024-09-21T06:04:32.761919+0000","last_clean":"2024-09-21T06:04:32.761919+0000","last_became_active":"2024-09-21T06:04:32.760303+0000","last_became_peered":"2024-09-21T06:04:32.760303+0000","last_unstale":"2024-09-21T06:04:32.761919+0000","last_undegraded":"2024-09-21T06:04:32.761919+0000","last_fullsized":"2024-09-21T06:04:32.761919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:04.139988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793365+0000","last_change":"2024-09-21T06:04:32.783570+0000","last_active":"2024-09-21T06:04:32.793365+0000","last_peered":"2024-09-21T06:04:32.793365+0000","last_clean":"2024-09-21T06:04:32.793365+0000","last_became_active":"2024-09-21T06:04:32.783265+0000","last_became_peered":"2024-09-21T06:04:32.783265+0000","last_unstale":"2024-09-21T06:04:32.793365+0000","last_undegraded":"2024-09-21T06:04:32.793365+0000","last_fullsized":"2024-09-21T06:04:32.793365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:49.490698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.778520+0000","last_change":"2024-09-21T06:04:32.776322+0000","last_active":"2024-09-21T06:04:32.778520+0000","last_peered":"2024-09-21T06:04:32.778520+0000","last_clean":"2024-09-21T06:04:32.778520+0000","last_became_active":"2024-09-21T06:04:32.776027+0000","last_became_peered":"2024-09-21T06:04:32.776027+0000","last_unstale":"2024-09-21T06:04:32.778520+0000","last_undegraded":"2024-09-21T06:04:32.778520+0000","last_fullsized":"2024-09-21T06:04:32.778520+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:14.913586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.760440+0000","last_change":"2024-09-21T06:04:31.760440+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.760440+0000","last_undegraded":"2024-09-21T06:04:31.760440+0000","last_fullsized":"2024-09-21T06:04:31.760440+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.773202+0000","last_change":"2024-09-21T06:04:32.771994+0000","last_active":"2024-09-21T06:04:32.773202+0000","last_peered":"2024-09-21T06:04:32.773202+0000","last_clean":"2024-09-21T06:04:32.773202+0000","last_became_active":"2024-09-21T06:04:32.771597+0000","last_became_peered":"2024-09-21T06:04:32.771597+0000","last_unstale":"2024-09-21T06:04:32.773202+0000","last_undegraded":"2024-09-21T06:04:32.773202+0000","last_fullsized":"2024-09-21T06:04:32.773202+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:00.871520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.769001+0000","last_change":"2024-09-21T06:04:32.766745+0000","last_active":"2024-09-21T06:04:32.769001+0000","last_peered":"2024-09-21T06:04:32.769001+0000","last_clean":"2024-09-21T06:04:32.769001+0000","last_became_active":"2024-09-21T06:04:32.766362+0000","last_became_peered":"2024-09-21T06:04:32.766362+0000","last_unstale":"2024-09-21T06:04:32.769001+0000","last_undegraded":"2024-09-21T06:04:32.769001+0000","last_fullsized":"2024-09-21T06:04:32.769001+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:12.666560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.765784+0000","last_change":"2024-09-21T06:04:32.760980+0000","last_active":"2024-09-21T06:04:32.765784+0000","last_peered":"2024-09-21T06:04:32.765784+0000","last_clean":"2024-09-21T06:04:32.765784+0000","last_became_active":"2024-09-21T06:04:32.760648+0000","last_became_peered":"2024-09-21T06:04:32.760648+0000","last_unstale":"2024-09-21T06:04:32.765784+0000","last_undegraded":"2024-09-21T06:04:32.765784+0000","last_fullsized":"2024-09-21T06:04:32.765784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:58.412247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.767175+0000","last_change":"2024-09-21T06:04:32.762713+0000","last_active":"2024-09-21T06:04:32.767175+0000","last_peered":"2024-09-21T06:04:32.767175+0000","last_clean":"2024-09-21T06:04:32.767175+0000","last_became_active":"2024-09-21T06:04:32.762345+0000","last_became_peered":"2024-09-21T06:04:32.762345+0000","last_unstale":"2024-09-21T06:04:32.767175+0000","last_undegraded":"2024-09-21T06:04:32.767175+0000","last_fullsized":"2024-09-21T06:04:32.767175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:54.763471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793898+0000","last_change":"2024-09-21T06:04:32.788061+0000","last_active":"2024-09-21T06:04:32.793898+0000","last_peered":"2024-09-21T06:04:32.793898+0000","last_clean":"2024-09-21T06:04:32.793898+0000","last_became_active":"2024-09-21T06:04:32.787777+0000","last_became_peered":"2024-09-21T06:04:32.787777+0000","last_unstale":"2024-09-21T06:04:32.793898+0000","last_undegraded":"2024-09-21T06:04:32.793898+0000","last_fullsized":"2024-09-21T06:04:32.793898+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:53.442251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794388+0000","last_change":"2024-09-21T06:04:32.790938+0000","last_active":"2024-09-21T06:04:32.794388+0000","last_peered":"2024-09-21T06:04:32.794388+0000","last_clean":"2024-09-21T06:04:32.794388+0000","last_became_active":"2024-09-21T06:04:32.790267+0000","last_became_peered":"2024-09-21T06:04:32.790267+0000","last_unstale":"2024-09-21T06:04:32.794388+0000","last_undegraded":"2024-09-21T06:04:32.794388+0000","last_fullsized":"2024-09-21T06:04:32.794388+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:19.004848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.789947+0000","last_change":"2024-09-21T06:04:32.779963+0000","last_active":"2024-09-21T06:04:32.789947+0000","last_peered":"2024-09-21T06:04:32.789947+0000","last_clean":"2024-09-21T06:04:32.789947+0000","last_became_active":"2024-09-21T06:04:32.779659+0000","last_became_peered":"2024-09-21T06:04:32.779659+0000","last_unstale":"2024-09-21T06:04:32.789947+0000","last_undegraded":"2024-09-21T06:04:32.789947+0000","last_fullsized":"2024-09-21T06:04:32.789947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:08.847286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783071+0000","last_change":"2024-09-21T06:04:32.778117+0000","last_active":"2024-09-21T06:04:32.783071+0000","last_peered":"2024-09-21T06:04:32.783071+0000","last_clean":"2024-09-21T06:04:32.783071+0000","last_became_active":"2024-09-21T06:04:32.777105+0000","last_became_peered":"2024-09-21T06:04:32.777105+0000","last_unstale":"2024-09-21T06:04:32.783071+0000","last_undegraded":"2024-09-21T06:04:32.783071+0000","last_fullsized":"2024-09-21T06:04:32.783071+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:37.388005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781124+0000","last_change":"2024-09-21T06:04:32.771946+0000","last_active":"2024-09-21T06:04:32.781124+0000","last_peered":"2024-09-21T06:04:32.781124+0000","last_clean":"2024-09-21T06:04:32.781124+0000","last_became_active":"2024-09-21T06:04:32.771652+0000","last_became_peered":"2024-09-21T06:04:32.771652+0000","last_unstale":"2024-09-21T06:04:32.781124+0000","last_undegraded":"2024-09-21T06:04:32.781124+0000","last_fullsized":"2024-09-21T06:04:32.781124+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:02.037656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792043+0000","last_change":"2024-09-21T06:04:32.780368+0000","last_active":"2024-09-21T06:04:32.792043+0000","last_peered":"2024-09-21T06:04:32.792043+0000","last_clean":"2024-09-21T06:04:32.792043+0000","last_became_active":"2024-09-21T06:04:32.780021+0000","last_became_peered":"2024-09-21T06:04:32.780021+0000","last_unstale":"2024-09-21T06:04:32.792043+0000","last_undegraded":"2024-09-21T06:04:32.792043+0000","last_fullsized":"2024-09-21T06:04:32.792043+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:57.984993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.779808+0000","last_change":"2024-09-21T06:04:32.773537+0000","last_active":"2024-09-21T06:04:32.779808+0000","last_peered":"2024-09-21T06:04:32.779808+0000","last_clean":"2024-09-21T06:04:32.779808+0000","last_became_active":"2024-09-21T06:04:32.773240+0000","last_became_peered":"2024-09-21T06:04:32.773240+0000","last_unstale":"2024-09-21T06:04:32.779808+0000","last_undegraded":"2024-09-21T06:04:32.779808+0000","last_fullsized":"2024-09-21T06:04:32.779808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:04.990837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.789169+0000","last_change":"2024-09-21T06:04:32.782666+0000","last_active":"2024-09-21T06:04:32.789169+0000","last_peered":"2024-09-21T06:04:32.789169+0000","last_clean":"2024-09-21T06:04:32.789169+0000","last_became_active":"2024-09-21T06:04:32.777455+0000","last_became_peered":"2024-09-21T06:04:32.777455+0000","last_unstale":"2024-09-21T06:04:32.789169+0000","last_undegraded":"2024-09-21T06:04:32.789169+0000","last_fullsized":"2024-09-21T06:04:32.789169+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:53.442251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.768646+0000","last_change":"2024-09-21T06:04:32.764381+0000","last_active":"2024-09-21T06:04:32.768646+0000","last_peered":"2024-09-21T06:04:32.768646+0000","last_clean":"2024-09-21T06:04:32.768646+0000","last_became_active":"2024-09-21T06:04:32.764040+0000","last_became_peered":"2024-09-21T06:04:32.764040+0000","last_unstale":"2024-09-21T06:04:32.768646+0000","last_undegraded":"2024-09-21T06:04:32.768646+0000","last_fullsized":"2024-09-21T06:04:32.768646+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:33.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783350+0000","last_change":"2024-09-21T06:04:32.774466+0000","last_active":"2024-09-21T06:04:32.783350+0000","last_peered":"2024-09-21T06:04:32.783350+0000","last_clean":"2024-09-21T06:04:32.783350+0000","last_became_active":"2024-09-21T06:04:32.774133+0000","last_became_peered":"2024-09-21T06:04:32.774133+0000","last_unstale":"2024-09-21T06:04:32.783350+0000","last_undegraded":"2024-09-21T06:04:32.783350+0000","last_fullsized":"2024-09-21T06:04:32.783350+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:58.412247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794461+0000","last_change":"2024-09-21T06:04:32.792389+0000","last_active":"2024-09-21T06:04:32.794461+0000","last_peered":"2024-09-21T06:04:32.794461+0000","last_clean":"2024-09-21T06:04:32.794461+0000","last_became_active":"2024-09-21T06:04:32.792097+0000","last_became_peered":"2024-09-21T06:04:32.792097+0000","last_unstale":"2024-09-21T06:04:32.794461+0000","last_undegraded":"2024-09-21T06:04:32.794461+0000","last_fullsized":"2024-09-21T06:04:32.794461+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:46.730158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793238+0000","last_change":"2024-09-21T06:04:31.777083+0000","last_active":"2024-09-21T06:04:32.793238+0000","last_peered":"2024-09-21T06:04:32.793238+0000","last_clean":"2024-09-21T06:04:32.793238+0000","last_became_active":"2024-09-21T06:04:31.776739+0000","last_became_peered":"2024-09-21T06:04:31.776739+0000","last_unstale":"2024-09-21T06:04:32.793238+0000","last_undegraded":"2024-09-21T06:04:32.793238+0000","last_fullsized":"2024-09-21T06:04:32.793238+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:59.686723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783664+0000","last_change":"2024-09-21T06:04:32.776263+0000","last_active":"2024-09-21T06:04:32.783664+0000","last_peered":"2024-09-21T06:04:32.783664+0000","last_clean":"2024-09-21T06:04:32.783664+0000","last_became_active":"2024-09-21T06:04:32.775659+0000","last_became_peered":"2024-09-21T06:04:32.775659+0000","last_unstale":"2024-09-21T06:04:32.783664+0000","last_undegraded":"2024-09-21T06:04:32.783664+0000","last_fullsized":"2024-09-21T06:04:32.783664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:22.506119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.784109+0000","last_change":"2024-09-21T06:04:31.769588+0000","last_active":"2024-09-21T06:04:32.784109+0000","last_peered":"2024-09-21T06:04:32.784109+0000","last_clean":"2024-09-21T06:04:32.784109+0000","last_became_active":"2024-09-21T06:04:31.768905+0000","last_became_peered":"2024-09-21T06:04:31.768905+0000","last_unstale":"2024-09-21T06:04:32.784109+0000","last_undegraded":"2024-09-21T06:04:32.784109+0000","last_fullsized":"2024-09-21T06:04:32.784109+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:27.975438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.789767+0000","last_change":"2024-09-21T06:04:32.788286+0000","last_active":"2024-09-21T06:04:32.789767+0000","last_peered":"2024-09-21T06:04:32.789767+0000","last_clean":"2024-09-21T06:04:32.789767+0000","last_became_active":"2024-09-21T06:04:32.787926+0000","last_became_peered":"2024-09-21T06:04:32.787926+0000","last_unstale":"2024-09-21T06:04:32.789767+0000","last_undegraded":"2024-09-21T06:04:32.789767+0000","last_fullsized":"2024-09-21T06:04:32.789767+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:08.847286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794349+0000","last_change":"2024-09-21T06:04:31.772963+0000","last_active":"2024-09-21T06:04:32.794349+0000","last_peered":"2024-09-21T06:04:32.794349+0000","last_clean":"2024-09-21T06:04:32.794349+0000","last_became_active":"2024-09-21T06:04:31.772653+0000","last_became_peered":"2024-09-21T06:04:31.772653+0000","last_unstale":"2024-09-21T06:04:32.794349+0000","last_undegraded":"2024-09-21T06:04:32.794349+0000","last_fullsized":"2024-09-21T06:04:32.794349+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:59.868880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793482+0000","last_change":"2024-09-21T06:04:32.785502+0000","last_active":"2024-09-21T06:04:32.793482+0000","last_peered":"2024-09-21T06:04:32.793482+0000","last_clean":"2024-09-21T06:04:32.793482+0000","last_became_active":"2024-09-21T06:04:32.785091+0000","last_became_peered":"2024-09-21T06:04:32.785091+0000","last_unstale":"2024-09-21T06:04:32.793482+0000","last_undegraded":"2024-09-21T06:04:32.793482+0000","last_fullsized":"2024-09-21T06:04:32.793482+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:11.518104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.788989+0000","last_change":"2024-09-21T06:04:31.775276+0000","last_active":"2024-09-21T06:04:32.788989+0000","last_peered":"2024-09-21T06:04:32.788989+0000","last_clean":"2024-09-21T06:04:32.788989+0000","last_became_active":"2024-09-21T06:04:31.774028+0000","last_became_peered":"2024-09-21T06:04:31.774028+0000","last_unstale":"2024-09-21T06:04:32.788989+0000","last_undegraded":"2024-09-21T06:04:32.788989+0000","last_fullsized":"2024-09-21T06:04:32.788989+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:18.843912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783561+0000","last_change":"2024-09-21T06:04:32.777770+0000","last_active":"2024-09-21T06:04:32.783561+0000","last_peered":"2024-09-21T06:04:32.783561+0000","last_clean":"2024-09-21T06:04:32.783561+0000","last_became_active":"2024-09-21T06:04:32.777019+0000","last_became_peered":"2024-09-21T06:04:32.777019+0000","last_unstale":"2024-09-21T06:04:32.783561+0000","last_undegraded":"2024-09-21T06:04:32.783561+0000","last_fullsized":"2024-09-21T06:04:32.783561+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:59.802808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793522+0000","last_change":"2024-09-21T06:04:31.766081+0000","last_active":"2024-09-21T06:04:32.793522+0000","last_peered":"2024-09-21T06:04:32.793522+0000","last_clean":"2024-09-21T06:04:32.793522+0000","last_became_active":"2024-09-21T06:04:31.765541+0000","last_became_peered":"2024-09-21T06:04:31.765541+0000","last_unstale":"2024-09-21T06:04:32.793522+0000","last_undegraded":"2024-09-21T06:04:32.793522+0000","last_fullsized":"2024-09-21T06:04:32.793522+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:40.183984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792327+0000","last_change":"2024-09-21T06:04:32.780478+0000","last_active":"2024-09-21T06:04:32.792327+0000","last_peered":"2024-09-21T06:04:32.792327+0000","last_clean":"2024-09-21T06:04:32.792327+0000","last_became_active":"2024-09-21T06:04:32.778124+0000","last_became_peered":"2024-09-21T06:04:32.778124+0000","last_unstale":"2024-09-21T06:04:32.792327+0000","last_undegraded":"2024-09-21T06:04:32.792327+0000","last_fullsized":"2024-09-21T06:04:32.792327+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:26.746417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.787610+0000","last_change":"2024-09-21T06:04:31.772094+0000","last_active":"2024-09-21T06:04:32.787610+0000","last_peered":"2024-09-21T06:04:32.787610+0000","last_clean":"2024-09-21T06:04:32.787610+0000","last_became_active":"2024-09-21T06:04:31.771544+0000","last_became_peered":"2024-09-21T06:04:31.771544+0000","last_unstale":"2024-09-21T06:04:32.787610+0000","last_undegraded":"2024-09-21T06:04:32.787610+0000","last_fullsized":"2024-09-21T06:04:32.787610+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:03.600601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.767075+0000","last_change":"2024-09-21T06:04:32.761010+0000","last_active":"2024-09-21T06:04:32.767075+0000","last_peered":"2024-09-21T06:04:32.767075+0000","last_clean":"2024-09-21T06:04:32.767075+0000","last_became_active":"2024-09-21T06:04:32.760708+0000","last_became_peered":"2024-09-21T06:04:32.760708+0000","last_unstale":"2024-09-21T06:04:32.767075+0000","last_undegraded":"2024-09-21T06:04:32.767075+0000","last_fullsized":"2024-09-21T06:04:32.767075+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:45.868263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.769744+0000","last_change":"2024-09-21T06:04:31.768175+0000","last_active":"2024-09-21T06:04:31.769744+0000","last_peered":"2024-09-21T06:04:31.769744+0000","last_clean":"2024-09-21T06:04:31.769744+0000","last_became_active":"2024-09-21T06:04:31.767820+0000","last_became_peered":"2024-09-21T06:04:31.767820+0000","last_unstale":"2024-09-21T06:04:31.769744+0000","last_undegraded":"2024-09-21T06:04:31.769744+0000","last_fullsized":"2024-09-21T06:04:31.769744+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:24.459378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.775720+0000","last_change":"2024-09-21T06:04:32.772784+0000","last_active":"2024-09-21T06:04:32.775720+0000","last_peered":"2024-09-21T06:04:32.775720+0000","last_clean":"2024-09-21T06:04:32.775720+0000","last_became_active":"2024-09-21T06:04:32.772246+0000","last_became_peered":"2024-09-21T06:04:32.772246+0000","last_unstale":"2024-09-21T06:04:32.775720+0000","last_undegraded":"2024-09-21T06:04:32.775720+0000","last_fullsized":"2024-09-21T06:04:32.775720+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:39.036138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.752202+0000","last_change":"2024-09-21T06:04:31.759185+0000","last_active":"2024-09-21T06:04:32.752202+0000","last_peered":"2024-09-21T06:04:32.752202+0000","last_clean":"2024-09-21T06:04:32.752202+0000","last_became_active":"2024-09-21T06:04:31.758873+0000","last_became_peered":"2024-09-21T06:04:31.758873+0000","last_unstale":"2024-09-21T06:04:32.752202+0000","last_undegraded":"2024-09-21T06:04:32.752202+0000","last_fullsized":"2024-09-21T06:04:32.752202+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:37.617451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.770924+0000","last_change":"2024-09-21T06:04:32.769066+0000","last_active":"2024-09-21T06:04:32.770924+0000","last_peered":"2024-09-21T06:04:32.770924+0000","last_clean":"2024-09-21T06:04:32.770924+0000","last_became_active":"2024-09-21T06:04:32.768695+0000","last_became_peered":"2024-09-21T06:04:32.768695+0000","last_unstale":"2024-09-21T06:04:32.770924+0000","last_undegraded":"2024-09-21T06:04:32.770924+0000","last_fullsized":"2024-09-21T06:04:32.770924+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:41.742767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.755039+0000","last_change":"2024-09-21T06:04:31.770884+0000","last_active":"2024-09-21T06:04:32.755039+0000","last_peered":"2024-09-21T06:04:32.755039+0000","last_clean":"2024-09-21T06:04:32.755039+0000","last_became_active":"2024-09-21T06:04:31.768869+0000","last_became_peered":"2024-09-21T06:04:31.768869+0000","last_unstale":"2024-09-21T06:04:32.755039+0000","last_undegraded":"2024-09-21T06:04:32.755039+0000","last_fullsized":"2024-09-21T06:04:32.755039+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:18.843912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794408+0000","last_change":"2024-09-21T06:04:32.791807+0000","last_active":"2024-09-21T06:04:32.794408+0000","last_peered":"2024-09-21T06:04:32.794408+0000","last_clean":"2024-09-21T06:04:32.794408+0000","last_became_active":"2024-09-21T06:04:32.791461+0000","last_became_peered":"2024-09-21T06:04:32.791461+0000","last_unstale":"2024-09-21T06:04:32.794408+0000","last_undegraded":"2024-09-21T06:04:32.794408+0000","last_fullsized":"2024-09-21T06:04:32.794408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:47.832984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.766601+0000","last_change":"2024-09-21T06:04:31.770081+0000","last_active":"2024-09-21T06:04:32.766601+0000","last_peered":"2024-09-21T06:04:32.766601+0000","last_clean":"2024-09-21T06:04:32.766601+0000","last_became_active":"2024-09-21T06:04:31.768462+0000","last_became_peered":"2024-09-21T06:04:31.768462+0000","last_unstale":"2024-09-21T06:04:32.766601+0000","last_undegraded":"2024-09-21T06:04:32.766601+0000","last_fullsized":"2024-09-21T06:04:32.766601+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:10.542640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.796309+0000","last_change":"2024-09-21T06:04:32.795515+0000","last_active":"2024-09-21T06:04:32.796309+0000","last_peered":"2024-09-21T06:04:32.796309+0000","last_clean":"2024-09-21T06:04:32.796309+0000","last_became_active":"2024-09-21T06:04:32.795019+0000","last_became_peered":"2024-09-21T06:04:32.795019+0000","last_unstale":"2024-09-21T06:04:32.796309+0000","last_undegraded":"2024-09-21T06:04:32.796309+0000","last_fullsized":"2024-09-21T06:04:32.796309+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:26.746417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792146+0000","last_change":"2024-09-21T06:04:31.769723+0000","last_active":"2024-09-21T06:04:32.792146+0000","last_peered":"2024-09-21T06:04:32.792146+0000","last_clean":"2024-09-21T06:04:32.792146+0000","last_became_active":"2024-09-21T06:04:31.769344+0000","last_became_peered":"2024-09-21T06:04:31.769344+0000","last_unstale":"2024-09-21T06:04:32.792146+0000","last_undegraded":"2024-09-21T06:04:32.792146+0000","last_fullsized":"2024-09-21T06:04:32.792146+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:00.566146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.768354+0000","last_change":"2024-09-21T06:04:32.767569+0000","last_active":"2024-09-21T06:04:32.768354+0000","last_peered":"2024-09-21T06:04:32.768354+0000","last_clean":"2024-09-21T06:04:32.768354+0000","last_became_active":"2024-09-21T06:04:32.766934+0000","last_became_peered":"2024-09-21T06:04:32.766934+0000","last_unstale":"2024-09-21T06:04:32.768354+0000","last_undegraded":"2024-09-21T06:04:32.768354+0000","last_fullsized":"2024-09-21T06:04:32.768354+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:56.103744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.770166+0000","last_change":"2024-09-21T06:04:31.768773+0000","last_active":"2024-09-21T06:04:31.770166+0000","last_peered":"2024-09-21T06:04:31.770166+0000","last_clean":"2024-09-21T06:04:31.770166+0000","last_became_active":"2024-09-21T06:04:31.767292+0000","last_became_peered":"2024-09-21T06:04:31.767292+0000","last_unstale":"2024-09-21T06:04:31.770166+0000","last_undegraded":"2024-09-21T06:04:31.770166+0000","last_fullsized":"2024-09-21T06:04:31.770166+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:32.705430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.792225+0000","last_change":"2024-09-21T06:04:32.784585+0000","last_active":"2024-09-21T06:04:32.792225+0000","last_peered":"2024-09-21T06:04:32.792225+0000","last_clean":"2024-09-21T06:04:32.792225+0000","last_became_active":"2024-09-21T06:04:32.784259+0000","last_became_peered":"2024-09-21T06:04:32.784259+0000","last_unstale":"2024-09-21T06:04:32.792225+0000","last_undegraded":"2024-09-21T06:04:32.792225+0000","last_fullsized":"2024-09-21T06:04:32.792225+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:11.446707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.773742+0000","last_change":"2024-09-21T06:04:31.772181+0000","last_active":"2024-09-21T06:04:31.773742+0000","last_peered":"2024-09-21T06:04:31.773742+0000","last_clean":"2024-09-21T06:04:31.773742+0000","last_became_active":"2024-09-21T06:04:31.771816+0000","last_became_peered":"2024-09-21T06:04:31.771816+0000","last_unstale":"2024-09-21T06:04:31.773742+0000","last_undegraded":"2024-09-21T06:04:31.773742+0000","last_fullsized":"2024-09-21T06:04:31.773742+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:47.864470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.770503+0000","last_change":"2024-09-21T06:04:32.768202+0000","last_active":"2024-09-21T06:04:32.770503+0000","last_peered":"2024-09-21T06:04:32.770503+0000","last_clean":"2024-09-21T06:04:32.770503+0000","last_became_active":"2024-09-21T06:04:32.767810+0000","last_became_peered":"2024-09-21T06:04:32.767810+0000","last_unstale":"2024-09-21T06:04:32.770503+0000","last_undegraded":"2024-09-21T06:04:32.770503+0000","last_fullsized":"2024-09-21T06:04:32.770503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:11.518104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.763381+0000","last_change":"2024-09-21T06:04:31.775115+0000","last_active":"2024-09-21T06:04:32.763381+0000","last_peered":"2024-09-21T06:04:32.763381+0000","last_clean":"2024-09-21T06:04:32.763381+0000","last_became_active":"2024-09-21T06:04:31.773039+0000","last_became_peered":"2024-09-21T06:04:31.773039+0000","last_unstale":"2024-09-21T06:04:32.763381+0000","last_undegraded":"2024-09-21T06:04:32.763381+0000","last_fullsized":"2024-09-21T06:04:32.763381+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:54.006741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781404+0000","last_change":"2024-09-21T06:04:32.780256+0000","last_active":"2024-09-21T06:04:32.781404+0000","last_peered":"2024-09-21T06:04:32.781404+0000","last_clean":"2024-09-21T06:04:32.781404+0000","last_became_active":"2024-09-21T06:04:32.779924+0000","last_became_peered":"2024-09-21T06:04:32.779924+0000","last_unstale":"2024-09-21T06:04:32.781404+0000","last_undegraded":"2024-09-21T06:04:32.781404+0000","last_fullsized":"2024-09-21T06:04:32.781404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:52.831924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.756793+0000","last_change":"2024-09-21T06:04:31.775645+0000","last_active":"2024-09-21T06:04:32.756793+0000","last_peered":"2024-09-21T06:04:32.756793+0000","last_clean":"2024-09-21T06:04:32.756793+0000","last_became_active":"2024-09-21T06:04:31.775129+0000","last_became_peered":"2024-09-21T06:04:31.775129+0000","last_unstale":"2024-09-21T06:04:32.756793+0000","last_undegraded":"2024-09-21T06:04:32.756793+0000","last_fullsized":"2024-09-21T06:04:32.756793+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:55.569554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.771789+0000","last_change":"2024-09-21T06:04:32.770814+0000","last_active":"2024-09-21T06:04:32.771789+0000","last_peered":"2024-09-21T06:04:32.771789+0000","last_clean":"2024-09-21T06:04:32.771789+0000","last_became_active":"2024-09-21T06:04:32.770411+0000","last_became_peered":"2024-09-21T06:04:32.770411+0000","last_unstale":"2024-09-21T06:04:32.771789+0000","last_undegraded":"2024-09-21T06:04:32.771789+0000","last_fullsized":"2024-09-21T06:04:32.771789+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:17.290747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793295+0000","last_change":"2024-09-21T06:04:31.772137+0000","last_active":"2024-09-21T06:04:32.793295+0000","last_peered":"2024-09-21T06:04:32.793295+0000","last_clean":"2024-09-21T06:04:32.793295+0000","last_became_active":"2024-09-21T06:04:31.771622+0000","last_became_peered":"2024-09-21T06:04:31.771622+0000","last_unstale":"2024-09-21T06:04:32.793295+0000","last_undegraded":"2024-09-21T06:04:32.793295+0000","last_fullsized":"2024-09-21T06:04:32.793295+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:38.991457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.769708+0000","last_change":"2024-09-21T06:04:32.768582+0000","last_active":"2024-09-21T06:04:32.769708+0000","last_peered":"2024-09-21T06:04:32.769708+0000","last_clean":"2024-09-21T06:04:32.769708+0000","last_became_active":"2024-09-21T06:04:32.767918+0000","last_became_peered":"2024-09-21T06:04:32.767918+0000","last_unstale":"2024-09-21T06:04:32.769708+0000","last_undegraded":"2024-09-21T06:04:32.769708+0000","last_fullsized":"2024-09-21T06:04:32.769708+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:49.109988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.786154+0000","last_change":"2024-09-21T06:04:31.775324+0000","last_active":"2024-09-21T06:04:32.786154+0000","last_peered":"2024-09-21T06:04:32.786154+0000","last_clean":"2024-09-21T06:04:32.786154+0000","last_became_active":"2024-09-21T06:04:31.774989+0000","last_became_peered":"2024-09-21T06:04:31.774989+0000","last_unstale":"2024-09-21T06:04:32.786154+0000","last_undegraded":"2024-09-21T06:04:32.786154+0000","last_fullsized":"2024-09-21T06:04:32.786154+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:10.782962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.793956+0000","last_change":"2024-09-21T06:04:32.787340+0000","last_active":"2024-09-21T06:04:32.793956+0000","last_peered":"2024-09-21T06:04:32.793956+0000","last_clean":"2024-09-21T06:04:32.793956+0000","last_became_active":"2024-09-21T06:04:32.787027+0000","last_became_peered":"2024-09-21T06:04:32.787027+0000","last_unstale":"2024-09-21T06:04:32.793956+0000","last_undegraded":"2024-09-21T06:04:32.793956+0000","last_fullsized":"2024-09-21T06:04:32.793956+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:37.388005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.791893+0000","last_change":"2024-09-21T06:04:31.774380+0000","last_active":"2024-09-21T06:04:32.791893+0000","last_peered":"2024-09-21T06:04:32.791893+0000","last_clean":"2024-09-21T06:04:32.791893+0000","last_became_active":"2024-09-21T06:04:31.773835+0000","last_became_peered":"2024-09-21T06:04:31.773835+0000","last_unstale":"2024-09-21T06:04:32.791893+0000","last_undegraded":"2024-09-21T06:04:32.791893+0000","last_fullsized":"2024-09-21T06:04:32.791893+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:53.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.759154+0000","last_change":"2024-09-21T06:04:31.759154+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.759154+0000","last_undegraded":"2024-09-21T06:04:31.759154+0000","last_fullsized":"2024-09-21T06:04:31.759154+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.795017+0000","last_change":"2024-09-21T06:04:31.768595+0000","last_active":"2024-09-21T06:04:32.795017+0000","last_peered":"2024-09-21T06:04:32.795017+0000","last_clean":"2024-09-21T06:04:32.795017+0000","last_became_active":"2024-09-21T06:04:31.767603+0000","last_became_peered":"2024-09-21T06:04:31.767603+0000","last_unstale":"2024-09-21T06:04:32.795017+0000","last_undegraded":"2024-09-21T06:04:32.795017+0000","last_fullsized":"2024-09-21T06:04:32.795017+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:53.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.774981+0000","last_change":"2024-09-21T06:04:32.774290+0000","last_active":"2024-09-21T06:04:32.774981+0000","last_peered":"2024-09-21T06:04:32.774981+0000","last_clean":"2024-09-21T06:04:32.774981+0000","last_became_active":"2024-09-21T06:04:32.774023+0000","last_became_peered":"2024-09-21T06:04:32.774023+0000","last_unstale":"2024-09-21T06:04:32.774981+0000","last_undegraded":"2024-09-21T06:04:32.774981+0000","last_fullsized":"2024-09-21T06:04:32.774981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.944676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.774811+0000","last_change":"2024-09-21T06:04:31.774500+0000","last_active":"2024-09-21T06:04:32.774811+0000","last_peered":"2024-09-21T06:04:32.774811+0000","last_clean":"2024-09-21T06:04:32.774811+0000","last_became_active":"2024-09-21T06:04:31.774161+0000","last_became_peered":"2024-09-21T06:04:31.774161+0000","last_unstale":"2024-09-21T06:04:32.774811+0000","last_undegraded":"2024-09-21T06:04:32.774811+0000","last_fullsized":"2024-09-21T06:04:32.774811+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:32.288314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.773796+0000","last_change":"2024-09-21T06:04:32.769815+0000","last_active":"2024-09-21T06:04:32.773796+0000","last_peered":"2024-09-21T06:04:32.773796+0000","last_clean":"2024-09-21T06:04:32.773796+0000","last_became_active":"2024-09-21T06:04:32.769467+0000","last_became_peered":"2024-09-21T06:04:32.769467+0000","last_unstale":"2024-09-21T06:04:32.773796+0000","last_undegraded":"2024-09-21T06:04:32.773796+0000","last_fullsized":"2024-09-21T06:04:32.773796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:44.954801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.763651+0000","last_change":"2024-09-21T06:04:31.772131+0000","last_active":"2024-09-21T06:04:32.763651+0000","last_peered":"2024-09-21T06:04:32.763651+0000","last_clean":"2024-09-21T06:04:32.763651+0000","last_became_active":"2024-09-21T06:04:31.771792+0000","last_became_peered":"2024-09-21T06:04:31.771792+0000","last_unstale":"2024-09-21T06:04:32.763651+0000","last_undegraded":"2024-09-21T06:04:32.763651+0000","last_fullsized":"2024-09-21T06:04:32.763651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.561776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.783512+0000","last_change":"2024-09-21T06:04:32.782337+0000","last_active":"2024-09-21T06:04:32.783512+0000","last_peered":"2024-09-21T06:04:32.783512+0000","last_clean":"2024-09-21T06:04:32.783512+0000","last_became_active":"2024-09-21T06:04:32.782011+0000","last_became_peered":"2024-09-21T06:04:32.782011+0000","last_unstale":"2024-09-21T06:04:32.783512+0000","last_undegraded":"2024-09-21T06:04:32.783512+0000","last_fullsized":"2024-09-21T06:04:32.783512+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:45.735018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.757070+0000","last_change":"2024-09-21T06:04:31.766119+0000","last_active":"2024-09-21T06:04:32.757070+0000","last_peered":"2024-09-21T06:04:32.757070+0000","last_clean":"2024-09-21T06:04:32.757070+0000","last_became_active":"2024-09-21T06:04:31.765398+0000","last_became_peered":"2024-09-21T06:04:31.765398+0000","last_unstale":"2024-09-21T06:04:32.757070+0000","last_undegraded":"2024-09-21T06:04:32.757070+0000","last_fullsized":"2024-09-21T06:04:32.757070+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:23.042000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.781363+0000","last_change":"2024-09-21T06:04:32.777061+0000","last_active":"2024-09-21T06:04:32.781363+0000","last_peered":"2024-09-21T06:04:32.781363+0000","last_clean":"2024-09-21T06:04:32.781363+0000","last_became_active":"2024-09-21T06:04:32.776703+0000","last_became_peered":"2024-09-21T06:04:32.776703+0000","last_unstale":"2024-09-21T06:04:32.781363+0000","last_undegraded":"2024-09-21T06:04:32.781363+0000","last_fullsized":"2024-09-21T06:04:32.781363+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:12.555174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.756499+0000","last_change":"2024-09-21T06:04:31.770508+0000","last_active":"2024-09-21T06:04:32.756499+0000","last_peered":"2024-09-21T06:04:32.756499+0000","last_clean":"2024-09-21T06:04:32.756499+0000","last_became_active":"2024-09-21T06:04:31.769020+0000","last_became_peered":"2024-09-21T06:04:31.769020+0000","last_unstale":"2024-09-21T06:04:32.756499+0000","last_undegraded":"2024-09-21T06:04:32.756499+0000","last_fullsized":"2024-09-21T06:04:32.756499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:40.893581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.765322+0000","last_change":"2024-09-21T06:04:32.762522+0000","last_active":"2024-09-21T06:04:32.765322+0000","last_peered":"2024-09-21T06:04:32.765322+0000","last_clean":"2024-09-21T06:04:32.765322+0000","last_became_active":"2024-09-21T06:04:32.762144+0000","last_became_peered":"2024-09-21T06:04:32.762144+0000","last_unstale":"2024-09-21T06:04:32.765322+0000","last_undegraded":"2024-09-21T06:04:32.765322+0000","last_fullsized":"2024-09-21T06:04:32.765322+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:36.679619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.786027+0000","last_change":"2024-09-21T06:04:31.774958+0000","last_active":"2024-09-21T06:04:32.786027+0000","last_peered":"2024-09-21T06:04:32.786027+0000","last_clean":"2024-09-21T06:04:32.786027+0000","last_became_active":"2024-09-21T06:04:31.774595+0000","last_became_peered":"2024-09-21T06:04:31.774595+0000","last_unstale":"2024-09-21T06:04:32.786027+0000","last_undegraded":"2024-09-21T06:04:32.786027+0000","last_fullsized":"2024-09-21T06:04:32.786027+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:05.289920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.770815+0000","last_change":"2024-09-21T06:04:32.766158+0000","last_active":"2024-09-21T06:04:32.770815+0000","last_peered":"2024-09-21T06:04:32.770815+0000","last_clean":"2024-09-21T06:04:32.770815+0000","last_became_active":"2024-09-21T06:04:32.765811+0000","last_became_peered":"2024-09-21T06:04:32.765811+0000","last_unstale":"2024-09-21T06:04:32.770815+0000","last_undegraded":"2024-09-21T06:04:32.770815+0000","last_fullsized":"2024-09-21T06:04:32.770815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:11.446707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.757924+0000","last_change":"2024-09-21T06:04:31.775745+0000","last_active":"2024-09-21T06:04:32.757924+0000","last_peered":"2024-09-21T06:04:32.757924+0000","last_clean":"2024-09-21T06:04:32.757924+0000","last_became_active":"2024-09-21T06:04:31.775037+0000","last_became_peered":"2024-09-21T06:04:31.775037+0000","last_unstale":"2024-09-21T06:04:32.757924+0000","last_undegraded":"2024-09-21T06:04:32.757924+0000","last_fullsized":"2024-09-21T06:04:32.757924+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:40.310390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794553+0000","last_change":"2024-09-21T06:04:32.791087+0000","last_active":"2024-09-21T06:04:32.794553+0000","last_peered":"2024-09-21T06:04:32.794553+0000","last_clean":"2024-09-21T06:04:32.794553+0000","last_became_active":"2024-09-21T06:04:32.790735+0000","last_became_peered":"2024-09-21T06:04:32.790735+0000","last_unstale":"2024-09-21T06:04:32.794553+0000","last_undegraded":"2024-09-21T06:04:32.794553+0000","last_fullsized":"2024-09-21T06:04:32.794553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:30.386322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.751374+0000","last_change":"2024-09-21T06:04:31.761815+0000","last_active":"2024-09-21T06:04:32.751374+0000","last_peered":"2024-09-21T06:04:32.751374+0000","last_clean":"2024-09-21T06:04:32.751374+0000","last_became_active":"2024-09-21T06:04:31.761487+0000","last_became_peered":"2024-09-21T06:04:31.761487+0000","last_unstale":"2024-09-21T06:04:32.751374+0000","last_undegraded":"2024-09-21T06:04:32.751374+0000","last_fullsized":"2024-09-21T06:04:32.751374+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:02:35.169801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.794037+0000","last_change":"2024-09-21T06:04:32.788888+0000","last_active":"2024-09-21T06:04:32.794037+0000","last_peered":"2024-09-21T06:04:32.794037+0000","last_clean":"2024-09-21T06:04:32.794037+0000","last_became_active":"2024-09-21T06:04:32.788597+0000","last_became_peered":"2024-09-21T06:04:32.788597+0000","last_unstale":"2024-09-21T06:04:32.794037+0000","last_undegraded":"2024-09-21T06:04:32.794037+0000","last_fullsized":"2024-09-21T06:04:32.794037+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:12.183155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.761494+0000","last_change":"2024-09-21T06:04:31.762529+0000","last_active":"2024-09-21T06:04:32.761494+0000","last_peered":"2024-09-21T06:04:32.761494+0000","last_clean":"2024-09-21T06:04:32.761494+0000","last_became_active":"2024-09-21T06:04:31.762189+0000","last_became_peered":"2024-09-21T06:04:31.762189+0000","last_unstale":"2024-09-21T06:04:32.761494+0000","last_undegraded":"2024-09-21T06:04:32.761494+0000","last_fullsized":"2024-09-21T06:04:32.761494+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:51.822586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.773682+0000","last_change":"2024-09-21T06:04:32.770727+0000","last_active":"2024-09-21T06:04:32.773682+0000","last_peered":"2024-09-21T06:04:32.773682+0000","last_clean":"2024-09-21T06:04:32.773682+0000","last_became_active":"2024-09-21T06:04:32.770393+0000","last_became_peered":"2024-09-21T06:04:32.770393+0000","last_unstale":"2024-09-21T06:04:32.773682+0000","last_undegraded":"2024-09-21T06:04:32.773682+0000","last_fullsized":"2024-09-21T06:04:32.773682+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:23.599523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.751186+0000","last_change":"2024-09-21T06:04:31.768859+0000","last_active":"2024-09-21T06:04:32.751186+0000","last_peered":"2024-09-21T06:04:32.751186+0000","last_clean":"2024-09-21T06:04:32.751186+0000","last_became_active":"2024-09-21T06:04:31.768535+0000","last_became_peered":"2024-09-21T06:04:31.768535+0000","last_unstale":"2024-09-21T06:04:32.751186+0000","last_undegraded":"2024-09-21T06:04:32.751186+0000","last_fullsized":"2024-09-21T06:04:32.751186+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:05.572103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.786505+0000","last_change":"2024-09-21T06:04:32.782856+0000","last_active":"2024-09-21T06:04:32.786505+0000","last_peered":"2024-09-21T06:04:32.786505+0000","last_clean":"2024-09-21T06:04:32.786505+0000","last_became_active":"2024-09-21T06:04:32.782532+0000","last_became_peered":"2024-09-21T06:04:32.782532+0000","last_unstale":"2024-09-21T06:04:32.786505+0000","last_undegraded":"2024-09-21T06:04:32.786505+0000","last_fullsized":"2024-09-21T06:04:32.786505+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:49.109988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-21T06:04:31.770750+0000","last_change":"2024-09-21T06:04:31.768251+0000","last_active":"2024-09-21T06:04:31.770750+0000","last_peered":"2024-09-21T06:04:31.770750+0000","last_clean":"2024-09-21T06:04:31.770750+0000","last_became_active":"2024-09-21T06:04:31.767918+0000","last_became_peered":"2024-09-21T06:04:31.767918+0000","last_unstale":"2024-09-21T06:04:31.770750+0000","last_undegraded":"2024-09-21T06:04:31.770750+0000","last_fullsized":"2024-09-21T06:04:31.770750+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:28.832034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T06:04:31.760288+0000","last_change":"2024-09-21T06:04:31.760288+0000","last_active":"2024-09-21T06:04:31.738639+0000","last_peered":"2024-09-21T06:04:31.738639+0000","last_clean":"2024-09-21T06:04:31.738639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:04:31.760288+0000","last_undegraded":"2024-09-21T06:04:31.760288+0000","last_fullsized":"2024-09-21T06:04:31.760288+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.755302+0000","last_change":"2024-09-21T06:04:31.769840+0000","last_active":"2024-09-21T06:04:32.755302+0000","last_peered":"2024-09-21T06:04:32.755302+0000","last_clean":"2024-09-21T06:04:32.755302+0000","last_became_active":"2024-09-21T06:04:31.769534+0000","last_became_peered":"2024-09-21T06:04:31.769534+0000","last_unstale":"2024-09-21T06:04:32.755302+0000","last_undegraded":"2024-09-21T06:04:32.755302+0000","last_fullsized":"2024-09-21T06:04:32.755302+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:10.542640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.784911+0000","last_change":"2024-09-21T06:04:32.779505+0000","last_active":"2024-09-21T06:04:32.784911+0000","last_peered":"2024-09-21T06:04:32.784911+0000","last_clean":"2024-09-21T06:04:32.784911+0000","last_became_active":"2024-09-21T06:04:32.779079+0000","last_became_peered":"2024-09-21T06:04:32.779079+0000","last_unstale":"2024-09-21T06:04:32.784911+0000","last_undegraded":"2024-09-21T06:04:32.784911+0000","last_fullsized":"2024-09-21T06:04:32.784911+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:37.256481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.765120+0000","last_change":"2024-09-21T06:04:31.768166+0000","last_active":"2024-09-21T06:04:32.765120+0000","last_peered":"2024-09-21T06:04:32.765120+0000","last_clean":"2024-09-21T06:04:32.765120+0000","last_became_active":"2024-09-21T06:04:31.767622+0000","last_became_peered":"2024-09-21T06:04:31.767622+0000","last_unstale":"2024-09-21T06:04:32.765120+0000","last_undegraded":"2024-09-21T06:04:32.765120+0000","last_fullsized":"2024-09-21T06:04:32.765120+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:14.758437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.780297+0000","last_change":"2024-09-21T06:04:32.776172+0000","last_active":"2024-09-21T06:04:32.780297+0000","last_peered":"2024-09-21T06:04:32.780297+0000","last_clean":"2024-09-21T06:04:32.780297+0000","last_became_active":"2024-09-21T06:04:32.775877+0000","last_became_peered":"2024-09-21T06:04:32.775877+0000","last_unstale":"2024-09-21T06:04:32.780297+0000","last_undegraded":"2024-09-21T06:04:32.780297+0000","last_fullsized":"2024-09-21T06:04:32.780297+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:54.870810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.788947+0000","last_change":"2024-09-21T06:04:31.772282+0000","last_active":"2024-09-21T06:04:32.788947+0000","last_peered":"2024-09-21T06:04:32.788947+0000","last_clean":"2024-09-21T06:04:32.788947+0000","last_became_active":"2024-09-21T06:04:31.770583+0000","last_became_peered":"2024-09-21T06:04:31.770583+0000","last_unstale":"2024-09-21T06:04:32.788947+0000","last_undegraded":"2024-09-21T06:04:32.788947+0000","last_fullsized":"2024-09-21T06:04:32.788947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:35.915894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":201,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T06:04:32.775042+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:04:32.775042+0000","last_peered":"2024-09-21T06:04:32.775042+0000","last_clean":"2024-09-21T06:04:32.775042+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:04:32.775042+0000","last_undegraded":"2024-09-21T06:04:32.775042+0000","last_fullsized":"2024-09-21T06:04:32.775042+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50656,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321184,"statfs":{"total":96636764160,"available":96584892416,"internally_reserved":0,"allocated":1445888,"data_stored":944799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509542,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":851968,"data_stored":353907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509541,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":847872,"data_stored":353905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50064,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321776,"statfs":{"total":96636764160,"available":96585498624,"internally_reserved":0,"allocated":847872,"data_stored":353885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54232,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317608,"statfs":{"total":96636764160,"available":96581230592,"internally_reserved":0,"allocated":851968,"data_stored":353927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50048,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321792,"statfs":{"total":96636764160,"available":96585515008,"internally_reserved":0,"allocated":831488,"data_stored":347012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509541,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54788,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317052,"statfs":{"total":96636764160,"available":96580661248,"internally_reserved":0,"allocated":1425408,"data_stored":937380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58368,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313472,"statfs":{"total":96636764160,"available":96576995328,"internally_reserved":0,"allocated":831488,"data_stored":347012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:04:41.652 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:04:41.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.822+0000 7f289bc00640 1 -- 172.21.15.50:0/312882873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c005160 msgr2=0x7f287c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:41.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.822+0000 7f289bc00640 1 --2- 172.21.15.50:0/312882873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c005160 0x7f287c00b9d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f288c00e400 tx=0x7f288c01d7d0 comp rx=0 tx=0).stop 2024-09-21T06:04:41.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.822+0000 7f2899c00640 1 -- 172.21.15.50:0/312882873 reap_dead start 2024-09-21T06:04:41.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.822+0000 7f289bc00640 1 -- 172.21.15.50:0/312882873 shutdown_connections 2024-09-21T06:04:41.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.822+0000 7f289bc00640 1 -- 172.21.15.50:0/312882873 >> 172.21.15.50:0/312882873 conn(0x7f28940b2040 msgr2=0x7f28940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:41.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.822+0000 7f289bc00640 1 -- 172.21.15.50:0/312882873 shutdown_connections 2024-09-21T06:04:41.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.822+0000 7f289bc00640 1 -- 172.21.15.50:0/312882873 wait complete. 2024-09-21T06:04:41.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.826+0000 7f289bc00640 1 Processor -- start 2024-09-21T06:04:41.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.826+0000 7f289bc00640 1 -- start start 2024-09-21T06:04:41.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.826+0000 7f289bc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f287c005160 0x7f28941e6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:41.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.826+0000 7f289bc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28941f5580 0x7f28941f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:41.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.826+0000 7f289bc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941506c0 0x7f2894150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:41.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.826+0000 7f289bc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f289415cdb0 con 0x7f28941506c0 2024-09-21T06:04:41.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.826+0000 7f289bc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f289415bf20 con 0x7f28941f5580 2024-09-21T06:04:41.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.826+0000 7f289bc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f289415cf30 con 0x7f287c005160 2024-09-21T06:04:41.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2899c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941506c0 0x7f2894150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:41.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2899200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f287c005160 0x7f28941e6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:41.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2899c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941506c0 0x7f2894150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36704/0 (socket says 172.21.15.50:36704) 2024-09-21T06:04:41.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2899c00640 1 -- 172.21.15.50:0/658992596 learned_addr learned my addr 172.21.15.50:0/658992596 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:41.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2893e00640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28941f5580 0x7f28941f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:41.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2899c00640 1 -- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f287c005160 msgr2=0x7f28941e6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:41.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2899c00640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f287c005160 0x7f28941e6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:41.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2899c00640 1 -- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28941f5580 msgr2=0x7f28941f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:41.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2899c00640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28941f5580 0x7f28941f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:41.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2899c00640 1 -- 172.21.15.50:0/658992596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2894151010 con 0x7f28941506c0 2024-09-21T06:04:41.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2899c00640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941506c0 0x7f2894150ad0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f288c00e400 tx=0x7f288c0a8f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:41.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2890c00640 1 -- 172.21.15.50:0/658992596 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c009c60 con 0x7f28941506c0 2024-09-21T06:04:41.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2890c00640 1 -- 172.21.15.50:0/658992596 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f288c004c60 con 0x7f28941506c0 2024-09-21T06:04:41.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f289bc00640 1 -- 172.21.15.50:0/658992596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28941e72f0 con 0x7f28941506c0 2024-09-21T06:04:41.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2890c00640 1 -- 172.21.15.50:0/658992596 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c009200 con 0x7f28941506c0 2024-09-21T06:04:41.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f289bc00640 1 -- 172.21.15.50:0/658992596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28941f6490 con 0x7f28941506c0 2024-09-21T06:04:41.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f289bc00640 1 -- 172.21.15.50:0/658992596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28941f5f20 con 0x7f28941506c0 2024-09-21T06:04:41.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.830+0000 7f2890c00640 1 -- 172.21.15.50:0/658992596 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f288c0c1060 con 0x7f28941506c0 2024-09-21T06:04:41.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.834+0000 7f2890c00640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2874073e90 0x7f28740763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:41.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.834+0000 7f2890c00640 1 -- 172.21.15.50:0/658992596 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f288c12db60 con 0x7f28941506c0 2024-09-21T06:04:41.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.834+0000 7f2899200640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2874073e90 0x7f28740763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:41.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.834+0000 7f2899200640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2874073e90 0x7f28740763a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2894071cb0 tx=0x7f28880091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:41.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:41.838+0000 7f2890c00640 1 -- 172.21.15.50:0/658992596 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2894151010 con 0x7f28941506c0 2024-09-21T06:04:42.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.010+0000 7f289bc00640 1 -- 172.21.15.50:0/658992596 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f289406b540 con 0x7f2874073e90 2024-09-21T06:04:42.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.018+0000 7f2890c00640 1 -- 172.21.15.50:0/658992596 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252030 (secure 0 0 0) 0x7f2888009c90 con 0x7f2874073e90 2024-09-21T06:04:42.021 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:42.023 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:04:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.022+0000 7f289bc00640 1 -- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2874073e90 msgr2=0x7f28740763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.022+0000 7f289bc00640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2874073e90 0x7f28740763a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2894071cb0 tx=0x7f28880091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.022+0000 7f289bc00640 1 -- 172.21.15.50:0/658992596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941506c0 msgr2=0x7f2894150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.022+0000 7f289bc00640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941506c0 0x7f2894150ad0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f288c00e400 tx=0x7f288c0a8f10 comp rx=0 tx=0).stop 2024-09-21T06:04:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.022+0000 7f289bc00640 1 -- 172.21.15.50:0/658992596 shutdown_connections 2024-09-21T06:04:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.022+0000 7f289bc00640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2874073e90 0x7f28740763a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.022+0000 7f289bc00640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941506c0 0x7f2894150ad0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.022+0000 7f289bc00640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28941f5580 0x7f28941f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.022+0000 7f289bc00640 1 --2- 172.21.15.50:0/658992596 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f287c005160 0x7f28941e6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.022+0000 7f289bc00640 1 -- 172.21.15.50:0/658992596 >> 172.21.15.50:0/658992596 conn(0x7f28940b2040 msgr2=0x7f2894151db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.022+0000 7f289bc00640 1 -- 172.21.15.50:0/658992596 shutdown_connections 2024-09-21T06:04:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.022+0000 7f289bc00640 1 -- 172.21.15.50:0/658992596 wait complete. 2024-09-21T06:04:42.043 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-21T06:04:41.024271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":418468,"kb_used_data":7912,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754556252,"statfs":{"total":773094113280,"available":772665602048,"internally_reserved":0,"allocated":8101888,"data_stored":4066700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.273663"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230033+0000","last_change":"2024-09-21T06:04:32.787310+0000","last_active":"2024-09-21T06:04:34.230033+0000","last_peered":"2024-09-21T06:04:34.230033+0000","last_clean":"2024-09-21T06:04:34.230033+0000","last_became_active":"2024-09-21T06:04:32.786715+0000","last_became_peered":"2024-09-21T06:04:32.786715+0000","last_unstale":"2024-09-21T06:04:34.230033+0000","last_undegraded":"2024-09-21T06:04:34.230033+0000","last_fullsized":"2024-09-21T06:04:34.230033+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:00.582472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230085+0000","last_change":"2024-09-21T06:04:32.778255+0000","last_active":"2024-09-21T06:04:34.230085+0000","last_peered":"2024-09-21T06:04:34.230085+0000","last_clean":"2024-09-21T06:04:34.230085+0000","last_became_active":"2024-09-21T06:04:32.777803+0000","last_became_peered":"2024-09-21T06:04:32.777803+0000","last_unstale":"2024-09-21T06:04:34.230085+0000","last_undegraded":"2024-09-21T06:04:34.230085+0000","last_fullsized":"2024-09-21T06:04:34.230085+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:59.802808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228661+0000","last_change":"2024-09-21T06:04:32.772652+0000","last_active":"2024-09-21T06:04:34.228661+0000","last_peered":"2024-09-21T06:04:34.228661+0000","last_clean":"2024-09-21T06:04:34.228661+0000","last_became_active":"2024-09-21T06:04:32.772322+0000","last_became_peered":"2024-09-21T06:04:32.772322+0000","last_unstale":"2024-09-21T06:04:34.228661+0000","last_undegraded":"2024-09-21T06:04:34.228661+0000","last_fullsized":"2024-09-21T06:04:34.228661+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:33.161847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230082+0000","last_change":"2024-09-21T06:04:32.774804+0000","last_active":"2024-09-21T06:04:34.230082+0000","last_peered":"2024-09-21T06:04:34.230082+0000","last_clean":"2024-09-21T06:04:34.230082+0000","last_became_active":"2024-09-21T06:04:32.774467+0000","last_became_peered":"2024-09-21T06:04:32.774467+0000","last_unstale":"2024-09-21T06:04:34.230082+0000","last_undegraded":"2024-09-21T06:04:34.230082+0000","last_fullsized":"2024-09-21T06:04:34.230082+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.739273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228944+0000","last_change":"2024-09-21T06:04:32.777265+0000","last_active":"2024-09-21T06:04:34.228944+0000","last_peered":"2024-09-21T06:04:34.228944+0000","last_clean":"2024-09-21T06:04:34.228944+0000","last_became_active":"2024-09-21T06:04:32.776943+0000","last_became_peered":"2024-09-21T06:04:32.776943+0000","last_unstale":"2024-09-21T06:04:34.228944+0000","last_undegraded":"2024-09-21T06:04:34.228944+0000","last_fullsized":"2024-09-21T06:04:34.228944+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:18.206195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229437+0000","last_change":"2024-09-21T06:04:32.794992+0000","last_active":"2024-09-21T06:04:34.229437+0000","last_peered":"2024-09-21T06:04:34.229437+0000","last_clean":"2024-09-21T06:04:34.229437+0000","last_became_active":"2024-09-21T06:04:32.794580+0000","last_became_peered":"2024-09-21T06:04:32.794580+0000","last_unstale":"2024-09-21T06:04:34.229437+0000","last_undegraded":"2024-09-21T06:04:34.229437+0000","last_fullsized":"2024-09-21T06:04:34.229437+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:11.446707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772322+0000","last_change":"2024-09-21T06:04:32.765123+0000","last_active":"2024-09-21T06:04:33.772322+0000","last_peered":"2024-09-21T06:04:33.772322+0000","last_clean":"2024-09-21T06:04:33.772322+0000","last_became_active":"2024-09-21T06:04:32.764446+0000","last_became_peered":"2024-09-21T06:04:32.764446+0000","last_unstale":"2024-09-21T06:04:33.772322+0000","last_undegraded":"2024-09-21T06:04:33.772322+0000","last_fullsized":"2024-09-21T06:04:33.772322+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:04.139988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228803+0000","last_change":"2024-09-21T06:04:32.773644+0000","last_active":"2024-09-21T06:04:34.228803+0000","last_peered":"2024-09-21T06:04:34.228803+0000","last_clean":"2024-09-21T06:04:34.228803+0000","last_became_active":"2024-09-21T06:04:32.771506+0000","last_became_peered":"2024-09-21T06:04:32.771506+0000","last_unstale":"2024-09-21T06:04:34.228803+0000","last_undegraded":"2024-09-21T06:04:34.228803+0000","last_fullsized":"2024-09-21T06:04:34.228803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:56.572194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229169+0000","last_change":"2024-09-21T06:04:32.775939+0000","last_active":"2024-09-21T06:04:34.229169+0000","last_peered":"2024-09-21T06:04:34.229169+0000","last_clean":"2024-09-21T06:04:34.229169+0000","last_became_active":"2024-09-21T06:04:32.775526+0000","last_became_peered":"2024-09-21T06:04:32.775526+0000","last_unstale":"2024-09-21T06:04:34.229169+0000","last_undegraded":"2024-09-21T06:04:34.229169+0000","last_fullsized":"2024-09-21T06:04:34.229169+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:25.765599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.771961+0000","last_change":"2024-09-21T06:04:32.768237+0000","last_active":"2024-09-21T06:04:33.771961+0000","last_peered":"2024-09-21T06:04:33.771961+0000","last_clean":"2024-09-21T06:04:33.771961+0000","last_became_active":"2024-09-21T06:04:32.767480+0000","last_became_peered":"2024-09-21T06:04:32.767480+0000","last_unstale":"2024-09-21T06:04:33.771961+0000","last_undegraded":"2024-09-21T06:04:33.771961+0000","last_fullsized":"2024-09-21T06:04:33.771961+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:57.290592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229217+0000","last_change":"2024-09-21T06:04:32.786243+0000","last_active":"2024-09-21T06:04:34.229217+0000","last_peered":"2024-09-21T06:04:34.229217+0000","last_clean":"2024-09-21T06:04:34.229217+0000","last_became_active":"2024-09-21T06:04:32.785278+0000","last_became_peered":"2024-09-21T06:04:32.785278+0000","last_unstale":"2024-09-21T06:04:34.229217+0000","last_undegraded":"2024-09-21T06:04:34.229217+0000","last_fullsized":"2024-09-21T06:04:34.229217+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:52.831924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230017+0000","last_change":"2024-09-21T06:04:32.771093+0000","last_active":"2024-09-21T06:04:34.230017+0000","last_peered":"2024-09-21T06:04:34.230017+0000","last_clean":"2024-09-21T06:04:34.230017+0000","last_became_active":"2024-09-21T06:04:32.770795+0000","last_became_peered":"2024-09-21T06:04:32.770795+0000","last_unstale":"2024-09-21T06:04:34.230017+0000","last_undegraded":"2024-09-21T06:04:34.230017+0000","last_fullsized":"2024-09-21T06:04:34.230017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:57.395413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.770871+0000","last_change":"2024-09-21T06:04:32.774414+0000","last_active":"2024-09-21T06:04:33.770871+0000","last_peered":"2024-09-21T06:04:33.770871+0000","last_clean":"2024-09-21T06:04:33.770871+0000","last_became_active":"2024-09-21T06:04:32.774110+0000","last_became_peered":"2024-09-21T06:04:32.774110+0000","last_unstale":"2024-09-21T06:04:33.770871+0000","last_undegraded":"2024-09-21T06:04:33.770871+0000","last_fullsized":"2024-09-21T06:04:33.770871+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:00.582472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229518+0000","last_change":"2024-09-21T06:04:32.760679+0000","last_active":"2024-09-21T06:04:34.229518+0000","last_peered":"2024-09-21T06:04:34.229518+0000","last_clean":"2024-09-21T06:04:34.229518+0000","last_became_active":"2024-09-21T06:04:32.760378+0000","last_became_peered":"2024-09-21T06:04:32.760378+0000","last_unstale":"2024-09-21T06:04:34.229518+0000","last_undegraded":"2024-09-21T06:04:34.229518+0000","last_fullsized":"2024-09-21T06:04:34.229518+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:06.652846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.770600+0000","last_change":"2024-09-21T06:04:32.760660+0000","last_active":"2024-09-21T06:04:33.770600+0000","last_peered":"2024-09-21T06:04:33.770600+0000","last_clean":"2024-09-21T06:04:33.770600+0000","last_became_active":"2024-09-21T06:04:32.760303+0000","last_became_peered":"2024-09-21T06:04:32.760303+0000","last_unstale":"2024-09-21T06:04:33.770600+0000","last_undegraded":"2024-09-21T06:04:33.770600+0000","last_fullsized":"2024-09-21T06:04:33.770600+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:51.415459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.771764+0000","last_change":"2024-09-21T06:04:32.783570+0000","last_active":"2024-09-21T06:04:33.771764+0000","last_peered":"2024-09-21T06:04:33.771764+0000","last_clean":"2024-09-21T06:04:33.771764+0000","last_became_active":"2024-09-21T06:04:32.783265+0000","last_became_peered":"2024-09-21T06:04:32.783265+0000","last_unstale":"2024-09-21T06:04:33.771764+0000","last_undegraded":"2024-09-21T06:04:33.771764+0000","last_fullsized":"2024-09-21T06:04:33.771764+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:42.385243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229258+0000","last_change":"2024-09-21T06:04:32.776322+0000","last_active":"2024-09-21T06:04:34.229258+0000","last_peered":"2024-09-21T06:04:34.229258+0000","last_clean":"2024-09-21T06:04:34.229258+0000","last_became_active":"2024-09-21T06:04:32.776027+0000","last_became_peered":"2024-09-21T06:04:32.776027+0000","last_unstale":"2024-09-21T06:04:34.229258+0000","last_undegraded":"2024-09-21T06:04:34.229258+0000","last_fullsized":"2024-09-21T06:04:34.229258+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:22.855687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230058+0000","last_change":"2024-09-21T06:04:32.774442+0000","last_active":"2024-09-21T06:04:34.230058+0000","last_peered":"2024-09-21T06:04:34.230058+0000","last_clean":"2024-09-21T06:04:34.230058+0000","last_became_active":"2024-09-21T06:04:32.774148+0000","last_became_peered":"2024-09-21T06:04:32.774148+0000","last_unstale":"2024-09-21T06:04:34.230058+0000","last_undegraded":"2024-09-21T06:04:34.230058+0000","last_fullsized":"2024-09-21T06:04:34.230058+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:03.651120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229005+0000","last_change":"2024-09-21T06:04:32.771994+0000","last_active":"2024-09-21T06:04:34.229005+0000","last_peered":"2024-09-21T06:04:34.229005+0000","last_clean":"2024-09-21T06:04:34.229005+0000","last_became_active":"2024-09-21T06:04:32.771597+0000","last_became_peered":"2024-09-21T06:04:32.771597+0000","last_unstale":"2024-09-21T06:04:34.229005+0000","last_undegraded":"2024-09-21T06:04:34.229005+0000","last_fullsized":"2024-09-21T06:04:34.229005+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:41.313030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.770901+0000","last_change":"2024-09-21T06:04:32.766745+0000","last_active":"2024-09-21T06:04:33.770901+0000","last_peered":"2024-09-21T06:04:33.770901+0000","last_clean":"2024-09-21T06:04:33.770901+0000","last_became_active":"2024-09-21T06:04:32.766362+0000","last_became_peered":"2024-09-21T06:04:32.766362+0000","last_unstale":"2024-09-21T06:04:33.770901+0000","last_undegraded":"2024-09-21T06:04:33.770901+0000","last_fullsized":"2024-09-21T06:04:33.770901+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:20.870190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772438+0000","last_change":"2024-09-21T06:04:32.760980+0000","last_active":"2024-09-21T06:04:33.772438+0000","last_peered":"2024-09-21T06:04:33.772438+0000","last_clean":"2024-09-21T06:04:33.772438+0000","last_became_active":"2024-09-21T06:04:32.760648+0000","last_became_peered":"2024-09-21T06:04:32.760648+0000","last_unstale":"2024-09-21T06:04:33.772438+0000","last_undegraded":"2024-09-21T06:04:33.772438+0000","last_fullsized":"2024-09-21T06:04:33.772438+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:31.640086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772052+0000","last_change":"2024-09-21T06:04:32.762713+0000","last_active":"2024-09-21T06:04:33.772052+0000","last_peered":"2024-09-21T06:04:33.772052+0000","last_clean":"2024-09-21T06:04:33.772052+0000","last_became_active":"2024-09-21T06:04:32.762345+0000","last_became_peered":"2024-09-21T06:04:32.762345+0000","last_unstale":"2024-09-21T06:04:33.772052+0000","last_undegraded":"2024-09-21T06:04:33.772052+0000","last_fullsized":"2024-09-21T06:04:33.772052+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:10.351688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228639+0000","last_change":"2024-09-21T06:04:32.788061+0000","last_active":"2024-09-21T06:04:34.228639+0000","last_peered":"2024-09-21T06:04:34.228639+0000","last_clean":"2024-09-21T06:04:34.228639+0000","last_became_active":"2024-09-21T06:04:32.787777+0000","last_became_peered":"2024-09-21T06:04:32.787777+0000","last_unstale":"2024-09-21T06:04:34.228639+0000","last_undegraded":"2024-09-21T06:04:34.228639+0000","last_fullsized":"2024-09-21T06:04:34.228639+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:11.805943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228446+0000","last_change":"2024-09-21T06:04:32.790938+0000","last_active":"2024-09-21T06:04:34.228446+0000","last_peered":"2024-09-21T06:04:34.228446+0000","last_clean":"2024-09-21T06:04:34.228446+0000","last_became_active":"2024-09-21T06:04:32.790267+0000","last_became_peered":"2024-09-21T06:04:32.790267+0000","last_unstale":"2024-09-21T06:04:34.228446+0000","last_undegraded":"2024-09-21T06:04:34.228446+0000","last_fullsized":"2024-09-21T06:04:34.228446+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:39.036138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772961+0000","last_change":"2024-09-21T06:04:32.779963+0000","last_active":"2024-09-21T06:04:33.772961+0000","last_peered":"2024-09-21T06:04:33.772961+0000","last_clean":"2024-09-21T06:04:33.772961+0000","last_became_active":"2024-09-21T06:04:32.779659+0000","last_became_peered":"2024-09-21T06:04:32.779659+0000","last_unstale":"2024-09-21T06:04:33.772961+0000","last_undegraded":"2024-09-21T06:04:33.772961+0000","last_fullsized":"2024-09-21T06:04:33.772961+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:33.129220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.773479+0000","last_change":"2024-09-21T06:04:32.778117+0000","last_active":"2024-09-21T06:04:33.773479+0000","last_peered":"2024-09-21T06:04:33.773479+0000","last_clean":"2024-09-21T06:04:33.773479+0000","last_became_active":"2024-09-21T06:04:32.777105+0000","last_became_peered":"2024-09-21T06:04:32.777105+0000","last_unstale":"2024-09-21T06:04:33.773479+0000","last_undegraded":"2024-09-21T06:04:33.773479+0000","last_fullsized":"2024-09-21T06:04:33.773479+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:39.036138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228914+0000","last_change":"2024-09-21T06:04:32.771946+0000","last_active":"2024-09-21T06:04:34.228914+0000","last_peered":"2024-09-21T06:04:34.228914+0000","last_clean":"2024-09-21T06:04:34.228914+0000","last_became_active":"2024-09-21T06:04:32.771652+0000","last_became_peered":"2024-09-21T06:04:32.771652+0000","last_unstale":"2024-09-21T06:04:34.228914+0000","last_undegraded":"2024-09-21T06:04:34.228914+0000","last_fullsized":"2024-09-21T06:04:34.228914+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:12.296413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772366+0000","last_change":"2024-09-21T06:04:32.780368+0000","last_active":"2024-09-21T06:04:33.772366+0000","last_peered":"2024-09-21T06:04:33.772366+0000","last_clean":"2024-09-21T06:04:33.772366+0000","last_became_active":"2024-09-21T06:04:32.780021+0000","last_became_peered":"2024-09-21T06:04:32.780021+0000","last_unstale":"2024-09-21T06:04:33.772366+0000","last_undegraded":"2024-09-21T06:04:33.772366+0000","last_fullsized":"2024-09-21T06:04:33.772366+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:12.581470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228946+0000","last_change":"2024-09-21T06:04:32.773537+0000","last_active":"2024-09-21T06:04:34.228946+0000","last_peered":"2024-09-21T06:04:34.228946+0000","last_clean":"2024-09-21T06:04:34.228946+0000","last_became_active":"2024-09-21T06:04:32.773240+0000","last_became_peered":"2024-09-21T06:04:32.773240+0000","last_unstale":"2024-09-21T06:04:34.228946+0000","last_undegraded":"2024-09-21T06:04:34.228946+0000","last_fullsized":"2024-09-21T06:04:34.228946+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:53.456264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772343+0000","last_change":"2024-09-21T06:04:32.782666+0000","last_active":"2024-09-21T06:04:33.772343+0000","last_peered":"2024-09-21T06:04:33.772343+0000","last_clean":"2024-09-21T06:04:33.772343+0000","last_became_active":"2024-09-21T06:04:32.777455+0000","last_became_peered":"2024-09-21T06:04:32.777455+0000","last_unstale":"2024-09-21T06:04:33.772343+0000","last_undegraded":"2024-09-21T06:04:33.772343+0000","last_fullsized":"2024-09-21T06:04:33.772343+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:52.825226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772582+0000","last_change":"2024-09-21T06:04:32.764381+0000","last_active":"2024-09-21T06:04:33.772582+0000","last_peered":"2024-09-21T06:04:33.772582+0000","last_clean":"2024-09-21T06:04:33.772582+0000","last_became_active":"2024-09-21T06:04:32.764040+0000","last_became_peered":"2024-09-21T06:04:32.764040+0000","last_unstale":"2024-09-21T06:04:33.772582+0000","last_undegraded":"2024-09-21T06:04:33.772582+0000","last_fullsized":"2024-09-21T06:04:33.772582+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:28.978078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229087+0000","last_change":"2024-09-21T06:04:32.774466+0000","last_active":"2024-09-21T06:04:34.229087+0000","last_peered":"2024-09-21T06:04:34.229087+0000","last_clean":"2024-09-21T06:04:34.229087+0000","last_became_active":"2024-09-21T06:04:32.774133+0000","last_became_peered":"2024-09-21T06:04:32.774133+0000","last_unstale":"2024-09-21T06:04:34.229087+0000","last_undegraded":"2024-09-21T06:04:34.229087+0000","last_fullsized":"2024-09-21T06:04:34.229087+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:33.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230584+0000","last_change":"2024-09-21T06:04:32.792389+0000","last_active":"2024-09-21T06:04:34.230584+0000","last_peered":"2024-09-21T06:04:34.230584+0000","last_clean":"2024-09-21T06:04:34.230584+0000","last_became_active":"2024-09-21T06:04:32.792097+0000","last_became_peered":"2024-09-21T06:04:32.792097+0000","last_unstale":"2024-09-21T06:04:34.230584+0000","last_undegraded":"2024-09-21T06:04:34.230584+0000","last_fullsized":"2024-09-21T06:04:34.230584+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:24.387854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230292+0000","last_change":"2024-09-21T06:04:31.777083+0000","last_active":"2024-09-21T06:04:34.230292+0000","last_peered":"2024-09-21T06:04:34.230292+0000","last_clean":"2024-09-21T06:04:34.230292+0000","last_became_active":"2024-09-21T06:04:31.776739+0000","last_became_peered":"2024-09-21T06:04:31.776739+0000","last_unstale":"2024-09-21T06:04:34.230292+0000","last_undegraded":"2024-09-21T06:04:34.230292+0000","last_fullsized":"2024-09-21T06:04:34.230292+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:59.686723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229380+0000","last_change":"2024-09-21T06:04:32.776263+0000","last_active":"2024-09-21T06:04:34.229380+0000","last_peered":"2024-09-21T06:04:34.229380+0000","last_clean":"2024-09-21T06:04:34.229380+0000","last_became_active":"2024-09-21T06:04:32.775659+0000","last_became_peered":"2024-09-21T06:04:32.775659+0000","last_unstale":"2024-09-21T06:04:34.229380+0000","last_undegraded":"2024-09-21T06:04:34.229380+0000","last_fullsized":"2024-09-21T06:04:34.229380+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:42.107381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.771278+0000","last_change":"2024-09-21T06:04:31.769588+0000","last_active":"2024-09-21T06:04:33.771278+0000","last_peered":"2024-09-21T06:04:33.771278+0000","last_clean":"2024-09-21T06:04:33.771278+0000","last_became_active":"2024-09-21T06:04:31.768905+0000","last_became_peered":"2024-09-21T06:04:31.768905+0000","last_unstale":"2024-09-21T06:04:33.771278+0000","last_undegraded":"2024-09-21T06:04:33.771278+0000","last_fullsized":"2024-09-21T06:04:33.771278+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:27.975438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229075+0000","last_change":"2024-09-21T06:04:32.788286+0000","last_active":"2024-09-21T06:04:34.229075+0000","last_peered":"2024-09-21T06:04:34.229075+0000","last_clean":"2024-09-21T06:04:34.229075+0000","last_became_active":"2024-09-21T06:04:32.787926+0000","last_became_peered":"2024-09-21T06:04:32.787926+0000","last_unstale":"2024-09-21T06:04:34.229075+0000","last_undegraded":"2024-09-21T06:04:34.229075+0000","last_fullsized":"2024-09-21T06:04:34.229075+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:00.689363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772098+0000","last_change":"2024-09-21T06:04:31.772963+0000","last_active":"2024-09-21T06:04:33.772098+0000","last_peered":"2024-09-21T06:04:33.772098+0000","last_clean":"2024-09-21T06:04:33.772098+0000","last_became_active":"2024-09-21T06:04:31.772653+0000","last_became_peered":"2024-09-21T06:04:31.772653+0000","last_unstale":"2024-09-21T06:04:33.772098+0000","last_undegraded":"2024-09-21T06:04:33.772098+0000","last_fullsized":"2024-09-21T06:04:33.772098+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:59.868880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229990+0000","last_change":"2024-09-21T06:04:32.785502+0000","last_active":"2024-09-21T06:04:34.229990+0000","last_peered":"2024-09-21T06:04:34.229990+0000","last_clean":"2024-09-21T06:04:34.229990+0000","last_became_active":"2024-09-21T06:04:32.785091+0000","last_became_peered":"2024-09-21T06:04:32.785091+0000","last_unstale":"2024-09-21T06:04:34.229990+0000","last_undegraded":"2024-09-21T06:04:34.229990+0000","last_fullsized":"2024-09-21T06:04:34.229990+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:43.257993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229298+0000","last_change":"2024-09-21T06:04:31.775276+0000","last_active":"2024-09-21T06:04:34.229298+0000","last_peered":"2024-09-21T06:04:34.229298+0000","last_clean":"2024-09-21T06:04:34.229298+0000","last_became_active":"2024-09-21T06:04:31.774028+0000","last_became_peered":"2024-09-21T06:04:31.774028+0000","last_unstale":"2024-09-21T06:04:34.229298+0000","last_undegraded":"2024-09-21T06:04:34.229298+0000","last_fullsized":"2024-09-21T06:04:34.229298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:18.843912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772697+0000","last_change":"2024-09-21T06:04:32.777770+0000","last_active":"2024-09-21T06:04:33.772697+0000","last_peered":"2024-09-21T06:04:33.772697+0000","last_clean":"2024-09-21T06:04:33.772697+0000","last_became_active":"2024-09-21T06:04:32.777019+0000","last_became_peered":"2024-09-21T06:04:32.777019+0000","last_unstale":"2024-09-21T06:04:33.772697+0000","last_undegraded":"2024-09-21T06:04:33.772697+0000","last_fullsized":"2024-09-21T06:04:33.772697+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:36.679619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229135+0000","last_change":"2024-09-21T06:04:31.766081+0000","last_active":"2024-09-21T06:04:34.229135+0000","last_peered":"2024-09-21T06:04:34.229135+0000","last_clean":"2024-09-21T06:04:34.229135+0000","last_became_active":"2024-09-21T06:04:31.765541+0000","last_became_peered":"2024-09-21T06:04:31.765541+0000","last_unstale":"2024-09-21T06:04:34.229135+0000","last_undegraded":"2024-09-21T06:04:34.229135+0000","last_fullsized":"2024-09-21T06:04:34.229135+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:40.183984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772698+0000","last_change":"2024-09-21T06:04:32.780478+0000","last_active":"2024-09-21T06:04:33.772698+0000","last_peered":"2024-09-21T06:04:33.772698+0000","last_clean":"2024-09-21T06:04:33.772698+0000","last_became_active":"2024-09-21T06:04:32.778124+0000","last_became_peered":"2024-09-21T06:04:32.778124+0000","last_unstale":"2024-09-21T06:04:33.772698+0000","last_undegraded":"2024-09-21T06:04:33.772698+0000","last_fullsized":"2024-09-21T06:04:33.772698+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:33.487309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228766+0000","last_change":"2024-09-21T06:04:31.772094+0000","last_active":"2024-09-21T06:04:34.228766+0000","last_peered":"2024-09-21T06:04:34.228766+0000","last_clean":"2024-09-21T06:04:34.228766+0000","last_became_active":"2024-09-21T06:04:31.771544+0000","last_became_peered":"2024-09-21T06:04:31.771544+0000","last_unstale":"2024-09-21T06:04:34.228766+0000","last_undegraded":"2024-09-21T06:04:34.228766+0000","last_fullsized":"2024-09-21T06:04:34.228766+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:03.600601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228406+0000","last_change":"2024-09-21T06:04:32.761010+0000","last_active":"2024-09-21T06:04:34.228406+0000","last_peered":"2024-09-21T06:04:34.228406+0000","last_clean":"2024-09-21T06:04:34.228406+0000","last_became_active":"2024-09-21T06:04:32.760708+0000","last_became_peered":"2024-09-21T06:04:32.760708+0000","last_unstale":"2024-09-21T06:04:34.228406+0000","last_undegraded":"2024-09-21T06:04:34.228406+0000","last_fullsized":"2024-09-21T06:04:34.228406+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:54.853883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229709+0000","last_change":"2024-09-21T06:04:31.768175+0000","last_active":"2024-09-21T06:04:34.229709+0000","last_peered":"2024-09-21T06:04:34.229709+0000","last_clean":"2024-09-21T06:04:34.229709+0000","last_became_active":"2024-09-21T06:04:31.767820+0000","last_became_peered":"2024-09-21T06:04:31.767820+0000","last_unstale":"2024-09-21T06:04:34.229709+0000","last_undegraded":"2024-09-21T06:04:34.229709+0000","last_fullsized":"2024-09-21T06:04:34.229709+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:10.429860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.770673+0000","last_change":"2024-09-21T06:04:32.772784+0000","last_active":"2024-09-21T06:04:33.770673+0000","last_peered":"2024-09-21T06:04:33.770673+0000","last_clean":"2024-09-21T06:04:33.770673+0000","last_became_active":"2024-09-21T06:04:32.772246+0000","last_became_peered":"2024-09-21T06:04:32.772246+0000","last_unstale":"2024-09-21T06:04:33.770673+0000","last_undegraded":"2024-09-21T06:04:33.770673+0000","last_fullsized":"2024-09-21T06:04:33.770673+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:43.257993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228642+0000","last_change":"2024-09-21T06:04:31.759185+0000","last_active":"2024-09-21T06:04:34.228642+0000","last_peered":"2024-09-21T06:04:34.228642+0000","last_clean":"2024-09-21T06:04:34.228642+0000","last_became_active":"2024-09-21T06:04:31.758873+0000","last_became_peered":"2024-09-21T06:04:31.758873+0000","last_unstale":"2024-09-21T06:04:34.228642+0000","last_undegraded":"2024-09-21T06:04:34.228642+0000","last_fullsized":"2024-09-21T06:04:34.228642+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:37.617451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.773657+0000","last_change":"2024-09-21T06:04:32.769066+0000","last_active":"2024-09-21T06:04:33.773657+0000","last_peered":"2024-09-21T06:04:33.773657+0000","last_clean":"2024-09-21T06:04:33.773657+0000","last_became_active":"2024-09-21T06:04:32.768695+0000","last_became_peered":"2024-09-21T06:04:32.768695+0000","last_unstale":"2024-09-21T06:04:33.773657+0000","last_undegraded":"2024-09-21T06:04:33.773657+0000","last_fullsized":"2024-09-21T06:04:33.773657+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:52.831924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772985+0000","last_change":"2024-09-21T06:04:31.770884+0000","last_active":"2024-09-21T06:04:33.772985+0000","last_peered":"2024-09-21T06:04:33.772985+0000","last_clean":"2024-09-21T06:04:33.772985+0000","last_became_active":"2024-09-21T06:04:31.768869+0000","last_became_peered":"2024-09-21T06:04:31.768869+0000","last_unstale":"2024-09-21T06:04:33.772985+0000","last_undegraded":"2024-09-21T06:04:33.772985+0000","last_fullsized":"2024-09-21T06:04:33.772985+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:18.843912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230063+0000","last_change":"2024-09-21T06:04:32.791807+0000","last_active":"2024-09-21T06:04:34.230063+0000","last_peered":"2024-09-21T06:04:34.230063+0000","last_clean":"2024-09-21T06:04:34.230063+0000","last_became_active":"2024-09-21T06:04:32.791461+0000","last_became_peered":"2024-09-21T06:04:32.791461+0000","last_unstale":"2024-09-21T06:04:34.230063+0000","last_undegraded":"2024-09-21T06:04:34.230063+0000","last_fullsized":"2024-09-21T06:04:34.230063+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:20.870190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230285+0000","last_change":"2024-09-21T06:04:31.770081+0000","last_active":"2024-09-21T06:04:34.230285+0000","last_peered":"2024-09-21T06:04:34.230285+0000","last_clean":"2024-09-21T06:04:34.230285+0000","last_became_active":"2024-09-21T06:04:31.768462+0000","last_became_peered":"2024-09-21T06:04:31.768462+0000","last_unstale":"2024-09-21T06:04:34.230285+0000","last_undegraded":"2024-09-21T06:04:34.230285+0000","last_fullsized":"2024-09-21T06:04:34.230285+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:10.542640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229138+0000","last_change":"2024-09-21T06:04:32.795515+0000","last_active":"2024-09-21T06:04:34.229138+0000","last_peered":"2024-09-21T06:04:34.229138+0000","last_clean":"2024-09-21T06:04:34.229138+0000","last_became_active":"2024-09-21T06:04:32.795019+0000","last_became_peered":"2024-09-21T06:04:32.795019+0000","last_unstale":"2024-09-21T06:04:34.229138+0000","last_undegraded":"2024-09-21T06:04:34.229138+0000","last_fullsized":"2024-09-21T06:04:34.229138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:41.742767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229102+0000","last_change":"2024-09-21T06:04:31.769723+0000","last_active":"2024-09-21T06:04:34.229102+0000","last_peered":"2024-09-21T06:04:34.229102+0000","last_clean":"2024-09-21T06:04:34.229102+0000","last_became_active":"2024-09-21T06:04:31.769344+0000","last_became_peered":"2024-09-21T06:04:31.769344+0000","last_unstale":"2024-09-21T06:04:34.229102+0000","last_undegraded":"2024-09-21T06:04:34.229102+0000","last_fullsized":"2024-09-21T06:04:34.229102+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:00.566146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229386+0000","last_change":"2024-09-21T06:04:32.767569+0000","last_active":"2024-09-21T06:04:34.229386+0000","last_peered":"2024-09-21T06:04:34.229386+0000","last_clean":"2024-09-21T06:04:34.229386+0000","last_became_active":"2024-09-21T06:04:32.766934+0000","last_became_peered":"2024-09-21T06:04:32.766934+0000","last_unstale":"2024-09-21T06:04:34.229386+0000","last_undegraded":"2024-09-21T06:04:34.229386+0000","last_fullsized":"2024-09-21T06:04:34.229386+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:32.002829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230273+0000","last_change":"2024-09-21T06:04:31.768773+0000","last_active":"2024-09-21T06:04:34.230273+0000","last_peered":"2024-09-21T06:04:34.230273+0000","last_clean":"2024-09-21T06:04:34.230273+0000","last_became_active":"2024-09-21T06:04:31.767292+0000","last_became_peered":"2024-09-21T06:04:31.767292+0000","last_unstale":"2024-09-21T06:04:34.230273+0000","last_undegraded":"2024-09-21T06:04:34.230273+0000","last_fullsized":"2024-09-21T06:04:34.230273+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:43.524114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229733+0000","last_change":"2024-09-21T06:04:32.784585+0000","last_active":"2024-09-21T06:04:34.229733+0000","last_peered":"2024-09-21T06:04:34.229733+0000","last_clean":"2024-09-21T06:04:34.229733+0000","last_became_active":"2024-09-21T06:04:32.784259+0000","last_became_peered":"2024-09-21T06:04:32.784259+0000","last_unstale":"2024-09-21T06:04:34.229733+0000","last_undegraded":"2024-09-21T06:04:34.229733+0000","last_fullsized":"2024-09-21T06:04:34.229733+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:48.829674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230021+0000","last_change":"2024-09-21T06:04:31.772181+0000","last_active":"2024-09-21T06:04:34.230021+0000","last_peered":"2024-09-21T06:04:34.230021+0000","last_clean":"2024-09-21T06:04:34.230021+0000","last_became_active":"2024-09-21T06:04:31.771816+0000","last_became_peered":"2024-09-21T06:04:31.771816+0000","last_unstale":"2024-09-21T06:04:34.230021+0000","last_undegraded":"2024-09-21T06:04:34.230021+0000","last_fullsized":"2024-09-21T06:04:34.230021+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:34.818183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.773381+0000","last_change":"2024-09-21T06:04:32.768202+0000","last_active":"2024-09-21T06:04:33.773381+0000","last_peered":"2024-09-21T06:04:33.773381+0000","last_clean":"2024-09-21T06:04:33.773381+0000","last_became_active":"2024-09-21T06:04:32.767810+0000","last_became_peered":"2024-09-21T06:04:32.767810+0000","last_unstale":"2024-09-21T06:04:33.773381+0000","last_undegraded":"2024-09-21T06:04:33.773381+0000","last_fullsized":"2024-09-21T06:04:33.773381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:12.666560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228915+0000","last_change":"2024-09-21T06:04:31.775115+0000","last_active":"2024-09-21T06:04:34.228915+0000","last_peered":"2024-09-21T06:04:34.228915+0000","last_clean":"2024-09-21T06:04:34.228915+0000","last_became_active":"2024-09-21T06:04:31.773039+0000","last_became_peered":"2024-09-21T06:04:31.773039+0000","last_unstale":"2024-09-21T06:04:34.228915+0000","last_undegraded":"2024-09-21T06:04:34.228915+0000","last_fullsized":"2024-09-21T06:04:34.228915+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:54.006741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.770531+0000","last_change":"2024-09-21T06:04:32.780256+0000","last_active":"2024-09-21T06:04:33.770531+0000","last_peered":"2024-09-21T06:04:33.770531+0000","last_clean":"2024-09-21T06:04:33.770531+0000","last_became_active":"2024-09-21T06:04:32.779924+0000","last_became_peered":"2024-09-21T06:04:32.779924+0000","last_unstale":"2024-09-21T06:04:33.770531+0000","last_undegraded":"2024-09-21T06:04:33.770531+0000","last_fullsized":"2024-09-21T06:04:33.770531+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:48.829674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772351+0000","last_change":"2024-09-21T06:04:31.775645+0000","last_active":"2024-09-21T06:04:33.772351+0000","last_peered":"2024-09-21T06:04:33.772351+0000","last_clean":"2024-09-21T06:04:33.772351+0000","last_became_active":"2024-09-21T06:04:31.775129+0000","last_became_peered":"2024-09-21T06:04:31.775129+0000","last_unstale":"2024-09-21T06:04:33.772351+0000","last_undegraded":"2024-09-21T06:04:33.772351+0000","last_fullsized":"2024-09-21T06:04:33.772351+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:55.569554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229037+0000","last_change":"2024-09-21T06:04:32.770814+0000","last_active":"2024-09-21T06:04:34.229037+0000","last_peered":"2024-09-21T06:04:34.229037+0000","last_clean":"2024-09-21T06:04:34.229037+0000","last_became_active":"2024-09-21T06:04:32.770411+0000","last_became_peered":"2024-09-21T06:04:32.770411+0000","last_unstale":"2024-09-21T06:04:34.229037+0000","last_undegraded":"2024-09-21T06:04:34.229037+0000","last_fullsized":"2024-09-21T06:04:34.229037+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:33.148573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.771763+0000","last_change":"2024-09-21T06:04:31.772137+0000","last_active":"2024-09-21T06:04:33.771763+0000","last_peered":"2024-09-21T06:04:33.771763+0000","last_clean":"2024-09-21T06:04:33.771763+0000","last_became_active":"2024-09-21T06:04:31.771622+0000","last_became_peered":"2024-09-21T06:04:31.771622+0000","last_unstale":"2024-09-21T06:04:33.771763+0000","last_undegraded":"2024-09-21T06:04:33.771763+0000","last_fullsized":"2024-09-21T06:04:33.771763+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:38.991457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772964+0000","last_change":"2024-09-21T06:04:32.768582+0000","last_active":"2024-09-21T06:04:33.772964+0000","last_peered":"2024-09-21T06:04:33.772964+0000","last_clean":"2024-09-21T06:04:33.772964+0000","last_became_active":"2024-09-21T06:04:32.767918+0000","last_became_peered":"2024-09-21T06:04:32.767918+0000","last_unstale":"2024-09-21T06:04:33.772964+0000","last_undegraded":"2024-09-21T06:04:33.772964+0000","last_fullsized":"2024-09-21T06:04:33.772964+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:14.363193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230488+0000","last_change":"2024-09-21T06:04:31.775324+0000","last_active":"2024-09-21T06:04:34.230488+0000","last_peered":"2024-09-21T06:04:34.230488+0000","last_clean":"2024-09-21T06:04:34.230488+0000","last_became_active":"2024-09-21T06:04:31.774989+0000","last_became_peered":"2024-09-21T06:04:31.774989+0000","last_unstale":"2024-09-21T06:04:34.230488+0000","last_undegraded":"2024-09-21T06:04:34.230488+0000","last_fullsized":"2024-09-21T06:04:34.230488+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:10.782962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230384+0000","last_change":"2024-09-21T06:04:32.787340+0000","last_active":"2024-09-21T06:04:34.230384+0000","last_peered":"2024-09-21T06:04:34.230384+0000","last_clean":"2024-09-21T06:04:34.230384+0000","last_became_active":"2024-09-21T06:04:32.787027+0000","last_became_peered":"2024-09-21T06:04:32.787027+0000","last_unstale":"2024-09-21T06:04:34.230384+0000","last_undegraded":"2024-09-21T06:04:34.230384+0000","last_fullsized":"2024-09-21T06:04:34.230384+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:38.471875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772869+0000","last_change":"2024-09-21T06:04:31.774380+0000","last_active":"2024-09-21T06:04:33.772869+0000","last_peered":"2024-09-21T06:04:33.772869+0000","last_clean":"2024-09-21T06:04:33.772869+0000","last_became_active":"2024-09-21T06:04:31.773835+0000","last_became_peered":"2024-09-21T06:04:31.773835+0000","last_unstale":"2024-09-21T06:04:33.772869+0000","last_undegraded":"2024-09-21T06:04:33.772869+0000","last_fullsized":"2024-09-21T06:04:33.772869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:53.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230271+0000","last_change":"2024-09-21T06:04:32.772562+0000","last_active":"2024-09-21T06:04:34.230271+0000","last_peered":"2024-09-21T06:04:34.230271+0000","last_clean":"2024-09-21T06:04:34.230271+0000","last_became_active":"2024-09-21T06:04:32.771141+0000","last_became_peered":"2024-09-21T06:04:32.771141+0000","last_unstale":"2024-09-21T06:04:34.230271+0000","last_undegraded":"2024-09-21T06:04:34.230271+0000","last_fullsized":"2024-09-21T06:04:34.230271+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:30.093433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229705+0000","last_change":"2024-09-21T06:04:31.768595+0000","last_active":"2024-09-21T06:04:34.229705+0000","last_peered":"2024-09-21T06:04:34.229705+0000","last_clean":"2024-09-21T06:04:34.229705+0000","last_became_active":"2024-09-21T06:04:31.767603+0000","last_became_peered":"2024-09-21T06:04:31.767603+0000","last_unstale":"2024-09-21T06:04:34.229705+0000","last_undegraded":"2024-09-21T06:04:34.229705+0000","last_fullsized":"2024-09-21T06:04:34.229705+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:53.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229706+0000","last_change":"2024-09-21T06:04:32.774290+0000","last_active":"2024-09-21T06:04:34.229706+0000","last_peered":"2024-09-21T06:04:34.229706+0000","last_clean":"2024-09-21T06:04:34.229706+0000","last_became_active":"2024-09-21T06:04:32.774023+0000","last_became_peered":"2024-09-21T06:04:32.774023+0000","last_unstale":"2024-09-21T06:04:34.229706+0000","last_undegraded":"2024-09-21T06:04:34.229706+0000","last_fullsized":"2024-09-21T06:04:34.229706+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:15.692355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229659+0000","last_change":"2024-09-21T06:04:31.774500+0000","last_active":"2024-09-21T06:04:34.229659+0000","last_peered":"2024-09-21T06:04:34.229659+0000","last_clean":"2024-09-21T06:04:34.229659+0000","last_became_active":"2024-09-21T06:04:31.774161+0000","last_became_peered":"2024-09-21T06:04:31.774161+0000","last_unstale":"2024-09-21T06:04:34.229659+0000","last_undegraded":"2024-09-21T06:04:34.229659+0000","last_fullsized":"2024-09-21T06:04:34.229659+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:32.288314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229360+0000","last_change":"2024-09-21T06:04:32.769815+0000","last_active":"2024-09-21T06:04:34.229360+0000","last_peered":"2024-09-21T06:04:34.229360+0000","last_clean":"2024-09-21T06:04:34.229360+0000","last_became_active":"2024-09-21T06:04:32.769467+0000","last_became_peered":"2024-09-21T06:04:32.769467+0000","last_unstale":"2024-09-21T06:04:34.229360+0000","last_undegraded":"2024-09-21T06:04:34.229360+0000","last_fullsized":"2024-09-21T06:04:34.229360+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:01.951599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229512+0000","last_change":"2024-09-21T06:04:31.772131+0000","last_active":"2024-09-21T06:04:34.229512+0000","last_peered":"2024-09-21T06:04:34.229512+0000","last_clean":"2024-09-21T06:04:34.229512+0000","last_became_active":"2024-09-21T06:04:31.771792+0000","last_became_peered":"2024-09-21T06:04:31.771792+0000","last_unstale":"2024-09-21T06:04:34.229512+0000","last_undegraded":"2024-09-21T06:04:34.229512+0000","last_fullsized":"2024-09-21T06:04:34.229512+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.561776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228450+0000","last_change":"2024-09-21T06:04:32.782337+0000","last_active":"2024-09-21T06:04:34.228450+0000","last_peered":"2024-09-21T06:04:34.228450+0000","last_clean":"2024-09-21T06:04:34.228450+0000","last_became_active":"2024-09-21T06:04:32.782011+0000","last_became_peered":"2024-09-21T06:04:32.782011+0000","last_unstale":"2024-09-21T06:04:34.228450+0000","last_undegraded":"2024-09-21T06:04:34.228450+0000","last_fullsized":"2024-09-21T06:04:34.228450+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:00.534033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.772572+0000","last_change":"2024-09-21T06:04:31.766119+0000","last_active":"2024-09-21T06:04:33.772572+0000","last_peered":"2024-09-21T06:04:33.772572+0000","last_clean":"2024-09-21T06:04:33.772572+0000","last_became_active":"2024-09-21T06:04:31.765398+0000","last_became_peered":"2024-09-21T06:04:31.765398+0000","last_unstale":"2024-09-21T06:04:33.772572+0000","last_undegraded":"2024-09-21T06:04:33.772572+0000","last_fullsized":"2024-09-21T06:04:33.772572+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:23.042000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228624+0000","last_change":"2024-09-21T06:04:32.777061+0000","last_active":"2024-09-21T06:04:34.228624+0000","last_peered":"2024-09-21T06:04:34.228624+0000","last_clean":"2024-09-21T06:04:34.228624+0000","last_became_active":"2024-09-21T06:04:32.776703+0000","last_became_peered":"2024-09-21T06:04:32.776703+0000","last_unstale":"2024-09-21T06:04:34.228624+0000","last_undegraded":"2024-09-21T06:04:34.228624+0000","last_fullsized":"2024-09-21T06:04:34.228624+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:26.428611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.770443+0000","last_change":"2024-09-21T06:04:31.770508+0000","last_active":"2024-09-21T06:04:33.770443+0000","last_peered":"2024-09-21T06:04:33.770443+0000","last_clean":"2024-09-21T06:04:33.770443+0000","last_became_active":"2024-09-21T06:04:31.769020+0000","last_became_peered":"2024-09-21T06:04:31.769020+0000","last_unstale":"2024-09-21T06:04:33.770443+0000","last_undegraded":"2024-09-21T06:04:33.770443+0000","last_fullsized":"2024-09-21T06:04:33.770443+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:40.893581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.771309+0000","last_change":"2024-09-21T06:04:32.762522+0000","last_active":"2024-09-21T06:04:33.771309+0000","last_peered":"2024-09-21T06:04:33.771309+0000","last_clean":"2024-09-21T06:04:33.771309+0000","last_became_active":"2024-09-21T06:04:32.762144+0000","last_became_peered":"2024-09-21T06:04:32.762144+0000","last_unstale":"2024-09-21T06:04:33.771309+0000","last_undegraded":"2024-09-21T06:04:33.771309+0000","last_fullsized":"2024-09-21T06:04:33.771309+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:31.531894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230583+0000","last_change":"2024-09-21T06:04:31.774958+0000","last_active":"2024-09-21T06:04:34.230583+0000","last_peered":"2024-09-21T06:04:34.230583+0000","last_clean":"2024-09-21T06:04:34.230583+0000","last_became_active":"2024-09-21T06:04:31.774595+0000","last_became_peered":"2024-09-21T06:04:31.774595+0000","last_unstale":"2024-09-21T06:04:34.230583+0000","last_undegraded":"2024-09-21T06:04:34.230583+0000","last_fullsized":"2024-09-21T06:04:34.230583+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:05.289920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.773514+0000","last_change":"2024-09-21T06:04:32.766158+0000","last_active":"2024-09-21T06:04:33.773514+0000","last_peered":"2024-09-21T06:04:33.773514+0000","last_clean":"2024-09-21T06:04:33.773514+0000","last_became_active":"2024-09-21T06:04:32.765811+0000","last_became_peered":"2024-09-21T06:04:32.765811+0000","last_unstale":"2024-09-21T06:04:33.773514+0000","last_undegraded":"2024-09-21T06:04:33.773514+0000","last_fullsized":"2024-09-21T06:04:33.773514+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:11.805943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"70'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.773745+0000","last_change":"2024-09-21T06:04:31.775745+0000","last_active":"2024-09-21T06:04:33.773745+0000","last_peered":"2024-09-21T06:04:33.773745+0000","last_clean":"2024-09-21T06:04:33.773745+0000","last_became_active":"2024-09-21T06:04:31.775037+0000","last_became_peered":"2024-09-21T06:04:31.775037+0000","last_unstale":"2024-09-21T06:04:33.773745+0000","last_undegraded":"2024-09-21T06:04:33.773745+0000","last_fullsized":"2024-09-21T06:04:33.773745+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:40.310390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229659+0000","last_change":"2024-09-21T06:04:32.791087+0000","last_active":"2024-09-21T06:04:34.229659+0000","last_peered":"2024-09-21T06:04:34.229659+0000","last_clean":"2024-09-21T06:04:34.229659+0000","last_became_active":"2024-09-21T06:04:32.790735+0000","last_became_peered":"2024-09-21T06:04:32.790735+0000","last_unstale":"2024-09-21T06:04:34.229659+0000","last_undegraded":"2024-09-21T06:04:34.229659+0000","last_fullsized":"2024-09-21T06:04:34.229659+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:51.415459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229719+0000","last_change":"2024-09-21T06:04:31.761815+0000","last_active":"2024-09-21T06:04:34.229719+0000","last_peered":"2024-09-21T06:04:34.229719+0000","last_clean":"2024-09-21T06:04:34.229719+0000","last_became_active":"2024-09-21T06:04:31.761487+0000","last_became_peered":"2024-09-21T06:04:31.761487+0000","last_unstale":"2024-09-21T06:04:34.229719+0000","last_undegraded":"2024-09-21T06:04:34.229719+0000","last_fullsized":"2024-09-21T06:04:34.229719+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:02:35.169801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.230116+0000","last_change":"2024-09-21T06:04:32.788888+0000","last_active":"2024-09-21T06:04:34.230116+0000","last_peered":"2024-09-21T06:04:34.230116+0000","last_clean":"2024-09-21T06:04:34.230116+0000","last_became_active":"2024-09-21T06:04:32.788597+0000","last_became_peered":"2024-09-21T06:04:32.788597+0000","last_unstale":"2024-09-21T06:04:34.230116+0000","last_undegraded":"2024-09-21T06:04:34.230116+0000","last_fullsized":"2024-09-21T06:04:34.230116+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:31.531894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.771108+0000","last_change":"2024-09-21T06:04:31.762529+0000","last_active":"2024-09-21T06:04:33.771108+0000","last_peered":"2024-09-21T06:04:33.771108+0000","last_clean":"2024-09-21T06:04:33.771108+0000","last_became_active":"2024-09-21T06:04:31.762189+0000","last_became_peered":"2024-09-21T06:04:31.762189+0000","last_unstale":"2024-09-21T06:04:33.771108+0000","last_undegraded":"2024-09-21T06:04:33.771108+0000","last_fullsized":"2024-09-21T06:04:33.771108+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:51.822586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229663+0000","last_change":"2024-09-21T06:04:32.770727+0000","last_active":"2024-09-21T06:04:34.229663+0000","last_peered":"2024-09-21T06:04:34.229663+0000","last_clean":"2024-09-21T06:04:34.229663+0000","last_became_active":"2024-09-21T06:04:32.770393+0000","last_became_peered":"2024-09-21T06:04:32.770393+0000","last_unstale":"2024-09-21T06:04:34.229663+0000","last_undegraded":"2024-09-21T06:04:34.229663+0000","last_fullsized":"2024-09-21T06:04:34.229663+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:10.950332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229132+0000","last_change":"2024-09-21T06:04:31.768859+0000","last_active":"2024-09-21T06:04:34.229132+0000","last_peered":"2024-09-21T06:04:34.229132+0000","last_clean":"2024-09-21T06:04:34.229132+0000","last_became_active":"2024-09-21T06:04:31.768535+0000","last_became_peered":"2024-09-21T06:04:31.768535+0000","last_unstale":"2024-09-21T06:04:34.229132+0000","last_undegraded":"2024-09-21T06:04:34.229132+0000","last_fullsized":"2024-09-21T06:04:34.229132+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:05.572103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229461+0000","last_change":"2024-09-21T06:04:32.782856+0000","last_active":"2024-09-21T06:04:34.229461+0000","last_peered":"2024-09-21T06:04:34.229461+0000","last_clean":"2024-09-21T06:04:34.229461+0000","last_became_active":"2024-09-21T06:04:32.782532+0000","last_became_peered":"2024-09-21T06:04:32.782532+0000","last_unstale":"2024-09-21T06:04:34.229461+0000","last_undegraded":"2024-09-21T06:04:34.229461+0000","last_fullsized":"2024-09-21T06:04:34.229461+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:11.518104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229890+0000","last_change":"2024-09-21T06:04:31.768251+0000","last_active":"2024-09-21T06:04:34.229890+0000","last_peered":"2024-09-21T06:04:34.229890+0000","last_clean":"2024-09-21T06:04:34.229890+0000","last_became_active":"2024-09-21T06:04:31.767918+0000","last_became_peered":"2024-09-21T06:04:31.767918+0000","last_unstale":"2024-09-21T06:04:34.229890+0000","last_undegraded":"2024-09-21T06:04:34.229890+0000","last_fullsized":"2024-09-21T06:04:34.229890+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:57.751518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229404+0000","last_change":"2024-09-21T06:04:32.769572+0000","last_active":"2024-09-21T06:04:34.229404+0000","last_peered":"2024-09-21T06:04:34.229404+0000","last_clean":"2024-09-21T06:04:34.229404+0000","last_became_active":"2024-09-21T06:04:32.769178+0000","last_became_peered":"2024-09-21T06:04:32.769178+0000","last_unstale":"2024-09-21T06:04:34.229404+0000","last_undegraded":"2024-09-21T06:04:34.229404+0000","last_fullsized":"2024-09-21T06:04:34.229404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:57.347204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:33.771752+0000","last_change":"2024-09-21T06:04:31.769840+0000","last_active":"2024-09-21T06:04:33.771752+0000","last_peered":"2024-09-21T06:04:33.771752+0000","last_clean":"2024-09-21T06:04:33.771752+0000","last_became_active":"2024-09-21T06:04:31.769534+0000","last_became_peered":"2024-09-21T06:04:31.769534+0000","last_unstale":"2024-09-21T06:04:33.771752+0000","last_undegraded":"2024-09-21T06:04:33.771752+0000","last_fullsized":"2024-09-21T06:04:33.771752+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:10.542640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228843+0000","last_change":"2024-09-21T06:04:32.779505+0000","last_active":"2024-09-21T06:04:34.228843+0000","last_peered":"2024-09-21T06:04:34.228843+0000","last_clean":"2024-09-21T06:04:34.228843+0000","last_became_active":"2024-09-21T06:04:32.779079+0000","last_became_peered":"2024-09-21T06:04:32.779079+0000","last_unstale":"2024-09-21T06:04:34.228843+0000","last_undegraded":"2024-09-21T06:04:34.228843+0000","last_fullsized":"2024-09-21T06:04:34.228843+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:24.044640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.229284+0000","last_change":"2024-09-21T06:04:31.768166+0000","last_active":"2024-09-21T06:04:34.229284+0000","last_peered":"2024-09-21T06:04:34.229284+0000","last_clean":"2024-09-21T06:04:34.229284+0000","last_became_active":"2024-09-21T06:04:31.767622+0000","last_became_peered":"2024-09-21T06:04:31.767622+0000","last_unstale":"2024-09-21T06:04:34.229284+0000","last_undegraded":"2024-09-21T06:04:34.229284+0000","last_fullsized":"2024-09-21T06:04:34.229284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:14.758437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228647+0000","last_change":"2024-09-21T06:04:32.776172+0000","last_active":"2024-09-21T06:04:34.228647+0000","last_peered":"2024-09-21T06:04:34.228647+0000","last_clean":"2024-09-21T06:04:34.228647+0000","last_became_active":"2024-09-21T06:04:32.775877+0000","last_became_peered":"2024-09-21T06:04:32.775877+0000","last_unstale":"2024-09-21T06:04:34.228647+0000","last_undegraded":"2024-09-21T06:04:34.228647+0000","last_fullsized":"2024-09-21T06:04:34.228647+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:31.738639+0000","last_clean_scrub_stamp":"2024-09-21T06:04:31.738639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:47.071247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228644+0000","last_change":"2024-09-21T06:04:31.772282+0000","last_active":"2024-09-21T06:04:34.228644+0000","last_peered":"2024-09-21T06:04:34.228644+0000","last_clean":"2024-09-21T06:04:34.228644+0000","last_became_active":"2024-09-21T06:04:31.770583+0000","last_became_peered":"2024-09-21T06:04:31.770583+0000","last_unstale":"2024-09-21T06:04:34.228644+0000","last_undegraded":"2024-09-21T06:04:34.228644+0000","last_fullsized":"2024-09-21T06:04:34.228644+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:04:30.735999+0000","last_clean_scrub_stamp":"2024-09-21T06:04:30.735999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:35.915894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":203,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T06:04:34.228454+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:04:34.228454+0000","last_peered":"2024-09-21T06:04:34.228454+0000","last_clean":"2024-09-21T06:04:34.228454+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:04:34.228454+0000","last_undegraded":"2024-09-21T06:04:34.228454+0000","last_fullsized":"2024-09-21T06:04:34.228454+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509543,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50660,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321180,"statfs":{"total":96636764160,"available":96584888320,"internally_reserved":0,"allocated":1458176,"data_stored":951127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509543,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50080,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321760,"statfs":{"total":96636764160,"available":96585482240,"internally_reserved":0,"allocated":864256,"data_stored":360235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509542,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50076,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321764,"statfs":{"total":96636764160,"available":96585486336,"internally_reserved":0,"allocated":860160,"data_stored":360233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509543,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50080,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321760,"statfs":{"total":96636764160,"available":96585482240,"internally_reserved":0,"allocated":860160,"data_stored":360213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54248,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317592,"statfs":{"total":96636764160,"available":96581214208,"internally_reserved":0,"allocated":864256,"data_stored":360255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50084,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321756,"statfs":{"total":96636764160,"available":96585478144,"internally_reserved":0,"allocated":868352,"data_stored":361755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509542,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54820,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317020,"statfs":{"total":96636764160,"available":96580628480,"internally_reserved":0,"allocated":1449984,"data_stored":950511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542247,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58420,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313420,"statfs":{"total":96636764160,"available":96576942080,"internally_reserved":0,"allocated":876544,"data_stored":362371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:04:42.048 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:04:42.048 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:04:42.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.250+0000 7f8f51000640 1 -- 172.21.15.50:0/1991980477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f30004810 msgr2=0x7f8f30004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:42.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.250+0000 7f8f51000640 1 --2- 172.21.15.50:0/1991980477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f30004810 0x7f8f30004c20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0b33f0 tx=0x7f8f3401cb90 comp rx=0 tx=0).stop 2024-09-21T06:04:42.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.250+0000 7f8f4a800640 1 -- 172.21.15.50:0/1991980477 reap_dead start 2024-09-21T06:04:42.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.250+0000 7f8f51000640 1 -- 172.21.15.50:0/1991980477 shutdown_connections 2024-09-21T06:04:42.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.250+0000 7f8f51000640 1 -- 172.21.15.50:0/1991980477 >> 172.21.15.50:0/1991980477 conn(0x7f8f4c0b2040 msgr2=0x7f8f4c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:42.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.250+0000 7f8f51000640 1 -- 172.21.15.50:0/1991980477 shutdown_connections 2024-09-21T06:04:42.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.250+0000 7f8f51000640 1 -- 172.21.15.50:0/1991980477 wait complete. 2024-09-21T06:04:42.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f51000640 1 Processor -- start 2024-09-21T06:04:42.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f51000640 1 -- start start 2024-09-21T06:04:42.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f51000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f30004810 0x7f8f4c150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:42.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f51000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f4c1f6830 0x7f8f4c159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:42.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f51000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4c1f9e20 0x7f8f4c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:42.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f51000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c167bc0 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f51000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c167a40 con 0x7f8f4c1f6830 2024-09-21T06:04:42.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f51000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8f4c167d40 con 0x7f8f30004810 2024-09-21T06:04:42.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f4a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4c1f9e20 0x7f8f4c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:42.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f4a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4c1f9e20 0x7f8f4c159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36714/0 (socket says 172.21.15.50:36714) 2024-09-21T06:04:42.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f4a800640 1 -- 172.21.15.50:0/3331448216 learned_addr learned my addr 172.21.15.50:0/3331448216 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:42.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f43e00640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f4c1f6830 0x7f8f4c159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:42.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f49e00640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f30004810 0x7f8f4c150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:42.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f4a800640 1 -- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f30004810 msgr2=0x7f8f4c150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:42.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f4a800640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f30004810 0x7f8f4c150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f4a800640 1 -- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f4c1f6830 msgr2=0x7f8f4c159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:42.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f4a800640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f4c1f6830 0x7f8f4c159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f4a800640 1 -- 172.21.15.50:0/3331448216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4c1f7010 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f4a800640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4c1f9e20 0x7f8f4c159c60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c156170 tx=0x7f8f44013d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:42.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f42000640 1 -- 172.21.15.50:0/3331448216 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4401c440 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f42000640 1 -- 172.21.15.50:0/3331448216 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f4401c5e0 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f42000640 1 -- 172.21.15.50:0/3331448216 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4401ca30 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f51000640 1 -- 172.21.15.50:0/3331448216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4c1f72f0 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.254+0000 7f8f51000640 1 -- 172.21.15.50:0/3331448216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4c1f7bb0 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.258+0000 7f8f51000640 1 -- 172.21.15.50:0/3331448216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f4c1f6d80 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.258+0000 7f8f42000640 1 -- 172.21.15.50:0/3331448216 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f8f440253f0 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.258+0000 7f8f42000640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f28073e60 0x7f8f28076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:42.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.258+0000 7f8f42000640 1 -- 172.21.15.50:0/3331448216 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f8f44097500 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.258+0000 7f8f49e00640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f28073e60 0x7f8f28076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:42.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.262+0000 7f8f49e00640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f28073e60 0x7f8f28076370 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c000c10 tx=0x7f8f3c006b80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:42.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.262+0000 7f8f42000640 1 -- 172.21.15.50:0/3331448216 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8f440639b0 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.506+0000 7f8f51000640 1 -- 172.21.15.50:0/3331448216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f8f4c0b51f0 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f42000640 1 -- 172.21.15.50:0/3331448216 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v33) ==== 190+0+147 (secure 0 0 0) 0x7f8f440683e0 con 0x7f8f4c1f9e20 2024-09-21T06:04:42.511 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-09-21T06:04:42.511 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw== 2024-09-21T06:04:42.512 INFO:teuthology.orchestra.run.smithi050.stdout: caps mds = "allow" 2024-09-21T06:04:42.512 INFO:teuthology.orchestra.run.smithi050.stdout: caps mon = "allow r" 2024-09-21T06:04:42.512 INFO:teuthology.orchestra.run.smithi050.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:04:42.512 INFO:teuthology.orchestra.run.smithi050.stderr:updated caps for client.0 2024-09-21T06:04:42.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f51000640 1 -- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f28073e60 msgr2=0x7f8f28076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:42.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f51000640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f28073e60 0x7f8f28076370 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c000c10 tx=0x7f8f3c006b80 comp rx=0 tx=0).stop 2024-09-21T06:04:42.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f51000640 1 -- 172.21.15.50:0/3331448216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4c1f9e20 msgr2=0x7f8f4c159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:42.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f51000640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4c1f9e20 0x7f8f4c159c60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c156170 tx=0x7f8f44013d60 comp rx=0 tx=0).stop 2024-09-21T06:04:42.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f51000640 1 -- 172.21.15.50:0/3331448216 shutdown_connections 2024-09-21T06:04:42.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f51000640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f28073e60 0x7f8f28076370 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f51000640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4c1f9e20 0x7f8f4c159c60 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f51000640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f4c1f6830 0x7f8f4c159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f51000640 1 --2- 172.21.15.50:0/3331448216 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f30004810 0x7f8f4c150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f51000640 1 -- 172.21.15.50:0/3331448216 >> 172.21.15.50:0/3331448216 conn(0x7f8f4c0b2040 msgr2=0x7f8f4c15a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:42.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f51000640 1 -- 172.21.15.50:0/3331448216 shutdown_connections 2024-09-21T06:04:42.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.510+0000 7f8f51000640 1 -- 172.21.15.50:0/3331448216 wait complete. 2024-09-21T06:04:42.535 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:04:42.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.690+0000 7fa41e600640 1 -- 172.21.15.50:0/785216633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa418160060 msgr2=0x7fa418074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:42.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.690+0000 7fa41e600640 1 --2- 172.21.15.50:0/785216633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa418160060 0x7fa418074240 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa404009900 tx=0x7fa40401c850 comp rx=0 tx=0).stop 2024-09-21T06:04:42.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.690+0000 7fa41e600640 1 -- 172.21.15.50:0/785216633 shutdown_connections 2024-09-21T06:04:42.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.690+0000 7fa41e600640 1 --2- 172.21.15.50:0/785216633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa418160060 0x7fa418074240 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.690+0000 7fa41e600640 1 --2- 172.21.15.50:0/785216633 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4181536f0 0x7fa41815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.690+0000 7fa41e600640 1 -- 172.21.15.50:0/785216633 >> 172.21.15.50:0/785216633 conn(0x7fa4180b4560 msgr2=0x7fa4180d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:42.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.690+0000 7fa41e600640 1 -- 172.21.15.50:0/785216633 shutdown_connections 2024-09-21T06:04:42.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.690+0000 7fa41e600640 1 -- 172.21.15.50:0/785216633 wait complete. 2024-09-21T06:04:42.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa41e600640 1 Processor -- start 2024-09-21T06:04:42.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa41e600640 1 -- start start 2024-09-21T06:04:42.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa41e600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4181536f0 0x7fa418155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:42.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa41e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa418160060 0x7fa418151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:42.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa41e600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa418160470 0x7fa4181f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:42.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa41e600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa41816a490 con 0x7fa418160060 2024-09-21T06:04:42.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa41e600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa41816a310 con 0x7fa4181536f0 2024-09-21T06:04:42.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa41e600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa41816a610 con 0x7fa418160470 2024-09-21T06:04:42.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa417e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa418160060 0x7fa418151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:42.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa417e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa418160060 0x7fa418151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36732/0 (socket says 172.21.15.50:36732) 2024-09-21T06:04:42.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa417e00640 1 -- 172.21.15.50:0/2465300438 learned_addr learned my addr 172.21.15.50:0/2465300438 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:42.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa417e00640 1 -- 172.21.15.50:0/2465300438 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa418160470 msgr2=0x7fa4181f99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:42.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa417e00640 1 --2- 172.21.15.50:0/2465300438 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa418160470 0x7fa4181f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa417e00640 1 -- 172.21.15.50:0/2465300438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4181536f0 msgr2=0x7fa418155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:42.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa417e00640 1 --2- 172.21.15.50:0/2465300438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4181536f0 0x7fa418155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa417e00640 1 -- 172.21.15.50:0/2465300438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa418152690 con 0x7fa418160060 2024-09-21T06:04:42.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa417e00640 1 --2- 172.21.15.50:0/2465300438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa418160060 0x7fa418151cf0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa404009850 tx=0x7fa4040a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:42.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa414c00640 1 -- 172.21.15.50:0/2465300438 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040ba020 con 0x7fa418160060 2024-09-21T06:04:42.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa414c00640 1 -- 172.21.15.50:0/2465300438 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa404004430 con 0x7fa418160060 2024-09-21T06:04:42.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa414c00640 1 -- 172.21.15.50:0/2465300438 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040b2040 con 0x7fa418160060 2024-09-21T06:04:42.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.694+0000 7fa41e600640 1 -- 172.21.15.50:0/2465300438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4181fcf60 con 0x7fa418160060 2024-09-21T06:04:42.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.698+0000 7fa41e600640 1 -- 172.21.15.50:0/2465300438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4181fd520 con 0x7fa418160060 2024-09-21T06:04:42.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.698+0000 7fa41e600640 1 -- 172.21.15.50:0/2465300438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa41815d480 con 0x7fa418160060 2024-09-21T06:04:42.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.698+0000 7fa414c00640 1 -- 172.21.15.50:0/2465300438 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7fa4040bd070 con 0x7fa418160060 2024-09-21T06:04:42.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.698+0000 7fa414c00640 1 --2- 172.21.15.50:0/2465300438 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa3f8073e60 0x7fa3f8076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:42.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.702+0000 7fa414c00640 1 -- 172.21.15.50:0/2465300438 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7fa40412c6b0 con 0x7fa418160060 2024-09-21T06:04:42.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.702+0000 7fa41d200640 1 --2- 172.21.15.50:0/2465300438 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa3f8073e60 0x7fa3f8076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:42.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.702+0000 7fa41d200640 1 --2- 172.21.15.50:0/2465300438 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa3f8073e60 0x7fa3f8076370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009690 tx=0x7fa40c006c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:42.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.706+0000 7fa414c00640 1 -- 172.21.15.50:0/2465300438 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa418152690 con 0x7fa418160060 2024-09-21T06:04:42.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.918+0000 7fa41e600640 1 -- 172.21.15.50:0/2465300438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4180b5470 con 0x7fa418160060 2024-09-21T06:04:42.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.918+0000 7fa414c00640 1 -- 172.21.15.50:0/2465300438 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5222 (secure 0 0 0) 0x7fa4040164d0 con 0x7fa418160060 2024-09-21T06:04:42.920 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:04:42.920 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":45,"btime":"2024-09-21T06:04:35:777183+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/2233580616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2233580616},{"type":"v1","addr":"172.21.15.50:6833","nonce":2233580616}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3749582023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3749582023},{"type":"v1","addr":"172.21.15.50:6835","nonce":3749582023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6139,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/382971646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":382971646},{"type":"v1","addr":"172.21.15.88:6836","nonce":382971646}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:04:32.178456+0000","modified":"2024-09-21T06:04:35.197824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6423":{"gid":6423,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/2131576448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2131576448},{"type":"v1","addr":"172.21.15.88:6837","nonce":2131576448}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6423,"qdb_cluster":[6423]},"id":5}]} 2024-09-21T06:04:42.920 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 45 2024-09-21T06:04:42.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.922+0000 7fa41e600640 1 -- 172.21.15.50:0/2465300438 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa3f8073e60 msgr2=0x7fa3f8076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:42.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.922+0000 7fa41e600640 1 --2- 172.21.15.50:0/2465300438 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa3f8073e60 0x7fa3f8076370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009690 tx=0x7fa40c006c00 comp rx=0 tx=0).stop 2024-09-21T06:04:42.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.922+0000 7fa41e600640 1 -- 172.21.15.50:0/2465300438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa418160060 msgr2=0x7fa418151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:42.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.922+0000 7fa41e600640 1 --2- 172.21.15.50:0/2465300438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa418160060 0x7fa418151cf0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa404009850 tx=0x7fa4040a5620 comp rx=0 tx=0).stop 2024-09-21T06:04:42.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.922+0000 7fa41e600640 1 -- 172.21.15.50:0/2465300438 shutdown_connections 2024-09-21T06:04:42.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.922+0000 7fa41e600640 1 --2- 172.21.15.50:0/2465300438 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa418160470 0x7fa4181f99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.922+0000 7fa41e600640 1 --2- 172.21.15.50:0/2465300438 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa3f8073e60 0x7fa3f8076370 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.922+0000 7fa41e600640 1 --2- 172.21.15.50:0/2465300438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa418160060 0x7fa418151cf0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.922+0000 7fa41e600640 1 --2- 172.21.15.50:0/2465300438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4181536f0 0x7fa418155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:42.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.922+0000 7fa41e600640 1 -- 172.21.15.50:0/2465300438 >> 172.21.15.50:0/2465300438 conn(0x7fa4180b4560 msgr2=0x7fa4180b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:42.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.922+0000 7fa41e600640 1 -- 172.21.15.50:0/2465300438 shutdown_connections 2024-09-21T06:04:42.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:42.922+0000 7fa41e600640 1 -- 172.21.15.50:0/2465300438 wait complete. 2024-09-21T06:04:43.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:04:32.178456+0000', 'modified': '2024-09-21T06:04:35.197824+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6423}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6423': {'gid': 6423, 'name': 'd', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6837/2131576448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6835', 'nonce': 2131576448}, {'type': 'v1', 'addr': '172.21.15.88:6837', 'nonce': 2131576448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6423, 'qdb_cluster': [6423]} 2024-09-21T06:04:43.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:04:43.346 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T06:04:43.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.506+0000 7eff33800640 1 -- 172.21.15.50:0/843455232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff2c074950 msgr2=0x7eff2c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.506+0000 7eff33800640 1 --2- 172.21.15.50:0/843455232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff2c074950 0x7eff2c15cf90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7eff24009bc0 tx=0x7eff2401cba0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.506+0000 7eff33800640 1 -- 172.21.15.50:0/843455232 shutdown_connections 2024-09-21T06:04:43.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.506+0000 7eff33800640 1 --2- 172.21.15.50:0/843455232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff2c074950 0x7eff2c15cf90 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.506+0000 7eff33800640 1 --2- 172.21.15.50:0/843455232 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7eff2c0b5a80 0x7eff2c074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.506+0000 7eff33800640 1 -- 172.21.15.50:0/843455232 >> 172.21.15.50:0/843455232 conn(0x7eff2c0708c0 msgr2=0x7eff2c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:43.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.506+0000 7eff33800640 1 -- 172.21.15.50:0/843455232 shutdown_connections 2024-09-21T06:04:43.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.506+0000 7eff33800640 1 -- 172.21.15.50:0/843455232 wait complete. 2024-09-21T06:04:43.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.510+0000 7eff33800640 1 Processor -- start 2024-09-21T06:04:43.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.510+0000 7eff33800640 1 -- start start 2024-09-21T06:04:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.510+0000 7eff33800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7eff2c074950 0x7eff2c147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.510+0000 7eff33800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff2c0b5a80 0x7eff2c151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.510+0000 7eff33800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff2c157030 0x7eff2c151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.510+0000 7eff33800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c16a360 con 0x7eff2c0b5a80 2024-09-21T06:04:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.510+0000 7eff33800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c16a1e0 con 0x7eff2c157030 2024-09-21T06:04:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.510+0000 7eff33800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c16a4e0 con 0x7eff2c074950 2024-09-21T06:04:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff2c0b5a80 0x7eff2c151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff2c0b5a80 0x7eff2c151770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36750/0 (socket says 172.21.15.50:36750) 2024-09-21T06:04:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff2be00640 1 -- 172.21.15.50:0/1763590580 learned_addr learned my addr 172.21.15.50:0/1763590580 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff30e00640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7eff2c074950 0x7eff2c147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff31800640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff2c157030 0x7eff2c151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff2be00640 1 -- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7eff2c074950 msgr2=0x7eff2c147d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff2be00640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7eff2c074950 0x7eff2c147d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff2be00640 1 -- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff2c157030 msgr2=0x7eff2c151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff2be00640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff2c157030 0x7eff2c151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff2be00640 1 -- 172.21.15.50:0/1763590580 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c148250 con 0x7eff2c0b5a80 2024-09-21T06:04:43.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff2be00640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff2c0b5a80 0x7eff2c151770 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7eff1800c970 tx=0x7eff1800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:43.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff28c00640 1 -- 172.21.15.50:0/1763590580 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18014070 con 0x7eff2c0b5a80 2024-09-21T06:04:43.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff28c00640 1 -- 172.21.15.50:0/1763590580 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff180077a0 con 0x7eff2c0b5a80 2024-09-21T06:04:43.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff28c00640 1 -- 172.21.15.50:0/1763590580 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18007bf0 con 0x7eff2c0b5a80 2024-09-21T06:04:43.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff33800640 1 -- 172.21.15.50:0/1763590580 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c15cb00 con 0x7eff2c0b5a80 2024-09-21T06:04:43.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff33800640 1 -- 172.21.15.50:0/1763590580 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c15c540 con 0x7eff2c0b5a80 2024-09-21T06:04:43.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.514+0000 7eff28c00640 1 -- 172.21.15.50:0/1763590580 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7eff18017070 con 0x7eff2c0b5a80 2024-09-21T06:04:43.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff28c00640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7eff0c073e60 0x7eff0c076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff28c00640 1 -- 172.21.15.50:0/1763590580 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7eff18092ef0 con 0x7eff2c0b5a80 2024-09-21T06:04:43.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff30e00640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7eff0c073e60 0x7eff0c076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff3af2a000 1 Processor -- start 2024-09-21T06:04:43.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff3af2a000 1 -- start start 2024-09-21T06:04:43.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff30e00640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7eff0c073e60 0x7eff0c076370 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7eff20007fd0 tx=0x7eff20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:43.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff3af2a000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec016a460 0x55bec06ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff3af2a000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec055a950 0x55bec06abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff3af2a000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec0543ca0 0x55bec06ac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff3af2a000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bec0580410 con 0x55bec055a950 2024-09-21T06:04:43.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff3af2a000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bec04bdb40 con 0x55bec0543ca0 2024-09-21T06:04:43.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff3af2a000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55bec04acf70 con 0x55bec016a460 2024-09-21T06:04:43.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff30e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec016a460 0x55bec06ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff30e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec016a460 0x55bec06ab6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:35768/0 (socket says 172.21.15.50:35768) 2024-09-21T06:04:43.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff30e00640 1 -- 172.21.15.50:0/4143707918 learned_addr learned my addr 172.21.15.50:0/4143707918 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:43.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff2be00640 1 --2- 172.21.15.50:0/4143707918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec055a950 0x55bec06abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff31800640 1 --2- 172.21.15.50:0/4143707918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec0543ca0 0x55bec06ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff30e00640 1 -- 172.21.15.50:0/4143707918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec0543ca0 msgr2=0x55bec06ac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:43.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff30e00640 1 --2- 172.21.15.50:0/4143707918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec0543ca0 0x55bec06ac120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff30e00640 1 -- 172.21.15.50:0/4143707918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec055a950 msgr2=0x55bec06abbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:43.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff30e00640 1 --2- 172.21.15.50:0/4143707918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec055a950 0x55bec06abbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff30e00640 1 -- 172.21.15.50:0/4143707918 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bec04745b0 con 0x55bec016a460 2024-09-21T06:04:43.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.518+0000 7eff2be00640 1 --2- 172.21.15.50:0/4143707918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec055a950 0x55bec06abbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:43.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff30e00640 1 --2- 172.21.15.50:0/4143707918 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec016a460 0x55bec06ab6a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7eff2000fb40 tx=0x7eff20012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab883408c8022118 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:43.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff1d600640 1 -- 172.21.15.50:0/4143707918 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20025070 con 0x55bec016a460 2024-09-21T06:04:43.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff1d600640 1 -- 172.21.15.50:0/4143707918 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff2001b050 con 0x55bec016a460 2024-09-21T06:04:43.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff1d600640 1 -- 172.21.15.50:0/4143707918 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20014e40 con 0x55bec016a460 2024-09-21T06:04:43.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff3af2a000 1 -- 172.21.15.50:0/4143707918 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec016a460 msgr2=0x55bec06ab6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff3af2a000 1 --2- 172.21.15.50:0/4143707918 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec016a460 0x55bec06ab6a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7eff2000fb40 tx=0x7eff20012000 comp rx=0 tx=0).stop 2024-09-21T06:04:43.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff3af2a000 1 -- 172.21.15.50:0/4143707918 shutdown_connections 2024-09-21T06:04:43.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff3af2a000 1 --2- 172.21.15.50:0/4143707918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec0543ca0 0x55bec06ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff3af2a000 1 --2- 172.21.15.50:0/4143707918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec055a950 0x55bec06abbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff3af2a000 1 --2- 172.21.15.50:0/4143707918 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec016a460 0x55bec06ab6a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff3af2a000 1 -- 172.21.15.50:0/4143707918 >> 172.21.15.50:0/4143707918 conn(0x55bec0630d90 msgr2=0x55bec0660690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:43.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff3af2a000 1 -- 172.21.15.50:0/4143707918 shutdown_connections 2024-09-21T06:04:43.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff3af2a000 1 -- 172.21.15.50:0/4143707918 wait complete. 2024-09-21T06:04:43.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff3af2a000 1 -- start start 2024-09-21T06:04:43.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.522+0000 7eff3af2a000 1 Processor -- start 2024-09-21T06:04:43.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec016a460 0x55bec06b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec06b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec055a950 0x55bec06b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bec04475f0 con 0x55bec0543ca0 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bec04b0d80 con 0x55bec055a950 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55bec0453520 con 0x55bec016a460 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 10 client.0 _ll_register_callbacks cb 0x55bec0457830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec06b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec06b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36766/0 (socket says 172.21.15.50:36766) 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff2be00640 1 -- 172.21.15.50:0/800614487 learned_addr learned my addr 172.21.15.50:0/800614487 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff30e00640 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec016a460 0x55bec06b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff31800640 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec055a950 0x55bec06b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff2be00640 1 -- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec016a460 msgr2=0x55bec06b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:43.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff2be00640 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec016a460 0x55bec06b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff2be00640 1 -- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec055a950 msgr2=0x55bec06b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff2be00640 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec055a950 0x55bec06b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff2be00640 1 -- 172.21.15.50:0/800614487 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bec044b190 con 0x55bec0543ca0 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff31800640 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec055a950 0x55bec06b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff2be00640 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec06b2e60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7eff18007590 tx=0x7eff1805f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61183286f2db1a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff12000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff12000640 1 -- 172.21.15.50:0/800614487 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1804c070 con 0x55bec0543ca0 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff12000640 1 -- 172.21.15.50:0/800614487 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7eff1805dfe0 con 0x55bec0543ca0 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 1 -- 172.21.15.50:0/800614487 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bec0445110 con 0x55bec0543ca0 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff12000640 1 -- 172.21.15.50:0/800614487 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7eff1807e050 con 0x55bec0543ca0 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 10 client.6576 fetch_fsmap learned FSMap version 45 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 1 -- 172.21.15.50:0/800614487 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55bec0531f10 con 0x55bec0543ca0 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff12000640 1 -- 172.21.15.50:0/800614487 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55bec0445110 con 0x55bec0543ca0 2024-09-21T06:04:43.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff12000640 1 -- 172.21.15.50:0/800614487 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7eff1805e1e0 con 0x55bec0543ca0 2024-09-21T06:04:43.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 10 client.6576 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T06:04:43.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 10 client.6576 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:04:43.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 20 client.6576 populate_metadata read hostname 'smithi050' 2024-09-21T06:04:43.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:6835/2131576448,v1:172.21.15.88:6837/2131576448] conn(0x55bec014f7d0 0x55bec06bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 4 client.6576 mds_command: new command op to 6423 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:43.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff3af2a000 1 -- 172.21.15.50:0/800614487 --> [v2:172.21.15.88:6835/2131576448,v1:172.21.15.88:6837/2131576448] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bec04bfa90 con 0x55bec014f7d0 2024-09-21T06:04:43.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff30e00640 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:6835/2131576448,v1:172.21.15.88:6837/2131576448] conn(0x55bec014f7d0 0x55bec06bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff30e00640 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:6835/2131576448,v1:172.21.15.88:6837/2131576448] conn(0x55bec014f7d0 0x55bec06bcd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6892edc66a270264 server_cookie=6bd5a2d37e15027e in_seq=0 out_seq=0 2024-09-21T06:04:43.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.526+0000 7eff12000640 10 client.6576 ms_handle_connect on v2:172.21.15.88:6835/2131576448 2024-09-21T06:04:43.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff12000640 1 -- 172.21.15.50:0/800614487 <== mds.0 v2:172.21.15.88:6835/2131576448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bec04bfa90 con 0x55bec014f7d0 2024-09-21T06:04:43.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff12000640 10 client.6576 handle_command_reply: tid=0 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff3af2a000 1 client.6576 shutdown 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff3af2a000 1 -- 172.21.15.50:0/800614487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 msgr2=0x55bec06b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff3af2a000 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec06b2e60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7eff18007590 tx=0x7eff1805f860 comp rx=0 tx=0).stop 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff3af2a000 1 -- 172.21.15.50:0/800614487 shutdown_connections 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff3af2a000 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:6835/2131576448,v1:172.21.15.88:6837/2131576448] conn(0x55bec014f7d0 0x55bec06bcd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff3af2a000 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec055a950 0x55bec06b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff3af2a000 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec06b2e60 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff3af2a000 1 --2- 172.21.15.50:0/800614487 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec016a460 0x55bec06b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff3af2a000 1 -- 172.21.15.50:0/800614487 >> 172.21.15.50:0/800614487 conn(0x55bec0630d90 msgr2=0x55bec0667440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff12000640 0 client.6576 ms_handle_reset on v2:172.21.15.88:6835/2131576448 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff3af2a000 1 -- 172.21.15.50:0/800614487 shutdown_connections 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff3af2a000 1 -- 172.21.15.50:0/800614487 wait complete. 2024-09-21T06:04:43.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.530+0000 7eff3af2a000 20 client.6576 trim_cache size 0 max 16384 2024-09-21T06:04:43.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff3af2a000 1 Processor -- start 2024-09-21T06:04:43.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff3af2a000 1 -- start start 2024-09-21T06:04:43.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff3af2a000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec016a460 0x55bec071d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff3af2a000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec071db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff3af2a000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec055a950 0x55bec071e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff3af2a000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c205880 con 0x55bec0543ca0 2024-09-21T06:04:43.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff3af2a000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c06e420 con 0x55bec016a460 2024-09-21T06:04:43.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff3af2a000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c06d780 con 0x55bec055a950 2024-09-21T06:04:43.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec071db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec071db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36768/0 (socket says 172.21.15.50:36768) 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff2be00640 1 -- 172.21.15.50:0/3599710741 learned_addr learned my addr 172.21.15.50:0/3599710741 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff30e00640 1 --2- 172.21.15.50:0/3599710741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec016a460 0x55bec071d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff31800640 1 --2- 172.21.15.50:0/3599710741 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec055a950 0x55bec071e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff2be00640 1 -- 172.21.15.50:0/3599710741 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec055a950 msgr2=0x55bec071e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff2be00640 1 --2- 172.21.15.50:0/3599710741 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec055a950 0x55bec071e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff2be00640 1 -- 172.21.15.50:0/3599710741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec016a460 msgr2=0x55bec071d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff2be00640 1 --2- 172.21.15.50:0/3599710741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec016a460 0x55bec071d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff2be00640 1 -- 172.21.15.50:0/3599710741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bec0508db0 con 0x55bec0543ca0 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff31800640 1 --2- 172.21.15.50:0/3599710741 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec055a950 0x55bec071e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff2be00640 1 --2- 172.21.15.50:0/3599710741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec071db70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7eff1800cf80 tx=0x7eff1805ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3ff3be04386634 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff1cc00640 1 -- 172.21.15.50:0/3599710741 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1804c040 con 0x55bec0543ca0 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff1cc00640 1 -- 172.21.15.50:0/3599710741 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff18068c80 con 0x55bec0543ca0 2024-09-21T06:04:43.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.550+0000 7eff1cc00640 1 -- 172.21.15.50:0/3599710741 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1801e030 con 0x55bec0543ca0 2024-09-21T06:04:43.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 -- 172.21.15.50:0/3599710741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 msgr2=0x55bec071db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 --2- 172.21.15.50:0/3599710741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec071db70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7eff1800cf80 tx=0x7eff1805ec60 comp rx=0 tx=0).stop 2024-09-21T06:04:43.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 -- 172.21.15.50:0/3599710741 shutdown_connections 2024-09-21T06:04:43.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 --2- 172.21.15.50:0/3599710741 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec055a950 0x55bec071e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 --2- 172.21.15.50:0/3599710741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec071db70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 --2- 172.21.15.50:0/3599710741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec016a460 0x55bec071d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 -- 172.21.15.50:0/3599710741 >> 172.21.15.50:0/3599710741 conn(0x55bec0630d90 msgr2=0x55bec065fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:43.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 -- 172.21.15.50:0/3599710741 shutdown_connections 2024-09-21T06:04:43.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 -- 172.21.15.50:0/3599710741 wait complete. 2024-09-21T06:04:43.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 -- start start 2024-09-21T06:04:43.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 Processor -- start 2024-09-21T06:04:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec016a460 0x55bec07213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec0721910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec055a950 0x55bec0721e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bec04bc910 con 0x55bec0543ca0 2024-09-21T06:04:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bec04b53b0 con 0x55bec016a460 2024-09-21T06:04:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55bec056c750 con 0x55bec055a950 2024-09-21T06:04:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff3af2a000 10 client.0 _ll_register_callbacks cb 0x7eff1805ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec0721910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec0721910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36778/0 (socket says 172.21.15.50:36778) 2024-09-21T06:04:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff2be00640 1 -- 172.21.15.50:0/1993180458 learned_addr learned my addr 172.21.15.50:0/1993180458 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff30e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec016a460 0x55bec07213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.554+0000 7eff31800640 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec055a950 0x55bec0721e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff2be00640 1 -- 172.21.15.50:0/1993180458 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec055a950 msgr2=0x55bec0721e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff2be00640 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec055a950 0x55bec0721e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff2be00640 1 -- 172.21.15.50:0/1993180458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec016a460 msgr2=0x55bec07213d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff2be00640 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec016a460 0x55bec07213d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff2be00640 1 -- 172.21.15.50:0/1993180458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bec0473ab0 con 0x55bec0543ca0 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff31800640 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec055a950 0x55bec0721e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff2be00640 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec0721910 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7eff1805f2a0 tx=0x7eff1805f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b4c81d32ab6792 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff12000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff12000640 1 -- 172.21.15.50:0/1993180458 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1801e070 con 0x55bec0543ca0 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff12000640 1 -- 172.21.15.50:0/1993180458 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7eff180650d0 con 0x55bec0543ca0 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 -- 172.21.15.50:0/1993180458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff18068c80 con 0x55bec0543ca0 2024-09-21T06:04:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff12000640 1 -- 172.21.15.50:0/1993180458 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7eff18074050 con 0x55bec0543ca0 2024-09-21T06:04:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 10 client.6582 fetch_fsmap learned FSMap version 45 2024-09-21T06:04:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 -- 172.21.15.50:0/1993180458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55bec0452070 con 0x55bec0543ca0 2024-09-21T06:04:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff12000640 1 -- 172.21.15.50:0/1993180458 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff18068c80 con 0x55bec0543ca0 2024-09-21T06:04:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff12000640 1 -- 172.21.15.50:0/1993180458 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7eff180791b0 con 0x55bec0543ca0 2024-09-21T06:04:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 10 client.6582 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T06:04:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 10 client.6582 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:04:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 20 client.6582 populate_metadata read hostname 'smithi050' 2024-09-21T06:04:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.88:6835/2131576448,v1:172.21.15.88:6837/2131576448] conn(0x55bec013d830 0x55bec06dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 4 client.6582 mds_command: new command op to 6423 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:04:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 -- 172.21.15.50:0/1993180458 --> [v2:172.21.15.88:6835/2131576448,v1:172.21.15.88:6837/2131576448] -- command(tid 0: {"prefix": "status"}) -- 0x55bec04bb540 con 0x55bec013d830 2024-09-21T06:04:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff30e00640 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.88:6835/2131576448,v1:172.21.15.88:6837/2131576448] conn(0x55bec013d830 0x55bec06dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff12000640 10 client.6582 ms_handle_connect on v2:172.21.15.88:6835/2131576448 2024-09-21T06:04:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff30e00640 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.88:6835/2131576448,v1:172.21.15.88:6837/2131576448] conn(0x55bec013d830 0x55bec06dd0b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36e2acab5be267ac server_cookie=364d0b8e1f2fbc0b in_seq=0 out_seq=0 2024-09-21T06:04:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff12000640 1 -- 172.21.15.50:0/1993180458 <== mds.0 v2:172.21.15.88:6835/2131576448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55bec04bb540 con 0x55bec013d830 2024-09-21T06:04:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff12000640 10 client.6582 handle_command_reply: tid=0 2024-09-21T06:04:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 client.6582 shutdown 2024-09-21T06:04:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 -- 172.21.15.50:0/1993180458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 msgr2=0x55bec0721910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec0721910 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7eff1805f2a0 tx=0x7eff1805f490 comp rx=0 tx=0).stop 2024-09-21T06:04:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 -- 172.21.15.50:0/1993180458 shutdown_connections 2024-09-21T06:04:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55bec055a950 0x55bec0721e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bec0543ca0 0x55bec0721910 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.88:6835/2131576448,v1:172.21.15.88:6837/2131576448] conn(0x55bec013d830 0x55bec06dd0b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 --2- 172.21.15.50:0/1993180458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bec016a460 0x55bec07213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff12000640 0 client.6582 ms_handle_reset on v2:172.21.15.88:6835/2131576448 2024-09-21T06:04:43.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 -- 172.21.15.50:0/1993180458 >> 172.21.15.50:0/1993180458 conn(0x55bec0630d90 msgr2=0x55bec0660690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:43.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 -- 172.21.15.50:0/1993180458 shutdown_connections 2024-09-21T06:04:43.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 1 -- 172.21.15.50:0/1993180458 wait complete. 2024-09-21T06:04:43.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff3af2a000 20 client.6582 trim_cache size 0 max 16384 2024-09-21T06:04:43.565 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:04:43.565 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:04:43.565 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-21T06:04:43.565 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 6423, 2024-09-21T06:04:43.565 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-21T06:04:43.565 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-21T06:04:43.565 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-21T06:04:43.565 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 10.814670156, 2024-09-21T06:04:43.566 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 45, 2024-09-21T06:04:43.566 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 71, 2024-09-21T06:04:43.566 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 71, 2024-09-21T06:04:43.566 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 15.224108242 2024-09-21T06:04:43.566 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:04:43.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff33800640 1 -- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7eff0c073e60 msgr2=0x7eff0c076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff33800640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7eff0c073e60 0x7eff0c076370 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7eff20007fd0 tx=0x7eff20009040 comp rx=0 tx=0).stop 2024-09-21T06:04:43.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff33800640 1 -- 172.21.15.50:0/1763590580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff2c0b5a80 msgr2=0x7eff2c151770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.558+0000 7eff33800640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff2c0b5a80 0x7eff2c151770 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7eff1800c970 tx=0x7eff1800ce30 comp rx=0 tx=0).stop 2024-09-21T06:04:43.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.562+0000 7eff33800640 1 -- 172.21.15.50:0/1763590580 shutdown_connections 2024-09-21T06:04:43.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.562+0000 7eff33800640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff2c157030 0x7eff2c151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.562+0000 7eff33800640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7eff0c073e60 0x7eff0c076370 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.562+0000 7eff33800640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff2c0b5a80 0x7eff2c151770 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.562+0000 7eff33800640 1 --2- 172.21.15.50:0/1763590580 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7eff2c074950 0x7eff2c147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.562+0000 7eff33800640 1 -- 172.21.15.50:0/1763590580 >> 172.21.15.50:0/1763590580 conn(0x7eff2c0708c0 msgr2=0x7eff2c15f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:43.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.562+0000 7eff33800640 1 -- 172.21.15.50:0/1763590580 shutdown_connections 2024-09-21T06:04:43.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.562+0000 7eff33800640 1 -- 172.21.15.50:0/1763590580 wait complete. 2024-09-21T06:04:43.582 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:04:43.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.738+0000 7f84ca400640 1 -- 172.21.15.50:0/4286198659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84c40b5c70 msgr2=0x7f84c415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.738+0000 7f84ca400640 1 --2- 172.21.15.50:0/4286198659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84c40b5c70 0x7f84c415bb60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f84c4074b20 tx=0x7f84b401c760 comp rx=0 tx=0).stop 2024-09-21T06:04:43.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.738+0000 7f84ca400640 1 -- 172.21.15.50:0/4286198659 shutdown_connections 2024-09-21T06:04:43.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.738+0000 7f84ca400640 1 --2- 172.21.15.50:0/4286198659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84c40b5c70 0x7f84c415bb60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.738+0000 7f84ca400640 1 -- 172.21.15.50:0/4286198659 >> 172.21.15.50:0/4286198659 conn(0x7f84c40b2040 msgr2=0x7f84c40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:43.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.738+0000 7f84ca400640 1 -- 172.21.15.50:0/4286198659 shutdown_connections 2024-09-21T06:04:43.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.738+0000 7f84ca400640 1 -- 172.21.15.50:0/4286198659 wait complete. 2024-09-21T06:04:43.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84ca400640 1 Processor -- start 2024-09-21T06:04:43.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84ca400640 1 -- start start 2024-09-21T06:04:43.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84ca400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84c40b4de0 0x7f84c4158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84ca400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84c40b5c70 0x7f84c40b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84ca400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84c41527b0 0x7f84c40b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84ca400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f84c4169b30 con 0x7f84c40b4de0 2024-09-21T06:04:43.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84ca400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f84c41699b0 con 0x7f84c41527b0 2024-09-21T06:04:43.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84ca400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f84c4169cb0 con 0x7f84c40b5c70 2024-09-21T06:04:43.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84c3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84c40b4de0 0x7f84c4158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84c3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84c40b4de0 0x7f84c4158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36780/0 (socket says 172.21.15.50:36780) 2024-09-21T06:04:43.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84c3400640 1 -- 172.21.15.50:0/1413796913 learned_addr learned my addr 172.21.15.50:0/1413796913 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:43.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84bbe00640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84c40b5c70 0x7f84c40b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84c3e00640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84c41527b0 0x7f84c40b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84c3400640 1 -- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84c40b5c70 msgr2=0x7f84c40b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84c3400640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84c40b5c70 0x7f84c40b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84c3400640 1 -- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84c41527b0 msgr2=0x7f84c40b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84c3400640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84c41527b0 0x7f84c40b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84c3400640 1 -- 172.21.15.50:0/1413796913 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c41fc140 con 0x7f84c40b4de0 2024-09-21T06:04:43.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.742+0000 7f84c3e00640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84c41527b0 0x7f84c40b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:43.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.746+0000 7f84c3400640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84c40b4de0 0x7f84c4158d20 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f84b401cc40 tx=0x7f84b4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:43.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.746+0000 7f84bb400640 1 -- 172.21.15.50:0/1413796913 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40bb070 con 0x7f84c40b4de0 2024-09-21T06:04:43.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.746+0000 7f84ca400640 1 -- 172.21.15.50:0/1413796913 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c41fc420 con 0x7f84c40b4de0 2024-09-21T06:04:43.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.746+0000 7f84bb400640 1 -- 172.21.15.50:0/1413796913 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84b4004330 con 0x7f84c40b4de0 2024-09-21T06:04:43.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.746+0000 7f84bb400640 1 -- 172.21.15.50:0/1413796913 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40b2040 con 0x7f84c40b4de0 2024-09-21T06:04:43.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.746+0000 7f84ca400640 1 -- 172.21.15.50:0/1413796913 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c41fc9e0 con 0x7f84c40b4de0 2024-09-21T06:04:43.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.746+0000 7f84ca400640 1 -- 172.21.15.50:0/1413796913 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c415bb60 con 0x7f84c40b4de0 2024-09-21T06:04:43.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.746+0000 7f84bb400640 1 -- 172.21.15.50:0/1413796913 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f84b4005270 con 0x7f84c40b4de0 2024-09-21T06:04:43.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.746+0000 7f84bb400640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f84a0073e60 0x7f84a0076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:43.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.746+0000 7f84bb400640 1 -- 172.21.15.50:0/1413796913 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f84b412c5e0 con 0x7f84c40b4de0 2024-09-21T06:04:43.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.750+0000 7f84bbe00640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f84a0073e60 0x7f84a0076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:43.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.750+0000 7f84bbe00640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f84a0073e60 0x7f84a0076370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f84c4154060 tx=0x7f84ac001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:43.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.754+0000 7f84bb400640 1 -- 172.21.15.50:0/1413796913 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f84b4016410 con 0x7f84c40b4de0 2024-09-21T06:04:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 -- 172.21.15.50:0/1413796913 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f84c40b5960 con 0x7f84c40b4de0 2024-09-21T06:04:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84bb400640 1 -- 172.21.15.50:0/1413796913 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7f84b4005560 con 0x7f84c40b4de0 2024-09-21T06:04:43.967 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:04:43.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 -- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f84a0073e60 msgr2=0x7f84a0076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f84a0073e60 0x7f84a0076370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f84c4154060 tx=0x7f84ac001380 comp rx=0 tx=0).stop 2024-09-21T06:04:43.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 -- 172.21.15.50:0/1413796913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84c40b4de0 msgr2=0x7f84c4158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:43.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84c40b4de0 0x7f84c4158d20 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f84b401cc40 tx=0x7f84b4005d50 comp rx=0 tx=0).stop 2024-09-21T06:04:43.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 -- 172.21.15.50:0/1413796913 shutdown_connections 2024-09-21T06:04:43.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84c41527b0 0x7f84c40b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84c40b5c70 0x7f84c40b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f84a0073e60 0x7f84a0076370 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 --2- 172.21.15.50:0/1413796913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84c40b4de0 0x7f84c4158d20 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 -- 172.21.15.50:0/1413796913 >> 172.21.15.50:0/1413796913 conn(0x7f84c40b2040 msgr2=0x7f84c4159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 -- 172.21.15.50:0/1413796913 shutdown_connections 2024-09-21T06:04:43.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:43.966+0000 7f84ca400640 1 -- 172.21.15.50:0/1413796913 wait complete. 2024-09-21T06:04:43.990 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:04:43.990 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:04:43.990 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-21T06:04:43.991 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:04:43.991 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:04:43.991 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:04:43.991 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:04:43.991 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:04:43.991 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:04:43.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:43.991 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-21T06:04:44.001 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 41911sec preferred_lft 41911sec 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:8b:d0:0a:5d brd ff:ff:ff:ff:ff:ff 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:04:44.002 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run.smithi050.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:04:44.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:44.004 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:04:44.063 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-21T06:04:44.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:44.065 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-21T06:04:44.115 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-21T06:04:44.115 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:04:44.115 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-21T06:04:44.115 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T06:04:44.115 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-21T06:04:44.115 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:04:44.115 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:04:44.115 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T06:04:44.115 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:04:44.115 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:04:44.116 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:04:44.116 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T06:04:44.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:04:44.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:04:44.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:44.254 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T06:04:44.263 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:04:44.264 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:04:44.264 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:04:44.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:44.264 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-09-21T06:04:44.325 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:04:44.326 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:04:44.374 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:04:44.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:04:44.374 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:04:44.425 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:04:44.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:44.426 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:04:44.531 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:04:44.532 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:04:44.532 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:04:44.550 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:04:44.550 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:04:44.550 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:04:44.550 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:04:44.550 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:04:44.560 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:04:44.560 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:04:44.560 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:04:44.560 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:04:44.560 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:04:44.560 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T06:04:44.560 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T06:04:44.560 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T06:04:44.564 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:04:44.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:44.564 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:04:44.584 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:04:44.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:04:44.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:04:44.700 INFO:teuthology.orchestra.run.smithi050.stdout:client5876 2024-09-21T06:04:44.700 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:04:44.700 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:04:44.709 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T06:04:44.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.914+0000 7f54c8200640 1 -- 172.21.15.50:0/1356734072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54c00b4de0 msgr2=0x7f54c00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:44.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.914+0000 7f54c8200640 1 --2- 172.21.15.50:0/1356734072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54c00b4de0 0x7f54c00b51f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f54c0071cb0 tx=0x7f54b801ccc0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.914+0000 7f54c8200640 1 -- 172.21.15.50:0/1356734072 shutdown_connections 2024-09-21T06:04:44.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.914+0000 7f54c8200640 1 --2- 172.21.15.50:0/1356734072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54c00b4de0 0x7f54c00b51f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.914+0000 7f54c8200640 1 --2- 172.21.15.50:0/1356734072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54c01527b0 0x7f54c0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.914+0000 7f54c8200640 1 --2- 172.21.15.50:0/1356734072 >> v2:172.21.15.88:3300/0 conn(0x7f54c00b5c70 0x7f54c015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.914+0000 7f54c8200640 1 -- 172.21.15.50:0/1356734072 >> 172.21.15.50:0/1356734072 conn(0x7f54c00b2040 msgr2=0x7f54c00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:44.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.914+0000 7f54c8200640 1 -- 172.21.15.50:0/1356734072 shutdown_connections 2024-09-21T06:04:44.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.914+0000 7f54c8200640 1 -- 172.21.15.50:0/1356734072 wait complete. 2024-09-21T06:04:44.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c8200640 1 Processor -- start 2024-09-21T06:04:44.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c8200640 1 -- start start 2024-09-21T06:04:44.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c8200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c00b4de0 0x7f54c00af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c8200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54c00b5c70 0x7f54c00aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c8200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54c01527b0 0x7f54c0158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c8200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f54c015f730 con 0x7f54c01527b0 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c8200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f54c015f5b0 con 0x7f54c00b4de0 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c8200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f54c015f8b0 con 0x7f54c00b5c70 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c6200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54c01527b0 0x7f54c0158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c6200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54c01527b0 0x7f54c0158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36804/0 (socket says 172.21.15.50:36804) 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c6200640 1 -- 172.21.15.50:0/585102489 learned_addr learned my addr 172.21.15.50:0/585102489 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c4e00640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54c00b5c70 0x7f54c00aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c5800640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c00b4de0 0x7f54c00af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c6200640 1 -- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54c00b5c70 msgr2=0x7f54c00aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c6200640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54c00b5c70 0x7f54c00aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c6200640 1 -- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c00b4de0 msgr2=0x7f54c00af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c6200640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c00b4de0 0x7f54c00af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c6200640 1 -- 172.21.15.50:0/585102489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c00afb70 con 0x7f54c01527b0 2024-09-21T06:04:44.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c5800640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c00b4de0 0x7f54c00af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:44.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c4e00640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54c00b5c70 0x7f54c00aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:44.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c6200640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54c01527b0 0x7f54c0158390 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009bd0 tx=0x7f54b8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:44.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54bd600640 1 -- 172.21.15.50:0/585102489 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b80bb070 con 0x7f54c01527b0 2024-09-21T06:04:44.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54bd600640 1 -- 172.21.15.50:0/585102489 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54b8004d40 con 0x7f54c01527b0 2024-09-21T06:04:44.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54bd600640 1 -- 172.21.15.50:0/585102489 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b80b2040 con 0x7f54c01527b0 2024-09-21T06:04:44.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.918+0000 7f54c8200640 1 -- 172.21.15.50:0/585102489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c0151730 con 0x7f54c01527b0 2024-09-21T06:04:44.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.922+0000 7f54c8200640 1 -- 172.21.15.50:0/585102489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c013c980 con 0x7f54c01527b0 2024-09-21T06:04:44.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.922+0000 7f54c8200640 1 -- 172.21.15.50:0/585102489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c015fdd0 con 0x7f54c01527b0 2024-09-21T06:04:44.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.922+0000 7f54bd600640 1 -- 172.21.15.50:0/585102489 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f54b8004240 con 0x7f54c01527b0 2024-09-21T06:04:44.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.922+0000 7f54bd600640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54a0073e60 0x7f54a0076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.922+0000 7f54bd600640 1 -- 172.21.15.50:0/585102489 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f54b812cf80 con 0x7f54c01527b0 2024-09-21T06:04:44.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.922+0000 7f54c5800640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54a0073e60 0x7f54a0076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.926+0000 7f54c5800640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54a0073e60 0x7f54a0076370 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f54b4005fd0 tx=0x7f54b4005d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:44.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:44.926+0000 7f54bd600640 1 -- 172.21.15.50:0/585102489 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f54b80c4020 con 0x7f54c01527b0 2024-09-21T06:04:45.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.142+0000 7f54c8200640 1 -- 172.21.15.50:0/585102489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f54c015edd0 con 0x7f54c01527b0 2024-09-21T06:04:45.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.518+0000 7f54bd600640 1 -- 172.21.15.50:0/585102489 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v46) ==== 136+0+0 (secure 0 0 0) 0x7f54c00afb70 con 0x7f54c01527b0 2024-09-21T06:04:45.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.522+0000 7f54c8200640 1 -- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54a0073e60 msgr2=0x7f54a0076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:45.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.522+0000 7f54c8200640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54a0073e60 0x7f54a0076370 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f54b4005fd0 tx=0x7f54b4005d40 comp rx=0 tx=0).stop 2024-09-21T06:04:45.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.522+0000 7f54c8200640 1 -- 172.21.15.50:0/585102489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54c01527b0 msgr2=0x7f54c0158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:45.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.522+0000 7f54c8200640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54c01527b0 0x7f54c0158390 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009bd0 tx=0x7f54b8005b00 comp rx=0 tx=0).stop 2024-09-21T06:04:45.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.522+0000 7f54c8200640 1 -- 172.21.15.50:0/585102489 shutdown_connections 2024-09-21T06:04:45.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.522+0000 7f54c8200640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54a0073e60 0x7f54a0076370 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.522+0000 7f54c8200640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54c01527b0 0x7f54c0158390 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.522+0000 7f54c8200640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54c00b5c70 0x7f54c00aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.522+0000 7f54c8200640 1 --2- 172.21.15.50:0/585102489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c00b4de0 0x7f54c00af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.522+0000 7f54c8200640 1 -- 172.21.15.50:0/585102489 >> 172.21.15.50:0/585102489 conn(0x7f54c00b2040 msgr2=0x7f54c015e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:45.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.522+0000 7f54c8200640 1 -- 172.21.15.50:0/585102489 shutdown_connections 2024-09-21T06:04:45.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.522+0000 7f54c8200640 1 -- 172.21.15.50:0/585102489 wait complete. 2024-09-21T06:04:45.546 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:04:45.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.706+0000 7f6936400640 1 -- 172.21.15.50:0/2071215372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69301527b0 msgr2=0x7f693015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:45.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.706+0000 7f6936400640 1 --2- 172.21.15.50:0/2071215372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69301527b0 0x7f693015bb60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6930074b20 tx=0x7f691c01c820 comp rx=0 tx=0).stop 2024-09-21T06:04:45.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.706+0000 7f6936400640 1 -- 172.21.15.50:0/2071215372 shutdown_connections 2024-09-21T06:04:45.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.706+0000 7f6936400640 1 --2- 172.21.15.50:0/2071215372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69301527b0 0x7f693015bb60 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.706+0000 7f6936400640 1 -- 172.21.15.50:0/2071215372 >> 172.21.15.50:0/2071215372 conn(0x7f69300b2040 msgr2=0x7f69300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:45.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.706+0000 7f6936400640 1 -- 172.21.15.50:0/2071215372 shutdown_connections 2024-09-21T06:04:45.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.706+0000 7f6936400640 1 -- 172.21.15.50:0/2071215372 wait complete. 2024-09-21T06:04:45.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.706+0000 7f6936400640 1 Processor -- start 2024-09-21T06:04:45.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.706+0000 7f6936400640 1 -- start start 2024-09-21T06:04:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f6936400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69300b4de0 0x7f69301fc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f6936400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69300b5c70 0x7f6930154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f6936400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69301527b0 0x7f69301435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f6936400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6930169c00 con 0x7f69300b5c70 2024-09-21T06:04:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f6936400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6930169a80 con 0x7f69300b4de0 2024-09-21T06:04:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f6936400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6930169d80 con 0x7f69301527b0 2024-09-21T06:04:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f692ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69300b5c70 0x7f6930154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f692ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69300b5c70 0x7f6930154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36824/0 (socket says 172.21.15.50:36824) 2024-09-21T06:04:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f692ea00640 1 -- 172.21.15.50:0/442657259 learned_addr learned my addr 172.21.15.50:0/442657259 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f692f400640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69300b4de0 0x7f69301fc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f692fe00640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69301527b0 0x7f69301435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f692ea00640 1 -- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69301527b0 msgr2=0x7f69301435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f692ea00640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69301527b0 0x7f69301435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f692ea00640 1 -- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69300b4de0 msgr2=0x7f69301fc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f692ea00640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69300b4de0 0x7f69301fc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f692ea00640 1 -- 172.21.15.50:0/442657259 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6930158700 con 0x7f69300b5c70 2024-09-21T06:04:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f692ea00640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69300b5c70 0x7f6930154e00 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f691c019430 tx=0x7f691c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f6927400640 1 -- 172.21.15.50:0/442657259 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c0bd070 con 0x7f69300b5c70 2024-09-21T06:04:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f6927400640 1 -- 172.21.15.50:0/442657259 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f691c003600 con 0x7f69300b5c70 2024-09-21T06:04:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f6927400640 1 -- 172.21.15.50:0/442657259 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c0b2040 con 0x7f69300b5c70 2024-09-21T06:04:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f6936400640 1 -- 172.21.15.50:0/442657259 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69301589e0 con 0x7f69300b5c70 2024-09-21T06:04:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f6936400640 1 -- 172.21.15.50:0/442657259 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6930158fa0 con 0x7f69300b5c70 2024-09-21T06:04:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.710+0000 7f6936400640 1 -- 172.21.15.50:0/442657259 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6930158470 con 0x7f69300b5c70 2024-09-21T06:04:45.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.714+0000 7f6927400640 1 -- 172.21.15.50:0/442657259 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f691c003a50 con 0x7f69300b5c70 2024-09-21T06:04:45.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.714+0000 7f6927400640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6910073e90 0x7f69100763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:45.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.714+0000 7f6927400640 1 -- 172.21.15.50:0/442657259 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f691c12cd70 con 0x7f69300b5c70 2024-09-21T06:04:45.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.714+0000 7f692f400640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6910073e90 0x7f69100763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:45.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.714+0000 7f692f400640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6910073e90 0x7f69100763a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6920007fd0 tx=0x7f6920009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:45.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.718+0000 7f6927400640 1 -- 172.21.15.50:0/442657259 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f691c0164d0 con 0x7f69300b5c70 2024-09-21T06:04:45.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.922+0000 7f6936400640 1 -- 172.21.15.50:0/442657259 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f693006b540 con 0x7f6910073e90 2024-09-21T06:04:45.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.922+0000 7f6927400640 1 -- 172.21.15.50:0/442657259 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6920001490 con 0x7f6910073e90 2024-09-21T06:04:45.925 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:04:45.925 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:04:45.925 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:04:45.925 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:04:45.925 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:04:45.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.926+0000 7f6936400640 1 -- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6910073e90 msgr2=0x7f69100763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:45.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.926+0000 7f6936400640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6910073e90 0x7f69100763a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6920007fd0 tx=0x7f6920009040 comp rx=0 tx=0).stop 2024-09-21T06:04:45.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.926+0000 7f6936400640 1 -- 172.21.15.50:0/442657259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69300b5c70 msgr2=0x7f6930154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:45.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.926+0000 7f6936400640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69300b5c70 0x7f6930154e00 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f691c019430 tx=0x7f691c0040c0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.926+0000 7f6936400640 1 -- 172.21.15.50:0/442657259 shutdown_connections 2024-09-21T06:04:45.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.926+0000 7f6936400640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f69301527b0 0x7f69301435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.926+0000 7f6936400640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6910073e90 0x7f69100763a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.926+0000 7f6936400640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69300b5c70 0x7f6930154e00 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.926+0000 7f6936400640 1 --2- 172.21.15.50:0/442657259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69300b4de0 0x7f69301fc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.926+0000 7f6936400640 1 -- 172.21.15.50:0/442657259 >> 172.21.15.50:0/442657259 conn(0x7f69300b2040 msgr2=0x7f693015aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:45.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.926+0000 7f6936400640 1 -- 172.21.15.50:0/442657259 shutdown_connections 2024-09-21T06:04:45.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:45.926+0000 7f6936400640 1 -- 172.21.15.50:0/442657259 wait complete. 2024-09-21T06:04:45.948 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:04:46.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.106+0000 7f5cfda00640 1 -- 172.21.15.50:0/4073379051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf8074b20 msgr2=0x7f5cf815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.106+0000 7f5cfda00640 1 --2- 172.21.15.50:0/4073379051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf8074b20 0x7f5cf815cf90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009bc0 tx=0x7f5cf001cb10 comp rx=0 tx=0).stop 2024-09-21T06:04:46.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.106+0000 7f5cfda00640 1 -- 172.21.15.50:0/4073379051 shutdown_connections 2024-09-21T06:04:46.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.106+0000 7f5cfda00640 1 --2- 172.21.15.50:0/4073379051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf8074b20 0x7f5cf815cf90 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.106+0000 7f5cfda00640 1 -- 172.21.15.50:0/4073379051 >> 172.21.15.50:0/4073379051 conn(0x7f5cf80708c0 msgr2=0x7f5cf8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:46.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.106+0000 7f5cfda00640 1 -- 172.21.15.50:0/4073379051 shutdown_connections 2024-09-21T06:04:46.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.106+0000 7f5cfda00640 1 -- 172.21.15.50:0/4073379051 wait complete. 2024-09-21T06:04:46.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.110+0000 7f5cfda00640 1 Processor -- start 2024-09-21T06:04:46.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.110+0000 7f5cfda00640 1 -- start start 2024-09-21T06:04:46.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.110+0000 7f5cfda00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf80b5a80 0x7f5cf8146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:46.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.110+0000 7f5cfda00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5cf8157030 0x7f5cf814a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:46.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.110+0000 7f5cfda00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf80b2570 0x7f5cf814aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:46.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.110+0000 7f5cfda00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf816b590 con 0x7f5cf80b2570 2024-09-21T06:04:46.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.110+0000 7f5cfda00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf816b410 con 0x7f5cf80b5a80 2024-09-21T06:04:46.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.110+0000 7f5cfda00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf816b710 con 0x7f5cf8157030 2024-09-21T06:04:46.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.110+0000 7f5cf7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf80b2570 0x7f5cf814aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:46.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.110+0000 7f5cf7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf80b2570 0x7f5cf814aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36852/0 (socket says 172.21.15.50:36852) 2024-09-21T06:04:46.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.110+0000 7f5cf7400640 1 -- 172.21.15.50:0/2070424270 learned_addr learned my addr 172.21.15.50:0/2070424270 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:46.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cf7400640 1 -- 172.21.15.50:0/2070424270 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5cf8157030 msgr2=0x7f5cf814a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cf7400640 1 --2- 172.21.15.50:0/2070424270 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5cf8157030 0x7f5cf814a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cf7400640 1 -- 172.21.15.50:0/2070424270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf80b5a80 msgr2=0x7f5cf8146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cf7400640 1 --2- 172.21.15.50:0/2070424270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf80b5a80 0x7f5cf8146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cf7400640 1 -- 172.21.15.50:0/2070424270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf80b4070 con 0x7f5cf80b2570 2024-09-21T06:04:46.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cf7400640 1 --2- 172.21.15.50:0/2070424270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf80b2570 0x7f5cf814aae0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0019720 tx=0x7f5cf00a5940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:46.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cef400640 1 -- 172.21.15.50:0/2070424270 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00bb070 con 0x7f5cf80b2570 2024-09-21T06:04:46.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cef400640 1 -- 172.21.15.50:0/2070424270 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cf0004d40 con 0x7f5cf80b2570 2024-09-21T06:04:46.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cef400640 1 -- 172.21.15.50:0/2070424270 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00b2040 con 0x7f5cf80b2570 2024-09-21T06:04:46.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cfda00640 1 -- 172.21.15.50:0/2070424270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf80b4350 con 0x7f5cf80b2570 2024-09-21T06:04:46.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cfda00640 1 -- 172.21.15.50:0/2070424270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf80b4910 con 0x7f5cf80b2570 2024-09-21T06:04:46.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cfda00640 1 -- 172.21.15.50:0/2070424270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf80b3df0 con 0x7f5cf80b2570 2024-09-21T06:04:46.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.114+0000 7f5cef400640 1 -- 172.21.15.50:0/2070424270 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f5cf0004320 con 0x7f5cf80b2570 2024-09-21T06:04:46.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.118+0000 7f5cef400640 1 --2- 172.21.15.50:0/2070424270 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5cd4073e60 0x7f5cd4076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:46.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.118+0000 7f5cef400640 1 -- 172.21.15.50:0/2070424270 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f5cf012cc50 con 0x7f5cf80b2570 2024-09-21T06:04:46.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.118+0000 7f5cf6a00640 1 --2- 172.21.15.50:0/2070424270 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5cd4073e60 0x7f5cd4076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:46.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.118+0000 7f5cf6a00640 1 --2- 172.21.15.50:0/2070424270 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5cd4073e60 0x7f5cd4076370 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8007fd0 tx=0x7f5ce8007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:46.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.122+0000 7f5cef400640 1 -- 172.21.15.50:0/2070424270 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5cf80b3df0 con 0x7f5cf80b2570 2024-09-21T06:04:46.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.298+0000 7f5cfda00640 1 -- 172.21.15.50:0/2070424270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5cf8150880 con 0x7f5cf80b2570 2024-09-21T06:04:46.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.298+0000 7f5cef400640 1 -- 172.21.15.50:0/2070424270 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7f5cf0100e20 con 0x7f5cf80b2570 2024-09-21T06:04:46.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.302+0000 7f5cfda00640 1 -- 172.21.15.50:0/2070424270 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5cd4073e60 msgr2=0x7f5cd4076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.302+0000 7f5cfda00640 1 --2- 172.21.15.50:0/2070424270 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5cd4073e60 0x7f5cd4076370 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8007fd0 tx=0x7f5ce8007980 comp rx=0 tx=0).stop 2024-09-21T06:04:46.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.302+0000 7f5cfda00640 1 -- 172.21.15.50:0/2070424270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf80b2570 msgr2=0x7f5cf814aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.302+0000 7f5cfda00640 1 --2- 172.21.15.50:0/2070424270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf80b2570 0x7f5cf814aae0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0019720 tx=0x7f5cf00a5940 comp rx=0 tx=0).stop 2024-09-21T06:04:46.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.302+0000 7f5cfda00640 1 -- 172.21.15.50:0/2070424270 shutdown_connections 2024-09-21T06:04:46.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.302+0000 7f5cfda00640 1 --2- 172.21.15.50:0/2070424270 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5cd4073e60 0x7f5cd4076370 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.302+0000 7f5cfda00640 1 --2- 172.21.15.50:0/2070424270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf80b2570 0x7f5cf814aae0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.302+0000 7f5cfda00640 1 --2- 172.21.15.50:0/2070424270 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5cf8157030 0x7f5cf814a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.302+0000 7f5cfda00640 1 --2- 172.21.15.50:0/2070424270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf80b5a80 0x7f5cf8146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.302+0000 7f5cfda00640 1 -- 172.21.15.50:0/2070424270 >> 172.21.15.50:0/2070424270 conn(0x7f5cf80708c0 msgr2=0x7f5cf8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:46.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.302+0000 7f5cfda00640 1 -- 172.21.15.50:0/2070424270 shutdown_connections 2024-09-21T06:04:46.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.302+0000 7f5cfda00640 1 -- 172.21.15.50:0/2070424270 wait complete. 2024-09-21T06:04:46.324 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-21T06:04:46.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.482+0000 7fb939c00640 1 -- 172.21.15.50:0/3652020212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9340b4de0 msgr2=0x7fb9340b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.482+0000 7fb939c00640 1 --2- 172.21.15.50:0/3652020212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9340b4de0 0x7fb9340b51f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009c80 tx=0x7fb92c01cb00 comp rx=0 tx=0).stop 2024-09-21T06:04:46.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.482+0000 7fb939c00640 1 -- 172.21.15.50:0/3652020212 shutdown_connections 2024-09-21T06:04:46.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.482+0000 7fb939c00640 1 --2- 172.21.15.50:0/3652020212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9340b4de0 0x7fb9340b51f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.482+0000 7fb939c00640 1 --2- 172.21.15.50:0/3652020212 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9341527b0 0x7fb93415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.482+0000 7fb939c00640 1 -- 172.21.15.50:0/3652020212 >> 172.21.15.50:0/3652020212 conn(0x7fb9340b2040 msgr2=0x7fb9340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:46.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.482+0000 7fb939c00640 1 -- 172.21.15.50:0/3652020212 shutdown_connections 2024-09-21T06:04:46.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.482+0000 7fb939c00640 1 -- 172.21.15.50:0/3652020212 wait complete. 2024-09-21T06:04:46.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb939c00640 1 Processor -- start 2024-09-21T06:04:46.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb939c00640 1 -- start start 2024-09-21T06:04:46.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb939c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9340b4de0 0x7fb9340cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:46.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb939c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9340b5c70 0x7fb9341413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:46.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb939c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9341527b0 0x7fb93413e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:46.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb939c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb934168ac0 con 0x7fb9340b4de0 2024-09-21T06:04:46.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb939c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb934168940 con 0x7fb9341527b0 2024-09-21T06:04:46.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb939c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb934168c40 con 0x7fb9340b5c70 2024-09-21T06:04:46.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb932a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9340b4de0 0x7fb9340cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:46.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb932a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9340b4de0 0x7fb9340cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36882/0 (socket says 172.21.15.50:36882) 2024-09-21T06:04:46.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb932a00640 1 -- 172.21.15.50:0/2947548105 learned_addr learned my addr 172.21.15.50:0/2947548105 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:46.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb92a000640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9340b5c70 0x7fb9341413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:46.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.486+0000 7fb933400640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9341527b0 0x7fb93413e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:46.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb932a00640 1 -- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9340b5c70 msgr2=0x7fb9341413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb932a00640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9340b5c70 0x7fb9341413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb932a00640 1 -- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9341527b0 msgr2=0x7fb93413e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb932a00640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9341527b0 0x7fb93413e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb932a00640 1 -- 172.21.15.50:0/2947548105 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93413f300 con 0x7fb9340b4de0 2024-09-21T06:04:46.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb932a00640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9340b4de0 0x7fb9340cc950 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb92400b410 tx=0x7fb92400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:46.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb92b400640 1 -- 172.21.15.50:0/2947548105 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924015030 con 0x7fb9340b4de0 2024-09-21T06:04:46.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb92b400640 1 -- 172.21.15.50:0/2947548105 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb924004020 con 0x7fb9340b4de0 2024-09-21T06:04:46.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb92b400640 1 -- 172.21.15.50:0/2947548105 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240044b0 con 0x7fb9340b4de0 2024-09-21T06:04:46.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb939c00640 1 -- 172.21.15.50:0/2947548105 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93413f5e0 con 0x7fb9340b4de0 2024-09-21T06:04:46.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb939c00640 1 -- 172.21.15.50:0/2947548105 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93413fba0 con 0x7fb9340b4de0 2024-09-21T06:04:46.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb939c00640 1 -- 172.21.15.50:0/2947548105 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9340b51f0 con 0x7fb9340b4de0 2024-09-21T06:04:46.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.490+0000 7fb92b400640 1 -- 172.21.15.50:0/2947548105 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7fb924018070 con 0x7fb9340b4de0 2024-09-21T06:04:46.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.494+0000 7fb92b400640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb910073e60 0x7fb910076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:46.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.494+0000 7fb92b400640 1 -- 172.21.15.50:0/2947548105 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7fb924092cd0 con 0x7fb9340b4de0 2024-09-21T06:04:46.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.494+0000 7fb92a000640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb910073e60 0x7fb910076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:46.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.494+0000 7fb92a000640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb910073e60 0x7fb910076370 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb93415bf60 tx=0x7fb91c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:46.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.494+0000 7fb92b400640 1 -- 172.21.15.50:0/2947548105 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb92405eeb0 con 0x7fb9340b4de0 2024-09-21T06:04:46.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 -- 172.21.15.50:0/2947548105 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fb934155cd0 con 0x7fb9340b4de0 2024-09-21T06:04:46.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb92b400640 1 -- 172.21.15.50:0/2947548105 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v41) ==== 116+0+2 (secure 0 0 0) 0x7fb9240638e0 con 0x7fb9340b4de0 2024-09-21T06:04:46.675 INFO:teuthology.orchestra.run.smithi050.stdout:4 2024-09-21T06:04:46.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 -- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb910073e60 msgr2=0x7fb910076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb910073e60 0x7fb910076370 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb93415bf60 tx=0x7fb91c00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 -- 172.21.15.50:0/2947548105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9340b4de0 msgr2=0x7fb9340cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9340b4de0 0x7fb9340cc950 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb92400b410 tx=0x7fb92400b8d0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 -- 172.21.15.50:0/2947548105 shutdown_connections 2024-09-21T06:04:46.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9341527b0 0x7fb93413e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9340b5c70 0x7fb9341413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb910073e60 0x7fb910076370 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 --2- 172.21.15.50:0/2947548105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9340b4de0 0x7fb9340cc950 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 -- 172.21.15.50:0/2947548105 >> 172.21.15.50:0/2947548105 conn(0x7fb9340b2040 msgr2=0x7fb93415b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:46.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 -- 172.21.15.50:0/2947548105 shutdown_connections 2024-09-21T06:04:46.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.674+0000 7fb939c00640 1 -- 172.21.15.50:0/2947548105 wait complete. 2024-09-21T06:04:46.698 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:04:46.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.858+0000 7fd3dd000640 1 -- 172.21.15.50:0/2458132302 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd3d815c5d0 msgr2=0x7fd3d8073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.858+0000 7fd3dd000640 1 --2- 172.21.15.50:0/2458132302 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd3d815c5d0 0x7fd3d8073dc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009840 tx=0x7fd3c801c6b0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.858+0000 7fd3dd000640 1 -- 172.21.15.50:0/2458132302 shutdown_connections 2024-09-21T06:04:46.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.858+0000 7fd3dd000640 1 --2- 172.21.15.50:0/2458132302 >> v2:172.21.15.88:3300/0 conn(0x7fd3d8074300 0x7fd3d815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.858+0000 7fd3dd000640 1 --2- 172.21.15.50:0/2458132302 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd3d815c5d0 0x7fd3d8073dc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.858+0000 7fd3dd000640 1 -- 172.21.15.50:0/2458132302 >> 172.21.15.50:0/2458132302 conn(0x7fd3d80b2c40 msgr2=0x7fd3d80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:46.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.858+0000 7fd3dd000640 1 -- 172.21.15.50:0/2458132302 shutdown_connections 2024-09-21T06:04:46.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.858+0000 7fd3dd000640 1 -- 172.21.15.50:0/2458132302 wait complete. 2024-09-21T06:04:46.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3dd000640 1 Processor -- start 2024-09-21T06:04:46.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3dd000640 1 -- start start 2024-09-21T06:04:46.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3dd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3d8074300 0x7fd3d814c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3dd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3d815c5d0 0x7fd3d80b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3dd000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd3d815f6b0 0x7fd3d80b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3dd000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d806f980 con 0x7fd3d8074300 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3dd000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d80b3d50 con 0x7fd3d815c5d0 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3dd000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d806fb00 con 0x7fd3d815f6b0 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3d5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3d8074300 0x7fd3d814c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3d5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3d8074300 0x7fd3d814c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38814/0 (socket says 172.21.15.50:38814) 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3d5e00640 1 -- 172.21.15.50:0/1587141129 learned_addr learned my addr 172.21.15.50:0/1587141129 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3cfe00640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3d815c5d0 0x7fd3d80b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3d6800640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd3d815f6b0 0x7fd3d80b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3d5e00640 1 -- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd3d815f6b0 msgr2=0x7fd3d80b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3d5e00640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd3d815f6b0 0x7fd3d80b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3d5e00640 1 -- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3d815c5d0 msgr2=0x7fd3d80b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3d5e00640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3d815c5d0 0x7fd3d80b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3d5e00640 1 -- 172.21.15.50:0/1587141129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d8159320 con 0x7fd3d8074300 2024-09-21T06:04:46.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3cfe00640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3d815c5d0 0x7fd3d80b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:46.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3d5e00640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3d8074300 0x7fd3d814c170 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd3c801a340 tx=0x7fd3c8005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:46.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3ce000640 1 -- 172.21.15.50:0/1587141129 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80a7af0 con 0x7fd3d8074300 2024-09-21T06:04:46.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3ce000640 1 -- 172.21.15.50:0/1587141129 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3c80bf070 con 0x7fd3d8074300 2024-09-21T06:04:46.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3ce000640 1 -- 172.21.15.50:0/1587141129 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8003600 con 0x7fd3d8074300 2024-09-21T06:04:46.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3dd000640 1 -- 172.21.15.50:0/1587141129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d81fa7c0 con 0x7fd3d8074300 2024-09-21T06:04:46.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.862+0000 7fd3dd000640 1 -- 172.21.15.50:0/1587141129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d81fad80 con 0x7fd3d8074300 2024-09-21T06:04:46.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.866+0000 7fd3dd000640 1 -- 172.21.15.50:0/1587141129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d815f170 con 0x7fd3d8074300 2024-09-21T06:04:46.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.866+0000 7fd3ce000640 1 -- 172.21.15.50:0/1587141129 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7fd3c80037a0 con 0x7fd3d8074300 2024-09-21T06:04:46.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.866+0000 7fd3ce000640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd3b4073e60 0x7fd3b4076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:46.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.866+0000 7fd3ce000640 1 -- 172.21.15.50:0/1587141129 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7fd3c812e030 con 0x7fd3d8074300 2024-09-21T06:04:46.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.866+0000 7fd3cfe00640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd3b4073e60 0x7fd3b4076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:46.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.870+0000 7fd3cfe00640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd3b4073e60 0x7fd3b4076370 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80b1670 tx=0x7fd3c00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:46.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:46.870+0000 7fd3ce000640 1 -- 172.21.15.50:0/1587141129 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd3c80c9020 con 0x7fd3d8074300 2024-09-21T06:04:47.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.046+0000 7fd3dd000640 1 -- 172.21.15.50:0/1587141129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fd3d8076250 con 0x7fd3d8074300 2024-09-21T06:04:47.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3ce000640 1 -- 172.21.15.50:0/1587141129 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v42)=0 v42) ==== 131+0+0 (secure 0 0 0) 0x7fd3d8159320 con 0x7fd3d8074300 2024-09-21T06:04:47.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3dd000640 1 -- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd3b4073e60 msgr2=0x7fd3b4076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3dd000640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd3b4073e60 0x7fd3b4076370 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80b1670 tx=0x7fd3c00091e0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3dd000640 1 -- 172.21.15.50:0/1587141129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3d8074300 msgr2=0x7fd3d814c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3dd000640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3d8074300 0x7fd3d814c170 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd3c801a340 tx=0x7fd3c8005790 comp rx=0 tx=0).stop 2024-09-21T06:04:47.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3dd000640 1 -- 172.21.15.50:0/1587141129 shutdown_connections 2024-09-21T06:04:47.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3dd000640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd3d815f6b0 0x7fd3d80b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3dd000640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3d815c5d0 0x7fd3d80b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3dd000640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd3b4073e60 0x7fd3b4076370 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3dd000640 1 --2- 172.21.15.50:0/1587141129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3d8074300 0x7fd3d814c170 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3dd000640 1 -- 172.21.15.50:0/1587141129 >> 172.21.15.50:0/1587141129 conn(0x7fd3d80b2c40 msgr2=0x7fd3d815bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3dd000640 1 -- 172.21.15.50:0/1587141129 shutdown_connections 2024-09-21T06:04:47.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.054+0000 7fd3dd000640 1 -- 172.21.15.50:0/1587141129 wait complete. 2024-09-21T06:04:47.075 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:47.074+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:47.075 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:47.074+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:47.075 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:47.074+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:47.076 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:47.074+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:47.076 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:47.074+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:47.078 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-21T06:04:47.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.238+0000 7f55d8a00640 1 -- 172.21.15.50:0/3718552121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55d40b5c70 msgr2=0x7f55d415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.238+0000 7f55d8a00640 1 --2- 172.21.15.50:0/3718552121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55d40b5c70 0x7f55d415bb60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f55d40cca90 tx=0x7f55c401c800 comp rx=0 tx=0).stop 2024-09-21T06:04:47.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.238+0000 7f55d8a00640 1 -- 172.21.15.50:0/3718552121 shutdown_connections 2024-09-21T06:04:47.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.238+0000 7f55d8a00640 1 --2- 172.21.15.50:0/3718552121 >> v2:172.21.15.88:3300/0 conn(0x7f55d40b4de0 0x7f55d40b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.238+0000 7f55d8a00640 1 --2- 172.21.15.50:0/3718552121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55d40b5c70 0x7f55d415bb60 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.238+0000 7f55d8a00640 1 -- 172.21.15.50:0/3718552121 >> 172.21.15.50:0/3718552121 conn(0x7f55d40b2040 msgr2=0x7f55d40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.238+0000 7f55d8a00640 1 -- 172.21.15.50:0/3718552121 shutdown_connections 2024-09-21T06:04:47.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.238+0000 7f55d8a00640 1 -- 172.21.15.50:0/3718552121 wait complete. 2024-09-21T06:04:47.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.238+0000 7f55d8a00640 1 Processor -- start 2024-09-21T06:04:47.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55d8a00640 1 -- start start 2024-09-21T06:04:47.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55d8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55d40b4de0 0x7f55d4154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55d8a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55d40b5c70 0x7f55d41f8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55d8a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55d41527b0 0x7f55d4142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55d8a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4168970 con 0x7f55d40b5c70 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55d8a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f55d41687f0 con 0x7f55d40b4de0 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55d8a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f55d4168af0 con 0x7f55d41527b0 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55cbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55d40b5c70 0x7f55d41f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55cbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55d40b5c70 0x7f55d41f8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38830/0 (socket says 172.21.15.50:38830) 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55cbe00640 1 -- 172.21.15.50:0/1503966838 learned_addr learned my addr 172.21.15.50:0/1503966838 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55d2800640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55d41527b0 0x7f55d4142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55d1e00640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55d40b4de0 0x7f55d4154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55cbe00640 1 -- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55d41527b0 msgr2=0x7f55d4142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55cbe00640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55d41527b0 0x7f55d4142320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55cbe00640 1 -- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55d40b4de0 msgr2=0x7f55d4154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55cbe00640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55d40b4de0 0x7f55d4154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55cbe00640 1 -- 172.21.15.50:0/1503966838 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d4154c00 con 0x7f55d40b5c70 2024-09-21T06:04:47.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55cbe00640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55d40b5c70 0x7f55d41f8200 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00c3c0 tx=0x7f55bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55ca000640 1 -- 172.21.15.50:0/1503966838 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc017070 con 0x7f55d40b5c70 2024-09-21T06:04:47.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55ca000640 1 -- 172.21.15.50:0/1503966838 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55bc0053c0 con 0x7f55d40b5c70 2024-09-21T06:04:47.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55d8a00640 1 -- 172.21.15.50:0/1503966838 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d4154ee0 con 0x7f55d40b5c70 2024-09-21T06:04:47.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55ca000640 1 -- 172.21.15.50:0/1503966838 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc003860 con 0x7f55d40b5c70 2024-09-21T06:04:47.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55d8a00640 1 -- 172.21.15.50:0/1503966838 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d41554a0 con 0x7f55d40b5c70 2024-09-21T06:04:47.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.242+0000 7f55d8a00640 1 -- 172.21.15.50:0/1503966838 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d40b51f0 con 0x7f55d40b5c70 2024-09-21T06:04:47.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.246+0000 7f55ca000640 1 -- 172.21.15.50:0/1503966838 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f55bc01a070 con 0x7f55d40b5c70 2024-09-21T06:04:47.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.246+0000 7f55ca000640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55b0073e60 0x7f55b0076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.246+0000 7f55ca000640 1 -- 172.21.15.50:0/1503966838 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f55bc092590 con 0x7f55d40b5c70 2024-09-21T06:04:47.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.246+0000 7f55d1e00640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55b0073e60 0x7f55b0076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.246+0000 7f55d1e00640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55b0073e60 0x7f55b0076370 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f55c40af000 tx=0x7f55c40a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.250+0000 7f55ca000640 1 -- 172.21.15.50:0/1503966838 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f55bc021020 con 0x7f55d40b5c70 2024-09-21T06:04:47.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.426+0000 7f55d8a00640 1 -- 172.21.15.50:0/1503966838 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f55d4154990 con 0x7f55d40b5c70 2024-09-21T06:04:47.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.430+0000 7f55ca000640 1 -- 172.21.15.50:0/1503966838 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v43)=0 v43) ==== 136+0+0 (secure 0 0 0) 0x7f55d40b5730 con 0x7f55d40b5c70 2024-09-21T06:04:47.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.434+0000 7f55d8a00640 1 -- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55b0073e60 msgr2=0x7f55b0076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.434+0000 7f55d8a00640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55b0073e60 0x7f55b0076370 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f55c40af000 tx=0x7f55c40a7040 comp rx=0 tx=0).stop 2024-09-21T06:04:47.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.434+0000 7f55d8a00640 1 -- 172.21.15.50:0/1503966838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55d40b5c70 msgr2=0x7f55d41f8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.434+0000 7f55d8a00640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55d40b5c70 0x7f55d41f8200 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00c3c0 tx=0x7f55bc00c880 comp rx=0 tx=0).stop 2024-09-21T06:04:47.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.434+0000 7f55d8a00640 1 -- 172.21.15.50:0/1503966838 shutdown_connections 2024-09-21T06:04:47.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.434+0000 7f55d8a00640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55d41527b0 0x7f55d4142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.434+0000 7f55d8a00640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55b0073e60 0x7f55b0076370 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.434+0000 7f55d8a00640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55d40b5c70 0x7f55d41f8200 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.434+0000 7f55d8a00640 1 --2- 172.21.15.50:0/1503966838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55d40b4de0 0x7f55d4154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.434+0000 7f55d8a00640 1 -- 172.21.15.50:0/1503966838 >> 172.21.15.50:0/1503966838 conn(0x7f55d40b2040 msgr2=0x7f55d40cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.434+0000 7f55d8a00640 1 -- 172.21.15.50:0/1503966838 shutdown_connections 2024-09-21T06:04:47.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.434+0000 7f55d8a00640 1 -- 172.21.15.50:0/1503966838 wait complete. 2024-09-21T06:04:47.457 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-21T06:04:47.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.618+0000 7f80e6400640 1 -- 172.21.15.50:0/2076397973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c4003a10 msgr2=0x7f80c4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.618+0000 7f80e6400640 1 --2- 172.21.15.50:0/2076397973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c4003a10 0x7f80c4003e80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009b60 tx=0x7f80d001d280 comp rx=0 tx=0).stop 2024-09-21T06:04:47.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.618+0000 7f80dfe00640 1 -- 172.21.15.50:0/2076397973 reap_dead start 2024-09-21T06:04:47.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.618+0000 7f80e6400640 1 -- 172.21.15.50:0/2076397973 shutdown_connections 2024-09-21T06:04:47.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.618+0000 7f80e6400640 1 -- 172.21.15.50:0/2076397973 >> 172.21.15.50:0/2076397973 conn(0x7f80e00b2040 msgr2=0x7f80e00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.618+0000 7f80e6400640 1 -- 172.21.15.50:0/2076397973 shutdown_connections 2024-09-21T06:04:47.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.618+0000 7f80e6400640 1 -- 172.21.15.50:0/2076397973 wait complete. 2024-09-21T06:04:47.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.622+0000 7f80e6400640 1 Processor -- start 2024-09-21T06:04:47.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.622+0000 7f80e6400640 1 -- start start 2024-09-21T06:04:47.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.622+0000 7f80e6400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80c4003a10 0x7f80e014afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.622+0000 7f80e6400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80e01fc880 0x7f80e01fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.622+0000 7f80e6400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e00ae970 0x7f80e00aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.622+0000 7f80e6400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f80e0166b90 con 0x7f80e00ae970 2024-09-21T06:04:47.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.622+0000 7f80e6400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f80e0166a10 con 0x7f80e01fc880 2024-09-21T06:04:47.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.622+0000 7f80e6400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f80e0166d10 con 0x7f80c4003a10 2024-09-21T06:04:47.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.622+0000 7f80dfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e00ae970 0x7f80e00aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.622+0000 7f80dfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e00ae970 0x7f80e00aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38860/0 (socket says 172.21.15.50:38860) 2024-09-21T06:04:47.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.622+0000 7f80dfe00640 1 -- 172.21.15.50:0/3898336506 learned_addr learned my addr 172.21.15.50:0/3898336506 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:47.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80df400640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80c4003a10 0x7f80e014afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80d7e00640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80e01fc880 0x7f80e01fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80dfe00640 1 -- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80c4003a10 msgr2=0x7f80e014afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80dfe00640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80c4003a10 0x7f80e014afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80dfe00640 1 -- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80e01fc880 msgr2=0x7f80e01fcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80dfe00640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80e01fc880 0x7f80e01fcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80dfe00640 1 -- 172.21.15.50:0/3898336506 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e01fd1d0 con 0x7f80e00ae970 2024-09-21T06:04:47.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80dfe00640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e00ae970 0x7f80e00aed80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f80d8002240 tx=0x7f80d800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80d7400640 1 -- 172.21.15.50:0/3898336506 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d800ca10 con 0x7f80e00ae970 2024-09-21T06:04:47.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80d7400640 1 -- 172.21.15.50:0/3898336506 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80d8017070 con 0x7f80e00ae970 2024-09-21T06:04:47.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80e6400640 1 -- 172.21.15.50:0/3898336506 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e00ad8d0 con 0x7f80e00ae970 2024-09-21T06:04:47.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80d7400640 1 -- 172.21.15.50:0/3898336506 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8011460 con 0x7f80e00ae970 2024-09-21T06:04:47.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80e6400640 1 -- 172.21.15.50:0/3898336506 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e00ade90 con 0x7f80e00ae970 2024-09-21T06:04:47.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80e6400640 1 -- 172.21.15.50:0/3898336506 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e0154580 con 0x7f80e00ae970 2024-09-21T06:04:47.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.626+0000 7f80d7400640 1 -- 172.21.15.50:0/3898336506 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f80d8011810 con 0x7f80e00ae970 2024-09-21T06:04:47.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.630+0000 7f80d7400640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80bc073e90 0x7f80bc0763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.630+0000 7f80d7400640 1 -- 172.21.15.50:0/3898336506 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f80d8093230 con 0x7f80e00ae970 2024-09-21T06:04:47.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.630+0000 7f80df400640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80bc073e90 0x7f80bc0763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.630+0000 7f80df400640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80bc073e90 0x7f80bc0763a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f80d00ae000 tx=0x7f80d001a020 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.634+0000 7f80d7400640 1 -- 172.21.15.50:0/3898336506 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f80e01fd1d0 con 0x7f80e00ae970 2024-09-21T06:04:47.766 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:04:47.771 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:04:47.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.842+0000 7f80e6400640 1 -- 172.21.15.50:0/3898336506 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f80e006b540 con 0x7f80bc073e90 2024-09-21T06:04:47.854 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:47.850+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:47.854 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:47.850+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:47.854 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:47.850+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:47.854 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:47.850+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:47.854 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:04:47.850+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:04:47.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80d7400640 1 -- 172.21.15.50:0/3898336506 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f80d0004040 con 0x7f80bc073e90 2024-09-21T06:04:47.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80e6400640 1 -- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80bc073e90 msgr2=0x7f80bc0763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80e6400640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80bc073e90 0x7f80bc0763a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f80d00ae000 tx=0x7f80d001a020 comp rx=0 tx=0).stop 2024-09-21T06:04:47.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80e6400640 1 -- 172.21.15.50:0/3898336506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e00ae970 msgr2=0x7f80e00aed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80e6400640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e00ae970 0x7f80e00aed80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f80d8002240 tx=0x7f80d800ecf0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80e6400640 1 -- 172.21.15.50:0/3898336506 shutdown_connections 2024-09-21T06:04:47.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80e6400640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80bc073e90 0x7f80bc0763a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80e6400640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e00ae970 0x7f80e00aed80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80e6400640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80e01fc880 0x7f80e01fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80e6400640 1 --2- 172.21.15.50:0/3898336506 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80c4003a10 0x7f80e014afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80e6400640 1 -- 172.21.15.50:0/3898336506 >> 172.21.15.50:0/3898336506 conn(0x7f80e00b2040 msgr2=0x7f80e0151b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80e6400640 1 -- 172.21.15.50:0/3898336506 shutdown_connections 2024-09-21T06:04:47.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:47.894+0000 7f80e6400640 1 -- 172.21.15.50:0/3898336506 wait complete. 2024-09-21T06:04:47.919 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:04:48.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.126+0000 7f80ed400640 1 -- 172.21.15.50:0/3994113401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e8161160 msgr2=0x7f80e8074950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:48.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.126+0000 7f80ed400640 1 --2- 172.21.15.50:0/3994113401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e8161160 0x7f80e8074950 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f80e80b20f0 tx=0x7f80e001cda0 comp rx=0 tx=0).stop 2024-09-21T06:04:48.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.126+0000 7f80ed400640 1 -- 172.21.15.50:0/3994113401 shutdown_connections 2024-09-21T06:04:48.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.126+0000 7f80ed400640 1 --2- 172.21.15.50:0/3994113401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e8161160 0x7f80e8074950 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:48.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.126+0000 7f80ed400640 1 --2- 172.21.15.50:0/3994113401 >> v2:172.21.15.88:3300/0 conn(0x7f80e8160d50 0x7f80e8074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:48.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.126+0000 7f80ed400640 1 --2- 172.21.15.50:0/3994113401 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80e8157030 0x7f80e80b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:48.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.126+0000 7f80ed400640 1 -- 172.21.15.50:0/3994113401 >> 172.21.15.50:0/3994113401 conn(0x7f80e80708c0 msgr2=0x7f80e8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:48.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.126+0000 7f80ed400640 1 -- 172.21.15.50:0/3994113401 shutdown_connections 2024-09-21T06:04:48.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.126+0000 7f80ed400640 1 -- 172.21.15.50:0/3994113401 wait complete. 2024-09-21T06:04:48.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.126+0000 7f80ed400640 1 Processor -- start 2024-09-21T06:04:48.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.126+0000 7f80ed400640 1 -- start start 2024-09-21T06:04:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80ed400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e8157030 0x7f80e815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80ed400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80e8160d50 0x7f80e806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80ed400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80e8161160 0x7f80e806de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80ed400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f80e80722c0 con 0x7f80e8157030 2024-09-21T06:04:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80ed400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f80e8072140 con 0x7f80e8161160 2024-09-21T06:04:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80ed400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f80e8072440 con 0x7f80e8160d50 2024-09-21T06:04:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80e6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e8157030 0x7f80e815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80e6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e8157030 0x7f80e815bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38882/0 (socket says 172.21.15.50:38882) 2024-09-21T06:04:48.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80e6800640 1 -- 172.21.15.50:0/2627840662 learned_addr learned my addr 172.21.15.50:0/2627840662 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:04:48.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80e6800640 1 -- 172.21.15.50:0/2627840662 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80e8160d50 msgr2=0x7f80e806d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80e6800640 1 --2- 172.21.15.50:0/2627840662 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80e8160d50 0x7f80e806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80e6800640 1 -- 172.21.15.50:0/2627840662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80e8161160 msgr2=0x7f80e806de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80e6800640 1 --2- 172.21.15.50:0/2627840662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80e8161160 0x7f80e806de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80e6800640 1 -- 172.21.15.50:0/2627840662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e814ab70 con 0x7f80e8157030 2024-09-21T06:04:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80e6800640 1 --2- 172.21.15.50:0/2627840662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e8157030 0x7f80e815bc90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f80d800b410 tx=0x7f80d800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80dea00640 1 -- 172.21.15.50:0/2627840662 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8015020 con 0x7f80e8157030 2024-09-21T06:04:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80dea00640 1 -- 172.21.15.50:0/2627840662 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80d8011920 con 0x7f80e8157030 2024-09-21T06:04:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80ed400640 1 -- 172.21.15.50:0/2627840662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e806cdf0 con 0x7f80e8157030 2024-09-21T06:04:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80dea00640 1 -- 172.21.15.50:0/2627840662 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8011d70 con 0x7f80e8157030 2024-09-21T06:04:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80ed400640 1 -- 172.21.15.50:0/2627840662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e806d3b0 con 0x7f80e8157030 2024-09-21T06:04:48.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.130+0000 7f80ed400640 1 -- 172.21.15.50:0/2627840662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e8072970 con 0x7f80e8157030 2024-09-21T06:04:48.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.134+0000 7f80dea00640 1 -- 172.21.15.50:0/2627840662 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 28) ==== 98356+0+0 (secure 0 0 0) 0x7f80d8018020 con 0x7f80e8157030 2024-09-21T06:04:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.134+0000 7f80dea00640 1 --2- 172.21.15.50:0/2627840662 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80c4073e60 0x7f80c4076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.134+0000 7f80dea00640 1 -- 172.21.15.50:0/2627840662 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f80d8092e80 con 0x7f80e8157030 2024-09-21T06:04:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.134+0000 7f80dfe00640 1 --2- 172.21.15.50:0/2627840662 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80c4073e60 0x7f80c4076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:48.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.134+0000 7f80dfe00640 1 --2- 172.21.15.50:0/2627840662 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80c4073e60 0x7f80c4076370 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f80e815cc50 tx=0x7f80d0006c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:48.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.138+0000 7f80dea00640 1 -- 172.21.15.50:0/2627840662 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f80d801f020 con 0x7f80e8157030 2024-09-21T06:04:48.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.342+0000 7f80ed400640 1 -- 172.21.15.50:0/2627840662 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f80e814a920 con 0x7f80c4073e60 2024-09-21T06:04:48.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.346+0000 7f80dea00640 1 -- 172.21.15.50:0/2627840662 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f80e814a920 con 0x7f80c4073e60 2024-09-21T06:04:48.350 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/b8565bbf-4697-4374-a50b-f645a4f5306e 2024-09-21T06:04:48.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.350+0000 7f80ed400640 1 -- 172.21.15.50:0/2627840662 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80c4073e60 msgr2=0x7f80c4076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:48.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.350+0000 7f80ed400640 1 --2- 172.21.15.50:0/2627840662 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80c4073e60 0x7f80c4076370 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f80e815cc50 tx=0x7f80d0006c00 comp rx=0 tx=0).stop 2024-09-21T06:04:48.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.350+0000 7f80ed400640 1 -- 172.21.15.50:0/2627840662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e8157030 msgr2=0x7f80e815bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:48.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.350+0000 7f80ed400640 1 --2- 172.21.15.50:0/2627840662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e8157030 0x7f80e815bc90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f80d800b410 tx=0x7f80d800b8d0 comp rx=0 tx=0).stop 2024-09-21T06:04:48.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.350+0000 7f80ed400640 1 -- 172.21.15.50:0/2627840662 shutdown_connections 2024-09-21T06:04:48.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.350+0000 7f80ed400640 1 --2- 172.21.15.50:0/2627840662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80e8161160 0x7f80e806de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:48.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.350+0000 7f80ed400640 1 --2- 172.21.15.50:0/2627840662 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80e8160d50 0x7f80e806d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:48.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.350+0000 7f80ed400640 1 --2- 172.21.15.50:0/2627840662 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80c4073e60 0x7f80c4076370 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:48.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.350+0000 7f80ed400640 1 --2- 172.21.15.50:0/2627840662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e8157030 0x7f80e815bc90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:48.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.350+0000 7f80ed400640 1 -- 172.21.15.50:0/2627840662 >> 172.21.15.50:0/2627840662 conn(0x7f80e80708c0 msgr2=0x7f80e8151290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:48.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.350+0000 7f80ed400640 1 -- 172.21.15.50:0/2627840662 shutdown_connections 2024-09-21T06:04:48.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:04:48.350+0000 7f80ed400640 1 -- 172.21.15.50:0/2627840662 wait complete. 2024-09-21T06:04:48.373 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/b8565bbf-4697-4374-a50b-f645a4f5306e 2024-09-21T06:04:48.373 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:04:48.373 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:04:48.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:48.382 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b8565bbf-4697-4374-a50b-f645a4f5306e/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:04:57.744 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:04:57.744 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:04:57.744 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.30247 s, 115 MB/s 2024-09-21T06:04:57.745 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:04:57.745 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:04:57.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:04:57.754 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b8565bbf-4697-4374-a50b-f645a4f5306e/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:05:07.130 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:05:07.130 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:05:07.130 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.32248 s, 115 MB/s 2024-09-21T06:05:07.131 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:05:07.131 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:05:07.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:05:07.140 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b8565bbf-4697-4374-a50b-f645a4f5306e/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:05:16.715 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:05:16.715 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:05:16.715 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.5244 s, 113 MB/s 2024-09-21T06:05:16.717 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-21T06:05:16.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.878+0000 7f8f4dc00640 1 -- 172.21.15.50:0/2680777086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4815b2d0 msgr2=0x7f8f4815b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:16.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.878+0000 7f8f4dc00640 1 --2- 172.21.15.50:0/2680777086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4815b2d0 0x7f8f4815b6e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009900 tx=0x7f8f3401c8e0 comp rx=0 tx=0).stop 2024-09-21T06:05:16.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.878+0000 7f8f4dc00640 1 -- 172.21.15.50:0/2680777086 shutdown_connections 2024-09-21T06:05:16.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.878+0000 7f8f4dc00640 1 --2- 172.21.15.50:0/2680777086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4815b2d0 0x7f8f4815b6e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:16.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.878+0000 7f8f4dc00640 1 --2- 172.21.15.50:0/2680777086 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f4815e8c0 0x7f8f4815ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:16.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.878+0000 7f8f4dc00640 1 -- 172.21.15.50:0/2680777086 >> 172.21.15.50:0/2680777086 conn(0x7f8f480b2040 msgr2=0x7f8f480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:16.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.878+0000 7f8f4dc00640 1 -- 172.21.15.50:0/2680777086 shutdown_connections 2024-09-21T06:05:16.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.878+0000 7f8f4dc00640 1 -- 172.21.15.50:0/2680777086 wait complete. 2024-09-21T06:05:16.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f4dc00640 1 Processor -- start 2024-09-21T06:05:16.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f4dc00640 1 -- start start 2024-09-21T06:05:16.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f4dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f4815b2d0 0x7f8f480cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:16.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f4dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4815bc20 0x7f8f480c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:16.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f4dc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f4815e8c0 0x7f8f480cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:16.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f4dc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f480b54e0 con 0x7f8f4815bc20 2024-09-21T06:05:16.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f4dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8f480b5360 con 0x7f8f4815b2d0 2024-09-21T06:05:16.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f4dc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8f4815d990 con 0x7f8f4815e8c0 2024-09-21T06:05:16.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f46000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4815bc20 0x7f8f480c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:16.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f46000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4815bc20 0x7f8f480c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53480/0 (socket says 172.21.15.50:53480) 2024-09-21T06:05:16.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f46000640 1 -- 172.21.15.50:0/3139168470 learned_addr learned my addr 172.21.15.50:0/3139168470 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:16.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f46a00640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f4815b2d0 0x7f8f480cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:16.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f47400640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f4815e8c0 0x7f8f480cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:16.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f46000640 1 -- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f4815e8c0 msgr2=0x7f8f480cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:16.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f46000640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f4815e8c0 0x7f8f480cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:16.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f46000640 1 -- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f4815b2d0 msgr2=0x7f8f480cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:16.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f46000640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f4815b2d0 0x7f8f480cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:16.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.882+0000 7f8f46000640 1 -- 172.21.15.50:0/3139168470 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f480b1910 con 0x7f8f4815bc20 2024-09-21T06:05:16.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.886+0000 7f8f46000640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4815bc20 0x7f8f480c5c10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009850 tx=0x7f8f34005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:16.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.886+0000 7f8f3ea00640 1 -- 172.21.15.50:0/3139168470 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f340b2040 con 0x7f8f4815bc20 2024-09-21T06:05:16.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.886+0000 7f8f3ea00640 1 -- 172.21.15.50:0/3139168470 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f340bd070 con 0x7f8f4815bc20 2024-09-21T06:05:16.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.886+0000 7f8f4dc00640 1 -- 172.21.15.50:0/3139168470 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f480cd430 con 0x7f8f4815bc20 2024-09-21T06:05:16.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.886+0000 7f8f3ea00640 1 -- 172.21.15.50:0/3139168470 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34003600 con 0x7f8f4815bc20 2024-09-21T06:05:16.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.886+0000 7f8f4dc00640 1 -- 172.21.15.50:0/3139168470 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f480b1bb0 con 0x7f8f4815bc20 2024-09-21T06:05:16.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.886+0000 7f8f4dc00640 1 -- 172.21.15.50:0/3139168470 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f480cd1a0 con 0x7f8f4815bc20 2024-09-21T06:05:16.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.886+0000 7f8f3ea00640 1 -- 172.21.15.50:0/3139168470 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 29) ==== 98458+0+0 (secure 0 0 0) 0x7f8f340037a0 con 0x7f8f4815bc20 2024-09-21T06:05:16.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.886+0000 7f8f3ea00640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f28073f80 0x7f8f28076490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:16.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.886+0000 7f8f3ea00640 1 -- 172.21.15.50:0/3139168470 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f8f3412d390 con 0x7f8f4815bc20 2024-09-21T06:05:16.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.890+0000 7f8f46a00640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f28073f80 0x7f8f28076490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:16.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.890+0000 7f8f46a00640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f28073f80 0x7f8f28076490 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800bb50 tx=0x7f8f3800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:16.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:16.890+0000 7f8f3ea00640 1 -- 172.21.15.50:0/3139168470 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8f340c7020 con 0x7f8f4815bc20 2024-09-21T06:05:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.098+0000 7f8f4dc00640 1 -- 172.21.15.50:0/3139168470 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f8f4806b540 con 0x7f8f28073f80 2024-09-21T06:05:17.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.110+0000 7f8f3ea00640 1 -- 172.21.15.50:0/3139168470 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8f38015070 con 0x7f8f28073f80 2024-09-21T06:05:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.114+0000 7f8f4dc00640 1 -- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f28073f80 msgr2=0x7f8f28076490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.114+0000 7f8f4dc00640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f28073f80 0x7f8f28076490 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800bb50 tx=0x7f8f3800b2e0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.114+0000 7f8f4dc00640 1 -- 172.21.15.50:0/3139168470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4815bc20 msgr2=0x7f8f480c5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.114+0000 7f8f4dc00640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4815bc20 0x7f8f480c5c10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009850 tx=0x7f8f34005de0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.114+0000 7f8f4dc00640 1 -- 172.21.15.50:0/3139168470 shutdown_connections 2024-09-21T06:05:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.114+0000 7f8f4dc00640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f4815e8c0 0x7f8f480cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.114+0000 7f8f4dc00640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f28073f80 0x7f8f28076490 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.114+0000 7f8f4dc00640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f4815bc20 0x7f8f480c5c10 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.114+0000 7f8f4dc00640 1 --2- 172.21.15.50:0/3139168470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f4815b2d0 0x7f8f480cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.114+0000 7f8f4dc00640 1 -- 172.21.15.50:0/3139168470 >> 172.21.15.50:0/3139168470 conn(0x7f8f480b2040 msgr2=0x7f8f480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.114+0000 7f8f4dc00640 1 -- 172.21.15.50:0/3139168470 shutdown_connections 2024-09-21T06:05:17.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.114+0000 7f8f4dc00640 1 -- 172.21.15.50:0/3139168470 wait complete. 2024-09-21T06:05:17.138 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:05:17.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 -- 172.21.15.50:0/318021770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fe0160d50 msgr2=0x7f2fe0074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:17.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 --2- 172.21.15.50:0/318021770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fe0160d50 0x7f2fe0074410 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009900 tx=0x7f2fc801c8e0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 -- 172.21.15.50:0/318021770 shutdown_connections 2024-09-21T06:05:17.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 --2- 172.21.15.50:0/318021770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fe0160d50 0x7f2fe0074410 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 --2- 172.21.15.50:0/318021770 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2fe0157030 0x7f2fe00b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 -- 172.21.15.50:0/318021770 >> 172.21.15.50:0/318021770 conn(0x7f2fe00708c0 msgr2=0x7f2fe0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:17.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 -- 172.21.15.50:0/318021770 shutdown_connections 2024-09-21T06:05:17.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 -- 172.21.15.50:0/318021770 wait complete. 2024-09-21T06:05:17.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 Processor -- start 2024-09-21T06:05:17.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 -- start start 2024-09-21T06:05:17.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fe0157030 0x7f2fe01566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:17.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fe0160d50 0x7f2fe0142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:17.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2fe0161160 0x7f2fe00d0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:17.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe016a850 con 0x7f2fe0160d50 2024-09-21T06:05:17.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe016a6d0 con 0x7f2fe0157030 2024-09-21T06:05:17.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.298+0000 7f2fe5800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe016a9d0 con 0x7f2fe0161160 2024-09-21T06:05:17.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fd7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fe0160d50 0x7f2fe0142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:17.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fdea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fe0157030 0x7f2fe01566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:17.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fdea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fe0157030 0x7f2fe01566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:33216/0 (socket says 172.21.15.50:33216) 2024-09-21T06:05:17.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fd7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fe0160d50 0x7f2fe0142aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53492/0 (socket says 172.21.15.50:53492) 2024-09-21T06:05:17.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fdea00640 1 -- 172.21.15.50:0/1662783280 learned_addr learned my addr 172.21.15.50:0/1662783280 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:17.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fdf400640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2fe0161160 0x7f2fe00d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:17.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fd7e00640 1 -- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2fe0161160 msgr2=0x7f2fe00d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:17.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fd7e00640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2fe0161160 0x7f2fe00d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fd7e00640 1 -- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fe0157030 msgr2=0x7f2fe01566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:17.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fd7e00640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fe0157030 0x7f2fe01566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fd7e00640 1 -- 172.21.15.50:0/1662783280 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe01537b0 con 0x7f2fe0160d50 2024-09-21T06:05:17.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fd7e00640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fe0160d50 0x7f2fe0142aa0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00d2c80 tx=0x7f2fc8005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:17.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fd6a00640 1 -- 172.21.15.50:0/1662783280 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80b2040 con 0x7f2fe0160d50 2024-09-21T06:05:17.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fd6a00640 1 -- 172.21.15.50:0/1662783280 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fc8004340 con 0x7f2fe0160d50 2024-09-21T06:05:17.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fe5800640 1 -- 172.21.15.50:0/1662783280 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe0143480 con 0x7f2fe0160d50 2024-09-21T06:05:17.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fd6a00640 1 -- 172.21.15.50:0/1662783280 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8003860 con 0x7f2fe0160d50 2024-09-21T06:05:17.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fe5800640 1 -- 172.21.15.50:0/1662783280 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe0153a50 con 0x7f2fe0160d50 2024-09-21T06:05:17.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fe5800640 1 -- 172.21.15.50:0/1662783280 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe0156490 con 0x7f2fe0160d50 2024-09-21T06:05:17.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.302+0000 7f2fd6a00640 1 -- 172.21.15.50:0/1662783280 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 29) ==== 98458+0+0 (secure 0 0 0) 0x7f2fc80be060 con 0x7f2fe0160d50 2024-09-21T06:05:17.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.306+0000 7f2fd6a00640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2fbc073f80 0x7f2fbc076490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:17.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.306+0000 7f2fd6a00640 1 -- 172.21.15.50:0/1662783280 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f2fc812c2d0 con 0x7f2fe0160d50 2024-09-21T06:05:17.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.306+0000 7f2fdea00640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2fbc073f80 0x7f2fbc076490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:17.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.306+0000 7f2fdea00640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2fbc073f80 0x7f2fbc076490 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009c40 tx=0x7f2fd00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:17.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.310+0000 7f2fd6a00640 1 -- 172.21.15.50:0/1662783280 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2fc80164d0 con 0x7f2fe0160d50 2024-09-21T06:05:17.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.510+0000 7f2fe5800640 1 -- 172.21.15.50:0/1662783280 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f2fe0143230 con 0x7f2fbc073f80 2024-09-21T06:05:17.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.514+0000 7f2fd6a00640 1 -- 172.21.15.50:0/1662783280 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f2fe0143230 con 0x7f2fbc073f80 2024-09-21T06:05:17.516 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/b8565bbf-4697-4374-a50b-f645a4f5306e 2024-09-21T06:05:17.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.518+0000 7f2fe5800640 1 -- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2fbc073f80 msgr2=0x7f2fbc076490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:17.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.518+0000 7f2fe5800640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2fbc073f80 0x7f2fbc076490 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009c40 tx=0x7f2fd00091e0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.518+0000 7f2fe5800640 1 -- 172.21.15.50:0/1662783280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fe0160d50 msgr2=0x7f2fe0142aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:17.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.518+0000 7f2fe5800640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fe0160d50 0x7f2fe0142aa0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00d2c80 tx=0x7f2fc8005e00 comp rx=0 tx=0).stop 2024-09-21T06:05:17.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.518+0000 7f2fe5800640 1 -- 172.21.15.50:0/1662783280 shutdown_connections 2024-09-21T06:05:17.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.518+0000 7f2fe5800640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2fe0161160 0x7f2fe00d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.518+0000 7f2fe5800640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2fbc073f80 0x7f2fbc076490 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.518+0000 7f2fe5800640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fe0160d50 0x7f2fe0142aa0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.518+0000 7f2fe5800640 1 --2- 172.21.15.50:0/1662783280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fe0157030 0x7f2fe01566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:17.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.518+0000 7f2fe5800640 1 -- 172.21.15.50:0/1662783280 >> 172.21.15.50:0/1662783280 conn(0x7f2fe00708c0 msgr2=0x7f2fe00b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:17.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.518+0000 7f2fe5800640 1 -- 172.21.15.50:0/1662783280 shutdown_connections 2024-09-21T06:05:17.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:17.518+0000 7f2fe5800640 1 -- 172.21.15.50:0/1662783280 wait complete. 2024-09-21T06:05:17.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:05:17.539 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/b8565bbf-4697-4374-a50b-f645a4f5306e 2024-09-21T06:05:17.539 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:05:17.549 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/b8565bbf-4697-4374-a50b-f645a4f5306e 2024-09-21T06:05:17.549 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="3055190016" 2024-09-21T06:05:17.549 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:17.550 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-21T06:05:17.821 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:05:17.824 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:05:19.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:05:19.551 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/b8565bbf-4697-4374-a50b-f645a4f5306e 2024-09-21T06:05:19.551 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:05:19.563 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/b8565bbf-4697-4374-a50b-f645a4f5306e 2024-09-21T06:05:19.563 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="3221225472" 2024-09-21T06:05:19.563 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:19.564 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-21T06:05:19.564 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone91230 2024-09-21T06:05:19.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.782+0000 7fef2b200640 1 -- 172.21.15.50:0/3905198437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24160470 msgr2=0x7fef24074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:19.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.782+0000 7fef2b200640 1 --2- 172.21.15.50:0/3905198437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24160470 0x7fef24074780 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fef240ced40 tx=0x7fef2001ccf0 comp rx=0 tx=0).stop 2024-09-21T06:05:19.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.782+0000 7fef2b200640 1 -- 172.21.15.50:0/3905198437 shutdown_connections 2024-09-21T06:05:19.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.782+0000 7fef2b200640 1 --2- 172.21.15.50:0/3905198437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24160470 0x7fef24074780 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:19.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.782+0000 7fef2b200640 1 --2- 172.21.15.50:0/3905198437 >> v2:172.21.15.88:3300/0 conn(0x7fef24160060 0x7fef24074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:19.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.782+0000 7fef2b200640 1 -- 172.21.15.50:0/3905198437 >> 172.21.15.50:0/3905198437 conn(0x7fef240b4560 msgr2=0x7fef240d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:19.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.782+0000 7fef2b200640 1 -- 172.21.15.50:0/3905198437 shutdown_connections 2024-09-21T06:05:19.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.782+0000 7fef2b200640 1 -- 172.21.15.50:0/3905198437 wait complete. 2024-09-21T06:05:19.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef2b200640 1 Processor -- start 2024-09-21T06:05:19.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef2b200640 1 -- start start 2024-09-21T06:05:19.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef2b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef241536f0 0x7fef2414f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:19.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef2b200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef24160060 0x7fef24154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:19.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef2b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef24160470 0x7fef241610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:19.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef2b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef241696e0 con 0x7fef241536f0 2024-09-21T06:05:19.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef2b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fef24169560 con 0x7fef24160470 2024-09-21T06:05:19.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef2b200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fef24169860 con 0x7fef24160060 2024-09-21T06:05:19.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef29e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef241536f0 0x7fef2414f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:19.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef29e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef241536f0 0x7fef2414f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53508/0 (socket says 172.21.15.50:53508) 2024-09-21T06:05:19.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef29e00640 1 -- 172.21.15.50:0/2486058729 learned_addr learned my addr 172.21.15.50:0/2486058729 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:19.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef2a800640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef24160470 0x7fef241610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:19.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.786+0000 7fef29400640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef24160060 0x7fef24154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:19.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef29e00640 1 -- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef24160060 msgr2=0x7fef24154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:19.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef29e00640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef24160060 0x7fef24154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:19.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef29e00640 1 -- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef24160470 msgr2=0x7fef241610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:19.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef29e00640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef24160470 0x7fef241610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:19.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef29e00640 1 -- 172.21.15.50:0/2486058729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef24162820 con 0x7fef241536f0 2024-09-21T06:05:19.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef29e00640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef241536f0 0x7fef2414f480 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fef1800b3e0 tx=0x7fef1800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:19.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef1e000640 1 -- 172.21.15.50:0/2486058729 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18015020 con 0x7fef241536f0 2024-09-21T06:05:19.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef1e000640 1 -- 172.21.15.50:0/2486058729 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef18011940 con 0x7fef241536f0 2024-09-21T06:05:19.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef1e000640 1 -- 172.21.15.50:0/2486058729 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18011d90 con 0x7fef241536f0 2024-09-21T06:05:19.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef2b200640 1 -- 172.21.15.50:0/2486058729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2414f9c0 con 0x7fef241536f0 2024-09-21T06:05:19.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef2b200640 1 -- 172.21.15.50:0/2486058729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef24162540 con 0x7fef241536f0 2024-09-21T06:05:19.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef2b200640 1 -- 172.21.15.50:0/2486058729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef2415d480 con 0x7fef241536f0 2024-09-21T06:05:19.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.790+0000 7fef1e000640 1 -- 172.21.15.50:0/2486058729 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 29) ==== 98458+0+0 (secure 0 0 0) 0x7fef18018020 con 0x7fef241536f0 2024-09-21T06:05:19.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.794+0000 7fef1e000640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef08073fb0 0x7fef080764c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:19.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.794+0000 7fef1e000640 1 -- 172.21.15.50:0/2486058729 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fef18092e90 con 0x7fef241536f0 2024-09-21T06:05:19.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.794+0000 7fef29400640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef08073fb0 0x7fef080764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:19.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.794+0000 7fef29400640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef08073fb0 0x7fef080764c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fef14005fd0 tx=0x7fef14005d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:19.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:19.798+0000 7fef1e000640 1 -- 172.21.15.50:0/2486058729 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fef1801f020 con 0x7fef241536f0 2024-09-21T06:05:20.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.010+0000 7fef2b200640 1 -- 172.21.15.50:0/2486058729 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone91230", "target": ["mon-mgr", ""]}) -- 0x7fef2406b540 con 0x7fef08073fb0 2024-09-21T06:05:20.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef1e000640 1 -- 172.21.15.50:0/2486058729 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fef1400ea60 con 0x7fef08073fb0 2024-09-21T06:05:20.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef2b200640 1 -- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef08073fb0 msgr2=0x7fef080764c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:20.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef2b200640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef08073fb0 0x7fef080764c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fef14005fd0 tx=0x7fef14005d10 comp rx=0 tx=0).stop 2024-09-21T06:05:20.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef2b200640 1 -- 172.21.15.50:0/2486058729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef241536f0 msgr2=0x7fef2414f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:20.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef2b200640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef241536f0 0x7fef2414f480 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fef1800b3e0 tx=0x7fef1800b8a0 comp rx=0 tx=0).stop 2024-09-21T06:05:20.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef2b200640 1 -- 172.21.15.50:0/2486058729 shutdown_connections 2024-09-21T06:05:20.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef2b200640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef24160470 0x7fef241610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:20.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef2b200640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef24160060 0x7fef24154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:20.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef2b200640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef08073fb0 0x7fef080764c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:20.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef2b200640 1 --2- 172.21.15.50:0/2486058729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef241536f0 0x7fef2414f480 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:20.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef2b200640 1 -- 172.21.15.50:0/2486058729 >> 172.21.15.50:0/2486058729 conn(0x7fef240b4560 msgr2=0x7fef24152870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:20.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef2b200640 1 -- 172.21.15.50:0/2486058729 shutdown_connections 2024-09-21T06:05:20.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.082+0000 7fef2b200640 1 -- 172.21.15.50:0/2486058729 wait complete. 2024-09-21T06:05:20.088 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:20.090+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:20.088 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:20.090+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:20.088 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:20.090+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:20.088 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:20.090+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:20.089 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:20.090+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:20.107 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone15181 2024-09-21T06:05:20.108 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:20.110+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:20.109 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:20.110+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:20.109 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:20.110+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:20.109 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:20.110+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:20.109 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:20.110+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:20.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.282+0000 7fb236800640 1 -- 172.21.15.50:0/1073797814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2300b3130 msgr2=0x7fb2300b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:20.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.282+0000 7fb236800640 1 --2- 172.21.15.50:0/1073797814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2300b3130 0x7fb2300b3540 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb228009c20 tx=0x7fb22801ca20 comp rx=0 tx=0).stop 2024-09-21T06:05:20.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.282+0000 7fb236800640 1 -- 172.21.15.50:0/1073797814 shutdown_connections 2024-09-21T06:05:20.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.282+0000 7fb236800640 1 --2- 172.21.15.50:0/1073797814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2300b3130 0x7fb2300b3540 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:20.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.282+0000 7fb236800640 1 -- 172.21.15.50:0/1073797814 >> 172.21.15.50:0/1073797814 conn(0x7fb23006cdf0 msgr2=0x7fb23006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:20.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.282+0000 7fb236800640 1 -- 172.21.15.50:0/1073797814 shutdown_connections 2024-09-21T06:05:20.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.282+0000 7fb236800640 1 -- 172.21.15.50:0/1073797814 wait complete. 2024-09-21T06:05:20.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb236800640 1 Processor -- start 2024-09-21T06:05:20.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb236800640 1 -- start start 2024-09-21T06:05:20.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb236800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2300b1dc0 0x7fb23014bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:20.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb236800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2300b3130 0x7fb2300ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:20.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb236800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb230155a00 0x7fb2300aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:20.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb236800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb23016a080 con 0x7fb2300b1dc0 2024-09-21T06:05:20.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb236800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb230169f00 con 0x7fb230155a00 2024-09-21T06:05:20.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb236800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb23016a200 con 0x7fb2300b3130 2024-09-21T06:05:20.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb235400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2300b1dc0 0x7fb23014bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:20.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb235400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2300b1dc0 0x7fb23014bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53518/0 (socket says 172.21.15.50:53518) 2024-09-21T06:05:20.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb235400640 1 -- 172.21.15.50:0/2803979308 learned_addr learned my addr 172.21.15.50:0/2803979308 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:20.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb235e00640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb230155a00 0x7fb2300aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:20.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb234a00640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2300b3130 0x7fb2300ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:20.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb235400640 1 -- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2300b3130 msgr2=0x7fb2300ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:20.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb235400640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2300b3130 0x7fb2300ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:20.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb235400640 1 -- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb230155a00 msgr2=0x7fb2300aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:20.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb235400640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb230155a00 0x7fb2300aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:20.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb235400640 1 -- 172.21.15.50:0/2803979308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2301fcfb0 con 0x7fb2300b1dc0 2024-09-21T06:05:20.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb234a00640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2300b3130 0x7fb2300ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:20.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb235400640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2300b1dc0 0x7fb23014bcd0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb22400c400 tx=0x7fb22400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:20.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb22d600640 1 -- 172.21.15.50:0/2803979308 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224017070 con 0x7fb2300b1dc0 2024-09-21T06:05:20.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb22d600640 1 -- 172.21.15.50:0/2803979308 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb224005400 con 0x7fb2300b1dc0 2024-09-21T06:05:20.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb236800640 1 -- 172.21.15.50:0/2803979308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2300cd480 con 0x7fb2300b1dc0 2024-09-21T06:05:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb22d600640 1 -- 172.21.15.50:0/2803979308 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224003860 con 0x7fb2300b1dc0 2024-09-21T06:05:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.286+0000 7fb236800640 1 -- 172.21.15.50:0/2803979308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2300cd720 con 0x7fb2300b1dc0 2024-09-21T06:05:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.290+0000 7fb236800640 1 -- 172.21.15.50:0/2803979308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2300b3540 con 0x7fb2300b1dc0 2024-09-21T06:05:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.290+0000 7fb22d600640 1 -- 172.21.15.50:0/2803979308 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 29) ==== 98458+0+0 (secure 0 0 0) 0x7fb22401a070 con 0x7fb2300b1dc0 2024-09-21T06:05:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.290+0000 7fb22d600640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb210073fb0 0x7fb2100764c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.290+0000 7fb22d600640 1 -- 172.21.15.50:0/2803979308 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fb224092250 con 0x7fb2300b1dc0 2024-09-21T06:05:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.290+0000 7fb234a00640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb210073fb0 0x7fb2100764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:20.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.290+0000 7fb234a00640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb210073fb0 0x7fb2100764c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb21c000c10 tx=0x7fb21c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:20.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.294+0000 7fb22d600640 1 -- 172.21.15.50:0/2803979308 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb22405e3d0 con 0x7fb2300b1dc0 2024-09-21T06:05:20.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:20.502+0000 7fb236800640 1 -- 172.21.15.50:0/2803979308 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone15181", "target": ["mon-mgr", ""]}) -- 0x7fb23006b540 con 0x7fb210073fb0 2024-09-21T06:05:21.088 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:21.090+0000 7f6596c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:21.088 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:21.090+0000 7f6596c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:21.088 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:21.090+0000 7f6596c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:21.088 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:21.090+0000 7f6596c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:21.088 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:21.090+0000 7f6596c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.178+0000 7fb22d600640 1 -- 172.21.15.50:0/2803979308 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb21c001b40 con 0x7fb210073fb0 2024-09-21T06:05:21.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.182+0000 7fb236800640 1 -- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb210073fb0 msgr2=0x7fb2100764c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:21.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.182+0000 7fb236800640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb210073fb0 0x7fb2100764c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb21c000c10 tx=0x7fb21c009040 comp rx=0 tx=0).stop 2024-09-21T06:05:21.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.182+0000 7fb236800640 1 -- 172.21.15.50:0/2803979308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2300b1dc0 msgr2=0x7fb23014bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:21.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.182+0000 7fb236800640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2300b1dc0 0x7fb23014bcd0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb22400c400 tx=0x7fb22400c8c0 comp rx=0 tx=0).stop 2024-09-21T06:05:21.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.182+0000 7fb236800640 1 -- 172.21.15.50:0/2803979308 shutdown_connections 2024-09-21T06:05:21.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.182+0000 7fb236800640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb230155a00 0x7fb2300aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:21.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.182+0000 7fb236800640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2300b3130 0x7fb2300ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:21.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.182+0000 7fb236800640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb210073fb0 0x7fb2100764c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:21.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.182+0000 7fb236800640 1 --2- 172.21.15.50:0/2803979308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2300b1dc0 0x7fb23014bcd0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:21.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.182+0000 7fb236800640 1 -- 172.21.15.50:0/2803979308 >> 172.21.15.50:0/2803979308 conn(0x7fb23006cdf0 msgr2=0x7fb23006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:21.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.182+0000 7fb236800640 1 -- 172.21.15.50:0/2803979308 shutdown_connections 2024-09-21T06:05:21.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.182+0000 7fb236800640 1 -- 172.21.15.50:0/2803979308 wait complete. 2024-09-21T06:05:21.223 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone51512 2024-09-21T06:05:21.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.482+0000 7f5eada00640 1 -- 172.21.15.50:0/3317660298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea80b3130 msgr2=0x7f5ea80b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:21.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.482+0000 7f5eada00640 1 --2- 172.21.15.50:0/3317660298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea80b3130 0x7f5ea80b3540 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009900 tx=0x7f5ea001c8b0 comp rx=0 tx=0).stop 2024-09-21T06:05:21.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.482+0000 7f5eada00640 1 -- 172.21.15.50:0/3317660298 shutdown_connections 2024-09-21T06:05:21.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.482+0000 7f5eada00640 1 --2- 172.21.15.50:0/3317660298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea80b3130 0x7f5ea80b3540 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:21.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.482+0000 7f5eada00640 1 --2- 172.21.15.50:0/3317660298 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5ea80b1dc0 0x7f5ea80b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:21.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.482+0000 7f5eada00640 1 -- 172.21.15.50:0/3317660298 >> 172.21.15.50:0/3317660298 conn(0x7f5ea806cdf0 msgr2=0x7f5ea806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:21.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.482+0000 7f5eada00640 1 -- 172.21.15.50:0/3317660298 shutdown_connections 2024-09-21T06:05:21.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.482+0000 7f5eada00640 1 -- 172.21.15.50:0/3317660298 wait complete. 2024-09-21T06:05:21.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.486+0000 7f5eada00640 1 Processor -- start 2024-09-21T06:05:21.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.486+0000 7f5eada00640 1 -- start start 2024-09-21T06:05:21.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5eada00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea80b1dc0 0x7f5ea80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:21.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5eada00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea80b3130 0x7f5ea80d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:21.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5eada00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5ea8155a00 0x7f5ea814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:21.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5eada00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea8168e00 con 0x7f5ea80b3130 2024-09-21T06:05:21.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5eada00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea8168c80 con 0x7f5ea80b1dc0 2024-09-21T06:05:21.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5eada00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea8168f80 con 0x7f5ea8155a00 2024-09-21T06:05:21.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5e9f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea80b3130 0x7f5ea80d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:21.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5e9f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea80b3130 0x7f5ea80d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53538/0 (socket says 172.21.15.50:53538) 2024-09-21T06:05:21.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5e9f400640 1 -- 172.21.15.50:0/2393848769 learned_addr learned my addr 172.21.15.50:0/2393848769 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:21.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5ea7e00640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea80b1dc0 0x7f5ea80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:21.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5e9f400640 1 -- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5ea8155a00 msgr2=0x7f5ea814bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:21.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5e9f400640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5ea8155a00 0x7f5ea814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:21.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5e9f400640 1 -- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea80b1dc0 msgr2=0x7f5ea80cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:21.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5e9f400640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea80b1dc0 0x7f5ea80cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:21.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5e9f400640 1 -- 172.21.15.50:0/2393848769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea80cdc50 con 0x7f5ea80b3130 2024-09-21T06:05:21.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5ea7e00640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea80b1dc0 0x7f5ea80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:21.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5e9f400640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea80b3130 0x7f5ea80d03c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000e7f0 tx=0x7f5e9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:21.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5ea4c00640 1 -- 172.21.15.50:0/2393848769 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9000ccc0 con 0x7f5ea80b3130 2024-09-21T06:05:21.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5eada00640 1 -- 172.21.15.50:0/2393848769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea80cdf30 con 0x7f5ea80b3130 2024-09-21T06:05:21.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5eada00640 1 -- 172.21.15.50:0/2393848769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea80cfbc0 con 0x7f5ea80b3130 2024-09-21T06:05:21.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5ea4c00640 1 -- 172.21.15.50:0/2393848769 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e9000ce60 con 0x7f5ea80b3130 2024-09-21T06:05:21.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.490+0000 7f5ea4c00640 1 -- 172.21.15.50:0/2393848769 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90005240 con 0x7f5ea80b3130 2024-09-21T06:05:21.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.494+0000 7f5eada00640 1 -- 172.21.15.50:0/2393848769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e800052a0 con 0x7f5ea80b3130 2024-09-21T06:05:21.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.498+0000 7f5ea4c00640 1 -- 172.21.15.50:0/2393848769 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 29) ==== 98458+0+0 (secure 0 0 0) 0x7f5e900053e0 con 0x7f5ea80b3130 2024-09-21T06:05:21.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.498+0000 7f5ea4c00640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5e84073f80 0x7f5e84076490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:21.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.498+0000 7f5ea4c00640 1 -- 172.21.15.50:0/2393848769 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f5e90093bd0 con 0x7f5ea80b3130 2024-09-21T06:05:21.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.502+0000 7f5ea7e00640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5e84073f80 0x7f5e84076490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:21.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.502+0000 7f5ea7e00640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5e84073f80 0x7f5e84076490 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5e98007fd0 tx=0x7f5e98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:21.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.506+0000 7f5ea4c00640 1 -- 172.21.15.50:0/2393848769 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5e90060480 con 0x7f5ea80b3130 2024-09-21T06:05:21.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.690+0000 7f5ea4c00640 1 -- 172.21.15.50:0/2393848769 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 30) ==== 98560+0+0 (secure 0 0 0) 0x7f5e9005bff0 con 0x7f5ea80b3130 2024-09-21T06:05:21.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:21.854+0000 7f5eada00640 1 -- 172.21.15.50:0/2393848769 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone51512", "target": ["mon-mgr", ""]}) -- 0x7f5e80000c90 con 0x7f5e84073f80 2024-09-21T06:05:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.574+0000 7f5ea4c00640 1 -- 172.21.15.50:0/2393848769 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5e80000c90 con 0x7f5e84073f80 2024-09-21T06:05:22.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.582+0000 7f5eada00640 1 -- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5e84073f80 msgr2=0x7f5e84076490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:22.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.582+0000 7f5eada00640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5e84073f80 0x7f5e84076490 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5e98007fd0 tx=0x7f5e98009040 comp rx=0 tx=0).stop 2024-09-21T06:05:22.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.582+0000 7f5eada00640 1 -- 172.21.15.50:0/2393848769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea80b3130 msgr2=0x7f5ea80d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:22.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.582+0000 7f5eada00640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea80b3130 0x7f5ea80d03c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000e7f0 tx=0x7f5e9000ecb0 comp rx=0 tx=0).stop 2024-09-21T06:05:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.582+0000 7f5eada00640 1 -- 172.21.15.50:0/2393848769 shutdown_connections 2024-09-21T06:05:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.582+0000 7f5eada00640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5ea8155a00 0x7f5ea814bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.582+0000 7f5eada00640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5e84073f80 0x7f5e84076490 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.582+0000 7f5eada00640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea80b3130 0x7f5ea80d03c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.582+0000 7f5eada00640 1 --2- 172.21.15.50:0/2393848769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea80b1dc0 0x7f5ea80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.582+0000 7f5eada00640 1 -- 172.21.15.50:0/2393848769 >> 172.21.15.50:0/2393848769 conn(0x7f5ea806cdf0 msgr2=0x7f5ea814c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.582+0000 7f5eada00640 1 -- 172.21.15.50:0/2393848769 shutdown_connections 2024-09-21T06:05:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.586+0000 7f5eada00640 1 -- 172.21.15.50:0/2393848769 wait complete. 2024-09-21T06:05:22.631 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone26043 2024-09-21T06:05:22.646 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:22.646+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:22.646 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:22.646+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:22.646 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:22.646+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:22.646 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:22.646+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:22.646 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:22.646+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:22.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.898+0000 7f0585600640 1 -- 172.21.15.50:0/40191698 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f058006efa0 msgr2=0x7f058006f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:22.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.898+0000 7f0585600640 1 --2- 172.21.15.50:0/40191698 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f058006efa0 0x7f058006f3b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0578009bc0 tx=0x7f057801cb70 comp rx=0 tx=0).stop 2024-09-21T06:05:22.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.902+0000 7f0585600640 1 -- 172.21.15.50:0/40191698 shutdown_connections 2024-09-21T06:05:22.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.902+0000 7f0585600640 1 --2- 172.21.15.50:0/40191698 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f058006efa0 0x7f058006f3b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:22.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.902+0000 7f0585600640 1 --2- 172.21.15.50:0/40191698 >> v2:172.21.15.88:3300/0 conn(0x7f05800744d0 0x7f058006ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:22.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.902+0000 7f0585600640 1 -- 172.21.15.50:0/40191698 >> 172.21.15.50:0/40191698 conn(0x7f05800b2c40 msgr2=0x7f05800b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:22.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.902+0000 7f0585600640 1 -- 172.21.15.50:0/40191698 shutdown_connections 2024-09-21T06:05:22.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.902+0000 7f0585600640 1 -- 172.21.15.50:0/40191698 wait complete. 2024-09-21T06:05:22.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.902+0000 7f0585600640 1 Processor -- start 2024-09-21T06:05:22.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.902+0000 7f0585600640 1 -- start start 2024-09-21T06:05:22.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.906+0000 7f0585600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f058006efa0 0x7f058015a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:22.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.906+0000 7f0585600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f058015c5d0 0x7f058014e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:22.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.906+0000 7f0585600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0580200a60 0x7f05800cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:22.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.906+0000 7f0585600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0580073040 con 0x7f0580200a60 2024-09-21T06:05:22.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.906+0000 7f0585600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0580072ec0 con 0x7f058015c5d0 2024-09-21T06:05:22.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.906+0000 7f0585600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f05800731c0 con 0x7f058006efa0 2024-09-21T06:05:22.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.906+0000 7f057f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0580200a60 0x7f05800cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:22.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f057f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0580200a60 0x7f05800cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53550/0 (socket says 172.21.15.50:53550) 2024-09-21T06:05:22.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f057f400640 1 -- 172.21.15.50:0/3018599296 learned_addr learned my addr 172.21.15.50:0/3018599296 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:22.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f057f400640 1 -- 172.21.15.50:0/3018599296 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f058006efa0 msgr2=0x7f058015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:22.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f057f400640 1 --2- 172.21.15.50:0/3018599296 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f058006efa0 0x7f058015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:22.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f057f400640 1 -- 172.21.15.50:0/3018599296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f058015c5d0 msgr2=0x7f058014e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:22.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f057f400640 1 --2- 172.21.15.50:0/3018599296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f058015c5d0 0x7f058014e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:22.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f057f400640 1 -- 172.21.15.50:0/3018599296 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058015ace0 con 0x7f0580200a60 2024-09-21T06:05:22.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f057f400640 1 --2- 172.21.15.50:0/3018599296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0580200a60 0x7f05800cdaf0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f057801a7a0 tx=0x7f0578009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:22.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f0576a00640 1 -- 172.21.15.50:0/3018599296 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05780bb070 con 0x7f0580200a60 2024-09-21T06:05:22.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f0576a00640 1 -- 172.21.15.50:0/3018599296 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0578004d40 con 0x7f0580200a60 2024-09-21T06:05:22.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f0585600640 1 -- 172.21.15.50:0/3018599296 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0580152780 con 0x7f0580200a60 2024-09-21T06:05:22.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f0576a00640 1 -- 172.21.15.50:0/3018599296 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05780b2040 con 0x7f0580200a60 2024-09-21T06:05:22.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f0585600640 1 -- 172.21.15.50:0/3018599296 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f058014edd0 con 0x7f0580200a60 2024-09-21T06:05:22.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.910+0000 7f0585600640 1 -- 172.21.15.50:0/3018599296 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f058006f3b0 con 0x7f0580200a60 2024-09-21T06:05:22.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.914+0000 7f0576a00640 1 -- 172.21.15.50:0/3018599296 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 30) ==== 98560+0+0 (secure 0 0 0) 0x7f05780be070 con 0x7f0580200a60 2024-09-21T06:05:22.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.918+0000 7f0576a00640 1 --2- 172.21.15.50:0/3018599296 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f05600740a0 0x7f05600765b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:22.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.918+0000 7f0576a00640 1 -- 172.21.15.50:0/3018599296 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f057812cb00 con 0x7f0580200a60 2024-09-21T06:05:22.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.926+0000 7f0576a00640 1 -- 172.21.15.50:0/3018599296 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0578016790 con 0x7f0580200a60 2024-09-21T06:05:22.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.926+0000 7f057ea00640 1 --2- 172.21.15.50:0/3018599296 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f05600740a0 0x7f05600765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:22.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:22.930+0000 7f057ea00640 1 --2- 172.21.15.50:0/3018599296 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f05600740a0 0x7f05600765b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f05700046c0 tx=0x7f0570007910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:23.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.270+0000 7f0585600640 1 -- 172.21.15.50:0/3018599296 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone26043", "target": ["mon-mgr", ""]}) -- 0x7f058015aa70 con 0x7f05600740a0 2024-09-21T06:05:23.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.766+0000 7f0576a00640 1 -- 172.21.15.50:0/3018599296 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 31) ==== 98662+0+0 (secure 0 0 0) 0x7f05780f4e60 con 0x7f0580200a60 2024-09-21T06:05:23.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.914+0000 7f0576a00640 1 -- 172.21.15.50:0/3018599296 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f058015aa70 con 0x7f05600740a0 2024-09-21T06:05:23.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.918+0000 7f0585600640 1 -- 172.21.15.50:0/3018599296 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f05600740a0 msgr2=0x7f05600765b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:23.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.918+0000 7f0585600640 1 --2- 172.21.15.50:0/3018599296 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f05600740a0 0x7f05600765b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f05700046c0 tx=0x7f0570007910 comp rx=0 tx=0).stop 2024-09-21T06:05:23.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.918+0000 7f0585600640 1 -- 172.21.15.50:0/3018599296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0580200a60 msgr2=0x7f05800cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:23.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.918+0000 7f0585600640 1 --2- 172.21.15.50:0/3018599296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0580200a60 0x7f05800cdaf0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f057801a7a0 tx=0x7f0578009260 comp rx=0 tx=0).stop 2024-09-21T06:05:23.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.918+0000 7f0585600640 1 -- 172.21.15.50:0/3018599296 shutdown_connections 2024-09-21T06:05:23.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.918+0000 7f0585600640 1 --2- 172.21.15.50:0/3018599296 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f05600740a0 0x7f05600765b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:23.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.918+0000 7f0585600640 1 --2- 172.21.15.50:0/3018599296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0580200a60 0x7f05800cdaf0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:23.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.918+0000 7f0585600640 1 --2- 172.21.15.50:0/3018599296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f058015c5d0 0x7f058014e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:23.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.918+0000 7f0585600640 1 --2- 172.21.15.50:0/3018599296 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f058006efa0 0x7f058015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:23.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.918+0000 7f0585600640 1 -- 172.21.15.50:0/3018599296 >> 172.21.15.50:0/3018599296 conn(0x7f05800b2c40 msgr2=0x7f058015cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:23.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.918+0000 7f0585600640 1 -- 172.21.15.50:0/3018599296 shutdown_connections 2024-09-21T06:05:23.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:23.922+0000 7f0585600640 1 -- 172.21.15.50:0/3018599296 wait complete. 2024-09-21T06:05:23.965 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone93454 2024-09-21T06:05:23.969 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:23.966+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:23.969 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:23.966+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:23.969 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:23.966+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:23.969 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:23.966+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:23.969 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:23.966+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:24.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.246+0000 7fa37f200640 1 -- 172.21.15.50:0/715900370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38006ac60 msgr2=0x7fa3800af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:24.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.246+0000 7fa37f200640 1 --2- 172.21.15.50:0/715900370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38006ac60 0x7fa3800af730 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009930 tx=0x7fa36c01c850 comp rx=0 tx=0).stop 2024-09-21T06:05:24.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.246+0000 7fa37f200640 1 -- 172.21.15.50:0/715900370 shutdown_connections 2024-09-21T06:05:24.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.246+0000 7fa37f200640 1 --2- 172.21.15.50:0/715900370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38006ac60 0x7fa3800af730 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:24.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.246+0000 7fa37f200640 1 --2- 172.21.15.50:0/715900370 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa3800b5a00 0x7fa380151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:24.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.246+0000 7fa37f200640 1 -- 172.21.15.50:0/715900370 >> 172.21.15.50:0/715900370 conn(0x7fa38006d3f0 msgr2=0x7fa3800b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:24.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.250+0000 7fa37f200640 1 -- 172.21.15.50:0/715900370 shutdown_connections 2024-09-21T06:05:24.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.250+0000 7fa37f200640 1 -- 172.21.15.50:0/715900370 wait complete. 2024-09-21T06:05:24.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.254+0000 7fa37f200640 1 Processor -- start 2024-09-21T06:05:24.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.254+0000 7fa37f200640 1 -- start start 2024-09-21T06:05:24.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38006ac60 0x7fa3800ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:24.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37f200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa38006b070 0x7fa3800ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:24.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3800b5a00 0x7fa380153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:24.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa380168fc0 con 0x7fa38006ac60 2024-09-21T06:05:24.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa380168e40 con 0x7fa3800b5a00 2024-09-21T06:05:24.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37f200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa380169140 con 0x7fa38006b070 2024-09-21T06:05:24.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38006ac60 0x7fa3800ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:24.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38006ac60 0x7fa3800ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53556/0 (socket says 172.21.15.50:53556) 2024-09-21T06:05:24.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37de00640 1 -- 172.21.15.50:0/380835352 learned_addr learned my addr 172.21.15.50:0/380835352 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:24.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37d400640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa38006b070 0x7fa3800ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:24.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37d400640 1 -- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3800b5a00 msgr2=0x7fa380153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:24.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37d400640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3800b5a00 0x7fa380153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:24.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37d400640 1 -- 172.21.15.50:0/380835352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38006ac60 msgr2=0x7fa3800ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:24.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37d400640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38006ac60 0x7fa3800ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:24.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.258+0000 7fa37d400640 1 -- 172.21.15.50:0/380835352 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3800ace40 con 0x7fa38006b070 2024-09-21T06:05:24.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.262+0000 7fa37d400640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa38006b070 0x7fa3800ae9e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0ac000 tx=0x7fa36c0a5680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.262+0000 7fa376000640 1 -- 172.21.15.50:0/380835352 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c0b2040 con 0x7fa38006b070 2024-09-21T06:05:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.262+0000 7fa376000640 1 -- 172.21.15.50:0/380835352 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa36c0a5cf0 con 0x7fa38006b070 2024-09-21T06:05:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.262+0000 7fa376000640 1 -- 172.21.15.50:0/380835352 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c003860 con 0x7fa38006b070 2024-09-21T06:05:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.262+0000 7fa37f200640 1 -- 172.21.15.50:0/380835352 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3800ad0f0 con 0x7fa38006b070 2024-09-21T06:05:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.262+0000 7fa37f200640 1 -- 172.21.15.50:0/380835352 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa380206060 con 0x7fa38006b070 2024-09-21T06:05:24.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.262+0000 7fa36b400640 1 -- 172.21.15.50:0/380835352 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3540052a0 con 0x7fa38006b070 2024-09-21T06:05:24.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.270+0000 7fa376000640 1 -- 172.21.15.50:0/380835352 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 31) ==== 98662+0+0 (secure 0 0 0) 0x7fa36c003a00 con 0x7fa38006b070 2024-09-21T06:05:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.274+0000 7fa376000640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa3600741c0 0x7fa3600766d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.274+0000 7fa376000640 1 -- 172.21.15.50:0/380835352 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fa36c12cf10 con 0x7fa38006b070 2024-09-21T06:05:24.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.286+0000 7fa376000640 1 -- 172.21.15.50:0/380835352 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa36c0164d0 con 0x7fa38006b070 2024-09-21T06:05:24.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.298+0000 7fa37de00640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa3600741c0 0x7fa3600766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:24.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.306+0000 7fa37de00640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa3600741c0 0x7fa3600766d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa370005fd0 tx=0x7fa370005dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:24.622+0000 7fa36b400640 1 -- 172.21.15.50:0/380835352 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone93454", "target": ["mon-mgr", ""]}) -- 0x7fa354000c90 con 0x7fa3600741c0 2024-09-21T06:05:25.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.198+0000 7fa376000640 1 -- 172.21.15.50:0/380835352 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa354000c90 con 0x7fa3600741c0 2024-09-21T06:05:25.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.202+0000 7fa36b400640 1 -- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa3600741c0 msgr2=0x7fa3600766d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:25.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.202+0000 7fa36b400640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa3600741c0 0x7fa3600766d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa370005fd0 tx=0x7fa370005dd0 comp rx=0 tx=0).stop 2024-09-21T06:05:25.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.202+0000 7fa36b400640 1 -- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa38006b070 msgr2=0x7fa3800ae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:25.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.202+0000 7fa36b400640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa38006b070 0x7fa3800ae9e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0ac000 tx=0x7fa36c0a5680 comp rx=0 tx=0).stop 2024-09-21T06:05:25.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.202+0000 7fa36b400640 1 -- 172.21.15.50:0/380835352 shutdown_connections 2024-09-21T06:05:25.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.202+0000 7fa36b400640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3800b5a00 0x7fa380153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:25.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.202+0000 7fa36b400640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa38006b070 0x7fa3800ae9e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:25.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.202+0000 7fa36b400640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa3600741c0 0x7fa3600766d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:25.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.202+0000 7fa36b400640 1 --2- 172.21.15.50:0/380835352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38006ac60 0x7fa3800ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:25.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.202+0000 7fa36b400640 1 -- 172.21.15.50:0/380835352 >> 172.21.15.50:0/380835352 conn(0x7fa38006d3f0 msgr2=0x7fa3800d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:25.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.202+0000 7fa36b400640 1 -- 172.21.15.50:0/380835352 shutdown_connections 2024-09-21T06:05:25.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.202+0000 7fa36b400640 1 -- 172.21.15.50:0/380835352 wait complete. 2024-09-21T06:05:25.252 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone71335 2024-09-21T06:05:25.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.594+0000 7f75bc400640 1 -- 172.21.15.50:0/3727131541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f759c003a10 msgr2=0x7f759c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:25.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.594+0000 7f75bc400640 1 --2- 172.21.15.50:0/3727131541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f759c003a10 0x7f759c003e60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f75a800c310 tx=0x7f75a801cfc0 comp rx=0 tx=0).stop 2024-09-21T06:05:25.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.594+0000 7f75bc400640 1 -- 172.21.15.50:0/3727131541 shutdown_connections 2024-09-21T06:05:25.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.594+0000 7f75bc400640 1 --2- 172.21.15.50:0/3727131541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f759c003a10 0x7f759c003e60 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:25.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.594+0000 7f75bc400640 1 -- 172.21.15.50:0/3727131541 >> 172.21.15.50:0/3727131541 conn(0x7f75b40b2040 msgr2=0x7f75b40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:25.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.594+0000 7f75bc400640 1 -- 172.21.15.50:0/3727131541 shutdown_connections 2024-09-21T06:05:25.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.594+0000 7f75bc400640 1 -- 172.21.15.50:0/3727131541 wait complete. 2024-09-21T06:05:25.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.598+0000 7f75bc400640 1 Processor -- start 2024-09-21T06:05:25.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.598+0000 7f75bc400640 1 -- start start 2024-09-21T06:05:25.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75bc400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f759c003a10 0x7f75b41527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:25.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75bc400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75b4145260 0x7f75b40aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:25.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75bc400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b41444f0 0x7f75b4144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:25.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75bc400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f75b40b3ae0 con 0x7f759c003a10 2024-09-21T06:05:25.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75bc400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f75b40b3960 con 0x7f75b41444f0 2024-09-21T06:05:25.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75bc400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f75b40b3c60 con 0x7f75b4145260 2024-09-21T06:05:25.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75b9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f759c003a10 0x7f75b41527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:25.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75b9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f759c003a10 0x7f75b41527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53588/0 (socket says 172.21.15.50:53588) 2024-09-21T06:05:25.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75b9a00640 1 -- 172.21.15.50:0/1837672889 learned_addr learned my addr 172.21.15.50:0/1837672889 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:25.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75ba400640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b41444f0 0x7f75b4144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:25.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75b9a00640 1 -- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75b4145260 msgr2=0x7f75b40aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:25.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75b9a00640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75b4145260 0x7f75b40aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:25.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75b9a00640 1 -- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b41444f0 msgr2=0x7f75b4144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:25.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75b9a00640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b41444f0 0x7f75b4144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:25.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75b9a00640 1 -- 172.21.15.50:0/1837672889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b40af330 con 0x7f759c003a10 2024-09-21T06:05:25.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75ba400640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b41444f0 0x7f75b4144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:25.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.602+0000 7f75b9a00640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f759c003a10 0x7f75b41527f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f75a801b890 tx=0x7f75a80aeac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:25.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.606+0000 7f75b1600640 1 -- 172.21.15.50:0/1837672889 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8006c80 con 0x7f759c003a10 2024-09-21T06:05:25.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.606+0000 7f75bc400640 1 -- 172.21.15.50:0/1837672889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b4153120 con 0x7f759c003a10 2024-09-21T06:05:25.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.606+0000 7f75bc400640 1 -- 172.21.15.50:0/1837672889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b41536e0 con 0x7f759c003a10 2024-09-21T06:05:25.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.606+0000 7f75b1600640 1 -- 172.21.15.50:0/1837672889 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75a80aecb0 con 0x7f759c003a10 2024-09-21T06:05:25.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.606+0000 7f75b1600640 1 -- 172.21.15.50:0/1837672889 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8001d00 con 0x7f759c003a10 2024-09-21T06:05:25.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.606+0000 7f75bc400640 1 -- 172.21.15.50:0/1837672889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75900052a0 con 0x7f759c003a10 2024-09-21T06:05:25.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.610+0000 7f75b1600640 1 -- 172.21.15.50:0/1837672889 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 31) ==== 98662+0+0 (secure 0 0 0) 0x7f75a80bf060 con 0x7f759c003a10 2024-09-21T06:05:25.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.610+0000 7f75b1600640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7594074290 0x7f75940767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:25.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.614+0000 7f75b1600640 1 -- 172.21.15.50:0/1837672889 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f75a812d3b0 con 0x7f759c003a10 2024-09-21T06:05:25.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.618+0000 7f75b1600640 1 -- 172.21.15.50:0/1837672889 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f75a8016d70 con 0x7f759c003a10 2024-09-21T06:05:25.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.630+0000 7f75b9000640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7594074290 0x7f75940767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:25.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.666+0000 7f75b9000640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7594074290 0x7f75940767a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f75b4154cd0 tx=0x7f75a0006d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:25.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:25.842+0000 7f75b1600640 1 -- 172.21.15.50:0/1837672889 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 32) ==== 98713+0+0 (secure 0 0 0) 0x7f75a80f5990 con 0x7f759c003a10 2024-09-21T06:05:26.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:26.010+0000 7f75bc400640 1 -- 172.21.15.50:0/1837672889 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone71335", "target": ["mon-mgr", ""]}) -- 0x7f7590000c90 con 0x7f7594074290 2024-09-21T06:05:27.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.326+0000 7f75b1600640 1 -- 172.21.15.50:0/1837672889 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7590000c90 con 0x7f7594074290 2024-09-21T06:05:27.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.330+0000 7f75bc400640 1 -- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7594074290 msgr2=0x7f75940767a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.330+0000 7f75bc400640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7594074290 0x7f75940767a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f75b4154cd0 tx=0x7f75a0006d10 comp rx=0 tx=0).stop 2024-09-21T06:05:27.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.330+0000 7f75bc400640 1 -- 172.21.15.50:0/1837672889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f759c003a10 msgr2=0x7f75b41527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.330+0000 7f75bc400640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f759c003a10 0x7f75b41527f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f75a801b890 tx=0x7f75a80aeac0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.330+0000 7f75bc400640 1 -- 172.21.15.50:0/1837672889 shutdown_connections 2024-09-21T06:05:27.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.330+0000 7f75bc400640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b41444f0 0x7f75b4144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.330+0000 7f75bc400640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75b4145260 0x7f75b40aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.330+0000 7f75bc400640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7594074290 0x7f75940767a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.330+0000 7f75bc400640 1 --2- 172.21.15.50:0/1837672889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f759c003a10 0x7f75b41527f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.330+0000 7f75bc400640 1 -- 172.21.15.50:0/1837672889 >> 172.21.15.50:0/1837672889 conn(0x7f75b40b2040 msgr2=0x7f75b4158ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:27.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.330+0000 7f75bc400640 1 -- 172.21.15.50:0/1837672889 shutdown_connections 2024-09-21T06:05:27.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.330+0000 7f75bc400640 1 -- 172.21.15.50:0/1837672889 wait complete. 2024-09-21T06:05:27.359 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone12326 2024-09-21T06:05:27.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.606+0000 7fe7a0000640 1 -- 172.21.15.50:0/2145505306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe798074950 msgr2=0x7fe79815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.606+0000 7fe7a0000640 1 --2- 172.21.15.50:0/2145505306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe798074950 0x7fe79815cf90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe790009bc0 tx=0x7fe79001cba0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.606+0000 7fe7a0000640 1 -- 172.21.15.50:0/2145505306 shutdown_connections 2024-09-21T06:05:27.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.606+0000 7fe7a0000640 1 --2- 172.21.15.50:0/2145505306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe798074950 0x7fe79815cf90 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.606+0000 7fe7a0000640 1 --2- 172.21.15.50:0/2145505306 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe7980b5a80 0x7fe798074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.606+0000 7fe7a0000640 1 -- 172.21.15.50:0/2145505306 >> 172.21.15.50:0/2145505306 conn(0x7fe7980708c0 msgr2=0x7fe798070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.610+0000 7fe7a0000640 1 -- 172.21.15.50:0/2145505306 shutdown_connections 2024-09-21T06:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.610+0000 7fe7a0000640 1 -- 172.21.15.50:0/2145505306 wait complete. 2024-09-21T06:05:27.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.614+0000 7fe7a0000640 1 Processor -- start 2024-09-21T06:05:27.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.614+0000 7fe7a0000640 1 -- start start 2024-09-21T06:05:27.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.614+0000 7fe7a0000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe798074950 0x7fe7981fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.614+0000 7fe7a0000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7980b5a80 0x7fe7981495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.614+0000 7fe7a0000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe798157030 0x7fe7981523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.614+0000 7fe7a0000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe79816a3c0 con 0x7fe7980b5a80 2024-09-21T06:05:27.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.614+0000 7fe7a0000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe79816a240 con 0x7fe798074950 2024-09-21T06:05:27.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.614+0000 7fe7a0000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe79816a540 con 0x7fe798157030 2024-09-21T06:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe79d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe798074950 0x7fe7981fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe79d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe798074950 0x7fe7981fd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:49654/0 (socket says 172.21.15.50:49654) 2024-09-21T06:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe79d600640 1 -- 172.21.15.50:0/1454283501 learned_addr learned my addr 172.21.15.50:0/1454283501 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe79cc00640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7980b5a80 0x7fe7981495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe79e000640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe798157030 0x7fe7981523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe79cc00640 1 -- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe798157030 msgr2=0x7fe7981523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe79cc00640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe798157030 0x7fe7981523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe79cc00640 1 -- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe798074950 msgr2=0x7fe7981fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe79cc00640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe798074950 0x7fe7981fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe79cc00640 1 -- 172.21.15.50:0/1454283501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7981fdc70 con 0x7fe7980b5a80 2024-09-21T06:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe79cc00640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7980b5a80 0x7fe7981495e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe7980b1740 tx=0x7fe78400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe795600640 1 -- 172.21.15.50:0/1454283501 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78400cab0 con 0x7fe7980b5a80 2024-09-21T06:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe7a0000640 1 -- 172.21.15.50:0/1454283501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7981fdf50 con 0x7fe7980b5a80 2024-09-21T06:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe795600640 1 -- 172.21.15.50:0/1454283501 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe78400cd40 con 0x7fe7980b5a80 2024-09-21T06:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe795600640 1 -- 172.21.15.50:0/1454283501 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784029690 con 0x7fe7980b5a80 2024-09-21T06:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.618+0000 7fe7a0000640 1 -- 172.21.15.50:0/1454283501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7981ffd70 con 0x7fe7980b5a80 2024-09-21T06:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.622+0000 7fe7a0000640 1 -- 172.21.15.50:0/1454283501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7740052a0 con 0x7fe7980b5a80 2024-09-21T06:05:27.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.622+0000 7fe795600640 1 -- 172.21.15.50:0/1454283501 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 32) ==== 98713+0+0 (secure 0 0 0) 0x7fe784029830 con 0x7fe7980b5a80 2024-09-21T06:05:27.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.622+0000 7fe795600640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe778074250 0x7fe778076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.626+0000 7fe795600640 1 -- 172.21.15.50:0/1454283501 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fe7840a4810 con 0x7fe7980b5a80 2024-09-21T06:05:27.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.630+0000 7fe795600640 1 -- 172.21.15.50:0/1454283501 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe784070980 con 0x7fe7980b5a80 2024-09-21T06:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.638+0000 7fe79d600640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe778074250 0x7fe778076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.642+0000 7fe79d600640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe778074250 0x7fe778076760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe78c000f70 tx=0x7fe78c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:27.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:27.982+0000 7fe7a0000640 1 -- 172.21.15.50:0/1454283501 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone12326", "target": ["mon-mgr", ""]}) -- 0x7fe774000c90 con 0x7fe778074250 2024-09-21T06:05:29.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.378+0000 7fe795600640 1 -- 172.21.15.50:0/1454283501 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe774000c90 con 0x7fe778074250 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.382+0000 7fe7a0000640 1 -- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe778074250 msgr2=0x7fe778076760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.382+0000 7fe7a0000640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe778074250 0x7fe778076760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe78c000f70 tx=0x7fe78c009040 comp rx=0 tx=0).stop 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.382+0000 7fe7a0000640 1 -- 172.21.15.50:0/1454283501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7980b5a80 msgr2=0x7fe7981495e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.382+0000 7fe7a0000640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7980b5a80 0x7fe7981495e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe7980b1740 tx=0x7fe78400eb50 comp rx=0 tx=0).stop 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.382+0000 7fe7a0000640 1 -- 172.21.15.50:0/1454283501 shutdown_connections 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.382+0000 7fe7a0000640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe798157030 0x7fe7981523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.382+0000 7fe7a0000640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe778074250 0x7fe778076760 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.382+0000 7fe7a0000640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7980b5a80 0x7fe7981495e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.382+0000 7fe7a0000640 1 --2- 172.21.15.50:0/1454283501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe798074950 0x7fe7981fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.382+0000 7fe7a0000640 1 -- 172.21.15.50:0/1454283501 >> 172.21.15.50:0/1454283501 conn(0x7fe7980708c0 msgr2=0x7fe798154030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:29.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.382+0000 7fe7a0000640 1 -- 172.21.15.50:0/1454283501 shutdown_connections 2024-09-21T06:05:29.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.382+0000 7fe7a0000640 1 -- 172.21.15.50:0/1454283501 wait complete. 2024-09-21T06:05:29.405 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:05:29.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.694+0000 7fec47200640 1 -- 172.21.15.50:0/3315031352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec2c0039c0 msgr2=0x7fec2c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.694+0000 7fec47200640 1 --2- 172.21.15.50:0/3315031352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec2c0039c0 0x7fec2c003e10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c730 tx=0x7fec3801c6b0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.694+0000 7fec47200640 1 -- 172.21.15.50:0/3315031352 shutdown_connections 2024-09-21T06:05:29.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.694+0000 7fec47200640 1 --2- 172.21.15.50:0/3315031352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec2c0039c0 0x7fec2c003e10 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.694+0000 7fec47200640 1 -- 172.21.15.50:0/3315031352 >> 172.21.15.50:0/3315031352 conn(0x7fec4806cdf0 msgr2=0x7fec4806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:29.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.694+0000 7fec47200640 1 -- 172.21.15.50:0/3315031352 shutdown_connections 2024-09-21T06:05:29.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.694+0000 7fec47200640 1 -- 172.21.15.50:0/3315031352 wait complete. 2024-09-21T06:05:29.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.698+0000 7fec47200640 1 Processor -- start 2024-09-21T06:05:29.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.698+0000 7fec47200640 1 -- start start 2024-09-21T06:05:29.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.698+0000 7fec47200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec2c0039c0 0x7fec4813dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.698+0000 7fec47200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec48148990 0x7fec4814eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.698+0000 7fec47200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec4814f090 0x7fec4814a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.698+0000 7fec47200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fec480b39b0 con 0x7fec48148990 2024-09-21T06:05:29.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.698+0000 7fec47200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec480b3830 con 0x7fec2c0039c0 2024-09-21T06:05:29.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.698+0000 7fec47200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fec480b3b30 con 0x7fec4814f090 2024-09-21T06:05:29.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec46800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec4814f090 0x7fec4814a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec46800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec4814f090 0x7fec4814a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:35976/0 (socket says 172.21.15.50:35976) 2024-09-21T06:05:29.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec46800640 1 -- 172.21.15.50:0/3323833629 learned_addr learned my addr 172.21.15.50:0/3323833629 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:29.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec45e00640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec2c0039c0 0x7fec4813dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec3fe00640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec48148990 0x7fec4814eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec45e00640 1 -- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec4814f090 msgr2=0x7fec4814a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec45e00640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec4814f090 0x7fec4814a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec45e00640 1 -- 172.21.15.50:0/3323833629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec48148990 msgr2=0x7fec4814eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec45e00640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec48148990 0x7fec4814eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec45e00640 1 -- 172.21.15.50:0/3323833629 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4814fc60 con 0x7fec2c0039c0 2024-09-21T06:05:29.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec45e00640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec2c0039c0 0x7fec4813dbf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fec38006fd0 tx=0x7fec38004c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:29.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec3e000640 1 -- 172.21.15.50:0/3323833629 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec380ba070 con 0x7fec2c0039c0 2024-09-21T06:05:29.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec3e000640 1 -- 172.21.15.50:0/3323833629 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec38006500 con 0x7fec2c0039c0 2024-09-21T06:05:29.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.702+0000 7fec3e000640 1 -- 172.21.15.50:0/3323833629 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec380ad070 con 0x7fec2c0039c0 2024-09-21T06:05:29.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.706+0000 7fec47200640 1 -- 172.21.15.50:0/3323833629 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4815e3b0 con 0x7fec2c0039c0 2024-09-21T06:05:29.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.706+0000 7fec47200640 1 -- 172.21.15.50:0/3323833629 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4815e970 con 0x7fec2c0039c0 2024-09-21T06:05:29.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.706+0000 7fec47200640 1 -- 172.21.15.50:0/3323833629 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec480720c0 con 0x7fec2c0039c0 2024-09-21T06:05:29.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.706+0000 7fec3e000640 1 -- 172.21.15.50:0/3323833629 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) ==== 98713+0+0 (secure 0 0 0) 0x7fec380a76a0 con 0x7fec2c0039c0 2024-09-21T06:05:29.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.710+0000 7fec3e000640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec24074250 0x7fec24076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.710+0000 7fec3e000640 1 -- 172.21.15.50:0/3323833629 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fec3812c650 con 0x7fec2c0039c0 2024-09-21T06:05:29.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.718+0000 7fec3e000640 1 -- 172.21.15.50:0/3323833629 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fec380163f0 con 0x7fec2c0039c0 2024-09-21T06:05:29.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.750+0000 7fec3fe00640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec24074250 0x7fec24076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:29.774+0000 7fec3fe00640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec24074250 0x7fec24076760 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fec30001ec0 tx=0x7fec30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:30.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.158+0000 7fec47200640 1 -- 172.21.15.50:0/3323833629 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fec48153610 con 0x7fec2c0039c0 2024-09-21T06:05:30.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.158+0000 7fec3e000640 1 -- 172.21.15.50:0/3323833629 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+3532 (secure 0 0 0) 0x7fec380f8ab0 con 0x7fec2c0039c0 2024-09-21T06:05:30.161 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:30.161 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:05:30.161 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:05:30.162 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:05:30.162 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:05:30.162 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:05:30.162 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:05:30.162 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:30.162 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:05:30.162 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:05:30.162 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:05:30.162 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:05:30.162 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:05:30.162 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:05:30.163 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:05:30.163 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:05:30.163 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:05:30.163 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:05:30.163 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:05:30.163 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 566, 2024-09-21T06:05:30.163 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:05:30.164 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:05:30.164 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:05:30.164 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:05:30.164 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:30.164 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:05:30.164 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 73, 2024-09-21T06:05:30.164 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:05:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 1924, 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 7928507345, 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 15597027328, 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 757497085952, 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 403957391, 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 459169183, 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 97, 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 124 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 46, 2024-09-21T06:05:30.166 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T06:04:45:517919+0000", 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 5, 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 5, 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "d", 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 6423 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:05:30.167 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 13, 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:05:29.047569+0000", 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "services": { 2024-09-21T06:05:30.168 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-09-21T06:05:30.169 INFO:teuthology.orchestra.run.smithi050.stdout: "daemons": { 2024-09-21T06:05:30.169 INFO:teuthology.orchestra.run.smithi050.stdout: "summary": "", 2024-09-21T06:05:30.169 INFO:teuthology.orchestra.run.smithi050.stdout: "a": { 2024-09-21T06:05:30.169 INFO:teuthology.orchestra.run.smithi050.stdout: "start_epoch": 0, 2024-09-21T06:05:30.169 INFO:teuthology.orchestra.run.smithi050.stdout: "start_stamp": "0.000000", 2024-09-21T06:05:30.169 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 0, 2024-09-21T06:05:30.169 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-21T06:05:30.169 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": {}, 2024-09-21T06:05:30.169 INFO:teuthology.orchestra.run.smithi050.stdout: "task_status": {} 2024-09-21T06:05:30.169 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:30.169 INFO:teuthology.orchestra.run.smithi050.stdout: "c": { 2024-09-21T06:05:30.169 INFO:teuthology.orchestra.run.smithi050.stdout: "start_epoch": 0, 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: "start_stamp": "0.000000", 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 0, 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": {}, 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: "task_status": {} 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: "d": { 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: "start_epoch": 0, 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: "start_stamp": "0.000000", 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 0, 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": {}, 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: "task_status": {} 2024-09-21T06:05:30.170 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": { 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr-vol-ongoing-clones": { 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: "message": "4 ongoing clones - average progress is 32.357% (6s)\n [=========...................] (remaining: 14s)", 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: "progress": 0.32357001304626465, 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: "add_to_ceph_s": true 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr-vol-total-clones": { 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: "message": "Total 5 clones - average progress is 29.063% (0s)\n [========....................] ", 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: "progress": 0.29063001275062561, 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: "add_to_ceph_s": true 2024-09-21T06:05:30.171 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:05:30.172 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:05:30.172 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:05:30.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.162+0000 7fec47200640 1 -- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec24074250 msgr2=0x7fec24076760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:30.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.162+0000 7fec47200640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec24074250 0x7fec24076760 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fec30001ec0 tx=0x7fec30009040 comp rx=0 tx=0).stop 2024-09-21T06:05:30.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.162+0000 7fec47200640 1 -- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec2c0039c0 msgr2=0x7fec4813dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:30.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.162+0000 7fec47200640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec2c0039c0 0x7fec4813dbf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fec38006fd0 tx=0x7fec38004c90 comp rx=0 tx=0).stop 2024-09-21T06:05:30.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.162+0000 7fec47200640 1 -- 172.21.15.50:0/3323833629 shutdown_connections 2024-09-21T06:05:30.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.162+0000 7fec47200640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec4814f090 0x7fec4814a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.162+0000 7fec47200640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec24074250 0x7fec24076760 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.162+0000 7fec47200640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec48148990 0x7fec4814eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.162+0000 7fec47200640 1 --2- 172.21.15.50:0/3323833629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec2c0039c0 0x7fec4813dbf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.162+0000 7fec47200640 1 -- 172.21.15.50:0/3323833629 >> 172.21.15.50:0/3323833629 conn(0x7fec4806cdf0 msgr2=0x7fec4806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:30.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.162+0000 7fec47200640 1 -- 172.21.15.50:0/3323833629 shutdown_connections 2024-09-21T06:05:30.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.162+0000 7fec47200640 1 -- 172.21.15.50:0/3323833629 wait complete. 2024-09-21T06:05:30.206 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone91230 2024-09-21T06:05:30.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.446+0000 7f28bbe00640 1 -- 172.21.15.50:0/3780931015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28bc15f5c0 msgr2=0x7f28bc15f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:30.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.446+0000 7f28bbe00640 1 --2- 172.21.15.50:0/3780931015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28bc15f5c0 0x7f28bc15f9d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0ced20 tx=0x7f28b401caf0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.446+0000 7f28bbe00640 1 -- 172.21.15.50:0/3780931015 shutdown_connections 2024-09-21T06:05:30.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.446+0000 7f28bbe00640 1 --2- 172.21.15.50:0/3780931015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28bc15f5c0 0x7f28bc15f9d0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.446+0000 7f28bbe00640 1 --2- 172.21.15.50:0/3780931015 >> v2:172.21.15.88:3300/0 conn(0x7f28bc0d1140 0x7f28bc0b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.446+0000 7f28bbe00640 1 -- 172.21.15.50:0/3780931015 >> 172.21.15.50:0/3780931015 conn(0x7f28bc0b41f0 msgr2=0x7f28bc0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:30.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.450+0000 7f28bbe00640 1 -- 172.21.15.50:0/3780931015 shutdown_connections 2024-09-21T06:05:30.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.450+0000 7f28bbe00640 1 -- 172.21.15.50:0/3780931015 wait complete. 2024-09-21T06:05:30.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.450+0000 7f28bbe00640 1 Processor -- start 2024-09-21T06:05:30.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.454+0000 7f28bbe00640 1 -- start start 2024-09-21T06:05:30.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.454+0000 7f28bbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28bc06dbc0 0x7f28bc158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:30.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.454+0000 7f28bbe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28bc0d1140 0x7f28bc0c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:30.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.454+0000 7f28bbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28bc15f5c0 0x7f28bc0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:30.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.454+0000 7f28bbe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc168ff0 con 0x7f28bc06dbc0 2024-09-21T06:05:30.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.454+0000 7f28bbe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc168e70 con 0x7f28bc15f5c0 2024-09-21T06:05:30.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.454+0000 7f28bbe00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f28bc169170 con 0x7f28bc0d1140 2024-09-21T06:05:30.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.454+0000 7f28baa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28bc06dbc0 0x7f28bc158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:30.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.454+0000 7f28baa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28bc06dbc0 0x7f28bc158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50388/0 (socket says 172.21.15.50:50388) 2024-09-21T06:05:30.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.454+0000 7f28baa00640 1 -- 172.21.15.50:0/1067367566 learned_addr learned my addr 172.21.15.50:0/1067367566 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:30.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.458+0000 7f28baa00640 1 -- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28bc0d1140 msgr2=0x7f28bc0c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:30.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.458+0000 7f28ba000640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28bc0d1140 0x7f28bc0c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:30.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.458+0000 7f28baa00640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28bc0d1140 0x7f28bc0c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.458+0000 7f28baa00640 1 -- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28bc15f5c0 msgr2=0x7f28bc0ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:30.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.458+0000 7f28baa00640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28bc15f5c0 0x7f28bc0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.458+0000 7f28baa00640 1 -- 172.21.15.50:0/1067367566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28bc1fc900 con 0x7f28bc06dbc0 2024-09-21T06:05:30.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.458+0000 7f28ba000640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28bc0d1140 0x7f28bc0c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:30.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.458+0000 7f28baa00640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28bc06dbc0 0x7f28bc158f70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00b410 tx=0x7f28ac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:30.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.458+0000 7f28b2a00640 1 -- 172.21.15.50:0/1067367566 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac015020 con 0x7f28bc06dbc0 2024-09-21T06:05:30.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.458+0000 7f28b2a00640 1 -- 172.21.15.50:0/1067367566 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28ac011920 con 0x7f28bc06dbc0 2024-09-21T06:05:30.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.458+0000 7f28b2a00640 1 -- 172.21.15.50:0/1067367566 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac011d70 con 0x7f28bc06dbc0 2024-09-21T06:05:30.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.462+0000 7f28bbe00640 1 -- 172.21.15.50:0/1067367566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28bc1fcbe0 con 0x7f28bc06dbc0 2024-09-21T06:05:30.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.462+0000 7f28bbe00640 1 -- 172.21.15.50:0/1067367566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28bc15cdb0 con 0x7f28bc06dbc0 2024-09-21T06:05:30.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.466+0000 7f28bbe00640 1 -- 172.21.15.50:0/1067367566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28bc0b0730 con 0x7f28bc06dbc0 2024-09-21T06:05:30.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.466+0000 7f28b2a00640 1 -- 172.21.15.50:0/1067367566 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 32) ==== 98713+0+0 (secure 0 0 0) 0x7f28ac018020 con 0x7f28bc06dbc0 2024-09-21T06:05:30.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.466+0000 7f28b2a00640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f289c074250 0x7f289c076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:30.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.466+0000 7f28b2a00640 1 -- 172.21.15.50:0/1067367566 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f28ac092b60 con 0x7f28bc06dbc0 2024-09-21T06:05:30.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.474+0000 7f28b2a00640 1 -- 172.21.15.50:0/1067367566 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f28ac05f630 con 0x7f28bc06dbc0 2024-09-21T06:05:30.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.486+0000 7f28ba000640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f289c074250 0x7f289c076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:30.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.546+0000 7f28ba000640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f289c074250 0x7f289c076760 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0cac10 tx=0x7f28a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:30.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:30.886+0000 7f28bbe00640 1 -- 172.21.15.50:0/1067367566 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone91230", "target": ["mon-mgr", ""]}) -- 0x7f28bc06b540 con 0x7f289c074250 2024-09-21T06:05:31.115 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:31.114+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:31.116 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:31.114+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:31.116 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:31.114+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:31.116 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:31.114+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:31.116 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:31.114+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:33.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28b2a00640 1 -- 172.21.15.50:0/1067367566 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f28a80012b0 con 0x7f289c074250 2024-09-21T06:05:33.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28bbe00640 1 -- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f289c074250 msgr2=0x7f289c076760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:33.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28bbe00640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f289c074250 0x7f289c076760 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0cac10 tx=0x7f28a8009040 comp rx=0 tx=0).stop 2024-09-21T06:05:33.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28bbe00640 1 -- 172.21.15.50:0/1067367566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28bc06dbc0 msgr2=0x7f28bc158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:33.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28bbe00640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28bc06dbc0 0x7f28bc158f70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00b410 tx=0x7f28ac00b8d0 comp rx=0 tx=0).stop 2024-09-21T06:05:33.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28bbe00640 1 -- 172.21.15.50:0/1067367566 shutdown_connections 2024-09-21T06:05:33.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28bbe00640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28bc15f5c0 0x7f28bc0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:33.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28bbe00640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f28bc0d1140 0x7f28bc0c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:33.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28bbe00640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f289c074250 0x7f289c076760 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:33.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28bbe00640 1 --2- 172.21.15.50:0/1067367566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28bc06dbc0 0x7f28bc158f70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:33.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28bbe00640 1 -- 172.21.15.50:0/1067367566 >> 172.21.15.50:0/1067367566 conn(0x7f28bc0b41f0 msgr2=0x7f28bc0cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:33.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28bbe00640 1 -- 172.21.15.50:0/1067367566 shutdown_connections 2024-09-21T06:05:33.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:33.862+0000 7f28bbe00640 1 -- 172.21.15.50:0/1067367566 wait complete. 2024-09-21T06:05:33.903 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone15181 2024-09-21T06:05:34.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.170+0000 7f9ba0800640 1 -- 172.21.15.50:0/1939548453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b9806cdf0 msgr2=0x7f9b9806d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:34.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.170+0000 7f9ba0800640 1 --2- 172.21.15.50:0/1939548453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b9806cdf0 0x7f9b9806d200 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009840 tx=0x7f9b8801c7b0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.174+0000 7f9ba0800640 1 -- 172.21.15.50:0/1939548453 shutdown_connections 2024-09-21T06:05:34.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.174+0000 7f9ba0800640 1 --2- 172.21.15.50:0/1939548453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b9806cdf0 0x7f9b9806d200 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.174+0000 7f9ba0800640 1 -- 172.21.15.50:0/1939548453 >> 172.21.15.50:0/1939548453 conn(0x7f9b980b35f0 msgr2=0x7f9b980b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:34.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.174+0000 7f9ba0800640 1 -- 172.21.15.50:0/1939548453 shutdown_connections 2024-09-21T06:05:34.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.174+0000 7f9ba0800640 1 -- 172.21.15.50:0/1939548453 wait complete. 2024-09-21T06:05:34.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.178+0000 7f9ba0800640 1 Processor -- start 2024-09-21T06:05:34.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.182+0000 7f9ba0800640 1 -- start start 2024-09-21T06:05:34.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.182+0000 7f9ba0800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b9806cdf0 0x7f9b981474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.182+0000 7f9b9de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b9806cdf0 0x7f9b981474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.182+0000 7f9b9de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b9806cdf0 0x7f9b981474d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50400/0 (socket says 172.21.15.50:50400) 2024-09-21T06:05:34.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.182+0000 7f9b9de00640 1 -- 172.21.15.50:0/351046471 learned_addr learned my addr 172.21.15.50:0/351046471 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:34.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9ba0800640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9b9806fc90 0x7f9b9806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9ba0800640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b98159130 0x7f9b9806dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9b980693c0 con 0x7f9b9806cdf0 2024-09-21T06:05:34.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9b98069240 con 0x7f9b98159130 2024-09-21T06:05:34.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9b98069540 con 0x7f9b9806fc90 2024-09-21T06:05:34.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9b9de00640 1 -- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9b9806fc90 msgr2=0x7f9b9806d760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:05:34.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9b9de00640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9b9806fc90 0x7f9b9806d760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9b9de00640 1 -- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b98159130 msgr2=0x7f9b9806dca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:05:34.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9b9de00640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b98159130 0x7f9b9806dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9b9de00640 1 -- 172.21.15.50:0/351046471 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b980d0ab0 con 0x7f9b9806cdf0 2024-09-21T06:05:34.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9b9de00640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b9806cdf0 0x7f9b981474d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00c400 tx=0x7f9b8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:34.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9b92000640 1 -- 172.21.15.50:0/351046471 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c011790 con 0x7f9b9806cdf0 2024-09-21T06:05:34.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.186+0000 7f9b92000640 1 -- 172.21.15.50:0/351046471 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b8c017070 con 0x7f9b9806cdf0 2024-09-21T06:05:34.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.190+0000 7f9b92000640 1 -- 172.21.15.50:0/351046471 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c011cc0 con 0x7f9b9806cdf0 2024-09-21T06:05:34.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.190+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b98159e20 con 0x7f9b9806cdf0 2024-09-21T06:05:34.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.190+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9815a3e0 con 0x7f9b9806cdf0 2024-09-21T06:05:34.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.190+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b98069b10 con 0x7f9b9806cdf0 2024-09-21T06:05:34.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.194+0000 7f9b92000640 1 -- 172.21.15.50:0/351046471 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7f9b8c01c460 con 0x7f9b9806cdf0 2024-09-21T06:05:34.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.194+0000 7f9b92000640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9b7c0742e0 0x7f9b7c0767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.194+0000 7f9b92000640 1 -- 172.21.15.50:0/351046471 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f9b8c092b60 con 0x7f9b9806cdf0 2024-09-21T06:05:34.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.202+0000 7f9b9d400640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9b7c0742e0 0x7f9b7c0767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.206+0000 7f9b92000640 1 -- 172.21.15.50:0/351046471 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9b980d0ab0 con 0x7f9b9806cdf0 2024-09-21T06:05:34.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.210+0000 7f9b9d400640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9b7c0742e0 0x7f9b7c0767f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9b880af000 tx=0x7f9b8801aff0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:34.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:34.554+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone15181", "target": ["mon-mgr", ""]}) -- 0x7f9b9806b540 con 0x7f9b7c0742e0 2024-09-21T06:05:35.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.434+0000 7f9b92000640 1 -- 172.21.15.50:0/351046471 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9b880a68f0 con 0x7f9b7c0742e0 2024-09-21T06:05:35.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.438+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9b7c0742e0 msgr2=0x7f9b7c0767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:35.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.438+0000 7f9ba0800640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9b7c0742e0 0x7f9b7c0767f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9b880af000 tx=0x7f9b8801aff0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.438+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b9806cdf0 msgr2=0x7f9b981474d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:35.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.438+0000 7f9ba0800640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b9806cdf0 0x7f9b981474d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00c400 tx=0x7f9b8c00c8c0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.438+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 shutdown_connections 2024-09-21T06:05:35.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.438+0000 7f9ba0800640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b98159130 0x7f9b9806dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.438+0000 7f9ba0800640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9b9806fc90 0x7f9b9806d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.438+0000 7f9ba0800640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9b7c0742e0 0x7f9b7c0767f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.438+0000 7f9ba0800640 1 --2- 172.21.15.50:0/351046471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b9806cdf0 0x7f9b981474d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.438+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 >> 172.21.15.50:0/351046471 conn(0x7f9b980b35f0 msgr2=0x7f9b980b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:35.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.438+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 shutdown_connections 2024-09-21T06:05:35.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.438+0000 7f9ba0800640 1 -- 172.21.15.50:0/351046471 wait complete. 2024-09-21T06:05:35.471 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone51512 2024-09-21T06:05:35.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.738+0000 7fa029e00640 1 -- 172.21.15.50:0/3170814578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa00c003a60 msgr2=0x7fa00c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:35.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.738+0000 7fa029e00640 1 --2- 172.21.15.50:0/3170814578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa00c003a60 0x7fa00c003ed0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa01800c260 tx=0x7fa01801ced0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.738+0000 7fa029e00640 1 -- 172.21.15.50:0/3170814578 shutdown_connections 2024-09-21T06:05:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.738+0000 7fa029e00640 1 --2- 172.21.15.50:0/3170814578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa00c003a60 0x7fa00c003ed0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.738+0000 7fa029e00640 1 -- 172.21.15.50:0/3170814578 >> 172.21.15.50:0/3170814578 conn(0x7fa02406d3f0 msgr2=0x7fa0240b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.738+0000 7fa029e00640 1 -- 172.21.15.50:0/3170814578 shutdown_connections 2024-09-21T06:05:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.738+0000 7fa029e00640 1 -- 172.21.15.50:0/3170814578 wait complete. 2024-09-21T06:05:35.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa029e00640 1 Processor -- start 2024-09-21T06:05:35.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa029e00640 1 -- start start 2024-09-21T06:05:35.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa029e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa00c003a60 0x7fa02415da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa029e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02415c050 0x7fa02414e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa029e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0240b1200 0x7fa0240b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa029e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa024073390 con 0x7fa00c003a60 2024-09-21T06:05:35.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa029e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa024073210 con 0x7fa02415c050 2024-09-21T06:05:35.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa029e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa024073510 con 0x7fa0240b1200 2024-09-21T06:05:35.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa028a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa00c003a60 0x7fa02415da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa028a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa00c003a60 0x7fa02415da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50408/0 (socket says 172.21.15.50:50408) 2024-09-21T06:05:35.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa028a00640 1 -- 172.21.15.50:0/3882601112 learned_addr learned my addr 172.21.15.50:0/3882601112 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:35.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa028a00640 1 -- 172.21.15.50:0/3882601112 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0240b1200 msgr2=0x7fa0240b1610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:05:35.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa028a00640 1 --2- 172.21.15.50:0/3882601112 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0240b1200 0x7fa0240b1610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa028a00640 1 -- 172.21.15.50:0/3882601112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02415c050 msgr2=0x7fa02414e470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:05:35.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa028a00640 1 --2- 172.21.15.50:0/3882601112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02415c050 0x7fa02414e470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa028a00640 1 -- 172.21.15.50:0/3882601112 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa024075d10 con 0x7fa00c003a60 2024-09-21T06:05:35.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa028a00640 1 --2- 172.21.15.50:0/3882601112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa00c003a60 0x7fa02415da10 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa01801c990 tx=0x7fa018001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:35.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.742+0000 7fa020c00640 1 -- 172.21.15.50:0/3882601112 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0180010d0 con 0x7fa00c003a60 2024-09-21T06:05:35.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.746+0000 7fa029e00640 1 -- 172.21.15.50:0/3882601112 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0240b1b50 con 0x7fa00c003a60 2024-09-21T06:05:35.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.746+0000 7fa020c00640 1 -- 172.21.15.50:0/3882601112 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0180060d0 con 0x7fa00c003a60 2024-09-21T06:05:35.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.746+0000 7fa020c00640 1 -- 172.21.15.50:0/3882601112 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018003800 con 0x7fa00c003a60 2024-09-21T06:05:35.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.746+0000 7fa029e00640 1 -- 172.21.15.50:0/3882601112 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa02415c730 con 0x7fa00c003a60 2024-09-21T06:05:35.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.746+0000 7fa029e00640 1 -- 172.21.15.50:0/3882601112 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa02406e830 con 0x7fa00c003a60 2024-09-21T06:05:35.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.746+0000 7fa020c00640 1 -- 172.21.15.50:0/3882601112 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7fa0180039a0 con 0x7fa00c003a60 2024-09-21T06:05:35.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.750+0000 7fa020c00640 1 --2- 172.21.15.50:0/3882601112 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa0040744e0 0x7fa0040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.750+0000 7fa020c00640 1 -- 172.21.15.50:0/3882601112 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fa01812d3c0 con 0x7fa00c003a60 2024-09-21T06:05:35.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.750+0000 7fa023e00640 1 --2- 172.21.15.50:0/3882601112 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa0040744e0 0x7fa0040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.750+0000 7fa023e00640 1 --2- 172.21.15.50:0/3882601112 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa0040744e0 0x7fa0040769f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa010000c10 tx=0x7fa010006050 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:35.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:35.754+0000 7fa020c00640 1 -- 172.21.15.50:0/3882601112 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa0180c0030 con 0x7fa00c003a60 2024-09-21T06:05:36.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:36.098+0000 7fa029e00640 1 -- 172.21.15.50:0/3882601112 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone51512", "target": ["mon-mgr", ""]}) -- 0x7fa02406b540 con 0x7fa0040744e0 2024-09-21T06:05:38.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.206+0000 7fa020c00640 1 -- 172.21.15.50:0/3882601112 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa010006e50 con 0x7fa0040744e0 2024-09-21T06:05:38.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.214+0000 7fa029e00640 1 -- 172.21.15.50:0/3882601112 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa0040744e0 msgr2=0x7fa0040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.214+0000 7fa029e00640 1 --2- 172.21.15.50:0/3882601112 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa0040744e0 0x7fa0040769f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa010000c10 tx=0x7fa010006050 comp rx=0 tx=0).stop 2024-09-21T06:05:38.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.214+0000 7fa029e00640 1 -- 172.21.15.50:0/3882601112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa00c003a60 msgr2=0x7fa02415da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.214+0000 7fa029e00640 1 --2- 172.21.15.50:0/3882601112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa00c003a60 0x7fa02415da10 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa01801c990 tx=0x7fa018001d00 comp rx=0 tx=0).stop 2024-09-21T06:05:38.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.218+0000 7fa029e00640 1 -- 172.21.15.50:0/3882601112 shutdown_connections 2024-09-21T06:05:38.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.218+0000 7fa029e00640 1 --2- 172.21.15.50:0/3882601112 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0240b1200 0x7fa0240b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.218+0000 7fa029e00640 1 --2- 172.21.15.50:0/3882601112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02415c050 0x7fa02414e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.218+0000 7fa029e00640 1 --2- 172.21.15.50:0/3882601112 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa0040744e0 0x7fa0040769f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.218+0000 7fa029e00640 1 --2- 172.21.15.50:0/3882601112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa00c003a60 0x7fa02415da10 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.218+0000 7fa029e00640 1 -- 172.21.15.50:0/3882601112 >> 172.21.15.50:0/3882601112 conn(0x7fa02406d3f0 msgr2=0x7fa024163940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:38.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.218+0000 7fa029e00640 1 -- 172.21.15.50:0/3882601112 shutdown_connections 2024-09-21T06:05:38.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.218+0000 7fa029e00640 1 -- 172.21.15.50:0/3882601112 wait complete. 2024-09-21T06:05:38.273 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone26043 2024-09-21T06:05:38.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.562+0000 7ff2c3000640 1 -- 172.21.15.50:0/55565773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2bc1556b0 msgr2=0x7ff2bc155ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.562+0000 7ff2c3000640 1 --2- 172.21.15.50:0/55565773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2bc1556b0 0x7ff2bc155ac0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc0b20f0 tx=0x7ff2b401cb30 comp rx=0 tx=0).stop 2024-09-21T06:05:38.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.562+0000 7ff2c3000640 1 -- 172.21.15.50:0/55565773 shutdown_connections 2024-09-21T06:05:38.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.562+0000 7ff2c3000640 1 --2- 172.21.15.50:0/55565773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2bc1556b0 0x7ff2bc155ac0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.562+0000 7ff2c3000640 1 --2- 172.21.15.50:0/55565773 >> v2:172.21.15.50:3300/0 conn(0x7ff2bc0b7270 0x7ff2bc06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.562+0000 7ff2c3000640 1 -- 172.21.15.50:0/55565773 >> 172.21.15.50:0/55565773 conn(0x7ff2bc0708c0 msgr2=0x7ff2bc070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:38.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.562+0000 7ff2c3000640 1 -- 172.21.15.50:0/55565773 shutdown_connections 2024-09-21T06:05:38.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.562+0000 7ff2c3000640 1 -- 172.21.15.50:0/55565773 wait complete. 2024-09-21T06:05:38.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c3000640 1 Processor -- start 2024-09-21T06:05:38.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c3000640 1 -- start start 2024-09-21T06:05:38.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c3000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2bc0b7270 0x7ff2bc0b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c3000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2bc0d29c0 0x7ff2bc15dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c3000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2bc1556b0 0x7ff2bc15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c3000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc170530 con 0x7ff2bc1556b0 2024-09-21T06:05:38.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c3000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc15ffe0 con 0x7ff2bc0b7270 2024-09-21T06:05:38.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c3000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff2bc1706b0 con 0x7ff2bc0d29c0 2024-09-21T06:05:38.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2bc1556b0 0x7ff2bc15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2bc1556b0 0x7ff2bc15e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51614/0 (socket says 172.21.15.50:51614) 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c1000640 1 -- 172.21.15.50:0/546809162 learned_addr learned my addr 172.21.15.50:0/546809162 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2bb400640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2bc0d29c0 0x7ff2bc15dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c1000640 1 -- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2bc0d29c0 msgr2=0x7ff2bc15dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c1000640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2bc0d29c0 0x7ff2bc15dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c1000640 1 -- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2bc0b7270 msgr2=0x7ff2bc0b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c1000640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2bc0b7270 0x7ff2bc0b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c1000640 1 -- 172.21.15.50:0/546809162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc155070 con 0x7ff2bc1556b0 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c1000640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2bc1556b0 0x7ff2bc15e320 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff2b401a760 tx=0x7ff2b4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2b3e00640 1 -- 172.21.15.50:0/546809162 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40bc070 con 0x7ff2bc1556b0 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2b3e00640 1 -- 172.21.15.50:0/546809162 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2b40042f0 con 0x7ff2bc1556b0 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c3000640 1 -- 172.21.15.50:0/546809162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2bc155350 con 0x7ff2bc1556b0 2024-09-21T06:05:38.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2b3e00640 1 -- 172.21.15.50:0/546809162 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40b2040 con 0x7ff2bc1556b0 2024-09-21T06:05:38.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.566+0000 7ff2c3000640 1 -- 172.21.15.50:0/546809162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2bc14a870 con 0x7ff2bc1556b0 2024-09-21T06:05:38.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.570+0000 7ff2c3000640 1 -- 172.21.15.50:0/546809162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2bc06e520 con 0x7ff2bc1556b0 2024-09-21T06:05:38.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.570+0000 7ff2b3e00640 1 -- 172.21.15.50:0/546809162 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7ff2b40bf070 con 0x7ff2bc1556b0 2024-09-21T06:05:38.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.570+0000 7ff2b3e00640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff29c0742e0 0x7ff29c0767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.570+0000 7ff2b3e00640 1 -- 172.21.15.50:0/546809162 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7ff2b412cc50 con 0x7ff2bc1556b0 2024-09-21T06:05:38.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.578+0000 7ff2b3e00640 1 -- 172.21.15.50:0/546809162 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff2b40167b0 con 0x7ff2bc1556b0 2024-09-21T06:05:38.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.586+0000 7ff2bbe00640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff29c0742e0 0x7ff29c0767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.594+0000 7ff2bbe00640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff29c0742e0 0x7ff29c0767f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0058d0 tx=0x7ff2ac005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:38.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:38.890+0000 7ff2c3000640 1 -- 172.21.15.50:0/546809162 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone26043", "target": ["mon-mgr", ""]}) -- 0x7ff2bc06b540 con 0x7ff29c0742e0 2024-09-21T06:05:40.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.434+0000 7ff2b3e00640 1 -- 172.21.15.50:0/546809162 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff2ac0096c0 con 0x7ff29c0742e0 2024-09-21T06:05:40.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.434+0000 7ff2c3000640 1 -- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff29c0742e0 msgr2=0x7ff29c0767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.434+0000 7ff2c3000640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff29c0742e0 0x7ff29c0767f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0058d0 tx=0x7ff2ac005860 comp rx=0 tx=0).stop 2024-09-21T06:05:40.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.434+0000 7ff2c3000640 1 -- 172.21.15.50:0/546809162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2bc1556b0 msgr2=0x7ff2bc15e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.434+0000 7ff2c3000640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2bc1556b0 0x7ff2bc15e320 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff2b401a760 tx=0x7ff2b4009260 comp rx=0 tx=0).stop 2024-09-21T06:05:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.434+0000 7ff2c3000640 1 -- 172.21.15.50:0/546809162 shutdown_connections 2024-09-21T06:05:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.434+0000 7ff2c3000640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff29c0742e0 0x7ff29c0767f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.434+0000 7ff2c3000640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2bc1556b0 0x7ff2bc15e320 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.434+0000 7ff2c3000640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2bc0d29c0 0x7ff2bc15dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.434+0000 7ff2c3000640 1 --2- 172.21.15.50:0/546809162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2bc0b7270 0x7ff2bc0b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.434+0000 7ff2c3000640 1 -- 172.21.15.50:0/546809162 >> 172.21.15.50:0/546809162 conn(0x7ff2bc0708c0 msgr2=0x7ff2bc16d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.438+0000 7ff2c3000640 1 -- 172.21.15.50:0/546809162 shutdown_connections 2024-09-21T06:05:40.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.438+0000 7ff2c3000640 1 -- 172.21.15.50:0/546809162 wait complete. 2024-09-21T06:05:40.463 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone93454 2024-09-21T06:05:40.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.758+0000 7f3f34c00640 1 -- 172.21.15.50:0/2580466947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f300d1140 msgr2=0x7f3f300b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.758+0000 7f3f34c00640 1 --2- 172.21.15.50:0/2580466947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f300d1140 0x7f3f300b0730 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3f300cb7e0 tx=0x7f3f2001c850 comp rx=0 tx=0).stop 2024-09-21T06:05:40.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.758+0000 7f3f34c00640 1 -- 172.21.15.50:0/2580466947 shutdown_connections 2024-09-21T06:05:40.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.758+0000 7f3f34c00640 1 --2- 172.21.15.50:0/2580466947 >> v2:172.21.15.88:3300/0 conn(0x7f3f3015f5c0 0x7f3f3015f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.758+0000 7f3f34c00640 1 --2- 172.21.15.50:0/2580466947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f300d1140 0x7f3f300b0730 secure :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3f300cb7e0 tx=0x7f3f2001c850 comp rx=0 tx=0).stop 2024-09-21T06:05:40.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.758+0000 7f3f34c00640 1 -- 172.21.15.50:0/2580466947 >> 172.21.15.50:0/2580466947 conn(0x7f3f300b41f0 msgr2=0x7f3f300b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:40.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.762+0000 7f3f34c00640 1 -- 172.21.15.50:0/2580466947 shutdown_connections 2024-09-21T06:05:40.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.762+0000 7f3f34c00640 1 -- 172.21.15.50:0/2580466947 wait complete. 2024-09-21T06:05:40.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.766+0000 7f3f34c00640 1 Processor -- start 2024-09-21T06:05:40.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.766+0000 7f3f34c00640 1 -- start start 2024-09-21T06:05:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f34c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3006dbc0 0x7f3f30156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f34c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f3015f5c0 0x7f3f30157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f34c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f301585d0 0x7f3f301575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f34c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30168ff0 con 0x7f3f3006dbc0 2024-09-21T06:05:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f34c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30168e70 con 0x7f3f301585d0 2024-09-21T06:05:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f34c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3f30169170 con 0x7f3f3015f5c0 2024-09-21T06:05:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f2f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3006dbc0 0x7f3f30156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f2f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3006dbc0 0x7f3f30156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51630/0 (socket says 172.21.15.50:51630) 2024-09-21T06:05:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f2f400640 1 -- 172.21.15.50:0/2122364067 learned_addr learned my addr 172.21.15.50:0/2122364067 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f2ea00640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f3015f5c0 0x7f3f30157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f2f400640 1 -- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f3015f5c0 msgr2=0x7f3f30157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f2f400640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f3015f5c0 0x7f3f30157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f2f400640 1 -- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f301585d0 msgr2=0x7f3f301575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.770+0000 7f3f2f400640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f301585d0 0x7f3f301575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.774+0000 7f3f2f400640 1 -- 172.21.15.50:0/2122364067 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f300b3850 con 0x7f3f3006dbc0 2024-09-21T06:05:40.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.778+0000 7f3f2f400640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3006dbc0 0x7f3f30156840 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009850 tx=0x7f3f20004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:40.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.778+0000 7f3f27400640 1 -- 172.21.15.50:0/2122364067 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f200b2040 con 0x7f3f3006dbc0 2024-09-21T06:05:40.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.778+0000 7f3f27400640 1 -- 172.21.15.50:0/2122364067 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f20005020 con 0x7f3f3006dbc0 2024-09-21T06:05:40.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.778+0000 7f3f27400640 1 -- 172.21.15.50:0/2122364067 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20005430 con 0x7f3f3006dbc0 2024-09-21T06:05:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.778+0000 7f3f34c00640 1 -- 172.21.15.50:0/2122364067 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3009c290 con 0x7f3f3006dbc0 2024-09-21T06:05:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.778+0000 7f3f34c00640 1 -- 172.21.15.50:0/2122364067 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3009d540 con 0x7f3f3006dbc0 2024-09-21T06:05:40.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.778+0000 7f3f34c00640 1 -- 172.21.15.50:0/2122364067 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f30069be0 con 0x7f3f3006dbc0 2024-09-21T06:05:40.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.782+0000 7f3f27400640 1 -- 172.21.15.50:0/2122364067 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7f3f200bf060 con 0x7f3f3006dbc0 2024-09-21T06:05:40.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.786+0000 7f3f27400640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f100742e0 0x7f3f100767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.786+0000 7f3f27400640 1 -- 172.21.15.50:0/2122364067 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f3f2012d450 con 0x7f3f3006dbc0 2024-09-21T06:05:40.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.790+0000 7f3f27400640 1 -- 172.21.15.50:0/2122364067 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3f3009c290 con 0x7f3f3006dbc0 2024-09-21T06:05:40.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.794+0000 7f3f2ea00640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f100742e0 0x7f3f100767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:40.814+0000 7f3f2ea00640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f100742e0 0x7f3f100767f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3f30165d00 tx=0x7f3f1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:41.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:41.162+0000 7f3f34c00640 1 -- 172.21.15.50:0/2122364067 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone93454", "target": ["mon-mgr", ""]}) -- 0x7f3f30070530 con 0x7f3f100742e0 2024-09-21T06:05:42.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.030+0000 7f3f27400640 1 -- 172.21.15.50:0/2122364067 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3f30070530 con 0x7f3f100742e0 2024-09-21T06:05:42.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.034+0000 7f3f34c00640 1 -- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f100742e0 msgr2=0x7f3f100767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:42.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.034+0000 7f3f34c00640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f100742e0 0x7f3f100767f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3f30165d00 tx=0x7f3f1c008040 comp rx=0 tx=0).stop 2024-09-21T06:05:42.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.034+0000 7f3f34c00640 1 -- 172.21.15.50:0/2122364067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3006dbc0 msgr2=0x7f3f30156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:42.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.034+0000 7f3f34c00640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3006dbc0 0x7f3f30156840 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009850 tx=0x7f3f20004040 comp rx=0 tx=0).stop 2024-09-21T06:05:42.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.034+0000 7f3f34c00640 1 -- 172.21.15.50:0/2122364067 shutdown_connections 2024-09-21T06:05:42.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.034+0000 7f3f34c00640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f301585d0 0x7f3f301575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:42.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.034+0000 7f3f34c00640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f3015f5c0 0x7f3f30157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:42.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.034+0000 7f3f34c00640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f100742e0 0x7f3f100767f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:42.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.034+0000 7f3f34c00640 1 --2- 172.21.15.50:0/2122364067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3006dbc0 0x7f3f30156840 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:42.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.034+0000 7f3f34c00640 1 -- 172.21.15.50:0/2122364067 >> 172.21.15.50:0/2122364067 conn(0x7f3f300b41f0 msgr2=0x7f3f300b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:42.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.034+0000 7f3f34c00640 1 -- 172.21.15.50:0/2122364067 shutdown_connections 2024-09-21T06:05:42.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.034+0000 7f3f34c00640 1 -- 172.21.15.50:0/2122364067 wait complete. 2024-09-21T06:05:42.072 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone71335 2024-09-21T06:05:42.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.314+0000 7f3767e00640 1 -- 172.21.15.50:0/2787871256 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375400b640 con 0x7f3750004810 2024-09-21T06:05:42.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.314+0000 7f3770200640 1 -- 172.21.15.50:0/2787871256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3750004810 msgr2=0x7f3750004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:42.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.314+0000 7f3770200640 1 --2- 172.21.15.50:0/2787871256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3750004810 0x7f3750004c20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f37680722b0 tx=0x7f375401cba0 comp rx=0 tx=0).stop 2024-09-21T06:05:42.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.314+0000 7f376e200640 1 -- 172.21.15.50:0/2787871256 reap_dead start 2024-09-21T06:05:42.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.314+0000 7f3770200640 1 -- 172.21.15.50:0/2787871256 shutdown_connections 2024-09-21T06:05:42.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.314+0000 7f3770200640 1 -- 172.21.15.50:0/2787871256 >> 172.21.15.50:0/2787871256 conn(0x7f37680b2040 msgr2=0x7f37680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:42.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.314+0000 7f3770200640 1 -- 172.21.15.50:0/2787871256 shutdown_connections 2024-09-21T06:05:42.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.314+0000 7f3770200640 1 -- 172.21.15.50:0/2787871256 wait complete. 2024-09-21T06:05:42.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.318+0000 7f3770200640 1 Processor -- start 2024-09-21T06:05:42.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.318+0000 7f3770200640 1 -- start start 2024-09-21T06:05:42.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.318+0000 7f3770200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3750004810 0x7f376813d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:42.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.318+0000 7f3770200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3768134800 0x7f376813c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:42.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.318+0000 7f3770200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f376813c640 0x7f37680748b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:42.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.318+0000 7f3770200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3768140590 con 0x7f3768134800 2024-09-21T06:05:42.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.318+0000 7f3770200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f37680b5670 con 0x7f376813c640 2024-09-21T06:05:42.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.318+0000 7f3770200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3768140710 con 0x7f3750004810 2024-09-21T06:05:42.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.318+0000 7f376ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3768134800 0x7f376813c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:42.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.318+0000 7f376ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3768134800 0x7f376813c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51640/0 (socket says 172.21.15.50:51640) 2024-09-21T06:05:42.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.318+0000 7f376ce00640 1 -- 172.21.15.50:0/154830875 learned_addr learned my addr 172.21.15.50:0/154830875 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:42.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.318+0000 7f376e200640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f376813c640 0x7f37680748b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:42.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f376ce00640 1 -- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3750004810 msgr2=0x7f376813d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:42.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f376ce00640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3750004810 0x7f376813d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:42.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f376ce00640 1 -- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f376813c640 msgr2=0x7f37680748b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:42.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f376ce00640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f376813c640 0x7f37680748b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:42.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f376ce00640 1 -- 172.21.15.50:0/154830875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376813b840 con 0x7f3768134800 2024-09-21T06:05:42.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f376ce00640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3768134800 0x7f376813c100 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f37540b1040 tx=0x7f375400b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:42.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f3765600640 1 -- 172.21.15.50:0/154830875 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37540ab040 con 0x7f3768134800 2024-09-21T06:05:42.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f3765600640 1 -- 172.21.15.50:0/154830875 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37540be070 con 0x7f3768134800 2024-09-21T06:05:42.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f3765600640 1 -- 172.21.15.50:0/154830875 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37540b9990 con 0x7f3768134800 2024-09-21T06:05:42.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f3770200640 1 -- 172.21.15.50:0/154830875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3768134d30 con 0x7f3768134800 2024-09-21T06:05:42.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f3770200640 1 -- 172.21.15.50:0/154830875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681342e0 con 0x7f3768134800 2024-09-21T06:05:42.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f3770200640 1 -- 172.21.15.50:0/154830875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376813ba10 con 0x7f3768134800 2024-09-21T06:05:42.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.322+0000 7f3765600640 1 -- 172.21.15.50:0/154830875 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7f37540b9b30 con 0x7f3768134800 2024-09-21T06:05:42.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.326+0000 7f3765600640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f37480742e0 0x7f37480767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:42.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.326+0000 7f3765600640 1 -- 172.21.15.50:0/154830875 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f375412d4d0 con 0x7f3768134800 2024-09-21T06:05:42.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.326+0000 7f376d800640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f37480742e0 0x7f37480767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:42.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.330+0000 7f3765600640 1 -- 172.21.15.50:0/154830875 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f376813ba10 con 0x7f3768134800 2024-09-21T06:05:42.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.330+0000 7f376d800640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f37480742e0 0x7f37480767f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f375c001a40 tx=0x7f375c000f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:42.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:42.718+0000 7f3770200640 1 -- 172.21.15.50:0/154830875 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone71335", "target": ["mon-mgr", ""]}) -- 0x7f376806b540 con 0x7f37480742e0 2024-09-21T06:05:43.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.370+0000 7f3765600640 1 -- 172.21.15.50:0/154830875 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f375c001120 con 0x7f37480742e0 2024-09-21T06:05:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.374+0000 7f3770200640 1 -- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f37480742e0 msgr2=0x7f37480767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.374+0000 7f3770200640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f37480742e0 0x7f37480767f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f375c001a40 tx=0x7f375c000f90 comp rx=0 tx=0).stop 2024-09-21T06:05:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.374+0000 7f3770200640 1 -- 172.21.15.50:0/154830875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3768134800 msgr2=0x7f376813c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.374+0000 7f3770200640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3768134800 0x7f376813c100 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f37540b1040 tx=0x7f375400b640 comp rx=0 tx=0).stop 2024-09-21T06:05:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.374+0000 7f3770200640 1 -- 172.21.15.50:0/154830875 shutdown_connections 2024-09-21T06:05:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.374+0000 7f3770200640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f376813c640 0x7f37680748b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.374+0000 7f3770200640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f37480742e0 0x7f37480767f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.374+0000 7f3770200640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3768134800 0x7f376813c100 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.374+0000 7f3770200640 1 --2- 172.21.15.50:0/154830875 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3750004810 0x7f376813d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.374+0000 7f3770200640 1 -- 172.21.15.50:0/154830875 >> 172.21.15.50:0/154830875 conn(0x7f37680b2040 msgr2=0x7f37680aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:43.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.374+0000 7f3770200640 1 -- 172.21.15.50:0/154830875 shutdown_connections 2024-09-21T06:05:43.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.374+0000 7f3770200640 1 -- 172.21.15.50:0/154830875 wait complete. 2024-09-21T06:05:43.414 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone12326 2024-09-21T06:05:43.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.634+0000 7ff886c00640 1 -- 172.21.15.50:0/4042390403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8800b3340 msgr2=0x7ff8800b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.634+0000 7ff886c00640 1 --2- 172.21.15.50:0/4042390403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8800b3340 0x7ff8800b3750 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff878009c80 tx=0x7ff87801cda0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.634+0000 7ff886c00640 1 -- 172.21.15.50:0/4042390403 shutdown_connections 2024-09-21T06:05:43.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.634+0000 7ff886c00640 1 --2- 172.21.15.50:0/4042390403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8800b3340 0x7ff8800b3750 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.634+0000 7ff886c00640 1 --2- 172.21.15.50:0/4042390403 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8800749a0 0x7ff880074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.634+0000 7ff886c00640 1 -- 172.21.15.50:0/4042390403 >> 172.21.15.50:0/4042390403 conn(0x7ff8800b2040 msgr2=0x7ff8800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:43.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.634+0000 7ff886c00640 1 -- 172.21.15.50:0/4042390403 shutdown_connections 2024-09-21T06:05:43.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.638+0000 7ff886c00640 1 -- 172.21.15.50:0/4042390403 wait complete. 2024-09-21T06:05:43.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.642+0000 7ff886c00640 1 Processor -- start 2024-09-21T06:05:43.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.642+0000 7ff886c00640 1 -- start start 2024-09-21T06:05:43.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.642+0000 7ff886c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8800749a0 0x7ff880209220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.642+0000 7ff886c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8800b3340 0x7ff8801540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.642+0000 7ff886c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8800cefa0 0x7ff8800aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.642+0000 7ff886c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff880169de0 con 0x7ff8800cefa0 2024-09-21T06:05:43.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.642+0000 7ff886c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff880169c60 con 0x7ff8800b3340 2024-09-21T06:05:43.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.642+0000 7ff886c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff880169f60 con 0x7ff8800749a0 2024-09-21T06:05:43.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.642+0000 7ff884c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8800cefa0 0x7ff8800aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.642+0000 7ff884c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8800cefa0 0x7ff8800aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51660/0 (socket says 172.21.15.50:51660) 2024-09-21T06:05:43.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.642+0000 7ff884c00640 1 -- 172.21.15.50:0/1304999952 learned_addr learned my addr 172.21.15.50:0/1304999952 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.646+0000 7ff884c00640 1 -- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8800749a0 msgr2=0x7ff880209220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.646+0000 7ff884c00640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8800749a0 0x7ff880209220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.646+0000 7ff87f400640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8800b3340 0x7ff8801540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.646+0000 7ff884c00640 1 -- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8800b3340 msgr2=0x7ff8801540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.646+0000 7ff884c00640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8800b3340 0x7ff8801540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.646+0000 7ff884c00640 1 -- 172.21.15.50:0/1304999952 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff880209760 con 0x7ff8800cefa0 2024-09-21T06:05:43.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.646+0000 7ff87f400640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8800b3340 0x7ff8801540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:43.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.646+0000 7ff884c00640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8800cefa0 0x7ff8800aee50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff878009bd0 tx=0x7ff878005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:43.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.646+0000 7ff877e00640 1 -- 172.21.15.50:0/1304999952 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780b2040 con 0x7ff8800cefa0 2024-09-21T06:05:43.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.650+0000 7ff877e00640 1 -- 172.21.15.50:0/1304999952 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff878005560 con 0x7ff8800cefa0 2024-09-21T06:05:43.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.650+0000 7ff877e00640 1 -- 172.21.15.50:0/1304999952 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878005970 con 0x7ff8800cefa0 2024-09-21T06:05:43.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.650+0000 7ff886c00640 1 -- 172.21.15.50:0/1304999952 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff880205830 con 0x7ff8800cefa0 2024-09-21T06:05:43.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.650+0000 7ff886c00640 1 -- 172.21.15.50:0/1304999952 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff880205df0 con 0x7ff8800cefa0 2024-09-21T06:05:43.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.650+0000 7ff886c00640 1 -- 172.21.15.50:0/1304999952 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8802055a0 con 0x7ff8800cefa0 2024-09-21T06:05:43.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.654+0000 7ff877e00640 1 -- 172.21.15.50:0/1304999952 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7ff8780ad440 con 0x7ff8800cefa0 2024-09-21T06:05:43.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.654+0000 7ff877e00640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8600742e0 0x7ff8600767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.654+0000 7ff877e00640 1 -- 172.21.15.50:0/1304999952 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7ff878002020 con 0x7ff8800cefa0 2024-09-21T06:05:43.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.658+0000 7ff87fe00640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8600742e0 0x7ff8600767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.662+0000 7ff87fe00640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8600742e0 0x7ff8600767f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff870009690 tx=0x7ff870006c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:43.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:43.662+0000 7ff877e00640 1 -- 172.21.15.50:0/1304999952 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff880209760 con 0x7ff8800cefa0 2024-09-21T06:05:44.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.014+0000 7ff886c00640 1 -- 172.21.15.50:0/1304999952 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone12326", "target": ["mon-mgr", ""]}) -- 0x7ff880074db0 con 0x7ff8600742e0 2024-09-21T06:05:44.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.226+0000 7ff877e00640 1 -- 172.21.15.50:0/1304999952 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff880074db0 con 0x7ff8600742e0 2024-09-21T06:05:44.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.230+0000 7ff886c00640 1 -- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8600742e0 msgr2=0x7ff8600767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:44.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.230+0000 7ff886c00640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8600742e0 0x7ff8600767f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff870009690 tx=0x7ff870006c00 comp rx=0 tx=0).stop 2024-09-21T06:05:44.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.230+0000 7ff886c00640 1 -- 172.21.15.50:0/1304999952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8800cefa0 msgr2=0x7ff8800aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:44.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.230+0000 7ff886c00640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8800cefa0 0x7ff8800aee50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff878009bd0 tx=0x7ff878005040 comp rx=0 tx=0).stop 2024-09-21T06:05:44.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.230+0000 7ff886c00640 1 -- 172.21.15.50:0/1304999952 shutdown_connections 2024-09-21T06:05:44.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.230+0000 7ff886c00640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8600742e0 0x7ff8600767f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.230+0000 7ff886c00640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8800cefa0 0x7ff8800aee50 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.230+0000 7ff886c00640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8800b3340 0x7ff8801540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.230+0000 7ff886c00640 1 --2- 172.21.15.50:0/1304999952 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8800749a0 0x7ff880209220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.230+0000 7ff886c00640 1 -- 172.21.15.50:0/1304999952 >> 172.21.15.50:0/1304999952 conn(0x7ff8800b2040 msgr2=0x7ff88006e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:44.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.230+0000 7ff886c00640 1 -- 172.21.15.50:0/1304999952 shutdown_connections 2024-09-21T06:05:44.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.230+0000 7ff886c00640 1 -- 172.21.15.50:0/1304999952 wait complete. 2024-09-21T06:05:44.250 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:05:44.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.410+0000 7f8c24e00640 1 -- 172.21.15.50:0/2371811821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c201527b0 msgr2=0x7f8c2015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:44.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.410+0000 7f8c24e00640 1 --2- 172.21.15.50:0/2371811821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c201527b0 0x7f8c2015bb60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8c20074b20 tx=0x7f8c0801c680 comp rx=0 tx=0).stop 2024-09-21T06:05:44.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.410+0000 7f8c24e00640 1 -- 172.21.15.50:0/2371811821 shutdown_connections 2024-09-21T06:05:44.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.410+0000 7f8c24e00640 1 --2- 172.21.15.50:0/2371811821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c201527b0 0x7f8c2015bb60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.410+0000 7f8c24e00640 1 -- 172.21.15.50:0/2371811821 >> 172.21.15.50:0/2371811821 conn(0x7f8c200b2040 msgr2=0x7f8c200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:44.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.410+0000 7f8c24e00640 1 -- 172.21.15.50:0/2371811821 shutdown_connections 2024-09-21T06:05:44.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.410+0000 7f8c24e00640 1 -- 172.21.15.50:0/2371811821 wait complete. 2024-09-21T06:05:44.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c24e00640 1 Processor -- start 2024-09-21T06:05:44.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c24e00640 1 -- start start 2024-09-21T06:05:44.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c24e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c200b4de0 0x7f8c201585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:44.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c24e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c200b5c70 0x7f8c200cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:44.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c24e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c201527b0 0x7f8c2013f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:44.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c24e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c20169bb0 con 0x7f8c200b4de0 2024-09-21T06:05:44.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c24e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8c20169a30 con 0x7f8c200b5c70 2024-09-21T06:05:44.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c24e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8c20169d30 con 0x7f8c201527b0 2024-09-21T06:05:44.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c1de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c200b4de0 0x7f8c201585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:44.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c1de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c200b4de0 0x7f8c201585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51676/0 (socket says 172.21.15.50:51676) 2024-09-21T06:05:44.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c1de00640 1 -- 172.21.15.50:0/1463473246 learned_addr learned my addr 172.21.15.50:0/1463473246 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:44.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c17e00640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c200b5c70 0x7f8c200cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:44.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.414+0000 7f8c1e800640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c201527b0 0x7f8c2013f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:44.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c1de00640 1 -- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c201527b0 msgr2=0x7f8c2013f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:44.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c1de00640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c201527b0 0x7f8c2013f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c1de00640 1 -- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c200b5c70 msgr2=0x7f8c200cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:44.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c1de00640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c200b5c70 0x7f8c200cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c1de00640 1 -- 172.21.15.50:0/1463473246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c201fbdb0 con 0x7f8c200b4de0 2024-09-21T06:05:44.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c1de00640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c200b4de0 0x7f8c201585a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000c400 tx=0x7f8c1000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:44.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c16000640 1 -- 172.21.15.50:0/1463473246 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10017070 con 0x7f8c200b4de0 2024-09-21T06:05:44.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c16000640 1 -- 172.21.15.50:0/1463473246 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c10005400 con 0x7f8c200b4de0 2024-09-21T06:05:44.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c24e00640 1 -- 172.21.15.50:0/1463473246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c20158d60 con 0x7f8c200b4de0 2024-09-21T06:05:44.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c16000640 1 -- 172.21.15.50:0/1463473246 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10003860 con 0x7f8c200b4de0 2024-09-21T06:05:44.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c24e00640 1 -- 172.21.15.50:0/1463473246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c201fc350 con 0x7f8c200b4de0 2024-09-21T06:05:44.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c24e00640 1 -- 172.21.15.50:0/1463473246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c2015bb60 con 0x7f8c200b4de0 2024-09-21T06:05:44.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.418+0000 7f8c16000640 1 -- 172.21.15.50:0/1463473246 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7f8c1001a070 con 0x7f8c200b4de0 2024-09-21T06:05:44.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.422+0000 7f8c16000640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8bfc0742e0 0x7f8bfc0767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:44.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.422+0000 7f8c16000640 1 -- 172.21.15.50:0/1463473246 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f8c100923b0 con 0x7f8c200b4de0 2024-09-21T06:05:44.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.422+0000 7f8c17e00640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8bfc0742e0 0x7f8bfc0767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:44.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.422+0000 7f8c17e00640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8bfc0742e0 0x7f8bfc0767f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8c080af000 tx=0x7f8c0801aec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:44.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.426+0000 7f8c16000640 1 -- 172.21.15.50:0/1463473246 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8c1005e500 con 0x7f8c200b4de0 2024-09-21T06:05:44.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.622+0000 7f8c24e00640 1 -- 172.21.15.50:0/1463473246 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f8c2006b540 con 0x7f8bfc0742e0 2024-09-21T06:05:44.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.626+0000 7f8c16000640 1 -- 172.21.15.50:0/1463473246 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f8c080040c0 con 0x7f8bfc0742e0 2024-09-21T06:05:44.627 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:05:44.627 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:05:44.627 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:05:44.627 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:05:44.627 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:05:44.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.626+0000 7f8c24e00640 1 -- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8bfc0742e0 msgr2=0x7f8bfc0767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:44.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.626+0000 7f8c24e00640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8bfc0742e0 0x7f8bfc0767f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8c080af000 tx=0x7f8c0801aec0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.626+0000 7f8c24e00640 1 -- 172.21.15.50:0/1463473246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c200b4de0 msgr2=0x7f8c201585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:44.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.626+0000 7f8c24e00640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c200b4de0 0x7f8c201585a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000c400 tx=0x7f8c1000c8c0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.626+0000 7f8c24e00640 1 -- 172.21.15.50:0/1463473246 shutdown_connections 2024-09-21T06:05:44.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.626+0000 7f8c24e00640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8c201527b0 0x7f8c2013f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.626+0000 7f8c24e00640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c200b5c70 0x7f8c200cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.626+0000 7f8c24e00640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8bfc0742e0 0x7f8bfc0767f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.626+0000 7f8c24e00640 1 --2- 172.21.15.50:0/1463473246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c200b4de0 0x7f8c201585a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.626+0000 7f8c24e00640 1 -- 172.21.15.50:0/1463473246 >> 172.21.15.50:0/1463473246 conn(0x7f8c200b2040 msgr2=0x7f8c2015a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:44.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.630+0000 7f8c24e00640 1 -- 172.21.15.50:0/1463473246 shutdown_connections 2024-09-21T06:05:44.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.630+0000 7f8c24e00640 1 -- 172.21.15.50:0/1463473246 wait complete. 2024-09-21T06:05:44.651 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-21T06:05:44.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.814+0000 7f92e6e00640 1 -- 172.21.15.50:0/1673717725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e00746e0 msgr2=0x7f92e015c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:44.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.814+0000 7f92e6e00640 1 --2- 172.21.15.50:0/1673717725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e00746e0 0x7f92e015c920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009840 tx=0x7f92d401c790 comp rx=0 tx=0).stop 2024-09-21T06:05:44.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.814+0000 7f92e6e00640 1 -- 172.21.15.50:0/1673717725 shutdown_connections 2024-09-21T06:05:44.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.814+0000 7f92e6e00640 1 --2- 172.21.15.50:0/1673717725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e00746e0 0x7f92e015c920 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.814+0000 7f92e6e00640 1 -- 172.21.15.50:0/1673717725 >> 172.21.15.50:0/1673717725 conn(0x7f92e00b41f0 msgr2=0x7f92e00b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:44.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.814+0000 7f92e6e00640 1 -- 172.21.15.50:0/1673717725 shutdown_connections 2024-09-21T06:05:44.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.814+0000 7f92e6e00640 1 -- 172.21.15.50:0/1673717725 wait complete. 2024-09-21T06:05:44.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.814+0000 7f92e6e00640 1 Processor -- start 2024-09-21T06:05:44.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.814+0000 7f92e6e00640 1 -- start start 2024-09-21T06:05:44.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f92e00746e0 0x7f92e009a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:44.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e0074c30 0x7f92e015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:44.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92e00cda80 0x7f92e00b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:44.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f92e016b490 con 0x7f92e00cda80 2024-09-21T06:05:44.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f92e016b310 con 0x7f92e0074c30 2024-09-21T06:05:44.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f92e016b610 con 0x7f92e00746e0 2024-09-21T06:05:44.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92e00cda80 0x7f92e00b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:44.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92e00cda80 0x7f92e00b24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51692/0 (socket says 172.21.15.50:51692) 2024-09-21T06:05:44.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6400640 1 -- 172.21.15.50:0/1230348720 learned_addr learned my addr 172.21.15.50:0/1230348720 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e5a00640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f92e00746e0 0x7f92e009a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e5000640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e0074c30 0x7f92e015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6400640 1 -- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f92e00746e0 msgr2=0x7f92e009a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6400640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f92e00746e0 0x7f92e009a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6400640 1 -- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e0074c30 msgr2=0x7f92e015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6400640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e0074c30 0x7f92e015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6400640 1 -- 172.21.15.50:0/1230348720 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e015be70 con 0x7f92e00cda80 2024-09-21T06:05:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6400640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92e00cda80 0x7f92e00b24e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c710 tx=0x7f92d800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92dd600640 1 -- 172.21.15.50:0/1230348720 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8003b80 con 0x7f92e00cda80 2024-09-21T06:05:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92dd600640 1 -- 172.21.15.50:0/1230348720 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92d8003d20 con 0x7f92e00cda80 2024-09-21T06:05:44.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6e00640 1 -- 172.21.15.50:0/1230348720 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e015b5d0 con 0x7f92e00cda80 2024-09-21T06:05:44.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92dd600640 1 -- 172.21.15.50:0/1230348720 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d800aab0 con 0x7f92e00cda80 2024-09-21T06:05:44.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6e00640 1 -- 172.21.15.50:0/1230348720 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e015bb90 con 0x7f92e00cda80 2024-09-21T06:05:44.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.818+0000 7f92e6e00640 1 -- 172.21.15.50:0/1230348720 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e015b340 con 0x7f92e00cda80 2024-09-21T06:05:44.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.822+0000 7f92dd600640 1 -- 172.21.15.50:0/1230348720 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7f92d800ac50 con 0x7f92e00cda80 2024-09-21T06:05:44.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.822+0000 7f92dd600640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f92c00742e0 0x7f92c00767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:44.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.822+0000 7f92dd600640 1 -- 172.21.15.50:0/1230348720 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f92d8092d80 con 0x7f92e00cda80 2024-09-21T06:05:44.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.822+0000 7f92e5a00640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f92c00742e0 0x7f92c00767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:44.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.822+0000 7f92e5a00640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f92c00742e0 0x7f92c00767f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f92d40af000 tx=0x7f92d401afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:44.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:44.826+0000 7f92dd600640 1 -- 172.21.15.50:0/1230348720 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f92d800ce20 con 0x7f92e00cda80 2024-09-21T06:05:45.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.026+0000 7f92e6e00640 1 -- 172.21.15.50:0/1230348720 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92e015c920 con 0x7f92c00742e0 2024-09-21T06:05:45.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92dd600640 1 -- 172.21.15.50:0/1230348720 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7f92e015c920 con 0x7f92c00742e0 2024-09-21T06:05:45.037 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:45.037 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone91230" 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone71335" 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sv1" 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone26043" 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:05:45.038 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone93454" 2024-09-21T06:05:45.039 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:45.039 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:05:45.039 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone15181" 2024-09-21T06:05:45.039 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:45.039 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:05:45.039 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone12326" 2024-09-21T06:05:45.039 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:05:45.039 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:05:45.039 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone51512" 2024-09-21T06:05:45.039 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:05:45.039 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:05:45.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92e6e00640 1 -- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f92c00742e0 msgr2=0x7f92c00767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:45.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92e6e00640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f92c00742e0 0x7f92c00767f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f92d40af000 tx=0x7f92d401afd0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92e6e00640 1 -- 172.21.15.50:0/1230348720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92e00cda80 msgr2=0x7f92e00b24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:45.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92e6e00640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92e00cda80 0x7f92e00b24e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c710 tx=0x7f92d800cbd0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92e6e00640 1 -- 172.21.15.50:0/1230348720 shutdown_connections 2024-09-21T06:05:45.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92e6e00640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f92c00742e0 0x7f92c00767f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92e6e00640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92e00cda80 0x7f92e00b24e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92e6e00640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e0074c30 0x7f92e015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92e6e00640 1 --2- 172.21.15.50:0/1230348720 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f92e00746e0 0x7f92e009a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92e6e00640 1 -- 172.21.15.50:0/1230348720 >> 172.21.15.50:0/1230348720 conn(0x7f92e00b41f0 msgr2=0x7f92e00cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:45.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92e6e00640 1 -- 172.21.15.50:0/1230348720 shutdown_connections 2024-09-21T06:05:45.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.038+0000 7f92e6e00640 1 -- 172.21.15.50:0/1230348720 wait complete. 2024-09-21T06:05:45.060 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone91230 2024-09-21T06:05:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.222+0000 7ff88e600640 1 -- 172.21.15.50:0/1751271065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8880cf280 msgr2=0x7ff8880cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.222+0000 7ff88e600640 1 --2- 172.21.15.50:0/1751271065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8880cf280 0x7ff8880cf690 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff880009870 tx=0x7ff88001c760 comp rx=0 tx=0).stop 2024-09-21T06:05:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.222+0000 7ff88e600640 1 -- 172.21.15.50:0/1751271065 shutdown_connections 2024-09-21T06:05:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.222+0000 7ff88e600640 1 --2- 172.21.15.50:0/1751271065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8880cf280 0x7ff8880cf690 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.222+0000 7ff88e600640 1 --2- 172.21.15.50:0/1751271065 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff88815e890 0x7ff88815eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.222+0000 7ff88e600640 1 -- 172.21.15.50:0/1751271065 >> 172.21.15.50:0/1751271065 conn(0x7ff8880b2040 msgr2=0x7ff8880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.222+0000 7ff88e600640 1 -- 172.21.15.50:0/1751271065 shutdown_connections 2024-09-21T06:05:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.222+0000 7ff88e600640 1 -- 172.21.15.50:0/1751271065 wait complete. 2024-09-21T06:05:45.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff88e600640 1 Processor -- start 2024-09-21T06:05:45.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff88e600640 1 -- start start 2024-09-21T06:05:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff88e600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8880cf280 0x7ff888142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff88e600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff88813e790 0x7ff8881513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff88e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff88815e890 0x7ff888138fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff88e600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff8880b54b0 con 0x7ff88815e890 2024-09-21T06:05:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff88e600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff8880b5330 con 0x7ff88813e790 2024-09-21T06:05:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff88e600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff8880b5630 con 0x7ff8880cf280 2024-09-21T06:05:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff887e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff88815e890 0x7ff888138fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff887e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff88815e890 0x7ff888138fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51698/0 (socket says 172.21.15.50:51698) 2024-09-21T06:05:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff887e00640 1 -- 172.21.15.50:0/2348090341 learned_addr learned my addr 172.21.15.50:0/2348090341 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff87ea00640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff88813e790 0x7ff8881513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff887400640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8880cf280 0x7ff888142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff887e00640 1 -- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8880cf280 msgr2=0x7ff888142460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff887e00640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8880cf280 0x7ff888142460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff887e00640 1 -- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff88813e790 msgr2=0x7ff8881513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff887e00640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff88813e790 0x7ff8881513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff887e00640 1 -- 172.21.15.50:0/2348090341 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8881518f0 con 0x7ff88815e890 2024-09-21T06:05:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff887e00640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff88815e890 0x7ff888138fc0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff880098340 tx=0x7ff8800056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff87fe00640 1 -- 172.21.15.50:0/2348090341 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8800b2040 con 0x7ff88815e890 2024-09-21T06:05:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff87fe00640 1 -- 172.21.15.50:0/2348090341 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8800bb070 con 0x7ff88815e890 2024-09-21T06:05:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff88e600640 1 -- 172.21.15.50:0/2348090341 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff888135a80 con 0x7ff88815e890 2024-09-21T06:05:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff87fe00640 1 -- 172.21.15.50:0/2348090341 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880003600 con 0x7ff88815e890 2024-09-21T06:05:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.226+0000 7ff88e600640 1 -- 172.21.15.50:0/2348090341 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88813bbf0 con 0x7ff88815e890 2024-09-21T06:05:45.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.230+0000 7ff88e600640 1 -- 172.21.15.50:0/2348090341 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8880cf690 con 0x7ff88815e890 2024-09-21T06:05:45.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.230+0000 7ff87fe00640 1 -- 172.21.15.50:0/2348090341 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7ff8800037a0 con 0x7ff88815e890 2024-09-21T06:05:45.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.230+0000 7ff87fe00640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8640742e0 0x7ff8640767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:45.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.230+0000 7ff87fe00640 1 -- 172.21.15.50:0/2348090341 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7ff88012d080 con 0x7ff88815e890 2024-09-21T06:05:45.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.230+0000 7ff887400640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8640742e0 0x7ff8640767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:45.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.230+0000 7ff887400640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8640742e0 0x7ff8640767f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff878001780 tx=0x7ff878001e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:45.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.234+0000 7ff87fe00640 1 -- 172.21.15.50:0/2348090341 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff880016410 con 0x7ff88815e890 2024-09-21T06:05:45.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.438+0000 7ff88e600640 1 -- 172.21.15.50:0/2348090341 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone91230", "target": ["mon-mgr", ""]}) -- 0x7ff8880b6360 con 0x7ff8640742e0 2024-09-21T06:05:45.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff87fe00640 1 -- 172.21.15.50:0/2348090341 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff8880b6360 con 0x7ff8640742e0 2024-09-21T06:05:45.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff88e600640 1 -- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8640742e0 msgr2=0x7ff8640767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:45.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff88e600640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8640742e0 0x7ff8640767f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff878001780 tx=0x7ff878001e80 comp rx=0 tx=0).stop 2024-09-21T06:05:45.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff88e600640 1 -- 172.21.15.50:0/2348090341 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff88815e890 msgr2=0x7ff888138fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:45.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff88e600640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff88815e890 0x7ff888138fc0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff880098340 tx=0x7ff8800056e0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff88e600640 1 -- 172.21.15.50:0/2348090341 shutdown_connections 2024-09-21T06:05:45.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff88e600640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8640742e0 0x7ff8640767f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff88e600640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff88815e890 0x7ff888138fc0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff88e600640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff88813e790 0x7ff8881513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff88e600640 1 --2- 172.21.15.50:0/2348090341 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8880cf280 0x7ff888142460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff88e600640 1 -- 172.21.15.50:0/2348090341 >> 172.21.15.50:0/2348090341 conn(0x7ff8880b2040 msgr2=0x7ff88813ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:45.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff88e600640 1 -- 172.21.15.50:0/2348090341 shutdown_connections 2024-09-21T06:05:45.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.462+0000 7ff88e600640 1 -- 172.21.15.50:0/2348090341 wait complete. 2024-09-21T06:05:45.476 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:45.477+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:45.477 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:45.477+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:45.477 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:45.477+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:45.477 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:45.477+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:45.477 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:05:45.477+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:05:45.486 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone71335 2024-09-21T06:05:45.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.650+0000 7f55b1600640 1 -- 172.21.15.50:0/2355597312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac072100 msgr2=0x7f55ac155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:45.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.650+0000 7f55b1600640 1 --2- 172.21.15.50:0/2355597312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac072100 0x7f55ac155b50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f55ac071130 tx=0x7f559c01c8c0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.650+0000 7f55b1600640 1 -- 172.21.15.50:0/2355597312 shutdown_connections 2024-09-21T06:05:45.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.650+0000 7f55b1600640 1 --2- 172.21.15.50:0/2355597312 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55ac15f0e0 0x7f55ac15f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.650+0000 7f55b1600640 1 --2- 172.21.15.50:0/2355597312 >> v2:172.21.15.88:3300/0 conn(0x7f55ac0b4de0 0x7f55ac0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.650+0000 7f55b1600640 1 --2- 172.21.15.50:0/2355597312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac072100 0x7f55ac155b50 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.650+0000 7f55b1600640 1 -- 172.21.15.50:0/2355597312 >> 172.21.15.50:0/2355597312 conn(0x7f55ac0b2040 msgr2=0x7f55ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.650+0000 7f55b1600640 1 -- 172.21.15.50:0/2355597312 shutdown_connections 2024-09-21T06:05:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.650+0000 7f55b1600640 1 -- 172.21.15.50:0/2355597312 wait complete. 2024-09-21T06:05:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.650+0000 7f55b1600640 1 Processor -- start 2024-09-21T06:05:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.650+0000 7f55b1600640 1 -- start start 2024-09-21T06:05:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55b1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac072100 0x7f55ac1f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55b1600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55ac0b4de0 0x7f55ac1f9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55b1600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55ac15f0e0 0x7f55ac0aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55b1600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac074990 con 0x7f55ac072100 2024-09-21T06:05:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55b1600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac074810 con 0x7f55ac0b4de0 2024-09-21T06:05:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55b1600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f55ac074b10 con 0x7f55ac15f0e0 2024-09-21T06:05:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55aaa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac072100 0x7f55ac1f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55aaa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac072100 0x7f55ac1f6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51722/0 (socket says 172.21.15.50:51722) 2024-09-21T06:05:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55aaa00640 1 -- 172.21.15.50:0/2097156960 learned_addr learned my addr 172.21.15.50:0/2097156960 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55a3e00640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55ac0b4de0 0x7f55ac1f9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55ab400640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55ac15f0e0 0x7f55ac0aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55aaa00640 1 -- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55ac15f0e0 msgr2=0x7f55ac0aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55aaa00640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55ac15f0e0 0x7f55ac0aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55aaa00640 1 -- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55ac0b4de0 msgr2=0x7f55ac1f9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55aaa00640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55ac0b4de0 0x7f55ac1f9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55aaa00640 1 -- 172.21.15.50:0/2097156960 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ac1f8080 con 0x7f55ac072100 2024-09-21T06:05:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55ab400640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55ac15f0e0 0x7f55ac0aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55aaa00640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac072100 0x7f55ac1f6b90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f559c0b0040 tx=0x7f559c005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55a2a00640 1 -- 172.21.15.50:0/2097156960 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c0b2040 con 0x7f55ac072100 2024-09-21T06:05:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55a2a00640 1 -- 172.21.15.50:0/2097156960 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f559c004330 con 0x7f55ac072100 2024-09-21T06:05:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55b1600640 1 -- 172.21.15.50:0/2097156960 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ac1f7680 con 0x7f55ac072100 2024-09-21T06:05:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55a2a00640 1 -- 172.21.15.50:0/2097156960 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c005280 con 0x7f55ac072100 2024-09-21T06:05:45.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55b1600640 1 -- 172.21.15.50:0/2097156960 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ac158c00 con 0x7f55ac072100 2024-09-21T06:05:45.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.654+0000 7f55b1600640 1 -- 172.21.15.50:0/2097156960 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55ac1f8250 con 0x7f55ac072100 2024-09-21T06:05:45.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.658+0000 7f55a2a00640 1 -- 172.21.15.50:0/2097156960 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7f559c005420 con 0x7f55ac072100 2024-09-21T06:05:45.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.658+0000 7f55a2a00640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55880742e0 0x7f55880767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:45.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.658+0000 7f55a2a00640 1 -- 172.21.15.50:0/2097156960 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f559c12cf60 con 0x7f55ac072100 2024-09-21T06:05:45.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.658+0000 7f55a3e00640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55880742e0 0x7f55880767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:45.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.658+0000 7f55a3e00640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55880742e0 0x7f55880767f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f55ac15f8d0 tx=0x7f55940091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:45.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.662+0000 7f55a2a00640 1 -- 172.21.15.50:0/2097156960 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f559c0164b0 con 0x7f55ac072100 2024-09-21T06:05:45.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.866+0000 7f55b1600640 1 -- 172.21.15.50:0/2097156960 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone71335", "target": ["mon-mgr", ""]}) -- 0x7f55ac06b540 con 0x7f55880742e0 2024-09-21T06:05:45.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55a2a00640 1 -- 172.21.15.50:0/2097156960 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5594011b50 con 0x7f55880742e0 2024-09-21T06:05:45.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55b1600640 1 -- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55880742e0 msgr2=0x7f55880767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:45.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55b1600640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55880742e0 0x7f55880767f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f55ac15f8d0 tx=0x7f55940091e0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55b1600640 1 -- 172.21.15.50:0/2097156960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac072100 msgr2=0x7f55ac1f6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:45.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55b1600640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac072100 0x7f55ac1f6b90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f559c0b0040 tx=0x7f559c005d60 comp rx=0 tx=0).stop 2024-09-21T06:05:45.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55b1600640 1 -- 172.21.15.50:0/2097156960 shutdown_connections 2024-09-21T06:05:45.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55b1600640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55ac15f0e0 0x7f55ac0aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55b1600640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55ac0b4de0 0x7f55ac1f9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55b1600640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55880742e0 0x7f55880767f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55b1600640 1 --2- 172.21.15.50:0/2097156960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac072100 0x7f55ac1f6b90 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:45.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55b1600640 1 -- 172.21.15.50:0/2097156960 >> 172.21.15.50:0/2097156960 conn(0x7f55ac0b2040 msgr2=0x7f55ac0b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:45.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55b1600640 1 -- 172.21.15.50:0/2097156960 shutdown_connections 2024-09-21T06:05:45.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:45.886+0000 7f55b1600640 1 -- 172.21.15.50:0/2097156960 wait complete. 2024-09-21T06:05:45.910 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-21T06:05:46.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.074+0000 7ff54b800640 1 -- 172.21.15.50:0/37104111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5440b4de0 msgr2=0x7ff5440b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.074+0000 7ff54b800640 1 --2- 172.21.15.50:0/37104111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5440b4de0 0x7ff5440b51f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009c80 tx=0x7ff53c01cce0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.074+0000 7ff54b800640 1 -- 172.21.15.50:0/37104111 shutdown_connections 2024-09-21T06:05:46.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.074+0000 7ff54b800640 1 --2- 172.21.15.50:0/37104111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5440b4de0 0x7ff5440b51f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.074+0000 7ff54b800640 1 --2- 172.21.15.50:0/37104111 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff5440b5c70 0x7ff54415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.074+0000 7ff54b800640 1 -- 172.21.15.50:0/37104111 >> 172.21.15.50:0/37104111 conn(0x7ff5440b2040 msgr2=0x7ff5440b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:46.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.074+0000 7ff54b800640 1 -- 172.21.15.50:0/37104111 shutdown_connections 2024-09-21T06:05:46.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.074+0000 7ff54b800640 1 -- 172.21.15.50:0/37104111 wait complete. 2024-09-21T06:05:46.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.074+0000 7ff54b800640 1 Processor -- start 2024-09-21T06:05:46.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.074+0000 7ff54b800640 1 -- start start 2024-09-21T06:05:46.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff54b800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff5440b4de0 0x7ff5441fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:46.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff54b800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5440b5c70 0x7ff5441561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:46.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff54b800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5441527b0 0x7ff5441f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:46.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff54b800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff544168ab0 con 0x7ff5440b5c70 2024-09-21T06:05:46.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff54b800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff544168930 con 0x7ff5441527b0 2024-09-21T06:05:46.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff54b800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff544168c30 con 0x7ff5440b4de0 2024-09-21T06:05:46.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff53be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5440b5c70 0x7ff5441561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:46.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff53be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5440b5c70 0x7ff5441561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51736/0 (socket says 172.21.15.50:51736) 2024-09-21T06:05:46.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff53be00640 1 -- 172.21.15.50:0/4131857513 learned_addr learned my addr 172.21.15.50:0/4131857513 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:46.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff549800640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5441527b0 0x7ff5441f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:46.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff548e00640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff5440b4de0 0x7ff5441fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff53be00640 1 -- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff5440b4de0 msgr2=0x7ff5441fc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff53be00640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff5440b4de0 0x7ff5441fc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff53be00640 1 -- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5441527b0 msgr2=0x7ff5441f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff53be00640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5441527b0 0x7ff5441f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff53be00640 1 -- 172.21.15.50:0/4131857513 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff544156700 con 0x7ff5440b5c70 2024-09-21T06:05:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff53be00640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5440b5c70 0x7ff5441561c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00c3c0 tx=0x7ff52c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff541600640 1 -- 172.21.15.50:0/4131857513 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c017070 con 0x7ff5440b5c70 2024-09-21T06:05:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff541600640 1 -- 172.21.15.50:0/4131857513 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff52c0053c0 con 0x7ff5440b5c70 2024-09-21T06:05:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff54b800640 1 -- 172.21.15.50:0/4131857513 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5440c5500 con 0x7ff5440b5c70 2024-09-21T06:05:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff541600640 1 -- 172.21.15.50:0/4131857513 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c003840 con 0x7ff5440b5c70 2024-09-21T06:05:46.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff54b800640 1 -- 172.21.15.50:0/4131857513 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5441f8a50 con 0x7ff5440b5c70 2024-09-21T06:05:46.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.078+0000 7ff54b800640 1 -- 172.21.15.50:0/4131857513 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5440b51f0 con 0x7ff5440b5c70 2024-09-21T06:05:46.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.082+0000 7ff541600640 1 -- 172.21.15.50:0/4131857513 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7ff52c01a070 con 0x7ff5440b5c70 2024-09-21T06:05:46.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.082+0000 7ff541600640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5200742e0 0x7ff5200767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:46.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.082+0000 7ff541600640 1 -- 172.21.15.50:0/4131857513 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7ff52c092370 con 0x7ff5440b5c70 2024-09-21T06:05:46.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.082+0000 7ff548e00640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5200742e0 0x7ff5200767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:46.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.082+0000 7ff548e00640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5200742e0 0x7ff5200767f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff53400bb50 tx=0x7ff53400b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:46.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.086+0000 7ff541600640 1 -- 172.21.15.50:0/4131857513 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff5440b5730 con 0x7ff5440b5c70 2024-09-21T06:05:46.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.294+0000 7ff54b800640 1 -- 172.21.15.50:0/4131857513 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5440c52b0 con 0x7ff5200742e0 2024-09-21T06:05:46.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff541600640 1 -- 172.21.15.50:0/4131857513 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff5440c52b0 con 0x7ff5200742e0 2024-09-21T06:05:46.307 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:46.307 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:05:46.307 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:05:46.307 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ss1" 2024-09-21T06:05:46.307 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:05:46.307 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:05:46.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff54b800640 1 -- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5200742e0 msgr2=0x7ff5200767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff54b800640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5200742e0 0x7ff5200767f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff53400bb50 tx=0x7ff53400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff54b800640 1 -- 172.21.15.50:0/4131857513 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5440b5c70 msgr2=0x7ff5441561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff54b800640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5440b5c70 0x7ff5441561c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00c3c0 tx=0x7ff52c00c880 comp rx=0 tx=0).stop 2024-09-21T06:05:46.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff54b800640 1 -- 172.21.15.50:0/4131857513 shutdown_connections 2024-09-21T06:05:46.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff54b800640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5441527b0 0x7ff5441f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff54b800640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5200742e0 0x7ff5200767f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff54b800640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5440b5c70 0x7ff5441561c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff54b800640 1 --2- 172.21.15.50:0/4131857513 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff5440b4de0 0x7ff5441fc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff54b800640 1 -- 172.21.15.50:0/4131857513 >> 172.21.15.50:0/4131857513 conn(0x7ff5440b2040 msgr2=0x7ff54415b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:46.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff54b800640 1 -- 172.21.15.50:0/4131857513 shutdown_connections 2024-09-21T06:05:46.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.306+0000 7ff54b800640 1 -- 172.21.15.50:0/4131857513 wait complete. 2024-09-21T06:05:46.330 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-21T06:05:46.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.494+0000 7f2de9600640 1 -- 172.21.15.50:0/1071745933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2de40b4de0 msgr2=0x7f2de40b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.494+0000 7f2de9600640 1 --2- 172.21.15.50:0/1071745933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2de40b4de0 0x7f2de40b51f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2de4071cb0 tx=0x7f2ddc01ccc0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.494+0000 7f2de9600640 1 -- 172.21.15.50:0/1071745933 shutdown_connections 2024-09-21T06:05:46.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.494+0000 7f2de9600640 1 --2- 172.21.15.50:0/1071745933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2de40b4de0 0x7f2de40b51f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.494+0000 7f2de9600640 1 --2- 172.21.15.50:0/1071745933 >> v2:172.21.15.88:3300/0 conn(0x7f2de40b5c70 0x7f2de415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.494+0000 7f2de9600640 1 -- 172.21.15.50:0/1071745933 >> 172.21.15.50:0/1071745933 conn(0x7f2de40b2040 msgr2=0x7f2de40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:46.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.494+0000 7f2de9600640 1 -- 172.21.15.50:0/1071745933 shutdown_connections 2024-09-21T06:05:46.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.494+0000 7f2de9600640 1 -- 172.21.15.50:0/1071745933 wait complete. 2024-09-21T06:05:46.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2de9600640 1 Processor -- start 2024-09-21T06:05:46.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2de9600640 1 -- start start 2024-09-21T06:05:46.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2de9600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2de40b4de0 0x7f2de41fc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:46.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2de9600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2de40b5c70 0x7f2de414eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:46.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2de9600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2de41527b0 0x7f2de41f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:46.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2de9600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4168920 con 0x7f2de40b5c70 2024-09-21T06:05:46.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2de9600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41687a0 con 0x7f2de40b4de0 2024-09-21T06:05:46.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2de9600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2de4168aa0 con 0x7f2de41527b0 2024-09-21T06:05:46.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2ddbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2de40b5c70 0x7f2de414eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:46.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2ddbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2de40b5c70 0x7f2de414eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51748/0 (socket says 172.21.15.50:51748) 2024-09-21T06:05:46.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2de3400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2de41527b0 0x7f2de41f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:46.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2ddbe00640 1 -- 172.21.15.50:0/987044208 learned_addr learned my addr 172.21.15.50:0/987044208 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:46.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2de2a00640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2de40b4de0 0x7f2de41fc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:46.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2ddbe00640 1 -- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2de41527b0 msgr2=0x7f2de41f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2ddbe00640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2de41527b0 0x7f2de41f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2ddbe00640 1 -- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2de40b4de0 msgr2=0x7f2de41fc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2ddbe00640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2de40b4de0 0x7f2de41fc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2ddbe00640 1 -- 172.21.15.50:0/987044208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de40cf440 con 0x7f2de40b5c70 2024-09-21T06:05:46.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2ddbe00640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2de40b5c70 0x7f2de414eb50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00c3f0 tx=0x7f2dcc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:46.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2ddaa00640 1 -- 172.21.15.50:0/987044208 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc017070 con 0x7f2de40b5c70 2024-09-21T06:05:46.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2ddaa00640 1 -- 172.21.15.50:0/987044208 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dcc0053c0 con 0x7f2de40b5c70 2024-09-21T06:05:46.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2de9600640 1 -- 172.21.15.50:0/987044208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de40cf720 con 0x7f2de40b5c70 2024-09-21T06:05:46.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2ddaa00640 1 -- 172.21.15.50:0/987044208 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc003860 con 0x7f2de40b5c70 2024-09-21T06:05:46.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.498+0000 7f2de9600640 1 -- 172.21.15.50:0/987044208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de40cf110 con 0x7f2de40b5c70 2024-09-21T06:05:46.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.502+0000 7f2de9600640 1 -- 172.21.15.50:0/987044208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de40b51f0 con 0x7f2de40b5c70 2024-09-21T06:05:46.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.502+0000 7f2ddaa00640 1 -- 172.21.15.50:0/987044208 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7f2dcc01a070 con 0x7f2de40b5c70 2024-09-21T06:05:46.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.502+0000 7f2ddaa00640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2dc0074310 0x7f2dc0076820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:46.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.502+0000 7f2ddaa00640 1 -- 172.21.15.50:0/987044208 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f2dcc092330 con 0x7f2de40b5c70 2024-09-21T06:05:46.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.502+0000 7f2de2a00640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2dc0074310 0x7f2dc0076820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:46.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.506+0000 7f2de2a00640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2dc0074310 0x7f2dc0076820 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4004630 tx=0x7f2dd400b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:46.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.506+0000 7f2ddaa00640 1 -- 172.21.15.50:0/987044208 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2de40b5730 con 0x7f2de40b5c70 2024-09-21T06:05:46.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.718+0000 7f2de9600640 1 -- 172.21.15.50:0/987044208 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2de414f090 con 0x7f2dc0074310 2024-09-21T06:05:46.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.746+0000 7f2ddaa00640 1 -- 172.21.15.50:0/987044208 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2de414f090 con 0x7f2dc0074310 2024-09-21T06:05:46.747 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:46.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.746+0000 7f2de9600640 1 -- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2dc0074310 msgr2=0x7f2dc0076820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.746+0000 7f2de9600640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2dc0074310 0x7f2dc0076820 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4004630 tx=0x7f2dd400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.746+0000 7f2de9600640 1 -- 172.21.15.50:0/987044208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2de40b5c70 msgr2=0x7f2de414eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.746+0000 7f2de9600640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2de40b5c70 0x7f2de414eb50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00c3f0 tx=0x7f2dcc00c8b0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.746+0000 7f2de9600640 1 -- 172.21.15.50:0/987044208 shutdown_connections 2024-09-21T06:05:46.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.746+0000 7f2de9600640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2de41527b0 0x7f2de41f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.746+0000 7f2de9600640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2dc0074310 0x7f2dc0076820 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.746+0000 7f2de9600640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2de40b5c70 0x7f2de414eb50 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.746+0000 7f2de9600640 1 --2- 172.21.15.50:0/987044208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2de40b4de0 0x7f2de41fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.746+0000 7f2de9600640 1 -- 172.21.15.50:0/987044208 >> 172.21.15.50:0/987044208 conn(0x7f2de40b2040 msgr2=0x7f2de4151c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:46.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.750+0000 7f2de9600640 1 -- 172.21.15.50:0/987044208 shutdown_connections 2024-09-21T06:05:46.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.750+0000 7f2de9600640 1 -- 172.21.15.50:0/987044208 wait complete. 2024-09-21T06:05:46.770 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-21T06:05:46.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.930+0000 7efc63600640 1 -- 172.21.15.50:0/1588642287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c072100 msgr2=0x7efc5c072510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.930+0000 7efc63600640 1 --2- 172.21.15.50:0/1588642287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c072100 0x7efc5c072510 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efc54009bc0 tx=0x7efc5401ca10 comp rx=0 tx=0).stop 2024-09-21T06:05:46.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.930+0000 7efc63600640 1 -- 172.21.15.50:0/1588642287 shutdown_connections 2024-09-21T06:05:46.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.930+0000 7efc63600640 1 --2- 172.21.15.50:0/1588642287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c072100 0x7efc5c072510 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.930+0000 7efc63600640 1 --2- 172.21.15.50:0/1588642287 >> v2:172.21.15.88:3300/0 conn(0x7efc5c0cefa0 0x7efc5c0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.930+0000 7efc63600640 1 -- 172.21.15.50:0/1588642287 >> 172.21.15.50:0/1588642287 conn(0x7efc5c0b2040 msgr2=0x7efc5c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:46.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.930+0000 7efc63600640 1 -- 172.21.15.50:0/1588642287 shutdown_connections 2024-09-21T06:05:46.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.930+0000 7efc63600640 1 -- 172.21.15.50:0/1588642287 wait complete. 2024-09-21T06:05:46.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc63600640 1 Processor -- start 2024-09-21T06:05:46.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc63600640 1 -- start start 2024-09-21T06:05:46.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc63600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c072100 0x7efc5c0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:46.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc63600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efc5c0cefa0 0x7efc5c0ce400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:46.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc63600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc5c15cbb0 0x7efc5c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:46.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc63600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c169ed0 con 0x7efc5c072100 2024-09-21T06:05:46.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc63600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c169d50 con 0x7efc5c15cbb0 2024-09-21T06:05:46.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc63600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7efc5c16a050 con 0x7efc5c0cefa0 2024-09-21T06:05:46.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc60c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c072100 0x7efc5c0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:46.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc60c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c072100 0x7efc5c0b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46334/0 (socket says 172.21.15.50:46334) 2024-09-21T06:05:46.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc60c00640 1 -- 172.21.15.50:0/3886152772 learned_addr learned my addr 172.21.15.50:0/3886152772 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:46.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc5be00640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efc5c0cefa0 0x7efc5c0ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:46.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc61600640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc5c15cbb0 0x7efc5c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc60c00640 1 -- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efc5c0cefa0 msgr2=0x7efc5c0ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc60c00640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efc5c0cefa0 0x7efc5c0ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc60c00640 1 -- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc5c15cbb0 msgr2=0x7efc5c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc60c00640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc5c15cbb0 0x7efc5c0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc60c00640 1 -- 172.21.15.50:0/3886152772 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c13edb0 con 0x7efc5c072100 2024-09-21T06:05:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.934+0000 7efc60c00640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c072100 0x7efc5c0b18b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efc5000e7f0 tx=0x7efc5000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.938+0000 7efc58c00640 1 -- 172.21.15.50:0/3886152772 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5000ccc0 con 0x7efc5c072100 2024-09-21T06:05:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.938+0000 7efc63600640 1 -- 172.21.15.50:0/3886152772 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5c13f090 con 0x7efc5c072100 2024-09-21T06:05:46.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.938+0000 7efc58c00640 1 -- 172.21.15.50:0/3886152772 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc5000ce80 con 0x7efc5c072100 2024-09-21T06:05:46.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.938+0000 7efc58c00640 1 -- 172.21.15.50:0/3886152772 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50005280 con 0x7efc5c072100 2024-09-21T06:05:46.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.938+0000 7efc63600640 1 -- 172.21.15.50:0/3886152772 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5c13f650 con 0x7efc5c072100 2024-09-21T06:05:46.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.938+0000 7efc63600640 1 -- 172.21.15.50:0/3886152772 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5c072510 con 0x7efc5c072100 2024-09-21T06:05:46.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.938+0000 7efc58c00640 1 -- 172.21.15.50:0/3886152772 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 33) ==== 98764+0+0 (secure 0 0 0) 0x7efc50005420 con 0x7efc5c072100 2024-09-21T06:05:46.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.938+0000 7efc58c00640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efc3c074310 0x7efc3c076820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:46.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.938+0000 7efc58c00640 1 -- 172.21.15.50:0/3886152772 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7efc50093e10 con 0x7efc5c072100 2024-09-21T06:05:46.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.938+0000 7efc5be00640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efc3c074310 0x7efc3c076820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:46.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.942+0000 7efc5be00640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efc3c074310 0x7efc3c076820 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efc5c1382a0 tx=0x7efc48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:46.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:46.942+0000 7efc58c00640 1 -- 172.21.15.50:0/3886152772 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efc5005fef0 con 0x7efc5c072100 2024-09-21T06:05:47.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.142+0000 7efc63600640 1 -- 172.21.15.50:0/3886152772 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7efc5c06b540 con 0x7efc3c074310 2024-09-21T06:05:47.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.158+0000 7efc58c00640 1 -- 172.21.15.50:0/3886152772 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc48001490 con 0x7efc3c074310 2024-09-21T06:05:47.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.162+0000 7efc63600640 1 -- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efc3c074310 msgr2=0x7efc3c076820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:47.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.162+0000 7efc63600640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efc3c074310 0x7efc3c076820 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efc5c1382a0 tx=0x7efc48009040 comp rx=0 tx=0).stop 2024-09-21T06:05:47.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.162+0000 7efc63600640 1 -- 172.21.15.50:0/3886152772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c072100 msgr2=0x7efc5c0b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:47.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.162+0000 7efc63600640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c072100 0x7efc5c0b18b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efc5000e7f0 tx=0x7efc5000ecb0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.162+0000 7efc63600640 1 -- 172.21.15.50:0/3886152772 shutdown_connections 2024-09-21T06:05:47.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.162+0000 7efc63600640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc5c15cbb0 0x7efc5c0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.162+0000 7efc63600640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efc5c0cefa0 0x7efc5c0ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.162+0000 7efc63600640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efc3c074310 0x7efc3c076820 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.162+0000 7efc63600640 1 --2- 172.21.15.50:0/3886152772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c072100 0x7efc5c0b18b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.162+0000 7efc63600640 1 -- 172.21.15.50:0/3886152772 >> 172.21.15.50:0/3886152772 conn(0x7efc5c0b2040 msgr2=0x7efc5c13cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:47.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.162+0000 7efc63600640 1 -- 172.21.15.50:0/3886152772 shutdown_connections 2024-09-21T06:05:47.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.162+0000 7efc63600640 1 -- 172.21.15.50:0/3886152772 wait complete. 2024-09-21T06:05:47.184 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone26043 2024-09-21T06:05:47.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.350+0000 7f6f67c00640 1 -- 172.21.15.50:0/1744704768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60074b20 msgr2=0x7f6f6015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:47.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.350+0000 7f6f67c00640 1 --2- 172.21.15.50:0/1744704768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60074b20 0x7f6f6015cf90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009c80 tx=0x7f6f5801cda0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.350+0000 7f6f67c00640 1 -- 172.21.15.50:0/1744704768 shutdown_connections 2024-09-21T06:05:47.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.350+0000 7f6f67c00640 1 --2- 172.21.15.50:0/1744704768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60074b20 0x7f6f6015cf90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.350+0000 7f6f67c00640 1 --2- 172.21.15.50:0/1744704768 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6f60157030 0x7f6f600b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.350+0000 7f6f67c00640 1 -- 172.21.15.50:0/1744704768 >> 172.21.15.50:0/1744704768 conn(0x7f6f600708c0 msgr2=0x7f6f60070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:47.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f67c00640 1 -- 172.21.15.50:0/1744704768 shutdown_connections 2024-09-21T06:05:47.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f67c00640 1 -- 172.21.15.50:0/1744704768 wait complete. 2024-09-21T06:05:47.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f67c00640 1 Processor -- start 2024-09-21T06:05:47.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f67c00640 1 -- start start 2024-09-21T06:05:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f67c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f60157030 0x7f6f600d1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f67c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60160d50 0x7f6f60155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f67c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6f600d1a60 0x7f6f60155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f67c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6016a800 con 0x7f6f60160d50 2024-09-21T06:05:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f67c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6016a680 con 0x7f6f60157030 2024-09-21T06:05:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f67c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6016a980 con 0x7f6f600d1a60 2024-09-21T06:05:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f5fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60160d50 0x7f6f60155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f65200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f60157030 0x7f6f600d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f5fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60160d50 0x7f6f60155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46350/0 (socket says 172.21.15.50:46350) 2024-09-21T06:05:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.354+0000 7f6f5fe00640 1 -- 172.21.15.50:0/3969924295 learned_addr learned my addr 172.21.15.50:0/3969924295 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f65c00640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6f600d1a60 0x7f6f60155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f65200640 1 -- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6f600d1a60 msgr2=0x7f6f60155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f65200640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6f600d1a60 0x7f6f60155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f65200640 1 -- 172.21.15.50:0/3969924295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60160d50 msgr2=0x7f6f60155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f65200640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60160d50 0x7f6f60155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f65200640 1 -- 172.21.15.50:0/3969924295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f601fc7e0 con 0x7f6f60157030 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f5fe00640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60160d50 0x7f6f60155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f65200640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f60157030 0x7f6f600d1520 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400e6c0 tx=0x7f6f5400eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f5cc00640 1 -- 172.21.15.50:0/3969924295 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54004220 con 0x7f6f60157030 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f5cc00640 1 -- 172.21.15.50:0/3969924295 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f5400c8b0 con 0x7f6f60157030 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f5cc00640 1 -- 172.21.15.50:0/3969924295 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5400cd00 con 0x7f6f60157030 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f67c00640 1 -- 172.21.15.50:0/3969924295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f601fcc50 con 0x7f6f60157030 2024-09-21T06:05:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f67c00640 1 -- 172.21.15.50:0/3969924295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f601fd210 con 0x7f6f60157030 2024-09-21T06:05:47.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.358+0000 7f6f67c00640 1 -- 172.21.15.50:0/3969924295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f600d2260 con 0x7f6f60157030 2024-09-21T06:05:47.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.362+0000 7f6f5cc00640 1 -- 172.21.15.50:0/3969924295 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f6f54005030 con 0x7f6f60157030 2024-09-21T06:05:47.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.362+0000 7f6f5cc00640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f40074370 0x7f6f40076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:47.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.362+0000 7f6f5cc00640 1 -- 172.21.15.50:0/3969924295 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f6f54093660 con 0x7f6f60157030 2024-09-21T06:05:47.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.362+0000 7f6f5fe00640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f40074370 0x7f6f40076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:47.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.362+0000 7f6f5fe00640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f40074370 0x7f6f40076880 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6f600b4550 tx=0x7f6f4c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:47.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.366+0000 7f6f5cc00640 1 -- 172.21.15.50:0/3969924295 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6f5405f7c0 con 0x7f6f60157030 2024-09-21T06:05:47.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.566+0000 7f6f67c00640 1 -- 172.21.15.50:0/3969924295 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone26043", "target": ["mon-mgr", ""]}) -- 0x7f6f6006b540 con 0x7f6f40074370 2024-09-21T06:05:47.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f5cc00640 1 -- 172.21.15.50:0/3969924295 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6f4c0018e0 con 0x7f6f40074370 2024-09-21T06:05:47.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f67c00640 1 -- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f40074370 msgr2=0x7f6f40076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:47.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f67c00640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f40074370 0x7f6f40076880 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6f600b4550 tx=0x7f6f4c009040 comp rx=0 tx=0).stop 2024-09-21T06:05:47.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f67c00640 1 -- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f60157030 msgr2=0x7f6f600d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:47.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f67c00640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f60157030 0x7f6f600d1520 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400e6c0 tx=0x7f6f5400eb80 comp rx=0 tx=0).stop 2024-09-21T06:05:47.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f67c00640 1 -- 172.21.15.50:0/3969924295 shutdown_connections 2024-09-21T06:05:47.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f67c00640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6f600d1a60 0x7f6f60155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f67c00640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f40074370 0x7f6f40076880 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f67c00640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60160d50 0x7f6f60155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f67c00640 1 --2- 172.21.15.50:0/3969924295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f60157030 0x7f6f600d1520 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f67c00640 1 -- 172.21.15.50:0/3969924295 >> 172.21.15.50:0/3969924295 conn(0x7f6f600708c0 msgr2=0x7f6f6015f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:47.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f67c00640 1 -- 172.21.15.50:0/3969924295 shutdown_connections 2024-09-21T06:05:47.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.590+0000 7f6f67c00640 1 -- 172.21.15.50:0/3969924295 wait complete. 2024-09-21T06:05:47.614 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone93454 2024-09-21T06:05:47.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.782+0000 7f1912400640 1 -- 172.21.15.50:0/1550847858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f190c1527b0 msgr2=0x7f190c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:47.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.782+0000 7f1912400640 1 --2- 172.21.15.50:0/1550847858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f190c1527b0 0x7f190c15bb60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f190c0cca90 tx=0x7f18f401c800 comp rx=0 tx=0).stop 2024-09-21T06:05:47.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.782+0000 7f1912400640 1 -- 172.21.15.50:0/1550847858 shutdown_connections 2024-09-21T06:05:47.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.782+0000 7f1912400640 1 --2- 172.21.15.50:0/1550847858 >> v2:172.21.15.88:3300/0 conn(0x7f190c0b4de0 0x7f190c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.782+0000 7f1912400640 1 --2- 172.21.15.50:0/1550847858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f190c1527b0 0x7f190c15bb60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.782+0000 7f1912400640 1 -- 172.21.15.50:0/1550847858 >> 172.21.15.50:0/1550847858 conn(0x7f190c0b2040 msgr2=0x7f190c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:47.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.782+0000 7f1912400640 1 -- 172.21.15.50:0/1550847858 shutdown_connections 2024-09-21T06:05:47.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.782+0000 7f1912400640 1 -- 172.21.15.50:0/1550847858 wait complete. 2024-09-21T06:05:47.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.782+0000 7f1912400640 1 Processor -- start 2024-09-21T06:05:47.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.782+0000 7f1912400640 1 -- start start 2024-09-21T06:05:47.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1912400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f190c0b4de0 0x7f190c1fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:47.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1912400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f190c0b5c70 0x7f190c1f9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:47.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1912400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f190c1527b0 0x7f190c153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:47.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1912400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f190c168920 con 0x7f190c0b5c70 2024-09-21T06:05:47.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1912400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1687a0 con 0x7f190c0b4de0 2024-09-21T06:05:47.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1912400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f190c168aa0 con 0x7f190c1527b0 2024-09-21T06:05:47.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f190b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f190c0b4de0 0x7f190c1fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:47.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1903e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f190c0b5c70 0x7f190c1f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:47.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f190b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f190c0b4de0 0x7f190c1fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:43904/0 (socket says 172.21.15.50:43904) 2024-09-21T06:05:47.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f190b400640 1 -- 172.21.15.50:0/231731944 learned_addr learned my addr 172.21.15.50:0/231731944 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:47.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f190be00640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f190c1527b0 0x7f190c153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:47.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f190b400640 1 -- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f190c1527b0 msgr2=0x7f190c153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:47.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f190b400640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f190c1527b0 0x7f190c153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f190b400640 1 -- 172.21.15.50:0/231731944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f190c0b5c70 msgr2=0x7f190c1f9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:47.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f190b400640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f190c0b5c70 0x7f190c1f9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:47.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f190b400640 1 -- 172.21.15.50:0/231731944 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c1fc770 con 0x7f190c0b4de0 2024-09-21T06:05:47.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f190b400640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f190c0b4de0 0x7f190c1fbf20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c3c0 tx=0x7f18fc00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:47.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1903400640 1 -- 172.21.15.50:0/231731944 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc017070 con 0x7f190c0b4de0 2024-09-21T06:05:47.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1903400640 1 -- 172.21.15.50:0/231731944 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18fc0053c0 con 0x7f190c0b4de0 2024-09-21T06:05:47.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1912400640 1 -- 172.21.15.50:0/231731944 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c1fca50 con 0x7f190c0b4de0 2024-09-21T06:05:47.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1903400640 1 -- 172.21.15.50:0/231731944 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc003860 con 0x7f190c0b4de0 2024-09-21T06:05:47.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1912400640 1 -- 172.21.15.50:0/231731944 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c1f8a40 con 0x7f190c0b4de0 2024-09-21T06:05:47.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.786+0000 7f1912400640 1 -- 172.21.15.50:0/231731944 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190c0b51f0 con 0x7f190c0b4de0 2024-09-21T06:05:47.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.790+0000 7f1903400640 1 -- 172.21.15.50:0/231731944 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f18fc01a070 con 0x7f190c0b4de0 2024-09-21T06:05:47.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.790+0000 7f1903400640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18e8074370 0x7f18e8076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:47.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.790+0000 7f1903400640 1 -- 172.21.15.50:0/231731944 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f18fc092360 con 0x7f190c0b4de0 2024-09-21T06:05:47.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.790+0000 7f1903e00640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18e8074370 0x7f18e8076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:47.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.790+0000 7f1903e00640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18e8074370 0x7f18e8076880 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f18f40af000 tx=0x7f18f40a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:47.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:47.794+0000 7f1903400640 1 -- 172.21.15.50:0/231731944 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f190c0b5730 con 0x7f190c0b4de0 2024-09-21T06:05:47.904 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:05:47.910 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:05:48.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.118+0000 7f1912400640 1 -- 172.21.15.50:0/231731944 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone93454", "target": ["mon-mgr", ""]}) -- 0x7f190c06b540 con 0x7f18e8074370 2024-09-21T06:05:48.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.158+0000 7f1903400640 1 -- 172.21.15.50:0/231731944 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f18f40b3040 con 0x7f18e8074370 2024-09-21T06:05:48.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.162+0000 7f1912400640 1 -- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18e8074370 msgr2=0x7f18e8076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:48.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.162+0000 7f1912400640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18e8074370 0x7f18e8076880 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f18f40af000 tx=0x7f18f40a7040 comp rx=0 tx=0).stop 2024-09-21T06:05:48.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.162+0000 7f1912400640 1 -- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f190c0b4de0 msgr2=0x7f190c1fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:48.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.162+0000 7f1912400640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f190c0b4de0 0x7f190c1fbf20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c3c0 tx=0x7f18fc00c880 comp rx=0 tx=0).stop 2024-09-21T06:05:48.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.166+0000 7f1912400640 1 -- 172.21.15.50:0/231731944 shutdown_connections 2024-09-21T06:05:48.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.166+0000 7f1912400640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f190c1527b0 0x7f190c153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.166+0000 7f1912400640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18e8074370 0x7f18e8076880 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.166+0000 7f1912400640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f190c0b5c70 0x7f190c1f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.166+0000 7f1912400640 1 --2- 172.21.15.50:0/231731944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f190c0b4de0 0x7f190c1fbf20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.166+0000 7f1912400640 1 -- 172.21.15.50:0/231731944 >> 172.21.15.50:0/231731944 conn(0x7f190c0b2040 msgr2=0x7f190c15a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:48.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.166+0000 7f1912400640 1 -- 172.21.15.50:0/231731944 shutdown_connections 2024-09-21T06:05:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.166+0000 7f1912400640 1 -- 172.21.15.50:0/231731944 wait complete. 2024-09-21T06:05:48.202 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone15181 2024-09-21T06:05:48.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.430+0000 7f3a1c200640 1 -- 172.21.15.50:0/4115488798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a140b5c70 msgr2=0x7f3a1415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:48.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.430+0000 7f3a1c200640 1 --2- 172.21.15.50:0/4115488798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a140b5c70 0x7f3a1415bb60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009840 tx=0x7f3a0801c7f0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.430+0000 7f3a1c200640 1 -- 172.21.15.50:0/4115488798 shutdown_connections 2024-09-21T06:05:48.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.430+0000 7f3a1c200640 1 --2- 172.21.15.50:0/4115488798 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3a140b4de0 0x7f3a140b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.430+0000 7f3a1c200640 1 --2- 172.21.15.50:0/4115488798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a140b5c70 0x7f3a1415bb60 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.430+0000 7f3a1c200640 1 -- 172.21.15.50:0/4115488798 >> 172.21.15.50:0/4115488798 conn(0x7f3a140b2040 msgr2=0x7f3a140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:48.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.430+0000 7f3a1c200640 1 -- 172.21.15.50:0/4115488798 shutdown_connections 2024-09-21T06:05:48.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.430+0000 7f3a1c200640 1 -- 172.21.15.50:0/4115488798 wait complete. 2024-09-21T06:05:48.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1c200640 1 Processor -- start 2024-09-21T06:05:48.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1c200640 1 -- start start 2024-09-21T06:05:48.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1c200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3a140b4de0 0x7f3a140af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:48.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1c200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a140b5c70 0x7f3a141f8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:48.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1c200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a141527b0 0x7f3a140b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:48.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1c200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14168970 con 0x7f3a141527b0 2024-09-21T06:05:48.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1c200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141687f0 con 0x7f3a140b5c70 2024-09-21T06:05:48.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1c200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3a14168af0 con 0x7f3a140b4de0 2024-09-21T06:05:48.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1a200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a141527b0 0x7f3a140b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:48.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1a200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a141527b0 0x7f3a140b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46368/0 (socket says 172.21.15.50:46368) 2024-09-21T06:05:48.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1a200640 1 -- 172.21.15.50:0/1961946994 learned_addr learned my addr 172.21.15.50:0/1961946994 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:48.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a19800640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3a140b4de0 0x7f3a140af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:48.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a18e00640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a140b5c70 0x7f3a141f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:48.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1a200640 1 -- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3a140b4de0 msgr2=0x7f3a140af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:48.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1a200640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3a140b4de0 0x7f3a140af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1a200640 1 -- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a140b5c70 msgr2=0x7f3a141f8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:48.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1a200640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a140b5c70 0x7f3a141f8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1a200640 1 -- 172.21.15.50:0/1961946994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a141f8870 con 0x7f3a141527b0 2024-09-21T06:05:48.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a18e00640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a140b5c70 0x7f3a141f8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:48.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1a200640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a141527b0 0x7f3a140b1090 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00e970 tx=0x7f3a0c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:48.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a11600640 1 -- 172.21.15.50:0/1961946994 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c004d30 con 0x7f3a141527b0 2024-09-21T06:05:48.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a11600640 1 -- 172.21.15.50:0/1961946994 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a0c00cac0 con 0x7f3a141527b0 2024-09-21T06:05:48.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a11600640 1 -- 172.21.15.50:0/1961946994 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c005020 con 0x7f3a141527b0 2024-09-21T06:05:48.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1c200640 1 -- 172.21.15.50:0/1961946994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a140ae980 con 0x7f3a141527b0 2024-09-21T06:05:48.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.434+0000 7f3a1c200640 1 -- 172.21.15.50:0/1961946994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a140b1ae0 con 0x7f3a141527b0 2024-09-21T06:05:48.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.438+0000 7f3a1c200640 1 -- 172.21.15.50:0/1961946994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a140b1850 con 0x7f3a141527b0 2024-09-21T06:05:48.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.438+0000 7f3a11600640 1 -- 172.21.15.50:0/1961946994 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f3a0c0051c0 con 0x7f3a141527b0 2024-09-21T06:05:48.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.438+0000 7f3a11600640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39f4074370 0x7f39f4076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:48.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.438+0000 7f3a11600640 1 -- 172.21.15.50:0/1961946994 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f3a0c093980 con 0x7f3a141527b0 2024-09-21T06:05:48.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.438+0000 7f3a19800640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39f4074370 0x7f39f4076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:48.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.442+0000 7f3a19800640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39f4074370 0x7f39f4076880 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3a080af000 tx=0x7f3a0801b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:48.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.442+0000 7f3a11600640 1 -- 172.21.15.50:0/1961946994 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3a141f8870 con 0x7f3a141527b0 2024-09-21T06:05:48.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.662+0000 7f3a1c200640 1 -- 172.21.15.50:0/1961946994 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone15181", "target": ["mon-mgr", ""]}) -- 0x7f3a1406b540 con 0x7f39f4074370 2024-09-21T06:05:48.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.678+0000 7f3a11600640 1 -- 172.21.15.50:0/1961946994 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3a08004040 con 0x7f39f4074370 2024-09-21T06:05:48.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.682+0000 7f3a1c200640 1 -- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39f4074370 msgr2=0x7f39f4076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:48.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.682+0000 7f3a1c200640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39f4074370 0x7f39f4076880 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3a080af000 tx=0x7f3a0801b030 comp rx=0 tx=0).stop 2024-09-21T06:05:48.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.682+0000 7f3a1c200640 1 -- 172.21.15.50:0/1961946994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a141527b0 msgr2=0x7f3a140b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:48.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.682+0000 7f3a1c200640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a141527b0 0x7f3a140b1090 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00e970 tx=0x7f3a0c00ee30 comp rx=0 tx=0).stop 2024-09-21T06:05:48.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.682+0000 7f3a1c200640 1 -- 172.21.15.50:0/1961946994 shutdown_connections 2024-09-21T06:05:48.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.682+0000 7f3a1c200640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39f4074370 0x7f39f4076880 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.682+0000 7f3a1c200640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a141527b0 0x7f3a140b1090 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.682+0000 7f3a1c200640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a140b5c70 0x7f3a141f8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.682+0000 7f3a1c200640 1 --2- 172.21.15.50:0/1961946994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3a140b4de0 0x7f3a140af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.682+0000 7f3a1c200640 1 -- 172.21.15.50:0/1961946994 >> 172.21.15.50:0/1961946994 conn(0x7f3a140b2040 msgr2=0x7f3a141581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:48.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.682+0000 7f3a1c200640 1 -- 172.21.15.50:0/1961946994 shutdown_connections 2024-09-21T06:05:48.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.682+0000 7f3a1c200640 1 -- 172.21.15.50:0/1961946994 wait complete. 2024-09-21T06:05:48.705 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone12326 2024-09-21T06:05:48.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.866+0000 7f790ba00640 1 -- 172.21.15.50:0/279940488 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79040b5c70 msgr2=0x7f790415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:48.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.866+0000 7f790ba00640 1 --2- 172.21.15.50:0/279940488 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79040b5c70 0x7f790415bb60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009840 tx=0x7f78f801c760 comp rx=0 tx=0).stop 2024-09-21T06:05:48.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.866+0000 7f790ba00640 1 -- 172.21.15.50:0/279940488 shutdown_connections 2024-09-21T06:05:48.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.866+0000 7f790ba00640 1 --2- 172.21.15.50:0/279940488 >> v2:172.21.15.88:3300/0 conn(0x7f79040b4de0 0x7f79040b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.866+0000 7f790ba00640 1 --2- 172.21.15.50:0/279940488 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79040b5c70 0x7f790415bb60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.866+0000 7f790ba00640 1 -- 172.21.15.50:0/279940488 >> 172.21.15.50:0/279940488 conn(0x7f79040b2040 msgr2=0x7f79040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:48.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.866+0000 7f790ba00640 1 -- 172.21.15.50:0/279940488 shutdown_connections 2024-09-21T06:05:48.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.866+0000 7f790ba00640 1 -- 172.21.15.50:0/279940488 wait complete. 2024-09-21T06:05:48.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f790ba00640 1 Processor -- start 2024-09-21T06:05:48.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f790ba00640 1 -- start start 2024-09-21T06:05:48.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f790ba00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79040b4de0 0x7f79040cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f790ba00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79040b5c70 0x7f7904150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f790ba00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79041527b0 0x7f79040ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f790ba00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7904168ab0 con 0x7f79041527b0 2024-09-21T06:05:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f790ba00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7904168930 con 0x7f79040b4de0 2024-09-21T06:05:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f790ba00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7904168c30 con 0x7f79040b5c70 2024-09-21T06:05:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f7909a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79041527b0 0x7f79040ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f7909a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79041527b0 0x7f79040ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46380/0 (socket says 172.21.15.50:46380) 2024-09-21T06:05:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f7909a00640 1 -- 172.21.15.50:0/1879197471 learned_addr learned my addr 172.21.15.50:0/1879197471 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f7909000640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79040b4de0 0x7f79040cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f7903e00640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79040b5c70 0x7f7904150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f7909a00640 1 -- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79040b5c70 msgr2=0x7f7904150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:48.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f7909a00640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79040b5c70 0x7f7904150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f7909a00640 1 -- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79040b4de0 msgr2=0x7f79040cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:48.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f7909a00640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79040b4de0 0x7f79040cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:48.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f7909a00640 1 -- 172.21.15.50:0/1879197471 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79041f8970 con 0x7f79041527b0 2024-09-21T06:05:48.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f7909a00640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79041527b0 0x7f79040ae8e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00eca0 tx=0x7f78fc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.870+0000 7f7900c00640 1 -- 172.21.15.50:0/1879197471 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc003b80 con 0x7f79041527b0 2024-09-21T06:05:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.874+0000 7f7900c00640 1 -- 172.21.15.50:0/1879197471 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78fc003d20 con 0x7f79041527b0 2024-09-21T06:05:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.874+0000 7f7900c00640 1 -- 172.21.15.50:0/1879197471 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc004280 con 0x7f79041527b0 2024-09-21T06:05:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.874+0000 7f790ba00640 1 -- 172.21.15.50:0/1879197471 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79041f8de0 con 0x7f79041527b0 2024-09-21T06:05:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.874+0000 7f790ba00640 1 -- 172.21.15.50:0/1879197471 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79041f93a0 con 0x7f79041527b0 2024-09-21T06:05:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.874+0000 7f790ba00640 1 -- 172.21.15.50:0/1879197471 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79040b51f0 con 0x7f79041527b0 2024-09-21T06:05:48.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.874+0000 7f7900c00640 1 -- 172.21.15.50:0/1879197471 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f78fc004420 con 0x7f79041527b0 2024-09-21T06:05:48.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.878+0000 7f7900c00640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f78e4074370 0x7f78e4076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:48.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.878+0000 7f7900c00640 1 -- 172.21.15.50:0/1879197471 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f78fc093c10 con 0x7f79041527b0 2024-09-21T06:05:48.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.878+0000 7f7909000640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f78e4074370 0x7f78e4076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:48.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.878+0000 7f7909000640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f78e4074370 0x7f78e4076880 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f78f80af000 tx=0x7f78f801afa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:48.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:48.882+0000 7f7900c00640 1 -- 172.21.15.50:0/1879197471 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f78fc05fdd0 con 0x7f79041527b0 2024-09-21T06:05:49.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.082+0000 7f790ba00640 1 -- 172.21.15.50:0/1879197471 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone12326", "target": ["mon-mgr", ""]}) -- 0x7f790406b540 con 0x7f78e4074370 2024-09-21T06:05:49.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.102+0000 7f7900c00640 1 -- 172.21.15.50:0/1879197471 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f78f8004040 con 0x7f78e4074370 2024-09-21T06:05:49.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.106+0000 7f790ba00640 1 -- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f78e4074370 msgr2=0x7f78e4076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:49.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.106+0000 7f790ba00640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f78e4074370 0x7f78e4076880 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f78f80af000 tx=0x7f78f801afa0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.106+0000 7f790ba00640 1 -- 172.21.15.50:0/1879197471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79041527b0 msgr2=0x7f79040ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:49.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.106+0000 7f790ba00640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79041527b0 0x7f79040ae8e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00eca0 tx=0x7f78fc00c6a0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.106+0000 7f790ba00640 1 -- 172.21.15.50:0/1879197471 shutdown_connections 2024-09-21T06:05:49.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.106+0000 7f790ba00640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f78e4074370 0x7f78e4076880 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.106+0000 7f790ba00640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79041527b0 0x7f79040ae8e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.106+0000 7f790ba00640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79040b5c70 0x7f7904150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.106+0000 7f790ba00640 1 --2- 172.21.15.50:0/1879197471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79040b4de0 0x7f79040cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.106+0000 7f790ba00640 1 -- 172.21.15.50:0/1879197471 >> 172.21.15.50:0/1879197471 conn(0x7f79040b2040 msgr2=0x7f79040b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:49.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.106+0000 7f790ba00640 1 -- 172.21.15.50:0/1879197471 shutdown_connections 2024-09-21T06:05:49.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.106+0000 7f790ba00640 1 -- 172.21.15.50:0/1879197471 wait complete. 2024-09-21T06:05:49.127 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone51512 2024-09-21T06:05:49.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.290+0000 7f792a000640 1 -- 172.21.15.50:0/3395321536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f792415c5d0 msgr2=0x7f7924073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:49.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.290+0000 7f792a000640 1 --2- 172.21.15.50:0/3395321536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f792415c5d0 0x7f7924073dc0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f792406ae60 tx=0x7f791401c890 comp rx=0 tx=0).stop 2024-09-21T06:05:49.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.290+0000 7f792a000640 1 -- 172.21.15.50:0/3395321536 shutdown_connections 2024-09-21T06:05:49.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.290+0000 7f792a000640 1 --2- 172.21.15.50:0/3395321536 >> v2:172.21.15.88:3300/0 conn(0x7f7924074300 0x7f792415f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.290+0000 7f792a000640 1 --2- 172.21.15.50:0/3395321536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f792415c5d0 0x7f7924073dc0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.290+0000 7f792a000640 1 -- 172.21.15.50:0/3395321536 >> 172.21.15.50:0/3395321536 conn(0x7f79240b2c40 msgr2=0x7f79240b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:49.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.290+0000 7f792a000640 1 -- 172.21.15.50:0/3395321536 shutdown_connections 2024-09-21T06:05:49.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.290+0000 7f792a000640 1 -- 172.21.15.50:0/3395321536 wait complete. 2024-09-21T06:05:49.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.290+0000 7f792a000640 1 Processor -- start 2024-09-21T06:05:49.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.290+0000 7f792a000640 1 -- start start 2024-09-21T06:05:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f792a000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7924074300 0x7f792415a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f792a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f792415c5d0 0x7f792414ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f792a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f792415f6b0 0x7f79240b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f792a000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f792406f980 con 0x7f792415c5d0 2024-09-21T06:05:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f792a000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f79240b3d50 con 0x7f792415f6b0 2024-09-21T06:05:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f792a000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f792406fb00 con 0x7f7924074300 2024-09-21T06:05:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f791aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f792415c5d0 0x7f792414ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f791aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f792415c5d0 0x7f792414ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46392/0 (socket says 172.21.15.50:46392) 2024-09-21T06:05:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f7923e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f792415f6b0 0x7f79240b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f791aa00640 1 -- 172.21.15.50:0/1866709831 learned_addr learned my addr 172.21.15.50:0/1866709831 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f7923400640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7924074300 0x7f792415a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:49.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f791aa00640 1 -- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7924074300 msgr2=0x7f792415a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:49.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f791aa00640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7924074300 0x7f792415a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f791aa00640 1 -- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f792415f6b0 msgr2=0x7f79240b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:49.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f791aa00640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f792415f6b0 0x7f79240b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f791aa00640 1 -- 172.21.15.50:0/1866709831 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792414f720 con 0x7f792415c5d0 2024-09-21T06:05:49.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f791aa00640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f792415c5d0 0x7f792414ca40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f790c00d6d0 tx=0x7f790c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:49.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f791be00640 1 -- 172.21.15.50:0/1866709831 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c017070 con 0x7f792415c5d0 2024-09-21T06:05:49.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f791be00640 1 -- 172.21.15.50:0/1866709831 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f790c005310 con 0x7f792415c5d0 2024-09-21T06:05:49.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f792a000640 1 -- 172.21.15.50:0/1866709831 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792414fa00 con 0x7f792415c5d0 2024-09-21T06:05:49.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f791be00640 1 -- 172.21.15.50:0/1866709831 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c009820 con 0x7f792415c5d0 2024-09-21T06:05:49.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f792a000640 1 -- 172.21.15.50:0/1866709831 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792414ffc0 con 0x7f792415c5d0 2024-09-21T06:05:49.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.294+0000 7f792a000640 1 -- 172.21.15.50:0/1866709831 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79241f6dd0 con 0x7f792415c5d0 2024-09-21T06:05:49.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.298+0000 7f791be00640 1 -- 172.21.15.50:0/1866709831 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f790c0099c0 con 0x7f792415c5d0 2024-09-21T06:05:49.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.298+0000 7f791be00640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79000743a0 0x7f79000768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:49.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.298+0000 7f791be00640 1 -- 172.21.15.50:0/1866709831 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f790c0930b0 con 0x7f792415c5d0 2024-09-21T06:05:49.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.298+0000 7f7923400640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79000743a0 0x7f79000768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:49.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.298+0000 7f7923400640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79000743a0 0x7f79000768b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f79140af000 tx=0x7f791401b0d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:49.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.302+0000 7f791be00640 1 -- 172.21.15.50:0/1866709831 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f790c05f210 con 0x7f792415c5d0 2024-09-21T06:05:49.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.514+0000 7f792a000640 1 -- 172.21.15.50:0/1866709831 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone51512", "target": ["mon-mgr", ""]}) -- 0x7f792414f4d0 con 0x7f79000743a0 2024-09-21T06:05:49.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.530+0000 7f791be00640 1 -- 172.21.15.50:0/1866709831 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f792414f4d0 con 0x7f79000743a0 2024-09-21T06:05:49.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.534+0000 7f792a000640 1 -- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79000743a0 msgr2=0x7f79000768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:49.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.534+0000 7f792a000640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79000743a0 0x7f79000768b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f79140af000 tx=0x7f791401b0d0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.534+0000 7f792a000640 1 -- 172.21.15.50:0/1866709831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f792415c5d0 msgr2=0x7f792414ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:49.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.534+0000 7f792a000640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f792415c5d0 0x7f792414ca40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f790c00d6d0 tx=0x7f790c00db90 comp rx=0 tx=0).stop 2024-09-21T06:05:49.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.534+0000 7f792a000640 1 -- 172.21.15.50:0/1866709831 shutdown_connections 2024-09-21T06:05:49.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.534+0000 7f792a000640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f792415f6b0 0x7f79240b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.534+0000 7f792a000640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79000743a0 0x7f79000768b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.534+0000 7f792a000640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f792415c5d0 0x7f792414ca40 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.534+0000 7f792a000640 1 --2- 172.21.15.50:0/1866709831 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7924074300 0x7f792415a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.534+0000 7f792a000640 1 -- 172.21.15.50:0/1866709831 >> 172.21.15.50:0/1866709831 conn(0x7f79240b2c40 msgr2=0x7f792415dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:49.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.534+0000 7f792a000640 1 -- 172.21.15.50:0/1866709831 shutdown_connections 2024-09-21T06:05:49.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.534+0000 7f792a000640 1 -- 172.21.15.50:0/1866709831 wait complete. 2024-09-21T06:05:49.559 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T06:05:49.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:05:49.615 INFO:teuthology.orchestra.run.smithi050.stdout:2 2024-09-21T06:05:49.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:05:49.623 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:05:49.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.830+0000 7fc350a00640 1 -- 172.21.15.50:0/1449891005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc34c157030 msgr2=0x7fc34c0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:49.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.830+0000 7fc350a00640 1 --2- 172.21.15.50:0/1449891005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc34c157030 0x7fc34c0b5a80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009840 tx=0x7fc33c01c790 comp rx=0 tx=0).stop 2024-09-21T06:05:49.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.830+0000 7fc350a00640 1 -- 172.21.15.50:0/1449891005 shutdown_connections 2024-09-21T06:05:49.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.830+0000 7fc350a00640 1 --2- 172.21.15.50:0/1449891005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc34c157030 0x7fc34c0b5a80 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.830+0000 7fc350a00640 1 -- 172.21.15.50:0/1449891005 >> 172.21.15.50:0/1449891005 conn(0x7fc34c0708c0 msgr2=0x7fc34c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:49.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.830+0000 7fc350a00640 1 -- 172.21.15.50:0/1449891005 shutdown_connections 2024-09-21T06:05:49.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.830+0000 7fc350a00640 1 -- 172.21.15.50:0/1449891005 wait complete. 2024-09-21T06:05:49.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.834+0000 7fc350a00640 1 Processor -- start 2024-09-21T06:05:49.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.834+0000 7fc350a00640 1 -- start start 2024-09-21T06:05:49.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.834+0000 7fc350a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc34c157030 0x7fc34c13fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:49.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.834+0000 7fc350a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc34c160d50 0x7fc34c1fc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:49.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.834+0000 7fc350a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc34c14df50 0x7fc34c1fcea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:49.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.834+0000 7fc350a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c16b8c0 con 0x7fc34c14df50 2024-09-21T06:05:49.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.834+0000 7fc350a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c16b740 con 0x7fc34c160d50 2024-09-21T06:05:49.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.834+0000 7fc350a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc34c16ba40 con 0x7fc34c157030 2024-09-21T06:05:49.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.834+0000 7fc349400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc34c160d50 0x7fc34c1fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:49.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc34a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc34c14df50 0x7fc34c1fcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:49.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc349e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc34c157030 0x7fc34c13fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:49.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc349e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc34c157030 0x7fc34c13fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:49270/0 (socket says 172.21.15.50:49270) 2024-09-21T06:05:49.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc349e00640 1 -- 172.21.15.50:0/960083248 learned_addr learned my addr 172.21.15.50:0/960083248 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:49.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc349400640 1 -- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc34c157030 msgr2=0x7fc34c13fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:49.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc349400640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc34c157030 0x7fc34c13fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc349400640 1 -- 172.21.15.50:0/960083248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc34c14df50 msgr2=0x7fc34c1fcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:49.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc349400640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc34c14df50 0x7fc34c1fcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc349400640 1 -- 172.21.15.50:0/960083248 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c150860 con 0x7fc34c160d50 2024-09-21T06:05:49.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc34a800640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc34c14df50 0x7fc34c1fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:05:49.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc349400640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc34c160d50 0x7fc34c1fc960 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc33800c3f0 tx=0x7fc33800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:49.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc342000640 1 -- 172.21.15.50:0/960083248 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc338017070 con 0x7fc34c160d50 2024-09-21T06:05:49.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc342000640 1 -- 172.21.15.50:0/960083248 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3380053c0 con 0x7fc34c160d50 2024-09-21T06:05:49.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc350a00640 1 -- 172.21.15.50:0/960083248 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34c140370 con 0x7fc34c160d50 2024-09-21T06:05:49.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc342000640 1 -- 172.21.15.50:0/960083248 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc338003860 con 0x7fc34c160d50 2024-09-21T06:05:49.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc350a00640 1 -- 172.21.15.50:0/960083248 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc34c1624d0 con 0x7fc34c160d50 2024-09-21T06:05:49.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc350a00640 1 -- 172.21.15.50:0/960083248 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34c0d2260 con 0x7fc34c160d50 2024-09-21T06:05:49.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.838+0000 7fc342000640 1 -- 172.21.15.50:0/960083248 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fc33801a070 con 0x7fc34c160d50 2024-09-21T06:05:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.842+0000 7fc342000640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc32c074370 0x7fc32c076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.842+0000 7fc342000640 1 -- 172.21.15.50:0/960083248 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7fc338092350 con 0x7fc34c160d50 2024-09-21T06:05:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.842+0000 7fc349e00640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc32c074370 0x7fc32c076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:49.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.842+0000 7fc349e00640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc32c074370 0x7fc32c076880 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc33c01a3c0 tx=0x7fc33c01afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:49.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:49.846+0000 7fc342000640 1 -- 172.21.15.50:0/960083248 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc33809a070 con 0x7fc34c160d50 2024-09-21T06:05:50.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.150+0000 7fc350a00640 1 -- 172.21.15.50:0/960083248 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fc34c15cb10 con 0x7fc34c160d50 2024-09-21T06:05:50.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.154+0000 7fc342000640 1 -- 172.21.15.50:0/960083248 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v43)=0 v43) ==== 131+0+0 (secure 0 0 0) 0x7fc33805e570 con 0x7fc34c160d50 2024-09-21T06:05:50.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.158+0000 7fc350a00640 1 -- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc32c074370 msgr2=0x7fc32c076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:50.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.158+0000 7fc350a00640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc32c074370 0x7fc32c076880 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc33c01a3c0 tx=0x7fc33c01afd0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.158+0000 7fc350a00640 1 -- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc34c160d50 msgr2=0x7fc34c1fc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:50.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.158+0000 7fc350a00640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc34c160d50 0x7fc34c1fc960 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc33800c3f0 tx=0x7fc33800c8b0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.158+0000 7fc350a00640 1 -- 172.21.15.50:0/960083248 shutdown_connections 2024-09-21T06:05:50.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.158+0000 7fc350a00640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc32c074370 0x7fc32c076880 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.158+0000 7fc350a00640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc34c14df50 0x7fc34c1fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.158+0000 7fc350a00640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc34c160d50 0x7fc34c1fc960 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.158+0000 7fc350a00640 1 --2- 172.21.15.50:0/960083248 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc34c157030 0x7fc34c13fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.158+0000 7fc350a00640 1 -- 172.21.15.50:0/960083248 >> 172.21.15.50:0/960083248 conn(0x7fc34c0708c0 msgr2=0x7fc34c075ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:50.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.158+0000 7fc350a00640 1 -- 172.21.15.50:0/960083248 shutdown_connections 2024-09-21T06:05:50.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.158+0000 7fc350a00640 1 -- 172.21.15.50:0/960083248 wait complete. 2024-09-21T06:05:50.190 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:05:50.190 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:05:50.296 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:05:50.379 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:05:50.379 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:05:50.379 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:05:50.379 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:05:50.390 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:05:50.390 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:05:50.448 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:05:50.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.718+0000 7f69b3400640 1 -- 172.21.15.50:0/4054478580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6994005920 msgr2=0x7f6994005d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:50.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.718+0000 7f69b3400640 1 --2- 172.21.15.50:0/4054478580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6994005920 0x7f6994005d30 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f699800c3f0 tx=0x7f699801cec0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.718+0000 7f69b3400640 1 -- 172.21.15.50:0/4054478580 shutdown_connections 2024-09-21T06:05:50.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.718+0000 7f69b3400640 1 --2- 172.21.15.50:0/4054478580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6994005920 0x7f6994005d30 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.718+0000 7f69b3400640 1 -- 172.21.15.50:0/4054478580 >> 172.21.15.50:0/4054478580 conn(0x7f69ac0b2040 msgr2=0x7f69ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:50.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.718+0000 7f69b3400640 1 -- 172.21.15.50:0/4054478580 shutdown_connections 2024-09-21T06:05:50.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.718+0000 7f69b3400640 1 -- 172.21.15.50:0/4054478580 wait complete. 2024-09-21T06:05:50.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.718+0000 7f69b3400640 1 Processor -- start 2024-09-21T06:05:50.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.718+0000 7f69b3400640 1 -- start start 2024-09-21T06:05:50.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b3400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6994005920 0x7f69ac13d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:50.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ac13de20 0x7f69ac0747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:50.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69ac153c60 0x7f69ac074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:50.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b3400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac15b270 con 0x7f69ac153c60 2024-09-21T06:05:50.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b3400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac15df60 con 0x7f69ac13de20 2024-09-21T06:05:50.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b3400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f69ac15b3f0 con 0x7f6994005920 2024-09-21T06:05:50.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69ac153c60 0x7f69ac074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:50.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b0a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6994005920 0x7f69ac13d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:50.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69ac153c60 0x7f69ac074ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46422/0 (socket says 172.21.15.50:46422) 2024-09-21T06:05:50.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b1400640 1 -- 172.21.15.50:0/4190335110 learned_addr learned my addr 172.21.15.50:0/4190335110 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:50.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b0a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6994005920 0x7f69ac13d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:49296/0 (socket says 172.21.15.50:49296) 2024-09-21T06:05:50.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69abe00640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ac13de20 0x7f69ac0747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:50.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b1400640 1 -- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6994005920 msgr2=0x7f69ac13d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:50.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b1400640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6994005920 0x7f69ac13d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b1400640 1 -- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ac13de20 msgr2=0x7f69ac0747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:50.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b1400640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ac13de20 0x7f69ac0747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b1400640 1 -- 172.21.15.50:0/4190335110 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ac13dc10 con 0x7f69ac153c60 2024-09-21T06:05:50.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b1400640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69ac153c60 0x7f69ac074ce0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f69ac158ec0 tx=0x7f69a4001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:50.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69a8c00640 1 -- 172.21.15.50:0/4190335110 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4011870 con 0x7f69ac153c60 2024-09-21T06:05:50.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69a8c00640 1 -- 172.21.15.50:0/4190335110 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69a4017070 con 0x7f69ac153c60 2024-09-21T06:05:50.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b3400640 1 -- 172.21.15.50:0/4190335110 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ac155cc0 con 0x7f69ac153c60 2024-09-21T06:05:50.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69a8c00640 1 -- 172.21.15.50:0/4190335110 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4011dc0 con 0x7f69ac153c60 2024-09-21T06:05:50.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b3400640 1 -- 172.21.15.50:0/4190335110 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ac156280 con 0x7f69ac153c60 2024-09-21T06:05:50.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69b3400640 1 -- 172.21.15.50:0/4190335110 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ac15b8c0 con 0x7f69ac153c60 2024-09-21T06:05:50.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.722+0000 7f69a8c00640 1 -- 172.21.15.50:0/4190335110 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f69a401c460 con 0x7f69ac153c60 2024-09-21T06:05:50.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.726+0000 7f69a8c00640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f698c0743a0 0x7f698c0768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:50.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.726+0000 7f69a8c00640 1 -- 172.21.15.50:0/4190335110 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7f69a4093750 con 0x7f69ac153c60 2024-09-21T06:05:50.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.726+0000 7f69b0a00640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f698c0743a0 0x7f698c0768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:50.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.726+0000 7f69b0a00640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f698c0743a0 0x7f698c0768b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009dc0 tx=0x7f69a00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:50.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.730+0000 7f69a8c00640 1 -- 172.21.15.50:0/4190335110 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f69a405f8b0 con 0x7f69ac153c60 2024-09-21T06:05:50.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.954+0000 7f69b3400640 1 -- 172.21.15.50:0/4190335110 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f69ac15a020 con 0x7f69ac153c60 2024-09-21T06:05:50.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.954+0000 7f69a8c00640 1 -- 172.21.15.50:0/4190335110 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) ==== 80+0+0 (secure 0 0 0) 0x7f69a40642e0 con 0x7f69ac153c60 2024-09-21T06:05:50.957 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T06:05:50.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.958+0000 7f69b3400640 1 -- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f698c0743a0 msgr2=0x7f698c0768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.958+0000 7f69b3400640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f698c0743a0 0x7f698c0768b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009dc0 tx=0x7f69a00091e0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.958+0000 7f69b3400640 1 -- 172.21.15.50:0/4190335110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69ac153c60 msgr2=0x7f69ac074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.958+0000 7f69b3400640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69ac153c60 0x7f69ac074ce0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f69ac158ec0 tx=0x7f69a4001b90 comp rx=0 tx=0).stop 2024-09-21T06:05:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.958+0000 7f69b3400640 1 -- 172.21.15.50:0/4190335110 shutdown_connections 2024-09-21T06:05:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.958+0000 7f69b3400640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f698c0743a0 0x7f698c0768b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.958+0000 7f69b3400640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69ac153c60 0x7f69ac074ce0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.958+0000 7f69b3400640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ac13de20 0x7f69ac0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.958+0000 7f69b3400640 1 --2- 172.21.15.50:0/4190335110 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6994005920 0x7f69ac13d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.958+0000 7f69b3400640 1 -- 172.21.15.50:0/4190335110 >> 172.21.15.50:0/4190335110 conn(0x7f69ac0b2040 msgr2=0x7f69ac0b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.958+0000 7f69b3400640 1 -- 172.21.15.50:0/4190335110 shutdown_connections 2024-09-21T06:05:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:50.958+0000 7f69b3400640 1 -- 172.21.15.50:0/4190335110 wait complete. 2024-09-21T06:05:50.994 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:05:50.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:05:50.994 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:05:51.068 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:05:51.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:05:51.068 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:05:51.092 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:05:51.092 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:05:51.092 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:05:51.111 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:05:51.112 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:05:51.112 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:05:51.112 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:05:51.150 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:05:51.151 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:05:51.151 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:05:51.151 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:05:51.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:05:51.167 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:05:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.454+0000 7fa95b000640 1 -- 172.21.15.50:0/2214382186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa954155ad0 msgr2=0x7fa95406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.454+0000 7fa95b000640 1 --2- 172.21.15.50:0/2214382186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa954155ad0 0x7fa95406e520 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa9540b61e0 tx=0x7fa94401c770 comp rx=0 tx=0).stop 2024-09-21T06:05:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.454+0000 7fa95b000640 1 -- 172.21.15.50:0/2214382186 shutdown_connections 2024-09-21T06:05:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.454+0000 7fa95b000640 1 --2- 172.21.15.50:0/2214382186 >> v2:172.21.15.88:3300/0 conn(0x7fa9540d29c0 0x7fa9540d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.454+0000 7fa95b000640 1 --2- 172.21.15.50:0/2214382186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa954155ad0 0x7fa95406e520 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.454+0000 7fa95b000640 1 -- 172.21.15.50:0/2214382186 >> 172.21.15.50:0/2214382186 conn(0x7fa9540708c0 msgr2=0x7fa954070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:51.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.454+0000 7fa95b000640 1 -- 172.21.15.50:0/2214382186 shutdown_connections 2024-09-21T06:05:51.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.454+0000 7fa95b000640 1 -- 172.21.15.50:0/2214382186 wait complete. 2024-09-21T06:05:51.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.458+0000 7fa95b000640 1 Processor -- start 2024-09-21T06:05:51.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.458+0000 7fa95b000640 1 -- start start 2024-09-21T06:05:51.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.462+0000 7fa95b000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa954074410 0x7fa95414c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:51.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.462+0000 7fa95b000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9540d29c0 0x7fa954151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:51.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.462+0000 7fa95b000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa954152110 0x7fa9540b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:51.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.462+0000 7fa95b000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa95416c4f0 con 0x7fa954074410 2024-09-21T06:05:51.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.462+0000 7fa95b000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa95416c370 con 0x7fa9540d29c0 2024-09-21T06:05:51.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.462+0000 7fa95b000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa95416c670 con 0x7fa954152110 2024-09-21T06:05:51.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.462+0000 7fa953e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa954074410 0x7fa95414c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:51.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.462+0000 7fa953e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa954074410 0x7fa95414c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46444/0 (socket says 172.21.15.50:46444) 2024-09-21T06:05:51.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.462+0000 7fa953e00640 1 -- 172.21.15.50:0/3825510554 learned_addr learned my addr 172.21.15.50:0/3825510554 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:51.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.462+0000 7fa953400640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9540d29c0 0x7fa954151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:51.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa953e00640 1 -- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa954152110 msgr2=0x7fa9540b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa959000640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa954152110 0x7fa9540b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa953e00640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa954152110 0x7fa9540b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa953e00640 1 -- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9540d29c0 msgr2=0x7fa954151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa953e00640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9540d29c0 0x7fa954151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa953e00640 1 -- 172.21.15.50:0/3825510554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9541fdb80 con 0x7fa954074410 2024-09-21T06:05:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa959000640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa954152110 0x7fa9540b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa953e00640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa954074410 0x7fa95414c740 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa944000c10 tx=0x7fa9440a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa94be00640 1 -- 172.21.15.50:0/3825510554 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440b2040 con 0x7fa954074410 2024-09-21T06:05:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa94be00640 1 -- 172.21.15.50:0/3825510554 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9440bb070 con 0x7fa954074410 2024-09-21T06:05:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa94be00640 1 -- 172.21.15.50:0/3825510554 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944005630 con 0x7fa954074410 2024-09-21T06:05:51.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa95b000640 1 -- 172.21.15.50:0/3825510554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9541fde60 con 0x7fa954074410 2024-09-21T06:05:51.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.466+0000 7fa95b000640 1 -- 172.21.15.50:0/3825510554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541fe420 con 0x7fa954074410 2024-09-21T06:05:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.470+0000 7fa95b000640 1 -- 172.21.15.50:0/3825510554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9541532e0 con 0x7fa954074410 2024-09-21T06:05:51.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.470+0000 7fa94be00640 1 -- 172.21.15.50:0/3825510554 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fa9440046d0 con 0x7fa954074410 2024-09-21T06:05:51.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.474+0000 7fa94be00640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa9340743a0 0x7fa9340768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:51.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.474+0000 7fa94be00640 1 -- 172.21.15.50:0/3825510554 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7fa94412d150 con 0x7fa954074410 2024-09-21T06:05:51.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.474+0000 7fa953400640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa9340743a0 0x7fa9340768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:51.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.474+0000 7fa953400640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa9340743a0 0x7fa9340768b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa954151680 tx=0x7fa9400091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:51.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.482+0000 7fa94be00640 1 -- 172.21.15.50:0/3825510554 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa9440163f0 con 0x7fa954074410 2024-09-21T06:05:51.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.706+0000 7fa95b000640 1 -- 172.21.15.50:0/3825510554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa954071050 con 0x7fa954074410 2024-09-21T06:05:51.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.706+0000 7fa94be00640 1 -- 172.21.15.50:0/3825510554 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5222 (secure 0 0 0) 0x7fa9440f9870 con 0x7fa954074410 2024-09-21T06:05:51.709 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:51.709 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":46,"btime":"2024-09-21T06:04:45:517919+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/2233580616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2233580616},{"type":"v1","addr":"172.21.15.50:6833","nonce":2233580616}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3749582023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3749582023},{"type":"v1","addr":"172.21.15.50:6835","nonce":3749582023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6139,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/382971646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":382971646},{"type":"v1","addr":"172.21.15.88:6836","nonce":382971646}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:04:32.178456+0000","modified":"2024-09-21T06:04:35.197824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6423":{"gid":6423,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/2131576448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2131576448},{"type":"v1","addr":"172.21.15.88:6837","nonce":2131576448}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6423,"qdb_cluster":[6423]},"id":5}]} 2024-09-21T06:05:51.710 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 46 2024-09-21T06:05:51.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.710+0000 7fa95b000640 1 -- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa9340743a0 msgr2=0x7fa9340768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:51.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.710+0000 7fa95b000640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa9340743a0 0x7fa9340768b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa954151680 tx=0x7fa9400091e0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.710+0000 7fa95b000640 1 -- 172.21.15.50:0/3825510554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa954074410 msgr2=0x7fa95414c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:51.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.710+0000 7fa95b000640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa954074410 0x7fa95414c740 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa944000c10 tx=0x7fa9440a55a0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.710+0000 7fa95b000640 1 -- 172.21.15.50:0/3825510554 shutdown_connections 2024-09-21T06:05:51.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.710+0000 7fa95b000640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa954152110 0x7fa9540b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.710+0000 7fa95b000640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9540d29c0 0x7fa954151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.710+0000 7fa95b000640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa9340743a0 0x7fa9340768b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.710+0000 7fa95b000640 1 --2- 172.21.15.50:0/3825510554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa954074410 0x7fa95414c740 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.710+0000 7fa95b000640 1 -- 172.21.15.50:0/3825510554 >> 172.21.15.50:0/3825510554 conn(0x7fa9540708c0 msgr2=0x7fa95414e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:51.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.710+0000 7fa95b000640 1 -- 172.21.15.50:0/3825510554 shutdown_connections 2024-09-21T06:05:51.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.710+0000 7fa95b000640 1 -- 172.21.15.50:0/3825510554 wait complete. 2024-09-21T06:05:51.733 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:05:51.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.902+0000 7f689d400640 1 -- 172.21.15.50:0/1220967918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6898069330 msgr2=0x7f689806cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.902+0000 7f689d400640 1 --2- 172.21.15.50:0/1220967918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6898069330 0x7f689806cdf0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f68980724a0 tx=0x7f688801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.902+0000 7f689d400640 1 -- 172.21.15.50:0/1220967918 shutdown_connections 2024-09-21T06:05:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.902+0000 7f689d400640 1 --2- 172.21.15.50:0/1220967918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6898069330 0x7f689806cdf0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.902+0000 7f689d400640 1 --2- 172.21.15.50:0/1220967918 >> v2:172.21.15.88:3300/0 conn(0x7f68980b0810 0x7f6898068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.902+0000 7f689d400640 1 -- 172.21.15.50:0/1220967918 >> 172.21.15.50:0/1220967918 conn(0x7f68980b35f0 msgr2=0x7f68980b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.902+0000 7f689d400640 1 -- 172.21.15.50:0/1220967918 shutdown_connections 2024-09-21T06:05:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.902+0000 7f689d400640 1 -- 172.21.15.50:0/1220967918 wait complete. 2024-09-21T06:05:51.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.906+0000 7f689d400640 1 Processor -- start 2024-09-21T06:05:51.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.906+0000 7f689d400640 1 -- start start 2024-09-21T06:05:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f689d400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6898069330 0x7f689816de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f689d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f689806d330 0x7f6898143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f689d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68980b0810 0x7f6898161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f689d400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6898154a10 con 0x7f689806d330 2024-09-21T06:05:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f689d400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6898154890 con 0x7f68980b0810 2024-09-21T06:05:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f689d400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6898154b90 con 0x7f6898069330 2024-09-21T06:05:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f6895e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f689806d330 0x7f6898143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f6895e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f689806d330 0x7f6898143fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46454/0 (socket says 172.21.15.50:46454) 2024-09-21T06:05:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f6895e00640 1 -- 172.21.15.50:0/3299730987 learned_addr learned my addr 172.21.15.50:0/3299730987 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f6896800640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6898069330 0x7f689816de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f6895e00640 1 -- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6898069330 msgr2=0x7f689816de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f6895e00640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6898069330 0x7f689816de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f6895e00640 1 -- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68980b0810 msgr2=0x7f6898161380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f6895e00640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68980b0810 0x7f6898161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f6895e00640 1 -- 172.21.15.50:0/3299730987 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689816e8a0 con 0x7f689806d330 2024-09-21T06:05:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.910+0000 7f6895e00640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f689806d330 0x7f6898143fe0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f68880194d0 tx=0x7f68880a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.914+0000 7f688ea00640 1 -- 172.21.15.50:0/3299730987 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68880bc070 con 0x7f689806d330 2024-09-21T06:05:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.914+0000 7f688ea00640 1 -- 172.21.15.50:0/3299730987 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6888004470 con 0x7f689806d330 2024-09-21T06:05:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.914+0000 7f689d400640 1 -- 172.21.15.50:0/3299730987 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68980aa600 con 0x7f689806d330 2024-09-21T06:05:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.914+0000 7f688ea00640 1 -- 172.21.15.50:0/3299730987 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68880b2040 con 0x7f689806d330 2024-09-21T06:05:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.914+0000 7f689d400640 1 -- 172.21.15.50:0/3299730987 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68980aab80 con 0x7f689806d330 2024-09-21T06:05:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.914+0000 7f689d400640 1 -- 172.21.15.50:0/3299730987 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689806cdf0 con 0x7f689806d330 2024-09-21T06:05:51.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.918+0000 7f688ea00640 1 -- 172.21.15.50:0/3299730987 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f6888003860 con 0x7f689806d330 2024-09-21T06:05:51.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.918+0000 7f688ea00640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6874074370 0x7f6874076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:51.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.918+0000 7f688ea00640 1 -- 172.21.15.50:0/3299730987 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7f688812cb80 con 0x7f689806d330 2024-09-21T06:05:51.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.918+0000 7f6896800640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6874074370 0x7f6874076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:51.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.922+0000 7f6896800640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6874074370 0x7f6874076880 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6880009b40 tx=0x7f68800091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:51.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:51.926+0000 7f688ea00640 1 -- 172.21.15.50:0/3299730987 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f68880164b0 con 0x7f689806d330 2024-09-21T06:05:52.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.330+0000 7f689d400640 1 -- 172.21.15.50:0/3299730987 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68980715c0 con 0x7f689806d330 2024-09-21T06:05:52.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.330+0000 7f688ea00640 1 -- 172.21.15.50:0/3299730987 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5222 (secure 0 0 0) 0x7f68880f8ce0 con 0x7f689806d330 2024-09-21T06:05:52.333 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:52.334 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":46,"btime":"2024-09-21T06:04:45:517919+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/2233580616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2233580616},{"type":"v1","addr":"172.21.15.50:6833","nonce":2233580616}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3749582023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3749582023},{"type":"v1","addr":"172.21.15.50:6835","nonce":3749582023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6139,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/382971646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":382971646},{"type":"v1","addr":"172.21.15.88:6836","nonce":382971646}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:04:32.178456+0000","modified":"2024-09-21T06:04:35.197824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6423":{"gid":6423,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/2131576448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2131576448},{"type":"v1","addr":"172.21.15.88:6837","nonce":2131576448}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6423,"qdb_cluster":[6423]},"id":5}]} 2024-09-21T06:05:52.334 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 46 2024-09-21T06:05:52.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.334+0000 7f689d400640 1 -- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6874074370 msgr2=0x7f6874076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:52.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.334+0000 7f689d400640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6874074370 0x7f6874076880 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6880009b40 tx=0x7f68800091e0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.334+0000 7f689d400640 1 -- 172.21.15.50:0/3299730987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f689806d330 msgr2=0x7f6898143fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:52.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.334+0000 7f689d400640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f689806d330 0x7f6898143fe0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f68880194d0 tx=0x7f68880a5700 comp rx=0 tx=0).stop 2024-09-21T06:05:52.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.334+0000 7f689d400640 1 -- 172.21.15.50:0/3299730987 shutdown_connections 2024-09-21T06:05:52.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.334+0000 7f689d400640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68980b0810 0x7f6898161380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.334+0000 7f689d400640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6874074370 0x7f6874076880 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.334+0000 7f689d400640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f689806d330 0x7f6898143fe0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.334+0000 7f689d400640 1 --2- 172.21.15.50:0/3299730987 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6898069330 0x7f689816de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.334+0000 7f689d400640 1 -- 172.21.15.50:0/3299730987 >> 172.21.15.50:0/3299730987 conn(0x7f68980b35f0 msgr2=0x7f68980b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:52.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.338+0000 7f689d400640 1 -- 172.21.15.50:0/3299730987 shutdown_connections 2024-09-21T06:05:52.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.338+0000 7f689d400640 1 -- 172.21.15.50:0/3299730987 wait complete. 2024-09-21T06:05:52.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:05:52.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.554+0000 7ff218a00640 1 -- 172.21.15.50:0/3159066267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2141527b0 msgr2=0x7ff214074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:52.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.554+0000 7ff218a00640 1 --2- 172.21.15.50:0/3159066267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2141527b0 0x7ff214074e30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009840 tx=0x7ff1fc01c760 comp rx=0 tx=0).stop 2024-09-21T06:05:52.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.554+0000 7ff218a00640 1 -- 172.21.15.50:0/3159066267 shutdown_connections 2024-09-21T06:05:52.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.554+0000 7ff218a00640 1 --2- 172.21.15.50:0/3159066267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2141527b0 0x7ff214074e30 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.554+0000 7ff218a00640 1 --2- 172.21.15.50:0/3159066267 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2140b5c70 0x7ff21415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.554+0000 7ff218a00640 1 -- 172.21.15.50:0/3159066267 >> 172.21.15.50:0/3159066267 conn(0x7ff2140b2040 msgr2=0x7ff2140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:52.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.554+0000 7ff218a00640 1 -- 172.21.15.50:0/3159066267 shutdown_connections 2024-09-21T06:05:52.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.554+0000 7ff218a00640 1 -- 172.21.15.50:0/3159066267 wait complete. 2024-09-21T06:05:52.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.554+0000 7ff218a00640 1 Processor -- start 2024-09-21T06:05:52.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.554+0000 7ff218a00640 1 -- start start 2024-09-21T06:05:52.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff218a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2140b4de0 0x7ff2140ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:52.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff218a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2140b5c70 0x7ff214158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:52.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff218a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2141527b0 0x7ff2141588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:52.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff218a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff214168c50 con 0x7ff2140b4de0 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff218a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff214168ad0 con 0x7ff2140b5c70 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff218a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff214168dd0 con 0x7ff2141527b0 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff211e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2140b4de0 0x7ff2140ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff211e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2140b4de0 0x7ff2140ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46480/0 (socket says 172.21.15.50:46480) 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff211e00640 1 -- 172.21.15.50:0/4200354839 learned_addr learned my addr 172.21.15.50:0/4200354839 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff209400640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2140b5c70 0x7ff214158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff212800640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2141527b0 0x7ff2141588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff209400640 1 -- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2141527b0 msgr2=0x7ff2141588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff209400640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2141527b0 0x7ff2141588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff209400640 1 -- 172.21.15.50:0/4200354839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2140b4de0 msgr2=0x7ff2140ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff209400640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2140b4de0 0x7ff2140ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff209400640 1 -- 172.21.15.50:0/4200354839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff214156eb0 con 0x7ff2140b5c70 2024-09-21T06:05:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff209400640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2140b5c70 0x7ff214158360 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0ac000 tx=0x7ff1fc0a57a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:52.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff20aa00640 1 -- 172.21.15.50:0/4200354839 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0b2040 con 0x7ff2140b5c70 2024-09-21T06:05:52.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff20aa00640 1 -- 172.21.15.50:0/4200354839 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1fc005020 con 0x7ff2140b5c70 2024-09-21T06:05:52.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff218a00640 1 -- 172.21.15.50:0/4200354839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff214157190 con 0x7ff2140b5c70 2024-09-21T06:05:52.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff20aa00640 1 -- 172.21.15.50:0/4200354839 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc005470 con 0x7ff2140b5c70 2024-09-21T06:05:52.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff218a00640 1 -- 172.21.15.50:0/4200354839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff214157750 con 0x7ff2140b5c70 2024-09-21T06:05:52.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.558+0000 7ff218a00640 1 -- 172.21.15.50:0/4200354839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff21415bb60 con 0x7ff2140b5c70 2024-09-21T06:05:52.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.562+0000 7ff20aa00640 1 -- 172.21.15.50:0/4200354839 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7ff1fc0046d0 con 0x7ff2140b5c70 2024-09-21T06:05:52.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.562+0000 7ff20aa00640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff1f00743a0 0x7ff1f00768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:52.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.562+0000 7ff20aa00640 1 -- 172.21.15.50:0/4200354839 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7ff1fc12cbc0 con 0x7ff2140b5c70 2024-09-21T06:05:52.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.562+0000 7ff211e00640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff1f00743a0 0x7ff1f00768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:52.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.562+0000 7ff211e00640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff1f00743a0 0x7ff1f00768b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff204005fd0 tx=0x7ff20400b410 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:52.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.566+0000 7ff20aa00640 1 -- 172.21.15.50:0/4200354839 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff214075580 con 0x7ff2140b5c70 2024-09-21T06:05:52.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.746+0000 7ff218a00640 1 -- 172.21.15.50:0/4200354839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff214074ff0 con 0x7ff2140b5c70 2024-09-21T06:05:52.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.746+0000 7ff20aa00640 1 -- 172.21.15.50:0/4200354839 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16932 (secure 0 0 0) 0x7ff1fc016410 con 0x7ff2140b5c70 2024-09-21T06:05:52.748 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:52.748 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":74,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:05:49.334633+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:04:29.936073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T06:04:31.146344+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":74,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:05:52.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.750+0000 7ff218a00640 1 -- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff1f00743a0 msgr2=0x7ff1f00768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:52.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.750+0000 7ff218a00640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff1f00743a0 0x7ff1f00768b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff204005fd0 tx=0x7ff20400b410 comp rx=0 tx=0).stop 2024-09-21T06:05:52.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.750+0000 7ff218a00640 1 -- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2140b5c70 msgr2=0x7ff214158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:52.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.750+0000 7ff218a00640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2140b5c70 0x7ff214158360 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0ac000 tx=0x7ff1fc0a57a0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.750+0000 7ff218a00640 1 -- 172.21.15.50:0/4200354839 shutdown_connections 2024-09-21T06:05:52.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.750+0000 7ff218a00640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2141527b0 0x7ff2141588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.750+0000 7ff218a00640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2140b5c70 0x7ff214158360 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.750+0000 7ff218a00640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff1f00743a0 0x7ff1f00768b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.750+0000 7ff218a00640 1 --2- 172.21.15.50:0/4200354839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2140b4de0 0x7ff2140ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.750+0000 7ff218a00640 1 -- 172.21.15.50:0/4200354839 >> 172.21.15.50:0/4200354839 conn(0x7ff2140b2040 msgr2=0x7ff2140b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:52.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.750+0000 7ff218a00640 1 -- 172.21.15.50:0/4200354839 shutdown_connections 2024-09-21T06:05:52.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.750+0000 7ff218a00640 1 -- 172.21.15.50:0/4200354839 wait complete. 2024-09-21T06:05:52.770 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:05:52.770 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:05:52.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.930+0000 7f5fa6600640 1 -- 172.21.15.50:0/1756241605 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fa0072100 msgr2=0x7f5fa0155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:52.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.930+0000 7f5fa6600640 1 --2- 172.21.15.50:0/1756241605 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fa0072100 0x7f5fa0155b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009900 tx=0x7f5f9001c8e0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.930+0000 7f5fa6600640 1 -- 172.21.15.50:0/1756241605 shutdown_connections 2024-09-21T06:05:52.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.930+0000 7f5fa6600640 1 --2- 172.21.15.50:0/1756241605 >> v2:172.21.15.88:3300/0 conn(0x7f5fa0073a90 0x7f5fa0073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.930+0000 7f5fa6600640 1 --2- 172.21.15.50:0/1756241605 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fa0072100 0x7f5fa0155b50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.930+0000 7f5fa6600640 1 -- 172.21.15.50:0/1756241605 >> 172.21.15.50:0/1756241605 conn(0x7f5fa00b2040 msgr2=0x7f5fa00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:52.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.930+0000 7f5fa6600640 1 -- 172.21.15.50:0/1756241605 shutdown_connections 2024-09-21T06:05:52.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.930+0000 7f5fa6600640 1 -- 172.21.15.50:0/1756241605 wait complete. 2024-09-21T06:05:52.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.930+0000 7f5fa6600640 1 Processor -- start 2024-09-21T06:05:52.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.930+0000 7f5fa6600640 1 -- start start 2024-09-21T06:05:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5fa6600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa0072100 0x7f5fa0164450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5fa6600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa0073a90 0x7f5fa014d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5fa6600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fa00b4de0 0x7f5fa01fa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5fa6600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa0168f30 con 0x7f5fa0072100 2024-09-21T06:05:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5fa6600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa0168db0 con 0x7f5fa0073a90 2024-09-21T06:05:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5fa6600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa01690b0 con 0x7f5fa00b4de0 2024-09-21T06:05:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f9f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa0072100 0x7f5fa0164450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f9f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa0072100 0x7f5fa0164450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46488/0 (socket says 172.21.15.50:46488) 2024-09-21T06:05:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f9f400640 1 -- 172.21.15.50:0/3240789101 learned_addr learned my addr 172.21.15.50:0/3240789101 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f9fe00640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fa00b4de0 0x7f5fa01fa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f96a00640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa0073a90 0x7f5fa014d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f9f400640 1 -- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fa00b4de0 msgr2=0x7f5fa01fa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f9f400640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fa00b4de0 0x7f5fa01fa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f9f400640 1 -- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa0073a90 msgr2=0x7f5fa014d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f9f400640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa0073a90 0x7f5fa014d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f9f400640 1 -- 172.21.15.50:0/3240789101 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa00aa670 con 0x7f5fa0072100 2024-09-21T06:05:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f9f400640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa0072100 0x7f5fa0164450 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5f900b0040 tx=0x7f5f90005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f97e00640 1 -- 172.21.15.50:0/3240789101 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f900b2040 con 0x7f5fa0072100 2024-09-21T06:05:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f97e00640 1 -- 172.21.15.50:0/3240789101 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f90004320 con 0x7f5fa0072100 2024-09-21T06:05:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5fa6600640 1 -- 172.21.15.50:0/3240789101 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa00aa910 con 0x7f5fa0072100 2024-09-21T06:05:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5f97e00640 1 -- 172.21.15.50:0/3240789101 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f900044f0 con 0x7f5fa0072100 2024-09-21T06:05:52.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5fa6600640 1 -- 172.21.15.50:0/3240789101 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa00aaed0 con 0x7f5fa0072100 2024-09-21T06:05:52.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.934+0000 7f5fa6600640 1 -- 172.21.15.50:0/3240789101 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa0155b50 con 0x7f5fa0072100 2024-09-21T06:05:52.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.938+0000 7f5f97e00640 1 -- 172.21.15.50:0/3240789101 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f5f90005290 con 0x7f5fa0072100 2024-09-21T06:05:52.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.938+0000 7f5f97e00640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5f7c074370 0x7f5f7c076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:52.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.938+0000 7f5f97e00640 1 -- 172.21.15.50:0/3240789101 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7f5f9012c390 con 0x7f5fa0072100 2024-09-21T06:05:52.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.938+0000 7f5f96a00640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5f7c074370 0x7f5f7c076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:52.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.938+0000 7f5f96a00640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5f7c074370 0x7f5f7c076880 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00b6430 tx=0x7f5f88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:52.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:52.942+0000 7f5f97e00640 1 -- 172.21.15.50:0/3240789101 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5f900164d0 con 0x7f5fa0072100 2024-09-21T06:05:53.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 -- 172.21.15.50:0/3240789101 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fa00b3740 con 0x7f5fa0072100 2024-09-21T06:05:53.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5f97e00640 1 -- 172.21.15.50:0/3240789101 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5222 (secure 0 0 0) 0x7f5f90005580 con 0x7f5fa0072100 2024-09-21T06:05:53.155 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:53.155 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":46,"btime":"2024-09-21T06:04:45:517919+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/2233580616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2233580616},{"type":"v1","addr":"172.21.15.50:6833","nonce":2233580616}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3749582023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3749582023},{"type":"v1","addr":"172.21.15.50:6835","nonce":3749582023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6139,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/382971646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":382971646},{"type":"v1","addr":"172.21.15.88:6836","nonce":382971646}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:04:32.178456+0000","modified":"2024-09-21T06:04:35.197824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6423":{"gid":6423,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/2131576448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2131576448},{"type":"v1","addr":"172.21.15.88:6837","nonce":2131576448}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6423,"qdb_cluster":[6423]},"id":5}]} 2024-09-21T06:05:53.155 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 46 2024-09-21T06:05:53.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 -- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5f7c074370 msgr2=0x7f5f7c076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5f7c074370 0x7f5f7c076880 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00b6430 tx=0x7f5f88009040 comp rx=0 tx=0).stop 2024-09-21T06:05:53.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 -- 172.21.15.50:0/3240789101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa0072100 msgr2=0x7f5fa0164450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa0072100 0x7f5fa0164450 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5f900b0040 tx=0x7f5f90005760 comp rx=0 tx=0).stop 2024-09-21T06:05:53.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 -- 172.21.15.50:0/3240789101 shutdown_connections 2024-09-21T06:05:53.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fa00b4de0 0x7f5fa01fa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa0073a90 0x7f5fa014d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5f7c074370 0x7f5f7c076880 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 --2- 172.21.15.50:0/3240789101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa0072100 0x7f5fa0164450 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 -- 172.21.15.50:0/3240789101 >> 172.21.15.50:0/3240789101 conn(0x7f5fa00b2040 msgr2=0x7f5fa0159a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:53.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 -- 172.21.15.50:0/3240789101 shutdown_connections 2024-09-21T06:05:53.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.154+0000 7f5fa6600640 1 -- 172.21.15.50:0/3240789101 wait complete. 2024-09-21T06:05:53.177 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:05:53.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.334+0000 7f620e200640 1 -- 172.21.15.50:0/1523529192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62080b4de0 msgr2=0x7f62080b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.334+0000 7f620e200640 1 --2- 172.21.15.50:0/1523529192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62080b4de0 0x7f62080b51f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f62080703e0 tx=0x7f620001ccc0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.334+0000 7f620e200640 1 -- 172.21.15.50:0/1523529192 shutdown_connections 2024-09-21T06:05:53.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.334+0000 7f620e200640 1 --2- 172.21.15.50:0/1523529192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62080b4de0 0x7f62080b51f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.334+0000 7f620e200640 1 --2- 172.21.15.50:0/1523529192 >> v2:172.21.15.88:3300/0 conn(0x7f62081527b0 0x7f6208074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.334+0000 7f620e200640 1 --2- 172.21.15.50:0/1523529192 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62080b5c70 0x7f620815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.334+0000 7f620e200640 1 -- 172.21.15.50:0/1523529192 >> 172.21.15.50:0/1523529192 conn(0x7f62080b2040 msgr2=0x7f62080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:53.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.334+0000 7f620e200640 1 -- 172.21.15.50:0/1523529192 shutdown_connections 2024-09-21T06:05:53.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.334+0000 7f620e200640 1 -- 172.21.15.50:0/1523529192 wait complete. 2024-09-21T06:05:53.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f620e200640 1 Processor -- start 2024-09-21T06:05:53.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f620e200640 1 -- start start 2024-09-21T06:05:53.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f620e200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62080b4de0 0x7f62081f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f620e200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62080b5c70 0x7f620814e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f620e200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62081527b0 0x7f62081577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f620e200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f620815f6b0 con 0x7f62080b4de0 2024-09-21T06:05:53.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f620e200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f620815f530 con 0x7f62081527b0 2024-09-21T06:05:53.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f620e200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f620815f830 con 0x7f62080b5c70 2024-09-21T06:05:53.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f6207400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62080b4de0 0x7f62081f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f6207400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62080b4de0 0x7f62081f88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46510/0 (socket says 172.21.15.50:46510) 2024-09-21T06:05:53.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f6207400640 1 -- 172.21.15.50:0/3137616861 learned_addr learned my addr 172.21.15.50:0/3137616861 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:53.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f61ffe00640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62080b5c70 0x7f620814e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f6207e00640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62081527b0 0x7f62081577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f6207400640 1 -- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62080b5c70 msgr2=0x7f620814e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f6207400640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62080b5c70 0x7f620814e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f6207400640 1 -- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62081527b0 msgr2=0x7f62081577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f6207400640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62081527b0 0x7f62081577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f6207400640 1 -- 172.21.15.50:0/3137616861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62081f8e00 con 0x7f62080b4de0 2024-09-21T06:05:53.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f6207400640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62080b4de0 0x7f62081f88c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f61f800e740 tx=0x7f61f800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:53.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f61ff400640 1 -- 172.21.15.50:0/3137616861 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f800ccc0 con 0x7f62080b4de0 2024-09-21T06:05:53.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f61ff400640 1 -- 172.21.15.50:0/3137616861 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61f800ce60 con 0x7f62080b4de0 2024-09-21T06:05:53.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f620e200640 1 -- 172.21.15.50:0/3137616861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6208156730 con 0x7f62080b4de0 2024-09-21T06:05:53.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f61ff400640 1 -- 172.21.15.50:0/3137616861 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8005280 con 0x7f62080b4de0 2024-09-21T06:05:53.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.338+0000 7f620e200640 1 -- 172.21.15.50:0/3137616861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6208156cf0 con 0x7f62080b4de0 2024-09-21T06:05:53.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.342+0000 7f620e200640 1 -- 172.21.15.50:0/3137616861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f620815fd50 con 0x7f62080b4de0 2024-09-21T06:05:53.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.342+0000 7f61ff400640 1 -- 172.21.15.50:0/3137616861 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f61f8005420 con 0x7f62080b4de0 2024-09-21T06:05:53.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.346+0000 7f61ff400640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f61e4074370 0x7f61e4076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.346+0000 7f61ff400640 1 -- 172.21.15.50:0/3137616861 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7f61f8093d50 con 0x7f62080b4de0 2024-09-21T06:05:53.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.346+0000 7f61ffe00640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f61e4074370 0x7f61e4076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.346+0000 7f61ffe00640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f61e4074370 0x7f61e4076880 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6208159890 tx=0x7f61f00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:53.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.350+0000 7f61ff400640 1 -- 172.21.15.50:0/3137616861 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f61f805ff10 con 0x7f62080b4de0 2024-09-21T06:05:53.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 -- 172.21.15.50:0/3137616861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6208074f20 con 0x7f62080b4de0 2024-09-21T06:05:53.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f61ff400640 1 -- 172.21.15.50:0/3137616861 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5222 (secure 0 0 0) 0x7f61f8064700 con 0x7f62080b4de0 2024-09-21T06:05:53.566 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:53.566 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":46,"btime":"2024-09-21T06:04:45:517919+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/2233580616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2233580616},{"type":"v1","addr":"172.21.15.50:6833","nonce":2233580616}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3749582023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3749582023},{"type":"v1","addr":"172.21.15.50:6835","nonce":3749582023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6139,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/382971646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":382971646},{"type":"v1","addr":"172.21.15.88:6836","nonce":382971646}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:04:32.178456+0000","modified":"2024-09-21T06:04:35.197824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6423":{"gid":6423,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/2131576448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2131576448},{"type":"v1","addr":"172.21.15.88:6837","nonce":2131576448}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6423,"qdb_cluster":[6423]},"id":5}]} 2024-09-21T06:05:53.567 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 46 2024-09-21T06:05:53.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 -- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f61e4074370 msgr2=0x7f61e4076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f61e4074370 0x7f61e4076880 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6208159890 tx=0x7f61f00091e0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 -- 172.21.15.50:0/3137616861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62080b4de0 msgr2=0x7f62081f88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62080b4de0 0x7f62081f88c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f61f800e740 tx=0x7f61f800ec00 comp rx=0 tx=0).stop 2024-09-21T06:05:53.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 -- 172.21.15.50:0/3137616861 shutdown_connections 2024-09-21T06:05:53.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62081527b0 0x7f62081577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62080b5c70 0x7f620814e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f61e4074370 0x7f61e4076880 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 --2- 172.21.15.50:0/3137616861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62080b4de0 0x7f62081f88c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 -- 172.21.15.50:0/3137616861 >> 172.21.15.50:0/3137616861 conn(0x7f62080b2040 msgr2=0x7f620806e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:53.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 -- 172.21.15.50:0/3137616861 shutdown_connections 2024-09-21T06:05:53.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.566+0000 7f620e200640 1 -- 172.21.15.50:0/3137616861 wait complete. 2024-09-21T06:05:53.590 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:05:53.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.750+0000 7fd1dbc00640 1 -- 172.21.15.50:0/987916568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1d40b5c70 msgr2=0x7fd1d415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.750+0000 7fd1dbc00640 1 --2- 172.21.15.50:0/987916568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1d40b5c70 0x7fd1d415bb60 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009900 tx=0x7fd1c801c8b0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.750+0000 7fd1dbc00640 1 -- 172.21.15.50:0/987916568 shutdown_connections 2024-09-21T06:05:53.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.750+0000 7fd1dbc00640 1 --2- 172.21.15.50:0/987916568 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1d40b4de0 0x7fd1d40b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.750+0000 7fd1dbc00640 1 --2- 172.21.15.50:0/987916568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1d40b5c70 0x7fd1d415bb60 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.750+0000 7fd1dbc00640 1 -- 172.21.15.50:0/987916568 >> 172.21.15.50:0/987916568 conn(0x7fd1d40b2040 msgr2=0x7fd1d40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:53.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.750+0000 7fd1dbc00640 1 -- 172.21.15.50:0/987916568 shutdown_connections 2024-09-21T06:05:53.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.750+0000 7fd1dbc00640 1 -- 172.21.15.50:0/987916568 wait complete. 2024-09-21T06:05:53.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.750+0000 7fd1dbc00640 1 Processor -- start 2024-09-21T06:05:53.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.750+0000 7fd1dbc00640 1 -- start start 2024-09-21T06:05:53.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1dbc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1d40b4de0 0x7fd1d41555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1dbc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1d40b5c70 0x7fd1d4157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1dbc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1d41527b0 0x7fd1d4157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1dbc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d4168970 con 0x7fd1d40b5c70 2024-09-21T06:05:53.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1dbc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d41687f0 con 0x7fd1d41527b0 2024-09-21T06:05:53.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1dbc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d4168af0 con 0x7fd1d40b4de0 2024-09-21T06:05:53.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1d40b5c70 0x7fd1d4157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d9200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1d40b4de0 0x7fd1d41555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1d40b5c70 0x7fd1d4157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46522/0 (socket says 172.21.15.50:46522) 2024-09-21T06:05:53.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d9c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1d41527b0 0x7fd1d4157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d9200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1d40b4de0 0x7fd1d41555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:49378/0 (socket says 172.21.15.50:49378) 2024-09-21T06:05:53.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d3e00640 1 -- 172.21.15.50:0/186781566 learned_addr learned my addr 172.21.15.50:0/186781566 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:53.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d3e00640 1 -- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1d40b4de0 msgr2=0x7fd1d41555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d3e00640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1d40b4de0 0x7fd1d41555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d3e00640 1 -- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1d41527b0 msgr2=0x7fd1d4157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d3e00640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1d41527b0 0x7fd1d4157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d3e00640 1 -- 172.21.15.50:0/186781566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d4156fb0 con 0x7fd1d40b5c70 2024-09-21T06:05:53.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d9200640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1d40b4de0 0x7fd1d41555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:05:53.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d3e00640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1d40b5c70 0x7fd1d4157a80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0001ec0 tx=0x7fd1c000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:53.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d0c00640 1 -- 172.21.15.50:0/186781566 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0017070 con 0x7fd1d40b5c70 2024-09-21T06:05:53.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1dbc00640 1 -- 172.21.15.50:0/186781566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d4152bc0 con 0x7fd1d40b5c70 2024-09-21T06:05:53.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d0c00640 1 -- 172.21.15.50:0/186781566 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1c0003600 con 0x7fd1d40b5c70 2024-09-21T06:05:53.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1d0c00640 1 -- 172.21.15.50:0/186781566 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0003a50 con 0x7fd1d40b5c70 2024-09-21T06:05:53.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1dbc00640 1 -- 172.21.15.50:0/186781566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d4157570 con 0x7fd1d40b5c70 2024-09-21T06:05:53.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.754+0000 7fd1dbc00640 1 -- 172.21.15.50:0/186781566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d40b51f0 con 0x7fd1d40b5c70 2024-09-21T06:05:53.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.758+0000 7fd1d0c00640 1 -- 172.21.15.50:0/186781566 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fd1c001a070 con 0x7fd1d40b5c70 2024-09-21T06:05:53.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.758+0000 7fd1d0c00640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd1b4074370 0x7fd1b4076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.758+0000 7fd1d0c00640 1 -- 172.21.15.50:0/186781566 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7fd1c0092440 con 0x7fd1d40b5c70 2024-09-21T06:05:53.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.758+0000 7fd1d9200640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd1b4074370 0x7fd1b4076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.758+0000 7fd1d9200640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd1b4074370 0x7fd1b4076880 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80af000 tx=0x7fd1c801b0f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.762+0000 7fd1d0c00640 1 -- 172.21.15.50:0/186781566 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd1d40b5730 con 0x7fd1d40b5c70 2024-09-21T06:05:53.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.938+0000 7fd1dbc00640 1 -- 172.21.15.50:0/186781566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1d4156d20 con 0x7fd1d40b5c70 2024-09-21T06:05:53.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1d0c00640 1 -- 172.21.15.50:0/186781566 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16932 (secure 0 0 0) 0x7fd1c005e520 con 0x7fd1d40b5c70 2024-09-21T06:05:53.942 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:53.943 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":74,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:05:49.334633+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:04:29.936073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T06:04:31.146344+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":74,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:05:53.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1dbc00640 1 -- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd1b4074370 msgr2=0x7fd1b4076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1dbc00640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd1b4074370 0x7fd1b4076880 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80af000 tx=0x7fd1c801b0f0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1dbc00640 1 -- 172.21.15.50:0/186781566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1d40b5c70 msgr2=0x7fd1d4157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1dbc00640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1d40b5c70 0x7fd1d4157a80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0001ec0 tx=0x7fd1c000c890 comp rx=0 tx=0).stop 2024-09-21T06:05:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1dbc00640 1 -- 172.21.15.50:0/186781566 shutdown_connections 2024-09-21T06:05:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1dbc00640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1d41527b0 0x7fd1d4157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1dbc00640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd1b4074370 0x7fd1b4076880 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1dbc00640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1d40b5c70 0x7fd1d4157a80 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1dbc00640 1 --2- 172.21.15.50:0/186781566 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1d40b4de0 0x7fd1d41555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1dbc00640 1 -- 172.21.15.50:0/186781566 >> 172.21.15.50:0/186781566 conn(0x7fd1d40b2040 msgr2=0x7fd1d40ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1dbc00640 1 -- 172.21.15.50:0/186781566 shutdown_connections 2024-09-21T06:05:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:53.942+0000 7fd1dbc00640 1 -- 172.21.15.50:0/186781566 wait complete. 2024-09-21T06:05:53.966 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:05:54.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.122+0000 7f0ae2000640 1 -- 172.21.15.50:0/3320707456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0adc0b3340 msgr2=0x7f0adc0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:54.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.122+0000 7f0ae2000640 1 --2- 172.21.15.50:0/3320707456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0adc0b3340 0x7f0adc0b3750 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0726f0 tx=0x7f0ad401cd80 comp rx=0 tx=0).stop 2024-09-21T06:05:54.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.122+0000 7f0ae2000640 1 -- 172.21.15.50:0/3320707456 shutdown_connections 2024-09-21T06:05:54.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.122+0000 7f0ae2000640 1 --2- 172.21.15.50:0/3320707456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0adc0b3340 0x7f0adc0b3750 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.122+0000 7f0ae2000640 1 --2- 172.21.15.50:0/3320707456 >> v2:172.21.15.88:3300/0 conn(0x7f0adc0cefa0 0x7f0adc0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.122+0000 7f0ae2000640 1 -- 172.21.15.50:0/3320707456 >> 172.21.15.50:0/3320707456 conn(0x7f0adc0b2040 msgr2=0x7f0adc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:54.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.122+0000 7f0ae2000640 1 -- 172.21.15.50:0/3320707456 shutdown_connections 2024-09-21T06:05:54.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.122+0000 7f0ae2000640 1 -- 172.21.15.50:0/3320707456 wait complete. 2024-09-21T06:05:54.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0ae2000640 1 Processor -- start 2024-09-21T06:05:54.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0ae2000640 1 -- start start 2024-09-21T06:05:54.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0ae2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0adc0749a0 0x7f0adc15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:54.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0ae2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0adc0b3340 0x7f0adc0af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:54.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0ae2000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0adc0cefa0 0x7f0adc160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:54.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0ae2000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc169d70 con 0x7f0adc0749a0 2024-09-21T06:05:54.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0ae2000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc169bf0 con 0x7f0adc0b3340 2024-09-21T06:05:54.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0ae2000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0adc169ef0 con 0x7f0adc0cefa0 2024-09-21T06:05:54.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0adb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0adc0749a0 0x7f0adc15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:54.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0adb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0adc0749a0 0x7f0adc15a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46546/0 (socket says 172.21.15.50:46546) 2024-09-21T06:05:54.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0adb400640 1 -- 172.21.15.50:0/3117600037 learned_addr learned my addr 172.21.15.50:0/3117600037 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:54.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0adbe00640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0adc0cefa0 0x7f0adc160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:54.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.126+0000 7f0ad2a00640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0adc0b3340 0x7f0adc0af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:54.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0adb400640 1 -- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0adc0cefa0 msgr2=0x7f0adc160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:54.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0adb400640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0adc0cefa0 0x7f0adc160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0adb400640 1 -- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0adc0b3340 msgr2=0x7f0adc0af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:54.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0adb400640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0adc0b3340 0x7f0adc0af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0adb400640 1 -- 172.21.15.50:0/3117600037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0adc15a6d0 con 0x7f0adc0749a0 2024-09-21T06:05:54.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0adb400640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0adc0749a0 0x7f0adc15a190 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00c3c0 tx=0x7f0acc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:54.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0ad3e00640 1 -- 172.21.15.50:0/3117600037 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc017070 con 0x7f0adc0749a0 2024-09-21T06:05:54.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0ad3e00640 1 -- 172.21.15.50:0/3117600037 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0acc0053c0 con 0x7f0adc0749a0 2024-09-21T06:05:54.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0ae2000640 1 -- 172.21.15.50:0/3117600037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0adc0afa90 con 0x7f0adc0749a0 2024-09-21T06:05:54.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0ad3e00640 1 -- 172.21.15.50:0/3117600037 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc003860 con 0x7f0adc0749a0 2024-09-21T06:05:54.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0ae2000640 1 -- 172.21.15.50:0/3117600037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0adc0ce7b0 con 0x7f0adc0749a0 2024-09-21T06:05:54.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0ae2000640 1 -- 172.21.15.50:0/3117600037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0adc0b3750 con 0x7f0adc0749a0 2024-09-21T06:05:54.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0ad3e00640 1 -- 172.21.15.50:0/3117600037 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f0acc01a070 con 0x7f0adc0749a0 2024-09-21T06:05:54.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0ad3e00640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ab8074370 0x7f0ab8076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:54.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.130+0000 7f0ad3e00640 1 -- 172.21.15.50:0/3117600037 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7f0acc092470 con 0x7f0adc0749a0 2024-09-21T06:05:54.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.134+0000 7f0ad2a00640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ab8074370 0x7f0ab8076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:54.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.134+0000 7f0ad2a00640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ab8074370 0x7f0ab8076880 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0adc13e770 tx=0x7f0ac400b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:54.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.138+0000 7f0ad3e00640 1 -- 172.21.15.50:0/3117600037 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0acc05e540 con 0x7f0adc0749a0 2024-09-21T06:05:54.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.350+0000 7f0ae2000640 1 -- 172.21.15.50:0/3117600037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0adc15af00 con 0x7f0adc0749a0 2024-09-21T06:05:54.350 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-09-21T06:05:54.350+0000 7f0d45000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:05:54.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.350+0000 7f0ad3e00640 1 -- 172.21.15.50:0/3117600037 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) ==== 161+0+0 (secure 0 0 0) 0x7f0acc062f70 con 0x7f0adc0749a0 2024-09-21T06:05:54.352 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:05:54.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.354+0000 7f0ae2000640 1 -- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ab8074370 msgr2=0x7f0ab8076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:54.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.354+0000 7f0ae2000640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ab8074370 0x7f0ab8076880 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0adc13e770 tx=0x7f0ac400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.354+0000 7f0ae2000640 1 -- 172.21.15.50:0/3117600037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0adc0749a0 msgr2=0x7f0adc15a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:54.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.354+0000 7f0ae2000640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0adc0749a0 0x7f0adc15a190 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00c3c0 tx=0x7f0acc00c880 comp rx=0 tx=0).stop 2024-09-21T06:05:54.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.354+0000 7f0ae2000640 1 -- 172.21.15.50:0/3117600037 shutdown_connections 2024-09-21T06:05:54.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.354+0000 7f0ae2000640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0adc0cefa0 0x7f0adc160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.354+0000 7f0ae2000640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0adc0b3340 0x7f0adc0af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.354+0000 7f0ae2000640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ab8074370 0x7f0ab8076880 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.354+0000 7f0ae2000640 1 --2- 172.21.15.50:0/3117600037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0adc0749a0 0x7f0adc15a190 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.354+0000 7f0ae2000640 1 -- 172.21.15.50:0/3117600037 >> 172.21.15.50:0/3117600037 conn(0x7f0adc0b2040 msgr2=0x7f0adc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:54.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.354+0000 7f0ae2000640 1 -- 172.21.15.50:0/3117600037 shutdown_connections 2024-09-21T06:05:54.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.354+0000 7f0ae2000640 1 -- 172.21.15.50:0/3117600037 wait complete. 2024-09-21T06:05:54.378 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:05:54.495 INFO:tasks.ceph.mds.d.smithi088.stdout:starting mds.d at 2024-09-21T06:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.542+0000 7fce88400640 1 -- 172.21.15.50:0/650023236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce800b54f0 msgr2=0x7fce8015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.542+0000 7fce88400640 1 --2- 172.21.15.50:0/650023236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce800b54f0 0x7fce8015bb60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fce78009bc0 tx=0x7fce7801cae0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.542+0000 7fce88400640 1 -- 172.21.15.50:0/650023236 shutdown_connections 2024-09-21T06:05:54.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.542+0000 7fce88400640 1 --2- 172.21.15.50:0/650023236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce800b54f0 0x7fce8015bb60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.542+0000 7fce88400640 1 -- 172.21.15.50:0/650023236 >> 172.21.15.50:0/650023236 conn(0x7fce800b2040 msgr2=0x7fce800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:54.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.542+0000 7fce88400640 1 -- 172.21.15.50:0/650023236 shutdown_connections 2024-09-21T06:05:54.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.542+0000 7fce88400640 1 -- 172.21.15.50:0/650023236 wait complete. 2024-09-21T06:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce88400640 1 Processor -- start 2024-09-21T06:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce88400640 1 -- start start 2024-09-21T06:05:54.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce88400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce800b5c70 0x7fce80158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:54.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce88400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fce801527b0 0x7fce80140440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:54.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce88400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce80151bf0 0x7fce80140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:54.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce88400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce80169b30 con 0x7fce800b5c70 2024-09-21T06:05:54.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce88400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fce801699b0 con 0x7fce80151bf0 2024-09-21T06:05:54.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce88400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fce80169cb0 con 0x7fce801527b0 2024-09-21T06:05:54.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce85a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce800b5c70 0x7fce80158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:54.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce85a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce800b5c70 0x7fce80158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46560/0 (socket says 172.21.15.50:46560) 2024-09-21T06:05:54.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce85a00640 1 -- 172.21.15.50:0/2969542254 learned_addr learned my addr 172.21.15.50:0/2969542254 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:54.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce85000640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fce801527b0 0x7fce80140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:54.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce85a00640 1 -- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fce801527b0 msgr2=0x7fce80140440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:54.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce86400640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce80151bf0 0x7fce80140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:54.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce85a00640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fce801527b0 0x7fce80140440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce85a00640 1 -- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce80151bf0 msgr2=0x7fce80140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:54.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce85a00640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce80151bf0 0x7fce80140980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:54.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce85a00640 1 -- 172.21.15.50:0/2969542254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce80158fc0 con 0x7fce800b5c70 2024-09-21T06:05:54.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce86400640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce80151bf0 0x7fce80140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:54.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce85000640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fce801527b0 0x7fce80140440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:54.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce85a00640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce800b5c70 0x7fce80158790 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fce74001ec0 tx=0x7fce7400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:54.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce7d600640 1 -- 172.21.15.50:0/2969542254 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74017070 con 0x7fce800b5c70 2024-09-21T06:05:54.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce88400640 1 -- 172.21.15.50:0/2969542254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce801592a0 con 0x7fce800b5c70 2024-09-21T06:05:54.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce7d600640 1 -- 172.21.15.50:0/2969542254 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce74003600 con 0x7fce800b5c70 2024-09-21T06:05:54.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce7d600640 1 -- 172.21.15.50:0/2969542254 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74003a50 con 0x7fce800b5c70 2024-09-21T06:05:54.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce88400640 1 -- 172.21.15.50:0/2969542254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce80158cd0 con 0x7fce800b5c70 2024-09-21T06:05:54.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.546+0000 7fce88400640 1 -- 172.21.15.50:0/2969542254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce8015bb60 con 0x7fce800b5c70 2024-09-21T06:05:54.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.550+0000 7fce7d600640 1 -- 172.21.15.50:0/2969542254 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fce7401a070 con 0x7fce800b5c70 2024-09-21T06:05:54.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.550+0000 7fce7d600640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fce60074370 0x7fce60076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:54.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.550+0000 7fce7d600640 1 -- 172.21.15.50:0/2969542254 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5909+0+0 (secure 0 0 0) 0x7fce74092520 con 0x7fce800b5c70 2024-09-21T06:05:54.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.550+0000 7fce85000640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fce60074370 0x7fce60076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:54.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.550+0000 7fce85000640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fce60074370 0x7fce60076880 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fce801671e0 tx=0x7fce6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:54.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.554+0000 7fce7d600640 1 -- 172.21.15.50:0/2969542254 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fce7405ebc0 con 0x7fce800b5c70 2024-09-21T06:05:54.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:54.770+0000 7fce88400640 1 -- 172.21.15.50:0/2969542254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fce8015c740 con 0x7fce800b5c70 2024-09-21T06:05:55.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.370+0000 7fce7d600640 1 -- 172.21.15.50:0/2969542254 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) ==== 104+0+0 (secure 0 0 0) 0x7fce7405e4c0 con 0x7fce800b5c70 2024-09-21T06:05:55.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.374+0000 7fce88400640 1 -- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fce60074370 msgr2=0x7fce60076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:55.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.374+0000 7fce88400640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fce60074370 0x7fce60076880 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fce801671e0 tx=0x7fce6c009040 comp rx=0 tx=0).stop 2024-09-21T06:05:55.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.374+0000 7fce88400640 1 -- 172.21.15.50:0/2969542254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce800b5c70 msgr2=0x7fce80158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:55.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.374+0000 7fce88400640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce800b5c70 0x7fce80158790 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fce74001ec0 tx=0x7fce7400c890 comp rx=0 tx=0).stop 2024-09-21T06:05:55.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.374+0000 7fce88400640 1 -- 172.21.15.50:0/2969542254 shutdown_connections 2024-09-21T06:05:55.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.374+0000 7fce88400640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce80151bf0 0x7fce80140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:55.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.374+0000 7fce88400640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fce801527b0 0x7fce80140440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:55.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.374+0000 7fce88400640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fce60074370 0x7fce60076880 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:55.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.374+0000 7fce88400640 1 --2- 172.21.15.50:0/2969542254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce800b5c70 0x7fce80158790 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:55.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.374+0000 7fce88400640 1 -- 172.21.15.50:0/2969542254 >> 172.21.15.50:0/2969542254 conn(0x7fce800b2040 msgr2=0x7fce800b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:55.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.374+0000 7fce88400640 1 -- 172.21.15.50:0/2969542254 shutdown_connections 2024-09-21T06:05:55.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.374+0000 7fce88400640 1 -- 172.21.15.50:0/2969542254 wait complete. 2024-09-21T06:05:55.398 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:05:55.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.562+0000 7fef56400640 1 -- 172.21.15.50:0/209123791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef501527b0 msgr2=0x7fef5015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:55.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.562+0000 7fef56400640 1 --2- 172.21.15.50:0/209123791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef501527b0 0x7fef5015bb60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fef38009840 tx=0x7fef3801c7f0 comp rx=0 tx=0).stop 2024-09-21T06:05:55.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.562+0000 7fef56400640 1 -- 172.21.15.50:0/209123791 shutdown_connections 2024-09-21T06:05:55.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.562+0000 7fef56400640 1 --2- 172.21.15.50:0/209123791 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef500b4de0 0x7fef500b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:55.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.562+0000 7fef56400640 1 --2- 172.21.15.50:0/209123791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef501527b0 0x7fef5015bb60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:55.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.562+0000 7fef56400640 1 -- 172.21.15.50:0/209123791 >> 172.21.15.50:0/209123791 conn(0x7fef500b2040 msgr2=0x7fef500b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:55.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.562+0000 7fef56400640 1 -- 172.21.15.50:0/209123791 shutdown_connections 2024-09-21T06:05:55.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.562+0000 7fef56400640 1 -- 172.21.15.50:0/209123791 wait complete. 2024-09-21T06:05:55.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef56400640 1 Processor -- start 2024-09-21T06:05:55.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef56400640 1 -- start start 2024-09-21T06:05:55.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef56400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef500b4de0 0x7fef50154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:55.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef56400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef500b5c70 0x7fef50140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:55.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef56400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef501527b0 0x7fef50140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:55.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef56400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef50168950 con 0x7fef501527b0 2024-09-21T06:05:55.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef56400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fef501687d0 con 0x7fef500b5c70 2024-09-21T06:05:55.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef56400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fef50168ad0 con 0x7fef500b4de0 2024-09-21T06:05:55.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef4fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef501527b0 0x7fef50140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:55.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef4fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef501527b0 0x7fef50140d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46566/0 (socket says 172.21.15.50:46566) 2024-09-21T06:05:55.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef4fe00640 1 -- 172.21.15.50:0/3746056891 learned_addr learned my addr 172.21.15.50:0/3746056891 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:55.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef4f400640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef500b4de0 0x7fef50154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:55.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef47e00640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef500b5c70 0x7fef50140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:55.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef4fe00640 1 -- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef500b4de0 msgr2=0x7fef50154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:55.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef4fe00640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef500b4de0 0x7fef50154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:55.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef4fe00640 1 -- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef500b5c70 msgr2=0x7fef50140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:55.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef4fe00640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef500b5c70 0x7fef50140820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:55.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef4fe00640 1 -- 172.21.15.50:0/3746056891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef50150160 con 0x7fef501527b0 2024-09-21T06:05:55.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef4fe00640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef501527b0 0x7fef50140d60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fef4800ec00 tx=0x7fef4800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:55.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef47400640 1 -- 172.21.15.50:0/3746056891 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48003b80 con 0x7fef501527b0 2024-09-21T06:05:55.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef47400640 1 -- 172.21.15.50:0/3746056891 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef48003e10 con 0x7fef501527b0 2024-09-21T06:05:55.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef56400640 1 -- 172.21.15.50:0/3746056891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef50155710 con 0x7fef501527b0 2024-09-21T06:05:55.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef47400640 1 -- 172.21.15.50:0/3746056891 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48004280 con 0x7fef501527b0 2024-09-21T06:05:55.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.566+0000 7fef56400640 1 -- 172.21.15.50:0/3746056891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5013ea30 con 0x7fef501527b0 2024-09-21T06:05:55.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.570+0000 7fef56400640 1 -- 172.21.15.50:0/3746056891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef500b51f0 con 0x7fef501527b0 2024-09-21T06:05:55.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.570+0000 7fef47400640 1 -- 172.21.15.50:0/3746056891 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fef48004420 con 0x7fef501527b0 2024-09-21T06:05:55.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.570+0000 7fef47400640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef2c074370 0x7fef2c076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:55.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.570+0000 7fef47400640 1 -- 172.21.15.50:0/3746056891 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5909+0+0 (secure 0 0 0) 0x7fef48093f60 con 0x7fef501527b0 2024-09-21T06:05:55.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.574+0000 7fef4f400640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef2c074370 0x7fef2c076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:55.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.574+0000 7fef4f400640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef2c074370 0x7fef2c076880 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fef40001400 tx=0x7fef40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:55.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.578+0000 7fef47400640 1 -- 172.21.15.50:0/3746056891 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fef50150160 con 0x7fef501527b0 2024-09-21T06:05:55.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:55.758+0000 7fef56400640 1 -- 172.21.15.50:0/3746056891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fef500b0bd0 con 0x7fef501527b0 2024-09-21T06:05:56.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.382+0000 7fef47400640 1 -- 172.21.15.50:0/3746056891 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) ==== 181+0+0 (secure 0 0 0) 0x7fef480606e0 con 0x7fef501527b0 2024-09-21T06:05:56.382 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:05:56.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.386+0000 7fef56400640 1 -- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef2c074370 msgr2=0x7fef2c076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:56.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.386+0000 7fef56400640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef2c074370 0x7fef2c076880 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fef40001400 tx=0x7fef40009040 comp rx=0 tx=0).stop 2024-09-21T06:05:56.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.386+0000 7fef56400640 1 -- 172.21.15.50:0/3746056891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef501527b0 msgr2=0x7fef50140d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:56.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.386+0000 7fef56400640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef501527b0 0x7fef50140d60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fef4800ec00 tx=0x7fef4800c6a0 comp rx=0 tx=0).stop 2024-09-21T06:05:56.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.386+0000 7fef56400640 1 -- 172.21.15.50:0/3746056891 shutdown_connections 2024-09-21T06:05:56.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.386+0000 7fef56400640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef2c074370 0x7fef2c076880 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:56.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.386+0000 7fef56400640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef501527b0 0x7fef50140d60 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:56.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.386+0000 7fef56400640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef500b5c70 0x7fef50140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:56.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.386+0000 7fef56400640 1 --2- 172.21.15.50:0/3746056891 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef500b4de0 0x7fef50154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:56.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.386+0000 7fef56400640 1 -- 172.21.15.50:0/3746056891 >> 172.21.15.50:0/3746056891 conn(0x7fef500b2040 msgr2=0x7fef5015b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:56.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.386+0000 7fef56400640 1 -- 172.21.15.50:0/3746056891 shutdown_connections 2024-09-21T06:05:56.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.386+0000 7fef56400640 1 -- 172.21.15.50:0/3746056891 wait complete. 2024-09-21T06:05:56.410 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:05:56.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.574+0000 7fd681c00640 1 -- 172.21.15.50:0/1003866967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd660003a60 msgr2=0x7fd660003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:56.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.574+0000 7fd681c00640 1 --2- 172.21.15.50:0/1003866967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd660003a60 0x7fd660003ed0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00c310 tx=0x7fd66c01cef0 comp rx=0 tx=0).stop 2024-09-21T06:05:56.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.578+0000 7fd681c00640 1 -- 172.21.15.50:0/1003866967 shutdown_connections 2024-09-21T06:05:56.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.578+0000 7fd681c00640 1 --2- 172.21.15.50:0/1003866967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd660003a60 0x7fd660003ed0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:56.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.578+0000 7fd681c00640 1 -- 172.21.15.50:0/1003866967 >> 172.21.15.50:0/1003866967 conn(0x7fd67c0b2040 msgr2=0x7fd67c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:56.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.578+0000 7fd681c00640 1 -- 172.21.15.50:0/1003866967 shutdown_connections 2024-09-21T06:05:56.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.578+0000 7fd681c00640 1 -- 172.21.15.50:0/1003866967 wait complete. 2024-09-21T06:05:56.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.578+0000 7fd681c00640 1 Processor -- start 2024-09-21T06:05:56.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.578+0000 7fd681c00640 1 -- start start 2024-09-21T06:05:56.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.578+0000 7fd681c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd660003a60 0x7fd67c1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:56.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.578+0000 7fd681c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd67c14cc70 0x7fd67c1fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:56.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.578+0000 7fd681c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd67c1fc6e0 0x7fd67c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:56.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd681c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c15d180 con 0x7fd67c1fc6e0 2024-09-21T06:05:56.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd681c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c15d000 con 0x7fd660003a60 2024-09-21T06:05:56.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd681c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd67c15d300 con 0x7fd67c14cc70 2024-09-21T06:05:56.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd67b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd67c1fc6e0 0x7fd67c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:56.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd67b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd67c1fc6e0 0x7fd67c0cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46578/0 (socket says 172.21.15.50:46578) 2024-09-21T06:05:56.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd67b400640 1 -- 172.21.15.50:0/516551196 learned_addr learned my addr 172.21.15.50:0/516551196 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:56.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd673e00640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd67c14cc70 0x7fd67c1fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:56.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd67aa00640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd660003a60 0x7fd67c1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:56.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd67b400640 1 -- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd67c14cc70 msgr2=0x7fd67c1fc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:56.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd67b400640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd67c14cc70 0x7fd67c1fc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:56.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd67b400640 1 -- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd660003a60 msgr2=0x7fd67c1591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:56.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd67b400640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd660003a60 0x7fd67c1591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:56.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd67b400640 1 -- 172.21.15.50:0/516551196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c1fcde0 con 0x7fd67c1fc6e0 2024-09-21T06:05:56.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd67aa00640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd660003a60 0x7fd67c1591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:56.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd67b400640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd67c1fc6e0 0x7fd67c0cef10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd67c1596f0 tx=0x7fd67400f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd672a00640 1 -- 172.21.15.50:0/516551196 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674009c80 con 0x7fd67c1fc6e0 2024-09-21T06:05:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd672a00640 1 -- 172.21.15.50:0/516551196 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd674004d30 con 0x7fd67c1fc6e0 2024-09-21T06:05:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd672a00640 1 -- 172.21.15.50:0/516551196 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674003bc0 con 0x7fd67c1fc6e0 2024-09-21T06:05:56.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd681c00640 1 -- 172.21.15.50:0/516551196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c14d260 con 0x7fd67c1fc6e0 2024-09-21T06:05:56.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd681c00640 1 -- 172.21.15.50:0/516551196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c1573d0 con 0x7fd67c1fc6e0 2024-09-21T06:05:56.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd681c00640 1 -- 172.21.15.50:0/516551196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67c157630 con 0x7fd67c1fc6e0 2024-09-21T06:05:56.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.582+0000 7fd672a00640 1 -- 172.21.15.50:0/516551196 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fd674004030 con 0x7fd67c1fc6e0 2024-09-21T06:05:56.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.586+0000 7fd672a00640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd658074370 0x7fd658076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:56.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.586+0000 7fd672a00640 1 -- 172.21.15.50:0/516551196 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5471+0+0 (secure 0 0 0) 0x7fd674093b30 con 0x7fd67c1fc6e0 2024-09-21T06:05:56.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.586+0000 7fd67aa00640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd658074370 0x7fd658076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:56.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.586+0000 7fd67aa00640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd658074370 0x7fd658076880 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0ab000 tx=0x7fd66c019d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:56.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.590+0000 7fd672a00640 1 -- 172.21.15.50:0/516551196 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd67405fc90 con 0x7fd67c1fc6e0 2024-09-21T06:05:56.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:56.774+0000 7fd681c00640 1 -- 172.21.15.50:0/516551196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd67c1fcb40 con 0x7fd67c1fc6e0 2024-09-21T06:05:57.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.386+0000 7fd672a00640 1 -- 172.21.15.50:0/516551196 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) ==== 169+0+0 (secure 0 0 0) 0x7fd6740646c0 con 0x7fd67c1fc6e0 2024-09-21T06:05:57.389 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-09-21T06:05:57.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.394+0000 7fd681c00640 1 -- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd658074370 msgr2=0x7fd658076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:57.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.394+0000 7fd681c00640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd658074370 0x7fd658076880 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0ab000 tx=0x7fd66c019d30 comp rx=0 tx=0).stop 2024-09-21T06:05:57.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.394+0000 7fd681c00640 1 -- 172.21.15.50:0/516551196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd67c1fc6e0 msgr2=0x7fd67c0cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:57.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.394+0000 7fd681c00640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd67c1fc6e0 0x7fd67c0cef10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd67c1596f0 tx=0x7fd67400f9c0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.394+0000 7fd681c00640 1 -- 172.21.15.50:0/516551196 shutdown_connections 2024-09-21T06:05:57.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.394+0000 7fd681c00640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd658074370 0x7fd658076880 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.394+0000 7fd681c00640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd67c1fc6e0 0x7fd67c0cef10 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.394+0000 7fd681c00640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd67c14cc70 0x7fd67c1fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.394+0000 7fd681c00640 1 --2- 172.21.15.50:0/516551196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd660003a60 0x7fd67c1591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.394+0000 7fd681c00640 1 -- 172.21.15.50:0/516551196 >> 172.21.15.50:0/516551196 conn(0x7fd67c0b2040 msgr2=0x7fd67c0af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:57.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.394+0000 7fd681c00640 1 -- 172.21.15.50:0/516551196 shutdown_connections 2024-09-21T06:05:57.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.394+0000 7fd681c00640 1 -- 172.21.15.50:0/516551196 wait complete. 2024-09-21T06:05:57.422 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:05:57.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.586+0000 7fb0cf000640 1 -- 172.21.15.50:0/3027146854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0c8157030 msgr2=0x7fb0c80b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:57.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.586+0000 7fb0cf000640 1 --2- 172.21.15.50:0/3027146854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0c8157030 0x7fb0c80b5a80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009840 tx=0x7fb0b801c820 comp rx=0 tx=0).stop 2024-09-21T06:05:57.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.586+0000 7fb0cf000640 1 -- 172.21.15.50:0/3027146854 shutdown_connections 2024-09-21T06:05:57.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.586+0000 7fb0cf000640 1 --2- 172.21.15.50:0/3027146854 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0c8074b20 0x7fb0c815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.586+0000 7fb0cf000640 1 --2- 172.21.15.50:0/3027146854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0c8157030 0x7fb0c80b5a80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.586+0000 7fb0cf000640 1 -- 172.21.15.50:0/3027146854 >> 172.21.15.50:0/3027146854 conn(0x7fb0c80708c0 msgr2=0x7fb0c8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:57.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.586+0000 7fb0cf000640 1 -- 172.21.15.50:0/3027146854 shutdown_connections 2024-09-21T06:05:57.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.586+0000 7fb0cf000640 1 -- 172.21.15.50:0/3027146854 wait complete. 2024-09-21T06:05:57.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.586+0000 7fb0cf000640 1 Processor -- start 2024-09-21T06:05:57.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.586+0000 7fb0cf000640 1 -- start start 2024-09-21T06:05:57.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cf000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0c8157030 0x7fb0c814bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:57.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cf000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0c8160d50 0x7fb0c814b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:57.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cf000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c80d14c0 0x7fb0c80d18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:57.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cf000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c816a740 con 0x7fb0c80d14c0 2024-09-21T06:05:57.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cf000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c816a5c0 con 0x7fb0c8160d50 2024-09-21T06:05:57.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cf000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c816a8c0 con 0x7fb0c8157030 2024-09-21T06:05:57.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c80d14c0 0x7fb0c80d18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:57.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0c7e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0c8157030 0x7fb0c814bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:57.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c80d14c0 0x7fb0c80d18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40306/0 (socket says 172.21.15.50:40306) 2024-09-21T06:05:57.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cd000640 1 -- 172.21.15.50:0/3466621391 learned_addr learned my addr 172.21.15.50:0/3466621391 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:57.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0bfe00640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0c8160d50 0x7fb0c814b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:57.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cd000640 1 -- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0c8157030 msgr2=0x7fb0c814bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:57.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cd000640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0c8157030 0x7fb0c814bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cd000640 1 -- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0c8160d50 msgr2=0x7fb0c814b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:57.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cd000640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0c8160d50 0x7fb0c814b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cd000640 1 -- 172.21.15.50:0/3466621391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c80d1e10 con 0x7fb0c80d14c0 2024-09-21T06:05:57.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cd000640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c80d14c0 0x7fb0c80d18d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000ec70 tx=0x7fb0c000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:57.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0c4c00640 1 -- 172.21.15.50:0/3466621391 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0003b80 con 0x7fb0c80d14c0 2024-09-21T06:05:57.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cf000640 1 -- 172.21.15.50:0/3466621391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c814c450 con 0x7fb0c80d14c0 2024-09-21T06:05:57.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0c4c00640 1 -- 172.21.15.50:0/3466621391 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0c0003d20 con 0x7fb0c80d14c0 2024-09-21T06:05:57.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0c4c00640 1 -- 172.21.15.50:0/3466621391 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0004280 con 0x7fb0c80d14c0 2024-09-21T06:05:57.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cf000640 1 -- 172.21.15.50:0/3466621391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c814c960 con 0x7fb0c80d14c0 2024-09-21T06:05:57.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.590+0000 7fb0cf000640 1 -- 172.21.15.50:0/3466621391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c814c6d0 con 0x7fb0c80d14c0 2024-09-21T06:05:57.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.594+0000 7fb0c4c00640 1 -- 172.21.15.50:0/3466621391 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fb0c003c080 con 0x7fb0c80d14c0 2024-09-21T06:05:57.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.594+0000 7fb0c4c00640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0a40743a0 0x7fb0a40768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:57.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.594+0000 7fb0c4c00640 1 -- 172.21.15.50:0/3466621391 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7fb0c0093fd0 con 0x7fb0c80d14c0 2024-09-21T06:05:57.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.594+0000 7fb0c7e00640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0a40743a0 0x7fb0a40768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:57.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.594+0000 7fb0c7e00640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0a40743a0 0x7fb0a40768b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80af000 tx=0x7fb0b801b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:57.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.598+0000 7fb0c4c00640 1 -- 172.21.15.50:0/3466621391 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb0c80d1e10 con 0x7fb0c80d14c0 2024-09-21T06:05:57.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.786+0000 7fb0cf000640 1 -- 172.21.15.50:0/3466621391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fb0c815cf90 con 0x7fb0c80d14c0 2024-09-21T06:05:57.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0c4c00640 1 -- 172.21.15.50:0/3466621391 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v44) ==== 71+0+0 (secure 0 0 0) 0x7fb0c0060130 con 0x7fb0c80d14c0 2024-09-21T06:05:57.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0cf000640 1 -- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0a40743a0 msgr2=0x7fb0a40768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:57.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0cf000640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0a40743a0 0x7fb0a40768b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80af000 tx=0x7fb0b801b060 comp rx=0 tx=0).stop 2024-09-21T06:05:57.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0cf000640 1 -- 172.21.15.50:0/3466621391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c80d14c0 msgr2=0x7fb0c80d18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:57.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0cf000640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c80d14c0 0x7fb0c80d18d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000ec70 tx=0x7fb0c000c6a0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0cf000640 1 -- 172.21.15.50:0/3466621391 shutdown_connections 2024-09-21T06:05:57.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0cf000640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0a40743a0 0x7fb0a40768b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0cf000640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c80d14c0 0x7fb0c80d18d0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0cf000640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0c8160d50 0x7fb0c814b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0cf000640 1 --2- 172.21.15.50:0/3466621391 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0c8157030 0x7fb0c814bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0cf000640 1 -- 172.21.15.50:0/3466621391 >> 172.21.15.50:0/3466621391 conn(0x7fb0c80708c0 msgr2=0x7fb0c815f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:57.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0cf000640 1 -- 172.21.15.50:0/3466621391 shutdown_connections 2024-09-21T06:05:57.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.790+0000 7fb0cf000640 1 -- 172.21.15.50:0/3466621391 wait complete. 2024-09-21T06:05:57.814 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-21T06:05:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.970+0000 7f4e43e00640 1 -- 172.21.15.50:0/858752360 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4e3c1527b0 msgr2=0x7f4e3c15bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:57.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.970+0000 7f4e43e00640 1 --2- 172.21.15.50:0/858752360 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4e3c1527b0 0x7f4e3c15bb90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009840 tx=0x7f4e2801c7f0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 -- 172.21.15.50:0/858752360 shutdown_connections 2024-09-21T06:05:57.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 --2- 172.21.15.50:0/858752360 >> v2:172.21.15.88:3300/0 conn(0x7f4e3c0b4de0 0x7f4e3c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 --2- 172.21.15.50:0/858752360 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4e3c1527b0 0x7f4e3c15bb90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 -- 172.21.15.50:0/858752360 >> 172.21.15.50:0/858752360 conn(0x7f4e3c0b2040 msgr2=0x7f4e3c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:57.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 -- 172.21.15.50:0/858752360 shutdown_connections 2024-09-21T06:05:57.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 -- 172.21.15.50:0/858752360 wait complete. 2024-09-21T06:05:57.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 Processor -- start 2024-09-21T06:05:57.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 -- start start 2024-09-21T06:05:57.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c0b4de0 0x7f4e3c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:57.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4e3c0b5c70 0x7f4e3c1f7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:57.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e3c1527b0 0x7f4e3c1f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:57.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1689a0 con 0x7f4e3c0b4de0 2024-09-21T06:05:57.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c168820 con 0x7f4e3c1527b0 2024-09-21T06:05:57.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.974+0000 7f4e43e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3c168b20 con 0x7f4e3c0b5c70 2024-09-21T06:05:57.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e41400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c0b4de0 0x7f4e3c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:57.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e41400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c0b4de0 0x7f4e3c158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40312/0 (socket says 172.21.15.50:40312) 2024-09-21T06:05:57.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e41400640 1 -- 172.21.15.50:0/4135223233 learned_addr learned my addr 172.21.15.50:0/4135223233 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:57.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e41e00640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e3c1527b0 0x7f4e3c1f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:57.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e40a00640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4e3c0b5c70 0x7f4e3c1f7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:57.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e41400640 1 -- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4e3c0b5c70 msgr2=0x7f4e3c1f7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:57.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e41400640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4e3c0b5c70 0x7f4e3c1f7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e41400640 1 -- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e3c1527b0 msgr2=0x7f4e3c1f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:57.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e41400640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e3c1527b0 0x7f4e3c1f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:57.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e41400640 1 -- 172.21.15.50:0/4135223233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c153350 con 0x7f4e3c0b4de0 2024-09-21T06:05:57.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e41e00640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e3c1527b0 0x7f4e3c1f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:05:57.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e41400640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c0b4de0 0x7f4e3c158e00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000c760 tx=0x7f4e3000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:57.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e39600640 1 -- 172.21.15.50:0/4135223233 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e300052d0 con 0x7f4e3c0b4de0 2024-09-21T06:05:57.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e39600640 1 -- 172.21.15.50:0/4135223233 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e30001d20 con 0x7f4e3c0b4de0 2024-09-21T06:05:57.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e39600640 1 -- 172.21.15.50:0/4135223233 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3000a880 con 0x7f4e3c0b4de0 2024-09-21T06:05:57.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e43e00640 1 -- 172.21.15.50:0/4135223233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3c153630 con 0x7f4e3c0b4de0 2024-09-21T06:05:57.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e43e00640 1 -- 172.21.15.50:0/4135223233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3c153bf0 con 0x7f4e3c0b4de0 2024-09-21T06:05:57.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e43e00640 1 -- 172.21.15.50:0/4135223233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3c0b51f0 con 0x7f4e3c0b4de0 2024-09-21T06:05:57.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.978+0000 7f4e39600640 1 -- 172.21.15.50:0/4135223233 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f4e3000aa20 con 0x7f4e3c0b4de0 2024-09-21T06:05:57.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.982+0000 7f4e39600640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4e1c0743a0 0x7f4e1c0768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:57.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.982+0000 7f4e39600640 1 -- 172.21.15.50:0/4135223233 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7f4e30092b00 con 0x7f4e3c0b4de0 2024-09-21T06:05:57.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.982+0000 7f4e40a00640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4e1c0743a0 0x7f4e1c0768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:57.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.982+0000 7f4e40a00640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4e1c0743a0 0x7f4e1c0768b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4e280af000 tx=0x7f4e2801b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:57.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:57.986+0000 7f4e39600640 1 -- 172.21.15.50:0/4135223233 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4e3c0b5730 con 0x7f4e3c0b4de0 2024-09-21T06:05:58.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.230+0000 7f4e43e00640 1 -- 172.21.15.50:0/4135223233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7f4e3c14d050 con 0x7f4e3c0b4de0 2024-09-21T06:05:58.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.390+0000 7f4e39600640 1 -- 172.21.15.50:0/4135223233 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v417) ==== 167+0+0 (secure 0 0 0) 0x7f4e3005ebd0 con 0x7f4e3c0b4de0 2024-09-21T06:05:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.394+0000 7f4e43e00640 1 -- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4e1c0743a0 msgr2=0x7f4e1c0768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.394+0000 7f4e43e00640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4e1c0743a0 0x7f4e1c0768b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4e280af000 tx=0x7f4e2801b030 comp rx=0 tx=0).stop 2024-09-21T06:05:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.394+0000 7f4e43e00640 1 -- 172.21.15.50:0/4135223233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c0b4de0 msgr2=0x7f4e3c158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.394+0000 7f4e43e00640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c0b4de0 0x7f4e3c158e00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000c760 tx=0x7f4e3000cc20 comp rx=0 tx=0).stop 2024-09-21T06:05:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.394+0000 7f4e43e00640 1 -- 172.21.15.50:0/4135223233 shutdown_connections 2024-09-21T06:05:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.394+0000 7f4e43e00640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e3c1527b0 0x7f4e3c1f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.394+0000 7f4e43e00640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4e3c0b5c70 0x7f4e3c1f7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.394+0000 7f4e43e00640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4e1c0743a0 0x7f4e1c0768b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.394+0000 7f4e43e00640 1 --2- 172.21.15.50:0/4135223233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c0b4de0 0x7f4e3c158e00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.394+0000 7f4e43e00640 1 -- 172.21.15.50:0/4135223233 >> 172.21.15.50:0/4135223233 conn(0x7f4e3c0b2040 msgr2=0x7f4e3c14d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.394+0000 7f4e43e00640 1 -- 172.21.15.50:0/4135223233 shutdown_connections 2024-09-21T06:05:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.394+0000 7f4e43e00640 1 -- 172.21.15.50:0/4135223233 wait complete. 2024-09-21T06:05:58.415 INFO:tasks.cephfs_test_runner:Test that 2 progress bars are printed in output of "ceph status" ... ok 2024-09-21T06:05:58.415 INFO:tasks.cephfs_test_runner:Starting test: test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress indicated by progress bar for ongoing+pending clones 2024-09-21T06:05:58.415 INFO:tasks.cephfs_test_runner:test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-21T06:05:58.415 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:05:58.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.586+0000 7f8012e00640 1 -- 172.21.15.50:0/1121067864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f800c071940 msgr2=0x7f800c068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:58.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.586+0000 7f8012e00640 1 --2- 172.21.15.50:0/1121067864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f800c071940 0x7f800c068df0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009840 tx=0x7f7ffc01c760 comp rx=0 tx=0).stop 2024-09-21T06:05:58.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.586+0000 7f8012e00640 1 -- 172.21.15.50:0/1121067864 shutdown_connections 2024-09-21T06:05:58.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.586+0000 7f8012e00640 1 --2- 172.21.15.50:0/1121067864 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f800c069330 0x7f800c0cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.586+0000 7f8012e00640 1 --2- 172.21.15.50:0/1121067864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f800c071940 0x7f800c068df0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.586+0000 7f8012e00640 1 -- 172.21.15.50:0/1121067864 >> 172.21.15.50:0/1121067864 conn(0x7f800c0b35f0 msgr2=0x7f800c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:58.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.586+0000 7f8012e00640 1 -- 172.21.15.50:0/1121067864 shutdown_connections 2024-09-21T06:05:58.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.586+0000 7f8012e00640 1 -- 172.21.15.50:0/1121067864 wait complete. 2024-09-21T06:05:58.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f8012e00640 1 Processor -- start 2024-09-21T06:05:58.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f8012e00640 1 -- start start 2024-09-21T06:05:58.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f8012e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f800c069330 0x7f800c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:58.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f8012e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f800c071940 0x7f800c1fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f8012e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f800c0cfcd0 0x7f800c165720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f8012e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f800c16a210 con 0x7f800c0cfcd0 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f8012e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f800c16a090 con 0x7f800c071940 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f8012e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f800c16a390 con 0x7f800c069330 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f8010e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f800c0cfcd0 0x7f800c165720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f800be00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f800c069330 0x7f800c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f8010e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f800c0cfcd0 0x7f800c165720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40324/0 (socket says 172.21.15.50:40324) 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f8010e00640 1 -- 172.21.15.50:0/2416930093 learned_addr learned my addr 172.21.15.50:0/2416930093 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f800b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f800c071940 0x7f800c1fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f800be00640 1 -- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f800c071940 msgr2=0x7f800c1fcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f800be00640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f800c071940 0x7f800c1fcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f800be00640 1 -- 172.21.15.50:0/2416930093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f800c0cfcd0 msgr2=0x7f800c165720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f800be00640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f800c0cfcd0 0x7f800c165720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f800be00640 1 -- 172.21.15.50:0/2416930093 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c0b67e0 con 0x7f800c069330 2024-09-21T06:05:58.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f8010e00640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f800c0cfcd0 0x7f800c165720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:05:58.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.590+0000 7f800be00640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f800c069330 0x7f800c15b380 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0ac000 tx=0x7f7ffc0a56a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:58.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.594+0000 7f8003e00640 1 -- 172.21.15.50:0/2416930093 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0b2040 con 0x7f800c069330 2024-09-21T06:05:58.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.594+0000 7f8003e00640 1 -- 172.21.15.50:0/2416930093 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ffc004470 con 0x7f800c069330 2024-09-21T06:05:58.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.594+0000 7f8003e00640 1 -- 172.21.15.50:0/2416930093 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc005280 con 0x7f800c069330 2024-09-21T06:05:58.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.594+0000 7f8012e00640 1 -- 172.21.15.50:0/2416930093 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c0b6220 con 0x7f800c069330 2024-09-21T06:05:58.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.594+0000 7f8012e00640 1 -- 172.21.15.50:0/2416930093 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c1fc5a0 con 0x7f800c069330 2024-09-21T06:05:58.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.594+0000 7f8012e00640 1 -- 172.21.15.50:0/2416930093 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800c0cf790 con 0x7f800c069330 2024-09-21T06:05:58.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.594+0000 7f8003e00640 1 -- 172.21.15.50:0/2416930093 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f7ffc005420 con 0x7f800c069330 2024-09-21T06:05:58.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.594+0000 7f8003e00640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7fec074370 0x7f7fec076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:58.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.594+0000 7f8003e00640 1 -- 172.21.15.50:0/2416930093 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7f7ffc12caa0 con 0x7f800c069330 2024-09-21T06:05:58.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.594+0000 7f800b400640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7fec074370 0x7f7fec076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:58.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.598+0000 7f800b400640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7fec074370 0x7f7fec076880 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f800c0b3ed0 tx=0x7f7ff8009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:58.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.598+0000 7f8003e00640 1 -- 172.21.15.50:0/2416930093 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7ffc0bc060 con 0x7f800c069330 2024-09-21T06:05:58.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.778+0000 7f8012e00640 1 -- 172.21.15.50:0/2416930093 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f800c0d04f0 con 0x7f800c069330 2024-09-21T06:05:58.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.778+0000 7f8003e00640 1 -- 172.21.15.50:0/2416930093 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+13073 (secure 0 0 0) 0x7f7ffc016410 con 0x7f800c069330 2024-09-21T06:05:58.779 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:05:58.779 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":77,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:05:57.386698+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6837/2131576448":"2024-09-22T06:05:54.354144+0000","172.21.15.88:6835/2131576448":"2024-09-22T06:05:54.354144+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:05:58.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.782+0000 7f8012e00640 1 -- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7fec074370 msgr2=0x7f7fec076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:58.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.782+0000 7f8012e00640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7fec074370 0x7f7fec076880 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f800c0b3ed0 tx=0x7f7ff8009260 comp rx=0 tx=0).stop 2024-09-21T06:05:58.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.782+0000 7f8012e00640 1 -- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f800c069330 msgr2=0x7f800c15b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:58.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.782+0000 7f8012e00640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f800c069330 0x7f800c15b380 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0ac000 tx=0x7f7ffc0a56a0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.782+0000 7f8012e00640 1 -- 172.21.15.50:0/2416930093 shutdown_connections 2024-09-21T06:05:58.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.782+0000 7f8012e00640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7fec074370 0x7f7fec076880 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.782+0000 7f8012e00640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f800c0cfcd0 0x7f800c165720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.782+0000 7f8012e00640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f800c071940 0x7f800c1fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.782+0000 7f8012e00640 1 --2- 172.21.15.50:0/2416930093 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f800c069330 0x7f800c15b380 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.782+0000 7f8012e00640 1 -- 172.21.15.50:0/2416930093 >> 172.21.15.50:0/2416930093 conn(0x7f800c0b35f0 msgr2=0x7f800c15dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:58.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.782+0000 7f8012e00640 1 -- 172.21.15.50:0/2416930093 shutdown_connections 2024-09-21T06:05:58.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.782+0000 7f8012e00640 1 -- 172.21.15.50:0/2416930093 wait complete. 2024-09-21T06:05:58.802 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:56:18.921197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:05:58.803 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:05:58.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.966+0000 7f0c64c00640 1 -- 172.21.15.50:0/2014458443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c440039f0 msgr2=0x7f0c44003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:58.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.966+0000 7f0c64c00640 1 --2- 172.21.15.50:0/2014458443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c440039f0 0x7f0c44003e20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009b60 tx=0x7f0c5001d260 comp rx=0 tx=0).stop 2024-09-21T06:05:58.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.966+0000 7f0c5e800640 1 -- 172.21.15.50:0/2014458443 reap_dead start 2024-09-21T06:05:58.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.966+0000 7f0c64c00640 1 -- 172.21.15.50:0/2014458443 shutdown_connections 2024-09-21T06:05:58.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.966+0000 7f0c64c00640 1 -- 172.21.15.50:0/2014458443 >> 172.21.15.50:0/2014458443 conn(0x7f0c600b2040 msgr2=0x7f0c600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:58.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.966+0000 7f0c64c00640 1 -- 172.21.15.50:0/2014458443 shutdown_connections 2024-09-21T06:05:58.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.966+0000 7f0c64c00640 1 -- 172.21.15.50:0/2014458443 wait complete. 2024-09-21T06:05:58.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.966+0000 7f0c64c00640 1 Processor -- start 2024-09-21T06:05:58.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.966+0000 7f0c64c00640 1 -- start start 2024-09-21T06:05:58.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c64c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c440039f0 0x7f0c60140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:58.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c64c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60158d40 0x7f0c60159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:58.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c64c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60151650 0x7f0c60151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:58.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c64c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c60167bc0 con 0x7f0c60158d40 2024-09-21T06:05:58.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c64c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0c60167a40 con 0x7f0c60151650 2024-09-21T06:05:58.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c64c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0c60167d40 con 0x7f0c440039f0 2024-09-21T06:05:58.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c57e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60158d40 0x7f0c60159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:58.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c57e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60158d40 0x7f0c60159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40342/0 (socket says 172.21.15.50:40342) 2024-09-21T06:05:58.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c57e00640 1 -- 172.21.15.50:0/2873092711 learned_addr learned my addr 172.21.15.50:0/2873092711 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:58.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c5de00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c440039f0 0x7f0c60140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:58.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c5e800640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60151650 0x7f0c60151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:58.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c57e00640 1 -- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c440039f0 msgr2=0x7f0c60140ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:58.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c57e00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c440039f0 0x7f0c60140ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c57e00640 1 -- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60151650 msgr2=0x7f0c60151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:58.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c57e00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60151650 0x7f0c60151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:58.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c57e00640 1 -- 172.21.15.50:0/2873092711 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c60151fa0 con 0x7f0c60158d40 2024-09-21T06:05:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c5de00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c440039f0 0x7f0c60140ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c5e800640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60151650 0x7f0c60151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c57e00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60158d40 0x7f0c60159150 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0c48004dd0 tx=0x7f0c48001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c56000640 1 -- 172.21.15.50:0/2873092711 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48010890 con 0x7f0c60158d40 2024-09-21T06:05:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c56000640 1 -- 172.21.15.50:0/2873092711 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c48016070 con 0x7f0c60158d40 2024-09-21T06:05:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c64c00640 1 -- 172.21.15.50:0/2873092711 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c60156630 con 0x7f0c60158d40 2024-09-21T06:05:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c56000640 1 -- 172.21.15.50:0/2873092711 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48010da0 con 0x7f0c60158d40 2024-09-21T06:05:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c64c00640 1 -- 172.21.15.50:0/2873092711 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c60156bf0 con 0x7f0c60158d40 2024-09-21T06:05:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.970+0000 7f0c64c00640 1 -- 172.21.15.50:0/2873092711 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c600b51f0 con 0x7f0c60158d40 2024-09-21T06:05:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.974+0000 7f0c56000640 1 -- 172.21.15.50:0/2873092711 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f0c4801b460 con 0x7f0c60158d40 2024-09-21T06:05:58.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.974+0000 7f0c56000640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c3c0743a0 0x7f0c3c0768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:58.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.974+0000 7f0c56000640 1 -- 172.21.15.50:0/2873092711 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7f0c48094560 con 0x7f0c60158d40 2024-09-21T06:05:58.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.974+0000 7f0c5de00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c3c0743a0 0x7f0c3c0768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:58.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.974+0000 7f0c5de00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c3c0743a0 0x7f0c3c0768b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0c500ae000 tx=0x7f0c5001a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:58.978+0000 7f0c56000640 1 -- 172.21.15.50:0/2873092711 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0c60151fa0 con 0x7f0c60158d40 2024-09-21T06:05:59.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.154+0000 7f0c64c00640 1 -- 172.21.15.50:0/2873092711 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0c600b2660 con 0x7f0c60158d40 2024-09-21T06:05:59.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.154+0000 7f0c56000640 1 -- 172.21.15.50:0/2873092711 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f0c48060d10 con 0x7f0c60158d40 2024-09-21T06:05:59.156 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-09-21T06:05:59.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.158+0000 7f0c64c00640 1 -- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c3c0743a0 msgr2=0x7f0c3c0768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:59.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.158+0000 7f0c64c00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c3c0743a0 0x7f0c3c0768b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0c500ae000 tx=0x7f0c5001a000 comp rx=0 tx=0).stop 2024-09-21T06:05:59.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.158+0000 7f0c64c00640 1 -- 172.21.15.50:0/2873092711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60158d40 msgr2=0x7f0c60159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.158+0000 7f0c64c00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60158d40 0x7f0c60159150 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0c48004dd0 tx=0x7f0c48001720 comp rx=0 tx=0).stop 2024-09-21T06:05:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.158+0000 7f0c64c00640 1 -- 172.21.15.50:0/2873092711 shutdown_connections 2024-09-21T06:05:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.158+0000 7f0c64c00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60151650 0x7f0c60151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.158+0000 7f0c64c00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c3c0743a0 0x7f0c3c0768b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.158+0000 7f0c64c00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60158d40 0x7f0c60159150 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.158+0000 7f0c64c00640 1 --2- 172.21.15.50:0/2873092711 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c440039f0 0x7f0c60140ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.158+0000 7f0c64c00640 1 -- 172.21.15.50:0/2873092711 >> 172.21.15.50:0/2873092711 conn(0x7f0c600b2040 msgr2=0x7f0c60154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.158+0000 7f0c64c00640 1 -- 172.21.15.50:0/2873092711 shutdown_connections 2024-09-21T06:05:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.158+0000 7f0c64c00640 1 -- 172.21.15.50:0/2873092711 wait complete. 2024-09-21T06:05:59.179 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-21T06:05:59.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.338+0000 7f54a6400640 1 -- 172.21.15.50:0/767440022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a00749a0 msgr2=0x7f54a0074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:59.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.338+0000 7f54a6400640 1 --2- 172.21.15.50:0/767440022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a00749a0 0x7f54a0074db0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5490009840 tx=0x7f549001c740 comp rx=0 tx=0).stop 2024-09-21T06:05:59.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.338+0000 7f54a6400640 1 -- 172.21.15.50:0/767440022 shutdown_connections 2024-09-21T06:05:59.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.338+0000 7f54a6400640 1 --2- 172.21.15.50:0/767440022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a00749a0 0x7f54a0074db0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:59.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.338+0000 7f54a6400640 1 -- 172.21.15.50:0/767440022 >> 172.21.15.50:0/767440022 conn(0x7f54a00b2040 msgr2=0x7f54a00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:59.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.338+0000 7f54a6400640 1 -- 172.21.15.50:0/767440022 shutdown_connections 2024-09-21T06:05:59.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.338+0000 7f54a6400640 1 -- 172.21.15.50:0/767440022 wait complete. 2024-09-21T06:05:59.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f54a6400640 1 Processor -- start 2024-09-21T06:05:59.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f54a6400640 1 -- start start 2024-09-21T06:05:59.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f54a6400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a00749a0 0x7f54a015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:59.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f54a6400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54a00b3340 0x7f54a00ce810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:59.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f54a6400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a00cefa0 0x7f54a015b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:59.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f54a6400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f54a016b2e0 con 0x7f54a00cefa0 2024-09-21T06:05:59.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f54a6400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f54a016b160 con 0x7f54a00749a0 2024-09-21T06:05:59.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f54a6400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f54a016b460 con 0x7f54a00b3340 2024-09-21T06:05:59.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f549fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a00cefa0 0x7f54a015b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:59.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f549fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a00cefa0 0x7f54a015b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40344/0 (socket says 172.21.15.50:40344) 2024-09-21T06:05:59.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f549fe00640 1 -- 172.21.15.50:0/2862741874 learned_addr learned my addr 172.21.15.50:0/2862741874 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:05:59.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f549f400640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a00749a0 0x7f54a015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:59.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f5497e00640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54a00b3340 0x7f54a00ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f549fe00640 1 -- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54a00b3340 msgr2=0x7f54a00ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f549fe00640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54a00b3340 0x7f54a00ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f549fe00640 1 -- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a00749a0 msgr2=0x7f54a015dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f549fe00640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a00749a0 0x7f54a015dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f549fe00640 1 -- 172.21.15.50:0/2862741874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a013aa10 con 0x7f54a00cefa0 2024-09-21T06:05:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f549fe00640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a00cefa0 0x7f54a015b620 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f54980015b0 tx=0x7f549800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f5497400640 1 -- 172.21.15.50:0/2862741874 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498016020 con 0x7f54a00cefa0 2024-09-21T06:05:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f5497400640 1 -- 172.21.15.50:0/2862741874 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f549800ab30 con 0x7f54a00cefa0 2024-09-21T06:05:59.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f54a6400640 1 -- 172.21.15.50:0/2862741874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a013acf0 con 0x7f54a00cefa0 2024-09-21T06:05:59.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f5497400640 1 -- 172.21.15.50:0/2862741874 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498012420 con 0x7f54a00cefa0 2024-09-21T06:05:59.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.342+0000 7f54a6400640 1 -- 172.21.15.50:0/2862741874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a013b800 con 0x7f54a00cefa0 2024-09-21T06:05:59.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.346+0000 7f54a6400640 1 -- 172.21.15.50:0/2862741874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a0074db0 con 0x7f54a00cefa0 2024-09-21T06:05:59.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.346+0000 7f5497400640 1 -- 172.21.15.50:0/2862741874 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f5498019020 con 0x7f54a00cefa0 2024-09-21T06:05:59.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.346+0000 7f5497400640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f547c074370 0x7f547c076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:59.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.346+0000 7f5497400640 1 -- 172.21.15.50:0/2862741874 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7f5498093320 con 0x7f54a00cefa0 2024-09-21T06:05:59.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.346+0000 7f549f400640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f547c074370 0x7f547c076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:59.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.346+0000 7f549f400640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f547c074370 0x7f547c076880 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f54900af000 tx=0x7f54900b2040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:59.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.350+0000 7f5497400640 1 -- 172.21.15.50:0/2862741874 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5498020020 con 0x7f54a00cefa0 2024-09-21T06:05:59.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:05:59.594+0000 7f54a6400640 1 -- 172.21.15.50:0/2862741874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f54a00b5d70 con 0x7f54a00cefa0 2024-09-21T06:06:00.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.398+0000 7f5497400640 1 -- 172.21.15.50:0/2862741874 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v419) ==== 177+0+0 (secure 0 0 0) 0x7f549805f400 con 0x7f54a00cefa0 2024-09-21T06:06:00.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.402+0000 7f54a6400640 1 -- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f547c074370 msgr2=0x7f547c076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:00.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.402+0000 7f54a6400640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f547c074370 0x7f547c076880 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f54900af000 tx=0x7f54900b2040 comp rx=0 tx=0).stop 2024-09-21T06:06:00.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.402+0000 7f54a6400640 1 -- 172.21.15.50:0/2862741874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a00cefa0 msgr2=0x7f54a015b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:00.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.402+0000 7f54a6400640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a00cefa0 0x7f54a015b620 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f54980015b0 tx=0x7f549800cc80 comp rx=0 tx=0).stop 2024-09-21T06:06:00.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.402+0000 7f54a6400640 1 -- 172.21.15.50:0/2862741874 shutdown_connections 2024-09-21T06:06:00.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.402+0000 7f54a6400640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f547c074370 0x7f547c076880 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.402+0000 7f54a6400640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54a00cefa0 0x7f54a015b620 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.402+0000 7f54a6400640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f54a00b3340 0x7f54a00ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.402+0000 7f54a6400640 1 --2- 172.21.15.50:0/2862741874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a00749a0 0x7f54a015dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.402+0000 7f54a6400640 1 -- 172.21.15.50:0/2862741874 >> 172.21.15.50:0/2862741874 conn(0x7f54a00b2040 msgr2=0x7f54a00cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.402+0000 7f54a6400640 1 -- 172.21.15.50:0/2862741874 shutdown_connections 2024-09-21T06:06:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.402+0000 7f54a6400640 1 -- 172.21.15.50:0/2862741874 wait complete. 2024-09-21T06:06:00.423 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:06:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.582+0000 7f57abe00640 1 -- 172.21.15.50:0/3674825265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a41527b0 msgr2=0x7f57a415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.582+0000 7f57abe00640 1 --2- 172.21.15.50:0/3674825265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a41527b0 0x7f57a415bb60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5790009840 tx=0x7f579001c7f0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.582+0000 7f57abe00640 1 -- 172.21.15.50:0/3674825265 shutdown_connections 2024-09-21T06:06:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.582+0000 7f57abe00640 1 --2- 172.21.15.50:0/3674825265 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57a40b4de0 0x7f57a40b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.582+0000 7f57abe00640 1 --2- 172.21.15.50:0/3674825265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a41527b0 0x7f57a415bb60 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.582+0000 7f57abe00640 1 -- 172.21.15.50:0/3674825265 >> 172.21.15.50:0/3674825265 conn(0x7f57a40b2040 msgr2=0x7f57a40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.582+0000 7f57abe00640 1 -- 172.21.15.50:0/3674825265 shutdown_connections 2024-09-21T06:06:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.582+0000 7f57abe00640 1 -- 172.21.15.50:0/3674825265 wait complete. 2024-09-21T06:06:00.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.582+0000 7f57abe00640 1 Processor -- start 2024-09-21T06:06:00.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.582+0000 7f57abe00640 1 -- start start 2024-09-21T06:06:00.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57abe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57a40b4de0 0x7f57a40ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:00.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57abe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a40b5c70 0x7f57a4151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:00.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57abe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57a41527b0 0x7f57a40ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57abe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4168950 con 0x7f57a40b5c70 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57abe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41687d0 con 0x7f57a40b4de0 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57abe00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f57a4168ad0 con 0x7f57a41527b0 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a8a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a40b5c70 0x7f57a4151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a8a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a40b5c70 0x7f57a4151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40372/0 (socket says 172.21.15.50:40372) 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a8a00640 1 -- 172.21.15.50:0/651781681 learned_addr learned my addr 172.21.15.50:0/651781681 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a9400640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57a40b4de0 0x7f57a40ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a9e00640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57a41527b0 0x7f57a40ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a8a00640 1 -- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57a41527b0 msgr2=0x7f57a40ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a8a00640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57a41527b0 0x7f57a40ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a8a00640 1 -- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57a40b4de0 msgr2=0x7f57a40ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a8a00640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57a40b4de0 0x7f57a40ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a8a00640 1 -- 172.21.15.50:0/651781681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a41f8fc0 con 0x7f57a40b5c70 2024-09-21T06:06:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a8a00640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a40b5c70 0x7f57a4151c80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5790000c10 tx=0x7f5790005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a1600640 1 -- 172.21.15.50:0/651781681 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57900b2040 con 0x7f57a40b5c70 2024-09-21T06:06:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57abe00640 1 -- 172.21.15.50:0/651781681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a40ce740 con 0x7f57a40b5c70 2024-09-21T06:06:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a1600640 1 -- 172.21.15.50:0/651781681 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5790004330 con 0x7f57a40b5c70 2024-09-21T06:06:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57a1600640 1 -- 172.21.15.50:0/651781681 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790003800 con 0x7f57a40b5c70 2024-09-21T06:06:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57abe00640 1 -- 172.21.15.50:0/651781681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a40c5350 con 0x7f57a40b5c70 2024-09-21T06:06:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.586+0000 7f57abe00640 1 -- 172.21.15.50:0/651781681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a40b51f0 con 0x7f57a40b5c70 2024-09-21T06:06:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.590+0000 7f57a1600640 1 -- 172.21.15.50:0/651781681 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f57900bd060 con 0x7f57a40b5c70 2024-09-21T06:06:00.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.590+0000 7f57a1600640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5784074370 0x7f5784076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:00.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.590+0000 7f57a1600640 1 -- 172.21.15.50:0/651781681 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7f579012ca90 con 0x7f57a40b5c70 2024-09-21T06:06:00.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.590+0000 7f57a9400640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5784074370 0x7f5784076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:00.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.590+0000 7f57a9400640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5784074370 0x7f5784076880 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5798001400 tx=0x7f5798009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.594+0000 7f57a1600640 1 -- 172.21.15.50:0/651781681 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5790016410 con 0x7f57a40b5c70 2024-09-21T06:06:00.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 -- 172.21.15.50:0/651781681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f57a40b42f0 con 0x7f57a40b5c70 2024-09-21T06:06:00.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57a1600640 1 -- 172.21.15.50:0/651781681 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v44) ==== 59+0+2294 (secure 0 0 0) 0x7f57900f9130 con 0x7f57a40b5c70 2024-09-21T06:06:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:06:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:06:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:06:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_client 20/20 2024-09-21T06:06:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:06:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:06:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:06:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:06:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3735.149299 2024-09-21T06:06:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4471.400731 2024-09-21T06:06:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3640.131623 2024-09-21T06:06:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3663.645087 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3762.055672 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3661.558528 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3661.456695 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3796.187852 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds 20/20 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_ms 2/2 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_frag true 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_verify_scatter true 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:06:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced client_mount_timeout 600 2024-09-21T06:06:00.777 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_client 20/20 2024-09-21T06:06:00.777 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_ms 1/1 2024-09-21T06:06:00.777 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:06:00.777 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:06:00.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 -- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5784074370 msgr2=0x7f5784076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:00.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5784074370 0x7f5784076880 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5798001400 tx=0x7f5798009040 comp rx=0 tx=0).stop 2024-09-21T06:06:00.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 -- 172.21.15.50:0/651781681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a40b5c70 msgr2=0x7f57a4151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:00.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a40b5c70 0x7f57a4151c80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5790000c10 tx=0x7f5790005db0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 -- 172.21.15.50:0/651781681 shutdown_connections 2024-09-21T06:06:00.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57a41527b0 0x7f57a40ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5784074370 0x7f5784076880 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57a40b5c70 0x7f57a4151c80 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 --2- 172.21.15.50:0/651781681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57a40b4de0 0x7f57a40ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 -- 172.21.15.50:0/651781681 >> 172.21.15.50:0/651781681 conn(0x7f57a40b2040 msgr2=0x7f57a40b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:00.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 -- 172.21.15.50:0/651781681 shutdown_connections 2024-09-21T06:06:00.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.774+0000 7f57abe00640 1 -- 172.21.15.50:0/651781681 wait complete. 2024-09-21T06:06:00.798 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:06:00.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.958+0000 7f8d90a00640 1 -- 172.21.15.50:0/2485505420 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d8c0b4de0 msgr2=0x7f8d8c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:00.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.958+0000 7f8d90a00640 1 --2- 172.21.15.50:0/2485505420 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d8c0b4de0 0x7f8d8c0b51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009bc0 tx=0x7f8d8401cb70 comp rx=0 tx=0).stop 2024-09-21T06:06:00.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.958+0000 7f8d90a00640 1 -- 172.21.15.50:0/2485505420 shutdown_connections 2024-09-21T06:06:00.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.958+0000 7f8d90a00640 1 --2- 172.21.15.50:0/2485505420 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d8c0b4de0 0x7f8d8c0b51f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.958+0000 7f8d90a00640 1 --2- 172.21.15.50:0/2485505420 >> v2:172.21.15.88:3300/0 conn(0x7f8d8c1527b0 0x7f8d8c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.958+0000 7f8d90a00640 1 -- 172.21.15.50:0/2485505420 >> 172.21.15.50:0/2485505420 conn(0x7f8d8c0b2040 msgr2=0x7f8d8c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:00.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.958+0000 7f8d90a00640 1 -- 172.21.15.50:0/2485505420 shutdown_connections 2024-09-21T06:06:00.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.958+0000 7f8d90a00640 1 -- 172.21.15.50:0/2485505420 wait complete. 2024-09-21T06:06:00.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d90a00640 1 Processor -- start 2024-09-21T06:06:00.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d90a00640 1 -- start start 2024-09-21T06:06:00.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d90a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d8c0b4de0 0x7f8d8c14b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:00.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d90a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d8c0b5c70 0x7f8d8c14ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:00.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d90a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d8c1527b0 0x7f8d8c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:00.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d90a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8c1689d0 con 0x7f8d8c0b4de0 2024-09-21T06:06:00.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d90a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8c168850 con 0x7f8d8c0b5c70 2024-09-21T06:06:00.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d90a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8d8c168b50 con 0x7f8d8c1527b0 2024-09-21T06:06:00.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d89e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d8c0b4de0 0x7f8d8c14b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:00.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d89e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d8c0b4de0 0x7f8d8c14b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40376/0 (socket says 172.21.15.50:40376) 2024-09-21T06:06:00.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d89e00640 1 -- 172.21.15.50:0/981405849 learned_addr learned my addr 172.21.15.50:0/981405849 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:00.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d81400640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d8c0b5c70 0x7f8d8c14ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:00.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d8a800640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d8c1527b0 0x7f8d8c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:00.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d89e00640 1 -- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d8c1527b0 msgr2=0x7f8d8c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:00.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d89e00640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d8c1527b0 0x7f8d8c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d89e00640 1 -- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d8c0b5c70 msgr2=0x7f8d8c14ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:00.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d89e00640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d8c0b5c70 0x7f8d8c14ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:00.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d89e00640 1 -- 172.21.15.50:0/981405849 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8c14edd0 con 0x7f8d8c0b4de0 2024-09-21T06:06:00.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d89e00640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d8c0b4de0 0x7f8d8c14b860 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00c3c0 tx=0x7f8d7c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:00.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d82a00640 1 -- 172.21.15.50:0/981405849 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c017070 con 0x7f8d8c0b4de0 2024-09-21T06:06:00.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d82a00640 1 -- 172.21.15.50:0/981405849 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d7c0053c0 con 0x7f8d8c0b4de0 2024-09-21T06:06:00.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d82a00640 1 -- 172.21.15.50:0/981405849 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c003860 con 0x7f8d8c0b4de0 2024-09-21T06:06:00.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d90a00640 1 -- 172.21.15.50:0/981405849 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d8c14f870 con 0x7f8d8c0b4de0 2024-09-21T06:06:00.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.962+0000 7f8d90a00640 1 -- 172.21.15.50:0/981405849 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d8c14d690 con 0x7f8d8c0b4de0 2024-09-21T06:06:00.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.966+0000 7f8d90a00640 1 -- 172.21.15.50:0/981405849 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d8c0b51f0 con 0x7f8d8c0b4de0 2024-09-21T06:06:00.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.966+0000 7f8d82a00640 1 -- 172.21.15.50:0/981405849 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f8d7c01a070 con 0x7f8d8c0b4de0 2024-09-21T06:06:00.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.966+0000 7f8d82a00640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8d68074370 0x7f8d68076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:00.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.966+0000 7f8d82a00640 1 -- 172.21.15.50:0/981405849 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7f8d7c0923e0 con 0x7f8d8c0b4de0 2024-09-21T06:06:00.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.966+0000 7f8d81400640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8d68074370 0x7f8d68076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:00.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.970+0000 7f8d81400640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8d68074370 0x7f8d68076880 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0af4a0 tx=0x7f8d74009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:00.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:00.970+0000 7f8d82a00640 1 -- 172.21.15.50:0/981405849 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8d8c0b5730 con 0x7f8d8c0b4de0 2024-09-21T06:06:01.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.150+0000 7f8d90a00640 1 -- 172.21.15.50:0/981405849 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8d8c152ed0 con 0x7f8d8c0b4de0 2024-09-21T06:06:01.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.150+0000 7f8d82a00640 1 -- 172.21.15.50:0/981405849 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f8d7c05e540 con 0x7f8d8c0b4de0 2024-09-21T06:06:01.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.154+0000 7f8d90a00640 1 -- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8d68074370 msgr2=0x7f8d68076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:01.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.154+0000 7f8d90a00640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8d68074370 0x7f8d68076880 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0af4a0 tx=0x7f8d74009260 comp rx=0 tx=0).stop 2024-09-21T06:06:01.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.154+0000 7f8d90a00640 1 -- 172.21.15.50:0/981405849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d8c0b4de0 msgr2=0x7f8d8c14b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:01.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.154+0000 7f8d90a00640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d8c0b4de0 0x7f8d8c14b860 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00c3c0 tx=0x7f8d7c00c880 comp rx=0 tx=0).stop 2024-09-21T06:06:01.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.154+0000 7f8d90a00640 1 -- 172.21.15.50:0/981405849 shutdown_connections 2024-09-21T06:06:01.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.154+0000 7f8d90a00640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d8c1527b0 0x7f8d8c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:01.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.154+0000 7f8d90a00640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d8c0b5c70 0x7f8d8c14ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:01.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.154+0000 7f8d90a00640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8d68074370 0x7f8d68076880 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:01.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.154+0000 7f8d90a00640 1 --2- 172.21.15.50:0/981405849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d8c0b4de0 0x7f8d8c14b860 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:01.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.154+0000 7f8d90a00640 1 -- 172.21.15.50:0/981405849 >> 172.21.15.50:0/981405849 conn(0x7f8d8c0b2040 msgr2=0x7f8d8c0b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:01.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.154+0000 7f8d90a00640 1 -- 172.21.15.50:0/981405849 shutdown_connections 2024-09-21T06:06:01.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.154+0000 7f8d90a00640 1 -- 172.21.15.50:0/981405849 wait complete. 2024-09-21T06:06:01.176 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:06:01.176 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:06:01.264 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:06:01.353 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:06:01.353 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:06:01.353 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:06:01.362 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:06:01.362 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:06:01.362 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:06:01.371 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:06:01.371 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:06:01.371 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:06:01.413 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:06:01.413 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:06:01.413 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:06:01.426 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:06:01.426 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:06:01.426 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:06:01.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.618+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3019074816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3ec0703a0 msgr2=0x7fe3ec06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:01.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.618+0000 7fe3f3e00640 1 --2- 172.21.15.50:0/3019074816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3ec0703a0 0x7fe3ec06d200 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009bc0 tx=0x7fe3e401cb10 comp rx=0 tx=0).stop 2024-09-21T06:06:01.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.618+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3019074816 shutdown_connections 2024-09-21T06:06:01.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.618+0000 7fe3f3e00640 1 --2- 172.21.15.50:0/3019074816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3ec0703a0 0x7fe3ec06d200 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:01.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.618+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3019074816 >> 172.21.15.50:0/3019074816 conn(0x7fe3ec0b35f0 msgr2=0x7fe3ec0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:01.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.618+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3019074816 shutdown_connections 2024-09-21T06:06:01.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.618+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3019074816 wait complete. 2024-09-21T06:06:01.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f3e00640 1 Processor -- start 2024-09-21T06:06:01.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f3e00640 1 -- start start 2024-09-21T06:06:01.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3ec06cdf0 0x7fe3ec0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:01.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3ec0703a0 0x7fe3ec1fd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:01.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f3e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3ec159130 0x7fe3ec15deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:01.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec16a470 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec16a2f0 con 0x7fe3ec0703a0 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f3e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe3ec16a5f0 con 0x7fe3ec159130 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3ec06cdf0 0x7fe3ec0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3ec06cdf0 0x7fe3ec0cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40384/0 (socket says 172.21.15.50:40384) 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f1400640 1 -- 172.21.15.50:0/3309183673 learned_addr learned my addr 172.21.15.50:0/3309183673 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f0a00640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3ec0703a0 0x7fe3ec1fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f1e00640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3ec159130 0x7fe3ec15deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f1400640 1 -- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3ec159130 msgr2=0x7fe3ec15deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f1400640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3ec159130 0x7fe3ec15deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f1400640 1 -- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3ec0703a0 msgr2=0x7fe3ec1fd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f1400640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3ec0703a0 0x7fe3ec1fd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f1400640 1 -- 172.21.15.50:0/3309183673 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ec1fd790 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f1400640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3ec06cdf0 0x7fe3ec0cea90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000c3c0 tx=0x7fe3e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:01.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3e9600640 1 -- 172.21.15.50:0/3309183673 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0017070 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3309183673 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ec0cf2c0 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3e9600640 1 -- 172.21.15.50:0/3309183673 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3e0003600 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3e9600640 1 -- 172.21.15.50:0/3309183673 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0003a50 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.622+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3309183673 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ec0cf880 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.626+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3309183673 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ec06d200 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.626+0000 7fe3e9600640 1 -- 172.21.15.50:0/3309183673 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fe3e001a070 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.626+0000 7fe3e9600640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe3cc074370 0x7fe3cc076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:01.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.626+0000 7fe3e9600640 1 -- 172.21.15.50:0/3309183673 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7fe3e0092790 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.626+0000 7fe3f0a00640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe3cc074370 0x7fe3cc076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:01.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.626+0000 7fe3f0a00640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe3cc074370 0x7fe3cc076880 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0aea00 tx=0x7fe3d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:01.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.630+0000 7fe3e9600640 1 -- 172.21.15.50:0/3309183673 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe3e0021020 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.842+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3309183673 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3ec06e350 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.842+0000 7fe3e9600640 1 -- 172.21.15.50:0/3309183673 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+3908 (secure 0 0 0) 0x7fe3e005e860 con 0x7fe3ec06cdf0 2024-09-21T06:06:01.844 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:01.844 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":48,"btime":"2024-09-21T06:05:55:368886+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/2233580616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2233580616},{"type":"v1","addr":"172.21.15.50:6833","nonce":2233580616}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3749582023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3749582023},{"type":"v1","addr":"172.21.15.50:6835","nonce":3749582023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6139,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/382971646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":382971646},{"type":"v1","addr":"172.21.15.88:6836","nonce":382971646}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1235632361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1235632361},{"type":"v1","addr":"172.21.15.88:6837","nonce":1235632361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[]} 2024-09-21T06:06:01.845 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 48 2024-09-21T06:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.846+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe3cc074370 msgr2=0x7fe3cc076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.846+0000 7fe3f3e00640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe3cc074370 0x7fe3cc076880 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0aea00 tx=0x7fe3d8009040 comp rx=0 tx=0).stop 2024-09-21T06:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.846+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3309183673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3ec06cdf0 msgr2=0x7fe3ec0cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.846+0000 7fe3f3e00640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3ec06cdf0 0x7fe3ec0cea90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000c3c0 tx=0x7fe3e000c880 comp rx=0 tx=0).stop 2024-09-21T06:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.846+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3309183673 shutdown_connections 2024-09-21T06:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.846+0000 7fe3f3e00640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3ec159130 0x7fe3ec15deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.846+0000 7fe3f3e00640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3ec0703a0 0x7fe3ec1fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.846+0000 7fe3f3e00640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe3cc074370 0x7fe3cc076880 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.846+0000 7fe3f3e00640 1 --2- 172.21.15.50:0/3309183673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3ec06cdf0 0x7fe3ec0cea90 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.846+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3309183673 >> 172.21.15.50:0/3309183673 conn(0x7fe3ec0b35f0 msgr2=0x7fe3ec152180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.846+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3309183673 shutdown_connections 2024-09-21T06:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:01.846+0000 7fe3f3e00640 1 -- 172.21.15.50:0/3309183673 wait complete. 2024-09-21T06:06:01.867 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:06:01.868 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:06:01.868 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:06:01.868 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:06:01.868 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:06:01.868 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:06:01.868 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:06:01.868 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:06:01.869 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:06:01.869 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:06:01.869 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:06:01.869 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:06:01.869 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:06:01.869 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:06:01.869 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:06:01.869 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:06:01.883 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:06:01.884 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:06:01.885 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:06:01.885 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:06:01.887 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:06:01.887 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:06:01.897 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:06:01.897 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:06:01.935 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:06:01.935 DEBUG:teuthology.parallel:result is None 2024-09-21T06:06:01.937 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:06:01.937 DEBUG:teuthology.parallel:result is None 2024-09-21T06:06:01.940 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:06:01.940 DEBUG:teuthology.parallel:result is None 2024-09-21T06:06:01.942 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:06:01.942 DEBUG:teuthology.parallel:result is None 2024-09-21T06:06:01.943 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:06:02.037 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-09-21T06:06:02.037 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T06:06:02.043 INFO:tasks.ceph.mds.d.smithi088.stdout:starting mds.d at 2024-09-21T06:06:02.045 INFO:tasks.ceph.mds.b.smithi088.stdout:starting mds.b at 2024-09-21T06:06:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.110+0000 7fb6d5e00640 1 -- 172.21.15.50:0/293443267 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6d00b5a80 msgr2=0x7fb6d0074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.110+0000 7fb6d5e00640 1 --2- 172.21.15.50:0/293443267 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6d00b5a80 0x7fb6d0074410 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009840 tx=0x7fb6b801c740 comp rx=0 tx=0).stop 2024-09-21T06:06:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.110+0000 7fb6d5e00640 1 -- 172.21.15.50:0/293443267 shutdown_connections 2024-09-21T06:06:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.110+0000 7fb6d5e00640 1 --2- 172.21.15.50:0/293443267 >> v2:172.21.15.88:3300/0 conn(0x7fb6d0074950 0x7fb6d015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.110+0000 7fb6d5e00640 1 --2- 172.21.15.50:0/293443267 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6d00b5a80 0x7fb6d0074410 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.110+0000 7fb6d5e00640 1 -- 172.21.15.50:0/293443267 >> 172.21.15.50:0/293443267 conn(0x7fb6d00708c0 msgr2=0x7fb6d0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.110+0000 7fb6d5e00640 1 -- 172.21.15.50:0/293443267 shutdown_connections 2024-09-21T06:06:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.110+0000 7fb6d5e00640 1 -- 172.21.15.50:0/293443267 wait complete. 2024-09-21T06:06:02.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.110+0000 7fb6d5e00640 1 Processor -- start 2024-09-21T06:06:02.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.110+0000 7fb6d5e00640 1 -- start start 2024-09-21T06:06:02.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6d5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d0074950 0x7fb6d0139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:02.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6d5e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6d00b5a80 0x7fb6d015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:02.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6d5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6d0157030 0x7fb6d01550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:02.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6d5e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d016a360 con 0x7fb6d0157030 2024-09-21T06:06:02.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6d5e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d016a1e0 con 0x7fb6d0074950 2024-09-21T06:06:02.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6d5e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d016a4e0 con 0x7fb6d00b5a80 2024-09-21T06:06:02.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6cf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6d0157030 0x7fb6d01550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:02.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6cf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6d0157030 0x7fb6d01550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40418/0 (socket says 172.21.15.50:40418) 2024-09-21T06:06:02.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6cf400640 1 -- 172.21.15.50:0/1040732809 learned_addr learned my addr 172.21.15.50:0/1040732809 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:02.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6cea00640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d0074950 0x7fb6d0139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:02.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6c7e00640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6d00b5a80 0x7fb6d015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:02.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6cf400640 1 -- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6d00b5a80 msgr2=0x7fb6d015ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:02.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6cf400640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6d00b5a80 0x7fb6d015ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6cf400640 1 -- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d0074950 msgr2=0x7fb6d0139ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:02.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6cf400640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d0074950 0x7fb6d0139ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6cf400640 1 -- 172.21.15.50:0/1040732809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d01fc9b0 con 0x7fb6d0157030 2024-09-21T06:06:02.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6cf400640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6d0157030 0x7fb6d01550b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800eae0 tx=0x7fb6c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:02.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6c7400640 1 -- 172.21.15.50:0/1040732809 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c800ce50 con 0x7fb6d0157030 2024-09-21T06:06:02.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6c7400640 1 -- 172.21.15.50:0/1040732809 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6c8004b30 con 0x7fb6d0157030 2024-09-21T06:06:02.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6c7400640 1 -- 172.21.15.50:0/1040732809 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8005020 con 0x7fb6d0157030 2024-09-21T06:06:02.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6d5e00640 1 -- 172.21.15.50:0/1040732809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d01fcc90 con 0x7fb6d0157030 2024-09-21T06:06:02.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6d5e00640 1 -- 172.21.15.50:0/1040732809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d01fd250 con 0x7fb6d0157030 2024-09-21T06:06:02.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6d5e00640 1 -- 172.21.15.50:0/1040732809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d015cf90 con 0x7fb6d0157030 2024-09-21T06:06:02.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.114+0000 7fb6c7400640 1 -- 172.21.15.50:0/1040732809 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fb6c80051c0 con 0x7fb6d0157030 2024-09-21T06:06:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.118+0000 7fb6c7400640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb6ac0743a0 0x7fb6ac0768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.118+0000 7fb6c7400640 1 -- 172.21.15.50:0/1040732809 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7fb6c8093bc0 con 0x7fb6d0157030 2024-09-21T06:06:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.118+0000 7fb6cea00640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb6ac0743a0 0x7fb6ac0768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.118+0000 7fb6cea00640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb6ac0743a0 0x7fb6ac0768b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0007990 tx=0x7fb6c00012e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.122+0000 7fb6c7400640 1 -- 172.21.15.50:0/1040732809 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb6c805fc90 con 0x7fb6d0157030 2024-09-21T06:06:02.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.310+0000 7fb6d5e00640 1 -- 172.21.15.50:0/1040732809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb6d00d2260 con 0x7fb6d0157030 2024-09-21T06:06:02.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.422+0000 7fb6c7400640 1 -- 172.21.15.50:0/1040732809 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) ==== 97+0+0 (secure 0 0 0) 0x7fb6c80646c0 con 0x7fb6d0157030 2024-09-21T06:06:02.425 INFO:teuthology.orchestra.run.smithi050.stderr: removed all blocklist entries 2024-09-21T06:06:02.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.426+0000 7fb6d5e00640 1 -- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb6ac0743a0 msgr2=0x7fb6ac0768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:02.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.426+0000 7fb6d5e00640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb6ac0743a0 0x7fb6ac0768b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0007990 tx=0x7fb6c00012e0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.426+0000 7fb6d5e00640 1 -- 172.21.15.50:0/1040732809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6d0157030 msgr2=0x7fb6d01550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:02.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.426+0000 7fb6d5e00640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6d0157030 0x7fb6d01550b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800eae0 tx=0x7fb6c800c2d0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.426+0000 7fb6d5e00640 1 -- 172.21.15.50:0/1040732809 shutdown_connections 2024-09-21T06:06:02.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.426+0000 7fb6d5e00640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb6ac0743a0 0x7fb6ac0768b0 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0007990 tx=0x7fb6c00012e0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.426+0000 7fb6d5e00640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6d0157030 0x7fb6d01550b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.426+0000 7fb6d5e00640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6d00b5a80 0x7fb6d015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.426+0000 7fb6d5e00640 1 --2- 172.21.15.50:0/1040732809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d0074950 0x7fb6d0139ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.426+0000 7fb6d5e00640 1 -- 172.21.15.50:0/1040732809 >> 172.21.15.50:0/1040732809 conn(0x7fb6d00708c0 msgr2=0x7fb6d0155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:02.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.426+0000 7fb6d5e00640 1 -- 172.21.15.50:0/1040732809 shutdown_connections 2024-09-21T06:06:02.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.426+0000 7fb6d5e00640 1 -- 172.21.15.50:0/1040732809 wait complete. 2024-09-21T06:06:02.451 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:06:02.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.610+0000 7effb0200640 1 -- 172.21.15.50:0/2200293853 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7effa8074300 msgr2=0x7effa815f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:02.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.610+0000 7effb0200640 1 --2- 172.21.15.50:0/2200293853 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7effa8074300 0x7effa815f170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7eff94009900 tx=0x7eff9401c850 comp rx=0 tx=0).stop 2024-09-21T06:06:02.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.610+0000 7effb0200640 1 -- 172.21.15.50:0/2200293853 shutdown_connections 2024-09-21T06:06:02.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.610+0000 7effb0200640 1 --2- 172.21.15.50:0/2200293853 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7effa8074300 0x7effa815f170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.610+0000 7effb0200640 1 --2- 172.21.15.50:0/2200293853 >> v2:172.21.15.88:3300/0 conn(0x7effa815c5d0 0x7effa8073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.610+0000 7effb0200640 1 -- 172.21.15.50:0/2200293853 >> 172.21.15.50:0/2200293853 conn(0x7effa80b2c40 msgr2=0x7effa80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:02.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.610+0000 7effb0200640 1 -- 172.21.15.50:0/2200293853 shutdown_connections 2024-09-21T06:06:02.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.610+0000 7effb0200640 1 -- 172.21.15.50:0/2200293853 wait complete. 2024-09-21T06:06:02.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.610+0000 7effb0200640 1 Processor -- start 2024-09-21T06:06:02.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effb0200640 1 -- start start 2024-09-21T06:06:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effb0200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effa8074300 0x7effa814dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effb0200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7effa815c5d0 0x7effa81552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effb0200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effa815f6b0 0x7effa80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effb0200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7effa806f980 con 0x7effa815f6b0 2024-09-21T06:06:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effb0200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7effa80b3d50 con 0x7effa8074300 2024-09-21T06:06:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effb0200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7effa806fb00 con 0x7effa815c5d0 2024-09-21T06:06:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effae200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effa815f6b0 0x7effa80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effae200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effa815f6b0 0x7effa80cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40422/0 (socket says 172.21.15.50:40422) 2024-09-21T06:06:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effae200640 1 -- 172.21.15.50:0/2975845927 learned_addr learned my addr 172.21.15.50:0/2975845927 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7efface00640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7effa815c5d0 0x7effa81552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:02.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effad800640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effa8074300 0x7effa814dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effae200640 1 -- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7effa815c5d0 msgr2=0x7effa81552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effae200640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7effa815c5d0 0x7effa81552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effae200640 1 -- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effa8074300 msgr2=0x7effa814dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effae200640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effa8074300 0x7effa814dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effae200640 1 -- 172.21.15.50:0/2975845927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa81f8eb0 con 0x7effa815f6b0 2024-09-21T06:06:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effad800640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effa8074300 0x7effa814dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effae200640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effa815f6b0 0x7effa80cec70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7effa000c710 tx=0x7effa000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effa5600640 1 -- 172.21.15.50:0/2975845927 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0003b80 con 0x7effa815f6b0 2024-09-21T06:06:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effa5600640 1 -- 172.21.15.50:0/2975845927 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effa0003d20 con 0x7effa815f6b0 2024-09-21T06:06:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effa5600640 1 -- 172.21.15.50:0/2975845927 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa000aab0 con 0x7effa815f6b0 2024-09-21T06:06:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effb0200640 1 -- 172.21.15.50:0/2975845927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa81fc970 con 0x7effa815f6b0 2024-09-21T06:06:02.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effb0200640 1 -- 172.21.15.50:0/2975845927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa81fcf30 con 0x7effa815f6b0 2024-09-21T06:06:02.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.614+0000 7effb0200640 1 -- 172.21.15.50:0/2975845927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa81fc6e0 con 0x7effa815f6b0 2024-09-21T06:06:02.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.618+0000 7effa5600640 1 -- 172.21.15.50:0/2975845927 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7effa000ac50 con 0x7effa815f6b0 2024-09-21T06:06:02.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.618+0000 7effa5600640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7eff88074370 0x7eff88076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:02.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.618+0000 7effa5600640 1 -- 172.21.15.50:0/2975845927 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4913+0+0 (secure 0 0 0) 0x7effa0092e40 con 0x7effa815f6b0 2024-09-21T06:06:02.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.618+0000 7effad800640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7eff88074370 0x7eff88076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:02.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.618+0000 7effad800640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7eff88074370 0x7eff88076880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009f30 tx=0x7eff9c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:02.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.622+0000 7effa5600640 1 -- 172.21.15.50:0/2975845927 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7effa000ce20 con 0x7effa815f6b0 2024-09-21T06:06:02.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 -- 172.21.15.50:0/2975845927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7effa8073660 con 0x7effa815f6b0 2024-09-21T06:06:02.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effa5600640 1 -- 172.21.15.50:0/2975845927 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v33) ==== 80+0+5304 (secure 0 0 0) 0x7effa005ef10 con 0x7effa815f6b0 2024-09-21T06:06:02.862 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:02.863 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:06:02.863 INFO:teuthology.orchestra.run.smithi050.stdout: "auth_dump": [ 2024-09-21T06:06:02.863 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.863 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.a", 2024-09-21T06:06:02.863 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mSKq0ARAA2egaHGYWhRUEq6oRl1CIMg==", 2024-09-21T06:06:02.863 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.863 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:06:02.863 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:06:02.863 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:06:02.863 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.863 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.863 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.b", 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mwv0+DxAAY+eZPxv5uY9PnLl0asdr6w==", 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.c", 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5me6maCRAA4jvHgTLTKpb/mxzzG71SfQ==", 2024-09-21T06:06:02.864 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.d", 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mzYnuFhAA/hEigjluXspzs/hyeWxBjQ==", 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:06:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.0", 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDIX+5mrZF/NhAA33M6ummG4r4yXU8/YOL5WA==", 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.1", 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDMX+5myVz5DxAAMYfTPep6ubD7Cxnfa9dMCg==", 2024-09-21T06:06:02.866 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.2", 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDPX+5mA7ahIRAAf/osnY5l83p8ZWna0ErD2Q==", 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.867 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.3", 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDSX+5m+rDpNRAApGPZJIlAaA0DyjtPXPgoaQ==", 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.4", 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDYX+5mKL9SORAA21ITQUwUSoDUjXaD0Cwb3g==", 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.868 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.5", 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDcX+5minRqFBAAMJOBQNvJFD4TNdKbPMLBig==", 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.6", 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDfX+5m6HadJhAAAv670Tvyi4IKMByFPiBlAw==", 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:06:02.869 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.7", 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDjX+5muPrrAhAAuRB7Muamiht+f7LvSmI6UQ==", 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.0", 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw==", 2024-09-21T06:06:02.870 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow r", 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.1", 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mo94GJBAAMBSb1NPb2L93vzvnAh59kw==", 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:06:02.871 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.2", 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5m4IrnHxAAFjER3ywuS+FlbuA7bPZVqw==", 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.3", 2024-09-21T06:06:02.872 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mysQeKBAAq5c4bD3hbsU1H7JabaMjwQ==", 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.admin", 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mcUXKHhAAV1Rs0OMNVPeAsRiAVt67jA==", 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:06:02.873 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.x", 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5m2F9WOBAAZuQTUyR7oTT01CRD1jo+/Q==", 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.y", 2024-09-21T06:06:02.874 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mEjC/MxAAHgwIu4CLDgFajwD2rL5WKA==", 2024-09-21T06:06:02.875 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:06:02.875 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:06:02.875 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:06:02.875 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:06:02.875 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.875 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:02.875 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-21T06:06:02.875 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:06:02.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 -- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7eff88074370 msgr2=0x7eff88076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:02.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7eff88074370 0x7eff88076880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009f30 tx=0x7eff9c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 -- 172.21.15.50:0/2975845927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effa815f6b0 msgr2=0x7effa80cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:02.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effa815f6b0 0x7effa80cec70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7effa000c710 tx=0x7effa000cbd0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 -- 172.21.15.50:0/2975845927 shutdown_connections 2024-09-21T06:06:02.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7eff88074370 0x7eff88076880 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effa815f6b0 0x7effa80cec70 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7effa815c5d0 0x7effa81552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 --2- 172.21.15.50:0/2975845927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effa8074300 0x7effa814dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:02.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 -- 172.21.15.50:0/2975845927 >> 172.21.15.50:0/2975845927 conn(0x7effa80b2c40 msgr2=0x7effa80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:02.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 -- 172.21.15.50:0/2975845927 shutdown_connections 2024-09-21T06:06:02.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:02.862+0000 7effb0200640 1 -- 172.21.15.50:0/2975845927 wait complete. 2024-09-21T06:06:02.886 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.042+0000 7f4d0f600640 1 -- 172.21.15.50:0/1732993254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d080749a0 msgr2=0x7f4d08074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.042+0000 7f4d0f600640 1 --2- 172.21.15.50:0/1732993254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d080749a0 0x7f4d08074db0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4d0815a930 tx=0x7f4cf801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.042+0000 7f4d0f600640 1 -- 172.21.15.50:0/1732993254 shutdown_connections 2024-09-21T06:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.042+0000 7f4d0f600640 1 --2- 172.21.15.50:0/1732993254 >> v2:172.21.15.88:3300/0 conn(0x7f4d08160330 0x7f4d08160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.042+0000 7f4d0f600640 1 --2- 172.21.15.50:0/1732993254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4d080cefa0 0x7f4d080cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.042+0000 7f4d0f600640 1 --2- 172.21.15.50:0/1732993254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d080749a0 0x7f4d08074db0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.042+0000 7f4d0f600640 1 -- 172.21.15.50:0/1732993254 >> 172.21.15.50:0/1732993254 conn(0x7f4d080b2040 msgr2=0x7f4d080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.042+0000 7f4d0f600640 1 -- 172.21.15.50:0/1732993254 shutdown_connections 2024-09-21T06:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.042+0000 7f4d0f600640 1 -- 172.21.15.50:0/1732993254 wait complete. 2024-09-21T06:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4d0f600640 1 Processor -- start 2024-09-21T06:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4d0f600640 1 -- start start 2024-09-21T06:06:03.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4d0f600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d080749a0 0x7f4d08163c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:03.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4d0f600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d080cefa0 0x7f4d080b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:03.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4d0f600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4d08160330 0x7f4d080b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:03.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4d0f600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4d080b3ee0 con 0x7f4d080cefa0 2024-09-21T06:06:03.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4d0f600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4d080b3d60 con 0x7f4d080749a0 2024-09-21T06:06:03.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4d0f600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4d080b4060 con 0x7f4d08160330 2024-09-21T06:06:03.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4cffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d080cefa0 0x7f4d080b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:03.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4cffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d080cefa0 0x7f4d080b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40448/0 (socket says 172.21.15.50:40448) 2024-09-21T06:06:03.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4cffe00640 1 -- 172.21.15.50:0/409959475 learned_addr learned my addr 172.21.15.50:0/409959475 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:03.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4d0cc00640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d080749a0 0x7f4d08163c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:03.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4d0d600640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4d08160330 0x7f4d080b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:03.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4cffe00640 1 -- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4d08160330 msgr2=0x7f4d080b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:03.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4cffe00640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4d08160330 0x7f4d080b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4cffe00640 1 -- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d080749a0 msgr2=0x7f4d08163c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:03.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4cffe00640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d080749a0 0x7f4d08163c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4cffe00640 1 -- 172.21.15.50:0/409959475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d08166f00 con 0x7f4d080cefa0 2024-09-21T06:06:03.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.046+0000 7f4cffe00640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d080cefa0 0x7f4d080b0150 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000e740 tx=0x7f4cf000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:03.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.050+0000 7f4d05600640 1 -- 172.21.15.50:0/409959475 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf000cc60 con 0x7f4d080cefa0 2024-09-21T06:06:03.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.050+0000 7f4d05600640 1 -- 172.21.15.50:0/409959475 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cf000ce00 con 0x7f4d080cefa0 2024-09-21T06:06:03.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.050+0000 7f4d0f600640 1 -- 172.21.15.50:0/409959475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d081671e0 con 0x7f4d080cefa0 2024-09-21T06:06:03.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.050+0000 7f4d05600640 1 -- 172.21.15.50:0/409959475 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0005280 con 0x7f4d080cefa0 2024-09-21T06:06:03.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.050+0000 7f4d0f600640 1 -- 172.21.15.50:0/409959475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d08161730 con 0x7f4d080cefa0 2024-09-21T06:06:03.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.050+0000 7f4d0f600640 1 -- 172.21.15.50:0/409959475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d08166c70 con 0x7f4d080cefa0 2024-09-21T06:06:03.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.050+0000 7f4d05600640 1 -- 172.21.15.50:0/409959475 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f4cf0005420 con 0x7f4d080cefa0 2024-09-21T06:06:03.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.050+0000 7f4d05600640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ce4074370 0x7f4ce4076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:03.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.050+0000 7f4d05600640 1 -- 172.21.15.50:0/409959475 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4913+0+0 (secure 0 0 0) 0x7f4cf00938a0 con 0x7f4d080cefa0 2024-09-21T06:06:03.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.050+0000 7f4d0cc00640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ce4074370 0x7f4ce4076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:03.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.054+0000 7f4d0cc00640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ce4074370 0x7f4ce4076880 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80af000 tx=0x7f4cf801b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:03.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.054+0000 7f4d05600640 1 -- 172.21.15.50:0/409959475 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4cf005fa00 con 0x7f4d080cefa0 2024-09-21T06:06:03.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 -- 172.21.15.50:0/409959475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4d080cf3b0 con 0x7f4d080cefa0 2024-09-21T06:06:03.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d05600640 1 -- 172.21.15.50:0/409959475 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f4cf0064430 con 0x7f4d080cefa0 2024-09-21T06:06:03.239 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:03.239 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:06:03.239 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:03.239 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-09-21T06:06:03.239 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-09-21T06:06:03.239 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:03.239 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:06:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 -- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ce4074370 msgr2=0x7f4ce4076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ce4074370 0x7f4ce4076880 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80af000 tx=0x7f4cf801b100 comp rx=0 tx=0).stop 2024-09-21T06:06:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 -- 172.21.15.50:0/409959475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d080cefa0 msgr2=0x7f4d080b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d080cefa0 0x7f4d080b0150 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000e740 tx=0x7f4cf000ec00 comp rx=0 tx=0).stop 2024-09-21T06:06:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 -- 172.21.15.50:0/409959475 shutdown_connections 2024-09-21T06:06:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4d08160330 0x7f4d080b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4ce4074370 0x7f4ce4076880 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d080cefa0 0x7f4d080b0150 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 --2- 172.21.15.50:0/409959475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d080749a0 0x7f4d08163c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 -- 172.21.15.50:0/409959475 >> 172.21.15.50:0/409959475 conn(0x7f4d080b2040 msgr2=0x7f4d08141530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:03.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 -- 172.21.15.50:0/409959475 shutdown_connections 2024-09-21T06:06:03.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.238+0000 7f4d0f600640 1 -- 172.21.15.50:0/409959475 wait complete. 2024-09-21T06:06:03.261 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:06:03.262 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:06:03.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.426+0000 7f0334a00640 1 -- 172.21.15.50:0/4030901342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0314004810 msgr2=0x7f0314004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:03.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.426+0000 7f0334a00640 1 --2- 172.21.15.50:0/4030901342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0314004810 0x7f0314004c20 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0330072810 tx=0x7f031c01c510 comp rx=0 tx=0).stop 2024-09-21T06:06:03.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.426+0000 7f0334a00640 1 -- 172.21.15.50:0/4030901342 shutdown_connections 2024-09-21T06:06:03.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.426+0000 7f0334a00640 1 --2- 172.21.15.50:0/4030901342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0314004810 0x7f0314004c20 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.426+0000 7f0334a00640 1 -- 172.21.15.50:0/4030901342 >> 172.21.15.50:0/4030901342 conn(0x7f03300b2c40 msgr2=0x7f03300b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:03.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.426+0000 7f0334a00640 1 -- 172.21.15.50:0/4030901342 shutdown_connections 2024-09-21T06:06:03.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.426+0000 7f0334a00640 1 -- 172.21.15.50:0/4030901342 wait complete. 2024-09-21T06:06:03.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.426+0000 7f0334a00640 1 Processor -- start 2024-09-21T06:06:03.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.426+0000 7f0334a00640 1 -- start start 2024-09-21T06:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f0334a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0314004810 0x7f03300b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f0334a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03300b2070 0x7f03301f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f0334a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f03301f93a0 0x7f0330147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f0334a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f03300b3b40 con 0x7f03300b2070 2024-09-21T06:06:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f0334a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f03300b6000 con 0x7f0314004810 2024-09-21T06:06:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f0334a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f03300b3cc0 con 0x7f03301f93a0 2024-09-21T06:06:03.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f032d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03300b2070 0x7f03301f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:03.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f032d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03300b2070 0x7f03301f8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40468/0 (socket says 172.21.15.50:40468) 2024-09-21T06:06:03.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f032d400640 1 -- 172.21.15.50:0/2720511652 learned_addr learned my addr 172.21.15.50:0/2720511652 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f032de00640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0314004810 0x7f03300b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f032e800640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f03301f93a0 0x7f0330147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f032d400640 1 -- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f03301f93a0 msgr2=0x7f0330147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f032d400640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f03301f93a0 0x7f0330147ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f032d400640 1 -- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0314004810 msgr2=0x7f03300b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f032d400640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0314004810 0x7f03300b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f032d400640 1 -- 172.21.15.50:0/2720511652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03300ab6c0 con 0x7f03300b2070 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f032d400640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03300b2070 0x7f03301f8e60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f031c01c9f0 tx=0x7f031c003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f0326000640 1 -- 172.21.15.50:0/2720511652 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c0ba070 con 0x7f03300b2070 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f0326000640 1 -- 172.21.15.50:0/2720511652 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f031c0af040 con 0x7f03300b2070 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f0334a00640 1 -- 172.21.15.50:0/2720511652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03300ab9a0 con 0x7f03300b2070 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f0326000640 1 -- 172.21.15.50:0/2720511652 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c0ae070 con 0x7f03300b2070 2024-09-21T06:06:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f0334a00640 1 -- 172.21.15.50:0/2720511652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03300ac480 con 0x7f03300b2070 2024-09-21T06:06:03.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.430+0000 7f0334a00640 1 -- 172.21.15.50:0/2720511652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f033015f410 con 0x7f03300b2070 2024-09-21T06:06:03.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.434+0000 7f0326000640 1 -- 172.21.15.50:0/2720511652 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f031c0a7070 con 0x7f03300b2070 2024-09-21T06:06:03.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.434+0000 7f0326000640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f03100743a0 0x7f03100768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:03.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.434+0000 7f0326000640 1 -- 172.21.15.50:0/2720511652 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4913+0+0 (secure 0 0 0) 0x7f031c12c130 con 0x7f03300b2070 2024-09-21T06:06:03.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.434+0000 7f032de00640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f03100743a0 0x7f03100768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:03.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.434+0000 7f032de00640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f03100743a0 0x7f03100768b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f033006bb90 tx=0x7f03200091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:03.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.438+0000 7f0326000640 1 -- 172.21.15.50:0/2720511652 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0330074710 con 0x7f03300b2070 2024-09-21T06:06:03.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:03.618+0000 7f0334a00640 1 -- 172.21.15.50:0/2720511652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f03301f9960 con 0x7f03300b2070 2024-09-21T06:06:04.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.442+0000 7f0326000640 1 -- 172.21.15.50:0/2720511652 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f031c0f8270 con 0x7f03300b2070 2024-09-21T06:06:04.445 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-09-21T06:06:04.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.446+0000 7f0334a00640 1 -- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f03100743a0 msgr2=0x7f03100768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:04.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.446+0000 7f0334a00640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f03100743a0 0x7f03100768b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f033006bb90 tx=0x7f03200091e0 comp rx=0 tx=0).stop 2024-09-21T06:06:04.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.446+0000 7f0334a00640 1 -- 172.21.15.50:0/2720511652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03300b2070 msgr2=0x7f03301f8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:04.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.446+0000 7f0334a00640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03300b2070 0x7f03301f8e60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f031c01c9f0 tx=0x7f031c003c60 comp rx=0 tx=0).stop 2024-09-21T06:06:04.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.446+0000 7f0334a00640 1 -- 172.21.15.50:0/2720511652 shutdown_connections 2024-09-21T06:06:04.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.446+0000 7f0334a00640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f03301f93a0 0x7f0330147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:04.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.446+0000 7f0334a00640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f03100743a0 0x7f03100768b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:04.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.446+0000 7f0334a00640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03300b2070 0x7f03301f8e60 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:04.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.446+0000 7f0334a00640 1 --2- 172.21.15.50:0/2720511652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0314004810 0x7f03300b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:04.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.446+0000 7f0334a00640 1 -- 172.21.15.50:0/2720511652 >> 172.21.15.50:0/2720511652 conn(0x7f03300b2c40 msgr2=0x7f03300b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:04.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.446+0000 7f0334a00640 1 -- 172.21.15.50:0/2720511652 shutdown_connections 2024-09-21T06:06:04.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.446+0000 7f0334a00640 1 -- 172.21.15.50:0/2720511652 wait complete. 2024-09-21T06:06:04.478 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:06:04.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.646+0000 7f7373e00640 1 -- 172.21.15.50:0/732864724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736c074410 msgr2=0x7f736c074820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.646+0000 7f7373e00640 1 --2- 172.21.15.50:0/732864724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736c074410 0x7f736c074820 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f736c0b7b80 tx=0x7f736401cd00 comp rx=0 tx=0).stop 2024-09-21T06:06:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.646+0000 7f7373e00640 1 -- 172.21.15.50:0/732864724 shutdown_connections 2024-09-21T06:06:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.646+0000 7f7373e00640 1 --2- 172.21.15.50:0/732864724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736c074410 0x7f736c074820 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.646+0000 7f7373e00640 1 --2- 172.21.15.50:0/732864724 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f736c0d29c0 0x7f736c0d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.646+0000 7f7373e00640 1 --2- 172.21.15.50:0/732864724 >> v2:172.21.15.88:3300/0 conn(0x7f736c155ad0 0x7f736c06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.646+0000 7f7373e00640 1 -- 172.21.15.50:0/732864724 >> 172.21.15.50:0/732864724 conn(0x7f736c0708c0 msgr2=0x7f736c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.650+0000 7f7373e00640 1 -- 172.21.15.50:0/732864724 shutdown_connections 2024-09-21T06:06:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.650+0000 7f7373e00640 1 -- 172.21.15.50:0/732864724 wait complete. 2024-09-21T06:06:04.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.650+0000 7f7373e00640 1 Processor -- start 2024-09-21T06:06:04.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.650+0000 7f7373e00640 1 -- start start 2024-09-21T06:06:04.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.650+0000 7f7373e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f736c074410 0x7f736c162120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:04.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.650+0000 7f7373e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f736c0d29c0 0x7f736c152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:04.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.650+0000 7f7373e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736c155ad0 0x7f736c158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:04.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.650+0000 7f7373e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f736c15fd20 con 0x7f736c155ad0 2024-09-21T06:06:04.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.650+0000 7f7373e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f736c15fba0 con 0x7f736c074410 2024-09-21T06:06:04.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.650+0000 7f7373e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f736c15fea0 con 0x7f736c0d29c0 2024-09-21T06:06:04.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.650+0000 7f7371e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736c155ad0 0x7f736c158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:04.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7371e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736c155ad0 0x7f736c158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40478/0 (socket says 172.21.15.50:40478) 2024-09-21T06:06:04.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7371e00640 1 -- 172.21.15.50:0/794058216 learned_addr learned my addr 172.21.15.50:0/794058216 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:04.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7370a00640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f736c0d29c0 0x7f736c152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:04.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7371400640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f736c074410 0x7f736c162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:04.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7371e00640 1 -- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f736c0d29c0 msgr2=0x7f736c152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:04.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7371e00640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f736c0d29c0 0x7f736c152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:04.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7371e00640 1 -- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f736c074410 msgr2=0x7f736c162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:04.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7371e00640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f736c074410 0x7f736c162120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:04.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7371e00640 1 -- 172.21.15.50:0/794058216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736c1fcb60 con 0x7f736c155ad0 2024-09-21T06:06:04.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7371e00640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736c155ad0 0x7f736c158ae0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f736401b410 tx=0x7f73640092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:04.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7369600640 1 -- 172.21.15.50:0/794058216 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73640b2040 con 0x7f736c155ad0 2024-09-21T06:06:04.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7369600640 1 -- 172.21.15.50:0/794058216 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7364004d40 con 0x7f736c155ad0 2024-09-21T06:06:04.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7373e00640 1 -- 172.21.15.50:0/794058216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736c1fce40 con 0x7f736c155ad0 2024-09-21T06:06:04.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7369600640 1 -- 172.21.15.50:0/794058216 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364004260 con 0x7f736c155ad0 2024-09-21T06:06:04.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7373e00640 1 -- 172.21.15.50:0/794058216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f736c1fd400 con 0x7f736c155ad0 2024-09-21T06:06:04.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7373e00640 1 -- 172.21.15.50:0/794058216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f736c0d2dd0 con 0x7f736c155ad0 2024-09-21T06:06:04.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.654+0000 7f7369600640 1 -- 172.21.15.50:0/794058216 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f73640be060 con 0x7f736c155ad0 2024-09-21T06:06:04.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.658+0000 7f7369600640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f734c074370 0x7f734c076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:04.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.658+0000 7f7369600640 1 -- 172.21.15.50:0/794058216 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5283+0+0 (secure 0 0 0) 0x7f736412d070 con 0x7f736c155ad0 2024-09-21T06:06:04.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.658+0000 7f7371400640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f734c074370 0x7f734c076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:04.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.658+0000 7f7371400640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f734c074370 0x7f734c076880 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f736000bad0 tx=0x7f736000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:04.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.662+0000 7f7369600640 1 -- 172.21.15.50:0/794058216 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7364016950 con 0x7f736c155ad0 2024-09-21T06:06:04.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:04.838+0000 7f7373e00640 1 -- 172.21.15.50:0/794058216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f736c150ce0 con 0x7f736c155ad0 2024-09-21T06:06:05.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.450+0000 7f7369600640 1 -- 172.21.15.50:0/794058216 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f73640f91d0 con 0x7f736c155ad0 2024-09-21T06:06:05.453 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-09-21T06:06:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.458+0000 7f7373e00640 1 -- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f734c074370 msgr2=0x7f734c076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.458+0000 7f7373e00640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f734c074370 0x7f734c076880 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f736000bad0 tx=0x7f736000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:06:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.458+0000 7f7373e00640 1 -- 172.21.15.50:0/794058216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736c155ad0 msgr2=0x7f736c158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.458+0000 7f7373e00640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736c155ad0 0x7f736c158ae0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f736401b410 tx=0x7f73640092c0 comp rx=0 tx=0).stop 2024-09-21T06:06:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.458+0000 7f7373e00640 1 -- 172.21.15.50:0/794058216 shutdown_connections 2024-09-21T06:06:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.458+0000 7f7373e00640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f734c074370 0x7f734c076880 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.458+0000 7f7373e00640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736c155ad0 0x7f736c158ae0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.458+0000 7f7373e00640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f736c0d29c0 0x7f736c152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.458+0000 7f7373e00640 1 --2- 172.21.15.50:0/794058216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f736c074410 0x7f736c162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:05.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.458+0000 7f7373e00640 1 -- 172.21.15.50:0/794058216 >> 172.21.15.50:0/794058216 conn(0x7f736c0708c0 msgr2=0x7f736c154050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:05.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.458+0000 7f7373e00640 1 -- 172.21.15.50:0/794058216 shutdown_connections 2024-09-21T06:06:05.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.458+0000 7f7373e00640 1 -- 172.21.15.50:0/794058216 wait complete. 2024-09-21T06:06:05.498 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:06:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.662+0000 7f11d2000640 1 -- 172.21.15.50:0/1397883386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11cc0b4de0 msgr2=0x7f11cc0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.662+0000 7f11d2000640 1 --2- 172.21.15.50:0/1397883386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11cc0b4de0 0x7f11cc0b51f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0703e0 tx=0x7f11c401ccc0 comp rx=0 tx=0).stop 2024-09-21T06:06:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.662+0000 7f11d2000640 1 -- 172.21.15.50:0/1397883386 shutdown_connections 2024-09-21T06:06:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.662+0000 7f11d2000640 1 --2- 172.21.15.50:0/1397883386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11cc0b4de0 0x7f11cc0b51f0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.662+0000 7f11d2000640 1 --2- 172.21.15.50:0/1397883386 >> v2:172.21.15.88:3300/0 conn(0x7f11cc1527b0 0x7f11cc074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.662+0000 7f11d2000640 1 --2- 172.21.15.50:0/1397883386 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11cc0b5c70 0x7f11cc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.662+0000 7f11d2000640 1 -- 172.21.15.50:0/1397883386 >> 172.21.15.50:0/1397883386 conn(0x7f11cc0b2040 msgr2=0x7f11cc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.662+0000 7f11d2000640 1 -- 172.21.15.50:0/1397883386 shutdown_connections 2024-09-21T06:06:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.662+0000 7f11d2000640 1 -- 172.21.15.50:0/1397883386 wait complete. 2024-09-21T06:06:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11d2000640 1 Processor -- start 2024-09-21T06:06:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11d2000640 1 -- start start 2024-09-21T06:06:05.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11d2000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11cc0b4de0 0x7f11cc0ab290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:05.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11d2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11cc0b5c70 0x7f11cc0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:05.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11d2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11cc1527b0 0x7f11cc0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:05.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11d2000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc15f730 con 0x7f11cc1527b0 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11d2000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc15f5b0 con 0x7f11cc0b5c70 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11d2000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc15f8b0 con 0x7f11cc0b4de0 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11cc1527b0 0x7f11cc0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11cc1527b0 0x7f11cc0cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40490/0 (socket says 172.21.15.50:40490) 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11cbe00640 1 -- 172.21.15.50:0/4204677331 learned_addr learned my addr 172.21.15.50:0/4204677331 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11cb400640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11cc0b4de0 0x7f11cc0ab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11c2a00640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11cc0b5c70 0x7f11cc0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11cbe00640 1 -- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11cc0b4de0 msgr2=0x7f11cc0ab290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11cbe00640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11cc0b4de0 0x7f11cc0ab290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11cbe00640 1 -- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11cc0b5c70 msgr2=0x7f11cc0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11cbe00640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11cc0b5c70 0x7f11cc0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11cbe00640 1 -- 172.21.15.50:0/4204677331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc0cfed0 con 0x7f11cc1527b0 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11cbe00640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11cc1527b0 0x7f11cc0cd9a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f11c401b370 tx=0x7f11c40092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:05.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11c3e00640 1 -- 172.21.15.50:0/4204677331 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c40bb070 con 0x7f11cc1527b0 2024-09-21T06:06:05.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11c3e00640 1 -- 172.21.15.50:0/4204677331 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11c4004d40 con 0x7f11cc1527b0 2024-09-21T06:06:05.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11c3e00640 1 -- 172.21.15.50:0/4204677331 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c40b2040 con 0x7f11cc1527b0 2024-09-21T06:06:05.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11d2000640 1 -- 172.21.15.50:0/4204677331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc0aa4d0 con 0x7f11cc1527b0 2024-09-21T06:06:05.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.666+0000 7f11d2000640 1 -- 172.21.15.50:0/4204677331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc0aaa90 con 0x7f11cc1527b0 2024-09-21T06:06:05.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.670+0000 7f11d2000640 1 -- 172.21.15.50:0/4204677331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11cc0cfc40 con 0x7f11cc1527b0 2024-09-21T06:06:05.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.670+0000 7f11c3e00640 1 -- 172.21.15.50:0/4204677331 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f11c4004200 con 0x7f11cc1527b0 2024-09-21T06:06:05.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.670+0000 7f11c3e00640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f11a8074370 0x7f11a8076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:05.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.670+0000 7f11c3e00640 1 -- 172.21.15.50:0/4204677331 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5665+0+0 (secure 0 0 0) 0x7f11c412d3e0 con 0x7f11cc1527b0 2024-09-21T06:06:05.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.670+0000 7f11cb400640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f11a8074370 0x7f11a8076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:05.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.674+0000 7f11cb400640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f11a8074370 0x7f11a8076880 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00bad0 tx=0x7f11bc00b260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.674+0000 7f11c3e00640 1 -- 172.21.15.50:0/4204677331 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f11c40c4020 con 0x7f11cc1527b0 2024-09-21T06:06:05.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:05.886+0000 7f11d2000640 1 -- 172.21.15.50:0/4204677331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f11cc15fdd0 con 0x7f11cc1527b0 2024-09-21T06:06:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.454+0000 7f11c3e00640 1 -- 172.21.15.50:0/4204677331 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v51) ==== 174+0+0 (secure 0 0 0) 0x7f11c4016970 con 0x7f11cc1527b0 2024-09-21T06:06:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 12 and data pool 13 2024-09-21T06:06:06.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.458+0000 7f11d2000640 1 -- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f11a8074370 msgr2=0x7f11a8076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.458+0000 7f11d2000640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f11a8074370 0x7f11a8076880 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00bad0 tx=0x7f11bc00b260 comp rx=0 tx=0).stop 2024-09-21T06:06:06.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.458+0000 7f11d2000640 1 -- 172.21.15.50:0/4204677331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11cc1527b0 msgr2=0x7f11cc0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.458+0000 7f11d2000640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11cc1527b0 0x7f11cc0cd9a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f11c401b370 tx=0x7f11c40092c0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.458+0000 7f11d2000640 1 -- 172.21.15.50:0/4204677331 shutdown_connections 2024-09-21T06:06:06.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.458+0000 7f11d2000640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f11a8074370 0x7f11a8076880 secure :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00bad0 tx=0x7f11bc00b260 comp rx=0 tx=0).stop 2024-09-21T06:06:06.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.458+0000 7f11d2000640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11cc1527b0 0x7f11cc0cd9a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.458+0000 7f11d2000640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11cc0b5c70 0x7f11cc0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.458+0000 7f11d2000640 1 --2- 172.21.15.50:0/4204677331 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11cc0b4de0 0x7f11cc0ab290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.458+0000 7f11d2000640 1 -- 172.21.15.50:0/4204677331 >> 172.21.15.50:0/4204677331 conn(0x7f11cc0b2040 msgr2=0x7f11cc0b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:06.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.458+0000 7f11d2000640 1 -- 172.21.15.50:0/4204677331 shutdown_connections 2024-09-21T06:06:06.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.458+0000 7f11d2000640 1 -- 172.21.15.50:0/4204677331 wait complete. 2024-09-21T06:06:06.514 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:06:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.669+0000 7fe89c600640 1 -- 172.21.15.50:0/479936752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8940cff30 msgr2=0x7fe894073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.669+0000 7fe89c600640 1 --2- 172.21.15.50:0/479936752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8940cff30 0x7fe894073e80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe888009840 tx=0x7fe88801c7f0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.669+0000 7fe89c600640 1 -- 172.21.15.50:0/479936752 shutdown_connections 2024-09-21T06:06:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.669+0000 7fe89c600640 1 --2- 172.21.15.50:0/479936752 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe89415c3e0 0x7fe89406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.669+0000 7fe89c600640 1 --2- 172.21.15.50:0/479936752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8940cff30 0x7fe894073e80 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.669+0000 7fe89c600640 1 -- 172.21.15.50:0/479936752 >> 172.21.15.50:0/479936752 conn(0x7fe8940b2c40 msgr2=0x7fe8940b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.669+0000 7fe89c600640 1 -- 172.21.15.50:0/479936752 shutdown_connections 2024-09-21T06:06:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.669+0000 7fe89c600640 1 -- 172.21.15.50:0/479936752 wait complete. 2024-09-21T06:06:06.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.673+0000 7fe89c600640 1 Processor -- start 2024-09-21T06:06:06.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.673+0000 7fe89c600640 1 -- start start 2024-09-21T06:06:06.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.673+0000 7fe89c600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8940743c0 0x7fe89414d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.673+0000 7fe89c600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8940cff30 0x7fe8940ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.673+0000 7fe89c600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe89415c3e0 0x7fe8940ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.673+0000 7fe89c600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe894152480 con 0x7fe8940743c0 2024-09-21T06:06:06.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.673+0000 7fe89c600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe894152300 con 0x7fe89415c3e0 2024-09-21T06:06:06.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.673+0000 7fe89c600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe894152600 con 0x7fe8940cff30 2024-09-21T06:06:06.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.673+0000 7fe899c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8940743c0 0x7fe89414d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.673+0000 7fe899c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8940743c0 0x7fe89414d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52966/0 (socket says 172.21.15.50:52966) 2024-09-21T06:06:06.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.673+0000 7fe899c00640 1 -- 172.21.15.50:0/800136175 learned_addr learned my addr 172.21.15.50:0/800136175 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe899200640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8940cff30 0x7fe8940ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe89a600640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe89415c3e0 0x7fe8940ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe899c00640 1 -- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8940cff30 msgr2=0x7fe8940ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe899c00640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8940cff30 0x7fe8940ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe899c00640 1 -- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe89415c3e0 msgr2=0x7fe8940ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe899c00640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe89415c3e0 0x7fe8940ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe899c00640 1 -- 172.21.15.50:0/800136175 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8940af420 con 0x7fe8940743c0 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe899c00640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8940743c0 0x7fe89414d670 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe888009790 tx=0x7fe8880a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe891600640 1 -- 172.21.15.50:0/800136175 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8880b2040 con 0x7fe8940743c0 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe891600640 1 -- 172.21.15.50:0/800136175 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe888004430 con 0x7fe8940743c0 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe89c600640 1 -- 172.21.15.50:0/800136175 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8940af700 con 0x7fe8940743c0 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe891600640 1 -- 172.21.15.50:0/800136175 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888003820 con 0x7fe8940743c0 2024-09-21T06:06:06.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe89c600640 1 -- 172.21.15.50:0/800136175 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe894151280 con 0x7fe8940743c0 2024-09-21T06:06:06.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe89c600640 1 -- 172.21.15.50:0/800136175 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89415b630 con 0x7fe8940743c0 2024-09-21T06:06:06.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.677+0000 7fe891600640 1 -- 172.21.15.50:0/800136175 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fe8880bd060 con 0x7fe8940743c0 2024-09-21T06:06:06.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.681+0000 7fe891600640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8740743a0 0x7fe8740768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.681+0000 7fe891600640 1 -- 172.21.15.50:0/800136175 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5765+0+0 (secure 0 0 0) 0x7fe88812c660 con 0x7fe8940743c0 2024-09-21T06:06:06.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.681+0000 7fe899200640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8740743a0 0x7fe8740768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.681+0000 7fe899200640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8740743a0 0x7fe8740768b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe89406fe10 tx=0x7fe880009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:06.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.685+0000 7fe891600640 1 -- 172.21.15.50:0/800136175 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe888016410 con 0x7fe8940743c0 2024-09-21T06:06:06.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.861+0000 7fe89c600640 1 -- 172.21.15.50:0/800136175 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8941595c0 con 0x7fe8940743c0 2024-09-21T06:06:06.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.861+0000 7fe891600640 1 -- 172.21.15.50:0/800136175 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16773 (secure 0 0 0) 0x7fe8880f8e10 con 0x7fe8940743c0 2024-09-21T06:06:06.865 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:06.865 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":84,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:06:05.890779+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:06:03.633659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T06:06:04.852705+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:06:06.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.865+0000 7fe89c600640 1 -- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8740743a0 msgr2=0x7fe8740768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.865+0000 7fe89c600640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8740743a0 0x7fe8740768b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe89406fe10 tx=0x7fe880009040 comp rx=0 tx=0).stop 2024-09-21T06:06:06.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.865+0000 7fe89c600640 1 -- 172.21.15.50:0/800136175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8940743c0 msgr2=0x7fe89414d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.865+0000 7fe89c600640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8940743c0 0x7fe89414d670 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe888009790 tx=0x7fe8880a57c0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.865+0000 7fe89c600640 1 -- 172.21.15.50:0/800136175 shutdown_connections 2024-09-21T06:06:06.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.865+0000 7fe89c600640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe89415c3e0 0x7fe8940ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.865+0000 7fe89c600640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8940cff30 0x7fe8940ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.865+0000 7fe89c600640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8740743a0 0x7fe8740768b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.865+0000 7fe89c600640 1 --2- 172.21.15.50:0/800136175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8940743c0 0x7fe89414d670 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.865+0000 7fe89c600640 1 -- 172.21.15.50:0/800136175 >> 172.21.15.50:0/800136175 conn(0x7fe8940b2c40 msgr2=0x7fe89415e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:06.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.865+0000 7fe89c600640 1 -- 172.21.15.50:0/800136175 shutdown_connections 2024-09-21T06:06:06.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:06.865+0000 7fe89c600640 1 -- 172.21.15.50:0/800136175 wait complete. 2024-09-21T06:06:06.888 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:06:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.045+0000 7f48fca00640 1 -- 172.21.15.50:0/853382911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48f81527b0 msgr2=0x7f48f815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.045+0000 7f48fca00640 1 --2- 172.21.15.50:0/853382911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48f81527b0 0x7f48f815bb60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f48f80cca90 tx=0x7f48e001c800 comp rx=0 tx=0).stop 2024-09-21T06:06:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.045+0000 7f48fca00640 1 -- 172.21.15.50:0/853382911 shutdown_connections 2024-09-21T06:06:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.045+0000 7f48fca00640 1 --2- 172.21.15.50:0/853382911 >> v2:172.21.15.88:3300/0 conn(0x7f48f80b4de0 0x7f48f80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.045+0000 7f48fca00640 1 --2- 172.21.15.50:0/853382911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48f81527b0 0x7f48f815bb60 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.045+0000 7f48fca00640 1 -- 172.21.15.50:0/853382911 >> 172.21.15.50:0/853382911 conn(0x7f48f80b2040 msgr2=0x7f48f80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.045+0000 7f48fca00640 1 -- 172.21.15.50:0/853382911 shutdown_connections 2024-09-21T06:06:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.045+0000 7f48fca00640 1 -- 172.21.15.50:0/853382911 wait complete. 2024-09-21T06:06:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.045+0000 7f48fca00640 1 Processor -- start 2024-09-21T06:06:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.045+0000 7f48fca00640 1 -- start start 2024-09-21T06:06:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48fca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48f80b4de0 0x7f48f81fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48fca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48f80b5c70 0x7f48f81f9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48fca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48f81527b0 0x7f48f8153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48fca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f48f8168920 con 0x7f48f80b4de0 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48fca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f48f81687a0 con 0x7f48f80b5c70 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48fca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f48f8168aa0 con 0x7f48f81527b0 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48f5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48f80b4de0 0x7f48f81fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48f5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48f80b4de0 0x7f48f81fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52978/0 (socket says 172.21.15.50:52978) 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48f5e00640 1 -- 172.21.15.50:0/1811221761 learned_addr learned my addr 172.21.15.50:0/1811221761 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48ed400640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48f80b5c70 0x7f48f81f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48f6800640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48f81527b0 0x7f48f8153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48f5e00640 1 -- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48f81527b0 msgr2=0x7f48f8153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48f5e00640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48f81527b0 0x7f48f8153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48f5e00640 1 -- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48f80b5c70 msgr2=0x7f48f81f9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48f5e00640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48f80b5c70 0x7f48f81f9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48f5e00640 1 -- 172.21.15.50:0/1811221761 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f81fc770 con 0x7f48f80b4de0 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48f5e00640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48f80b4de0 0x7f48f81fbf20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f48e800c3c0 tx=0x7f48e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48eea00640 1 -- 172.21.15.50:0/1811221761 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8017070 con 0x7f48f80b4de0 2024-09-21T06:06:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48eea00640 1 -- 172.21.15.50:0/1811221761 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48e80053c0 con 0x7f48f80b4de0 2024-09-21T06:06:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48eea00640 1 -- 172.21.15.50:0/1811221761 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8003860 con 0x7f48f80b4de0 2024-09-21T06:06:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48fca00640 1 -- 172.21.15.50:0/1811221761 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f81fca50 con 0x7f48f80b4de0 2024-09-21T06:06:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48fca00640 1 -- 172.21.15.50:0/1811221761 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f81f8a40 con 0x7f48f80b4de0 2024-09-21T06:06:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.049+0000 7f48fca00640 1 -- 172.21.15.50:0/1811221761 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f80b51f0 con 0x7f48f80b4de0 2024-09-21T06:06:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.053+0000 7f48eea00640 1 -- 172.21.15.50:0/1811221761 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f48e801a070 con 0x7f48f80b4de0 2024-09-21T06:06:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.053+0000 7f48eea00640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f48d4074370 0x7f48d4076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.053+0000 7f48eea00640 1 -- 172.21.15.50:0/1811221761 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5765+0+0 (secure 0 0 0) 0x7f48e8092360 con 0x7f48f80b4de0 2024-09-21T06:06:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.053+0000 7f48ed400640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f48d4074370 0x7f48d4076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.053+0000 7f48ed400640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f48d4074370 0x7f48d4076880 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f48e00af000 tx=0x7f48e00a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:07.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.057+0000 7f48eea00640 1 -- 172.21.15.50:0/1811221761 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f48f80b5730 con 0x7f48f80b4de0 2024-09-21T06:06:07.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.237+0000 7f48fca00640 1 -- 172.21.15.50:0/1811221761 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48f8159ba0 con 0x7f48f80b4de0 2024-09-21T06:06:07.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48eea00640 1 -- 172.21.15.50:0/1811221761 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16773 (secure 0 0 0) 0x7f48e805e4c0 con 0x7f48f80b4de0 2024-09-21T06:06:07.242 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:07.242 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":84,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:06:05.890779+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:06:03.633659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T06:06:04.852705+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:06:07.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48fca00640 1 -- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f48d4074370 msgr2=0x7f48d4076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:07.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48fca00640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f48d4074370 0x7f48d4076880 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f48e00af000 tx=0x7f48e00a7040 comp rx=0 tx=0).stop 2024-09-21T06:06:07.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48fca00640 1 -- 172.21.15.50:0/1811221761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48f80b4de0 msgr2=0x7f48f81fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:07.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48fca00640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48f80b4de0 0x7f48f81fbf20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f48e800c3c0 tx=0x7f48e800c880 comp rx=0 tx=0).stop 2024-09-21T06:06:07.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48fca00640 1 -- 172.21.15.50:0/1811221761 shutdown_connections 2024-09-21T06:06:07.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48fca00640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48f81527b0 0x7f48f8153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:07.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48fca00640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48f80b5c70 0x7f48f81f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:07.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48fca00640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f48d4074370 0x7f48d4076880 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:07.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48fca00640 1 --2- 172.21.15.50:0/1811221761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48f80b4de0 0x7f48f81fbf20 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:07.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48fca00640 1 -- 172.21.15.50:0/1811221761 >> 172.21.15.50:0/1811221761 conn(0x7f48f80b2040 msgr2=0x7f48f815a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:07.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48fca00640 1 -- 172.21.15.50:0/1811221761 shutdown_connections 2024-09-21T06:06:07.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.241+0000 7f48fca00640 1 -- 172.21.15.50:0/1811221761 wait complete. 2024-09-21T06:06:07.266 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:06:07.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.425+0000 7f44ab200640 1 -- 172.21.15.50:0/1153594540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44ac15ebd0 msgr2=0x7f44ac068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:07.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.425+0000 7f44ab200640 1 --2- 172.21.15.50:0/1153594540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44ac15ebd0 0x7f44ac068df0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f4494009900 tx=0x7f449401c8e0 comp rx=0 tx=0).stop 2024-09-21T06:06:07.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.425+0000 7f44ab200640 1 -- 172.21.15.50:0/1153594540 shutdown_connections 2024-09-21T06:06:07.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.425+0000 7f44ab200640 1 --2- 172.21.15.50:0/1153594540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44ac15ebd0 0x7f44ac068df0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:07.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.425+0000 7f44ab200640 1 -- 172.21.15.50:0/1153594540 >> 172.21.15.50:0/1153594540 conn(0x7f44ac0b41f0 msgr2=0x7f44ac0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:07.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.425+0000 7f44ab200640 1 -- 172.21.15.50:0/1153594540 shutdown_connections 2024-09-21T06:06:07.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.425+0000 7f44ab200640 1 -- 172.21.15.50:0/1153594540 wait complete. 2024-09-21T06:06:07.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44ab200640 1 Processor -- start 2024-09-21T06:06:07.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44ab200640 1 -- start start 2024-09-21T06:06:07.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44ab200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ac069330 0x7f44ac1596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:07.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44ab200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44ac075160 0x7f44ac098900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:07.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44ab200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44ac15ebd0 0x7f44ac153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:07.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44ab200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac16db80 con 0x7f44ac15ebd0 2024-09-21T06:06:07.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44ab200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac16da00 con 0x7f44ac069330 2024-09-21T06:06:07.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44ab200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac16dd00 con 0x7f44ac075160 2024-09-21T06:06:07.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44aa800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44ac15ebd0 0x7f44ac153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:07.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44aa800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44ac15ebd0 0x7f44ac153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52996/0 (socket says 172.21.15.50:52996) 2024-09-21T06:06:07.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44aa800640 1 -- 172.21.15.50:0/3795618521 learned_addr learned my addr 172.21.15.50:0/3795618521 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:07.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44a9e00640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ac069330 0x7f44ac1596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:07.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44a1400640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44ac075160 0x7f44ac098900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:07.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44aa800640 1 -- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44ac075160 msgr2=0x7f44ac098900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:07.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44aa800640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44ac075160 0x7f44ac098900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:07.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44aa800640 1 -- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ac069330 msgr2=0x7f44ac1596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:07.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44aa800640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ac069330 0x7f44ac1596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:07.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44aa800640 1 -- 172.21.15.50:0/3795618521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac0cc440 con 0x7f44ac15ebd0 2024-09-21T06:06:07.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44aa800640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44ac15ebd0 0x7f44ac153c00 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f44a400c730 tx=0x7f44a400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:07.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44a2a00640 1 -- 172.21.15.50:0/3795618521 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4003b80 con 0x7f44ac15ebd0 2024-09-21T06:06:07.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44a2a00640 1 -- 172.21.15.50:0/3795618521 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44a4003d40 con 0x7f44ac15ebd0 2024-09-21T06:06:07.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44ab200640 1 -- 172.21.15.50:0/3795618521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac1558b0 con 0x7f44ac15ebd0 2024-09-21T06:06:07.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44a2a00640 1 -- 172.21.15.50:0/3795618521 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a40052d0 con 0x7f44ac15ebd0 2024-09-21T06:06:07.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.429+0000 7f44ab200640 1 -- 172.21.15.50:0/3795618521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac09a8a0 con 0x7f44ac15ebd0 2024-09-21T06:06:07.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.433+0000 7f44ab200640 1 -- 172.21.15.50:0/3795618521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ac09a690 con 0x7f44ac15ebd0 2024-09-21T06:06:07.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.433+0000 7f44a2a00640 1 -- 172.21.15.50:0/3795618521 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f44a4005680 con 0x7f44ac15ebd0 2024-09-21T06:06:07.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.433+0000 7f44a2a00640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4488074370 0x7f4488076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:07.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.433+0000 7f44a2a00640 1 -- 172.21.15.50:0/3795618521 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5765+0+0 (secure 0 0 0) 0x7f44a4092ff0 con 0x7f44ac15ebd0 2024-09-21T06:06:07.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.433+0000 7f44a9e00640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4488074370 0x7f4488076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:07.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.437+0000 7f44a9e00640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4488074370 0x7f4488076880 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f449c007fd0 tx=0x7f449c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:07.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.437+0000 7f44a2a00640 1 -- 172.21.15.50:0/3795618521 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f44ac069740 con 0x7f44ac15ebd0 2024-09-21T06:06:07.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:07.657+0000 7f44ab200640 1 -- 172.21.15.50:0/3795618521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f44ac068df0 con 0x7f44ac15ebd0 2024-09-21T06:06:08.434 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-09-21T06:06:08.433+0000 7fcfaca00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:06:08.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.489+0000 7f44a2a00640 1 -- 172.21.15.50:0/3795618521 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) ==== 118+0+0 (secure 0 0 0) 0x7f44a405f150 con 0x7f44ac15ebd0 2024-09-21T06:06:08.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.493+0000 7f44ab200640 1 -- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4488074370 msgr2=0x7f4488076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:08.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.493+0000 7f44ab200640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4488074370 0x7f4488076880 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f449c007fd0 tx=0x7f449c009040 comp rx=0 tx=0).stop 2024-09-21T06:06:08.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.493+0000 7f44ab200640 1 -- 172.21.15.50:0/3795618521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44ac15ebd0 msgr2=0x7f44ac153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:08.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.493+0000 7f44ab200640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44ac15ebd0 0x7f44ac153c00 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f44a400c730 tx=0x7f44a400cbf0 comp rx=0 tx=0).stop 2024-09-21T06:06:08.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.493+0000 7f44ab200640 1 -- 172.21.15.50:0/3795618521 shutdown_connections 2024-09-21T06:06:08.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.493+0000 7f44ab200640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4488074370 0x7f4488076880 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:08.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.493+0000 7f44ab200640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44ac15ebd0 0x7f44ac153c00 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:08.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.493+0000 7f44ab200640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44ac075160 0x7f44ac098900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:08.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.493+0000 7f44ab200640 1 --2- 172.21.15.50:0/3795618521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ac069330 0x7f44ac1596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:08.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.493+0000 7f44ab200640 1 -- 172.21.15.50:0/3795618521 >> 172.21.15.50:0/3795618521 conn(0x7f44ac0b41f0 msgr2=0x7f44ac0b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:08.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.493+0000 7f44ab200640 1 -- 172.21.15.50:0/3795618521 shutdown_connections 2024-09-21T06:06:08.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.493+0000 7f44ab200640 1 -- 172.21.15.50:0/3795618521 wait complete. 2024-09-21T06:06:08.519 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:06:08.519 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:06:08.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.685+0000 7fc9e6400640 1 -- 172.21.15.50:0/1450356838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e00ce600 msgr2=0x7fc9e0068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:08.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.685+0000 7fc9e6400640 1 --2- 172.21.15.50:0/1450356838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e00ce600 0x7fc9e0068df0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0073ce0 tx=0x7fc9c801c860 comp rx=0 tx=0).stop 2024-09-21T06:06:08.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.685+0000 7fc9e6400640 1 -- 172.21.15.50:0/1450356838 shutdown_connections 2024-09-21T06:06:08.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.685+0000 7fc9e6400640 1 --2- 172.21.15.50:0/1450356838 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc9e0069330 0x7fc9e00b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:08.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.685+0000 7fc9e6400640 1 --2- 172.21.15.50:0/1450356838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e00ce600 0x7fc9e0068df0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:08.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.685+0000 7fc9e6400640 1 --2- 172.21.15.50:0/1450356838 >> v2:172.21.15.88:3300/0 conn(0x7fc9e0075160 0x7fc9e015ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:08.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.685+0000 7fc9e6400640 1 -- 172.21.15.50:0/1450356838 >> 172.21.15.50:0/1450356838 conn(0x7fc9e00b41f0 msgr2=0x7fc9e00b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:08.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.685+0000 7fc9e6400640 1 -- 172.21.15.50:0/1450356838 shutdown_connections 2024-09-21T06:06:08.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.685+0000 7fc9e6400640 1 -- 172.21.15.50:0/1450356838 wait complete. 2024-09-21T06:06:08.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e6400640 1 Processor -- start 2024-09-21T06:06:08.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e6400640 1 -- start start 2024-09-21T06:06:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e6400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc9e0069330 0x7fc9e009a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e6400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0075160 0x7fc9e015d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e6400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e00ce600 0x7fc9e015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e6400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e0162dd0 con 0x7fc9e0075160 2024-09-21T06:06:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e6400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e0162c50 con 0x7fc9e00ce600 2024-09-21T06:06:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e6400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e0162f50 con 0x7fc9e0069330 2024-09-21T06:06:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9d7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0075160 0x7fc9e015d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9d7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0075160 0x7fc9e015d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53004/0 (socket says 172.21.15.50:53004) 2024-09-21T06:06:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9d7e00640 1 -- 172.21.15.50:0/780715462 learned_addr learned my addr 172.21.15.50:0/780715462 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e5000640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc9e0069330 0x7fc9e009a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e5a00640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e00ce600 0x7fc9e015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9d7e00640 1 -- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc9e0069330 msgr2=0x7fc9e009a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9d7e00640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc9e0069330 0x7fc9e009a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9d7e00640 1 -- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e00ce600 msgr2=0x7fc9e015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9d7e00640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e00ce600 0x7fc9e015dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9d7e00640 1 -- 172.21.15.50:0/780715462 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e00cb5c0 con 0x7fc9e0075160 2024-09-21T06:06:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e5a00640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e00ce600 0x7fc9e015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9d7e00640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0075160 0x7fc9e015d980 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80b0040 tx=0x7fc9c8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9dd600640 1 -- 172.21.15.50:0/780715462 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80bb070 con 0x7fc9e0075160 2024-09-21T06:06:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9dd600640 1 -- 172.21.15.50:0/780715462 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9c8004330 con 0x7fc9e0075160 2024-09-21T06:06:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e6400640 1 -- 172.21.15.50:0/780715462 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e01566c0 con 0x7fc9e0075160 2024-09-21T06:06:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9dd600640 1 -- 172.21.15.50:0/780715462 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80b2040 con 0x7fc9e0075160 2024-09-21T06:06:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.689+0000 7fc9e6400640 1 -- 172.21.15.50:0/780715462 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e015ab20 con 0x7fc9e0075160 2024-09-21T06:06:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.693+0000 7fc9e6400640 1 -- 172.21.15.50:0/780715462 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e0163470 con 0x7fc9e0075160 2024-09-21T06:06:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.693+0000 7fc9dd600640 1 -- 172.21.15.50:0/780715462 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fc9c8005290 con 0x7fc9e0075160 2024-09-21T06:06:08.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.693+0000 7fc9dd600640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc9bc074370 0x7fc9bc076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:08.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.693+0000 7fc9dd600640 1 -- 172.21.15.50:0/780715462 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fc9c812c4e0 con 0x7fc9e0075160 2024-09-21T06:06:08.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.693+0000 7fc9e5000640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc9bc074370 0x7fc9bc076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:08.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.697+0000 7fc9e5000640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc9bc074370 0x7fc9bc076880 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009b40 tx=0x7fc9d00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:08.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.697+0000 7fc9dd600640 1 -- 172.21.15.50:0/780715462 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc9c80164b0 con 0x7fc9e0075160 2024-09-21T06:06:08.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:08.913+0000 7fc9e6400640 1 -- 172.21.15.50:0/780715462 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc9e00b7120 con 0x7fc9e0075160 2024-09-21T06:06:09.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9dd600640 1 -- 172.21.15.50:0/780715462 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) ==== 122+0+0 (secure 0 0 0) 0x7fc9c80f8ab0 con 0x7fc9e0075160 2024-09-21T06:06:09.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9e6400640 1 -- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc9bc074370 msgr2=0x7fc9bc076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:09.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9e6400640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc9bc074370 0x7fc9bc076880 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009b40 tx=0x7fc9d00091e0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9e6400640 1 -- 172.21.15.50:0/780715462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0075160 msgr2=0x7fc9e015d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:09.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9e6400640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0075160 0x7fc9e015d980 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80b0040 tx=0x7fc9c8005de0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9e6400640 1 -- 172.21.15.50:0/780715462 shutdown_connections 2024-09-21T06:06:09.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9e6400640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e00ce600 0x7fc9e015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9e6400640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc9bc074370 0x7fc9bc076880 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9e6400640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0075160 0x7fc9e015d980 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9e6400640 1 --2- 172.21.15.50:0/780715462 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc9e0069330 0x7fc9e009a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9e6400640 1 -- 172.21.15.50:0/780715462 >> 172.21.15.50:0/780715462 conn(0x7fc9e00b41f0 msgr2=0x7fc9e0156f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:09.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9e6400640 1 -- 172.21.15.50:0/780715462 shutdown_connections 2024-09-21T06:06:09.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.497+0000 7fc9e6400640 1 -- 172.21.15.50:0/780715462 wait complete. 2024-09-21T06:06:09.525 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:06:09.525 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.685+0000 7f48f0a00640 1 -- 172.21.15.50:0/1778438363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ec1602a0 msgr2=0x7f48ec1606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.685+0000 7f48f0a00640 1 --2- 172.21.15.50:0/1778438363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ec1602a0 0x7f48ec1606b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f48ec160da0 tx=0x7f48d801c8a0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.685+0000 7f48f0a00640 1 -- 172.21.15.50:0/1778438363 shutdown_connections 2024-09-21T06:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.685+0000 7f48f0a00640 1 --2- 172.21.15.50:0/1778438363 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48ec15cd20 0x7f48ec15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.685+0000 7f48f0a00640 1 --2- 172.21.15.50:0/1778438363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ec1602a0 0x7f48ec1606b0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.685+0000 7f48f0a00640 1 --2- 172.21.15.50:0/1778438363 >> v2:172.21.15.88:3300/0 conn(0x7f48ec0cd2e0 0x7f48ec070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.685+0000 7f48f0a00640 1 -- 172.21.15.50:0/1778438363 >> 172.21.15.50:0/1778438363 conn(0x7f48ec0b2c40 msgr2=0x7f48ec0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.685+0000 7f48f0a00640 1 -- 172.21.15.50:0/1778438363 shutdown_connections 2024-09-21T06:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.685+0000 7f48f0a00640 1 -- 172.21.15.50:0/1778438363 wait complete. 2024-09-21T06:06:09.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48f0a00640 1 Processor -- start 2024-09-21T06:06:09.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48f0a00640 1 -- start start 2024-09-21T06:06:09.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48f0a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48ec0cd2e0 0x7f48ec12abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:09.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48f0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ec15cd20 0x7f48ec13e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:09.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48f0a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48ec1602a0 0x7f48ec1fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:09.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48f0a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0cfef0 con 0x7f48ec15cd20 2024-09-21T06:06:09.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48f0a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0cfd70 con 0x7f48ec0cd2e0 2024-09-21T06:06:09.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48f0a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f48ec0d0070 con 0x7f48ec1602a0 2024-09-21T06:06:09.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48e9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ec15cd20 0x7f48ec13e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:09.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48e9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ec15cd20 0x7f48ec13e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53014/0 (socket says 172.21.15.50:53014) 2024-09-21T06:06:09.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48e9400640 1 -- 172.21.15.50:0/4272172209 learned_addr learned my addr 172.21.15.50:0/4272172209 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48e9e00640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48ec0cd2e0 0x7f48ec12abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48ea800640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48ec1602a0 0x7f48ec1fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48e9400640 1 -- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48ec1602a0 msgr2=0x7f48ec1fa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48e9400640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48ec1602a0 0x7f48ec1fa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48e9400640 1 -- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48ec0cd2e0 msgr2=0x7f48ec12abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48e9400640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48ec0cd2e0 0x7f48ec12abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48e9400640 1 -- 172.21.15.50:0/4272172209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec1459c0 con 0x7f48ec15cd20 2024-09-21T06:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.689+0000 7f48e9400640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ec15cd20 0x7f48ec13e480 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f48d801af50 tx=0x7f48d80a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.693+0000 7f48e2000640 1 -- 172.21.15.50:0/4272172209 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d80b2040 con 0x7f48ec15cd20 2024-09-21T06:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.693+0000 7f48e2000640 1 -- 172.21.15.50:0/4272172209 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48d80043b0 con 0x7f48ec15cd20 2024-09-21T06:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.693+0000 7f48f0a00640 1 -- 172.21.15.50:0/4272172209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec12d340 con 0x7f48ec15cd20 2024-09-21T06:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.693+0000 7f48e2000640 1 -- 172.21.15.50:0/4272172209 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8005290 con 0x7f48ec15cd20 2024-09-21T06:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.693+0000 7f48f0a00640 1 -- 172.21.15.50:0/4272172209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec12d900 con 0x7f48ec15cd20 2024-09-21T06:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.693+0000 7f48f0a00640 1 -- 172.21.15.50:0/4272172209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ec15d130 con 0x7f48ec15cd20 2024-09-21T06:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.693+0000 7f48e2000640 1 -- 172.21.15.50:0/4272172209 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f48d8005430 con 0x7f48ec15cd20 2024-09-21T06:06:09.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.693+0000 7f48e2000640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f48cc0743a0 0x7f48cc0768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:09.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.693+0000 7f48e2000640 1 -- 172.21.15.50:0/4272172209 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f48d812cb30 con 0x7f48ec15cd20 2024-09-21T06:06:09.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.693+0000 7f48e9e00640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f48cc0743a0 0x7f48cc0768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.697+0000 7f48e9e00640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f48cc0743a0 0x7f48cc0768b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00bb50 tx=0x7f48dc00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.697+0000 7f48e2000640 1 -- 172.21.15.50:0/4272172209 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f48d80be060 con 0x7f48ec15cd20 2024-09-21T06:06:09.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.909+0000 7f48f0a00640 1 -- 172.21.15.50:0/4272172209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48ec1539f0 con 0x7f48ec15cd20 2024-09-21T06:06:09.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.909+0000 7f48e2000640 1 -- 172.21.15.50:0/4272172209 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5225 (secure 0 0 0) 0x7f48ec1459c0 con 0x7f48ec15cd20 2024-09-21T06:06:09.913 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:09.913 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":55,"btime":"2024-09-21T06:06:09:495753+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3691304879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3691304879},{"type":"v1","addr":"172.21.15.50:6834","nonce":3691304879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1924969859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1924969859},{"type":"v1","addr":"172.21.15.88:6837","nonce":1924969859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3025915209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3025915209},{"type":"v1","addr":"172.21.15.88:6835","nonce":3025915209}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:06:05.890737+0000","modified":"2024-09-21T06:06:08.920852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/1211403675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1211403675},{"type":"v1","addr":"172.21.15.50:6835","nonce":1211403675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7029,"qdb_cluster":[7029]},"id":6}]} 2024-09-21T06:06:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 55 2024-09-21T06:06:09.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.913+0000 7f48f0a00640 1 -- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f48cc0743a0 msgr2=0x7f48cc0768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:09.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.913+0000 7f48f0a00640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f48cc0743a0 0x7f48cc0768b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00bb50 tx=0x7f48dc00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.913+0000 7f48f0a00640 1 -- 172.21.15.50:0/4272172209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ec15cd20 msgr2=0x7f48ec13e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:09.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.913+0000 7f48f0a00640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ec15cd20 0x7f48ec13e480 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f48d801af50 tx=0x7f48d80a5690 comp rx=0 tx=0).stop 2024-09-21T06:06:09.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.913+0000 7f48f0a00640 1 -- 172.21.15.50:0/4272172209 shutdown_connections 2024-09-21T06:06:09.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.913+0000 7f48f0a00640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48ec1602a0 0x7f48ec1fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.913+0000 7f48f0a00640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f48cc0743a0 0x7f48cc0768b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.913+0000 7f48f0a00640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ec15cd20 0x7f48ec13e480 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.913+0000 7f48f0a00640 1 --2- 172.21.15.50:0/4272172209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48ec0cd2e0 0x7f48ec12abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:09.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.913+0000 7f48f0a00640 1 -- 172.21.15.50:0/4272172209 >> 172.21.15.50:0/4272172209 conn(0x7f48ec0b2c40 msgr2=0x7f48ec13f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:09.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.913+0000 7f48f0a00640 1 -- 172.21.15.50:0/4272172209 shutdown_connections 2024-09-21T06:06:09.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:09.913+0000 7f48f0a00640 1 -- 172.21.15.50:0/4272172209 wait complete. 2024-09-21T06:06:09.936 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:06:10.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.093+0000 7f0206000640 1 -- 172.21.15.50:0/33615840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e8003a60 msgr2=0x7f01e8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.093+0000 7f0206000640 1 --2- 172.21.15.50:0/33615840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e8003a60 0x7f01e8003ed0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009b60 tx=0x7f01f401d270 comp rx=0 tx=0).stop 2024-09-21T06:06:10.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.093+0000 7f0206000640 1 -- 172.21.15.50:0/33615840 shutdown_connections 2024-09-21T06:06:10.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.093+0000 7f0206000640 1 --2- 172.21.15.50:0/33615840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e8003a60 0x7f01e8003ed0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.093+0000 7f0206000640 1 -- 172.21.15.50:0/33615840 >> 172.21.15.50:0/33615840 conn(0x7f02000b41f0 msgr2=0x7f02000b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:10.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.097+0000 7f0206000640 1 -- 172.21.15.50:0/33615840 shutdown_connections 2024-09-21T06:06:10.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.097+0000 7f0206000640 1 -- 172.21.15.50:0/33615840 wait complete. 2024-09-21T06:06:10.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.097+0000 7f0206000640 1 Processor -- start 2024-09-21T06:06:10.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.097+0000 7f0206000640 1 -- start start 2024-09-21T06:06:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.097+0000 7f0206000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e8003a60 0x7f02000ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.097+0000 7f0206000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f020015cf20 0x7f020015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.097+0000 7f0206000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f020015e0c0 0x7f02000cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.097+0000 7f0206000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f02000b1d70 con 0x7f020015e0c0 2024-09-21T06:06:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.097+0000 7f0206000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f02000b1bf0 con 0x7f020015cf20 2024-09-21T06:06:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.097+0000 7f0206000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0200074430 con 0x7f01e8003a60 2024-09-21T06:06:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0205600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f020015e0c0 0x7f02000cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:10.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0205600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f020015e0c0 0x7f02000cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53028/0 (socket says 172.21.15.50:53028) 2024-09-21T06:06:10.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0205600640 1 -- 172.21.15.50:0/220860206 learned_addr learned my addr 172.21.15.50:0/220860206 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:10.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0204c00640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e8003a60 0x7f02000ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:10.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f01ffe00640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f020015cf20 0x7f020015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:10.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0205600640 1 -- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e8003a60 msgr2=0x7f02000ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0205600640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e8003a60 0x7f02000ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0205600640 1 -- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f020015cf20 msgr2=0x7f020015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0205600640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f020015cf20 0x7f020015db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0205600640 1 -- 172.21.15.50:0/220860206 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02000764d0 con 0x7f020015e0c0 2024-09-21T06:06:10.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f01ffe00640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f020015cf20 0x7f020015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:10.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0204c00640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e8003a60 0x7f02000ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:10.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0205600640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f020015e0c0 0x7f02000cb830 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f01f8000f90 tx=0x7f01f800dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:10.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f01fcc00640 1 -- 172.21.15.50:0/220860206 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8003d90 con 0x7f020015e0c0 2024-09-21T06:06:10.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0206000640 1 -- 172.21.15.50:0/220860206 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020015e4d0 con 0x7f020015e0c0 2024-09-21T06:06:10.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f01fcc00640 1 -- 172.21.15.50:0/220860206 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01f800b800 con 0x7f020015e0c0 2024-09-21T06:06:10.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f01fcc00640 1 -- 172.21.15.50:0/220860206 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f800bc50 con 0x7f020015e0c0 2024-09-21T06:06:10.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0206000640 1 -- 172.21.15.50:0/220860206 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f020015c160 con 0x7f020015e0c0 2024-09-21T06:06:10.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f0206000640 1 -- 172.21.15.50:0/220860206 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02000ce690 con 0x7f020015e0c0 2024-09-21T06:06:10.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.101+0000 7f01fcc00640 1 -- 172.21.15.50:0/220860206 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f01f8006070 con 0x7f020015e0c0 2024-09-21T06:06:10.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.105+0000 7f01fcc00640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f01e00743a0 0x7f01e00768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:10.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.105+0000 7f01fcc00640 1 -- 172.21.15.50:0/220860206 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f01f8093200 con 0x7f020015e0c0 2024-09-21T06:06:10.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.105+0000 7f0204c00640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f01e00743a0 0x7f01e00768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:10.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.105+0000 7f0204c00640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f01e00743a0 0x7f01e00768b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f01f40ae000 tx=0x7f01f401a010 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:10.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.109+0000 7f01fcc00640 1 -- 172.21.15.50:0/220860206 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f020015e4d0 con 0x7f020015e0c0 2024-09-21T06:06:10.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.285+0000 7f0206000640 1 -- 172.21.15.50:0/220860206 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0200074990 con 0x7f020015e0c0 2024-09-21T06:06:10.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.285+0000 7f01fcc00640 1 -- 172.21.15.50:0/220860206 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16760 (secure 0 0 0) 0x7f01f805f5a0 con 0x7f020015e0c0 2024-09-21T06:06:10.289 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:10.289 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":85,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:06:07.459596+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:06:03.633659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T06:06:04.852705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:06:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.289+0000 7f0206000640 1 -- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f01e00743a0 msgr2=0x7f01e00768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.289+0000 7f0206000640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f01e00743a0 0x7f01e00768b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f01f40ae000 tx=0x7f01f401a010 comp rx=0 tx=0).stop 2024-09-21T06:06:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.289+0000 7f0206000640 1 -- 172.21.15.50:0/220860206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f020015e0c0 msgr2=0x7f02000cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.289+0000 7f0206000640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f020015e0c0 0x7f02000cb830 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f01f8000f90 tx=0x7f01f800dd00 comp rx=0 tx=0).stop 2024-09-21T06:06:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.289+0000 7f0206000640 1 -- 172.21.15.50:0/220860206 shutdown_connections 2024-09-21T06:06:10.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.289+0000 7f0206000640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f01e00743a0 0x7f01e00768b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.289+0000 7f0206000640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f020015e0c0 0x7f02000cb830 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.289+0000 7f0206000640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f020015cf20 0x7f020015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.289+0000 7f0206000640 1 --2- 172.21.15.50:0/220860206 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f01e8003a60 0x7f02000ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.289+0000 7f0206000640 1 -- 172.21.15.50:0/220860206 >> 172.21.15.50:0/220860206 conn(0x7f02000b41f0 msgr2=0x7f02000caee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:10.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.289+0000 7f0206000640 1 -- 172.21.15.50:0/220860206 shutdown_connections 2024-09-21T06:06:10.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.289+0000 7f0206000640 1 -- 172.21.15.50:0/220860206 wait complete. 2024-09-21T06:06:10.312 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:06:10.312 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:06:10.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.469+0000 7f6d92a00640 1 -- 172.21.15.50:0/3083477324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d8c159130 msgr2=0x7f6d8c15bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.469+0000 7f6d92a00640 1 --2- 172.21.15.50:0/3083477324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d8c159130 0x7f6d8c15bb90 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0d14d0 tx=0x7f6d7c01c830 comp rx=0 tx=0).stop 2024-09-21T06:06:10.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.469+0000 7f6d92a00640 1 -- 172.21.15.50:0/3083477324 shutdown_connections 2024-09-21T06:06:10.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.469+0000 7f6d92a00640 1 --2- 172.21.15.50:0/3083477324 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d8c06fc90 0x7f6d8c0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.469+0000 7f6d92a00640 1 --2- 172.21.15.50:0/3083477324 >> v2:172.21.15.88:3300/0 conn(0x7f6d8c06cdf0 0x7f6d8c06d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.469+0000 7f6d92a00640 1 --2- 172.21.15.50:0/3083477324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d8c159130 0x7f6d8c15bb90 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.469+0000 7f6d92a00640 1 -- 172.21.15.50:0/3083477324 >> 172.21.15.50:0/3083477324 conn(0x7f6d8c0b35f0 msgr2=0x7f6d8c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:10.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.469+0000 7f6d92a00640 1 -- 172.21.15.50:0/3083477324 shutdown_connections 2024-09-21T06:06:10.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.469+0000 7f6d92a00640 1 -- 172.21.15.50:0/3083477324 wait complete. 2024-09-21T06:06:10.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.469+0000 7f6d92a00640 1 Processor -- start 2024-09-21T06:06:10.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.469+0000 7f6d92a00640 1 -- start start 2024-09-21T06:06:10.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d92a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d8c06cdf0 0x7f6d8c1f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:10.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d92a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d8c06fc90 0x7f6d8c0d0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:10.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d92a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d8c159130 0x7f6d8c159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:10.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d92a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c15fa10 con 0x7f6d8c06cdf0 2024-09-21T06:06:10.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d92a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c15f890 con 0x7f6d8c06fc90 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d92a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8c15fb90 con 0x7f6d8c159130 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d8be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d8c06cdf0 0x7f6d8c1f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d8be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d8c06cdf0 0x7f6d8c1f8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53046/0 (socket says 172.21.15.50:53046) 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d8be00640 1 -- 172.21.15.50:0/2799511985 learned_addr learned my addr 172.21.15.50:0/2799511985 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d90a00640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d8c159130 0x7f6d8c159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d83400640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d8c06fc90 0x7f6d8c0d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d8be00640 1 -- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d8c159130 msgr2=0x7f6d8c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d8be00640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d8c159130 0x7f6d8c159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d8be00640 1 -- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d8c06fc90 msgr2=0x7f6d8c0d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d8be00640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d8c06fc90 0x7f6d8c0d0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d8be00640 1 -- 172.21.15.50:0/2799511985 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c14fa10 con 0x7f6d8c06cdf0 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d90a00640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d8c159130 0x7f6d8c159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d83400640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d8c06fc90 0x7f6d8c0d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:10.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d8be00640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d8c06cdf0 0x7f6d8c1f8d60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c01aee0 tx=0x7f6d7c0a65d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d88c00640 1 -- 172.21.15.50:0/2799511985 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c0bd070 con 0x7f6d8c06cdf0 2024-09-21T06:06:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d88c00640 1 -- 172.21.15.50:0/2799511985 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d7c0043f0 con 0x7f6d8c06cdf0 2024-09-21T06:06:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d92a00640 1 -- 172.21.15.50:0/2799511985 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c14fcf0 con 0x7f6d8c06cdf0 2024-09-21T06:06:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d88c00640 1 -- 172.21.15.50:0/2799511985 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c005290 con 0x7f6d8c06cdf0 2024-09-21T06:06:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d92a00640 1 -- 172.21.15.50:0/2799511985 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c1502b0 con 0x7f6d8c06cdf0 2024-09-21T06:06:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.473+0000 7f6d92a00640 1 -- 172.21.15.50:0/2799511985 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8c0700a0 con 0x7f6d8c06cdf0 2024-09-21T06:06:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.477+0000 7f6d88c00640 1 -- 172.21.15.50:0/2799511985 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f6d7c0c0070 con 0x7f6d8c06cdf0 2024-09-21T06:06:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.477+0000 7f6d88c00640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d68074370 0x7f6d68076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.477+0000 7f6d88c00640 1 -- 172.21.15.50:0/2799511985 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f6d7c12d0c0 con 0x7f6d8c06cdf0 2024-09-21T06:06:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.477+0000 7f6d83400640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d68074370 0x7f6d68076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.477+0000 7f6d83400640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d68074370 0x7f6d68076880 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0cea70 tx=0x7f6d74009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.481+0000 7f6d88c00640 1 -- 172.21.15.50:0/2799511985 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6d7c0164b0 con 0x7f6d8c06cdf0 2024-09-21T06:06:10.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.657+0000 7f6d92a00640 1 -- 172.21.15.50:0/2799511985 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d8c06b540 con 0x7f6d68074370 2024-09-21T06:06:10.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.665+0000 7f6d88c00640 1 -- 172.21.15.50:0/2799511985 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249412 (secure 0 0 0) 0x7f6d7401c490 con 0x7f6d68074370 2024-09-21T06:06:10.668 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:10.670 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:06:10.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.669+0000 7f6d92a00640 1 -- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d68074370 msgr2=0x7f6d68076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.669+0000 7f6d92a00640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d68074370 0x7f6d68076880 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0cea70 tx=0x7f6d74009500 comp rx=0 tx=0).stop 2024-09-21T06:06:10.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.669+0000 7f6d92a00640 1 -- 172.21.15.50:0/2799511985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d8c06cdf0 msgr2=0x7f6d8c1f8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.669+0000 7f6d92a00640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d8c06cdf0 0x7f6d8c1f8d60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c01aee0 tx=0x7f6d7c0a65d0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.669+0000 7f6d92a00640 1 -- 172.21.15.50:0/2799511985 shutdown_connections 2024-09-21T06:06:10.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.669+0000 7f6d92a00640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d8c159130 0x7f6d8c159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.669+0000 7f6d92a00640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d8c06fc90 0x7f6d8c0d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.669+0000 7f6d92a00640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d68074370 0x7f6d68076880 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.669+0000 7f6d92a00640 1 --2- 172.21.15.50:0/2799511985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d8c06cdf0 0x7f6d8c1f8d60 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.669+0000 7f6d92a00640 1 -- 172.21.15.50:0/2799511985 >> 172.21.15.50:0/2799511985 conn(0x7f6d8c0b35f0 msgr2=0x7f6d8c075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:10.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.669+0000 7f6d92a00640 1 -- 172.21.15.50:0/2799511985 shutdown_connections 2024-09-21T06:06:10.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.669+0000 7f6d92a00640 1 -- 172.21.15.50:0/2799511985 wait complete. 2024-09-21T06:06:10.690 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-21T06:06:09.074622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592084,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":136,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":156,"acting":156,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":452932,"kb_used_data":9212,"kb_used_omap":12,"kb_used_meta":443507,"kb_avail":754521788,"statfs":{"total":773094113280,"available":772630310912,"internally_reserved":0,"allocated":9433088,"data_stored":4675191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":454151728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1716,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.620756"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:05.449321+0000","last_change":"2024-09-21T06:06:05.449321+0000","last_active":"2024-09-21T06:06:05.449321+0000","last_peered":"2024-09-21T06:06:05.449321+0000","last_clean":"2024-09-21T06:06:05.449321+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:05.449321+0000","last_undegraded":"2024-09-21T06:06:05.449321+0000","last_fullsized":"2024-09-21T06:06:05.449321+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.503126+0000","last_change":"2024-09-21T06:06:06.500877+0000","last_active":"2024-09-21T06:06:06.503126+0000","last_peered":"2024-09-21T06:06:06.503126+0000","last_clean":"2024-09-21T06:06:06.503126+0000","last_became_active":"2024-09-21T06:06:06.500461+0000","last_became_peered":"2024-09-21T06:06:06.500461+0000","last_unstale":"2024-09-21T06:06:06.503126+0000","last_undegraded":"2024-09-21T06:06:06.503126+0000","last_fullsized":"2024-09-21T06:06:06.503126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:37.984268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807507+0000","last_change":"2024-09-21T06:06:06.487046+0000","last_active":"2024-09-21T06:06:07.807507+0000","last_peered":"2024-09-21T06:06:07.807507+0000","last_clean":"2024-09-21T06:06:07.807507+0000","last_became_active":"2024-09-21T06:06:06.485377+0000","last_became_peered":"2024-09-21T06:06:06.485377+0000","last_unstale":"2024-09-21T06:06:07.807507+0000","last_undegraded":"2024-09-21T06:06:07.807507+0000","last_fullsized":"2024-09-21T06:06:07.807507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:09.751571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464575+0000","last_change":"2024-09-21T06:06:06.503549+0000","last_active":"2024-09-21T06:06:07.464575+0000","last_peered":"2024-09-21T06:06:07.464575+0000","last_clean":"2024-09-21T06:06:07.464575+0000","last_became_active":"2024-09-21T06:06:06.503067+0000","last_became_peered":"2024-09-21T06:06:06.503067+0000","last_unstale":"2024-09-21T06:06:07.464575+0000","last_undegraded":"2024-09-21T06:06:07.464575+0000","last_fullsized":"2024-09-21T06:06:07.464575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:26.535908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467347+0000","last_change":"2024-09-21T06:06:06.482492+0000","last_active":"2024-09-21T06:06:07.467347+0000","last_peered":"2024-09-21T06:06:07.467347+0000","last_clean":"2024-09-21T06:06:07.467347+0000","last_became_active":"2024-09-21T06:06:06.482149+0000","last_became_peered":"2024-09-21T06:06:06.482149+0000","last_unstale":"2024-09-21T06:06:07.467347+0000","last_undegraded":"2024-09-21T06:06:07.467347+0000","last_fullsized":"2024-09-21T06:06:07.467347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:53.189736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467489+0000","last_change":"2024-09-21T06:06:06.496236+0000","last_active":"2024-09-21T06:06:07.467489+0000","last_peered":"2024-09-21T06:06:07.467489+0000","last_clean":"2024-09-21T06:06:07.467489+0000","last_became_active":"2024-09-21T06:06:06.495865+0000","last_became_peered":"2024-09-21T06:06:06.495865+0000","last_unstale":"2024-09-21T06:06:07.467489+0000","last_undegraded":"2024-09-21T06:06:07.467489+0000","last_fullsized":"2024-09-21T06:06:07.467489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:56.025511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.463935+0000","last_change":"2024-09-21T06:06:06.484412+0000","last_active":"2024-09-21T06:06:07.463935+0000","last_peered":"2024-09-21T06:06:07.463935+0000","last_clean":"2024-09-21T06:06:07.463935+0000","last_became_active":"2024-09-21T06:06:06.484047+0000","last_became_peered":"2024-09-21T06:06:06.484047+0000","last_unstale":"2024-09-21T06:06:07.463935+0000","last_undegraded":"2024-09-21T06:06:07.463935+0000","last_fullsized":"2024-09-21T06:06:07.463935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:48.073876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.466929+0000","last_change":"2024-09-21T06:06:06.491734+0000","last_active":"2024-09-21T06:06:07.466929+0000","last_peered":"2024-09-21T06:06:07.466929+0000","last_clean":"2024-09-21T06:06:07.466929+0000","last_became_active":"2024-09-21T06:06:06.488184+0000","last_became_peered":"2024-09-21T06:06:06.488184+0000","last_unstale":"2024-09-21T06:06:07.466929+0000","last_undegraded":"2024-09-21T06:06:07.466929+0000","last_fullsized":"2024-09-21T06:06:07.466929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:52.578612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467914+0000","last_change":"2024-09-21T06:06:06.491618+0000","last_active":"2024-09-21T06:06:07.467914+0000","last_peered":"2024-09-21T06:06:07.467914+0000","last_clean":"2024-09-21T06:06:07.467914+0000","last_became_active":"2024-09-21T06:06:06.490986+0000","last_became_peered":"2024-09-21T06:06:06.490986+0000","last_unstale":"2024-09-21T06:06:07.467914+0000","last_undegraded":"2024-09-21T06:06:07.467914+0000","last_fullsized":"2024-09-21T06:06:07.467914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:03.153608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:05.449321+0000","last_change":"2024-09-21T06:06:05.449321+0000","last_active":"2024-09-21T06:06:05.449321+0000","last_peered":"2024-09-21T06:06:05.449321+0000","last_clean":"2024-09-21T06:06:05.449321+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:05.449321+0000","last_undegraded":"2024-09-21T06:06:05.449321+0000","last_fullsized":"2024-09-21T06:06:05.449321+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.497903+0000","last_change":"2024-09-21T06:06:06.486032+0000","last_active":"2024-09-21T06:06:06.497903+0000","last_peered":"2024-09-21T06:06:06.497903+0000","last_clean":"2024-09-21T06:06:06.497903+0000","last_became_active":"2024-09-21T06:06:06.485644+0000","last_became_peered":"2024-09-21T06:06:06.485644+0000","last_unstale":"2024-09-21T06:06:06.497903+0000","last_undegraded":"2024-09-21T06:06:06.497903+0000","last_fullsized":"2024-09-21T06:06:06.497903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:34.583606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:05.449321+0000","last_change":"2024-09-21T06:06:05.449321+0000","last_active":"2024-09-21T06:06:05.449321+0000","last_peered":"2024-09-21T06:06:05.449321+0000","last_clean":"2024-09-21T06:06:05.449321+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:05.449321+0000","last_undegraded":"2024-09-21T06:06:05.449321+0000","last_fullsized":"2024-09-21T06:06:05.449321+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807703+0000","last_change":"2024-09-21T06:06:06.481595+0000","last_active":"2024-09-21T06:06:07.807703+0000","last_peered":"2024-09-21T06:06:07.807703+0000","last_clean":"2024-09-21T06:06:07.807703+0000","last_became_active":"2024-09-21T06:06:06.481215+0000","last_became_peered":"2024-09-21T06:06:06.481215+0000","last_unstale":"2024-09-21T06:06:07.807703+0000","last_undegraded":"2024-09-21T06:06:07.807703+0000","last_fullsized":"2024-09-21T06:06:07.807703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:27.583014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464158+0000","last_change":"2024-09-21T06:06:06.502877+0000","last_active":"2024-09-21T06:06:07.464158+0000","last_peered":"2024-09-21T06:06:07.464158+0000","last_clean":"2024-09-21T06:06:07.464158+0000","last_became_active":"2024-09-21T06:06:06.502483+0000","last_became_peered":"2024-09-21T06:06:06.502483+0000","last_unstale":"2024-09-21T06:06:07.464158+0000","last_undegraded":"2024-09-21T06:06:07.464158+0000","last_fullsized":"2024-09-21T06:06:07.464158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:28.522489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808252+0000","last_change":"2024-09-21T06:06:06.480427+0000","last_active":"2024-09-21T06:06:07.808252+0000","last_peered":"2024-09-21T06:06:07.808252+0000","last_clean":"2024-09-21T06:06:07.808252+0000","last_became_active":"2024-09-21T06:06:06.479937+0000","last_became_peered":"2024-09-21T06:06:06.479937+0000","last_unstale":"2024-09-21T06:06:07.808252+0000","last_undegraded":"2024-09-21T06:06:07.808252+0000","last_fullsized":"2024-09-21T06:06:07.808252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:33.706114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.466795+0000","last_change":"2024-09-21T06:06:06.481281+0000","last_active":"2024-09-21T06:06:07.466795+0000","last_peered":"2024-09-21T06:06:07.466795+0000","last_clean":"2024-09-21T06:06:07.466795+0000","last_became_active":"2024-09-21T06:06:06.479451+0000","last_became_peered":"2024-09-21T06:06:06.479451+0000","last_unstale":"2024-09-21T06:06:07.466795+0000","last_undegraded":"2024-09-21T06:06:07.466795+0000","last_fullsized":"2024-09-21T06:06:07.466795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:09.534869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467899+0000","last_change":"2024-09-21T06:06:06.490406+0000","last_active":"2024-09-21T06:06:07.467899+0000","last_peered":"2024-09-21T06:06:07.467899+0000","last_clean":"2024-09-21T06:06:07.467899+0000","last_became_active":"2024-09-21T06:06:06.490067+0000","last_became_peered":"2024-09-21T06:06:06.490067+0000","last_unstale":"2024-09-21T06:06:07.467899+0000","last_undegraded":"2024-09-21T06:06:07.467899+0000","last_fullsized":"2024-09-21T06:06:07.467899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.650255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464945+0000","last_change":"2024-09-21T06:06:06.499507+0000","last_active":"2024-09-21T06:06:07.464945+0000","last_peered":"2024-09-21T06:06:07.464945+0000","last_clean":"2024-09-21T06:06:07.464945+0000","last_became_active":"2024-09-21T06:06:06.499192+0000","last_became_peered":"2024-09-21T06:06:06.499192+0000","last_unstale":"2024-09-21T06:06:07.464945+0000","last_undegraded":"2024-09-21T06:06:07.464945+0000","last_fullsized":"2024-09-21T06:06:07.464945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:10.390301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809337+0000","last_change":"2024-09-21T06:06:06.480694+0000","last_active":"2024-09-21T06:06:07.809337+0000","last_peered":"2024-09-21T06:06:07.809337+0000","last_clean":"2024-09-21T06:06:07.809337+0000","last_became_active":"2024-09-21T06:06:06.480324+0000","last_became_peered":"2024-09-21T06:06:06.480324+0000","last_unstale":"2024-09-21T06:06:07.809337+0000","last_undegraded":"2024-09-21T06:06:07.809337+0000","last_fullsized":"2024-09-21T06:06:07.809337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:18.934480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464737+0000","last_change":"2024-09-21T06:06:06.492181+0000","last_active":"2024-09-21T06:06:07.464737+0000","last_peered":"2024-09-21T06:06:07.464737+0000","last_clean":"2024-09-21T06:06:07.464737+0000","last_became_active":"2024-09-21T06:06:06.491891+0000","last_became_peered":"2024-09-21T06:06:06.491891+0000","last_unstale":"2024-09-21T06:06:07.464737+0000","last_undegraded":"2024-09-21T06:06:07.464737+0000","last_fullsized":"2024-09-21T06:06:07.464737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:02.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464537+0000","last_change":"2024-09-21T06:06:06.492752+0000","last_active":"2024-09-21T06:06:07.464537+0000","last_peered":"2024-09-21T06:06:07.464537+0000","last_clean":"2024-09-21T06:06:07.464537+0000","last_became_active":"2024-09-21T06:06:06.492444+0000","last_became_peered":"2024-09-21T06:06:06.492444+0000","last_unstale":"2024-09-21T06:06:07.464537+0000","last_undegraded":"2024-09-21T06:06:07.464537+0000","last_fullsized":"2024-09-21T06:06:07.464537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:37.850671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.481926+0000","last_change":"2024-09-21T06:06:06.479811+0000","last_active":"2024-09-21T06:06:06.481926+0000","last_peered":"2024-09-21T06:06:06.481926+0000","last_clean":"2024-09-21T06:06:06.481926+0000","last_became_active":"2024-09-21T06:06:06.479512+0000","last_became_peered":"2024-09-21T06:06:06.479512+0000","last_unstale":"2024-09-21T06:06:06.481926+0000","last_undegraded":"2024-09-21T06:06:06.481926+0000","last_fullsized":"2024-09-21T06:06:06.481926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:14.167900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:05.449321+0000","last_change":"2024-09-21T06:06:05.449321+0000","last_active":"2024-09-21T06:06:05.449321+0000","last_peered":"2024-09-21T06:06:05.449321+0000","last_clean":"2024-09-21T06:06:05.449321+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:05.449321+0000","last_undegraded":"2024-09-21T06:06:05.449321+0000","last_fullsized":"2024-09-21T06:06:05.449321+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.463917+0000","last_change":"2024-09-21T06:06:06.490803+0000","last_active":"2024-09-21T06:06:07.463917+0000","last_peered":"2024-09-21T06:06:07.463917+0000","last_clean":"2024-09-21T06:06:07.463917+0000","last_became_active":"2024-09-21T06:06:06.490532+0000","last_became_peered":"2024-09-21T06:06:06.490532+0000","last_unstale":"2024-09-21T06:06:07.463917+0000","last_undegraded":"2024-09-21T06:06:07.463917+0000","last_fullsized":"2024-09-21T06:06:07.463917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:06.839902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.501129+0000","last_change":"2024-09-21T06:06:06.497051+0000","last_active":"2024-09-21T06:06:06.501129+0000","last_peered":"2024-09-21T06:06:06.501129+0000","last_clean":"2024-09-21T06:06:06.501129+0000","last_became_active":"2024-09-21T06:06:06.496735+0000","last_became_peered":"2024-09-21T06:06:06.496735+0000","last_unstale":"2024-09-21T06:06:06.501129+0000","last_undegraded":"2024-09-21T06:06:06.501129+0000","last_fullsized":"2024-09-21T06:06:06.501129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:48.240604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807545+0000","last_change":"2024-09-21T06:06:06.484207+0000","last_active":"2024-09-21T06:06:07.807545+0000","last_peered":"2024-09-21T06:06:07.807545+0000","last_clean":"2024-09-21T06:06:07.807545+0000","last_became_active":"2024-09-21T06:06:06.482308+0000","last_became_peered":"2024-09-21T06:06:06.482308+0000","last_unstale":"2024-09-21T06:06:07.807545+0000","last_undegraded":"2024-09-21T06:06:07.807545+0000","last_fullsized":"2024-09-21T06:06:07.807545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:02:51.670834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467726+0000","last_change":"2024-09-21T06:06:06.490320+0000","last_active":"2024-09-21T06:06:07.467726+0000","last_peered":"2024-09-21T06:06:07.467726+0000","last_clean":"2024-09-21T06:06:07.467726+0000","last_became_active":"2024-09-21T06:06:06.489578+0000","last_became_peered":"2024-09-21T06:06:06.489578+0000","last_unstale":"2024-09-21T06:06:07.467726+0000","last_undegraded":"2024-09-21T06:06:07.467726+0000","last_fullsized":"2024-09-21T06:06:07.467726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:56.025511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:05.449321+0000","last_change":"2024-09-21T06:06:05.449321+0000","last_active":"2024-09-21T06:06:05.449321+0000","last_peered":"2024-09-21T06:06:05.449321+0000","last_clean":"2024-09-21T06:06:05.449321+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:05.449321+0000","last_undegraded":"2024-09-21T06:06:05.449321+0000","last_fullsized":"2024-09-21T06:06:05.449321+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.495283+0000","last_change":"2024-09-21T06:06:06.485833+0000","last_active":"2024-09-21T06:06:06.495283+0000","last_peered":"2024-09-21T06:06:06.495283+0000","last_clean":"2024-09-21T06:06:06.495283+0000","last_became_active":"2024-09-21T06:06:06.485486+0000","last_became_peered":"2024-09-21T06:06:06.485486+0000","last_unstale":"2024-09-21T06:06:06.495283+0000","last_undegraded":"2024-09-21T06:06:06.495283+0000","last_fullsized":"2024-09-21T06:06:06.495283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:56.693436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.501670+0000","last_change":"2024-09-21T06:06:06.498605+0000","last_active":"2024-09-21T06:06:06.501670+0000","last_peered":"2024-09-21T06:06:06.501670+0000","last_clean":"2024-09-21T06:06:06.501670+0000","last_became_active":"2024-09-21T06:06:06.497923+0000","last_became_peered":"2024-09-21T06:06:06.497923+0000","last_unstale":"2024-09-21T06:06:06.501670+0000","last_undegraded":"2024-09-21T06:06:06.501670+0000","last_fullsized":"2024-09-21T06:06:06.501670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:17.364354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.466613+0000","last_change":"2024-09-21T06:06:06.494033+0000","last_active":"2024-09-21T06:06:07.466613+0000","last_peered":"2024-09-21T06:06:07.466613+0000","last_clean":"2024-09-21T06:06:07.466613+0000","last_became_active":"2024-09-21T06:06:06.493680+0000","last_became_peered":"2024-09-21T06:06:06.493680+0000","last_unstale":"2024-09-21T06:06:07.466613+0000","last_undegraded":"2024-09-21T06:06:07.466613+0000","last_fullsized":"2024-09-21T06:06:07.466613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:52.578612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:05.449321+0000","last_change":"2024-09-21T06:06:05.449321+0000","last_active":"2024-09-21T06:06:05.449321+0000","last_peered":"2024-09-21T06:06:05.449321+0000","last_clean":"2024-09-21T06:06:05.449321+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:05.449321+0000","last_undegraded":"2024-09-21T06:06:05.449321+0000","last_fullsized":"2024-09-21T06:06:05.449321+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467914+0000","last_change":"2024-09-21T06:06:06.499984+0000","last_active":"2024-09-21T06:06:07.467914+0000","last_peered":"2024-09-21T06:06:07.467914+0000","last_clean":"2024-09-21T06:06:07.467914+0000","last_became_active":"2024-09-21T06:06:06.499597+0000","last_became_peered":"2024-09-21T06:06:06.499597+0000","last_unstale":"2024-09-21T06:06:07.467914+0000","last_undegraded":"2024-09-21T06:06:07.467914+0000","last_fullsized":"2024-09-21T06:06:07.467914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:03.153608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:04.446388+0000","last_change":"2024-09-21T06:06:04.446388+0000","last_active":"2024-09-21T06:06:04.446388+0000","last_peered":"2024-09-21T06:06:04.446388+0000","last_clean":"2024-09-21T06:06:04.446388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:04.446388+0000","last_undegraded":"2024-09-21T06:06:04.446388+0000","last_fullsized":"2024-09-21T06:06:04.446388+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:05.449321+0000","last_change":"2024-09-21T06:06:05.449321+0000","last_active":"2024-09-21T06:06:05.449321+0000","last_peered":"2024-09-21T06:06:05.449321+0000","last_clean":"2024-09-21T06:06:05.449321+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:05.449321+0000","last_undegraded":"2024-09-21T06:06:05.449321+0000","last_fullsized":"2024-09-21T06:06:05.449321+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.468178+0000","last_change":"2024-09-21T06:06:05.483146+0000","last_active":"2024-09-21T06:06:07.468178+0000","last_peered":"2024-09-21T06:06:07.468178+0000","last_clean":"2024-09-21T06:06:07.468178+0000","last_became_active":"2024-09-21T06:06:05.482652+0000","last_became_peered":"2024-09-21T06:06:05.482652+0000","last_unstale":"2024-09-21T06:06:07.468178+0000","last_undegraded":"2024-09-21T06:06:07.468178+0000","last_fullsized":"2024-09-21T06:06:07.468178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:07.574298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:05.449321+0000","last_change":"2024-09-21T06:06:05.449321+0000","last_active":"2024-09-21T06:06:05.449321+0000","last_peered":"2024-09-21T06:06:05.449321+0000","last_clean":"2024-09-21T06:06:05.449321+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:05.449321+0000","last_undegraded":"2024-09-21T06:06:05.449321+0000","last_fullsized":"2024-09-21T06:06:05.449321+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809273+0000","last_change":"2024-09-21T06:06:05.476004+0000","last_active":"2024-09-21T06:06:07.809273+0000","last_peered":"2024-09-21T06:06:07.809273+0000","last_clean":"2024-09-21T06:06:07.809273+0000","last_became_active":"2024-09-21T06:06:05.475651+0000","last_became_peered":"2024-09-21T06:06:05.475651+0000","last_unstale":"2024-09-21T06:06:07.809273+0000","last_undegraded":"2024-09-21T06:06:07.809273+0000","last_fullsized":"2024-09-21T06:06:07.809273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.863279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807685+0000","last_change":"2024-09-21T06:06:06.493257+0000","last_active":"2024-09-21T06:06:07.807685+0000","last_peered":"2024-09-21T06:06:07.807685+0000","last_clean":"2024-09-21T06:06:07.807685+0000","last_became_active":"2024-09-21T06:06:06.492949+0000","last_became_peered":"2024-09-21T06:06:06.492949+0000","last_unstale":"2024-09-21T06:06:07.807685+0000","last_undegraded":"2024-09-21T06:06:07.807685+0000","last_fullsized":"2024-09-21T06:06:07.807685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:40.715275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808676+0000","last_change":"2024-09-21T06:06:05.477663+0000","last_active":"2024-09-21T06:06:07.808676+0000","last_peered":"2024-09-21T06:06:07.808676+0000","last_clean":"2024-09-21T06:06:07.808676+0000","last_became_active":"2024-09-21T06:06:05.476398+0000","last_became_peered":"2024-09-21T06:06:05.476398+0000","last_unstale":"2024-09-21T06:06:07.808676+0000","last_undegraded":"2024-09-21T06:06:07.808676+0000","last_fullsized":"2024-09-21T06:06:07.808676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:12.882276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.501593+0000","last_change":"2024-09-21T06:06:06.498268+0000","last_active":"2024-09-21T06:06:06.501593+0000","last_peered":"2024-09-21T06:06:06.501593+0000","last_clean":"2024-09-21T06:06:06.501593+0000","last_became_active":"2024-09-21T06:06:06.497552+0000","last_became_peered":"2024-09-21T06:06:06.497552+0000","last_unstale":"2024-09-21T06:06:06.501593+0000","last_undegraded":"2024-09-21T06:06:06.501593+0000","last_fullsized":"2024-09-21T06:06:06.501593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:29.882040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467629+0000","last_change":"2024-09-21T06:06:05.483942+0000","last_active":"2024-09-21T06:06:07.467629+0000","last_peered":"2024-09-21T06:06:07.467629+0000","last_clean":"2024-09-21T06:06:07.467629+0000","last_became_active":"2024-09-21T06:06:05.483448+0000","last_became_peered":"2024-09-21T06:06:05.483448+0000","last_unstale":"2024-09-21T06:06:07.467629+0000","last_undegraded":"2024-09-21T06:06:07.467629+0000","last_fullsized":"2024-09-21T06:06:07.467629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:39.097894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467772+0000","last_change":"2024-09-21T06:06:06.498433+0000","last_active":"2024-09-21T06:06:07.467772+0000","last_peered":"2024-09-21T06:06:07.467772+0000","last_clean":"2024-09-21T06:06:07.467772+0000","last_became_active":"2024-09-21T06:06:06.496844+0000","last_became_peered":"2024-09-21T06:06:06.496844+0000","last_unstale":"2024-09-21T06:06:07.467772+0000","last_undegraded":"2024-09-21T06:06:07.467772+0000","last_fullsized":"2024-09-21T06:06:07.467772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:13.885209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.466662+0000","last_change":"2024-09-21T06:06:05.484136+0000","last_active":"2024-09-21T06:06:07.466662+0000","last_peered":"2024-09-21T06:06:07.466662+0000","last_clean":"2024-09-21T06:06:07.466662+0000","last_became_active":"2024-09-21T06:06:05.483784+0000","last_became_peered":"2024-09-21T06:06:05.483784+0000","last_unstale":"2024-09-21T06:06:07.466662+0000","last_undegraded":"2024-09-21T06:06:07.466662+0000","last_fullsized":"2024-09-21T06:06:07.466662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:26.423218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808164+0000","last_change":"2024-09-21T06:06:06.489834+0000","last_active":"2024-09-21T06:06:07.808164+0000","last_peered":"2024-09-21T06:06:07.808164+0000","last_clean":"2024-09-21T06:06:07.808164+0000","last_became_active":"2024-09-21T06:06:06.489509+0000","last_became_peered":"2024-09-21T06:06:06.489509+0000","last_unstale":"2024-09-21T06:06:07.808164+0000","last_undegraded":"2024-09-21T06:06:07.808164+0000","last_fullsized":"2024-09-21T06:06:07.808164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:50.663244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808033+0000","last_change":"2024-09-21T06:06:05.468846+0000","last_active":"2024-09-21T06:06:07.808033+0000","last_peered":"2024-09-21T06:06:07.808033+0000","last_clean":"2024-09-21T06:06:07.808033+0000","last_became_active":"2024-09-21T06:06:05.468326+0000","last_became_peered":"2024-09-21T06:06:05.468326+0000","last_unstale":"2024-09-21T06:06:07.808033+0000","last_undegraded":"2024-09-21T06:06:07.808033+0000","last_fullsized":"2024-09-21T06:06:07.808033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:55.022578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808472+0000","last_change":"2024-09-21T06:06:06.481080+0000","last_active":"2024-09-21T06:06:07.808472+0000","last_peered":"2024-09-21T06:06:07.808472+0000","last_clean":"2024-09-21T06:06:07.808472+0000","last_became_active":"2024-09-21T06:06:06.480795+0000","last_became_peered":"2024-09-21T06:06:06.480795+0000","last_unstale":"2024-09-21T06:06:07.808472+0000","last_undegraded":"2024-09-21T06:06:07.808472+0000","last_fullsized":"2024-09-21T06:06:07.808472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:49.332796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.467093+0000","last_change":"2024-09-21T06:06:05.483474+0000","last_active":"2024-09-21T06:06:06.467093+0000","last_peered":"2024-09-21T06:06:06.467093+0000","last_clean":"2024-09-21T06:06:06.467093+0000","last_became_active":"2024-09-21T06:06:05.483164+0000","last_became_peered":"2024-09-21T06:06:05.483164+0000","last_unstale":"2024-09-21T06:06:06.467093+0000","last_undegraded":"2024-09-21T06:06:06.467093+0000","last_fullsized":"2024-09-21T06:06:06.467093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:02.829753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809090+0000","last_change":"2024-09-21T06:06:06.483225+0000","last_active":"2024-09-21T06:06:07.809090+0000","last_peered":"2024-09-21T06:06:07.809090+0000","last_clean":"2024-09-21T06:06:07.809090+0000","last_became_active":"2024-09-21T06:06:06.482919+0000","last_became_peered":"2024-09-21T06:06:06.482919+0000","last_unstale":"2024-09-21T06:06:07.809090+0000","last_undegraded":"2024-09-21T06:06:07.809090+0000","last_fullsized":"2024-09-21T06:06:07.809090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:41.231378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:04.446388+0000","last_change":"2024-09-21T06:06:04.446388+0000","last_active":"2024-09-21T06:06:04.446388+0000","last_peered":"2024-09-21T06:06:04.446388+0000","last_clean":"2024-09-21T06:06:04.446388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:04.446388+0000","last_undegraded":"2024-09-21T06:06:04.446388+0000","last_fullsized":"2024-09-21T06:06:04.446388+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467630+0000","last_change":"2024-09-21T06:06:06.491687+0000","last_active":"2024-09-21T06:06:07.467630+0000","last_peered":"2024-09-21T06:06:07.467630+0000","last_clean":"2024-09-21T06:06:07.467630+0000","last_became_active":"2024-09-21T06:06:06.491374+0000","last_became_peered":"2024-09-21T06:06:06.491374+0000","last_unstale":"2024-09-21T06:06:07.467630+0000","last_undegraded":"2024-09-21T06:06:07.467630+0000","last_fullsized":"2024-09-21T06:06:07.467630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.650255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:04.446388+0000","last_change":"2024-09-21T06:06:04.446388+0000","last_active":"2024-09-21T06:06:04.446388+0000","last_peered":"2024-09-21T06:06:04.446388+0000","last_clean":"2024-09-21T06:06:04.446388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:04.446388+0000","last_undegraded":"2024-09-21T06:06:04.446388+0000","last_fullsized":"2024-09-21T06:06:04.446388+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808541+0000","last_change":"2024-09-21T06:06:06.502662+0000","last_active":"2024-09-21T06:06:07.808541+0000","last_peered":"2024-09-21T06:06:07.808541+0000","last_clean":"2024-09-21T06:06:07.808541+0000","last_became_active":"2024-09-21T06:06:06.501378+0000","last_became_peered":"2024-09-21T06:06:06.501378+0000","last_unstale":"2024-09-21T06:06:07.808541+0000","last_undegraded":"2024-09-21T06:06:07.808541+0000","last_fullsized":"2024-09-21T06:06:07.808541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:46.482537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:04.446388+0000","last_change":"2024-09-21T06:06:04.446388+0000","last_active":"2024-09-21T06:06:04.446388+0000","last_peered":"2024-09-21T06:06:04.446388+0000","last_clean":"2024-09-21T06:06:04.446388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:04.446388+0000","last_undegraded":"2024-09-21T06:06:04.446388+0000","last_fullsized":"2024-09-21T06:06:04.446388+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467970+0000","last_change":"2024-09-21T06:06:06.496327+0000","last_active":"2024-09-21T06:06:07.467970+0000","last_peered":"2024-09-21T06:06:07.467970+0000","last_clean":"2024-09-21T06:06:07.467970+0000","last_became_active":"2024-09-21T06:06:06.494798+0000","last_became_peered":"2024-09-21T06:06:06.494798+0000","last_unstale":"2024-09-21T06:06:07.467970+0000","last_undegraded":"2024-09-21T06:06:07.467970+0000","last_fullsized":"2024-09-21T06:06:07.467970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:55.866212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467680+0000","last_change":"2024-09-21T06:06:05.482802+0000","last_active":"2024-09-21T06:06:07.467680+0000","last_peered":"2024-09-21T06:06:07.467680+0000","last_clean":"2024-09-21T06:06:07.467680+0000","last_became_active":"2024-09-21T06:06:05.482468+0000","last_became_peered":"2024-09-21T06:06:05.482468+0000","last_unstale":"2024-09-21T06:06:07.467680+0000","last_undegraded":"2024-09-21T06:06:07.467680+0000","last_fullsized":"2024-09-21T06:06:07.467680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:40.202862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809830+0000","last_change":"2024-09-21T06:06:06.489206+0000","last_active":"2024-09-21T06:06:07.809830+0000","last_peered":"2024-09-21T06:06:07.809830+0000","last_clean":"2024-09-21T06:06:07.809830+0000","last_became_active":"2024-09-21T06:06:06.488865+0000","last_became_peered":"2024-09-21T06:06:06.488865+0000","last_unstale":"2024-09-21T06:06:07.809830+0000","last_undegraded":"2024-09-21T06:06:07.809830+0000","last_fullsized":"2024-09-21T06:06:07.809830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:13.837098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.497253+0000","last_change":"2024-09-21T06:06:05.487366+0000","last_active":"2024-09-21T06:06:06.497253+0000","last_peered":"2024-09-21T06:06:06.497253+0000","last_clean":"2024-09-21T06:06:06.497253+0000","last_became_active":"2024-09-21T06:06:05.487114+0000","last_became_peered":"2024-09-21T06:06:05.487114+0000","last_unstale":"2024-09-21T06:06:06.497253+0000","last_undegraded":"2024-09-21T06:06:06.497253+0000","last_fullsized":"2024-09-21T06:06:06.497253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:18.791827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807954+0000","last_change":"2024-09-21T06:06:06.489504+0000","last_active":"2024-09-21T06:06:07.807954+0000","last_peered":"2024-09-21T06:06:07.807954+0000","last_clean":"2024-09-21T06:06:07.807954+0000","last_became_active":"2024-09-21T06:06:06.489173+0000","last_became_peered":"2024-09-21T06:06:06.489173+0000","last_unstale":"2024-09-21T06:06:07.807954+0000","last_undegraded":"2024-09-21T06:06:07.807954+0000","last_fullsized":"2024-09-21T06:06:07.807954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:50.456245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808309+0000","last_change":"2024-09-21T06:06:05.480471+0000","last_active":"2024-09-21T06:06:07.808309+0000","last_peered":"2024-09-21T06:06:07.808309+0000","last_clean":"2024-09-21T06:06:07.808309+0000","last_became_active":"2024-09-21T06:06:05.480151+0000","last_became_peered":"2024-09-21T06:06:05.480151+0000","last_unstale":"2024-09-21T06:06:07.808309+0000","last_undegraded":"2024-09-21T06:06:07.808309+0000","last_fullsized":"2024-09-21T06:06:07.808309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:53.008784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.501884+0000","last_change":"2024-09-21T06:06:06.498633+0000","last_active":"2024-09-21T06:06:06.501884+0000","last_peered":"2024-09-21T06:06:06.501884+0000","last_clean":"2024-09-21T06:06:06.501884+0000","last_became_active":"2024-09-21T06:06:06.498359+0000","last_became_peered":"2024-09-21T06:06:06.498359+0000","last_unstale":"2024-09-21T06:06:06.501884+0000","last_undegraded":"2024-09-21T06:06:06.501884+0000","last_fullsized":"2024-09-21T06:06:06.501884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:20.848345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467139+0000","last_change":"2024-09-21T06:06:05.479196+0000","last_active":"2024-09-21T06:06:07.467139+0000","last_peered":"2024-09-21T06:06:07.467139+0000","last_clean":"2024-09-21T06:06:07.467139+0000","last_became_active":"2024-09-21T06:06:05.478857+0000","last_became_peered":"2024-09-21T06:06:05.478857+0000","last_unstale":"2024-09-21T06:06:07.467139+0000","last_undegraded":"2024-09-21T06:06:07.467139+0000","last_fullsized":"2024-09-21T06:06:07.467139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:12.428211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.495401+0000","last_change":"2024-09-21T06:06:06.486854+0000","last_active":"2024-09-21T06:06:06.495401+0000","last_peered":"2024-09-21T06:06:06.495401+0000","last_clean":"2024-09-21T06:06:06.495401+0000","last_became_active":"2024-09-21T06:06:06.486536+0000","last_became_peered":"2024-09-21T06:06:06.486536+0000","last_unstale":"2024-09-21T06:06:06.495401+0000","last_undegraded":"2024-09-21T06:06:06.495401+0000","last_fullsized":"2024-09-21T06:06:06.495401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:47.530625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807944+0000","last_change":"2024-09-21T06:06:05.471801+0000","last_active":"2024-09-21T06:06:07.807944+0000","last_peered":"2024-09-21T06:06:07.807944+0000","last_clean":"2024-09-21T06:06:07.807944+0000","last_became_active":"2024-09-21T06:06:05.471359+0000","last_became_peered":"2024-09-21T06:06:05.471359+0000","last_unstale":"2024-09-21T06:06:07.807944+0000","last_undegraded":"2024-09-21T06:06:07.807944+0000","last_fullsized":"2024-09-21T06:06:07.807944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:52.186803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807799+0000","last_change":"2024-09-21T06:06:06.487470+0000","last_active":"2024-09-21T06:06:07.807799+0000","last_peered":"2024-09-21T06:06:07.807799+0000","last_clean":"2024-09-21T06:06:07.807799+0000","last_became_active":"2024-09-21T06:06:06.487176+0000","last_became_peered":"2024-09-21T06:06:06.487176+0000","last_unstale":"2024-09-21T06:06:07.807799+0000","last_undegraded":"2024-09-21T06:06:07.807799+0000","last_fullsized":"2024-09-21T06:06:07.807799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:30.279574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:04.446388+0000","last_change":"2024-09-21T06:06:04.446388+0000","last_active":"2024-09-21T06:06:04.446388+0000","last_peered":"2024-09-21T06:06:04.446388+0000","last_clean":"2024-09-21T06:06:04.446388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:04.446388+0000","last_undegraded":"2024-09-21T06:06:04.446388+0000","last_fullsized":"2024-09-21T06:06:04.446388+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:05.449321+0000","last_change":"2024-09-21T06:06:05.449321+0000","last_active":"2024-09-21T06:06:05.449321+0000","last_peered":"2024-09-21T06:06:05.449321+0000","last_clean":"2024-09-21T06:06:05.449321+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:05.449321+0000","last_undegraded":"2024-09-21T06:06:05.449321+0000","last_fullsized":"2024-09-21T06:06:05.449321+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809510+0000","last_change":"2024-09-21T06:06:05.483759+0000","last_active":"2024-09-21T06:06:07.809510+0000","last_peered":"2024-09-21T06:06:07.809510+0000","last_clean":"2024-09-21T06:06:07.809510+0000","last_became_active":"2024-09-21T06:06:05.480097+0000","last_became_peered":"2024-09-21T06:06:05.480097+0000","last_unstale":"2024-09-21T06:06:07.809510+0000","last_undegraded":"2024-09-21T06:06:07.809510+0000","last_fullsized":"2024-09-21T06:06:07.809510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:40.346749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.468319+0000","last_change":"2024-09-21T06:06:06.488128+0000","last_active":"2024-09-21T06:06:07.468319+0000","last_peered":"2024-09-21T06:06:07.468319+0000","last_clean":"2024-09-21T06:06:07.468319+0000","last_became_active":"2024-09-21T06:06:06.487775+0000","last_became_peered":"2024-09-21T06:06:06.487775+0000","last_unstale":"2024-09-21T06:06:07.468319+0000","last_undegraded":"2024-09-21T06:06:07.468319+0000","last_fullsized":"2024-09-21T06:06:07.468319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:09.123021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808792+0000","last_change":"2024-09-21T06:06:05.469921+0000","last_active":"2024-09-21T06:06:07.808792+0000","last_peered":"2024-09-21T06:06:07.808792+0000","last_clean":"2024-09-21T06:06:07.808792+0000","last_became_active":"2024-09-21T06:06:05.469492+0000","last_became_peered":"2024-09-21T06:06:05.469492+0000","last_unstale":"2024-09-21T06:06:07.808792+0000","last_undegraded":"2024-09-21T06:06:07.808792+0000","last_fullsized":"2024-09-21T06:06:07.808792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:10.647321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464338+0000","last_change":"2024-09-21T06:06:06.502840+0000","last_active":"2024-09-21T06:06:07.464338+0000","last_peered":"2024-09-21T06:06:07.464338+0000","last_clean":"2024-09-21T06:06:07.464338+0000","last_became_active":"2024-09-21T06:06:06.502287+0000","last_became_peered":"2024-09-21T06:06:06.502287+0000","last_unstale":"2024-09-21T06:06:07.464338+0000","last_undegraded":"2024-09-21T06:06:07.464338+0000","last_fullsized":"2024-09-21T06:06:07.464338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:15.606903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808698+0000","last_change":"2024-09-21T06:06:05.491217+0000","last_active":"2024-09-21T06:06:07.808698+0000","last_peered":"2024-09-21T06:06:07.808698+0000","last_clean":"2024-09-21T06:06:07.808698+0000","last_became_active":"2024-09-21T06:06:05.490974+0000","last_became_peered":"2024-09-21T06:06:05.490974+0000","last_unstale":"2024-09-21T06:06:07.808698+0000","last_undegraded":"2024-09-21T06:06:07.808698+0000","last_fullsized":"2024-09-21T06:06:07.808698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:07.626683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467220+0000","last_change":"2024-09-21T06:06:06.482651+0000","last_active":"2024-09-21T06:06:07.467220+0000","last_peered":"2024-09-21T06:06:07.467220+0000","last_clean":"2024-09-21T06:06:07.467220+0000","last_became_active":"2024-09-21T06:06:06.481936+0000","last_became_peered":"2024-09-21T06:06:06.481936+0000","last_unstale":"2024-09-21T06:06:07.467220+0000","last_undegraded":"2024-09-21T06:06:07.467220+0000","last_fullsized":"2024-09-21T06:06:07.467220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:09.534869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464302+0000","last_change":"2024-09-21T06:06:05.484732+0000","last_active":"2024-09-21T06:06:07.464302+0000","last_peered":"2024-09-21T06:06:07.464302+0000","last_clean":"2024-09-21T06:06:07.464302+0000","last_became_active":"2024-09-21T06:06:05.484237+0000","last_became_peered":"2024-09-21T06:06:05.484237+0000","last_unstale":"2024-09-21T06:06:07.464302+0000","last_undegraded":"2024-09-21T06:06:07.464302+0000","last_fullsized":"2024-09-21T06:06:07.464302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:45.088459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.463943+0000","last_change":"2024-09-21T06:06:06.490135+0000","last_active":"2024-09-21T06:06:07.463943+0000","last_peered":"2024-09-21T06:06:07.463943+0000","last_clean":"2024-09-21T06:06:07.463943+0000","last_became_active":"2024-09-21T06:06:06.489717+0000","last_became_peered":"2024-09-21T06:06:06.489717+0000","last_unstale":"2024-09-21T06:06:07.463943+0000","last_undegraded":"2024-09-21T06:06:07.463943+0000","last_fullsized":"2024-09-21T06:06:07.463943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:33.513491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.463948+0000","last_change":"2024-09-21T06:06:05.480150+0000","last_active":"2024-09-21T06:06:07.463948+0000","last_peered":"2024-09-21T06:06:07.463948+0000","last_clean":"2024-09-21T06:06:07.463948+0000","last_became_active":"2024-09-21T06:06:05.479612+0000","last_became_peered":"2024-09-21T06:06:05.479612+0000","last_unstale":"2024-09-21T06:06:07.463948+0000","last_undegraded":"2024-09-21T06:06:07.463948+0000","last_fullsized":"2024-09-21T06:06:07.463948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:21.817737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467297+0000","last_change":"2024-09-21T06:06:06.494317+0000","last_active":"2024-09-21T06:06:07.467297+0000","last_peered":"2024-09-21T06:06:07.467297+0000","last_clean":"2024-09-21T06:06:07.467297+0000","last_became_active":"2024-09-21T06:06:06.493079+0000","last_became_peered":"2024-09-21T06:06:06.493079+0000","last_unstale":"2024-09-21T06:06:07.467297+0000","last_undegraded":"2024-09-21T06:06:07.467297+0000","last_fullsized":"2024-09-21T06:06:07.467297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:53.189736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807950+0000","last_change":"2024-09-21T06:06:05.484776+0000","last_active":"2024-09-21T06:06:07.807950+0000","last_peered":"2024-09-21T06:06:07.807950+0000","last_clean":"2024-09-21T06:06:07.807950+0000","last_became_active":"2024-09-21T06:06:05.484459+0000","last_became_peered":"2024-09-21T06:06:05.484459+0000","last_unstale":"2024-09-21T06:06:07.807950+0000","last_undegraded":"2024-09-21T06:06:07.807950+0000","last_fullsized":"2024-09-21T06:06:07.807950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:44.048829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.468371+0000","last_change":"2024-09-21T06:06:06.487876+0000","last_active":"2024-09-21T06:06:07.468371+0000","last_peered":"2024-09-21T06:06:07.468371+0000","last_clean":"2024-09-21T06:06:07.468371+0000","last_became_active":"2024-09-21T06:06:06.486615+0000","last_became_peered":"2024-09-21T06:06:06.486615+0000","last_unstale":"2024-09-21T06:06:07.468371+0000","last_undegraded":"2024-09-21T06:06:07.468371+0000","last_fullsized":"2024-09-21T06:06:07.468371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:15.269467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.495286+0000","last_change":"2024-09-21T06:06:05.490629+0000","last_active":"2024-09-21T06:06:06.495286+0000","last_peered":"2024-09-21T06:06:06.495286+0000","last_clean":"2024-09-21T06:06:06.495286+0000","last_became_active":"2024-09-21T06:06:05.490286+0000","last_became_peered":"2024-09-21T06:06:05.490286+0000","last_unstale":"2024-09-21T06:06:06.495286+0000","last_undegraded":"2024-09-21T06:06:06.495286+0000","last_fullsized":"2024-09-21T06:06:06.495286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:51.319347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808092+0000","last_change":"2024-09-21T06:06:06.487398+0000","last_active":"2024-09-21T06:06:07.808092+0000","last_peered":"2024-09-21T06:06:07.808092+0000","last_clean":"2024-09-21T06:06:07.808092+0000","last_became_active":"2024-09-21T06:06:06.486064+0000","last_became_peered":"2024-09-21T06:06:06.486064+0000","last_unstale":"2024-09-21T06:06:07.808092+0000","last_undegraded":"2024-09-21T06:06:07.808092+0000","last_fullsized":"2024-09-21T06:06:07.808092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.877978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.467001+0000","last_change":"2024-09-21T06:06:05.489211+0000","last_active":"2024-09-21T06:06:06.467001+0000","last_peered":"2024-09-21T06:06:06.467001+0000","last_clean":"2024-09-21T06:06:06.467001+0000","last_became_active":"2024-09-21T06:06:05.488870+0000","last_became_peered":"2024-09-21T06:06:05.488870+0000","last_unstale":"2024-09-21T06:06:06.467001+0000","last_undegraded":"2024-09-21T06:06:06.467001+0000","last_fullsized":"2024-09-21T06:06:06.467001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:51.520528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:05.449321+0000","last_change":"2024-09-21T06:06:05.449321+0000","last_active":"2024-09-21T06:06:05.449321+0000","last_peered":"2024-09-21T06:06:05.449321+0000","last_clean":"2024-09-21T06:06:05.449321+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:05.449321+0000","last_undegraded":"2024-09-21T06:06:05.449321+0000","last_fullsized":"2024-09-21T06:06:05.449321+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:04.446388+0000","last_change":"2024-09-21T06:06:04.446388+0000","last_active":"2024-09-21T06:06:04.446388+0000","last_peered":"2024-09-21T06:06:04.446388+0000","last_clean":"2024-09-21T06:06:04.446388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:04.446388+0000","last_undegraded":"2024-09-21T06:06:04.446388+0000","last_fullsized":"2024-09-21T06:06:04.446388+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.468274+0000","last_change":"2024-09-21T06:06:06.494320+0000","last_active":"2024-09-21T06:06:07.468274+0000","last_peered":"2024-09-21T06:06:07.468274+0000","last_clean":"2024-09-21T06:06:07.468274+0000","last_became_active":"2024-09-21T06:06:06.490006+0000","last_became_peered":"2024-09-21T06:06:06.490006+0000","last_unstale":"2024-09-21T06:06:07.468274+0000","last_undegraded":"2024-09-21T06:06:07.468274+0000","last_fullsized":"2024-09-21T06:06:07.468274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:41.349682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:04.446388+0000","last_change":"2024-09-21T06:06:04.446388+0000","last_active":"2024-09-21T06:06:04.446388+0000","last_peered":"2024-09-21T06:06:04.446388+0000","last_clean":"2024-09-21T06:06:04.446388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:04.446388+0000","last_undegraded":"2024-09-21T06:06:04.446388+0000","last_fullsized":"2024-09-21T06:06:04.446388+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467524+0000","last_change":"2024-09-21T06:06:06.481625+0000","last_active":"2024-09-21T06:06:07.467524+0000","last_peered":"2024-09-21T06:06:07.467524+0000","last_clean":"2024-09-21T06:06:07.467524+0000","last_became_active":"2024-09-21T06:06:06.481282+0000","last_became_peered":"2024-09-21T06:06:06.481282+0000","last_unstale":"2024-09-21T06:06:07.467524+0000","last_undegraded":"2024-09-21T06:06:07.467524+0000","last_fullsized":"2024-09-21T06:06:07.467524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:13.885209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808704+0000","last_change":"2024-09-21T06:06:05.473001+0000","last_active":"2024-09-21T06:06:07.808704+0000","last_peered":"2024-09-21T06:06:07.808704+0000","last_clean":"2024-09-21T06:06:07.808704+0000","last_became_active":"2024-09-21T06:06:05.472727+0000","last_became_peered":"2024-09-21T06:06:05.472727+0000","last_unstale":"2024-09-21T06:06:07.808704+0000","last_undegraded":"2024-09-21T06:06:07.808704+0000","last_fullsized":"2024-09-21T06:06:07.808704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:02.150675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808584+0000","last_change":"2024-09-21T06:06:06.491604+0000","last_active":"2024-09-21T06:06:07.808584+0000","last_peered":"2024-09-21T06:06:07.808584+0000","last_clean":"2024-09-21T06:06:07.808584+0000","last_became_active":"2024-09-21T06:06:06.491238+0000","last_became_peered":"2024-09-21T06:06:06.491238+0000","last_unstale":"2024-09-21T06:06:07.808584+0000","last_undegraded":"2024-09-21T06:06:07.808584+0000","last_fullsized":"2024-09-21T06:06:07.808584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:17.926778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.466835+0000","last_change":"2024-09-21T06:06:05.484019+0000","last_active":"2024-09-21T06:06:07.466835+0000","last_peered":"2024-09-21T06:06:07.466835+0000","last_clean":"2024-09-21T06:06:07.466835+0000","last_became_active":"2024-09-21T06:06:05.483668+0000","last_became_peered":"2024-09-21T06:06:05.483668+0000","last_unstale":"2024-09-21T06:06:07.466835+0000","last_undegraded":"2024-09-21T06:06:07.466835+0000","last_fullsized":"2024-09-21T06:06:07.466835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:57.375406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808304+0000","last_change":"2024-09-21T06:06:06.498345+0000","last_active":"2024-09-21T06:06:07.808304+0000","last_peered":"2024-09-21T06:06:07.808304+0000","last_clean":"2024-09-21T06:06:07.808304+0000","last_became_active":"2024-09-21T06:06:06.495657+0000","last_became_peered":"2024-09-21T06:06:06.495657+0000","last_unstale":"2024-09-21T06:06:07.808304+0000","last_undegraded":"2024-09-21T06:06:07.808304+0000","last_fullsized":"2024-09-21T06:06:07.808304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:21.621271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:04.446388+0000","last_change":"2024-09-21T06:06:04.446388+0000","last_active":"2024-09-21T06:06:04.446388+0000","last_peered":"2024-09-21T06:06:04.446388+0000","last_clean":"2024-09-21T06:06:04.446388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:04.446388+0000","last_undegraded":"2024-09-21T06:06:04.446388+0000","last_fullsized":"2024-09-21T06:06:04.446388+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T06:06:05.449321+0000","last_change":"2024-09-21T06:06:05.449321+0000","last_active":"2024-09-21T06:06:05.449321+0000","last_peered":"2024-09-21T06:06:05.449321+0000","last_clean":"2024-09-21T06:06:05.449321+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:06:05.449321+0000","last_undegraded":"2024-09-21T06:06:05.449321+0000","last_fullsized":"2024-09-21T06:06:05.449321+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.466580+0000","last_change":"2024-09-21T06:06:05.485498+0000","last_active":"2024-09-21T06:06:07.466580+0000","last_peered":"2024-09-21T06:06:07.466580+0000","last_clean":"2024-09-21T06:06:07.466580+0000","last_became_active":"2024-09-21T06:06:05.485102+0000","last_became_peered":"2024-09-21T06:06:05.485102+0000","last_unstale":"2024-09-21T06:06:07.466580+0000","last_undegraded":"2024-09-21T06:06:07.466580+0000","last_fullsized":"2024-09-21T06:06:07.466580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:14.106756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.480240+0000","last_change":"2024-09-21T06:06:06.477652+0000","last_active":"2024-09-21T06:06:06.480240+0000","last_peered":"2024-09-21T06:06:06.480240+0000","last_clean":"2024-09-21T06:06:06.480240+0000","last_became_active":"2024-09-21T06:06:06.477385+0000","last_became_peered":"2024-09-21T06:06:06.477385+0000","last_unstale":"2024-09-21T06:06:06.480240+0000","last_undegraded":"2024-09-21T06:06:06.480240+0000","last_fullsized":"2024-09-21T06:06:06.480240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:07.571592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807632+0000","last_change":"2024-09-21T06:06:05.482619+0000","last_active":"2024-09-21T06:06:07.807632+0000","last_peered":"2024-09-21T06:06:07.807632+0000","last_clean":"2024-09-21T06:06:07.807632+0000","last_became_active":"2024-09-21T06:06:05.482257+0000","last_became_peered":"2024-09-21T06:06:05.482257+0000","last_unstale":"2024-09-21T06:06:07.807632+0000","last_undegraded":"2024-09-21T06:06:07.807632+0000","last_fullsized":"2024-09-21T06:06:07.807632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:23.611208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":228,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808293+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:06:07.808293+0000","last_peered":"2024-09-21T06:06:07.808293+0000","last_clean":"2024-09-21T06:06:07.808293+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:06:07.808293+0000","last_undegraded":"2024-09-21T06:06:07.808293+0000","last_fullsized":"2024-09-21T06:06:07.808293+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":1716,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":48,"acting":48,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509561,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46668,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325172,"statfs":{"total":96636764160,"available":96588976128,"internally_reserved":0,"allocated":1622016,"data_stored":1026725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46044,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325796,"statfs":{"total":96636764160,"available":96589615104,"internally_reserved":0,"allocated":987136,"data_stored":417252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509560,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54416,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317424,"statfs":{"total":96636764160,"available":96581042176,"internally_reserved":0,"allocated":1040384,"data_stored":437921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509561,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58572,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313268,"statfs":{"total":96636764160,"available":96576786432,"internally_reserved":0,"allocated":1036288,"data_stored":442643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509560,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68236,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303604,"statfs":{"total":96636764160,"available":96566890496,"internally_reserved":0,"allocated":1036288,"data_stored":437899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509562,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64084,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307756,"statfs":{"total":96636764160,"available":96571142144,"internally_reserved":0,"allocated":1040384,"data_stored":443225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509560,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46672,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325168,"statfs":{"total":96636764160,"available":96588972032,"internally_reserved":0,"allocated":1634304,"data_stored":1033045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542265,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68240,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303600,"statfs":{"total":96636764160,"available":96566886400,"internally_reserved":0,"allocated":1036288,"data_stored":436481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:06:10.694 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:06:10.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.889+0000 7f25f5600640 1 -- 172.21.15.50:0/2986741736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25f01527b0 msgr2=0x7f25f0074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.889+0000 7f25f5600640 1 --2- 172.21.15.50:0/2986741736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25f01527b0 0x7f25f0074e30 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009900 tx=0x7f25d801c820 comp rx=0 tx=0).stop 2024-09-21T06:06:10.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.889+0000 7f25f5600640 1 -- 172.21.15.50:0/2986741736 shutdown_connections 2024-09-21T06:06:10.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.889+0000 7f25f5600640 1 --2- 172.21.15.50:0/2986741736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25f01527b0 0x7f25f0074e30 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.889+0000 7f25f5600640 1 --2- 172.21.15.50:0/2986741736 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25f00b5c70 0x7f25f015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.889+0000 7f25f5600640 1 -- 172.21.15.50:0/2986741736 >> 172.21.15.50:0/2986741736 conn(0x7f25f00b2040 msgr2=0x7f25f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:10.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.889+0000 7f25f5600640 1 -- 172.21.15.50:0/2986741736 shutdown_connections 2024-09-21T06:06:10.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.889+0000 7f25f5600640 1 -- 172.21.15.50:0/2986741736 wait complete. 2024-09-21T06:06:10.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25f5600640 1 Processor -- start 2024-09-21T06:06:10.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25f5600640 1 -- start start 2024-09-21T06:06:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25f5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25f00b4de0 0x7f25f0157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25f5600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f00b5c70 0x7f25f0149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25f5600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25f01527b0 0x7f25f00b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25f5600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f25f0168c50 con 0x7f25f00b4de0 2024-09-21T06:06:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25f5600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f25f0168ad0 con 0x7f25f00b5c70 2024-09-21T06:06:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25f5600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f25f0168dd0 con 0x7f25f01527b0 2024-09-21T06:06:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25eea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25f00b4de0 0x7f25f0157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25eea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25f00b4de0 0x7f25f0157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53054/0 (socket says 172.21.15.50:53054) 2024-09-21T06:06:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25eea00640 1 -- 172.21.15.50:0/3599975040 learned_addr learned my addr 172.21.15.50:0/3599975040 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25e7e00640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f00b5c70 0x7f25f0149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:10.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25ef400640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25f01527b0 0x7f25f00b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:10.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25eea00640 1 -- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25f01527b0 msgr2=0x7f25f00b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25eea00640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25f01527b0 0x7f25f00b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25eea00640 1 -- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f00b5c70 msgr2=0x7f25f0149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:10.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25eea00640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f00b5c70 0x7f25f0149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:10.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25eea00640 1 -- 172.21.15.50:0/3599975040 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f00ce660 con 0x7f25f00b4de0 2024-09-21T06:06:10.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25eea00640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25f00b4de0 0x7f25f0157d50 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e7f0 tx=0x7f25e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:10.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25e6a00640 1 -- 172.21.15.50:0/3599975040 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e000ccc0 con 0x7f25f00b4de0 2024-09-21T06:06:10.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25e6a00640 1 -- 172.21.15.50:0/3599975040 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25e000ce80 con 0x7f25f00b4de0 2024-09-21T06:06:10.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25e6a00640 1 -- 172.21.15.50:0/3599975040 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0005280 con 0x7f25f00b4de0 2024-09-21T06:06:10.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25f5600640 1 -- 172.21.15.50:0/3599975040 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f00ce940 con 0x7f25f00b4de0 2024-09-21T06:06:10.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.893+0000 7f25f5600640 1 -- 172.21.15.50:0/3599975040 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f00cef00 con 0x7f25f00b4de0 2024-09-21T06:06:10.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.897+0000 7f25f5600640 1 -- 172.21.15.50:0/3599975040 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f015bb60 con 0x7f25f00b4de0 2024-09-21T06:06:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.897+0000 7f25e6a00640 1 -- 172.21.15.50:0/3599975040 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f25e0005420 con 0x7f25f00b4de0 2024-09-21T06:06:10.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.897+0000 7f25e6a00640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25cc0743a0 0x7f25cc0768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:10.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.897+0000 7f25e6a00640 1 -- 172.21.15.50:0/3599975040 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f25e0093940 con 0x7f25f00b4de0 2024-09-21T06:06:10.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.897+0000 7f25e7e00640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25cc0743a0 0x7f25cc0768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:10.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.901+0000 7f25e7e00640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25cc0743a0 0x7f25cc0768b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f25d80af000 tx=0x7f25d801b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:10.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:10.901+0000 7f25e6a00640 1 -- 172.21.15.50:0/3599975040 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f25e005faa0 con 0x7f25f00b4de0 2024-09-21T06:06:11.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.077+0000 7f25f5600640 1 -- 172.21.15.50:0/3599975040 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25f006b540 con 0x7f25cc0743a0 2024-09-21T06:06:11.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.081+0000 7f25e6a00640 1 -- 172.21.15.50:0/3599975040 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252071 (secure 0 0 0) 0x7f25d80a6a80 con 0x7f25cc0743a0 2024-09-21T06:06:11.087 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:11.089 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:06:11.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.085+0000 7f25f5600640 1 -- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25cc0743a0 msgr2=0x7f25cc0768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:11.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.085+0000 7f25f5600640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25cc0743a0 0x7f25cc0768b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f25d80af000 tx=0x7f25d801b060 comp rx=0 tx=0).stop 2024-09-21T06:06:11.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.085+0000 7f25f5600640 1 -- 172.21.15.50:0/3599975040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25f00b4de0 msgr2=0x7f25f0157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:11.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.085+0000 7f25f5600640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25f00b4de0 0x7f25f0157d50 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e7f0 tx=0x7f25e000ecb0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.089+0000 7f25f5600640 1 -- 172.21.15.50:0/3599975040 shutdown_connections 2024-09-21T06:06:11.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.089+0000 7f25f5600640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25f01527b0 0x7f25f00b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.089+0000 7f25f5600640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f00b5c70 0x7f25f0149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.089+0000 7f25f5600640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25cc0743a0 0x7f25cc0768b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.089+0000 7f25f5600640 1 --2- 172.21.15.50:0/3599975040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25f00b4de0 0x7f25f0157d50 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.089+0000 7f25f5600640 1 -- 172.21.15.50:0/3599975040 >> 172.21.15.50:0/3599975040 conn(0x7f25f00b2040 msgr2=0x7f25f00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:11.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.089+0000 7f25f5600640 1 -- 172.21.15.50:0/3599975040 shutdown_connections 2024-09-21T06:06:11.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.089+0000 7f25f5600640 1 -- 172.21.15.50:0/3599975040 wait complete. 2024-09-21T06:06:11.109 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-21T06:06:11.075309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":470980,"kb_used_data":9272,"kb_used_omap":12,"kb_used_meta":461491,"kb_avail":754503740,"statfs":{"total":773094113280,"available":772611829760,"internally_reserved":0,"allocated":9494528,"data_stored":4701276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":472567344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001612"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808873+0000","last_change":"2024-09-21T06:06:06.491858+0000","last_active":"2024-09-21T06:06:07.808873+0000","last_peered":"2024-09-21T06:06:07.808873+0000","last_clean":"2024-09-21T06:06:07.808873+0000","last_became_active":"2024-09-21T06:06:06.490924+0000","last_became_peered":"2024-09-21T06:06:06.490924+0000","last_unstale":"2024-09-21T06:06:07.808873+0000","last_undegraded":"2024-09-21T06:06:07.808873+0000","last_fullsized":"2024-09-21T06:06:07.808873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:07.808090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.503126+0000","last_change":"2024-09-21T06:06:06.500877+0000","last_active":"2024-09-21T06:06:06.503126+0000","last_peered":"2024-09-21T06:06:06.503126+0000","last_clean":"2024-09-21T06:06:06.503126+0000","last_became_active":"2024-09-21T06:06:06.500461+0000","last_became_peered":"2024-09-21T06:06:06.500461+0000","last_unstale":"2024-09-21T06:06:06.503126+0000","last_undegraded":"2024-09-21T06:06:06.503126+0000","last_fullsized":"2024-09-21T06:06:06.503126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:37.984268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807507+0000","last_change":"2024-09-21T06:06:06.487046+0000","last_active":"2024-09-21T06:06:07.807507+0000","last_peered":"2024-09-21T06:06:07.807507+0000","last_clean":"2024-09-21T06:06:07.807507+0000","last_became_active":"2024-09-21T06:06:06.485377+0000","last_became_peered":"2024-09-21T06:06:06.485377+0000","last_unstale":"2024-09-21T06:06:07.807507+0000","last_undegraded":"2024-09-21T06:06:07.807507+0000","last_fullsized":"2024-09-21T06:06:07.807507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:09.751571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464575+0000","last_change":"2024-09-21T06:06:06.503549+0000","last_active":"2024-09-21T06:06:07.464575+0000","last_peered":"2024-09-21T06:06:07.464575+0000","last_clean":"2024-09-21T06:06:07.464575+0000","last_became_active":"2024-09-21T06:06:06.503067+0000","last_became_peered":"2024-09-21T06:06:06.503067+0000","last_unstale":"2024-09-21T06:06:07.464575+0000","last_undegraded":"2024-09-21T06:06:07.464575+0000","last_fullsized":"2024-09-21T06:06:07.464575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:26.535908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467347+0000","last_change":"2024-09-21T06:06:06.482492+0000","last_active":"2024-09-21T06:06:07.467347+0000","last_peered":"2024-09-21T06:06:07.467347+0000","last_clean":"2024-09-21T06:06:07.467347+0000","last_became_active":"2024-09-21T06:06:06.482149+0000","last_became_peered":"2024-09-21T06:06:06.482149+0000","last_unstale":"2024-09-21T06:06:07.467347+0000","last_undegraded":"2024-09-21T06:06:07.467347+0000","last_fullsized":"2024-09-21T06:06:07.467347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:53.189736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467489+0000","last_change":"2024-09-21T06:06:06.496236+0000","last_active":"2024-09-21T06:06:07.467489+0000","last_peered":"2024-09-21T06:06:07.467489+0000","last_clean":"2024-09-21T06:06:07.467489+0000","last_became_active":"2024-09-21T06:06:06.495865+0000","last_became_peered":"2024-09-21T06:06:06.495865+0000","last_unstale":"2024-09-21T06:06:07.467489+0000","last_undegraded":"2024-09-21T06:06:07.467489+0000","last_fullsized":"2024-09-21T06:06:07.467489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:56.025511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.463935+0000","last_change":"2024-09-21T06:06:06.484412+0000","last_active":"2024-09-21T06:06:07.463935+0000","last_peered":"2024-09-21T06:06:07.463935+0000","last_clean":"2024-09-21T06:06:07.463935+0000","last_became_active":"2024-09-21T06:06:06.484047+0000","last_became_peered":"2024-09-21T06:06:06.484047+0000","last_unstale":"2024-09-21T06:06:07.463935+0000","last_undegraded":"2024-09-21T06:06:07.463935+0000","last_fullsized":"2024-09-21T06:06:07.463935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:48.073876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.466929+0000","last_change":"2024-09-21T06:06:06.491734+0000","last_active":"2024-09-21T06:06:07.466929+0000","last_peered":"2024-09-21T06:06:07.466929+0000","last_clean":"2024-09-21T06:06:07.466929+0000","last_became_active":"2024-09-21T06:06:06.488184+0000","last_became_peered":"2024-09-21T06:06:06.488184+0000","last_unstale":"2024-09-21T06:06:07.466929+0000","last_undegraded":"2024-09-21T06:06:07.466929+0000","last_fullsized":"2024-09-21T06:06:07.466929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:52.578612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467914+0000","last_change":"2024-09-21T06:06:06.491618+0000","last_active":"2024-09-21T06:06:07.467914+0000","last_peered":"2024-09-21T06:06:07.467914+0000","last_clean":"2024-09-21T06:06:07.467914+0000","last_became_active":"2024-09-21T06:06:06.490986+0000","last_became_peered":"2024-09-21T06:06:06.490986+0000","last_unstale":"2024-09-21T06:06:07.467914+0000","last_undegraded":"2024-09-21T06:06:07.467914+0000","last_fullsized":"2024-09-21T06:06:07.467914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:03.153608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808883+0000","last_change":"2024-09-21T06:06:06.483455+0000","last_active":"2024-09-21T06:06:07.808883+0000","last_peered":"2024-09-21T06:06:07.808883+0000","last_clean":"2024-09-21T06:06:07.808883+0000","last_became_active":"2024-09-21T06:06:06.483130+0000","last_became_peered":"2024-09-21T06:06:06.483130+0000","last_unstale":"2024-09-21T06:06:07.808883+0000","last_undegraded":"2024-09-21T06:06:07.808883+0000","last_fullsized":"2024-09-21T06:06:07.808883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:37.111741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.497903+0000","last_change":"2024-09-21T06:06:06.486032+0000","last_active":"2024-09-21T06:06:06.497903+0000","last_peered":"2024-09-21T06:06:06.497903+0000","last_clean":"2024-09-21T06:06:06.497903+0000","last_became_active":"2024-09-21T06:06:06.485644+0000","last_became_peered":"2024-09-21T06:06:06.485644+0000","last_unstale":"2024-09-21T06:06:06.497903+0000","last_undegraded":"2024-09-21T06:06:06.497903+0000","last_fullsized":"2024-09-21T06:06:06.497903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:34.583606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808504+0000","last_change":"2024-09-21T06:06:06.483634+0000","last_active":"2024-09-21T06:06:07.808504+0000","last_peered":"2024-09-21T06:06:07.808504+0000","last_clean":"2024-09-21T06:06:07.808504+0000","last_became_active":"2024-09-21T06:06:06.483278+0000","last_became_peered":"2024-09-21T06:06:06.483278+0000","last_unstale":"2024-09-21T06:06:07.808504+0000","last_undegraded":"2024-09-21T06:06:07.808504+0000","last_fullsized":"2024-09-21T06:06:07.808504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:32.897434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807703+0000","last_change":"2024-09-21T06:06:06.481595+0000","last_active":"2024-09-21T06:06:07.807703+0000","last_peered":"2024-09-21T06:06:07.807703+0000","last_clean":"2024-09-21T06:06:07.807703+0000","last_became_active":"2024-09-21T06:06:06.481215+0000","last_became_peered":"2024-09-21T06:06:06.481215+0000","last_unstale":"2024-09-21T06:06:07.807703+0000","last_undegraded":"2024-09-21T06:06:07.807703+0000","last_fullsized":"2024-09-21T06:06:07.807703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:27.583014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464158+0000","last_change":"2024-09-21T06:06:06.502877+0000","last_active":"2024-09-21T06:06:07.464158+0000","last_peered":"2024-09-21T06:06:07.464158+0000","last_clean":"2024-09-21T06:06:07.464158+0000","last_became_active":"2024-09-21T06:06:06.502483+0000","last_became_peered":"2024-09-21T06:06:06.502483+0000","last_unstale":"2024-09-21T06:06:07.464158+0000","last_undegraded":"2024-09-21T06:06:07.464158+0000","last_fullsized":"2024-09-21T06:06:07.464158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:28.522489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808252+0000","last_change":"2024-09-21T06:06:06.480427+0000","last_active":"2024-09-21T06:06:07.808252+0000","last_peered":"2024-09-21T06:06:07.808252+0000","last_clean":"2024-09-21T06:06:07.808252+0000","last_became_active":"2024-09-21T06:06:06.479937+0000","last_became_peered":"2024-09-21T06:06:06.479937+0000","last_unstale":"2024-09-21T06:06:07.808252+0000","last_undegraded":"2024-09-21T06:06:07.808252+0000","last_fullsized":"2024-09-21T06:06:07.808252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:33.706114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.466795+0000","last_change":"2024-09-21T06:06:06.481281+0000","last_active":"2024-09-21T06:06:07.466795+0000","last_peered":"2024-09-21T06:06:07.466795+0000","last_clean":"2024-09-21T06:06:07.466795+0000","last_became_active":"2024-09-21T06:06:06.479451+0000","last_became_peered":"2024-09-21T06:06:06.479451+0000","last_unstale":"2024-09-21T06:06:07.466795+0000","last_undegraded":"2024-09-21T06:06:07.466795+0000","last_fullsized":"2024-09-21T06:06:07.466795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:09.534869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467899+0000","last_change":"2024-09-21T06:06:06.490406+0000","last_active":"2024-09-21T06:06:07.467899+0000","last_peered":"2024-09-21T06:06:07.467899+0000","last_clean":"2024-09-21T06:06:07.467899+0000","last_became_active":"2024-09-21T06:06:06.490067+0000","last_became_peered":"2024-09-21T06:06:06.490067+0000","last_unstale":"2024-09-21T06:06:07.467899+0000","last_undegraded":"2024-09-21T06:06:07.467899+0000","last_fullsized":"2024-09-21T06:06:07.467899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.650255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464945+0000","last_change":"2024-09-21T06:06:06.499507+0000","last_active":"2024-09-21T06:06:07.464945+0000","last_peered":"2024-09-21T06:06:07.464945+0000","last_clean":"2024-09-21T06:06:07.464945+0000","last_became_active":"2024-09-21T06:06:06.499192+0000","last_became_peered":"2024-09-21T06:06:06.499192+0000","last_unstale":"2024-09-21T06:06:07.464945+0000","last_undegraded":"2024-09-21T06:06:07.464945+0000","last_fullsized":"2024-09-21T06:06:07.464945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:10.390301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809337+0000","last_change":"2024-09-21T06:06:06.480694+0000","last_active":"2024-09-21T06:06:07.809337+0000","last_peered":"2024-09-21T06:06:07.809337+0000","last_clean":"2024-09-21T06:06:07.809337+0000","last_became_active":"2024-09-21T06:06:06.480324+0000","last_became_peered":"2024-09-21T06:06:06.480324+0000","last_unstale":"2024-09-21T06:06:07.809337+0000","last_undegraded":"2024-09-21T06:06:07.809337+0000","last_fullsized":"2024-09-21T06:06:07.809337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:18.934480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464737+0000","last_change":"2024-09-21T06:06:06.492181+0000","last_active":"2024-09-21T06:06:07.464737+0000","last_peered":"2024-09-21T06:06:07.464737+0000","last_clean":"2024-09-21T06:06:07.464737+0000","last_became_active":"2024-09-21T06:06:06.491891+0000","last_became_peered":"2024-09-21T06:06:06.491891+0000","last_unstale":"2024-09-21T06:06:07.464737+0000","last_undegraded":"2024-09-21T06:06:07.464737+0000","last_fullsized":"2024-09-21T06:06:07.464737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:02.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464537+0000","last_change":"2024-09-21T06:06:06.492752+0000","last_active":"2024-09-21T06:06:07.464537+0000","last_peered":"2024-09-21T06:06:07.464537+0000","last_clean":"2024-09-21T06:06:07.464537+0000","last_became_active":"2024-09-21T06:06:06.492444+0000","last_became_peered":"2024-09-21T06:06:06.492444+0000","last_unstale":"2024-09-21T06:06:07.464537+0000","last_undegraded":"2024-09-21T06:06:07.464537+0000","last_fullsized":"2024-09-21T06:06:07.464537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:37.850671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.481926+0000","last_change":"2024-09-21T06:06:06.479811+0000","last_active":"2024-09-21T06:06:06.481926+0000","last_peered":"2024-09-21T06:06:06.481926+0000","last_clean":"2024-09-21T06:06:06.481926+0000","last_became_active":"2024-09-21T06:06:06.479512+0000","last_became_peered":"2024-09-21T06:06:06.479512+0000","last_unstale":"2024-09-21T06:06:06.481926+0000","last_undegraded":"2024-09-21T06:06:06.481926+0000","last_fullsized":"2024-09-21T06:06:06.481926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:14.167900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807868+0000","last_change":"2024-09-21T06:06:06.498397+0000","last_active":"2024-09-21T06:06:07.807868+0000","last_peered":"2024-09-21T06:06:07.807868+0000","last_clean":"2024-09-21T06:06:07.807868+0000","last_became_active":"2024-09-21T06:06:06.498051+0000","last_became_peered":"2024-09-21T06:06:06.498051+0000","last_unstale":"2024-09-21T06:06:07.807868+0000","last_undegraded":"2024-09-21T06:06:07.807868+0000","last_fullsized":"2024-09-21T06:06:07.807868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:38.910081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.463917+0000","last_change":"2024-09-21T06:06:06.490803+0000","last_active":"2024-09-21T06:06:07.463917+0000","last_peered":"2024-09-21T06:06:07.463917+0000","last_clean":"2024-09-21T06:06:07.463917+0000","last_became_active":"2024-09-21T06:06:06.490532+0000","last_became_peered":"2024-09-21T06:06:06.490532+0000","last_unstale":"2024-09-21T06:06:07.463917+0000","last_undegraded":"2024-09-21T06:06:07.463917+0000","last_fullsized":"2024-09-21T06:06:07.463917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:06.839902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.501129+0000","last_change":"2024-09-21T06:06:06.497051+0000","last_active":"2024-09-21T06:06:06.501129+0000","last_peered":"2024-09-21T06:06:06.501129+0000","last_clean":"2024-09-21T06:06:06.501129+0000","last_became_active":"2024-09-21T06:06:06.496735+0000","last_became_peered":"2024-09-21T06:06:06.496735+0000","last_unstale":"2024-09-21T06:06:06.501129+0000","last_undegraded":"2024-09-21T06:06:06.501129+0000","last_fullsized":"2024-09-21T06:06:06.501129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:48.240604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807545+0000","last_change":"2024-09-21T06:06:06.484207+0000","last_active":"2024-09-21T06:06:07.807545+0000","last_peered":"2024-09-21T06:06:07.807545+0000","last_clean":"2024-09-21T06:06:07.807545+0000","last_became_active":"2024-09-21T06:06:06.482308+0000","last_became_peered":"2024-09-21T06:06:06.482308+0000","last_unstale":"2024-09-21T06:06:07.807545+0000","last_undegraded":"2024-09-21T06:06:07.807545+0000","last_fullsized":"2024-09-21T06:06:07.807545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:02:51.670834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467726+0000","last_change":"2024-09-21T06:06:06.490320+0000","last_active":"2024-09-21T06:06:07.467726+0000","last_peered":"2024-09-21T06:06:07.467726+0000","last_clean":"2024-09-21T06:06:07.467726+0000","last_became_active":"2024-09-21T06:06:06.489578+0000","last_became_peered":"2024-09-21T06:06:06.489578+0000","last_unstale":"2024-09-21T06:06:07.467726+0000","last_undegraded":"2024-09-21T06:06:07.467726+0000","last_fullsized":"2024-09-21T06:06:07.467726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:56.025511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808640+0000","last_change":"2024-09-21T06:06:06.497134+0000","last_active":"2024-09-21T06:06:07.808640+0000","last_peered":"2024-09-21T06:06:07.808640+0000","last_clean":"2024-09-21T06:06:07.808640+0000","last_became_active":"2024-09-21T06:06:06.493404+0000","last_became_peered":"2024-09-21T06:06:06.493404+0000","last_unstale":"2024-09-21T06:06:07.808640+0000","last_undegraded":"2024-09-21T06:06:07.808640+0000","last_fullsized":"2024-09-21T06:06:07.808640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:48.783966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.495283+0000","last_change":"2024-09-21T06:06:06.485833+0000","last_active":"2024-09-21T06:06:06.495283+0000","last_peered":"2024-09-21T06:06:06.495283+0000","last_clean":"2024-09-21T06:06:06.495283+0000","last_became_active":"2024-09-21T06:06:06.485486+0000","last_became_peered":"2024-09-21T06:06:06.485486+0000","last_unstale":"2024-09-21T06:06:06.495283+0000","last_undegraded":"2024-09-21T06:06:06.495283+0000","last_fullsized":"2024-09-21T06:06:06.495283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:56.693436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.501670+0000","last_change":"2024-09-21T06:06:06.498605+0000","last_active":"2024-09-21T06:06:06.501670+0000","last_peered":"2024-09-21T06:06:06.501670+0000","last_clean":"2024-09-21T06:06:06.501670+0000","last_became_active":"2024-09-21T06:06:06.497923+0000","last_became_peered":"2024-09-21T06:06:06.497923+0000","last_unstale":"2024-09-21T06:06:06.501670+0000","last_undegraded":"2024-09-21T06:06:06.501670+0000","last_fullsized":"2024-09-21T06:06:06.501670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:17.364354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.466613+0000","last_change":"2024-09-21T06:06:06.494033+0000","last_active":"2024-09-21T06:06:07.466613+0000","last_peered":"2024-09-21T06:06:07.466613+0000","last_clean":"2024-09-21T06:06:07.466613+0000","last_became_active":"2024-09-21T06:06:06.493680+0000","last_became_peered":"2024-09-21T06:06:06.493680+0000","last_unstale":"2024-09-21T06:06:07.466613+0000","last_undegraded":"2024-09-21T06:06:07.466613+0000","last_fullsized":"2024-09-21T06:06:07.466613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:52.578612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807405+0000","last_change":"2024-09-21T06:06:06.496963+0000","last_active":"2024-09-21T06:06:07.807405+0000","last_peered":"2024-09-21T06:06:07.807405+0000","last_clean":"2024-09-21T06:06:07.807405+0000","last_became_active":"2024-09-21T06:06:06.489504+0000","last_became_peered":"2024-09-21T06:06:06.489504+0000","last_unstale":"2024-09-21T06:06:07.807405+0000","last_undegraded":"2024-09-21T06:06:07.807405+0000","last_fullsized":"2024-09-21T06:06:07.807405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:19.632077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467914+0000","last_change":"2024-09-21T06:06:06.499984+0000","last_active":"2024-09-21T06:06:07.467914+0000","last_peered":"2024-09-21T06:06:07.467914+0000","last_clean":"2024-09-21T06:06:07.467914+0000","last_became_active":"2024-09-21T06:06:06.499597+0000","last_became_peered":"2024-09-21T06:06:06.499597+0000","last_unstale":"2024-09-21T06:06:07.467914+0000","last_undegraded":"2024-09-21T06:06:07.467914+0000","last_fullsized":"2024-09-21T06:06:07.467914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:03.153608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809071+0000","last_change":"2024-09-21T06:06:05.488638+0000","last_active":"2024-09-21T06:06:07.809071+0000","last_peered":"2024-09-21T06:06:07.809071+0000","last_clean":"2024-09-21T06:06:07.809071+0000","last_became_active":"2024-09-21T06:06:05.488274+0000","last_became_peered":"2024-09-21T06:06:05.488274+0000","last_unstale":"2024-09-21T06:06:07.809071+0000","last_undegraded":"2024-09-21T06:06:07.809071+0000","last_fullsized":"2024-09-21T06:06:07.809071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:45.004162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808948+0000","last_change":"2024-09-21T06:06:06.483703+0000","last_active":"2024-09-21T06:06:07.808948+0000","last_peered":"2024-09-21T06:06:07.808948+0000","last_clean":"2024-09-21T06:06:07.808948+0000","last_became_active":"2024-09-21T06:06:06.483326+0000","last_became_peered":"2024-09-21T06:06:06.483326+0000","last_unstale":"2024-09-21T06:06:07.808948+0000","last_undegraded":"2024-09-21T06:06:07.808948+0000","last_fullsized":"2024-09-21T06:06:07.808948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:17.926778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.468178+0000","last_change":"2024-09-21T06:06:05.483146+0000","last_active":"2024-09-21T06:06:07.468178+0000","last_peered":"2024-09-21T06:06:07.468178+0000","last_clean":"2024-09-21T06:06:07.468178+0000","last_became_active":"2024-09-21T06:06:05.482652+0000","last_became_peered":"2024-09-21T06:06:05.482652+0000","last_unstale":"2024-09-21T06:06:07.468178+0000","last_undegraded":"2024-09-21T06:06:07.468178+0000","last_fullsized":"2024-09-21T06:06:07.468178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:07.574298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808807+0000","last_change":"2024-09-21T06:06:06.493760+0000","last_active":"2024-09-21T06:06:07.808807+0000","last_peered":"2024-09-21T06:06:07.808807+0000","last_clean":"2024-09-21T06:06:07.808807+0000","last_became_active":"2024-09-21T06:06:06.490862+0000","last_became_peered":"2024-09-21T06:06:06.490862+0000","last_unstale":"2024-09-21T06:06:07.808807+0000","last_undegraded":"2024-09-21T06:06:07.808807+0000","last_fullsized":"2024-09-21T06:06:07.808807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:29.548861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809273+0000","last_change":"2024-09-21T06:06:05.476004+0000","last_active":"2024-09-21T06:06:07.809273+0000","last_peered":"2024-09-21T06:06:07.809273+0000","last_clean":"2024-09-21T06:06:07.809273+0000","last_became_active":"2024-09-21T06:06:05.475651+0000","last_became_peered":"2024-09-21T06:06:05.475651+0000","last_unstale":"2024-09-21T06:06:07.809273+0000","last_undegraded":"2024-09-21T06:06:07.809273+0000","last_fullsized":"2024-09-21T06:06:07.809273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.863279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807685+0000","last_change":"2024-09-21T06:06:06.493257+0000","last_active":"2024-09-21T06:06:07.807685+0000","last_peered":"2024-09-21T06:06:07.807685+0000","last_clean":"2024-09-21T06:06:07.807685+0000","last_became_active":"2024-09-21T06:06:06.492949+0000","last_became_peered":"2024-09-21T06:06:06.492949+0000","last_unstale":"2024-09-21T06:06:07.807685+0000","last_undegraded":"2024-09-21T06:06:07.807685+0000","last_fullsized":"2024-09-21T06:06:07.807685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:40.715275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808676+0000","last_change":"2024-09-21T06:06:05.477663+0000","last_active":"2024-09-21T06:06:07.808676+0000","last_peered":"2024-09-21T06:06:07.808676+0000","last_clean":"2024-09-21T06:06:07.808676+0000","last_became_active":"2024-09-21T06:06:05.476398+0000","last_became_peered":"2024-09-21T06:06:05.476398+0000","last_unstale":"2024-09-21T06:06:07.808676+0000","last_undegraded":"2024-09-21T06:06:07.808676+0000","last_fullsized":"2024-09-21T06:06:07.808676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:12.882276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.501593+0000","last_change":"2024-09-21T06:06:06.498268+0000","last_active":"2024-09-21T06:06:06.501593+0000","last_peered":"2024-09-21T06:06:06.501593+0000","last_clean":"2024-09-21T06:06:06.501593+0000","last_became_active":"2024-09-21T06:06:06.497552+0000","last_became_peered":"2024-09-21T06:06:06.497552+0000","last_unstale":"2024-09-21T06:06:06.501593+0000","last_undegraded":"2024-09-21T06:06:06.501593+0000","last_fullsized":"2024-09-21T06:06:06.501593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:29.882040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467629+0000","last_change":"2024-09-21T06:06:05.483942+0000","last_active":"2024-09-21T06:06:07.467629+0000","last_peered":"2024-09-21T06:06:07.467629+0000","last_clean":"2024-09-21T06:06:07.467629+0000","last_became_active":"2024-09-21T06:06:05.483448+0000","last_became_peered":"2024-09-21T06:06:05.483448+0000","last_unstale":"2024-09-21T06:06:07.467629+0000","last_undegraded":"2024-09-21T06:06:07.467629+0000","last_fullsized":"2024-09-21T06:06:07.467629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:39.097894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467772+0000","last_change":"2024-09-21T06:06:06.498433+0000","last_active":"2024-09-21T06:06:07.467772+0000","last_peered":"2024-09-21T06:06:07.467772+0000","last_clean":"2024-09-21T06:06:07.467772+0000","last_became_active":"2024-09-21T06:06:06.496844+0000","last_became_peered":"2024-09-21T06:06:06.496844+0000","last_unstale":"2024-09-21T06:06:07.467772+0000","last_undegraded":"2024-09-21T06:06:07.467772+0000","last_fullsized":"2024-09-21T06:06:07.467772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:13.885209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.466662+0000","last_change":"2024-09-21T06:06:05.484136+0000","last_active":"2024-09-21T06:06:07.466662+0000","last_peered":"2024-09-21T06:06:07.466662+0000","last_clean":"2024-09-21T06:06:07.466662+0000","last_became_active":"2024-09-21T06:06:05.483784+0000","last_became_peered":"2024-09-21T06:06:05.483784+0000","last_unstale":"2024-09-21T06:06:07.466662+0000","last_undegraded":"2024-09-21T06:06:07.466662+0000","last_fullsized":"2024-09-21T06:06:07.466662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:26.423218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808164+0000","last_change":"2024-09-21T06:06:06.489834+0000","last_active":"2024-09-21T06:06:07.808164+0000","last_peered":"2024-09-21T06:06:07.808164+0000","last_clean":"2024-09-21T06:06:07.808164+0000","last_became_active":"2024-09-21T06:06:06.489509+0000","last_became_peered":"2024-09-21T06:06:06.489509+0000","last_unstale":"2024-09-21T06:06:07.808164+0000","last_undegraded":"2024-09-21T06:06:07.808164+0000","last_fullsized":"2024-09-21T06:06:07.808164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:50.663244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808033+0000","last_change":"2024-09-21T06:06:05.468846+0000","last_active":"2024-09-21T06:06:07.808033+0000","last_peered":"2024-09-21T06:06:07.808033+0000","last_clean":"2024-09-21T06:06:07.808033+0000","last_became_active":"2024-09-21T06:06:05.468326+0000","last_became_peered":"2024-09-21T06:06:05.468326+0000","last_unstale":"2024-09-21T06:06:07.808033+0000","last_undegraded":"2024-09-21T06:06:07.808033+0000","last_fullsized":"2024-09-21T06:06:07.808033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:55.022578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808472+0000","last_change":"2024-09-21T06:06:06.481080+0000","last_active":"2024-09-21T06:06:07.808472+0000","last_peered":"2024-09-21T06:06:07.808472+0000","last_clean":"2024-09-21T06:06:07.808472+0000","last_became_active":"2024-09-21T06:06:06.480795+0000","last_became_peered":"2024-09-21T06:06:06.480795+0000","last_unstale":"2024-09-21T06:06:07.808472+0000","last_undegraded":"2024-09-21T06:06:07.808472+0000","last_fullsized":"2024-09-21T06:06:07.808472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:49.332796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.467093+0000","last_change":"2024-09-21T06:06:05.483474+0000","last_active":"2024-09-21T06:06:06.467093+0000","last_peered":"2024-09-21T06:06:06.467093+0000","last_clean":"2024-09-21T06:06:06.467093+0000","last_became_active":"2024-09-21T06:06:05.483164+0000","last_became_peered":"2024-09-21T06:06:05.483164+0000","last_unstale":"2024-09-21T06:06:06.467093+0000","last_undegraded":"2024-09-21T06:06:06.467093+0000","last_fullsized":"2024-09-21T06:06:06.467093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:02.829753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809090+0000","last_change":"2024-09-21T06:06:06.483225+0000","last_active":"2024-09-21T06:06:07.809090+0000","last_peered":"2024-09-21T06:06:07.809090+0000","last_clean":"2024-09-21T06:06:07.809090+0000","last_became_active":"2024-09-21T06:06:06.482919+0000","last_became_peered":"2024-09-21T06:06:06.482919+0000","last_unstale":"2024-09-21T06:06:07.809090+0000","last_undegraded":"2024-09-21T06:06:07.809090+0000","last_fullsized":"2024-09-21T06:06:07.809090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:41.231378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809141+0000","last_change":"2024-09-21T06:06:05.482413+0000","last_active":"2024-09-21T06:06:07.809141+0000","last_peered":"2024-09-21T06:06:07.809141+0000","last_clean":"2024-09-21T06:06:07.809141+0000","last_became_active":"2024-09-21T06:06:05.481145+0000","last_became_peered":"2024-09-21T06:06:05.481145+0000","last_unstale":"2024-09-21T06:06:07.809141+0000","last_undegraded":"2024-09-21T06:06:07.809141+0000","last_fullsized":"2024-09-21T06:06:07.809141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:34.659348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467630+0000","last_change":"2024-09-21T06:06:06.491687+0000","last_active":"2024-09-21T06:06:07.467630+0000","last_peered":"2024-09-21T06:06:07.467630+0000","last_clean":"2024-09-21T06:06:07.467630+0000","last_became_active":"2024-09-21T06:06:06.491374+0000","last_became_peered":"2024-09-21T06:06:06.491374+0000","last_unstale":"2024-09-21T06:06:07.467630+0000","last_undegraded":"2024-09-21T06:06:07.467630+0000","last_fullsized":"2024-09-21T06:06:07.467630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.650255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808707+0000","last_change":"2024-09-21T06:06:05.480729+0000","last_active":"2024-09-21T06:06:07.808707+0000","last_peered":"2024-09-21T06:06:07.808707+0000","last_clean":"2024-09-21T06:06:07.808707+0000","last_became_active":"2024-09-21T06:06:05.480392+0000","last_became_peered":"2024-09-21T06:06:05.480392+0000","last_unstale":"2024-09-21T06:06:07.808707+0000","last_undegraded":"2024-09-21T06:06:07.808707+0000","last_fullsized":"2024-09-21T06:06:07.808707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:42.985855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808541+0000","last_change":"2024-09-21T06:06:06.502662+0000","last_active":"2024-09-21T06:06:07.808541+0000","last_peered":"2024-09-21T06:06:07.808541+0000","last_clean":"2024-09-21T06:06:07.808541+0000","last_became_active":"2024-09-21T06:06:06.501378+0000","last_became_peered":"2024-09-21T06:06:06.501378+0000","last_unstale":"2024-09-21T06:06:07.808541+0000","last_undegraded":"2024-09-21T06:06:07.808541+0000","last_fullsized":"2024-09-21T06:06:07.808541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:46.482537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808306+0000","last_change":"2024-09-21T06:06:05.488090+0000","last_active":"2024-09-21T06:06:07.808306+0000","last_peered":"2024-09-21T06:06:07.808306+0000","last_clean":"2024-09-21T06:06:07.808306+0000","last_became_active":"2024-09-21T06:06:05.487715+0000","last_became_peered":"2024-09-21T06:06:05.487715+0000","last_unstale":"2024-09-21T06:06:07.808306+0000","last_undegraded":"2024-09-21T06:06:07.808306+0000","last_fullsized":"2024-09-21T06:06:07.808306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:33.241782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467970+0000","last_change":"2024-09-21T06:06:06.496327+0000","last_active":"2024-09-21T06:06:07.467970+0000","last_peered":"2024-09-21T06:06:07.467970+0000","last_clean":"2024-09-21T06:06:07.467970+0000","last_became_active":"2024-09-21T06:06:06.494798+0000","last_became_peered":"2024-09-21T06:06:06.494798+0000","last_unstale":"2024-09-21T06:06:07.467970+0000","last_undegraded":"2024-09-21T06:06:07.467970+0000","last_fullsized":"2024-09-21T06:06:07.467970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:55.866212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467680+0000","last_change":"2024-09-21T06:06:05.482802+0000","last_active":"2024-09-21T06:06:07.467680+0000","last_peered":"2024-09-21T06:06:07.467680+0000","last_clean":"2024-09-21T06:06:07.467680+0000","last_became_active":"2024-09-21T06:06:05.482468+0000","last_became_peered":"2024-09-21T06:06:05.482468+0000","last_unstale":"2024-09-21T06:06:07.467680+0000","last_undegraded":"2024-09-21T06:06:07.467680+0000","last_fullsized":"2024-09-21T06:06:07.467680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:40.202862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809830+0000","last_change":"2024-09-21T06:06:06.489206+0000","last_active":"2024-09-21T06:06:07.809830+0000","last_peered":"2024-09-21T06:06:07.809830+0000","last_clean":"2024-09-21T06:06:07.809830+0000","last_became_active":"2024-09-21T06:06:06.488865+0000","last_became_peered":"2024-09-21T06:06:06.488865+0000","last_unstale":"2024-09-21T06:06:07.809830+0000","last_undegraded":"2024-09-21T06:06:07.809830+0000","last_fullsized":"2024-09-21T06:06:07.809830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:13.837098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.497253+0000","last_change":"2024-09-21T06:06:05.487366+0000","last_active":"2024-09-21T06:06:06.497253+0000","last_peered":"2024-09-21T06:06:06.497253+0000","last_clean":"2024-09-21T06:06:06.497253+0000","last_became_active":"2024-09-21T06:06:05.487114+0000","last_became_peered":"2024-09-21T06:06:05.487114+0000","last_unstale":"2024-09-21T06:06:06.497253+0000","last_undegraded":"2024-09-21T06:06:06.497253+0000","last_fullsized":"2024-09-21T06:06:06.497253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:18.791827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807954+0000","last_change":"2024-09-21T06:06:06.489504+0000","last_active":"2024-09-21T06:06:07.807954+0000","last_peered":"2024-09-21T06:06:07.807954+0000","last_clean":"2024-09-21T06:06:07.807954+0000","last_became_active":"2024-09-21T06:06:06.489173+0000","last_became_peered":"2024-09-21T06:06:06.489173+0000","last_unstale":"2024-09-21T06:06:07.807954+0000","last_undegraded":"2024-09-21T06:06:07.807954+0000","last_fullsized":"2024-09-21T06:06:07.807954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:50.456245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808309+0000","last_change":"2024-09-21T06:06:05.480471+0000","last_active":"2024-09-21T06:06:07.808309+0000","last_peered":"2024-09-21T06:06:07.808309+0000","last_clean":"2024-09-21T06:06:07.808309+0000","last_became_active":"2024-09-21T06:06:05.480151+0000","last_became_peered":"2024-09-21T06:06:05.480151+0000","last_unstale":"2024-09-21T06:06:07.808309+0000","last_undegraded":"2024-09-21T06:06:07.808309+0000","last_fullsized":"2024-09-21T06:06:07.808309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:53.008784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.501884+0000","last_change":"2024-09-21T06:06:06.498633+0000","last_active":"2024-09-21T06:06:06.501884+0000","last_peered":"2024-09-21T06:06:06.501884+0000","last_clean":"2024-09-21T06:06:06.501884+0000","last_became_active":"2024-09-21T06:06:06.498359+0000","last_became_peered":"2024-09-21T06:06:06.498359+0000","last_unstale":"2024-09-21T06:06:06.501884+0000","last_undegraded":"2024-09-21T06:06:06.501884+0000","last_fullsized":"2024-09-21T06:06:06.501884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:20.848345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467139+0000","last_change":"2024-09-21T06:06:05.479196+0000","last_active":"2024-09-21T06:06:07.467139+0000","last_peered":"2024-09-21T06:06:07.467139+0000","last_clean":"2024-09-21T06:06:07.467139+0000","last_became_active":"2024-09-21T06:06:05.478857+0000","last_became_peered":"2024-09-21T06:06:05.478857+0000","last_unstale":"2024-09-21T06:06:07.467139+0000","last_undegraded":"2024-09-21T06:06:07.467139+0000","last_fullsized":"2024-09-21T06:06:07.467139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:12.428211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.495401+0000","last_change":"2024-09-21T06:06:06.486854+0000","last_active":"2024-09-21T06:06:06.495401+0000","last_peered":"2024-09-21T06:06:06.495401+0000","last_clean":"2024-09-21T06:06:06.495401+0000","last_became_active":"2024-09-21T06:06:06.486536+0000","last_became_peered":"2024-09-21T06:06:06.486536+0000","last_unstale":"2024-09-21T06:06:06.495401+0000","last_undegraded":"2024-09-21T06:06:06.495401+0000","last_fullsized":"2024-09-21T06:06:06.495401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:47.530625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807944+0000","last_change":"2024-09-21T06:06:05.471801+0000","last_active":"2024-09-21T06:06:07.807944+0000","last_peered":"2024-09-21T06:06:07.807944+0000","last_clean":"2024-09-21T06:06:07.807944+0000","last_became_active":"2024-09-21T06:06:05.471359+0000","last_became_peered":"2024-09-21T06:06:05.471359+0000","last_unstale":"2024-09-21T06:06:07.807944+0000","last_undegraded":"2024-09-21T06:06:07.807944+0000","last_fullsized":"2024-09-21T06:06:07.807944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:52.186803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807799+0000","last_change":"2024-09-21T06:06:06.487470+0000","last_active":"2024-09-21T06:06:07.807799+0000","last_peered":"2024-09-21T06:06:07.807799+0000","last_clean":"2024-09-21T06:06:07.807799+0000","last_became_active":"2024-09-21T06:06:06.487176+0000","last_became_peered":"2024-09-21T06:06:06.487176+0000","last_unstale":"2024-09-21T06:06:07.807799+0000","last_undegraded":"2024-09-21T06:06:07.807799+0000","last_fullsized":"2024-09-21T06:06:07.807799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:30.279574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809242+0000","last_change":"2024-09-21T06:06:05.476969+0000","last_active":"2024-09-21T06:06:07.809242+0000","last_peered":"2024-09-21T06:06:07.809242+0000","last_clean":"2024-09-21T06:06:07.809242+0000","last_became_active":"2024-09-21T06:06:05.476469+0000","last_became_peered":"2024-09-21T06:06:05.476469+0000","last_unstale":"2024-09-21T06:06:07.809242+0000","last_undegraded":"2024-09-21T06:06:07.809242+0000","last_fullsized":"2024-09-21T06:06:07.809242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:05.509640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809094+0000","last_change":"2024-09-21T06:06:06.492232+0000","last_active":"2024-09-21T06:06:07.809094+0000","last_peered":"2024-09-21T06:06:07.809094+0000","last_clean":"2024-09-21T06:06:07.809094+0000","last_became_active":"2024-09-21T06:06:06.491367+0000","last_became_peered":"2024-09-21T06:06:06.491367+0000","last_unstale":"2024-09-21T06:06:07.809094+0000","last_undegraded":"2024-09-21T06:06:07.809094+0000","last_fullsized":"2024-09-21T06:06:07.809094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:13.837098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.809510+0000","last_change":"2024-09-21T06:06:05.483759+0000","last_active":"2024-09-21T06:06:07.809510+0000","last_peered":"2024-09-21T06:06:07.809510+0000","last_clean":"2024-09-21T06:06:07.809510+0000","last_became_active":"2024-09-21T06:06:05.480097+0000","last_became_peered":"2024-09-21T06:06:05.480097+0000","last_unstale":"2024-09-21T06:06:07.809510+0000","last_undegraded":"2024-09-21T06:06:07.809510+0000","last_fullsized":"2024-09-21T06:06:07.809510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:40.346749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.468319+0000","last_change":"2024-09-21T06:06:06.488128+0000","last_active":"2024-09-21T06:06:07.468319+0000","last_peered":"2024-09-21T06:06:07.468319+0000","last_clean":"2024-09-21T06:06:07.468319+0000","last_became_active":"2024-09-21T06:06:06.487775+0000","last_became_peered":"2024-09-21T06:06:06.487775+0000","last_unstale":"2024-09-21T06:06:07.468319+0000","last_undegraded":"2024-09-21T06:06:07.468319+0000","last_fullsized":"2024-09-21T06:06:07.468319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:09.123021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808792+0000","last_change":"2024-09-21T06:06:05.469921+0000","last_active":"2024-09-21T06:06:07.808792+0000","last_peered":"2024-09-21T06:06:07.808792+0000","last_clean":"2024-09-21T06:06:07.808792+0000","last_became_active":"2024-09-21T06:06:05.469492+0000","last_became_peered":"2024-09-21T06:06:05.469492+0000","last_unstale":"2024-09-21T06:06:07.808792+0000","last_undegraded":"2024-09-21T06:06:07.808792+0000","last_fullsized":"2024-09-21T06:06:07.808792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:10.647321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464338+0000","last_change":"2024-09-21T06:06:06.502840+0000","last_active":"2024-09-21T06:06:07.464338+0000","last_peered":"2024-09-21T06:06:07.464338+0000","last_clean":"2024-09-21T06:06:07.464338+0000","last_became_active":"2024-09-21T06:06:06.502287+0000","last_became_peered":"2024-09-21T06:06:06.502287+0000","last_unstale":"2024-09-21T06:06:07.464338+0000","last_undegraded":"2024-09-21T06:06:07.464338+0000","last_fullsized":"2024-09-21T06:06:07.464338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:15.606903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808698+0000","last_change":"2024-09-21T06:06:05.491217+0000","last_active":"2024-09-21T06:06:07.808698+0000","last_peered":"2024-09-21T06:06:07.808698+0000","last_clean":"2024-09-21T06:06:07.808698+0000","last_became_active":"2024-09-21T06:06:05.490974+0000","last_became_peered":"2024-09-21T06:06:05.490974+0000","last_unstale":"2024-09-21T06:06:07.808698+0000","last_undegraded":"2024-09-21T06:06:07.808698+0000","last_fullsized":"2024-09-21T06:06:07.808698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:07.626683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467220+0000","last_change":"2024-09-21T06:06:06.482651+0000","last_active":"2024-09-21T06:06:07.467220+0000","last_peered":"2024-09-21T06:06:07.467220+0000","last_clean":"2024-09-21T06:06:07.467220+0000","last_became_active":"2024-09-21T06:06:06.481936+0000","last_became_peered":"2024-09-21T06:06:06.481936+0000","last_unstale":"2024-09-21T06:06:07.467220+0000","last_undegraded":"2024-09-21T06:06:07.467220+0000","last_fullsized":"2024-09-21T06:06:07.467220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:09.534869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.464302+0000","last_change":"2024-09-21T06:06:05.484732+0000","last_active":"2024-09-21T06:06:07.464302+0000","last_peered":"2024-09-21T06:06:07.464302+0000","last_clean":"2024-09-21T06:06:07.464302+0000","last_became_active":"2024-09-21T06:06:05.484237+0000","last_became_peered":"2024-09-21T06:06:05.484237+0000","last_unstale":"2024-09-21T06:06:07.464302+0000","last_undegraded":"2024-09-21T06:06:07.464302+0000","last_fullsized":"2024-09-21T06:06:07.464302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:45.088459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.463943+0000","last_change":"2024-09-21T06:06:06.490135+0000","last_active":"2024-09-21T06:06:07.463943+0000","last_peered":"2024-09-21T06:06:07.463943+0000","last_clean":"2024-09-21T06:06:07.463943+0000","last_became_active":"2024-09-21T06:06:06.489717+0000","last_became_peered":"2024-09-21T06:06:06.489717+0000","last_unstale":"2024-09-21T06:06:07.463943+0000","last_undegraded":"2024-09-21T06:06:07.463943+0000","last_fullsized":"2024-09-21T06:06:07.463943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:33.513491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.463948+0000","last_change":"2024-09-21T06:06:05.480150+0000","last_active":"2024-09-21T06:06:07.463948+0000","last_peered":"2024-09-21T06:06:07.463948+0000","last_clean":"2024-09-21T06:06:07.463948+0000","last_became_active":"2024-09-21T06:06:05.479612+0000","last_became_peered":"2024-09-21T06:06:05.479612+0000","last_unstale":"2024-09-21T06:06:07.463948+0000","last_undegraded":"2024-09-21T06:06:07.463948+0000","last_fullsized":"2024-09-21T06:06:07.463948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:21.817737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467297+0000","last_change":"2024-09-21T06:06:06.494317+0000","last_active":"2024-09-21T06:06:07.467297+0000","last_peered":"2024-09-21T06:06:07.467297+0000","last_clean":"2024-09-21T06:06:07.467297+0000","last_became_active":"2024-09-21T06:06:06.493079+0000","last_became_peered":"2024-09-21T06:06:06.493079+0000","last_unstale":"2024-09-21T06:06:07.467297+0000","last_undegraded":"2024-09-21T06:06:07.467297+0000","last_fullsized":"2024-09-21T06:06:07.467297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:53.189736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807950+0000","last_change":"2024-09-21T06:06:05.484776+0000","last_active":"2024-09-21T06:06:07.807950+0000","last_peered":"2024-09-21T06:06:07.807950+0000","last_clean":"2024-09-21T06:06:07.807950+0000","last_became_active":"2024-09-21T06:06:05.484459+0000","last_became_peered":"2024-09-21T06:06:05.484459+0000","last_unstale":"2024-09-21T06:06:07.807950+0000","last_undegraded":"2024-09-21T06:06:07.807950+0000","last_fullsized":"2024-09-21T06:06:07.807950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:44.048829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.468371+0000","last_change":"2024-09-21T06:06:06.487876+0000","last_active":"2024-09-21T06:06:07.468371+0000","last_peered":"2024-09-21T06:06:07.468371+0000","last_clean":"2024-09-21T06:06:07.468371+0000","last_became_active":"2024-09-21T06:06:06.486615+0000","last_became_peered":"2024-09-21T06:06:06.486615+0000","last_unstale":"2024-09-21T06:06:07.468371+0000","last_undegraded":"2024-09-21T06:06:07.468371+0000","last_fullsized":"2024-09-21T06:06:07.468371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:15.269467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.495286+0000","last_change":"2024-09-21T06:06:05.490629+0000","last_active":"2024-09-21T06:06:06.495286+0000","last_peered":"2024-09-21T06:06:06.495286+0000","last_clean":"2024-09-21T06:06:06.495286+0000","last_became_active":"2024-09-21T06:06:05.490286+0000","last_became_peered":"2024-09-21T06:06:05.490286+0000","last_unstale":"2024-09-21T06:06:06.495286+0000","last_undegraded":"2024-09-21T06:06:06.495286+0000","last_fullsized":"2024-09-21T06:06:06.495286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:51.319347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808092+0000","last_change":"2024-09-21T06:06:06.487398+0000","last_active":"2024-09-21T06:06:07.808092+0000","last_peered":"2024-09-21T06:06:07.808092+0000","last_clean":"2024-09-21T06:06:07.808092+0000","last_became_active":"2024-09-21T06:06:06.486064+0000","last_became_peered":"2024-09-21T06:06:06.486064+0000","last_unstale":"2024-09-21T06:06:07.808092+0000","last_undegraded":"2024-09-21T06:06:07.808092+0000","last_fullsized":"2024-09-21T06:06:07.808092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.877978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.467001+0000","last_change":"2024-09-21T06:06:05.489211+0000","last_active":"2024-09-21T06:06:06.467001+0000","last_peered":"2024-09-21T06:06:06.467001+0000","last_clean":"2024-09-21T06:06:06.467001+0000","last_became_active":"2024-09-21T06:06:05.488870+0000","last_became_peered":"2024-09-21T06:06:05.488870+0000","last_unstale":"2024-09-21T06:06:06.467001+0000","last_undegraded":"2024-09-21T06:06:06.467001+0000","last_fullsized":"2024-09-21T06:06:06.467001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:51.520528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808961+0000","last_change":"2024-09-21T06:06:06.497606+0000","last_active":"2024-09-21T06:06:07.808961+0000","last_peered":"2024-09-21T06:06:07.808961+0000","last_clean":"2024-09-21T06:06:07.808961+0000","last_became_active":"2024-09-21T06:06:06.496819+0000","last_became_peered":"2024-09-21T06:06:06.496819+0000","last_unstale":"2024-09-21T06:06:07.808961+0000","last_undegraded":"2024-09-21T06:06:07.808961+0000","last_fullsized":"2024-09-21T06:06:07.808961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:18.934480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808468+0000","last_change":"2024-09-21T06:06:05.488646+0000","last_active":"2024-09-21T06:06:07.808468+0000","last_peered":"2024-09-21T06:06:07.808468+0000","last_clean":"2024-09-21T06:06:07.808468+0000","last_became_active":"2024-09-21T06:06:05.488292+0000","last_became_peered":"2024-09-21T06:06:05.488292+0000","last_unstale":"2024-09-21T06:06:07.808468+0000","last_undegraded":"2024-09-21T06:06:07.808468+0000","last_fullsized":"2024-09-21T06:06:07.808468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:59.136360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.468274+0000","last_change":"2024-09-21T06:06:06.494320+0000","last_active":"2024-09-21T06:06:07.468274+0000","last_peered":"2024-09-21T06:06:07.468274+0000","last_clean":"2024-09-21T06:06:07.468274+0000","last_became_active":"2024-09-21T06:06:06.490006+0000","last_became_peered":"2024-09-21T06:06:06.490006+0000","last_unstale":"2024-09-21T06:06:07.468274+0000","last_undegraded":"2024-09-21T06:06:07.468274+0000","last_fullsized":"2024-09-21T06:06:07.468274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:41.349682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808906+0000","last_change":"2024-09-21T06:06:05.479924+0000","last_active":"2024-09-21T06:06:07.808906+0000","last_peered":"2024-09-21T06:06:07.808906+0000","last_clean":"2024-09-21T06:06:07.808906+0000","last_became_active":"2024-09-21T06:06:05.479574+0000","last_became_peered":"2024-09-21T06:06:05.479574+0000","last_unstale":"2024-09-21T06:06:07.808906+0000","last_undegraded":"2024-09-21T06:06:07.808906+0000","last_fullsized":"2024-09-21T06:06:07.808906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:19.778996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.467524+0000","last_change":"2024-09-21T06:06:06.481625+0000","last_active":"2024-09-21T06:06:07.467524+0000","last_peered":"2024-09-21T06:06:07.467524+0000","last_clean":"2024-09-21T06:06:07.467524+0000","last_became_active":"2024-09-21T06:06:06.481282+0000","last_became_peered":"2024-09-21T06:06:06.481282+0000","last_unstale":"2024-09-21T06:06:07.467524+0000","last_undegraded":"2024-09-21T06:06:07.467524+0000","last_fullsized":"2024-09-21T06:06:07.467524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:13.885209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808704+0000","last_change":"2024-09-21T06:06:05.473001+0000","last_active":"2024-09-21T06:06:07.808704+0000","last_peered":"2024-09-21T06:06:07.808704+0000","last_clean":"2024-09-21T06:06:07.808704+0000","last_became_active":"2024-09-21T06:06:05.472727+0000","last_became_peered":"2024-09-21T06:06:05.472727+0000","last_unstale":"2024-09-21T06:06:07.808704+0000","last_undegraded":"2024-09-21T06:06:07.808704+0000","last_fullsized":"2024-09-21T06:06:07.808704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:02.150675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808584+0000","last_change":"2024-09-21T06:06:06.491604+0000","last_active":"2024-09-21T06:06:07.808584+0000","last_peered":"2024-09-21T06:06:07.808584+0000","last_clean":"2024-09-21T06:06:07.808584+0000","last_became_active":"2024-09-21T06:06:06.491238+0000","last_became_peered":"2024-09-21T06:06:06.491238+0000","last_unstale":"2024-09-21T06:06:07.808584+0000","last_undegraded":"2024-09-21T06:06:07.808584+0000","last_fullsized":"2024-09-21T06:06:07.808584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:17.926778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.466835+0000","last_change":"2024-09-21T06:06:05.484019+0000","last_active":"2024-09-21T06:06:07.466835+0000","last_peered":"2024-09-21T06:06:07.466835+0000","last_clean":"2024-09-21T06:06:07.466835+0000","last_became_active":"2024-09-21T06:06:05.483668+0000","last_became_peered":"2024-09-21T06:06:05.483668+0000","last_unstale":"2024-09-21T06:06:07.466835+0000","last_undegraded":"2024-09-21T06:06:07.466835+0000","last_fullsized":"2024-09-21T06:06:07.466835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:57.375406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808304+0000","last_change":"2024-09-21T06:06:06.498345+0000","last_active":"2024-09-21T06:06:07.808304+0000","last_peered":"2024-09-21T06:06:07.808304+0000","last_clean":"2024-09-21T06:06:07.808304+0000","last_became_active":"2024-09-21T06:06:06.495657+0000","last_became_peered":"2024-09-21T06:06:06.495657+0000","last_unstale":"2024-09-21T06:06:07.808304+0000","last_undegraded":"2024-09-21T06:06:07.808304+0000","last_fullsized":"2024-09-21T06:06:07.808304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:21.621271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808245+0000","last_change":"2024-09-21T06:06:05.480466+0000","last_active":"2024-09-21T06:06:07.808245+0000","last_peered":"2024-09-21T06:06:07.808245+0000","last_clean":"2024-09-21T06:06:07.808245+0000","last_became_active":"2024-09-21T06:06:05.480189+0000","last_became_peered":"2024-09-21T06:06:05.480189+0000","last_unstale":"2024-09-21T06:06:07.808245+0000","last_undegraded":"2024-09-21T06:06:07.808245+0000","last_fullsized":"2024-09-21T06:06:07.808245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:26.164013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808244+0000","last_change":"2024-09-21T06:06:06.492925+0000","last_active":"2024-09-21T06:06:07.808244+0000","last_peered":"2024-09-21T06:06:07.808244+0000","last_clean":"2024-09-21T06:06:07.808244+0000","last_became_active":"2024-09-21T06:06:06.491269+0000","last_became_peered":"2024-09-21T06:06:06.491269+0000","last_unstale":"2024-09-21T06:06:07.808244+0000","last_undegraded":"2024-09-21T06:06:07.808244+0000","last_fullsized":"2024-09-21T06:06:07.808244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:14.206215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.466580+0000","last_change":"2024-09-21T06:06:05.485498+0000","last_active":"2024-09-21T06:06:07.466580+0000","last_peered":"2024-09-21T06:06:07.466580+0000","last_clean":"2024-09-21T06:06:07.466580+0000","last_became_active":"2024-09-21T06:06:05.485102+0000","last_became_peered":"2024-09-21T06:06:05.485102+0000","last_unstale":"2024-09-21T06:06:07.466580+0000","last_undegraded":"2024-09-21T06:06:07.466580+0000","last_fullsized":"2024-09-21T06:06:07.466580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:14.106756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:06:06.480240+0000","last_change":"2024-09-21T06:06:06.477652+0000","last_active":"2024-09-21T06:06:06.480240+0000","last_peered":"2024-09-21T06:06:06.480240+0000","last_clean":"2024-09-21T06:06:06.480240+0000","last_became_active":"2024-09-21T06:06:06.477385+0000","last_became_peered":"2024-09-21T06:06:06.477385+0000","last_unstale":"2024-09-21T06:06:06.480240+0000","last_undegraded":"2024-09-21T06:06:06.480240+0000","last_fullsized":"2024-09-21T06:06:06.480240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:05.449321+0000","last_clean_scrub_stamp":"2024-09-21T06:06:05.449321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:07.571592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.807632+0000","last_change":"2024-09-21T06:06:05.482619+0000","last_active":"2024-09-21T06:06:07.807632+0000","last_peered":"2024-09-21T06:06:07.807632+0000","last_clean":"2024-09-21T06:06:07.807632+0000","last_became_active":"2024-09-21T06:06:05.482257+0000","last_became_peered":"2024-09-21T06:06:05.482257+0000","last_unstale":"2024-09-21T06:06:07.807632+0000","last_undegraded":"2024-09-21T06:06:07.807632+0000","last_fullsized":"2024-09-21T06:06:07.807632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:06:04.446388+0000","last_clean_scrub_stamp":"2024-09-21T06:06:04.446388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:23.611208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":228,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:06:07.808293+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:06:07.808293+0000","last_peered":"2024-09-21T06:06:07.808293+0000","last_clean":"2024-09-21T06:06:07.808293+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:06:07.808293+0000","last_undegraded":"2024-09-21T06:06:07.808293+0000","last_fullsized":"2024-09-21T06:06:07.808293+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509561,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46668,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325172,"statfs":{"total":96636764160,"available":96588976128,"internally_reserved":0,"allocated":1622016,"data_stored":1026725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509561,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64092,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307748,"statfs":{"total":96636764160,"available":96571133952,"internally_reserved":0,"allocated":1048576,"data_stored":443337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509560,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54416,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317424,"statfs":{"total":96636764160,"available":96581042176,"internally_reserved":0,"allocated":1040384,"data_stored":437921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509561,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58572,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313268,"statfs":{"total":96636764160,"available":96576786432,"internally_reserved":0,"allocated":1036288,"data_stored":442643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509560,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68236,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303604,"statfs":{"total":96636764160,"available":96566890496,"internally_reserved":0,"allocated":1036288,"data_stored":437899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509562,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64084,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307756,"statfs":{"total":96636764160,"available":96571142144,"internally_reserved":0,"allocated":1040384,"data_stored":443225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509560,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46672,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325168,"statfs":{"total":96636764160,"available":96588972032,"internally_reserved":0,"allocated":1634304,"data_stored":1033045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542265,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68240,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303600,"statfs":{"total":96636764160,"available":96566886400,"internally_reserved":0,"allocated":1036288,"data_stored":436481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:06:11.113 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:06:11.113 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:06:11.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.317+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2952711856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0f00b5c70 msgr2=0x7fa0f015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:11.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.317+0000 7fa0f6a00640 1 --2- 172.21.15.50:0/2952711856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0f00b5c70 0x7fa0f015bb60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009900 tx=0x7fa0e001c6d0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.317+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2952711856 shutdown_connections 2024-09-21T06:06:11.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.317+0000 7fa0f6a00640 1 --2- 172.21.15.50:0/2952711856 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0f00b4de0 0x7fa0f00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.317+0000 7fa0f6a00640 1 --2- 172.21.15.50:0/2952711856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0f00b5c70 0x7fa0f015bb60 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.317+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2952711856 >> 172.21.15.50:0/2952711856 conn(0x7fa0f00b2040 msgr2=0x7fa0f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:11.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.317+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2952711856 shutdown_connections 2024-09-21T06:06:11.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.317+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2952711856 wait complete. 2024-09-21T06:06:11.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.317+0000 7fa0f6a00640 1 Processor -- start 2024-09-21T06:06:11.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.317+0000 7fa0f6a00640 1 -- start start 2024-09-21T06:06:11.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f6a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0f00b4de0 0x7fa0f0158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:11.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0f00b5c70 0x7fa0f014fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:11.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0f01527b0 0x7fa0f01f9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:11.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f6a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f0168970 con 0x7fa0f01527b0 2024-09-21T06:06:11.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f6a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f01687f0 con 0x7fa0f00b5c70 2024-09-21T06:06:11.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f6a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f0168af0 con 0x7fa0f00b4de0 2024-09-21T06:06:11.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0f01527b0 0x7fa0f01f9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:11.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0f01527b0 0x7fa0f01f9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53082/0 (socket says 172.21.15.50:53082) 2024-09-21T06:06:11.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f4a00640 1 -- 172.21.15.50:0/2078538096 learned_addr learned my addr 172.21.15.50:0/2078538096 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:11.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0efe00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0f00b4de0 0x7fa0f0158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:11.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0e7400640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0f00b5c70 0x7fa0f014fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:11.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f4a00640 1 -- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0f00b4de0 msgr2=0x7fa0f0158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:11.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f4a00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0f00b4de0 0x7fa0f0158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f4a00640 1 -- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0f00b5c70 msgr2=0x7fa0f014fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:11.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f4a00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0f00b5c70 0x7fa0f014fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f4a00640 1 -- 172.21.15.50:0/2078538096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f00ae7c0 con 0x7fa0f01527b0 2024-09-21T06:06:11.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f4a00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0f01527b0 0x7fa0f01f9fd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800ca30 tx=0x7fa0e800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:11.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0ecc00640 1 -- 172.21.15.50:0/2078538096 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8007d90 con 0x7fa0f01527b0 2024-09-21T06:06:11.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0ecc00640 1 -- 172.21.15.50:0/2078538096 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0e8004c20 con 0x7fa0f01527b0 2024-09-21T06:06:11.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0ecc00640 1 -- 172.21.15.50:0/2078538096 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8005280 con 0x7fa0f01527b0 2024-09-21T06:06:11.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2078538096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f00af910 con 0x7fa0f01527b0 2024-09-21T06:06:11.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2078538096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f00aea40 con 0x7fa0f01527b0 2024-09-21T06:06:11.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.321+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2078538096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f00b51f0 con 0x7fa0f01527b0 2024-09-21T06:06:11.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.325+0000 7fa0ecc00640 1 -- 172.21.15.50:0/2078538096 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fa0e8005420 con 0x7fa0f01527b0 2024-09-21T06:06:11.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.325+0000 7fa0ecc00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa0cc074370 0x7fa0cc076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:11.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.325+0000 7fa0ecc00640 1 -- 172.21.15.50:0/2078538096 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fa0e8093ba0 con 0x7fa0f01527b0 2024-09-21T06:06:11.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.325+0000 7fa0efe00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa0cc074370 0x7fa0cc076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:11.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.325+0000 7fa0efe00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa0cc074370 0x7fa0cc076880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00af000 tx=0x7fa0e001af10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:11.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.329+0000 7fa0ecc00640 1 -- 172.21.15.50:0/2078538096 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa0f00ae7c0 con 0x7fa0f01527b0 2024-09-21T06:06:11.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.569+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2078538096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa0f00cc7e0 con 0x7fa0f01527b0 2024-09-21T06:06:11.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0ecc00640 1 -- 172.21.15.50:0/2078538096 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v34) ==== 190+0+147 (secure 0 0 0) 0x7fa0e8014070 con 0x7fa0f01527b0 2024-09-21T06:06:11.574 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-09-21T06:06:11.574 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw== 2024-09-21T06:06:11.574 INFO:teuthology.orchestra.run.smithi050.stdout: caps mds = "allow" 2024-09-21T06:06:11.574 INFO:teuthology.orchestra.run.smithi050.stdout: caps mon = "allow r" 2024-09-21T06:06:11.574 INFO:teuthology.orchestra.run.smithi050.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:06:11.575 INFO:teuthology.orchestra.run.smithi050.stderr:updated caps for client.0 2024-09-21T06:06:11.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa0cc074370 msgr2=0x7fa0cc076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:11.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0f6a00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa0cc074370 0x7fa0cc076880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00af000 tx=0x7fa0e001af10 comp rx=0 tx=0).stop 2024-09-21T06:06:11.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2078538096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0f01527b0 msgr2=0x7fa0f01f9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:11.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0f6a00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0f01527b0 0x7fa0f01f9fd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800ca30 tx=0x7fa0e800cef0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2078538096 shutdown_connections 2024-09-21T06:06:11.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0f6a00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa0cc074370 0x7fa0cc076880 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0f6a00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0f01527b0 0x7fa0f01f9fd0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0f6a00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0f00b5c70 0x7fa0f014fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0f6a00640 1 --2- 172.21.15.50:0/2078538096 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0f00b4de0 0x7fa0f0158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2078538096 >> 172.21.15.50:0/2078538096 conn(0x7fa0f00b2040 msgr2=0x7fa0f015ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:11.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2078538096 shutdown_connections 2024-09-21T06:06:11.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.573+0000 7fa0f6a00640 1 -- 172.21.15.50:0/2078538096 wait complete. 2024-09-21T06:06:11.597 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:06:11.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.753+0000 7f0b0d200640 1 -- 172.21.15.50:0/2302585446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b080746e0 msgr2=0x7f0b0815c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:11.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.753+0000 7f0b0d200640 1 --2- 172.21.15.50:0/2302585446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b080746e0 0x7f0b0815c920 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009840 tx=0x7f0af801c760 comp rx=0 tx=0).stop 2024-09-21T06:06:11.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.753+0000 7f0b0d200640 1 -- 172.21.15.50:0/2302585446 shutdown_connections 2024-09-21T06:06:11.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.753+0000 7f0b0d200640 1 --2- 172.21.15.50:0/2302585446 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b08074c30 0x7f0b08075040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.753+0000 7f0b0d200640 1 --2- 172.21.15.50:0/2302585446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b080746e0 0x7f0b0815c920 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.753+0000 7f0b0d200640 1 -- 172.21.15.50:0/2302585446 >> 172.21.15.50:0/2302585446 conn(0x7f0b080b41f0 msgr2=0x7f0b080b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:11.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.753+0000 7f0b0d200640 1 -- 172.21.15.50:0/2302585446 shutdown_connections 2024-09-21T06:06:11.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.753+0000 7f0b0d200640 1 -- 172.21.15.50:0/2302585446 wait complete. 2024-09-21T06:06:11.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.757+0000 7f0b0d200640 1 Processor -- start 2024-09-21T06:06:11.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.757+0000 7f0b0d200640 1 -- start start 2024-09-21T06:06:11.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.757+0000 7f0b0d200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b080746e0 0x7f0b0815b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:11.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.757+0000 7f0b0d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b08074c30 0x7f0b081fa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.757+0000 7f0b0d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b080d1190 0x7f0b081fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.757+0000 7f0b0d200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0816a310 con 0x7f0b08074c30 2024-09-21T06:06:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.757+0000 7f0b0d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0816a190 con 0x7f0b080d1190 2024-09-21T06:06:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.757+0000 7f0b0d200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0b0816a490 con 0x7f0b080746e0 2024-09-21T06:06:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.757+0000 7f0afea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b08074c30 0x7f0b081fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0b07400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b080746e0 0x7f0b0815b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0afea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b08074c30 0x7f0b081fa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53098/0 (socket says 172.21.15.50:53098) 2024-09-21T06:06:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0afea00640 1 -- 172.21.15.50:0/633966462 learned_addr learned my addr 172.21.15.50:0/633966462 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0b07e00640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b080d1190 0x7f0b081fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0b07400640 1 -- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b080d1190 msgr2=0x7f0b081fa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0b07400640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b080d1190 0x7f0b081fa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0b07400640 1 -- 172.21.15.50:0/633966462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b08074c30 msgr2=0x7f0b081fa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0b07400640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b08074c30 0x7f0b081fa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0b07400640 1 -- 172.21.15.50:0/633966462 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b081faf00 con 0x7f0b080746e0 2024-09-21T06:06:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0afea00640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b08074c30 0x7f0b081fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0b07400640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b080746e0 0x7f0b0815b480 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0af80ac000 tx=0x7f0af8005ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0affe00640 1 -- 172.21.15.50:0/633966462 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80bb070 con 0x7f0b080746e0 2024-09-21T06:06:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0affe00640 1 -- 172.21.15.50:0/633966462 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0af8004330 con 0x7f0b080746e0 2024-09-21T06:06:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0affe00640 1 -- 172.21.15.50:0/633966462 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80b2040 con 0x7f0b080746e0 2024-09-21T06:06:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0b0d200640 1 -- 172.21.15.50:0/633966462 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b081fb370 con 0x7f0b080746e0 2024-09-21T06:06:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0b0d200640 1 -- 172.21.15.50:0/633966462 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0809e3e0 con 0x7f0b080746e0 2024-09-21T06:06:11.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0b0d200640 1 -- 172.21.15.50:0/633966462 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0815c920 con 0x7f0b080746e0 2024-09-21T06:06:11.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.761+0000 7f0affe00640 1 -- 172.21.15.50:0/633966462 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f0af80be070 con 0x7f0b080746e0 2024-09-21T06:06:11.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.765+0000 7f0affe00640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ae4074370 0x7f0ae4076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:11.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.765+0000 7f0affe00640 1 -- 172.21.15.50:0/633966462 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f0af812ca20 con 0x7f0b080746e0 2024-09-21T06:06:11.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.765+0000 7f0afea00640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ae4074370 0x7f0ae4076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:11.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.765+0000 7f0afea00640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ae4074370 0x7f0ae4076880 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0b080697d0 tx=0x7f0af00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:11.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.769+0000 7f0affe00640 1 -- 172.21.15.50:0/633966462 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0b08075580 con 0x7f0b080746e0 2024-09-21T06:06:11.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.981+0000 7f0b0d200640 1 -- 172.21.15.50:0/633966462 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b080b4f30 con 0x7f0b080746e0 2024-09-21T06:06:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.981+0000 7f0affe00640 1 -- 172.21.15.50:0/633966462 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5225 (secure 0 0 0) 0x7f0af8016410 con 0x7f0b080746e0 2024-09-21T06:06:11.983 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:11.983 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":55,"btime":"2024-09-21T06:06:09:495753+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3691304879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3691304879},{"type":"v1","addr":"172.21.15.50:6834","nonce":3691304879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1924969859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1924969859},{"type":"v1","addr":"172.21.15.88:6837","nonce":1924969859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3025915209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3025915209},{"type":"v1","addr":"172.21.15.88:6835","nonce":3025915209}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:06:05.890737+0000","modified":"2024-09-21T06:06:08.920852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/1211403675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1211403675},{"type":"v1","addr":"172.21.15.50:6835","nonce":1211403675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7029,"qdb_cluster":[7029]},"id":6}]} 2024-09-21T06:06:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 55 2024-09-21T06:06:11.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.985+0000 7f0b0d200640 1 -- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ae4074370 msgr2=0x7f0ae4076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:11.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.985+0000 7f0b0d200640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ae4074370 0x7f0ae4076880 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0b080697d0 tx=0x7f0af00091e0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.985+0000 7f0b0d200640 1 -- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b080746e0 msgr2=0x7f0b0815b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:11.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.985+0000 7f0b0d200640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b080746e0 0x7f0b0815b480 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0af80ac000 tx=0x7f0af8005ca0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.985+0000 7f0b0d200640 1 -- 172.21.15.50:0/633966462 shutdown_connections 2024-09-21T06:06:11.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.985+0000 7f0b0d200640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b080d1190 0x7f0b081fa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.985+0000 7f0b0d200640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0ae4074370 0x7f0ae4076880 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.985+0000 7f0b0d200640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b08074c30 0x7f0b081fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.985+0000 7f0b0d200640 1 --2- 172.21.15.50:0/633966462 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b080746e0 0x7f0b0815b480 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:11.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.985+0000 7f0b0d200640 1 -- 172.21.15.50:0/633966462 >> 172.21.15.50:0/633966462 conn(0x7f0b080b41f0 msgr2=0x7f0b080b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:11.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.985+0000 7f0b0d200640 1 -- 172.21.15.50:0/633966462 shutdown_connections 2024-09-21T06:06:11.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:11.985+0000 7f0b0d200640 1 -- 172.21.15.50:0/633966462 wait complete. 2024-09-21T06:06:12.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:06:05.890737+0000', 'modified': '2024-09-21T06:06:08.920852+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7029}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7029': {'gid': 7029, 'name': 'c', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6835/1211403675', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6833', 'nonce': 1211403675}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 1211403675}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7029, 'qdb_cluster': [7029]} 2024-09-21T06:06:12.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:06:12.408 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T06:06:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.569+0000 7f9ee5400640 1 -- 172.21.15.50:0/1784127306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ee0164560 msgr2=0x7f9ee0166980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.569+0000 7f9ee5400640 1 --2- 172.21.15.50:0/1784127306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ee0164560 0x7f9ee0166980 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009bc0 tx=0x7f9ed801cba0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.569+0000 7f9ee5400640 1 -- 172.21.15.50:0/1784127306 shutdown_connections 2024-09-21T06:06:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.569+0000 7f9ee5400640 1 --2- 172.21.15.50:0/1784127306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ee0164560 0x7f9ee0166980 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.569+0000 7f9ee5400640 1 -- 172.21.15.50:0/1784127306 >> 172.21.15.50:0/1784127306 conn(0x7f9ee00b2040 msgr2=0x7f9ee00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.569+0000 7f9ee5400640 1 -- 172.21.15.50:0/1784127306 shutdown_connections 2024-09-21T06:06:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.569+0000 7f9ee5400640 1 -- 172.21.15.50:0/1784127306 wait complete. 2024-09-21T06:06:12.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.573+0000 7f9ee5400640 1 Processor -- start 2024-09-21T06:06:12.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.573+0000 7f9ee5400640 1 -- start start 2024-09-21T06:06:12.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.573+0000 7f9ee5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ee015e8c0 0x7f9ee0160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.573+0000 7f9ee5400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ee0160a20 0x7f9ee013dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.573+0000 7f9ee5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee0164560 0x7f9ee00ce810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.573+0000 7f9ee5400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0169ee0 con 0x7f9ee0164560 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.573+0000 7f9ee5400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0169d60 con 0x7f9ee015e8c0 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.573+0000 7f9ee5400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee016a060 con 0x7f9ee0160a20 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.573+0000 7f9edf200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee0164560 0x7f9ee00ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.573+0000 7f9edf200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee0164560 0x7f9ee00ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53114/0 (socket says 172.21.15.50:53114) 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.573+0000 7f9edf200640 1 -- 172.21.15.50:0/3383206072 learned_addr learned my addr 172.21.15.50:0/3383206072 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9ede800640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ee015e8c0 0x7f9ee0160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ee0160a20 0x7f9ee013dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9edf200640 1 -- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ee0160a20 msgr2=0x7f9ee013dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9edf200640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ee0160a20 0x7f9ee013dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9edf200640 1 -- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ee015e8c0 msgr2=0x7f9ee0160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9edf200640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ee015e8c0 0x7f9ee0160e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9edf200640 1 -- 172.21.15.50:0/3383206072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee00cd770 con 0x7f9ee0164560 2024-09-21T06:06:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9edf200640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee0164560 0x7f9ee00ce810 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80197b0 tx=0x7f9ed8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:12.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9ed6a00640 1 -- 172.21.15.50:0/3383206072 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80b2040 con 0x7f9ee0164560 2024-09-21T06:06:12.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9ed6a00640 1 -- 172.21.15.50:0/3383206072 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ed8004040 con 0x7f9ee0164560 2024-09-21T06:06:12.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9ee5400640 1 -- 172.21.15.50:0/3383206072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee00cda50 con 0x7f9ee0164560 2024-09-21T06:06:12.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9ed6a00640 1 -- 172.21.15.50:0/3383206072 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80044b0 con 0x7f9ee0164560 2024-09-21T06:06:12.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9ee5400640 1 -- 172.21.15.50:0/3383206072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee00ce010 con 0x7f9ee0164560 2024-09-21T06:06:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.577+0000 7f9ed6a00640 1 -- 172.21.15.50:0/3383206072 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f9ed80bd030 con 0x7f9ee0164560 2024-09-21T06:06:12.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ed6a00640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ebc0743a0 0x7f9ebc0768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ed6a00640 1 -- 172.21.15.50:0/3383206072 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f9ed812cdb0 con 0x7f9ee0164560 2024-09-21T06:06:12.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ede800640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ebc0743a0 0x7f9ebc0768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9eecc82000 1 Processor -- start 2024-09-21T06:06:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9eecc82000 1 -- start start 2024-09-21T06:06:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ede800640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ebc0743a0 0x7f9ebc0768b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0007fd0 tx=0x7f9ed0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9eecc82000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9eecc82000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4130950 0x563cd4281be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9eecc82000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4119ca0 0x563cd4282120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9eecc82000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563cd4156410 con 0x563cd3d40460 2024-09-21T06:06:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9eecc82000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563cd4093b40 con 0x563cd4130950 2024-09-21T06:06:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9eecc82000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x563cd4082f70 con 0x563cd4119ca0 2024-09-21T06:06:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ede800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ede800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42816a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53116/0 (socket says 172.21.15.50:53116) 2024-09-21T06:06:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ede800640 1 -- 172.21.15.50:0/3349110459 learned_addr learned my addr 172.21.15.50:0/3349110459 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/3349110459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4130950 0x563cd4281be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9edf200640 1 --2- 172.21.15.50:0/3349110459 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4119ca0 0x563cd4282120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ede800640 1 -- 172.21.15.50:0/3349110459 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4119ca0 msgr2=0x563cd4282120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ede800640 1 --2- 172.21.15.50:0/3349110459 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4119ca0 0x563cd4282120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ede800640 1 -- 172.21.15.50:0/3349110459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4130950 msgr2=0x563cd4281be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ede800640 1 --2- 172.21.15.50:0/3349110459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4130950 0x563cd4281be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.581+0000 7f9ede800640 1 -- 172.21.15.50:0/3349110459 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cd404a5b0 con 0x563cd3d40460 2024-09-21T06:06:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9ede800640 1 --2- 172.21.15.50:0/3349110459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42816a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000f8c0 tx=0x7f9ed0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0654c0333313a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9ecf000640 1 -- 172.21.15.50:0/3349110459 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0025070 con 0x563cd3d40460 2024-09-21T06:06:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9ecf000640 1 -- 172.21.15.50:0/3349110459 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ed001b050 con 0x563cd3d40460 2024-09-21T06:06:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9ecf000640 1 -- 172.21.15.50:0/3349110459 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0014de0 con 0x563cd3d40460 2024-09-21T06:06:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9eecc82000 1 -- 172.21.15.50:0/3349110459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 msgr2=0x563cd42816a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9eecc82000 1 --2- 172.21.15.50:0/3349110459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42816a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000f8c0 tx=0x7f9ed0012000 comp rx=0 tx=0).stop 2024-09-21T06:06:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9eecc82000 1 -- 172.21.15.50:0/3349110459 shutdown_connections 2024-09-21T06:06:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9eecc82000 1 --2- 172.21.15.50:0/3349110459 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4119ca0 0x563cd4282120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9eecc82000 1 --2- 172.21.15.50:0/3349110459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4130950 0x563cd4281be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9eecc82000 1 --2- 172.21.15.50:0/3349110459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42816a0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9eecc82000 1 -- 172.21.15.50:0/3349110459 >> 172.21.15.50:0/3349110459 conn(0x563cd4206d90 msgr2=0x563cd4236690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:12.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9eecc82000 1 -- 172.21.15.50:0/3349110459 shutdown_connections 2024-09-21T06:06:12.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9eecc82000 1 -- 172.21.15.50:0/3349110459 wait complete. 2024-09-21T06:06:12.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9eecc82000 1 -- start start 2024-09-21T06:06:12.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.585+0000 7f9eecc82000 1 Processor -- start 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd3d40460 0x563cd4288920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd4119ca0 0x563cd4288e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 0x563cd42893a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563cd401d5f0 con 0x563cd4119ca0 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563cd4086d80 con 0x563cd3d40460 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x563cd4029520 con 0x563cd4130950 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 10 client.0 _ll_register_callbacks cb 0x563cd402d830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ed7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd4119ca0 0x563cd4288e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ed7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd4119ca0 0x563cd4288e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53118/0 (socket says 172.21.15.50:53118) 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ed7e00640 1 -- 172.21.15.50:0/363337954 learned_addr learned my addr 172.21.15.50:0/363337954 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ed7e00640 1 -- 172.21.15.50:0/363337954 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 msgr2=0x563cd42893a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/363337954 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 0x563cd42893a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ede800640 1 --2- 172.21.15.50:0/363337954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd3d40460 0x563cd4288920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ed7e00640 1 -- 172.21.15.50:0/363337954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd3d40460 msgr2=0x563cd4288920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/363337954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd3d40460 0x563cd4288920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ed7e00640 1 -- 172.21.15.50:0/363337954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cd4021190 con 0x563cd4119ca0 2024-09-21T06:06:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/363337954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd4119ca0 0x563cd4288e60 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8012520 tx=0x7f9ec80129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1521caaae67a6dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ec3400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:06:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ec3400640 1 -- 172.21.15.50:0/363337954 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec801a030 con 0x563cd4119ca0 2024-09-21T06:06:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ec3400640 1 -- 172.21.15.50:0/363337954 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f9ec8010040 con 0x563cd4119ca0 2024-09-21T06:06:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 1 -- 172.21.15.50:0/363337954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563cd401b110 con 0x563cd4119ca0 2024-09-21T06:06:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ec3400640 1 -- 172.21.15.50:0/363337954 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9ec8023050 con 0x563cd4119ca0 2024-09-21T06:06:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 10 client.7149 fetch_fsmap learned FSMap version 55 2024-09-21T06:06:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 1 -- 172.21.15.50:0/363337954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x563cd4107f10 con 0x563cd4119ca0 2024-09-21T06:06:12.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ec3400640 1 -- 172.21.15.50:0/363337954 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563cd401b110 con 0x563cd4119ca0 2024-09-21T06:06:12.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ec3400640 1 -- 172.21.15.50:0/363337954 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7f9ec8026020 con 0x563cd4119ca0 2024-09-21T06:06:12.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 10 client.7149 fetch_fsmap finished waiting for FSMap version 55 2024-09-21T06:06:12.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 10 client.7149 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:06:12.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 20 client.7149 populate_metadata read hostname 'smithi050' 2024-09-21T06:06:12.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 1 --2- 172.21.15.50:0/363337954 >> [v2:172.21.15.50:6833/1211403675,v1:172.21.15.50:6835/1211403675] conn(0x563cd3d257d0 0x563cd4292d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 4 client.7149 mds_command: new command op to 7029 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9eecc82000 1 -- 172.21.15.50:0/363337954 --> [v2:172.21.15.50:6833/1211403675,v1:172.21.15.50:6835/1211403675] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cd4095a90 con 0x563cd3d257d0 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ede800640 1 --2- 172.21.15.50:0/363337954 >> [v2:172.21.15.50:6833/1211403675,v1:172.21.15.50:6835/1211403675] conn(0x563cd3d257d0 0x563cd4292d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ede800640 1 --2- 172.21.15.50:0/363337954 >> [v2:172.21.15.50:6833/1211403675,v1:172.21.15.50:6835/1211403675] conn(0x563cd3d257d0 0x563cd4292d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f894a20be9e6cc96 server_cookie=6985c02b7201838f in_seq=0 out_seq=0 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.589+0000 7f9ec3400640 10 client.7149 ms_handle_connect on v2:172.21.15.50:6833/1211403675 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9ec3400640 1 -- 172.21.15.50:0/363337954 <== mds.0 v2:172.21.15.50:6833/1211403675 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563cd4095a90 con 0x563cd3d257d0 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9ec3400640 10 client.7149 handle_command_reply: tid=0 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9eecc82000 1 client.7149 shutdown 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9eecc82000 1 -- 172.21.15.50:0/363337954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd4119ca0 msgr2=0x563cd4288e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9eecc82000 1 --2- 172.21.15.50:0/363337954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd4119ca0 0x563cd4288e60 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8012520 tx=0x7f9ec80129e0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9eecc82000 1 -- 172.21.15.50:0/363337954 shutdown_connections 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9eecc82000 1 --2- 172.21.15.50:0/363337954 >> [v2:172.21.15.50:6833/1211403675,v1:172.21.15.50:6835/1211403675] conn(0x563cd3d257d0 0x563cd4292d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9eecc82000 1 --2- 172.21.15.50:0/363337954 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 0x563cd42893a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9eecc82000 1 --2- 172.21.15.50:0/363337954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd4119ca0 0x563cd4288e60 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9eecc82000 1 --2- 172.21.15.50:0/363337954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd3d40460 0x563cd4288920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9ec3400640 0 client.7149 ms_handle_reset on v2:172.21.15.50:6833/1211403675 2024-09-21T06:06:12.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9eecc82000 1 -- 172.21.15.50:0/363337954 >> 172.21.15.50:0/363337954 conn(0x563cd4206d90 msgr2=0x563cd423d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:12.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9eecc82000 1 -- 172.21.15.50:0/363337954 shutdown_connections 2024-09-21T06:06:12.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9eecc82000 1 -- 172.21.15.50:0/363337954 wait complete. 2024-09-21T06:06:12.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.593+0000 7f9eecc82000 20 client.7149 trim_cache size 0 max 16384 2024-09-21T06:06:12.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.609+0000 7f9eecc82000 1 Processor -- start 2024-09-21T06:06:12.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.609+0000 7f9eecc82000 1 -- start start 2024-09-21T06:06:12.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42f3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4119ca0 0x563cd42f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 0x563cd42f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee02053d0 con 0x563cd3d40460 2024-09-21T06:06:12.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0204410 con 0x563cd4119ca0 2024-09-21T06:06:12.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee0142cc0 con 0x563cd4130950 2024-09-21T06:06:12.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ede800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ede800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53126/0 (socket says 172.21.15.50:53126) 2024-09-21T06:06:12.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ede800640 1 -- 172.21.15.50:0/2238212342 learned_addr learned my addr 172.21.15.50:0/2238212342 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:12.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/2238212342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4119ca0 0x563cd42f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9edf200640 1 --2- 172.21.15.50:0/2238212342 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 0x563cd42f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ede800640 1 -- 172.21.15.50:0/2238212342 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 msgr2=0x563cd42f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ede800640 1 --2- 172.21.15.50:0/2238212342 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 0x563cd42f40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ede800640 1 -- 172.21.15.50:0/2238212342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4119ca0 msgr2=0x563cd42f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ede800640 1 --2- 172.21.15.50:0/2238212342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4119ca0 0x563cd42f3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ede800640 1 -- 172.21.15.50:0/2238212342 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cd40dedb0 con 0x563cd3d40460 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ede800640 1 --2- 172.21.15.50:0/2238212342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42f3620 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0001150 tx=0x7f9ed0000ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f857f1d0ce0d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ece600640 1 -- 172.21.15.50:0/2238212342 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0025040 con 0x563cd3d40460 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ece600640 1 -- 172.21.15.50:0/2238212342 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ed001b030 con 0x563cd3d40460 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9ece600640 1 -- 172.21.15.50:0/2238212342 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0006c60 con 0x563cd3d40460 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 -- 172.21.15.50:0/2238212342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 msgr2=0x563cd42f3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 --2- 172.21.15.50:0/2238212342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42f3620 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0001150 tx=0x7f9ed0000ff0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 -- 172.21.15.50:0/2238212342 shutdown_connections 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 --2- 172.21.15.50:0/2238212342 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 0x563cd42f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 --2- 172.21.15.50:0/2238212342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4119ca0 0x563cd42f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 --2- 172.21.15.50:0/2238212342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42f3620 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 -- 172.21.15.50:0/2238212342 >> 172.21.15.50:0/2238212342 conn(0x563cd4206d90 msgr2=0x563cd427f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 -- 172.21.15.50:0/2238212342 shutdown_connections 2024-09-21T06:06:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.613+0000 7f9eecc82000 1 -- 172.21.15.50:0/2238212342 wait complete. 2024-09-21T06:06:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9eecc82000 1 -- start start 2024-09-21T06:06:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9eecc82000 1 Processor -- start 2024-09-21T06:06:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9eecc82000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42f7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9eecc82000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4119ca0 0x563cd42f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9eecc82000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 0x563cd42f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9eecc82000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563cd408b3b0 con 0x563cd3d40460 2024-09-21T06:06:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9eecc82000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563cd4142750 con 0x563cd4119ca0 2024-09-21T06:06:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9eecc82000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x563cd4092910 con 0x563cd4130950 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9eecc82000 10 client.0 _ll_register_callbacks cb 0x7f9ec80013e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ede800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ede800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42f7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53142/0 (socket says 172.21.15.50:53142) 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ede800640 1 -- 172.21.15.50:0/3330734125 learned_addr learned my addr 172.21.15.50:0/3330734125 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9edf200640 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 0x563cd42f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4119ca0 0x563cd42f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ede800640 1 -- 172.21.15.50:0/3330734125 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 msgr2=0x563cd42f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ede800640 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 0x563cd42f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ede800640 1 -- 172.21.15.50:0/3330734125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4119ca0 msgr2=0x563cd42f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ede800640 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4119ca0 0x563cd42f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ede800640 1 -- 172.21.15.50:0/3330734125 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cd4049ab0 con 0x563cd3d40460 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9edf200640 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 0x563cd42f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4119ca0 0x563cd42f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ede800640 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42f7660 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0001150 tx=0x7f9ed0007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d6196f52e8775e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:12.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ec3400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:06:12.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ec3400640 1 -- 172.21.15.50:0/3330734125 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0036020 con 0x563cd3d40460 2024-09-21T06:06:12.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ec3400640 1 -- 172.21.15.50:0/3330734125 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f9ed00069e0 con 0x563cd3d40460 2024-09-21T06:06:12.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9eecc82000 1 -- 172.21.15.50:0/3330734125 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed001b030 con 0x563cd3d40460 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9ec3400640 1 -- 172.21.15.50:0/3330734125 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9ed0040050 con 0x563cd3d40460 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9eecc82000 10 client.7155 fetch_fsmap learned FSMap version 55 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.617+0000 7f9eecc82000 1 -- 172.21.15.50:0/3330734125 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x563cd4028070 con 0x563cd3d40460 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ec3400640 1 -- 172.21.15.50:0/3330734125 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ed001b030 con 0x563cd3d40460 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ec3400640 1 -- 172.21.15.50:0/3330734125 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7f9ed0043020 con 0x563cd3d40460 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 10 client.7155 fetch_fsmap finished waiting for FSMap version 55 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 10 client.7155 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 20 client.7155 populate_metadata read hostname 'smithi050' 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.50:6833/1211403675,v1:172.21.15.50:6835/1211403675] conn(0x563cd3d13830 0x563cd42b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 4 client.7155 mds_command: new command op to 7029 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 -- 172.21.15.50:0/3330734125 --> [v2:172.21.15.50:6833/1211403675,v1:172.21.15.50:6835/1211403675] -- command(tid 0: {"prefix": "status"}) -- 0x563cd4091540 con 0x563cd3d13830 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.50:6833/1211403675,v1:172.21.15.50:6835/1211403675] conn(0x563cd3d13830 0x563cd42b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ec3400640 10 client.7155 ms_handle_connect on v2:172.21.15.50:6833/1211403675 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ed7e00640 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.50:6833/1211403675,v1:172.21.15.50:6835/1211403675] conn(0x563cd3d13830 0x563cd42b3610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1e704de04a822 server_cookie=6b5283edbf5e617b in_seq=0 out_seq=0 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ec3400640 1 -- 172.21.15.50:0/3330734125 <== mds.0 v2:172.21.15.50:6833/1211403675 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x563cd4091540 con 0x563cd3d13830 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ec3400640 10 client.7155 handle_command_reply: tid=0 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 client.7155 shutdown 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 -- 172.21.15.50:0/3330734125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 msgr2=0x563cd42f7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42f7660 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0001150 tx=0x7f9ed0007b40 comp rx=0 tx=0).stop 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 -- 172.21.15.50:0/3330734125 shutdown_connections 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.50:6833/1211403675,v1:172.21.15.50:6835/1211403675] conn(0x563cd3d13830 0x563cd42b3610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x563cd4130950 0x563cd42f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563cd4119ca0 0x563cd42f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ec3400640 0 client.7155 ms_handle_reset on v2:172.21.15.50:6833/1211403675 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 --2- 172.21.15.50:0/3330734125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cd3d40460 0x563cd42f7660 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 -- 172.21.15.50:0/3330734125 >> 172.21.15.50:0/3330734125 conn(0x563cd4206d90 msgr2=0x563cd42926a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 -- 172.21.15.50:0/3330734125 shutdown_connections 2024-09-21T06:06:12.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 1 -- 172.21.15.50:0/3330734125 wait complete. 2024-09-21T06:06:12.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9eecc82000 20 client.7155 trim_cache size 0 max 16384 2024-09-21T06:06:12.626 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:06:12.626 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:06:12.626 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-21T06:06:12.626 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 7029, 2024-09-21T06:06:12.626 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-21T06:06:12.626 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-21T06:06:12.626 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-21T06:06:12.626 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 6.1661728499999997, 2024-09-21T06:06:12.626 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 55, 2024-09-21T06:06:12.627 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 85, 2024-09-21T06:06:12.627 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 85, 2024-09-21T06:06:12.627 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 10.580325669 2024-09-21T06:06:12.627 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:06:12.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ee5400640 1 -- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ebc0743a0 msgr2=0x7f9ebc0768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ee5400640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ebc0743a0 0x7f9ebc0768b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0007fd0 tx=0x7f9ed0009040 comp rx=0 tx=0).stop 2024-09-21T06:06:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ee5400640 1 -- 172.21.15.50:0/3383206072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee0164560 msgr2=0x7f9ee00ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ee5400640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee0164560 0x7f9ee00ce810 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80197b0 tx=0x7f9ed8003f60 comp rx=0 tx=0).stop 2024-09-21T06:06:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ee5400640 1 -- 172.21.15.50:0/3383206072 shutdown_connections 2024-09-21T06:06:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ee5400640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9ebc0743a0 0x7f9ebc0768b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ee5400640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee0164560 0x7f9ee00ce810 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ee5400640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9ee0160a20 0x7f9ee013dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ee5400640 1 --2- 172.21.15.50:0/3383206072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ee015e8c0 0x7f9ee0160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ee5400640 1 -- 172.21.15.50:0/3383206072 >> 172.21.15.50:0/3383206072 conn(0x7f9ee00b2040 msgr2=0x7f9ee015ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ee5400640 1 -- 172.21.15.50:0/3383206072 shutdown_connections 2024-09-21T06:06:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.621+0000 7f9ee5400640 1 -- 172.21.15.50:0/3383206072 wait complete. 2024-09-21T06:06:12.643 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:06:12.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.801+0000 7f6d0c000640 1 -- 172.21.15.50:0/3657152463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cec003a10 msgr2=0x7f6cec003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:12.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.801+0000 7f6d0c000640 1 --2- 172.21.15.50:0/3657152463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cec003a10 0x7f6cec003e80 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c4a0 tx=0x7f6cf801cf50 comp rx=0 tx=0).stop 2024-09-21T06:06:12.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.801+0000 7f6d0a000640 1 -- 172.21.15.50:0/3657152463 reap_dead start 2024-09-21T06:06:12.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.801+0000 7f6d0c000640 1 -- 172.21.15.50:0/3657152463 shutdown_connections 2024-09-21T06:06:12.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.801+0000 7f6d0c000640 1 -- 172.21.15.50:0/3657152463 >> 172.21.15.50:0/3657152463 conn(0x7f6d040b2040 msgr2=0x7f6d040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:12.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.801+0000 7f6d0c000640 1 -- 172.21.15.50:0/3657152463 shutdown_connections 2024-09-21T06:06:12.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.801+0000 7f6d0c000640 1 -- 172.21.15.50:0/3657152463 wait complete. 2024-09-21T06:06:12.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0c000640 1 Processor -- start 2024-09-21T06:06:12.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0c000640 1 -- start start 2024-09-21T06:06:12.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0c000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cec003a10 0x7f6d04158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0c000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d041f9cb0 0x7f6d041fc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0c000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d041fcaf0 0x7f6d04165470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0c000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04166910 con 0x7f6d041fcaf0 2024-09-21T06:06:12.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0c000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04166790 con 0x7f6d041f9cb0 2024-09-21T06:06:12.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0c000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6d04166a90 con 0x7f6cec003a10 2024-09-21T06:06:12.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d041fcaf0 0x7f6d04165470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d041fcaf0 0x7f6d04165470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53158/0 (socket says 172.21.15.50:53158) 2024-09-21T06:06:12.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0a000640 1 -- 172.21.15.50:0/68266745 learned_addr learned my addr 172.21.15.50:0/68266745 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:12.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d08c00640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d041f9cb0 0x7f6d041fc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d09600640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cec003a10 0x7f6d04158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0a000640 1 -- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cec003a10 msgr2=0x7f6d04158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:12.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0a000640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cec003a10 0x7f6d04158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0a000640 1 -- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d041f9cb0 msgr2=0x7f6d041fc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:12.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0a000640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d041f9cb0 0x7f6d041fc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:12.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0a000640 1 -- 172.21.15.50:0/68266745 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d041f8f60 con 0x7f6d041fcaf0 2024-09-21T06:06:12.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0a000640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d041fcaf0 0x7f6d04165470 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6d041596f0 tx=0x7f6cfc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:12.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d01600640 1 -- 172.21.15.50:0/68266745 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00ecc0 con 0x7f6d041fcaf0 2024-09-21T06:06:12.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d01600640 1 -- 172.21.15.50:0/68266745 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cfc003980 con 0x7f6d041fcaf0 2024-09-21T06:06:12.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d01600640 1 -- 172.21.15.50:0/68266745 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc003dd0 con 0x7f6d041fcaf0 2024-09-21T06:06:12.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0c000640 1 -- 172.21.15.50:0/68266745 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d041fd2a0 con 0x7f6d041fcaf0 2024-09-21T06:06:12.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.805+0000 7f6d0c000640 1 -- 172.21.15.50:0/68266745 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d041f94b0 con 0x7f6d041fcaf0 2024-09-21T06:06:12.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.809+0000 7f6d0c000640 1 -- 172.21.15.50:0/68266745 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d040b51f0 con 0x7f6d041fcaf0 2024-09-21T06:06:12.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.809+0000 7f6d01600640 1 -- 172.21.15.50:0/68266745 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f6cfc01b030 con 0x7f6d041fcaf0 2024-09-21T06:06:12.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.809+0000 7f6d01600640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ce4074370 0x7f6ce4076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:12.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.809+0000 7f6d01600640 1 -- 172.21.15.50:0/68266745 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f6cfc092d40 con 0x7f6d041fcaf0 2024-09-21T06:06:12.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.809+0000 7f6d09600640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ce4074370 0x7f6ce4076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:12.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.809+0000 7f6d09600640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ce4074370 0x7f6ce4076880 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80aa000 tx=0x7f6cf8019d90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:12.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:12.813+0000 7f6d01600640 1 -- 172.21.15.50:0/68266745 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6cfc022020 con 0x7f6d041fcaf0 2024-09-21T06:06:13.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.025+0000 7f6d0c000640 1 -- 172.21.15.50:0/68266745 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6d041581e0 con 0x7f6d041fcaf0 2024-09-21T06:06:13.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d01600640 1 -- 172.21.15.50:0/68266745 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) ==== 53+0+73 (secure 0 0 0) 0x7f6cfc05f1f0 con 0x7f6d041fcaf0 2024-09-21T06:06:13.029 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:06:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d0c000640 1 -- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ce4074370 msgr2=0x7f6ce4076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d0c000640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ce4074370 0x7f6ce4076880 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80aa000 tx=0x7f6cf8019d90 comp rx=0 tx=0).stop 2024-09-21T06:06:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d0c000640 1 -- 172.21.15.50:0/68266745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d041fcaf0 msgr2=0x7f6d04165470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d0c000640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d041fcaf0 0x7f6d04165470 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6d041596f0 tx=0x7f6cfc00c6a0 comp rx=0 tx=0).stop 2024-09-21T06:06:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d0c000640 1 -- 172.21.15.50:0/68266745 shutdown_connections 2024-09-21T06:06:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d0c000640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ce4074370 0x7f6ce4076880 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d0c000640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d041fcaf0 0x7f6d04165470 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d0c000640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d041f9cb0 0x7f6d041fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d0c000640 1 --2- 172.21.15.50:0/68266745 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cec003a10 0x7f6d04158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d0c000640 1 -- 172.21.15.50:0/68266745 >> 172.21.15.50:0/68266745 conn(0x7f6d040b2040 msgr2=0x7f6d04159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d0c000640 1 -- 172.21.15.50:0/68266745 shutdown_connections 2024-09-21T06:06:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.029+0000 7f6d0c000640 1 -- 172.21.15.50:0/68266745 wait complete. 2024-09-21T06:06:13.052 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:06:13.052 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:06:13.052 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-21T06:06:13.052 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:06:13.052 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:06:13.052 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:06:13.052 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:06:13.052 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:06:13.052 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:06:13.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:13.053 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-21T06:06:13.062 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:06:13.062 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:06:13.062 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:06:13.062 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:06:13.062 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T06:06:13.062 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:06:13.062 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:06:13.062 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:06:13.062 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-21T06:06:13.062 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 41822sec preferred_lft 41822sec 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T06:06:13.063 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:8b:d0:0a:5d brd ff:ff:ff:ff:ff:ff 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:06:13.064 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:06:13.065 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T06:06:13.065 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:06:13.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:13.065 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:06:13.130 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-21T06:06:13.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:13.132 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-21T06:06:13.182 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-21T06:06:13.182 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:06:13.182 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-21T06:06:13.183 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T06:06:13.183 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-21T06:06:13.183 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:06:13.183 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:06:13.183 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T06:06:13.183 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:06:13.183 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:06:13.183 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:06:13.183 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T06:06:13.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:06:13.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:06:13.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:13.324 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T06:06:13.332 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:06:13.332 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:06:13.333 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:06:13.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:13.333 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-09-21T06:06:13.396 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:06:13.396 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:06:13.444 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:06:13.445 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:06:13.445 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:06:13.496 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:06:13.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:13.496 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:06:13.598 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:06:13.598 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:06:13.598 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:06:13.616 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:06:13.616 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:06:13.616 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:06:13.616 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:06:13.617 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:06:13.626 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:06:13.626 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:06:13.626 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:06:13.626 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:06:13.626 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:06:13.626 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T06:06:13.626 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T06:06:13.626 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T06:06:13.630 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:06:13.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:13.630 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:06:13.650 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:06:13.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:06:13.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:06:13.756 INFO:teuthology.orchestra.run.smithi050.stdout:client7164 2024-09-21T06:06:13.756 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:06:13.756 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:06:13.765 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T06:06:13.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.969+0000 7f4192e00640 1 -- 172.21.15.50:0/1748340923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f418c0b5c70 msgr2=0x7f418c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:13.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.969+0000 7f4192e00640 1 --2- 172.21.15.50:0/1748340923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f418c0b5c70 0x7f418c15bb60 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f417c009900 tx=0x7f417c01c6d0 comp rx=0 tx=0).stop 2024-09-21T06:06:13.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.969+0000 7f4192e00640 1 -- 172.21.15.50:0/1748340923 shutdown_connections 2024-09-21T06:06:13.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.969+0000 7f4192e00640 1 --2- 172.21.15.50:0/1748340923 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f418c0b4de0 0x7f418c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:13.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.969+0000 7f4192e00640 1 --2- 172.21.15.50:0/1748340923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f418c0b5c70 0x7f418c15bb60 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:13.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.969+0000 7f4192e00640 1 -- 172.21.15.50:0/1748340923 >> 172.21.15.50:0/1748340923 conn(0x7f418c0b2040 msgr2=0x7f418c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:13.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.969+0000 7f4192e00640 1 -- 172.21.15.50:0/1748340923 shutdown_connections 2024-09-21T06:06:13.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.969+0000 7f4192e00640 1 -- 172.21.15.50:0/1748340923 wait complete. 2024-09-21T06:06:13.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.969+0000 7f4192e00640 1 Processor -- start 2024-09-21T06:06:13.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.969+0000 7f4192e00640 1 -- start start 2024-09-21T06:06:13.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4192e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f418c0b4de0 0x7f418c144930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:13.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4192e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f418c0b5c70 0x7f418c13f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:13.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4192e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f418c1527b0 0x7f418c13f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:13.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4192e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f418c168a50 con 0x7f418c0b4de0 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4192e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f418c1688d0 con 0x7f418c1527b0 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4192e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f418c168bd0 con 0x7f418c0b5c70 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f418be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f418c0b4de0 0x7f418c144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f418be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f418c0b4de0 0x7f418c144930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53174/0 (socket says 172.21.15.50:53174) 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f418be00640 1 -- 172.21.15.50:0/4120948396 learned_addr learned my addr 172.21.15.50:0/4120948396 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4183e00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f418c0b5c70 0x7f418c13f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4190e00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f418c1527b0 0x7f418c13f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f418be00640 1 -- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f418c0b5c70 msgr2=0x7f418c13f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f418be00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f418c0b5c70 0x7f418c13f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f418be00640 1 -- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f418c1527b0 msgr2=0x7f418c13f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f418be00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f418c1527b0 0x7f418c13f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f418be00640 1 -- 172.21.15.50:0/4120948396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418c157980 con 0x7f418c0b4de0 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4190e00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f418c1527b0 0x7f418c13f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:13.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4183e00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f418c0b5c70 0x7f418c13f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:13.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f418be00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f418c0b4de0 0x7f418c144930 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f417c0b0040 tx=0x7f417c0a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:13.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4188c00640 1 -- 172.21.15.50:0/4120948396 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c0ba020 con 0x7f418c0b4de0 2024-09-21T06:06:13.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4188c00640 1 -- 172.21.15.50:0/4120948396 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f417c004430 con 0x7f418c0b4de0 2024-09-21T06:06:13.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4192e00640 1 -- 172.21.15.50:0/4120948396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418c157c60 con 0x7f418c0b4de0 2024-09-21T06:06:13.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4188c00640 1 -- 172.21.15.50:0/4120948396 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c0b2040 con 0x7f418c0b4de0 2024-09-21T06:06:13.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4192e00640 1 -- 172.21.15.50:0/4120948396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f418c158220 con 0x7f418c0b4de0 2024-09-21T06:06:13.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.973+0000 7f4192e00640 1 -- 172.21.15.50:0/4120948396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f418c0b51f0 con 0x7f418c0b4de0 2024-09-21T06:06:13.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.977+0000 7f4188c00640 1 -- 172.21.15.50:0/4120948396 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f417c0bd070 con 0x7f418c0b4de0 2024-09-21T06:06:13.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.977+0000 7f4188c00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4168074370 0x7f4168076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:13.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.977+0000 7f4188c00640 1 -- 172.21.15.50:0/4120948396 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f417c12c960 con 0x7f418c0b4de0 2024-09-21T06:06:13.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.977+0000 7f4183e00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4168074370 0x7f4168076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:13.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.977+0000 7f4183e00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4168074370 0x7f4168076880 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f418c15bb60 tx=0x7f4174008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:13.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:13.981+0000 7f4188c00640 1 -- 172.21.15.50:0/4120948396 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f418c0b5730 con 0x7f418c0b4de0 2024-09-21T06:06:14.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.197+0000 7f4192e00640 1 -- 172.21.15.50:0/4120948396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f418c15bb90 con 0x7f418c0b4de0 2024-09-21T06:06:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.513+0000 7f4188c00640 1 -- 172.21.15.50:0/4120948396 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v56) ==== 136+0+0 (secure 0 0 0) 0x7f417c0f9140 con 0x7f418c0b4de0 2024-09-21T06:06:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.517+0000 7f4192e00640 1 -- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4168074370 msgr2=0x7f4168076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.517+0000 7f4192e00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4168074370 0x7f4168076880 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f418c15bb60 tx=0x7f4174008040 comp rx=0 tx=0).stop 2024-09-21T06:06:14.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.517+0000 7f4192e00640 1 -- 172.21.15.50:0/4120948396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f418c0b4de0 msgr2=0x7f418c144930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.517+0000 7f4192e00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f418c0b4de0 0x7f418c144930 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f417c0b0040 tx=0x7f417c0a5620 comp rx=0 tx=0).stop 2024-09-21T06:06:14.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.517+0000 7f4192e00640 1 -- 172.21.15.50:0/4120948396 shutdown_connections 2024-09-21T06:06:14.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.517+0000 7f4192e00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f418c1527b0 0x7f418c13f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.517+0000 7f4192e00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f418c0b5c70 0x7f418c13f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.517+0000 7f4192e00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4168074370 0x7f4168076880 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.517+0000 7f4192e00640 1 --2- 172.21.15.50:0/4120948396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f418c0b4de0 0x7f418c144930 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.517+0000 7f4192e00640 1 -- 172.21.15.50:0/4120948396 >> 172.21.15.50:0/4120948396 conn(0x7f418c0b2040 msgr2=0x7f418c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:14.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.517+0000 7f4192e00640 1 -- 172.21.15.50:0/4120948396 shutdown_connections 2024-09-21T06:06:14.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.517+0000 7f4192e00640 1 -- 172.21.15.50:0/4120948396 wait complete. 2024-09-21T06:06:14.540 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:06:14.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.697+0000 7f5614600640 1 -- 172.21.15.50:0/3031953368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560c072100 msgr2=0x7f560c155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.697+0000 7f5614600640 1 --2- 172.21.15.50:0/3031953368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560c072100 0x7f560c155b50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5600009900 tx=0x7f560001c8e0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.697+0000 7f5614600640 1 -- 172.21.15.50:0/3031953368 shutdown_connections 2024-09-21T06:06:14.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.697+0000 7f5614600640 1 --2- 172.21.15.50:0/3031953368 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f560c073a90 0x7f560c073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.697+0000 7f5614600640 1 --2- 172.21.15.50:0/3031953368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560c072100 0x7f560c155b50 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.697+0000 7f5614600640 1 -- 172.21.15.50:0/3031953368 >> 172.21.15.50:0/3031953368 conn(0x7f560c0b2040 msgr2=0x7f560c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:14.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.697+0000 7f5614600640 1 -- 172.21.15.50:0/3031953368 shutdown_connections 2024-09-21T06:06:14.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.697+0000 7f5614600640 1 -- 172.21.15.50:0/3031953368 wait complete. 2024-09-21T06:06:14.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.701+0000 7f5614600640 1 Processor -- start 2024-09-21T06:06:14.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.701+0000 7f5614600640 1 -- start start 2024-09-21T06:06:14.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.701+0000 7f5614600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f560c072100 0x7f560c14b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.701+0000 7f5614600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560c073a90 0x7f560c157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.701+0000 7f5614600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f560c0b4de0 0x7f560c14e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.701+0000 7f5614600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f560c168e80 con 0x7f560c073a90 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.701+0000 7f5614600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f560c168d00 con 0x7f560c0b4de0 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.701+0000 7f5614600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f560c169000 con 0x7f560c072100 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5611200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560c073a90 0x7f560c157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5611200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560c073a90 0x7f560c157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53198/0 (socket says 172.21.15.50:53198) 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5611200640 1 -- 172.21.15.50:0/2092345328 learned_addr learned my addr 172.21.15.50:0/2092345328 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5611c00640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f560c072100 0x7f560c14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5612600640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f560c0b4de0 0x7f560c14e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5611200640 1 -- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f560c072100 msgr2=0x7f560c14b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5611200640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f560c072100 0x7f560c14b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5611200640 1 -- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f560c0b4de0 msgr2=0x7f560c14e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5611200640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f560c0b4de0 0x7f560c14e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5611200640 1 -- 172.21.15.50:0/2092345328 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c072fb0 con 0x7f560c073a90 2024-09-21T06:06:14.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5612600640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f560c0b4de0 0x7f560c14e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:14.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5611200640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560c073a90 0x7f560c157600 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f55f800c370 tx=0x7f55f800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:14.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5609600640 1 -- 172.21.15.50:0/2092345328 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8017070 con 0x7f560c073a90 2024-09-21T06:06:14.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5609600640 1 -- 172.21.15.50:0/2092345328 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55f8003600 con 0x7f560c073a90 2024-09-21T06:06:14.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5609600640 1 -- 172.21.15.50:0/2092345328 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8003ab0 con 0x7f560c073a90 2024-09-21T06:06:14.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5614600640 1 -- 172.21.15.50:0/2092345328 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c1f6f00 con 0x7f560c073a90 2024-09-21T06:06:14.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5614600640 1 -- 172.21.15.50:0/2092345328 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c15a840 con 0x7f560c073a90 2024-09-21T06:06:14.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5614600640 1 -- 172.21.15.50:0/2092345328 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560c155b50 con 0x7f560c073a90 2024-09-21T06:06:14.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.705+0000 7f5609600640 1 -- 172.21.15.50:0/2092345328 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f55f801a070 con 0x7f560c073a90 2024-09-21T06:06:14.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.709+0000 7f5609600640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55ec074370 0x7f55ec076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.709+0000 7f5609600640 1 -- 172.21.15.50:0/2092345328 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f55f80928e0 con 0x7f560c073a90 2024-09-21T06:06:14.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.709+0000 7f5611c00640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55ec074370 0x7f55ec076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.709+0000 7f5611c00640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55ec074370 0x7f55ec076880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f56000af000 tx=0x7f560001b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:14.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.709+0000 7f5609600640 1 -- 172.21.15.50:0/2092345328 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f55f8021020 con 0x7f560c073a90 2024-09-21T06:06:14.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.913+0000 7f5614600640 1 -- 172.21.15.50:0/2092345328 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f560c06b540 con 0x7f55ec074370 2024-09-21T06:06:14.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.913+0000 7f5609600640 1 -- 172.21.15.50:0/2092345328 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f56000b3040 con 0x7f55ec074370 2024-09-21T06:06:14.915 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:06:14.915 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:06:14.915 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:06:14.915 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:06:14.916 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:06:14.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.917+0000 7f5614600640 1 -- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55ec074370 msgr2=0x7f55ec076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.917+0000 7f5614600640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55ec074370 0x7f55ec076880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f56000af000 tx=0x7f560001b120 comp rx=0 tx=0).stop 2024-09-21T06:06:14.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.917+0000 7f5614600640 1 -- 172.21.15.50:0/2092345328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560c073a90 msgr2=0x7f560c157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.917+0000 7f5614600640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560c073a90 0x7f560c157600 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f55f800c370 tx=0x7f55f800c830 comp rx=0 tx=0).stop 2024-09-21T06:06:14.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.917+0000 7f5614600640 1 -- 172.21.15.50:0/2092345328 shutdown_connections 2024-09-21T06:06:14.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.917+0000 7f5614600640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f560c0b4de0 0x7f560c14e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.917+0000 7f5614600640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f55ec074370 0x7f55ec076880 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.917+0000 7f5614600640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560c073a90 0x7f560c157600 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.917+0000 7f5614600640 1 --2- 172.21.15.50:0/2092345328 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f560c072100 0x7f560c14b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.917+0000 7f5614600640 1 -- 172.21.15.50:0/2092345328 >> 172.21.15.50:0/2092345328 conn(0x7f560c0b2040 msgr2=0x7f560c1569f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:14.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.917+0000 7f5614600640 1 -- 172.21.15.50:0/2092345328 shutdown_connections 2024-09-21T06:06:14.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:14.917+0000 7f5614600640 1 -- 172.21.15.50:0/2092345328 wait complete. 2024-09-21T06:06:14.938 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:06:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.097+0000 7f9475e00640 1 -- 172.21.15.50:0/390396087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94700b3340 msgr2=0x7f94700b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.097+0000 7f9475e00640 1 --2- 172.21.15.50:0/390396087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94700b3340 0x7f94700b3750 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9468009bc0 tx=0x7f946801cba0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.097+0000 7f9475e00640 1 -- 172.21.15.50:0/390396087 shutdown_connections 2024-09-21T06:06:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.097+0000 7f9475e00640 1 --2- 172.21.15.50:0/390396087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94700b3340 0x7f94700b3750 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.097+0000 7f9475e00640 1 --2- 172.21.15.50:0/390396087 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f94700cefa0 0x7f94700cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.097+0000 7f9475e00640 1 -- 172.21.15.50:0/390396087 >> 172.21.15.50:0/390396087 conn(0x7f94700b2040 msgr2=0x7f94700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.097+0000 7f9475e00640 1 -- 172.21.15.50:0/390396087 shutdown_connections 2024-09-21T06:06:15.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.097+0000 7f9475e00640 1 -- 172.21.15.50:0/390396087 wait complete. 2024-09-21T06:06:15.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f9475e00640 1 Processor -- start 2024-09-21T06:06:15.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f9475e00640 1 -- start start 2024-09-21T06:06:15.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f9475e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94700749a0 0x7f9470165a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:15.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f9475e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f94700b3340 0x7f94700aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:15.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f9475e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94700cefa0 0x7f94700af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:15.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f9475e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9470169cb0 con 0x7f94700749a0 2024-09-21T06:06:15.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f9475e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9470169b30 con 0x7f94700cefa0 2024-09-21T06:06:15.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f9475e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9470169e30 con 0x7f94700b3340 2024-09-21T06:06:15.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f946ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94700749a0 0x7f9470165a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:15.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f946ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94700749a0 0x7f9470165a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53210/0 (socket says 172.21.15.50:53210) 2024-09-21T06:06:15.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f946ea00640 1 -- 172.21.15.50:0/222639688 learned_addr learned my addr 172.21.15.50:0/222639688 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:15.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f946f400640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94700cefa0 0x7f94700af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:15.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f946e000640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f94700b3340 0x7f94700aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:15.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f946ea00640 1 -- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f94700b3340 msgr2=0x7f94700aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f946ea00640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f94700b3340 0x7f94700aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f946ea00640 1 -- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94700cefa0 msgr2=0x7f94700af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f946ea00640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94700cefa0 0x7f94700af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f946ea00640 1 -- 172.21.15.50:0/222639688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947013c9e0 con 0x7f94700749a0 2024-09-21T06:06:15.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f946ea00640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94700749a0 0x7f9470165a40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f946000c400 tx=0x7f946000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:15.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f9467400640 1 -- 172.21.15.50:0/222639688 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460011790 con 0x7f94700749a0 2024-09-21T06:06:15.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f9467400640 1 -- 172.21.15.50:0/222639688 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9460017070 con 0x7f94700749a0 2024-09-21T06:06:15.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f9475e00640 1 -- 172.21.15.50:0/222639688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947013ccc0 con 0x7f94700749a0 2024-09-21T06:06:15.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.101+0000 7f9467400640 1 -- 172.21.15.50:0/222639688 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460011cc0 con 0x7f94700749a0 2024-09-21T06:06:15.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.105+0000 7f9475e00640 1 -- 172.21.15.50:0/222639688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947013d280 con 0x7f94700749a0 2024-09-21T06:06:15.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.105+0000 7f9475e00640 1 -- 172.21.15.50:0/222639688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94700b3750 con 0x7f94700749a0 2024-09-21T06:06:15.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.105+0000 7f9467400640 1 -- 172.21.15.50:0/222639688 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f946001c460 con 0x7f94700749a0 2024-09-21T06:06:15.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.105+0000 7f9467400640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9450074370 0x7f9450076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:15.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.105+0000 7f9467400640 1 -- 172.21.15.50:0/222639688 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f9460093880 con 0x7f94700749a0 2024-09-21T06:06:15.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.109+0000 7f946e000640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9450074370 0x7f9450076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:15.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.109+0000 7f946e000640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9450074370 0x7f9450076880 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9470146330 tx=0x7f945c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.109+0000 7f9467400640 1 -- 172.21.15.50:0/222639688 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f94701430e0 con 0x7f94700749a0 2024-09-21T06:06:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.285+0000 7f9475e00640 1 -- 172.21.15.50:0/222639688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9470165f80 con 0x7f94700749a0 2024-09-21T06:06:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.285+0000 7f9467400640 1 -- 172.21.15.50:0/222639688 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f946005f9e0 con 0x7f94700749a0 2024-09-21T06:06:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.289+0000 7f9475e00640 1 -- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9450074370 msgr2=0x7f9450076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.289+0000 7f9475e00640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9450074370 0x7f9450076880 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9470146330 tx=0x7f945c00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.289+0000 7f9475e00640 1 -- 172.21.15.50:0/222639688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94700749a0 msgr2=0x7f9470165a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.289+0000 7f9475e00640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94700749a0 0x7f9470165a40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f946000c400 tx=0x7f946000c8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.289+0000 7f9475e00640 1 -- 172.21.15.50:0/222639688 shutdown_connections 2024-09-21T06:06:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.289+0000 7f9475e00640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94700cefa0 0x7f94700af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.289+0000 7f9475e00640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f94700b3340 0x7f94700aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.289+0000 7f9475e00640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9450074370 0x7f9450076880 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.289+0000 7f9475e00640 1 --2- 172.21.15.50:0/222639688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94700749a0 0x7f9470165a40 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.289+0000 7f9475e00640 1 -- 172.21.15.50:0/222639688 >> 172.21.15.50:0/222639688 conn(0x7f94700b2040 msgr2=0x7f9470074db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.289+0000 7f9475e00640 1 -- 172.21.15.50:0/222639688 shutdown_connections 2024-09-21T06:06:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.289+0000 7f9475e00640 1 -- 172.21.15.50:0/222639688 wait complete. 2024-09-21T06:06:15.310 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-21T06:06:15.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.469+0000 7fde94e00640 1 -- 172.21.15.50:0/2764979868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde74005160 msgr2=0x7fde7400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.469+0000 7fde94e00640 1 --2- 172.21.15.50:0/2764979868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde74005160 0x7fde7400b9d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fde88001dc0 tx=0x7fde8801caf0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.469+0000 7fde8fe00640 1 -- 172.21.15.50:0/2764979868 reap_dead start 2024-09-21T06:06:15.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.469+0000 7fde94e00640 1 -- 172.21.15.50:0/2764979868 shutdown_connections 2024-09-21T06:06:15.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.469+0000 7fde94e00640 1 -- 172.21.15.50:0/2764979868 >> 172.21.15.50:0/2764979868 conn(0x7fde9006d3f0 msgr2=0x7fde900b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:15.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.469+0000 7fde94e00640 1 -- 172.21.15.50:0/2764979868 shutdown_connections 2024-09-21T06:06:15.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.469+0000 7fde94e00640 1 -- 172.21.15.50:0/2764979868 wait complete. 2024-09-21T06:06:15.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde94e00640 1 Processor -- start 2024-09-21T06:06:15.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde94e00640 1 -- start start 2024-09-21T06:06:15.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde94e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde74005160 0x7fde901f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:15.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde94e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fde900aa4d0 0x7fde901f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:15.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde94e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde901f3110 0x7fde901f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde94e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde900b58f0 con 0x7fde74005160 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde94e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde900b0230 con 0x7fde901f3110 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde94e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fde900cfe60 con 0x7fde900aa4d0 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde8f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde74005160 0x7fde901f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde8f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde74005160 0x7fde901f6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53230/0 (socket says 172.21.15.50:53230) 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde8f400640 1 -- 172.21.15.50:0/3771205649 learned_addr learned my addr 172.21.15.50:0/3771205649 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde8fe00640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde901f3110 0x7fde901f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde87e00640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fde900aa4d0 0x7fde901f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde8f400640 1 -- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fde900aa4d0 msgr2=0x7fde901f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde8f400640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fde900aa4d0 0x7fde901f66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde8f400640 1 -- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde901f3110 msgr2=0x7fde901f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde8f400640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde901f3110 0x7fde901f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde8f400640 1 -- 172.21.15.50:0/3771205649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde900aacb0 con 0x7fde74005160 2024-09-21T06:06:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde87e00640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fde900aa4d0 0x7fde901f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde8f400640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde74005160 0x7fde901f6190 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fde80000fc0 tx=0x7fde80012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde87400640 1 -- 172.21.15.50:0/3771205649 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80010b40 con 0x7fde74005160 2024-09-21T06:06:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde87400640 1 -- 172.21.15.50:0/3771205649 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde80017030 con 0x7fde74005160 2024-09-21T06:06:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde94e00640 1 -- 172.21.15.50:0/3771205649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde900aaf90 con 0x7fde74005160 2024-09-21T06:06:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde87400640 1 -- 172.21.15.50:0/3771205649 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80013450 con 0x7fde74005160 2024-09-21T06:06:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.473+0000 7fde94e00640 1 -- 172.21.15.50:0/3771205649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde900ab550 con 0x7fde74005160 2024-09-21T06:06:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.477+0000 7fde94e00640 1 -- 172.21.15.50:0/3771205649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde90070140 con 0x7fde74005160 2024-09-21T06:06:15.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.477+0000 7fde87400640 1 -- 172.21.15.50:0/3771205649 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7fde800135f0 con 0x7fde74005160 2024-09-21T06:06:15.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.477+0000 7fde87400640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fde6c074370 0x7fde6c076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:15.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.477+0000 7fde87400640 1 -- 172.21.15.50:0/3771205649 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fde80097440 con 0x7fde74005160 2024-09-21T06:06:15.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.477+0000 7fde87e00640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fde6c074370 0x7fde6c076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:15.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.481+0000 7fde87e00640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fde6c074370 0x7fde6c076880 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fde900b2930 tx=0x7fde780091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:15.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.481+0000 7fde87400640 1 -- 172.21.15.50:0/3771205649 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fde80063510 con 0x7fde74005160 2024-09-21T06:06:15.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.657+0000 7fde94e00640 1 -- 172.21.15.50:0/3771205649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fde90069d20 con 0x7fde74005160 2024-09-21T06:06:15.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde87400640 1 -- 172.21.15.50:0/3771205649 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v45) ==== 116+0+2 (secure 0 0 0) 0x7fde80067f40 con 0x7fde74005160 2024-09-21T06:06:15.661 INFO:teuthology.orchestra.run.smithi050.stdout:4 2024-09-21T06:06:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde94e00640 1 -- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fde6c074370 msgr2=0x7fde6c076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde94e00640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fde6c074370 0x7fde6c076880 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fde900b2930 tx=0x7fde780091e0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde94e00640 1 -- 172.21.15.50:0/3771205649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde74005160 msgr2=0x7fde901f6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde94e00640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde74005160 0x7fde901f6190 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fde80000fc0 tx=0x7fde80012b20 comp rx=0 tx=0).stop 2024-09-21T06:06:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde94e00640 1 -- 172.21.15.50:0/3771205649 shutdown_connections 2024-09-21T06:06:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde94e00640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde901f3110 0x7fde901f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde94e00640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fde900aa4d0 0x7fde901f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde94e00640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fde6c074370 0x7fde6c076880 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde94e00640 1 --2- 172.21.15.50:0/3771205649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde74005160 0x7fde901f6190 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde94e00640 1 -- 172.21.15.50:0/3771205649 >> 172.21.15.50:0/3771205649 conn(0x7fde9006d3f0 msgr2=0x7fde900b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde94e00640 1 -- 172.21.15.50:0/3771205649 shutdown_connections 2024-09-21T06:06:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.661+0000 7fde94e00640 1 -- 172.21.15.50:0/3771205649 wait complete. 2024-09-21T06:06:15.684 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:06:15.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.845+0000 7f873ca00640 1 -- 172.21.15.50:0/2574842676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87380b4de0 msgr2=0x7f87380b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.845+0000 7f873ca00640 1 --2- 172.21.15.50:0/2574842676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87380b4de0 0x7f87380b51f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8730009900 tx=0x7f873001c820 comp rx=0 tx=0).stop 2024-09-21T06:06:15.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.845+0000 7f873ca00640 1 -- 172.21.15.50:0/2574842676 shutdown_connections 2024-09-21T06:06:15.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.845+0000 7f873ca00640 1 --2- 172.21.15.50:0/2574842676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87380b4de0 0x7f87380b51f0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.845+0000 7f873ca00640 1 --2- 172.21.15.50:0/2574842676 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87381527b0 0x7f873815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.845+0000 7f873ca00640 1 -- 172.21.15.50:0/2574842676 >> 172.21.15.50:0/2574842676 conn(0x7f87380b2040 msgr2=0x7f87380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:15.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.845+0000 7f873ca00640 1 -- 172.21.15.50:0/2574842676 shutdown_connections 2024-09-21T06:06:15.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.845+0000 7f873ca00640 1 -- 172.21.15.50:0/2574842676 wait complete. 2024-09-21T06:06:15.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f873ca00640 1 Processor -- start 2024-09-21T06:06:15.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f873ca00640 1 -- start start 2024-09-21T06:06:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f873ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87380b4de0 0x7f87380ab050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f873ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87380b5c70 0x7f87380cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f873ca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87381527b0 0x7f87380ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f873ca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8738168ab0 con 0x7f87380b5c70 2024-09-21T06:06:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f873ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8738168930 con 0x7f87380b4de0 2024-09-21T06:06:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f873ca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8738168c30 con 0x7f87381527b0 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f872d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87380b5c70 0x7f87380cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f872d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87380b5c70 0x7f87380cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53250/0 (socket says 172.21.15.50:53250) 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f872d400640 1 -- 172.21.15.50:0/3149116670 learned_addr learned my addr 172.21.15.50:0/3149116670 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f8735e00640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87380b4de0 0x7f87380ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f8736800640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87381527b0 0x7f87380ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f872d400640 1 -- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87381527b0 msgr2=0x7f87380ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f872d400640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87381527b0 0x7f87380ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f872d400640 1 -- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87380b4de0 msgr2=0x7f87380ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f872d400640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87380b4de0 0x7f87380ab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f872d400640 1 -- 172.21.15.50:0/3149116670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87381f78f0 con 0x7f87380b5c70 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f8736800640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87381527b0 0x7f87380ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f872d400640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87380b5c70 0x7f87380cb930 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8720005d10 tx=0x7f872000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f872ea00640 1 -- 172.21.15.50:0/3149116670 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720017070 con 0x7f87380b5c70 2024-09-21T06:06:15.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f872ea00640 1 -- 172.21.15.50:0/3149116670 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8720009ac0 con 0x7f87380b5c70 2024-09-21T06:06:15.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f872ea00640 1 -- 172.21.15.50:0/3149116670 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720010460 con 0x7f87380b5c70 2024-09-21T06:06:15.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f873ca00640 1 -- 172.21.15.50:0/3149116670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87381f7610 con 0x7f87380b5c70 2024-09-21T06:06:15.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.849+0000 7f873ca00640 1 -- 172.21.15.50:0/3149116670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87380aa850 con 0x7f87380b5c70 2024-09-21T06:06:15.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.853+0000 7f873ca00640 1 -- 172.21.15.50:0/3149116670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87380b51f0 con 0x7f87380b5c70 2024-09-21T06:06:15.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.853+0000 7f872ea00640 1 -- 172.21.15.50:0/3149116670 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f872001a070 con 0x7f87380b5c70 2024-09-21T06:06:15.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.853+0000 7f872ea00640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87140743a0 0x7f87140768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:15.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.853+0000 7f872ea00640 1 -- 172.21.15.50:0/3149116670 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f8720093070 con 0x7f87380b5c70 2024-09-21T06:06:15.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.853+0000 7f8735e00640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87140743a0 0x7f87140768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:15.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.857+0000 7f8735e00640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87140743a0 0x7f87140768b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8728001e40 tx=0x7f8728009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:15.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:15.857+0000 7f872ea00640 1 -- 172.21.15.50:0/3149116670 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f872005f230 con 0x7f87380b5c70 2024-09-21T06:06:16.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.037+0000 7f873ca00640 1 -- 172.21.15.50:0/3149116670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f873815a790 con 0x7f87380b5c70 2024-09-21T06:06:16.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f872ea00640 1 -- 172.21.15.50:0/3149116670 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7f8720063c60 con 0x7f87380b5c70 2024-09-21T06:06:16.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f873ca00640 1 -- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87140743a0 msgr2=0x7f87140768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:16.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f873ca00640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87140743a0 0x7f87140768b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8728001e40 tx=0x7f8728009040 comp rx=0 tx=0).stop 2024-09-21T06:06:16.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f873ca00640 1 -- 172.21.15.50:0/3149116670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87380b5c70 msgr2=0x7f87380cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:16.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f873ca00640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87380b5c70 0x7f87380cb930 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8720005d10 tx=0x7f872000dbb0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f873ca00640 1 -- 172.21.15.50:0/3149116670 shutdown_connections 2024-09-21T06:06:16.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f873ca00640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87381527b0 0x7f87380ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f873ca00640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87140743a0 0x7f87140768b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f873ca00640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87380b5c70 0x7f87380cb930 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f873ca00640 1 --2- 172.21.15.50:0/3149116670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87380b4de0 0x7f87380ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f873ca00640 1 -- 172.21.15.50:0/3149116670 >> 172.21.15.50:0/3149116670 conn(0x7f87380b2040 msgr2=0x7f873815af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:16.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f873ca00640 1 -- 172.21.15.50:0/3149116670 shutdown_connections 2024-09-21T06:06:16.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.041+0000 7f873ca00640 1 -- 172.21.15.50:0/3149116670 wait complete. 2024-09-21T06:06:16.065 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-21T06:06:16.065 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:16.065+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:16.065 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:16.065+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:16.065 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:16.065+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:16.065 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:16.065+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:16.066 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:16.065+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:16.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.225+0000 7f0eafc00640 1 -- 172.21.15.50:0/3140758707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e90004d50 msgr2=0x7f0e900051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:16.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.225+0000 7f0eafc00640 1 --2- 172.21.15.50:0/3140758707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e90004d50 0x7f0e900051a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009eb0 tx=0x7f0ea001d5d0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.225+0000 7f0eadc00640 1 -- 172.21.15.50:0/3140758707 reap_dead start 2024-09-21T06:06:16.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.225+0000 7f0eafc00640 1 -- 172.21.15.50:0/3140758707 shutdown_connections 2024-09-21T06:06:16.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.225+0000 7f0eafc00640 1 -- 172.21.15.50:0/3140758707 >> 172.21.15.50:0/3140758707 conn(0x7f0ea80b2040 msgr2=0x7f0ea80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:16.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.225+0000 7f0eafc00640 1 -- 172.21.15.50:0/3140758707 shutdown_connections 2024-09-21T06:06:16.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.225+0000 7f0eafc00640 1 -- 172.21.15.50:0/3140758707 wait complete. 2024-09-21T06:06:16.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eafc00640 1 Processor -- start 2024-09-21T06:06:16.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eafc00640 1 -- start start 2024-09-21T06:06:16.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eafc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e90004d50 0x7f0ea815c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:16.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eafc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ea8145f70 0x7f0ea8144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:16.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eafc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea8144540 0x7f0ea80cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:16.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eafc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8165ac0 con 0x7f0ea8144540 2024-09-21T06:06:16.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eafc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8165c40 con 0x7f0e90004d50 2024-09-21T06:06:16.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eafc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea8165dc0 con 0x7f0ea8145f70 2024-09-21T06:06:16.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eadc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea8144540 0x7f0ea80cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:16.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eadc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea8144540 0x7f0ea80cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53278/0 (socket says 172.21.15.50:53278) 2024-09-21T06:06:16.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eadc00640 1 -- 172.21.15.50:0/918977872 learned_addr learned my addr 172.21.15.50:0/918977872 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:16.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0ead200640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e90004d50 0x7f0ea815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:16.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0ea7e00640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ea8145f70 0x7f0ea8144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:16.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eadc00640 1 -- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ea8145f70 msgr2=0x7f0ea8144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:16.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eadc00640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ea8145f70 0x7f0ea8144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eadc00640 1 -- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e90004d50 msgr2=0x7f0ea815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:16.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eadc00640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e90004d50 0x7f0ea815c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0eadc00640 1 -- 172.21.15.50:0/918977872 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea80cd490 con 0x7f0ea8144540 2024-09-21T06:06:16.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0ea7e00640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ea8145f70 0x7f0ea8144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:16.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.229+0000 7f0ead200640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e90004d50 0x7f0ea815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:16.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.233+0000 7f0eadc00640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea8144540 0x7f0ea80cd940 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009eb0 tx=0x7f0ea0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:16.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.233+0000 7f0ea4c00640 1 -- 172.21.15.50:0/918977872 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0009200 con 0x7f0ea8144540 2024-09-21T06:06:16.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.233+0000 7f0ea4c00640 1 -- 172.21.15.50:0/918977872 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ea00a9510 con 0x7f0ea8144540 2024-09-21T06:06:16.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.233+0000 7f0ea4c00640 1 -- 172.21.15.50:0/918977872 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0003ba0 con 0x7f0ea8144540 2024-09-21T06:06:16.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.233+0000 7f0eafc00640 1 -- 172.21.15.50:0/918977872 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea8144950 con 0x7f0ea8144540 2024-09-21T06:06:16.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.233+0000 7f0eafc00640 1 -- 172.21.15.50:0/918977872 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea81f9260 con 0x7f0ea8144540 2024-09-21T06:06:16.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.233+0000 7f0eafc00640 1 -- 172.21.15.50:0/918977872 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea81f8cf0 con 0x7f0ea8144540 2024-09-21T06:06:16.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.233+0000 7f0ea4c00640 1 -- 172.21.15.50:0/918977872 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f0ea00b9030 con 0x7f0ea8144540 2024-09-21T06:06:16.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.237+0000 7f0ea4c00640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e88074370 0x7f0e88076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:16.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.237+0000 7f0ea4c00640 1 -- 172.21.15.50:0/918977872 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f0ea012dd80 con 0x7f0ea8144540 2024-09-21T06:06:16.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.237+0000 7f0ead200640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e88074370 0x7f0e88076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:16.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.237+0000 7f0ead200640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e88074370 0x7f0e88076880 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0017c0 tx=0x7f0e9c001e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:16.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.237+0000 7f0ea4c00640 1 -- 172.21.15.50:0/918977872 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0ea8144950 con 0x7f0ea8144540 2024-09-21T06:06:16.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.417+0000 7f0eafc00640 1 -- 172.21.15.50:0/918977872 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f0ea8074db0 con 0x7f0ea8144540 2024-09-21T06:06:16.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.421+0000 7f0ea4c00640 1 -- 172.21.15.50:0/918977872 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v47)=0 v47) ==== 136+0+0 (secure 0 0 0) 0x7f0ea00ad030 con 0x7f0ea8144540 2024-09-21T06:06:16.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.421+0000 7f0eafc00640 1 -- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e88074370 msgr2=0x7f0e88076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:16.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.421+0000 7f0eafc00640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e88074370 0x7f0e88076880 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0017c0 tx=0x7f0e9c001e80 comp rx=0 tx=0).stop 2024-09-21T06:06:16.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.421+0000 7f0eafc00640 1 -- 172.21.15.50:0/918977872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea8144540 msgr2=0x7f0ea80cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:16.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.421+0000 7f0eafc00640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea8144540 0x7f0ea80cd940 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009eb0 tx=0x7f0ea0009780 comp rx=0 tx=0).stop 2024-09-21T06:06:16.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.421+0000 7f0eafc00640 1 -- 172.21.15.50:0/918977872 shutdown_connections 2024-09-21T06:06:16.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.421+0000 7f0eafc00640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0e88074370 0x7f0e88076880 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.421+0000 7f0eafc00640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea8144540 0x7f0ea80cd940 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.421+0000 7f0eafc00640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ea8145f70 0x7f0ea8144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.421+0000 7f0eafc00640 1 --2- 172.21.15.50:0/918977872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e90004d50 0x7f0ea815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.421+0000 7f0eafc00640 1 -- 172.21.15.50:0/918977872 >> 172.21.15.50:0/918977872 conn(0x7f0ea80b2040 msgr2=0x7f0ea80b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:16.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.421+0000 7f0eafc00640 1 -- 172.21.15.50:0/918977872 shutdown_connections 2024-09-21T06:06:16.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.425+0000 7f0eafc00640 1 -- 172.21.15.50:0/918977872 wait complete. 2024-09-21T06:06:16.446 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-21T06:06:16.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.605+0000 7f5a04000640 1 -- 172.21.15.50:0/3649893672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59fc069500 msgr2=0x7f59fc15dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:16.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.605+0000 7f5a04000640 1 --2- 172.21.15.50:0/3649893672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59fc069500 0x7f59fc15dee0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009bc0 tx=0x7f59f401cba0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 -- 172.21.15.50:0/3649893672 shutdown_connections 2024-09-21T06:06:16.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 --2- 172.21.15.50:0/3649893672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59fc069500 0x7f59fc15dee0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 -- 172.21.15.50:0/3649893672 >> 172.21.15.50:0/3649893672 conn(0x7f59fc0b35f0 msgr2=0x7f59fc0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:16.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 -- 172.21.15.50:0/3649893672 shutdown_connections 2024-09-21T06:06:16.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 -- 172.21.15.50:0/3649893672 wait complete. 2024-09-21T06:06:16.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 Processor -- start 2024-09-21T06:06:16.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 -- start start 2024-09-21T06:06:16.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59fc0d0e40 0x7f59fc153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:16.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f59fc155ce0 0x7f59fc071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:16.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59fc1560f0 0x7f59fc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:16.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc16b4f0 con 0x7f59fc0d0e40 2024-09-21T06:06:16.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc16b370 con 0x7f59fc1560f0 2024-09-21T06:06:16.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.609+0000 7f5a04000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f59fc16b670 con 0x7f59fc155ce0 2024-09-21T06:06:16.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a01600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59fc0d0e40 0x7f59fc153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:16.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a01600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59fc0d0e40 0x7f59fc153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53296/0 (socket says 172.21.15.50:53296) 2024-09-21T06:06:16.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a01600640 1 -- 172.21.15.50:0/3693533730 learned_addr learned my addr 172.21.15.50:0/3693533730 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:16.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a02000640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59fc1560f0 0x7f59fc071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:16.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a00c00640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f59fc155ce0 0x7f59fc071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:16.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a01600640 1 -- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f59fc155ce0 msgr2=0x7f59fc071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:16.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a01600640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f59fc155ce0 0x7f59fc071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a01600640 1 -- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59fc1560f0 msgr2=0x7f59fc071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:16.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a01600640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59fc1560f0 0x7f59fc071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a01600640 1 -- 172.21.15.50:0/3693533730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc157870 con 0x7f59fc0d0e40 2024-09-21T06:06:16.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a01600640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59fc0d0e40 0x7f59fc153270 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f59f000c3c0 tx=0x7f59f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:16.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f59f9600640 1 -- 172.21.15.50:0/3693533730 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0011790 con 0x7f59fc0d0e40 2024-09-21T06:06:16.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f59f9600640 1 -- 172.21.15.50:0/3693533730 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59f0017070 con 0x7f59fc0d0e40 2024-09-21T06:06:16.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f59f9600640 1 -- 172.21.15.50:0/3693533730 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0011d40 con 0x7f59fc0d0e40 2024-09-21T06:06:16.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a04000640 1 -- 172.21.15.50:0/3693533730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc157b50 con 0x7f59fc0d0e40 2024-09-21T06:06:16.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a04000640 1 -- 172.21.15.50:0/3693533730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc158110 con 0x7f59fc0d0e40 2024-09-21T06:06:16.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f5a04000640 1 -- 172.21.15.50:0/3693533730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59fc0d1250 con 0x7f59fc0d0e40 2024-09-21T06:06:16.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.613+0000 7f59f9600640 1 -- 172.21.15.50:0/3693533730 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7f59f001c460 con 0x7f59fc0d0e40 2024-09-21T06:06:16.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.617+0000 7f59f9600640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f59dc0743a0 0x7f59dc0768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:16.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.617+0000 7f59f9600640 1 -- 172.21.15.50:0/3693533730 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f59f0092f10 con 0x7f59fc0d0e40 2024-09-21T06:06:16.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.617+0000 7f5a00c00640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f59dc0743a0 0x7f59dc0768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:16.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.617+0000 7f5a00c00640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f59dc0743a0 0x7f59dc0768b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0aee30 tx=0x7f59e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:16.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.621+0000 7f59f9600640 1 -- 172.21.15.50:0/3693533730 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f59f005f630 con 0x7f59fc0d0e40 2024-09-21T06:06:16.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.821+0000 7f5a04000640 1 -- 172.21.15.50:0/3693533730 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f59fc06b540 con 0x7f59dc0743a0 2024-09-21T06:06:16.836 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:16.837+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:16.836 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:16.837+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:16.836 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:16.837+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:16.836 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:16.837+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:16.836 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:16.837+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:16.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.873+0000 7f59f9600640 1 -- 172.21.15.50:0/3693533730 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f59e800bd80 con 0x7f59dc0743a0 2024-09-21T06:06:16.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.877+0000 7f5a04000640 1 -- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f59dc0743a0 msgr2=0x7f59dc0768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:16.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.877+0000 7f5a04000640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f59dc0743a0 0x7f59dc0768b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0aee30 tx=0x7f59e8008040 comp rx=0 tx=0).stop 2024-09-21T06:06:16.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.877+0000 7f5a04000640 1 -- 172.21.15.50:0/3693533730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59fc0d0e40 msgr2=0x7f59fc153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:16.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.877+0000 7f5a04000640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59fc0d0e40 0x7f59fc153270 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f59f000c3c0 tx=0x7f59f000c880 comp rx=0 tx=0).stop 2024-09-21T06:06:16.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.877+0000 7f5a04000640 1 -- 172.21.15.50:0/3693533730 shutdown_connections 2024-09-21T06:06:16.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.877+0000 7f5a04000640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59fc1560f0 0x7f59fc071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.877+0000 7f5a04000640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f59fc155ce0 0x7f59fc071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.877+0000 7f5a04000640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f59dc0743a0 0x7f59dc0768b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.877+0000 7f5a04000640 1 --2- 172.21.15.50:0/3693533730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59fc0d0e40 0x7f59fc153270 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:16.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.877+0000 7f5a04000640 1 -- 172.21.15.50:0/3693533730 >> 172.21.15.50:0/3693533730 conn(0x7f59fc0b35f0 msgr2=0x7f59fc0b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:16.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.877+0000 7f5a04000640 1 -- 172.21.15.50:0/3693533730 shutdown_connections 2024-09-21T06:06:16.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:16.877+0000 7f5a04000640 1 -- 172.21.15.50:0/3693533730 wait complete. 2024-09-21T06:06:16.899 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:06:17.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.057+0000 7efdec200640 1 -- 172.21.15.50:0/3139613482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4160d50 msgr2=0x7efde4074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:17.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.057+0000 7efdec200640 1 --2- 172.21.15.50:0/3139613482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4160d50 0x7efde4074410 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009900 tx=0x7efdd001c8e0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.057+0000 7efdec200640 1 -- 172.21.15.50:0/3139613482 shutdown_connections 2024-09-21T06:06:17.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.057+0000 7efdec200640 1 --2- 172.21.15.50:0/3139613482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4160d50 0x7efde4074410 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.057+0000 7efdec200640 1 --2- 172.21.15.50:0/3139613482 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efde4157030 0x7efde40b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.057+0000 7efdec200640 1 -- 172.21.15.50:0/3139613482 >> 172.21.15.50:0/3139613482 conn(0x7efde40708c0 msgr2=0x7efde4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:17.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.057+0000 7efdec200640 1 -- 172.21.15.50:0/3139613482 shutdown_connections 2024-09-21T06:06:17.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.057+0000 7efdec200640 1 -- 172.21.15.50:0/3139613482 wait complete. 2024-09-21T06:06:17.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.061+0000 7efdec200640 1 Processor -- start 2024-09-21T06:06:17.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.061+0000 7efdec200640 1 -- start start 2024-09-21T06:06:17.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.061+0000 7efdec200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde4157030 0x7efde4149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:17.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.061+0000 7efdec200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efde4160d50 0x7efde41fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:17.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.061+0000 7efdec200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4161160 0x7efde41fd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:17.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.061+0000 7efdec200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efde416a850 con 0x7efde4161160 2024-09-21T06:06:17.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.061+0000 7efdec200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efde416a6d0 con 0x7efde4157030 2024-09-21T06:06:17.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.061+0000 7efdec200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7efde416a9d0 con 0x7efde4160d50 2024-09-21T06:06:17.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.061+0000 7efdea200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4161160 0x7efde41fd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:17.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efdea200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4161160 0x7efde41fd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37596/0 (socket says 172.21.15.50:37596) 2024-09-21T06:06:17.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efdea200640 1 -- 172.21.15.50:0/693254132 learned_addr learned my addr 172.21.15.50:0/693254132 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:17.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efde8e00640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efde4160d50 0x7efde41fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:17.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efde9800640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde4157030 0x7efde4149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:17.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efdea200640 1 -- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efde4160d50 msgr2=0x7efde41fcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:17.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efdea200640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efde4160d50 0x7efde41fcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efdea200640 1 -- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde4157030 msgr2=0x7efde4149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:17.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efdea200640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde4157030 0x7efde4149810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efdea200640 1 -- 172.21.15.50:0/693254132 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde414be40 con 0x7efde4161160 2024-09-21T06:06:17.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efdea200640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4161160 0x7efde41fd4c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7efddc00c6b0 tx=0x7efddc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:17.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efde1600640 1 -- 172.21.15.50:0/693254132 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc003b80 con 0x7efde4161160 2024-09-21T06:06:17.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efde1600640 1 -- 172.21.15.50:0/693254132 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efddc003d40 con 0x7efde4161160 2024-09-21T06:06:17.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efdec200640 1 -- 172.21.15.50:0/693254132 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde414c340 con 0x7efde4161160 2024-09-21T06:06:17.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efde1600640 1 -- 172.21.15.50:0/693254132 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc005330 con 0x7efde4161160 2024-09-21T06:06:17.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efdec200640 1 -- 172.21.15.50:0/693254132 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde414c900 con 0x7efde4161160 2024-09-21T06:06:17.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efdec200640 1 -- 172.21.15.50:0/693254132 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde414cb40 con 0x7efde4161160 2024-09-21T06:06:17.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.065+0000 7efde1600640 1 -- 172.21.15.50:0/693254132 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 34) ==== 98815+0+0 (secure 0 0 0) 0x7efddc0054d0 con 0x7efde4161160 2024-09-21T06:06:17.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.069+0000 7efde1600640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdc4074370 0x7efdc4076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:17.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.069+0000 7efde1600640 1 -- 172.21.15.50:0/693254132 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7efddc092f50 con 0x7efde4161160 2024-09-21T06:06:17.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.069+0000 7efde9800640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdc4074370 0x7efdc4076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:17.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.069+0000 7efde9800640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdc4074370 0x7efdc4076880 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efdd80058d0 tx=0x7efdd8005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:17.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.073+0000 7efde1600640 1 -- 172.21.15.50:0/693254132 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efde414cb40 con 0x7efde4161160 2024-09-21T06:06:17.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.273+0000 7efdec200640 1 -- 172.21.15.50:0/693254132 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7efde40b5070 con 0x7efdc4074370 2024-09-21T06:06:17.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.281+0000 7efde1600640 1 -- 172.21.15.50:0/693254132 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7efde40b5070 con 0x7efdc4074370 2024-09-21T06:06:17.284 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/14242017-d837-4e5e-aab2-fde54a141f79 2024-09-21T06:06:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.285+0000 7efdec200640 1 -- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdc4074370 msgr2=0x7efdc4076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.285+0000 7efdec200640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdc4074370 0x7efdc4076880 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efdd80058d0 tx=0x7efdd8005860 comp rx=0 tx=0).stop 2024-09-21T06:06:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.285+0000 7efdec200640 1 -- 172.21.15.50:0/693254132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4161160 msgr2=0x7efde41fd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.285+0000 7efdec200640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4161160 0x7efde41fd4c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7efddc00c6b0 tx=0x7efddc00cb70 comp rx=0 tx=0).stop 2024-09-21T06:06:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.285+0000 7efdec200640 1 -- 172.21.15.50:0/693254132 shutdown_connections 2024-09-21T06:06:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.285+0000 7efdec200640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdc4074370 0x7efdc4076880 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.285+0000 7efdec200640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4161160 0x7efde41fd4c0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.285+0000 7efdec200640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efde4160d50 0x7efde41fcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.285+0000 7efdec200640 1 --2- 172.21.15.50:0/693254132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde4157030 0x7efde4149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.285+0000 7efdec200640 1 -- 172.21.15.50:0/693254132 >> 172.21.15.50:0/693254132 conn(0x7efde40708c0 msgr2=0x7efde4151870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:17.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.285+0000 7efdec200640 1 -- 172.21.15.50:0/693254132 shutdown_connections 2024-09-21T06:06:17.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:17.285+0000 7efdec200640 1 -- 172.21.15.50:0/693254132 wait complete. 2024-09-21T06:06:17.306 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/14242017-d837-4e5e-aab2-fde54a141f79 2024-09-21T06:06:17.307 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:06:17.307 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:06:17.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:17.316 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/14242017-d837-4e5e-aab2-fde54a141f79/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:06:18.004 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:06:18.008 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:06:27.749 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:06:27.749 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:06:27.749 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.3769 s, 103 MB/s 2024-09-21T06:06:27.750 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:06:27.750 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:06:27.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:27.759 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/14242017-d837-4e5e-aab2-fde54a141f79/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:06:38.073 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:06:38.073 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:06:38.073 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.2591 s, 105 MB/s 2024-09-21T06:06:38.074 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:06:38.074 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:06:38.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:38.083 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/14242017-d837-4e5e-aab2-fde54a141f79/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:06:48.062 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:06:48.065 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:06:48.226 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:06:48.226 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:06:48.227 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.093 s, 106 MB/s 2024-09-21T06:06:48.228 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-21T06:06:48.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.393+0000 7f347dc00640 1 -- 172.21.15.50:0/2381483283 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34780cd7a0 msgr2=0x7f34780cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.393+0000 7f347dc00640 1 --2- 172.21.15.50:0/2381483283 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34780cd7a0 0x7f34780cdbb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3460009840 tx=0x7f346001c820 comp rx=0 tx=0).stop 2024-09-21T06:06:48.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.393+0000 7f347dc00640 1 -- 172.21.15.50:0/2381483283 shutdown_connections 2024-09-21T06:06:48.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.393+0000 7f347dc00640 1 --2- 172.21.15.50:0/2381483283 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34780cd7a0 0x7f34780cdbb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.393+0000 7f347dc00640 1 --2- 172.21.15.50:0/2381483283 >> v2:172.21.15.88:3300/0 conn(0x7f347815aa50 0x7f34780cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.393+0000 7f347dc00640 1 -- 172.21.15.50:0/2381483283 >> 172.21.15.50:0/2381483283 conn(0x7f34780b2040 msgr2=0x7f34780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:48.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.393+0000 7f347dc00640 1 -- 172.21.15.50:0/2381483283 shutdown_connections 2024-09-21T06:06:48.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.393+0000 7f347dc00640 1 -- 172.21.15.50:0/2381483283 wait complete. 2024-09-21T06:06:48.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.393+0000 7f347dc00640 1 Processor -- start 2024-09-21T06:06:48.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.393+0000 7f347dc00640 1 -- start start 2024-09-21T06:06:48.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f347dc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34780cd7a0 0x7f34780740a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:48.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f347dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347815aa50 0x7f34781fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:48.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f347dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f347815e4e0 0x7f3478152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:48.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f347dc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34781663a0 con 0x7f347815aa50 2024-09-21T06:06:48.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f347dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3478166220 con 0x7f347815e4e0 2024-09-21T06:06:48.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f347dc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3478166520 con 0x7f34780cd7a0 2024-09-21T06:06:48.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f346fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347815aa50 0x7f34781fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:48.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f346fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347815aa50 0x7f34781fd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49814/0 (socket says 172.21.15.50:49814) 2024-09-21T06:06:48.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f346fe00640 1 -- 172.21.15.50:0/137574764 learned_addr learned my addr 172.21.15.50:0/137574764 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:48.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f3476a00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34780cd7a0 0x7f34780740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:48.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f3477400640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f347815e4e0 0x7f3478152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:48.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f346fe00640 1 -- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34780cd7a0 msgr2=0x7f34780740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f346fe00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34780cd7a0 0x7f34780740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f346fe00640 1 -- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f347815e4e0 msgr2=0x7f3478152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f346fe00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f347815e4e0 0x7f3478152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f346fe00640 1 -- 172.21.15.50:0/137574764 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34780748f0 con 0x7f347815aa50 2024-09-21T06:06:48.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f3477400640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f347815e4e0 0x7f3478152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:48.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f346fe00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347815aa50 0x7f34781fd280 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3460009790 tx=0x7f3460005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:48.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f346ea00640 1 -- 172.21.15.50:0/137574764 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34600bb070 con 0x7f347815aa50 2024-09-21T06:06:48.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f346ea00640 1 -- 172.21.15.50:0/137574764 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3460004330 con 0x7f347815aa50 2024-09-21T06:06:48.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f347dc00640 1 -- 172.21.15.50:0/137574764 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3478074bd0 con 0x7f347815aa50 2024-09-21T06:06:48.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f346ea00640 1 -- 172.21.15.50:0/137574764 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34600b2040 con 0x7f347815aa50 2024-09-21T06:06:48.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f347dc00640 1 -- 172.21.15.50:0/137574764 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34781fca80 con 0x7f347815aa50 2024-09-21T06:06:48.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.397+0000 7f347dc00640 1 -- 172.21.15.50:0/137574764 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3478074660 con 0x7f347815aa50 2024-09-21T06:06:48.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.401+0000 7f346ea00640 1 -- 172.21.15.50:0/137574764 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 35) ==== 98917+0+0 (secure 0 0 0) 0x7f3460003840 con 0x7f347815aa50 2024-09-21T06:06:48.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.401+0000 7f346ea00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f34540744c0 0x7f34540769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:48.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.401+0000 7f346ea00640 1 -- 172.21.15.50:0/137574764 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f346012ca50 con 0x7f347815aa50 2024-09-21T06:06:48.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.401+0000 7f3476a00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f34540744c0 0x7f34540769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:48.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.401+0000 7f3476a00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f34540744c0 0x7f34540769d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3468009ad0 tx=0x7f34680091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:48.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.405+0000 7f346ea00640 1 -- 172.21.15.50:0/137574764 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f34600f2290 con 0x7f347815aa50 2024-09-21T06:06:48.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.609+0000 7f347dc00640 1 -- 172.21.15.50:0/137574764 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f347806b540 con 0x7f34540744c0 2024-09-21T06:06:48.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f346ea00640 1 -- 172.21.15.50:0/137574764 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3468011b20 con 0x7f34540744c0 2024-09-21T06:06:48.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f347dc00640 1 -- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f34540744c0 msgr2=0x7f34540769d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f347dc00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f34540744c0 0x7f34540769d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3468009ad0 tx=0x7f34680091e0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f347dc00640 1 -- 172.21.15.50:0/137574764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347815aa50 msgr2=0x7f34781fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f347dc00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347815aa50 0x7f34781fd280 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3460009790 tx=0x7f3460005d20 comp rx=0 tx=0).stop 2024-09-21T06:06:48.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f347dc00640 1 -- 172.21.15.50:0/137574764 shutdown_connections 2024-09-21T06:06:48.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f347dc00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f347815e4e0 0x7f3478152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f347dc00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f34540744c0 0x7f34540769d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f347dc00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347815aa50 0x7f34781fd280 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f347dc00640 1 --2- 172.21.15.50:0/137574764 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f34780cd7a0 0x7f34780740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f347dc00640 1 -- 172.21.15.50:0/137574764 >> 172.21.15.50:0/137574764 conn(0x7f34780b2040 msgr2=0x7f34780ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:48.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f347dc00640 1 -- 172.21.15.50:0/137574764 shutdown_connections 2024-09-21T06:06:48.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.629+0000 7f347dc00640 1 -- 172.21.15.50:0/137574764 wait complete. 2024-09-21T06:06:48.651 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:06:48.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.813+0000 7f2775000640 1 -- 172.21.15.50:0/2509041765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27701527b0 msgr2=0x7f277015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.813+0000 7f2775000640 1 --2- 172.21.15.50:0/2509041765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27701527b0 0x7f277015bb60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2758009840 tx=0x7f275801c710 comp rx=0 tx=0).stop 2024-09-21T06:06:48.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.813+0000 7f2775000640 1 -- 172.21.15.50:0/2509041765 shutdown_connections 2024-09-21T06:06:48.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.813+0000 7f2775000640 1 --2- 172.21.15.50:0/2509041765 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f27700b4de0 0x7f27700b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.813+0000 7f2775000640 1 --2- 172.21.15.50:0/2509041765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27701527b0 0x7f277015bb60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.813+0000 7f2775000640 1 -- 172.21.15.50:0/2509041765 >> 172.21.15.50:0/2509041765 conn(0x7f27700b2040 msgr2=0x7f27700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:48.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.813+0000 7f2775000640 1 -- 172.21.15.50:0/2509041765 shutdown_connections 2024-09-21T06:06:48.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.813+0000 7f2775000640 1 -- 172.21.15.50:0/2509041765 wait complete. 2024-09-21T06:06:48.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2775000640 1 Processor -- start 2024-09-21T06:06:48.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2775000640 1 -- start start 2024-09-21T06:06:48.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2775000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27700b4de0 0x7f2770158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:48.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2775000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27700b5c70 0x7f2770154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:48.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2775000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f27701527b0 0x7f27701f8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:48.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2775000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2770168970 con 0x7f27700b5c70 2024-09-21T06:06:48.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2775000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f27701687f0 con 0x7f27700b4de0 2024-09-21T06:06:48.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2775000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2770168af0 con 0x7f27701527b0 2024-09-21T06:06:48.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2767e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27700b5c70 0x7f2770154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:48.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2767e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27700b5c70 0x7f2770154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49824/0 (socket says 172.21.15.50:49824) 2024-09-21T06:06:48.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2767e00640 1 -- 172.21.15.50:0/4197004649 learned_addr learned my addr 172.21.15.50:0/4197004649 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:48.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f276e800640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f27701527b0 0x7f27701f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:48.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f276de00640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27700b4de0 0x7f2770158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:48.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2767e00640 1 -- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f27701527b0 msgr2=0x7f27701f8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2767e00640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f27701527b0 0x7f27701f8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2767e00640 1 -- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27700b4de0 msgr2=0x7f2770158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2767e00640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27700b4de0 0x7f2770158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2767e00640 1 -- 172.21.15.50:0/4197004649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2770158aa0 con 0x7f27700b5c70 2024-09-21T06:06:48.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2767e00640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27700b5c70 0x7f2770154da0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2758009790 tx=0x7f2758004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:48.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2766000640 1 -- 172.21.15.50:0/4197004649 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580bb070 con 0x7f27700b5c70 2024-09-21T06:06:48.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2766000640 1 -- 172.21.15.50:0/4197004649 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27580a5be0 con 0x7f27700b5c70 2024-09-21T06:06:48.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2766000640 1 -- 172.21.15.50:0/4197004649 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580b2040 con 0x7f27700b5c70 2024-09-21T06:06:48.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2775000640 1 -- 172.21.15.50:0/4197004649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277014f030 con 0x7f27700b5c70 2024-09-21T06:06:48.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.817+0000 7f2775000640 1 -- 172.21.15.50:0/4197004649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2770157840 con 0x7f27700b5c70 2024-09-21T06:06:48.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.821+0000 7f2775000640 1 -- 172.21.15.50:0/4197004649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27701575b0 con 0x7f27700b5c70 2024-09-21T06:06:48.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.821+0000 7f2766000640 1 -- 172.21.15.50:0/4197004649 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 35) ==== 98917+0+0 (secure 0 0 0) 0x7f2758005030 con 0x7f27700b5c70 2024-09-21T06:06:48.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.821+0000 7f2766000640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f274c0744b0 0x7f274c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:48.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.821+0000 7f2766000640 1 -- 172.21.15.50:0/4197004649 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f275812c650 con 0x7f27700b5c70 2024-09-21T06:06:48.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.821+0000 7f276de00640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f274c0744b0 0x7f274c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:48.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.821+0000 7f276de00640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f274c0744b0 0x7f274c0769c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2760007990 tx=0x7f27600012e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:48.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:48.825+0000 7f2766000640 1 -- 172.21.15.50:0/4197004649 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2770158aa0 con 0x7f27700b5c70 2024-09-21T06:06:49.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.029+0000 7f2775000640 1 -- 172.21.15.50:0/4197004649 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f277006b540 con 0x7f274c0744b0 2024-09-21T06:06:49.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.033+0000 7f2766000640 1 -- 172.21.15.50:0/4197004649 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f276000f850 con 0x7f274c0744b0 2024-09-21T06:06:49.034 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/14242017-d837-4e5e-aab2-fde54a141f79 2024-09-21T06:06:49.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.033+0000 7f2775000640 1 -- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f274c0744b0 msgr2=0x7f274c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:49.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.033+0000 7f2775000640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f274c0744b0 0x7f274c0769c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2760007990 tx=0x7f27600012e0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.033+0000 7f2775000640 1 -- 172.21.15.50:0/4197004649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27700b5c70 msgr2=0x7f2770154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:49.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.033+0000 7f2775000640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27700b5c70 0x7f2770154da0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2758009790 tx=0x7f2758004620 comp rx=0 tx=0).stop 2024-09-21T06:06:49.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.033+0000 7f2775000640 1 -- 172.21.15.50:0/4197004649 shutdown_connections 2024-09-21T06:06:49.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.033+0000 7f2775000640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f27701527b0 0x7f27701f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.033+0000 7f2775000640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f274c0744b0 0x7f274c0769c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.033+0000 7f2775000640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27700b5c70 0x7f2770154da0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.033+0000 7f2775000640 1 --2- 172.21.15.50:0/4197004649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27700b4de0 0x7f2770158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.033+0000 7f2775000640 1 -- 172.21.15.50:0/4197004649 >> 172.21.15.50:0/4197004649 conn(0x7f27700b2040 msgr2=0x7f27700af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:49.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.037+0000 7f2775000640 1 -- 172.21.15.50:0/4197004649 shutdown_connections 2024-09-21T06:06:49.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:49.037+0000 7f2775000640 1 -- 172.21.15.50:0/4197004649 wait complete. 2024-09-21T06:06:49.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:49.056 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/14242017-d837-4e5e-aab2-fde54a141f79 2024-09-21T06:06:49.057 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:06:49.065 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/14242017-d837-4e5e-aab2-fde54a141f79 2024-09-21T06:06:49.066 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="3055190016" 2024-09-21T06:06:49.066 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:49.066 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-21T06:06:51.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:51.067 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/14242017-d837-4e5e-aab2-fde54a141f79 2024-09-21T06:06:51.067 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:06:51.078 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/14242017-d837-4e5e-aab2-fde54a141f79 2024-09-21T06:06:51.078 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="3055190016" 2024-09-21T06:06:51.078 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:51.081 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-21T06:06:53.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:06:53.083 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/14242017-d837-4e5e-aab2-fde54a141f79 2024-09-21T06:06:53.083 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:06:53.094 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/14242017-d837-4e5e-aab2-fde54a141f79 2024-09-21T06:06:53.094 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="3221225472" 2024-09-21T06:06:53.094 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:06:53.095 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-21T06:06:53.096 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81850 2024-09-21T06:06:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.325+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1643289705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbb00041d0 msgr2=0x7fcbb0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.325+0000 7fcbd0e00640 1 --2- 172.21.15.50:0/1643289705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbb00041d0 0x7fcbb0004810 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc072810 tx=0x7fcbb401cea0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.325+0000 7fcbca800640 1 -- 172.21.15.50:0/1643289705 reap_dead start 2024-09-21T06:06:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.325+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1643289705 shutdown_connections 2024-09-21T06:06:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.325+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1643289705 >> 172.21.15.50:0/1643289705 conn(0x7fcbcc0b2c40 msgr2=0x7fcbcc0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.325+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1643289705 shutdown_connections 2024-09-21T06:06:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.325+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1643289705 wait complete. 2024-09-21T06:06:53.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbd0e00640 1 Processor -- start 2024-09-21T06:06:53.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbd0e00640 1 -- start start 2024-09-21T06:06:53.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbd0e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbb00041d0 0x7fcbcc13d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:53.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbd0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbcc14ebc0 0x7fcbcc14dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:53.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbd0e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcbcc14a8e0 0x7fcbcc14e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:53.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbd0e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc0b3b40 con 0x7fcbb00041d0 2024-09-21T06:06:53.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbd0e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc15c9e0 con 0x7fcbcc14ebc0 2024-09-21T06:06:53.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbd0e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fcbcc0b3cc0 con 0x7fcbcc14a8e0 2024-09-21T06:06:53.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbb00041d0 0x7fcbcc13d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:53.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbb00041d0 0x7fcbcc13d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49838/0 (socket says 172.21.15.50:49838) 2024-09-21T06:06:53.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc9e00640 1 -- 172.21.15.50:0/1300952869 learned_addr learned my addr 172.21.15.50:0/1300952869 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:53.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc3e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbcc14ebc0 0x7fcbcc14dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:53.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbca800640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcbcc14a8e0 0x7fcbcc14e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:53.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc9e00640 1 -- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcbcc14a8e0 msgr2=0x7fcbcc14e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc9e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcbcc14a8e0 0x7fcbcc14e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc9e00640 1 -- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbcc14ebc0 msgr2=0x7fcbcc14dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc9e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbcc14ebc0 0x7fcbcc14dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc9e00640 1 -- 172.21.15.50:0/1300952869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbcc13d9f0 con 0x7fcbb00041d0 2024-09-21T06:06:53.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc3e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbcc14ebc0 0x7fcbcc14dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:53.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc9e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbb00041d0 0x7fcbcc13d4b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc011ae0 tx=0x7fcbbc009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:53.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc2000640 1 -- 172.21.15.50:0/1300952869 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc01e070 con 0x7fcbb00041d0 2024-09-21T06:06:53.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc2000640 1 -- 172.21.15.50:0/1300952869 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbbc00fcd0 con 0x7fcbb00041d0 2024-09-21T06:06:53.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1300952869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbcc14ce00 con 0x7fcbb00041d0 2024-09-21T06:06:53.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbc2000640 1 -- 172.21.15.50:0/1300952869 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc019670 con 0x7fcbb00041d0 2024-09-21T06:06:53.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.329+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1300952869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbcc14d3c0 con 0x7fcbb00041d0 2024-09-21T06:06:53.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.333+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1300952869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbcc15f170 con 0x7fcbb00041d0 2024-09-21T06:06:53.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.333+0000 7fcbc2000640 1 -- 172.21.15.50:0/1300952869 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 35) ==== 98917+0+0 (secure 0 0 0) 0x7fcbbc021070 con 0x7fcbb00041d0 2024-09-21T06:06:53.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.333+0000 7fcbc2000640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcba80744c0 0x7fcba80769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:53.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.333+0000 7fcbc2000640 1 -- 172.21.15.50:0/1300952869 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fcbbc0935c0 con 0x7fcbb00041d0 2024-09-21T06:06:53.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.333+0000 7fcbc3e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcba80744c0 0x7fcba80769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:53.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.333+0000 7fcbc3e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcba80744c0 0x7fcba80769d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40ac000 tx=0x7fcbb4019ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:53.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.337+0000 7fcbc2000640 1 -- 172.21.15.50:0/1300952869 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcbbc05fa70 con 0x7fcbb00041d0 2024-09-21T06:06:53.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.553+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1300952869 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81850", "target": ["mon-mgr", ""]}) -- 0x7fcbcc06b540 con 0x7fcba80744c0 2024-09-21T06:06:53.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbc2000640 1 -- 172.21.15.50:0/1300952869 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcbb4001600 con 0x7fcba80744c0 2024-09-21T06:06:53.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcba80744c0 msgr2=0x7fcba80769d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbd0e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcba80744c0 0x7fcba80769d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40ac000 tx=0x7fcbb4019ce0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1300952869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbb00041d0 msgr2=0x7fcbcc13d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbd0e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbb00041d0 0x7fcbcc13d4b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc011ae0 tx=0x7fcbbc009ee0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1300952869 shutdown_connections 2024-09-21T06:06:53.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbd0e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcbcc14a8e0 0x7fcbcc14e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbd0e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbcc14ebc0 0x7fcbcc14dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbd0e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcba80744c0 0x7fcba80769d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbd0e00640 1 --2- 172.21.15.50:0/1300952869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbb00041d0 0x7fcbcc13d4b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1300952869 >> 172.21.15.50:0/1300952869 conn(0x7fcbcc0b2c40 msgr2=0x7fcbcc075f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:53.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1300952869 shutdown_connections 2024-09-21T06:06:53.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.625+0000 7fcbd0e00640 1 -- 172.21.15.50:0/1300952869 wait complete. 2024-09-21T06:06:53.631 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:53.633+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:53.632 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:53.633+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:53.632 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:53.633+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:53.632 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:53.633+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:53.632 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:53.633+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:53.646 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone80951 2024-09-21T06:06:53.650 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:53.649+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:53.650 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:53.649+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:53.650 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:53.649+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:53.650 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:53.649+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:53.650 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:53.649+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:53.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.825+0000 7f7b11600640 1 -- 172.21.15.50:0/1618603101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b0c15bdf0 msgr2=0x7f7b0c165a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.825+0000 7f7b11600640 1 --2- 172.21.15.50:0/1618603101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b0c15bdf0 0x7f7b0c165a50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009c80 tx=0x7f7b0401cda0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.825+0000 7f7b11600640 1 -- 172.21.15.50:0/1618603101 shutdown_connections 2024-09-21T06:06:53.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.825+0000 7f7b11600640 1 --2- 172.21.15.50:0/1618603101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b0c15bdf0 0x7f7b0c165a50 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.825+0000 7f7b11600640 1 --2- 172.21.15.50:0/1618603101 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7b0c15e8c0 0x7f7b0c15ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.825+0000 7f7b11600640 1 -- 172.21.15.50:0/1618603101 >> 172.21.15.50:0/1618603101 conn(0x7f7b0c0b2040 msgr2=0x7f7b0c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:53.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.825+0000 7f7b11600640 1 -- 172.21.15.50:0/1618603101 shutdown_connections 2024-09-21T06:06:53.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.825+0000 7f7b11600640 1 -- 172.21.15.50:0/1618603101 wait complete. 2024-09-21T06:06:53.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.829+0000 7f7b11600640 1 Processor -- start 2024-09-21T06:06:53.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.829+0000 7f7b11600640 1 -- start start 2024-09-21T06:06:53.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.829+0000 7f7b11600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b0c15b2d0 0x7f7b0c13e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:53.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.829+0000 7f7b11600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7b0c15bdf0 0x7f7b0c143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:53.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.829+0000 7f7b11600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b0c15e8c0 0x7f7b0c1fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:53.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.829+0000 7f7b11600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c168b40 con 0x7f7b0c15b2d0 2024-09-21T06:06:53.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.829+0000 7f7b11600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c1689c0 con 0x7f7b0c15e8c0 2024-09-21T06:06:53.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.829+0000 7f7b11600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0c168cc0 con 0x7f7b0c15bdf0 2024-09-21T06:06:53.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.829+0000 7f7b0aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b0c15b2d0 0x7f7b0c13e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:53.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b0aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b0c15b2d0 0x7f7b0c13e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49868/0 (socket says 172.21.15.50:49868) 2024-09-21T06:06:53.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b0aa00640 1 -- 172.21.15.50:0/2846116709 learned_addr learned my addr 172.21.15.50:0/2846116709 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:53.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b0b400640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b0c15e8c0 0x7f7b0c1fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:53.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b02000640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7b0c15bdf0 0x7f7b0c143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:53.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b0aa00640 1 -- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7b0c15bdf0 msgr2=0x7f7b0c143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b0aa00640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7b0c15bdf0 0x7f7b0c143810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b0aa00640 1 -- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b0c15e8c0 msgr2=0x7f7b0c1fbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b0aa00640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b0c15e8c0 0x7f7b0c1fbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b0aa00640 1 -- 172.21.15.50:0/2846116709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c157490 con 0x7f7b0c15b2d0 2024-09-21T06:06:53.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b0aa00640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b0c15b2d0 0x7f7b0c13e420 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00e7c0 tx=0x7f7afc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:53.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b03400640 1 -- 172.21.15.50:0/2846116709 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00ccc0 con 0x7f7b0c15b2d0 2024-09-21T06:06:53.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b03400640 1 -- 172.21.15.50:0/2846116709 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7afc00ce80 con 0x7f7b0c15b2d0 2024-09-21T06:06:53.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b11600640 1 -- 172.21.15.50:0/2846116709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0c157700 con 0x7f7b0c15b2d0 2024-09-21T06:06:53.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b03400640 1 -- 172.21.15.50:0/2846116709 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc005280 con 0x7f7b0c15b2d0 2024-09-21T06:06:53.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b11600640 1 -- 172.21.15.50:0/2846116709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0c157cc0 con 0x7f7b0c15b2d0 2024-09-21T06:06:53.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b11600640 1 -- 172.21.15.50:0/2846116709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b0c15ecd0 con 0x7f7b0c15b2d0 2024-09-21T06:06:53.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.833+0000 7f7b03400640 1 -- 172.21.15.50:0/2846116709 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 35) ==== 98917+0+0 (secure 0 0 0) 0x7f7afc005420 con 0x7f7b0c15b2d0 2024-09-21T06:06:53.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.837+0000 7f7b03400640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ae80744c0 0x7f7ae80769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:53.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.837+0000 7f7b03400640 1 -- 172.21.15.50:0/2846116709 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f7afc093630 con 0x7f7b0c15b2d0 2024-09-21T06:06:53.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.837+0000 7f7b02000640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ae80744c0 0x7f7ae80769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:53.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.837+0000 7f7b02000640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ae80744c0 0x7f7ae80769d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c1573c0 tx=0x7f7af4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:53.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:53.841+0000 7f7b03400640 1 -- 172.21.15.50:0/2846116709 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7afc05fae0 con 0x7f7b0c15b2d0 2024-09-21T06:06:54.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.049+0000 7f7b11600640 1 -- 172.21.15.50:0/2846116709 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone80951", "target": ["mon-mgr", ""]}) -- 0x7f7b0c144820 con 0x7f7ae80744c0 2024-09-21T06:06:54.627 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:54.629+0000 7f656c400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:54.627 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:54.629+0000 7f656c400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:54.627 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:54.629+0000 7f656c400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:54.627 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:54.629+0000 7f656c400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:54.627 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:54.629+0000 7f656c400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:54.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.737+0000 7f7b03400640 1 -- 172.21.15.50:0/2846116709 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7b0c144820 con 0x7f7ae80744c0 2024-09-21T06:06:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.741+0000 7f7b11600640 1 -- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ae80744c0 msgr2=0x7f7ae80769d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.741+0000 7f7b11600640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ae80744c0 0x7f7ae80769d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c1573c0 tx=0x7f7af4009040 comp rx=0 tx=0).stop 2024-09-21T06:06:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.741+0000 7f7b11600640 1 -- 172.21.15.50:0/2846116709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b0c15b2d0 msgr2=0x7f7b0c13e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.741+0000 7f7b11600640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b0c15b2d0 0x7f7b0c13e420 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00e7c0 tx=0x7f7afc00ec80 comp rx=0 tx=0).stop 2024-09-21T06:06:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.741+0000 7f7b11600640 1 -- 172.21.15.50:0/2846116709 shutdown_connections 2024-09-21T06:06:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.741+0000 7f7b11600640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b0c15e8c0 0x7f7b0c1fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.741+0000 7f7b11600640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7b0c15bdf0 0x7f7b0c143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.741+0000 7f7b11600640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ae80744c0 0x7f7ae80769d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.741+0000 7f7b11600640 1 --2- 172.21.15.50:0/2846116709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b0c15b2d0 0x7f7b0c13e420 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:54.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.741+0000 7f7b11600640 1 -- 172.21.15.50:0/2846116709 >> 172.21.15.50:0/2846116709 conn(0x7f7b0c0b2040 msgr2=0x7f7b0c13d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:54.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.741+0000 7f7b11600640 1 -- 172.21.15.50:0/2846116709 shutdown_connections 2024-09-21T06:06:54.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:54.741+0000 7f7b11600640 1 -- 172.21.15.50:0/2846116709 wait complete. 2024-09-21T06:06:54.765 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone72702 2024-09-21T06:06:55.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.033+0000 7faf8a400640 1 -- 172.21.15.50:0/3433893758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf84157030 msgr2=0x7faf840b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:55.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.033+0000 7faf8a400640 1 --2- 172.21.15.50:0/3433893758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf84157030 0x7faf840b5a80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7faf74009900 tx=0x7faf7401c8e0 comp rx=0 tx=0).stop 2024-09-21T06:06:55.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.033+0000 7faf8a400640 1 -- 172.21.15.50:0/3433893758 shutdown_connections 2024-09-21T06:06:55.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.033+0000 7faf8a400640 1 --2- 172.21.15.50:0/3433893758 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faf84074b20 0x7faf8415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:55.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.033+0000 7faf8a400640 1 --2- 172.21.15.50:0/3433893758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf84157030 0x7faf840b5a80 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:55.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.033+0000 7faf8a400640 1 -- 172.21.15.50:0/3433893758 >> 172.21.15.50:0/3433893758 conn(0x7faf840708c0 msgr2=0x7faf84070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:55.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.033+0000 7faf8a400640 1 -- 172.21.15.50:0/3433893758 shutdown_connections 2024-09-21T06:06:55.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.033+0000 7faf8a400640 1 -- 172.21.15.50:0/3433893758 wait complete. 2024-09-21T06:06:55.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.037+0000 7faf8a400640 1 Processor -- start 2024-09-21T06:06:55.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.037+0000 7faf8a400640 1 -- start start 2024-09-21T06:06:55.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf8a400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faf84157030 0x7faf8413ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:55.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf8a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf84160d50 0x7faf8406e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:55.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf8a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf840b3530 0x7faf840b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:55.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf8a400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faf8416a810 con 0x7faf840b3530 2024-09-21T06:06:55.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf8a400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf8416a690 con 0x7faf84160d50 2024-09-21T06:06:55.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf8a400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7faf8416a990 con 0x7faf84157030 2024-09-21T06:06:55.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf83e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf840b3530 0x7faf840b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:55.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf83e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf840b3530 0x7faf840b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49878/0 (socket says 172.21.15.50:49878) 2024-09-21T06:06:55.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf83e00640 1 -- 172.21.15.50:0/135342208 learned_addr learned my addr 172.21.15.50:0/135342208 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:55.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf83400640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faf84157030 0x7faf8413ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:55.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf82a00640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf84160d50 0x7faf8406e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:55.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf83e00640 1 -- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faf84157030 msgr2=0x7faf8413ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:55.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf83e00640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faf84157030 0x7faf8413ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:55.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf83e00640 1 -- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf84160d50 msgr2=0x7faf8406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:55.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf83e00640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf84160d50 0x7faf8406e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:55.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf83e00640 1 -- 172.21.15.50:0/135342208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf840b3e80 con 0x7faf840b3530 2024-09-21T06:06:55.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf83e00640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf840b3530 0x7faf840b3940 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00b9a0 tx=0x7faf7c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:55.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf7b400640 1 -- 172.21.15.50:0/135342208 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c019070 con 0x7faf840b3530 2024-09-21T06:06:55.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf7b400640 1 -- 172.21.15.50:0/135342208 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf7c00ccb0 con 0x7faf840b3530 2024-09-21T06:06:55.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf8a400640 1 -- 172.21.15.50:0/135342208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf8413d360 con 0x7faf840b3530 2024-09-21T06:06:55.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf7b400640 1 -- 172.21.15.50:0/135342208 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0146a0 con 0x7faf840b3530 2024-09-21T06:06:55.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf8a400640 1 -- 172.21.15.50:0/135342208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf841f9c20 con 0x7faf840b3530 2024-09-21T06:06:55.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.041+0000 7faf8a400640 1 -- 172.21.15.50:0/135342208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf8415cf90 con 0x7faf840b3530 2024-09-21T06:06:55.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.045+0000 7faf7b400640 1 -- 172.21.15.50:0/135342208 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 35) ==== 98917+0+0 (secure 0 0 0) 0x7faf7c01c070 con 0x7faf840b3530 2024-09-21T06:06:55.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.045+0000 7faf7b400640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf640744c0 0x7faf640769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:55.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.045+0000 7faf7b400640 1 -- 172.21.15.50:0/135342208 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7faf7c092dd0 con 0x7faf840b3530 2024-09-21T06:06:55.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.045+0000 7faf83400640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf640744c0 0x7faf640769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:55.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.049+0000 7faf7b400640 1 -- 172.21.15.50:0/135342208 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7faf840b3e80 con 0x7faf840b3530 2024-09-21T06:06:55.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.057+0000 7faf83400640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf640744c0 0x7faf640769d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faf740af000 tx=0x7faf7401b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:55.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.445+0000 7faf8a400640 1 -- 172.21.15.50:0/135342208 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone72702", "target": ["mon-mgr", ""]}) -- 0x7faf8406b540 con 0x7faf640744c0 2024-09-21T06:06:55.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:55.701+0000 7faf7b400640 1 -- 172.21.15.50:0/135342208 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 36) ==== 99070+0+0 (secure 0 0 0) 0x7faf7c05b150 con 0x7faf840b3530 2024-09-21T06:06:56.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.185+0000 7faf7b400640 1 -- 172.21.15.50:0/135342208 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf74004140 con 0x7faf640744c0 2024-09-21T06:06:56.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.189+0000 7faf8a400640 1 -- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf640744c0 msgr2=0x7faf640769d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:56.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.189+0000 7faf8a400640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf640744c0 0x7faf640769d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faf740af000 tx=0x7faf7401b120 comp rx=0 tx=0).stop 2024-09-21T06:06:56.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.189+0000 7faf8a400640 1 -- 172.21.15.50:0/135342208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf840b3530 msgr2=0x7faf840b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:56.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.189+0000 7faf8a400640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf840b3530 0x7faf840b3940 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00b9a0 tx=0x7faf7c00be60 comp rx=0 tx=0).stop 2024-09-21T06:06:56.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.189+0000 7faf8a400640 1 -- 172.21.15.50:0/135342208 shutdown_connections 2024-09-21T06:06:56.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.189+0000 7faf8a400640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf640744c0 0x7faf640769d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:56.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.189+0000 7faf8a400640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf840b3530 0x7faf840b3940 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:56.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.189+0000 7faf8a400640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf84160d50 0x7faf8406e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:56.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.189+0000 7faf8a400640 1 --2- 172.21.15.50:0/135342208 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faf84157030 0x7faf8413ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:56.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.189+0000 7faf8a400640 1 -- 172.21.15.50:0/135342208 >> 172.21.15.50:0/135342208 conn(0x7faf840708c0 msgr2=0x7faf8415e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:56.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.189+0000 7faf8a400640 1 -- 172.21.15.50:0/135342208 shutdown_connections 2024-09-21T06:06:56.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.189+0000 7faf8a400640 1 -- 172.21.15.50:0/135342208 wait complete. 2024-09-21T06:06:56.215 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone2443 2024-09-21T06:06:56.233 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:56.233+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:56.233 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:56.233+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:56.233 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:56.233+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:56.233 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:56.233+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:56.233 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:56.233+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.489+0000 7f6ca4a00640 1 -- 172.21.15.50:0/13548453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca006de00 msgr2=0x7f6ca006e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.489+0000 7f6ca4a00640 1 --2- 172.21.15.50:0/13548453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca006de00 0x7f6ca006e210 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009900 tx=0x7f6c9001c8e0 comp rx=0 tx=0).stop 2024-09-21T06:06:56.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.493+0000 7f6ca4a00640 1 -- 172.21.15.50:0/13548453 shutdown_connections 2024-09-21T06:06:56.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.493+0000 7f6ca4a00640 1 --2- 172.21.15.50:0/13548453 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ca00cfbd0 0x7f6ca00cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:56.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.493+0000 7f6ca4a00640 1 --2- 172.21.15.50:0/13548453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca006de00 0x7f6ca006e210 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:56.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.493+0000 7f6ca4a00640 1 -- 172.21.15.50:0/13548453 >> 172.21.15.50:0/13548453 conn(0x7f6ca0073380 msgr2=0x7f6ca0073790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:56.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.493+0000 7f6ca4a00640 1 -- 172.21.15.50:0/13548453 shutdown_connections 2024-09-21T06:06:56.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.493+0000 7f6ca4a00640 1 -- 172.21.15.50:0/13548453 wait complete. 2024-09-21T06:06:56.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.497+0000 7f6ca4a00640 1 Processor -- start 2024-09-21T06:06:56.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.501+0000 7f6ca4a00640 1 -- start start 2024-09-21T06:06:56.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.501+0000 7f6ca4a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ca006de00 0x7f6ca014ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:56.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.501+0000 7f6ca4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0070460 0x7f6ca014ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:56.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.501+0000 7f6ca4a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca00cfbd0 0x7f6ca0200a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:56.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.501+0000 7f6ca4a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0069820 con 0x7f6ca0070460 2024-09-21T06:06:56.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.501+0000 7f6ca4a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca00696a0 con 0x7f6ca00cfbd0 2024-09-21T06:06:56.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.501+0000 7f6ca4a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca00699a0 con 0x7f6ca006de00 2024-09-21T06:06:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca00cfbd0 0x7f6ca0200a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9de00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ca006de00 0x7f6ca014ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca00cfbd0 0x7f6ca0200a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:57380/0 (socket says 172.21.15.50:57380) 2024-09-21T06:06:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9de00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ca006de00 0x7f6ca014ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:60320/0 (socket says 172.21.15.50:60320) 2024-09-21T06:06:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9e800640 1 -- 172.21.15.50:0/845810845 learned_addr learned my addr 172.21.15.50:0/845810845 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9d400640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0070460 0x7f6ca014ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9e800640 1 -- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ca006de00 msgr2=0x7f6ca014ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9e800640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ca006de00 0x7f6ca014ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9e800640 1 -- 172.21.15.50:0/845810845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0070460 msgr2=0x7f6ca014ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9e800640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0070460 0x7f6ca014ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9e800640 1 -- 172.21.15.50:0/845810845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca014f860 con 0x7f6ca00cfbd0 2024-09-21T06:06:56.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9d400640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0070460 0x7f6ca014ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:56.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.505+0000 7f6c9e800640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca00cfbd0 0x7f6ca0200a60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800e970 tx=0x7f6c9800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:56.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.509+0000 7f6c96000640 1 -- 172.21.15.50:0/845810845 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98004d30 con 0x7f6ca00cfbd0 2024-09-21T06:06:56.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.509+0000 7f6c96000640 1 -- 172.21.15.50:0/845810845 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c9800cb20 con 0x7f6ca00cfbd0 2024-09-21T06:06:56.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.509+0000 7f6c96000640 1 -- 172.21.15.50:0/845810845 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98005020 con 0x7f6ca00cfbd0 2024-09-21T06:06:56.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.509+0000 7f6ca4a00640 1 -- 172.21.15.50:0/845810845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca014fb40 con 0x7f6ca00cfbd0 2024-09-21T06:06:56.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.509+0000 7f6ca4a00640 1 -- 172.21.15.50:0/845810845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca0150100 con 0x7f6ca00cfbd0 2024-09-21T06:06:56.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.509+0000 7f6ca4a00640 1 -- 172.21.15.50:0/845810845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca014f5d0 con 0x7f6ca00cfbd0 2024-09-21T06:06:56.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.517+0000 7f6c96000640 1 -- 172.21.15.50:0/845810845 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 36) ==== 99070+0+0 (secure 0 0 0) 0x7f6c980051c0 con 0x7f6ca00cfbd0 2024-09-21T06:06:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.517+0000 7f6c96000640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6c80074740 0x7f6c80076c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.517+0000 7f6c96000640 1 -- 172.21.15.50:0/845810845 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f6c98093fb0 con 0x7f6ca00cfbd0 2024-09-21T06:06:56.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.521+0000 7f6c9de00640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6c80074740 0x7f6c80076c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:56.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.521+0000 7f6c9de00640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6c80074740 0x7f6c80076c50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6c900af000 tx=0x7f6c9001b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:56.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.525+0000 7f6c96000640 1 -- 172.21.15.50:0/845810845 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6c98016030 con 0x7f6ca00cfbd0 2024-09-21T06:06:56.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:56.893+0000 7f6ca4a00640 1 -- 172.21.15.50:0/845810845 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone2443", "target": ["mon-mgr", ""]}) -- 0x7f6ca006b540 con 0x7f6c80074740 2024-09-21T06:06:57.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.653+0000 7f6c96000640 1 -- 172.21.15.50:0/845810845 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6c900b3040 con 0x7f6c80074740 2024-09-21T06:06:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.657+0000 7f6ca4a00640 1 -- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6c80074740 msgr2=0x7f6c80076c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.657+0000 7f6ca4a00640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6c80074740 0x7f6c80076c50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6c900af000 tx=0x7f6c9001b120 comp rx=0 tx=0).stop 2024-09-21T06:06:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.657+0000 7f6ca4a00640 1 -- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca00cfbd0 msgr2=0x7f6ca0200a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.657+0000 7f6ca4a00640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca00cfbd0 0x7f6ca0200a60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800e970 tx=0x7f6c9800ee30 comp rx=0 tx=0).stop 2024-09-21T06:06:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.657+0000 7f6ca4a00640 1 -- 172.21.15.50:0/845810845 shutdown_connections 2024-09-21T06:06:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.657+0000 7f6ca4a00640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca00cfbd0 0x7f6ca0200a60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.657+0000 7f6ca4a00640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6c80074740 0x7f6c80076c50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.657+0000 7f6ca4a00640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0070460 0x7f6ca014ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.657+0000 7f6ca4a00640 1 --2- 172.21.15.50:0/845810845 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ca006de00 0x7f6ca014ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.657+0000 7f6ca4a00640 1 -- 172.21.15.50:0/845810845 >> 172.21.15.50:0/845810845 conn(0x7f6ca0073380 msgr2=0x7f6ca00b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.657+0000 7f6ca4a00640 1 -- 172.21.15.50:0/845810845 shutdown_connections 2024-09-21T06:06:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.657+0000 7f6ca4a00640 1 -- 172.21.15.50:0/845810845 wait complete. 2024-09-21T06:06:57.693 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone96264 2024-09-21T06:06:57.700 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:57.701+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:57.701 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:57.701+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:57.701 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:57.701+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:57.701 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:57.701+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:57.701 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:06:57.701+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:06:57.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.965+0000 7f0b80e00640 1 -- 172.21.15.50:0/1830948285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b7c0b5c70 msgr2=0x7f0b7c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:57.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.965+0000 7f0b80e00640 1 --2- 172.21.15.50:0/1830948285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b7c0b5c70 0x7f0b7c15bb60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009900 tx=0x7f0b6c01c6d0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.965+0000 7f0b80e00640 1 -- 172.21.15.50:0/1830948285 shutdown_connections 2024-09-21T06:06:57.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.965+0000 7f0b80e00640 1 --2- 172.21.15.50:0/1830948285 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b7c0b4de0 0x7f0b7c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.965+0000 7f0b80e00640 1 --2- 172.21.15.50:0/1830948285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b7c0b5c70 0x7f0b7c15bb60 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.965+0000 7f0b80e00640 1 -- 172.21.15.50:0/1830948285 >> 172.21.15.50:0/1830948285 conn(0x7f0b7c0b2040 msgr2=0x7f0b7c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:57.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.965+0000 7f0b80e00640 1 -- 172.21.15.50:0/1830948285 shutdown_connections 2024-09-21T06:06:57.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.965+0000 7f0b80e00640 1 -- 172.21.15.50:0/1830948285 wait complete. 2024-09-21T06:06:57.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.965+0000 7f0b80e00640 1 Processor -- start 2024-09-21T06:06:57.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.965+0000 7f0b80e00640 1 -- start start 2024-09-21T06:06:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b80e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b7c0b4de0 0x7f0b7c157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b80e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b7c0b5c70 0x7f0b7c14fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b80e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b7c1527b0 0x7f0b7c1f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b80e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c168a10 con 0x7f0b7c0b4de0 2024-09-21T06:06:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b80e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c168890 con 0x7f0b7c0b5c70 2024-09-21T06:06:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b80e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c168b90 con 0x7f0b7c1527b0 2024-09-21T06:06:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b79e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b7c0b4de0 0x7f0b7c157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b79e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b7c0b4de0 0x7f0b7c157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47168/0 (socket says 172.21.15.50:47168) 2024-09-21T06:06:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b79e00640 1 -- 172.21.15.50:0/2740697208 learned_addr learned my addr 172.21.15.50:0/2740697208 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:06:57.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b79e00640 1 -- 172.21.15.50:0/2740697208 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b7c1527b0 msgr2=0x7f0b7c1f9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:57.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b79e00640 1 --2- 172.21.15.50:0/2740697208 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b7c1527b0 0x7f0b7c1f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b79e00640 1 -- 172.21.15.50:0/2740697208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b7c0b5c70 msgr2=0x7f0b7c14fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:57.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b79e00640 1 --2- 172.21.15.50:0/2740697208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b7c0b5c70 0x7f0b7c14fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b79e00640 1 -- 172.21.15.50:0/2740697208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c1561c0 con 0x7f0b7c0b4de0 2024-09-21T06:06:57.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b79e00640 1 --2- 172.21.15.50:0/2740697208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b7c0b4de0 0x7f0b7c157760 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009850 tx=0x7f0b6c0a5590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:57.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b72000640 1 -- 172.21.15.50:0/2740697208 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0b2040 con 0x7f0b7c0b4de0 2024-09-21T06:06:57.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b72000640 1 -- 172.21.15.50:0/2740697208 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b6c0043b0 con 0x7f0b7c0b4de0 2024-09-21T06:06:57.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b72000640 1 -- 172.21.15.50:0/2740697208 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c003860 con 0x7f0b7c0b4de0 2024-09-21T06:06:57.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b80e00640 1 -- 172.21.15.50:0/2740697208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c1566c0 con 0x7f0b7c0b4de0 2024-09-21T06:06:57.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b80e00640 1 -- 172.21.15.50:0/2740697208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c156c80 con 0x7f0b7c0b4de0 2024-09-21T06:06:57.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.969+0000 7f0b80e00640 1 -- 172.21.15.50:0/2740697208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c0b51f0 con 0x7f0b7c0b4de0 2024-09-21T06:06:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.973+0000 7f0b72000640 1 -- 172.21.15.50:0/2740697208 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 37) ==== 99121+0+0 (secure 0 0 0) 0x7f0b6c0c0060 con 0x7f0b7c0b4de0 2024-09-21T06:06:57.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.973+0000 7f0b72000640 1 --2- 172.21.15.50:0/2740697208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b5c0746d0 0x7f0b5c076be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:57.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.973+0000 7f0b72000640 1 -- 172.21.15.50:0/2740697208 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f0b6c12cf30 con 0x7f0b7c0b4de0 2024-09-21T06:06:57.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:57.981+0000 7f0b72000640 1 -- 172.21.15.50:0/2740697208 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0b7c0b5730 con 0x7f0b7c0b4de0 2024-09-21T06:06:58.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:58.013+0000 7f0b79400640 1 --2- 172.21.15.50:0/2740697208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b5c0746d0 0x7f0b5c076be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:58.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:58.085+0000 7f0b79400640 1 --2- 172.21.15.50:0/2740697208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b5c0746d0 0x7f0b5c076be0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009650 tx=0x7f0b68006bb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:58.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:58.381+0000 7f0b80e00640 1 -- 172.21.15.50:0/2740697208 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone96264", "target": ["mon-mgr", ""]}) -- 0x7f0b7c06b540 con 0x7f0b5c0746d0 2024-09-21T06:06:59.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:06:59.809+0000 7f0b72000640 1 -- 172.21.15.50:0/2740697208 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f0b6c0f5580 con 0x7f0b7c0b4de0 2024-09-21T06:07:00.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.657+0000 7f0b72000640 1 -- 172.21.15.50:0/2740697208 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0b68009280 con 0x7f0b5c0746d0 2024-09-21T06:07:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.661+0000 7f0b80e00640 1 -- 172.21.15.50:0/2740697208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b5c0746d0 msgr2=0x7f0b5c076be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.661+0000 7f0b80e00640 1 --2- 172.21.15.50:0/2740697208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b5c0746d0 0x7f0b5c076be0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009650 tx=0x7f0b68006bb0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.661+0000 7f0b80e00640 1 -- 172.21.15.50:0/2740697208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b7c0b4de0 msgr2=0x7f0b7c157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.661+0000 7f0b80e00640 1 --2- 172.21.15.50:0/2740697208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b7c0b4de0 0x7f0b7c157760 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009850 tx=0x7f0b6c0a5590 comp rx=0 tx=0).stop 2024-09-21T06:07:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.661+0000 7f0b80e00640 1 -- 172.21.15.50:0/2740697208 shutdown_connections 2024-09-21T06:07:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.661+0000 7f0b80e00640 1 --2- 172.21.15.50:0/2740697208 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b7c1527b0 0x7f0b7c1f9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.661+0000 7f0b80e00640 1 --2- 172.21.15.50:0/2740697208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b7c0b5c70 0x7f0b7c14fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.661+0000 7f0b80e00640 1 --2- 172.21.15.50:0/2740697208 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b5c0746d0 0x7f0b5c076be0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.661+0000 7f0b80e00640 1 --2- 172.21.15.50:0/2740697208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b7c0b4de0 0x7f0b7c157760 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.661+0000 7f0b80e00640 1 -- 172.21.15.50:0/2740697208 >> 172.21.15.50:0/2740697208 conn(0x7f0b7c0b2040 msgr2=0x7f0b7c0b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.661+0000 7f0b80e00640 1 -- 172.21.15.50:0/2740697208 shutdown_connections 2024-09-21T06:07:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.661+0000 7f0b80e00640 1 -- 172.21.15.50:0/2740697208 wait complete. 2024-09-21T06:07:00.717 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:07:00.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.973+0000 7f27cb200640 1 -- 172.21.15.50:0/3224379368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27cc06dbc0 msgr2=0x7f27cc06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:00.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.973+0000 7f27cb200640 1 --2- 172.21.15.50:0/3224379368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27cc06dbc0 0x7f27cc06dfd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009870 tx=0x7f27b801c790 comp rx=0 tx=0).stop 2024-09-21T06:07:00.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.973+0000 7f27cb200640 1 -- 172.21.15.50:0/3224379368 shutdown_connections 2024-09-21T06:07:00.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.973+0000 7f27cb200640 1 --2- 172.21.15.50:0/3224379368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27cc06dbc0 0x7f27cc06dfd0 secure :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009870 tx=0x7f27b801c790 comp rx=0 tx=0).stop 2024-09-21T06:07:00.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.973+0000 7f27cb200640 1 --2- 172.21.15.50:0/3224379368 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f27cc0d1140 0x7f27cc0b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.973+0000 7f27cb200640 1 -- 172.21.15.50:0/3224379368 >> 172.21.15.50:0/3224379368 conn(0x7f27cc0b41f0 msgr2=0x7f27cc0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:00.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.973+0000 7f27cb200640 1 -- 172.21.15.50:0/3224379368 shutdown_connections 2024-09-21T06:07:00.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.973+0000 7f27cb200640 1 -- 172.21.15.50:0/3224379368 wait complete. 2024-09-21T06:07:00.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.977+0000 7f27cb200640 1 Processor -- start 2024-09-21T06:07:00.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.977+0000 7f27cb200640 1 -- start start 2024-09-21T06:07:00.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27cb200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27cc0d1140 0x7f27cc0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:00.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27cb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27cc15f5c0 0x7f27cc0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:00.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27cb200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f27cc0ae610 0x7f27cc1fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:00.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27cb200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc168de0 con 0x7f27cc0d1140 2024-09-21T06:07:00.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27cb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc168c60 con 0x7f27cc15f5c0 2024-09-21T06:07:00.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27cb200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f27cc168f60 con 0x7f27cc0ae610 2024-09-21T06:07:00.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27c9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27cc0d1140 0x7f27cc0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:00.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27c9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27cc0d1140 0x7f27cc0cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47182/0 (socket says 172.21.15.50:47182) 2024-09-21T06:07:00.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27c9e00640 1 -- 172.21.15.50:0/1938271023 learned_addr learned my addr 172.21.15.50:0/1938271023 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:00.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27c9e00640 1 -- 172.21.15.50:0/1938271023 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f27cc0ae610 msgr2=0x7f27cc1fc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27c9e00640 1 --2- 172.21.15.50:0/1938271023 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f27cc0ae610 0x7f27cc1fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27c9e00640 1 -- 172.21.15.50:0/1938271023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27cc15f5c0 msgr2=0x7f27cc0ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27c9e00640 1 --2- 172.21.15.50:0/1938271023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27cc15f5c0 0x7f27cc0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27c9e00640 1 -- 172.21.15.50:0/1938271023 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27cc0ce690 con 0x7f27cc0d1140 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27c9e00640 1 --2- 172.21.15.50:0/1938271023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27cc0d1140 0x7f27cc0cbab0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0058d0 tx=0x7f27bc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27c2000640 1 -- 172.21.15.50:0/1938271023 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc017070 con 0x7f27cc0d1140 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27c2000640 1 -- 172.21.15.50:0/1938271023 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27bc009d20 con 0x7f27cc0d1140 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27c2000640 1 -- 172.21.15.50:0/1938271023 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0106c0 con 0x7f27cc0d1140 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27cb200640 1 -- 172.21.15.50:0/1938271023 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27cc0ce080 con 0x7f27cc0d1140 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.981+0000 7f27cb200640 1 -- 172.21.15.50:0/1938271023 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27cc0ce320 con 0x7f27cc0d1140 2024-09-21T06:07:00.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.985+0000 7f27cb200640 1 -- 172.21.15.50:0/1938271023 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27cc06dfd0 con 0x7f27cc0d1140 2024-09-21T06:07:00.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.985+0000 7f27c2000640 1 -- 172.21.15.50:0/1938271023 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f27bc01a070 con 0x7f27cc0d1140 2024-09-21T06:07:00.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.989+0000 7f27c2000640 1 --2- 172.21.15.50:0/1938271023 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f27ac074790 0x7f27ac076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:00.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.989+0000 7f27c2000640 1 -- 172.21.15.50:0/1938271023 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f27bc093480 con 0x7f27cc0d1140 2024-09-21T06:07:00.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:00.993+0000 7f27c2000640 1 -- 172.21.15.50:0/1938271023 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f27bc05fbe0 con 0x7f27cc0d1140 2024-09-21T06:07:01.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.017+0000 7f27c9400640 1 --2- 172.21.15.50:0/1938271023 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f27ac074790 0x7f27ac076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:01.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.033+0000 7f27c9400640 1 --2- 172.21.15.50:0/1938271023 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f27ac074790 0x7f27ac076ca0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f27b80af000 tx=0x7f27b8019530 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:01.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.429+0000 7f27cb200640 1 -- 172.21.15.50:0/1938271023 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f27cc155f80 con 0x7f27cc0d1140 2024-09-21T06:07:01.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.433+0000 7f27c2000640 1 -- 172.21.15.50:0/1938271023 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2182 (secure 0 0 0) 0x7f27bc05f4e0 con 0x7f27cc0d1140 2024-09-21T06:07:01.430 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:07:01.431 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 657, 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 85, 2024-09-21T06:07:01.432 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:01.433 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 1260, 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 5139180649, 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 10410532864, 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 762683580416, 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 177135751, 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 244950322, 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 43, 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 67 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:07:01.434 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 56, 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T06:06:14:518361+0000", 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 6, 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 6, 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "c", 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 7029 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:01.435 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 15, 2024-09-21T06:07:01.436 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:05:41.065723+0000", 2024-09-21T06:07:01.437 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:07:01.437 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:01.437 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": { 2024-09-21T06:07:01.437 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr-vol-ongoing-clones": { 2024-09-21T06:07:01.437 INFO:teuthology.orchestra.run.smithi050.stdout: "message": "2 ongoing clones - average progress is 22.265% (2s)\n [======......................] (remaining: 7s)", 2024-09-21T06:07:01.437 INFO:teuthology.orchestra.run.smithi050.stdout: "progress": 0.22265000641345978, 2024-09-21T06:07:01.437 INFO:teuthology.orchestra.run.smithi050.stdout: "add_to_ceph_s": true 2024-09-21T06:07:01.437 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:01.437 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:01.437 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:07:01.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.437+0000 7f27cb200640 1 -- 172.21.15.50:0/1938271023 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f27ac074790 msgr2=0x7f27ac076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:01.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.437+0000 7f27cb200640 1 --2- 172.21.15.50:0/1938271023 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f27ac074790 0x7f27ac076ca0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f27b80af000 tx=0x7f27b8019530 comp rx=0 tx=0).stop 2024-09-21T06:07:01.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.437+0000 7f27cb200640 1 -- 172.21.15.50:0/1938271023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27cc0d1140 msgr2=0x7f27cc0cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:01.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.437+0000 7f27cb200640 1 --2- 172.21.15.50:0/1938271023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27cc0d1140 0x7f27cc0cbab0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0058d0 tx=0x7f27bc00dde0 comp rx=0 tx=0).stop 2024-09-21T06:07:01.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.437+0000 7f27cb200640 1 -- 172.21.15.50:0/1938271023 shutdown_connections 2024-09-21T06:07:01.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.437+0000 7f27cb200640 1 --2- 172.21.15.50:0/1938271023 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f27cc0ae610 0x7f27cc1fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:01.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.437+0000 7f27cb200640 1 --2- 172.21.15.50:0/1938271023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27cc15f5c0 0x7f27cc0ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:01.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.437+0000 7f27cb200640 1 --2- 172.21.15.50:0/1938271023 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f27ac074790 0x7f27ac076ca0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:01.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.437+0000 7f27cb200640 1 --2- 172.21.15.50:0/1938271023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27cc0d1140 0x7f27cc0cbab0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:01.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.437+0000 7f27cb200640 1 -- 172.21.15.50:0/1938271023 >> 172.21.15.50:0/1938271023 conn(0x7f27cc0b41f0 msgr2=0x7f27cc0705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:01.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.437+0000 7f27cb200640 1 -- 172.21.15.50:0/1938271023 shutdown_connections 2024-09-21T06:07:01.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:01.437+0000 7f27cb200640 1 -- 172.21.15.50:0/1938271023 wait complete. 2024-09-21T06:07:02.470 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:07:02.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.777+0000 7fa1a3200640 1 -- 172.21.15.50:0/321752997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1a4155710 msgr2=0x7fa1a4162c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.777+0000 7fa1a3200640 1 --2- 172.21.15.50:0/321752997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1a4155710 0x7fa1a4162c30 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa194009840 tx=0x7fa19401c790 comp rx=0 tx=0).stop 2024-09-21T06:07:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.777+0000 7fa1a3200640 1 -- 172.21.15.50:0/321752997 shutdown_connections 2024-09-21T06:07:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.777+0000 7fa1a3200640 1 --2- 172.21.15.50:0/321752997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1a4155710 0x7fa1a4162c30 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.777+0000 7fa1a3200640 1 -- 172.21.15.50:0/321752997 >> 172.21.15.50:0/321752997 conn(0x7fa1a40b4560 msgr2=0x7fa1a40d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.777+0000 7fa1a3200640 1 -- 172.21.15.50:0/321752997 shutdown_connections 2024-09-21T06:07:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.777+0000 7fa1a3200640 1 -- 172.21.15.50:0/321752997 wait complete. 2024-09-21T06:07:02.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.777+0000 7fa1a3200640 1 Processor -- start 2024-09-21T06:07:02.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.777+0000 7fa1a3200640 1 -- start start 2024-09-21T06:07:02.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa1a3200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa1a40b6b10 0x7fa1a4139c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:02.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa1a3200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1a40d1270 0x7fa1a4150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:02.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa1a3200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1a4155710 0x7fa1a4156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa1a3200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a416d310 con 0x7fa1a40d1270 2024-09-21T06:07:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa1a3200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a416d190 con 0x7fa1a4155710 2024-09-21T06:07:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa1a3200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a416d490 con 0x7fa1a40b6b10 2024-09-21T06:07:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa19be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1a40d1270 0x7fa1a4150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa19be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1a40d1270 0x7fa1a4150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47202/0 (socket says 172.21.15.50:47202) 2024-09-21T06:07:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa19be00640 1 -- 172.21.15.50:0/450536146 learned_addr learned my addr 172.21.15.50:0/450536146 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa1a1e00640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa1a40b6b10 0x7fa1a4139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa1a2800640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1a4155710 0x7fa1a4156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa19be00640 1 -- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa1a40b6b10 msgr2=0x7fa1a4139c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa19be00640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa1a40b6b10 0x7fa1a4139c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa19be00640 1 -- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1a4155710 msgr2=0x7fa1a4156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa19be00640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1a4155710 0x7fa1a4156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa19be00640 1 -- 172.21.15.50:0/450536146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a4147d00 con 0x7fa1a40d1270 2024-09-21T06:07:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa19be00640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1a40d1270 0x7fa1a4150840 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00b410 tx=0x7fa18c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa19a000640 1 -- 172.21.15.50:0/450536146 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c015020 con 0x7fa1a40d1270 2024-09-21T06:07:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa19a000640 1 -- 172.21.15.50:0/450536146 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa18c011920 con 0x7fa1a40d1270 2024-09-21T06:07:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa1a3200640 1 -- 172.21.15.50:0/450536146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a414e420 con 0x7fa1a40d1270 2024-09-21T06:07:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa19a000640 1 -- 172.21.15.50:0/450536146 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c011d70 con 0x7fa1a40d1270 2024-09-21T06:07:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.781+0000 7fa1a3200640 1 -- 172.21.15.50:0/450536146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a4153870 con 0x7fa1a40d1270 2024-09-21T06:07:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.785+0000 7fa1a3200640 1 -- 172.21.15.50:0/450536146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a4162c30 con 0x7fa1a40d1270 2024-09-21T06:07:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.785+0000 7fa19a000640 1 -- 172.21.15.50:0/450536146 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7fa18c018020 con 0x7fa1a40d1270 2024-09-21T06:07:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.785+0000 7fa19a000640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa180074760 0x7fa180076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.789+0000 7fa19a000640 1 -- 172.21.15.50:0/450536146 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fa18c093280 con 0x7fa1a40d1270 2024-09-21T06:07:02.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.793+0000 7fa19a000640 1 -- 172.21.15.50:0/450536146 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa18c01f020 con 0x7fa1a40d1270 2024-09-21T06:07:02.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.805+0000 7fa1a1e00640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa180074760 0x7fa180076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:02.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:02.825+0000 7fa1a1e00640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa180074760 0x7fa180076c70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa1940ae000 tx=0x7fa19401afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:03.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.277+0000 7fa1a3200640 1 -- 172.21.15.50:0/450536146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa1a41534c0 con 0x7fa1a40d1270 2024-09-21T06:07:03.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.277+0000 7fa19a000640 1 -- 172.21.15.50:0/450536146 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+3290 (secure 0 0 0) 0x7fa18c05f960 con 0x7fa1a40d1270 2024-09-21T06:07:03.276 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:07:03.276 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:07:03.276 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:07:03.277 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:07:03.277 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:07:03.277 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:07:03.277 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:07:03.277 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:03.277 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:07:03.277 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:07:03.277 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:07:03.277 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:07:03.278 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:07:03.278 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:03.278 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:07:03.278 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:07:03.278 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:07:03.278 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:07:03.278 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:03.278 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 659, 2024-09-21T06:07:03.278 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:07:03.279 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:07:03.279 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:07:03.279 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:07:03.279 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:03.279 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:07:03.279 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 85, 2024-09-21T06:07:03.279 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:07:03.279 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:07:03.280 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:07:03.280 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:07:03.280 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:07:03.280 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:07:03.280 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:03.280 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:07:03.280 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:07:03.280 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:07:03.280 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:07:03.280 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:07:03.281 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:03.281 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:03.281 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:07:03.281 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:07:03.281 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 1557, 2024-09-21T06:07:03.281 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 6389083300, 2024-09-21T06:07:03.281 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 12600033280, 2024-09-21T06:07:03.281 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 760494080000, 2024-09-21T06:07:03.281 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:07:03.282 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 275248080, 2024-09-21T06:07:03.282 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 265153921, 2024-09-21T06:07:03.282 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 67, 2024-09-21T06:07:03.282 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 74 2024-09-21T06:07:03.282 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:03.282 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:07:03.282 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 56, 2024-09-21T06:07:03.282 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T06:06:14:518361+0000", 2024-09-21T06:07:03.282 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 6, 2024-09-21T06:07:03.283 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:07:03.283 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:07:03.283 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:07:03.283 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:07:03.283 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:07:03.283 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 6, 2024-09-21T06:07:03.283 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:07:03.283 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "c", 2024-09-21T06:07:03.283 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:07:03.284 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 7029 2024-09-21T06:07:03.284 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:03.284 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:03.284 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:07:03.284 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:03.284 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:07:03.284 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:07:03.284 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:07:03.284 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 16, 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:07:03.094270+0000", 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: "services": { 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: "daemons": { 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: "summary": "", 2024-09-21T06:07:03.285 INFO:teuthology.orchestra.run.smithi050.stdout: "b": { 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "start_epoch": 0, 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "start_stamp": "0.000000", 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 0, 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": {}, 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "task_status": {} 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "c": { 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "start_epoch": 0, 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "start_stamp": "0.000000", 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 0, 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": {}, 2024-09-21T06:07:03.286 INFO:teuthology.orchestra.run.smithi050.stdout: "task_status": {} 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: "d": { 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: "start_epoch": 0, 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: "start_stamp": "0.000000", 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 0, 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": {}, 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: "task_status": {} 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": { 2024-09-21T06:07:03.287 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr-vol-ongoing-clones": { 2024-09-21T06:07:03.288 INFO:teuthology.orchestra.run.smithi050.stdout: "message": "4 ongoing clones - average progress is 34.7% (6s)\n [=========...................] (remaining: 12s)", 2024-09-21T06:07:03.288 INFO:teuthology.orchestra.run.smithi050.stdout: "progress": 0.34700000286102295, 2024-09-21T06:07:03.288 INFO:teuthology.orchestra.run.smithi050.stdout: "add_to_ceph_s": true 2024-09-21T06:07:03.288 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:03.288 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:03.288 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:07:03.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.281+0000 7fa1a3200640 1 -- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa180074760 msgr2=0x7fa180076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:03.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.281+0000 7fa1a3200640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa180074760 0x7fa180076c70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa1940ae000 tx=0x7fa19401afd0 comp rx=0 tx=0).stop 2024-09-21T06:07:03.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.281+0000 7fa1a3200640 1 -- 172.21.15.50:0/450536146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1a40d1270 msgr2=0x7fa1a4150840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:03.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.281+0000 7fa1a3200640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1a40d1270 0x7fa1a4150840 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00b410 tx=0x7fa18c00b8d0 comp rx=0 tx=0).stop 2024-09-21T06:07:03.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.281+0000 7fa1a3200640 1 -- 172.21.15.50:0/450536146 shutdown_connections 2024-09-21T06:07:03.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.281+0000 7fa1a3200640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1a4155710 0x7fa1a4156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:03.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.281+0000 7fa1a3200640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa180074760 0x7fa180076c70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:03.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.281+0000 7fa1a3200640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1a40d1270 0x7fa1a4150840 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:03.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.281+0000 7fa1a3200640 1 --2- 172.21.15.50:0/450536146 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa1a40b6b10 0x7fa1a4139c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:03.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.281+0000 7fa1a3200640 1 -- 172.21.15.50:0/450536146 >> 172.21.15.50:0/450536146 conn(0x7fa1a40b4560 msgr2=0x7fa1a40d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:03.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.281+0000 7fa1a3200640 1 -- 172.21.15.50:0/450536146 shutdown_connections 2024-09-21T06:07:03.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:03.281+0000 7fa1a3200640 1 -- 172.21.15.50:0/450536146 wait complete. 2024-09-21T06:07:04.322 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:07:04.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.585+0000 7f6237e00640 1 -- 172.21.15.50:0/2346042017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6238074c30 msgr2=0x7f6238075040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:04.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.585+0000 7f6237e00640 1 --2- 172.21.15.50:0/2346042017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6238074c30 0x7f6238075040 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6238075760 tx=0x7f622001c800 comp rx=0 tx=0).stop 2024-09-21T06:07:04.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.585+0000 7f6237e00640 1 -- 172.21.15.50:0/2346042017 shutdown_connections 2024-09-21T06:07:04.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.585+0000 7f6237e00640 1 --2- 172.21.15.50:0/2346042017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6238074c30 0x7f6238075040 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:04.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.585+0000 7f6237e00640 1 --2- 172.21.15.50:0/2346042017 >> v2:172.21.15.88:3300/0 conn(0x7f62380746e0 0x7f623815c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:04.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.585+0000 7f6237e00640 1 -- 172.21.15.50:0/2346042017 >> 172.21.15.50:0/2346042017 conn(0x7f62380b41f0 msgr2=0x7f62380b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:04.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.585+0000 7f6237e00640 1 -- 172.21.15.50:0/2346042017 shutdown_connections 2024-09-21T06:07:04.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.585+0000 7f6237e00640 1 -- 172.21.15.50:0/2346042017 wait complete. 2024-09-21T06:07:04.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.589+0000 7f6237e00640 1 Processor -- start 2024-09-21T06:07:04.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.589+0000 7f6237e00640 1 -- start start 2024-09-21T06:07:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.589+0000 7f6237e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62380746e0 0x7f623814bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.589+0000 7f6237e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6238074c30 0x7f623814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.589+0000 7f6237e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62380d1190 0x7f62381f9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.589+0000 7f6237e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f623816a190 con 0x7f6238074c30 2024-09-21T06:07:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.589+0000 7f6237e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6238069a30 con 0x7f62380746e0 2024-09-21T06:07:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.589+0000 7f6237e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f623816a310 con 0x7f62380d1190 2024-09-21T06:07:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.589+0000 7f622fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6238074c30 0x7f623814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.589+0000 7f622fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6238074c30 0x7f623814c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47234/0 (socket says 172.21.15.50:47234) 2024-09-21T06:07:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.589+0000 7f622fe00640 1 -- 172.21.15.50:0/1346815806 learned_addr learned my addr 172.21.15.50:0/1346815806 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:04.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f622fe00640 1 -- 172.21.15.50:0/1346815806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62380d1190 msgr2=0x7f62381f9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:04.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f622fe00640 1 --2- 172.21.15.50:0/1346815806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62380d1190 0x7f62381f9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:04.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f622fe00640 1 -- 172.21.15.50:0/1346815806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62380746e0 msgr2=0x7f623814bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:04.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f622fe00640 1 --2- 172.21.15.50:0/1346815806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62380746e0 0x7f623814bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:04.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f622fe00640 1 -- 172.21.15.50:0/1346815806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62380cff80 con 0x7f6238074c30 2024-09-21T06:07:04.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f622fe00640 1 --2- 172.21.15.50:0/1346815806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6238074c30 0x7f623814c200 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f62200b0040 tx=0x7f6220004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:04.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f622ea00640 1 -- 172.21.15.50:0/1346815806 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62200b2040 con 0x7f6238074c30 2024-09-21T06:07:04.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f622ea00640 1 -- 172.21.15.50:0/1346815806 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6220003600 con 0x7f6238074c30 2024-09-21T06:07:04.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f6237e00640 1 -- 172.21.15.50:0/1346815806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62380d0260 con 0x7f6238074c30 2024-09-21T06:07:04.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f622ea00640 1 -- 172.21.15.50:0/1346815806 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6220003a30 con 0x7f6238074c30 2024-09-21T06:07:04.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f6237e00640 1 -- 172.21.15.50:0/1346815806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62380d0820 con 0x7f6238074c30 2024-09-21T06:07:04.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f6237e00640 1 -- 172.21.15.50:0/1346815806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623815c920 con 0x7f6238074c30 2024-09-21T06:07:04.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.593+0000 7f622ea00640 1 -- 172.21.15.50:0/1346815806 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f6220003bd0 con 0x7f6238074c30 2024-09-21T06:07:04.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.597+0000 7f622ea00640 1 --2- 172.21.15.50:0/1346815806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6214074760 0x7f6214076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:04.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.597+0000 7f622ea00640 1 -- 172.21.15.50:0/1346815806 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f622012ce30 con 0x7f6238074c30 2024-09-21T06:07:04.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.597+0000 7f6236a00640 1 --2- 172.21.15.50:0/1346815806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6214074760 0x7f6214076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.601+0000 7f622ea00640 1 -- 172.21.15.50:0/1346815806 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f62200bd060 con 0x7f6238074c30 2024-09-21T06:07:04.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:04.609+0000 7f6236a00640 1 --2- 172.21.15.50:0/1346815806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6214074760 0x7f6214076c70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6228009690 tx=0x7f6228006c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:04.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.001+0000 7f6237e00640 1 -- 172.21.15.50:0/1346815806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f62380b4f30 con 0x7f6238074c30 2024-09-21T06:07:04.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.001+0000 7f622ea00640 1 -- 172.21.15.50:0/1346815806 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+3290 (secure 0 0 0) 0x7f6238075580 con 0x7f6238074c30 2024-09-21T06:07:04.998 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:07:04.998 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:07:04.998 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:07:04.998 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:07:04.998 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:07:04.998 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:07:04.998 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:07:04.998 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:04.998 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:07:04.998 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:07:04.998 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 661, 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:07:04.999 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 85, 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:07:05.000 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 1557, 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 6389083300, 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 12600033280, 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 760494080000, 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 275248080, 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 265153921, 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 67, 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 74 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:05.001 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 56, 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T06:06:14:518361+0000", 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 6, 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 6, 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "c", 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 7029 2024-09-21T06:07:05.002 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:05.003 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 16, 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:07:03.094270+0000", 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "services": { 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "daemons": { 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "summary": "", 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "b": { 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "start_epoch": 0, 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "start_stamp": "0.000000", 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 0, 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": {}, 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: "task_status": {} 2024-09-21T06:07:05.004 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "c": { 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "start_epoch": 0, 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "start_stamp": "0.000000", 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 0, 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": {}, 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "task_status": {} 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "d": { 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "start_epoch": 0, 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "start_stamp": "0.000000", 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 0, 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": {}, 2024-09-21T06:07:05.005 INFO:teuthology.orchestra.run.smithi050.stdout: "task_status": {} 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": { 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr-vol-ongoing-clones": { 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout: "message": "4 ongoing clones - average progress is 34.7% (6s)\n [=========...................] (remaining: 12s)", 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout: "progress": 0.34700000286102295, 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout: "add_to_ceph_s": true 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:05.006 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:07:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.001+0000 7f6237e00640 1 -- 172.21.15.50:0/1346815806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6214074760 msgr2=0x7f6214076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.001+0000 7f6237e00640 1 --2- 172.21.15.50:0/1346815806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6214074760 0x7f6214076c70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6228009690 tx=0x7f6228006c00 comp rx=0 tx=0).stop 2024-09-21T06:07:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.001+0000 7f6237e00640 1 -- 172.21.15.50:0/1346815806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6238074c30 msgr2=0x7f623814c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.001+0000 7f6237e00640 1 --2- 172.21.15.50:0/1346815806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6238074c30 0x7f623814c200 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f62200b0040 tx=0x7f6220004040 comp rx=0 tx=0).stop 2024-09-21T06:07:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.005+0000 7f6237e00640 1 -- 172.21.15.50:0/1346815806 shutdown_connections 2024-09-21T06:07:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.005+0000 7f6237e00640 1 --2- 172.21.15.50:0/1346815806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62380d1190 0x7f62381f9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.005+0000 7f6237e00640 1 --2- 172.21.15.50:0/1346815806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6214074760 0x7f6214076c70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.005+0000 7f6237e00640 1 --2- 172.21.15.50:0/1346815806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6238074c30 0x7f623814c200 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.005+0000 7f6237e00640 1 --2- 172.21.15.50:0/1346815806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62380746e0 0x7f623814bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.005+0000 7f6237e00640 1 -- 172.21.15.50:0/1346815806 >> 172.21.15.50:0/1346815806 conn(0x7f62380b41f0 msgr2=0x7f623806f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.005+0000 7f6237e00640 1 -- 172.21.15.50:0/1346815806 shutdown_connections 2024-09-21T06:07:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:05.005+0000 7f6237e00640 1 -- 172.21.15.50:0/1346815806 wait complete. 2024-09-21T06:07:06.036 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:07:06.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.285+0000 7f6409200640 1 -- 172.21.15.50:0/2680262133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e8005340 msgr2=0x7f63e800b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:06.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.285+0000 7f6409200640 1 --2- 172.21.15.50:0/2680262133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e8005340 0x7f63e800b9f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00c640 tx=0x7f63fc01fa80 comp rx=0 tx=0).stop 2024-09-21T06:07:06.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.285+0000 7f6409200640 1 -- 172.21.15.50:0/2680262133 shutdown_connections 2024-09-21T06:07:06.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.285+0000 7f6409200640 1 --2- 172.21.15.50:0/2680262133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e8005340 0x7f63e800b9f0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:06.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.285+0000 7f6409200640 1 -- 172.21.15.50:0/2680262133 >> 172.21.15.50:0/2680262133 conn(0x7f64040b35f0 msgr2=0x7f64040b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:06.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.285+0000 7f6409200640 1 -- 172.21.15.50:0/2680262133 shutdown_connections 2024-09-21T06:07:06.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.285+0000 7f6409200640 1 -- 172.21.15.50:0/2680262133 wait complete. 2024-09-21T06:07:06.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.289+0000 7f6409200640 1 Processor -- start 2024-09-21T06:07:06.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.289+0000 7f6409200640 1 -- start start 2024-09-21T06:07:06.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6409200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e8005340 0x7f64040af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:06.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6409200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64041556f0 0x7f640414bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:06.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6409200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f640415a570 0x7f640415a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:06.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6409200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6404070220 con 0x7f64041556f0 2024-09-21T06:07:06.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6409200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64040700a0 con 0x7f63e8005340 2024-09-21T06:07:06.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6409200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f64040703a0 con 0x7f640415a570 2024-09-21T06:07:06.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6401400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64041556f0 0x7f640414bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:06.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6401400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64041556f0 0x7f640414bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47258/0 (socket says 172.21.15.50:47258) 2024-09-21T06:07:06.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6401400640 1 -- 172.21.15.50:0/2707876935 learned_addr learned my addr 172.21.15.50:0/2707876935 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:06.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6401e00640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e8005340 0x7f64040af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:06.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6401400640 1 -- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f640415a570 msgr2=0x7f640415a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:06.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6401400640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f640415a570 0x7f640415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:06.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6401400640 1 -- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e8005340 msgr2=0x7f64040af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:06.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6401400640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e8005340 0x7f64040af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:06.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6401400640 1 -- 172.21.15.50:0/2707876935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640415aec0 con 0x7f64041556f0 2024-09-21T06:07:06.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6401e00640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e8005340 0x7f64040af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:06.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6401400640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64041556f0 0x7f640414bfd0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f63f000d610 tx=0x7f63f000dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:06.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f63fa000640 1 -- 172.21.15.50:0/2707876935 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0017070 con 0x7f64041556f0 2024-09-21T06:07:06.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f63fa000640 1 -- 172.21.15.50:0/2707876935 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63f0009a40 con 0x7f64041556f0 2024-09-21T06:07:06.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f63fa000640 1 -- 172.21.15.50:0/2707876935 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0010460 con 0x7f64041556f0 2024-09-21T06:07:06.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.293+0000 7f6409200640 1 -- 172.21.15.50:0/2707876935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64040afb50 con 0x7f64041556f0 2024-09-21T06:07:06.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.297+0000 7f6409200640 1 -- 172.21.15.50:0/2707876935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64040b0110 con 0x7f64041556f0 2024-09-21T06:07:06.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.297+0000 7f6409200640 1 -- 172.21.15.50:0/2707876935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64040b0310 con 0x7f64041556f0 2024-09-21T06:07:06.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.297+0000 7f63fa000640 1 -- 172.21.15.50:0/2707876935 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f63f001a070 con 0x7f64041556f0 2024-09-21T06:07:06.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.301+0000 7f63fa000640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63e4074760 0x7f63e4076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:06.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.301+0000 7f63fa000640 1 -- 172.21.15.50:0/2707876935 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f63f00933a0 con 0x7f64041556f0 2024-09-21T06:07:06.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.309+0000 7f63fa000640 1 -- 172.21.15.50:0/2707876935 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f64040b0310 con 0x7f64041556f0 2024-09-21T06:07:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.309+0000 7f6401e00640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63e4074760 0x7f63e4076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:06.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.317+0000 7f6401e00640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63e4074760 0x7f63e4076c70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f63f4003ea0 tx=0x7f63f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:06.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.713+0000 7f6409200640 1 -- 172.21.15.50:0/2707876935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f64040af8b0 con 0x7f64041556f0 2024-09-21T06:07:06.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.713+0000 7f63fa000640 1 -- 172.21.15.50:0/2707876935 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2877 (secure 0 0 0) 0x7f640415aec0 con 0x7f64041556f0 2024-09-21T06:07:06.710 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:07:06.710 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:07:06.710 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:07:06.710 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:07:06.710 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:07:06.710 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:07:06.710 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:07:06.710 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:06.710 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 663, 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:07:06.711 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 85, 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:07:06.712 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 2349, 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 9711162609, 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 19078012928, 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 754016100352, 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 548707256, 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 540377272, 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 132, 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 146 2024-09-21T06:07:06.713 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 56, 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T06:06:14:518361+0000", 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 6, 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 6, 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "c", 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:07:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 7029 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:07:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:07:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 17, 2024-09-21T06:07:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:07:05.096225+0000", 2024-09-21T06:07:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: "services": { 2024-09-21T06:07:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-09-21T06:07:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: "daemons": { 2024-09-21T06:07:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: "summary": "", 2024-09-21T06:07:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: "c": { 2024-09-21T06:07:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: "start_epoch": 0, 2024-09-21T06:07:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: "start_stamp": "0.000000", 2024-09-21T06:07:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 0, 2024-09-21T06:07:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": {}, 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: "task_status": {} 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": { 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr-vol-ongoing-clones": { 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: "message": "4 ongoing clones - average progress is 35.026% (9s)\n [=========...................] (remaining: 18s)", 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: "progress": 0.35025998950004578, 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: "add_to_ceph_s": true 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr-vol-total-clones": { 2024-09-21T06:07:06.717 INFO:teuthology.orchestra.run.smithi050.stdout: "message": "Total 5 clones - average progress is 43.125% (0s)\n [============................] ", 2024-09-21T06:07:06.718 INFO:teuthology.orchestra.run.smithi050.stdout: "progress": 0.43125000596046448, 2024-09-21T06:07:06.718 INFO:teuthology.orchestra.run.smithi050.stdout: "add_to_ceph_s": true 2024-09-21T06:07:06.718 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:06.718 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:06.718 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:07:06.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.717+0000 7f6409200640 1 -- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63e4074760 msgr2=0x7f63e4076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:06.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.717+0000 7f6409200640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63e4074760 0x7f63e4076c70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f63f4003ea0 tx=0x7f63f4009040 comp rx=0 tx=0).stop 2024-09-21T06:07:06.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.717+0000 7f6409200640 1 -- 172.21.15.50:0/2707876935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64041556f0 msgr2=0x7f640414bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:06.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.717+0000 7f6409200640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64041556f0 0x7f640414bfd0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f63f000d610 tx=0x7f63f000dad0 comp rx=0 tx=0).stop 2024-09-21T06:07:06.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.717+0000 7f6409200640 1 -- 172.21.15.50:0/2707876935 shutdown_connections 2024-09-21T06:07:06.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.717+0000 7f6409200640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f640415a570 0x7f640415a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:06.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.717+0000 7f6409200640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63e4074760 0x7f63e4076c70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:06.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.717+0000 7f6409200640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64041556f0 0x7f640414bfd0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:06.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.717+0000 7f6409200640 1 --2- 172.21.15.50:0/2707876935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e8005340 0x7f64040af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:06.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.717+0000 7f6409200640 1 -- 172.21.15.50:0/2707876935 >> 172.21.15.50:0/2707876935 conn(0x7f64040b35f0 msgr2=0x7f640406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:06.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.717+0000 7f6409200640 1 -- 172.21.15.50:0/2707876935 shutdown_connections 2024-09-21T06:07:06.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:06.717+0000 7f6409200640 1 -- 172.21.15.50:0/2707876935 wait complete. 2024-09-21T06:07:06.756 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 35.026% (9s)\n [=========...................] (remaining: 18s)', 'progress': 0.3502599895000458, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 5 clones - average progress is 43.125% (0s)\n [============................] ', 'progress': 0.4312500059604645, 'add_to_ceph_s': True}} 2024-09-21T06:07:06.756 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone64155 2024-09-21T06:07:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.021+0000 7ff320400640 1 -- 172.21.15.50:0/4219412562 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff31813e790 msgr2=0x7ff3180cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.021+0000 7ff320400640 1 --2- 172.21.15.50:0/4219412562 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff31813e790 0x7ff3180cc320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff304009930 tx=0x7ff30401c850 comp rx=0 tx=0).stop 2024-09-21T06:07:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.021+0000 7ff320400640 1 -- 172.21.15.50:0/4219412562 shutdown_connections 2024-09-21T06:07:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.021+0000 7ff320400640 1 --2- 172.21.15.50:0/4219412562 >> v2:172.21.15.88:3300/0 conn(0x7ff31815efa0 0x7ff3180cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.021+0000 7ff320400640 1 --2- 172.21.15.50:0/4219412562 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff31813e790 0x7ff3180cc320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.021+0000 7ff320400640 1 -- 172.21.15.50:0/4219412562 >> 172.21.15.50:0/4219412562 conn(0x7ff3180b2040 msgr2=0x7ff3180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.021+0000 7ff320400640 1 -- 172.21.15.50:0/4219412562 shutdown_connections 2024-09-21T06:07:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.021+0000 7ff320400640 1 -- 172.21.15.50:0/4219412562 wait complete. 2024-09-21T06:07:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.025+0000 7ff320400640 1 Processor -- start 2024-09-21T06:07:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.025+0000 7ff320400640 1 -- start start 2024-09-21T06:07:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.025+0000 7ff320400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff3180cc860 0x7ff318153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.025+0000 7ff320400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff31813e790 0x7ff31813cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.025+0000 7ff320400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3181f94c0 0x7ff318142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.025+0000 7ff320400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff318140ff0 con 0x7ff3181f94c0 2024-09-21T06:07:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.025+0000 7ff320400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff318140e70 con 0x7ff31813e790 2024-09-21T06:07:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.025+0000 7ff320400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff318168a30 con 0x7ff3180cc860 2024-09-21T06:07:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.025+0000 7ff31e400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3181f94c0 0x7ff318142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.025+0000 7ff31e400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3181f94c0 0x7ff318142d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42114/0 (socket says 172.21.15.50:42114) 2024-09-21T06:07:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.025+0000 7ff31e400640 1 -- 172.21.15.50:0/203030324 learned_addr learned my addr 172.21.15.50:0/203030324 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff31e400640 1 -- 172.21.15.50:0/203030324 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff3180cc860 msgr2=0x7ff318153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff31e400640 1 --2- 172.21.15.50:0/203030324 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff3180cc860 0x7ff318153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff31e400640 1 -- 172.21.15.50:0/203030324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff31813e790 msgr2=0x7ff31813cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff31e400640 1 --2- 172.21.15.50:0/203030324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff31813e790 0x7ff31813cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff31e400640 1 -- 172.21.15.50:0/203030324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31813d0e0 con 0x7ff3181f94c0 2024-09-21T06:07:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff31e400640 1 --2- 172.21.15.50:0/203030324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3181f94c0 0x7ff318142d10 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff31000ba10 tx=0x7ff31000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff315600640 1 -- 172.21.15.50:0/203030324 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310019070 con 0x7ff3181f94c0 2024-09-21T06:07:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff315600640 1 -- 172.21.15.50:0/203030324 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff31000cc50 con 0x7ff3181f94c0 2024-09-21T06:07:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff320400640 1 -- 172.21.15.50:0/203030324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff318154780 con 0x7ff3181f94c0 2024-09-21T06:07:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff315600640 1 -- 172.21.15.50:0/203030324 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3100146a0 con 0x7ff3181f94c0 2024-09-21T06:07:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff320400640 1 -- 172.21.15.50:0/203030324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff318141620 con 0x7ff3181f94c0 2024-09-21T06:07:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff320400640 1 -- 172.21.15.50:0/203030324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3181544f0 con 0x7ff3181f94c0 2024-09-21T06:07:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.029+0000 7ff315600640 1 -- 172.21.15.50:0/203030324 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7ff31001c070 con 0x7ff3181f94c0 2024-09-21T06:07:07.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.033+0000 7ff315600640 1 --2- 172.21.15.50:0/203030324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff2f8074760 0x7ff2f8076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:07.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.037+0000 7ff315600640 1 -- 172.21.15.50:0/203030324 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7ff310093030 con 0x7ff3181f94c0 2024-09-21T06:07:07.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.041+0000 7ff315600640 1 -- 172.21.15.50:0/203030324 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff31005f710 con 0x7ff3181f94c0 2024-09-21T06:07:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.049+0000 7ff31da00640 1 --2- 172.21.15.50:0/203030324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff2f8074760 0x7ff2f8076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:07.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.073+0000 7ff31da00640 1 --2- 172.21.15.50:0/203030324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff2f8074760 0x7ff2f8076c70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff3040af000 tx=0x7ff3040195f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:07.473+0000 7ff320400640 1 -- 172.21.15.50:0/203030324 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone64155", "target": ["mon-mgr", ""]}) -- 0x7ff31806b540 con 0x7ff2f8074760 2024-09-21T06:07:08.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.241+0000 7ff315600640 1 -- 172.21.15.50:0/203030324 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff304004020 con 0x7ff2f8074760 2024-09-21T06:07:08.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.249+0000 7ff320400640 1 -- 172.21.15.50:0/203030324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff2f8074760 msgr2=0x7ff2f8076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:08.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.249+0000 7ff320400640 1 --2- 172.21.15.50:0/203030324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff2f8074760 0x7ff2f8076c70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff3040af000 tx=0x7ff3040195f0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.249+0000 7ff320400640 1 -- 172.21.15.50:0/203030324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3181f94c0 msgr2=0x7ff318142d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:08.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.249+0000 7ff320400640 1 --2- 172.21.15.50:0/203030324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3181f94c0 0x7ff318142d10 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff31000ba10 tx=0x7ff31000bed0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.253+0000 7ff320400640 1 -- 172.21.15.50:0/203030324 shutdown_connections 2024-09-21T06:07:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.253+0000 7ff320400640 1 --2- 172.21.15.50:0/203030324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff2f8074760 0x7ff2f8076c70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.253+0000 7ff320400640 1 --2- 172.21.15.50:0/203030324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3181f94c0 0x7ff318142d10 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.253+0000 7ff320400640 1 --2- 172.21.15.50:0/203030324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff31813e790 0x7ff31813cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.253+0000 7ff320400640 1 --2- 172.21.15.50:0/203030324 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff3180cc860 0x7ff318153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.253+0000 7ff320400640 1 -- 172.21.15.50:0/203030324 >> 172.21.15.50:0/203030324 conn(0x7ff3180b2040 msgr2=0x7ff3180ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.253+0000 7ff320400640 1 -- 172.21.15.50:0/203030324 shutdown_connections 2024-09-21T06:07:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.253+0000 7ff320400640 1 -- 172.21.15.50:0/203030324 wait complete. 2024-09-21T06:07:08.312 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8636 2024-09-21T06:07:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.637+0000 7f1a15e00640 1 -- 172.21.15.50:0/4208036196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a100ceba0 msgr2=0x7f1a100cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.637+0000 7f1a15e00640 1 --2- 172.21.15.50:0/4208036196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a100ceba0 0x7f1a100cefb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1a1006f140 tx=0x7f19fc01d320 comp rx=0 tx=0).stop 2024-09-21T06:07:08.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.645+0000 7f1a15e00640 1 -- 172.21.15.50:0/4208036196 shutdown_connections 2024-09-21T06:07:08.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.645+0000 7f1a15e00640 1 --2- 172.21.15.50:0/4208036196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a100ceba0 0x7f1a100cefb0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.645+0000 7f1a15e00640 1 --2- 172.21.15.50:0/4208036196 >> v2:172.21.15.88:3300/0 conn(0x7f1a1015ef10 0x7f1a1015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.645+0000 7f1a15e00640 1 -- 172.21.15.50:0/4208036196 >> 172.21.15.50:0/4208036196 conn(0x7f1a1006e1d0 msgr2=0x7f1a1006e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:08.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.645+0000 7f1a15e00640 1 -- 172.21.15.50:0/4208036196 shutdown_connections 2024-09-21T06:07:08.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.645+0000 7f1a15e00640 1 -- 172.21.15.50:0/4208036196 wait complete. 2024-09-21T06:07:08.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.645+0000 7f1a15e00640 1 Processor -- start 2024-09-21T06:07:08.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a15e00640 1 -- start start 2024-09-21T06:07:08.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a15e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a100cd260 0x7f1a1015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:08.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a15e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a100ceba0 0x7f1a1014cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:08.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a15e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a1015ef10 0x7f1a1014d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:08.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a15e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1a10168230 con 0x7f1a100cd260 2024-09-21T06:07:08.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a15e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101680b0 con 0x7f1a1015ef10 2024-09-21T06:07:08.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a15e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1a101683b0 con 0x7f1a100ceba0 2024-09-21T06:07:08.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a0ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a100cd260 0x7f1a1015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:08.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a0ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a100cd260 0x7f1a1015d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42138/0 (socket says 172.21.15.50:42138) 2024-09-21T06:07:08.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a0ea00640 1 -- 172.21.15.50:0/3430082942 learned_addr learned my addr 172.21.15.50:0/3430082942 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:08.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a0ea00640 1 -- 172.21.15.50:0/3430082942 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a100ceba0 msgr2=0x7f1a1014cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:08.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a0ea00640 1 --2- 172.21.15.50:0/3430082942 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a100ceba0 0x7f1a1014cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a0ea00640 1 -- 172.21.15.50:0/3430082942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a1015ef10 msgr2=0x7f1a1014d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:08.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a0ea00640 1 --2- 172.21.15.50:0/3430082942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a1015ef10 0x7f1a1014d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.649+0000 7f1a0ea00640 1 -- 172.21.15.50:0/3430082942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a10158e60 con 0x7f1a100cd260 2024-09-21T06:07:08.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.653+0000 7f1a0ea00640 1 --2- 172.21.15.50:0/3430082942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a100cd260 0x7f1a1015d100 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800ba50 tx=0x7f1a0800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:08.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.653+0000 7f1a07400640 1 -- 172.21.15.50:0/3430082942 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08019070 con 0x7f1a100cd260 2024-09-21T06:07:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.653+0000 7f1a15e00640 1 -- 172.21.15.50:0/3430082942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a10159100 con 0x7f1a100cd260 2024-09-21T06:07:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.653+0000 7f1a07400640 1 -- 172.21.15.50:0/3430082942 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a0800ccf0 con 0x7f1a100cd260 2024-09-21T06:07:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.653+0000 7f1a07400640 1 -- 172.21.15.50:0/3430082942 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080146a0 con 0x7f1a100cd260 2024-09-21T06:07:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.653+0000 7f1a15e00640 1 -- 172.21.15.50:0/3430082942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a101596c0 con 0x7f1a100cd260 2024-09-21T06:07:08.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.657+0000 7f1a15e00640 1 -- 172.21.15.50:0/3430082942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a100cefb0 con 0x7f1a100cd260 2024-09-21T06:07:08.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.665+0000 7f1a07400640 1 -- 172.21.15.50:0/3430082942 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f1a0801c070 con 0x7f1a100cd260 2024-09-21T06:07:08.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.669+0000 7f1a07400640 1 --2- 172.21.15.50:0/3430082942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19f0074760 0x7f19f0076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:08.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.669+0000 7f1a07400640 1 -- 172.21.15.50:0/3430082942 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f1a080931d0 con 0x7f1a100cd260 2024-09-21T06:07:08.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.673+0000 7f1a0e000640 1 --2- 172.21.15.50:0/3430082942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19f0074760 0x7f19f0076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:08.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.677+0000 7f1a0e000640 1 --2- 172.21.15.50:0/3430082942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19f0074760 0x7f19f0076c70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0af000 tx=0x7f19fc01a0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:08.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:08.677+0000 7f1a07400640 1 -- 172.21.15.50:0/3430082942 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1a10158e60 con 0x7f1a100cd260 2024-09-21T06:07:09.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.093+0000 7f1a15e00640 1 -- 172.21.15.50:0/3430082942 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8636", "target": ["mon-mgr", ""]}) -- 0x7f1a1006b540 con 0x7f19f0074760 2024-09-21T06:07:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.665+0000 7f1a07400640 1 -- 172.21.15.50:0/3430082942 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19fc008e50 con 0x7f19f0074760 2024-09-21T06:07:09.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.673+0000 7f1a15e00640 1 -- 172.21.15.50:0/3430082942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19f0074760 msgr2=0x7f19f0076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:09.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.673+0000 7f1a15e00640 1 --2- 172.21.15.50:0/3430082942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19f0074760 0x7f19f0076c70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0af000 tx=0x7f19fc01a0c0 comp rx=0 tx=0).stop 2024-09-21T06:07:09.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.673+0000 7f1a15e00640 1 -- 172.21.15.50:0/3430082942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a100cd260 msgr2=0x7f1a1015d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:09.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.673+0000 7f1a15e00640 1 --2- 172.21.15.50:0/3430082942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a100cd260 0x7f1a1015d100 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800ba50 tx=0x7f1a0800bf10 comp rx=0 tx=0).stop 2024-09-21T06:07:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.673+0000 7f1a15e00640 1 -- 172.21.15.50:0/3430082942 shutdown_connections 2024-09-21T06:07:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.673+0000 7f1a15e00640 1 --2- 172.21.15.50:0/3430082942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a1015ef10 0x7f1a1014d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.673+0000 7f1a15e00640 1 --2- 172.21.15.50:0/3430082942 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a100ceba0 0x7f1a1014cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.673+0000 7f1a15e00640 1 --2- 172.21.15.50:0/3430082942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19f0074760 0x7f19f0076c70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.673+0000 7f1a15e00640 1 --2- 172.21.15.50:0/3430082942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a100cd260 0x7f1a1015d100 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.673+0000 7f1a15e00640 1 -- 172.21.15.50:0/3430082942 >> 172.21.15.50:0/3430082942 conn(0x7f1a1006e1d0 msgr2=0x7f1a10157930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:09.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.677+0000 7f1a15e00640 1 -- 172.21.15.50:0/3430082942 shutdown_connections 2024-09-21T06:07:09.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:09.677+0000 7f1a15e00640 1 -- 172.21.15.50:0/3430082942 wait complete. 2024-09-21T06:07:09.731 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone67087 2024-09-21T06:07:10.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.085+0000 7fcabce00640 1 -- 172.21.15.50:0/1680962636 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcab80d0ce0 msgr2=0x7fcab80d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:10.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.085+0000 7fcabce00640 1 --2- 172.21.15.50:0/1680962636 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcab80d0ce0 0x7fcab80d10f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009960 tx=0x7fcaa401c870 comp rx=0 tx=0).stop 2024-09-21T06:07:10.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.085+0000 7fcabce00640 1 -- 172.21.15.50:0/1680962636 shutdown_connections 2024-09-21T06:07:10.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.085+0000 7fcabce00640 1 --2- 172.21.15.50:0/1680962636 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcab80d0ce0 0x7fcab80d10f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:10.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.085+0000 7fcabce00640 1 -- 172.21.15.50:0/1680962636 >> 172.21.15.50:0/1680962636 conn(0x7fcab8073a10 msgr2=0x7fcab806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:10.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.085+0000 7fcabce00640 1 -- 172.21.15.50:0/1680962636 shutdown_connections 2024-09-21T06:07:10.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.089+0000 7fcabce00640 1 -- 172.21.15.50:0/1680962636 wait complete. 2024-09-21T06:07:10.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.093+0000 7fcabce00640 1 Processor -- start 2024-09-21T06:07:10.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.093+0000 7fcabce00640 1 -- start start 2024-09-21T06:07:10.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.097+0000 7fcabce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcab80cf1c0 0x7fcab8152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:10.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.097+0000 7fcabce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcab80d0ce0 0x7fcab80ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:10.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcabce00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcab81625f0 0x7fcab80b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:10.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcabce00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcab816d850 con 0x7fcab80d0ce0 2024-09-21T06:07:10.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcab7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcab80cf1c0 0x7fcab8152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:10.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcab7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcab80cf1c0 0x7fcab8152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:32958/0 (socket says 172.21.15.50:32958) 2024-09-21T06:07:10.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcab7400640 1 -- 172.21.15.50:0/3471491375 learned_addr learned my addr 172.21.15.50:0/3471491375 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:10.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcabce00640 1 -- 172.21.15.50:0/3471491375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcab816d6d0 con 0x7fcab80cf1c0 2024-09-21T06:07:10.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcabce00640 1 -- 172.21.15.50:0/3471491375 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fcab816d9d0 con 0x7fcab81625f0 2024-09-21T06:07:10.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcab6a00640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcab80d0ce0 0x7fcab80ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:10.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcab6a00640 1 -- 172.21.15.50:0/3471491375 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcab81625f0 msgr2=0x7fcab80b63d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:07:10.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcab6a00640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcab81625f0 0x7fcab80b63d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:10.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcab6a00640 1 -- 172.21.15.50:0/3471491375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcab80cf1c0 msgr2=0x7fcab8152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:10.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcab6a00640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcab80cf1c0 0x7fcab8152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:10.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcab6a00640 1 -- 172.21.15.50:0/3471491375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab80b40a0 con 0x7fcab80d0ce0 2024-09-21T06:07:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcab6a00640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcab80d0ce0 0x7fcab80ccd50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fcab80b7760 tx=0x7fcab000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcaaf400640 1 -- 172.21.15.50:0/3471491375 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0003980 con 0x7fcab80d0ce0 2024-09-21T06:07:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcaaf400640 1 -- 172.21.15.50:0/3471491375 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcab0003c10 con 0x7fcab80d0ce0 2024-09-21T06:07:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcaaf400640 1 -- 172.21.15.50:0/3471491375 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0004020 con 0x7fcab80d0ce0 2024-09-21T06:07:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcabce00640 1 -- 172.21.15.50:0/3471491375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab80b4380 con 0x7fcab80d0ce0 2024-09-21T06:07:10.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.101+0000 7fcabce00640 1 -- 172.21.15.50:0/3471491375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab81fed60 con 0x7fcab80d0ce0 2024-09-21T06:07:10.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.105+0000 7fcabce00640 1 -- 172.21.15.50:0/3471491375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab815e890 con 0x7fcab80d0ce0 2024-09-21T06:07:10.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.109+0000 7fcaaf400640 1 -- 172.21.15.50:0/3471491375 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7fcab00041c0 con 0x7fcab80d0ce0 2024-09-21T06:07:10.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.109+0000 7fcaaf400640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca98074760 0x7fca98076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:10.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.113+0000 7fcaaf400640 1 -- 172.21.15.50:0/3471491375 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fcab0093b30 con 0x7fcab80d0ce0 2024-09-21T06:07:10.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.117+0000 7fcab7400640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca98074760 0x7fca98076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:10.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.133+0000 7fcab7400640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca98074760 0x7fca98076c70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4000c10 tx=0x7fcaa400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:10.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.133+0000 7fcaaf400640 1 -- 172.21.15.50:0/3471491375 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcab0060270 con 0x7fcab80d0ce0 2024-09-21T06:07:10.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:10.533+0000 7fcabce00640 1 -- 172.21.15.50:0/3471491375 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone67087", "target": ["mon-mgr", ""]}) -- 0x7fcab806b540 con 0x7fca98074760 2024-09-21T06:07:12.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.189+0000 7fcaaf400640 1 -- 172.21.15.50:0/3471491375 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcaa400b0b0 con 0x7fca98074760 2024-09-21T06:07:12.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.193+0000 7fcabce00640 1 -- 172.21.15.50:0/3471491375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca98074760 msgr2=0x7fca98076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.193+0000 7fcabce00640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca98074760 0x7fca98076c70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4000c10 tx=0x7fcaa400b040 comp rx=0 tx=0).stop 2024-09-21T06:07:12.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.193+0000 7fcabce00640 1 -- 172.21.15.50:0/3471491375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcab80d0ce0 msgr2=0x7fcab80ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.193+0000 7fcabce00640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcab80d0ce0 0x7fcab80ccd50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fcab80b7760 tx=0x7fcab000be00 comp rx=0 tx=0).stop 2024-09-21T06:07:12.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.193+0000 7fcabce00640 1 -- 172.21.15.50:0/3471491375 shutdown_connections 2024-09-21T06:07:12.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.193+0000 7fcabce00640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcab81625f0 0x7fcab80b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.193+0000 7fcabce00640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca98074760 0x7fca98076c70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.193+0000 7fcabce00640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcab80d0ce0 0x7fcab80ccd50 secure :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fcab80b7760 tx=0x7fcab000be00 comp rx=0 tx=0).stop 2024-09-21T06:07:12.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.193+0000 7fcabce00640 1 --2- 172.21.15.50:0/3471491375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcab80cf1c0 0x7fcab8152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.193+0000 7fcabce00640 1 -- 172.21.15.50:0/3471491375 >> 172.21.15.50:0/3471491375 conn(0x7fcab8073a10 msgr2=0x7fcab80b9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:12.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.197+0000 7fcabce00640 1 -- 172.21.15.50:0/3471491375 shutdown_connections 2024-09-21T06:07:12.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.197+0000 7fcabce00640 1 -- 172.21.15.50:0/3471491375 wait complete. 2024-09-21T06:07:12.243 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone71268 2024-09-21T06:07:12.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.513+0000 7f3248000640 1 -- 172.21.15.50:0/127404129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32400703a0 msgr2=0x7f324006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.513+0000 7f3248000640 1 --2- 172.21.15.50:0/127404129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32400703a0 0x7f324006d200 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f32400b6bf0 tx=0x7f323801cb10 comp rx=0 tx=0).stop 2024-09-21T06:07:12.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.513+0000 7f3248000640 1 -- 172.21.15.50:0/127404129 shutdown_connections 2024-09-21T06:07:12.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.513+0000 7f3248000640 1 --2- 172.21.15.50:0/127404129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32400703a0 0x7f324006d200 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.513+0000 7f3248000640 1 --2- 172.21.15.50:0/127404129 >> v2:172.21.15.88:3300/0 conn(0x7f3240159130 0x7f324015bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.513+0000 7f3248000640 1 -- 172.21.15.50:0/127404129 >> 172.21.15.50:0/127404129 conn(0x7f32400b35f0 msgr2=0x7f32400b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:12.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.513+0000 7f3248000640 1 -- 172.21.15.50:0/127404129 shutdown_connections 2024-09-21T06:07:12.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.513+0000 7f3248000640 1 -- 172.21.15.50:0/127404129 wait complete. 2024-09-21T06:07:12.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.517+0000 7f3248000640 1 Processor -- start 2024-09-21T06:07:12.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.517+0000 7f3248000640 1 -- start start 2024-09-21T06:07:12.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3248000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324006cdf0 0x7f3240157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3248000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f32400703a0 0x7f3240161880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3248000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3240159130 0x7f324015a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3248000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3240168de0 con 0x7f3240159130 2024-09-21T06:07:12.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3248000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3240168c60 con 0x7f324006cdf0 2024-09-21T06:07:12.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3248000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3240168f60 con 0x7f32400703a0 2024-09-21T06:07:12.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3246000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3240159130 0x7f324015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3246000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3240159130 0x7f324015a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42156/0 (socket says 172.21.15.50:42156) 2024-09-21T06:07:12.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3246000640 1 -- 172.21.15.50:0/400328019 learned_addr learned my addr 172.21.15.50:0/400328019 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:12.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3245600640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324006cdf0 0x7f3240157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3246000640 1 -- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f32400703a0 msgr2=0x7f3240161880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:07:12.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3246000640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f32400703a0 0x7f3240161880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3246000640 1 -- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324006cdf0 msgr2=0x7f3240157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3246000640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324006cdf0 0x7f3240157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3246000640 1 -- 172.21.15.50:0/400328019 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32400cf550 con 0x7f3240159130 2024-09-21T06:07:12.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3246000640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3240159130 0x7f324015a800 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f323801b1c0 tx=0x7f3238003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:12.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f323d600640 1 -- 172.21.15.50:0/400328019 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32380bc070 con 0x7f3240159130 2024-09-21T06:07:12.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f323d600640 1 -- 172.21.15.50:0/400328019 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32380042f0 con 0x7f3240159130 2024-09-21T06:07:12.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f3248000640 1 -- 172.21.15.50:0/400328019 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32400cdbb0 con 0x7f3240159130 2024-09-21T06:07:12.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.521+0000 7f323d600640 1 -- 172.21.15.50:0/400328019 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32380b2040 con 0x7f3240159130 2024-09-21T06:07:12.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.525+0000 7f3248000640 1 -- 172.21.15.50:0/400328019 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32400ce790 con 0x7f3240159130 2024-09-21T06:07:12.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.529+0000 7f3248000640 1 -- 172.21.15.50:0/400328019 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324006d200 con 0x7f3240159130 2024-09-21T06:07:12.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.529+0000 7f323d600640 1 -- 172.21.15.50:0/400328019 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f32380bf070 con 0x7f3240159130 2024-09-21T06:07:12.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.529+0000 7f323d600640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3220074760 0x7f3220076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.529+0000 7f323d600640 1 -- 172.21.15.50:0/400328019 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f323812d110 con 0x7f3240159130 2024-09-21T06:07:12.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.537+0000 7f323d600640 1 -- 172.21.15.50:0/400328019 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f324015c0d0 con 0x7f3240159130 2024-09-21T06:07:12.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.541+0000 7f3245600640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3220074760 0x7f3220076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.549+0000 7f3245600640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3220074760 0x7f3220076c70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3234006fd0 tx=0x7f3234008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:12.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:12.913+0000 7f3248000640 1 -- 172.21.15.50:0/400328019 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone71268", "target": ["mon-mgr", ""]}) -- 0x7f324006b540 con 0x7f3220074760 2024-09-21T06:07:13.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.628+0000 7f323d600640 1 -- 172.21.15.50:0/400328019 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f323400bda0 con 0x7f3220074760 2024-09-21T06:07:13.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.636+0000 7f3248000640 1 -- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3220074760 msgr2=0x7f3220076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.636+0000 7f3248000640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3220074760 0x7f3220076c70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3234006fd0 tx=0x7f3234008040 comp rx=0 tx=0).stop 2024-09-21T06:07:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.636+0000 7f3248000640 1 -- 172.21.15.50:0/400328019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3240159130 msgr2=0x7f324015a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.636+0000 7f3248000640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3240159130 0x7f324015a800 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f323801b1c0 tx=0x7f3238003f80 comp rx=0 tx=0).stop 2024-09-21T06:07:13.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.636+0000 7f3248000640 1 -- 172.21.15.50:0/400328019 shutdown_connections 2024-09-21T06:07:13.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.636+0000 7f3248000640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3220074760 0x7f3220076c70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:13.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.636+0000 7f3248000640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3240159130 0x7f324015a800 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:13.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.636+0000 7f3248000640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f32400703a0 0x7f3240161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:13.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.636+0000 7f3248000640 1 --2- 172.21.15.50:0/400328019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324006cdf0 0x7f3240157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:13.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.636+0000 7f3248000640 1 -- 172.21.15.50:0/400328019 >> 172.21.15.50:0/400328019 conn(0x7f32400b35f0 msgr2=0x7f3240075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:13.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.636+0000 7f3248000640 1 -- 172.21.15.50:0/400328019 shutdown_connections 2024-09-21T06:07:13.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:13.636+0000 7f3248000640 1 -- 172.21.15.50:0/400328019 wait complete. 2024-09-21T06:07:13.691 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone71879 2024-09-21T06:07:14.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.004+0000 7f228b200640 1 -- 172.21.15.50:0/3537226111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2270004d50 msgr2=0x7f2270005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:14.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.004+0000 7f228b200640 1 --2- 172.21.15.50:0/3537226111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2270004d50 0x7f2270005160 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f227c00bb10 tx=0x7f227c01cd60 comp rx=0 tx=0).stop 2024-09-21T06:07:14.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.004+0000 7f228a800640 1 -- 172.21.15.50:0/3537226111 reap_dead start 2024-09-21T06:07:14.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.008+0000 7f228b200640 1 -- 172.21.15.50:0/3537226111 shutdown_connections 2024-09-21T06:07:14.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.008+0000 7f228b200640 1 -- 172.21.15.50:0/3537226111 >> 172.21.15.50:0/3537226111 conn(0x7f228c06d3f0 msgr2=0x7f228c0b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:14.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.008+0000 7f228b200640 1 -- 172.21.15.50:0/3537226111 shutdown_connections 2024-09-21T06:07:14.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.008+0000 7f228b200640 1 -- 172.21.15.50:0/3537226111 wait complete. 2024-09-21T06:07:14.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.008+0000 7f228b200640 1 Processor -- start 2024-09-21T06:07:14.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.008+0000 7f228b200640 1 -- start start 2024-09-21T06:07:14.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f228b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f228c14c4e0 0x7f228c13de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:14.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f228b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f228c152ae0 0x7f228c152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:14.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f228b200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f228c0b1220 0x7f228c0b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:14.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f228b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f228c0cf7d0 con 0x7f228c152ae0 2024-09-21T06:07:14.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f228b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f228c0cf650 con 0x7f228c14c4e0 2024-09-21T06:07:14.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f228b200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f228c0cf950 con 0x7f228c0b1220 2024-09-21T06:07:14.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f2289400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f228c152ae0 0x7f228c152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:14.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f2289400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f228c152ae0 0x7f228c152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42176/0 (socket says 172.21.15.50:42176) 2024-09-21T06:07:14.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f2289400640 1 -- 172.21.15.50:0/575853065 learned_addr learned my addr 172.21.15.50:0/575853065 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f2289400640 1 -- 172.21.15.50:0/575853065 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f228c0b1220 msgr2=0x7f228c0b1630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:07:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f2289400640 1 --2- 172.21.15.50:0/575853065 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f228c0b1220 0x7f228c0b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f2289400640 1 -- 172.21.15.50:0/575853065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f228c14c4e0 msgr2=0x7f228c13de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f2289400640 1 --2- 172.21.15.50:0/575853065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f228c14c4e0 0x7f228c13de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f2289400640 1 -- 172.21.15.50:0/575853065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c13e370 con 0x7f228c152ae0 2024-09-21T06:07:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.012+0000 7f2289400640 1 --2- 172.21.15.50:0/575853065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f228c152ae0 0x7f228c152ef0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2284011d30 tx=0x7f228400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:14.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.016+0000 7f2282000640 1 -- 172.21.15.50:0/575853065 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284003b80 con 0x7f228c152ae0 2024-09-21T06:07:14.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.016+0000 7f228b200640 1 -- 172.21.15.50:0/575853065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228c153430 con 0x7f228c152ae0 2024-09-21T06:07:14.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.016+0000 7f228b200640 1 -- 172.21.15.50:0/575853065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228c14fd90 con 0x7f228c152ae0 2024-09-21T06:07:14.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.016+0000 7f2282000640 1 -- 172.21.15.50:0/575853065 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2284003d20 con 0x7f228c152ae0 2024-09-21T06:07:14.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.016+0000 7f2282000640 1 -- 172.21.15.50:0/575853065 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284004290 con 0x7f228c152ae0 2024-09-21T06:07:14.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.016+0000 7f228b200640 1 -- 172.21.15.50:0/575853065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f228c0b3dc0 con 0x7f228c152ae0 2024-09-21T06:07:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.016+0000 7f2282000640 1 -- 172.21.15.50:0/575853065 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f2284004430 con 0x7f228c152ae0 2024-09-21T06:07:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.020+0000 7f2282000640 1 --2- 172.21.15.50:0/575853065 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f226c074760 0x7f226c076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.020+0000 7f2282000640 1 -- 172.21.15.50:0/575853065 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f2284093f60 con 0x7f228c152ae0 2024-09-21T06:07:14.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.028+0000 7f2282000640 1 -- 172.21.15.50:0/575853065 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f228c153430 con 0x7f228c152ae0 2024-09-21T06:07:14.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.028+0000 7f2289e00640 1 --2- 172.21.15.50:0/575853065 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f226c074760 0x7f226c076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:14.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.028+0000 7f2289e00640 1 --2- 172.21.15.50:0/575853065 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f226c074760 0x7f226c076c70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2278006c30 tx=0x7f2278006bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:14.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:14.456+0000 7f228b200640 1 -- 172.21.15.50:0/575853065 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone71879", "target": ["mon-mgr", ""]}) -- 0x7f228c06b540 con 0x7f226c074760 2024-09-21T06:07:15.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.428+0000 7f2282000640 1 -- 172.21.15.50:0/575853065 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f22780064f0 con 0x7f226c074760 2024-09-21T06:07:15.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.432+0000 7f228b200640 1 -- 172.21.15.50:0/575853065 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f226c074760 msgr2=0x7f226c076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:15.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.432+0000 7f228b200640 1 --2- 172.21.15.50:0/575853065 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f226c074760 0x7f226c076c70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2278006c30 tx=0x7f2278006bc0 comp rx=0 tx=0).stop 2024-09-21T06:07:15.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.432+0000 7f228b200640 1 -- 172.21.15.50:0/575853065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f228c152ae0 msgr2=0x7f228c152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:15.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.432+0000 7f228b200640 1 --2- 172.21.15.50:0/575853065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f228c152ae0 0x7f228c152ef0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2284011d30 tx=0x7f228400f790 comp rx=0 tx=0).stop 2024-09-21T06:07:15.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.432+0000 7f228b200640 1 -- 172.21.15.50:0/575853065 shutdown_connections 2024-09-21T06:07:15.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.432+0000 7f228b200640 1 --2- 172.21.15.50:0/575853065 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f228c0b1220 0x7f228c0b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:15.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.432+0000 7f228b200640 1 --2- 172.21.15.50:0/575853065 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f226c074760 0x7f226c076c70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:15.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.432+0000 7f228b200640 1 --2- 172.21.15.50:0/575853065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f228c152ae0 0x7f228c152ef0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:15.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.432+0000 7f228b200640 1 --2- 172.21.15.50:0/575853065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f228c14c4e0 0x7f228c13de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:15.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.432+0000 7f228b200640 1 -- 172.21.15.50:0/575853065 >> 172.21.15.50:0/575853065 conn(0x7f228c06d3f0 msgr2=0x7f228c153f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:15.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.432+0000 7f228b200640 1 -- 172.21.15.50:0/575853065 shutdown_connections 2024-09-21T06:07:15.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.432+0000 7f228b200640 1 -- 172.21.15.50:0/575853065 wait complete. 2024-09-21T06:07:15.474 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone647110 2024-09-21T06:07:15.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.824+0000 7fde5c400640 1 -- 172.21.15.50:0/1501061090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde5415cbb0 msgr2=0x7fde5415cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:15.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.824+0000 7fde5c400640 1 --2- 172.21.15.50:0/1501061090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde5415cbb0 0x7fde5415cfc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fde40009840 tx=0x7fde4001c690 comp rx=0 tx=0).stop 2024-09-21T06:07:15.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.824+0000 7fde5c400640 1 -- 172.21.15.50:0/1501061090 shutdown_connections 2024-09-21T06:07:15.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.824+0000 7fde5c400640 1 --2- 172.21.15.50:0/1501061090 >> v2:172.21.15.88:3300/0 conn(0x7fde54072100 0x7fde54072510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:15.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.824+0000 7fde5c400640 1 --2- 172.21.15.50:0/1501061090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde5415cbb0 0x7fde5415cfc0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:15.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.824+0000 7fde5c400640 1 -- 172.21.15.50:0/1501061090 >> 172.21.15.50:0/1501061090 conn(0x7fde540b2040 msgr2=0x7fde540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:15.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.824+0000 7fde5c400640 1 -- 172.21.15.50:0/1501061090 shutdown_connections 2024-09-21T06:07:15.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.824+0000 7fde5c400640 1 -- 172.21.15.50:0/1501061090 wait complete. 2024-09-21T06:07:15.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.828+0000 7fde5c400640 1 Processor -- start 2024-09-21T06:07:15.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.828+0000 7fde5c400640 1 -- start start 2024-09-21T06:07:15.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.828+0000 7fde5c400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde54072100 0x7fde54134cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:15.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.828+0000 7fde5c400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cefa0 0x7fde54145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:15.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.828+0000 7fde5c400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fde5415cbb0 0x7fde541458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:15.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.828+0000 7fde5c400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde54169ed0 con 0x7fde54072100 2024-09-21T06:07:15.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.828+0000 7fde5c400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde54169d50 con 0x7fde540cefa0 2024-09-21T06:07:15.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.828+0000 7fde5c400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fde5416a050 con 0x7fde5415cbb0 2024-09-21T06:07:15.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.828+0000 7fde59a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde54072100 0x7fde54134cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:15.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde59a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde54072100 0x7fde54134cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42198/0 (socket says 172.21.15.50:42198) 2024-09-21T06:07:15.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde59a00640 1 -- 172.21.15.50:0/594985698 learned_addr learned my addr 172.21.15.50:0/594985698 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:15.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde5a400640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fde5415cbb0 0x7fde541458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:15.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde59a00640 1 -- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fde5415cbb0 msgr2=0x7fde541458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:15.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde59a00640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fde5415cbb0 0x7fde541458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:15.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde59000640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cefa0 0x7fde54145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:15.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde59a00640 1 -- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cefa0 msgr2=0x7fde54145370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:15.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde59a00640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cefa0 0x7fde54145370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:15.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde59a00640 1 -- 172.21.15.50:0/594985698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde54155990 con 0x7fde54072100 2024-09-21T06:07:15.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde59000640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cefa0 0x7fde54145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:15.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde59a00640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde54072100 0x7fde54134cd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c400 tx=0x7fde4800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:15.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde51600640 1 -- 172.21.15.50:0/594985698 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48017070 con 0x7fde54072100 2024-09-21T06:07:15.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde51600640 1 -- 172.21.15.50:0/594985698 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde48003600 con 0x7fde54072100 2024-09-21T06:07:15.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde51600640 1 -- 172.21.15.50:0/594985698 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48003a50 con 0x7fde54072100 2024-09-21T06:07:15.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde5c400640 1 -- 172.21.15.50:0/594985698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5413e7f0 con 0x7fde54072100 2024-09-21T06:07:15.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde5c400640 1 -- 172.21.15.50:0/594985698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde5413edb0 con 0x7fde54072100 2024-09-21T06:07:15.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.832+0000 7fde5c400640 1 -- 172.21.15.50:0/594985698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde54136720 con 0x7fde54072100 2024-09-21T06:07:15.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.836+0000 7fde51600640 1 -- 172.21.15.50:0/594985698 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7fde4801a070 con 0x7fde54072100 2024-09-21T06:07:15.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.836+0000 7fde51600640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fde34074790 0x7fde34076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:15.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.836+0000 7fde51600640 1 -- 172.21.15.50:0/594985698 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7fde480925c0 con 0x7fde54072100 2024-09-21T06:07:15.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.836+0000 7fde59000640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fde34074790 0x7fde34076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.840+0000 7fde59000640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fde34074790 0x7fde34076ca0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fde400af000 tx=0x7fde4001aed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:15.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:15.844+0000 7fde51600640 1 -- 172.21.15.50:0/594985698 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fde4805ec30 con 0x7fde54072100 2024-09-21T06:07:16.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:16.224+0000 7fde5c400640 1 -- 172.21.15.50:0/594985698 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone647110", "target": ["mon-mgr", ""]}) -- 0x7fde5413f5b0 con 0x7fde34074790 2024-09-21T06:07:17.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.144+0000 7fde51600640 1 -- 172.21.15.50:0/594985698 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fde5413f5b0 con 0x7fde34074790 2024-09-21T06:07:17.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.152+0000 7fde5c400640 1 -- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fde34074790 msgr2=0x7fde34076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:17.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.152+0000 7fde5c400640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fde34074790 0x7fde34076ca0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fde400af000 tx=0x7fde4001aed0 comp rx=0 tx=0).stop 2024-09-21T06:07:17.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.152+0000 7fde5c400640 1 -- 172.21.15.50:0/594985698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde54072100 msgr2=0x7fde54134cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:17.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.152+0000 7fde5c400640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde54072100 0x7fde54134cd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c400 tx=0x7fde4800c8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:17.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.152+0000 7fde5c400640 1 -- 172.21.15.50:0/594985698 shutdown_connections 2024-09-21T06:07:17.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.152+0000 7fde5c400640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fde5415cbb0 0x7fde541458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:17.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.152+0000 7fde5c400640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cefa0 0x7fde54145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:17.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.152+0000 7fde5c400640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fde34074790 0x7fde34076ca0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:17.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.152+0000 7fde5c400640 1 --2- 172.21.15.50:0/594985698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde54072100 0x7fde54134cd0 secure :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c400 tx=0x7fde4800c8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:17.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.152+0000 7fde5c400640 1 -- 172.21.15.50:0/594985698 >> 172.21.15.50:0/594985698 conn(0x7fde540b2040 msgr2=0x7fde54142f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:17.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.152+0000 7fde5c400640 1 -- 172.21.15.50:0/594985698 shutdown_connections 2024-09-21T06:07:17.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.152+0000 7fde5c400640 1 -- 172.21.15.50:0/594985698 wait complete. 2024-09-21T06:07:17.200 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone189911 2024-09-21T06:07:17.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.476+0000 7efdcb200640 1 -- 172.21.15.50:0/3349572418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0004e50 msgr2=0x7efdb000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:17.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.476+0000 7efdcb200640 1 --2- 172.21.15.50:0/3349572418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0004e50 0x7efdb000b9d0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009f90 tx=0x7efdc401c930 comp rx=0 tx=0).stop 2024-09-21T06:07:17.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.476+0000 7efdca800640 1 -- 172.21.15.50:0/3349572418 reap_dead start 2024-09-21T06:07:17.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.476+0000 7efdcb200640 1 -- 172.21.15.50:0/3349572418 shutdown_connections 2024-09-21T06:07:17.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.476+0000 7efdcb200640 1 -- 172.21.15.50:0/3349572418 >> 172.21.15.50:0/3349572418 conn(0x7efdcc0b41f0 msgr2=0x7efdcc0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:17.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.480+0000 7efdcb200640 1 -- 172.21.15.50:0/3349572418 shutdown_connections 2024-09-21T06:07:17.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.480+0000 7efdcb200640 1 -- 172.21.15.50:0/3349572418 wait complete. 2024-09-21T06:07:17.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.484+0000 7efdcb200640 1 Processor -- start 2024-09-21T06:07:17.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.488+0000 7efdcb200640 1 -- start start 2024-09-21T06:07:17.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.488+0000 7efdcb200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0004e50 0x7efdcc1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:17.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.488+0000 7efdcb200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdcc1fc620 0x7efdcc1fca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:17.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.488+0000 7efdcb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdcc070160 0x7efdcc070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:17.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdc9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0004e50 0x7efdcc1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:17.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdcb200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc15fb50 con 0x7efdb0004e50 2024-09-21T06:07:17.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdcb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc15f9d0 con 0x7efdcc070160 2024-09-21T06:07:17.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdcb200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7efdcc15fcd0 con 0x7efdcc1fc620 2024-09-21T06:07:17.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdc9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0004e50 0x7efdcc1569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50522/0 (socket says 172.21.15.50:50522) 2024-09-21T06:07:17.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdc9e00640 1 -- 172.21.15.50:0/1159666954 learned_addr learned my addr 172.21.15.50:0/1159666954 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:17.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdca800640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdcc070160 0x7efdcc070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:17.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdc9e00640 1 -- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdcc1fc620 msgr2=0x7efdcc1fca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:17.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdc9e00640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdcc1fc620 0x7efdcc1fca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:17.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdc9e00640 1 -- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdcc070160 msgr2=0x7efdcc070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:17.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdc9e00640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdcc070160 0x7efdcc070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:17.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdc9e00640 1 -- 172.21.15.50:0/1159666954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdcc070ab0 con 0x7efdb0004e50 2024-09-21T06:07:17.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdc9e00640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0004e50 0x7efdcc1569e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00f410 tx=0x7efdbc00f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:17.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdc2000640 1 -- 172.21.15.50:0/1159666954 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc01a070 con 0x7efdb0004e50 2024-09-21T06:07:17.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdc2000640 1 -- 172.21.15.50:0/1159666954 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdbc0084e0 con 0x7efdb0004e50 2024-09-21T06:07:17.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdc2000640 1 -- 172.21.15.50:0/1159666954 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc006980 con 0x7efdb0004e50 2024-09-21T06:07:17.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.492+0000 7efdcb200640 1 -- 172.21.15.50:0/1159666954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdcc1573c0 con 0x7efdb0004e50 2024-09-21T06:07:17.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.496+0000 7efdcb200640 1 -- 172.21.15.50:0/1159666954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdcc0c1970 con 0x7efdb0004e50 2024-09-21T06:07:17.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.496+0000 7efdcb200640 1 -- 172.21.15.50:0/1159666954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdcc0b0730 con 0x7efdb0004e50 2024-09-21T06:07:17.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.496+0000 7efdc2000640 1 -- 172.21.15.50:0/1159666954 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7efdbc01d070 con 0x7efdb0004e50 2024-09-21T06:07:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.500+0000 7efdc2000640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efda8074790 0x7efda8076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.500+0000 7efdc2000640 1 -- 172.21.15.50:0/1159666954 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7efdbc095530 con 0x7efdb0004e50 2024-09-21T06:07:17.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.508+0000 7efdc2000640 1 -- 172.21.15.50:0/1159666954 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efdcc070ab0 con 0x7efdb0004e50 2024-09-21T06:07:17.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.512+0000 7efdc9400640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efda8074790 0x7efda8076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:17.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.536+0000 7efdc9400640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efda8074790 0x7efda8076ca0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efdcc09f3d0 tx=0x7efdb4006bb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:17.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:17.848+0000 7efdcb200640 1 -- 172.21.15.50:0/1159666954 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone189911", "target": ["mon-mgr", ""]}) -- 0x7efdcc06b540 con 0x7efda8074790 2024-09-21T06:07:18.135 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:07:18.140 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:07:19.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.048+0000 7efdc2000640 1 -- 172.21.15.50:0/1159666954 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efdb4009200 con 0x7efda8074790 2024-09-21T06:07:19.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.060+0000 7efdcb200640 1 -- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efda8074790 msgr2=0x7efda8076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:19.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.060+0000 7efdcb200640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efda8074790 0x7efda8076ca0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efdcc09f3d0 tx=0x7efdb4006bb0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.060+0000 7efdcb200640 1 -- 172.21.15.50:0/1159666954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0004e50 msgr2=0x7efdcc1569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:19.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.060+0000 7efdcb200640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0004e50 0x7efdcc1569e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00f410 tx=0x7efdbc00f8d0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.060+0000 7efdcb200640 1 -- 172.21.15.50:0/1159666954 shutdown_connections 2024-09-21T06:07:19.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.060+0000 7efdcb200640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdcc070160 0x7efdcc070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.060+0000 7efdcb200640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdcc1fc620 0x7efdcc1fca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.060+0000 7efdcb200640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efda8074790 0x7efda8076ca0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.060+0000 7efdcb200640 1 --2- 172.21.15.50:0/1159666954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0004e50 0x7efdcc1569e0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.060+0000 7efdcb200640 1 -- 172.21.15.50:0/1159666954 >> 172.21.15.50:0/1159666954 conn(0x7efdcc0b41f0 msgr2=0x7efdcc159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:19.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.060+0000 7efdcb200640 1 -- 172.21.15.50:0/1159666954 shutdown_connections 2024-09-21T06:07:19.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.064+0000 7efdcb200640 1 -- 172.21.15.50:0/1159666954 wait complete. 2024-09-21T06:07:19.109 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone837612 2024-09-21T06:07:19.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.392+0000 7f806e000640 1 -- 172.21.15.50:0/3345235733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80680ce4c0 msgr2=0x7f80680cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:19.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.392+0000 7f806e000640 1 --2- 172.21.15.50:0/3345235733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80680ce4c0 0x7f80680cdaf0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f80680ce8d0 tx=0x7f805801c8e0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.392+0000 7f806e000640 1 -- 172.21.15.50:0/3345235733 shutdown_connections 2024-09-21T06:07:19.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.392+0000 7f806e000640 1 --2- 172.21.15.50:0/3345235733 >> v2:172.21.15.88:3300/0 conn(0x7f806815ebf0 0x7f8068165660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.392+0000 7f806e000640 1 --2- 172.21.15.50:0/3345235733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80680ce4c0 0x7f80680cdaf0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.392+0000 7f806e000640 1 -- 172.21.15.50:0/3345235733 >> 172.21.15.50:0/3345235733 conn(0x7f80680b2040 msgr2=0x7f80680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:19.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.392+0000 7f806e000640 1 -- 172.21.15.50:0/3345235733 shutdown_connections 2024-09-21T06:07:19.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.392+0000 7f806e000640 1 -- 172.21.15.50:0/3345235733 wait complete. 2024-09-21T06:07:19.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.400+0000 7f806e000640 1 Processor -- start 2024-09-21T06:07:19.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.400+0000 7f806e000640 1 -- start start 2024-09-21T06:07:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.408+0000 7f806e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80680ce4c0 0x7f80680aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:19.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.408+0000 7f806e000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f806815aa50 0x7f8068205580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:19.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.408+0000 7f806e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f806815ebf0 0x7f8068073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:19.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.408+0000 7f806e000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8068165fe0 con 0x7f806815ebf0 2024-09-21T06:07:19.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.408+0000 7f806e000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8068165e60 con 0x7f80680ce4c0 2024-09-21T06:07:19.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.408+0000 7f806e000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8068166160 con 0x7f806815aa50 2024-09-21T06:07:19.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.408+0000 7f8067400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80680ce4c0 0x7f80680aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:19.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.408+0000 7f8067400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80680ce4c0 0x7f80680aafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:48654/0 (socket says 172.21.15.50:48654) 2024-09-21T06:07:19.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.408+0000 7f8067400640 1 -- 172.21.15.50:0/1159951711 learned_addr learned my addr 172.21.15.50:0/1159951711 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:19.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.408+0000 7f8067e00640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f806815ebf0 0x7f8068073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:19.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.412+0000 7f8067400640 1 -- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f806815aa50 msgr2=0x7f8068205580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:19.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.412+0000 7f8067400640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f806815aa50 0x7f8068205580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.412+0000 7f8067400640 1 -- 172.21.15.50:0/1159951711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f806815ebf0 msgr2=0x7f8068073cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:19.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.412+0000 7f8067400640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f806815ebf0 0x7f8068073cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.412+0000 7f8067400640 1 -- 172.21.15.50:0/1159951711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8068205ac0 con 0x7f80680ce4c0 2024-09-21T06:07:19.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.412+0000 7f8067400640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80680ce4c0 0x7f80680aafe0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8058009850 tx=0x7f8058005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:19.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.416+0000 7f805f400640 1 -- 172.21.15.50:0/1159951711 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80580bc070 con 0x7f80680ce4c0 2024-09-21T06:07:19.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.416+0000 7f805f400640 1 -- 172.21.15.50:0/1159951711 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8058003600 con 0x7f80680ce4c0 2024-09-21T06:07:19.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.416+0000 7f805f400640 1 -- 172.21.15.50:0/1159951711 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80580b2040 con 0x7f80680ce4c0 2024-09-21T06:07:19.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.420+0000 7f806e000640 1 -- 172.21.15.50:0/1159951711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80680aea60 con 0x7f80680ce4c0 2024-09-21T06:07:19.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.420+0000 7f806e000640 1 -- 172.21.15.50:0/1159951711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80680abec0 con 0x7f80680ce4c0 2024-09-21T06:07:19.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.420+0000 7f806e000640 1 -- 172.21.15.50:0/1159951711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80400052a0 con 0x7f80680ce4c0 2024-09-21T06:07:19.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.428+0000 7f805f400640 1 -- 172.21.15.50:0/1159951711 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f80580039d0 con 0x7f80680ce4c0 2024-09-21T06:07:19.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.428+0000 7f805f400640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8048074760 0x7f8048076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:19.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.428+0000 7f805f400640 1 -- 172.21.15.50:0/1159951711 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f805812d130 con 0x7f80680ce4c0 2024-09-21T06:07:19.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.444+0000 7f805f400640 1 -- 172.21.15.50:0/1159951711 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f80580164d0 con 0x7f80680ce4c0 2024-09-21T06:07:19.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.472+0000 7f8066a00640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8048074760 0x7f8048076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:19.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.484+0000 7f8066a00640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8048074760 0x7f8048076c70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f80680b2010 tx=0x7f8054006bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:19.880+0000 7f806e000640 1 -- 172.21.15.50:0/1159951711 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone837612", "target": ["mon-mgr", ""]}) -- 0x7f8040000c90 con 0x7f8048074760 2024-09-21T06:07:20.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.592+0000 7f805f400640 1 -- 172.21.15.50:0/1159951711 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8040000c90 con 0x7f8048074760 2024-09-21T06:07:20.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.600+0000 7f806e000640 1 -- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8048074760 msgr2=0x7f8048076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:20.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.600+0000 7f806e000640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8048074760 0x7f8048076c70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f80680b2010 tx=0x7f8054006bd0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.600+0000 7f806e000640 1 -- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80680ce4c0 msgr2=0x7f80680aafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:20.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.600+0000 7f806e000640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80680ce4c0 0x7f80680aafe0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8058009850 tx=0x7f8058005770 comp rx=0 tx=0).stop 2024-09-21T06:07:20.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.600+0000 7f806e000640 1 -- 172.21.15.50:0/1159951711 shutdown_connections 2024-09-21T06:07:20.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.600+0000 7f806e000640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8048074760 0x7f8048076c70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.600+0000 7f806e000640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f806815ebf0 0x7f8068073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.600+0000 7f806e000640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f806815aa50 0x7f8068205580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.600+0000 7f806e000640 1 --2- 172.21.15.50:0/1159951711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80680ce4c0 0x7f80680aafe0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.600+0000 7f806e000640 1 -- 172.21.15.50:0/1159951711 >> 172.21.15.50:0/1159951711 conn(0x7f80680b2040 msgr2=0x7f80680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:20.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.608+0000 7f806e000640 1 -- 172.21.15.50:0/1159951711 shutdown_connections 2024-09-21T06:07:20.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.608+0000 7f806e000640 1 -- 172.21.15.50:0/1159951711 wait complete. 2024-09-21T06:07:20.655 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone986813 2024-09-21T06:07:20.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.992+0000 7f8178a00640 1 -- 172.21.15.50:0/3173891921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81580039f0 msgr2=0x7f8158003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:20.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.992+0000 7f8178a00640 1 --2- 172.21.15.50:0/3173891921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81580039f0 0x7f8158003e20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8174066c60 tx=0x7f8174099ae0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.992+0000 7f8172800640 1 -- 172.21.15.50:0/3173891921 reap_dead start 2024-09-21T06:07:20.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.996+0000 7f8178a00640 1 -- 172.21.15.50:0/3173891921 shutdown_connections 2024-09-21T06:07:20.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.996+0000 7f8178a00640 1 -- 172.21.15.50:0/3173891921 >> 172.21.15.50:0/3173891921 conn(0x7f816c01fbf0 msgr2=0x7f816c056950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:20.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.996+0000 7f8178a00640 1 -- 172.21.15.50:0/3173891921 shutdown_connections 2024-09-21T06:07:20.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:20.996+0000 7f8178a00640 1 -- 172.21.15.50:0/3173891921 wait complete. 2024-09-21T06:07:20.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.000+0000 7f8178a00640 1 Processor -- start 2024-09-21T06:07:21.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.000+0000 7f8178a00640 1 -- start start 2024-09-21T06:07:21.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.004+0000 7f8178a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81580039f0 0x7f816c0e2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:21.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.004+0000 7f8178a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816c0334e0 0x7f816c04c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:21.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.004+0000 7f8178a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f816c17f530 0x7f816c04cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:21.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.004+0000 7f8178a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f816c0ded30 con 0x7f816c17f530 2024-09-21T06:07:21.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.004+0000 7f8178a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f816c0debb0 con 0x7f816c0334e0 2024-09-21T06:07:21.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.004+0000 7f8178a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f816c0deeb0 con 0x7f81580039f0 2024-09-21T06:07:21.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.008+0000 7f8171e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81580039f0 0x7f816c0e2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:21.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.008+0000 7f8172800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f816c17f530 0x7f816c04cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:21.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.008+0000 7f8171400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816c0334e0 0x7f816c04c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:21.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.008+0000 7f8171400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816c0334e0 0x7f816c04c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:48672/0 (socket says 172.21.15.50:48672) 2024-09-21T06:07:21.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.008+0000 7f8171400640 1 -- 172.21.15.50:0/2197659024 learned_addr learned my addr 172.21.15.50:0/2197659024 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:21.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.008+0000 7f8172800640 1 -- 172.21.15.50:0/2197659024 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81580039f0 msgr2=0x7f816c0e2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:21.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.008+0000 7f8172800640 1 --2- 172.21.15.50:0/2197659024 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81580039f0 0x7f816c0e2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:21.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.008+0000 7f8172800640 1 -- 172.21.15.50:0/2197659024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816c0334e0 msgr2=0x7f816c04c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:21.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.008+0000 7f8172800640 1 --2- 172.21.15.50:0/2197659024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816c0334e0 0x7f816c04c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:21.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.008+0000 7f8172800640 1 -- 172.21.15.50:0/2197659024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c04aae0 con 0x7f816c17f530 2024-09-21T06:07:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.008+0000 7f8172800640 1 --2- 172.21.15.50:0/2197659024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f816c17f530 0x7f816c04cf20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f816c009640 tx=0x7f816000ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:21.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.008+0000 7f816a000640 1 -- 172.21.15.50:0/2197659024 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160014030 con 0x7f816c17f530 2024-09-21T06:07:21.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.012+0000 7f8178a00640 1 -- 172.21.15.50:0/2197659024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c04a520 con 0x7f816c17f530 2024-09-21T06:07:21.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.012+0000 7f8178a00640 1 -- 172.21.15.50:0/2197659024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c04c1e0 con 0x7f816c17f530 2024-09-21T06:07:21.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.012+0000 7f816a000640 1 -- 172.21.15.50:0/2197659024 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f816000f780 con 0x7f816c17f530 2024-09-21T06:07:21.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.012+0000 7f816a000640 1 -- 172.21.15.50:0/2197659024 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816000fbb0 con 0x7f816c17f530 2024-09-21T06:07:21.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.012+0000 7f8178a00640 1 -- 172.21.15.50:0/2197659024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c0052a0 con 0x7f816c17f530 2024-09-21T06:07:21.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.016+0000 7f816a000640 1 -- 172.21.15.50:0/2197659024 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f8160017030 con 0x7f816c17f530 2024-09-21T06:07:21.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.020+0000 7f816a000640 1 --2- 172.21.15.50:0/2197659024 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8154074760 0x7f8154076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:21.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.020+0000 7f816a000640 1 -- 172.21.15.50:0/2197659024 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f81600937b0 con 0x7f816c17f530 2024-09-21T06:07:21.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.024+0000 7f8171e00640 1 --2- 172.21.15.50:0/2197659024 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8154074760 0x7f8154076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:21.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.024+0000 7f8171e00640 1 --2- 172.21.15.50:0/2197659024 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8154074760 0x7f8154076c70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f815800c330 tx=0x7f8174096920 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:21.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.028+0000 7f816a000640 1 -- 172.21.15.50:0/2197659024 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f816001e070 con 0x7f816c17f530 2024-09-21T06:07:21.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:21.552+0000 7f8178a00640 1 -- 172.21.15.50:0/2197659024 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone986813", "target": ["mon-mgr", ""]}) -- 0x7f814c000c90 con 0x7f8154074760 2024-09-21T06:07:22.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.384+0000 7f816a000640 1 -- 172.21.15.50:0/2197659024 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f814c000c90 con 0x7f8154074760 2024-09-21T06:07:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.388+0000 7f8178a00640 1 -- 172.21.15.50:0/2197659024 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8154074760 msgr2=0x7f8154076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.388+0000 7f8178a00640 1 --2- 172.21.15.50:0/2197659024 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8154074760 0x7f8154076c70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f815800c330 tx=0x7f8174096920 comp rx=0 tx=0).stop 2024-09-21T06:07:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.388+0000 7f8178a00640 1 -- 172.21.15.50:0/2197659024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f816c17f530 msgr2=0x7f816c04cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.388+0000 7f8178a00640 1 --2- 172.21.15.50:0/2197659024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f816c17f530 0x7f816c04cf20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f816c009640 tx=0x7f816000ba00 comp rx=0 tx=0).stop 2024-09-21T06:07:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.388+0000 7f8178a00640 1 -- 172.21.15.50:0/2197659024 shutdown_connections 2024-09-21T06:07:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.388+0000 7f8178a00640 1 --2- 172.21.15.50:0/2197659024 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8154074760 0x7f8154076c70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.388+0000 7f8178a00640 1 --2- 172.21.15.50:0/2197659024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f816c17f530 0x7f816c04cf20 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.388+0000 7f8178a00640 1 --2- 172.21.15.50:0/2197659024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816c0334e0 0x7f816c04c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.388+0000 7f8178a00640 1 --2- 172.21.15.50:0/2197659024 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f81580039f0 0x7f816c0e2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.388+0000 7f8178a00640 1 -- 172.21.15.50:0/2197659024 >> 172.21.15.50:0/2197659024 conn(0x7f816c01fbf0 msgr2=0x7f816c051dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.388+0000 7f8178a00640 1 -- 172.21.15.50:0/2197659024 shutdown_connections 2024-09-21T06:07:22.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.388+0000 7f8178a00640 1 -- 172.21.15.50:0/2197659024 wait complete. 2024-09-21T06:07:22.424 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone216314 2024-09-21T06:07:22.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.720+0000 7f2ed1000640 1 -- 172.21.15.50:0/2022546688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ecc0d1140 msgr2=0x7f2ecc0b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.720+0000 7f2ed1000640 1 --2- 172.21.15.50:0/2022546688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ecc0d1140 0x7f2ecc0b0730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009900 tx=0x7f2ebc01c850 comp rx=0 tx=0).stop 2024-09-21T06:07:22.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.724+0000 7f2ed1000640 1 -- 172.21.15.50:0/2022546688 shutdown_connections 2024-09-21T06:07:22.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.724+0000 7f2ed1000640 1 --2- 172.21.15.50:0/2022546688 >> v2:172.21.15.88:3300/0 conn(0x7f2ecc15f5c0 0x7f2ecc15f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.724+0000 7f2ed1000640 1 --2- 172.21.15.50:0/2022546688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ecc0d1140 0x7f2ecc0b0730 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009900 tx=0x7f2ebc01c850 comp rx=0 tx=0).stop 2024-09-21T06:07:22.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.724+0000 7f2ed1000640 1 -- 172.21.15.50:0/2022546688 >> 172.21.15.50:0/2022546688 conn(0x7f2ecc0b41f0 msgr2=0x7f2ecc0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:22.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.724+0000 7f2ed1000640 1 -- 172.21.15.50:0/2022546688 shutdown_connections 2024-09-21T06:07:22.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.724+0000 7f2ed1000640 1 -- 172.21.15.50:0/2022546688 wait complete. 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.728+0000 7f2ed1000640 1 Processor -- start 2024-09-21T06:07:22.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.728+0000 7f2ed1000640 1 -- start start 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ed1000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ecc06dbc0 0x7f2ecc1fd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ed1000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ecc15f5c0 0x7f2ecc157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ed1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ecc06fdd0 0x7f2ecc159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ed1000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc168ff0 con 0x7f2ecc06fdd0 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ed1000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc168e70 con 0x7f2ecc06dbc0 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ed1000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2ecc169170 con 0x7f2ecc15f5c0 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ecbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ecc06fdd0 0x7f2ecc159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ecbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ecc06fdd0 0x7f2ecc159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50570/0 (socket says 172.21.15.50:50570) 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ecbe00640 1 -- 172.21.15.50:0/2971103110 learned_addr learned my addr 172.21.15.50:0/2971103110 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ec2a00640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ecc15f5c0 0x7f2ecc157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ecbe00640 1 -- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ecc15f5c0 msgr2=0x7f2ecc157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ecbe00640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ecc15f5c0 0x7f2ecc157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ecbe00640 1 -- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ecc06dbc0 msgr2=0x7f2ecc1fd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ecbe00640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ecc06dbc0 0x7f2ecc1fd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ecbe00640 1 -- 172.21.15.50:0/2971103110 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ecc0af580 con 0x7f2ecc06fdd0 2024-09-21T06:07:22.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.732+0000 7f2ecbe00640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ecc06fdd0 0x7f2ecc159c30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400e740 tx=0x7f2ec400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:22.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.736+0000 7f2ec2a00640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ecc15f5c0 0x7f2ecc157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:22.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.736+0000 7f2ec3e00640 1 -- 172.21.15.50:0/2971103110 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec400cc60 con 0x7f2ecc06fdd0 2024-09-21T06:07:22.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.736+0000 7f2ec3e00640 1 -- 172.21.15.50:0/2971103110 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ec400ce00 con 0x7f2ecc06fdd0 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.736+0000 7f2ec3e00640 1 -- 172.21.15.50:0/2971103110 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4005280 con 0x7f2ecc06fdd0 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.736+0000 7f2ed1000640 1 -- 172.21.15.50:0/2971103110 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ecc0af860 con 0x7f2ecc06fdd0 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.736+0000 7f2ed1000640 1 -- 172.21.15.50:0/2971103110 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ecc0afe20 con 0x7f2ecc06fdd0 2024-09-21T06:07:22.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.736+0000 7f2ed1000640 1 -- 172.21.15.50:0/2971103110 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea40052a0 con 0x7f2ecc06fdd0 2024-09-21T06:07:22.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.740+0000 7f2ec3e00640 1 -- 172.21.15.50:0/2971103110 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f2ec4005420 con 0x7f2ecc06fdd0 2024-09-21T06:07:22.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.744+0000 7f2ec3e00640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ea8074760 0x7f2ea8076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.744+0000 7f2ec3e00640 1 -- 172.21.15.50:0/2971103110 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f2ec40938a0 con 0x7f2ecc06fdd0 2024-09-21T06:07:22.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.748+0000 7f2ec3e00640 1 -- 172.21.15.50:0/2971103110 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2ec40603a0 con 0x7f2ecc06fdd0 2024-09-21T06:07:22.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.760+0000 7f2ecb400640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ea8074760 0x7f2ea8076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:22.768+0000 7f2ecb400640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ea8074760 0x7f2ea8076c70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0af000 tx=0x7f2ebc0195f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:23.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:23.204+0000 7f2ed1000640 1 -- 172.21.15.50:0/2971103110 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone216314", "target": ["mon-mgr", ""]}) -- 0x7f2ea4000c90 con 0x7f2ea8074760 2024-09-21T06:07:24.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ec3e00640 1 -- 172.21.15.50:0/2971103110 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2ea4000c90 con 0x7f2ea8074760 2024-09-21T06:07:24.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ed1000640 1 -- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ea8074760 msgr2=0x7f2ea8076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ed1000640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ea8074760 0x7f2ea8076c70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0af000 tx=0x7f2ebc0195f0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ed1000640 1 -- 172.21.15.50:0/2971103110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ecc06fdd0 msgr2=0x7f2ecc159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ed1000640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ecc06fdd0 0x7f2ecc159c30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400e740 tx=0x7f2ec400ec00 comp rx=0 tx=0).stop 2024-09-21T06:07:24.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ed1000640 1 -- 172.21.15.50:0/2971103110 shutdown_connections 2024-09-21T06:07:24.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ed1000640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ea8074760 0x7f2ea8076c70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ed1000640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ecc06fdd0 0x7f2ecc159c30 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ed1000640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2ecc15f5c0 0x7f2ecc157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ed1000640 1 --2- 172.21.15.50:0/2971103110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ecc06dbc0 0x7f2ecc1fd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ed1000640 1 -- 172.21.15.50:0/2971103110 >> 172.21.15.50:0/2971103110 conn(0x7f2ecc0b41f0 msgr2=0x7f2ecc0d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:24.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ed1000640 1 -- 172.21.15.50:0/2971103110 shutdown_connections 2024-09-21T06:07:24.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.072+0000 7f2ed1000640 1 -- 172.21.15.50:0/2971103110 wait complete. 2024-09-21T06:07:24.097 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone877615 2024-09-21T06:07:24.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.372+0000 7f9513200640 1 -- 172.21.15.50:0/229381256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95140d1140 msgr2=0x7f95140b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.372+0000 7f9513200640 1 --2- 172.21.15.50:0/229381256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95140d1140 0x7f95140b0730 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9504009930 tx=0x7f950401c670 comp rx=0 tx=0).stop 2024-09-21T06:07:24.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.372+0000 7f9513200640 1 -- 172.21.15.50:0/229381256 shutdown_connections 2024-09-21T06:07:24.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.372+0000 7f9513200640 1 --2- 172.21.15.50:0/229381256 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f951406dbc0 0x7f951406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.372+0000 7f9513200640 1 --2- 172.21.15.50:0/229381256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95140d1140 0x7f95140b0730 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.372+0000 7f9513200640 1 -- 172.21.15.50:0/229381256 >> 172.21.15.50:0/229381256 conn(0x7f95140b41f0 msgr2=0x7f95140b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:24.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.372+0000 7f9513200640 1 -- 172.21.15.50:0/229381256 shutdown_connections 2024-09-21T06:07:24.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.372+0000 7f9513200640 1 -- 172.21.15.50:0/229381256 wait complete. 2024-09-21T06:07:24.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.380+0000 7f9513200640 1 Processor -- start 2024-09-21T06:07:24.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.380+0000 7f9513200640 1 -- start start 2024-09-21T06:07:24.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.384+0000 7f9513200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951406dbc0 0x7f9514158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.384+0000 7f9513200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95140d1140 0x7f95140ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.384+0000 7f9513200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f951415f5c0 0x7f951415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.384+0000 7f9513200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9514166dd0 con 0x7f95140d1140 2024-09-21T06:07:24.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.384+0000 7f9513200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9514166c50 con 0x7f951406dbc0 2024-09-21T06:07:24.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.384+0000 7f9513200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9514166f50 con 0x7f951415f5c0 2024-09-21T06:07:24.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.384+0000 7f9511e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951406dbc0 0x7f9514158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.384+0000 7f9511e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951406dbc0 0x7f9514158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:48712/0 (socket says 172.21.15.50:48712) 2024-09-21T06:07:24.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.384+0000 7f9511e00640 1 -- 172.21.15.50:0/2195828265 learned_addr learned my addr 172.21.15.50:0/2195828265 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:24.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.384+0000 7f9511400640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95140d1140 0x7f95140ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.384+0000 7f9512800640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f951415f5c0 0x7f951415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.388+0000 7f9511e00640 1 -- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f951415f5c0 msgr2=0x7f951415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.388+0000 7f9511e00640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f951415f5c0 0x7f951415a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.388+0000 7f9511e00640 1 -- 172.21.15.50:0/2195828265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95140d1140 msgr2=0x7f95140ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.388+0000 7f9511e00640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95140d1140 0x7f95140ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.388+0000 7f9511e00640 1 -- 172.21.15.50:0/2195828265 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95141554b0 con 0x7f951406dbc0 2024-09-21T06:07:24.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.388+0000 7f9512800640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f951415f5c0 0x7f951415a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:24.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.392+0000 7f9511e00640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951406dbc0 0x7f9514158b30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9504000c10 tx=0x7f95040a8680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:24.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.392+0000 7f950a000640 1 -- 172.21.15.50:0/2195828265 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9504004340 con 0x7f951406dbc0 2024-09-21T06:07:24.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.392+0000 7f950a000640 1 -- 172.21.15.50:0/2195828265 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9504005030 con 0x7f951406dbc0 2024-09-21T06:07:24.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.392+0000 7f950a000640 1 -- 172.21.15.50:0/2195828265 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9504005480 con 0x7f951406dbc0 2024-09-21T06:07:24.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.392+0000 7f9513200640 1 -- 172.21.15.50:0/2195828265 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9514155790 con 0x7f951406dbc0 2024-09-21T06:07:24.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.392+0000 7f9513200640 1 -- 172.21.15.50:0/2195828265 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9514167680 con 0x7f951406dbc0 2024-09-21T06:07:24.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.396+0000 7f9513200640 1 -- 172.21.15.50:0/2195828265 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f951406dfd0 con 0x7f951406dbc0 2024-09-21T06:07:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.404+0000 7f950a000640 1 -- 172.21.15.50:0/2195828265 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f95040046d0 con 0x7f951406dbc0 2024-09-21T06:07:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.404+0000 7f950a000640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f94f4074790 0x7f94f4076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.404+0000 7f950a000640 1 -- 172.21.15.50:0/2195828265 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f950412d880 con 0x7f951406dbc0 2024-09-21T06:07:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.404+0000 7f9511400640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f94f4074790 0x7f94f4076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.404+0000 7f950a000640 1 -- 172.21.15.50:0/2195828265 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f95040c0070 con 0x7f951406dbc0 2024-09-21T06:07:24.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.408+0000 7f9511400640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f94f4074790 0x7f94f4076ca0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f95141539b0 tx=0x7f9500005f70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:24.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:24.772+0000 7f9513200640 1 -- 172.21.15.50:0/2195828265 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone877615", "target": ["mon-mgr", ""]}) -- 0x7f9514155260 con 0x7f94f4074790 2024-09-21T06:07:25.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.608+0000 7f950a000640 1 -- 172.21.15.50:0/2195828265 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9514155260 con 0x7f94f4074790 2024-09-21T06:07:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.620+0000 7f9513200640 1 -- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f94f4074790 msgr2=0x7f94f4076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.620+0000 7f9513200640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f94f4074790 0x7f94f4076ca0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f95141539b0 tx=0x7f9500005f70 comp rx=0 tx=0).stop 2024-09-21T06:07:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.620+0000 7f9513200640 1 -- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951406dbc0 msgr2=0x7f9514158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.620+0000 7f9513200640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951406dbc0 0x7f9514158b30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9504000c10 tx=0x7f95040a8680 comp rx=0 tx=0).stop 2024-09-21T06:07:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.620+0000 7f9513200640 1 -- 172.21.15.50:0/2195828265 shutdown_connections 2024-09-21T06:07:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.620+0000 7f9513200640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f951415f5c0 0x7f951415a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.620+0000 7f9513200640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f94f4074790 0x7f94f4076ca0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.620+0000 7f9513200640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95140d1140 0x7f95140ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.620+0000 7f9513200640 1 --2- 172.21.15.50:0/2195828265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951406dbc0 0x7f9514158b30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.620+0000 7f9513200640 1 -- 172.21.15.50:0/2195828265 >> 172.21.15.50:0/2195828265 conn(0x7f95140b41f0 msgr2=0x7f95140d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.620+0000 7f9513200640 1 -- 172.21.15.50:0/2195828265 shutdown_connections 2024-09-21T06:07:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.620+0000 7f9513200640 1 -- 172.21.15.50:0/2195828265 wait complete. 2024-09-21T06:07:25.667 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone832116 2024-09-21T06:07:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.992+0000 7f64d7000640 1 -- 172.21.15.50:0/655663899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64b4003a10 msgr2=0x7f64b4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.992+0000 7f64d7000640 1 --2- 172.21.15.50:0/655663899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64b4003a10 0x7f64b4003e80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0016f0 tx=0x7f64bc01c6f0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.992+0000 7f64d7000640 1 -- 172.21.15.50:0/655663899 shutdown_connections 2024-09-21T06:07:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.992+0000 7f64d7000640 1 --2- 172.21.15.50:0/655663899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64b4003a10 0x7f64b4003e80 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.992+0000 7f64d7000640 1 -- 172.21.15.50:0/655663899 >> 172.21.15.50:0/655663899 conn(0x7f64d00b3a20 msgr2=0x7f64d00d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.992+0000 7f64d7000640 1 -- 172.21.15.50:0/655663899 shutdown_connections 2024-09-21T06:07:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.992+0000 7f64d7000640 1 -- 172.21.15.50:0/655663899 wait complete. 2024-09-21T06:07:25.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.996+0000 7f64d7000640 1 Processor -- start 2024-09-21T06:07:25.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.996+0000 7f64d7000640 1 -- start start 2024-09-21T06:07:25.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.996+0000 7f64d7000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f64b4003a10 0x7f64d01f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.996+0000 7f64d7000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d0147da0 0x7f64d0149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.996+0000 7f64d7000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64d014a340 0x7f64d015df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.996+0000 7f64d7000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01586a0 con 0x7f64d014a340 2024-09-21T06:07:25.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.996+0000 7f64d7000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01632a0 con 0x7f64d0147da0 2024-09-21T06:07:25.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:25.996+0000 7f64d7000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f64d0158820 con 0x7f64b4003a10 2024-09-21T06:07:25.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64d5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64d014a340 0x7f64d015df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64d5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64d014a340 0x7f64d015df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50608/0 (socket says 172.21.15.50:50608) 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64d5000640 1 -- 172.21.15.50:0/165511324 learned_addr learned my addr 172.21.15.50:0/165511324 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64d5000640 1 -- 172.21.15.50:0/165511324 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f64b4003a10 msgr2=0x7f64d01f9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64d5000640 1 --2- 172.21.15.50:0/165511324 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f64b4003a10 0x7f64d01f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64d5000640 1 -- 172.21.15.50:0/165511324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d0147da0 msgr2=0x7f64d0149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64d5000640 1 --2- 172.21.15.50:0/165511324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d0147da0 0x7f64d0149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64d5000640 1 -- 172.21.15.50:0/165511324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d00cf030 con 0x7f64d014a340 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64d5000640 1 --2- 172.21.15.50:0/165511324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64d014a340 0x7f64d015df50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f64d00b0fc0 tx=0x7f64c800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64c7e00640 1 -- 172.21.15.50:0/165511324 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8017070 con 0x7f64d014a340 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64c7e00640 1 -- 172.21.15.50:0/165511324 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64c8005070 con 0x7f64d014a340 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64c7e00640 1 -- 172.21.15.50:0/165511324 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c80054c0 con 0x7f64d014a340 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64d7000640 1 -- 172.21.15.50:0/165511324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d014ab60 con 0x7f64d014a340 2024-09-21T06:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64d7000640 1 -- 172.21.15.50:0/165511324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d00cf5d0 con 0x7f64d014a340 2024-09-21T06:07:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.000+0000 7f64d7000640 1 -- 172.21.15.50:0/165511324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d0158d80 con 0x7f64d014a340 2024-09-21T06:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.004+0000 7f64c7e00640 1 -- 172.21.15.50:0/165511324 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f64c801a070 con 0x7f64d014a340 2024-09-21T06:07:26.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.008+0000 7f64c7e00640 1 --2- 172.21.15.50:0/165511324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f64b0074760 0x7f64b0076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:26.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.008+0000 7f64c7e00640 1 -- 172.21.15.50:0/165511324 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f64c80929e0 con 0x7f64d014a340 2024-09-21T06:07:26.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.012+0000 7f64cfe00640 1 --2- 172.21.15.50:0/165511324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f64b0074760 0x7f64b0076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.012+0000 7f64c7e00640 1 -- 172.21.15.50:0/165511324 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f64d0069740 con 0x7f64d014a340 2024-09-21T06:07:26.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.044+0000 7f64cfe00640 1 --2- 172.21.15.50:0/165511324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f64b0074760 0x7f64b0076c70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0016c0 tx=0x7f64b4005840 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:26.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.400+0000 7f64d7000640 1 -- 172.21.15.50:0/165511324 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone832116", "target": ["mon-mgr", ""]}) -- 0x7f64d014a8f0 con 0x7f64b0074760 2024-09-21T06:07:26.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.948+0000 7f64c7e00640 1 -- 172.21.15.50:0/165511324 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f64d014a8f0 con 0x7f64b0074760 2024-09-21T06:07:26.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.952+0000 7f64d7000640 1 -- 172.21.15.50:0/165511324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f64b0074760 msgr2=0x7f64b0076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:26.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.952+0000 7f64d7000640 1 --2- 172.21.15.50:0/165511324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f64b0074760 0x7f64b0076c70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0016c0 tx=0x7f64b4005840 comp rx=0 tx=0).stop 2024-09-21T06:07:26.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.952+0000 7f64d7000640 1 -- 172.21.15.50:0/165511324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64d014a340 msgr2=0x7f64d015df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:26.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.952+0000 7f64d7000640 1 --2- 172.21.15.50:0/165511324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64d014a340 0x7f64d015df50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f64d00b0fc0 tx=0x7f64c800cda0 comp rx=0 tx=0).stop 2024-09-21T06:07:26.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.952+0000 7f64d7000640 1 -- 172.21.15.50:0/165511324 shutdown_connections 2024-09-21T06:07:26.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.952+0000 7f64d7000640 1 --2- 172.21.15.50:0/165511324 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f64b0074760 0x7f64b0076c70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:26.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.952+0000 7f64d7000640 1 --2- 172.21.15.50:0/165511324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64d014a340 0x7f64d015df50 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:26.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.952+0000 7f64d7000640 1 --2- 172.21.15.50:0/165511324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d0147da0 0x7f64d0149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:26.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.952+0000 7f64d7000640 1 --2- 172.21.15.50:0/165511324 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f64b4003a10 0x7f64d01f9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:26.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.952+0000 7f64d7000640 1 -- 172.21.15.50:0/165511324 >> 172.21.15.50:0/165511324 conn(0x7f64d00b3a20 msgr2=0x7f64d00704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:26.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.952+0000 7f64d7000640 1 -- 172.21.15.50:0/165511324 shutdown_connections 2024-09-21T06:07:26.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:26.952+0000 7f64d7000640 1 -- 172.21.15.50:0/165511324 wait complete. 2024-09-21T06:07:26.995 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone935917 2024-09-21T06:07:27.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.304+0000 7fd3c3000640 1 -- 172.21.15.50:0/1337623311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3a00039c0 msgr2=0x7fd3a0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.304+0000 7fd3c3000640 1 --2- 172.21.15.50:0/1337623311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3a00039c0 0x7fd3a0003e30 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009f90 tx=0x7fd3b401c990 comp rx=0 tx=0).stop 2024-09-21T06:07:27.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.304+0000 7fd3c1000640 1 -- 172.21.15.50:0/1337623311 reap_dead start 2024-09-21T06:07:27.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.308+0000 7fd3c3000640 1 -- 172.21.15.50:0/1337623311 shutdown_connections 2024-09-21T06:07:27.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.308+0000 7fd3c3000640 1 -- 172.21.15.50:0/1337623311 >> 172.21.15.50:0/1337623311 conn(0x7fd3bc0b2040 msgr2=0x7fd3bc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:27.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.308+0000 7fd3c3000640 1 -- 172.21.15.50:0/1337623311 shutdown_connections 2024-09-21T06:07:27.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.308+0000 7fd3c3000640 1 -- 172.21.15.50:0/1337623311 wait complete. 2024-09-21T06:07:27.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.308+0000 7fd3c3000640 1 Processor -- start 2024-09-21T06:07:27.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.308+0000 7fd3c3000640 1 -- start start 2024-09-21T06:07:27.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3c3000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd3a00039c0 0x7fd3bc13dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3c3000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3bc167b60 0x7fd3bc157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3c3000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc1584a0 0x7fd3bc163e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3c3000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc165c00 con 0x7fd3bc167b60 2024-09-21T06:07:27.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3c3000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc165a80 con 0x7fd3bc1584a0 2024-09-21T06:07:27.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3c3000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd3bc165d80 con 0x7fd3a00039c0 2024-09-21T06:07:27.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3bb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3bc167b60 0x7fd3bc157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3bb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3bc167b60 0x7fd3bc157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57724/0 (socket says 172.21.15.50:57724) 2024-09-21T06:07:27.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3bb400640 1 -- 172.21.15.50:0/3472322504 learned_addr learned my addr 172.21.15.50:0/3472322504 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:27.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3bb400640 1 -- 172.21.15.50:0/3472322504 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd3a00039c0 msgr2=0x7fd3bc13dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3bb400640 1 --2- 172.21.15.50:0/3472322504 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd3a00039c0 0x7fd3bc13dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3bb400640 1 -- 172.21.15.50:0/3472322504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc1584a0 msgr2=0x7fd3bc163e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3bb400640 1 --2- 172.21.15.50:0/3472322504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc1584a0 0x7fd3bc163e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3bb400640 1 -- 172.21.15.50:0/3472322504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3bc13e0f0 con 0x7fd3bc167b60 2024-09-21T06:07:27.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3bb400640 1 --2- 172.21.15.50:0/3472322504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3bc167b60 0x7fd3bc157f60 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00c730 tx=0x7fd3ac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:27.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3b3e00640 1 -- 172.21.15.50:0/3472322504 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac017070 con 0x7fd3bc167b60 2024-09-21T06:07:27.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3c3000640 1 -- 172.21.15.50:0/3472322504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3bc1fc1b0 con 0x7fd3bc167b60 2024-09-21T06:07:27.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3b3e00640 1 -- 172.21.15.50:0/3472322504 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3ac00ce40 con 0x7fd3bc167b60 2024-09-21T06:07:27.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3b3e00640 1 -- 172.21.15.50:0/3472322504 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac003860 con 0x7fd3bc167b60 2024-09-21T06:07:27.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.312+0000 7fd3c3000640 1 -- 172.21.15.50:0/3472322504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3bc1fc770 con 0x7fd3bc167b60 2024-09-21T06:07:27.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.316+0000 7fd3c3000640 1 -- 172.21.15.50:0/3472322504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3bc1588b0 con 0x7fd3bc167b60 2024-09-21T06:07:27.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.316+0000 7fd3b3e00640 1 -- 172.21.15.50:0/3472322504 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7fd3ac01a070 con 0x7fd3bc167b60 2024-09-21T06:07:27.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.320+0000 7fd3b3e00640 1 --2- 172.21.15.50:0/3472322504 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd39c074760 0x7fd39c076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.320+0000 7fd3b3e00640 1 -- 172.21.15.50:0/3472322504 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7fd3ac0929e0 con 0x7fd3bc167b60 2024-09-21T06:07:27.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.320+0000 7fd3bbe00640 1 --2- 172.21.15.50:0/3472322504 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd39c074760 0x7fd39c076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.328+0000 7fd3b3e00640 1 -- 172.21.15.50:0/3472322504 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd3bc1588b0 con 0x7fd3bc167b60 2024-09-21T06:07:27.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.332+0000 7fd3bbe00640 1 --2- 172.21.15.50:0/3472322504 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd39c074760 0x7fd39c076c70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40af000 tx=0x7fd3a0005840 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:27.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:27.684+0000 7fd3c3000640 1 -- 172.21.15.50:0/3472322504 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone935917", "target": ["mon-mgr", ""]}) -- 0x7fd3bc06b540 con 0x7fd39c074760 2024-09-21T06:07:28.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.144+0000 7fd3b3e00640 1 -- 172.21.15.50:0/3472322504 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd3b4003a80 con 0x7fd39c074760 2024-09-21T06:07:28.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.148+0000 7fd3c3000640 1 -- 172.21.15.50:0/3472322504 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd39c074760 msgr2=0x7fd39c076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:28.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.148+0000 7fd3c3000640 1 --2- 172.21.15.50:0/3472322504 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd39c074760 0x7fd39c076c70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40af000 tx=0x7fd3a0005840 comp rx=0 tx=0).stop 2024-09-21T06:07:28.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.148+0000 7fd3c3000640 1 -- 172.21.15.50:0/3472322504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3bc167b60 msgr2=0x7fd3bc157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:28.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.148+0000 7fd3c3000640 1 --2- 172.21.15.50:0/3472322504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3bc167b60 0x7fd3bc157f60 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00c730 tx=0x7fd3ac00cbf0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.148+0000 7fd3c3000640 1 -- 172.21.15.50:0/3472322504 shutdown_connections 2024-09-21T06:07:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.148+0000 7fd3c3000640 1 --2- 172.21.15.50:0/3472322504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc1584a0 0x7fd3bc163e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.148+0000 7fd3c3000640 1 --2- 172.21.15.50:0/3472322504 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd39c074760 0x7fd39c076c70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.148+0000 7fd3c3000640 1 --2- 172.21.15.50:0/3472322504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3bc167b60 0x7fd3bc157f60 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.148+0000 7fd3c3000640 1 --2- 172.21.15.50:0/3472322504 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd3a00039c0 0x7fd3bc13dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.148+0000 7fd3c3000640 1 -- 172.21.15.50:0/3472322504 >> 172.21.15.50:0/3472322504 conn(0x7fd3bc0b2040 msgr2=0x7fd3bc167240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.148+0000 7fd3c3000640 1 -- 172.21.15.50:0/3472322504 shutdown_connections 2024-09-21T06:07:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.148+0000 7fd3c3000640 1 -- 172.21.15.50:0/3472322504 wait complete. 2024-09-21T06:07:28.190 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone972718 2024-09-21T06:07:28.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.500+0000 7f18f0c00640 1 -- 172.21.15.50:0/2054953431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18ec155ad0 msgr2=0x7f18ec06e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:28.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.500+0000 7f18f0c00640 1 --2- 172.21.15.50:0/2054953431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18ec155ad0 0x7f18ec06e520 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0b61e0 tx=0x7f18dc01c790 comp rx=0 tx=0).stop 2024-09-21T06:07:28.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.500+0000 7f18f0c00640 1 -- 172.21.15.50:0/2054953431 shutdown_connections 2024-09-21T06:07:28.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.500+0000 7f18f0c00640 1 --2- 172.21.15.50:0/2054953431 >> v2:172.21.15.88:3300/0 conn(0x7f18ec0d29c0 0x7f18ec0d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.500+0000 7f18f0c00640 1 --2- 172.21.15.50:0/2054953431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18ec155ad0 0x7f18ec06e520 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.500+0000 7f18f0c00640 1 -- 172.21.15.50:0/2054953431 >> 172.21.15.50:0/2054953431 conn(0x7f18ec0708c0 msgr2=0x7f18ec070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:28.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.500+0000 7f18f0c00640 1 -- 172.21.15.50:0/2054953431 shutdown_connections 2024-09-21T06:07:28.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.500+0000 7f18f0c00640 1 -- 172.21.15.50:0/2054953431 wait complete. 2024-09-21T06:07:28.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.504+0000 7f18f0c00640 1 Processor -- start 2024-09-21T06:07:28.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.504+0000 7f18f0c00640 1 -- start start 2024-09-21T06:07:28.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.508+0000 7f18f0c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18ec074410 0x7f18ec158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:28.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.508+0000 7f18f0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18ec0d29c0 0x7f18ec13d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:28.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.508+0000 7f18f0c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18ec155ad0 0x7f18ec15dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:28.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.508+0000 7f18f0c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec16c550 con 0x7f18ec0d29c0 2024-09-21T06:07:28.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.508+0000 7f18f0c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec16c3d0 con 0x7f18ec155ad0 2024-09-21T06:07:28.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.508+0000 7f18f0c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f18ec16c6d0 con 0x7f18ec074410 2024-09-21T06:07:28.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.512+0000 7f18ea800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18ec155ad0 0x7f18ec15dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:28.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.512+0000 7f18e9e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18ec074410 0x7f18ec158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:28.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.512+0000 7f18e9e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18ec074410 0x7f18ec158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:55922/0 (socket says 172.21.15.50:55922) 2024-09-21T06:07:28.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.512+0000 7f18e9e00640 1 -- 172.21.15.50:0/4278784179 learned_addr learned my addr 172.21.15.50:0/4278784179 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:28.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.512+0000 7f18e3e00640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18ec0d29c0 0x7f18ec13d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:28.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.512+0000 7f18e9e00640 1 -- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18ec155ad0 msgr2=0x7f18ec15dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:28.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.512+0000 7f18e9e00640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18ec155ad0 0x7f18ec15dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.512+0000 7f18e9e00640 1 -- 172.21.15.50:0/4278784179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18ec0d29c0 msgr2=0x7f18ec13d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:28.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.512+0000 7f18e9e00640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18ec0d29c0 0x7f18ec13d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.516+0000 7f18e9e00640 1 -- 172.21.15.50:0/4278784179 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec157ce0 con 0x7f18ec074410 2024-09-21T06:07:28.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.516+0000 7f18ea800640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18ec155ad0 0x7f18ec15dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:28.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.520+0000 7f18e9e00640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18ec074410 0x7f18ec158d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000c10 tx=0x7f18dc005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:28.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.520+0000 7f18e2000640 1 -- 172.21.15.50:0/4278784179 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc0b2040 con 0x7f18ec074410 2024-09-21T06:07:28.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.520+0000 7f18e2000640 1 -- 172.21.15.50:0/4278784179 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18dc0042f0 con 0x7f18ec074410 2024-09-21T06:07:28.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.520+0000 7f18e2000640 1 -- 172.21.15.50:0/4278784179 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc005270 con 0x7f18ec074410 2024-09-21T06:07:28.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.524+0000 7f18f0c00640 1 -- 172.21.15.50:0/4278784179 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec157fc0 con 0x7f18ec074410 2024-09-21T06:07:28.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.524+0000 7f18f0c00640 1 -- 172.21.15.50:0/4278784179 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec158580 con 0x7f18ec074410 2024-09-21T06:07:28.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.524+0000 7f18db400640 1 -- 172.21.15.50:0/4278784179 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18bc0052a0 con 0x7f18ec074410 2024-09-21T06:07:28.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.528+0000 7f18e2000640 1 -- 172.21.15.50:0/4278784179 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f18dc005410 con 0x7f18ec074410 2024-09-21T06:07:28.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.532+0000 7f18e2000640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18c8074760 0x7f18c8076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:28.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.532+0000 7f18e2000640 1 -- 172.21.15.50:0/4278784179 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f18dc12d410 con 0x7f18ec074410 2024-09-21T06:07:28.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.532+0000 7f18e3e00640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18c8074760 0x7f18c8076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:28.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.532+0000 7f18e3e00640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18c8074760 0x7f18c8076c70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0d1bc0 tx=0x7f18d4009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:28.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.544+0000 7f18e2000640 1 -- 172.21.15.50:0/4278784179 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f18dc016410 con 0x7f18ec074410 2024-09-21T06:07:28.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:28.940+0000 7f18db400640 1 -- 172.21.15.50:0/4278784179 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone972718", "target": ["mon-mgr", ""]}) -- 0x7f18bc000c90 con 0x7f18c8074760 2024-09-21T06:07:29.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.888+0000 7f18e2000640 1 -- 172.21.15.50:0/4278784179 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f18bc000c90 con 0x7f18c8074760 2024-09-21T06:07:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.892+0000 7f18db400640 1 -- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18c8074760 msgr2=0x7f18c8076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.892+0000 7f18db400640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18c8074760 0x7f18c8076c70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0d1bc0 tx=0x7f18d4009060 comp rx=0 tx=0).stop 2024-09-21T06:07:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.892+0000 7f18db400640 1 -- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18ec074410 msgr2=0x7f18ec158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.892+0000 7f18db400640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18ec074410 0x7f18ec158d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000c10 tx=0x7f18dc005dd0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.892+0000 7f18db400640 1 -- 172.21.15.50:0/4278784179 shutdown_connections 2024-09-21T06:07:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.892+0000 7f18db400640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18ec155ad0 0x7f18ec15dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.892+0000 7f18db400640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18c8074760 0x7f18c8076c70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.892+0000 7f18db400640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18ec0d29c0 0x7f18ec13d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.892+0000 7f18db400640 1 --2- 172.21.15.50:0/4278784179 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18ec074410 0x7f18ec158d80 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000c10 tx=0x7f18dc005dd0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.892+0000 7f18db400640 1 -- 172.21.15.50:0/4278784179 >> 172.21.15.50:0/4278784179 conn(0x7f18ec0708c0 msgr2=0x7f18ec14e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.892+0000 7f18db400640 1 -- 172.21.15.50:0/4278784179 shutdown_connections 2024-09-21T06:07:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:29.892+0000 7f18db400640 1 -- 172.21.15.50:0/4278784179 wait complete. 2024-09-21T06:07:29.943 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone564119 2024-09-21T06:07:30.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.264+0000 7f7ff6800640 1 -- 172.21.15.50:0/2838896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fd4004810 msgr2=0x7f7fd4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:30.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.264+0000 7f7ff6800640 1 --2- 172.21.15.50:0/2838896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fd4004810 0x7f7fd4004c20 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00c310 tx=0x7f7fdc01cfa0 comp rx=0 tx=0).stop 2024-09-21T06:07:30.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.264+0000 7f7ff6800640 1 -- 172.21.15.50:0/2838896 shutdown_connections 2024-09-21T06:07:30.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.264+0000 7f7ff6800640 1 --2- 172.21.15.50:0/2838896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fd4004810 0x7f7fd4004c20 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:30.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.264+0000 7f7ff6800640 1 -- 172.21.15.50:0/2838896 >> 172.21.15.50:0/2838896 conn(0x7f7ff00708c0 msgr2=0x7f7ff0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:30.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.264+0000 7f7ff6800640 1 -- 172.21.15.50:0/2838896 shutdown_connections 2024-09-21T06:07:30.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.264+0000 7f7ff6800640 1 -- 172.21.15.50:0/2838896 wait complete. 2024-09-21T06:07:30.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.268+0000 7f7ff6800640 1 Processor -- start 2024-09-21T06:07:30.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.268+0000 7f7ff6800640 1 -- start start 2024-09-21T06:07:30.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7ff6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fd4004810 0x7f7ff014d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:30.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7ff6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ff014bf60 0x7f7ff00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:30.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7ff6800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ff00b2660 0x7f7ff013ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:30.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7ff6800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff015d3d0 con 0x7f7ff014bf60 2024-09-21T06:07:30.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7ff6800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff015d250 con 0x7f7fd4004810 2024-09-21T06:07:30.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7ff6800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff015d550 con 0x7f7ff00b2660 2024-09-21T06:07:30.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7feea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ff014bf60 0x7f7ff00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:30.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7feea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ff014bf60 0x7f7ff00b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57760/0 (socket says 172.21.15.50:57760) 2024-09-21T06:07:30.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7feea00640 1 -- 172.21.15.50:0/2113801061 learned_addr learned my addr 172.21.15.50:0/2113801061 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:30.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7feea00640 1 -- 172.21.15.50:0/2113801061 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ff00b2660 msgr2=0x7f7ff013ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:30.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7feea00640 1 --2- 172.21.15.50:0/2113801061 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ff00b2660 0x7f7ff013ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:30.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7feea00640 1 -- 172.21.15.50:0/2113801061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fd4004810 msgr2=0x7f7ff014d7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:07:30.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7feea00640 1 --2- 172.21.15.50:0/2113801061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fd4004810 0x7f7ff014d7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:30.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7feea00640 1 -- 172.21.15.50:0/2113801061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff014c700 con 0x7f7ff014bf60 2024-09-21T06:07:30.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7feea00640 1 --2- 172.21.15.50:0/2113801061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ff014bf60 0x7f7ff00b2120 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00c310 tx=0x7f7fdc001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:30.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7fe7400640 1 -- 172.21.15.50:0/2113801061 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc0a5cb0 con 0x7f7ff014bf60 2024-09-21T06:07:30.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7fe7400640 1 -- 172.21.15.50:0/2113801061 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fdc001500 con 0x7f7ff014bf60 2024-09-21T06:07:30.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7fe7400640 1 -- 172.21.15.50:0/2113801061 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc006330 con 0x7f7ff014bf60 2024-09-21T06:07:30.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7ff6800640 1 -- 172.21.15.50:0/2113801061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff014c9e0 con 0x7f7ff014bf60 2024-09-21T06:07:30.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.272+0000 7f7ff6800640 1 -- 172.21.15.50:0/2113801061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff014cfa0 con 0x7f7ff014bf60 2024-09-21T06:07:30.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.276+0000 7f7ff6800640 1 -- 172.21.15.50:0/2113801061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff0071c00 con 0x7f7ff014bf60 2024-09-21T06:07:30.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.276+0000 7f7fe7400640 1 -- 172.21.15.50:0/2113801061 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f7fdc0b7030 con 0x7f7ff014bf60 2024-09-21T06:07:30.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.280+0000 7f7fe7400640 1 --2- 172.21.15.50:0/2113801061 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7fd0074760 0x7f7fd0076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:30.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.280+0000 7f7fe7400640 1 -- 172.21.15.50:0/2113801061 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f7fdc12cda0 con 0x7f7ff014bf60 2024-09-21T06:07:30.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.284+0000 7f7fe7400640 1 -- 172.21.15.50:0/2113801061 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7fdc016d50 con 0x7f7ff014bf60 2024-09-21T06:07:30.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.284+0000 7f7fef400640 1 --2- 172.21.15.50:0/2113801061 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7fd0074760 0x7f7fd0076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:30.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.292+0000 7f7fef400640 1 --2- 172.21.15.50:0/2113801061 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7fd0074760 0x7f7fd0076c70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0007fd0 tx=0x7f7fe00079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:30.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:30.700+0000 7f7ff6800640 1 -- 172.21.15.50:0/2113801061 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone564119", "target": ["mon-mgr", ""]}) -- 0x7f7ff006b540 con 0x7f7fd0074760 2024-09-21T06:07:32.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.424+0000 7f7fe7400640 1 -- 172.21.15.50:0/2113801061 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7fe0014400 con 0x7f7fd0074760 2024-09-21T06:07:32.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.428+0000 7f7ff6800640 1 -- 172.21.15.50:0/2113801061 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7fd0074760 msgr2=0x7f7fd0076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.428+0000 7f7ff6800640 1 --2- 172.21.15.50:0/2113801061 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7fd0074760 0x7f7fd0076c70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0007fd0 tx=0x7f7fe00079c0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.428+0000 7f7ff6800640 1 -- 172.21.15.50:0/2113801061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ff014bf60 msgr2=0x7f7ff00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.428+0000 7f7ff6800640 1 --2- 172.21.15.50:0/2113801061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ff014bf60 0x7f7ff00b2120 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00c310 tx=0x7f7fdc001d70 comp rx=0 tx=0).stop 2024-09-21T06:07:32.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.428+0000 7f7ff6800640 1 -- 172.21.15.50:0/2113801061 shutdown_connections 2024-09-21T06:07:32.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.428+0000 7f7ff6800640 1 --2- 172.21.15.50:0/2113801061 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7ff00b2660 0x7f7ff013ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.428+0000 7f7ff6800640 1 --2- 172.21.15.50:0/2113801061 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7fd0074760 0x7f7fd0076c70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.428+0000 7f7ff6800640 1 --2- 172.21.15.50:0/2113801061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ff014bf60 0x7f7ff00b2120 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.428+0000 7f7ff6800640 1 --2- 172.21.15.50:0/2113801061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fd4004810 0x7f7ff014d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.428+0000 7f7ff6800640 1 -- 172.21.15.50:0/2113801061 >> 172.21.15.50:0/2113801061 conn(0x7f7ff00708c0 msgr2=0x7f7ff015f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:32.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.428+0000 7f7ff6800640 1 -- 172.21.15.50:0/2113801061 shutdown_connections 2024-09-21T06:07:32.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:32.428+0000 7f7ff6800640 1 -- 172.21.15.50:0/2113801061 wait complete. 2024-09-21T06:07:34.471 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:07:34.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.796+0000 7f07ec800640 1 -- 172.21.15.50:0/1044533955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07e40cdbb0 msgr2=0x7f07e415ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:34.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.796+0000 7f07ec800640 1 --2- 172.21.15.50:0/1044533955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07e40cdbb0 0x7f07e415ebf0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f07e000afd0 tx=0x7f07e001d8f0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.800+0000 7f07ec800640 1 -- 172.21.15.50:0/1044533955 shutdown_connections 2024-09-21T06:07:34.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.800+0000 7f07ec800640 1 --2- 172.21.15.50:0/1044533955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07e40cdbb0 0x7f07e415ebf0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.800+0000 7f07ec800640 1 --2- 172.21.15.50:0/1044533955 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07e415aa50 0x7f07e40cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.800+0000 7f07ec800640 1 -- 172.21.15.50:0/1044533955 >> 172.21.15.50:0/1044533955 conn(0x7f07e40b2040 msgr2=0x7f07e40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:34.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.800+0000 7f07ec800640 1 -- 172.21.15.50:0/1044533955 shutdown_connections 2024-09-21T06:07:34.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.800+0000 7f07ec800640 1 -- 172.21.15.50:0/1044533955 wait complete. 2024-09-21T06:07:34.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.804+0000 7f07ec800640 1 Processor -- start 2024-09-21T06:07:34.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.804+0000 7f07ec800640 1 -- start start 2024-09-21T06:07:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07ec800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07e40cd7a0 0x7f07e4156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07ec800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07e40cdbb0 0x7f07e4150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07ec800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07e415aa50 0x7f07e4158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07ec800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f07e4168000 con 0x7f07e40cdbb0 2024-09-21T06:07:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07ec800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f07e4167e80 con 0x7f07e415aa50 2024-09-21T06:07:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07ec800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f07e4168180 con 0x7f07e40cd7a0 2024-09-21T06:07:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07e9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07e40cdbb0 0x7f07e4150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07e9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07e40cdbb0 0x7f07e4150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57782/0 (socket says 172.21.15.50:57782) 2024-09-21T06:07:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07e9400640 1 -- 172.21.15.50:0/118915023 learned_addr learned my addr 172.21.15.50:0/118915023 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07e9e00640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07e40cd7a0 0x7f07e4156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07ea800640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07e415aa50 0x7f07e4158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07e9400640 1 -- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07e40cd7a0 msgr2=0x7f07e4156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07e9400640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07e40cd7a0 0x7f07e4156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07e9400640 1 -- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07e415aa50 msgr2=0x7f07e4158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07e9400640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07e415aa50 0x7f07e4158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07e9400640 1 -- 172.21.15.50:0/118915023 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e4151540 con 0x7f07e40cdbb0 2024-09-21T06:07:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07ea800640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07e415aa50 0x7f07e4158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07e9400640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07e40cdbb0 0x7f07e4150880 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f07d800b410 tx=0x7f07d800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07de000640 1 -- 172.21.15.50:0/118915023 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8015020 con 0x7f07e40cdbb0 2024-09-21T06:07:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07de000640 1 -- 172.21.15.50:0/118915023 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07d8011920 con 0x7f07e40cdbb0 2024-09-21T06:07:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07de000640 1 -- 172.21.15.50:0/118915023 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8011d70 con 0x7f07e40cdbb0 2024-09-21T06:07:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.808+0000 7f07ec800640 1 -- 172.21.15.50:0/118915023 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e4159470 con 0x7f07e40cdbb0 2024-09-21T06:07:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.812+0000 7f07ec800640 1 -- 172.21.15.50:0/118915023 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e4159a30 con 0x7f07e40cdbb0 2024-09-21T06:07:34.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.812+0000 7f07ec800640 1 -- 172.21.15.50:0/118915023 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e41591a0 con 0x7f07e40cdbb0 2024-09-21T06:07:34.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.812+0000 7f07de000640 1 -- 172.21.15.50:0/118915023 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f07d8018020 con 0x7f07e40cdbb0 2024-09-21T06:07:34.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.816+0000 7f07de000640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f07c8074760 0x7f07c8076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.816+0000 7f07de000640 1 -- 172.21.15.50:0/118915023 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f07d8093300 con 0x7f07e40cdbb0 2024-09-21T06:07:34.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.816+0000 7f07e9e00640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f07c8074760 0x7f07c8076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.824+0000 7f07e9e00640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f07c8074760 0x7f07c8076c70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f07d4006fd0 tx=0x7f07d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:34.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:34.828+0000 7f07de000640 1 -- 172.21.15.50:0/118915023 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f07d801f020 con 0x7f07e40cdbb0 2024-09-21T06:07:35.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.264+0000 7f07ec800640 1 -- 172.21.15.50:0/118915023 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f07e4157a90 con 0x7f07e40cdbb0 2024-09-21T06:07:35.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.264+0000 7f07de000640 1 -- 172.21.15.50:0/118915023 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2443 (secure 0 0 0) 0x7f07d805f970 con 0x7f07e40cdbb0 2024-09-21T06:07:35.265 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:07:35.266 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:07:35.266 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:07:35.266 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:07:35.266 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:07:35.266 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:07:35.266 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:07:35.266 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:35.266 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:07:35.266 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:07:35.266 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:07:35.266 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:07:35.266 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 691, 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 87, 2024-09-21T06:07:35.267 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:35.268 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 7482, 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 31171115815, 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 62121840640, 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 710972272640, 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 838814872, 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 851126197, 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 202, 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 227 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 56, 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T06:06:14:518361+0000", 2024-09-21T06:07:35.269 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 6, 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 6, 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "c", 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 7029 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:35.270 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 18, 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:07:15.102988+0000", 2024-09-21T06:07:35.271 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": { 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr-vol-ongoing-clones": { 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: "message": "4 ongoing clones - average progress is 46.81% (33s)\n [=============...............] (remaining: 38s)", 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: "progress": 0.46810001134872437, 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: "add_to_ceph_s": true 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr-vol-total-clones": { 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: "message": "Total 11 clones - average progress is 32.694% (24s)\n [=========...................] (remaining: 49s)", 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: "progress": 0.32694000005722046, 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: "add_to_ceph_s": true 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:07:35.272 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:07:35.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.268+0000 7f07ec800640 1 -- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f07c8074760 msgr2=0x7f07c8076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:35.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.268+0000 7f07ec800640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f07c8074760 0x7f07c8076c70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f07d4006fd0 tx=0x7f07d4008040 comp rx=0 tx=0).stop 2024-09-21T06:07:35.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.268+0000 7f07ec800640 1 -- 172.21.15.50:0/118915023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07e40cdbb0 msgr2=0x7f07e4150880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:35.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.268+0000 7f07ec800640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07e40cdbb0 0x7f07e4150880 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f07d800b410 tx=0x7f07d800b8d0 comp rx=0 tx=0).stop 2024-09-21T06:07:35.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.268+0000 7f07ec800640 1 -- 172.21.15.50:0/118915023 shutdown_connections 2024-09-21T06:07:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.268+0000 7f07ec800640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07e415aa50 0x7f07e4158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.268+0000 7f07ec800640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f07c8074760 0x7f07c8076c70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.268+0000 7f07ec800640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07e40cdbb0 0x7f07e4150880 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.268+0000 7f07ec800640 1 --2- 172.21.15.50:0/118915023 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07e40cd7a0 0x7f07e4156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.268+0000 7f07ec800640 1 -- 172.21.15.50:0/118915023 >> 172.21.15.50:0/118915023 conn(0x7f07e40b2040 msgr2=0x7f07e406e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.268+0000 7f07ec800640 1 -- 172.21.15.50:0/118915023 shutdown_connections 2024-09-21T06:07:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.268+0000 7f07ec800640 1 -- 172.21.15.50:0/118915023 wait complete. 2024-09-21T06:07:35.303 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 46.81% (33s)\n [=============...............] (remaining: 38s)', 'progress': 0.46810001134872437, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 11 clones - average progress is 32.694% (24s)\n [=========...................] (remaining: 49s)', 'progress': 0.32694000005722046, 'add_to_ceph_s': True}} 2024-09-21T06:07:35.303 INFO:tasks.cephfs.test_volumes:new_onpen_p is less than onpen_p. 2024-09-21T06:07:35.303 INFO:tasks.cephfs.test_volumes:new_onpen_p = 0.32694000005722046; onpen_p = 0.4312500059604645 2024-09-21T06:07:35.303 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81850 2024-09-21T06:07:35.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.592+0000 7f50c7200640 1 -- 172.21.15.50:0/2589335972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50c006f840 msgr2=0x7f50c0169590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:35.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.592+0000 7f50c7200640 1 --2- 172.21.15.50:0/2589335972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50c006f840 0x7f50c0169590 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009bc0 tx=0x7f50bc01cab0 comp rx=0 tx=0).stop 2024-09-21T06:07:35.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.592+0000 7f50c7200640 1 -- 172.21.15.50:0/2589335972 shutdown_connections 2024-09-21T06:07:35.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.592+0000 7f50c7200640 1 --2- 172.21.15.50:0/2589335972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50c006f840 0x7f50c0169590 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:35.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.592+0000 7f50c7200640 1 -- 172.21.15.50:0/2589335972 >> 172.21.15.50:0/2589335972 conn(0x7f50c006d3f0 msgr2=0x7f50c00b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:35.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.592+0000 7f50c7200640 1 -- 172.21.15.50:0/2589335972 shutdown_connections 2024-09-21T06:07:35.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.592+0000 7f50c7200640 1 -- 172.21.15.50:0/2589335972 wait complete. 2024-09-21T06:07:35.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.596+0000 7f50c7200640 1 Processor -- start 2024-09-21T06:07:35.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.596+0000 7f50c7200640 1 -- start start 2024-09-21T06:07:35.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c7200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50c006f840 0x7f50c00b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:35.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50c00702e0 0x7f50c01fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:35.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50c00ae8e0 0x7f50c01514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:35.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c7200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f50c016cfb0 con 0x7f50c00ae8e0 2024-09-21T06:07:35.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f50c016ce30 con 0x7f50c00702e0 2024-09-21T06:07:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c7200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f50c016d130 con 0x7f50c006f840 2024-09-21T06:07:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50c00ae8e0 0x7f50c01514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50c00ae8e0 0x7f50c01514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57798/0 (socket says 172.21.15.50:57798) 2024-09-21T06:07:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c6800640 1 -- 172.21.15.50:0/2421145158 learned_addr learned my addr 172.21.15.50:0/2421145158 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:35.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c5400640 1 --2- 172.21.15.50:0/2421145158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50c00702e0 0x7f50c01fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:35.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c6800640 1 -- 172.21.15.50:0/2421145158 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50c006f840 msgr2=0x7f50c00b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:35.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c6800640 1 --2- 172.21.15.50:0/2421145158 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50c006f840 0x7f50c00b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:35.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c6800640 1 -- 172.21.15.50:0/2421145158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50c00702e0 msgr2=0x7f50c01fd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:35.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c6800640 1 --2- 172.21.15.50:0/2421145158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50c00702e0 0x7f50c01fd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:35.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c6800640 1 -- 172.21.15.50:0/2421145158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c00ce390 con 0x7f50c00ae8e0 2024-09-21T06:07:35.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c5400640 1 --2- 172.21.15.50:0/2421145158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50c00702e0 0x7f50c01fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:35.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c6800640 1 --2- 172.21.15.50:0/2421145158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50c00ae8e0 0x7f50c01514b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0b0040 tx=0x7f50bc005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:35.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50ba000640 1 -- 172.21.15.50:0/2421145158 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0b2040 con 0x7f50c00ae8e0 2024-09-21T06:07:35.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50ba000640 1 -- 172.21.15.50:0/2421145158 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50bc004040 con 0x7f50c00ae8e0 2024-09-21T06:07:35.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50ba000640 1 -- 172.21.15.50:0/2421145158 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc004470 con 0x7f50c00ae8e0 2024-09-21T06:07:35.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.600+0000 7f50c7200640 1 -- 172.21.15.50:0/2421145158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c00cd5d0 con 0x7f50c00ae8e0 2024-09-21T06:07:35.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.604+0000 7f50c7200640 1 -- 172.21.15.50:0/2421145158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c00cdb90 con 0x7f50c00ae8e0 2024-09-21T06:07:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.604+0000 7f50c7200640 1 -- 172.21.15.50:0/2421145158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c00afc30 con 0x7f50c00ae8e0 2024-09-21T06:07:35.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.608+0000 7f50ba000640 1 -- 172.21.15.50:0/2421145158 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f50bc0bd030 con 0x7f50c00ae8e0 2024-09-21T06:07:35.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.608+0000 7f50ba000640 1 --2- 172.21.15.50:0/2421145158 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50a4074790 0x7f50a4076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:35.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.612+0000 7f50ba000640 1 -- 172.21.15.50:0/2421145158 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f50bc12d5e0 con 0x7f50c00ae8e0 2024-09-21T06:07:35.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.612+0000 7f50c5e00640 1 --2- 172.21.15.50:0/2421145158 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50a4074790 0x7f50a4076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:35.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.616+0000 7f50ba000640 1 -- 172.21.15.50:0/2421145158 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f50bc0c4060 con 0x7f50c00ae8e0 2024-09-21T06:07:35.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.620+0000 7f50c5e00640 1 --2- 172.21.15.50:0/2421145158 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50a4074790 0x7f50a4076ca0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f50b40046c0 tx=0x7f50b4007910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:35.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:35.980+0000 7f50c7200640 1 -- 172.21.15.50:0/2421145158 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81850", "target": ["mon-mgr", ""]}) -- 0x7f50c00cd380 con 0x7f50a4074790 2024-09-21T06:07:36.742 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:36.744+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:07:36.743 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:36.744+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:07:36.743 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:36.744+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:07:36.743 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:36.744+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:07:36.743 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:36.744+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:07:37.449 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:37.448+0000 7f65a0c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:37.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.468+0000 7f50ba000640 1 -- 172.21.15.50:0/2421145158 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f50c00cd380 con 0x7f50a4074790 2024-09-21T06:07:37.469 INFO:teuthology.orchestra.run.smithi050.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:37.482 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T06:07:37.483 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone80951 2024-09-21T06:07:37.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.796+0000 7fa7e1800640 1 -- 172.21.15.50:0/1565965163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c0004810 msgr2=0x7fa7c0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:37.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.796+0000 7fa7e1800640 1 --2- 172.21.15.50:0/1565965163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c0004810 0x7fa7c0004c20 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009940 tx=0x7fa7c401c720 comp rx=0 tx=0).stop 2024-09-21T06:07:37.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.796+0000 7fa7e1800640 1 -- 172.21.15.50:0/1565965163 shutdown_connections 2024-09-21T06:07:37.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.796+0000 7fa7e1800640 1 --2- 172.21.15.50:0/1565965163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c0004810 0x7fa7c0004c20 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.796+0000 7fa7e1800640 1 -- 172.21.15.50:0/1565965163 >> 172.21.15.50:0/1565965163 conn(0x7fa7dc0708c0 msgr2=0x7fa7dc070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:37.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.796+0000 7fa7e1800640 1 -- 172.21.15.50:0/1565965163 shutdown_connections 2024-09-21T06:07:37.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.796+0000 7fa7e1800640 1 -- 172.21.15.50:0/1565965163 wait complete. 2024-09-21T06:07:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.800+0000 7fa7e1800640 1 Processor -- start 2024-09-21T06:07:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.800+0000 7fa7e1800640 1 -- start start 2024-09-21T06:07:37.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7e1800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c0004810 0x7fa7dc152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7e1800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa7dc14e210 0x7fa7dc15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7e1800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7dc15c620 0x7fa7dc14d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7e1800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc15d320 con 0x7fa7c0004810 2024-09-21T06:07:37.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7e1800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc15d1a0 con 0x7fa7dc15c620 2024-09-21T06:07:37.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7e1800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa7dc15d4a0 con 0x7fa7dc14e210 2024-09-21T06:07:37.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c0004810 0x7fa7dc152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7d3e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa7dc14e210 0x7fa7dc15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c0004810 0x7fa7dc152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49974/0 (socket says 172.21.15.50:49974) 2024-09-21T06:07:37.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7daa00640 1 -- 172.21.15.50:0/1625185812 learned_addr learned my addr 172.21.15.50:0/1625185812 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:37.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7daa00640 1 -- 172.21.15.50:0/1625185812 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa7dc14e210 msgr2=0x7fa7dc15c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:37.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7daa00640 1 --2- 172.21.15.50:0/1625185812 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa7dc14e210 0x7fa7dc15c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7daa00640 1 -- 172.21.15.50:0/1625185812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7dc15c620 msgr2=0x7fa7dc14d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:37.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7daa00640 1 --2- 172.21.15.50:0/1625185812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7dc15c620 0x7fa7dc14d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7daa00640 1 -- 172.21.15.50:0/1625185812 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7dc159bf0 con 0x7fa7c0004810 2024-09-21T06:07:37.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.804+0000 7fa7daa00640 1 --2- 172.21.15.50:0/1625185812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c0004810 0x7fa7dc152170 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc006e10 tx=0x7fa7cc0049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:37.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.812+0000 7fa7d2a00640 1 -- 172.21.15.50:0/1625185812 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc015020 con 0x7fa7c0004810 2024-09-21T06:07:37.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.812+0000 7fa7e1800640 1 -- 172.21.15.50:0/1625185812 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7dc159ed0 con 0x7fa7c0004810 2024-09-21T06:07:37.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.812+0000 7fa7e1800640 1 -- 172.21.15.50:0/1625185812 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7dc15bbc0 con 0x7fa7c0004810 2024-09-21T06:07:37.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.812+0000 7fa7d2a00640 1 -- 172.21.15.50:0/1625185812 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7cc0109a0 con 0x7fa7c0004810 2024-09-21T06:07:37.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.812+0000 7fa7d2a00640 1 -- 172.21.15.50:0/1625185812 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc010df0 con 0x7fa7c0004810 2024-09-21T06:07:37.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.820+0000 7fa7cbe00640 1 -- 172.21.15.50:0/1625185812 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7dc15a110 con 0x7fa7c0004810 2024-09-21T06:07:37.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.820+0000 7fa7d2a00640 1 -- 172.21.15.50:0/1625185812 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7fa7cc018020 con 0x7fa7c0004810 2024-09-21T06:07:37.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.824+0000 7fa7d2a00640 1 --2- 172.21.15.50:0/1625185812 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa7b8074760 0x7fa7b8076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.824+0000 7fa7d2a00640 1 -- 172.21.15.50:0/1625185812 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7fa7cc094050 con 0x7fa7c0004810 2024-09-21T06:07:37.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.840+0000 7fa7d2a00640 1 -- 172.21.15.50:0/1625185812 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa7dc15a110 con 0x7fa7c0004810 2024-09-21T06:07:37.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.848+0000 7fa7d3e00640 1 --2- 172.21.15.50:0/1625185812 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa7b8074760 0x7fa7b8076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:37.888+0000 7fa7d3e00640 1 --2- 172.21.15.50:0/1625185812 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa7b8074760 0x7fa7b8076c70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40ae000 tx=0x7fa7c40b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:38.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:38.168+0000 7fa7cbe00640 1 -- 172.21.15.50:0/1625185812 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone80951", "target": ["mon-mgr", ""]}) -- 0x7fa7dc06b540 con 0x7fa7b8074760 2024-09-21T06:07:38.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:38.248+0000 7fa7d2a00640 1 -- 172.21.15.50:0/1625185812 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7fa7cc05c210 con 0x7fa7c0004810 2024-09-21T06:07:38.794 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:38.796+0000 7f65a0c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:38.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:38.808+0000 7fa7d2a00640 1 -- 172.21.15.50:0/1625185812 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fa7c40a9410 con 0x7fa7b8074760 2024-09-21T06:07:38.807 INFO:teuthology.orchestra.run.smithi050.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:38.819 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T06:07:38.820 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone72702 2024-09-21T06:07:39.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.172+0000 7f5259c00640 1 -- 172.21.15.50:0/2712439078 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52540ce4c0 msgr2=0x7f525415aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:39.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.172+0000 7f5259c00640 1 --2- 172.21.15.50:0/2712439078 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52540ce4c0 0x7f525415aa50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5244009840 tx=0x7f524401c840 comp rx=0 tx=0).stop 2024-09-21T06:07:39.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.172+0000 7f5259c00640 1 -- 172.21.15.50:0/2712439078 shutdown_connections 2024-09-21T06:07:39.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.172+0000 7f5259c00640 1 --2- 172.21.15.50:0/2712439078 >> v2:172.21.15.88:3300/0 conn(0x7f52540cdd80 0x7f5254165660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.172+0000 7f5259c00640 1 --2- 172.21.15.50:0/2712439078 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52540ce4c0 0x7f525415aa50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.172+0000 7f5259c00640 1 -- 172.21.15.50:0/2712439078 >> 172.21.15.50:0/2712439078 conn(0x7f52540b2040 msgr2=0x7f52540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:39.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.172+0000 7f5259c00640 1 -- 172.21.15.50:0/2712439078 shutdown_connections 2024-09-21T06:07:39.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.172+0000 7f5259c00640 1 -- 172.21.15.50:0/2712439078 wait complete. 2024-09-21T06:07:39.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.176+0000 7f5259c00640 1 Processor -- start 2024-09-21T06:07:39.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.176+0000 7f5259c00640 1 -- start start 2024-09-21T06:07:39.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5259c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52540cd260 0x7f5254208570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5259c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52540cdd80 0x7f5254208ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5259c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52540ce4c0 0x7f5254208ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5259c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5254168bd0 con 0x7f52540cdd80 2024-09-21T06:07:39.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5259c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5254168a50 con 0x7f52540ce4c0 2024-09-21T06:07:39.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5259c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5254168d50 con 0x7f52540cd260 2024-09-21T06:07:39.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5252a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52540cd260 0x7f5254208570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5252a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52540cd260 0x7f5254208570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:54550/0 (socket says 172.21.15.50:54550) 2024-09-21T06:07:39.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5252a00640 1 -- 172.21.15.50:0/380859682 learned_addr learned my addr 172.21.15.50:0/380859682 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:39.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5252a00640 1 -- 172.21.15.50:0/380859682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52540ce4c0 msgr2=0x7f5254208ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:39.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5252a00640 1 --2- 172.21.15.50:0/380859682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52540ce4c0 0x7f5254208ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5252a00640 1 -- 172.21.15.50:0/380859682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52540cdd80 msgr2=0x7f5254208ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:39.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5252a00640 1 --2- 172.21.15.50:0/380859682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52540cdd80 0x7f5254208ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.180+0000 7f5252a00640 1 -- 172.21.15.50:0/380859682 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52540aea20 con 0x7f52540cd260 2024-09-21T06:07:39.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.184+0000 7f5252a00640 1 --2- 172.21.15.50:0/380859682 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f52540cd260 0x7f5254208570 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5244019450 tx=0x7f5244005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:39.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.188+0000 7f524aa00640 1 -- 172.21.15.50:0/380859682 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52440ba020 con 0x7f52540cd260 2024-09-21T06:07:39.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.188+0000 7f5259c00640 1 -- 172.21.15.50:0/380859682 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52540aed00 con 0x7f52540cd260 2024-09-21T06:07:39.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.188+0000 7f524aa00640 1 -- 172.21.15.50:0/380859682 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52440055e0 con 0x7f52540cd260 2024-09-21T06:07:39.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.188+0000 7f5259c00640 1 -- 172.21.15.50:0/380859682 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5254207710 con 0x7f52540cd260 2024-09-21T06:07:39.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.188+0000 7f524aa00640 1 -- 172.21.15.50:0/380859682 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52440ba020 con 0x7f52540cd260 2024-09-21T06:07:39.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.196+0000 7f5259c00640 1 -- 172.21.15.50:0/380859682 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f525415aa50 con 0x7f52540cd260 2024-09-21T06:07:39.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.200+0000 7f524aa00640 1 -- 172.21.15.50:0/380859682 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f5244004030 con 0x7f52540cd260 2024-09-21T06:07:39.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.200+0000 7f524aa00640 1 --2- 172.21.15.50:0/380859682 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f52300749f0 0x7f5230076f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.204+0000 7f524aa00640 1 -- 172.21.15.50:0/380859682 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f524412c830 con 0x7f52540cd260 2024-09-21T06:07:39.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.216+0000 7f524aa00640 1 -- 172.21.15.50:0/380859682 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5244016430 con 0x7f52540cd260 2024-09-21T06:07:39.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.244+0000 7f524be00640 1 --2- 172.21.15.50:0/380859682 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f52300749f0 0x7f5230076f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.252+0000 7f524be00640 1 --2- 172.21.15.50:0/380859682 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f52300749f0 0x7f5230076f00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f523c0046c0 tx=0x7f523c007910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:39.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.336+0000 7f524aa00640 1 -- 172.21.15.50:0/380859682 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f52440f4e10 con 0x7f52540cd260 2024-09-21T06:07:39.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.608+0000 7f5259c00640 1 -- 172.21.15.50:0/380859682 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone72702", "target": ["mon-mgr", ""]}) -- 0x7f525406b540 con 0x7f52300749f0 2024-09-21T06:07:39.900 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:39.900+0000 7f65a0c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:39.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:39.908+0000 7f524aa00640 1 -- 172.21.15.50:0/380859682 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f523c014400 con 0x7f52300749f0 2024-09-21T06:07:39.904 INFO:teuthology.orchestra.run.smithi050.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:39.923 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T06:07:39.924 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone2443 2024-09-21T06:07:40.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.204+0000 7fd8f1200640 1 -- 172.21.15.50:0/658200714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8d00053b0 msgr2=0x7fd8d000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:40.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.204+0000 7fd8f1200640 1 --2- 172.21.15.50:0/658200714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8d00053b0 0x7fd8d000ba30 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009f60 tx=0x7fd8e401c970 comp rx=0 tx=0).stop 2024-09-21T06:07:40.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.204+0000 7fd8f1200640 1 -- 172.21.15.50:0/658200714 shutdown_connections 2024-09-21T06:07:40.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.204+0000 7fd8f1200640 1 --2- 172.21.15.50:0/658200714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8d00053b0 0x7fd8d000ba30 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:40.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.204+0000 7fd8f1200640 1 -- 172.21.15.50:0/658200714 >> 172.21.15.50:0/658200714 conn(0x7fd8ec0b2040 msgr2=0x7fd8ec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:40.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.204+0000 7fd8f1200640 1 -- 172.21.15.50:0/658200714 shutdown_connections 2024-09-21T06:07:40.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.204+0000 7fd8f1200640 1 -- 172.21.15.50:0/658200714 wait complete. 2024-09-21T06:07:40.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.208+0000 7fd8f1200640 1 Processor -- start 2024-09-21T06:07:40.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.208+0000 7fd8f1200640 1 -- start start 2024-09-21T06:07:40.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.208+0000 7fd8f1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8d00053b0 0x7fd8ec156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:40.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.208+0000 7fd8f1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8ec14e970 0x7fd8ec1f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:40.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.208+0000 7fd8f1200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8ec157110 0x7fd8ec157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:40.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.208+0000 7fd8f1200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec075010 con 0x7fd8ec14e970 2024-09-21T06:07:40.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.208+0000 7fd8f1200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec152a50 con 0x7fd8d00053b0 2024-09-21T06:07:40.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.208+0000 7fd8f1200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec075190 con 0x7fd8ec157110 2024-09-21T06:07:40.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.208+0000 7fd8e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8ec14e970 0x7fd8ec1f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:40.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.208+0000 7fd8e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8ec14e970 0x7fd8ec1f6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49990/0 (socket says 172.21.15.50:49990) 2024-09-21T06:07:40.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.208+0000 7fd8e3e00640 1 -- 172.21.15.50:0/2131369818 learned_addr learned my addr 172.21.15.50:0/2131369818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:40.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.208+0000 7fd8e9e00640 1 --2- 172.21.15.50:0/2131369818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8d00053b0 0x7fd8ec156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:40.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8ea800640 1 --2- 172.21.15.50:0/2131369818 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8ec157110 0x7fd8ec157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:40.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8e3e00640 1 -- 172.21.15.50:0/2131369818 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8ec157110 msgr2=0x7fd8ec157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:40.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8e3e00640 1 --2- 172.21.15.50:0/2131369818 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8ec157110 0x7fd8ec157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:40.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8e3e00640 1 -- 172.21.15.50:0/2131369818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8d00053b0 msgr2=0x7fd8ec156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:40.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8e3e00640 1 --2- 172.21.15.50:0/2131369818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8d00053b0 0x7fd8ec156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:40.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8e3e00640 1 -- 172.21.15.50:0/2131369818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec1f66c0 con 0x7fd8ec14e970 2024-09-21T06:07:40.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8e3e00640 1 --2- 172.21.15.50:0/2131369818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8ec14e970 0x7fd8ec1f6180 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400c410 tx=0x7fd8d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:40.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8e2000640 1 -- 172.21.15.50:0/2131369818 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4017070 con 0x7fd8ec14e970 2024-09-21T06:07:40.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8e2000640 1 -- 172.21.15.50:0/2131369818 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8d4003600 con 0x7fd8ec14e970 2024-09-21T06:07:40.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8e2000640 1 -- 172.21.15.50:0/2131369818 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4003a90 con 0x7fd8ec14e970 2024-09-21T06:07:40.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8f1200640 1 -- 172.21.15.50:0/2131369818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec0c6020 con 0x7fd8ec14e970 2024-09-21T06:07:40.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8f1200640 1 -- 172.21.15.50:0/2131369818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec155b60 con 0x7fd8ec14e970 2024-09-21T06:07:40.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.212+0000 7fd8f1200640 1 -- 172.21.15.50:0/2131369818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec0b51f0 con 0x7fd8ec14e970 2024-09-21T06:07:40.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.216+0000 7fd8e2000640 1 -- 172.21.15.50:0/2131369818 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7fd8d401a070 con 0x7fd8ec14e970 2024-09-21T06:07:40.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.216+0000 7fd8e2000640 1 --2- 172.21.15.50:0/2131369818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8c8074760 0x7fd8c8076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:40.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.216+0000 7fd8e2000640 1 -- 172.21.15.50:0/2131369818 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7fd8d40925c0 con 0x7fd8ec14e970 2024-09-21T06:07:40.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.224+0000 7fd8e2000640 1 -- 172.21.15.50:0/2131369818 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd8ec1f66c0 con 0x7fd8ec14e970 2024-09-21T06:07:40.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.228+0000 7fd8e9e00640 1 --2- 172.21.15.50:0/2131369818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8c8074760 0x7fd8c8076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:40.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.236+0000 7fd8e9e00640 1 --2- 172.21.15.50:0/2131369818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8c8074760 0x7fd8c8076c70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc007a90 tx=0x7fd8dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:40.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:40.556+0000 7fd8f1200640 1 -- 172.21.15.50:0/2131369818 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone2443", "target": ["mon-mgr", ""]}) -- 0x7fd8ec06b540 con 0x7fd8c8074760 2024-09-21T06:07:40.999 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:41.000+0000 7f65a0c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.020+0000 7fd8e2000640 1 -- 172.21.15.50:0/2131369818 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fd8dc00f7f0 con 0x7fd8c8074760 2024-09-21T06:07:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:41.042 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T06:07:41.043 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone96264 2024-09-21T06:07:41.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.320+0000 7fe285600640 1 -- 172.21.15.50:0/915958135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe280073b00 msgr2=0x7fe280073f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:41.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.320+0000 7fe285600640 1 --2- 172.21.15.50:0/915958135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe280073b00 0x7fe280073f10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe270009900 tx=0x7fe27001c700 comp rx=0 tx=0).stop 2024-09-21T06:07:41.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.320+0000 7fe285600640 1 -- 172.21.15.50:0/915958135 shutdown_connections 2024-09-21T06:07:41.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.320+0000 7fe285600640 1 --2- 172.21.15.50:0/915958135 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe28006fd90 0x7fe2800701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.320+0000 7fe285600640 1 --2- 172.21.15.50:0/915958135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe280073b00 0x7fe280073f10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.320+0000 7fe285600640 1 -- 172.21.15.50:0/915958135 >> 172.21.15.50:0/915958135 conn(0x7fe2800b3b30 msgr2=0x7fe2800733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:41.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.320+0000 7fe285600640 1 -- 172.21.15.50:0/915958135 shutdown_connections 2024-09-21T06:07:41.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.320+0000 7fe285600640 1 -- 172.21.15.50:0/915958135 wait complete. 2024-09-21T06:07:41.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.324+0000 7fe285600640 1 Processor -- start 2024-09-21T06:07:41.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.324+0000 7fe285600640 1 -- start start 2024-09-21T06:07:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe285600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe28006fd90 0x7fe280141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe285600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe280073b00 0x7fe280166a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe285600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2800d07a0 0x7fe2801446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe285600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe28016a030 con 0x7fe28006fd90 2024-09-21T06:07:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe285600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe280169eb0 con 0x7fe2800d07a0 2024-09-21T06:07:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe285600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe28016a1b0 con 0x7fe280073b00 2024-09-21T06:07:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe27e000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe280073b00 0x7fe280166a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe27e000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe280073b00 0x7fe280166a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:54588/0 (socket says 172.21.15.50:54588) 2024-09-21T06:07:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe27e000640 1 -- 172.21.15.50:0/386986892 learned_addr learned my addr 172.21.15.50:0/386986892 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe27ea00640 1 --2- 172.21.15.50:0/386986892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe28006fd90 0x7fe280141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe27e000640 1 -- 172.21.15.50:0/386986892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2800d07a0 msgr2=0x7fe2801446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:41.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe27e000640 1 --2- 172.21.15.50:0/386986892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2800d07a0 0x7fe2801446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe27e000640 1 -- 172.21.15.50:0/386986892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe28006fd90 msgr2=0x7fe280141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:41.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.328+0000 7fe27e000640 1 --2- 172.21.15.50:0/386986892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe28006fd90 0x7fe280141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.332+0000 7fe27e000640 1 -- 172.21.15.50:0/386986892 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28013e2d0 con 0x7fe280073b00 2024-09-21T06:07:41.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.340+0000 7fe27e000640 1 --2- 172.21.15.50:0/386986892 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe280073b00 0x7fe280166a40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00b3e0 tx=0x7fe26c00b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:41.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.372+0000 7fe276a00640 1 -- 172.21.15.50:0/386986892 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c015020 con 0x7fe280073b00 2024-09-21T06:07:41.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.372+0000 7fe276a00640 1 -- 172.21.15.50:0/386986892 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe26c011920 con 0x7fe280073b00 2024-09-21T06:07:41.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.372+0000 7fe276a00640 1 -- 172.21.15.50:0/386986892 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c011d90 con 0x7fe280073b00 2024-09-21T06:07:41.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.372+0000 7fe285600640 1 -- 172.21.15.50:0/386986892 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28013e5b0 con 0x7fe280073b00 2024-09-21T06:07:41.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.376+0000 7fe285600640 1 -- 172.21.15.50:0/386986892 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe280142020 con 0x7fe280073b00 2024-09-21T06:07:41.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.380+0000 7fe285600640 1 -- 172.21.15.50:0/386986892 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2800701a0 con 0x7fe280073b00 2024-09-21T06:07:41.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.384+0000 7fe276a00640 1 -- 172.21.15.50:0/386986892 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7fe26c018020 con 0x7fe280073b00 2024-09-21T06:07:41.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.384+0000 7fe276a00640 1 --2- 172.21.15.50:0/386986892 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe2600747b0 0x7fe260076cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.384+0000 7fe276a00640 1 -- 172.21.15.50:0/386986892 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7fe26c05b940 con 0x7fe280073b00 2024-09-21T06:07:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.396+0000 7fe276a00640 1 -- 172.21.15.50:0/386986892 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe26c01f020 con 0x7fe280073b00 2024-09-21T06:07:41.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.412+0000 7fe27ea00640 1 --2- 172.21.15.50:0/386986892 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe2600747b0 0x7fe260076cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.424+0000 7fe27ea00640 1 --2- 172.21.15.50:0/386986892 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe2600747b0 0x7fe260076cc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe2700ae000 tx=0x7fe27001af40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:41.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:41.832+0000 7fe285600640 1 -- 172.21.15.50:0/386986892 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone96264", "target": ["mon-mgr", ""]}) -- 0x7fe28006b540 con 0x7fe2600747b0 2024-09-21T06:07:42.141 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:42.136+0000 7f65a0c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:42.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.176+0000 7fe276a00640 1 -- 172.21.15.50:0/386986892 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fe2700052a0 con 0x7fe2600747b0 2024-09-21T06:07:42.173 INFO:teuthology.orchestra.run.smithi050.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:42.195 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T06:07:42.196 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone64155 2024-09-21T06:07:42.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.468+0000 7f35a3200640 1 -- 172.21.15.50:0/4220978666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a4160060 msgr2=0x7f35a4074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:42.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.468+0000 7f35a3200640 1 --2- 172.21.15.50:0/4220978666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a4160060 0x7f35a4074240 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f35a4073cf0 tx=0x7f359001c830 comp rx=0 tx=0).stop 2024-09-21T06:07:42.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.468+0000 7f35a3200640 1 -- 172.21.15.50:0/4220978666 shutdown_connections 2024-09-21T06:07:42.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.468+0000 7f35a3200640 1 --2- 172.21.15.50:0/4220978666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a4160060 0x7f35a4074240 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:42.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.468+0000 7f35a3200640 1 --2- 172.21.15.50:0/4220978666 >> v2:172.21.15.88:3300/0 conn(0x7f35a41536f0 0x7f35a415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:42.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.468+0000 7f35a3200640 1 -- 172.21.15.50:0/4220978666 >> 172.21.15.50:0/4220978666 conn(0x7f35a40b4560 msgr2=0x7f35a40d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:42.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.468+0000 7f35a3200640 1 -- 172.21.15.50:0/4220978666 shutdown_connections 2024-09-21T06:07:42.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.468+0000 7f35a3200640 1 -- 172.21.15.50:0/4220978666 wait complete. 2024-09-21T06:07:42.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.472+0000 7f35a3200640 1 Processor -- start 2024-09-21T06:07:42.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.472+0000 7f35a3200640 1 -- start start 2024-09-21T06:07:42.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.472+0000 7f35a3200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f35a41536f0 0x7f35a415abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:42.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.472+0000 7f35a3200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35a4160060 0x7f35a414ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:42.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.472+0000 7f35a3200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a4160470 0x7f35a414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:42.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.472+0000 7f35a3200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f35a416a3a0 con 0x7f35a4160470 2024-09-21T06:07:42.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.472+0000 7f35a3200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f35a416a220 con 0x7f35a4160060 2024-09-21T06:07:42.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.472+0000 7f35a3200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f35a416a520 con 0x7f35a41536f0 2024-09-21T06:07:42.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.472+0000 7f35a2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a4160470 0x7f35a414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:42.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.472+0000 7f35a2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a4160470 0x7f35a414f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50018/0 (socket says 172.21.15.50:50018) 2024-09-21T06:07:42.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.472+0000 7f35a2800640 1 -- 172.21.15.50:0/3986693806 learned_addr learned my addr 172.21.15.50:0/3986693806 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:42.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.472+0000 7f35a1400640 1 --2- 172.21.15.50:0/3986693806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35a4160060 0x7f35a414ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:42.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.476+0000 7f35a2800640 1 -- 172.21.15.50:0/3986693806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f35a41536f0 msgr2=0x7f35a415abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:42.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.476+0000 7f35a2800640 1 --2- 172.21.15.50:0/3986693806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f35a41536f0 0x7f35a415abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:42.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.476+0000 7f35a2800640 1 -- 172.21.15.50:0/3986693806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35a4160060 msgr2=0x7f35a414ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:42.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.476+0000 7f35a2800640 1 --2- 172.21.15.50:0/3986693806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35a4160060 0x7f35a414ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:42.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.476+0000 7f35a2800640 1 -- 172.21.15.50:0/3986693806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a4154460 con 0x7f35a4160470 2024-09-21T06:07:42.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.476+0000 7f35a2800640 1 --2- 172.21.15.50:0/3986693806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a4160470 0x7f35a414f370 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f359c00c740 tx=0x7f359c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:42.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.476+0000 7f359a000640 1 -- 172.21.15.50:0/3986693806 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c003b80 con 0x7f35a4160470 2024-09-21T06:07:42.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.476+0000 7f359a000640 1 -- 172.21.15.50:0/3986693806 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f359c00ce50 con 0x7f35a4160470 2024-09-21T06:07:42.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.476+0000 7f359a000640 1 -- 172.21.15.50:0/3986693806 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c00a850 con 0x7f35a4160470 2024-09-21T06:07:42.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.476+0000 7f35a3200640 1 -- 172.21.15.50:0/3986693806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a4154740 con 0x7f35a4160470 2024-09-21T06:07:42.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.476+0000 7f35a3200640 1 -- 172.21.15.50:0/3986693806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a41fdfc0 con 0x7f35a4160470 2024-09-21T06:07:42.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.480+0000 7f35a3200640 1 -- 172.21.15.50:0/3986693806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a4151060 con 0x7f35a4160470 2024-09-21T06:07:42.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.480+0000 7f359a000640 1 -- 172.21.15.50:0/3986693806 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f359c00a9f0 con 0x7f35a4160470 2024-09-21T06:07:42.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.480+0000 7f359a000640 1 --2- 172.21.15.50:0/3986693806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f35840747b0 0x7f3584076cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:42.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.480+0000 7f359a000640 1 -- 172.21.15.50:0/3986693806 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f359c0931a0 con 0x7f35a4160470 2024-09-21T06:07:42.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.488+0000 7f359a000640 1 -- 172.21.15.50:0/3986693806 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f35a415d9c0 con 0x7f35a4160470 2024-09-21T06:07:42.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.516+0000 7f35a1e00640 1 --2- 172.21.15.50:0/3986693806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f35840747b0 0x7f3584076cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:42.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.528+0000 7f35a1e00640 1 --2- 172.21.15.50:0/3986693806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f35840747b0 0x7f3584076cc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3594009690 tx=0x7f3594006c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:42.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:42.924+0000 7f35a3200640 1 -- 172.21.15.50:0/3986693806 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone64155", "target": ["mon-mgr", ""]}) -- 0x7f35a406b540 con 0x7f35840747b0 2024-09-21T06:07:43.326 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:43.328+0000 7f65a0c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:43.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.328+0000 7f359a000640 1 -- 172.21.15.50:0/3986693806 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f3594013430 con 0x7f35840747b0 2024-09-21T06:07:43.328 INFO:teuthology.orchestra.run.smithi050.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:43.345 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T06:07:43.345 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8636 2024-09-21T06:07:43.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.656+0000 7fea85a00640 1 -- 172.21.15.50:0/3196163443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea8015cb90 msgr2=0x7fea80070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:43.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.656+0000 7fea85a00640 1 --2- 172.21.15.50:0/3196163443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea8015cb90 0x7fea80070060 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fea800722e0 tx=0x7fea7801c650 comp rx=0 tx=0).stop 2024-09-21T06:07:43.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.660+0000 7fea85a00640 1 -- 172.21.15.50:0/3196163443 shutdown_connections 2024-09-21T06:07:43.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.660+0000 7fea85a00640 1 --2- 172.21.15.50:0/3196163443 >> v2:172.21.15.88:3300/0 conn(0x7fea800b4fd0 0x7fea800b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.660+0000 7fea85a00640 1 --2- 172.21.15.50:0/3196163443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea8015cb90 0x7fea80070060 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.660+0000 7fea85a00640 1 -- 172.21.15.50:0/3196163443 >> 172.21.15.50:0/3196163443 conn(0x7fea8006d3f0 msgr2=0x7fea800b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:43.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.660+0000 7fea85a00640 1 -- 172.21.15.50:0/3196163443 shutdown_connections 2024-09-21T06:07:43.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.660+0000 7fea85a00640 1 -- 172.21.15.50:0/3196163443 wait complete. 2024-09-21T06:07:43.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.664+0000 7fea85a00640 1 Processor -- start 2024-09-21T06:07:43.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.664+0000 7fea85a00640 1 -- start start 2024-09-21T06:07:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.664+0000 7fea85a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea800af520 0x7fea800c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.664+0000 7fea85a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea800b4fd0 0x7fea801466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.664+0000 7fea85a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea8015cb90 0x7fea8015b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.664+0000 7fea85a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fea800cfad0 con 0x7fea800b4fd0 2024-09-21T06:07:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.664+0000 7fea85a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fea800cf950 con 0x7fea800af520 2024-09-21T06:07:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.664+0000 7fea85a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fea800cfc50 con 0x7fea8015cb90 2024-09-21T06:07:43.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.664+0000 7fea7f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea800b4fd0 0x7fea801466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.664+0000 7fea7f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea800b4fd0 0x7fea801466a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50028/0 (socket says 172.21.15.50:50028) 2024-09-21T06:07:43.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.664+0000 7fea7f400640 1 -- 172.21.15.50:0/4261547252 learned_addr learned my addr 172.21.15.50:0/4261547252 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:43.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.664+0000 7fea7fe00640 1 --2- 172.21.15.50:0/4261547252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea800af520 0x7fea800c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.668+0000 7fea7f400640 1 -- 172.21.15.50:0/4261547252 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea8015cb90 msgr2=0x7fea8015b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:43.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.668+0000 7fea7f400640 1 --2- 172.21.15.50:0/4261547252 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea8015cb90 0x7fea8015b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.668+0000 7fea7f400640 1 -- 172.21.15.50:0/4261547252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea800af520 msgr2=0x7fea800c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:43.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.668+0000 7fea7f400640 1 --2- 172.21.15.50:0/4261547252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea800af520 0x7fea800c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.668+0000 7fea7f400640 1 -- 172.21.15.50:0/4261547252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8015a570 con 0x7fea800b4fd0 2024-09-21T06:07:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.668+0000 7fea7f400640 1 --2- 172.21.15.50:0/4261547252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea800b4fd0 0x7fea801466a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fea6800ba50 tx=0x7fea6800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.668+0000 7fea77e00640 1 -- 172.21.15.50:0/4261547252 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68019070 con 0x7fea800b4fd0 2024-09-21T06:07:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.668+0000 7fea77e00640 1 -- 172.21.15.50:0/4261547252 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea6800cd00 con 0x7fea800b4fd0 2024-09-21T06:07:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.668+0000 7fea77e00640 1 -- 172.21.15.50:0/4261547252 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea680146a0 con 0x7fea800b4fd0 2024-09-21T06:07:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.668+0000 7fea85a00640 1 -- 172.21.15.50:0/4261547252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8015a850 con 0x7fea800b4fd0 2024-09-21T06:07:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.668+0000 7fea85a00640 1 -- 172.21.15.50:0/4261547252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8015ae10 con 0x7fea800b4fd0 2024-09-21T06:07:43.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.668+0000 7fea85a00640 1 -- 172.21.15.50:0/4261547252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea800b53e0 con 0x7fea800b4fd0 2024-09-21T06:07:43.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.672+0000 7fea77e00640 1 -- 172.21.15.50:0/4261547252 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7fea6801c070 con 0x7fea800b4fd0 2024-09-21T06:07:43.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.672+0000 7fea77e00640 1 --2- 172.21.15.50:0/4261547252 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea5c074760 0x7fea5c076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.672+0000 7fea77e00640 1 -- 172.21.15.50:0/4261547252 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7fea68093500 con 0x7fea800b4fd0 2024-09-21T06:07:43.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.672+0000 7fea7fe00640 1 --2- 172.21.15.50:0/4261547252 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea5c074760 0x7fea5c076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.676+0000 7fea77e00640 1 -- 172.21.15.50:0/4261547252 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fea68023020 con 0x7fea800b4fd0 2024-09-21T06:07:43.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:43.680+0000 7fea7fe00640 1 --2- 172.21.15.50:0/4261547252 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea5c074760 0x7fea5c076c70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fea780af000 tx=0x7fea7801ae90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:44.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.152+0000 7fea85a00640 1 -- 172.21.15.50:0/4261547252 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8636", "target": ["mon-mgr", ""]}) -- 0x7fea8006b540 con 0x7fea5c074760 2024-09-21T06:07:44.483 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:44.484+0000 7f65a0c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:44.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.504+0000 7fea77e00640 1 -- 172.21.15.50:0/4261547252 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fea78004040 con 0x7fea5c074760 2024-09-21T06:07:44.500 INFO:teuthology.orchestra.run.smithi050.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:44.516 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T06:07:44.517 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone67087 2024-09-21T06:07:44.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.804+0000 7efdfee00640 1 -- 172.21.15.50:0/4213822139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf8068df0 msgr2=0x7efdf8069200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:44.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.804+0000 7efdfee00640 1 --2- 172.21.15.50:0/4213822139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf8068df0 0x7efdf8069200 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7efdf80b2f20 tx=0x7efdec01c830 comp rx=0 tx=0).stop 2024-09-21T06:07:44.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.804+0000 7efdfee00640 1 -- 172.21.15.50:0/4213822139 shutdown_connections 2024-09-21T06:07:44.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.804+0000 7efdfee00640 1 --2- 172.21.15.50:0/4213822139 >> v2:172.21.15.88:3300/0 conn(0x7efdf80752d0 0x7efdf80b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:44.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.804+0000 7efdfee00640 1 --2- 172.21.15.50:0/4213822139 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdf8074980 0x7efdf8074d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:44.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.804+0000 7efdfee00640 1 --2- 172.21.15.50:0/4213822139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf8068df0 0x7efdf8069200 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:44.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.804+0000 7efdfee00640 1 -- 172.21.15.50:0/4213822139 >> 172.21.15.50:0/4213822139 conn(0x7efdf8073470 msgr2=0x7efdf80b6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:44.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.804+0000 7efdfee00640 1 -- 172.21.15.50:0/4213822139 shutdown_connections 2024-09-21T06:07:44.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.804+0000 7efdfee00640 1 -- 172.21.15.50:0/4213822139 wait complete. 2024-09-21T06:07:44.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.808+0000 7efdfee00640 1 Processor -- start 2024-09-21T06:07:44.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.808+0000 7efdfee00640 1 -- start start 2024-09-21T06:07:44.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.812+0000 7efdfee00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdf8068df0 0x7efdf8148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:44.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.812+0000 7efdfee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf8074980 0x7efdf82006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:44.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.812+0000 7efdfee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80752d0 0x7efdf8200be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:44.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.812+0000 7efdfee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8162c50 con 0x7efdf80752d0 2024-09-21T06:07:44.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.812+0000 7efdfee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8162ad0 con 0x7efdf8074980 2024-09-21T06:07:44.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.812+0000 7efdfee00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7efdf8162dd0 con 0x7efdf8068df0 2024-09-21T06:07:44.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfe400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80752d0 0x7efdf8200be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:44.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfe400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80752d0 0x7efdf8200be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50040/0 (socket says 172.21.15.50:50040) 2024-09-21T06:07:44.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfda00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdf8068df0 0x7efdf8148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:44.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfda00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdf8068df0 0x7efdf8148290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:54626/0 (socket says 172.21.15.50:54626) 2024-09-21T06:07:44.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfda00640 1 -- 172.21.15.50:0/3650723384 learned_addr learned my addr 172.21.15.50:0/3650723384 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:44.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfd000640 1 --2- 172.21.15.50:0/3650723384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf8074980 0x7efdf82006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:44.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfd000640 1 -- 172.21.15.50:0/3650723384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdf8068df0 msgr2=0x7efdf8148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:44.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfd000640 1 --2- 172.21.15.50:0/3650723384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdf8068df0 0x7efdf8148290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:44.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfd000640 1 -- 172.21.15.50:0/3650723384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80752d0 msgr2=0x7efdf8200be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:44.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfd000640 1 --2- 172.21.15.50:0/3650723384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80752d0 0x7efdf8200be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:44.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfd000640 1 -- 172.21.15.50:0/3650723384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf809cca0 con 0x7efdf8074980 2024-09-21T06:07:44.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfe400640 1 --2- 172.21.15.50:0/3650723384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80752d0 0x7efdf8200be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:44.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.816+0000 7efdfd000640 1 --2- 172.21.15.50:0/3650723384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf8074980 0x7efdf82006a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efde400d610 tx=0x7efde400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:44.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.820+0000 7efdf5600640 1 -- 172.21.15.50:0/3650723384 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4017070 con 0x7efdf8074980 2024-09-21T06:07:44.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.820+0000 7efdf5600640 1 -- 172.21.15.50:0/3650723384 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efde4005310 con 0x7efdf8074980 2024-09-21T06:07:44.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.820+0000 7efdf5600640 1 -- 172.21.15.50:0/3650723384 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4017070 con 0x7efdf8074980 2024-09-21T06:07:44.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.824+0000 7efdfee00640 1 -- 172.21.15.50:0/3650723384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf81487d0 con 0x7efdf8074980 2024-09-21T06:07:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.824+0000 7efdfee00640 1 -- 172.21.15.50:0/3650723384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf809d220 con 0x7efdf8074980 2024-09-21T06:07:44.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.828+0000 7efdfee00640 1 -- 172.21.15.50:0/3650723384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf81632e0 con 0x7efdf8074980 2024-09-21T06:07:44.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.832+0000 7efdf5600640 1 -- 172.21.15.50:0/3650723384 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7efde4005520 con 0x7efdf8074980 2024-09-21T06:07:44.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.832+0000 7efdf5600640 1 --2- 172.21.15.50:0/3650723384 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdd8074960 0x7efdd8076e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:44.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.832+0000 7efdf5600640 1 -- 172.21.15.50:0/3650723384 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7efde4092e60 con 0x7efdf8074980 2024-09-21T06:07:44.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.836+0000 7efdfda00640 1 --2- 172.21.15.50:0/3650723384 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdd8074960 0x7efdd8076e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:44.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.836+0000 7efdfda00640 1 --2- 172.21.15.50:0/3650723384 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdd8074960 0x7efdd8076e70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efdec009230 tx=0x7efdec0a8040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:44.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:44.840+0000 7efdf5600640 1 -- 172.21.15.50:0/3650723384 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efde405ef10 con 0x7efdf8074980 2024-09-21T06:07:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:45.224+0000 7efdfee00640 1 -- 172.21.15.50:0/3650723384 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone67087", "target": ["mon-mgr", ""]}) -- 0x7efdf806b540 con 0x7efdd8074960 2024-09-21T06:07:45.881 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:45.880+0000 7f65a0c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:45.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:45.884+0000 7efdf5600640 1 -- 172.21.15.50:0/3650723384 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7efdec0a66a0 con 0x7efdd8074960 2024-09-21T06:07:45.882 INFO:teuthology.orchestra.run.smithi050.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:45.901 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T06:07:45.901 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone71268 2024-09-21T06:07:46.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.196+0000 7f438ca00640 1 -- 172.21.15.50:0/1440457590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43880cdd80 msgr2=0x7f43880b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:46.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.196+0000 7f438ca00640 1 --2- 172.21.15.50:0/1440457590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43880cdd80 0x7f43880b3a50 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f437400aa10 tx=0x7f437401dc30 comp rx=0 tx=0).stop 2024-09-21T06:07:46.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.196+0000 7f438ca00640 1 -- 172.21.15.50:0/1440457590 shutdown_connections 2024-09-21T06:07:46.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.196+0000 7f438ca00640 1 --2- 172.21.15.50:0/1440457590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43880cdd80 0x7f43880b3a50 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:46.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.196+0000 7f438ca00640 1 -- 172.21.15.50:0/1440457590 >> 172.21.15.50:0/1440457590 conn(0x7f43880b2040 msgr2=0x7f43880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:46.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.200+0000 7f438ca00640 1 -- 172.21.15.50:0/1440457590 shutdown_connections 2024-09-21T06:07:46.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.200+0000 7f438ca00640 1 -- 172.21.15.50:0/1440457590 wait complete. 2024-09-21T06:07:46.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.204+0000 7f438ca00640 1 Processor -- start 2024-09-21T06:07:46.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.204+0000 7f438ca00640 1 -- start start 2024-09-21T06:07:46.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.208+0000 7f438ca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43880cd970 0x7f43880aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:46.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.208+0000 7f438ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43880cdd80 0x7f438813c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:46.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.208+0000 7f438ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f438815aa50 0x7f438813c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:46.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.208+0000 7f438ca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43881695a0 con 0x7f438815aa50 2024-09-21T06:07:46.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.208+0000 7f438ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4388169420 con 0x7f43880cdd80 2024-09-21T06:07:46.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.208+0000 7f438ca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4388169720 con 0x7f43880cd970 2024-09-21T06:07:46.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.208+0000 7f4386800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f438815aa50 0x7f438813c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:46.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.208+0000 7f4386800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f438815aa50 0x7f438813c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50070/0 (socket says 172.21.15.50:50070) 2024-09-21T06:07:46.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.208+0000 7f4386800640 1 -- 172.21.15.50:0/3960341791 learned_addr learned my addr 172.21.15.50:0/3960341791 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:46.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.208+0000 7f4385400640 1 --2- 172.21.15.50:0/3960341791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43880cdd80 0x7f438813c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:46.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.208+0000 7f4385e00640 1 --2- 172.21.15.50:0/3960341791 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43880cd970 0x7f43880aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:46.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.212+0000 7f4386800640 1 -- 172.21.15.50:0/3960341791 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43880cd970 msgr2=0x7f43880aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.212+0000 7f4386800640 1 --2- 172.21.15.50:0/3960341791 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43880cd970 0x7f43880aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.212+0000 7f4386800640 1 -- 172.21.15.50:0/3960341791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43880cdd80 msgr2=0x7f438813c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.212+0000 7f4386800640 1 --2- 172.21.15.50:0/3960341791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43880cdd80 0x7f438813c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.212+0000 7f4386800640 1 -- 172.21.15.50:0/3960341791 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438813b2b0 con 0x7f438815aa50 2024-09-21T06:07:46.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.212+0000 7f4386800640 1 --2- 172.21.15.50:0/3960341791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f438815aa50 0x7f438813c890 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f43740b1040 tx=0x7f43740a6b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:46.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.212+0000 7f437e000640 1 -- 172.21.15.50:0/3960341791 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43740bc070 con 0x7f438815aa50 2024-09-21T06:07:46.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.212+0000 7f437e000640 1 -- 172.21.15.50:0/3960341791 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4374003b80 con 0x7f438815aa50 2024-09-21T06:07:46.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.212+0000 7f437e000640 1 -- 172.21.15.50:0/3960341791 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43740b3070 con 0x7f438815aa50 2024-09-21T06:07:46.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.212+0000 7f438ca00640 1 -- 172.21.15.50:0/3960341791 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438813b590 con 0x7f438815aa50 2024-09-21T06:07:46.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.212+0000 7f438ca00640 1 -- 172.21.15.50:0/3960341791 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f438813bb50 con 0x7f438815aa50 2024-09-21T06:07:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.216+0000 7f4373400640 1 -- 172.21.15.50:0/3960341791 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f438813b020 con 0x7f438815aa50 2024-09-21T06:07:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.216+0000 7f437e000640 1 -- 172.21.15.50:0/3960341791 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f43740ae440 con 0x7f438815aa50 2024-09-21T06:07:46.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.224+0000 7f437e000640 1 --2- 172.21.15.50:0/3960341791 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4368074760 0x7f4368076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:46.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.224+0000 7f437e000640 1 -- 172.21.15.50:0/3960341791 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f437412dae0 con 0x7f438815aa50 2024-09-21T06:07:46.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.224+0000 7f4385e00640 1 --2- 172.21.15.50:0/3960341791 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4368074760 0x7f4368076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:46.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.228+0000 7f437e000640 1 -- 172.21.15.50:0/3960341791 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f43740c5020 con 0x7f438815aa50 2024-09-21T06:07:46.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.232+0000 7f4385e00640 1 --2- 172.21.15.50:0/3960341791 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4368074760 0x7f4368076c70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4380001e40 tx=0x7f4380009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.628+0000 7f4373400640 1 -- 172.21.15.50:0/3960341791 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone71268", "target": ["mon-mgr", ""]}) -- 0x7f438806b540 con 0x7f4368074760 2024-09-21T06:07:46.892 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:46.892+0000 7f65a0c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:46.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:46.908+0000 7f437e000640 1 -- 172.21.15.50:0/3960341791 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f4380001ae0 con 0x7f4368074760 2024-09-21T06:07:46.906 INFO:teuthology.orchestra.run.smithi050.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:46.928 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T06:07:46.928 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone71879 2024-09-21T06:07:47.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.232+0000 7f2609800640 1 -- 172.21.15.50:0/1210308732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26040cd970 msgr2=0x7f26040b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:47.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.232+0000 7f2609800640 1 --2- 172.21.15.50:0/1210308732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26040cd970 0x7f26040b3340 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009900 tx=0x7f25f001c8e0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.236+0000 7f2609800640 1 -- 172.21.15.50:0/1210308732 shutdown_connections 2024-09-21T06:07:47.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.236+0000 7f2609800640 1 --2- 172.21.15.50:0/1210308732 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f26040cdd80 0x7f26040b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.236+0000 7f2609800640 1 --2- 172.21.15.50:0/1210308732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26040cd970 0x7f26040b3340 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.236+0000 7f2609800640 1 -- 172.21.15.50:0/1210308732 >> 172.21.15.50:0/1210308732 conn(0x7f26040b2040 msgr2=0x7f26040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:47.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.240+0000 7f2609800640 1 -- 172.21.15.50:0/1210308732 shutdown_connections 2024-09-21T06:07:47.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.240+0000 7f2609800640 1 -- 172.21.15.50:0/1210308732 wait complete. 2024-09-21T06:07:47.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.240+0000 7f2609800640 1 Processor -- start 2024-09-21T06:07:47.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2609800640 1 -- start start 2024-09-21T06:07:47.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2609800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26040cd970 0x7f26041f9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:47.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2609800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f26040cdd80 0x7f26041555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:47.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2609800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f260415aa50 0x7f260414f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:47.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2609800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26041661b0 con 0x7f26040cd970 2024-09-21T06:07:47.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2609800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2604166030 con 0x7f260415aa50 2024-09-21T06:07:47.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2609800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2604166330 con 0x7f26040cdd80 2024-09-21T06:07:47.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2602a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26040cd970 0x7f26041f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:47.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2602a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26040cd970 0x7f26041f9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46810/0 (socket says 172.21.15.50:46810) 2024-09-21T06:07:47.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2602a00640 1 -- 172.21.15.50:0/186108617 learned_addr learned my addr 172.21.15.50:0/186108617 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:47.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2602a00640 1 -- 172.21.15.50:0/186108617 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f26040cdd80 msgr2=0x7f26041555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:47.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2602a00640 1 --2- 172.21.15.50:0/186108617 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f26040cdd80 0x7f26041555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2602a00640 1 -- 172.21.15.50:0/186108617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f260415aa50 msgr2=0x7f260414f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:47.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2602a00640 1 --2- 172.21.15.50:0/186108617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f260415aa50 0x7f260414f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2602a00640 1 -- 172.21.15.50:0/186108617 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2604159f60 con 0x7f26040cd970 2024-09-21T06:07:47.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.244+0000 7f2602a00640 1 --2- 172.21.15.50:0/186108617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26040cd970 0x7f26041f9150 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f25f400b3e0 tx=0x7f25f400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:47.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.248+0000 7f25faa00640 1 -- 172.21.15.50:0/186108617 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4015020 con 0x7f26040cd970 2024-09-21T06:07:47.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.248+0000 7f2609800640 1 -- 172.21.15.50:0/186108617 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260415a240 con 0x7f26040cd970 2024-09-21T06:07:47.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.248+0000 7f2609800640 1 -- 172.21.15.50:0/186108617 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2604154da0 con 0x7f26040cd970 2024-09-21T06:07:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.248+0000 7f25faa00640 1 -- 172.21.15.50:0/186108617 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25f4011940 con 0x7f26040cd970 2024-09-21T06:07:47.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.248+0000 7f25faa00640 1 -- 172.21.15.50:0/186108617 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4011d90 con 0x7f26040cd970 2024-09-21T06:07:47.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.248+0000 7f2609800640 1 -- 172.21.15.50:0/186108617 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25dc0052a0 con 0x7f26040cd970 2024-09-21T06:07:47.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.248+0000 7f25faa00640 1 -- 172.21.15.50:0/186108617 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f25f4018020 con 0x7f26040cd970 2024-09-21T06:07:47.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.252+0000 7f25faa00640 1 --2- 172.21.15.50:0/186108617 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25e4074760 0x7f25e4076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:47.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.252+0000 7f25faa00640 1 -- 172.21.15.50:0/186108617 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f25f4093200 con 0x7f26040cd970 2024-09-21T06:07:47.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.252+0000 7f2602000640 1 --2- 172.21.15.50:0/186108617 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25e4074760 0x7f25e4076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:47.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.252+0000 7f2602000640 1 --2- 172.21.15.50:0/186108617 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25e4074760 0x7f25e4076c70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f25f00af000 tx=0x7f25f0019680 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:47.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.256+0000 7f25faa00640 1 -- 172.21.15.50:0/186108617 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f25f401f020 con 0x7f26040cd970 2024-09-21T06:07:47.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.604+0000 7f2609800640 1 -- 172.21.15.50:0/186108617 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone71879", "target": ["mon-mgr", ""]}) -- 0x7f25dc000bd0 con 0x7f25e4074760 2024-09-21T06:07:47.770 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:47.772+0000 7f65a0c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:47.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:47.772+0000 7f25faa00640 1 -- 172.21.15.50:0/186108617 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f25dc000bd0 con 0x7f25e4074760 2024-09-21T06:07:47.772 INFO:teuthology.orchestra.run.smithi050.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:47.786 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T06:07:47.787 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone647110 2024-09-21T06:07:48.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.052+0000 7efdf6000640 1 -- 172.21.15.50:0/1113358920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf0159130 msgr2=0x7efdf015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:48.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.052+0000 7efdf6000640 1 --2- 172.21.15.50:0/1113358920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf0159130 0x7efdf015bb90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7efde8009c80 tx=0x7efde801cca0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.052+0000 7efdf6000640 1 -- 172.21.15.50:0/1113358920 shutdown_connections 2024-09-21T06:07:48.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.052+0000 7efdf6000640 1 --2- 172.21.15.50:0/1113358920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf0159130 0x7efdf015bb90 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.052+0000 7efdf6000640 1 -- 172.21.15.50:0/1113358920 >> 172.21.15.50:0/1113358920 conn(0x7efdf00b35f0 msgr2=0x7efdf00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:48.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.052+0000 7efdf6000640 1 -- 172.21.15.50:0/1113358920 shutdown_connections 2024-09-21T06:07:48.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.052+0000 7efdf6000640 1 -- 172.21.15.50:0/1113358920 wait complete. 2024-09-21T06:07:48.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdf6000640 1 Processor -- start 2024-09-21T06:07:48.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdf6000640 1 -- start start 2024-09-21T06:07:48.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdf6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf006cdf0 0x7efdf015e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdf6000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf00703a0 0x7efdf0155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdf6000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdf0159130 0x7efdf00af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdf6000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efdf0169d40 con 0x7efdf006cdf0 2024-09-21T06:07:48.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdf6000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efdf006d5a0 con 0x7efdf00703a0 2024-09-21T06:07:48.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdf6000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7efdf0169ec0 con 0x7efdf0159130 2024-09-21T06:07:48.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdef400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf006cdf0 0x7efdf015e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdef400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf006cdf0 0x7efdf015e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46842/0 (socket says 172.21.15.50:46842) 2024-09-21T06:07:48.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdef400640 1 -- 172.21.15.50:0/2000772496 learned_addr learned my addr 172.21.15.50:0/2000772496 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:48.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdef400640 1 -- 172.21.15.50:0/2000772496 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdf0159130 msgr2=0x7efdf00af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:48.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdef400640 1 --2- 172.21.15.50:0/2000772496 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdf0159130 0x7efdf00af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdef400640 1 -- 172.21.15.50:0/2000772496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf00703a0 msgr2=0x7efdf0155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:48.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdef400640 1 --2- 172.21.15.50:0/2000772496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf00703a0 0x7efdf0155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdef400640 1 -- 172.21.15.50:0/2000772496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf01fc090 con 0x7efdf006cdf0 2024-09-21T06:07:48.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdef400640 1 --2- 172.21.15.50:0/2000772496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf006cdf0 0x7efdf015e7c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7efde8000f90 tx=0x7efde80a5950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:48.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efde7400640 1 -- 172.21.15.50:0/2000772496 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8004c50 con 0x7efdf006cdf0 2024-09-21T06:07:48.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efde7400640 1 -- 172.21.15.50:0/2000772496 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efde8005070 con 0x7efdf006cdf0 2024-09-21T06:07:48.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efdf6000640 1 -- 172.21.15.50:0/2000772496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf01fc370 con 0x7efdf006cdf0 2024-09-21T06:07:48.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.056+0000 7efde7400640 1 -- 172.21.15.50:0/2000772496 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8005420 con 0x7efdf006cdf0 2024-09-21T06:07:48.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.060+0000 7efdf6000640 1 -- 172.21.15.50:0/2000772496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf01fc930 con 0x7efdf006cdf0 2024-09-21T06:07:48.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.060+0000 7efdf6000640 1 -- 172.21.15.50:0/2000772496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf015bb90 con 0x7efdf006cdf0 2024-09-21T06:07:48.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.060+0000 7efde7400640 1 -- 172.21.15.50:0/2000772496 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7efde80055c0 con 0x7efdf006cdf0 2024-09-21T06:07:48.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.064+0000 7efde7400640 1 --2- 172.21.15.50:0/2000772496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdd0074790 0x7efdd0076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.064+0000 7efde7400640 1 -- 172.21.15.50:0/2000772496 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7efde8135ea0 con 0x7efdf006cdf0 2024-09-21T06:07:48.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.068+0000 7efdeea00640 1 --2- 172.21.15.50:0/2000772496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdd0074790 0x7efdd0076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.072+0000 7efde7400640 1 -- 172.21.15.50:0/2000772496 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efde80169b0 con 0x7efdf006cdf0 2024-09-21T06:07:48.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.076+0000 7efdeea00640 1 --2- 172.21.15.50:0/2000772496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdd0074790 0x7efdd0076ca0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efdf00cdf80 tx=0x7efddc006bb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:48.232 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:07:48.236 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:07:48.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:48.420+0000 7efdf6000640 1 -- 172.21.15.50:0/2000772496 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone647110", "target": ["mon-mgr", ""]}) -- 0x7efdf006b540 con 0x7efdd0074790 2024-09-21T06:07:50.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efde7400640 1 -- 172.21.15.50:0/2000772496 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efddc009280 con 0x7efdd0074790 2024-09-21T06:07:50.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efdf6000640 1 -- 172.21.15.50:0/2000772496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdd0074790 msgr2=0x7efdd0076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efdf6000640 1 --2- 172.21.15.50:0/2000772496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdd0074790 0x7efdd0076ca0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efdf00cdf80 tx=0x7efddc006bb0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efdf6000640 1 -- 172.21.15.50:0/2000772496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf006cdf0 msgr2=0x7efdf015e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efdf6000640 1 --2- 172.21.15.50:0/2000772496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf006cdf0 0x7efdf015e7c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7efde8000f90 tx=0x7efde80a5950 comp rx=0 tx=0).stop 2024-09-21T06:07:50.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efdf6000640 1 -- 172.21.15.50:0/2000772496 shutdown_connections 2024-09-21T06:07:50.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efdf6000640 1 --2- 172.21.15.50:0/2000772496 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efdf0159130 0x7efdf00af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efdf6000640 1 --2- 172.21.15.50:0/2000772496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf00703a0 0x7efdf0155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efdf6000640 1 --2- 172.21.15.50:0/2000772496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efdd0074790 0x7efdd0076ca0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efdf6000640 1 --2- 172.21.15.50:0/2000772496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf006cdf0 0x7efdf015e7c0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efdf6000640 1 -- 172.21.15.50:0/2000772496 >> 172.21.15.50:0/2000772496 conn(0x7efdf00b35f0 msgr2=0x7efdf006d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:50.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efdf6000640 1 -- 172.21.15.50:0/2000772496 shutdown_connections 2024-09-21T06:07:50.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.448+0000 7efdf6000640 1 -- 172.21.15.50:0/2000772496 wait complete. 2024-09-21T06:07:50.487 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone189911 2024-09-21T06:07:50.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.788+0000 7fbcffc00640 1 -- 172.21.15.50:0/1077808528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcf80ce4c0 msgr2=0x7fbcf815aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.788+0000 7fbcffc00640 1 --2- 172.21.15.50:0/1077808528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcf80ce4c0 0x7fbcf815aa50 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009900 tx=0x7fbce401c8e0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.788+0000 7fbcffc00640 1 -- 172.21.15.50:0/1077808528 shutdown_connections 2024-09-21T06:07:50.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.788+0000 7fbcffc00640 1 --2- 172.21.15.50:0/1077808528 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbcf80cd260 0x7fbcf80cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.788+0000 7fbcffc00640 1 --2- 172.21.15.50:0/1077808528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcf80ce4c0 0x7fbcf815aa50 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.788+0000 7fbcffc00640 1 -- 172.21.15.50:0/1077808528 >> 172.21.15.50:0/1077808528 conn(0x7fbcf80b2040 msgr2=0x7fbcf80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:50.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.792+0000 7fbcffc00640 1 -- 172.21.15.50:0/1077808528 shutdown_connections 2024-09-21T06:07:50.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.796+0000 7fbcffc00640 1 -- 172.21.15.50:0/1077808528 wait complete. 2024-09-21T06:07:50.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.800+0000 7fbcffc00640 1 Processor -- start 2024-09-21T06:07:50.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.800+0000 7fbcffc00640 1 -- start start 2024-09-21T06:07:50.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcffc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcf80cd260 0x7fbcf81fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcffc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcf80cdbb0 0x7fbcf815c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcffc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbcf80ce4c0 0x7fbcf815f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcffc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8167c00 con 0x7fbcf80cdbb0 2024-09-21T06:07:50.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcffc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8167a80 con 0x7fbcf80cd260 2024-09-21T06:07:50.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcffc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf8167d80 con 0x7fbcf80ce4c0 2024-09-21T06:07:50.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcfdc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbcf80ce4c0 0x7fbcf815f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcfdc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbcf80ce4c0 0x7fbcf815f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:46576/0 (socket says 172.21.15.50:46576) 2024-09-21T06:07:50.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcfdc00640 1 -- 172.21.15.50:0/2761595002 learned_addr learned my addr 172.21.15.50:0/2761595002 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:50.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcfd200640 1 --2- 172.21.15.50:0/2761595002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcf80cd260 0x7fbcf81fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcf7e00640 1 --2- 172.21.15.50:0/2761595002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcf80cdbb0 0x7fbcf815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcfdc00640 1 -- 172.21.15.50:0/2761595002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcf80cd260 msgr2=0x7fbcf81fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcfdc00640 1 --2- 172.21.15.50:0/2761595002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcf80cd260 0x7fbcf81fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcfdc00640 1 -- 172.21.15.50:0/2761595002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcf80cdbb0 msgr2=0x7fbcf815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcfdc00640 1 --2- 172.21.15.50:0/2761595002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcf80cdbb0 0x7fbcf815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.804+0000 7fbcfdc00640 1 -- 172.21.15.50:0/2761595002 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf8162360 con 0x7fbcf80ce4c0 2024-09-21T06:07:50.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.808+0000 7fbcfdc00640 1 --2- 172.21.15.50:0/2761595002 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbcf80ce4c0 0x7fbcf815f4f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000b9a0 tx=0x7fbcf000be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:50.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.808+0000 7fbcf4c00640 1 -- 172.21.15.50:0/2761595002 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0019070 con 0x7fbcf80ce4c0 2024-09-21T06:07:50.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.808+0000 7fbcf4c00640 1 -- 172.21.15.50:0/2761595002 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcf000cc90 con 0x7fbcf80ce4c0 2024-09-21T06:07:50.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.808+0000 7fbcf4c00640 1 -- 172.21.15.50:0/2761595002 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf00146a0 con 0x7fbcf80ce4c0 2024-09-21T06:07:50.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.808+0000 7fbcffc00640 1 -- 172.21.15.50:0/2761595002 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf8162600 con 0x7fbcf80ce4c0 2024-09-21T06:07:50.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.808+0000 7fbcffc00640 1 -- 172.21.15.50:0/2761595002 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf8074370 con 0x7fbcf80ce4c0 2024-09-21T06:07:50.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.808+0000 7fbcffc00640 1 -- 172.21.15.50:0/2761595002 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf80745d0 con 0x7fbcf80ce4c0 2024-09-21T06:07:50.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.816+0000 7fbcf4c00640 1 -- 172.21.15.50:0/2761595002 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7fbcf001c070 con 0x7fbcf80ce4c0 2024-09-21T06:07:50.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.816+0000 7fbcf4c00640 1 --2- 172.21.15.50:0/2761595002 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbcd80747b0 0x7fbcd8076cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.816+0000 7fbcf4c00640 1 -- 172.21.15.50:0/2761595002 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7fbcf0093360 con 0x7fbcf80ce4c0 2024-09-21T06:07:50.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.820+0000 7fbcfd200640 1 --2- 172.21.15.50:0/2761595002 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbcd80747b0 0x7fbcd8076cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.824+0000 7fbcf4c00640 1 -- 172.21.15.50:0/2761595002 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbcf005faa0 con 0x7fbcf80ce4c0 2024-09-21T06:07:50.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:50.828+0000 7fbcfd200640 1 --2- 172.21.15.50:0/2761595002 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbcd80747b0 0x7fbcd8076cc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbce40af000 tx=0x7fbce401b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:51.204+0000 7fbcffc00640 1 -- 172.21.15.50:0/2761595002 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone189911", "target": ["mon-mgr", ""]}) -- 0x7fbcf806b540 con 0x7fbcd80747b0 2024-09-21T06:07:51.767 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:07:51.768+0000 7f65a0c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:51.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:51.768+0000 7fbcf4c00640 1 -- 172.21.15.50:0/2761595002 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fbce4005de0 con 0x7fbcd80747b0 2024-09-21T06:07:51.767 INFO:teuthology.orchestra.run.smithi050.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-21T06:07:51.779 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T06:07:51.779 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone837612 2024-09-21T06:07:52.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.016+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1398533841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b00099c0 msgr2=0x7fa6b000bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:52.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.016+0000 7fa6d2e00640 1 --2- 172.21.15.50:0/1398533841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b00099c0 0x7fa6b000bde0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009c00 tx=0x7fa6c401ca20 comp rx=0 tx=0).stop 2024-09-21T06:07:52.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.016+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1398533841 shutdown_connections 2024-09-21T06:07:52.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.016+0000 7fa6d2e00640 1 --2- 172.21.15.50:0/1398533841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b00099c0 0x7fa6b000bde0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:52.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.016+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1398533841 >> 172.21.15.50:0/1398533841 conn(0x7fa6cc0b2040 msgr2=0x7fa6cc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:52.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.016+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1398533841 shutdown_connections 2024-09-21T06:07:52.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.016+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1398533841 wait complete. 2024-09-21T06:07:52.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.020+0000 7fa6d2e00640 1 Processor -- start 2024-09-21T06:07:52.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.020+0000 7fa6d2e00640 1 -- start start 2024-09-21T06:07:52.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.020+0000 7fa6d2e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6b00099c0 0x7fa6cc0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:52.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.020+0000 7fa6d2e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6cc0cec20 0x7fa6cc1f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:52.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.020+0000 7fa6d2e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6cc1f5830 0x7fa6cc1f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:52.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.020+0000 7fa6d2e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc0751c0 con 0x7fa6cc0cec20 2024-09-21T06:07:52.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.020+0000 7fa6d2e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc075040 con 0x7fa6b00099c0 2024-09-21T06:07:52.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.020+0000 7fa6d2e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa6cc075340 con 0x7fa6cc1f5830 2024-09-21T06:07:52.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.020+0000 7fa6cb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6cc0cec20 0x7fa6cc1f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:52.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.020+0000 7fa6cb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6cc0cec20 0x7fa6cc1f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46878/0 (socket says 172.21.15.50:46878) 2024-09-21T06:07:52.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.020+0000 7fa6cb400640 1 -- 172.21.15.50:0/1613696008 learned_addr learned my addr 172.21.15.50:0/1613696008 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:52.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.024+0000 7fa6cb400640 1 -- 172.21.15.50:0/1613696008 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6cc1f5830 msgr2=0x7fa6cc1f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:52.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.024+0000 7fa6cb400640 1 --2- 172.21.15.50:0/1613696008 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6cc1f5830 0x7fa6cc1f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:52.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.024+0000 7fa6cb400640 1 -- 172.21.15.50:0/1613696008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6b00099c0 msgr2=0x7fa6cc0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.024+0000 7fa6cb400640 1 --2- 172.21.15.50:0/1613696008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6b00099c0 0x7fa6cc0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.024+0000 7fa6cb400640 1 -- 172.21.15.50:0/1613696008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6cc1f4e30 con 0x7fa6cc0cec20 2024-09-21T06:07:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.024+0000 7fa6cb400640 1 --2- 172.21.15.50:0/1613696008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6cc0cec20 0x7fa6cc1f52f0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800c440 tx=0x7fa6b800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:52.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.024+0000 7fa6c3e00640 1 -- 172.21.15.50:0/1613696008 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8017070 con 0x7fa6cc0cec20 2024-09-21T06:07:52.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.024+0000 7fa6c3e00640 1 -- 172.21.15.50:0/1613696008 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6b8003600 con 0x7fa6cc0cec20 2024-09-21T06:07:52.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.024+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1613696008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6cc0cf030 con 0x7fa6cc0cec20 2024-09-21T06:07:52.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.024+0000 7fa6c3e00640 1 -- 172.21.15.50:0/1613696008 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8003a50 con 0x7fa6cc0cec20 2024-09-21T06:07:52.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.024+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1613696008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6cc14ad80 con 0x7fa6cc0cec20 2024-09-21T06:07:52.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.028+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1613696008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6cc14aaf0 con 0x7fa6cc0cec20 2024-09-21T06:07:52.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.028+0000 7fa6c3e00640 1 -- 172.21.15.50:0/1613696008 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7fa6b801a070 con 0x7fa6cc0cec20 2024-09-21T06:07:52.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.032+0000 7fa6c3e00640 1 --2- 172.21.15.50:0/1613696008 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6ac074760 0x7fa6ac076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:52.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.032+0000 7fa6c3e00640 1 -- 172.21.15.50:0/1613696008 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7fa6b8092630 con 0x7fa6cc0cec20 2024-09-21T06:07:52.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.036+0000 7fa6cbe00640 1 --2- 172.21.15.50:0/1613696008 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6ac074760 0x7fa6ac076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:52.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.044+0000 7fa6c3e00640 1 -- 172.21.15.50:0/1613696008 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa6cc1f4e30 con 0x7fa6cc0cec20 2024-09-21T06:07:52.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.056+0000 7fa6cbe00640 1 --2- 172.21.15.50:0/1613696008 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6ac074760 0x7fa6ac076c70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00adb0 tx=0x7fa6bc001420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:52.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:52.432+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1613696008 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone837612", "target": ["mon-mgr", ""]}) -- 0x7fa6cc15cd10 con 0x7fa6ac074760 2024-09-21T06:07:53.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.400+0000 7fa6c3e00640 1 -- 172.21.15.50:0/1613696008 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa6cc15cd10 con 0x7fa6ac074760 2024-09-21T06:07:53.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.404+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1613696008 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6ac074760 msgr2=0x7fa6ac076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.404+0000 7fa6d2e00640 1 --2- 172.21.15.50:0/1613696008 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6ac074760 0x7fa6ac076c70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00adb0 tx=0x7fa6bc001420 comp rx=0 tx=0).stop 2024-09-21T06:07:53.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.404+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1613696008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6cc0cec20 msgr2=0x7fa6cc1f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.404+0000 7fa6d2e00640 1 --2- 172.21.15.50:0/1613696008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6cc0cec20 0x7fa6cc1f52f0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800c440 tx=0x7fa6b800c900 comp rx=0 tx=0).stop 2024-09-21T06:07:53.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.404+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1613696008 shutdown_connections 2024-09-21T06:07:53.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.404+0000 7fa6d2e00640 1 --2- 172.21.15.50:0/1613696008 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6cc1f5830 0x7fa6cc1f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.404+0000 7fa6d2e00640 1 --2- 172.21.15.50:0/1613696008 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6ac074760 0x7fa6ac076c70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.404+0000 7fa6d2e00640 1 --2- 172.21.15.50:0/1613696008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6cc0cec20 0x7fa6cc1f52f0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.404+0000 7fa6d2e00640 1 --2- 172.21.15.50:0/1613696008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6b00099c0 0x7fa6cc0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.404+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1613696008 >> 172.21.15.50:0/1613696008 conn(0x7fa6cc0b2040 msgr2=0x7fa6cc15ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:53.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.404+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1613696008 shutdown_connections 2024-09-21T06:07:53.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.404+0000 7fa6d2e00640 1 -- 172.21.15.50:0/1613696008 wait complete. 2024-09-21T06:07:53.440 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone986813 2024-09-21T06:07:53.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.768+0000 7f09ee800640 1 -- 172.21.15.50:0/1542646259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09e8160a10 msgr2=0x7f09e8160e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.768+0000 7f09ee800640 1 --2- 172.21.15.50:0/1542646259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09e8160a10 0x7f09e8160e20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f09e80734a0 tx=0x7f09d401d2f0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.768+0000 7f09ee800640 1 -- 172.21.15.50:0/1542646259 shutdown_connections 2024-09-21T06:07:53.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.768+0000 7f09ee800640 1 --2- 172.21.15.50:0/1542646259 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f09e815cf30 0x7f09e815d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.768+0000 7f09ee800640 1 --2- 172.21.15.50:0/1542646259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09e8160a10 0x7f09e8160e20 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.768+0000 7f09ee800640 1 --2- 172.21.15.50:0/1542646259 >> v2:172.21.15.88:3300/0 conn(0x7f09e8156e60 0x7f09e81604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.768+0000 7f09ee800640 1 -- 172.21.15.50:0/1542646259 >> 172.21.15.50:0/1542646259 conn(0x7f09e809ba70 msgr2=0x7f09e80b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:53.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.768+0000 7f09ee800640 1 -- 172.21.15.50:0/1542646259 shutdown_connections 2024-09-21T06:07:53.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.768+0000 7f09ee800640 1 -- 172.21.15.50:0/1542646259 wait complete. 2024-09-21T06:07:53.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.772+0000 7f09ee800640 1 Processor -- start 2024-09-21T06:07:53.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.776+0000 7f09ee800640 1 -- start start 2024-09-21T06:07:53.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.776+0000 7f09ee800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09e8156e60 0x7f09e81f9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.776+0000 7f09ee800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09e815cf30 0x7f09e8153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.776+0000 7f09ee800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f09e8160a10 0x7f09e80d2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.776+0000 7f09ee800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f09e80b0a90 con 0x7f09e8156e60 2024-09-21T06:07:53.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.776+0000 7f09ee800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f09e80b0910 con 0x7f09e815cf30 2024-09-21T06:07:53.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.776+0000 7f09ee800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f09e80b0c10 con 0x7f09e8160a10 2024-09-21T06:07:53.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.776+0000 7f09e7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09e8156e60 0x7f09e81f9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.776+0000 7f09e7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09e8156e60 0x7f09e81f9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46890/0 (socket says 172.21.15.50:46890) 2024-09-21T06:07:53.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.776+0000 7f09e7400640 1 -- 172.21.15.50:0/1546487011 learned_addr learned my addr 172.21.15.50:0/1546487011 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:53.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.780+0000 7f09e7400640 1 -- 172.21.15.50:0/1546487011 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f09e8160a10 msgr2=0x7f09e80d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.780+0000 7f09e7400640 1 --2- 172.21.15.50:0/1546487011 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f09e8160a10 0x7f09e80d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.780+0000 7f09e7400640 1 -- 172.21.15.50:0/1546487011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09e815cf30 msgr2=0x7f09e8153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.780+0000 7f09e7400640 1 --2- 172.21.15.50:0/1546487011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09e815cf30 0x7f09e8153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.780+0000 7f09e7400640 1 -- 172.21.15.50:0/1546487011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e815a590 con 0x7f09e8156e60 2024-09-21T06:07:53.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.780+0000 7f09e7400640 1 --2- 172.21.15.50:0/1546487011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09e8156e60 0x7f09e81f9bd0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f09d800b410 tx=0x7f09d800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:53.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.780+0000 7f09df400640 1 -- 172.21.15.50:0/1546487011 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8015020 con 0x7f09e8156e60 2024-09-21T06:07:53.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.780+0000 7f09ee800640 1 -- 172.21.15.50:0/1546487011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e80ceed0 con 0x7f09e8156e60 2024-09-21T06:07:53.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.780+0000 7f09df400640 1 -- 172.21.15.50:0/1546487011 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09d8011920 con 0x7f09e8156e60 2024-09-21T06:07:53.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.780+0000 7f09df400640 1 -- 172.21.15.50:0/1546487011 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8015020 con 0x7f09e8156e60 2024-09-21T06:07:53.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.780+0000 7f09ee800640 1 -- 172.21.15.50:0/1546487011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e80cf490 con 0x7f09e8156e60 2024-09-21T06:07:53.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.784+0000 7f09ee800640 1 -- 172.21.15.50:0/1546487011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e815d340 con 0x7f09e8156e60 2024-09-21T06:07:53.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.788+0000 7f09df400640 1 -- 172.21.15.50:0/1546487011 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f09d8018020 con 0x7f09e8156e60 2024-09-21T06:07:53.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.792+0000 7f09df400640 1 --2- 172.21.15.50:0/1546487011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f09c8074960 0x7f09c8076e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.792+0000 7f09df400640 1 -- 172.21.15.50:0/1546487011 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f09d8092f00 con 0x7f09e8156e60 2024-09-21T06:07:53.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.792+0000 7f09e6a00640 1 --2- 172.21.15.50:0/1546487011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f09c8074960 0x7f09c8076e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.792+0000 7f09e6a00640 1 --2- 172.21.15.50:0/1546487011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f09c8074960 0x7f09c8076e70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f09d40af000 tx=0x7f09d4019630 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:53.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:53.800+0000 7f09df400640 1 -- 172.21.15.50:0/1546487011 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f09d801f020 con 0x7f09e8156e60 2024-09-21T06:07:54.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:54.228+0000 7f09ee800640 1 -- 172.21.15.50:0/1546487011 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone986813", "target": ["mon-mgr", ""]}) -- 0x7f09e806b540 con 0x7f09c8074960 2024-09-21T06:07:56.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.856+0000 7f09df400640 1 -- 172.21.15.50:0/1546487011 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f09d40a5a20 con 0x7f09c8074960 2024-09-21T06:07:56.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.860+0000 7f09ee800640 1 -- 172.21.15.50:0/1546487011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f09c8074960 msgr2=0x7f09c8076e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:56.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.860+0000 7f09ee800640 1 --2- 172.21.15.50:0/1546487011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f09c8074960 0x7f09c8076e70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f09d40af000 tx=0x7f09d4019630 comp rx=0 tx=0).stop 2024-09-21T06:07:56.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.860+0000 7f09ee800640 1 -- 172.21.15.50:0/1546487011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09e8156e60 msgr2=0x7f09e81f9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:56.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.860+0000 7f09ee800640 1 --2- 172.21.15.50:0/1546487011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09e8156e60 0x7f09e81f9bd0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f09d800b410 tx=0x7f09d800b8d0 comp rx=0 tx=0).stop 2024-09-21T06:07:56.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.860+0000 7f09ee800640 1 -- 172.21.15.50:0/1546487011 shutdown_connections 2024-09-21T06:07:56.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.860+0000 7f09ee800640 1 --2- 172.21.15.50:0/1546487011 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f09e8160a10 0x7f09e80d2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:56.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.860+0000 7f09ee800640 1 --2- 172.21.15.50:0/1546487011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09e815cf30 0x7f09e8153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:56.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.860+0000 7f09ee800640 1 --2- 172.21.15.50:0/1546487011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f09c8074960 0x7f09c8076e70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:56.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.860+0000 7f09ee800640 1 --2- 172.21.15.50:0/1546487011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09e8156e60 0x7f09e81f9bd0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:56.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.860+0000 7f09ee800640 1 -- 172.21.15.50:0/1546487011 >> 172.21.15.50:0/1546487011 conn(0x7f09e809ba70 msgr2=0x7f09e806d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:56.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.864+0000 7f09ee800640 1 -- 172.21.15.50:0/1546487011 shutdown_connections 2024-09-21T06:07:56.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:56.864+0000 7f09ee800640 1 -- 172.21.15.50:0/1546487011 wait complete. 2024-09-21T06:07:56.895 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone216314 2024-09-21T06:07:57.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.156+0000 7f7205200640 1 -- 172.21.15.50:0/214692908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f720006d500 msgr2=0x7f720015b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.156+0000 7f7205200640 1 --2- 172.21.15.50:0/214692908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f720006d500 0x7f720015b0b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009bc0 tx=0x7f71f801cbc0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.160+0000 7f7205200640 1 -- 172.21.15.50:0/214692908 shutdown_connections 2024-09-21T06:07:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.160+0000 7f7205200640 1 --2- 172.21.15.50:0/214692908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f720006d500 0x7f720015b0b0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.160+0000 7f7205200640 1 --2- 172.21.15.50:0/214692908 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72000cebd0 0x7f72000b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.160+0000 7f7205200640 1 -- 172.21.15.50:0/214692908 >> 172.21.15.50:0/214692908 conn(0x7f720006ee20 msgr2=0x7f72000d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.160+0000 7f7205200640 1 -- 172.21.15.50:0/214692908 shutdown_connections 2024-09-21T06:07:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.160+0000 7f7205200640 1 -- 172.21.15.50:0/214692908 wait complete. 2024-09-21T06:07:57.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.164+0000 7f7205200640 1 Processor -- start 2024-09-21T06:07:57.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.164+0000 7f7205200640 1 -- start start 2024-09-21T06:07:57.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.168+0000 7f7205200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72000b6100 0x7f72000b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.168+0000 7f7205200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72000cebd0 0x7f720014d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.168+0000 7f7205200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f720014aa20 0x7f720014d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.168+0000 7f7205200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f720015f970 con 0x7f72000cebd0 2024-09-21T06:07:57.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.168+0000 7f7205200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f720015f7f0 con 0x7f720014aa20 2024-09-21T06:07:57.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.168+0000 7f7205200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f720015faf0 con 0x7f72000b6100 2024-09-21T06:07:57.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.168+0000 7f71ff400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72000b6100 0x7f72000b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.168+0000 7f71ff400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72000b6100 0x7f72000b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:38056/0 (socket says 172.21.15.50:38056) 2024-09-21T06:07:57.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.168+0000 7f71ff400640 1 -- 172.21.15.50:0/2982203456 learned_addr learned my addr 172.21.15.50:0/2982203456 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:57.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.168+0000 7f71fea00640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72000cebd0 0x7f720014d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.168+0000 7f71ffe00640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f720014aa20 0x7f720014d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.172+0000 7f71ff400640 1 -- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f720014aa20 msgr2=0x7f720014d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.172+0000 7f71ff400640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f720014aa20 0x7f720014d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.172+0000 7f71ff400640 1 -- 172.21.15.50:0/2982203456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72000cebd0 msgr2=0x7f720014d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.176+0000 7f71ff400640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72000cebd0 0x7f720014d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.176+0000 7f71ff400640 1 -- 172.21.15.50:0/2982203456 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72000b0c10 con 0x7f72000b6100 2024-09-21T06:07:57.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.176+0000 7f71fea00640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72000cebd0 0x7f720014d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:57.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.180+0000 7f71ff400640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72000b6100 0x7f72000b06d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f71f000d950 tx=0x7f71f000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:57.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.180+0000 7f71f7400640 1 -- 172.21.15.50:0/2982203456 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0017070 con 0x7f72000b6100 2024-09-21T06:07:57.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.180+0000 7f71f7400640 1 -- 172.21.15.50:0/2982203456 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71f0009d20 con 0x7f72000b6100 2024-09-21T06:07:57.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.180+0000 7f71f7400640 1 -- 172.21.15.50:0/2982203456 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f00106c0 con 0x7f72000b6100 2024-09-21T06:07:57.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.180+0000 7f7205200640 1 -- 172.21.15.50:0/2982203456 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7200150380 con 0x7f72000b6100 2024-09-21T06:07:57.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.180+0000 7f7205200640 1 -- 172.21.15.50:0/2982203456 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f72001fbc40 con 0x7f72000b6100 2024-09-21T06:07:57.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.184+0000 7f7205200640 1 -- 172.21.15.50:0/2982203456 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720015b0b0 con 0x7f72000b6100 2024-09-21T06:07:57.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.192+0000 7f71f7400640 1 -- 172.21.15.50:0/2982203456 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f71f001a070 con 0x7f72000b6100 2024-09-21T06:07:57.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.192+0000 7f71f7400640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f71e00747e0 0x7f71e0076cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.192+0000 7f71f7400640 1 -- 172.21.15.50:0/2982203456 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f71f0093830 con 0x7f72000b6100 2024-09-21T06:07:57.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.196+0000 7f71fea00640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f71e00747e0 0x7f71e0076cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.200+0000 7f71f7400640 1 -- 172.21.15.50:0/2982203456 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f71f005f920 con 0x7f72000b6100 2024-09-21T06:07:57.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.200+0000 7f71fea00640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f71e00747e0 0x7f71e0076cf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7200151860 tx=0x7f71ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:57.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:57.560+0000 7f7205200640 1 -- 172.21.15.50:0/2982203456 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone216314", "target": ["mon-mgr", ""]}) -- 0x7f720006b540 con 0x7f71e00747e0 2024-09-21T06:07:58.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.380+0000 7f71f7400640 1 -- 172.21.15.50:0/2982203456 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f71ec0013b0 con 0x7f71e00747e0 2024-09-21T06:07:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.388+0000 7f7205200640 1 -- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f71e00747e0 msgr2=0x7f71e0076cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.388+0000 7f7205200640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f71e00747e0 0x7f71e0076cf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7200151860 tx=0x7f71ec009040 comp rx=0 tx=0).stop 2024-09-21T06:07:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.388+0000 7f7205200640 1 -- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72000b6100 msgr2=0x7f72000b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.388+0000 7f7205200640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72000b6100 0x7f72000b06d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f71f000d950 tx=0x7f71f000de10 comp rx=0 tx=0).stop 2024-09-21T06:07:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.388+0000 7f7205200640 1 -- 172.21.15.50:0/2982203456 shutdown_connections 2024-09-21T06:07:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.388+0000 7f7205200640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f720014aa20 0x7f720014d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.388+0000 7f7205200640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f71e00747e0 0x7f71e0076cf0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.388+0000 7f7205200640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72000cebd0 0x7f720014d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.388+0000 7f7205200640 1 --2- 172.21.15.50:0/2982203456 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f72000b6100 0x7f72000b06d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.388+0000 7f7205200640 1 -- 172.21.15.50:0/2982203456 >> 172.21.15.50:0/2982203456 conn(0x7f720006ee20 msgr2=0x7f72001599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.388+0000 7f7205200640 1 -- 172.21.15.50:0/2982203456 shutdown_connections 2024-09-21T06:07:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.388+0000 7f7205200640 1 -- 172.21.15.50:0/2982203456 wait complete. 2024-09-21T06:07:58.414 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone877615 2024-09-21T06:07:58.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.660+0000 7f7d5b600640 1 -- 172.21.15.50:0/3389503222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d541527b0 msgr2=0x7f7d54074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:58.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.660+0000 7f7d5b600640 1 --2- 172.21.15.50:0/3389503222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d541527b0 0x7f7d54074e30 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009900 tx=0x7f7d4001c820 comp rx=0 tx=0).stop 2024-09-21T06:07:58.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.660+0000 7f7d5b600640 1 -- 172.21.15.50:0/3389503222 shutdown_connections 2024-09-21T06:07:58.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.660+0000 7f7d5b600640 1 --2- 172.21.15.50:0/3389503222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d541527b0 0x7f7d54074e30 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.660+0000 7f7d5b600640 1 --2- 172.21.15.50:0/3389503222 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7d540b5c70 0x7f7d5415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.660+0000 7f7d5b600640 1 -- 172.21.15.50:0/3389503222 >> 172.21.15.50:0/3389503222 conn(0x7f7d540b2040 msgr2=0x7f7d540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:58.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.660+0000 7f7d5b600640 1 -- 172.21.15.50:0/3389503222 shutdown_connections 2024-09-21T06:07:58.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.660+0000 7f7d5b600640 1 -- 172.21.15.50:0/3389503222 wait complete. 2024-09-21T06:07:58.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.664+0000 7f7d5b600640 1 Processor -- start 2024-09-21T06:07:58.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.664+0000 7f7d5b600640 1 -- start start 2024-09-21T06:07:58.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d5b600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d540b4de0 0x7f7d540cead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:58.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d5b600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b5c70 0x7f7d54149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:58.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d5b600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7d541527b0 0x7f7d540ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:58.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d5b600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7d54168ca0 con 0x7f7d540b4de0 2024-09-21T06:07:58.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d5b600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7d54168b20 con 0x7f7d540b5c70 2024-09-21T06:07:58.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d5b600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7d54168e20 con 0x7f7d541527b0 2024-09-21T06:07:58.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d58c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d540b4de0 0x7f7d540cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:58.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d58c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d540b4de0 0x7f7d540cead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35050/0 (socket says 172.21.15.50:35050) 2024-09-21T06:07:58.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d58c00640 1 -- 172.21.15.50:0/3927220806 learned_addr learned my addr 172.21.15.50:0/3927220806 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:07:58.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d58c00640 1 -- 172.21.15.50:0/3927220806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7d541527b0 msgr2=0x7f7d540ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:58.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d58c00640 1 --2- 172.21.15.50:0/3927220806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7d541527b0 0x7f7d540ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d58c00640 1 -- 172.21.15.50:0/3927220806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b5c70 msgr2=0x7f7d54149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:58.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d58c00640 1 --2- 172.21.15.50:0/3927220806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b5c70 0x7f7d54149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d58c00640 1 -- 172.21.15.50:0/3927220806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d541fc970 con 0x7f7d540b4de0 2024-09-21T06:07:58.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.668+0000 7f7d58c00640 1 --2- 172.21.15.50:0/3927220806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d540b4de0 0x7f7d540cead0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800b410 tx=0x7f7d4800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:58.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.672+0000 7f7d50c00640 1 -- 172.21.15.50:0/3927220806 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48015020 con 0x7f7d540b4de0 2024-09-21T06:07:58.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.672+0000 7f7d50c00640 1 -- 172.21.15.50:0/3927220806 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d48011920 con 0x7f7d540b4de0 2024-09-21T06:07:58.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.672+0000 7f7d5b600640 1 -- 172.21.15.50:0/3927220806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d541fcc50 con 0x7f7d540b4de0 2024-09-21T06:07:58.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.672+0000 7f7d50c00640 1 -- 172.21.15.50:0/3927220806 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48011d70 con 0x7f7d540b4de0 2024-09-21T06:07:58.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.672+0000 7f7d5b600640 1 -- 172.21.15.50:0/3927220806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d54156ee0 con 0x7f7d540b4de0 2024-09-21T06:07:58.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.672+0000 7f7d5b600640 1 -- 172.21.15.50:0/3927220806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d5415bb60 con 0x7f7d540b4de0 2024-09-21T06:07:58.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.676+0000 7f7d50c00640 1 -- 172.21.15.50:0/3927220806 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f7d48018020 con 0x7f7d540b4de0 2024-09-21T06:07:58.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.680+0000 7f7d50c00640 1 --2- 172.21.15.50:0/3927220806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7d34074790 0x7f7d34076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:58.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.680+0000 7f7d50c00640 1 -- 172.21.15.50:0/3927220806 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f7d48093280 con 0x7f7d540b4de0 2024-09-21T06:07:58.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.680+0000 7f7d53e00640 1 --2- 172.21.15.50:0/3927220806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7d34074790 0x7f7d34076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:58.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.688+0000 7f7d50c00640 1 -- 172.21.15.50:0/3927220806 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7d4801f020 con 0x7f7d540b4de0 2024-09-21T06:07:58.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:58.688+0000 7f7d53e00640 1 --2- 172.21.15.50:0/3927220806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7d34074790 0x7f7d34076ca0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7d400af000 tx=0x7f7d400195c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:59.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:07:59.040+0000 7f7d5b600640 1 -- 172.21.15.50:0/3927220806 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone877615", "target": ["mon-mgr", ""]}) -- 0x7f7d5406b540 con 0x7f7d34074790 2024-09-21T06:08:01.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.012+0000 7f7d50c00640 1 -- 172.21.15.50:0/3927220806 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7d40004020 con 0x7f7d34074790 2024-09-21T06:08:01.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.016+0000 7f7d5b600640 1 -- 172.21.15.50:0/3927220806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7d34074790 msgr2=0x7f7d34076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:01.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.016+0000 7f7d5b600640 1 --2- 172.21.15.50:0/3927220806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7d34074790 0x7f7d34076ca0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7d400af000 tx=0x7f7d400195c0 comp rx=0 tx=0).stop 2024-09-21T06:08:01.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.016+0000 7f7d5b600640 1 -- 172.21.15.50:0/3927220806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d540b4de0 msgr2=0x7f7d540cead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:01.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.016+0000 7f7d5b600640 1 --2- 172.21.15.50:0/3927220806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d540b4de0 0x7f7d540cead0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800b410 tx=0x7f7d4800b8d0 comp rx=0 tx=0).stop 2024-09-21T06:08:01.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.016+0000 7f7d5b600640 1 -- 172.21.15.50:0/3927220806 shutdown_connections 2024-09-21T06:08:01.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.016+0000 7f7d5b600640 1 --2- 172.21.15.50:0/3927220806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7d541527b0 0x7f7d540ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:01.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.016+0000 7f7d5b600640 1 --2- 172.21.15.50:0/3927220806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b5c70 0x7f7d54149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:01.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.016+0000 7f7d5b600640 1 --2- 172.21.15.50:0/3927220806 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7d34074790 0x7f7d34076ca0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:01.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.016+0000 7f7d5b600640 1 --2- 172.21.15.50:0/3927220806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d540b4de0 0x7f7d540cead0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:01.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.016+0000 7f7d5b600640 1 -- 172.21.15.50:0/3927220806 >> 172.21.15.50:0/3927220806 conn(0x7f7d540b2040 msgr2=0x7f7d540b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:01.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.016+0000 7f7d5b600640 1 -- 172.21.15.50:0/3927220806 shutdown_connections 2024-09-21T06:08:01.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.016+0000 7f7d5b600640 1 -- 172.21.15.50:0/3927220806 wait complete. 2024-09-21T06:08:01.063 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone832116 2024-09-21T06:08:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.340+0000 7fc2e7e00640 1 -- 172.21.15.50:0/3003034481 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2e0036220 msgr2=0x7fc2e0036630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.340+0000 7fc2e7e00640 1 --2- 172.21.15.50:0/3003034481 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2e0036220 0x7fc2e0036630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009870 tx=0x7fc2d401c610 comp rx=0 tx=0).stop 2024-09-21T06:08:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.340+0000 7fc2e7e00640 1 -- 172.21.15.50:0/3003034481 shutdown_connections 2024-09-21T06:08:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.340+0000 7fc2e7e00640 1 --2- 172.21.15.50:0/3003034481 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2e0036220 0x7fc2e0036630 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.340+0000 7fc2e7e00640 1 --2- 172.21.15.50:0/3003034481 >> v2:172.21.15.88:3300/0 conn(0x7fc2e000b010 0x7fc2e000a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.340+0000 7fc2e7e00640 1 -- 172.21.15.50:0/3003034481 >> 172.21.15.50:0/3003034481 conn(0x7fc2e00513f0 msgr2=0x7fc2e0051830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:01.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.340+0000 7fc2e7e00640 1 -- 172.21.15.50:0/3003034481 shutdown_connections 2024-09-21T06:08:01.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.340+0000 7fc2e7e00640 1 -- 172.21.15.50:0/3003034481 wait complete. 2024-09-21T06:08:01.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.344+0000 7fc2e7e00640 1 Processor -- start 2024-09-21T06:08:01.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.344+0000 7fc2e7e00640 1 -- start start 2024-09-21T06:08:01.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2e0005cc0 0x7fc2e0035600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:01.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2e000b010 0x7fc2e00dd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:01.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e7e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2e0036220 0x7fc2e00de070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:01.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e7e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e001c950 con 0x7fc2e0005cc0 2024-09-21T06:08:01.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e7e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e001c7d0 con 0x7fc2e000b010 2024-09-21T06:08:01.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e7e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e001cad0 con 0x7fc2e0036220 2024-09-21T06:08:01.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2e0005cc0 0x7fc2e0035600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:01.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2e0005cc0 0x7fc2e0035600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35054/0 (socket says 172.21.15.50:35054) 2024-09-21T06:08:01.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e6a00640 1 -- 172.21.15.50:0/669979297 learned_addr learned my addr 172.21.15.50:0/669979297 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:01.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e6000640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2e000b010 0x7fc2e00dd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:01.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e6a00640 1 -- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2e0036220 msgr2=0x7fc2e00de070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:01.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e6a00640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2e0036220 0x7fc2e00de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:01.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e6a00640 1 -- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2e000b010 msgr2=0x7fc2e00dd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:01.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e6a00640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2e000b010 0x7fc2e00dd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:01.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e6a00640 1 -- 172.21.15.50:0/669979297 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e002b640 con 0x7fc2e0005cc0 2024-09-21T06:08:01.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e6000640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2e000b010 0x7fc2e00dd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:01.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e6a00640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2e0005cc0 0x7fc2e0035600 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800e9f0 tx=0x7fc2d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:01.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2dea00640 1 -- 172.21.15.50:0/669979297 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d800cd80 con 0x7fc2e0005cc0 2024-09-21T06:08:01.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2dea00640 1 -- 172.21.15.50:0/669979297 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2d8010430 con 0x7fc2e0005cc0 2024-09-21T06:08:01.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e7e00640 1 -- 172.21.15.50:0/669979297 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e002b920 con 0x7fc2e0005cc0 2024-09-21T06:08:01.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2dea00640 1 -- 172.21.15.50:0/669979297 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8010880 con 0x7fc2e0005cc0 2024-09-21T06:08:01.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.348+0000 7fc2e7e00640 1 -- 172.21.15.50:0/669979297 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e004f310 con 0x7fc2e0005cc0 2024-09-21T06:08:01.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.352+0000 7fc2e7e00640 1 -- 172.21.15.50:0/669979297 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e004f570 con 0x7fc2e0005cc0 2024-09-21T06:08:01.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.352+0000 7fc2dea00640 1 -- 172.21.15.50:0/669979297 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7fc2d8010a20 con 0x7fc2e0005cc0 2024-09-21T06:08:01.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.352+0000 7fc2dea00640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc2c8074760 0x7fc2c8076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:01.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.352+0000 7fc2dea00640 1 -- 172.21.15.50:0/669979297 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7fc2d8093bd0 con 0x7fc2e0005cc0 2024-09-21T06:08:01.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.356+0000 7fc2e6000640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc2c8074760 0x7fc2c8076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:01.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.360+0000 7fc2dea00640 1 -- 172.21.15.50:0/669979297 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc2d80606e0 con 0x7fc2e0005cc0 2024-09-21T06:08:01.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.368+0000 7fc2e6000640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc2c8074760 0x7fc2c8076c70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40ae000 tx=0x7fc2d401ae50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:01.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:01.712+0000 7fc2e7e00640 1 -- 172.21.15.50:0/669979297 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone832116", "target": ["mon-mgr", ""]}) -- 0x7fc2e002baf0 con 0x7fc2c8074760 2024-09-21T06:08:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.696+0000 7fc2dea00640 1 -- 172.21.15.50:0/669979297 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc2e002baf0 con 0x7fc2c8074760 2024-09-21T06:08:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.704+0000 7fc2e7e00640 1 -- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc2c8074760 msgr2=0x7fc2c8076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.704+0000 7fc2e7e00640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc2c8074760 0x7fc2c8076c70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40ae000 tx=0x7fc2d401ae50 comp rx=0 tx=0).stop 2024-09-21T06:08:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.704+0000 7fc2e7e00640 1 -- 172.21.15.50:0/669979297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2e0005cc0 msgr2=0x7fc2e0035600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.704+0000 7fc2e7e00640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2e0005cc0 0x7fc2e0035600 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800e9f0 tx=0x7fc2d800edf0 comp rx=0 tx=0).stop 2024-09-21T06:08:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.708+0000 7fc2e7e00640 1 -- 172.21.15.50:0/669979297 shutdown_connections 2024-09-21T06:08:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.708+0000 7fc2e7e00640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc2e0036220 0x7fc2e00de070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.708+0000 7fc2e7e00640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2e000b010 0x7fc2e00dd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.708+0000 7fc2e7e00640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc2c8074760 0x7fc2c8076c70 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.708+0000 7fc2e7e00640 1 --2- 172.21.15.50:0/669979297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2e0005cc0 0x7fc2e0035600 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.708+0000 7fc2e7e00640 1 -- 172.21.15.50:0/669979297 >> 172.21.15.50:0/669979297 conn(0x7fc2e00513f0 msgr2=0x7fc2e004a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.712+0000 7fc2e7e00640 1 -- 172.21.15.50:0/669979297 shutdown_connections 2024-09-21T06:08:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:02.712+0000 7fc2e7e00640 1 -- 172.21.15.50:0/669979297 wait complete. 2024-09-21T06:08:02.756 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone935917 2024-09-21T06:08:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.044+0000 7f666c000640 1 -- 172.21.15.50:0/3295484715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66640cdd80 msgr2=0x7f66640b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.044+0000 7f666c000640 1 --2- 172.21.15.50:0/3295484715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66640cdd80 0x7f66640b3880 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f665c00bd50 tx=0x7f665c01e930 comp rx=0 tx=0).stop 2024-09-21T06:08:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.044+0000 7f666c000640 1 -- 172.21.15.50:0/3295484715 shutdown_connections 2024-09-21T06:08:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.044+0000 7f666c000640 1 --2- 172.21.15.50:0/3295484715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66640cdd80 0x7f66640b3880 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.044+0000 7f666c000640 1 --2- 172.21.15.50:0/3295484715 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f66640cd970 0x7f66640b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.044+0000 7f666c000640 1 -- 172.21.15.50:0/3295484715 >> 172.21.15.50:0/3295484715 conn(0x7f66640b2040 msgr2=0x7f66640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.044+0000 7f666c000640 1 -- 172.21.15.50:0/3295484715 shutdown_connections 2024-09-21T06:08:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.044+0000 7f666c000640 1 -- 172.21.15.50:0/3295484715 wait complete. 2024-09-21T06:08:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f666c000640 1 Processor -- start 2024-09-21T06:08:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f666c000640 1 -- start start 2024-09-21T06:08:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f666c000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66640cd970 0x7f66640ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f666c000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66640cdd80 0x7f66640aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f666c000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f666415aa50 0x7f66641648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f666c000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6664168130 con 0x7f66640cd970 2024-09-21T06:08:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f666c000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6664167fb0 con 0x7f66640cdd80 2024-09-21T06:08:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f666c000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f66641682b0 con 0x7f666415aa50 2024-09-21T06:08:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f6669600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66640cd970 0x7f66640ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f6669600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66640cd970 0x7f66640ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35080/0 (socket says 172.21.15.50:35080) 2024-09-21T06:08:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f6669600640 1 -- 172.21.15.50:0/666426517 learned_addr learned my addr 172.21.15.50:0/666426517 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f6669600640 1 -- 172.21.15.50:0/666426517 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f666415aa50 msgr2=0x7f66641648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:03.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f6669600640 1 --2- 172.21.15.50:0/666426517 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f666415aa50 0x7f66641648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:03.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f6669600640 1 -- 172.21.15.50:0/666426517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66640cdd80 msgr2=0x7f66640aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:03.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f6669600640 1 --2- 172.21.15.50:0/666426517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66640cdd80 0x7f66640aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:03.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f6669600640 1 -- 172.21.15.50:0/666426517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66641fc970 con 0x7f66640cd970 2024-09-21T06:08:03.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.048+0000 7f6669600640 1 --2- 172.21.15.50:0/666426517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66640cd970 0x7f66640ae8e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f665800c400 tx=0x7f665800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:03.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.052+0000 7f6661600640 1 -- 172.21.15.50:0/666426517 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658011790 con 0x7f66640cd970 2024-09-21T06:08:03.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.052+0000 7f6661600640 1 -- 172.21.15.50:0/666426517 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6658017070 con 0x7f66640cd970 2024-09-21T06:08:03.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.052+0000 7f6661600640 1 -- 172.21.15.50:0/666426517 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658011cc0 con 0x7f66640cd970 2024-09-21T06:08:03.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.052+0000 7f666c000640 1 -- 172.21.15.50:0/666426517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66641fcc50 con 0x7f66640cd970 2024-09-21T06:08:03.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.052+0000 7f666c000640 1 -- 172.21.15.50:0/666426517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66641fd210 con 0x7f66640cd970 2024-09-21T06:08:03.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.052+0000 7f666c000640 1 -- 172.21.15.50:0/666426517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66640b0e10 con 0x7f66640cd970 2024-09-21T06:08:03.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.052+0000 7f6661600640 1 -- 172.21.15.50:0/666426517 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f665801c460 con 0x7f66640cd970 2024-09-21T06:08:03.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.056+0000 7f6661600640 1 --2- 172.21.15.50:0/666426517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6644074790 0x7f6644076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:03.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.056+0000 7f6661600640 1 -- 172.21.15.50:0/666426517 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f66580930e0 con 0x7f66640cd970 2024-09-21T06:08:03.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.064+0000 7f6661600640 1 -- 172.21.15.50:0/666426517 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f665805f830 con 0x7f66640cd970 2024-09-21T06:08:03.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.076+0000 7f6668c00640 1 --2- 172.21.15.50:0/666426517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6644074790 0x7f6644076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:03.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.084+0000 7f6668c00640 1 --2- 172.21.15.50:0/666426517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6644074790 0x7f6644076ca0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f66500058d0 tx=0x7f6650005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:03.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:03.444+0000 7f666c000640 1 -- 172.21.15.50:0/666426517 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone935917", "target": ["mon-mgr", ""]}) -- 0x7f66641fc3a0 con 0x7f6644074790 2024-09-21T06:08:04.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.127+0000 7f6661600640 1 -- 172.21.15.50:0/666426517 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f66500096c0 con 0x7f6644074790 2024-09-21T06:08:04.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.131+0000 7f666c000640 1 -- 172.21.15.50:0/666426517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6644074790 msgr2=0x7f6644076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.131+0000 7f666c000640 1 --2- 172.21.15.50:0/666426517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6644074790 0x7f6644076ca0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f66500058d0 tx=0x7f6650005860 comp rx=0 tx=0).stop 2024-09-21T06:08:04.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.131+0000 7f666c000640 1 -- 172.21.15.50:0/666426517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66640cd970 msgr2=0x7f66640ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.131+0000 7f666c000640 1 --2- 172.21.15.50:0/666426517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66640cd970 0x7f66640ae8e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f665800c400 tx=0x7f665800c8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.135+0000 7f666c000640 1 -- 172.21.15.50:0/666426517 shutdown_connections 2024-09-21T06:08:04.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.135+0000 7f666c000640 1 --2- 172.21.15.50:0/666426517 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f666415aa50 0x7f66641648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.135+0000 7f666c000640 1 --2- 172.21.15.50:0/666426517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66640cdd80 0x7f66640aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.135+0000 7f666c000640 1 --2- 172.21.15.50:0/666426517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6644074790 0x7f6644076ca0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.135+0000 7f666c000640 1 --2- 172.21.15.50:0/666426517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66640cd970 0x7f66640ae8e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.135+0000 7f666c000640 1 -- 172.21.15.50:0/666426517 >> 172.21.15.50:0/666426517 conn(0x7f66640b2040 msgr2=0x7f6664151db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:04.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.139+0000 7f666c000640 1 -- 172.21.15.50:0/666426517 shutdown_connections 2024-09-21T06:08:04.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.139+0000 7f666c000640 1 -- 172.21.15.50:0/666426517 wait complete. 2024-09-21T06:08:04.168 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone972718 2024-09-21T06:08:04.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.427+0000 7f671fe00640 1 -- 172.21.15.50:0/3516315920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6704004810 msgr2=0x7f6704004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.427+0000 7f671fe00640 1 --2- 172.21.15.50:0/3516315920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6704004810 0x7f6704004c20 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f670c009940 tx=0x7f670c01c720 comp rx=0 tx=0).stop 2024-09-21T06:08:04.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.427+0000 7f671fe00640 1 -- 172.21.15.50:0/3516315920 shutdown_connections 2024-09-21T06:08:04.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.427+0000 7f671fe00640 1 --2- 172.21.15.50:0/3516315920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6704004810 0x7f6704004c20 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.427+0000 7f671fe00640 1 -- 172.21.15.50:0/3516315920 >> 172.21.15.50:0/3516315920 conn(0x7f67200b41f0 msgr2=0x7f67200b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:04.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.427+0000 7f671fe00640 1 -- 172.21.15.50:0/3516315920 shutdown_connections 2024-09-21T06:08:04.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.427+0000 7f671fe00640 1 -- 172.21.15.50:0/3516315920 wait complete. 2024-09-21T06:08:04.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.427+0000 7f671fe00640 1 Processor -- start 2024-09-21T06:08:04.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.427+0000 7f671fe00640 1 -- start start 2024-09-21T06:08:04.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6704004810 0x7f6720157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671fe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f672009ee10 0x7f67200cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67200cfec0 0x7f67201eb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f67200ce460 con 0x7f6704004810 2024-09-21T06:08:04.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f67200ce2e0 con 0x7f67200cfec0 2024-09-21T06:08:04.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671fe00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f67200ce5e0 con 0x7f672009ee10 2024-09-21T06:08:04.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6704004810 0x7f6720157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6704004810 0x7f6720157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35090/0 (socket says 172.21.15.50:35090) 2024-09-21T06:08:04.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671ea00640 1 -- 172.21.15.50:0/3193276986 learned_addr learned my addr 172.21.15.50:0/3193276986 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:04.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671f400640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67200cfec0 0x7f67201eb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671ea00640 1 -- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f672009ee10 msgr2=0x7f67200cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671ea00640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f672009ee10 0x7f67200cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671ea00640 1 -- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67200cfec0 msgr2=0x7f67201eb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671ea00640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67200cfec0 0x7f67201eb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671ea00640 1 -- 172.21.15.50:0/3193276986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67201ea510 con 0x7f6704004810 2024-09-21T06:08:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671ea00640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6704004810 0x7f6720157f40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f671000c700 tx=0x7f671000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f6716a00640 1 -- 172.21.15.50:0/3193276986 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710017070 con 0x7f6704004810 2024-09-21T06:08:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671fe00640 1 -- 172.21.15.50:0/3193276986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67200d02d0 con 0x7f6704004810 2024-09-21T06:08:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f6716a00640 1 -- 172.21.15.50:0/3193276986 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f671000ce10 con 0x7f6704004810 2024-09-21T06:08:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f671fe00640 1 -- 172.21.15.50:0/3193276986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67201ea780 con 0x7f6704004810 2024-09-21T06:08:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.431+0000 7f6716a00640 1 -- 172.21.15.50:0/3193276986 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710003860 con 0x7f6704004810 2024-09-21T06:08:04.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.435+0000 7f671fe00640 1 -- 172.21.15.50:0/3193276986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67200d1190 con 0x7f6704004810 2024-09-21T06:08:04.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.435+0000 7f6716a00640 1 -- 172.21.15.50:0/3193276986 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f671001a070 con 0x7f6704004810 2024-09-21T06:08:04.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.435+0000 7f6716a00640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6700074790 0x7f6700076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.435+0000 7f6716a00640 1 -- 172.21.15.50:0/3193276986 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f6710092900 con 0x7f6704004810 2024-09-21T06:08:04.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.443+0000 7f6716a00640 1 -- 172.21.15.50:0/3193276986 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f67200d02d0 con 0x7f6704004810 2024-09-21T06:08:04.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.443+0000 7f671e000640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6700074790 0x7f6700076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.443+0000 7f671e000640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6700074790 0x7f6700076ca0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f670c0ae000 tx=0x7f6704005840 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:04.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:04.779+0000 7f671fe00640 1 -- 172.21.15.50:0/3193276986 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone972718", "target": ["mon-mgr", ""]}) -- 0x7f672006b540 con 0x7f6700074790 2024-09-21T06:08:05.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.415+0000 7f6716a00640 1 -- 172.21.15.50:0/3193276986 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f670c0b1450 con 0x7f6700074790 2024-09-21T06:08:05.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.419+0000 7f671fe00640 1 -- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6700074790 msgr2=0x7f6700076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:05.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.419+0000 7f671fe00640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6700074790 0x7f6700076ca0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f670c0ae000 tx=0x7f6704005840 comp rx=0 tx=0).stop 2024-09-21T06:08:05.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.419+0000 7f671fe00640 1 -- 172.21.15.50:0/3193276986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6704004810 msgr2=0x7f6720157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:05.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.419+0000 7f671fe00640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6704004810 0x7f6720157f40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f671000c700 tx=0x7f671000cbc0 comp rx=0 tx=0).stop 2024-09-21T06:08:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.427+0000 7f671fe00640 1 -- 172.21.15.50:0/3193276986 shutdown_connections 2024-09-21T06:08:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.427+0000 7f671fe00640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67200cfec0 0x7f67201eb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.427+0000 7f671fe00640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f672009ee10 0x7f67200cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.427+0000 7f671fe00640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6700074790 0x7f6700076ca0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.427+0000 7f671fe00640 1 --2- 172.21.15.50:0/3193276986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6704004810 0x7f6720157f40 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.427+0000 7f671fe00640 1 -- 172.21.15.50:0/3193276986 >> 172.21.15.50:0/3193276986 conn(0x7f67200b41f0 msgr2=0x7f672015ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.427+0000 7f671fe00640 1 -- 172.21.15.50:0/3193276986 shutdown_connections 2024-09-21T06:08:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.427+0000 7f671fe00640 1 -- 172.21.15.50:0/3193276986 wait complete. 2024-09-21T06:08:05.477 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone564119 2024-09-21T06:08:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.783+0000 7faeffe00640 1 -- 172.21.15.50:0/3217787621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faef80ccca0 msgr2=0x7faef816f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.783+0000 7faeffe00640 1 --2- 172.21.15.50:0/3217787621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faef80ccca0 0x7faef816f6a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7faef8071cb0 tx=0x7faee401d880 comp rx=0 tx=0).stop 2024-09-21T06:08:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.783+0000 7faeffe00640 1 -- 172.21.15.50:0/3217787621 shutdown_connections 2024-09-21T06:08:05.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.783+0000 7faeffe00640 1 --2- 172.21.15.50:0/3217787621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faef80ccca0 0x7faef816f6a0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:05.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.783+0000 7faeffe00640 1 --2- 172.21.15.50:0/3217787621 >> v2:172.21.15.88:3300/0 conn(0x7faef816b7c0 0x7faef816f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:05.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.783+0000 7faeffe00640 1 -- 172.21.15.50:0/3217787621 >> 172.21.15.50:0/3217787621 conn(0x7faef806e1d0 msgr2=0x7faef806e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:05.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.783+0000 7faeffe00640 1 -- 172.21.15.50:0/3217787621 shutdown_connections 2024-09-21T06:08:05.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.783+0000 7faeffe00640 1 -- 172.21.15.50:0/3217787621 wait complete. 2024-09-21T06:08:05.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.787+0000 7faeffe00640 1 Processor -- start 2024-09-21T06:08:05.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.787+0000 7faeffe00640 1 -- start start 2024-09-21T06:08:05.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faeffe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faef80ccca0 0x7faef8161460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:05.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faeffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faef816b7c0 0x7faef820aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:05.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faeffe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faef816fbe0 0x7faef820afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:05.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faeffe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faef8178ae0 con 0x7faef816b7c0 2024-09-21T06:08:05.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faeffe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faef8178960 con 0x7faef80ccca0 2024-09-21T06:08:05.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faeffe00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7faef8178c60 con 0x7faef816fbe0 2024-09-21T06:08:05.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faefca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faef816b7c0 0x7faef820aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:05.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faefca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faef816b7c0 0x7faef820aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35100/0 (socket says 172.21.15.50:35100) 2024-09-21T06:08:05.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faefca00640 1 -- 172.21.15.50:0/3981603273 learned_addr learned my addr 172.21.15.50:0/3981603273 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:05.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faefd400640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faef80ccca0 0x7faef8161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:05.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faefde00640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faef816fbe0 0x7faef820afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:05.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faefca00640 1 -- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faef816fbe0 msgr2=0x7faef820afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:05.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faefca00640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faef816fbe0 0x7faef820afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:05.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faefca00640 1 -- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faef80ccca0 msgr2=0x7faef8161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:05.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faefca00640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faef80ccca0 0x7faef8161460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:05.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faefca00640 1 -- 172.21.15.50:0/3981603273 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef80ab290 con 0x7faef816b7c0 2024-09-21T06:08:05.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faefca00640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faef816b7c0 0x7faef820aa80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7faee4009850 tx=0x7faee4005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:05.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faef5600640 1 -- 172.21.15.50:0/3981603273 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee40b3070 con 0x7faef816b7c0 2024-09-21T06:08:05.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faef5600640 1 -- 172.21.15.50:0/3981603273 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faee4004330 con 0x7faef816b7c0 2024-09-21T06:08:05.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faeffe00640 1 -- 172.21.15.50:0/3981603273 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef80ab570 con 0x7faef816b7c0 2024-09-21T06:08:05.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faef5600640 1 -- 172.21.15.50:0/3981603273 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4005280 con 0x7faef816b7c0 2024-09-21T06:08:05.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faeffe00640 1 -- 172.21.15.50:0/3981603273 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef8208650 con 0x7faef816b7c0 2024-09-21T06:08:05.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.791+0000 7faeffe00640 1 -- 172.21.15.50:0/3981603273 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef80b3340 con 0x7faef816b7c0 2024-09-21T06:08:05.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.795+0000 7faef5600640 1 -- 172.21.15.50:0/3981603273 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7faee40b7070 con 0x7faef816b7c0 2024-09-21T06:08:05.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.795+0000 7faef5600640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faed8074760 0x7faed8076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:05.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.795+0000 7faef5600640 1 -- 172.21.15.50:0/3981603273 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7faee412d230 con 0x7faef816b7c0 2024-09-21T06:08:05.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.799+0000 7faef5600640 1 -- 172.21.15.50:0/3981603273 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7faee40175c0 con 0x7faef816b7c0 2024-09-21T06:08:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.811+0000 7faefd400640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faed8074760 0x7faed8076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:05.815+0000 7faefd400640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faed8074760 0x7faed8076c70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faeec009b40 tx=0x7faeec009630 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:06.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.139+0000 7faeffe00640 1 -- 172.21.15.50:0/3981603273 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone564119", "target": ["mon-mgr", ""]}) -- 0x7faef806b540 con 0x7faed8074760 2024-09-21T06:08:06.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.295+0000 7faef5600640 1 -- 172.21.15.50:0/3981603273 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faeec011b80 con 0x7faed8074760 2024-09-21T06:08:06.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.299+0000 7faeffe00640 1 -- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faed8074760 msgr2=0x7faed8076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:06.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.299+0000 7faeffe00640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faed8074760 0x7faed8076c70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faeec009b40 tx=0x7faeec009630 comp rx=0 tx=0).stop 2024-09-21T06:08:06.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.299+0000 7faeffe00640 1 -- 172.21.15.50:0/3981603273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faef816b7c0 msgr2=0x7faef820aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:06.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.299+0000 7faeffe00640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faef816b7c0 0x7faef820aa80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7faee4009850 tx=0x7faee4005e70 comp rx=0 tx=0).stop 2024-09-21T06:08:06.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.299+0000 7faeffe00640 1 -- 172.21.15.50:0/3981603273 shutdown_connections 2024-09-21T06:08:06.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.299+0000 7faeffe00640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faef816fbe0 0x7faef820afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.299+0000 7faeffe00640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faed8074760 0x7faed8076c70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.299+0000 7faeffe00640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faef816b7c0 0x7faef820aa80 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.299+0000 7faeffe00640 1 --2- 172.21.15.50:0/3981603273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faef80ccca0 0x7faef8161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.299+0000 7faeffe00640 1 -- 172.21.15.50:0/3981603273 >> 172.21.15.50:0/3981603273 conn(0x7faef806e1d0 msgr2=0x7faef816bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:06.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.299+0000 7faeffe00640 1 -- 172.21.15.50:0/3981603273 shutdown_connections 2024-09-21T06:08:06.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.299+0000 7faeffe00640 1 -- 172.21.15.50:0/3981603273 wait complete. 2024-09-21T06:08:06.318 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:08:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.479+0000 7f77fe200640 1 -- 172.21.15.50:0/2103173462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77f8155a00 msgr2=0x7f77f815f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.479+0000 7f77fe200640 1 --2- 172.21.15.50:0/2103173462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77f8155a00 0x7f77f815f070 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009900 tx=0x7f77e801c6d0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.479+0000 7f77fe200640 1 -- 172.21.15.50:0/2103173462 shutdown_connections 2024-09-21T06:08:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.479+0000 7f77fe200640 1 --2- 172.21.15.50:0/2103173462 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f77f815f5b0 0x7f77f80b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.479+0000 7f77fe200640 1 --2- 172.21.15.50:0/2103173462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77f8155a00 0x7f77f815f070 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.479+0000 7f77fe200640 1 -- 172.21.15.50:0/2103173462 >> 172.21.15.50:0/2103173462 conn(0x7f77f806cdf0 msgr2=0x7f77f806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.479+0000 7f77fe200640 1 -- 172.21.15.50:0/2103173462 shutdown_connections 2024-09-21T06:08:06.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.479+0000 7f77fe200640 1 -- 172.21.15.50:0/2103173462 wait complete. 2024-09-21T06:08:06.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fe200640 1 Processor -- start 2024-09-21T06:08:06.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fe200640 1 -- start start 2024-09-21T06:08:06.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fe200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77f80b2300 0x7f77f8149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:06.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fe200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f77f8155a00 0x7f77f815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:06.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fe200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77f815f5b0 0x7f77f81f7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:06.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fe200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f77f81688f0 con 0x7f77f815f5b0 2024-09-21T06:08:06.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fe200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77f8168770 con 0x7f77f80b2300 2024-09-21T06:08:06.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fe200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f77f8168a70 con 0x7f77f8155a00 2024-09-21T06:08:06.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fd800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77f815f5b0 0x7f77f81f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fd800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77f815f5b0 0x7f77f81f7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35126/0 (socket says 172.21.15.50:35126) 2024-09-21T06:08:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fd800640 1 -- 172.21.15.50:0/1879784418 learned_addr learned my addr 172.21.15.50:0/1879784418 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77efe00640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f77f8155a00 0x7f77f815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fce00640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77f80b2300 0x7f77f8149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fd800640 1 -- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f77f8155a00 msgr2=0x7f77f815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fd800640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f77f8155a00 0x7f77f815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fd800640 1 -- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77f80b2300 msgr2=0x7f77f8149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fd800640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77f80b2300 0x7f77f8149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fd800640 1 -- 172.21.15.50:0/1879784418 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f8150550 con 0x7f77f815f5b0 2024-09-21T06:08:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fd800640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77f815f5b0 0x7f77f81f7280 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b970 tx=0x7f77f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77f5600640 1 -- 172.21.15.50:0/1879784418 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0019070 con 0x7f77f815f5b0 2024-09-21T06:08:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77f5600640 1 -- 172.21.15.50:0/1879784418 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77f000cbd0 con 0x7f77f815f5b0 2024-09-21T06:08:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77fe200640 1 -- 172.21.15.50:0/1879784418 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f815b0a0 con 0x7f77f815f5b0 2024-09-21T06:08:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.483+0000 7f77f5600640 1 -- 172.21.15.50:0/1879784418 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f00146a0 con 0x7f77f815f5b0 2024-09-21T06:08:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.487+0000 7f77fe200640 1 -- 172.21.15.50:0/1879784418 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f8150af0 con 0x7f77f815f5b0 2024-09-21T06:08:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.487+0000 7f77fe200640 1 -- 172.21.15.50:0/1879784418 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f815f070 con 0x7f77f815f5b0 2024-09-21T06:08:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.487+0000 7f77f5600640 1 -- 172.21.15.50:0/1879784418 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f77f001c070 con 0x7f77f815f5b0 2024-09-21T06:08:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.487+0000 7f77f5600640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77d4074790 0x7f77d4076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.487+0000 7f77f5600640 1 -- 172.21.15.50:0/1879784418 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f77f0092e80 con 0x7f77f815f5b0 2024-09-21T06:08:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.487+0000 7f77fce00640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77d4074790 0x7f77d4076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.491+0000 7f77fce00640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77d4074790 0x7f77d4076ca0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f77e80af000 tx=0x7f77e801af10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:06.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.491+0000 7f77f5600640 1 -- 172.21.15.50:0/1879784418 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f77f005ef90 con 0x7f77f815f5b0 2024-09-21T06:08:06.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.699+0000 7f77fe200640 1 -- 172.21.15.50:0/1879784418 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f77f806f1f0 con 0x7f77d4074790 2024-09-21T06:08:06.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77f5600640 1 -- 172.21.15.50:0/1879784418 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f77f806f1f0 con 0x7f77d4074790 2024-09-21T06:08:06.701 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:08:06.701 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:06.701 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:08:06.701 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:06.701 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:08:06.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77fe200640 1 -- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77d4074790 msgr2=0x7f77d4076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:06.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77fe200640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77d4074790 0x7f77d4076ca0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f77e80af000 tx=0x7f77e801af10 comp rx=0 tx=0).stop 2024-09-21T06:08:06.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77fe200640 1 -- 172.21.15.50:0/1879784418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77f815f5b0 msgr2=0x7f77f81f7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:06.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77fe200640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77f815f5b0 0x7f77f81f7280 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b970 tx=0x7f77f000be30 comp rx=0 tx=0).stop 2024-09-21T06:08:06.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77fe200640 1 -- 172.21.15.50:0/1879784418 shutdown_connections 2024-09-21T06:08:06.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77fe200640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f77d4074790 0x7f77d4076ca0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77fe200640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77f815f5b0 0x7f77f81f7280 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77fe200640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f77f8155a00 0x7f77f815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77fe200640 1 --2- 172.21.15.50:0/1879784418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77f80b2300 0x7f77f8149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77fe200640 1 -- 172.21.15.50:0/1879784418 >> 172.21.15.50:0/1879784418 conn(0x7f77f806cdf0 msgr2=0x7f77f815e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:06.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77fe200640 1 -- 172.21.15.50:0/1879784418 shutdown_connections 2024-09-21T06:08:06.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.703+0000 7f77fe200640 1 -- 172.21.15.50:0/1879784418 wait complete. 2024-09-21T06:08:06.725 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-21T06:08:06.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.887+0000 7f9b81800640 1 -- 172.21.15.50:0/2081106196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b7c0b5c70 msgr2=0x7f9b7c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:06.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.887+0000 7f9b81800640 1 --2- 172.21.15.50:0/2081106196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b7c0b5c70 0x7f9b7c15bb60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c072e30 tx=0x7f9b6c01c800 comp rx=0 tx=0).stop 2024-09-21T06:08:06.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.887+0000 7f9b81800640 1 -- 172.21.15.50:0/2081106196 shutdown_connections 2024-09-21T06:08:06.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.887+0000 7f9b81800640 1 --2- 172.21.15.50:0/2081106196 >> v2:172.21.15.88:3300/0 conn(0x7f9b7c0b4de0 0x7f9b7c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.887+0000 7f9b81800640 1 --2- 172.21.15.50:0/2081106196 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9b7c1527b0 0x7f9b7c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.887+0000 7f9b81800640 1 --2- 172.21.15.50:0/2081106196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b7c0b5c70 0x7f9b7c15bb60 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.887+0000 7f9b81800640 1 -- 172.21.15.50:0/2081106196 >> 172.21.15.50:0/2081106196 conn(0x7f9b7c0b2040 msgr2=0x7f9b7c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:06.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.887+0000 7f9b81800640 1 -- 172.21.15.50:0/2081106196 shutdown_connections 2024-09-21T06:08:06.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.887+0000 7f9b81800640 1 -- 172.21.15.50:0/2081106196 wait complete. 2024-09-21T06:08:06.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b81800640 1 Processor -- start 2024-09-21T06:08:06.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b81800640 1 -- start start 2024-09-21T06:08:06.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b81800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b7c0b4de0 0x7f9b7c158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:06.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b81800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9b7c0b5c70 0x7f9b7c1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:06.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b81800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b7c1527b0 0x7f9b7c1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:06.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b81800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c15f700 con 0x7f9b7c1527b0 2024-09-21T06:08:06.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b81800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c15f580 con 0x7f9b7c0b4de0 2024-09-21T06:08:06.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b81800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7c15f880 con 0x7f9b7c0b5c70 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b7b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b7c1527b0 0x7f9b7c1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b7b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b7c1527b0 0x7f9b7c1533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52912/0 (socket says 172.21.15.50:52912) 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b7aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b7c0b4de0 0x7f9b7c158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b7b400640 1 -- 172.21.15.50:0/2565398470 learned_addr learned my addr 172.21.15.50:0/2565398470 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b73e00640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9b7c0b5c70 0x7f9b7c1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b7b400640 1 -- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9b7c0b5c70 msgr2=0x7f9b7c1588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b7b400640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9b7c0b5c70 0x7f9b7c1588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b7b400640 1 -- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b7c0b4de0 msgr2=0x7f9b7c158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b7b400640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b7c0b4de0 0x7f9b7c158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b7b400640 1 -- 172.21.15.50:0/2565398470 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c152c20 con 0x7f9b7c1527b0 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b7b400640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b7c1527b0 0x7f9b7c1533e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400d930 tx=0x7f9b7400ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b72a00640 1 -- 172.21.15.50:0/2565398470 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74016070 con 0x7f9b7c1527b0 2024-09-21T06:08:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b72a00640 1 -- 172.21.15.50:0/2565398470 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b74004e20 con 0x7f9b7c1527b0 2024-09-21T06:08:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b81800640 1 -- 172.21.15.50:0/2565398470 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7c1515f0 con 0x7f9b7c1527b0 2024-09-21T06:08:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b72a00640 1 -- 172.21.15.50:0/2565398470 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7400bd50 con 0x7f9b7c1527b0 2024-09-21T06:08:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.891+0000 7f9b81800640 1 -- 172.21.15.50:0/2565398470 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7c0cff70 con 0x7f9b7c1527b0 2024-09-21T06:08:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.895+0000 7f9b81800640 1 -- 172.21.15.50:0/2565398470 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7c0afbc0 con 0x7f9b7c1527b0 2024-09-21T06:08:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.895+0000 7f9b72a00640 1 -- 172.21.15.50:0/2565398470 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f9b74003980 con 0x7f9b7c1527b0 2024-09-21T06:08:06.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.895+0000 7f9b72a00640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9b58074790 0x7f9b58076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:06.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.895+0000 7f9b72a00640 1 -- 172.21.15.50:0/2565398470 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f9b74094120 con 0x7f9b7c1527b0 2024-09-21T06:08:06.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.895+0000 7f9b7aa00640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9b58074790 0x7f9b58076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:06.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.899+0000 7f9b7aa00640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9b58074790 0x7f9b58076ca0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0af000 tx=0x7f9b6c0a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:06.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:06.899+0000 7f9b72a00640 1 -- 172.21.15.50:0/2565398470 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9b7401f020 con 0x7f9b7c1527b0 2024-09-21T06:08:07.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.111+0000 7f9b81800640 1 -- 172.21.15.50:0/2565398470 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b7c15fdc0 con 0x7f9b58074790 2024-09-21T06:08:07.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.131+0000 7f9b72a00640 1 -- 172.21.15.50:0/2565398470 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1026 (secure 0 0 0) 0x7f9b7c15fdc0 con 0x7f9b58074790 2024-09-21T06:08:07.131 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:07.131 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:08:07.131 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.131 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone80951" 2024-09-21T06:08:07.131 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.131 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.131 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone564119" 2024-09-21T06:08:07.131 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.131 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.131 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone832116" 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone71268" 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone96264" 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone81850" 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sv1" 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.132 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone877615" 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone972718" 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone216314" 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone67087" 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone64155" 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.133 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone647110" 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone189911" 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone935917" 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone2443" 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone8636" 2024-09-21T06:08:07.134 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone71879" 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone72702" 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone837612" 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone986813" 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:07.135 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:08:07.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.135+0000 7f9b81800640 1 -- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9b58074790 msgr2=0x7f9b58076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:07.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.135+0000 7f9b81800640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9b58074790 0x7f9b58076ca0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0af000 tx=0x7f9b6c0a7040 comp rx=0 tx=0).stop 2024-09-21T06:08:07.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.135+0000 7f9b81800640 1 -- 172.21.15.50:0/2565398470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b7c1527b0 msgr2=0x7f9b7c1533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:07.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.135+0000 7f9b81800640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b7c1527b0 0x7f9b7c1533e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400d930 tx=0x7f9b7400ddf0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.135+0000 7f9b81800640 1 -- 172.21.15.50:0/2565398470 shutdown_connections 2024-09-21T06:08:07.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.135+0000 7f9b81800640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9b58074790 0x7f9b58076ca0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.135+0000 7f9b81800640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b7c1527b0 0x7f9b7c1533e0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.135+0000 7f9b81800640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9b7c0b5c70 0x7f9b7c1588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.135+0000 7f9b81800640 1 --2- 172.21.15.50:0/2565398470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b7c0b4de0 0x7f9b7c158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.135+0000 7f9b81800640 1 -- 172.21.15.50:0/2565398470 >> 172.21.15.50:0/2565398470 conn(0x7f9b7c0b2040 msgr2=0x7f9b7c0b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:07.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.135+0000 7f9b81800640 1 -- 172.21.15.50:0/2565398470 shutdown_connections 2024-09-21T06:08:07.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.135+0000 7f9b81800640 1 -- 172.21.15.50:0/2565398470 wait complete. 2024-09-21T06:08:07.154 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone80951 2024-09-21T06:08:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.319+0000 7f9c61000640 1 -- 172.21.15.50:0/2520185332 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c5c0b5c70 msgr2=0x7f9c5c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.319+0000 7f9c61000640 1 --2- 172.21.15.50:0/2520185332 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c5c0b5c70 0x7f9c5c15bb60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009840 tx=0x7f9c4c01c760 comp rx=0 tx=0).stop 2024-09-21T06:08:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.319+0000 7f9c61000640 1 -- 172.21.15.50:0/2520185332 shutdown_connections 2024-09-21T06:08:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.319+0000 7f9c61000640 1 --2- 172.21.15.50:0/2520185332 >> v2:172.21.15.88:3300/0 conn(0x7f9c5c1527b0 0x7f9c5c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.319+0000 7f9c61000640 1 --2- 172.21.15.50:0/2520185332 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c5c0b5c70 0x7f9c5c15bb60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.319+0000 7f9c61000640 1 -- 172.21.15.50:0/2520185332 >> 172.21.15.50:0/2520185332 conn(0x7f9c5c0b2040 msgr2=0x7f9c5c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.319+0000 7f9c61000640 1 -- 172.21.15.50:0/2520185332 shutdown_connections 2024-09-21T06:08:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.319+0000 7f9c61000640 1 -- 172.21.15.50:0/2520185332 wait complete. 2024-09-21T06:08:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.319+0000 7f9c61000640 1 Processor -- start 2024-09-21T06:08:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.319+0000 7f9c61000640 1 -- start start 2024-09-21T06:08:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c61000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c5c0b4de0 0x7f9c5c1447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c61000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c5c0b5c70 0x7f9c5c151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c61000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c5c1527b0 0x7f9c5c0b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c61000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c168c50 con 0x7f9c5c1527b0 2024-09-21T06:08:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c61000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c168ad0 con 0x7f9c5c0b4de0 2024-09-21T06:08:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c61000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5c168dd0 con 0x7f9c5c0b5c70 2024-09-21T06:08:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c5a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c5c1527b0 0x7f9c5c0b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c59e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c5c0b4de0 0x7f9c5c1447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c5a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c5c1527b0 0x7f9c5c0b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52918/0 (socket says 172.21.15.50:52918) 2024-09-21T06:08:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c5a800640 1 -- 172.21.15.50:0/3216949408 learned_addr learned my addr 172.21.15.50:0/3216949408 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c51400640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c5c0b5c70 0x7f9c5c151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c5a800640 1 -- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c5c0b5c70 msgr2=0x7f9c5c151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:07.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c5a800640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c5c0b5c70 0x7f9c5c151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c5a800640 1 -- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c5c0b4de0 msgr2=0x7f9c5c1447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:07.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c5a800640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c5c0b4de0 0x7f9c5c1447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c5a800640 1 -- 172.21.15.50:0/3216949408 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c1f89f0 con 0x7f9c5c1527b0 2024-09-21T06:08:07.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c5a800640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c5c1527b0 0x7f9c5c0b1330 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400cb10 tx=0x7f9c5400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:07.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c52a00640 1 -- 172.21.15.50:0/3216949408 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54016020 con 0x7f9c5c1527b0 2024-09-21T06:08:07.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c52a00640 1 -- 172.21.15.50:0/3216949408 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c5400ce50 con 0x7f9c5c1527b0 2024-09-21T06:08:07.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c52a00640 1 -- 172.21.15.50:0/3216949408 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400ae00 con 0x7f9c5c1527b0 2024-09-21T06:08:07.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c61000640 1 -- 172.21.15.50:0/3216949408 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5c144d10 con 0x7f9c5c1527b0 2024-09-21T06:08:07.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c61000640 1 -- 172.21.15.50:0/3216949408 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5c1f8fb0 con 0x7f9c5c1527b0 2024-09-21T06:08:07.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.323+0000 7f9c61000640 1 -- 172.21.15.50:0/3216949408 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c5c15bb60 con 0x7f9c5c1527b0 2024-09-21T06:08:07.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.327+0000 7f9c52a00640 1 -- 172.21.15.50:0/3216949408 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f9c54019020 con 0x7f9c5c1527b0 2024-09-21T06:08:07.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.327+0000 7f9c52a00640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c38074760 0x7f9c38076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:07.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.327+0000 7f9c52a00640 1 -- 172.21.15.50:0/3216949408 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f9c54093430 con 0x7f9c5c1527b0 2024-09-21T06:08:07.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.327+0000 7f9c59e00640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c38074760 0x7f9c38076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:07.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.327+0000 7f9c59e00640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c38074760 0x7f9c38076c70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0af000 tx=0x7f9c4c01afa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:07.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.331+0000 7f9c52a00640 1 -- 172.21.15.50:0/3216949408 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9c54020020 con 0x7f9c5c1527b0 2024-09-21T06:08:07.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.531+0000 7f9c61000640 1 -- 172.21.15.50:0/3216949408 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone80951", "target": ["mon-mgr", ""]}) -- 0x7f9c5c06b540 con 0x7f9c38074760 2024-09-21T06:08:07.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c52a00640 1 -- 172.21.15.50:0/3216949408 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9c4c004040 con 0x7f9c38074760 2024-09-21T06:08:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c61000640 1 -- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c38074760 msgr2=0x7f9c38076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c61000640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c38074760 0x7f9c38076c70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0af000 tx=0x7f9c4c01afa0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c61000640 1 -- 172.21.15.50:0/3216949408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c5c1527b0 msgr2=0x7f9c5c0b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c61000640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c5c1527b0 0x7f9c5c0b1330 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400cb10 tx=0x7f9c5400cc00 comp rx=0 tx=0).stop 2024-09-21T06:08:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c61000640 1 -- 172.21.15.50:0/3216949408 shutdown_connections 2024-09-21T06:08:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c61000640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c38074760 0x7f9c38076c70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c61000640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c5c1527b0 0x7f9c5c0b1330 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c61000640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c5c0b5c70 0x7f9c5c151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c61000640 1 --2- 172.21.15.50:0/3216949408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c5c0b4de0 0x7f9c5c1447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c61000640 1 -- 172.21.15.50:0/3216949408 >> 172.21.15.50:0/3216949408 conn(0x7f9c5c0b2040 msgr2=0x7f9c5c15abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c61000640 1 -- 172.21.15.50:0/3216949408 shutdown_connections 2024-09-21T06:08:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.559+0000 7f9c61000640 1 -- 172.21.15.50:0/3216949408 wait complete. 2024-09-21T06:08:07.561 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:07.560+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:07.561 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:07.560+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:07.561 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:07.560+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:07.561 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:07.560+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:07.561 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:07.560+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:07.580 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:07.580+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:07.580 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:07.580+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:07.580 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:07.580+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:07.581 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:07.580+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:07.581 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:07.580+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:07.582 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone564119 2024-09-21T06:08:07.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.751+0000 7fd811c00640 1 -- 172.21.15.50:0/2433307397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd80c15f5c0 msgr2=0x7fd80c15f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:07.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.751+0000 7fd811c00640 1 --2- 172.21.15.50:0/2433307397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd80c15f5c0 0x7fd80c15f9d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd80c072250 tx=0x7fd80401caf0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.751+0000 7fd811c00640 1 -- 172.21.15.50:0/2433307397 shutdown_connections 2024-09-21T06:08:07.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.751+0000 7fd811c00640 1 --2- 172.21.15.50:0/2433307397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd80c15f5c0 0x7fd80c15f9d0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.751+0000 7fd811c00640 1 --2- 172.21.15.50:0/2433307397 >> v2:172.21.15.88:3300/0 conn(0x7fd80c06dbc0 0x7fd80c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.751+0000 7fd811c00640 1 -- 172.21.15.50:0/2433307397 >> 172.21.15.50:0/2433307397 conn(0x7fd80c0b41f0 msgr2=0x7fd80c0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:07.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.751+0000 7fd811c00640 1 -- 172.21.15.50:0/2433307397 shutdown_connections 2024-09-21T06:08:07.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.751+0000 7fd811c00640 1 -- 172.21.15.50:0/2433307397 wait complete. 2024-09-21T06:08:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.751+0000 7fd811c00640 1 Processor -- start 2024-09-21T06:08:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.751+0000 7fd811c00640 1 -- start start 2024-09-21T06:08:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd811c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd80c06dbc0 0x7fd80c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd811c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd80c0d1140 0x7fd80c1656e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd811c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd80c15f5c0 0x7fd80c165c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd811c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c168ff0 con 0x7fd80c0d1140 2024-09-21T06:08:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd811c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c168e70 con 0x7fd80c06dbc0 2024-09-21T06:08:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd811c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd80c169170 con 0x7fd80c15f5c0 2024-09-21T06:08:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd803e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd80c0d1140 0x7fd80c1656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd803e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd80c0d1140 0x7fd80c1656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52940/0 (socket says 172.21.15.50:52940) 2024-09-21T06:08:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd803e00640 1 -- 172.21.15.50:0/744348181 learned_addr learned my addr 172.21.15.50:0/744348181 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd80be00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd80c06dbc0 0x7fd80c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd811200640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd80c15f5c0 0x7fd80c165c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd803e00640 1 -- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd80c15f5c0 msgr2=0x7fd80c165c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd803e00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd80c15f5c0 0x7fd80c165c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd803e00640 1 -- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd80c06dbc0 msgr2=0x7fd80c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd803e00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd80c06dbc0 0x7fd80c15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd803e00640 1 -- 172.21.15.50:0/744348181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c0aed30 con 0x7fd80c0d1140 2024-09-21T06:08:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd803e00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd80c0d1140 0x7fd80c1656e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400e7c0 tx=0x7fd7f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:07.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd808c00640 1 -- 172.21.15.50:0/744348181 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f400ccc0 con 0x7fd80c0d1140 2024-09-21T06:08:07.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd808c00640 1 -- 172.21.15.50:0/744348181 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7f400ce80 con 0x7fd80c0d1140 2024-09-21T06:08:07.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd811c00640 1 -- 172.21.15.50:0/744348181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c1f9130 con 0x7fd80c0d1140 2024-09-21T06:08:07.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd808c00640 1 -- 172.21.15.50:0/744348181 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4005280 con 0x7fd80c0d1140 2024-09-21T06:08:07.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd811c00640 1 -- 172.21.15.50:0/744348181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c1f96f0 con 0x7fd80c0d1140 2024-09-21T06:08:07.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.755+0000 7fd811c00640 1 -- 172.21.15.50:0/744348181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80c0aeaa0 con 0x7fd80c0d1140 2024-09-21T06:08:07.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.759+0000 7fd808c00640 1 -- 172.21.15.50:0/744348181 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7fd7f4005420 con 0x7fd80c0d1140 2024-09-21T06:08:07.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.759+0000 7fd808c00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7e8074790 0x7fd7e8076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:07.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.759+0000 7fd808c00640 1 -- 172.21.15.50:0/744348181 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7fd7f40938a0 con 0x7fd80c0d1140 2024-09-21T06:08:07.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.759+0000 7fd80be00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7e8074790 0x7fd7e8076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:07.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.759+0000 7fd80be00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7e8074790 0x7fd7e8076ca0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc007fd0 tx=0x7fd7fc007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:07.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.763+0000 7fd808c00640 1 -- 172.21.15.50:0/744348181 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd7f405fc40 con 0x7fd80c0d1140 2024-09-21T06:08:07.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:07.971+0000 7fd811c00640 1 -- 172.21.15.50:0/744348181 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone564119", "target": ["mon-mgr", ""]}) -- 0x7fd80c06b540 con 0x7fd7e8074790 2024-09-21T06:08:08.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.643+0000 7fd808c00640 1 -- 172.21.15.50:0/744348181 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd7fc001d20 con 0x7fd7e8074790 2024-09-21T06:08:08.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.643+0000 7fd811c00640 1 -- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7e8074790 msgr2=0x7fd7e8076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:08.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.643+0000 7fd811c00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7e8074790 0x7fd7e8076ca0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc007fd0 tx=0x7fd7fc007ea0 comp rx=0 tx=0).stop 2024-09-21T06:08:08.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.643+0000 7fd811c00640 1 -- 172.21.15.50:0/744348181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd80c0d1140 msgr2=0x7fd80c1656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:08.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.643+0000 7fd811c00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd80c0d1140 0x7fd80c1656e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400e7c0 tx=0x7fd7f400ec80 comp rx=0 tx=0).stop 2024-09-21T06:08:08.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.643+0000 7fd811c00640 1 -- 172.21.15.50:0/744348181 shutdown_connections 2024-09-21T06:08:08.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.643+0000 7fd811c00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd80c15f5c0 0x7fd80c165c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:08.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.643+0000 7fd811c00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7e8074790 0x7fd7e8076ca0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:08.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.643+0000 7fd811c00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd80c0d1140 0x7fd80c1656e0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:08.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.643+0000 7fd811c00640 1 --2- 172.21.15.50:0/744348181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd80c06dbc0 0x7fd80c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:08.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.643+0000 7fd811c00640 1 -- 172.21.15.50:0/744348181 >> 172.21.15.50:0/744348181 conn(0x7fd80c0b41f0 msgr2=0x7fd80c0c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:08.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.647+0000 7fd811c00640 1 -- 172.21.15.50:0/744348181 shutdown_connections 2024-09-21T06:08:08.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.647+0000 7fd811c00640 1 -- 172.21.15.50:0/744348181 wait complete. 2024-09-21T06:08:08.675 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone832116 2024-09-21T06:08:08.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.843+0000 7f7a4b200640 1 -- 172.21.15.50:0/3333435867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a280041f0 msgr2=0x7f7a28004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:08.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.843+0000 7f7a4b200640 1 --2- 172.21.15.50:0/3333435867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a280041f0 0x7f7a28004810 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7a441612d0 tx=0x7f7a440cfab0 comp rx=0 tx=0).stop 2024-09-21T06:08:08.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.843+0000 7f7a49200640 1 -- 172.21.15.50:0/3333435867 reap_dead start 2024-09-21T06:08:08.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.843+0000 7f7a4b200640 1 -- 172.21.15.50:0/3333435867 shutdown_connections 2024-09-21T06:08:08.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.843+0000 7f7a4b200640 1 -- 172.21.15.50:0/3333435867 >> 172.21.15.50:0/3333435867 conn(0x7f7a440d06e0 msgr2=0x7f7a440d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:08.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.843+0000 7f7a4b200640 1 -- 172.21.15.50:0/3333435867 shutdown_connections 2024-09-21T06:08:08.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.843+0000 7f7a4b200640 1 -- 172.21.15.50:0/3333435867 wait complete. 2024-09-21T06:08:08.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a4b200640 1 Processor -- start 2024-09-21T06:08:08.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a4b200640 1 -- start start 2024-09-21T06:08:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a280041f0 0x7f7a4415d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a44168030 0x7f7a441433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a44142900 0x7f7a441438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a4b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a44168e90 con 0x7f7a44168030 2024-09-21T06:08:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a4b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7a44168d10 con 0x7f7a44142900 2024-09-21T06:08:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a4b200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7a44169010 con 0x7f7a280041f0 2024-09-21T06:08:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a3be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a44168030 0x7f7a441433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a3be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a44168030 0x7f7a441433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52960/0 (socket says 172.21.15.50:52960) 2024-09-21T06:08:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a3be00640 1 -- 172.21.15.50:0/4272110307 learned_addr learned my addr 172.21.15.50:0/4272110307 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a49200640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a44142900 0x7f7a441438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.847+0000 7f7a43e00640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a280041f0 0x7f7a4415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a3be00640 1 -- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a280041f0 msgr2=0x7f7a4415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a3be00640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a280041f0 0x7f7a4415d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a3be00640 1 -- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a44142900 msgr2=0x7f7a441438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a3be00640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a44142900 0x7f7a441438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a3be00640 1 -- 172.21.15.50:0/4272110307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a441456b0 con 0x7f7a44168030 2024-09-21T06:08:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a3be00640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a44168030 0x7f7a441433a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0af040 tx=0x7f7a2c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a40c00640 1 -- 172.21.15.50:0/4272110307 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0bf070 con 0x7f7a44168030 2024-09-21T06:08:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a40c00640 1 -- 172.21.15.50:0/4272110307 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a2c0b2040 con 0x7f7a44168030 2024-09-21T06:08:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a40c00640 1 -- 172.21.15.50:0/4272110307 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0b1070 con 0x7f7a44168030 2024-09-21T06:08:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a4b200640 1 -- 172.21.15.50:0/4272110307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a44145990 con 0x7f7a44168030 2024-09-21T06:08:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a4b200640 1 -- 172.21.15.50:0/4272110307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a44145360 con 0x7f7a44168030 2024-09-21T06:08:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a4b200640 1 -- 172.21.15.50:0/4272110307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a44145b90 con 0x7f7a44168030 2024-09-21T06:08:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.851+0000 7f7a40c00640 1 -- 172.21.15.50:0/4272110307 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f7a2c0aca10 con 0x7f7a44168030 2024-09-21T06:08:08.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.855+0000 7f7a40c00640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a20074760 0x7f7a20076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:08.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.855+0000 7f7a40c00640 1 -- 172.21.15.50:0/4272110307 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f7a2c12dbf0 con 0x7f7a44168030 2024-09-21T06:08:08.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.855+0000 7f7a43e00640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a20074760 0x7f7a20076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:08.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.855+0000 7f7a43e00640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a20074760 0x7f7a20076c70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7a34006fd0 tx=0x7f7a34006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:08.859+0000 7f7a40c00640 1 -- 172.21.15.50:0/4272110307 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7a44145b90 con 0x7f7a44168030 2024-09-21T06:08:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.059+0000 7f7a4b200640 1 -- 172.21.15.50:0/4272110307 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone832116", "target": ["mon-mgr", ""]}) -- 0x7f7a441421a0 con 0x7f7a20074760 2024-09-21T06:08:09.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.083+0000 7f7a40c00640 1 -- 172.21.15.50:0/4272110307 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7a340037a0 con 0x7f7a20074760 2024-09-21T06:08:09.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.087+0000 7f7a4b200640 1 -- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a20074760 msgr2=0x7f7a20076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:09.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.087+0000 7f7a4b200640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a20074760 0x7f7a20076c70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7a34006fd0 tx=0x7f7a34006b20 comp rx=0 tx=0).stop 2024-09-21T06:08:09.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.087+0000 7f7a4b200640 1 -- 172.21.15.50:0/4272110307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a44168030 msgr2=0x7f7a441433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:09.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.087+0000 7f7a4b200640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a44168030 0x7f7a441433a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0af040 tx=0x7f7a2c005520 comp rx=0 tx=0).stop 2024-09-21T06:08:09.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.087+0000 7f7a4b200640 1 -- 172.21.15.50:0/4272110307 shutdown_connections 2024-09-21T06:08:09.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.087+0000 7f7a4b200640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a44142900 0x7f7a441438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.087+0000 7f7a4b200640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7a20074760 0x7f7a20076c70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.087+0000 7f7a4b200640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a44168030 0x7f7a441433a0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.087+0000 7f7a4b200640 1 --2- 172.21.15.50:0/4272110307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a280041f0 0x7f7a4415d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.087+0000 7f7a4b200640 1 -- 172.21.15.50:0/4272110307 >> 172.21.15.50:0/4272110307 conn(0x7f7a440d06e0 msgr2=0x7f7a440725f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:09.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.087+0000 7f7a4b200640 1 -- 172.21.15.50:0/4272110307 shutdown_connections 2024-09-21T06:08:09.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.087+0000 7f7a4b200640 1 -- 172.21.15.50:0/4272110307 wait complete. 2024-09-21T06:08:09.113 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone71268 2024-09-21T06:08:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.279+0000 7f3835a00640 1 -- 172.21.15.50:0/3260282933 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f383006fa40 msgr2=0x7f38300af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.279+0000 7f3835a00640 1 --2- 172.21.15.50:0/3260282933 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f383006fa40 0x7f38300af520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3828009bf0 tx=0x7f382801cbe0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.279+0000 7f3835a00640 1 -- 172.21.15.50:0/3260282933 shutdown_connections 2024-09-21T06:08:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.279+0000 7f3835a00640 1 --2- 172.21.15.50:0/3260282933 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f383006fa40 0x7f38300af520 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.279+0000 7f3835a00640 1 -- 172.21.15.50:0/3260282933 >> 172.21.15.50:0/3260282933 conn(0x7f38300d06e0 msgr2=0x7f38300d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.279+0000 7f3835a00640 1 -- 172.21.15.50:0/3260282933 shutdown_connections 2024-09-21T06:08:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.279+0000 7f3835a00640 1 -- 172.21.15.50:0/3260282933 wait complete. 2024-09-21T06:08:09.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.279+0000 7f3835a00640 1 Processor -- start 2024-09-21T06:08:09.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.279+0000 7f3835a00640 1 -- start start 2024-09-21T06:08:09.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3835a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f383006fa40 0x7f38300aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3835a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3830073f90 0x7f3830151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3835a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f383015c730 0x7f3830154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3835a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f383016a470 con 0x7f383015c730 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3835a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f383016a2f0 con 0x7f383006fa40 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3835a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f383016a5f0 con 0x7f3830073f90 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f382f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f383015c730 0x7f3830154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f382f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f383015c730 0x7f3830154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52972/0 (socket says 172.21.15.50:52972) 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f382f400640 1 -- 172.21.15.50:0/1127906267 learned_addr learned my addr 172.21.15.50:0/1127906267 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f382ea00640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f383006fa40 0x7f38300aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3827e00640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3830073f90 0x7f3830151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f382f400640 1 -- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3830073f90 msgr2=0x7f3830151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f382f400640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3830073f90 0x7f3830151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f382f400640 1 -- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f383006fa40 msgr2=0x7f38300aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:09.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f382f400640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f383006fa40 0x7f38300aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f382f400640 1 -- 172.21.15.50:0/1127906267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38301580c0 con 0x7f383015c730 2024-09-21T06:08:09.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f382f400640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f383015c730 0x7f3830154d70 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f382801a810 tx=0x7f3828005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:09.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3826a00640 1 -- 172.21.15.50:0/1127906267 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38280b2040 con 0x7f383015c730 2024-09-21T06:08:09.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3826a00640 1 -- 172.21.15.50:0/1127906267 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3828004b40 con 0x7f383015c730 2024-09-21T06:08:09.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3826a00640 1 -- 172.21.15.50:0/1127906267 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828004040 con 0x7f383015c730 2024-09-21T06:08:09.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3835a00640 1 -- 172.21.15.50:0/1127906267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38301fa350 con 0x7f383015c730 2024-09-21T06:08:09.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3835a00640 1 -- 172.21.15.50:0/1127906267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38301539b0 con 0x7f383015c730 2024-09-21T06:08:09.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.283+0000 7f3835a00640 1 -- 172.21.15.50:0/1127906267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38300af520 con 0x7f383015c730 2024-09-21T06:08:09.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.287+0000 7f3826a00640 1 -- 172.21.15.50:0/1127906267 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7f38280be060 con 0x7f383015c730 2024-09-21T06:08:09.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.287+0000 7f3826a00640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f380c074760 0x7f380c076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:09.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.287+0000 7f3826a00640 1 -- 172.21.15.50:0/1127906267 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f382812cad0 con 0x7f383015c730 2024-09-21T06:08:09.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.287+0000 7f382ea00640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f380c074760 0x7f380c076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:09.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.287+0000 7f382ea00640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f380c074760 0x7f380c076c70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3820007f30 tx=0x7f3820007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.291+0000 7f3826a00640 1 -- 172.21.15.50:0/1127906267 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f38280167d0 con 0x7f383015c730 2024-09-21T06:08:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.495+0000 7f3835a00640 1 -- 172.21.15.50:0/1127906267 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone71268", "target": ["mon-mgr", ""]}) -- 0x7f383006b540 con 0x7f380c074760 2024-09-21T06:08:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.515+0000 7f3826a00640 1 -- 172.21.15.50:0/1127906267 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f38200014c0 con 0x7f380c074760 2024-09-21T06:08:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.519+0000 7f3835a00640 1 -- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f380c074760 msgr2=0x7f380c076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.519+0000 7f3835a00640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f380c074760 0x7f380c076c70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3820007f30 tx=0x7f3820007ec0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.519+0000 7f3835a00640 1 -- 172.21.15.50:0/1127906267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f383015c730 msgr2=0x7f3830154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.519+0000 7f3835a00640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f383015c730 0x7f3830154d70 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f382801a810 tx=0x7f3828005aa0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.519+0000 7f3835a00640 1 -- 172.21.15.50:0/1127906267 shutdown_connections 2024-09-21T06:08:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.519+0000 7f3835a00640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f380c074760 0x7f380c076c70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.519+0000 7f3835a00640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f383015c730 0x7f3830154d70 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.519+0000 7f3835a00640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3830073f90 0x7f3830151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.519+0000 7f3835a00640 1 --2- 172.21.15.50:0/1127906267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f383006fa40 0x7f38300aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.519+0000 7f3835a00640 1 -- 172.21.15.50:0/1127906267 >> 172.21.15.50:0/1127906267 conn(0x7f38300d06e0 msgr2=0x7f38300d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:09.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.519+0000 7f3835a00640 1 -- 172.21.15.50:0/1127906267 shutdown_connections 2024-09-21T06:08:09.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.519+0000 7f3835a00640 1 -- 172.21.15.50:0/1127906267 wait complete. 2024-09-21T06:08:09.546 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone96264 2024-09-21T06:08:09.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.763+0000 7fab87000640 1 -- 172.21.15.50:0/2916283652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab800cd7a0 msgr2=0x7fab800cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:09.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.763+0000 7fab87000640 1 --2- 172.21.15.50:0/2916283652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab800cd7a0 0x7fab800cdbb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fab800ccc30 tx=0x7fab6c01d470 comp rx=0 tx=0).stop 2024-09-21T06:08:09.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.763+0000 7fab87000640 1 -- 172.21.15.50:0/2916283652 shutdown_connections 2024-09-21T06:08:09.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.763+0000 7fab87000640 1 --2- 172.21.15.50:0/2916283652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab800cd7a0 0x7fab800cdbb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.763+0000 7fab87000640 1 --2- 172.21.15.50:0/2916283652 >> v2:172.21.15.50:3300/0 conn(0x7fab8015aa50 0x7fab800cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.763+0000 7fab87000640 1 -- 172.21.15.50:0/2916283652 >> 172.21.15.50:0/2916283652 conn(0x7fab800b2040 msgr2=0x7fab800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:09.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.763+0000 7fab87000640 1 -- 172.21.15.50:0/2916283652 shutdown_connections 2024-09-21T06:08:09.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.763+0000 7fab87000640 1 -- 172.21.15.50:0/2916283652 wait complete. 2024-09-21T06:08:09.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.767+0000 7fab87000640 1 Processor -- start 2024-09-21T06:08:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.767+0000 7fab87000640 1 -- start start 2024-09-21T06:08:09.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.771+0000 7fab87000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab800cd7a0 0x7fab80141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:09.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.771+0000 7fab87000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab8015aa50 0x7fab801eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:09.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab87000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab8015e4e0 0x7fab80142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:09.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab7fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab800cd7a0 0x7fab80141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:09.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab7fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab800cd7a0 0x7fab80141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:40014/0 (socket says 172.21.15.50:40014) 2024-09-21T06:08:09.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab7fe00640 1 -- 172.21.15.50:0/2608660468 learned_addr learned my addr 172.21.15.50:0/2608660468 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:09.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fab801683a0 con 0x7fab8015e4e0 2024-09-21T06:08:09.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fab80168220 con 0x7fab800cd7a0 2024-09-21T06:08:09.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fab80168520 con 0x7fab8015aa50 2024-09-21T06:08:09.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab7f400640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab8015aa50 0x7fab801eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:09.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab85000640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab8015e4e0 0x7fab80142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:09.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab7fe00640 1 -- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab8015aa50 msgr2=0x7fab801eeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:09.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab7fe00640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab8015aa50 0x7fab801eeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab7fe00640 1 -- 172.21.15.50:0/2608660468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab8015e4e0 msgr2=0x7fab80142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:09.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab7fe00640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab8015e4e0 0x7fab80142070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:09.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab7fe00640 1 -- 172.21.15.50:0/2608660468 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab801559c0 con 0x7fab800cd7a0 2024-09-21T06:08:09.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab85000640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab8015e4e0 0x7fab80142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:09.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab7fe00640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab800cd7a0 0x7fab80141930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fab7000e9f0 tx=0x7fab7000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:09.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab77e00640 1 -- 172.21.15.50:0/2608660468 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7000cd80 con 0x7fab800cd7a0 2024-09-21T06:08:09.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab77e00640 1 -- 172.21.15.50:0/2608660468 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab70004020 con 0x7fab800cd7a0 2024-09-21T06:08:09.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab77e00640 1 -- 172.21.15.50:0/2608660468 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70004470 con 0x7fab800cd7a0 2024-09-21T06:08:09.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab801ef0c0 con 0x7fab800cd7a0 2024-09-21T06:08:09.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.775+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8015f900 con 0x7fab800cd7a0 2024-09-21T06:08:09.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.779+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8015fb40 con 0x7fab800cd7a0 2024-09-21T06:08:09.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.779+0000 7fab77e00640 1 -- 172.21.15.50:0/2608660468 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 40) ==== 99172+0+0 (secure 0 0 0) 0x7fab70003600 con 0x7fab800cd7a0 2024-09-21T06:08:09.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.783+0000 7fab77e00640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fab60074760 0x7fab60076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:09.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.783+0000 7fab77e00640 1 -- 172.21.15.50:0/2608660468 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7fab70093870 con 0x7fab800cd7a0 2024-09-21T06:08:09.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.783+0000 7fab7f400640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fab60074760 0x7fab60076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:09.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.783+0000 7fab7f400640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fab60074760 0x7fab60076c70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0ae000 tx=0x7fab6c01bcb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:09.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:09.791+0000 7fab77e00640 1 -- 172.21.15.50:0/2608660468 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fab801ef0c0 con 0x7fab800cd7a0 2024-09-21T06:08:10.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.091+0000 7fab77e00640 1 -- 172.21.15.50:0/2608660468 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fab700093e0 con 0x7fab800cd7a0 2024-09-21T06:08:10.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.135+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone96264", "target": ["mon-mgr", ""]}) -- 0x7fab8006b540 con 0x7fab60074760 2024-09-21T06:08:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.151+0000 7fab77e00640 1 -- 172.21.15.50:0/2608660468 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fab6c0af040 con 0x7fab60074760 2024-09-21T06:08:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.151+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fab60074760 msgr2=0x7fab60076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.151+0000 7fab87000640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fab60074760 0x7fab60076c70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0ae000 tx=0x7fab6c01bcb0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.155+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab800cd7a0 msgr2=0x7fab80141930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.155+0000 7fab87000640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab800cd7a0 0x7fab80141930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fab7000e9f0 tx=0x7fab7000edf0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.155+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 shutdown_connections 2024-09-21T06:08:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.155+0000 7fab87000640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fab60074760 0x7fab60076c70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.155+0000 7fab87000640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab8015e4e0 0x7fab80142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.155+0000 7fab87000640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fab8015aa50 0x7fab801eeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.155+0000 7fab87000640 1 --2- 172.21.15.50:0/2608660468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab800cd7a0 0x7fab80141930 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.155+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 >> 172.21.15.50:0/2608660468 conn(0x7fab800b2040 msgr2=0x7fab8015d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.155+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 shutdown_connections 2024-09-21T06:08:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.155+0000 7fab87000640 1 -- 172.21.15.50:0/2608660468 wait complete. 2024-09-21T06:08:10.174 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81850 2024-09-21T06:08:10.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.339+0000 7fedeec00640 1 -- 172.21.15.50:0/765495695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fede81527b0 msgr2=0x7fede815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:10.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.339+0000 7fedeec00640 1 --2- 172.21.15.50:0/765495695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fede81527b0 0x7fede815bb60 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fede8074b20 tx=0x7fedd001cce0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.339+0000 7fedeec00640 1 -- 172.21.15.50:0/765495695 shutdown_connections 2024-09-21T06:08:10.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.339+0000 7fedeec00640 1 --2- 172.21.15.50:0/765495695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fede81527b0 0x7fede815bb60 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.339+0000 7fedeec00640 1 -- 172.21.15.50:0/765495695 >> 172.21.15.50:0/765495695 conn(0x7fede80b2040 msgr2=0x7fede80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:10.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.339+0000 7fedeec00640 1 -- 172.21.15.50:0/765495695 shutdown_connections 2024-09-21T06:08:10.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.339+0000 7fedeec00640 1 -- 172.21.15.50:0/765495695 wait complete. 2024-09-21T06:08:10.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.343+0000 7fedeec00640 1 Processor -- start 2024-09-21T06:08:10.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.343+0000 7fedeec00640 1 -- start start 2024-09-21T06:08:10.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.343+0000 7fedeec00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fede80b4de0 0x7fede814aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.343+0000 7fedeec00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fede80b5c70 0x7fede8157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.343+0000 7fedeec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fede81527b0 0x7fede815a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.343+0000 7fedeec00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fede8169b30 con 0x7fede80b4de0 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.343+0000 7fedeec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fede81699b0 con 0x7fede81527b0 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.343+0000 7fedeec00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fede8169cb0 con 0x7fede80b5c70 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.343+0000 7fede7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fede80b4de0 0x7fede814aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.343+0000 7fede7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fede80b4de0 0x7fede814aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53012/0 (socket says 172.21.15.50:53012) 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.343+0000 7fede7e00640 1 -- 172.21.15.50:0/3133045470 learned_addr learned my addr 172.21.15.50:0/3133045470 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7feddf400640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fede80b5c70 0x7fede8157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fedecc00640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fede81527b0 0x7fede815a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fede7e00640 1 -- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fede80b5c70 msgr2=0x7fede8157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fede7e00640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fede80b5c70 0x7fede8157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fede7e00640 1 -- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fede81527b0 msgr2=0x7fede815a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fede7e00640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fede81527b0 0x7fede815a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fede7e00640 1 -- 172.21.15.50:0/3133045470 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede8157e30 con 0x7fede80b4de0 2024-09-21T06:08:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fede7e00640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fede80b4de0 0x7fede814aaf0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fedd800c3c0 tx=0x7fedd800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fede4c00640 1 -- 172.21.15.50:0/3133045470 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8017070 con 0x7fede80b4de0 2024-09-21T06:08:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fede4c00640 1 -- 172.21.15.50:0/3133045470 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedd80053c0 con 0x7fede80b4de0 2024-09-21T06:08:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fede4c00640 1 -- 172.21.15.50:0/3133045470 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8003860 con 0x7fede80b4de0 2024-09-21T06:08:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fedeec00640 1 -- 172.21.15.50:0/3133045470 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede8158110 con 0x7fede80b4de0 2024-09-21T06:08:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fedeec00640 1 -- 172.21.15.50:0/3133045470 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede81586d0 con 0x7fede80b4de0 2024-09-21T06:08:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fedeec00640 1 -- 172.21.15.50:0/3133045470 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede815bb60 con 0x7fede80b4de0 2024-09-21T06:08:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.347+0000 7fede4c00640 1 -- 172.21.15.50:0/3133045470 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fedd801a070 con 0x7fede80b4de0 2024-09-21T06:08:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.351+0000 7fede4c00640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fedc4074880 0x7fedc4076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.351+0000 7fede4c00640 1 -- 172.21.15.50:0/3133045470 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7fedd80920d0 con 0x7fede80b4de0 2024-09-21T06:08:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.351+0000 7feddf400640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fedc4074880 0x7fedc4076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.351+0000 7feddf400640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fedc4074880 0x7fedc4076d90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fedd00af000 tx=0x7fedd001b520 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.355+0000 7fede4c00640 1 -- 172.21.15.50:0/3133045470 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fedd805e630 con 0x7fede80b4de0 2024-09-21T06:08:10.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.559+0000 7fedeec00640 1 -- 172.21.15.50:0/3133045470 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81850", "target": ["mon-mgr", ""]}) -- 0x7fede806b540 con 0x7fedc4074880 2024-09-21T06:08:10.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.575+0000 7fede4c00640 1 -- 172.21.15.50:0/3133045470 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fedd0004ad0 con 0x7fedc4074880 2024-09-21T06:08:10.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.579+0000 7fedeec00640 1 -- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fedc4074880 msgr2=0x7fedc4076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:10.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.579+0000 7fedeec00640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fedc4074880 0x7fedc4076d90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fedd00af000 tx=0x7fedd001b520 comp rx=0 tx=0).stop 2024-09-21T06:08:10.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.579+0000 7fedeec00640 1 -- 172.21.15.50:0/3133045470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fede80b4de0 msgr2=0x7fede814aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:10.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.579+0000 7fedeec00640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fede80b4de0 0x7fede814aaf0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fedd800c3c0 tx=0x7fedd800c880 comp rx=0 tx=0).stop 2024-09-21T06:08:10.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.579+0000 7fedeec00640 1 -- 172.21.15.50:0/3133045470 shutdown_connections 2024-09-21T06:08:10.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.579+0000 7fedeec00640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fede81527b0 0x7fede815a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.579+0000 7fedeec00640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fede80b5c70 0x7fede8157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.579+0000 7fedeec00640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fedc4074880 0x7fedc4076d90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.579+0000 7fedeec00640 1 --2- 172.21.15.50:0/3133045470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fede80b4de0 0x7fede814aaf0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.579+0000 7fedeec00640 1 -- 172.21.15.50:0/3133045470 >> 172.21.15.50:0/3133045470 conn(0x7fede80b2040 msgr2=0x7fede814c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:10.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.579+0000 7fedeec00640 1 -- 172.21.15.50:0/3133045470 shutdown_connections 2024-09-21T06:08:10.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.579+0000 7fedeec00640 1 -- 172.21.15.50:0/3133045470 wait complete. 2024-09-21T06:08:10.601 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-21T06:08:10.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.827+0000 7f083ca00640 1 -- 172.21.15.50:0/3301863912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0838070050 msgr2=0x7f0838072ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:10.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.827+0000 7f083ca00640 1 --2- 172.21.15.50:0/3301863912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0838070050 0x7f0838072ba0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f08380722e0 tx=0x7f082001c770 comp rx=0 tx=0).stop 2024-09-21T06:08:10.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.827+0000 7f083ca00640 1 -- 172.21.15.50:0/3301863912 shutdown_connections 2024-09-21T06:08:10.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.827+0000 7f083ca00640 1 --2- 172.21.15.50:0/3301863912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0838070050 0x7f0838072ba0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.827+0000 7f083ca00640 1 --2- 172.21.15.50:0/3301863912 >> v2:172.21.15.88:3300/0 conn(0x7f08380af5b0 0x7f083806fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.827+0000 7f083ca00640 1 -- 172.21.15.50:0/3301863912 >> 172.21.15.50:0/3301863912 conn(0x7f083806d3f0 msgr2=0x7f08380b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:10.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.827+0000 7f083ca00640 1 -- 172.21.15.50:0/3301863912 shutdown_connections 2024-09-21T06:08:10.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.827+0000 7f083ca00640 1 -- 172.21.15.50:0/3301863912 wait complete. 2024-09-21T06:08:10.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.827+0000 7f083ca00640 1 Processor -- start 2024-09-21T06:08:10.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.827+0000 7f083ca00640 1 -- start start 2024-09-21T06:08:10.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f083ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380730e0 0x7f083815a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:10.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f083ca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380af5b0 0x7f08381588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:10.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f083ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0838159630 0x7f08380ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:10.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f083ca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0838168e30 con 0x7f0838159630 2024-09-21T06:08:10.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f083ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0838168cb0 con 0x7f08380730e0 2024-09-21T06:08:10.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f083ca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0838168fb0 con 0x7f08380af5b0 2024-09-21T06:08:10.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f0837200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0838159630 0x7f08380ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:10.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082de00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380af5b0 0x7f08381588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:10.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082de00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380af5b0 0x7f08381588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:56712/0 (socket says 172.21.15.50:56712) 2024-09-21T06:08:10.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f0837200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0838159630 0x7f08380ad320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53022/0 (socket says 172.21.15.50:53022) 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082de00640 1 -- 172.21.15.50:0/2155319757 learned_addr learned my addr 172.21.15.50:0/2155319757 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f0836800640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380730e0 0x7f083815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082de00640 1 -- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380730e0 msgr2=0x7f083815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082de00640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380730e0 0x7f083815a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082de00640 1 -- 172.21.15.50:0/2155319757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0838159630 msgr2=0x7f08380ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082de00640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0838159630 0x7f08380ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082de00640 1 -- 172.21.15.50:0/2155319757 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08380acd90 con 0x7f08380af5b0 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f0837200640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0838159630 0x7f08380ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082de00640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380af5b0 0x7f08381588d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08200b0040 tx=0x7f0820005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082f400640 1 -- 172.21.15.50:0/2155319757 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08200b2040 con 0x7f08380af5b0 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082f400640 1 -- 172.21.15.50:0/2155319757 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0820004330 con 0x7f08380af5b0 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f083ca00640 1 -- 172.21.15.50:0/2155319757 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08380ad070 con 0x7f08380af5b0 2024-09-21T06:08:10.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082f400640 1 -- 172.21.15.50:0/2155319757 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820005280 con 0x7f08380af5b0 2024-09-21T06:08:10.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f083ca00640 1 -- 172.21.15.50:0/2155319757 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0838159f00 con 0x7f08380af5b0 2024-09-21T06:08:10.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f083ca00640 1 -- 172.21.15.50:0/2155319757 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0838072ba0 con 0x7f08380af5b0 2024-09-21T06:08:10.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.831+0000 7f082f400640 1 -- 172.21.15.50:0/2155319757 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f0820005420 con 0x7f08380af5b0 2024-09-21T06:08:10.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.835+0000 7f082f400640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f08140748b0 0x7f0814076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:10.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.835+0000 7f082f400640 1 -- 172.21.15.50:0/2155319757 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f082012ce20 con 0x7f08380af5b0 2024-09-21T06:08:10.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.835+0000 7f0836800640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f08140748b0 0x7f0814076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:10.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.835+0000 7f0836800640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f08140748b0 0x7f0814076dc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0828004630 tx=0x7f082800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:10.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:10.839+0000 7f082f400640 1 -- 172.21.15.50:0/2155319757 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f08200164b0 con 0x7f08380af5b0 2024-09-21T06:08:11.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.047+0000 7f083ca00640 1 -- 172.21.15.50:0/2155319757 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f083806b540 con 0x7f08140748b0 2024-09-21T06:08:11.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f082f400640 1 -- 172.21.15.50:0/2155319757 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f082800b7e0 con 0x7f08140748b0 2024-09-21T06:08:11.061 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:11.061 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:08:11.061 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:11.062 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ss1" 2024-09-21T06:08:11.062 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:11.062 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:08:11.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f083ca00640 1 -- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f08140748b0 msgr2=0x7f0814076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:11.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f083ca00640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f08140748b0 0x7f0814076dc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0828004630 tx=0x7f082800b2e0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f083ca00640 1 -- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380af5b0 msgr2=0x7f08381588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:11.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f083ca00640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380af5b0 0x7f08381588d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08200b0040 tx=0x7f0820005e70 comp rx=0 tx=0).stop 2024-09-21T06:08:11.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f083ca00640 1 -- 172.21.15.50:0/2155319757 shutdown_connections 2024-09-21T06:08:11.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f083ca00640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f08140748b0 0x7f0814076dc0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f083ca00640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0838159630 0x7f08380ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f083ca00640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380af5b0 0x7f08381588d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f083ca00640 1 --2- 172.21.15.50:0/2155319757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380730e0 0x7f083815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f083ca00640 1 -- 172.21.15.50:0/2155319757 >> 172.21.15.50:0/2155319757 conn(0x7f083806d3f0 msgr2=0x7f083815f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:11.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f083ca00640 1 -- 172.21.15.50:0/2155319757 shutdown_connections 2024-09-21T06:08:11.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.063+0000 7f083ca00640 1 -- 172.21.15.50:0/2155319757 wait complete. 2024-09-21T06:08:11.084 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-21T06:08:11.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.251+0000 7f58dc800640 1 -- 172.21.15.50:0/3103617125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58d41527b0 msgr2=0x7f58d415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:11.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.251+0000 7f58dc800640 1 --2- 172.21.15.50:0/3103617125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58d41527b0 0x7f58d415bb60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009900 tx=0x7f58c401c8b0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.251+0000 7f58dc800640 1 -- 172.21.15.50:0/3103617125 shutdown_connections 2024-09-21T06:08:11.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.251+0000 7f58dc800640 1 --2- 172.21.15.50:0/3103617125 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f58d40b4de0 0x7f58d40b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.251+0000 7f58dc800640 1 --2- 172.21.15.50:0/3103617125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58d41527b0 0x7f58d415bb60 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.251+0000 7f58dc800640 1 -- 172.21.15.50:0/3103617125 >> 172.21.15.50:0/3103617125 conn(0x7f58d40b2040 msgr2=0x7f58d40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:11.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.251+0000 7f58dc800640 1 -- 172.21.15.50:0/3103617125 shutdown_connections 2024-09-21T06:08:11.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.251+0000 7f58dc800640 1 -- 172.21.15.50:0/3103617125 wait complete. 2024-09-21T06:08:11.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58dc800640 1 Processor -- start 2024-09-21T06:08:11.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58dc800640 1 -- start start 2024-09-21T06:08:11.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58dc800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58d40b4de0 0x7f58d40cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:11.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58dc800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f58d40b5c70 0x7f58d40cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:11.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58dc800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58d41527b0 0x7f58d40cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:11.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58dc800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4168950 con 0x7f58d41527b0 2024-09-21T06:08:11.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58dc800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41687d0 con 0x7f58d40b4de0 2024-09-21T06:08:11.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58dc800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f58d4168ad0 con 0x7f58d40b5c70 2024-09-21T06:08:11.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58da800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58d41527b0 0x7f58d40cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:11.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58da800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58d41527b0 0x7f58d40cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53042/0 (socket says 172.21.15.50:53042) 2024-09-21T06:08:11.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58d9400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f58d40b5c70 0x7f58d40cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:11.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58da800640 1 -- 172.21.15.50:0/417550432 learned_addr learned my addr 172.21.15.50:0/417550432 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:11.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58d9e00640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58d40b4de0 0x7f58d40cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:11.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58da800640 1 -- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f58d40b5c70 msgr2=0x7f58d40cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:11.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58da800640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f58d40b5c70 0x7f58d40cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58da800640 1 -- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58d40b4de0 msgr2=0x7f58d40cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:11.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58da800640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58d40b4de0 0x7f58d40cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58da800640 1 -- 172.21.15.50:0/417550432 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d40cd170 con 0x7f58d41527b0 2024-09-21T06:08:11.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58da800640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58d41527b0 0x7f58d40cdec0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f58d000ea80 tx=0x7f58d000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:11.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58ce000640 1 -- 172.21.15.50:0/417550432 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d000ce40 con 0x7f58d41527b0 2024-09-21T06:08:11.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58ce000640 1 -- 172.21.15.50:0/417550432 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58d0004b30 con 0x7f58d41527b0 2024-09-21T06:08:11.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58ce000640 1 -- 172.21.15.50:0/417550432 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0005020 con 0x7f58d41527b0 2024-09-21T06:08:11.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58dc800640 1 -- 172.21.15.50:0/417550432 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d40cd3e0 con 0x7f58d41527b0 2024-09-21T06:08:11.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.255+0000 7f58dc800640 1 -- 172.21.15.50:0/417550432 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d40cd9a0 con 0x7f58d41527b0 2024-09-21T06:08:11.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.259+0000 7f58dc800640 1 -- 172.21.15.50:0/417550432 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d40b51f0 con 0x7f58d41527b0 2024-09-21T06:08:11.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.259+0000 7f58ce000640 1 -- 172.21.15.50:0/417550432 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f58d00051c0 con 0x7f58d41527b0 2024-09-21T06:08:11.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.259+0000 7f58ce000640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f58b8074880 0x7f58b8076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:11.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.259+0000 7f58ce000640 1 -- 172.21.15.50:0/417550432 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f58d00939d0 con 0x7f58d41527b0 2024-09-21T06:08:11.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.259+0000 7f58d9e00640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f58b8074880 0x7f58b8076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:11.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.259+0000 7f58d9e00640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f58b8074880 0x7f58b8076d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f58c8007fd0 tx=0x7f58c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:11.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.263+0000 7f58ce000640 1 -- 172.21.15.50:0/417550432 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f58d005fe90 con 0x7f58d41527b0 2024-09-21T06:08:11.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.471+0000 7f58dc800640 1 -- 172.21.15.50:0/417550432 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58d406b540 con 0x7f58b8074880 2024-09-21T06:08:11.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58ce000640 1 -- 172.21.15.50:0/417550432 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f58c8001410 con 0x7f58b8074880 2024-09-21T06:08:11.501 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:11.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58dc800640 1 -- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f58b8074880 msgr2=0x7f58b8076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:11.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58dc800640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f58b8074880 0x7f58b8076d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f58c8007fd0 tx=0x7f58c8009040 comp rx=0 tx=0).stop 2024-09-21T06:08:11.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58dc800640 1 -- 172.21.15.50:0/417550432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58d41527b0 msgr2=0x7f58d40cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:11.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58dc800640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58d41527b0 0x7f58d40cdec0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f58d000ea80 tx=0x7f58d000ef40 comp rx=0 tx=0).stop 2024-09-21T06:08:11.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58dc800640 1 -- 172.21.15.50:0/417550432 shutdown_connections 2024-09-21T06:08:11.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58dc800640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f58b8074880 0x7f58b8076d90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58dc800640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58d41527b0 0x7f58d40cdec0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58dc800640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f58d40b5c70 0x7f58d40cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58dc800640 1 --2- 172.21.15.50:0/417550432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58d40b4de0 0x7f58d40cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58dc800640 1 -- 172.21.15.50:0/417550432 >> 172.21.15.50:0/417550432 conn(0x7f58d40b2040 msgr2=0x7f58d40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:11.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58dc800640 1 -- 172.21.15.50:0/417550432 shutdown_connections 2024-09-21T06:08:11.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.503+0000 7f58dc800640 1 -- 172.21.15.50:0/417550432 wait complete. 2024-09-21T06:08:11.525 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-21T06:08:11.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.735+0000 7f2a86200640 1 -- 172.21.15.50:0/1840076889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8015efc0 msgr2=0x7f2a80070910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:11.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.735+0000 7f2a86200640 1 --2- 172.21.15.50:0/1840076889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8015efc0 0x7f2a80070910 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f2a80072d40 tx=0x7f2a7001c8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.735+0000 7f2a86200640 1 -- 172.21.15.50:0/1840076889 shutdown_connections 2024-09-21T06:08:11.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.735+0000 7f2a86200640 1 --2- 172.21.15.50:0/1840076889 >> v2:172.21.15.88:3300/0 conn(0x7f2a800b0810 0x7f2a80068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.735+0000 7f2a86200640 1 --2- 172.21.15.50:0/1840076889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8015efc0 0x7f2a80070910 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.735+0000 7f2a86200640 1 -- 172.21.15.50:0/1840076889 >> 172.21.15.50:0/1840076889 conn(0x7f2a800b35f0 msgr2=0x7f2a800b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:11.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.735+0000 7f2a86200640 1 -- 172.21.15.50:0/1840076889 shutdown_connections 2024-09-21T06:08:11.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.735+0000 7f2a86200640 1 -- 172.21.15.50:0/1840076889 wait complete. 2024-09-21T06:08:11.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.739+0000 7f2a86200640 1 Processor -- start 2024-09-21T06:08:11.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.739+0000 7f2a86200640 1 -- start start 2024-09-21T06:08:11.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.739+0000 7f2a86200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a800b0810 0x7f2a8014ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:11.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a86200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a800b0c20 0x7f2a80156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a7f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a800b0810 0x7f2a8014ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a7f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a800b0810 0x7f2a8014ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:56754/0 (socket says 172.21.15.50:56754) 2024-09-21T06:08:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a86200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8015efc0 0x7f2a801507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a86200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8016c960 con 0x7f2a8015efc0 2024-09-21T06:08:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a86200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2a80163520 con 0x7f2a800b0c20 2024-09-21T06:08:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a86200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8016cae0 con 0x7f2a800b0810 2024-09-21T06:08:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a7f400640 1 -- 172.21.15.50:0/92014226 learned_addr learned my addr 172.21.15.50:0/92014226 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a7ea00640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a800b0c20 0x7f2a80156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a7fe00640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8015efc0 0x7f2a801507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a7ea00640 1 -- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a800b0810 msgr2=0x7f2a8014ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a7ea00640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a800b0810 0x7f2a8014ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a7ea00640 1 -- 172.21.15.50:0/92014226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8015efc0 msgr2=0x7f2a801507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a7ea00640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8015efc0 0x7f2a801507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a7ea00640 1 -- 172.21.15.50:0/92014226 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a80201250 con 0x7f2a800b0c20 2024-09-21T06:08:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a7fe00640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8015efc0 0x7f2a801507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a7ea00640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a800b0c20 0x7f2a80156ab0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00d680 tx=0x7f2a6c00db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:11.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a77400640 1 -- 172.21.15.50:0/92014226 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c017070 con 0x7f2a800b0c20 2024-09-21T06:08:11.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a86200640 1 -- 172.21.15.50:0/92014226 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a80201530 con 0x7f2a800b0c20 2024-09-21T06:08:11.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a77400640 1 -- 172.21.15.50:0/92014226 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a6c009b10 con 0x7f2a800b0c20 2024-09-21T06:08:11.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.743+0000 7f2a77400640 1 -- 172.21.15.50:0/92014226 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c010460 con 0x7f2a800b0c20 2024-09-21T06:08:11.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.747+0000 7f2a86200640 1 -- 172.21.15.50:0/92014226 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a80201af0 con 0x7f2a800b0c20 2024-09-21T06:08:11.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.747+0000 7f2a86200640 1 -- 172.21.15.50:0/92014226 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a80068df0 con 0x7f2a800b0c20 2024-09-21T06:08:11.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.747+0000 7f2a77400640 1 -- 172.21.15.50:0/92014226 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f2a6c01a070 con 0x7f2a800b0c20 2024-09-21T06:08:11.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.751+0000 7f2a77400640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a60074880 0x7f2a60076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:11.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.751+0000 7f2a77400640 1 -- 172.21.15.50:0/92014226 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7f2a6c093080 con 0x7f2a800b0c20 2024-09-21T06:08:11.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.751+0000 7f2a7f400640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a60074880 0x7f2a60076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:11.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.751+0000 7f2a7f400640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a60074880 0x7f2a60076d90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009230 tx=0x7f2a7001b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:11.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:11.755+0000 7f2a77400640 1 -- 172.21.15.50:0/92014226 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2a6c05f630 con 0x7f2a800b0c20 2024-09-21T06:08:12.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.159+0000 7f2a86200640 1 -- 172.21.15.50:0/92014226 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f2a80201000 con 0x7f2a60074880 2024-09-21T06:08:12.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.195+0000 7f2a77400640 1 -- 172.21.15.50:0/92014226 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2a80201000 con 0x7f2a60074880 2024-09-21T06:08:12.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.199+0000 7f2a86200640 1 -- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a60074880 msgr2=0x7f2a60076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:12.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.199+0000 7f2a86200640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a60074880 0x7f2a60076d90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009230 tx=0x7f2a7001b100 comp rx=0 tx=0).stop 2024-09-21T06:08:12.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.199+0000 7f2a86200640 1 -- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a800b0c20 msgr2=0x7f2a80156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:12.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.199+0000 7f2a86200640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a800b0c20 0x7f2a80156ab0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00d680 tx=0x7f2a6c00db40 comp rx=0 tx=0).stop 2024-09-21T06:08:12.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.199+0000 7f2a86200640 1 -- 172.21.15.50:0/92014226 shutdown_connections 2024-09-21T06:08:12.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.199+0000 7f2a86200640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2a60074880 0x7f2a60076d90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.199+0000 7f2a86200640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8015efc0 0x7f2a801507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.199+0000 7f2a86200640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a800b0c20 0x7f2a80156ab0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.199+0000 7f2a86200640 1 --2- 172.21.15.50:0/92014226 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2a800b0810 0x7f2a8014ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.199+0000 7f2a86200640 1 -- 172.21.15.50:0/92014226 >> 172.21.15.50:0/92014226 conn(0x7f2a800b35f0 msgr2=0x7f2a800b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:12.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.199+0000 7f2a86200640 1 -- 172.21.15.50:0/92014226 shutdown_connections 2024-09-21T06:08:12.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.203+0000 7f2a86200640 1 -- 172.21.15.50:0/92014226 wait complete. 2024-09-21T06:08:12.254 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone877615 2024-09-21T06:08:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.427+0000 7efe3ae00640 1 -- 172.21.15.50:0/3315938238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe340b4de0 msgr2=0x7efe340b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.427+0000 7efe3ae00640 1 --2- 172.21.15.50:0/3315938238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe340b4de0 0x7efe340b51f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009bc0 tx=0x7efe2c01cae0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.427+0000 7efe3ae00640 1 -- 172.21.15.50:0/3315938238 shutdown_connections 2024-09-21T06:08:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.427+0000 7efe3ae00640 1 --2- 172.21.15.50:0/3315938238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe340b4de0 0x7efe340b51f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.427+0000 7efe3ae00640 1 --2- 172.21.15.50:0/3315938238 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efe341527b0 0x7efe3415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.427+0000 7efe3ae00640 1 -- 172.21.15.50:0/3315938238 >> 172.21.15.50:0/3315938238 conn(0x7efe340b2040 msgr2=0x7efe340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:12.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.427+0000 7efe3ae00640 1 -- 172.21.15.50:0/3315938238 shutdown_connections 2024-09-21T06:08:12.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.427+0000 7efe3ae00640 1 -- 172.21.15.50:0/3315938238 wait complete. 2024-09-21T06:08:12.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe3ae00640 1 Processor -- start 2024-09-21T06:08:12.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe3ae00640 1 -- start start 2024-09-21T06:08:12.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe3ae00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efe340b4de0 0x7efe3413f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:12.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe3ae00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe340b5c70 0x7efe341fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:12.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe3ae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe341527b0 0x7efe341fc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:12.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe3ae00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efe34168a00 con 0x7efe340b5c70 2024-09-21T06:08:12.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe3ae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe34168880 con 0x7efe341527b0 2024-09-21T06:08:12.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe3ae00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7efe34168b80 con 0x7efe340b4de0 2024-09-21T06:08:12.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe2b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe340b5c70 0x7efe341fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:12.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe2b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe340b5c70 0x7efe341fbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53056/0 (socket says 172.21.15.50:53056) 2024-09-21T06:08:12.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe2b400640 1 -- 172.21.15.50:0/2987498141 learned_addr learned my addr 172.21.15.50:0/2987498141 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:12.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe33e00640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efe340b4de0 0x7efe3413f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:12.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe38e00640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe341527b0 0x7efe341fc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:12.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe2b400640 1 -- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efe340b4de0 msgr2=0x7efe3413f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:12.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe2b400640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efe340b4de0 0x7efe3413f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe2b400640 1 -- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe341527b0 msgr2=0x7efe341fc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:12.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe2b400640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe341527b0 0x7efe341fc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe2b400640 1 -- 172.21.15.50:0/2987498141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe341fcc80 con 0x7efe340b5c70 2024-09-21T06:08:12.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe2b400640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe340b5c70 0x7efe341fbf30 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00e9c0 tx=0x7efe1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:12.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe30c00640 1 -- 172.21.15.50:0/2987498141 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c00cd80 con 0x7efe340b5c70 2024-09-21T06:08:12.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe30c00640 1 -- 172.21.15.50:0/2987498141 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe1c004020 con 0x7efe340b5c70 2024-09-21T06:08:12.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe3ae00640 1 -- 172.21.15.50:0/2987498141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe34142370 con 0x7efe340b5c70 2024-09-21T06:08:12.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe30c00640 1 -- 172.21.15.50:0/2987498141 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c004450 con 0x7efe340b5c70 2024-09-21T06:08:12.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.431+0000 7efe3ae00640 1 -- 172.21.15.50:0/2987498141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe341f9610 con 0x7efe340b5c70 2024-09-21T06:08:12.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.435+0000 7efe3ae00640 1 -- 172.21.15.50:0/2987498141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe34142a90 con 0x7efe340b5c70 2024-09-21T06:08:12.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.435+0000 7efe30c00640 1 -- 172.21.15.50:0/2987498141 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7efe1c003600 con 0x7efe340b5c70 2024-09-21T06:08:12.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.435+0000 7efe30c00640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efe10074880 0x7efe10076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:12.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.435+0000 7efe30c00640 1 -- 172.21.15.50:0/2987498141 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5794+0+0 (secure 0 0 0) 0x7efe1c093790 con 0x7efe340b5c70 2024-09-21T06:08:12.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.435+0000 7efe33e00640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efe10074880 0x7efe10076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:12.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.435+0000 7efe33e00640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efe10074880 0x7efe10076d90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efe24007d70 tx=0x7efe24007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.439+0000 7efe30c00640 1 -- 172.21.15.50:0/2987498141 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efe1c05fc50 con 0x7efe340b5c70 2024-09-21T06:08:12.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.671+0000 7efe3ae00640 1 -- 172.21.15.50:0/2987498141 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone877615", "target": ["mon-mgr", ""]}) -- 0x7efe3406b540 con 0x7efe10074880 2024-09-21T06:08:12.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.715+0000 7efe30c00640 1 -- 172.21.15.50:0/2987498141 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efe24007de0 con 0x7efe10074880 2024-09-21T06:08:12.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.719+0000 7efe3ae00640 1 -- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efe10074880 msgr2=0x7efe10076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:12.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.719+0000 7efe3ae00640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efe10074880 0x7efe10076d90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efe24007d70 tx=0x7efe24007d00 comp rx=0 tx=0).stop 2024-09-21T06:08:12.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.719+0000 7efe3ae00640 1 -- 172.21.15.50:0/2987498141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe340b5c70 msgr2=0x7efe341fbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:12.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.719+0000 7efe3ae00640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe340b5c70 0x7efe341fbf30 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00e9c0 tx=0x7efe1c00ee80 comp rx=0 tx=0).stop 2024-09-21T06:08:12.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.719+0000 7efe3ae00640 1 -- 172.21.15.50:0/2987498141 shutdown_connections 2024-09-21T06:08:12.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.719+0000 7efe3ae00640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe341527b0 0x7efe341fc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.719+0000 7efe3ae00640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efe10074880 0x7efe10076d90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.719+0000 7efe3ae00640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe340b5c70 0x7efe341fbf30 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.719+0000 7efe3ae00640 1 --2- 172.21.15.50:0/2987498141 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efe340b4de0 0x7efe3413f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:12.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.719+0000 7efe3ae00640 1 -- 172.21.15.50:0/2987498141 >> 172.21.15.50:0/2987498141 conn(0x7efe340b2040 msgr2=0x7efe340affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:12.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.719+0000 7efe3ae00640 1 -- 172.21.15.50:0/2987498141 shutdown_connections 2024-09-21T06:08:12.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:12.719+0000 7efe3ae00640 1 -- 172.21.15.50:0/2987498141 wait complete. 2024-09-21T06:08:12.755 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone972718 2024-09-21T06:08:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.031+0000 7fa942400640 1 -- 172.21.15.50:0/6703090 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa93c06cdf0 msgr2=0x7fa93c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.031+0000 7fa942400640 1 --2- 172.21.15.50:0/6703090 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa93c06cdf0 0x7fa93c06d200 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa928009930 tx=0x7fa92801c890 comp rx=0 tx=0).stop 2024-09-21T06:08:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.031+0000 7fa942400640 1 -- 172.21.15.50:0/6703090 shutdown_connections 2024-09-21T06:08:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.031+0000 7fa942400640 1 --2- 172.21.15.50:0/6703090 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa93c06cdf0 0x7fa93c06d200 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.031+0000 7fa942400640 1 -- 172.21.15.50:0/6703090 >> 172.21.15.50:0/6703090 conn(0x7fa93c0b35f0 msgr2=0x7fa93c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:13.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.035+0000 7fa942400640 1 -- 172.21.15.50:0/6703090 shutdown_connections 2024-09-21T06:08:13.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.035+0000 7fa942400640 1 -- 172.21.15.50:0/6703090 wait complete. 2024-09-21T06:08:13.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.039+0000 7fa942400640 1 Processor -- start 2024-09-21T06:08:13.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.039+0000 7fa942400640 1 -- start start 2024-09-21T06:08:13.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.039+0000 7fa942400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa93c06cdf0 0x7fa93c0cf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:13.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.039+0000 7fa942400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa93c06fc90 0x7fa93c147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:13.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.039+0000 7fa942400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa93c159130 0x7fa93c147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:13.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.039+0000 7fa93aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa93c06fc90 0x7fa93c147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:13.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.039+0000 7fa93aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa93c06fc90 0x7fa93c147150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53062/0 (socket says 172.21.15.50:53062) 2024-09-21T06:08:13.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.039+0000 7fa93aa00640 1 -- 172.21.15.50:0/2314104656 learned_addr learned my addr 172.21.15.50:0/2314104656 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:13.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c16a570 con 0x7fa93c06fc90 2024-09-21T06:08:13.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c16a3f0 con 0x7fa93c06cdf0 2024-09-21T06:08:13.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa93c16a6f0 con 0x7fa93c159130 2024-09-21T06:08:13.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa93b400640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa93c06cdf0 0x7fa93c0cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:13.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa93be00640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa93c159130 0x7fa93c147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:13.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa93b400640 1 -- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa93c159130 msgr2=0x7fa93c147690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:13.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa93b400640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa93c159130 0x7fa93c147690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:13.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa93b400640 1 -- 172.21.15.50:0/2314104656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa93c06fc90 msgr2=0x7fa93c147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:13.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa93b400640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa93c06fc90 0x7fa93c147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:13.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa93b400640 1 -- 172.21.15.50:0/2314104656 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93c155c50 con 0x7fa93c06cdf0 2024-09-21T06:08:13.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa93b400640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa93c06cdf0 0x7fa93c0cf300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa92c001e50 tx=0x7fa92c001e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:13.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa933400640 1 -- 172.21.15.50:0/2314104656 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c005220 con 0x7fa93c06cdf0 2024-09-21T06:08:13.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa933400640 1 -- 172.21.15.50:0/2314104656 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa92c0054b0 con 0x7fa93c06cdf0 2024-09-21T06:08:13.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa933400640 1 -- 172.21.15.50:0/2314104656 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c006a40 con 0x7fa93c06cdf0 2024-09-21T06:08:13.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93c0ce820 con 0x7fa93c06cdf0 2024-09-21T06:08:13.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.043+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa93c0cede0 con 0x7fa93c06cdf0 2024-09-21T06:08:13.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.047+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa93c06d200 con 0x7fa93c06cdf0 2024-09-21T06:08:13.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.051+0000 7fa933400640 1 -- 172.21.15.50:0/2314104656 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fa92c006be0 con 0x7fa93c06cdf0 2024-09-21T06:08:13.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.051+0000 7fa933400640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa91c0748d0 0x7fa91c076de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:13.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.051+0000 7fa933400640 1 -- 172.21.15.50:0/2314104656 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7fa92c0963a0 con 0x7fa93c06cdf0 2024-09-21T06:08:13.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.051+0000 7fa93aa00640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa91c0748d0 0x7fa91c076de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:13.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.051+0000 7fa93aa00640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa91c0748d0 0x7fa91c076de0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa928009930 tx=0x7fa92801b0d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:13.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.055+0000 7fa933400640 1 -- 172.21.15.50:0/2314104656 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa92c062af0 con 0x7fa93c06cdf0 2024-09-21T06:08:13.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.459+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone972718", "target": ["mon-mgr", ""]}) -- 0x7fa93c06b540 con 0x7fa91c0748d0 2024-09-21T06:08:13.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.503+0000 7fa933400640 1 -- 172.21.15.50:0/2314104656 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa928004230 con 0x7fa91c0748d0 2024-09-21T06:08:13.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.507+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa91c0748d0 msgr2=0x7fa91c076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:13.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.507+0000 7fa942400640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa91c0748d0 0x7fa91c076de0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa928009930 tx=0x7fa92801b0d0 comp rx=0 tx=0).stop 2024-09-21T06:08:13.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.507+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa93c06cdf0 msgr2=0x7fa93c0cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:13.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.507+0000 7fa942400640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa93c06cdf0 0x7fa93c0cf300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa92c001e50 tx=0x7fa92c001e80 comp rx=0 tx=0).stop 2024-09-21T06:08:13.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.507+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 shutdown_connections 2024-09-21T06:08:13.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.507+0000 7fa942400640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa93c159130 0x7fa93c147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:13.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.507+0000 7fa942400640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa91c0748d0 0x7fa91c076de0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:13.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.507+0000 7fa942400640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa93c06fc90 0x7fa93c147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:13.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.507+0000 7fa942400640 1 --2- 172.21.15.50:0/2314104656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa93c06cdf0 0x7fa93c0cf300 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:13.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.507+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 >> 172.21.15.50:0/2314104656 conn(0x7fa93c0b35f0 msgr2=0x7fa93c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:13.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.507+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 shutdown_connections 2024-09-21T06:08:13.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.507+0000 7fa942400640 1 -- 172.21.15.50:0/2314104656 wait complete. 2024-09-21T06:08:13.529 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone216314 2024-09-21T06:08:13.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.747+0000 7f599de00640 1 -- 172.21.15.50:0/1331080239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f597c006270 msgr2=0x7f597c003450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:13.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.747+0000 7f599de00640 1 --2- 172.21.15.50:0/1331080239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f597c006270 0x7f597c003450 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5990009ee0 tx=0x7f599001d600 comp rx=0 tx=0).stop 2024-09-21T06:08:13.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.747+0000 7f599de00640 1 -- 172.21.15.50:0/1331080239 shutdown_connections 2024-09-21T06:08:13.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.747+0000 7f599de00640 1 --2- 172.21.15.50:0/1331080239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f597c006270 0x7f597c003450 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:13.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.747+0000 7f599de00640 1 -- 172.21.15.50:0/1331080239 >> 172.21.15.50:0/1331080239 conn(0x7f599806e1d0 msgr2=0x7f599806e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:13.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.747+0000 7f599de00640 1 -- 172.21.15.50:0/1331080239 shutdown_connections 2024-09-21T06:08:13.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.747+0000 7f599de00640 1 -- 172.21.15.50:0/1331080239 wait complete. 2024-09-21T06:08:13.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.751+0000 7f599de00640 1 Processor -- start 2024-09-21T06:08:13.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.751+0000 7f599de00640 1 -- start start 2024-09-21T06:08:13.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f599de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f597c006270 0x7f5998154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:13.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f599de00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5998156920 0x7f59980aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:13.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f599de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59980aaf40 0x7f59980ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:13.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f599de00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f59980cf130 con 0x7f597c006270 2024-09-21T06:08:13.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f599de00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f59980cefb0 con 0x7f59980aaf40 2024-09-21T06:08:13.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f599de00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f59980cf2b0 con 0x7f5998156920 2024-09-21T06:08:13.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f5996a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f597c006270 0x7f5998154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:13.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f5996a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f597c006270 0x7f5998154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53086/0 (socket says 172.21.15.50:53086) 2024-09-21T06:08:13.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f5996a00640 1 -- 172.21.15.50:0/742656098 learned_addr learned my addr 172.21.15.50:0/742656098 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:13.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f5996000640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5998156920 0x7f59980aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:13.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f5996a00640 1 -- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5998156920 msgr2=0x7f59980aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:13.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f5996a00640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5998156920 0x7f59980aaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:13.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f5996a00640 1 -- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59980aaf40 msgr2=0x7f59980ab350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:08:13.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f5996a00640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59980aaf40 0x7f59980ab350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:13.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f5996a00640 1 -- 172.21.15.50:0/742656098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59981566f0 con 0x7f597c006270 2024-09-21T06:08:13.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.755+0000 7f5996a00640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f597c006270 0x7f5998154c20 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f5988004db0 tx=0x7f5988001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:13.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.759+0000 7f598f400640 1 -- 172.21.15.50:0/742656098 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59880107d0 con 0x7f597c006270 2024-09-21T06:08:13.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.759+0000 7f598f400640 1 -- 172.21.15.50:0/742656098 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5988016070 con 0x7f597c006270 2024-09-21T06:08:13.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.759+0000 7f598f400640 1 -- 172.21.15.50:0/742656098 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5988010cc0 con 0x7f597c006270 2024-09-21T06:08:13.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.759+0000 7f599de00640 1 -- 172.21.15.50:0/742656098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59980ab9e0 con 0x7f597c006270 2024-09-21T06:08:13.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.759+0000 7f599de00640 1 -- 172.21.15.50:0/742656098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5998160450 con 0x7f597c006270 2024-09-21T06:08:13.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.759+0000 7f599de00640 1 -- 172.21.15.50:0/742656098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59980cd3d0 con 0x7f597c006270 2024-09-21T06:08:13.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.763+0000 7f598f400640 1 -- 172.21.15.50:0/742656098 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f598801b460 con 0x7f597c006270 2024-09-21T06:08:13.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.763+0000 7f598f400640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5978074880 0x7f5978076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:13.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.767+0000 7f598f400640 1 -- 172.21.15.50:0/742656098 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7f5988093820 con 0x7f597c006270 2024-09-21T06:08:13.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.767+0000 7f5996000640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5978074880 0x7f5978076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:13.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.767+0000 7f5996000640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5978074880 0x7f5978076d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f59980b1bf0 tx=0x7f59840067b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:13.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:13.779+0000 7f598f400640 1 -- 172.21.15.50:0/742656098 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f599815f320 con 0x7f597c006270 2024-09-21T06:08:14.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.131+0000 7f599de00640 1 -- 172.21.15.50:0/742656098 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone216314", "target": ["mon-mgr", ""]}) -- 0x7f5998155160 con 0x7f5978074880 2024-09-21T06:08:14.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.151+0000 7f598f400640 1 -- 172.21.15.50:0/742656098 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5998155160 con 0x7f5978074880 2024-09-21T06:08:14.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.155+0000 7f599de00640 1 -- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5978074880 msgr2=0x7f5978076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:14.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.155+0000 7f599de00640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5978074880 0x7f5978076d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f59980b1bf0 tx=0x7f59840067b0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.155+0000 7f599de00640 1 -- 172.21.15.50:0/742656098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f597c006270 msgr2=0x7f5998154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:14.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.155+0000 7f599de00640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f597c006270 0x7f5998154c20 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f5988004db0 tx=0x7f5988001640 comp rx=0 tx=0).stop 2024-09-21T06:08:14.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.155+0000 7f599de00640 1 -- 172.21.15.50:0/742656098 shutdown_connections 2024-09-21T06:08:14.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.155+0000 7f599de00640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59980aaf40 0x7f59980ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.155+0000 7f599de00640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5998156920 0x7f59980aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.155+0000 7f599de00640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5978074880 0x7f5978076d90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.155+0000 7f599de00640 1 --2- 172.21.15.50:0/742656098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f597c006270 0x7f5998154c20 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.155+0000 7f599de00640 1 -- 172.21.15.50:0/742656098 >> 172.21.15.50:0/742656098 conn(0x7f599806e1d0 msgr2=0x7f599815cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:14.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.155+0000 7f599de00640 1 -- 172.21.15.50:0/742656098 shutdown_connections 2024-09-21T06:08:14.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.155+0000 7f599de00640 1 -- 172.21.15.50:0/742656098 wait complete. 2024-09-21T06:08:14.175 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone67087 2024-09-21T06:08:14.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.339+0000 7fd727600640 1 -- 172.21.15.50:0/1147350232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7200b5c70 msgr2=0x7fd72015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:14.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.339+0000 7fd727600640 1 --2- 172.21.15.50:0/1147350232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7200b5c70 0x7fd72015bb60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd714009840 tx=0x7fd71401c7f0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.339+0000 7fd727600640 1 -- 172.21.15.50:0/1147350232 shutdown_connections 2024-09-21T06:08:14.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.339+0000 7fd727600640 1 --2- 172.21.15.50:0/1147350232 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7200b4de0 0x7fd7200b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.339+0000 7fd727600640 1 --2- 172.21.15.50:0/1147350232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7200b5c70 0x7fd72015bb60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.339+0000 7fd727600640 1 -- 172.21.15.50:0/1147350232 >> 172.21.15.50:0/1147350232 conn(0x7fd7200b2040 msgr2=0x7fd7200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:14.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.339+0000 7fd727600640 1 -- 172.21.15.50:0/1147350232 shutdown_connections 2024-09-21T06:08:14.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.339+0000 7fd727600640 1 -- 172.21.15.50:0/1147350232 wait complete. 2024-09-21T06:08:14.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd727600640 1 Processor -- start 2024-09-21T06:08:14.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd727600640 1 -- start start 2024-09-21T06:08:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd727600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200b4de0 0x7fd7200ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd727600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7200b5c70 0x7fd720151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd727600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7201527b0 0x7fd72013c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd727600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd720168a20 con 0x7fd7200b4de0 2024-09-21T06:08:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd727600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201688a0 con 0x7fd7201527b0 2024-09-21T06:08:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd727600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd720168ba0 con 0x7fd7200b5c70 2024-09-21T06:08:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd724c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200b4de0 0x7fd7200ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd724c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200b4de0 0x7fd7200ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53100/0 (socket says 172.21.15.50:53100) 2024-09-21T06:08:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd724c00640 1 -- 172.21.15.50:0/2403418745 learned_addr learned my addr 172.21.15.50:0/2403418745 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd725600640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7201527b0 0x7fd72013c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd71fe00640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7200b5c70 0x7fd720151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd725600640 1 -- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7200b5c70 msgr2=0x7fd720151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:14.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd725600640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7200b5c70 0x7fd720151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd725600640 1 -- 172.21.15.50:0/2403418745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200b4de0 msgr2=0x7fd7200ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:14.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd725600640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200b4de0 0x7fd7200ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd725600640 1 -- 172.21.15.50:0/2403418745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7200cea80 con 0x7fd7201527b0 2024-09-21T06:08:14.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd724c00640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200b4de0 0x7fd7200ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:14.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd725600640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7201527b0 0x7fd72013c250 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd71800ebc0 tx=0x7fd71800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:14.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd71cc00640 1 -- 172.21.15.50:0/2403418745 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd718003b80 con 0x7fd7201527b0 2024-09-21T06:08:14.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd71cc00640 1 -- 172.21.15.50:0/2403418745 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd718003d20 con 0x7fd7201527b0 2024-09-21T06:08:14.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd71cc00640 1 -- 172.21.15.50:0/2403418745 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd718004280 con 0x7fd7201527b0 2024-09-21T06:08:14.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd727600640 1 -- 172.21.15.50:0/2403418745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7200ae6f0 con 0x7fd7201527b0 2024-09-21T06:08:14.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.343+0000 7fd727600640 1 -- 172.21.15.50:0/2403418745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7200aecb0 con 0x7fd7201527b0 2024-09-21T06:08:14.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.347+0000 7fd727600640 1 -- 172.21.15.50:0/2403418745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7200cf8f0 con 0x7fd7201527b0 2024-09-21T06:08:14.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.347+0000 7fd71cc00640 1 -- 172.21.15.50:0/2403418745 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fd718004420 con 0x7fd7201527b0 2024-09-21T06:08:14.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.347+0000 7fd71cc00640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd700074880 0x7fd700076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:14.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.347+0000 7fd71cc00640 1 -- 172.21.15.50:0/2403418745 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7fd7180939f0 con 0x7fd7201527b0 2024-09-21T06:08:14.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.347+0000 7fd724c00640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd700074880 0x7fd700076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:14.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.347+0000 7fd724c00640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd700074880 0x7fd700076d90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd71401a420 tx=0x7fd71401b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:14.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.351+0000 7fd71cc00640 1 -- 172.21.15.50:0/2403418745 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd71805feb0 con 0x7fd7201527b0 2024-09-21T06:08:14.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.555+0000 7fd727600640 1 -- 172.21.15.50:0/2403418745 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone67087", "target": ["mon-mgr", ""]}) -- 0x7fd72006b540 con 0x7fd700074880 2024-09-21T06:08:14.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.575+0000 7fd71cc00640 1 -- 172.21.15.50:0/2403418745 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd714004040 con 0x7fd700074880 2024-09-21T06:08:14.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.579+0000 7fd727600640 1 -- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd700074880 msgr2=0x7fd700076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:14.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.579+0000 7fd727600640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd700074880 0x7fd700076d90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd71401a420 tx=0x7fd71401b030 comp rx=0 tx=0).stop 2024-09-21T06:08:14.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.579+0000 7fd727600640 1 -- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7201527b0 msgr2=0x7fd72013c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:14.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.579+0000 7fd727600640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7201527b0 0x7fd72013c250 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd71800ebc0 tx=0x7fd71800c6a0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.579+0000 7fd727600640 1 -- 172.21.15.50:0/2403418745 shutdown_connections 2024-09-21T06:08:14.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.579+0000 7fd727600640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7201527b0 0x7fd72013c250 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.579+0000 7fd727600640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7200b5c70 0x7fd720151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.579+0000 7fd727600640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd700074880 0x7fd700076d90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.579+0000 7fd727600640 1 --2- 172.21.15.50:0/2403418745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200b4de0 0x7fd7200ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.579+0000 7fd727600640 1 -- 172.21.15.50:0/2403418745 >> 172.21.15.50:0/2403418745 conn(0x7fd7200b2040 msgr2=0x7fd7200af820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:14.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.579+0000 7fd727600640 1 -- 172.21.15.50:0/2403418745 shutdown_connections 2024-09-21T06:08:14.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.579+0000 7fd727600640 1 -- 172.21.15.50:0/2403418745 wait complete. 2024-09-21T06:08:14.608 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone64155 2024-09-21T06:08:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.875+0000 7f664b200640 1 -- 172.21.15.50:0/144768096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f664c06dbc0 msgr2=0x7f664c06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:14.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.875+0000 7f664b200640 1 --2- 172.21.15.50:0/144768096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f664c06dbc0 0x7f664c06dfd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6638009840 tx=0x7f663801c620 comp rx=0 tx=0).stop 2024-09-21T06:08:14.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.875+0000 7f664b200640 1 -- 172.21.15.50:0/144768096 shutdown_connections 2024-09-21T06:08:14.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.875+0000 7f664b200640 1 --2- 172.21.15.50:0/144768096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f664c06dbc0 0x7f664c06dfd0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.875+0000 7f664b200640 1 -- 172.21.15.50:0/144768096 >> 172.21.15.50:0/144768096 conn(0x7f664c0b41f0 msgr2=0x7f664c0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:14.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.875+0000 7f664b200640 1 -- 172.21.15.50:0/144768096 shutdown_connections 2024-09-21T06:08:14.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.875+0000 7f664b200640 1 -- 172.21.15.50:0/144768096 wait complete. 2024-09-21T06:08:14.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f664b200640 1 Processor -- start 2024-09-21T06:08:14.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f664b200640 1 -- start start 2024-09-21T06:08:14.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f664b200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f664c06dbc0 0x7f664c09eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f664b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f664c0d1140 0x7f664c0691c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f664b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f664c15f5c0 0x7f664c1fa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f664b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f664c169df0 con 0x7f664c0d1140 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f664b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f664c169c70 con 0x7f664c15f5c0 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f664b200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f664c169f70 con 0x7f664c06dbc0 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f6649400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f664c0d1140 0x7f664c0691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f6649400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f664c0d1140 0x7f664c0691c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53112/0 (socket says 172.21.15.50:53112) 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f6649400640 1 -- 172.21.15.50:0/2952223688 learned_addr learned my addr 172.21.15.50:0/2952223688 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f6649e00640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f664c06dbc0 0x7f664c09eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f664a800640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f664c15f5c0 0x7f664c1fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f6649400640 1 -- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f664c06dbc0 msgr2=0x7f664c09eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f6649400640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f664c06dbc0 0x7f664c09eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f6649400640 1 -- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f664c15f5c0 msgr2=0x7f664c1fa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f6649400640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f664c15f5c0 0x7f664c1fa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.879+0000 7f6649400640 1 -- 172.21.15.50:0/2952223688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664c1fa100 con 0x7f664c0d1140 2024-09-21T06:08:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.883+0000 7f6649400640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f664c0d1140 0x7f664c0691c0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6638009790 tx=0x7f66380a95c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.883+0000 7f6642000640 1 -- 172.21.15.50:0/2952223688 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66380ad070 con 0x7f664c0d1140 2024-09-21T06:08:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.883+0000 7f664b200640 1 -- 172.21.15.50:0/2952223688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664c09f0c0 con 0x7f664c0d1140 2024-09-21T06:08:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.883+0000 7f6642000640 1 -- 172.21.15.50:0/2952223688 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66380aad80 con 0x7f664c0d1140 2024-09-21T06:08:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.883+0000 7f6642000640 1 -- 172.21.15.50:0/2952223688 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638003820 con 0x7f664c0d1140 2024-09-21T06:08:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.883+0000 7f664b200640 1 -- 172.21.15.50:0/2952223688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664c0ce220 con 0x7f664c0d1140 2024-09-21T06:08:14.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.883+0000 7f664b200640 1 -- 172.21.15.50:0/2952223688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f664c06dfd0 con 0x7f664c0d1140 2024-09-21T06:08:14.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.883+0000 7f6642000640 1 -- 172.21.15.50:0/2952223688 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f66380b0070 con 0x7f664c0d1140 2024-09-21T06:08:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.883+0000 7f6642000640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f662c074880 0x7f662c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.883+0000 7f6642000640 1 -- 172.21.15.50:0/2952223688 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7f663812d450 con 0x7f664c0d1140 2024-09-21T06:08:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.883+0000 7f6649e00640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f662c074880 0x7f662c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.887+0000 7f6649e00640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f662c074880 0x7f662c076d90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f663c007fd0 tx=0x7f663c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:14.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:14.891+0000 7f6642000640 1 -- 172.21.15.50:0/2952223688 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f66380c3070 con 0x7f664c0d1140 2024-09-21T06:08:15.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.091+0000 7f664b200640 1 -- 172.21.15.50:0/2952223688 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone64155", "target": ["mon-mgr", ""]}) -- 0x7f664c167230 con 0x7f662c074880 2024-09-21T06:08:15.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f6642000640 1 -- 172.21.15.50:0/2952223688 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f664c167230 con 0x7f662c074880 2024-09-21T06:08:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f664b200640 1 -- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f662c074880 msgr2=0x7f662c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f664b200640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f662c074880 0x7f662c076d90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f663c007fd0 tx=0x7f663c009040 comp rx=0 tx=0).stop 2024-09-21T06:08:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f664b200640 1 -- 172.21.15.50:0/2952223688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f664c0d1140 msgr2=0x7f664c0691c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f664b200640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f664c0d1140 0x7f664c0691c0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6638009790 tx=0x7f66380a95c0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f664b200640 1 -- 172.21.15.50:0/2952223688 shutdown_connections 2024-09-21T06:08:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f664b200640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f664c15f5c0 0x7f664c1fa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f664b200640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f662c074880 0x7f662c076d90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f664b200640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f664c0d1140 0x7f664c0691c0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f664b200640 1 --2- 172.21.15.50:0/2952223688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f664c06dbc0 0x7f664c09eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f664b200640 1 -- 172.21.15.50:0/2952223688 >> 172.21.15.50:0/2952223688 conn(0x7f664c0b41f0 msgr2=0x7f664c070020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f664b200640 1 -- 172.21.15.50:0/2952223688 shutdown_connections 2024-09-21T06:08:15.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.111+0000 7f664b200640 1 -- 172.21.15.50:0/2952223688 wait complete. 2024-09-21T06:08:15.132 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone647110 2024-09-21T06:08:15.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.299+0000 7f6187200640 1 -- 172.21.15.50:0/1147059169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61800cefa0 msgr2=0x7f61800cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:15.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.299+0000 7f6187200640 1 --2- 172.21.15.50:0/1147059169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61800cefa0 0x7f61800cf3b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f6168009900 tx=0x7f616801c8e0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.299+0000 7f6187200640 1 -- 172.21.15.50:0/1147059169 shutdown_connections 2024-09-21T06:08:15.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.299+0000 7f6187200640 1 --2- 172.21.15.50:0/1147059169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61800cefa0 0x7f61800cf3b0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.299+0000 7f6187200640 1 --2- 172.21.15.50:0/1147059169 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f61800749a0 0x7f6180074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.299+0000 7f6187200640 1 -- 172.21.15.50:0/1147059169 >> 172.21.15.50:0/1147059169 conn(0x7f61800b2040 msgr2=0x7f61800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:15.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.299+0000 7f6187200640 1 -- 172.21.15.50:0/1147059169 shutdown_connections 2024-09-21T06:08:15.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.299+0000 7f6187200640 1 -- 172.21.15.50:0/1147059169 wait complete. 2024-09-21T06:08:15.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6187200640 1 Processor -- start 2024-09-21T06:08:15.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6187200640 1 -- start start 2024-09-21T06:08:15.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6187200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f61800749a0 0x7f61800af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:15.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6187200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61800cefa0 0x7f61801404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6187200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6180160330 0x7f6180159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6187200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f618016a060 con 0x7f61800cefa0 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6187200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6180169ee0 con 0x7f6180160330 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6187200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f618016a1e0 con 0x7f61800749a0 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6177e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61800cefa0 0x7f61801404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6177e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61800cefa0 0x7f61801404b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53130/0 (socket says 172.21.15.50:53130) 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6177e00640 1 -- 172.21.15.50:0/3489489086 learned_addr learned my addr 172.21.15.50:0/3489489086 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f617fe00640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f61800749a0 0x7f61800af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6185200640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6180160330 0x7f6180159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6177e00640 1 -- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f61800749a0 msgr2=0x7f61800af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6177e00640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f61800749a0 0x7f61800af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6177e00640 1 -- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6180160330 msgr2=0x7f6180159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6177e00640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6180160330 0x7f6180159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6177e00640 1 -- 172.21.15.50:0/3489489086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618015a4d0 con 0x7f61800cefa0 2024-09-21T06:08:15.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6185200640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6180160330 0x7f6180159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:15.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6177e00640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61800cefa0 0x7f61801404b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f61680b0040 tx=0x7f61680a5890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:15.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f617cc00640 1 -- 172.21.15.50:0/3489489086 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61680b2040 con 0x7f61800cefa0 2024-09-21T06:08:15.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f617cc00640 1 -- 172.21.15.50:0/3489489086 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6168004430 con 0x7f61800cefa0 2024-09-21T06:08:15.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f617cc00640 1 -- 172.21.15.50:0/3489489086 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168005260 con 0x7f61800cefa0 2024-09-21T06:08:15.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6187200640 1 -- 172.21.15.50:0/3489489086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618015df80 con 0x7f61800cefa0 2024-09-21T06:08:15.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.303+0000 7f6187200640 1 -- 172.21.15.50:0/3489489086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618015e540 con 0x7f61800cefa0 2024-09-21T06:08:15.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.307+0000 7f6187200640 1 -- 172.21.15.50:0/3489489086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6180074db0 con 0x7f61800cefa0 2024-09-21T06:08:15.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.307+0000 7f617cc00640 1 -- 172.21.15.50:0/3489489086 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f6168005400 con 0x7f61800cefa0 2024-09-21T06:08:15.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.307+0000 7f617cc00640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f615c074880 0x7f615c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:15.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.307+0000 7f617cc00640 1 -- 172.21.15.50:0/3489489086 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7f61680bd070 con 0x7f61800cefa0 2024-09-21T06:08:15.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.307+0000 7f617fe00640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f615c074880 0x7f615c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:15.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.307+0000 7f617fe00640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f615c074880 0x7f615c076d90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6170009b40 tx=0x7f61700091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:15.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.311+0000 7f617cc00640 1 -- 172.21.15.50:0/3489489086 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6168137080 con 0x7f61800cefa0 2024-09-21T06:08:15.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.515+0000 7f6187200640 1 -- 172.21.15.50:0/3489489086 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone647110", "target": ["mon-mgr", ""]}) -- 0x7f618006b540 con 0x7f615c074880 2024-09-21T06:08:15.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f617cc00640 1 -- 172.21.15.50:0/3489489086 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6170011be0 con 0x7f615c074880 2024-09-21T06:08:15.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f6187200640 1 -- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f615c074880 msgr2=0x7f615c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:15.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f6187200640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f615c074880 0x7f615c076d90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6170009b40 tx=0x7f61700091e0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f6187200640 1 -- 172.21.15.50:0/3489489086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61800cefa0 msgr2=0x7f61801404b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:15.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f6187200640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61800cefa0 0x7f61801404b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f61680b0040 tx=0x7f61680a5890 comp rx=0 tx=0).stop 2024-09-21T06:08:15.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f6187200640 1 -- 172.21.15.50:0/3489489086 shutdown_connections 2024-09-21T06:08:15.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f6187200640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6180160330 0x7f6180159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f6187200640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f615c074880 0x7f615c076d90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f6187200640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61800cefa0 0x7f61801404b0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f6187200640 1 --2- 172.21.15.50:0/3489489086 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f61800749a0 0x7f61800af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f6187200640 1 -- 172.21.15.50:0/3489489086 >> 172.21.15.50:0/3489489086 conn(0x7f61800b2040 msgr2=0x7f61800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:15.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f6187200640 1 -- 172.21.15.50:0/3489489086 shutdown_connections 2024-09-21T06:08:15.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.535+0000 7f6187200640 1 -- 172.21.15.50:0/3489489086 wait complete. 2024-09-21T06:08:15.558 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone189911 2024-09-21T06:08:15.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.787+0000 7f9c8a000640 1 -- 172.21.15.50:0/2628505146 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c780ac070 con 0x7f9c8c069500 2024-09-21T06:08:15.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.787+0000 7f9c92600640 1 -- 172.21.15.50:0/2628505146 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c8c069500 msgr2=0x7f9c8c0ae510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:15.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.787+0000 7f9c92600640 1 --2- 172.21.15.50:0/2628505146 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c8c069500 0x7f9c8c0ae510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009870 tx=0x7f9c7801c710 comp rx=0 tx=0).stop 2024-09-21T06:08:15.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.787+0000 7f9c92600640 1 -- 172.21.15.50:0/2628505146 shutdown_connections 2024-09-21T06:08:15.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.787+0000 7f9c92600640 1 --2- 172.21.15.50:0/2628505146 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c8c069500 0x7f9c8c0ae510 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.787+0000 7f9c92600640 1 -- 172.21.15.50:0/2628505146 >> 172.21.15.50:0/2628505146 conn(0x7f9c8c0b35f0 msgr2=0x7f9c8c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:15.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.787+0000 7f9c92600640 1 -- 172.21.15.50:0/2628505146 shutdown_connections 2024-09-21T06:08:15.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.787+0000 7f9c92600640 1 -- 172.21.15.50:0/2628505146 wait complete. 2024-09-21T06:08:15.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.791+0000 7f9c92600640 1 Processor -- start 2024-09-21T06:08:15.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.791+0000 7f9c92600640 1 -- start start 2024-09-21T06:08:15.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.791+0000 7f9c92600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c8c0aea50 0x7f9c8c16cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:15.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.791+0000 7f9c92600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0b0810 0x7f9c8c149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:15.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.791+0000 7f9c92600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c8c151fe0 0x7f9c8c0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:15.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.791+0000 7f9c92600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c1556d0 con 0x7f9c8c0b0810 2024-09-21T06:08:15.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.791+0000 7f9c92600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c155550 con 0x7f9c8c0aea50 2024-09-21T06:08:15.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.791+0000 7f9c92600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8c155850 con 0x7f9c8c151fe0 2024-09-21T06:08:15.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.791+0000 7f9c8aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0b0810 0x7f9c8c149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:15.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.791+0000 7f9c8aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0b0810 0x7f9c8c149f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53138/0 (socket says 172.21.15.50:53138) 2024-09-21T06:08:15.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.791+0000 7f9c8aa00640 1 -- 172.21.15.50:0/3055814073 learned_addr learned my addr 172.21.15.50:0/3055814073 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:15.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.791+0000 7f9c8be00640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c8c151fe0 0x7f9c8c0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:15.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c8aa00640 1 -- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c8c151fe0 msgr2=0x7f9c8c0cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:15.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c8aa00640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c8c151fe0 0x7f9c8c0cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c8aa00640 1 -- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c8c0aea50 msgr2=0x7f9c8c16cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:15.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c8aa00640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c8c0aea50 0x7f9c8c16cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:15.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c8aa00640 1 -- 172.21.15.50:0/3055814073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8c152750 con 0x7f9c8c0b0810 2024-09-21T06:08:15.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c8be00640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c8c151fe0 0x7f9c8c0cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:15.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c8aa00640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0b0810 0x7f9c8c149f10 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9c7801cbf0 tx=0x7f9c780051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:15.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c83400640 1 -- 172.21.15.50:0/3055814073 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c780ac070 con 0x7f9c8c0b0810 2024-09-21T06:08:15.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c83400640 1 -- 172.21.15.50:0/3055814073 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c78005300 con 0x7f9c8c0b0810 2024-09-21T06:08:15.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c83400640 1 -- 172.21.15.50:0/3055814073 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c780046d0 con 0x7f9c8c0b0810 2024-09-21T06:08:15.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c92600640 1 -- 172.21.15.50:0/3055814073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8c152a30 con 0x7f9c8c0b0810 2024-09-21T06:08:15.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c92600640 1 -- 172.21.15.50:0/3055814073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8c0cf7e0 con 0x7f9c8c0b0810 2024-09-21T06:08:15.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.795+0000 7f9c92600640 1 -- 172.21.15.50:0/3055814073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8c155d60 con 0x7f9c8c0b0810 2024-09-21T06:08:15.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.799+0000 7f9c83400640 1 -- 172.21.15.50:0/3055814073 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f9c780af070 con 0x7f9c8c0b0810 2024-09-21T06:08:15.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.799+0000 7f9c83400640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c6c074950 0x7f9c6c076e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:15.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.799+0000 7f9c83400640 1 -- 172.21.15.50:0/3055814073 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7f9c7812d690 con 0x7f9c8c0b0810 2024-09-21T06:08:15.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.803+0000 7f9c8b400640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c6c074950 0x7f9c6c076e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:15.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.803+0000 7f9c8b400640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c6c074950 0x7f9c6c076e60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9c84000f70 tx=0x7f9c840079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:15.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:15.811+0000 7f9c83400640 1 -- 172.21.15.50:0/3055814073 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9c780c1060 con 0x7f9c8c0b0810 2024-09-21T06:08:16.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.091+0000 7f9c92600640 1 -- 172.21.15.50:0/3055814073 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone189911", "target": ["mon-mgr", ""]}) -- 0x7f9c8c06b540 con 0x7f9c6c074950 2024-09-21T06:08:16.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.107+0000 7f9c83400640 1 -- 172.21.15.50:0/3055814073 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9c840050d0 con 0x7f9c6c074950 2024-09-21T06:08:16.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.111+0000 7f9c92600640 1 -- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c6c074950 msgr2=0x7f9c6c076e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:16.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.111+0000 7f9c92600640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c6c074950 0x7f9c6c076e60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9c84000f70 tx=0x7f9c840079c0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.111+0000 7f9c92600640 1 -- 172.21.15.50:0/3055814073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0b0810 msgr2=0x7f9c8c149f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:16.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.111+0000 7f9c92600640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0b0810 0x7f9c8c149f10 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9c7801cbf0 tx=0x7f9c780051a0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.111+0000 7f9c92600640 1 -- 172.21.15.50:0/3055814073 shutdown_connections 2024-09-21T06:08:16.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.111+0000 7f9c92600640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c8c151fe0 0x7f9c8c0cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.111+0000 7f9c92600640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9c6c074950 0x7f9c6c076e60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.111+0000 7f9c92600640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0b0810 0x7f9c8c149f10 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.111+0000 7f9c92600640 1 --2- 172.21.15.50:0/3055814073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c8c0aea50 0x7f9c8c16cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.111+0000 7f9c92600640 1 -- 172.21.15.50:0/3055814073 >> 172.21.15.50:0/3055814073 conn(0x7f9c8c0b35f0 msgr2=0x7f9c8c0b4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:16.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.111+0000 7f9c92600640 1 -- 172.21.15.50:0/3055814073 shutdown_connections 2024-09-21T06:08:16.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.111+0000 7f9c92600640 1 -- 172.21.15.50:0/3055814073 wait complete. 2024-09-21T06:08:16.132 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone935917 2024-09-21T06:08:16.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.299+0000 7f983ce00640 1 -- 172.21.15.50:0/2226018008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98380b5a80 msgr2=0x7f9838074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:16.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.299+0000 7f983ce00640 1 --2- 172.21.15.50:0/2226018008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98380b5a80 0x7f9838074410 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9820009840 tx=0x7f982001c820 comp rx=0 tx=0).stop 2024-09-21T06:08:16.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.299+0000 7f983ce00640 1 -- 172.21.15.50:0/2226018008 shutdown_connections 2024-09-21T06:08:16.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.299+0000 7f983ce00640 1 --2- 172.21.15.50:0/2226018008 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9838074950 0x7f983815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.299+0000 7f983ce00640 1 --2- 172.21.15.50:0/2226018008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98380b5a80 0x7f9838074410 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.299+0000 7f983ce00640 1 -- 172.21.15.50:0/2226018008 >> 172.21.15.50:0/2226018008 conn(0x7f98380708c0 msgr2=0x7f9838070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:16.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.299+0000 7f983ce00640 1 -- 172.21.15.50:0/2226018008 shutdown_connections 2024-09-21T06:08:16.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.299+0000 7f983ce00640 1 -- 172.21.15.50:0/2226018008 wait complete. 2024-09-21T06:08:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.299+0000 7f983ce00640 1 Processor -- start 2024-09-21T06:08:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.299+0000 7f983ce00640 1 -- start start 2024-09-21T06:08:16.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f983ce00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9838074950 0x7f9838148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:16.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f983ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98380b5a80 0x7f983815c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:16.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f983ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9838157030 0x7f983814eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:16.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f983ce00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f983816a310 con 0x7f98380b5a80 2024-09-21T06:08:16.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f983ce00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f983816a190 con 0x7f9838157030 2024-09-21T06:08:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f983ce00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f983816a490 con 0x7f9838074950 2024-09-21T06:08:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f982fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98380b5a80 0x7f983815c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f9835e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9838074950 0x7f9838148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f982fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98380b5a80 0x7f983815c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53142/0 (socket says 172.21.15.50:53142) 2024-09-21T06:08:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f982fe00640 1 -- 172.21.15.50:0/3191945674 learned_addr learned my addr 172.21.15.50:0/3191945674 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f9836800640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9838157030 0x7f983814eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f9835e00640 1 -- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9838157030 msgr2=0x7f983814eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f9835e00640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9838157030 0x7f983814eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f9835e00640 1 -- 172.21.15.50:0/3191945674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98380b5a80 msgr2=0x7f983815c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f9835e00640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98380b5a80 0x7f983815c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f9835e00640 1 -- 172.21.15.50:0/3191945674 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9838149330 con 0x7f9838074950 2024-09-21T06:08:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f982fe00640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98380b5a80 0x7f983815c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:16.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f9835e00640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9838074950 0x7f9838148b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f982800c3f0 tx=0x7f982800c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:16.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f982e000640 1 -- 172.21.15.50:0/3191945674 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828017070 con 0x7f9838074950 2024-09-21T06:08:16.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f982e000640 1 -- 172.21.15.50:0/3191945674 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9828003600 con 0x7f9838074950 2024-09-21T06:08:16.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f982e000640 1 -- 172.21.15.50:0/3191945674 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828003a50 con 0x7f9838074950 2024-09-21T06:08:16.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f983ce00640 1 -- 172.21.15.50:0/3191945674 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983814b770 con 0x7f9838074950 2024-09-21T06:08:16.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f983ce00640 1 -- 172.21.15.50:0/3191945674 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f983814d1b0 con 0x7f9838074950 2024-09-21T06:08:16.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.303+0000 7f983ce00640 1 -- 172.21.15.50:0/3191945674 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f983815cf90 con 0x7f9838074950 2024-09-21T06:08:16.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.307+0000 7f982e000640 1 -- 172.21.15.50:0/3191945674 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f982801a070 con 0x7f9838074950 2024-09-21T06:08:16.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.307+0000 7f982e000640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9814074880 0x7f9814076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:16.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.307+0000 7f982e000640 1 -- 172.21.15.50:0/3191945674 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7f9828092540 con 0x7f9838074950 2024-09-21T06:08:16.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.307+0000 7f982fe00640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9814074880 0x7f9814076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:16.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.307+0000 7f982fe00640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9814074880 0x7f9814076d90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9820009840 tx=0x7f982001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.311+0000 7f982e000640 1 -- 172.21.15.50:0/3191945674 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9828021020 con 0x7f9838074950 2024-09-21T06:08:16.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.515+0000 7f983ce00640 1 -- 172.21.15.50:0/3191945674 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone935917", "target": ["mon-mgr", ""]}) -- 0x7f983806b540 con 0x7f9814074880 2024-09-21T06:08:16.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f982e000640 1 -- 172.21.15.50:0/3191945674 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9820005ca0 con 0x7f9814074880 2024-09-21T06:08:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f983ce00640 1 -- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9814074880 msgr2=0x7f9814076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f983ce00640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9814074880 0x7f9814076d90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9820009840 tx=0x7f982001b060 comp rx=0 tx=0).stop 2024-09-21T06:08:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f983ce00640 1 -- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9838074950 msgr2=0x7f9838148b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f983ce00640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9838074950 0x7f9838148b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f982800c3f0 tx=0x7f982800c7f0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f983ce00640 1 -- 172.21.15.50:0/3191945674 shutdown_connections 2024-09-21T06:08:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f983ce00640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9838157030 0x7f983814eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f983ce00640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9814074880 0x7f9814076d90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f983ce00640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98380b5a80 0x7f983815c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f983ce00640 1 --2- 172.21.15.50:0/3191945674 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9838074950 0x7f9838148b90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f983ce00640 1 -- 172.21.15.50:0/3191945674 >> 172.21.15.50:0/3191945674 conn(0x7f98380708c0 msgr2=0x7f9838156130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:16.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f983ce00640 1 -- 172.21.15.50:0/3191945674 shutdown_connections 2024-09-21T06:08:16.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.535+0000 7f983ce00640 1 -- 172.21.15.50:0/3191945674 wait complete. 2024-09-21T06:08:16.557 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone2443 2024-09-21T06:08:16.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.795+0000 7f47c5400640 1 -- 172.21.15.50:0/1134642806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c00d1580 msgr2=0x7f47c00b67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:16.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.795+0000 7f47c5400640 1 --2- 172.21.15.50:0/1134642806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c00d1580 0x7f47c00b67e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009bc0 tx=0x7f47b801c930 comp rx=0 tx=0).stop 2024-09-21T06:08:16.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.795+0000 7f47c5400640 1 -- 172.21.15.50:0/1134642806 shutdown_connections 2024-09-21T06:08:16.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.795+0000 7f47c5400640 1 --2- 172.21.15.50:0/1134642806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c00d1580 0x7f47c00b67e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.795+0000 7f47c5400640 1 --2- 172.21.15.50:0/1134642806 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f47c00d25d0 0x7f47c00d29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.795+0000 7f47c5400640 1 -- 172.21.15.50:0/1134642806 >> 172.21.15.50:0/1134642806 conn(0x7f47c0073e00 msgr2=0x7f47c006cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:16.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.795+0000 7f47c5400640 1 -- 172.21.15.50:0/1134642806 shutdown_connections 2024-09-21T06:08:16.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.799+0000 7f47c5400640 1 -- 172.21.15.50:0/1134642806 wait complete. 2024-09-21T06:08:16.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47c5400640 1 Processor -- start 2024-09-21T06:08:16.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47c5400640 1 -- start start 2024-09-21T06:08:16.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47c5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c00d1170 0x7f47c006fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:16.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47c5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c00d1580 0x7f47c0071e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:16.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47c5400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f47c00d25d0 0x7f47c02005b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:16.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47c5400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f47c015d4b0 con 0x7f47c00d1170 2024-09-21T06:08:16.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47c5400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f47c00d4340 con 0x7f47c00d1580 2024-09-21T06:08:16.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47c5400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f47c015d630 con 0x7f47c00d25d0 2024-09-21T06:08:16.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47be800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c00d1170 0x7f47c006fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:16.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47bf200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f47c00d25d0 0x7f47c02005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:16.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47be800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c00d1170 0x7f47c006fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57542/0 (socket says 172.21.15.50:57542) 2024-09-21T06:08:16.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47be800640 1 -- 172.21.15.50:0/3682224338 learned_addr learned my addr 172.21.15.50:0/3682224338 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:16.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47bde00640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c00d1580 0x7f47c0071e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:16.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47be800640 1 -- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f47c00d25d0 msgr2=0x7f47c02005b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:16.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47be800640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f47c00d25d0 0x7f47c02005b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47be800640 1 -- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c00d1580 msgr2=0x7f47c0071e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:16.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47be800640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c00d1580 0x7f47c0071e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:16.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.803+0000 7f47be800640 1 -- 172.21.15.50:0/3682224338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c0201120 con 0x7f47c00d1170 2024-09-21T06:08:16.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.807+0000 7f47be800640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c00d1170 0x7f47c006fae0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f47b000e7a0 tx=0x7f47b000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:16.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.807+0000 7f47b6a00640 1 -- 172.21.15.50:0/3682224338 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b000ccc0 con 0x7f47c00d1170 2024-09-21T06:08:16.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.807+0000 7f47c5400640 1 -- 172.21.15.50:0/3682224338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c0201400 con 0x7f47c00d1170 2024-09-21T06:08:16.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.807+0000 7f47c5400640 1 -- 172.21.15.50:0/3682224338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c006f2e0 con 0x7f47c00d1170 2024-09-21T06:08:16.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.807+0000 7f47b6a00640 1 -- 172.21.15.50:0/3682224338 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47b0010430 con 0x7f47c00d1170 2024-09-21T06:08:16.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.807+0000 7f47b6a00640 1 -- 172.21.15.50:0/3682224338 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0010880 con 0x7f47c00d1170 2024-09-21T06:08:16.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.807+0000 7f47c5400640 1 -- 172.21.15.50:0/3682224338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47980052a0 con 0x7f47c00d1170 2024-09-21T06:08:16.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.811+0000 7f47b6a00640 1 -- 172.21.15.50:0/3682224338 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f47b0010a20 con 0x7f47c00d1170 2024-09-21T06:08:16.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.815+0000 7f47b6a00640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f47a0074880 0x7f47a0076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:16.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.815+0000 7f47b6a00640 1 -- 172.21.15.50:0/3682224338 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7f47b0093c50 con 0x7f47c00d1170 2024-09-21T06:08:16.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.815+0000 7f47bde00640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f47a0074880 0x7f47a0076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:16.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.815+0000 7f47bde00640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f47a0074880 0x7f47a0076d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f47c0072850 tx=0x7f47ac009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:16.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:16.815+0000 7f47b6a00640 1 -- 172.21.15.50:0/3682224338 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f47b0026020 con 0x7f47c00d1170 2024-09-21T06:08:17.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.143+0000 7f47c5400640 1 -- 172.21.15.50:0/3682224338 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone2443", "target": ["mon-mgr", ""]}) -- 0x7f4798000bd0 con 0x7f47a0074880 2024-09-21T06:08:17.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47b6a00640 1 -- 172.21.15.50:0/3682224338 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4798000bd0 con 0x7f47a0074880 2024-09-21T06:08:17.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47c5400640 1 -- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f47a0074880 msgr2=0x7f47a0076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47c5400640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f47a0074880 0x7f47a0076d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f47c0072850 tx=0x7f47ac009060 comp rx=0 tx=0).stop 2024-09-21T06:08:17.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47c5400640 1 -- 172.21.15.50:0/3682224338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c00d1170 msgr2=0x7f47c006fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47c5400640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c00d1170 0x7f47c006fae0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f47b000e7a0 tx=0x7f47b000ec60 comp rx=0 tx=0).stop 2024-09-21T06:08:17.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47c5400640 1 -- 172.21.15.50:0/3682224338 shutdown_connections 2024-09-21T06:08:17.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47c5400640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f47c00d25d0 0x7f47c02005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47c5400640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c00d1580 0x7f47c0071e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47c5400640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f47a0074880 0x7f47a0076d90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47c5400640 1 --2- 172.21.15.50:0/3682224338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c00d1170 0x7f47c006fae0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47c5400640 1 -- 172.21.15.50:0/3682224338 >> 172.21.15.50:0/3682224338 conn(0x7f47c0073e00 msgr2=0x7f47c00b5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:17.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47c5400640 1 -- 172.21.15.50:0/3682224338 shutdown_connections 2024-09-21T06:08:17.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.163+0000 7f47c5400640 1 -- 172.21.15.50:0/3682224338 wait complete. 2024-09-21T06:08:17.184 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8636 2024-09-21T06:08:17.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.351+0000 7efc63600640 1 -- 172.21.15.50:0/707602850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc44005160 msgr2=0x7efc4400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.351+0000 7efc63600640 1 --2- 172.21.15.50:0/707602850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc44005160 0x7efc4400b9d0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0731f0 tx=0x7efc5401d2e0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.351+0000 7efc61600640 1 -- 172.21.15.50:0/707602850 reap_dead start 2024-09-21T06:08:17.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.351+0000 7efc63600640 1 -- 172.21.15.50:0/707602850 shutdown_connections 2024-09-21T06:08:17.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.351+0000 7efc63600640 1 -- 172.21.15.50:0/707602850 >> 172.21.15.50:0/707602850 conn(0x7efc5c0b2040 msgr2=0x7efc5c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:17.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.351+0000 7efc63600640 1 -- 172.21.15.50:0/707602850 shutdown_connections 2024-09-21T06:08:17.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.351+0000 7efc63600640 1 -- 172.21.15.50:0/707602850 wait complete. 2024-09-21T06:08:17.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc63600640 1 Processor -- start 2024-09-21T06:08:17.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc63600640 1 -- start start 2024-09-21T06:08:17.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc63600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efc44005160 0x7efc5c152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc63600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc5c1f8e10 0x7efc5c1f9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc63600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c1579c0 0x7efc5c157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc63600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c15fd80 con 0x7efc5c1579c0 2024-09-21T06:08:17.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc63600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c15fc00 con 0x7efc5c1f8e10 2024-09-21T06:08:17.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc63600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7efc5c15ff00 con 0x7efc44005160 2024-09-21T06:08:17.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc61600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c1579c0 0x7efc5c157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc61600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c1579c0 0x7efc5c157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57548/0 (socket says 172.21.15.50:57548) 2024-09-21T06:08:17.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc61600640 1 -- 172.21.15.50:0/271214160 learned_addr learned my addr 172.21.15.50:0/271214160 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc5be00640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc5c1f8e10 0x7efc5c1f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc60c00640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efc44005160 0x7efc5c152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc61600640 1 -- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efc44005160 msgr2=0x7efc5c152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc61600640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efc44005160 0x7efc5c152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc61600640 1 -- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc5c1f8e10 msgr2=0x7efc5c1f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc61600640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc5c1f8e10 0x7efc5c1f9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc61600640 1 -- 172.21.15.50:0/271214160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c1f9760 con 0x7efc5c1579c0 2024-09-21T06:08:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc5be00640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc5c1f8e10 0x7efc5c1f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc60c00640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efc44005160 0x7efc5c152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc61600640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c1579c0 0x7efc5c157dd0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7efc5401cda0 tx=0x7efc54006c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc58c00640 1 -- 172.21.15.50:0/271214160 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc540011d0 con 0x7efc5c1579c0 2024-09-21T06:08:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc58c00640 1 -- 172.21.15.50:0/271214160 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc54003c70 con 0x7efc5c1579c0 2024-09-21T06:08:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc58c00640 1 -- 172.21.15.50:0/271214160 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54006310 con 0x7efc5c1579c0 2024-09-21T06:08:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc63600640 1 -- 172.21.15.50:0/271214160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5c0c5fd0 con 0x7efc5c1579c0 2024-09-21T06:08:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.355+0000 7efc63600640 1 -- 172.21.15.50:0/271214160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5c0c6590 con 0x7efc5c1579c0 2024-09-21T06:08:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.359+0000 7efc63600640 1 -- 172.21.15.50:0/271214160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5c155e40 con 0x7efc5c1579c0 2024-09-21T06:08:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.359+0000 7efc58c00640 1 -- 172.21.15.50:0/271214160 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7efc540064b0 con 0x7efc5c1579c0 2024-09-21T06:08:17.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.359+0000 7efc58c00640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efc3c0748b0 0x7efc3c076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.359+0000 7efc58c00640 1 -- 172.21.15.50:0/271214160 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7efc5412d360 con 0x7efc5c1579c0 2024-09-21T06:08:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.359+0000 7efc60c00640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efc3c0748b0 0x7efc3c076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.359+0000 7efc60c00640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efc3c0748b0 0x7efc3c076dc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0b2fd0 tx=0x7efc500091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:17.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.363+0000 7efc58c00640 1 -- 172.21.15.50:0/271214160 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efc5c1f9760 con 0x7efc5c1579c0 2024-09-21T06:08:17.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.567+0000 7efc63600640 1 -- 172.21.15.50:0/271214160 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8636", "target": ["mon-mgr", ""]}) -- 0x7efc5c0b3740 con 0x7efc3c0748b0 2024-09-21T06:08:17.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.587+0000 7efc58c00640 1 -- 172.21.15.50:0/271214160 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc5c0b3740 con 0x7efc3c0748b0 2024-09-21T06:08:17.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.591+0000 7efc63600640 1 -- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efc3c0748b0 msgr2=0x7efc3c076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.591+0000 7efc63600640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efc3c0748b0 0x7efc3c076dc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0b2fd0 tx=0x7efc500091e0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.591+0000 7efc63600640 1 -- 172.21.15.50:0/271214160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c1579c0 msgr2=0x7efc5c157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.591+0000 7efc63600640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c1579c0 0x7efc5c157dd0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7efc5401cda0 tx=0x7efc54006c40 comp rx=0 tx=0).stop 2024-09-21T06:08:17.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.591+0000 7efc63600640 1 -- 172.21.15.50:0/271214160 shutdown_connections 2024-09-21T06:08:17.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.591+0000 7efc63600640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efc3c0748b0 0x7efc3c076dc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.591+0000 7efc63600640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc5c1579c0 0x7efc5c157dd0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.591+0000 7efc63600640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc5c1f8e10 0x7efc5c1f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.591+0000 7efc63600640 1 --2- 172.21.15.50:0/271214160 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efc44005160 0x7efc5c152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.591+0000 7efc63600640 1 -- 172.21.15.50:0/271214160 >> 172.21.15.50:0/271214160 conn(0x7efc5c0b2040 msgr2=0x7efc5c154e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:17.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.591+0000 7efc63600640 1 -- 172.21.15.50:0/271214160 shutdown_connections 2024-09-21T06:08:17.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.591+0000 7efc63600640 1 -- 172.21.15.50:0/271214160 wait complete. 2024-09-21T06:08:17.614 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone71879 2024-09-21T06:08:17.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.891+0000 7fa1d7400640 1 -- 172.21.15.50:0/3726864893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1d006cdf0 msgr2=0x7fa1d006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.891+0000 7fa1d7400640 1 --2- 172.21.15.50:0/3726864893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1d006cdf0 0x7fa1d006d200 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009840 tx=0x7fa1bc01c790 comp rx=0 tx=0).stop 2024-09-21T06:08:17.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.891+0000 7fa1d7400640 1 -- 172.21.15.50:0/3726864893 shutdown_connections 2024-09-21T06:08:17.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.891+0000 7fa1d7400640 1 --2- 172.21.15.50:0/3726864893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1d006cdf0 0x7fa1d006d200 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.891+0000 7fa1d7400640 1 -- 172.21.15.50:0/3726864893 >> 172.21.15.50:0/3726864893 conn(0x7fa1d00b35f0 msgr2=0x7fa1d00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:17.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.891+0000 7fa1d7400640 1 -- 172.21.15.50:0/3726864893 shutdown_connections 2024-09-21T06:08:17.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.891+0000 7fa1d7400640 1 -- 172.21.15.50:0/3726864893 wait complete. 2024-09-21T06:08:17.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.895+0000 7fa1d7400640 1 Processor -- start 2024-09-21T06:08:17.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.895+0000 7fa1d7400640 1 -- start start 2024-09-21T06:08:17.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d006cdf0 0x7fa1d01513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d7400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa1d006fc90 0x7fa1d01508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1d0159130 0x7fa1d015e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d7400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d00693c0 con 0x7fa1d006cdf0 2024-09-21T06:08:17.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d7400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0069240 con 0x7fa1d0159130 2024-09-21T06:08:17.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d7400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d0069540 con 0x7fa1d006fc90 2024-09-21T06:08:17.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d006cdf0 0x7fa1d01513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d006cdf0 0x7fa1d01513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57564/0 (socket says 172.21.15.50:57564) 2024-09-21T06:08:17.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d4a00640 1 -- 172.21.15.50:0/2125733381 learned_addr learned my addr 172.21.15.50:0/2125733381 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:17.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1cfe00640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa1d006fc90 0x7fa1d01508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d5400640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1d0159130 0x7fa1d015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d4a00640 1 -- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa1d006fc90 msgr2=0x7fa1d01508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d4a00640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa1d006fc90 0x7fa1d01508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d4a00640 1 -- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1d0159130 msgr2=0x7fa1d015e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d4a00640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1d0159130 0x7fa1d015e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.899+0000 7fa1d4a00640 1 -- 172.21.15.50:0/2125733381 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d014fb10 con 0x7fa1d006cdf0 2024-09-21T06:08:17.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.903+0000 7fa1d4a00640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d006cdf0 0x7fa1d01513f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000c460 tx=0x7fa1c000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:17.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.903+0000 7fa1ccc00640 1 -- 172.21.15.50:0/2125733381 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0016020 con 0x7fa1d006cdf0 2024-09-21T06:08:17.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.903+0000 7fa1ccc00640 1 -- 172.21.15.50:0/2125733381 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1c0012a00 con 0x7fa1d006cdf0 2024-09-21T06:08:17.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.903+0000 7fa1ccc00640 1 -- 172.21.15.50:0/2125733381 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0012e50 con 0x7fa1d006cdf0 2024-09-21T06:08:17.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.903+0000 7fa1d7400640 1 -- 172.21.15.50:0/2125733381 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d014fdf0 con 0x7fa1d006cdf0 2024-09-21T06:08:17.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.903+0000 7fa1d7400640 1 -- 172.21.15.50:0/2125733381 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d015e5b0 con 0x7fa1d006cdf0 2024-09-21T06:08:17.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.903+0000 7fa1d7400640 1 -- 172.21.15.50:0/2125733381 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d0069b10 con 0x7fa1d006cdf0 2024-09-21T06:08:17.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.907+0000 7fa1ccc00640 1 -- 172.21.15.50:0/2125733381 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fa1c0019020 con 0x7fa1d006cdf0 2024-09-21T06:08:17.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.911+0000 7fa1ccc00640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa1b0074a20 0x7fa1b0076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.911+0000 7fa1ccc00640 1 -- 172.21.15.50:0/2125733381 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7fa1c0092e40 con 0x7fa1d006cdf0 2024-09-21T06:08:17.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.911+0000 7fa1cfe00640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa1b0074a20 0x7fa1b0076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.911+0000 7fa1cfe00640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa1b0074a20 0x7fa1b0076f30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0af000 tx=0x7fa1bc0b2040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:17.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:17.919+0000 7fa1ccc00640 1 -- 172.21.15.50:0/2125733381 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa1c0020020 con 0x7fa1d006cdf0 2024-09-21T06:08:18.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.327+0000 7fa1d7400640 1 -- 172.21.15.50:0/2125733381 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone71879", "target": ["mon-mgr", ""]}) -- 0x7fa1d006b540 con 0x7fa1b0074a20 2024-09-21T06:08:18.336 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:08:18.340 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:08:18.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.371+0000 7fa1ccc00640 1 -- 172.21.15.50:0/2125733381 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa1bc0a6560 con 0x7fa1b0074a20 2024-09-21T06:08:18.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.375+0000 7fa1d7400640 1 -- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa1b0074a20 msgr2=0x7fa1b0076f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:18.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.375+0000 7fa1d7400640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa1b0074a20 0x7fa1b0076f30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0af000 tx=0x7fa1bc0b2040 comp rx=0 tx=0).stop 2024-09-21T06:08:18.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.375+0000 7fa1d7400640 1 -- 172.21.15.50:0/2125733381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d006cdf0 msgr2=0x7fa1d01513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:18.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.375+0000 7fa1d7400640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d006cdf0 0x7fa1d01513f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000c460 tx=0x7fa1c000c920 comp rx=0 tx=0).stop 2024-09-21T06:08:18.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.375+0000 7fa1d7400640 1 -- 172.21.15.50:0/2125733381 shutdown_connections 2024-09-21T06:08:18.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.375+0000 7fa1d7400640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1d0159130 0x7fa1d015e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.375+0000 7fa1d7400640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa1d006fc90 0x7fa1d01508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.375+0000 7fa1d7400640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa1b0074a20 0x7fa1b0076f30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.375+0000 7fa1d7400640 1 --2- 172.21.15.50:0/2125733381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d006cdf0 0x7fa1d01513f0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.375+0000 7fa1d7400640 1 -- 172.21.15.50:0/2125733381 >> 172.21.15.50:0/2125733381 conn(0x7fa1d00b35f0 msgr2=0x7fa1d00b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:18.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.375+0000 7fa1d7400640 1 -- 172.21.15.50:0/2125733381 shutdown_connections 2024-09-21T06:08:18.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.375+0000 7fa1d7400640 1 -- 172.21.15.50:0/2125733381 wait complete. 2024-09-21T06:08:18.414 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone72702 2024-09-21T06:08:18.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.627+0000 7f7965e00640 1 -- 172.21.15.50:0/2211354769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79601527b0 msgr2=0x7f796015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:18.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.627+0000 7f7965e00640 1 --2- 172.21.15.50:0/2211354769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79601527b0 0x7f796015bb60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7948009870 tx=0x7f794801c7a0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.627+0000 7f7965e00640 1 -- 172.21.15.50:0/2211354769 shutdown_connections 2024-09-21T06:08:18.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.627+0000 7f7965e00640 1 --2- 172.21.15.50:0/2211354769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79601527b0 0x7f796015bb60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.627+0000 7f7965e00640 1 -- 172.21.15.50:0/2211354769 >> 172.21.15.50:0/2211354769 conn(0x7f79600b2040 msgr2=0x7f79600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:18.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.627+0000 7f7965e00640 1 -- 172.21.15.50:0/2211354769 shutdown_connections 2024-09-21T06:08:18.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.627+0000 7f7965e00640 1 -- 172.21.15.50:0/2211354769 wait complete. 2024-09-21T06:08:18.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.627+0000 7f7965e00640 1 Processor -- start 2024-09-21T06:08:18.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.627+0000 7f7965e00640 1 -- start start 2024-09-21T06:08:18.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7965e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79600b4de0 0x7f79601f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:18.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7965e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79600b5c70 0x7f796013d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:18.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7965e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79601527b0 0x7f79601fc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:18.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7965e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7960169c60 con 0x7f79600b4de0 2024-09-21T06:08:18.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7965e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7960169ae0 con 0x7f79600b5c70 2024-09-21T06:08:18.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7965e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7960169de0 con 0x7f79601527b0 2024-09-21T06:08:18.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f795ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79600b4de0 0x7f79601f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:18.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7956000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79600b5c70 0x7f796013d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:18.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f795ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79600b4de0 0x7f79601f8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57574/0 (socket says 172.21.15.50:57574) 2024-09-21T06:08:18.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f795ea00640 1 -- 172.21.15.50:0/2971448309 learned_addr learned my addr 172.21.15.50:0/2971448309 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:18.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f795f400640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79601527b0 0x7f79601fc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:18.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f795ea00640 1 -- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79601527b0 msgr2=0x7f79601fc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:18.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f795ea00640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79601527b0 0x7f79601fc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f795ea00640 1 -- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79600b5c70 msgr2=0x7f796013d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:18.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f795ea00640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79600b5c70 0x7f796013d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f795ea00640 1 -- 172.21.15.50:0/2971448309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79601f94b0 con 0x7f79600b4de0 2024-09-21T06:08:18.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7956000640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79600b5c70 0x7f796013d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:18.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f795ea00640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79600b4de0 0x7f79601f8a80 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f795000c430 tx=0x7f795000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:18.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7957e00640 1 -- 172.21.15.50:0/2971448309 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950011790 con 0x7f79600b4de0 2024-09-21T06:08:18.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7957e00640 1 -- 172.21.15.50:0/2971448309 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7950017070 con 0x7f79600b4de0 2024-09-21T06:08:18.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7965e00640 1 -- 172.21.15.50:0/2971448309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79601f9790 con 0x7f79600b4de0 2024-09-21T06:08:18.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7957e00640 1 -- 172.21.15.50:0/2971448309 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950011cc0 con 0x7f79600b4de0 2024-09-21T06:08:18.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.631+0000 7f7965e00640 1 -- 172.21.15.50:0/2971448309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79601f8fc0 con 0x7f79600b4de0 2024-09-21T06:08:18.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.635+0000 7f7965e00640 1 -- 172.21.15.50:0/2971448309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796015bb60 con 0x7f79600b4de0 2024-09-21T06:08:18.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.635+0000 7f7957e00640 1 -- 172.21.15.50:0/2971448309 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f795001c460 con 0x7f79600b4de0 2024-09-21T06:08:18.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.639+0000 7f7957e00640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f793c074880 0x7f793c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:18.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.639+0000 7f7957e00640 1 -- 172.21.15.50:0/2971448309 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7f7950092c30 con 0x7f79600b4de0 2024-09-21T06:08:18.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.639+0000 7f7956000640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f793c074880 0x7f793c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:18.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.647+0000 7f7956000640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f793c074880 0x7f793c076d90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f79480af000 tx=0x7f794801afe0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:18.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:18.647+0000 7f7957e00640 1 -- 172.21.15.50:0/2971448309 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f795005f130 con 0x7f79600b4de0 2024-09-21T06:08:19.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.055+0000 7f7965e00640 1 -- 172.21.15.50:0/2971448309 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone72702", "target": ["mon-mgr", ""]}) -- 0x7f79601f9260 con 0x7f793c074880 2024-09-21T06:08:19.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.099+0000 7f7957e00640 1 -- 172.21.15.50:0/2971448309 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f79601f9260 con 0x7f793c074880 2024-09-21T06:08:19.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.103+0000 7f7965e00640 1 -- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f793c074880 msgr2=0x7f793c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:19.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.103+0000 7f7965e00640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f793c074880 0x7f793c076d90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f79480af000 tx=0x7f794801afe0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.103+0000 7f7965e00640 1 -- 172.21.15.50:0/2971448309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79600b4de0 msgr2=0x7f79601f8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:19.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.103+0000 7f7965e00640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79600b4de0 0x7f79601f8a80 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f795000c430 tx=0x7f795000c8f0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.103+0000 7f7965e00640 1 -- 172.21.15.50:0/2971448309 shutdown_connections 2024-09-21T06:08:19.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.103+0000 7f7965e00640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f79601527b0 0x7f79601fc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.103+0000 7f7965e00640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79600b5c70 0x7f796013d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.103+0000 7f7965e00640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f793c074880 0x7f793c076d90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.103+0000 7f7965e00640 1 --2- 172.21.15.50:0/2971448309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79600b4de0 0x7f79601f8a80 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.103+0000 7f7965e00640 1 -- 172.21.15.50:0/2971448309 >> 172.21.15.50:0/2971448309 conn(0x7f79600b2040 msgr2=0x7f7960154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:19.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.103+0000 7f7965e00640 1 -- 172.21.15.50:0/2971448309 shutdown_connections 2024-09-21T06:08:19.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.103+0000 7f7965e00640 1 -- 172.21.15.50:0/2971448309 wait complete. 2024-09-21T06:08:19.141 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone837612 2024-09-21T06:08:19.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.331+0000 7fea62c00640 1 -- 172.21.15.50:0/3213690332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c1527b0 msgr2=0x7fea5c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:19.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.331+0000 7fea62c00640 1 --2- 172.21.15.50:0/3213690332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c1527b0 0x7fea5c15bb60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fea44009840 tx=0x7fea4401c710 comp rx=0 tx=0).stop 2024-09-21T06:08:19.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.331+0000 7fea62c00640 1 -- 172.21.15.50:0/3213690332 shutdown_connections 2024-09-21T06:08:19.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.331+0000 7fea62c00640 1 --2- 172.21.15.50:0/3213690332 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea5c0b4de0 0x7fea5c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.331+0000 7fea62c00640 1 --2- 172.21.15.50:0/3213690332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c1527b0 0x7fea5c15bb60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.331+0000 7fea62c00640 1 -- 172.21.15.50:0/3213690332 >> 172.21.15.50:0/3213690332 conn(0x7fea5c0b2040 msgr2=0x7fea5c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:19.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.331+0000 7fea62c00640 1 -- 172.21.15.50:0/3213690332 shutdown_connections 2024-09-21T06:08:19.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.331+0000 7fea62c00640 1 -- 172.21.15.50:0/3213690332 wait complete. 2024-09-21T06:08:19.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.335+0000 7fea62c00640 1 Processor -- start 2024-09-21T06:08:19.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.335+0000 7fea62c00640 1 -- start start 2024-09-21T06:08:19.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.335+0000 7fea62c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c0b4de0 0x7fea5c0ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:19.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.335+0000 7fea62c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea5c0b5c70 0x7fea5c15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:19.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.335+0000 7fea62c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea5c1527b0 0x7fea5c1f9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:19.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.335+0000 7fea62c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1689b0 con 0x7fea5c1527b0 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.335+0000 7fea62c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c168830 con 0x7fea5c0b4de0 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.335+0000 7fea62c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c168b30 con 0x7fea5c0b5c70 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea53400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea5c0b5c70 0x7fea5c15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea53400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea5c0b5c70 0x7fea5c15a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:32936/0 (socket says 172.21.15.50:32936) 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea53400640 1 -- 172.21.15.50:0/932576466 learned_addr learned my addr 172.21.15.50:0/932576466 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea5be00640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c0b4de0 0x7fea5c0ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea60c00640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea5c1527b0 0x7fea5c1f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea53400640 1 -- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c0b4de0 msgr2=0x7fea5c0ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea53400640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c0b4de0 0x7fea5c0ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea53400640 1 -- 172.21.15.50:0/932576466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea5c1527b0 msgr2=0x7fea5c1f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea53400640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea5c1527b0 0x7fea5c1f9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea53400640 1 -- 172.21.15.50:0/932576466 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c0aeac0 con 0x7fea5c0b5c70 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea53400640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea5c0b5c70 0x7fea5c15a940 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea44009790 tx=0x7fea44004620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea58c00640 1 -- 172.21.15.50:0/932576466 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea440bb070 con 0x7fea5c0b5c70 2024-09-21T06:08:19.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea62c00640 1 -- 172.21.15.50:0/932576466 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c1f6b50 con 0x7fea5c0b5c70 2024-09-21T06:08:19.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea58c00640 1 -- 172.21.15.50:0/932576466 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea440a5b20 con 0x7fea5c0b5c70 2024-09-21T06:08:19.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea58c00640 1 -- 172.21.15.50:0/932576466 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea440b2040 con 0x7fea5c0b5c70 2024-09-21T06:08:19.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea62c00640 1 -- 172.21.15.50:0/932576466 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c1f7110 con 0x7fea5c0b5c70 2024-09-21T06:08:19.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea62c00640 1 -- 172.21.15.50:0/932576466 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5c0b51f0 con 0x7fea5c0b5c70 2024-09-21T06:08:19.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.339+0000 7fea58c00640 1 -- 172.21.15.50:0/932576466 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fea44005030 con 0x7fea5c0b5c70 2024-09-21T06:08:19.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.343+0000 7fea58c00640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea38074880 0x7fea38076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:19.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.343+0000 7fea58c00640 1 -- 172.21.15.50:0/932576466 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7fea4412c6d0 con 0x7fea5c0b5c70 2024-09-21T06:08:19.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.343+0000 7fea5be00640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea38074880 0x7fea38076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:19.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.343+0000 7fea5be00640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea38074880 0x7fea38076d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea4c007990 tx=0x7fea4c0012e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:19.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.343+0000 7fea58c00640 1 -- 172.21.15.50:0/932576466 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fea44016330 con 0x7fea5c0b5c70 2024-09-21T06:08:19.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.547+0000 7fea62c00640 1 -- 172.21.15.50:0/932576466 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone837612", "target": ["mon-mgr", ""]}) -- 0x7fea5c06b540 con 0x7fea38074880 2024-09-21T06:08:19.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.567+0000 7fea58c00640 1 -- 172.21.15.50:0/932576466 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fea4c00f850 con 0x7fea38074880 2024-09-21T06:08:19.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.571+0000 7fea62c00640 1 -- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea38074880 msgr2=0x7fea38076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:19.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.571+0000 7fea62c00640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea38074880 0x7fea38076d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea4c007990 tx=0x7fea4c0012e0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.571+0000 7fea62c00640 1 -- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea5c0b5c70 msgr2=0x7fea5c15a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:19.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.571+0000 7fea62c00640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea5c0b5c70 0x7fea5c15a940 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea44009790 tx=0x7fea44004620 comp rx=0 tx=0).stop 2024-09-21T06:08:19.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.571+0000 7fea62c00640 1 -- 172.21.15.50:0/932576466 shutdown_connections 2024-09-21T06:08:19.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.571+0000 7fea62c00640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fea38074880 0x7fea38076d90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.571+0000 7fea62c00640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea5c1527b0 0x7fea5c1f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.571+0000 7fea62c00640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fea5c0b5c70 0x7fea5c15a940 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.571+0000 7fea62c00640 1 --2- 172.21.15.50:0/932576466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c0b4de0 0x7fea5c0ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.571+0000 7fea62c00640 1 -- 172.21.15.50:0/932576466 >> 172.21.15.50:0/932576466 conn(0x7fea5c0b2040 msgr2=0x7fea5c0b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:19.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.571+0000 7fea62c00640 1 -- 172.21.15.50:0/932576466 shutdown_connections 2024-09-21T06:08:19.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.571+0000 7fea62c00640 1 -- 172.21.15.50:0/932576466 wait complete. 2024-09-21T06:08:19.592 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone986813 2024-09-21T06:08:19.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.835+0000 7f5e56c00640 1 -- 172.21.15.50:0/2660969369 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e50159130 msgr2=0x7f5e5015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:19.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.835+0000 7f5e56c00640 1 --2- 172.21.15.50:0/2660969369 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e50159130 0x7f5e5015bb90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009840 tx=0x7f5e3c01c620 comp rx=0 tx=0).stop 2024-09-21T06:08:19.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.835+0000 7f5e56c00640 1 -- 172.21.15.50:0/2660969369 shutdown_connections 2024-09-21T06:08:19.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.835+0000 7f5e56c00640 1 --2- 172.21.15.50:0/2660969369 >> v2:172.21.15.50:3300/0 conn(0x7f5e500703a0 0x7f5e5006d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.835+0000 7f5e56c00640 1 --2- 172.21.15.50:0/2660969369 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e50159130 0x7f5e5015bb90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.835+0000 7f5e56c00640 1 -- 172.21.15.50:0/2660969369 >> 172.21.15.50:0/2660969369 conn(0x7f5e500b35f0 msgr2=0x7f5e500b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:19.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.835+0000 7f5e56c00640 1 -- 172.21.15.50:0/2660969369 shutdown_connections 2024-09-21T06:08:19.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.835+0000 7f5e56c00640 1 -- 172.21.15.50:0/2660969369 wait complete. 2024-09-21T06:08:19.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.839+0000 7f5e56c00640 1 Processor -- start 2024-09-21T06:08:19.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.839+0000 7f5e56c00640 1 -- start start 2024-09-21T06:08:19.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.843+0000 7f5e56c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e5006cdf0 0x7f5e50161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:19.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.843+0000 7f5e56c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e500703a0 0x7f5e5015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:19.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.843+0000 7f5e56c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e50159130 0x7f5e50156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:19.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.843+0000 7f5e56c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5e50168e70 con 0x7f5e5006cdf0 2024-09-21T06:08:19.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.843+0000 7f5e56c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e50168cf0 con 0x7f5e50159130 2024-09-21T06:08:19.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.843+0000 7f5e56c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5e50168ff0 con 0x7f5e500703a0 2024-09-21T06:08:19.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.843+0000 7f5e4f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e500703a0 0x7f5e5015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:19.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.843+0000 7f5e4f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e500703a0 0x7f5e5015aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:32942/0 (socket says 172.21.15.50:32942) 2024-09-21T06:08:19.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.843+0000 7f5e4f400640 1 -- 172.21.15.50:0/2844715983 learned_addr learned my addr 172.21.15.50:0/2844715983 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:19.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.843+0000 7f5e4f400640 1 -- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e50159130 msgr2=0x7f5e50156ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:08:19.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.843+0000 7f5e54c00640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e50159130 0x7f5e50156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:19.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.843+0000 7f5e4f400640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e50159130 0x7f5e50156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.847+0000 7f5e4f400640 1 -- 172.21.15.50:0/2844715983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e5006cdf0 msgr2=0x7f5e50161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:19.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.847+0000 7f5e4fe00640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e5006cdf0 0x7f5e50161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:19.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.847+0000 7f5e4f400640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e5006cdf0 0x7f5e50161ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.847+0000 7f5e4f400640 1 -- 172.21.15.50:0/2844715983 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e50160270 con 0x7f5e500703a0 2024-09-21T06:08:19.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.847+0000 7f5e4fe00640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e5006cdf0 0x7f5e50161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:19.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.847+0000 7f5e4f400640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e500703a0 0x7f5e5015aee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5e500b5390 tx=0x7f5e4000cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:19.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.847+0000 7f5e47e00640 1 -- 172.21.15.50:0/2844715983 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40017070 con 0x7f5e500703a0 2024-09-21T06:08:19.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.847+0000 7f5e47e00640 1 -- 172.21.15.50:0/2844715983 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e40005360 con 0x7f5e500703a0 2024-09-21T06:08:19.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.847+0000 7f5e47e00640 1 -- 172.21.15.50:0/2844715983 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000a8a0 con 0x7f5e500703a0 2024-09-21T06:08:19.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.847+0000 7f5e56c00640 1 -- 172.21.15.50:0/2844715983 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e50164ac0 con 0x7f5e500703a0 2024-09-21T06:08:19.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.847+0000 7f5e56c00640 1 -- 172.21.15.50:0/2844715983 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e50164190 con 0x7f5e500703a0 2024-09-21T06:08:19.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.847+0000 7f5e56c00640 1 -- 172.21.15.50:0/2844715983 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e5006d200 con 0x7f5e500703a0 2024-09-21T06:08:19.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.851+0000 7f5e47e00640 1 -- 172.21.15.50:0/2844715983 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f5e4001a070 con 0x7f5e500703a0 2024-09-21T06:08:19.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.851+0000 7f5e47e00640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5e30074880 0x7f5e30076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:19.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.851+0000 7f5e47e00640 1 -- 172.21.15.50:0/2844715983 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7f5e40092530 con 0x7f5e500703a0 2024-09-21T06:08:19.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.851+0000 7f5e4fe00640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5e30074880 0x7f5e30076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:19.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.855+0000 7f5e4fe00640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5e30074880 0x7f5e30076d90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0af000 tx=0x7f5e3c01ae60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:19.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:19.859+0000 7f5e47e00640 1 -- 172.21.15.50:0/2844715983 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5e5015c0d0 con 0x7f5e500703a0 2024-09-21T06:08:20.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.259+0000 7f5e56c00640 1 -- 172.21.15.50:0/2844715983 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone986813", "target": ["mon-mgr", ""]}) -- 0x7f5e501582c0 con 0x7f5e30074880 2024-09-21T06:08:20.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.303+0000 7f5e47e00640 1 -- 172.21.15.50:0/2844715983 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5e501582c0 con 0x7f5e30074880 2024-09-21T06:08:20.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.307+0000 7f5e56c00640 1 -- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5e30074880 msgr2=0x7f5e30076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:20.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.307+0000 7f5e56c00640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5e30074880 0x7f5e30076d90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0af000 tx=0x7f5e3c01ae60 comp rx=0 tx=0).stop 2024-09-21T06:08:20.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.307+0000 7f5e56c00640 1 -- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e500703a0 msgr2=0x7f5e5015aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:20.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.307+0000 7f5e56c00640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e500703a0 0x7f5e5015aee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5e500b5390 tx=0x7f5e4000cb70 comp rx=0 tx=0).stop 2024-09-21T06:08:20.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.311+0000 7f5e56c00640 1 -- 172.21.15.50:0/2844715983 shutdown_connections 2024-09-21T06:08:20.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.311+0000 7f5e56c00640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e50159130 0x7f5e50156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:20.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.311+0000 7f5e56c00640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5e500703a0 0x7f5e5015aee0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:20.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.311+0000 7f5e56c00640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5e30074880 0x7f5e30076d90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:20.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.311+0000 7f5e56c00640 1 --2- 172.21.15.50:0/2844715983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e5006cdf0 0x7f5e50161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:20.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.311+0000 7f5e56c00640 1 -- 172.21.15.50:0/2844715983 >> 172.21.15.50:0/2844715983 conn(0x7f5e500b35f0 msgr2=0x7f5e5015f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:20.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.311+0000 7f5e56c00640 1 -- 172.21.15.50:0/2844715983 shutdown_connections 2024-09-21T06:08:20.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.311+0000 7f5e56c00640 1 -- 172.21.15.50:0/2844715983 wait complete. 2024-09-21T06:08:20.347 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T06:08:20.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:08:20.446 INFO:teuthology.orchestra.run.smithi050.stdout:2 2024-09-21T06:08:20.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:08:20.458 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:08:20.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.631+0000 7f84fea00640 1 -- 172.21.15.50:0/3890991611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84f8157030 msgr2=0x7f84f80b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:20.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.631+0000 7f84fea00640 1 --2- 172.21.15.50:0/3890991611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84f8157030 0x7f84f80b5a80 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009900 tx=0x7f84e801c850 comp rx=0 tx=0).stop 2024-09-21T06:08:20.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.631+0000 7f84fea00640 1 -- 172.21.15.50:0/3890991611 shutdown_connections 2024-09-21T06:08:20.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.631+0000 7f84fea00640 1 --2- 172.21.15.50:0/3890991611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84f8157030 0x7f84f80b5a80 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:20.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.631+0000 7f84fea00640 1 -- 172.21.15.50:0/3890991611 >> 172.21.15.50:0/3890991611 conn(0x7f84f80708c0 msgr2=0x7f84f8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:20.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.631+0000 7f84fea00640 1 -- 172.21.15.50:0/3890991611 shutdown_connections 2024-09-21T06:08:20.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.631+0000 7f84fea00640 1 -- 172.21.15.50:0/3890991611 wait complete. 2024-09-21T06:08:20.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.631+0000 7f84fea00640 1 Processor -- start 2024-09-21T06:08:20.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.631+0000 7f84fea00640 1 -- start start 2024-09-21T06:08:20.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84fea00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84f8157030 0x7f84f8155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:20.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84fea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84f8160d50 0x7f84f814bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:20.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84fea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84f81f9740 0x7f84f80b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:20.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84fea00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f84f816b8c0 con 0x7f84f8160d50 2024-09-21T06:08:20.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84fea00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f84f816b740 con 0x7f84f81f9740 2024-09-21T06:08:20.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84fea00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f84f816ba40 con 0x7f84f8157030 2024-09-21T06:08:20.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84f8160d50 0x7f84f814bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:20.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84f7e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84f8157030 0x7f84f8155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:20.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84f8160d50 0x7f84f814bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57620/0 (socket says 172.21.15.50:57620) 2024-09-21T06:08:20.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84f7e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84f8157030 0x7f84f8155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:32950/0 (socket says 172.21.15.50:32950) 2024-09-21T06:08:20.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84efe00640 1 -- 172.21.15.50:0/2908110465 learned_addr learned my addr 172.21.15.50:0/2908110465 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:20.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84fca00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84f81f9740 0x7f84f80b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:20.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84f7e00640 1 -- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84f81f9740 msgr2=0x7f84f80b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:20.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84f7e00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84f81f9740 0x7f84f80b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:20.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84f7e00640 1 -- 172.21.15.50:0/2908110465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84f8160d50 msgr2=0x7f84f814bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:20.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84f7e00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84f8160d50 0x7f84f814bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:20.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84f7e00640 1 -- 172.21.15.50:0/2908110465 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f81543c0 con 0x7f84f8157030 2024-09-21T06:08:20.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84efe00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84f8160d50 0x7f84f814bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:20.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84f7e00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84f8157030 0x7f84f8155d60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f84e80ac000 tx=0x7f84e8005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:20.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84f4c00640 1 -- 172.21.15.50:0/2908110465 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e80b2040 con 0x7f84f8157030 2024-09-21T06:08:20.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84f4c00640 1 -- 172.21.15.50:0/2908110465 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84e8004330 con 0x7f84f8157030 2024-09-21T06:08:20.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84f4c00640 1 -- 172.21.15.50:0/2908110465 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8003860 con 0x7f84f8157030 2024-09-21T06:08:20.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84fea00640 1 -- 172.21.15.50:0/2908110465 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f814c7f0 con 0x7f84f8157030 2024-09-21T06:08:20.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84fea00640 1 -- 172.21.15.50:0/2908110465 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f81fa150 con 0x7f84f8157030 2024-09-21T06:08:20.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.635+0000 7f84fea00640 1 -- 172.21.15.50:0/2908110465 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f80729f0 con 0x7f84f8157030 2024-09-21T06:08:20.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.639+0000 7f84f4c00640 1 -- 172.21.15.50:0/2908110465 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f84e8003a00 con 0x7f84f8157030 2024-09-21T06:08:20.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.639+0000 7f84f4c00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f84d4074880 0x7f84d4076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:20.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.639+0000 7f84f4c00640 1 -- 172.21.15.50:0/2908110465 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7f84e812c920 con 0x7f84f8157030 2024-09-21T06:08:20.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.639+0000 7f84efe00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f84d4074880 0x7f84d4076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:20.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.643+0000 7f84efe00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f84d4074880 0x7f84d4076d90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f84f814eba0 tx=0x7f84e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:20.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.651+0000 7f84f4c00640 1 -- 172.21.15.50:0/2908110465 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f84f81543c0 con 0x7f84f8157030 2024-09-21T06:08:21.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:20.999+0000 7f84fea00640 1 -- 172.21.15.50:0/2908110465 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f84f80d2260 con 0x7f84f8157030 2024-09-21T06:08:21.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.003+0000 7f84f4c00640 1 -- 172.21.15.50:0/2908110465 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v47)=0 v47) ==== 131+0+0 (secure 0 0 0) 0x7f84e80164d0 con 0x7f84f8157030 2024-09-21T06:08:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.007+0000 7f84fea00640 1 -- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f84d4074880 msgr2=0x7f84d4076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.007+0000 7f84fea00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f84d4074880 0x7f84d4076d90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f84f814eba0 tx=0x7f84e0009040 comp rx=0 tx=0).stop 2024-09-21T06:08:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.007+0000 7f84fea00640 1 -- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84f8157030 msgr2=0x7f84f8155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.007+0000 7f84fea00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84f8157030 0x7f84f8155d60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f84e80ac000 tx=0x7f84e8005790 comp rx=0 tx=0).stop 2024-09-21T06:08:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.007+0000 7f84fea00640 1 -- 172.21.15.50:0/2908110465 shutdown_connections 2024-09-21T06:08:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.007+0000 7f84fea00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84f81f9740 0x7f84f80b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.007+0000 7f84fea00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f84d4074880 0x7f84d4076d90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.007+0000 7f84fea00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84f8160d50 0x7f84f814bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.007+0000 7f84fea00640 1 --2- 172.21.15.50:0/2908110465 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f84f8157030 0x7f84f8155d60 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f84e80ac000 tx=0x7f84e8005790 comp rx=0 tx=0).stop 2024-09-21T06:08:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.007+0000 7f84fea00640 1 -- 172.21.15.50:0/2908110465 >> 172.21.15.50:0/2908110465 conn(0x7f84f80708c0 msgr2=0x7f84f80d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:21.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.011+0000 7f84fea00640 1 -- 172.21.15.50:0/2908110465 shutdown_connections 2024-09-21T06:08:21.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.011+0000 7f84fea00640 1 -- 172.21.15.50:0/2908110465 wait complete. 2024-09-21T06:08:21.051 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:08:21.051 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:08:21.164 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:08:21.266 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:08:21.266 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:08:21.266 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:08:21.266 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:08:21.282 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:08:21.282 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:08:21.340 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:08:21.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.583+0000 7faf3e000640 1 -- 172.21.15.50:0/1560386420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf381527b0 msgr2=0x7faf38074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:21.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.583+0000 7faf3e000640 1 --2- 172.21.15.50:0/1560386420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf381527b0 0x7faf38074e30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7faf38072e30 tx=0x7faf2001c800 comp rx=0 tx=0).stop 2024-09-21T06:08:21.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.583+0000 7faf3e000640 1 -- 172.21.15.50:0/1560386420 shutdown_connections 2024-09-21T06:08:21.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.583+0000 7faf3e000640 1 --2- 172.21.15.50:0/1560386420 >> v2:172.21.15.88:3300/0 conn(0x7faf380b4de0 0x7faf380b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.583+0000 7faf3e000640 1 --2- 172.21.15.50:0/1560386420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf381527b0 0x7faf38074e30 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.583+0000 7faf3e000640 1 --2- 172.21.15.50:0/1560386420 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faf380b5c70 0x7faf3815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.583+0000 7faf3e000640 1 -- 172.21.15.50:0/1560386420 >> 172.21.15.50:0/1560386420 conn(0x7faf380b2040 msgr2=0x7faf380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:21.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.583+0000 7faf3e000640 1 -- 172.21.15.50:0/1560386420 shutdown_connections 2024-09-21T06:08:21.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.583+0000 7faf3e000640 1 -- 172.21.15.50:0/1560386420 wait complete. 2024-09-21T06:08:21.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf3e000640 1 Processor -- start 2024-09-21T06:08:21.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf3e000640 1 -- start start 2024-09-21T06:08:21.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf3e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf380b4de0 0x7faf3814bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:21.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf3e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf380b5c70 0x7faf38153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:21.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf3e000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faf381527b0 0x7faf38146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:21.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf3e000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faf3815f6b0 con 0x7faf380b4de0 2024-09-21T06:08:21.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf3e000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf3815f530 con 0x7faf380b5c70 2024-09-21T06:08:21.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf3e000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7faf3815f830 con 0x7faf381527b0 2024-09-21T06:08:21.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf37400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf380b4de0 0x7faf3814bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:21.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf37400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf380b4de0 0x7faf3814bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57644/0 (socket says 172.21.15.50:57644) 2024-09-21T06:08:21.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf37400640 1 -- 172.21.15.50:0/737912288 learned_addr learned my addr 172.21.15.50:0/737912288 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:21.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf2fe00640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf380b5c70 0x7faf38153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:21.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf37e00640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faf381527b0 0x7faf38146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:21.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf37400640 1 -- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faf381527b0 msgr2=0x7faf38146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:21.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf37400640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faf381527b0 0x7faf38146c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf37400640 1 -- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf380b5c70 msgr2=0x7faf38153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:21.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf37400640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf380b5c70 0x7faf38153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf37400640 1 -- 172.21.15.50:0/737912288 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf381538d0 con 0x7faf380b4de0 2024-09-21T06:08:21.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.587+0000 7faf2fe00640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf380b5c70 0x7faf38153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:21.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.591+0000 7faf37400640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf380b4de0 0x7faf3814bda0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7faf2800e7c0 tx=0x7faf2800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:21.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.591+0000 7faf2f400640 1 -- 172.21.15.50:0/737912288 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2800cab0 con 0x7faf380b4de0 2024-09-21T06:08:21.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.591+0000 7faf2f400640 1 -- 172.21.15.50:0/737912288 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf2800cc50 con 0x7faf380b4de0 2024-09-21T06:08:21.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.591+0000 7faf2f400640 1 -- 172.21.15.50:0/737912288 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28005020 con 0x7faf380b4de0 2024-09-21T06:08:21.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.591+0000 7faf3e000640 1 -- 172.21.15.50:0/737912288 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf38156e20 con 0x7faf380b4de0 2024-09-21T06:08:21.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.591+0000 7faf3e000640 1 -- 172.21.15.50:0/737912288 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf381573e0 con 0x7faf380b4de0 2024-09-21T06:08:21.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.591+0000 7faf3e000640 1 -- 172.21.15.50:0/737912288 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3815fce0 con 0x7faf380b4de0 2024-09-21T06:08:21.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.591+0000 7faf2f400640 1 -- 172.21.15.50:0/737912288 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7faf280051c0 con 0x7faf380b4de0 2024-09-21T06:08:21.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.595+0000 7faf2f400640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf140748b0 0x7faf14076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:21.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.595+0000 7faf2f400640 1 -- 172.21.15.50:0/737912288 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7faf28093760 con 0x7faf380b4de0 2024-09-21T06:08:21.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.595+0000 7faf2fe00640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf140748b0 0x7faf14076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:21.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.595+0000 7faf2fe00640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf140748b0 0x7faf14076dc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faf200af000 tx=0x7faf2001b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:21.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.599+0000 7faf2f400640 1 -- 172.21.15.50:0/737912288 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7faf2809e080 con 0x7faf380b4de0 2024-09-21T06:08:21.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.879+0000 7faf3e000640 1 -- 172.21.15.50:0/737912288 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faf380b5290 con 0x7faf380b4de0 2024-09-21T06:08:21.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.879+0000 7faf2f400640 1 -- 172.21.15.50:0/737912288 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v88) ==== 80+0+0 (secure 0 0 0) 0x7faf28060270 con 0x7faf380b4de0 2024-09-21T06:08:21.879 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T06:08:21.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.883+0000 7faf3e000640 1 -- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf140748b0 msgr2=0x7faf14076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:21.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.883+0000 7faf3e000640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf140748b0 0x7faf14076dc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faf200af000 tx=0x7faf2001b040 comp rx=0 tx=0).stop 2024-09-21T06:08:21.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.883+0000 7faf3e000640 1 -- 172.21.15.50:0/737912288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf380b4de0 msgr2=0x7faf3814bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:21.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.883+0000 7faf3e000640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf380b4de0 0x7faf3814bda0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7faf2800e7c0 tx=0x7faf2800ebc0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.883+0000 7faf3e000640 1 -- 172.21.15.50:0/737912288 shutdown_connections 2024-09-21T06:08:21.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.883+0000 7faf3e000640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faf381527b0 0x7faf38146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.883+0000 7faf3e000640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf380b5c70 0x7faf38153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.883+0000 7faf3e000640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf140748b0 0x7faf14076dc0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.883+0000 7faf3e000640 1 --2- 172.21.15.50:0/737912288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf380b4de0 0x7faf3814bda0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:21.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.883+0000 7faf3e000640 1 -- 172.21.15.50:0/737912288 >> 172.21.15.50:0/737912288 conn(0x7faf380b2040 msgr2=0x7faf3806e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:21.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.883+0000 7faf3e000640 1 -- 172.21.15.50:0/737912288 shutdown_connections 2024-09-21T06:08:21.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:21.883+0000 7faf3e000640 1 -- 172.21.15.50:0/737912288 wait complete. 2024-09-21T06:08:21.904 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:08:21.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:08:21.904 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:08:21.962 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:08:21.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:08:21.963 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:08:21.980 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:08:21.980 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:08:21.980 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:08:21.990 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:08:21.990 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:08:21.991 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:08:21.991 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:08:22.036 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:08:22.036 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:08:22.036 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:08:22.036 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:08:22.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:08:22.046 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:08:22.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.239+0000 7fa790800640 1 -- 172.21.15.50:0/1543356289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7880cdd80 msgr2=0x7fa7880b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.239+0000 7fa790800640 1 --2- 172.21.15.50:0/1543356289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7880cdd80 0x7fa7880b3a50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa78400acd0 tx=0x7fa78401def0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.239+0000 7fa790800640 1 -- 172.21.15.50:0/1543356289 shutdown_connections 2024-09-21T06:08:22.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.239+0000 7fa790800640 1 --2- 172.21.15.50:0/1543356289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7880cdd80 0x7fa7880b3a50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.239+0000 7fa790800640 1 -- 172.21.15.50:0/1543356289 >> 172.21.15.50:0/1543356289 conn(0x7fa7880b2040 msgr2=0x7fa7880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:22.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.239+0000 7fa790800640 1 -- 172.21.15.50:0/1543356289 shutdown_connections 2024-09-21T06:08:22.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.239+0000 7fa790800640 1 -- 172.21.15.50:0/1543356289 wait complete. 2024-09-21T06:08:22.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.239+0000 7fa790800640 1 Processor -- start 2024-09-21T06:08:22.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.239+0000 7fa790800640 1 -- start start 2024-09-21T06:08:22.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa790800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa7880cd970 0x7fa7881566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa790800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7880cdd80 0x7fa78814e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa790800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa78815aa50 0x7fa788158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa790800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881694d0 con 0x7fa7880cdd80 2024-09-21T06:08:22.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa790800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa788169350 con 0x7fa78815aa50 2024-09-21T06:08:22.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa790800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa788169650 con 0x7fa7880cd970 2024-09-21T06:08:22.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa78d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7880cdd80 0x7fa78814e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa78d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7880cdd80 0x7fa78814e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57654/0 (socket says 172.21.15.50:57654) 2024-09-21T06:08:22.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa78d400640 1 -- 172.21.15.50:0/1599359606 learned_addr learned my addr 172.21.15.50:0/1599359606 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:22.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa78e800640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa78815aa50 0x7fa788158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa78de00640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa7880cd970 0x7fa7881566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa78d400640 1 -- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa7880cd970 msgr2=0x7fa7881566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa78d400640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa7880cd970 0x7fa7881566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa78d400640 1 -- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa78815aa50 msgr2=0x7fa788158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa78d400640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa78815aa50 0x7fa788158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa78d400640 1 -- 172.21.15.50:0/1599359606 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa788156c20 con 0x7fa7880cdd80 2024-09-21T06:08:22.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa78d400640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7880cdd80 0x7fa78814e1d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa77800c400 tx=0x7fa77800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:22.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa782000640 1 -- 172.21.15.50:0/1599359606 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778011790 con 0x7fa7880cdd80 2024-09-21T06:08:22.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa790800640 1 -- 172.21.15.50:0/1599359606 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78814d130 con 0x7fa7880cdd80 2024-09-21T06:08:22.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa782000640 1 -- 172.21.15.50:0/1599359606 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa778017070 con 0x7fa7880cdd80 2024-09-21T06:08:22.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa782000640 1 -- 172.21.15.50:0/1599359606 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778011ce0 con 0x7fa7880cdd80 2024-09-21T06:08:22.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa790800640 1 -- 172.21.15.50:0/1599359606 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa78814d6f0 con 0x7fa7880cdd80 2024-09-21T06:08:22.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.243+0000 7fa790800640 1 -- 172.21.15.50:0/1599359606 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7880b1400 con 0x7fa7880cdd80 2024-09-21T06:08:22.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.247+0000 7fa782000640 1 -- 172.21.15.50:0/1599359606 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fa77801c460 con 0x7fa7880cdd80 2024-09-21T06:08:22.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.247+0000 7fa782000640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa76c074880 0x7fa76c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.247+0000 7fa782000640 1 -- 172.21.15.50:0/1599359606 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7fa778093710 con 0x7fa7880cdd80 2024-09-21T06:08:22.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.247+0000 7fa78de00640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa76c074880 0x7fa76c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.247+0000 7fa78de00640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa76c074880 0x7fa76c076d90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa77c000c10 tx=0x7fa77c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:22.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.251+0000 7fa782000640 1 -- 172.21.15.50:0/1599359606 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa77805fbd0 con 0x7fa7880cdd80 2024-09-21T06:08:22.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.459+0000 7fa790800640 1 -- 172.21.15.50:0/1599359606 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7880cd1e0 con 0x7fa7880cdd80 2024-09-21T06:08:22.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa782000640 1 -- 172.21.15.50:0/1599359606 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5225 (secure 0 0 0) 0x7fa778064600 con 0x7fa7880cdd80 2024-09-21T06:08:22.461 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:22.461 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":56,"btime":"2024-09-21T06:06:14:518361+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3691304879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3691304879},{"type":"v1","addr":"172.21.15.50:6834","nonce":3691304879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1924969859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1924969859},{"type":"v1","addr":"172.21.15.88:6837","nonce":1924969859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3025915209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3025915209},{"type":"v1","addr":"172.21.15.88:6835","nonce":3025915209}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:06:05.890737+0000","modified":"2024-09-21T06:06:08.920852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/1211403675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1211403675},{"type":"v1","addr":"172.21.15.50:6835","nonce":1211403675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7029,"qdb_cluster":[7029]},"id":6}]} 2024-09-21T06:08:22.461 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 56 2024-09-21T06:08:22.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa790800640 1 -- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa76c074880 msgr2=0x7fa76c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa790800640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa76c074880 0x7fa76c076d90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa77c000c10 tx=0x7fa77c009040 comp rx=0 tx=0).stop 2024-09-21T06:08:22.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa790800640 1 -- 172.21.15.50:0/1599359606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7880cdd80 msgr2=0x7fa78814e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa790800640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7880cdd80 0x7fa78814e1d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa77800c400 tx=0x7fa77800c8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa790800640 1 -- 172.21.15.50:0/1599359606 shutdown_connections 2024-09-21T06:08:22.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa790800640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa78815aa50 0x7fa788158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa790800640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa76c074880 0x7fa76c076d90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa790800640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7880cdd80 0x7fa78814e1d0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa790800640 1 --2- 172.21.15.50:0/1599359606 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa7880cd970 0x7fa7881566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa790800640 1 -- 172.21.15.50:0/1599359606 >> 172.21.15.50:0/1599359606 conn(0x7fa7880b2040 msgr2=0x7fa7880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:22.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa790800640 1 -- 172.21.15.50:0/1599359606 shutdown_connections 2024-09-21T06:08:22.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.463+0000 7fa790800640 1 -- 172.21.15.50:0/1599359606 wait complete. 2024-09-21T06:08:22.484 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:08:22.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.651+0000 7f168dc00640 1 -- 172.21.15.50:0/2388311682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f168815f5c0 msgr2=0x7f168815f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.651+0000 7f168dc00640 1 --2- 172.21.15.50:0/2388311682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f168815f5c0 0x7f168815f9d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f1680009bc0 tx=0x7f168001cb10 comp rx=0 tx=0).stop 2024-09-21T06:08:22.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.651+0000 7f168dc00640 1 -- 172.21.15.50:0/2388311682 shutdown_connections 2024-09-21T06:08:22.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.651+0000 7f168dc00640 1 --2- 172.21.15.50:0/2388311682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f168815f5c0 0x7f168815f9d0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.651+0000 7f168dc00640 1 -- 172.21.15.50:0/2388311682 >> 172.21.15.50:0/2388311682 conn(0x7f16880b41f0 msgr2=0x7f16880b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:22.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.651+0000 7f168dc00640 1 -- 172.21.15.50:0/2388311682 shutdown_connections 2024-09-21T06:08:22.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.651+0000 7f168dc00640 1 -- 172.21.15.50:0/2388311682 wait complete. 2024-09-21T06:08:22.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.655+0000 7f168dc00640 1 Processor -- start 2024-09-21T06:08:22.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.655+0000 7f168dc00640 1 -- start start 2024-09-21T06:08:22.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f168dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168806dbc0 0x7f168815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f168dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16880d1140 0x7f16880d0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f168dc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f168815f5c0 0x7f16880cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f168dc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f16880692c0 con 0x7f16880d1140 2024-09-21T06:08:22.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f168dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1688069140 con 0x7f168806dbc0 2024-09-21T06:08:22.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f168dc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1688069440 con 0x7f168815f5c0 2024-09-21T06:08:22.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f1687400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16880d1140 0x7f16880d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f1687400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16880d1140 0x7f16880d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57678/0 (socket says 172.21.15.50:57678) 2024-09-21T06:08:22.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f1687400640 1 -- 172.21.15.50:0/1006227301 learned_addr learned my addr 172.21.15.50:0/1006227301 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:22.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f168d200640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f168815f5c0 0x7f16880cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f1687e00640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168806dbc0 0x7f168815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f1687e00640 1 -- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f168815f5c0 msgr2=0x7f16880cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f1687e00640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f168815f5c0 0x7f16880cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f1687e00640 1 -- 172.21.15.50:0/1006227301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16880d1140 msgr2=0x7f16880d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f1687e00640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16880d1140 0x7f16880d0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f1687e00640 1 -- 172.21.15.50:0/1006227301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16880cd1a0 con 0x7f168806dbc0 2024-09-21T06:08:22.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f1687e00640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168806dbc0 0x7f168815db60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f167800c390 tx=0x7f167800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:22.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.659+0000 7f167fe00640 1 -- 172.21.15.50:0/1006227301 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678011790 con 0x7f168806dbc0 2024-09-21T06:08:22.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.663+0000 7f168dc00640 1 -- 172.21.15.50:0/1006227301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168815d080 con 0x7f168806dbc0 2024-09-21T06:08:22.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.663+0000 7f167fe00640 1 -- 172.21.15.50:0/1006227301 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1678017070 con 0x7f168806dbc0 2024-09-21T06:08:22.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.663+0000 7f167fe00640 1 -- 172.21.15.50:0/1006227301 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678011c60 con 0x7f168806dbc0 2024-09-21T06:08:22.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.663+0000 7f168dc00640 1 -- 172.21.15.50:0/1006227301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168815d640 con 0x7f168806dbc0 2024-09-21T06:08:22.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.663+0000 7f168dc00640 1 -- 172.21.15.50:0/1006227301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16880699d0 con 0x7f168806dbc0 2024-09-21T06:08:22.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.667+0000 7f167fe00640 1 -- 172.21.15.50:0/1006227301 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f167801c460 con 0x7f168806dbc0 2024-09-21T06:08:22.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.667+0000 7f167fe00640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f16680748b0 0x7f1668076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.667+0000 7f167fe00640 1 -- 172.21.15.50:0/1006227301 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7f1678092cc0 con 0x7f168806dbc0 2024-09-21T06:08:22.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.667+0000 7f1687400640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f16680748b0 0x7f1668076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.667+0000 7f1687400640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f16680748b0 0x7f1668076dc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f168809f0f0 tx=0x7f1674008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:22.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:22.675+0000 7f167fe00640 1 -- 172.21.15.50:0/1006227301 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f167805f740 con 0x7f168806dbc0 2024-09-21T06:08:23.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.011+0000 7f168dc00640 1 -- 172.21.15.50:0/1006227301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16880d1ea0 con 0x7f168806dbc0 2024-09-21T06:08:23.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.011+0000 7f167fe00640 1 -- 172.21.15.50:0/1006227301 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5225 (secure 0 0 0) 0x7f167805f040 con 0x7f168806dbc0 2024-09-21T06:08:23.012 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:23.012 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":56,"btime":"2024-09-21T06:06:14:518361+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3691304879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3691304879},{"type":"v1","addr":"172.21.15.50:6834","nonce":3691304879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1924969859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1924969859},{"type":"v1","addr":"172.21.15.88:6837","nonce":1924969859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3025915209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3025915209},{"type":"v1","addr":"172.21.15.88:6835","nonce":3025915209}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:06:05.890737+0000","modified":"2024-09-21T06:06:08.920852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/1211403675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1211403675},{"type":"v1","addr":"172.21.15.50:6835","nonce":1211403675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7029,"qdb_cluster":[7029]},"id":6}]} 2024-09-21T06:08:23.012 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 56 2024-09-21T06:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.015+0000 7f168dc00640 1 -- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f16680748b0 msgr2=0x7f1668076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.015+0000 7f168dc00640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f16680748b0 0x7f1668076dc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f168809f0f0 tx=0x7f1674008040 comp rx=0 tx=0).stop 2024-09-21T06:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.015+0000 7f168dc00640 1 -- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168806dbc0 msgr2=0x7f168815db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.015+0000 7f168dc00640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168806dbc0 0x7f168815db60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f167800c390 tx=0x7f167800c850 comp rx=0 tx=0).stop 2024-09-21T06:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.015+0000 7f168dc00640 1 -- 172.21.15.50:0/1006227301 shutdown_connections 2024-09-21T06:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.015+0000 7f168dc00640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f168815f5c0 0x7f16880cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.015+0000 7f168dc00640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f16680748b0 0x7f1668076dc0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.015+0000 7f168dc00640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16880d1140 0x7f16880d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.015+0000 7f168dc00640 1 --2- 172.21.15.50:0/1006227301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168806dbc0 0x7f168815db60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.015+0000 7f168dc00640 1 -- 172.21.15.50:0/1006227301 >> 172.21.15.50:0/1006227301 conn(0x7f16880b41f0 msgr2=0x7f16880b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.015+0000 7f168dc00640 1 -- 172.21.15.50:0/1006227301 shutdown_connections 2024-09-21T06:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.015+0000 7f168dc00640 1 -- 172.21.15.50:0/1006227301 wait complete. 2024-09-21T06:08:23.034 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:08:23.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.195+0000 7f25ee000640 1 -- 172.21.15.50:0/2732977708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e80b5c70 msgr2=0x7f25e815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.195+0000 7f25ee000640 1 --2- 172.21.15.50:0/2732977708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e80b5c70 0x7f25e815bb60 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f25e8070710 tx=0x7f25d801c740 comp rx=0 tx=0).stop 2024-09-21T06:08:23.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.195+0000 7f25ee000640 1 -- 172.21.15.50:0/2732977708 shutdown_connections 2024-09-21T06:08:23.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.195+0000 7f25ee000640 1 --2- 172.21.15.50:0/2732977708 >> v2:172.21.15.88:3300/0 conn(0x7f25e81527b0 0x7f25e8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.195+0000 7f25ee000640 1 --2- 172.21.15.50:0/2732977708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e80b5c70 0x7f25e815bb60 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.195+0000 7f25ee000640 1 -- 172.21.15.50:0/2732977708 >> 172.21.15.50:0/2732977708 conn(0x7f25e80b2040 msgr2=0x7f25e80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:23.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.195+0000 7f25ee000640 1 -- 172.21.15.50:0/2732977708 shutdown_connections 2024-09-21T06:08:23.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.195+0000 7f25ee000640 1 -- 172.21.15.50:0/2732977708 wait complete. 2024-09-21T06:08:23.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.195+0000 7f25ee000640 1 Processor -- start 2024-09-21T06:08:23.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.195+0000 7f25ee000640 1 -- start start 2024-09-21T06:08:23.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25ee000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e80b4de0 0x7f25e8147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25ee000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25e80b5c70 0x7f25e80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25ee000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25e81527b0 0x7f25e80aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25ee000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8168a80 con 0x7f25e80b4de0 2024-09-21T06:08:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25ee000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8168900 con 0x7f25e81527b0 2024-09-21T06:08:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25ee000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f25e8168c00 con 0x7f25e80b5c70 2024-09-21T06:08:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25e7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e80b4de0 0x7f25e8147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25e7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e80b4de0 0x7f25e8147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57684/0 (socket says 172.21.15.50:57684) 2024-09-21T06:08:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25e7400640 1 -- 172.21.15.50:0/678489262 learned_addr learned my addr 172.21.15.50:0/678489262 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25dea00640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25e80b5c70 0x7f25e80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25e7e00640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25e81527b0 0x7f25e80aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25e7400640 1 -- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25e80b5c70 msgr2=0x7f25e80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25e7400640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25e80b5c70 0x7f25e80ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25e7400640 1 -- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25e81527b0 msgr2=0x7f25e80aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25e7400640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25e81527b0 0x7f25e80aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25e7400640 1 -- 172.21.15.50:0/678489262 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e8148190 con 0x7f25e80b4de0 2024-09-21T06:08:23.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25e7400640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e80b4de0 0x7f25e8147c50 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c10 tx=0x7f25d80a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:23.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25dfe00640 1 -- 172.21.15.50:0/678489262 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d80bb070 con 0x7f25e80b4de0 2024-09-21T06:08:23.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25dfe00640 1 -- 172.21.15.50:0/678489262 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25d80043b0 con 0x7f25e80b4de0 2024-09-21T06:08:23.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25dfe00640 1 -- 172.21.15.50:0/678489262 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d80b2040 con 0x7f25e80b4de0 2024-09-21T06:08:23.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25ee000640 1 -- 172.21.15.50:0/678489262 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e81fcb70 con 0x7f25e80b4de0 2024-09-21T06:08:23.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25ee000640 1 -- 172.21.15.50:0/678489262 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e81fd130 con 0x7f25e80b4de0 2024-09-21T06:08:23.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.199+0000 7f25ee000640 1 -- 172.21.15.50:0/678489262 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e815bb60 con 0x7f25e80b4de0 2024-09-21T06:08:23.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.203+0000 7f25dfe00640 1 -- 172.21.15.50:0/678489262 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f25d8003800 con 0x7f25e80b4de0 2024-09-21T06:08:23.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.203+0000 7f25dfe00640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25c4074880 0x7f25c4076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:23.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.203+0000 7f25dfe00640 1 -- 172.21.15.50:0/678489262 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5850+0+0 (secure 0 0 0) 0x7f25d812cab0 con 0x7f25e80b4de0 2024-09-21T06:08:23.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.203+0000 7f25dea00640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25c4074880 0x7f25c4076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:23.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.203+0000 7f25dea00640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25c4074880 0x7f25c4076d90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f25d00058d0 tx=0x7f25d0005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:23.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.207+0000 7f25dfe00640 1 -- 172.21.15.50:0/678489262 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f25e8075580 con 0x7f25e80b4de0 2024-09-21T06:08:23.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.387+0000 7f25ee000640 1 -- 172.21.15.50:0/678489262 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25e81660c0 con 0x7f25e80b4de0 2024-09-21T06:08:23.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.387+0000 7f25dfe00640 1 -- 172.21.15.50:0/678489262 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16920 (secure 0 0 0) 0x7f25d80163f0 con 0x7f25e80b4de0 2024-09-21T06:08:23.387 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:23.387 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":88,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:08:12.807934+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:06:03.633659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T06:06:04.852705+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":88,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.0","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:08:23.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.391+0000 7f25ee000640 1 -- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25c4074880 msgr2=0x7f25c4076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.391+0000 7f25ee000640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25c4074880 0x7f25c4076d90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f25d00058d0 tx=0x7f25d0005860 comp rx=0 tx=0).stop 2024-09-21T06:08:23.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.391+0000 7f25ee000640 1 -- 172.21.15.50:0/678489262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e80b4de0 msgr2=0x7f25e8147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.391+0000 7f25ee000640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e80b4de0 0x7f25e8147c50 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c10 tx=0x7f25d80a5620 comp rx=0 tx=0).stop 2024-09-21T06:08:23.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.391+0000 7f25ee000640 1 -- 172.21.15.50:0/678489262 shutdown_connections 2024-09-21T06:08:23.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.391+0000 7f25ee000640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25e81527b0 0x7f25e80aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.391+0000 7f25ee000640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25e80b5c70 0x7f25e80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.391+0000 7f25ee000640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25c4074880 0x7f25c4076d90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.391+0000 7f25ee000640 1 --2- 172.21.15.50:0/678489262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e80b4de0 0x7f25e8147c50 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.391+0000 7f25ee000640 1 -- 172.21.15.50:0/678489262 >> 172.21.15.50:0/678489262 conn(0x7f25e80b2040 msgr2=0x7f25e814d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:23.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.391+0000 7f25ee000640 1 -- 172.21.15.50:0/678489262 shutdown_connections 2024-09-21T06:08:23.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.391+0000 7f25ee000640 1 -- 172.21.15.50:0/678489262 wait complete. 2024-09-21T06:08:23.411 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:08:23.411 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:08:23.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.579+0000 7fcd92400640 1 -- 172.21.15.50:0/1776046253 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd8c1527b0 msgr2=0x7fcd8c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.579+0000 7fcd92400640 1 --2- 172.21.15.50:0/1776046253 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd8c1527b0 0x7fcd8c074e30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009900 tx=0x7fcd7801c820 comp rx=0 tx=0).stop 2024-09-21T06:08:23.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.579+0000 7fcd92400640 1 -- 172.21.15.50:0/1776046253 shutdown_connections 2024-09-21T06:08:23.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.579+0000 7fcd92400640 1 --2- 172.21.15.50:0/1776046253 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd8c1527b0 0x7fcd8c074e30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.579+0000 7fcd92400640 1 --2- 172.21.15.50:0/1776046253 >> v2:172.21.15.88:3300/0 conn(0x7fcd8c0b5c70 0x7fcd8c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.579+0000 7fcd92400640 1 -- 172.21.15.50:0/1776046253 >> 172.21.15.50:0/1776046253 conn(0x7fcd8c0b2040 msgr2=0x7fcd8c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:23.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.579+0000 7fcd92400640 1 -- 172.21.15.50:0/1776046253 shutdown_connections 2024-09-21T06:08:23.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.579+0000 7fcd92400640 1 -- 172.21.15.50:0/1776046253 wait complete. 2024-09-21T06:08:23.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.579+0000 7fcd92400640 1 Processor -- start 2024-09-21T06:08:23.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.579+0000 7fcd92400640 1 -- start start 2024-09-21T06:08:23.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd92400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd8c0b4de0 0x7fcd8c149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:23.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd92400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd8c0b5c70 0x7fcd8c149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd92400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd8c1527b0 0x7fcd8c1fa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd92400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c168c00 con 0x7fcd8c1527b0 2024-09-21T06:08:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd92400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c168a80 con 0x7fcd8c0b4de0 2024-09-21T06:08:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd92400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8c168d80 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd8b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd8c0b4de0 0x7fcd8c149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd8aa00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd8c0b5c70 0x7fcd8c149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd8b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd8c0b4de0 0x7fcd8c149650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:39206/0 (socket says 172.21.15.50:39206) 2024-09-21T06:08:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd8be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd8c1527b0 0x7fcd8c1fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd8b400640 1 -- 172.21.15.50:0/1488069815 learned_addr learned my addr 172.21.15.50:0/1488069815 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd8aa00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd8c0b5c70 0x7fcd8c149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:33004/0 (socket says 172.21.15.50:33004) 2024-09-21T06:08:23.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd8aa00640 1 -- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd8c0b4de0 msgr2=0x7fcd8c149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd8aa00640 1 --2- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd8c0b4de0 0x7fcd8c149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd8aa00640 1 -- 172.21.15.50:0/1488069815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd8c1527b0 msgr2=0x7fcd8c1fa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd8aa00640 1 --2- 172.21.15.50:0/1488069815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd8c1527b0 0x7fcd8c1fa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd8aa00640 1 -- 172.21.15.50:0/1488069815 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8c14fda0 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd8aa00640 1 --2- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd8c0b5c70 0x7fcd8c149b90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcd780ac000 tx=0x7fcd780a5810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:23.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd83400640 1 -- 172.21.15.50:0/1488069815 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd780b2040 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd83400640 1 -- 172.21.15.50:0/1488069815 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd780bd070 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd92400640 1 -- 172.21.15.50:0/1488069815 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8c150080 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd83400640 1 -- 172.21.15.50:0/1488069815 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78003860 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd92400640 1 -- 172.21.15.50:0/1488069815 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd8c156410 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.583+0000 7fcd92400640 1 -- 172.21.15.50:0/1488069815 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd8c0cff10 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.587+0000 7fcd83400640 1 -- 172.21.15.50:0/1488069815 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fcd78003a00 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.587+0000 7fcd83400640 1 --2- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcd6c074880 0x7fcd6c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:23.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.587+0000 7fcd83400640 1 -- 172.21.15.50:0/1488069815 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5822+0+0 (secure 0 0 0) 0x7fcd7812e0c0 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.587+0000 7fcd8b400640 1 --2- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcd6c074880 0x7fcd6c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:23.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.587+0000 7fcd8b400640 1 --2- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcd6c074880 0x7fcd6c076d90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009680 tx=0x7fcd7c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:23.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.591+0000 7fcd83400640 1 -- 172.21.15.50:0/1488069815 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcd8c075580 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 -- 172.21.15.50:0/1488069815 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd8c15bb60 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd83400640 1 -- 172.21.15.50:0/1488069815 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5225 (secure 0 0 0) 0x7fcd780164d0 con 0x7fcd8c0b5c70 2024-09-21T06:08:23.810 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:23.810 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":56,"btime":"2024-09-21T06:06:14:518361+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3691304879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3691304879},{"type":"v1","addr":"172.21.15.50:6834","nonce":3691304879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1924969859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1924969859},{"type":"v1","addr":"172.21.15.88:6837","nonce":1924969859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3025915209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3025915209},{"type":"v1","addr":"172.21.15.88:6835","nonce":3025915209}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:06:05.890737+0000","modified":"2024-09-21T06:06:08.920852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/1211403675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1211403675},{"type":"v1","addr":"172.21.15.50:6835","nonce":1211403675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7029,"qdb_cluster":[7029]},"id":6}]} 2024-09-21T06:08:23.810 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 56 2024-09-21T06:08:23.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 -- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcd6c074880 msgr2=0x7fcd6c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 --2- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcd6c074880 0x7fcd6c076d90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009680 tx=0x7fcd7c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 -- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd8c0b5c70 msgr2=0x7fcd8c149b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 --2- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd8c0b5c70 0x7fcd8c149b90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcd780ac000 tx=0x7fcd780a5810 comp rx=0 tx=0).stop 2024-09-21T06:08:23.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 -- 172.21.15.50:0/1488069815 shutdown_connections 2024-09-21T06:08:23.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 --2- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcd6c074880 0x7fcd6c076d90 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 --2- 172.21.15.50:0/1488069815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd8c1527b0 0x7fcd8c1fa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 --2- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcd8c0b5c70 0x7fcd8c149b90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 --2- 172.21.15.50:0/1488069815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd8c0b4de0 0x7fcd8c149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 -- 172.21.15.50:0/1488069815 >> 172.21.15.50:0/1488069815 conn(0x7fcd8c0b2040 msgr2=0x7fcd8c154e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:23.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 -- 172.21.15.50:0/1488069815 shutdown_connections 2024-09-21T06:08:23.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.811+0000 7fcd92400640 1 -- 172.21.15.50:0/1488069815 wait complete. 2024-09-21T06:08:23.832 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:08:23.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.991+0000 7f873b200640 1 -- 172.21.15.50:0/1070108366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f873415f5c0 msgr2=0x7f873415f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.991+0000 7f873b200640 1 --2- 172.21.15.50:0/1070108366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f873415f5c0 0x7f873415f9d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8730009bc0 tx=0x7f873001cb10 comp rx=0 tx=0).stop 2024-09-21T06:08:23.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.991+0000 7f873b200640 1 -- 172.21.15.50:0/1070108366 shutdown_connections 2024-09-21T06:08:23.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.991+0000 7f873b200640 1 --2- 172.21.15.50:0/1070108366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f873415f5c0 0x7f873415f9d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.991+0000 7f873b200640 1 -- 172.21.15.50:0/1070108366 >> 172.21.15.50:0/1070108366 conn(0x7f87340b41f0 msgr2=0x7f87340b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:23.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.991+0000 7f873b200640 1 -- 172.21.15.50:0/1070108366 shutdown_connections 2024-09-21T06:08:23.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.991+0000 7f873b200640 1 -- 172.21.15.50:0/1070108366 wait complete. 2024-09-21T06:08:23.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f873b200640 1 Processor -- start 2024-09-21T06:08:23.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f873b200640 1 -- start start 2024-09-21T06:08:23.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f873b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f873406dbc0 0x7f87341562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:23.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f873b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87340d1140 0x7f873409a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:23.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f873b200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f873415f5c0 0x7f873409aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:23.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f873b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f873416a160 con 0x7f87340d1140 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f873b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8734169fe0 con 0x7f873406dbc0 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f873b200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f873416a2e0 con 0x7f873415f5c0 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f8739400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87340d1140 0x7f873409a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f8739400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87340d1140 0x7f873409a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57694/0 (socket says 172.21.15.50:57694) 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f8739400640 1 -- 172.21.15.50:0/53753375 learned_addr learned my addr 172.21.15.50:0/53753375 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f8739e00640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f873406dbc0 0x7f87341562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.995+0000 7f873a800640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f873415f5c0 0x7f873409aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f8739400640 1 -- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f873415f5c0 msgr2=0x7f873409aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f8739400640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f873415f5c0 0x7f873409aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f8739400640 1 -- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f873406dbc0 msgr2=0x7f87341562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f8739400640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f873406dbc0 0x7f87341562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f8739400640 1 -- 172.21.15.50:0/53753375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873415d300 con 0x7f87340d1140 2024-09-21T06:08:23.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f8739400640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87340d1140 0x7f873409a5b0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f872400c400 tx=0x7f872400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:23.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f872e000640 1 -- 172.21.15.50:0/53753375 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724011790 con 0x7f87340d1140 2024-09-21T06:08:23.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f872e000640 1 -- 172.21.15.50:0/53753375 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8724017070 con 0x7f87340d1140 2024-09-21T06:08:23.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f873b200640 1 -- 172.21.15.50:0/53753375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873415d060 con 0x7f87340d1140 2024-09-21T06:08:23.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f872e000640 1 -- 172.21.15.50:0/53753375 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724011ca0 con 0x7f87340d1140 2024-09-21T06:08:23.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f873b200640 1 -- 172.21.15.50:0/53753375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8734159a90 con 0x7f87340d1140 2024-09-21T06:08:23.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f873b200640 1 -- 172.21.15.50:0/53753375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f873415f9d0 con 0x7f87340d1140 2024-09-21T06:08:24.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:23.999+0000 7f872e000640 1 -- 172.21.15.50:0/53753375 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f872401c460 con 0x7f87340d1140 2024-09-21T06:08:24.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.003+0000 7f872e000640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87180748b0 0x7f8718076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.003+0000 7f872e000640 1 -- 172.21.15.50:0/53753375 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5822+0+0 (secure 0 0 0) 0x7f8724092c90 con 0x7f87340d1140 2024-09-21T06:08:24.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.003+0000 7f8739e00640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87180748b0 0x7f8718076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:24.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.003+0000 7f8739e00640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87180748b0 0x7f8718076dc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8728000c10 tx=0x7f8728009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:24.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.007+0000 7f872e000640 1 -- 172.21.15.50:0/53753375 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f872405f0c0 con 0x7f87340d1140 2024-09-21T06:08:24.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.215+0000 7f873b200640 1 -- 172.21.15.50:0/53753375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87340d1d00 con 0x7f87340d1140 2024-09-21T06:08:24.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.215+0000 7f872e000640 1 -- 172.21.15.50:0/53753375 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5225 (secure 0 0 0) 0x7f8724063af0 con 0x7f87340d1140 2024-09-21T06:08:24.216 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:24.216 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":56,"btime":"2024-09-21T06:06:14:518361+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3691304879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3691304879},{"type":"v1","addr":"172.21.15.50:6834","nonce":3691304879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1924969859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1924969859},{"type":"v1","addr":"172.21.15.88:6837","nonce":1924969859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3025915209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3025915209},{"type":"v1","addr":"172.21.15.88:6835","nonce":3025915209}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:06:05.890737+0000","modified":"2024-09-21T06:06:08.920852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/1211403675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1211403675},{"type":"v1","addr":"172.21.15.50:6835","nonce":1211403675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7029,"qdb_cluster":[7029]},"id":6}]} 2024-09-21T06:08:24.217 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 56 2024-09-21T06:08:24.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.219+0000 7f873b200640 1 -- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87180748b0 msgr2=0x7f8718076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.219+0000 7f873b200640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87180748b0 0x7f8718076dc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8728000c10 tx=0x7f8728009040 comp rx=0 tx=0).stop 2024-09-21T06:08:24.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.219+0000 7f873b200640 1 -- 172.21.15.50:0/53753375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87340d1140 msgr2=0x7f873409a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.219+0000 7f873b200640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87340d1140 0x7f873409a5b0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f872400c400 tx=0x7f872400c8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.219+0000 7f873b200640 1 -- 172.21.15.50:0/53753375 shutdown_connections 2024-09-21T06:08:24.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.219+0000 7f873b200640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f873415f5c0 0x7f873409aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.219+0000 7f873b200640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f87180748b0 0x7f8718076dc0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.219+0000 7f873b200640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87340d1140 0x7f873409a5b0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.219+0000 7f873b200640 1 --2- 172.21.15.50:0/53753375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f873406dbc0 0x7f87341562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.219+0000 7f873b200640 1 -- 172.21.15.50:0/53753375 >> 172.21.15.50:0/53753375 conn(0x7f87340b41f0 msgr2=0x7f87340b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:24.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.219+0000 7f873b200640 1 -- 172.21.15.50:0/53753375 shutdown_connections 2024-09-21T06:08:24.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.219+0000 7f873b200640 1 -- 172.21.15.50:0/53753375 wait complete. 2024-09-21T06:08:24.238 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:08:24.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.399+0000 7f1edd200640 1 -- 172.21.15.50:0/3127440070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed80b5c70 msgr2=0x7f1ed815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.399+0000 7f1edd200640 1 --2- 172.21.15.50:0/3127440070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed80b5c70 0x7f1ed815bb60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8074b20 tx=0x7f1ec801c760 comp rx=0 tx=0).stop 2024-09-21T06:08:24.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.399+0000 7f1edd200640 1 -- 172.21.15.50:0/3127440070 shutdown_connections 2024-09-21T06:08:24.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.399+0000 7f1edd200640 1 --2- 172.21.15.50:0/3127440070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed80b5c70 0x7f1ed815bb60 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.399+0000 7f1edd200640 1 -- 172.21.15.50:0/3127440070 >> 172.21.15.50:0/3127440070 conn(0x7f1ed80b2040 msgr2=0x7f1ed80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:24.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.399+0000 7f1edd200640 1 -- 172.21.15.50:0/3127440070 shutdown_connections 2024-09-21T06:08:24.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.399+0000 7f1edd200640 1 -- 172.21.15.50:0/3127440070 wait complete. 2024-09-21T06:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.399+0000 7f1edd200640 1 Processor -- start 2024-09-21T06:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.399+0000 7f1edd200640 1 -- start start 2024-09-21T06:08:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1edd200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed80b4de0 0x7f1ed815d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1edd200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1ed80b5c70 0x7f1ed814f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1edd200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ed81527b0 0x7f1ed8151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1edd200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed8169b30 con 0x7f1ed80b4de0 2024-09-21T06:08:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1edd200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed81699b0 con 0x7f1ed81527b0 2024-09-21T06:08:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1edd200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed8169cb0 con 0x7f1ed80b5c70 2024-09-21T06:08:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ed5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed80b4de0 0x7f1ed815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ed5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed80b4de0 0x7f1ed815d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57718/0 (socket says 172.21.15.50:57718) 2024-09-21T06:08:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ed5e00640 1 -- 172.21.15.50:0/1440410496 learned_addr learned my addr 172.21.15.50:0/1440410496 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ed6800640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ed81527b0 0x7f1ed8151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ecfe00640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1ed80b5c70 0x7f1ed814f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ed5e00640 1 -- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1ed80b5c70 msgr2=0x7f1ed814f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ed5e00640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1ed80b5c70 0x7f1ed814f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ed5e00640 1 -- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ed81527b0 msgr2=0x7f1ed8151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ed5e00640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ed81527b0 0x7f1ed8151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ed5e00640 1 -- 172.21.15.50:0/1440410496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed80ae8e0 con 0x7f1ed80b4de0 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ed6800640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ed81527b0 0x7f1ed8151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ed5e00640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed80b4de0 0x7f1ed815d860 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1ec801cc40 tx=0x7f1ec80a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ece000640 1 -- 172.21.15.50:0/1440410496 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec80b2040 con 0x7f1ed80b4de0 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ece000640 1 -- 172.21.15.50:0/1440410496 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ec80042f0 con 0x7f1ed80b4de0 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1ece000640 1 -- 172.21.15.50:0/1440410496 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8003840 con 0x7f1ed80b4de0 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1edd200640 1 -- 172.21.15.50:0/1440410496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed815deb0 con 0x7f1ed80b4de0 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1edd200640 1 -- 172.21.15.50:0/1440410496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed80aee80 con 0x7f1ed80b4de0 2024-09-21T06:08:24.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.403+0000 7f1edd200640 1 -- 172.21.15.50:0/1440410496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed815bb60 con 0x7f1ed80b4de0 2024-09-21T06:08:24.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.407+0000 7f1ece000640 1 -- 172.21.15.50:0/1440410496 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f1ec80bd030 con 0x7f1ed80b4de0 2024-09-21T06:08:24.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.407+0000 7f1ece000640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1eb4074880 0x7f1eb4076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.407+0000 7f1ece000640 1 -- 172.21.15.50:0/1440410496 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5822+0+0 (secure 0 0 0) 0x7f1ec812cde0 con 0x7f1ed80b4de0 2024-09-21T06:08:24.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.407+0000 7f1ecfe00640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1eb4074880 0x7f1eb4076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:24.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.407+0000 7f1ecfe00640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1eb4074880 0x7f1eb4076d90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0000f70 tx=0x7f1ec0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:24.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.411+0000 7f1ece000640 1 -- 172.21.15.50:0/1440410496 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1ec80c4060 con 0x7f1ed80b4de0 2024-09-21T06:08:24.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.587+0000 7f1edd200640 1 -- 172.21.15.50:0/1440410496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ed814f6c0 con 0x7f1ed80b4de0 2024-09-21T06:08:24.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1ece000640 1 -- 172.21.15.50:0/1440410496 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16875 (secure 0 0 0) 0x7f1ec8016410 con 0x7f1ed80b4de0 2024-09-21T06:08:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:24.589 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":89,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:08:23.449797+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:06:03.633659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T06:06:04.852705+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":88,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.0","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:08:24.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1edd200640 1 -- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1eb4074880 msgr2=0x7f1eb4076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1edd200640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1eb4074880 0x7f1eb4076d90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0000f70 tx=0x7f1ec0009040 comp rx=0 tx=0).stop 2024-09-21T06:08:24.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1edd200640 1 -- 172.21.15.50:0/1440410496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed80b4de0 msgr2=0x7f1ed815d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1edd200640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed80b4de0 0x7f1ed815d860 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1ec801cc40 tx=0x7f1ec80a5780 comp rx=0 tx=0).stop 2024-09-21T06:08:24.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1edd200640 1 -- 172.21.15.50:0/1440410496 shutdown_connections 2024-09-21T06:08:24.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1edd200640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ed81527b0 0x7f1ed8151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1edd200640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1ed80b5c70 0x7f1ed814f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1edd200640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1eb4074880 0x7f1eb4076d90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1edd200640 1 --2- 172.21.15.50:0/1440410496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed80b4de0 0x7f1ed815d860 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1edd200640 1 -- 172.21.15.50:0/1440410496 >> 172.21.15.50:0/1440410496 conn(0x7f1ed80b2040 msgr2=0x7f1ed80b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:24.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1edd200640 1 -- 172.21.15.50:0/1440410496 shutdown_connections 2024-09-21T06:08:24.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.591+0000 7f1edd200640 1 -- 172.21.15.50:0/1440410496 wait complete. 2024-09-21T06:08:24.612 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:08:24.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.771+0000 7fd55b800640 1 -- 172.21.15.50:0/1551844973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5540cefa0 msgr2=0x7fd5540cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.771+0000 7fd55b800640 1 --2- 172.21.15.50:0/1551844973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5540cefa0 0x7fd5540cf3b0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fd5540b3040 tx=0x7fd53c01c8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.771+0000 7fd55b800640 1 -- 172.21.15.50:0/1551844973 shutdown_connections 2024-09-21T06:08:24.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.771+0000 7fd55b800640 1 --2- 172.21.15.50:0/1551844973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5540cefa0 0x7fd5540cf3b0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.771+0000 7fd55b800640 1 --2- 172.21.15.50:0/1551844973 >> v2:172.21.15.88:3300/0 conn(0x7fd5540749a0 0x7fd554074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.771+0000 7fd55b800640 1 -- 172.21.15.50:0/1551844973 >> 172.21.15.50:0/1551844973 conn(0x7fd5540b2040 msgr2=0x7fd5540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:24.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.771+0000 7fd55b800640 1 -- 172.21.15.50:0/1551844973 shutdown_connections 2024-09-21T06:08:24.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.771+0000 7fd55b800640 1 -- 172.21.15.50:0/1551844973 wait complete. 2024-09-21T06:08:24.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd55b800640 1 Processor -- start 2024-09-21T06:08:24.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd55b800640 1 -- start start 2024-09-21T06:08:24.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd55b800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd5540749a0 0x7fd5541fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd55b800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5540cefa0 0x7fd554159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd55b800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd554160330 0x7fd5540cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd55b800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd55416a060 con 0x7fd5540cefa0 2024-09-21T06:08:24.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd55b800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd554169ee0 con 0x7fd554160330 2024-09-21T06:08:24.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd55b800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd55416a1e0 con 0x7fd5540749a0 2024-09-21T06:08:24.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd54be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5540cefa0 0x7fd554159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:24.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd54be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5540cefa0 0x7fd554159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57730/0 (socket says 172.21.15.50:57730) 2024-09-21T06:08:24.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd54be00640 1 -- 172.21.15.50:0/283725473 learned_addr learned my addr 172.21.15.50:0/283725473 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:24.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd54be00640 1 -- 172.21.15.50:0/283725473 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd5540749a0 msgr2=0x7fd5541fcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd54be00640 1 --2- 172.21.15.50:0/283725473 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd5540749a0 0x7fd5541fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd54be00640 1 -- 172.21.15.50:0/283725473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd554160330 msgr2=0x7fd5540cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd54be00640 1 --2- 172.21.15.50:0/283725473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd554160330 0x7fd5540cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.775+0000 7fd54be00640 1 -- 172.21.15.50:0/283725473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55413be60 con 0x7fd5540cefa0 2024-09-21T06:08:24.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.779+0000 7fd54be00640 1 --2- 172.21.15.50:0/283725473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5540cefa0 0x7fd554159f90 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0b0040 tx=0x7fd53c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:24.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.779+0000 7fd551600640 1 -- 172.21.15.50:0/283725473 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0b2040 con 0x7fd5540cefa0 2024-09-21T06:08:24.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.779+0000 7fd551600640 1 -- 172.21.15.50:0/283725473 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd53c0bd070 con 0x7fd5540cefa0 2024-09-21T06:08:24.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.779+0000 7fd551600640 1 -- 172.21.15.50:0/283725473 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c003930 con 0x7fd5540cefa0 2024-09-21T06:08:24.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.779+0000 7fd55b800640 1 -- 172.21.15.50:0/283725473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5541f9240 con 0x7fd5540cefa0 2024-09-21T06:08:24.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.779+0000 7fd55b800640 1 -- 172.21.15.50:0/283725473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5541f9800 con 0x7fd5540cefa0 2024-09-21T06:08:24.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.779+0000 7fd55b800640 1 -- 172.21.15.50:0/283725473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd554074db0 con 0x7fd5540cefa0 2024-09-21T06:08:24.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.779+0000 7fd551600640 1 -- 172.21.15.50:0/283725473 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fd53c003ad0 con 0x7fd5540cefa0 2024-09-21T06:08:24.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.779+0000 7fd551600640 1 --2- 172.21.15.50:0/283725473 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd5300748b0 0x7fd530076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.779+0000 7fd551600640 1 -- 172.21.15.50:0/283725473 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5822+0+0 (secure 0 0 0) 0x7fd53c12dd90 con 0x7fd5540cefa0 2024-09-21T06:08:24.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.783+0000 7fd558e00640 1 --2- 172.21.15.50:0/283725473 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd5300748b0 0x7fd530076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:24.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.783+0000 7fd558e00640 1 --2- 172.21.15.50:0/283725473 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd5300748b0 0x7fd530076dc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd5440058d0 tx=0x7fd544005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:24.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.787+0000 7fd551600640 1 -- 172.21.15.50:0/283725473 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd53c0164b0 con 0x7fd5540cefa0 2024-09-21T06:08:24.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.999+0000 7fd55b800640 1 -- 172.21.15.50:0/283725473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd5540cc990 con 0x7fd5540cefa0 2024-09-21T06:08:24.998 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-09-21T06:08:24.999+0000 7f0d45000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:08:25.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:24.999+0000 7fd551600640 1 -- 172.21.15.50:0/283725473 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) ==== 161+0+0 (secure 0 0 0) 0x7fd53c0fa8a0 con 0x7fd5540cefa0 2024-09-21T06:08:25.000 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:08:25.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.003+0000 7fd55b800640 1 -- 172.21.15.50:0/283725473 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd5300748b0 msgr2=0x7fd530076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:25.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.003+0000 7fd55b800640 1 --2- 172.21.15.50:0/283725473 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd5300748b0 0x7fd530076dc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd5440058d0 tx=0x7fd544005860 comp rx=0 tx=0).stop 2024-09-21T06:08:25.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.003+0000 7fd55b800640 1 -- 172.21.15.50:0/283725473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5540cefa0 msgr2=0x7fd554159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:25.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.003+0000 7fd55b800640 1 --2- 172.21.15.50:0/283725473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5540cefa0 0x7fd554159f90 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0b0040 tx=0x7fd53c005760 comp rx=0 tx=0).stop 2024-09-21T06:08:25.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.003+0000 7fd55b800640 1 -- 172.21.15.50:0/283725473 shutdown_connections 2024-09-21T06:08:25.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.003+0000 7fd55b800640 1 --2- 172.21.15.50:0/283725473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd554160330 0x7fd5540cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:25.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.003+0000 7fd55b800640 1 --2- 172.21.15.50:0/283725473 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd5300748b0 0x7fd530076dc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:25.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.003+0000 7fd55b800640 1 --2- 172.21.15.50:0/283725473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5540cefa0 0x7fd554159f90 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:25.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.003+0000 7fd55b800640 1 --2- 172.21.15.50:0/283725473 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd5540749a0 0x7fd5541fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:25.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.003+0000 7fd55b800640 1 -- 172.21.15.50:0/283725473 >> 172.21.15.50:0/283725473 conn(0x7fd5540b2040 msgr2=0x7fd55415b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:25.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.003+0000 7fd55b800640 1 -- 172.21.15.50:0/283725473 shutdown_connections 2024-09-21T06:08:25.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.007+0000 7fd55b800640 1 -- 172.21.15.50:0/283725473 wait complete. 2024-09-21T06:08:25.026 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:08:25.143 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T06:08:25.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.191+0000 7fb594e00640 1 -- 172.21.15.50:0/9713648 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb5900b54f0 msgr2=0x7fb590155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:25.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.191+0000 7fb594e00640 1 --2- 172.21.15.50:0/9713648 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb5900b54f0 0x7fb590155b50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb5780098a0 tx=0x7fb57801c6d0 comp rx=0 tx=0).stop 2024-09-21T06:08:25.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.191+0000 7fb594e00640 1 -- 172.21.15.50:0/9713648 shutdown_connections 2024-09-21T06:08:25.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.191+0000 7fb594e00640 1 --2- 172.21.15.50:0/9713648 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb5900b54f0 0x7fb590155b50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:25.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.191+0000 7fb594e00640 1 -- 172.21.15.50:0/9713648 >> 172.21.15.50:0/9713648 conn(0x7fb5900b2040 msgr2=0x7fb5900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:25.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.191+0000 7fb594e00640 1 -- 172.21.15.50:0/9713648 shutdown_connections 2024-09-21T06:08:25.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.191+0000 7fb594e00640 1 -- 172.21.15.50:0/9713648 wait complete. 2024-09-21T06:08:25.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb594e00640 1 Processor -- start 2024-09-21T06:08:25.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb594e00640 1 -- start start 2024-09-21T06:08:25.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb594e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb590072100 0x7fb5901444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:25.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb594e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb590073a90 0x7fb5900ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:25.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb594e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb590144b40 0x7fb5900aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:25.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb594e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb590169ef0 con 0x7fb590144b40 2024-09-21T06:08:25.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb594e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb590169d70 con 0x7fb590072100 2024-09-21T06:08:25.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb594e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb59016a070 con 0x7fb590073a90 2024-09-21T06:08:25.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb58e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb590144b40 0x7fb5900aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:25.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb58e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb590144b40 0x7fb5900aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57744/0 (socket says 172.21.15.50:57744) 2024-09-21T06:08:25.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb58e800640 1 -- 172.21.15.50:0/2043143688 learned_addr learned my addr 172.21.15.50:0/2043143688 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:25.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb58de00640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb590072100 0x7fb5901444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:25.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb587e00640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb590073a90 0x7fb5900ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:25.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb58e800640 1 -- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb590073a90 msgr2=0x7fb5900ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:25.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb58e800640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb590073a90 0x7fb5900ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:25.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb58e800640 1 -- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb590072100 msgr2=0x7fb5901444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:25.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb58e800640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb590072100 0x7fb5901444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:25.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb58e800640 1 -- 172.21.15.50:0/2043143688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5900b0460 con 0x7fb590144b40 2024-09-21T06:08:25.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb58e800640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb590144b40 0x7fb5900aee20 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb588001f70 tx=0x7fb588002060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:25.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.195+0000 7fb586000640 1 -- 172.21.15.50:0/2043143688 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588016020 con 0x7fb590144b40 2024-09-21T06:08:25.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.199+0000 7fb586000640 1 -- 172.21.15.50:0/2043143688 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb58800ac20 con 0x7fb590144b40 2024-09-21T06:08:25.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.199+0000 7fb594e00640 1 -- 172.21.15.50:0/2043143688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5901571c0 con 0x7fb590144b40 2024-09-21T06:08:25.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.199+0000 7fb586000640 1 -- 172.21.15.50:0/2043143688 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588012680 con 0x7fb590144b40 2024-09-21T06:08:25.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.199+0000 7fb594e00640 1 -- 172.21.15.50:0/2043143688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5901574a0 con 0x7fb590144b40 2024-09-21T06:08:25.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.199+0000 7fb594e00640 1 -- 172.21.15.50:0/2043143688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb590155b50 con 0x7fb590144b40 2024-09-21T06:08:25.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.199+0000 7fb586000640 1 -- 172.21.15.50:0/2043143688 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fb588019020 con 0x7fb590144b40 2024-09-21T06:08:25.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.199+0000 7fb586000640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb56c074880 0x7fb56c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:25.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.199+0000 7fb586000640 1 -- 172.21.15.50:0/2043143688 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5880+0+0 (secure 0 0 0) 0x7fb588093320 con 0x7fb590144b40 2024-09-21T06:08:25.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.199+0000 7fb58de00640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb56c074880 0x7fb56c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:25.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.203+0000 7fb58de00640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb56c074880 0x7fb56c076d90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb580007fd0 tx=0x7fb580009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:25.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.203+0000 7fb586000640 1 -- 172.21.15.50:0/2043143688 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb588020020 con 0x7fb590144b40 2024-09-21T06:08:25.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:25.415+0000 7fb594e00640 1 -- 172.21.15.50:0/2043143688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb590156f10 con 0x7fb590144b40 2024-09-21T06:08:26.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.007+0000 7fb586000640 1 -- 172.21.15.50:0/2043143688 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) ==== 104+0+0 (secure 0 0 0) 0x7fb58805f7f0 con 0x7fb590144b40 2024-09-21T06:08:26.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.011+0000 7fb594e00640 1 -- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb56c074880 msgr2=0x7fb56c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.011+0000 7fb594e00640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb56c074880 0x7fb56c076d90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb580007fd0 tx=0x7fb580009040 comp rx=0 tx=0).stop 2024-09-21T06:08:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.011+0000 7fb594e00640 1 -- 172.21.15.50:0/2043143688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb590144b40 msgr2=0x7fb5900aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.011+0000 7fb594e00640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb590144b40 0x7fb5900aee20 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb588001f70 tx=0x7fb588002060 comp rx=0 tx=0).stop 2024-09-21T06:08:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.011+0000 7fb594e00640 1 -- 172.21.15.50:0/2043143688 shutdown_connections 2024-09-21T06:08:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.011+0000 7fb594e00640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb56c074880 0x7fb56c076d90 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.011+0000 7fb594e00640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb590144b40 0x7fb5900aee20 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.011+0000 7fb594e00640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb590073a90 0x7fb5900ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.011+0000 7fb594e00640 1 --2- 172.21.15.50:0/2043143688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb590072100 0x7fb5901444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:26.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.011+0000 7fb594e00640 1 -- 172.21.15.50:0/2043143688 >> 172.21.15.50:0/2043143688 conn(0x7fb5900b2040 msgr2=0x7fb590073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:26.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.011+0000 7fb594e00640 1 -- 172.21.15.50:0/2043143688 shutdown_connections 2024-09-21T06:08:26.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.011+0000 7fb594e00640 1 -- 172.21.15.50:0/2043143688 wait complete. 2024-09-21T06:08:26.034 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:08:26.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.207+0000 7fc7a5c00640 1 -- 172.21.15.50:0/1561506449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7a00b5c70 msgr2=0x7fc7a015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:26.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.207+0000 7fc7a5c00640 1 --2- 172.21.15.50:0/1561506449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7a00b5c70 0x7fc7a015bb60 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0070710 tx=0x7fc79001c800 comp rx=0 tx=0).stop 2024-09-21T06:08:26.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.207+0000 7fc7a5c00640 1 -- 172.21.15.50:0/1561506449 shutdown_connections 2024-09-21T06:08:26.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.207+0000 7fc7a5c00640 1 --2- 172.21.15.50:0/1561506449 >> v2:172.21.15.88:3300/0 conn(0x7fc7a01527b0 0x7fc7a0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:26.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.207+0000 7fc7a5c00640 1 --2- 172.21.15.50:0/1561506449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7a00b5c70 0x7fc7a015bb60 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:26.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.211+0000 7fc7a5c00640 1 -- 172.21.15.50:0/1561506449 >> 172.21.15.50:0/1561506449 conn(0x7fc7a00b2040 msgr2=0x7fc7a00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:26.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.211+0000 7fc7a5c00640 1 -- 172.21.15.50:0/1561506449 shutdown_connections 2024-09-21T06:08:26.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.211+0000 7fc7a5c00640 1 -- 172.21.15.50:0/1561506449 wait complete. 2024-09-21T06:08:26.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.211+0000 7fc7a5c00640 1 Processor -- start 2024-09-21T06:08:26.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.211+0000 7fc7a5c00640 1 -- start start 2024-09-21T06:08:26.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.211+0000 7fc7a5c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7a00b4de0 0x7fc7a013fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:26.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.211+0000 7fc7a5c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc7a00b5c70 0x7fc7a0141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.211+0000 7fc7a5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a01527b0 0x7fc7a01f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.211+0000 7fc7a5c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a0168b20 con 0x7fc7a00b4de0 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.211+0000 7fc7a5c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a01689a0 con 0x7fc7a01527b0 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.211+0000 7fc7a5c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a0168ca0 con 0x7fc7a00b5c70 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc79ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7a00b4de0 0x7fc7a013fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc79ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7a00b4de0 0x7fc7a013fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57762/0 (socket says 172.21.15.50:57762) 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc79ea00640 1 -- 172.21.15.50:0/4236202024 learned_addr learned my addr 172.21.15.50:0/4236202024 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc79f400640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a01527b0 0x7fc7a01f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc797e00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc7a00b5c70 0x7fc7a0141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc79ea00640 1 -- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc7a00b5c70 msgr2=0x7fc7a0141860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc79ea00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc7a00b5c70 0x7fc7a0141860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc79ea00640 1 -- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a01527b0 msgr2=0x7fc7a01f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc79ea00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a01527b0 0x7fc7a01f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:26.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc79ea00640 1 -- 172.21.15.50:0/4236202024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a013eb50 con 0x7fc7a00b4de0 2024-09-21T06:08:26.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc797e00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc7a00b5c70 0x7fc7a0141860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:26.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc79f400640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a01527b0 0x7fc7a01f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:26.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc79ea00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7a00b4de0 0x7fc7a013fbf0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fc7900b0040 tx=0x7fc7900a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:26.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc796a00640 1 -- 172.21.15.50:0/4236202024 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7900b2040 con 0x7fc7a00b4de0 2024-09-21T06:08:26.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc796a00640 1 -- 172.21.15.50:0/4236202024 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc790004430 con 0x7fc7a00b4de0 2024-09-21T06:08:26.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc796a00640 1 -- 172.21.15.50:0/4236202024 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790005280 con 0x7fc7a00b4de0 2024-09-21T06:08:26.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc7a5c00640 1 -- 172.21.15.50:0/4236202024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a013ee30 con 0x7fc7a00b4de0 2024-09-21T06:08:26.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc7a5c00640 1 -- 172.21.15.50:0/4236202024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a013f3f0 con 0x7fc7a00b4de0 2024-09-21T06:08:26.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc7a5c00640 1 -- 172.21.15.50:0/4236202024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a015bb60 con 0x7fc7a00b4de0 2024-09-21T06:08:26.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.215+0000 7fc796a00640 1 -- 172.21.15.50:0/4236202024 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fc790005420 con 0x7fc7a00b4de0 2024-09-21T06:08:26.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.219+0000 7fc796a00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc77c074880 0x7fc77c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:26.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.219+0000 7fc796a00640 1 -- 172.21.15.50:0/4236202024 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5880+0+0 (secure 0 0 0) 0x7fc79012cba0 con 0x7fc7a00b4de0 2024-09-21T06:08:26.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.219+0000 7fc797e00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc77c074880 0x7fc77c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:26.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.219+0000 7fc797e00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc77c074880 0x7fc77c076d90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00ade70 tx=0x7fc7880091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:26.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.223+0000 7fc796a00640 1 -- 172.21.15.50:0/4236202024 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc790136020 con 0x7fc7a00b4de0 2024-09-21T06:08:26.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:26.399+0000 7fc7a5c00640 1 -- 172.21.15.50:0/4236202024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc7a00b0cd0 con 0x7fc7a00b4de0 2024-09-21T06:08:27.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.007+0000 7fc796a00640 1 -- 172.21.15.50:0/4236202024 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7fc7a0075580 con 0x7fc7a00b4de0 2024-09-21T06:08:27.008 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:08:27.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.011+0000 7fc7a5c00640 1 -- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc77c074880 msgr2=0x7fc77c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:27.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.011+0000 7fc7a5c00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc77c074880 0x7fc77c076d90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00ade70 tx=0x7fc7880091e0 comp rx=0 tx=0).stop 2024-09-21T06:08:27.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.011+0000 7fc7a5c00640 1 -- 172.21.15.50:0/4236202024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7a00b4de0 msgr2=0x7fc7a013fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:27.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.011+0000 7fc7a5c00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7a00b4de0 0x7fc7a013fbf0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fc7900b0040 tx=0x7fc7900a57f0 comp rx=0 tx=0).stop 2024-09-21T06:08:27.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.011+0000 7fc7a5c00640 1 -- 172.21.15.50:0/4236202024 shutdown_connections 2024-09-21T06:08:27.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.011+0000 7fc7a5c00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a01527b0 0x7fc7a01f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:27.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.011+0000 7fc7a5c00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc7a00b5c70 0x7fc7a0141860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:27.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.011+0000 7fc7a5c00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc77c074880 0x7fc77c076d90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:27.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.015+0000 7fc7a5c00640 1 --2- 172.21.15.50:0/4236202024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7a00b4de0 0x7fc7a013fbf0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:27.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.015+0000 7fc7a5c00640 1 -- 172.21.15.50:0/4236202024 >> 172.21.15.50:0/4236202024 conn(0x7fc7a00b2040 msgr2=0x7fc7a00cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:27.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.015+0000 7fc7a5c00640 1 -- 172.21.15.50:0/4236202024 shutdown_connections 2024-09-21T06:08:27.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.015+0000 7fc7a5c00640 1 -- 172.21.15.50:0/4236202024 wait complete. 2024-09-21T06:08:27.039 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:08:27.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.211+0000 7f373ca00640 1 -- 172.21.15.50:0/1390477817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f371c004810 msgr2=0x7f371c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:27.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.211+0000 7f373ca00640 1 --2- 172.21.15.50:0/1390477817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f371c004810 0x7f371c004c20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f3720009b40 tx=0x7f372001d270 comp rx=0 tx=0).stop 2024-09-21T06:08:27.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.211+0000 7f3737200640 1 -- 172.21.15.50:0/1390477817 reap_dead start 2024-09-21T06:08:27.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.211+0000 7f373ca00640 1 -- 172.21.15.50:0/1390477817 shutdown_connections 2024-09-21T06:08:27.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.211+0000 7f373ca00640 1 -- 172.21.15.50:0/1390477817 >> 172.21.15.50:0/1390477817 conn(0x7f373806cdf0 msgr2=0x7f373806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:27.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.211+0000 7f373ca00640 1 -- 172.21.15.50:0/1390477817 shutdown_connections 2024-09-21T06:08:27.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.211+0000 7f373ca00640 1 -- 172.21.15.50:0/1390477817 wait complete. 2024-09-21T06:08:27.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.211+0000 7f373ca00640 1 Processor -- start 2024-09-21T06:08:27.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.211+0000 7f373ca00640 1 -- start start 2024-09-21T06:08:27.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f373ca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f371c004810 0x7f373814d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:27.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f373ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3738150220 0x7f37380aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:27.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f373ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37380aeff0 0x7f37381ef240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:27.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f373ca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f37380b3dd0 con 0x7f37380aeff0 2024-09-21T06:08:27.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f373ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f37380b3c50 con 0x7f3738150220 2024-09-21T06:08:27.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f373ca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f37380b21d0 con 0x7f371c004810 2024-09-21T06:08:27.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f3737200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37380aeff0 0x7f37381ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:27.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f3737200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37380aeff0 0x7f37381ef240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37952/0 (socket says 172.21.15.50:37952) 2024-09-21T06:08:27.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f3737200640 1 -- 172.21.15.50:0/2879468352 learned_addr learned my addr 172.21.15.50:0/2879468352 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:27.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f3736800640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f371c004810 0x7f373814d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:27.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f372fe00640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3738150220 0x7f37380aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:27.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f3737200640 1 -- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f371c004810 msgr2=0x7f373814d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:27.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f3737200640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f371c004810 0x7f373814d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:27.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f3737200640 1 -- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3738150220 msgr2=0x7f37380aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:27.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f3737200640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3738150220 0x7f37380aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:27.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f3737200640 1 -- 172.21.15.50:0/2879468352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373814de50 con 0x7f37380aeff0 2024-09-21T06:08:27.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f372fe00640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3738150220 0x7f37380aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:27.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f3737200640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37380aeff0 0x7f37381ef240 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f37380744d0 tx=0x7f3730006710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:27.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f372ea00640 1 -- 172.21.15.50:0/2879468352 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373000e890 con 0x7f37380aeff0 2024-09-21T06:08:27.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f372ea00640 1 -- 172.21.15.50:0/2879468352 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f373000eb40 con 0x7f37380aeff0 2024-09-21T06:08:27.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f373ca00640 1 -- 172.21.15.50:0/2879468352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373814e130 con 0x7f37380aeff0 2024-09-21T06:08:27.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f372ea00640 1 -- 172.21.15.50:0/2879468352 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730011460 con 0x7f37380aeff0 2024-09-21T06:08:27.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f373ca00640 1 -- 172.21.15.50:0/2879468352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f373814e6f0 con 0x7f37380aeff0 2024-09-21T06:08:27.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.215+0000 7f373ca00640 1 -- 172.21.15.50:0/2879468352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f373814d860 con 0x7f37380aeff0 2024-09-21T06:08:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.219+0000 7f372ea00640 1 -- 172.21.15.50:0/2879468352 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f3730019020 con 0x7f37380aeff0 2024-09-21T06:08:27.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.219+0000 7f372ea00640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3714074880 0x7f3714076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:27.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.219+0000 7f372ea00640 1 -- 172.21.15.50:0/2879468352 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5442+0+0 (secure 0 0 0) 0x7f3730093220 con 0x7f37380aeff0 2024-09-21T06:08:27.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.219+0000 7f3736800640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3714074880 0x7f3714076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:27.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.219+0000 7f3736800640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3714074880 0x7f3714076d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3728006fd0 tx=0x7f3728006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.223+0000 7f372ea00640 1 -- 172.21.15.50:0/2879468352 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f37380b2710 con 0x7f37380aeff0 2024-09-21T06:08:27.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:27.399+0000 7f373ca00640 1 -- 172.21.15.50:0/2879468352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f37380b3540 con 0x7f37380aeff0 2024-09-21T06:08:28.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.019+0000 7f372ea00640 1 -- 172.21.15.50:0/2879468352 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7f373005fd20 con 0x7f37380aeff0 2024-09-21T06:08:28.017 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-09-21T06:08:28.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.023+0000 7f373ca00640 1 -- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3714074880 msgr2=0x7f3714076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:28.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.023+0000 7f373ca00640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3714074880 0x7f3714076d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3728006fd0 tx=0x7f3728006b20 comp rx=0 tx=0).stop 2024-09-21T06:08:28.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.023+0000 7f373ca00640 1 -- 172.21.15.50:0/2879468352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37380aeff0 msgr2=0x7f37381ef240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:28.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.023+0000 7f373ca00640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37380aeff0 0x7f37381ef240 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f37380744d0 tx=0x7f3730006710 comp rx=0 tx=0).stop 2024-09-21T06:08:28.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.023+0000 7f373ca00640 1 -- 172.21.15.50:0/2879468352 shutdown_connections 2024-09-21T06:08:28.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.023+0000 7f373ca00640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3714074880 0x7f3714076d90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.023+0000 7f373ca00640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37380aeff0 0x7f37381ef240 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.023+0000 7f373ca00640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3738150220 0x7f37380aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.023+0000 7f373ca00640 1 --2- 172.21.15.50:0/2879468352 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f371c004810 0x7f373814d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.023+0000 7f373ca00640 1 -- 172.21.15.50:0/2879468352 >> 172.21.15.50:0/2879468352 conn(0x7f373806cdf0 msgr2=0x7f37380b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:28.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.023+0000 7f373ca00640 1 -- 172.21.15.50:0/2879468352 shutdown_connections 2024-09-21T06:08:28.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.023+0000 7f373ca00640 1 -- 172.21.15.50:0/2879468352 wait complete. 2024-09-21T06:08:28.053 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:08:28.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.215+0000 7ffb2f400640 1 -- 172.21.15.50:0/3131133280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb281527b0 msgr2=0x7ffb2815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:28.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.215+0000 7ffb2f400640 1 --2- 172.21.15.50:0/3131133280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb281527b0 0x7ffb2815bb60 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009900 tx=0x7ffb1001c8b0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.215+0000 7ffb2f400640 1 -- 172.21.15.50:0/3131133280 shutdown_connections 2024-09-21T06:08:28.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.215+0000 7ffb2f400640 1 --2- 172.21.15.50:0/3131133280 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb280b4de0 0x7ffb280b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.215+0000 7ffb2f400640 1 --2- 172.21.15.50:0/3131133280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb281527b0 0x7ffb2815bb60 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.215+0000 7ffb2f400640 1 -- 172.21.15.50:0/3131133280 >> 172.21.15.50:0/3131133280 conn(0x7ffb280b2040 msgr2=0x7ffb280b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:28.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.215+0000 7ffb2f400640 1 -- 172.21.15.50:0/3131133280 shutdown_connections 2024-09-21T06:08:28.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.215+0000 7ffb2f400640 1 -- 172.21.15.50:0/3131133280 wait complete. 2024-09-21T06:08:28.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2f400640 1 Processor -- start 2024-09-21T06:08:28.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2f400640 1 -- start start 2024-09-21T06:08:28.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb280b4de0 0x7ffb28150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:28.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb280b5c70 0x7ffb2814e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:28.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb281527b0 0x7ffb28148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:28.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2f400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb28168950 con 0x7ffb280b4de0 2024-09-21T06:08:28.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2f400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb281687d0 con 0x7ffb281527b0 2024-09-21T06:08:28.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2f400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ffb28168ad0 con 0x7ffb280b5c70 2024-09-21T06:08:28.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb280b4de0 0x7ffb28150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:28.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb280b4de0 0x7ffb28150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37980/0 (socket says 172.21.15.50:37980) 2024-09-21T06:08:28.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2ca00640 1 -- 172.21.15.50:0/2869809650 learned_addr learned my addr 172.21.15.50:0/2869809650 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:28.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb1fe00640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb280b5c70 0x7ffb2814e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:28.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2d400640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb281527b0 0x7ffb28148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:28.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2ca00640 1 -- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb280b5c70 msgr2=0x7ffb2814e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:28.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2ca00640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb280b5c70 0x7ffb2814e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2ca00640 1 -- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb281527b0 msgr2=0x7ffb28148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:28.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2ca00640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb281527b0 0x7ffb28148970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2ca00640 1 -- 172.21.15.50:0/2869809650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2814ce50 con 0x7ffb280b4de0 2024-09-21T06:08:28.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb2ca00640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb280b4de0 0x7ffb28150960 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800c3f0 tx=0x7ffb1800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:28.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.219+0000 7ffb25600640 1 -- 172.21.15.50:0/2869809650 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb18017070 con 0x7ffb280b4de0 2024-09-21T06:08:28.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.223+0000 7ffb25600640 1 -- 172.21.15.50:0/2869809650 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb180053c0 con 0x7ffb280b4de0 2024-09-21T06:08:28.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.223+0000 7ffb2f400640 1 -- 172.21.15.50:0/2869809650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2814d130 con 0x7ffb280b4de0 2024-09-21T06:08:28.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.223+0000 7ffb25600640 1 -- 172.21.15.50:0/2869809650 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb18003860 con 0x7ffb280b4de0 2024-09-21T06:08:28.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.223+0000 7ffb2f400640 1 -- 172.21.15.50:0/2869809650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2814d6f0 con 0x7ffb280b4de0 2024-09-21T06:08:28.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.223+0000 7ffb2f400640 1 -- 172.21.15.50:0/2869809650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb280b51f0 con 0x7ffb280b4de0 2024-09-21T06:08:28.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.223+0000 7ffb25600640 1 -- 172.21.15.50:0/2869809650 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7ffb1801a070 con 0x7ffb280b4de0 2024-09-21T06:08:28.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.223+0000 7ffb25600640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ffb040748b0 0x7ffb04076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:28.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.223+0000 7ffb25600640 1 -- 172.21.15.50:0/2869809650 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7ffb18092180 con 0x7ffb280b4de0 2024-09-21T06:08:28.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.227+0000 7ffb1fe00640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ffb040748b0 0x7ffb04076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:28.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.227+0000 7ffb1fe00640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ffb040748b0 0x7ffb04076dc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ffb100af000 tx=0x7ffb1001b0f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:28.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.231+0000 7ffb25600640 1 -- 172.21.15.50:0/2869809650 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ffb280b5730 con 0x7ffb280b4de0 2024-09-21T06:08:28.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.407+0000 7ffb2f400640 1 -- 172.21.15.50:0/2869809650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7ffb281594d0 con 0x7ffb280b4de0 2024-09-21T06:08:28.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.411+0000 7ffb25600640 1 -- 172.21.15.50:0/2869809650 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v48) ==== 71+0+0 (secure 0 0 0) 0x7ffb1805e640 con 0x7ffb280b4de0 2024-09-21T06:08:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.415+0000 7ffb2f400640 1 -- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ffb040748b0 msgr2=0x7ffb04076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.415+0000 7ffb2f400640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ffb040748b0 0x7ffb04076dc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ffb100af000 tx=0x7ffb1001b0f0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.415+0000 7ffb2f400640 1 -- 172.21.15.50:0/2869809650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb280b4de0 msgr2=0x7ffb28150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.415+0000 7ffb2f400640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb280b4de0 0x7ffb28150960 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800c3f0 tx=0x7ffb1800c8b0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.415+0000 7ffb2f400640 1 -- 172.21.15.50:0/2869809650 shutdown_connections 2024-09-21T06:08:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.415+0000 7ffb2f400640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb281527b0 0x7ffb28148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.415+0000 7ffb2f400640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ffb280b5c70 0x7ffb2814e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.415+0000 7ffb2f400640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ffb040748b0 0x7ffb04076dc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.415+0000 7ffb2f400640 1 --2- 172.21.15.50:0/2869809650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb280b4de0 0x7ffb28150960 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.415+0000 7ffb2f400640 1 -- 172.21.15.50:0/2869809650 >> 172.21.15.50:0/2869809650 conn(0x7ffb280b2040 msgr2=0x7ffb280b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.415+0000 7ffb2f400640 1 -- 172.21.15.50:0/2869809650 shutdown_connections 2024-09-21T06:08:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.415+0000 7ffb2f400640 1 -- 172.21.15.50:0/2869809650 wait complete. 2024-09-21T06:08:28.435 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-21T06:08:28.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.595+0000 7f8452600640 1 -- 172.21.15.50:0/3911845020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c0b5c70 msgr2=0x7f844c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:28.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.595+0000 7f8452600640 1 --2- 172.21.15.50:0/3911845020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c0b5c70 0x7f844c15bb60 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f843c009900 tx=0x7f843c01c8b0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.595+0000 7f8452600640 1 -- 172.21.15.50:0/3911845020 shutdown_connections 2024-09-21T06:08:28.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.595+0000 7f8452600640 1 --2- 172.21.15.50:0/3911845020 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f844c0b4de0 0x7f844c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.595+0000 7f8452600640 1 --2- 172.21.15.50:0/3911845020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c0b5c70 0x7f844c15bb60 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.595+0000 7f8452600640 1 -- 172.21.15.50:0/3911845020 >> 172.21.15.50:0/3911845020 conn(0x7f844c0b2040 msgr2=0x7f844c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:28.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.595+0000 7f8452600640 1 -- 172.21.15.50:0/3911845020 shutdown_connections 2024-09-21T06:08:28.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.595+0000 7f8452600640 1 -- 172.21.15.50:0/3911845020 wait complete. 2024-09-21T06:08:28.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8452600640 1 Processor -- start 2024-09-21T06:08:28.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8452600640 1 -- start start 2024-09-21T06:08:28.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8452600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f844c0b4de0 0x7f844c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:28.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8452600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f844c0b5c70 0x7f844c1fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8452600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c1527b0 0x7f844c157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8452600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f844c168a20 con 0x7f844c1527b0 2024-09-21T06:08:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8452600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1688a0 con 0x7f844c0b4de0 2024-09-21T06:08:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8452600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f844c168ba0 con 0x7f844c0b5c70 2024-09-21T06:08:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f844be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c1527b0 0x7f844c157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f844be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c1527b0 0x7f844c157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37998/0 (socket says 172.21.15.50:37998) 2024-09-21T06:08:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f844be00640 1 -- 172.21.15.50:0/3108096532 learned_addr learned my addr 172.21.15.50:0/3108096532 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f844b400640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f844c0b4de0 0x7f844c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8443e00640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f844c0b5c70 0x7f844c1fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f844be00640 1 -- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f844c0b5c70 msgr2=0x7f844c1fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f844be00640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f844c0b5c70 0x7f844c1fc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f844be00640 1 -- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f844c0b4de0 msgr2=0x7f844c152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f844be00640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f844c0b4de0 0x7f844c152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f844be00640 1 -- 172.21.15.50:0/3108096532 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c158740 con 0x7f844c1527b0 2024-09-21T06:08:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f844be00640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c1527b0 0x7f844c157c10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f844400e970 tx=0x7f844400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8443400640 1 -- 172.21.15.50:0/3108096532 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444004d30 con 0x7f844c1527b0 2024-09-21T06:08:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8443400640 1 -- 172.21.15.50:0/3108096532 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f844400cac0 con 0x7f844c1527b0 2024-09-21T06:08:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8452600640 1 -- 172.21.15.50:0/3108096532 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844c158a20 con 0x7f844c1527b0 2024-09-21T06:08:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8443400640 1 -- 172.21.15.50:0/3108096532 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444005020 con 0x7f844c1527b0 2024-09-21T06:08:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.599+0000 7f8452600640 1 -- 172.21.15.50:0/3108096532 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844c1fce50 con 0x7f844c1527b0 2024-09-21T06:08:28.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.603+0000 7f8452600640 1 -- 172.21.15.50:0/3108096532 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f844c0b51f0 con 0x7f844c1527b0 2024-09-21T06:08:28.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.603+0000 7f8443400640 1 -- 172.21.15.50:0/3108096532 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f84440051c0 con 0x7f844c1527b0 2024-09-21T06:08:28.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.603+0000 7f8443400640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8428074880 0x7f8428076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:28.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.603+0000 7f8443400640 1 -- 172.21.15.50:0/3108096532 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7f8444093bc0 con 0x7f844c1527b0 2024-09-21T06:08:28.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.603+0000 7f844b400640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8428074880 0x7f8428076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:28.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.607+0000 7f844b400640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8428074880 0x7f8428076d90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f843c0af000 tx=0x7f843c01b0f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:28.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.607+0000 7f8443400640 1 -- 172.21.15.50:0/3108096532 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8444016030 con 0x7f844c1527b0 2024-09-21T06:08:28.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:28.855+0000 7f8452600640 1 -- 172.21.15.50:0/3108096532 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f844c0af820 con 0x7f844c1527b0 2024-09-21T06:08:29.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.019+0000 7f8443400640 1 -- 172.21.15.50:0/3108096532 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v534) ==== 174+0+0 (secure 0 0 0) 0x7f8444067e20 con 0x7f844c1527b0 2024-09-21T06:08:29.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.023+0000 7f8452600640 1 -- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8428074880 msgr2=0x7f8428076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.023+0000 7f8452600640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8428074880 0x7f8428076d90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f843c0af000 tx=0x7f843c01b0f0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.023+0000 7f8452600640 1 -- 172.21.15.50:0/3108096532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c1527b0 msgr2=0x7f844c157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.023+0000 7f8452600640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c1527b0 0x7f844c157c10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f844400e970 tx=0x7f844400ee30 comp rx=0 tx=0).stop 2024-09-21T06:08:29.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.023+0000 7f8452600640 1 -- 172.21.15.50:0/3108096532 shutdown_connections 2024-09-21T06:08:29.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.023+0000 7f8452600640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8428074880 0x7f8428076d90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.023+0000 7f8452600640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c1527b0 0x7f844c157c10 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.023+0000 7f8452600640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f844c0b5c70 0x7f844c1fc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.023+0000 7f8452600640 1 --2- 172.21.15.50:0/3108096532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f844c0b4de0 0x7f844c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.023+0000 7f8452600640 1 -- 172.21.15.50:0/3108096532 >> 172.21.15.50:0/3108096532 conn(0x7f844c0b2040 msgr2=0x7f844c159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:29.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.023+0000 7f8452600640 1 -- 172.21.15.50:0/3108096532 shutdown_connections 2024-09-21T06:08:29.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.023+0000 7f8452600640 1 -- 172.21.15.50:0/3108096532 wait complete. 2024-09-21T06:08:29.042 INFO:tasks.cephfs_test_runner:Test that progress indicated by progress bar for ongoing+pending clones ... ok 2024-09-21T06:08:29.042 INFO:tasks.cephfs_test_runner:Starting test: test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the command "ceph fs clone status" prints progress stats 2024-09-21T06:08:29.042 INFO:tasks.cephfs_test_runner:test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-21T06:08:29.043 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:08:29.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.203+0000 7f86f7200640 1 -- 172.21.15.50:0/3776490114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d4004810 msgr2=0x7f86d4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.203+0000 7f86f7200640 1 --2- 172.21.15.50:0/3776490114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d4004810 0x7f86d4004c20 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f86f00724a0 tx=0x7f86d801d020 comp rx=0 tx=0).stop 2024-09-21T06:08:29.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.203+0000 7f86f5200640 1 -- 172.21.15.50:0/3776490114 reap_dead start 2024-09-21T06:08:29.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.203+0000 7f86f7200640 1 -- 172.21.15.50:0/3776490114 shutdown_connections 2024-09-21T06:08:29.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.203+0000 7f86f7200640 1 -- 172.21.15.50:0/3776490114 >> 172.21.15.50:0/3776490114 conn(0x7f86f00b35f0 msgr2=0x7f86f00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:29.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.203+0000 7f86f7200640 1 -- 172.21.15.50:0/3776490114 shutdown_connections 2024-09-21T06:08:29.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.203+0000 7f86f7200640 1 -- 172.21.15.50:0/3776490114 wait complete. 2024-09-21T06:08:29.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.203+0000 7f86f7200640 1 Processor -- start 2024-09-21T06:08:29.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.203+0000 7f86f7200640 1 -- start start 2024-09-21T06:08:29.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86f7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d4004810 0x7f86f0151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:29.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86f7200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86f01fd1c0 0x7f86f014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:29.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86f7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86f014e430 0x7f86f0156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:29.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86f7200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0069780 con 0x7f86d4004810 2024-09-21T06:08:29.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86f7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0069600 con 0x7f86f014e430 2024-09-21T06:08:29.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86f7200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f86f0069900 con 0x7f86f01fd1c0 2024-09-21T06:08:29.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d4004810 0x7f86f0151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:29.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d4004810 0x7f86f0151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38014/0 (socket says 172.21.15.50:38014) 2024-09-21T06:08:29.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86efe00640 1 -- 172.21.15.50:0/2193113234 learned_addr learned my addr 172.21.15.50:0/2193113234 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:29.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86e7e00640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86f01fd1c0 0x7f86f014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:29.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86f5200640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86f014e430 0x7f86f0156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:29.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86efe00640 1 -- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86f01fd1c0 msgr2=0x7f86f014eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86efe00640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86f01fd1c0 0x7f86f014eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86efe00640 1 -- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86f014e430 msgr2=0x7f86f0156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86efe00640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86f014e430 0x7f86f0156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86efe00640 1 -- 172.21.15.50:0/2193113234 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f0151cd0 con 0x7f86d4004810 2024-09-21T06:08:29.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86f5200640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86f014e430 0x7f86f0156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:29.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86e7e00640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86f01fd1c0 0x7f86f014eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:29.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86efe00640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d4004810 0x7f86f0151480 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f86e0003f30 tx=0x7f86e00018d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:29.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86ecc00640 1 -- 172.21.15.50:0/2193113234 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0015070 con 0x7f86d4004810 2024-09-21T06:08:29.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86ecc00640 1 -- 172.21.15.50:0/2193113234 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86e0011900 con 0x7f86d4004810 2024-09-21T06:08:29.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86ecc00640 1 -- 172.21.15.50:0/2193113234 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0011d50 con 0x7f86d4004810 2024-09-21T06:08:29.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86f7200640 1 -- 172.21.15.50:0/2193113234 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f01fc6e0 con 0x7f86d4004810 2024-09-21T06:08:29.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86f7200640 1 -- 172.21.15.50:0/2193113234 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01fcca0 con 0x7f86d4004810 2024-09-21T06:08:29.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.207+0000 7f86f7200640 1 -- 172.21.15.50:0/2193113234 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f00700a0 con 0x7f86d4004810 2024-09-21T06:08:29.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.211+0000 7f86ecc00640 1 -- 172.21.15.50:0/2193113234 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f86e000f430 con 0x7f86d4004810 2024-09-21T06:08:29.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.211+0000 7f86ecc00640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f86cc074880 0x7f86cc076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:29.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.211+0000 7f86ecc00640 1 -- 172.21.15.50:0/2193113234 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7f86e0093d30 con 0x7f86d4004810 2024-09-21T06:08:29.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.211+0000 7f86e7e00640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f86cc074880 0x7f86cc076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:29.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.211+0000 7f86e7e00640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f86cc074880 0x7f86cc076d90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f86d80af000 tx=0x7f86d80a5f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:29.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.215+0000 7f86ecc00640 1 -- 172.21.15.50:0/2193113234 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f86e00607b0 con 0x7f86d4004810 2024-09-21T06:08:29.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.391+0000 7f86f7200640 1 -- 172.21.15.50:0/2193113234 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86f00b3200 con 0x7f86d4004810 2024-09-21T06:08:29.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.391+0000 7f86ecc00640 1 -- 172.21.15.50:0/2193113234 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+13073 (secure 0 0 0) 0x7f86e00600b0 con 0x7f86d4004810 2024-09-21T06:08:29.390 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:29.391 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":92,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:08:28.013296+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6835/1211403675":"2024-09-22T06:08:25.002959+0000","172.21.15.50:6833/1211403675":"2024-09-22T06:08:25.002959+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:08:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.395+0000 7f86f7200640 1 -- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f86cc074880 msgr2=0x7f86cc076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.395+0000 7f86f7200640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f86cc074880 0x7f86cc076d90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f86d80af000 tx=0x7f86d80a5f90 comp rx=0 tx=0).stop 2024-09-21T06:08:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.395+0000 7f86f7200640 1 -- 172.21.15.50:0/2193113234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d4004810 msgr2=0x7f86f0151480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.395+0000 7f86f7200640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d4004810 0x7f86f0151480 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f86e0003f30 tx=0x7f86e00018d0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.395+0000 7f86f7200640 1 -- 172.21.15.50:0/2193113234 shutdown_connections 2024-09-21T06:08:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.395+0000 7f86f7200640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86f014e430 0x7f86f0156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.395+0000 7f86f7200640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86f01fd1c0 0x7f86f014eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.395+0000 7f86f7200640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f86cc074880 0x7f86cc076d90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.395+0000 7f86f7200640 1 --2- 172.21.15.50:0/2193113234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d4004810 0x7f86f0151480 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.395+0000 7f86f7200640 1 -- 172.21.15.50:0/2193113234 >> 172.21.15.50:0/2193113234 conn(0x7f86f00b35f0 msgr2=0x7f86f01582e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.395+0000 7f86f7200640 1 -- 172.21.15.50:0/2193113234 shutdown_connections 2024-09-21T06:08:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.395+0000 7f86f7200640 1 -- 172.21.15.50:0/2193113234 wait complete. 2024-09-21T06:08:29.416 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:56:18.921197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:08:29.417 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:08:29.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.595+0000 7fd896200640 1 -- 172.21.15.50:0/3755467463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd890160470 msgr2=0x7fd890074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.595+0000 7fd896200640 1 --2- 172.21.15.50:0/3755467463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd890160470 0x7fd890074780 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fd8900ced40 tx=0x7fd88801caf0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.595+0000 7fd896200640 1 -- 172.21.15.50:0/3755467463 shutdown_connections 2024-09-21T06:08:29.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.595+0000 7fd896200640 1 --2- 172.21.15.50:0/3755467463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd890160470 0x7fd890074780 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.595+0000 7fd896200640 1 --2- 172.21.15.50:0/3755467463 >> v2:172.21.15.88:3300/0 conn(0x7fd890160060 0x7fd890074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.595+0000 7fd896200640 1 -- 172.21.15.50:0/3755467463 >> 172.21.15.50:0/3755467463 conn(0x7fd8900b4560 msgr2=0x7fd8900d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:29.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.595+0000 7fd896200640 1 -- 172.21.15.50:0/3755467463 shutdown_connections 2024-09-21T06:08:29.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.595+0000 7fd896200640 1 -- 172.21.15.50:0/3755467463 wait complete. 2024-09-21T06:08:29.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.595+0000 7fd896200640 1 Processor -- start 2024-09-21T06:08:29.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.595+0000 7fd896200640 1 -- start start 2024-09-21T06:08:29.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd896200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8901536f0 0x7fd8900d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:29.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd896200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd890160060 0x7fd890154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:29.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd896200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd890160470 0x7fd890142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:29.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd896200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd890169790 con 0x7fd8901536f0 2024-09-21T06:08:29.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd896200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd890169610 con 0x7fd890160060 2024-09-21T06:08:29.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd896200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd890169910 con 0x7fd890160470 2024-09-21T06:08:29.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd894e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8901536f0 0x7fd8900d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:29.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd894e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8901536f0 0x7fd8900d1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38028/0 (socket says 172.21.15.50:38028) 2024-09-21T06:08:29.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd894e00640 1 -- 172.21.15.50:0/131899484 learned_addr learned my addr 172.21.15.50:0/131899484 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:29.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd88fe00640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd890160060 0x7fd890154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:29.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd895800640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd890160470 0x7fd890142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:29.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd894e00640 1 -- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd890160470 msgr2=0x7fd890142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd894e00640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd890160470 0x7fd890142bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd894e00640 1 -- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd890160060 msgr2=0x7fd890154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd894e00640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd890160060 0x7fd890154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd894e00640 1 -- 172.21.15.50:0/131899484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8901fc7a0 con 0x7fd8901536f0 2024-09-21T06:08:29.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd894e00640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8901536f0 0x7fd8900d1d00 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c430 tx=0x7fd88400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:29.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd88cc00640 1 -- 172.21.15.50:0/131899484 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884011790 con 0x7fd8901536f0 2024-09-21T06:08:29.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd88cc00640 1 -- 172.21.15.50:0/131899484 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd884017070 con 0x7fd8901536f0 2024-09-21T06:08:29.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd896200640 1 -- 172.21.15.50:0/131899484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8901fbf00 con 0x7fd8901536f0 2024-09-21T06:08:29.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd88cc00640 1 -- 172.21.15.50:0/131899484 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884011ce0 con 0x7fd8901536f0 2024-09-21T06:08:29.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd896200640 1 -- 172.21.15.50:0/131899484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8901fc4c0 con 0x7fd8901536f0 2024-09-21T06:08:29.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.599+0000 7fd896200640 1 -- 172.21.15.50:0/131899484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd89015d480 con 0x7fd8901536f0 2024-09-21T06:08:29.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.603+0000 7fd88cc00640 1 -- 172.21.15.50:0/131899484 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fd88401c460 con 0x7fd8901536f0 2024-09-21T06:08:29.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.603+0000 7fd88cc00640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd870074880 0x7fd870076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:29.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.603+0000 7fd88cc00640 1 -- 172.21.15.50:0/131899484 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7fd884092be0 con 0x7fd8901536f0 2024-09-21T06:08:29.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.603+0000 7fd88fe00640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd870074880 0x7fd870076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:29.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.607+0000 7fd88fe00640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd870074880 0x7fd870076d90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd8900d2650 tx=0x7fd87c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:29.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.607+0000 7fd88cc00640 1 -- 172.21.15.50:0/131899484 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd88405f160 con 0x7fd8901536f0 2024-09-21T06:08:29.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.783+0000 7fd896200640 1 -- 172.21.15.50:0/131899484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8900b15d0 con 0x7fd8901536f0 2024-09-21T06:08:29.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.783+0000 7fd88cc00640 1 -- 172.21.15.50:0/131899484 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fd884063b90 con 0x7fd8901536f0 2024-09-21T06:08:29.784 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-09-21T06:08:29.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.787+0000 7fd896200640 1 -- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd870074880 msgr2=0x7fd870076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.787+0000 7fd896200640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd870074880 0x7fd870076d90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd8900d2650 tx=0x7fd87c00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.787+0000 7fd896200640 1 -- 172.21.15.50:0/131899484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8901536f0 msgr2=0x7fd8900d1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.787+0000 7fd896200640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8901536f0 0x7fd8900d1d00 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c430 tx=0x7fd88400c8f0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.787+0000 7fd896200640 1 -- 172.21.15.50:0/131899484 shutdown_connections 2024-09-21T06:08:29.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.787+0000 7fd896200640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd890160470 0x7fd890142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.787+0000 7fd896200640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd890160060 0x7fd890154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.787+0000 7fd896200640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd870074880 0x7fd870076d90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.787+0000 7fd896200640 1 --2- 172.21.15.50:0/131899484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8901536f0 0x7fd8900d1d00 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.787+0000 7fd896200640 1 -- 172.21.15.50:0/131899484 >> 172.21.15.50:0/131899484 conn(0x7fd8900b4560 msgr2=0x7fd89014fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:29.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.787+0000 7fd896200640 1 -- 172.21.15.50:0/131899484 shutdown_connections 2024-09-21T06:08:29.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.787+0000 7fd896200640 1 -- 172.21.15.50:0/131899484 wait complete. 2024-09-21T06:08:29.807 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-21T06:08:29.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.967+0000 7f7326800640 1 -- 172.21.15.50:0/3212895105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73201527b0 msgr2=0x7f732015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.967+0000 7f7326800640 1 --2- 172.21.15.50:0/3212895105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73201527b0 0x7f732015bb60 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7320074b20 tx=0x7f730801c760 comp rx=0 tx=0).stop 2024-09-21T06:08:29.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.967+0000 7f7326800640 1 -- 172.21.15.50:0/3212895105 shutdown_connections 2024-09-21T06:08:29.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.967+0000 7f7326800640 1 --2- 172.21.15.50:0/3212895105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73201527b0 0x7f732015bb60 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.967+0000 7f7326800640 1 -- 172.21.15.50:0/3212895105 >> 172.21.15.50:0/3212895105 conn(0x7f73200b2040 msgr2=0x7f73200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:29.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.967+0000 7f7326800640 1 -- 172.21.15.50:0/3212895105 shutdown_connections 2024-09-21T06:08:29.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.967+0000 7f7326800640 1 -- 172.21.15.50:0/3212895105 wait complete. 2024-09-21T06:08:29.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.967+0000 7f7326800640 1 Processor -- start 2024-09-21T06:08:29.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.967+0000 7f7326800640 1 -- start start 2024-09-21T06:08:29.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7326800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73200b4de0 0x7f7320143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:29.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7326800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73200b5c70 0x7f73200af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:29.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7326800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f73201527b0 0x7f73200af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:29.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7326800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7320169bd0 con 0x7f73200b4de0 2024-09-21T06:08:29.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7326800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7320169a50 con 0x7f73200b5c70 2024-09-21T06:08:29.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7326800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7320169d50 con 0x7f73201527b0 2024-09-21T06:08:29.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f731f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73200b4de0 0x7f7320143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:29.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f731f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73200b4de0 0x7f7320143b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38048/0 (socket says 172.21.15.50:38048) 2024-09-21T06:08:29.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f731f400640 1 -- 172.21.15.50:0/3036317687 learned_addr learned my addr 172.21.15.50:0/3036317687 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:29.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7317e00640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73200b5c70 0x7f73200af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:29.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f731f400640 1 -- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f73201527b0 msgr2=0x7f73200af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f731fe00640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f73201527b0 0x7f73200af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:29.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f731f400640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f73201527b0 0x7f73200af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f731f400640 1 -- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73200b5c70 msgr2=0x7f73200af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:29.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f731f400640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73200b5c70 0x7f73200af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:29.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f731f400640 1 -- 172.21.15.50:0/3036317687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7320155140 con 0x7f73200b4de0 2024-09-21T06:08:29.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7317e00640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73200b5c70 0x7f73200af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:29.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f731fe00640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f73201527b0 0x7f73200af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:29.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f731f400640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73200b4de0 0x7f7320143b10 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f731000c3c0 tx=0x7f731000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:29.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7317400640 1 -- 172.21.15.50:0/3036317687 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310017070 con 0x7f73200b4de0 2024-09-21T06:08:29.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7317400640 1 -- 172.21.15.50:0/3036317687 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7310003600 con 0x7f73200b4de0 2024-09-21T06:08:29.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7326800640 1 -- 172.21.15.50:0/3036317687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7320155420 con 0x7f73200b4de0 2024-09-21T06:08:29.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7317400640 1 -- 172.21.15.50:0/3036317687 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310003a50 con 0x7f73200b4de0 2024-09-21T06:08:29.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7326800640 1 -- 172.21.15.50:0/3036317687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7320156540 con 0x7f73200b4de0 2024-09-21T06:08:29.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.971+0000 7f7326800640 1 -- 172.21.15.50:0/3036317687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732015bb60 con 0x7f73200b4de0 2024-09-21T06:08:29.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.975+0000 7f7317400640 1 -- 172.21.15.50:0/3036317687 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f731001a070 con 0x7f73200b4de0 2024-09-21T06:08:29.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.975+0000 7f7317400640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f72fc0748b0 0x7f72fc076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:29.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.975+0000 7f7317400640 1 -- 172.21.15.50:0/3036317687 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7f73100921e0 con 0x7f73200b4de0 2024-09-21T06:08:29.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.975+0000 7f7317e00640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f72fc0748b0 0x7f72fc076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:29.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.975+0000 7f7317e00640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f72fc0748b0 0x7f72fc076dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f73080af000 tx=0x7f730801afa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:29.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:29.979+0000 7f7317400640 1 -- 172.21.15.50:0/3036317687 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f731005ec60 con 0x7f73200b4de0 2024-09-21T06:08:30.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:30.231+0000 7f7326800640 1 -- 172.21.15.50:0/3036317687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7f7320075d10 con 0x7f73200b4de0 2024-09-21T06:08:31.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.027+0000 7f7317400640 1 -- 172.21.15.50:0/3036317687 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v536) ==== 181+0+0 (secure 0 0 0) 0x7f731005e560 con 0x7f73200b4de0 2024-09-21T06:08:31.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.031+0000 7f7326800640 1 -- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f72fc0748b0 msgr2=0x7f72fc076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:31.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.031+0000 7f7326800640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f72fc0748b0 0x7f72fc076dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f73080af000 tx=0x7f730801afa0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.031+0000 7f7326800640 1 -- 172.21.15.50:0/3036317687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73200b4de0 msgr2=0x7f7320143b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:31.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.031+0000 7f7326800640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73200b4de0 0x7f7320143b10 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f731000c3c0 tx=0x7f731000c880 comp rx=0 tx=0).stop 2024-09-21T06:08:31.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.031+0000 7f7326800640 1 -- 172.21.15.50:0/3036317687 shutdown_connections 2024-09-21T06:08:31.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.031+0000 7f7326800640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f73201527b0 0x7f73200af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.031+0000 7f7326800640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73200b5c70 0x7f73200af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.031+0000 7f7326800640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f72fc0748b0 0x7f72fc076dc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.031+0000 7f7326800640 1 --2- 172.21.15.50:0/3036317687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73200b4de0 0x7f7320143b10 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.031+0000 7f7326800640 1 -- 172.21.15.50:0/3036317687 >> 172.21.15.50:0/3036317687 conn(0x7f73200b2040 msgr2=0x7f732015bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:31.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.031+0000 7f7326800640 1 -- 172.21.15.50:0/3036317687 shutdown_connections 2024-09-21T06:08:31.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.031+0000 7f7326800640 1 -- 172.21.15.50:0/3036317687 wait complete. 2024-09-21T06:08:31.052 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:08:31.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.215+0000 7f3925200640 1 -- 172.21.15.50:0/1837688565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3920157030 msgr2=0x7f39200b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:31.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.215+0000 7f3925200640 1 --2- 172.21.15.50:0/1837688565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3920157030 0x7f39200b5a80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f39200b1740 tx=0x7f391001ccf0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.215+0000 7f3925200640 1 -- 172.21.15.50:0/1837688565 shutdown_connections 2024-09-21T06:08:31.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.215+0000 7f3925200640 1 --2- 172.21.15.50:0/1837688565 >> v2:172.21.15.88:3300/0 conn(0x7f3920160d50 0x7f3920074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.215+0000 7f3925200640 1 --2- 172.21.15.50:0/1837688565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3920157030 0x7f39200b5a80 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.215+0000 7f3925200640 1 -- 172.21.15.50:0/1837688565 >> 172.21.15.50:0/1837688565 conn(0x7f39200708c0 msgr2=0x7f3920070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:31.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.215+0000 7f3925200640 1 -- 172.21.15.50:0/1837688565 shutdown_connections 2024-09-21T06:08:31.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.215+0000 7f3925200640 1 -- 172.21.15.50:0/1837688565 wait complete. 2024-09-21T06:08:31.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.215+0000 7f3925200640 1 Processor -- start 2024-09-21T06:08:31.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.215+0000 7f3925200640 1 -- start start 2024-09-21T06:08:31.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3925200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3920157030 0x7f392013ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:31.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3925200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3920160d50 0x7f39201faa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:31.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3925200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3920161160 0x7f39201faf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:31.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3925200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f392016a7c0 con 0x7f3920160d50 2024-09-21T06:08:31.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3925200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f392016a640 con 0x7f3920161160 2024-09-21T06:08:31.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3925200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f392016a940 con 0x7f3920157030 2024-09-21T06:08:31.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3915400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3920160d50 0x7f39201faa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:31.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3915400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3920160d50 0x7f39201faa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38056/0 (socket says 172.21.15.50:38056) 2024-09-21T06:08:31.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3915400640 1 -- 172.21.15.50:0/1444137135 learned_addr learned my addr 172.21.15.50:0/1444137135 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:31.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f391de00640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3920157030 0x7f392013ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:31.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f391e800640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3920161160 0x7f39201faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:31.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3915400640 1 -- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3920157030 msgr2=0x7f392013ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:31.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3915400640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3920157030 0x7f392013ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3915400640 1 -- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3920161160 msgr2=0x7f39201faf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:31.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3915400640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3920161160 0x7f39201faf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3915400640 1 -- 172.21.15.50:0/1444137135 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392013f9d0 con 0x7f3920160d50 2024-09-21T06:08:31.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3915400640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3920160d50 0x7f39201faa20 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f390800b410 tx=0x7f390800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:31.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3916a00640 1 -- 172.21.15.50:0/1444137135 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908015020 con 0x7f3920160d50 2024-09-21T06:08:31.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3916a00640 1 -- 172.21.15.50:0/1444137135 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3908011920 con 0x7f3920160d50 2024-09-21T06:08:31.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3925200640 1 -- 172.21.15.50:0/1444137135 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392013fcb0 con 0x7f3920160d50 2024-09-21T06:08:31.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3916a00640 1 -- 172.21.15.50:0/1444137135 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908011d70 con 0x7f3920160d50 2024-09-21T06:08:31.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3925200640 1 -- 172.21.15.50:0/1444137135 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3920140270 con 0x7f3920160d50 2024-09-21T06:08:31.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3925200640 1 -- 172.21.15.50:0/1444137135 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39200d2260 con 0x7f3920160d50 2024-09-21T06:08:31.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.219+0000 7f3916a00640 1 -- 172.21.15.50:0/1444137135 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f3908018020 con 0x7f3920160d50 2024-09-21T06:08:31.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.223+0000 7f3916a00640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f38fc074880 0x7f38fc076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:31.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.223+0000 7f3916a00640 1 -- 172.21.15.50:0/1444137135 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7f3908092cc0 con 0x7f3920160d50 2024-09-21T06:08:31.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.223+0000 7f391de00640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f38fc074880 0x7f38fc076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:31.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.223+0000 7f391de00640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f38fc074880 0x7f38fc076d90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f39100af000 tx=0x7f391001b530 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:31.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.227+0000 7f3916a00640 1 -- 172.21.15.50:0/1444137135 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f390801f020 con 0x7f3920160d50 2024-09-21T06:08:31.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.407+0000 7f3925200640 1 -- 172.21.15.50:0/1444137135 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f39201542f0 con 0x7f3920160d50 2024-09-21T06:08:31.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.407+0000 7f3916a00640 1 -- 172.21.15.50:0/1444137135 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v48) ==== 59+0+2294 (secure 0 0 0) 0x7f390805f220 con 0x7f3920160d50 2024-09-21T06:08:31.406 INFO:teuthology.orchestra.run.smithi050.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:08:31.406 INFO:teuthology.orchestra.run.smithi050.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:08:31.406 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:08:31.406 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_client 20/20 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3735.149299 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4471.400731 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3640.131623 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3663.645087 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3762.055672 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3661.558528 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3661.456695 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3796.187852 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds 20/20 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:08:31.407 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_ms 2/2 2024-09-21T06:08:31.408 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_frag true 2024-09-21T06:08:31.408 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:08:31.408 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:08:31.408 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_verify_scatter true 2024-09-21T06:08:31.408 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:08:31.408 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:08:31.408 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:08:31.408 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced client_mount_timeout 600 2024-09-21T06:08:31.408 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_client 20/20 2024-09-21T06:08:31.408 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_ms 1/1 2024-09-21T06:08:31.408 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:08:31.408 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:08:31.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.411+0000 7f3925200640 1 -- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f38fc074880 msgr2=0x7f38fc076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:31.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.411+0000 7f3925200640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f38fc074880 0x7f38fc076d90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f39100af000 tx=0x7f391001b530 comp rx=0 tx=0).stop 2024-09-21T06:08:31.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.411+0000 7f3925200640 1 -- 172.21.15.50:0/1444137135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3920160d50 msgr2=0x7f39201faa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:31.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.411+0000 7f3925200640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3920160d50 0x7f39201faa20 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f390800b410 tx=0x7f390800b8d0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.411+0000 7f3925200640 1 -- 172.21.15.50:0/1444137135 shutdown_connections 2024-09-21T06:08:31.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.411+0000 7f3925200640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3920161160 0x7f39201faf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.411+0000 7f3925200640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f38fc074880 0x7f38fc076d90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.411+0000 7f3925200640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3920160d50 0x7f39201faa20 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.411+0000 7f3925200640 1 --2- 172.21.15.50:0/1444137135 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3920157030 0x7f392013ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.411+0000 7f3925200640 1 -- 172.21.15.50:0/1444137135 >> 172.21.15.50:0/1444137135 conn(0x7f39200708c0 msgr2=0x7f392015e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:31.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.411+0000 7f3925200640 1 -- 172.21.15.50:0/1444137135 shutdown_connections 2024-09-21T06:08:31.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.411+0000 7f3925200640 1 -- 172.21.15.50:0/1444137135 wait complete. 2024-09-21T06:08:31.429 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:08:31.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.591+0000 7f051ee00640 1 -- 172.21.15.50:0/1177887839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04fc004810 msgr2=0x7f04fc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:31.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.591+0000 7f051ee00640 1 --2- 172.21.15.50:0/1177887839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04fc004810 0x7f04fc004c20 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f0500009b40 tx=0x7f050001d220 comp rx=0 tx=0).stop 2024-09-21T06:08:31.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.591+0000 7f051ee00640 1 -- 172.21.15.50:0/1177887839 shutdown_connections 2024-09-21T06:08:31.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.591+0000 7f051ee00640 1 --2- 172.21.15.50:0/1177887839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04fc004810 0x7f04fc004c20 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.591+0000 7f051ee00640 1 -- 172.21.15.50:0/1177887839 >> 172.21.15.50:0/1177887839 conn(0x7f05180b2040 msgr2=0x7f05180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:31.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.591+0000 7f051ee00640 1 -- 172.21.15.50:0/1177887839 shutdown_connections 2024-09-21T06:08:31.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.591+0000 7f051ee00640 1 -- 172.21.15.50:0/1177887839 wait complete. 2024-09-21T06:08:31.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ee00640 1 Processor -- start 2024-09-21T06:08:31.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ee00640 1 -- start start 2024-09-21T06:08:31.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ee00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f04fc004810 0x7f0518153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:31.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0518154da0 0x7f0518155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:31.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051814e770 0x7f05181f9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:31.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f05180b5580 con 0x7f051814e770 2024-09-21T06:08:31.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05180b5400 con 0x7f0518154da0 2024-09-21T06:08:31.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ee00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f05180b5700 con 0x7f04fc004810 2024-09-21T06:08:31.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051814e770 0x7f05181f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:31.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051814e770 0x7f05181f9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38074/0 (socket says 172.21.15.50:38074) 2024-09-21T06:08:31.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ce00640 1 -- 172.21.15.50:0/1113968311 learned_addr learned my addr 172.21.15.50:0/1113968311 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:31.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f0517e00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f04fc004810 0x7f0518153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:31.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ce00640 1 -- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f04fc004810 msgr2=0x7f0518153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:31.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f050fe00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0518154da0 0x7f0518155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:31.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ce00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f04fc004810 0x7f0518153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ce00640 1 -- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0518154da0 msgr2=0x7f0518155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:31.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ce00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0518154da0 0x7f0518155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ce00640 1 -- 172.21.15.50:0/1113968311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05180cfa40 con 0x7f051814e770 2024-09-21T06:08:31.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f050fe00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0518154da0 0x7f0518155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:31.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ce00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051814e770 0x7f05181f9470 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f0518074950 tx=0x7f0510001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:31.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f0514c00640 1 -- 172.21.15.50:0/1113968311 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05100107d0 con 0x7f051814e770 2024-09-21T06:08:31.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f0514c00640 1 -- 172.21.15.50:0/1113968311 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0510010990 con 0x7f051814e770 2024-09-21T06:08:31.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ee00640 1 -- 172.21.15.50:0/1113968311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05180cfd20 con 0x7f051814e770 2024-09-21T06:08:31.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f0514c00640 1 -- 172.21.15.50:0/1113968311 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510010de0 con 0x7f051814e770 2024-09-21T06:08:31.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.595+0000 7f051ee00640 1 -- 172.21.15.50:0/1113968311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05181f8f50 con 0x7f051814e770 2024-09-21T06:08:31.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.599+0000 7f051ee00640 1 -- 172.21.15.50:0/1113968311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051815f500 con 0x7f051814e770 2024-09-21T06:08:31.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.599+0000 7f0514c00640 1 -- 172.21.15.50:0/1113968311 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f051001a400 con 0x7f051814e770 2024-09-21T06:08:31.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.599+0000 7f0514c00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f04f4074880 0x7f04f4076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:31.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.599+0000 7f0514c00640 1 -- 172.21.15.50:0/1113968311 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7f05100935a0 con 0x7f051814e770 2024-09-21T06:08:31.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.599+0000 7f0517e00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f04f4074880 0x7f04f4076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:31.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.603+0000 7f0517e00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f04f4074880 0x7f04f4076d90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0508001870 tx=0x7f0508001d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:31.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.603+0000 7f0514c00640 1 -- 172.21.15.50:0/1113968311 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f05100600b0 con 0x7f051814e770 2024-09-21T06:08:31.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.783+0000 7f051ee00640 1 -- 172.21.15.50:0/1113968311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f051815dea0 con 0x7f051814e770 2024-09-21T06:08:31.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.787+0000 7f0514c00640 1 -- 172.21.15.50:0/1113968311 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v49)=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7f051005f9b0 con 0x7f051814e770 2024-09-21T06:08:31.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.791+0000 7f051ee00640 1 -- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f04f4074880 msgr2=0x7f04f4076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:31.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.791+0000 7f051ee00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f04f4074880 0x7f04f4076d90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0508001870 tx=0x7f0508001d00 comp rx=0 tx=0).stop 2024-09-21T06:08:31.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.791+0000 7f051ee00640 1 -- 172.21.15.50:0/1113968311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051814e770 msgr2=0x7f05181f9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:31.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.791+0000 7f051ee00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051814e770 0x7f05181f9470 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f0518074950 tx=0x7f0510001a80 comp rx=0 tx=0).stop 2024-09-21T06:08:31.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.791+0000 7f051ee00640 1 -- 172.21.15.50:0/1113968311 shutdown_connections 2024-09-21T06:08:31.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.791+0000 7f051ee00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f04f4074880 0x7f04f4076d90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.791+0000 7f051ee00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051814e770 0x7f05181f9470 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.791+0000 7f051ee00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0518154da0 0x7f0518155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.791+0000 7f051ee00640 1 --2- 172.21.15.50:0/1113968311 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f04fc004810 0x7f0518153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:31.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.791+0000 7f051ee00640 1 -- 172.21.15.50:0/1113968311 >> 172.21.15.50:0/1113968311 conn(0x7f05180b2040 msgr2=0x7f0518151cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:31.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.791+0000 7f051ee00640 1 -- 172.21.15.50:0/1113968311 shutdown_connections 2024-09-21T06:08:31.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:31.791+0000 7f051ee00640 1 -- 172.21.15.50:0/1113968311 wait complete. 2024-09-21T06:08:31.810 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:08:31.811 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:08:31.927 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:08:32.024 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:08:32.024 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:08:32.024 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:08:32.034 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:08:32.035 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:08:32.035 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:08:32.043 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:08:32.044 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:08:32.044 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:08:32.084 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:08:32.084 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:08:32.085 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:08:32.096 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:08:32.096 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:08:32.097 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:08:32.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.295+0000 7fa863e00640 1 -- 172.21.15.50:0/4176320845 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa864160060 msgr2=0x7fa864074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.295+0000 7fa863e00640 1 --2- 172.21.15.50:0/4176320845 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa864160060 0x7fa864074240 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009900 tx=0x7fa84c01c850 comp rx=0 tx=0).stop 2024-09-21T06:08:32.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.295+0000 7fa863e00640 1 -- 172.21.15.50:0/4176320845 shutdown_connections 2024-09-21T06:08:32.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.295+0000 7fa863e00640 1 --2- 172.21.15.50:0/4176320845 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa864160060 0x7fa864074240 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.295+0000 7fa863e00640 1 --2- 172.21.15.50:0/4176320845 >> v2:172.21.15.88:3300/0 conn(0x7fa8641536f0 0x7fa86415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.295+0000 7fa863e00640 1 -- 172.21.15.50:0/4176320845 >> 172.21.15.50:0/4176320845 conn(0x7fa8640b4560 msgr2=0x7fa8640d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:32.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.295+0000 7fa863e00640 1 -- 172.21.15.50:0/4176320845 shutdown_connections 2024-09-21T06:08:32.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.295+0000 7fa863e00640 1 -- 172.21.15.50:0/4176320845 wait complete. 2024-09-21T06:08:32.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa863e00640 1 Processor -- start 2024-09-21T06:08:32.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa863e00640 1 -- start start 2024-09-21T06:08:32.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa863e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8641536f0 0x7fa8641566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:32.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa863e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa864160060 0x7fa86414a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:32.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa863e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa864160470 0x7fa864142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:32.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa863e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa86416a3f0 con 0x7fa864160060 2024-09-21T06:08:32.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa863e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa86416a270 con 0x7fa864160470 2024-09-21T06:08:32.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa863e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa86416a570 con 0x7fa8641536f0 2024-09-21T06:08:32.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa85be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa864160060 0x7fa86414a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:32.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa85be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa864160060 0x7fa86414a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38082/0 (socket says 172.21.15.50:38082) 2024-09-21T06:08:32.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa85be00640 1 -- 172.21.15.50:0/3454860012 learned_addr learned my addr 172.21.15.50:0/3454860012 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:32.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa862a00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8641536f0 0x7fa8641566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:32.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa863400640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa864160470 0x7fa864142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:32.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa85be00640 1 -- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8641536f0 msgr2=0x7fa8641566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa85be00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8641536f0 0x7fa8641566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa85be00640 1 -- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa864160470 msgr2=0x7fa864142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa85be00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa864160470 0x7fa864142d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa85be00640 1 -- 172.21.15.50:0/3454860012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa864152d90 con 0x7fa864160060 2024-09-21T06:08:32.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.299+0000 7fa85be00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa864160060 0x7fa86414a6f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0b0040 tx=0x7fa84c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:32.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.303+0000 7fa85aa00640 1 -- 172.21.15.50:0/3454860012 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c0b2040 con 0x7fa864160060 2024-09-21T06:08:32.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.303+0000 7fa85aa00640 1 -- 172.21.15.50:0/3454860012 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa84c004040 con 0x7fa864160060 2024-09-21T06:08:32.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.303+0000 7fa85aa00640 1 -- 172.21.15.50:0/3454860012 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c004410 con 0x7fa864160060 2024-09-21T06:08:32.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.303+0000 7fa863e00640 1 -- 172.21.15.50:0/3454860012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa864153070 con 0x7fa864160060 2024-09-21T06:08:32.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.303+0000 7fa863e00640 1 -- 172.21.15.50:0/3454860012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa864154db0 con 0x7fa864160060 2024-09-21T06:08:32.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.303+0000 7fa863e00640 1 -- 172.21.15.50:0/3454860012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa86415d480 con 0x7fa864160060 2024-09-21T06:08:32.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.303+0000 7fa85aa00640 1 -- 172.21.15.50:0/3454860012 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fa84c0bf020 con 0x7fa864160060 2024-09-21T06:08:32.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.303+0000 7fa85aa00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa8400748b0 0x7fa840076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:32.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.303+0000 7fa85aa00640 1 -- 172.21.15.50:0/3454860012 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7fa84c12cf80 con 0x7fa864160060 2024-09-21T06:08:32.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.303+0000 7fa862a00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa8400748b0 0x7fa840076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:32.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.307+0000 7fa862a00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa8400748b0 0x7fa840076dc0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa854009600 tx=0x7fa8540091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:32.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.307+0000 7fa85aa00640 1 -- 172.21.15.50:0/3454860012 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa84c0c6020 con 0x7fa864160060 2024-09-21T06:08:32.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.519+0000 7fa863e00640 1 -- 172.21.15.50:0/3454860012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8641fd3f0 con 0x7fa864160060 2024-09-21T06:08:32.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.519+0000 7fa85aa00640 1 -- 172.21.15.50:0/3454860012 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+3911 (secure 0 0 0) 0x7fa84c0164d0 con 0x7fa864160060 2024-09-21T06:08:32.520 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:32.520 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":58,"btime":"2024-09-21T06:08:26:005597+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3691304879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3691304879},{"type":"v1","addr":"172.21.15.50:6834","nonce":3691304879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1924969859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1924969859},{"type":"v1","addr":"172.21.15.88:6837","nonce":1924969859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3025915209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3025915209},{"type":"v1","addr":"172.21.15.88:6835","nonce":3025915209}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7761,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1134773103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1134773103},{"type":"v1","addr":"172.21.15.50:6835","nonce":1134773103}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[]} 2024-09-21T06:08:32.520 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 58 2024-09-21T06:08:32.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.523+0000 7fa863e00640 1 -- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa8400748b0 msgr2=0x7fa840076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.523+0000 7fa863e00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa8400748b0 0x7fa840076dc0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa854009600 tx=0x7fa8540091e0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.523+0000 7fa863e00640 1 -- 172.21.15.50:0/3454860012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa864160060 msgr2=0x7fa86414a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.523+0000 7fa863e00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa864160060 0x7fa86414a6f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0b0040 tx=0x7fa84c005760 comp rx=0 tx=0).stop 2024-09-21T06:08:32.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.523+0000 7fa863e00640 1 -- 172.21.15.50:0/3454860012 shutdown_connections 2024-09-21T06:08:32.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.523+0000 7fa863e00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa864160470 0x7fa864142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.523+0000 7fa863e00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa8400748b0 0x7fa840076dc0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.523+0000 7fa863e00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa864160060 0x7fa86414a6f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.523+0000 7fa863e00640 1 --2- 172.21.15.50:0/3454860012 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa8641536f0 0x7fa8641566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.523+0000 7fa863e00640 1 -- 172.21.15.50:0/3454860012 >> 172.21.15.50:0/3454860012 conn(0x7fa8640b4560 msgr2=0x7fa8640715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:32.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.523+0000 7fa863e00640 1 -- 172.21.15.50:0/3454860012 shutdown_connections 2024-09-21T06:08:32.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.523+0000 7fa863e00640 1 -- 172.21.15.50:0/3454860012 wait complete. 2024-09-21T06:08:32.542 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:08:32.543 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:08:32.543 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:08:32.543 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:08:32.543 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:08:32.543 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:08:32.543 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:08:32.543 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:08:32.544 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:08:32.544 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:08:32.544 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:08:32.544 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:08:32.544 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:08:32.544 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:08:32.544 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:08:32.544 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:08:32.557 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:08:32.558 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:08:32.562 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:08:32.562 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:08:32.563 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:08:32.563 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:08:32.567 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:08:32.567 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:08:32.607 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:08:32.609 DEBUG:teuthology.parallel:result is None 2024-09-21T06:08:32.610 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:08:32.610 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:08:32.611 DEBUG:teuthology.parallel:result is None 2024-09-21T06:08:32.611 DEBUG:teuthology.parallel:result is None 2024-09-21T06:08:32.612 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:08:32.612 DEBUG:teuthology.parallel:result is None 2024-09-21T06:08:32.612 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:08:32.711 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T06:08:32.713 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-09-21T06:08:32.715 INFO:tasks.ceph.mds.b.smithi088.stdout:starting mds.b at 2024-09-21T06:08:32.715 INFO:tasks.ceph.mds.d.smithi088.stdout:starting mds.d at 2024-09-21T06:08:32.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.779+0000 7f9c04a00640 1 -- 172.21.15.50:0/1493862532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c0015c210 msgr2=0x7f9c0006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.779+0000 7f9c04a00640 1 --2- 172.21.15.50:0/1493862532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c0015c210 0x7f9c0006e7c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009bc0 tx=0x7f9bf801cb70 comp rx=0 tx=0).stop 2024-09-21T06:08:32.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.779+0000 7f9c04a00640 1 -- 172.21.15.50:0/1493862532 shutdown_connections 2024-09-21T06:08:32.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.779+0000 7f9c04a00640 1 --2- 172.21.15.50:0/1493862532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c0015c210 0x7f9c0006e7c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.779+0000 7f9c04a00640 1 --2- 172.21.15.50:0/1493862532 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c00074590 0x7f9c0015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.779+0000 7f9c04a00640 1 -- 172.21.15.50:0/1493862532 >> 172.21.15.50:0/1493862532 conn(0x7f9c000b2c40 msgr2=0x7f9c000b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:32.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.779+0000 7f9c04a00640 1 -- 172.21.15.50:0/1493862532 shutdown_connections 2024-09-21T06:08:32.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.779+0000 7f9c04a00640 1 -- 172.21.15.50:0/1493862532 wait complete. 2024-09-21T06:08:32.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.783+0000 7f9c04a00640 1 Processor -- start 2024-09-21T06:08:32.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.783+0000 7f9c04a00640 1 -- start start 2024-09-21T06:08:32.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.783+0000 7f9c04a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c000cff30 0x7f9c0015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:32.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.783+0000 7f9c04a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c0015c210 0x7f9c0015d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:32.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.783+0000 7f9c04a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c0014a1c0 0x7f9c0015de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:32.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.783+0000 7f9c04a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9c00152590 con 0x7f9c000cff30 2024-09-21T06:08:32.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.783+0000 7f9c04a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9c00152410 con 0x7f9c0015c210 2024-09-21T06:08:32.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.783+0000 7f9c04a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9c00152710 con 0x7f9c0014a1c0 2024-09-21T06:08:32.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.783+0000 7f9bfde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c000cff30 0x7f9c0015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:32.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.783+0000 7f9bfde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c000cff30 0x7f9c0015cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38114/0 (socket says 172.21.15.50:38114) 2024-09-21T06:08:32.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.783+0000 7f9bfde00640 1 -- 172.21.15.50:0/1159925011 learned_addr learned my addr 172.21.15.50:0/1159925011 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:32.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9bf7e00640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c0015c210 0x7f9c0015d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:32.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9bfe800640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c0014a1c0 0x7f9c0015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:32.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9bfde00640 1 -- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c0014a1c0 msgr2=0x7f9c0015de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9bfde00640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c0014a1c0 0x7f9c0015de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9bfde00640 1 -- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c0015c210 msgr2=0x7f9c0015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9bfde00640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c0015c210 0x7f9c0015d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9bfde00640 1 -- 172.21.15.50:0/1159925011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c001f9950 con 0x7f9c000cff30 2024-09-21T06:08:32.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9bfde00640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c000cff30 0x7f9c0015cd60 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000c430 tx=0x7f9bf000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:32.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9bf6000640 1 -- 172.21.15.50:0/1159925011 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0011790 con 0x7f9c000cff30 2024-09-21T06:08:32.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9bf6000640 1 -- 172.21.15.50:0/1159925011 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bf0017070 con 0x7f9c000cff30 2024-09-21T06:08:32.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9bf6000640 1 -- 172.21.15.50:0/1159925011 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0011ce0 con 0x7f9c000cff30 2024-09-21T06:08:32.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9c04a00640 1 -- 172.21.15.50:0/1159925011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c001f9c30 con 0x7f9c000cff30 2024-09-21T06:08:32.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9c04a00640 1 -- 172.21.15.50:0/1159925011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c001fa1f0 con 0x7f9c000cff30 2024-09-21T06:08:32.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9c04a00640 1 -- 172.21.15.50:0/1159925011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0015bcd0 con 0x7f9c000cff30 2024-09-21T06:08:32.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.787+0000 7f9bf6000640 1 -- 172.21.15.50:0/1159925011 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f9bf001c460 con 0x7f9c000cff30 2024-09-21T06:08:32.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.791+0000 7f9bf6000640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9bdc074880 0x7f9bdc076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:32.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.791+0000 7f9bf6000640 1 -- 172.21.15.50:0/1159925011 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7f9bf0092c40 con 0x7f9c000cff30 2024-09-21T06:08:32.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.791+0000 7f9bf7e00640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9bdc074880 0x7f9bdc076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:32.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.791+0000 7f9bf7e00640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9bdc074880 0x7f9bdc076d90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9c001565d0 tx=0x7f9be80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:32.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.795+0000 7f9bf6000640 1 -- 172.21.15.50:0/1159925011 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9bf005f1c0 con 0x7f9c000cff30 2024-09-21T06:08:32.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:32.983+0000 7f9c04a00640 1 -- 172.21.15.50:0/1159925011 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9c00154850 con 0x7f9c000cff30 2024-09-21T06:08:33.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.043+0000 7f9bf6000640 1 -- 172.21.15.50:0/1159925011 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v94) ==== 97+0+0 (secure 0 0 0) 0x7f9bf0063bf0 con 0x7f9c000cff30 2024-09-21T06:08:33.043 INFO:teuthology.orchestra.run.smithi050.stderr: removed all blocklist entries 2024-09-21T06:08:33.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.047+0000 7f9c04a00640 1 -- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9bdc074880 msgr2=0x7f9bdc076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:33.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.047+0000 7f9c04a00640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9bdc074880 0x7f9bdc076d90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9c001565d0 tx=0x7f9be80091e0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.047+0000 7f9c04a00640 1 -- 172.21.15.50:0/1159925011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c000cff30 msgr2=0x7f9c0015cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:33.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.047+0000 7f9c04a00640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c000cff30 0x7f9c0015cd60 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000c430 tx=0x7f9bf000c8f0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.047+0000 7f9c04a00640 1 -- 172.21.15.50:0/1159925011 shutdown_connections 2024-09-21T06:08:33.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.047+0000 7f9c04a00640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9c0014a1c0 0x7f9c0015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.047+0000 7f9c04a00640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c0015c210 0x7f9c0015d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.047+0000 7f9c04a00640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9bdc074880 0x7f9bdc076d90 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.047+0000 7f9c04a00640 1 --2- 172.21.15.50:0/1159925011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c000cff30 0x7f9c0015cd60 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.047+0000 7f9c04a00640 1 -- 172.21.15.50:0/1159925011 >> 172.21.15.50:0/1159925011 conn(0x7f9c000b2c40 msgr2=0x7f9c0014adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:33.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.047+0000 7f9c04a00640 1 -- 172.21.15.50:0/1159925011 shutdown_connections 2024-09-21T06:08:33.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.047+0000 7f9c04a00640 1 -- 172.21.15.50:0/1159925011 wait complete. 2024-09-21T06:08:33.070 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:08:33.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.231+0000 7f25afe00640 1 -- 172.21.15.50:0/151236547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25b00b2300 msgr2=0x7f25b00d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:33.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.231+0000 7f25afe00640 1 --2- 172.21.15.50:0/151236547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25b00b2300 0x7f25b00d0090 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f25b00d1170 tx=0x7f259801c890 comp rx=0 tx=0).stop 2024-09-21T06:08:33.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.231+0000 7f25afe00640 1 -- 172.21.15.50:0/151236547 shutdown_connections 2024-09-21T06:08:33.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.231+0000 7f25afe00640 1 --2- 172.21.15.50:0/151236547 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25b00d05d0 0x7f25b00d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.231+0000 7f25afe00640 1 --2- 172.21.15.50:0/151236547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25b00b2300 0x7f25b00d0090 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.231+0000 7f25afe00640 1 --2- 172.21.15.50:0/151236547 >> v2:172.21.15.88:3300/0 conn(0x7f25b0074d30 0x7f25b00b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.231+0000 7f25afe00640 1 -- 172.21.15.50:0/151236547 >> 172.21.15.50:0/151236547 conn(0x7f25b006cdf0 msgr2=0x7f25b006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:33.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.231+0000 7f25afe00640 1 -- 172.21.15.50:0/151236547 shutdown_connections 2024-09-21T06:08:33.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.231+0000 7f25afe00640 1 -- 172.21.15.50:0/151236547 wait complete. 2024-09-21T06:08:33.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.231+0000 7f25afe00640 1 Processor -- start 2024-09-21T06:08:33.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.231+0000 7f25afe00640 1 -- start start 2024-09-21T06:08:33.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25afe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25b0074d30 0x7f25b01665f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:33.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25afe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25b00b2300 0x7f25b00aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:33.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25afe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25b00d05d0 0x7f25b0073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:33.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25afe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f25b00cef00 con 0x7f25b0074d30 2024-09-21T06:08:33.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25afe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f25b00ced80 con 0x7f25b00d05d0 2024-09-21T06:08:33.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25afe00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f25b00cf080 con 0x7f25b00b2300 2024-09-21T06:08:33.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25aea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25b0074d30 0x7f25b01665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:33.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25aea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25b0074d30 0x7f25b01665f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38130/0 (socket says 172.21.15.50:38130) 2024-09-21T06:08:33.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25aea00640 1 -- 172.21.15.50:0/2197200876 learned_addr learned my addr 172.21.15.50:0/2197200876 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:33.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25af400640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25b00d05d0 0x7f25b0073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:33.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25a6000640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25b00b2300 0x7f25b00aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:33.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25aea00640 1 -- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25b00b2300 msgr2=0x7f25b00aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:33.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25aea00640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25b00b2300 0x7f25b00aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25aea00640 1 -- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25b00d05d0 msgr2=0x7f25b0073c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:33.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25aea00640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25b00d05d0 0x7f25b0073c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25aea00640 1 -- 172.21.15.50:0/2197200876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b00aa570 con 0x7f25b0074d30 2024-09-21T06:08:33.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25aea00640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25b0074d30 0x7f25b01665f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f25a000d6d0 tx=0x7f25a000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:33.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25a7400640 1 -- 172.21.15.50:0/2197200876 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0017070 con 0x7f25b0074d30 2024-09-21T06:08:33.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25a7400640 1 -- 172.21.15.50:0/2197200876 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25a0009b10 con 0x7f25b0074d30 2024-09-21T06:08:33.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25a7400640 1 -- 172.21.15.50:0/2197200876 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0010460 con 0x7f25b0074d30 2024-09-21T06:08:33.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25afe00640 1 -- 172.21.15.50:0/2197200876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b00aa850 con 0x7f25b0074d30 2024-09-21T06:08:33.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25afe00640 1 -- 172.21.15.50:0/2197200876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b00aae10 con 0x7f25b0074d30 2024-09-21T06:08:33.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.235+0000 7f25afe00640 1 -- 172.21.15.50:0/2197200876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b00cf5a0 con 0x7f25b0074d30 2024-09-21T06:08:33.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.239+0000 7f25a7400640 1 -- 172.21.15.50:0/2197200876 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f25a001a070 con 0x7f25b0074d30 2024-09-21T06:08:33.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.239+0000 7f25a7400640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f258c074880 0x7f258c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:33.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.239+0000 7f25a7400640 1 -- 172.21.15.50:0/2197200876 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4913+0+0 (secure 0 0 0) 0x7f25a0093000 con 0x7f25b0074d30 2024-09-21T06:08:33.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.239+0000 7f25a6000640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f258c074880 0x7f258c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:33.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.239+0000 7f25a6000640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f258c074880 0x7f258c076d90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f25980af000 tx=0x7f25980a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:33.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.243+0000 7f25a7400640 1 -- 172.21.15.50:0/2197200876 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f25a005f430 con 0x7f25b0074d30 2024-09-21T06:08:33.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.479+0000 7f25afe00640 1 -- 172.21.15.50:0/2197200876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f25b0156cb0 con 0x7f25b0074d30 2024-09-21T06:08:33.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.479+0000 7f25a7400640 1 -- 172.21.15.50:0/2197200876 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) ==== 80+0+5304 (secure 0 0 0) 0x7f25a0063e60 con 0x7f25b0074d30 2024-09-21T06:08:33.478 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout: "auth_dump": [ 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.a", 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mSKq0ARAA2egaHGYWhRUEq6oRl1CIMg==", 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.479 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.b", 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mwv0+DxAAY+eZPxv5uY9PnLl0asdr6w==", 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.c", 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5me6maCRAA4jvHgTLTKpb/mxzzG71SfQ==", 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.480 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.d", 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mzYnuFhAA/hEigjluXspzs/hyeWxBjQ==", 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.481 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.0", 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDIX+5mrZF/NhAA33M6ummG4r4yXU8/YOL5WA==", 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.1", 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDMX+5myVz5DxAAMYfTPep6ubD7Cxnfa9dMCg==", 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.482 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.2", 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDPX+5mA7ahIRAAf/osnY5l83p8ZWna0ErD2Q==", 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.483 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.3", 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDSX+5m+rDpNRAApGPZJIlAaA0DyjtPXPgoaQ==", 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.4", 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDYX+5mKL9SORAA21ITQUwUSoDUjXaD0Cwb3g==", 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.484 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.5", 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDcX+5minRqFBAAMJOBQNvJFD4TNdKbPMLBig==", 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.6", 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDfX+5m6HadJhAAAv670Tvyi4IKMByFPiBlAw==", 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.485 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.7", 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDjX+5muPrrAhAAuRB7Muamiht+f7LvSmI6UQ==", 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.486 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.0", 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw==", 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow r", 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.1", 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mo94GJBAAMBSb1NPb2L93vzvnAh59kw==", 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:08:33.487 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.2", 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5m4IrnHxAAFjER3ywuS+FlbuA7bPZVqw==", 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.488 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.3", 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mysQeKBAAq5c4bD3hbsU1H7JabaMjwQ==", 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.admin", 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mcUXKHhAAV1Rs0OMNVPeAsRiAVt67jA==", 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:08:33.489 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:08:33.490 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.490 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.490 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.490 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.490 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.x", 2024-09-21T06:08:33.490 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5m2F9WOBAAZuQTUyR7oTT01CRD1jo+/Q==", 2024-09-21T06:08:33.490 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.490 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:08:33.490 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:08:33.490 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.490 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.491 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:08:33.491 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.491 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.y", 2024-09-21T06:08:33.491 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mEjC/MxAAHgwIu4CLDgFajwD2rL5WKA==", 2024-09-21T06:08:33.491 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:08:33.491 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:08:33.491 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:08:33.491 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:08:33.491 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.491 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.491 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-21T06:08:33.491 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:08:33.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.483+0000 7f25afe00640 1 -- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f258c074880 msgr2=0x7f258c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:33.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.483+0000 7f25afe00640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f258c074880 0x7f258c076d90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f25980af000 tx=0x7f25980a7040 comp rx=0 tx=0).stop 2024-09-21T06:08:33.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.483+0000 7f25afe00640 1 -- 172.21.15.50:0/2197200876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25b0074d30 msgr2=0x7f25b01665f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:33.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.483+0000 7f25afe00640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25b0074d30 0x7f25b01665f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f25a000d6d0 tx=0x7f25a000db90 comp rx=0 tx=0).stop 2024-09-21T06:08:33.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.483+0000 7f25afe00640 1 -- 172.21.15.50:0/2197200876 shutdown_connections 2024-09-21T06:08:33.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.483+0000 7f25afe00640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25b00d05d0 0x7f25b0073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.483+0000 7f25afe00640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25b00b2300 0x7f25b00aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.483+0000 7f25afe00640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f258c074880 0x7f258c076d90 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.483+0000 7f25afe00640 1 --2- 172.21.15.50:0/2197200876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25b0074d30 0x7f25b01665f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.483+0000 7f25afe00640 1 -- 172.21.15.50:0/2197200876 >> 172.21.15.50:0/2197200876 conn(0x7f25b006cdf0 msgr2=0x7f25b006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:33.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.483+0000 7f25afe00640 1 -- 172.21.15.50:0/2197200876 shutdown_connections 2024-09-21T06:08:33.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.483+0000 7f25afe00640 1 -- 172.21.15.50:0/2197200876 wait complete. 2024-09-21T06:08:33.502 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:08:33.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.659+0000 7f6ccd800640 1 -- 172.21.15.50:0/3641905558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cc81527b0 msgr2=0x7f6cc8074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:33.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.659+0000 7f6ccd800640 1 --2- 172.21.15.50:0/3641905558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cc81527b0 0x7f6cc8074e30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8071cb0 tx=0x7f6cb001c890 comp rx=0 tx=0).stop 2024-09-21T06:08:33.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.659+0000 7f6ccd800640 1 -- 172.21.15.50:0/3641905558 shutdown_connections 2024-09-21T06:08:33.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.659+0000 7f6ccd800640 1 --2- 172.21.15.50:0/3641905558 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cc80b4de0 0x7f6cc80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.659+0000 7f6ccd800640 1 --2- 172.21.15.50:0/3641905558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cc81527b0 0x7f6cc8074e30 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.659+0000 7f6ccd800640 1 --2- 172.21.15.50:0/3641905558 >> v2:172.21.15.88:3300/0 conn(0x7f6cc80b5c70 0x7f6cc815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.659+0000 7f6ccd800640 1 -- 172.21.15.50:0/3641905558 >> 172.21.15.50:0/3641905558 conn(0x7f6cc80b2040 msgr2=0x7f6cc80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:33.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.659+0000 7f6ccd800640 1 -- 172.21.15.50:0/3641905558 shutdown_connections 2024-09-21T06:08:33.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.659+0000 7f6ccd800640 1 -- 172.21.15.50:0/3641905558 wait complete. 2024-09-21T06:08:33.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6ccd800640 1 Processor -- start 2024-09-21T06:08:33.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6ccd800640 1 -- start start 2024-09-21T06:08:33.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6ccd800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cc80b4de0 0x7f6cc80af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:33.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6ccd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80b5c70 0x7f6cc81511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:33.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6ccd800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cc81527b0 0x7f6cc80ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:33.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6ccd800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc815f6b0 con 0x7f6cc81527b0 2024-09-21T06:08:33.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6ccd800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc815f530 con 0x7f6cc80b5c70 2024-09-21T06:08:33.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6ccd800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc815f830 con 0x7f6cc80b4de0 2024-09-21T06:08:33.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6cc7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cc81527b0 0x7f6cc80ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:33.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6cc7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cc81527b0 0x7f6cc80ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38154/0 (socket says 172.21.15.50:38154) 2024-09-21T06:08:33.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6cc7400640 1 -- 172.21.15.50:0/4157961350 learned_addr learned my addr 172.21.15.50:0/4157961350 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:33.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6cc6a00640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cc80b4de0 0x7f6cc80af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:33.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.663+0000 7f6cbfe00640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80b5c70 0x7f6cc81511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:33.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6cc7400640 1 -- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cc80b4de0 msgr2=0x7f6cc80af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:33.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6cc7400640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cc80b4de0 0x7f6cc80af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6cc7400640 1 -- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80b5c70 msgr2=0x7f6cc81511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:33.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6cc7400640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80b5c70 0x7f6cc81511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6cc7400640 1 -- 172.21.15.50:0/4157961350 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc80ae5c0 con 0x7f6cc81527b0 2024-09-21T06:08:33.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6cc7400640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cc81527b0 0x7f6cc80ccae0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000d980 tx=0x7f6cc000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:33.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6cbea00640 1 -- 172.21.15.50:0/4157961350 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0016070 con 0x7f6cc81527b0 2024-09-21T06:08:33.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6cbea00640 1 -- 172.21.15.50:0/4157961350 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cc0004e20 con 0x7f6cc81527b0 2024-09-21T06:08:33.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6ccd800640 1 -- 172.21.15.50:0/4157961350 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc80cf4a0 con 0x7f6cc81527b0 2024-09-21T06:08:33.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6cbea00640 1 -- 172.21.15.50:0/4157961350 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000bcb0 con 0x7f6cc81527b0 2024-09-21T06:08:33.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6ccd800640 1 -- 172.21.15.50:0/4157961350 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc80aeb40 con 0x7f6cc81527b0 2024-09-21T06:08:33.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6ccd800640 1 -- 172.21.15.50:0/4157961350 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc80cf1f0 con 0x7f6cc81527b0 2024-09-21T06:08:33.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.667+0000 7f6cbea00640 1 -- 172.21.15.50:0/4157961350 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f6cc0003980 con 0x7f6cc81527b0 2024-09-21T06:08:33.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.671+0000 7f6cbea00640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ca4074880 0x7f6ca4076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:33.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.671+0000 7f6cbea00640 1 -- 172.21.15.50:0/4157961350 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4913+0+0 (secure 0 0 0) 0x7f6cc0093760 con 0x7f6cc81527b0 2024-09-21T06:08:33.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.671+0000 7f6cc6a00640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ca4074880 0x7f6ca4076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:33.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.671+0000 7f6cc6a00640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ca4074880 0x7f6ca4076d90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009b40 tx=0x7f6cb80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:33.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.675+0000 7f6cbea00640 1 -- 172.21.15.50:0/4157961350 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6cc001f020 con 0x7f6cc81527b0 2024-09-21T06:08:33.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.851+0000 7f6ccd800640 1 -- 172.21.15.50:0/4157961350 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6cc815fd50 con 0x7f6cc81527b0 2024-09-21T06:08:33.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.851+0000 7f6cbea00640 1 -- 172.21.15.50:0/4157961350 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) ==== 84+0+65 (secure 0 0 0) 0x7f6cc005fc30 con 0x7f6cc81527b0 2024-09-21T06:08:33.851 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:33.851 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:08:33.851 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:33.851 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-09-21T06:08:33.851 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-09-21T06:08:33.851 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:33.851 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:08:33.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.855+0000 7f6ccd800640 1 -- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ca4074880 msgr2=0x7f6ca4076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:33.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.855+0000 7f6ccd800640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ca4074880 0x7f6ca4076d90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009b40 tx=0x7f6cb80091e0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.855+0000 7f6ccd800640 1 -- 172.21.15.50:0/4157961350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cc81527b0 msgr2=0x7f6cc80ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:33.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.855+0000 7f6ccd800640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cc81527b0 0x7f6cc80ccae0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000d980 tx=0x7f6cc000de40 comp rx=0 tx=0).stop 2024-09-21T06:08:33.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.855+0000 7f6ccd800640 1 -- 172.21.15.50:0/4157961350 shutdown_connections 2024-09-21T06:08:33.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.855+0000 7f6ccd800640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ca4074880 0x7f6ca4076d90 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.855+0000 7f6ccd800640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cc81527b0 0x7f6cc80ccae0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.855+0000 7f6ccd800640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80b5c70 0x7f6cc81511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.855+0000 7f6ccd800640 1 --2- 172.21.15.50:0/4157961350 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cc80b4de0 0x7f6cc80af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:33.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.855+0000 7f6ccd800640 1 -- 172.21.15.50:0/4157961350 >> 172.21.15.50:0/4157961350 conn(0x7f6cc80b2040 msgr2=0x7f6cc8159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:33.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.855+0000 7f6ccd800640 1 -- 172.21.15.50:0/4157961350 shutdown_connections 2024-09-21T06:08:33.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:33.855+0000 7f6ccd800640 1 -- 172.21.15.50:0/4157961350 wait complete. 2024-09-21T06:08:33.874 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:08:33.874 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:08:34.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.039+0000 7fd451200640 1 -- 172.21.15.50:0/1881676233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c0b5c70 msgr2=0x7fd44c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:34.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.039+0000 7fd451200640 1 --2- 172.21.15.50:0/1881676233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c0b5c70 0x7fd44c15bb60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009900 tx=0x7fd43c01c820 comp rx=0 tx=0).stop 2024-09-21T06:08:34.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.039+0000 7fd451200640 1 -- 172.21.15.50:0/1881676233 shutdown_connections 2024-09-21T06:08:34.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.039+0000 7fd451200640 1 --2- 172.21.15.50:0/1881676233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd44c1527b0 0x7fd44c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.039+0000 7fd451200640 1 --2- 172.21.15.50:0/1881676233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c0b5c70 0x7fd44c15bb60 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.039+0000 7fd451200640 1 -- 172.21.15.50:0/1881676233 >> 172.21.15.50:0/1881676233 conn(0x7fd44c0b2040 msgr2=0x7fd44c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:34.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.039+0000 7fd451200640 1 -- 172.21.15.50:0/1881676233 shutdown_connections 2024-09-21T06:08:34.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.039+0000 7fd451200640 1 -- 172.21.15.50:0/1881676233 wait complete. 2024-09-21T06:08:34.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.039+0000 7fd451200640 1 Processor -- start 2024-09-21T06:08:34.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.039+0000 7fd451200640 1 -- start start 2024-09-21T06:08:34.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd451200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd44c0b4de0 0x7fd44c0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd451200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c0b5c70 0x7fd44c14bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd451200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd44c1527b0 0x7fd44c14f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd451200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c168c50 con 0x7fd44c0b5c70 2024-09-21T06:08:34.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd451200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c168ad0 con 0x7fd44c1527b0 2024-09-21T06:08:34.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd451200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd44c168dd0 con 0x7fd44c0b4de0 2024-09-21T06:08:34.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd443e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c0b5c70 0x7fd44c14bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd443e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c0b5c70 0x7fd44c14bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38164/0 (socket says 172.21.15.50:38164) 2024-09-21T06:08:34.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd449e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd44c0b4de0 0x7fd44c0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd443e00640 1 -- 172.21.15.50:0/1603414793 learned_addr learned my addr 172.21.15.50:0/1603414793 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:34.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd44a800640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd44c1527b0 0x7fd44c14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd443e00640 1 -- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd44c0b4de0 msgr2=0x7fd44c0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:34.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd443e00640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd44c0b4de0 0x7fd44c0ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd443e00640 1 -- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd44c1527b0 msgr2=0x7fd44c14f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:34.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd443e00640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd44c1527b0 0x7fd44c14f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd443e00640 1 -- 172.21.15.50:0/1603414793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c1f67a0 con 0x7fd44c0b5c70 2024-09-21T06:08:34.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.043+0000 7fd443e00640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c0b5c70 0x7fd44c14bc90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd43400d6a0 tx=0x7fd43400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:34.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.047+0000 7fd442000640 1 -- 172.21.15.50:0/1603414793 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434017070 con 0x7fd44c0b5c70 2024-09-21T06:08:34.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.047+0000 7fd442000640 1 -- 172.21.15.50:0/1603414793 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd434005310 con 0x7fd44c0b5c70 2024-09-21T06:08:34.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.047+0000 7fd442000640 1 -- 172.21.15.50:0/1603414793 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434009820 con 0x7fd44c0b5c70 2024-09-21T06:08:34.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.047+0000 7fd451200640 1 -- 172.21.15.50:0/1603414793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44c1f9d90 con 0x7fd44c0b5c70 2024-09-21T06:08:34.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.047+0000 7fd451200640 1 -- 172.21.15.50:0/1603414793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44c1fa5a0 con 0x7fd44c0b5c70 2024-09-21T06:08:34.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.047+0000 7fd451200640 1 -- 172.21.15.50:0/1603414793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd44c1fa030 con 0x7fd44c0b5c70 2024-09-21T06:08:34.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.047+0000 7fd442000640 1 -- 172.21.15.50:0/1603414793 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fd4340099c0 con 0x7fd44c0b5c70 2024-09-21T06:08:34.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.047+0000 7fd442000640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd428074880 0x7fd428076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.047+0000 7fd442000640 1 -- 172.21.15.50:0/1603414793 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4913+0+0 (secure 0 0 0) 0x7fd434092f90 con 0x7fd44c0b5c70 2024-09-21T06:08:34.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.051+0000 7fd449e00640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd428074880 0x7fd428076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.051+0000 7fd449e00640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd428074880 0x7fd428076d90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0af000 tx=0x7fd43c01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:34.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.055+0000 7fd442000640 1 -- 172.21.15.50:0/1603414793 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd43405f4f0 con 0x7fd44c0b5c70 2024-09-21T06:08:34.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:34.235+0000 7fd451200640 1 -- 172.21.15.50:0/1603414793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd44c15bb60 con 0x7fd44c0b5c70 2024-09-21T06:08:35.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.059+0000 7fd442000640 1 -- 172.21.15.50:0/1603414793 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) ==== 138+0+0 (secure 0 0 0) 0x7fd434063f20 con 0x7fd44c0b5c70 2024-09-21T06:08:35.058 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-09-21T06:08:35.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.063+0000 7fd451200640 1 -- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd428074880 msgr2=0x7fd428076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.063+0000 7fd451200640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd428074880 0x7fd428076d90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0af000 tx=0x7fd43c01b060 comp rx=0 tx=0).stop 2024-09-21T06:08:35.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.063+0000 7fd451200640 1 -- 172.21.15.50:0/1603414793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c0b5c70 msgr2=0x7fd44c14bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.063+0000 7fd451200640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c0b5c70 0x7fd44c14bc90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd43400d6a0 tx=0x7fd43400db60 comp rx=0 tx=0).stop 2024-09-21T06:08:35.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.063+0000 7fd451200640 1 -- 172.21.15.50:0/1603414793 shutdown_connections 2024-09-21T06:08:35.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.063+0000 7fd451200640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd44c1527b0 0x7fd44c14f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.063+0000 7fd451200640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd428074880 0x7fd428076d90 secure :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0af000 tx=0x7fd43c01b060 comp rx=0 tx=0).stop 2024-09-21T06:08:35.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.063+0000 7fd451200640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c0b5c70 0x7fd44c14bc90 secure :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd43400d6a0 tx=0x7fd43400db60 comp rx=0 tx=0).stop 2024-09-21T06:08:35.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.063+0000 7fd451200640 1 --2- 172.21.15.50:0/1603414793 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd44c0b4de0 0x7fd44c0ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.063+0000 7fd451200640 1 -- 172.21.15.50:0/1603414793 >> 172.21.15.50:0/1603414793 conn(0x7fd44c0b2040 msgr2=0x7fd44c150990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:35.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.063+0000 7fd451200640 1 -- 172.21.15.50:0/1603414793 shutdown_connections 2024-09-21T06:08:35.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.063+0000 7fd451200640 1 -- 172.21.15.50:0/1603414793 wait complete. 2024-09-21T06:08:35.092 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:08:35.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.267+0000 7fef63600640 1 -- 172.21.15.50:0/1263462065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef44004c60 msgr2=0x7fef44005090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.267+0000 7fef63600640 1 --2- 172.21.15.50:0/1263462065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef44004c60 0x7fef44005090 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fef50001430 tx=0x7fef5001cd90 comp rx=0 tx=0).stop 2024-09-21T06:08:35.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.267+0000 7fef61600640 1 -- 172.21.15.50:0/1263462065 reap_dead start 2024-09-21T06:08:35.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.267+0000 7fef63600640 1 -- 172.21.15.50:0/1263462065 shutdown_connections 2024-09-21T06:08:35.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.267+0000 7fef63600640 1 -- 172.21.15.50:0/1263462065 >> 172.21.15.50:0/1263462065 conn(0x7fef5c0d06e0 msgr2=0x7fef5c0d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:35.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.267+0000 7fef63600640 1 -- 172.21.15.50:0/1263462065 shutdown_connections 2024-09-21T06:08:35.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.267+0000 7fef63600640 1 -- 172.21.15.50:0/1263462065 wait complete. 2024-09-21T06:08:35.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.267+0000 7fef63600640 1 Processor -- start 2024-09-21T06:08:35.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.267+0000 7fef63600640 1 -- start start 2024-09-21T06:08:35.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef63600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef44004c60 0x7fef5c1fa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef63600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c141450 0x7fef5c140c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef63600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef5c144710 0x7fef5c144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef63600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c168950 con 0x7fef5c144710 2024-09-21T06:08:35.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef63600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c1687d0 con 0x7fef5c141450 2024-09-21T06:08:35.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef63600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fef5c168ad0 con 0x7fef44004c60 2024-09-21T06:08:35.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef61600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef5c144710 0x7fef5c144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef61600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef5c144710 0x7fef5c144b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38188/0 (socket says 172.21.15.50:38188) 2024-09-21T06:08:35.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef5be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c141450 0x7fef5c140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef61600640 1 -- 172.21.15.50:0/272843178 learned_addr learned my addr 172.21.15.50:0/272843178 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:35.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef60c00640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef44004c60 0x7fef5c1fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef61600640 1 -- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef44004c60 msgr2=0x7fef5c1fa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef61600640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef44004c60 0x7fef5c1fa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef61600640 1 -- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c141450 msgr2=0x7fef5c140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef61600640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c141450 0x7fef5c140c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef61600640 1 -- 172.21.15.50:0/272843178 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c145060 con 0x7fef5c144710 2024-09-21T06:08:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef61600640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef5c144710 0x7fef5c144b20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fef5c146800 tx=0x7fef5400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef58c00640 1 -- 172.21.15.50:0/272843178 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54004d30 con 0x7fef5c144710 2024-09-21T06:08:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef58c00640 1 -- 172.21.15.50:0/272843178 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef5400e5c0 con 0x7fef5c144710 2024-09-21T06:08:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef58c00640 1 -- 172.21.15.50:0/272843178 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5400e9d0 con 0x7fef5c144710 2024-09-21T06:08:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef63600640 1 -- 172.21.15.50:0/272843178 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5c162670 con 0x7fef5c144710 2024-09-21T06:08:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef63600640 1 -- 172.21.15.50:0/272843178 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5c162c30 con 0x7fef5c144710 2024-09-21T06:08:35.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.271+0000 7fef63600640 1 -- 172.21.15.50:0/272843178 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5c160270 con 0x7fef5c144710 2024-09-21T06:08:35.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.275+0000 7fef58c00640 1 -- 172.21.15.50:0/272843178 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fef5400eb70 con 0x7fef5c144710 2024-09-21T06:08:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.275+0000 7fef58c00640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef3c0748b0 0x7fef3c076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.275+0000 7fef58c00640 1 -- 172.21.15.50:0/272843178 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5283+0+0 (secure 0 0 0) 0x7fef54094170 con 0x7fef5c144710 2024-09-21T06:08:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.275+0000 7fef60c00640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef3c0748b0 0x7fef3c076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.275+0000 7fef60c00640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef3c0748b0 0x7fef3c076dc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fef500ab000 tx=0x7fef50019b80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:35.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.279+0000 7fef58c00640 1 -- 172.21.15.50:0/272843178 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fef5c145060 con 0x7fef5c144710 2024-09-21T06:08:35.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:35.455+0000 7fef63600640 1 -- 172.21.15.50:0/272843178 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fef5c145c20 con 0x7fef5c144710 2024-09-21T06:08:36.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.067+0000 7fef58c00640 1 -- 172.21.15.50:0/272843178 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) ==== 170+0+0 (secure 0 0 0) 0x7fef54060620 con 0x7fef5c144710 2024-09-21T06:08:36.065 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-09-21T06:08:36.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.067+0000 7fef63600640 1 -- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef3c0748b0 msgr2=0x7fef3c076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:36.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.067+0000 7fef63600640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef3c0748b0 0x7fef3c076dc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fef500ab000 tx=0x7fef50019b80 comp rx=0 tx=0).stop 2024-09-21T06:08:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.067+0000 7fef63600640 1 -- 172.21.15.50:0/272843178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef5c144710 msgr2=0x7fef5c144b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.067+0000 7fef63600640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef5c144710 0x7fef5c144b20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fef5c146800 tx=0x7fef5400f790 comp rx=0 tx=0).stop 2024-09-21T06:08:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.067+0000 7fef63600640 1 -- 172.21.15.50:0/272843178 shutdown_connections 2024-09-21T06:08:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.067+0000 7fef63600640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef3c0748b0 0x7fef3c076dc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.067+0000 7fef63600640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef5c144710 0x7fef5c144b20 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.067+0000 7fef63600640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c141450 0x7fef5c140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.067+0000 7fef63600640 1 --2- 172.21.15.50:0/272843178 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef44004c60 0x7fef5c1fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.067+0000 7fef63600640 1 -- 172.21.15.50:0/272843178 >> 172.21.15.50:0/272843178 conn(0x7fef5c0d06e0 msgr2=0x7fef5c141de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:36.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.071+0000 7fef63600640 1 -- 172.21.15.50:0/272843178 shutdown_connections 2024-09-21T06:08:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.071+0000 7fef63600640 1 -- 172.21.15.50:0/272843178 wait complete. 2024-09-21T06:08:36.113 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:08:36.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.275+0000 7f9111000640 1 -- 172.21.15.50:0/1505702523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910c15d280 msgr2=0x7f910c15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:36.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.275+0000 7f9111000640 1 --2- 172.21.15.50:0/1505702523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910c15d280 0x7f910c15d690 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f90f400a950 tx=0x7f90f401db70 comp rx=0 tx=0).stop 2024-09-21T06:08:36.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.275+0000 7f9111000640 1 -- 172.21.15.50:0/1505702523 shutdown_connections 2024-09-21T06:08:36.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.275+0000 7f9111000640 1 --2- 172.21.15.50:0/1505702523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910c15d280 0x7f910c15d690 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:36.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.275+0000 7f9111000640 1 -- 172.21.15.50:0/1505702523 >> 172.21.15.50:0/1505702523 conn(0x7f910c0b2040 msgr2=0x7f910c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:36.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.275+0000 7f9111000640 1 -- 172.21.15.50:0/1505702523 shutdown_connections 2024-09-21T06:08:36.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.275+0000 7f9111000640 1 -- 172.21.15.50:0/1505702523 wait complete. 2024-09-21T06:08:36.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9111000640 1 Processor -- start 2024-09-21T06:08:36.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9111000640 1 -- start start 2024-09-21T06:08:36.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9111000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f910c15d280 0x7f910c163e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:36.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9111000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910c15dbd0 0x7f910c13d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:36.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9111000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f910c15e8c0 0x7f910c13fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:36.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9111000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f910c1692a0 con 0x7f910c15dbd0 2024-09-21T06:08:36.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9111000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f910c169120 con 0x7f910c15d280 2024-09-21T06:08:36.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9111000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f910c169420 con 0x7f910c15e8c0 2024-09-21T06:08:36.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9101400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910c15dbd0 0x7f910c13d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:36.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9101400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910c15dbd0 0x7f910c13d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38210/0 (socket says 172.21.15.50:38210) 2024-09-21T06:08:36.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9101400640 1 -- 172.21.15.50:0/3098421452 learned_addr learned my addr 172.21.15.50:0/3098421452 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:36.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9101400640 1 -- 172.21.15.50:0/3098421452 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f910c15e8c0 msgr2=0x7f910c13fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:36.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9101400640 1 --2- 172.21.15.50:0/3098421452 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f910c15e8c0 0x7f910c13fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:36.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9101400640 1 -- 172.21.15.50:0/3098421452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f910c15d280 msgr2=0x7f910c163e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:36.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9101400640 1 --2- 172.21.15.50:0/3098421452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f910c15d280 0x7f910c163e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:36.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9101400640 1 -- 172.21.15.50:0/3098421452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c13caf0 con 0x7f910c15dbd0 2024-09-21T06:08:36.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9101400640 1 --2- 172.21.15.50:0/3098421452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910c15dbd0 0x7f910c13d590 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f90f400a8a0 tx=0x7f90f4003830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:36.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9102a00640 1 -- 172.21.15.50:0/3098421452 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f40b3040 con 0x7f910c15dbd0 2024-09-21T06:08:36.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9102a00640 1 -- 172.21.15.50:0/3098421452 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90f4003b10 con 0x7f910c15dbd0 2024-09-21T06:08:36.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9111000640 1 -- 172.21.15.50:0/3098421452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910c164340 con 0x7f910c15dbd0 2024-09-21T06:08:36.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9102a00640 1 -- 172.21.15.50:0/3098421452 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f40ae440 con 0x7f910c15dbd0 2024-09-21T06:08:36.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.279+0000 7f9111000640 1 -- 172.21.15.50:0/3098421452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910c13cd90 con 0x7f910c15dbd0 2024-09-21T06:08:36.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.283+0000 7f9111000640 1 -- 172.21.15.50:0/3098421452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910c0b1950 con 0x7f910c15dbd0 2024-09-21T06:08:36.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.283+0000 7f9102a00640 1 -- 172.21.15.50:0/3098421452 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f90f40ae5e0 con 0x7f910c15dbd0 2024-09-21T06:08:36.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.283+0000 7f9102a00640 1 --2- 172.21.15.50:0/3098421452 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f90e8074880 0x7f90e8076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:36.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.283+0000 7f9102a00640 1 -- 172.21.15.50:0/3098421452 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5665+0+0 (secure 0 0 0) 0x7f90f412dcf0 con 0x7f910c15dbd0 2024-09-21T06:08:36.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.283+0000 7f9109e00640 1 --2- 172.21.15.50:0/3098421452 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f90e8074880 0x7f90e8076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:36.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.287+0000 7f9109e00640 1 --2- 172.21.15.50:0/3098421452 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f90e8074880 0x7f90e8076d90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f90fc007fd0 tx=0x7f90fc007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:36.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.287+0000 7f9102a00640 1 -- 172.21.15.50:0/3098421452 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f90f40bf060 con 0x7f910c15dbd0 2024-09-21T06:08:36.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:36.503+0000 7f9111000640 1 -- 172.21.15.50:0/3098421452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f910c15d690 con 0x7f910c15dbd0 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.067+0000 7f9102a00640 1 -- 172.21.15.50:0/3098421452 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v61) ==== 174+0+0 (secure 0 0 0) 0x7f90f4017760 con 0x7f910c15dbd0 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 14 and data pool 15 2024-09-21T06:08:37.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.075+0000 7f9111000640 1 -- 172.21.15.50:0/3098421452 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f90e8074880 msgr2=0x7f90e8076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.075+0000 7f9111000640 1 --2- 172.21.15.50:0/3098421452 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f90e8074880 0x7f90e8076d90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f90fc007fd0 tx=0x7f90fc007980 comp rx=0 tx=0).stop 2024-09-21T06:08:37.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.075+0000 7f9111000640 1 -- 172.21.15.50:0/3098421452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910c15dbd0 msgr2=0x7f910c13d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.075+0000 7f9111000640 1 --2- 172.21.15.50:0/3098421452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910c15dbd0 0x7f910c13d590 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f90f400a8a0 tx=0x7f90f4003830 comp rx=0 tx=0).stop 2024-09-21T06:08:37.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.075+0000 7f9111000640 1 -- 172.21.15.50:0/3098421452 shutdown_connections 2024-09-21T06:08:37.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.075+0000 7f9111000640 1 --2- 172.21.15.50:0/3098421452 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f910c15e8c0 0x7f910c13fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.075+0000 7f9111000640 1 --2- 172.21.15.50:0/3098421452 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f90e8074880 0x7f90e8076d90 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.075+0000 7f9111000640 1 --2- 172.21.15.50:0/3098421452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910c15dbd0 0x7f910c13d590 secure :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f90f400a8a0 tx=0x7f90f4003830 comp rx=0 tx=0).stop 2024-09-21T06:08:37.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.075+0000 7f9111000640 1 --2- 172.21.15.50:0/3098421452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f910c15d280 0x7f910c163e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.075+0000 7f9111000640 1 -- 172.21.15.50:0/3098421452 >> 172.21.15.50:0/3098421452 conn(0x7f910c0b2040 msgr2=0x7f910c158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:37.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.079+0000 7f9111000640 1 -- 172.21.15.50:0/3098421452 shutdown_connections 2024-09-21T06:08:37.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.079+0000 7f9111000640 1 -- 172.21.15.50:0/3098421452 wait complete. 2024-09-21T06:08:37.127 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:08:37.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.287+0000 7ff60da00640 1 -- 172.21.15.50:0/1476029050 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6080cdbb0 msgr2=0x7ff60815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.287+0000 7ff60da00640 1 --2- 172.21.15.50:0/1476029050 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6080cdbb0 0x7ff60815ebf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff600009c80 tx=0x7ff60001d640 comp rx=0 tx=0).stop 2024-09-21T06:08:37.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.287+0000 7ff60da00640 1 -- 172.21.15.50:0/1476029050 shutdown_connections 2024-09-21T06:08:37.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.287+0000 7ff60da00640 1 --2- 172.21.15.50:0/1476029050 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6080cdbb0 0x7ff60815ebf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.287+0000 7ff60da00640 1 --2- 172.21.15.50:0/1476029050 >> v2:172.21.15.88:3300/0 conn(0x7ff60815aa50 0x7ff6080cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.287+0000 7ff60da00640 1 -- 172.21.15.50:0/1476029050 >> 172.21.15.50:0/1476029050 conn(0x7ff6080b2040 msgr2=0x7ff6080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:37.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.291+0000 7ff60da00640 1 -- 172.21.15.50:0/1476029050 shutdown_connections 2024-09-21T06:08:37.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.291+0000 7ff60da00640 1 -- 172.21.15.50:0/1476029050 wait complete. 2024-09-21T06:08:37.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.291+0000 7ff60da00640 1 Processor -- start 2024-09-21T06:08:37.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.291+0000 7ff60da00640 1 -- start start 2024-09-21T06:08:37.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.291+0000 7ff60da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6080cd7a0 0x7ff6081541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.291+0000 7ff60da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6080cdbb0 0x7ff6080ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.291+0000 7ff60da00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff60815aa50 0x7ff608151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.291+0000 7ff60da00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff608169110 con 0x7ff6080cdbb0 2024-09-21T06:08:37.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.291+0000 7ff60da00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff608168f90 con 0x7ff6080cd7a0 2024-09-21T06:08:37.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.291+0000 7ff60da00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff608169290 con 0x7ff60815aa50 2024-09-21T06:08:37.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5fe000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6080cdbb0 0x7ff6080ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5fe000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6080cdbb0 0x7ff6080ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50022/0 (socket says 172.21.15.50:50022) 2024-09-21T06:08:37.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5fe000640 1 -- 172.21.15.50:0/1825076065 learned_addr learned my addr 172.21.15.50:0/1825076065 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:37.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff606a00640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6080cd7a0 0x7ff6081541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff607400640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff60815aa50 0x7ff608151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5fe000640 1 -- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff60815aa50 msgr2=0x7ff608151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5fe000640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff60815aa50 0x7ff608151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5fe000640 1 -- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6080cd7a0 msgr2=0x7ff6081541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5fe000640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6080cd7a0 0x7ff6081541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5fe000640 1 -- 172.21.15.50:0/1825076065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6080adfd0 con 0x7ff6080cdbb0 2024-09-21T06:08:37.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5fe000640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6080cdbb0 0x7ff6080ada90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c3c0 tx=0x7ff5f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:37.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5ff400640 1 -- 172.21.15.50:0/1825076065 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0017070 con 0x7ff6080cdbb0 2024-09-21T06:08:37.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5ff400640 1 -- 172.21.15.50:0/1825076065 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5f00053c0 con 0x7ff6080cdbb0 2024-09-21T06:08:37.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5ff400640 1 -- 172.21.15.50:0/1825076065 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0003860 con 0x7ff6080cdbb0 2024-09-21T06:08:37.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff60da00640 1 -- 172.21.15.50:0/1825076065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff608152230 con 0x7ff6080cdbb0 2024-09-21T06:08:37.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff60da00640 1 -- 172.21.15.50:0/1825076065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff608153830 con 0x7ff6080cdbb0 2024-09-21T06:08:37.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff60da00640 1 -- 172.21.15.50:0/1825076065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60815da70 con 0x7ff6080cdbb0 2024-09-21T06:08:37.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.295+0000 7ff5ff400640 1 -- 172.21.15.50:0/1825076065 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7ff5f001a070 con 0x7ff6080cdbb0 2024-09-21T06:08:37.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.299+0000 7ff5ff400640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5e40748b0 0x7ff5e4076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.299+0000 7ff5ff400640 1 -- 172.21.15.50:0/1825076065 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5765+0+0 (secure 0 0 0) 0x7ff5f00921f0 con 0x7ff6080cdbb0 2024-09-21T06:08:37.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.299+0000 7ff606a00640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5e40748b0 0x7ff5e4076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.299+0000 7ff606a00640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5e40748b0 0x7ff5e4076dc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009b40 tx=0x7ff5f80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:37.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.303+0000 7ff5ff400640 1 -- 172.21.15.50:0/1825076065 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff5f005e6b0 con 0x7ff6080cdbb0 2024-09-21T06:08:37.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 -- 172.21.15.50:0/1825076065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff60815d790 con 0x7ff6080cdbb0 2024-09-21T06:08:37.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff5ff400640 1 -- 172.21.15.50:0/1825076065 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16721 (secure 0 0 0) 0x7ff5f00630e0 con 0x7ff6080cdbb0 2024-09-21T06:08:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:37.482 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":99,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:08:36.508684+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:08:34.249299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-21T06:08:35.470212+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:08:37.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 -- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5e40748b0 msgr2=0x7ff5e4076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5e40748b0 0x7ff5e4076dc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009b40 tx=0x7ff5f80091e0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 -- 172.21.15.50:0/1825076065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6080cdbb0 msgr2=0x7ff6080ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6080cdbb0 0x7ff6080ada90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c3c0 tx=0x7ff5f000c880 comp rx=0 tx=0).stop 2024-09-21T06:08:37.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 -- 172.21.15.50:0/1825076065 shutdown_connections 2024-09-21T06:08:37.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff60815aa50 0x7ff608151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5e40748b0 0x7ff5e4076dc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6080cdbb0 0x7ff6080ada90 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 --2- 172.21.15.50:0/1825076065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6080cd7a0 0x7ff6081541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 -- 172.21.15.50:0/1825076065 >> 172.21.15.50:0/1825076065 conn(0x7ff6080b2040 msgr2=0x7ff60815c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:37.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 -- 172.21.15.50:0/1825076065 shutdown_connections 2024-09-21T06:08:37.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.483+0000 7ff60da00640 1 -- 172.21.15.50:0/1825076065 wait complete. 2024-09-21T06:08:37.505 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:08:37.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.667+0000 7f32b4600640 1 -- 172.21.15.50:0/2800308103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3294004810 msgr2=0x7f3294004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.667+0000 7f32b4600640 1 --2- 172.21.15.50:0/2800308103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3294004810 0x7f3294004c20 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f32ac160100 tx=0x7f329801d240 comp rx=0 tx=0).stop 2024-09-21T06:08:37.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.667+0000 7f32b2600640 1 -- 172.21.15.50:0/2800308103 reap_dead start 2024-09-21T06:08:37.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.667+0000 7f32b4600640 1 -- 172.21.15.50:0/2800308103 shutdown_connections 2024-09-21T06:08:37.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.667+0000 7f32b4600640 1 -- 172.21.15.50:0/2800308103 >> 172.21.15.50:0/2800308103 conn(0x7f32ac0b2040 msgr2=0x7f32ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:37.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.667+0000 7f32b4600640 1 -- 172.21.15.50:0/2800308103 shutdown_connections 2024-09-21T06:08:37.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.667+0000 7f32b4600640 1 -- 172.21.15.50:0/2800308103 wait complete. 2024-09-21T06:08:37.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.667+0000 7f32b4600640 1 Processor -- start 2024-09-21T06:08:37.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b4600640 1 -- start start 2024-09-21T06:08:37.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b4600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3294004810 0x7f32ac1434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b4600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f32ac142fc0 0x7f32ac13db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b4600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32ac144340 0x7f32ac144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b4600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac167870 con 0x7f32ac144340 2024-09-21T06:08:37.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b4600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac1676f0 con 0x7f3294004810 2024-09-21T06:08:37.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b4600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac1679f0 con 0x7f32ac142fc0 2024-09-21T06:08:37.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b2600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32ac144340 0x7f32ac144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b2600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32ac144340 0x7f32ac144750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50032/0 (socket says 172.21.15.50:50032) 2024-09-21T06:08:37.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b2600640 1 -- 172.21.15.50:0/313234822 learned_addr learned my addr 172.21.15.50:0/313234822 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:37.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b1c00640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3294004810 0x7f32ac1434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b1200640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f32ac142fc0 0x7f32ac13db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b2600640 1 -- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f32ac142fc0 msgr2=0x7f32ac13db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b2600640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f32ac142fc0 0x7f32ac13db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b2600640 1 -- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3294004810 msgr2=0x7f32ac1434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b2600640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3294004810 0x7f32ac1434b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b2600640 1 -- 172.21.15.50:0/313234822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac144c90 con 0x7f32ac144340 2024-09-21T06:08:37.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b1c00640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3294004810 0x7f32ac1434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:37.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b2600640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32ac144340 0x7f32ac144750 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0b6130 tx=0x7f32a400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:37.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32a9600640 1 -- 172.21.15.50:0/313234822 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4017070 con 0x7f32ac144340 2024-09-21T06:08:37.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32a9600640 1 -- 172.21.15.50:0/313234822 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a4015070 con 0x7f32ac144340 2024-09-21T06:08:37.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b4600640 1 -- 172.21.15.50:0/313234822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac1fc470 con 0x7f32ac144340 2024-09-21T06:08:37.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32a9600640 1 -- 172.21.15.50:0/313234822 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a400eea0 con 0x7f32ac144340 2024-09-21T06:08:37.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b4600640 1 -- 172.21.15.50:0/313234822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac1fca30 con 0x7f32ac144340 2024-09-21T06:08:37.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.671+0000 7f32b4600640 1 -- 172.21.15.50:0/313234822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ac072510 con 0x7f32ac144340 2024-09-21T06:08:37.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.675+0000 7f32a9600640 1 -- 172.21.15.50:0/313234822 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f32a4003be0 con 0x7f32ac144340 2024-09-21T06:08:37.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.675+0000 7f32a9600640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f328c074880 0x7f328c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.675+0000 7f32a9600640 1 -- 172.21.15.50:0/313234822 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5765+0+0 (secure 0 0 0) 0x7f32a4093790 con 0x7f32ac144340 2024-09-21T06:08:37.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.675+0000 7f32b1c00640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f328c074880 0x7f328c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.675+0000 7f32b1c00640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f328c074880 0x7f328c076d90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f32a0001d60 tx=0x7f32a0001c10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:37.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.679+0000 7f32a9600640 1 -- 172.21.15.50:0/313234822 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f32ac144c90 con 0x7f32ac144340 2024-09-21T06:08:37.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.859+0000 7f32b4600640 1 -- 172.21.15.50:0/313234822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32ac165680 con 0x7f32ac144340 2024-09-21T06:08:37.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.859+0000 7f32a9600640 1 -- 172.21.15.50:0/313234822 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16721 (secure 0 0 0) 0x7f32a40602a0 con 0x7f32ac144340 2024-09-21T06:08:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:37.859 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":99,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:08:36.508684+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:08:34.249299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-21T06:08:35.470212+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:08:37.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.863+0000 7f32b4600640 1 -- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f328c074880 msgr2=0x7f328c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.863+0000 7f32b4600640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f328c074880 0x7f328c076d90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f32a0001d60 tx=0x7f32a0001c10 comp rx=0 tx=0).stop 2024-09-21T06:08:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.863+0000 7f32b4600640 1 -- 172.21.15.50:0/313234822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32ac144340 msgr2=0x7f32ac144750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.863+0000 7f32b4600640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32ac144340 0x7f32ac144750 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0b6130 tx=0x7f32a400e720 comp rx=0 tx=0).stop 2024-09-21T06:08:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.863+0000 7f32b4600640 1 -- 172.21.15.50:0/313234822 shutdown_connections 2024-09-21T06:08:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.863+0000 7f32b4600640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f328c074880 0x7f328c076d90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.863+0000 7f32b4600640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32ac144340 0x7f32ac144750 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.863+0000 7f32b4600640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f32ac142fc0 0x7f32ac13db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.863+0000 7f32b4600640 1 --2- 172.21.15.50:0/313234822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3294004810 0x7f32ac1434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.863+0000 7f32b4600640 1 -- 172.21.15.50:0/313234822 >> 172.21.15.50:0/313234822 conn(0x7f32ac0b2040 msgr2=0x7f32ac0b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.863+0000 7f32b4600640 1 -- 172.21.15.50:0/313234822 shutdown_connections 2024-09-21T06:08:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:37.863+0000 7f32b4600640 1 -- 172.21.15.50:0/313234822 wait complete. 2024-09-21T06:08:37.882 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:08:38.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.047+0000 7ff4d2200640 1 -- 172.21.15.50:0/4060072411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b0003fa0 msgr2=0x7ff4b0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:38.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.047+0000 7ff4d2200640 1 --2- 172.21.15.50:0/4060072411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b0003fa0 0x7ff4b0004810 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0b1740 tx=0x7ff4b401c880 comp rx=0 tx=0).stop 2024-09-21T06:08:38.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.047+0000 7ff4cbe00640 1 -- 172.21.15.50:0/4060072411 reap_dead start 2024-09-21T06:08:38.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.047+0000 7ff4d2200640 1 -- 172.21.15.50:0/4060072411 shutdown_connections 2024-09-21T06:08:38.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.047+0000 7ff4d2200640 1 -- 172.21.15.50:0/4060072411 >> 172.21.15.50:0/4060072411 conn(0x7ff4cc0708c0 msgr2=0x7ff4cc070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:38.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.047+0000 7ff4d2200640 1 -- 172.21.15.50:0/4060072411 shutdown_connections 2024-09-21T06:08:38.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.047+0000 7ff4d2200640 1 -- 172.21.15.50:0/4060072411 wait complete. 2024-09-21T06:08:38.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.047+0000 7ff4d2200640 1 Processor -- start 2024-09-21T06:08:38.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.047+0000 7ff4d2200640 1 -- start start 2024-09-21T06:08:38.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4d2200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b0003fa0 0x7ff4cc167440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4d2200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff4cc155410 0x7ff4cc145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4d2200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4cc15bac0 0x7ff4cc15bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4d2200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc1693c0 con 0x7ff4b0003fa0 2024-09-21T06:08:38.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4d2200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc169240 con 0x7ff4cc15bac0 2024-09-21T06:08:38.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4d2200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff4cc169540 con 0x7ff4cc155410 2024-09-21T06:08:38.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4cb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b0003fa0 0x7ff4cc167440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4cb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b0003fa0 0x7ff4cc167440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50048/0 (socket says 172.21.15.50:50048) 2024-09-21T06:08:38.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4cb400640 1 -- 172.21.15.50:0/2084704844 learned_addr learned my addr 172.21.15.50:0/2084704844 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:38.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4c3e00640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff4cc155410 0x7ff4cc145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4cbe00640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4cc15bac0 0x7ff4cc15bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4cb400640 1 -- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff4cc155410 msgr2=0x7ff4cc145f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:38.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4cb400640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff4cc155410 0x7ff4cc145f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4cb400640 1 -- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4cc15bac0 msgr2=0x7ff4cc15bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:38.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4cb400640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4cc15bac0 0x7ff4cc15bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4cb400640 1 -- 172.21.15.50:0/2084704844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc15c410 con 0x7ff4b0003fa0 2024-09-21T06:08:38.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4c3e00640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff4cc155410 0x7ff4cc145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:38.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4cbe00640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4cc15bac0 0x7ff4cc15bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:38.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4cb400640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b0003fa0 0x7ff4cc167440 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc011ae0 tx=0x7ff4bc009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:38.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4c3400640 1 -- 172.21.15.50:0/2084704844 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc004460 con 0x7ff4b0003fa0 2024-09-21T06:08:38.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4c3400640 1 -- 172.21.15.50:0/2084704844 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4bc00f9d0 con 0x7ff4b0003fa0 2024-09-21T06:08:38.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4c3400640 1 -- 172.21.15.50:0/2084704844 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc00fe20 con 0x7ff4b0003fa0 2024-09-21T06:08:38.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4d2200640 1 -- 172.21.15.50:0/2084704844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc1fc660 con 0x7ff4b0003fa0 2024-09-21T06:08:38.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4d2200640 1 -- 172.21.15.50:0/2084704844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc1fcc20 con 0x7ff4b0003fa0 2024-09-21T06:08:38.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.051+0000 7ff4d2200640 1 -- 172.21.15.50:0/2084704844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4cc15cf90 con 0x7ff4b0003fa0 2024-09-21T06:08:38.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.055+0000 7ff4c3400640 1 -- 172.21.15.50:0/2084704844 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7ff4bc003600 con 0x7ff4b0003fa0 2024-09-21T06:08:38.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.055+0000 7ff4c3400640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff4a80748b0 0x7ff4a8076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.055+0000 7ff4c3400640 1 -- 172.21.15.50:0/2084704844 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5765+0+0 (secure 0 0 0) 0x7ff4bc093fe0 con 0x7ff4b0003fa0 2024-09-21T06:08:38.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.055+0000 7ff4c3e00640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff4a80748b0 0x7ff4a8076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.055+0000 7ff4c3e00640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff4a80748b0 0x7ff4a8076dc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0751f0 tx=0x7ff4b4019620 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:38.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.059+0000 7ff4c3400640 1 -- 172.21.15.50:0/2084704844 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff4cc15c410 con 0x7ff4b0003fa0 2024-09-21T06:08:38.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:38.283+0000 7ff4d2200640 1 -- 172.21.15.50:0/2084704844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff4cc0d2260 con 0x7ff4b0003fa0 2024-09-21T06:08:39.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.099+0000 7ff4c3400640 1 -- 172.21.15.50:0/2084704844 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) ==== 118+0+0 (secure 0 0 0) 0x7ff4bc060bb0 con 0x7ff4b0003fa0 2024-09-21T06:08:39.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.103+0000 7ff4d2200640 1 -- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff4a80748b0 msgr2=0x7ff4a8076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:39.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.103+0000 7ff4d2200640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff4a80748b0 0x7ff4a8076dc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0751f0 tx=0x7ff4b4019620 comp rx=0 tx=0).stop 2024-09-21T06:08:39.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.103+0000 7ff4d2200640 1 -- 172.21.15.50:0/2084704844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b0003fa0 msgr2=0x7ff4cc167440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:39.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.103+0000 7ff4d2200640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b0003fa0 0x7ff4cc167440 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc011ae0 tx=0x7ff4bc009ea0 comp rx=0 tx=0).stop 2024-09-21T06:08:39.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.103+0000 7ff4d2200640 1 -- 172.21.15.50:0/2084704844 shutdown_connections 2024-09-21T06:08:39.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.103+0000 7ff4d2200640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4cc15bac0 0x7ff4cc15bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.103+0000 7ff4d2200640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff4cc155410 0x7ff4cc145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.103+0000 7ff4d2200640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff4a80748b0 0x7ff4a8076dc0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.103+0000 7ff4d2200640 1 --2- 172.21.15.50:0/2084704844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b0003fa0 0x7ff4cc167440 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.103+0000 7ff4d2200640 1 -- 172.21.15.50:0/2084704844 >> 172.21.15.50:0/2084704844 conn(0x7ff4cc0708c0 msgr2=0x7ff4cc15f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.103+0000 7ff4d2200640 1 -- 172.21.15.50:0/2084704844 shutdown_connections 2024-09-21T06:08:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.103+0000 7ff4d2200640 1 -- 172.21.15.50:0/2084704844 wait complete. 2024-09-21T06:08:39.126 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:08:39.126 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:08:39.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.291+0000 7f4845600640 1 -- 172.21.15.50:0/231010872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4824005360 msgr2=0x7f482400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:39.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.291+0000 7f4845600640 1 --2- 172.21.15.50:0/231010872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4824005360 0x7f482400b9d0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f4840074950 tx=0x7f483801f9e0 comp rx=0 tx=0).stop 2024-09-21T06:08:39.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.291+0000 7f4845600640 1 -- 172.21.15.50:0/231010872 shutdown_connections 2024-09-21T06:08:39.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.291+0000 7f4845600640 1 --2- 172.21.15.50:0/231010872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4824005360 0x7f482400b9d0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:39.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.291+0000 7f4845600640 1 -- 172.21.15.50:0/231010872 >> 172.21.15.50:0/231010872 conn(0x7f48400b2040 msgr2=0x7f48400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:39.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.291+0000 7f4845600640 1 -- 172.21.15.50:0/231010872 shutdown_connections 2024-09-21T06:08:39.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.291+0000 7f4845600640 1 -- 172.21.15.50:0/231010872 wait complete. 2024-09-21T06:08:39.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4845600640 1 Processor -- start 2024-09-21T06:08:39.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4845600640 1 -- start start 2024-09-21T06:08:39.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4845600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4824005360 0x7f484014a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:39.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4845600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48400af6e0 0x7f4840151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:39.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4845600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f484014ad40 0x7f48401515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:39.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4845600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f48400751c0 con 0x7f484014ad40 2024-09-21T06:08:39.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4845600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4840075040 con 0x7f4824005360 2024-09-21T06:08:39.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4845600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4840075340 con 0x7f48400af6e0 2024-09-21T06:08:39.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f483f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f484014ad40 0x7f48401515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:39.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f483f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f484014ad40 0x7f48401515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50068/0 (socket says 172.21.15.50:50068) 2024-09-21T06:08:39.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f483f400640 1 -- 172.21.15.50:0/1927465167 learned_addr learned my addr 172.21.15.50:0/1927465167 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:39.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f483ea00640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4824005360 0x7f484014a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:39.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4837e00640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48400af6e0 0x7f4840151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:39.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f483f400640 1 -- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48400af6e0 msgr2=0x7f4840151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:39.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f483f400640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48400af6e0 0x7f4840151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:39.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f483f400640 1 -- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4824005360 msgr2=0x7f484014a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:39.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f483f400640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4824005360 0x7f484014a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:39.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f483f400640 1 -- 172.21.15.50:0/1927465167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48400af0a0 con 0x7f484014ad40 2024-09-21T06:08:39.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f483f400640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f484014ad40 0x7f48401515b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f483809c2d0 tx=0x7f4838001c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:39.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4836a00640 1 -- 172.21.15.50:0/1927465167 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838002080 con 0x7f484014ad40 2024-09-21T06:08:39.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4836a00640 1 -- 172.21.15.50:0/1927465167 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48380b7020 con 0x7f484014ad40 2024-09-21T06:08:39.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4836a00640 1 -- 172.21.15.50:0/1927465167 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48380a95b0 con 0x7f484014ad40 2024-09-21T06:08:39.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4845600640 1 -- 172.21.15.50:0/1927465167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4840154c00 con 0x7f484014ad40 2024-09-21T06:08:39.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.295+0000 7f4845600640 1 -- 172.21.15.50:0/1927465167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4840154400 con 0x7f484014ad40 2024-09-21T06:08:39.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.299+0000 7f4845600640 1 -- 172.21.15.50:0/1927465167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48400b53b0 con 0x7f484014ad40 2024-09-21T06:08:39.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.299+0000 7f4836a00640 1 -- 172.21.15.50:0/1927465167 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f48380060d0 con 0x7f484014ad40 2024-09-21T06:08:39.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.299+0000 7f4836a00640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f481c074880 0x7f481c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:39.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.299+0000 7f4836a00640 1 -- 172.21.15.50:0/1927465167 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f4838130bd0 con 0x7f484014ad40 2024-09-21T06:08:39.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.299+0000 7f483ea00640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f481c074880 0x7f481c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:39.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.303+0000 7f483ea00640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f481c074880 0x7f481c076d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4830000c10 tx=0x7f4830006a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:39.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.303+0000 7f4836a00640 1 -- 172.21.15.50:0/1927465167 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f48380c1030 con 0x7f484014ad40 2024-09-21T06:08:39.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:39.515+0000 7f4845600640 1 -- 172.21.15.50:0/1927465167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f48400b24f0 con 0x7f484014ad40 2024-09-21T06:08:40.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.103+0000 7f4836a00640 1 -- 172.21.15.50:0/1927465167 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) ==== 122+0+0 (secure 0 0 0) 0x7f4838019790 con 0x7f484014ad40 2024-09-21T06:08:40.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.107+0000 7f4845600640 1 -- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f481c074880 msgr2=0x7f481c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.107+0000 7f4845600640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f481c074880 0x7f481c076d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4830000c10 tx=0x7f4830006a30 comp rx=0 tx=0).stop 2024-09-21T06:08:40.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.107+0000 7f4845600640 1 -- 172.21.15.50:0/1927465167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f484014ad40 msgr2=0x7f48401515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.107+0000 7f4845600640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f484014ad40 0x7f48401515b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f483809c2d0 tx=0x7f4838001c60 comp rx=0 tx=0).stop 2024-09-21T06:08:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.107+0000 7f4845600640 1 -- 172.21.15.50:0/1927465167 shutdown_connections 2024-09-21T06:08:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.107+0000 7f4845600640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f481c074880 0x7f481c076d90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.107+0000 7f4845600640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f484014ad40 0x7f48401515b0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.107+0000 7f4845600640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48400af6e0 0x7f4840151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.107+0000 7f4845600640 1 --2- 172.21.15.50:0/1927465167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4824005360 0x7f484014a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.107+0000 7f4845600640 1 -- 172.21.15.50:0/1927465167 >> 172.21.15.50:0/1927465167 conn(0x7f48400b2040 msgr2=0x7f484014cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:40.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.107+0000 7f4845600640 1 -- 172.21.15.50:0/1927465167 shutdown_connections 2024-09-21T06:08:40.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.107+0000 7f4845600640 1 -- 172.21.15.50:0/1927465167 wait complete. 2024-09-21T06:08:40.129 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:08:40.130 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:08:40.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.291+0000 7feca1c00640 1 -- 172.21.15.50:0/626848316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec9c0d1190 msgr2=0x7fec9c0d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.291+0000 7feca1c00640 1 --2- 172.21.15.50:0/626848316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec9c0d1190 0x7fec9c0d15a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fec9c075760 tx=0x7fec9401cc60 comp rx=0 tx=0).stop 2024-09-21T06:08:40.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.291+0000 7feca1c00640 1 -- 172.21.15.50:0/626848316 shutdown_connections 2024-09-21T06:08:40.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.291+0000 7feca1c00640 1 --2- 172.21.15.50:0/626848316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec9c0d1190 0x7fec9c0d15a0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.291+0000 7feca1c00640 1 --2- 172.21.15.50:0/626848316 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec9c074c30 0x7fec9c075040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.291+0000 7feca1c00640 1 --2- 172.21.15.50:0/626848316 >> v2:172.21.15.88:3300/0 conn(0x7fec9c0746e0 0x7fec9c15c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.291+0000 7feca1c00640 1 -- 172.21.15.50:0/626848316 >> 172.21.15.50:0/626848316 conn(0x7fec9c0b41f0 msgr2=0x7fec9c0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:40.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.291+0000 7feca1c00640 1 -- 172.21.15.50:0/626848316 shutdown_connections 2024-09-21T06:08:40.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.291+0000 7feca1c00640 1 -- 172.21.15.50:0/626848316 wait complete. 2024-09-21T06:08:40.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7feca1c00640 1 Processor -- start 2024-09-21T06:08:40.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7feca1c00640 1 -- start start 2024-09-21T06:08:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7feca1c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec9c0746e0 0x7fec9c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7feca1c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec9c074c30 0x7fec9c09e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7feca1c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec9c0d1190 0x7fec9c162b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7feca1c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c0ce410 con 0x7fec9c074c30 2024-09-21T06:08:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7feca1c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c0ce290 con 0x7fec9c0746e0 2024-09-21T06:08:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7feca1c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fec9c0ce590 con 0x7fec9c0d1190 2024-09-21T06:08:40.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7fec93400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec9c074c30 0x7fec9c09e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7fec93400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec9c074c30 0x7fec9c09e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50084/0 (socket says 172.21.15.50:50084) 2024-09-21T06:08:40.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7fec93400640 1 -- 172.21.15.50:0/2148702117 learned_addr learned my addr 172.21.15.50:0/2148702117 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:40.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7fec9be00640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec9c0746e0 0x7fec9c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7feca1200640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec9c0d1190 0x7fec9c162b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7fec93400640 1 -- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec9c0d1190 msgr2=0x7fec9c162b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7fec93400640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec9c0d1190 0x7fec9c162b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7fec93400640 1 -- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec9c0746e0 msgr2=0x7fec9c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7fec93400640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec9c0746e0 0x7fec9c0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.295+0000 7fec93400640 1 -- 172.21.15.50:0/2148702117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9c09ed70 con 0x7fec9c074c30 2024-09-21T06:08:40.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.299+0000 7fec93400640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec9c074c30 0x7fec9c09e830 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fec84005bd0 tx=0x7fec8400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:40.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.299+0000 7fec98c00640 1 -- 172.21.15.50:0/2148702117 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84004230 con 0x7fec9c074c30 2024-09-21T06:08:40.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.299+0000 7fec98c00640 1 -- 172.21.15.50:0/2148702117 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec840044c0 con 0x7fec9c074c30 2024-09-21T06:08:40.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.299+0000 7feca1c00640 1 -- 172.21.15.50:0/2148702117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9c1f9060 con 0x7fec9c074c30 2024-09-21T06:08:40.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.299+0000 7fec98c00640 1 -- 172.21.15.50:0/2148702117 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84009a80 con 0x7fec9c074c30 2024-09-21T06:08:40.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.299+0000 7feca1c00640 1 -- 172.21.15.50:0/2148702117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9c1f9620 con 0x7fec9c074c30 2024-09-21T06:08:40.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.299+0000 7feca1c00640 1 -- 172.21.15.50:0/2148702117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec9c075040 con 0x7fec9c074c30 2024-09-21T06:08:40.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.299+0000 7fec98c00640 1 -- 172.21.15.50:0/2148702117 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fec84003600 con 0x7fec9c074c30 2024-09-21T06:08:40.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.299+0000 7fec98c00640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec78074880 0x7fec78076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.299+0000 7fec98c00640 1 -- 172.21.15.50:0/2148702117 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fec84093360 con 0x7fec9c074c30 2024-09-21T06:08:40.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.303+0000 7fec9be00640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec78074880 0x7fec78076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.303+0000 7fec9be00640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec78074880 0x7fec78076d90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00bad0 tx=0x7fec8c00b260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:40.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.303+0000 7fec98c00640 1 -- 172.21.15.50:0/2148702117 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fec84017020 con 0x7fec9c074c30 2024-09-21T06:08:40.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.515+0000 7feca1c00640 1 -- 172.21.15.50:0/2148702117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec9c0b4f30 con 0x7fec9c074c30 2024-09-21T06:08:40.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7fec98c00640 1 -- 172.21.15.50:0/2148702117 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5222 (secure 0 0 0) 0x7fec8405f790 con 0x7fec9c074c30 2024-09-21T06:08:40.516 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:40.516 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":65,"btime":"2024-09-21T06:08:40:104993+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2607826086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2607826086},{"type":"v1","addr":"172.21.15.50:6835","nonce":2607826086}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1771536088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1771536088},{"type":"v1","addr":"172.21.15.88:6837","nonce":1771536088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/433905290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":433905290},{"type":"v1","addr":"172.21.15.88:6836","nonce":433905290}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:08:36.508641+0000","modified":"2024-09-21T06:08:39.519558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1418616207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1418616207},{"type":"v1","addr":"172.21.15.50:6834","nonce":1418616207}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7851,"qdb_cluster":[7851]},"id":7}]} 2024-09-21T06:08:40.517 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 65 2024-09-21T06:08:40.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7feca1c00640 1 -- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec78074880 msgr2=0x7fec78076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7feca1c00640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec78074880 0x7fec78076d90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00bad0 tx=0x7fec8c00b260 comp rx=0 tx=0).stop 2024-09-21T06:08:40.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7feca1c00640 1 -- 172.21.15.50:0/2148702117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec9c074c30 msgr2=0x7fec9c09e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7feca1c00640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec9c074c30 0x7fec9c09e830 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fec84005bd0 tx=0x7fec8400db90 comp rx=0 tx=0).stop 2024-09-21T06:08:40.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7feca1c00640 1 -- 172.21.15.50:0/2148702117 shutdown_connections 2024-09-21T06:08:40.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7feca1c00640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec9c0d1190 0x7fec9c162b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7feca1c00640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec78074880 0x7fec78076d90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7feca1c00640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec9c074c30 0x7fec9c09e830 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7feca1c00640 1 --2- 172.21.15.50:0/2148702117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec9c0746e0 0x7fec9c0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7feca1c00640 1 -- 172.21.15.50:0/2148702117 >> 172.21.15.50:0/2148702117 conn(0x7fec9c0b41f0 msgr2=0x7fec9c0ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:40.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7feca1c00640 1 -- 172.21.15.50:0/2148702117 shutdown_connections 2024-09-21T06:08:40.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.519+0000 7feca1c00640 1 -- 172.21.15.50:0/2148702117 wait complete. 2024-09-21T06:08:40.539 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:08:40.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.699+0000 7f863aa00640 1 -- 172.21.15.50:0/4155959289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86340b5c70 msgr2=0x7f863415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.699+0000 7f863aa00640 1 --2- 172.21.15.50:0/4155959289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86340b5c70 0x7f863415bb60 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8624009900 tx=0x7f862401c820 comp rx=0 tx=0).stop 2024-09-21T06:08:40.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.699+0000 7f863aa00640 1 -- 172.21.15.50:0/4155959289 shutdown_connections 2024-09-21T06:08:40.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.699+0000 7f863aa00640 1 --2- 172.21.15.50:0/4155959289 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86341527b0 0x7f8634074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.699+0000 7f863aa00640 1 --2- 172.21.15.50:0/4155959289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86340b5c70 0x7f863415bb60 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.699+0000 7f863aa00640 1 -- 172.21.15.50:0/4155959289 >> 172.21.15.50:0/4155959289 conn(0x7f86340b2040 msgr2=0x7f86340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:40.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.699+0000 7f863aa00640 1 -- 172.21.15.50:0/4155959289 shutdown_connections 2024-09-21T06:08:40.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.699+0000 7f863aa00640 1 -- 172.21.15.50:0/4155959289 wait complete. 2024-09-21T06:08:40.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.699+0000 7f863aa00640 1 Processor -- start 2024-09-21T06:08:40.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.699+0000 7f863aa00640 1 -- start start 2024-09-21T06:08:40.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f863aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86340b4de0 0x7f86340ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f863aa00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86340b5c70 0x7f863414bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f863aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86341527b0 0x7f863414f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f863aa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8634168c50 con 0x7f86340b4de0 2024-09-21T06:08:40.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f863aa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8634168ad0 con 0x7f86341527b0 2024-09-21T06:08:40.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f863aa00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8634168dd0 con 0x7f86340b5c70 2024-09-21T06:08:40.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8633e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86340b4de0 0x7f86340ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8633e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86340b4de0 0x7f86340ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50090/0 (socket says 172.21.15.50:50090) 2024-09-21T06:08:40.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8633e00640 1 -- 172.21.15.50:0/1131771878 learned_addr learned my addr 172.21.15.50:0/1131771878 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:40.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f862be00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86340b5c70 0x7f863414bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8638a00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86341527b0 0x7f863414f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8633e00640 1 -- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86340b5c70 msgr2=0x7f863414bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8633e00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86340b5c70 0x7f863414bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8633e00640 1 -- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86341527b0 msgr2=0x7f863414f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8633e00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86341527b0 0x7f863414f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8633e00640 1 -- 172.21.15.50:0/1131771878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86341f67a0 con 0x7f86340b4de0 2024-09-21T06:08:40.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8633e00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86340b4de0 0x7f86340ce1c0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8624009850 tx=0x7f8624004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:40.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8630c00640 1 -- 172.21.15.50:0/1131771878 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86240b2040 con 0x7f86340b4de0 2024-09-21T06:08:40.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8630c00640 1 -- 172.21.15.50:0/1131771878 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8624005030 con 0x7f86340b4de0 2024-09-21T06:08:40.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f863aa00640 1 -- 172.21.15.50:0/1131771878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86341f9d90 con 0x7f86340b4de0 2024-09-21T06:08:40.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f8630c00640 1 -- 172.21.15.50:0/1131771878 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624005420 con 0x7f86340b4de0 2024-09-21T06:08:40.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f863aa00640 1 -- 172.21.15.50:0/1131771878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86341fa5a0 con 0x7f86340b4de0 2024-09-21T06:08:40.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.703+0000 7f863aa00640 1 -- 172.21.15.50:0/1131771878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863415bb60 con 0x7f86340b4de0 2024-09-21T06:08:40.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.707+0000 7f8630c00640 1 -- 172.21.15.50:0/1131771878 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f86240bd060 con 0x7f86340b4de0 2024-09-21T06:08:40.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.707+0000 7f8630c00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8610074880 0x7f8610076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.707+0000 7f8630c00640 1 -- 172.21.15.50:0/1131771878 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f862412c5e0 con 0x7f86340b4de0 2024-09-21T06:08:40.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.707+0000 7f862be00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8610074880 0x7f8610076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.707+0000 7f862be00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8610074880 0x7f8610076d90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f861c0058d0 tx=0x7f861c005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:40.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.711+0000 7f8630c00640 1 -- 172.21.15.50:0/1131771878 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8634075580 con 0x7f86340b4de0 2024-09-21T06:08:40.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.891+0000 7f863aa00640 1 -- 172.21.15.50:0/1131771878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8634150190 con 0x7f86340b4de0 2024-09-21T06:08:40.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.891+0000 7f8630c00640 1 -- 172.21.15.50:0/1131771878 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16710 (secure 0 0 0) 0x7f86240164d0 con 0x7f86340b4de0 2024-09-21T06:08:40.891 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:40.891 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":100,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:08:38.087373+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:08:34.249299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-21T06:08:35.470212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:08:40.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.895+0000 7f863aa00640 1 -- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8610074880 msgr2=0x7f8610076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.895+0000 7f863aa00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8610074880 0x7f8610076d90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f861c0058d0 tx=0x7f861c005860 comp rx=0 tx=0).stop 2024-09-21T06:08:40.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.895+0000 7f863aa00640 1 -- 172.21.15.50:0/1131771878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86340b4de0 msgr2=0x7f86340ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.895+0000 7f863aa00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86340b4de0 0x7f86340ce1c0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8624009850 tx=0x7f8624004040 comp rx=0 tx=0).stop 2024-09-21T06:08:40.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.895+0000 7f863aa00640 1 -- 172.21.15.50:0/1131771878 shutdown_connections 2024-09-21T06:08:40.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.895+0000 7f863aa00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86341527b0 0x7f863414f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.895+0000 7f863aa00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f86340b5c70 0x7f863414bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.895+0000 7f863aa00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8610074880 0x7f8610076d90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.895+0000 7f863aa00640 1 --2- 172.21.15.50:0/1131771878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86340b4de0 0x7f86340ce1c0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.895+0000 7f863aa00640 1 -- 172.21.15.50:0/1131771878 >> 172.21.15.50:0/1131771878 conn(0x7f86340b2040 msgr2=0x7f8634150990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:40.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.895+0000 7f863aa00640 1 -- 172.21.15.50:0/1131771878 shutdown_connections 2024-09-21T06:08:40.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:40.895+0000 7f863aa00640 1 -- 172.21.15.50:0/1131771878 wait complete. 2024-09-21T06:08:40.914 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:08:40.914 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:08:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.075+0000 7f20b6e00640 1 -- 172.21.15.50:0/1582263428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20b01527b0 msgr2=0x7f20b0074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.075+0000 7f20b6e00640 1 --2- 172.21.15.50:0/1582263428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20b01527b0 0x7f20b0074e30 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f20b0071cb0 tx=0x7f209801c890 comp rx=0 tx=0).stop 2024-09-21T06:08:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.075+0000 7f20b6e00640 1 -- 172.21.15.50:0/1582263428 shutdown_connections 2024-09-21T06:08:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.075+0000 7f20b6e00640 1 --2- 172.21.15.50:0/1582263428 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20b00b4de0 0x7f20b00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.075+0000 7f20b6e00640 1 --2- 172.21.15.50:0/1582263428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20b01527b0 0x7f20b0074e30 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.075+0000 7f20b6e00640 1 --2- 172.21.15.50:0/1582263428 >> v2:172.21.15.88:3300/0 conn(0x7f20b00b5c70 0x7f20b015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.075+0000 7f20b6e00640 1 -- 172.21.15.50:0/1582263428 >> 172.21.15.50:0/1582263428 conn(0x7f20b00b2040 msgr2=0x7f20b00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.075+0000 7f20b6e00640 1 -- 172.21.15.50:0/1582263428 shutdown_connections 2024-09-21T06:08:41.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.075+0000 7f20b6e00640 1 -- 172.21.15.50:0/1582263428 wait complete. 2024-09-21T06:08:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.075+0000 7f20b6e00640 1 Processor -- start 2024-09-21T06:08:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.075+0000 7f20b6e00640 1 -- start start 2024-09-21T06:08:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20b6e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20b00b4de0 0x7f20b01f9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20b6e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20b00b5c70 0x7f20b014cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20b6e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b01527b0 0x7f20b0150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20b6e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f20b015f6b0 con 0x7f20b00b4de0 2024-09-21T06:08:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20b6e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f20b015f530 con 0x7f20b01527b0 2024-09-21T06:08:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20b6e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f20b015f830 con 0x7f20b00b5c70 2024-09-21T06:08:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20afe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20b00b4de0 0x7f20b01f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20afe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20b00b4de0 0x7f20b01f9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50108/0 (socket says 172.21.15.50:50108) 2024-09-21T06:08:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20afe00640 1 -- 172.21.15.50:0/1718302850 learned_addr learned my addr 172.21.15.50:0/1718302850 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20a7400640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20b00b5c70 0x7f20b014cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20b4e00640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b01527b0 0x7f20b0150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20afe00640 1 -- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20b00b5c70 msgr2=0x7f20b014cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20afe00640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20b00b5c70 0x7f20b014cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20afe00640 1 -- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b01527b0 msgr2=0x7f20b0150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20afe00640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b01527b0 0x7f20b0150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20afe00640 1 -- 172.21.15.50:0/1718302850 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b014d0e0 con 0x7f20b00b4de0 2024-09-21T06:08:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20a7400640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20b00b5c70 0x7f20b014cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20afe00640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20b00b4de0 0x7f20b01f9230 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d950 tx=0x7f20a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20acc00640 1 -- 172.21.15.50:0/1718302850 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0017070 con 0x7f20b00b4de0 2024-09-21T06:08:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20acc00640 1 -- 172.21.15.50:0/1718302850 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20a0009d20 con 0x7f20b00b4de0 2024-09-21T06:08:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20acc00640 1 -- 172.21.15.50:0/1718302850 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a00106c0 con 0x7f20b00b4de0 2024-09-21T06:08:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20b6e00640 1 -- 172.21.15.50:0/1718302850 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b0159260 con 0x7f20b00b4de0 2024-09-21T06:08:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20b6e00640 1 -- 172.21.15.50:0/1718302850 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b01fc1e0 con 0x7f20b00b4de0 2024-09-21T06:08:41.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.079+0000 7f20b6e00640 1 -- 172.21.15.50:0/1718302850 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b015fd50 con 0x7f20b00b4de0 2024-09-21T06:08:41.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.083+0000 7f20acc00640 1 -- 172.21.15.50:0/1718302850 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f20a001a070 con 0x7f20b00b4de0 2024-09-21T06:08:41.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.083+0000 7f20acc00640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f208c0748b0 0x7f208c076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.083+0000 7f20acc00640 1 -- 172.21.15.50:0/1718302850 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f20a0093130 con 0x7f20b00b4de0 2024-09-21T06:08:41.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.083+0000 7f20a7400640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f208c0748b0 0x7f208c076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.083+0000 7f20a7400640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f208c0748b0 0x7f208c076dc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f20980af000 tx=0x7f209801b0d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:41.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.087+0000 7f20acc00640 1 -- 172.21.15.50:0/1718302850 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f20a005fbb0 con 0x7f20b00b4de0 2024-09-21T06:08:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.259+0000 7f20b6e00640 1 -- 172.21.15.50:0/1718302850 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f20b006b540 con 0x7f208c0748b0 2024-09-21T06:08:41.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.267+0000 7f20acc00640 1 -- 172.21.15.50:0/1718302850 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252130 (secure 0 0 0) 0x7f20980a6a90 con 0x7f208c0748b0 2024-09-21T06:08:41.269 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:41.270 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:08:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.271+0000 7f20b6e00640 1 -- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f208c0748b0 msgr2=0x7f208c076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.271+0000 7f20b6e00640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f208c0748b0 0x7f208c076dc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f20980af000 tx=0x7f209801b0d0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.271+0000 7f20b6e00640 1 -- 172.21.15.50:0/1718302850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20b00b4de0 msgr2=0x7f20b01f9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.271+0000 7f20b6e00640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20b00b4de0 0x7f20b01f9230 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d950 tx=0x7f20a000de10 comp rx=0 tx=0).stop 2024-09-21T06:08:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.271+0000 7f20b6e00640 1 -- 172.21.15.50:0/1718302850 shutdown_connections 2024-09-21T06:08:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.271+0000 7f20b6e00640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b01527b0 0x7f20b0150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.271+0000 7f20b6e00640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20b00b5c70 0x7f20b014cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.271+0000 7f20b6e00640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f208c0748b0 0x7f208c076dc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.271+0000 7f20b6e00640 1 --2- 172.21.15.50:0/1718302850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20b00b4de0 0x7f20b01f9230 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.271+0000 7f20b6e00640 1 -- 172.21.15.50:0/1718302850 >> 172.21.15.50:0/1718302850 conn(0x7f20b00b2040 msgr2=0x7f20b00b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.271+0000 7f20b6e00640 1 -- 172.21.15.50:0/1718302850 shutdown_connections 2024-09-21T06:08:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.271+0000 7f20b6e00640 1 -- 172.21.15.50:0/1718302850 wait complete. 2024-09-21T06:08:41.291 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-21T06:08:41.164171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1027444,"kb_used_data":10752,"kb_used_omap":12,"kb_used_meta":1016499,"kb_avail":753947276,"statfs":{"total":773094113280,"available":772042010624,"internally_reserved":0,"allocated":11010048,"data_stored":5395569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1040895536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001679"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.115029+0000","last_change":"2024-09-21T06:08:37.111644+0000","last_active":"2024-09-21T06:08:37.115029+0000","last_peered":"2024-09-21T06:08:37.115029+0000","last_clean":"2024-09-21T06:08:37.115029+0000","last_became_active":"2024-09-21T06:08:37.109736+0000","last_became_peered":"2024-09-21T06:08:37.109736+0000","last_unstale":"2024-09-21T06:08:37.115029+0000","last_undegraded":"2024-09-21T06:08:37.115029+0000","last_fullsized":"2024-09-21T06:08:37.115029+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:18.299738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.108217+0000","last_change":"2024-09-21T06:08:37.103889+0000","last_active":"2024-09-21T06:08:37.108217+0000","last_peered":"2024-09-21T06:08:37.108217+0000","last_clean":"2024-09-21T06:08:37.108217+0000","last_became_active":"2024-09-21T06:08:37.103130+0000","last_became_peered":"2024-09-21T06:08:37.103130+0000","last_unstale":"2024-09-21T06:08:37.108217+0000","last_undegraded":"2024-09-21T06:08:37.108217+0000","last_fullsized":"2024-09-21T06:08:37.108217+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:58.527808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.118326+0000","last_change":"2024-09-21T06:08:37.114293+0000","last_active":"2024-09-21T06:08:37.118326+0000","last_peered":"2024-09-21T06:08:37.118326+0000","last_clean":"2024-09-21T06:08:37.118326+0000","last_became_active":"2024-09-21T06:08:37.113795+0000","last_became_peered":"2024-09-21T06:08:37.113795+0000","last_unstale":"2024-09-21T06:08:37.118326+0000","last_undegraded":"2024-09-21T06:08:37.118326+0000","last_fullsized":"2024-09-21T06:08:37.118326+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:38.848043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096880+0000","last_change":"2024-09-21T06:08:37.101236+0000","last_active":"2024-09-21T06:08:38.096880+0000","last_peered":"2024-09-21T06:08:38.096880+0000","last_clean":"2024-09-21T06:08:38.096880+0000","last_became_active":"2024-09-21T06:08:37.090014+0000","last_became_peered":"2024-09-21T06:08:37.090014+0000","last_unstale":"2024-09-21T06:08:38.096880+0000","last_undegraded":"2024-09-21T06:08:38.096880+0000","last_fullsized":"2024-09-21T06:08:38.096880+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:18.697748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052184+0000","last_change":"2024-09-21T06:08:37.109964+0000","last_active":"2024-09-21T06:08:39.052184+0000","last_peered":"2024-09-21T06:08:39.052184+0000","last_clean":"2024-09-21T06:08:39.052184+0000","last_became_active":"2024-09-21T06:08:37.109637+0000","last_became_peered":"2024-09-21T06:08:37.109637+0000","last_unstale":"2024-09-21T06:08:39.052184+0000","last_undegraded":"2024-09-21T06:08:39.052184+0000","last_fullsized":"2024-09-21T06:08:39.052184+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:56.569180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103088+0000","last_change":"2024-09-21T06:08:37.098656+0000","last_active":"2024-09-21T06:08:37.103088+0000","last_peered":"2024-09-21T06:08:37.103088+0000","last_clean":"2024-09-21T06:08:37.103088+0000","last_became_active":"2024-09-21T06:08:37.098236+0000","last_became_peered":"2024-09-21T06:08:37.098236+0000","last_unstale":"2024-09-21T06:08:37.103088+0000","last_undegraded":"2024-09-21T06:08:37.103088+0000","last_fullsized":"2024-09-21T06:08:37.103088+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:54.634271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103076+0000","last_change":"2024-09-21T06:08:37.098517+0000","last_active":"2024-09-21T06:08:37.103076+0000","last_peered":"2024-09-21T06:08:37.103076+0000","last_clean":"2024-09-21T06:08:37.103076+0000","last_became_active":"2024-09-21T06:08:37.098152+0000","last_became_peered":"2024-09-21T06:08:37.098152+0000","last_unstale":"2024-09-21T06:08:37.103076+0000","last_undegraded":"2024-09-21T06:08:37.103076+0000","last_fullsized":"2024-09-21T06:08:37.103076+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:10.378585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.102604+0000","last_change":"2024-09-21T06:08:37.094106+0000","last_active":"2024-09-21T06:08:37.102604+0000","last_peered":"2024-09-21T06:08:37.102604+0000","last_clean":"2024-09-21T06:08:37.102604+0000","last_became_active":"2024-09-21T06:08:37.093804+0000","last_became_peered":"2024-09-21T06:08:37.093804+0000","last_unstale":"2024-09-21T06:08:37.102604+0000","last_undegraded":"2024-09-21T06:08:37.102604+0000","last_fullsized":"2024-09-21T06:08:37.102604+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:42.918938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098883+0000","last_change":"2024-09-21T06:08:37.094564+0000","last_active":"2024-09-21T06:08:38.098883+0000","last_peered":"2024-09-21T06:08:38.098883+0000","last_clean":"2024-09-21T06:08:38.098883+0000","last_became_active":"2024-09-21T06:08:37.092224+0000","last_became_peered":"2024-09-21T06:08:37.092224+0000","last_unstale":"2024-09-21T06:08:38.098883+0000","last_undegraded":"2024-09-21T06:08:38.098883+0000","last_fullsized":"2024-09-21T06:08:38.098883+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:10.378585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052811+0000","last_change":"2024-09-21T06:08:37.112644+0000","last_active":"2024-09-21T06:08:39.052811+0000","last_peered":"2024-09-21T06:08:39.052811+0000","last_clean":"2024-09-21T06:08:39.052811+0000","last_became_active":"2024-09-21T06:08:37.112365+0000","last_became_peered":"2024-09-21T06:08:37.112365+0000","last_unstale":"2024-09-21T06:08:39.052811+0000","last_undegraded":"2024-09-21T06:08:39.052811+0000","last_fullsized":"2024-09-21T06:08:39.052811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:57.092581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103357+0000","last_change":"2024-09-21T06:08:37.100042+0000","last_active":"2024-09-21T06:08:37.103357+0000","last_peered":"2024-09-21T06:08:37.103357+0000","last_clean":"2024-09-21T06:08:37.103357+0000","last_became_active":"2024-09-21T06:08:37.099653+0000","last_became_peered":"2024-09-21T06:08:37.099653+0000","last_unstale":"2024-09-21T06:08:37.103357+0000","last_undegraded":"2024-09-21T06:08:37.103357+0000","last_fullsized":"2024-09-21T06:08:37.103357+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:20.619423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098116+0000","last_change":"2024-09-21T06:08:37.111023+0000","last_active":"2024-09-21T06:08:38.098116+0000","last_peered":"2024-09-21T06:08:38.098116+0000","last_clean":"2024-09-21T06:08:38.098116+0000","last_became_active":"2024-09-21T06:08:37.110651+0000","last_became_peered":"2024-09-21T06:08:37.110651+0000","last_unstale":"2024-09-21T06:08:38.098116+0000","last_undegraded":"2024-09-21T06:08:38.098116+0000","last_fullsized":"2024-09-21T06:08:38.098116+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:54.542954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096807+0000","last_change":"2024-09-21T06:08:37.102848+0000","last_active":"2024-09-21T06:08:38.096807+0000","last_peered":"2024-09-21T06:08:38.096807+0000","last_clean":"2024-09-21T06:08:38.096807+0000","last_became_active":"2024-09-21T06:08:37.102527+0000","last_became_peered":"2024-09-21T06:08:37.102527+0000","last_unstale":"2024-09-21T06:08:38.096807+0000","last_undegraded":"2024-09-21T06:08:38.096807+0000","last_fullsized":"2024-09-21T06:08:38.096807+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:57.092581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.101954+0000","last_change":"2024-09-21T06:08:37.094115+0000","last_active":"2024-09-21T06:08:37.101954+0000","last_peered":"2024-09-21T06:08:37.101954+0000","last_clean":"2024-09-21T06:08:37.101954+0000","last_became_active":"2024-09-21T06:08:37.093812+0000","last_became_peered":"2024-09-21T06:08:37.093812+0000","last_unstale":"2024-09-21T06:08:37.101954+0000","last_undegraded":"2024-09-21T06:08:37.101954+0000","last_fullsized":"2024-09-21T06:08:37.101954+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:23.803134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096227+0000","last_change":"2024-09-21T06:08:37.113343+0000","last_active":"2024-09-21T06:08:38.096227+0000","last_peered":"2024-09-21T06:08:38.096227+0000","last_clean":"2024-09-21T06:08:38.096227+0000","last_became_active":"2024-09-21T06:08:37.112810+0000","last_became_peered":"2024-09-21T06:08:37.112810+0000","last_unstale":"2024-09-21T06:08:38.096227+0000","last_undegraded":"2024-09-21T06:08:38.096227+0000","last_fullsized":"2024-09-21T06:08:38.096227+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:58.527808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.106132+0000","last_change":"2024-09-21T06:08:37.104315+0000","last_active":"2024-09-21T06:08:37.106132+0000","last_peered":"2024-09-21T06:08:37.106132+0000","last_clean":"2024-09-21T06:08:37.106132+0000","last_became_active":"2024-09-21T06:08:37.103962+0000","last_became_peered":"2024-09-21T06:08:37.103962+0000","last_unstale":"2024-09-21T06:08:37.106132+0000","last_undegraded":"2024-09-21T06:08:37.106132+0000","last_fullsized":"2024-09-21T06:08:37.106132+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:37.179525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103046+0000","last_change":"2024-09-21T06:08:37.096111+0000","last_active":"2024-09-21T06:08:37.103046+0000","last_peered":"2024-09-21T06:08:37.103046+0000","last_clean":"2024-09-21T06:08:37.103046+0000","last_became_active":"2024-09-21T06:08:37.095527+0000","last_became_peered":"2024-09-21T06:08:37.095527+0000","last_unstale":"2024-09-21T06:08:37.103046+0000","last_undegraded":"2024-09-21T06:08:37.103046+0000","last_fullsized":"2024-09-21T06:08:37.103046+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:10.378585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096920+0000","last_change":"2024-09-21T06:08:37.093479+0000","last_active":"2024-09-21T06:08:38.096920+0000","last_peered":"2024-09-21T06:08:38.096920+0000","last_clean":"2024-09-21T06:08:38.096920+0000","last_became_active":"2024-09-21T06:08:37.093080+0000","last_became_peered":"2024-09-21T06:08:37.093080+0000","last_unstale":"2024-09-21T06:08:38.096920+0000","last_undegraded":"2024-09-21T06:08:38.096920+0000","last_fullsized":"2024-09-21T06:08:38.096920+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:05.612097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.119214+0000","last_change":"2024-09-21T06:08:37.118195+0000","last_active":"2024-09-21T06:08:37.119214+0000","last_peered":"2024-09-21T06:08:37.119214+0000","last_clean":"2024-09-21T06:08:37.119214+0000","last_became_active":"2024-09-21T06:08:37.117853+0000","last_became_peered":"2024-09-21T06:08:37.117853+0000","last_unstale":"2024-09-21T06:08:37.119214+0000","last_undegraded":"2024-09-21T06:08:37.119214+0000","last_fullsized":"2024-09-21T06:08:37.119214+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:45.882864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098168+0000","last_change":"2024-09-21T06:08:37.093053+0000","last_active":"2024-09-21T06:08:38.098168+0000","last_peered":"2024-09-21T06:08:38.098168+0000","last_clean":"2024-09-21T06:08:38.098168+0000","last_became_active":"2024-09-21T06:08:37.092756+0000","last_became_peered":"2024-09-21T06:08:37.092756+0000","last_unstale":"2024-09-21T06:08:38.098168+0000","last_undegraded":"2024-09-21T06:08:38.098168+0000","last_fullsized":"2024-09-21T06:08:38.098168+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:12.213858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103026+0000","last_change":"2024-09-21T06:08:37.092381+0000","last_active":"2024-09-21T06:08:37.103026+0000","last_peered":"2024-09-21T06:08:37.103026+0000","last_clean":"2024-09-21T06:08:37.103026+0000","last_became_active":"2024-09-21T06:08:37.092047+0000","last_became_peered":"2024-09-21T06:08:37.092047+0000","last_unstale":"2024-09-21T06:08:37.103026+0000","last_undegraded":"2024-09-21T06:08:37.103026+0000","last_fullsized":"2024-09-21T06:08:37.103026+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:40.148267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.114756+0000","last_change":"2024-09-21T06:08:37.110704+0000","last_active":"2024-09-21T06:08:37.114756+0000","last_peered":"2024-09-21T06:08:37.114756+0000","last_clean":"2024-09-21T06:08:37.114756+0000","last_became_active":"2024-09-21T06:08:37.110329+0000","last_became_peered":"2024-09-21T06:08:37.110329+0000","last_unstale":"2024-09-21T06:08:37.114756+0000","last_undegraded":"2024-09-21T06:08:37.114756+0000","last_fullsized":"2024-09-21T06:08:37.114756+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:12.316093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.102852+0000","last_change":"2024-09-21T06:08:37.097419+0000","last_active":"2024-09-21T06:08:37.102852+0000","last_peered":"2024-09-21T06:08:37.102852+0000","last_clean":"2024-09-21T06:08:37.102852+0000","last_became_active":"2024-09-21T06:08:37.097111+0000","last_became_peered":"2024-09-21T06:08:37.097111+0000","last_unstale":"2024-09-21T06:08:37.102852+0000","last_undegraded":"2024-09-21T06:08:37.102852+0000","last_fullsized":"2024-09-21T06:08:37.102852+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:20.619423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.098424+0000","last_change":"2024-09-21T06:08:37.096439+0000","last_active":"2024-09-21T06:08:37.098424+0000","last_peered":"2024-09-21T06:08:37.098424+0000","last_clean":"2024-09-21T06:08:37.098424+0000","last_became_active":"2024-09-21T06:08:37.095640+0000","last_became_peered":"2024-09-21T06:08:37.095640+0000","last_unstale":"2024-09-21T06:08:37.098424+0000","last_undegraded":"2024-09-21T06:08:37.098424+0000","last_fullsized":"2024-09-21T06:08:37.098424+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:33.956003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052905+0000","last_change":"2024-09-21T06:08:37.105728+0000","last_active":"2024-09-21T06:08:39.052905+0000","last_peered":"2024-09-21T06:08:39.052905+0000","last_clean":"2024-09-21T06:08:39.052905+0000","last_became_active":"2024-09-21T06:08:37.104902+0000","last_became_peered":"2024-09-21T06:08:37.104902+0000","last_unstale":"2024-09-21T06:08:39.052905+0000","last_undegraded":"2024-09-21T06:08:39.052905+0000","last_fullsized":"2024-09-21T06:08:39.052905+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:37.177983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.115664+0000","last_change":"2024-09-21T06:08:37.112467+0000","last_active":"2024-09-21T06:08:37.115664+0000","last_peered":"2024-09-21T06:08:37.115664+0000","last_clean":"2024-09-21T06:08:37.115664+0000","last_became_active":"2024-09-21T06:08:37.110782+0000","last_became_peered":"2024-09-21T06:08:37.110782+0000","last_unstale":"2024-09-21T06:08:37.115664+0000","last_undegraded":"2024-09-21T06:08:37.115664+0000","last_fullsized":"2024-09-21T06:08:37.115664+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:24.499024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.096128+0000","last_change":"2024-09-21T06:08:37.093012+0000","last_active":"2024-09-21T06:08:37.096128+0000","last_peered":"2024-09-21T06:08:37.096128+0000","last_clean":"2024-09-21T06:08:37.096128+0000","last_became_active":"2024-09-21T06:08:37.092656+0000","last_became_peered":"2024-09-21T06:08:37.092656+0000","last_unstale":"2024-09-21T06:08:37.096128+0000","last_undegraded":"2024-09-21T06:08:37.096128+0000","last_fullsized":"2024-09-21T06:08:37.096128+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:42.918938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052348+0000","last_change":"2024-09-21T06:08:37.115603+0000","last_active":"2024-09-21T06:08:39.052348+0000","last_peered":"2024-09-21T06:08:39.052348+0000","last_clean":"2024-09-21T06:08:39.052348+0000","last_became_active":"2024-09-21T06:08:37.115374+0000","last_became_peered":"2024-09-21T06:08:37.115374+0000","last_unstale":"2024-09-21T06:08:39.052348+0000","last_undegraded":"2024-09-21T06:08:39.052348+0000","last_fullsized":"2024-09-21T06:08:39.052348+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:07.241230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.118330+0000","last_change":"2024-09-21T06:08:37.115735+0000","last_active":"2024-09-21T06:08:37.118330+0000","last_peered":"2024-09-21T06:08:37.118330+0000","last_clean":"2024-09-21T06:08:37.118330+0000","last_became_active":"2024-09-21T06:08:37.115427+0000","last_became_peered":"2024-09-21T06:08:37.115427+0000","last_unstale":"2024-09-21T06:08:37.118330+0000","last_undegraded":"2024-09-21T06:08:37.118330+0000","last_fullsized":"2024-09-21T06:08:37.118330+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:39.123832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.118383+0000","last_change":"2024-09-21T06:08:37.117199+0000","last_active":"2024-09-21T06:08:37.118383+0000","last_peered":"2024-09-21T06:08:37.118383+0000","last_clean":"2024-09-21T06:08:37.118383+0000","last_became_active":"2024-09-21T06:08:37.116852+0000","last_became_peered":"2024-09-21T06:08:37.116852+0000","last_unstale":"2024-09-21T06:08:37.118383+0000","last_undegraded":"2024-09-21T06:08:37.118383+0000","last_fullsized":"2024-09-21T06:08:37.118383+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:39.736419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096340+0000","last_change":"2024-09-21T06:08:37.093396+0000","last_active":"2024-09-21T06:08:38.096340+0000","last_peered":"2024-09-21T06:08:38.096340+0000","last_clean":"2024-09-21T06:08:38.096340+0000","last_became_active":"2024-09-21T06:08:37.093067+0000","last_became_peered":"2024-09-21T06:08:37.093067+0000","last_unstale":"2024-09-21T06:08:38.096340+0000","last_undegraded":"2024-09-21T06:08:38.096340+0000","last_fullsized":"2024-09-21T06:08:38.096340+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:24.499024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.115213+0000","last_change":"2024-09-21T06:08:37.104312+0000","last_active":"2024-09-21T06:08:37.115213+0000","last_peered":"2024-09-21T06:08:37.115213+0000","last_clean":"2024-09-21T06:08:37.115213+0000","last_became_active":"2024-09-21T06:08:37.103982+0000","last_became_peered":"2024-09-21T06:08:37.103982+0000","last_unstale":"2024-09-21T06:08:37.115213+0000","last_undegraded":"2024-09-21T06:08:37.115213+0000","last_fullsized":"2024-09-21T06:08:37.115213+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:12.827691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.092207+0000","last_change":"2024-09-21T06:08:37.090139+0000","last_active":"2024-09-21T06:08:37.092207+0000","last_peered":"2024-09-21T06:08:37.092207+0000","last_clean":"2024-09-21T06:08:37.092207+0000","last_became_active":"2024-09-21T06:08:37.089814+0000","last_became_peered":"2024-09-21T06:08:37.089814+0000","last_unstale":"2024-09-21T06:08:37.092207+0000","last_undegraded":"2024-09-21T06:08:37.092207+0000","last_fullsized":"2024-09-21T06:08:37.092207+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:23.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"99'4","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.113493+0000","last_change":"2024-09-21T06:08:36.087950+0000","last_active":"2024-09-21T06:08:37.113493+0000","last_peered":"2024-09-21T06:08:37.113493+0000","last_clean":"2024-09-21T06:08:37.113493+0000","last_became_active":"2024-09-21T06:08:36.087621+0000","last_became_peered":"2024-09-21T06:08:36.087621+0000","last_unstale":"2024-09-21T06:08:37.113493+0000","last_undegraded":"2024-09-21T06:08:37.113493+0000","last_fullsized":"2024-09-21T06:08:37.113493+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:32.953321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097088+0000","last_change":"2024-09-21T06:08:37.095434+0000","last_active":"2024-09-21T06:08:38.097088+0000","last_peered":"2024-09-21T06:08:38.097088+0000","last_clean":"2024-09-21T06:08:38.097088+0000","last_became_active":"2024-09-21T06:08:37.094577+0000","last_became_peered":"2024-09-21T06:08:37.094577+0000","last_unstale":"2024-09-21T06:08:38.097088+0000","last_undegraded":"2024-09-21T06:08:38.097088+0000","last_fullsized":"2024-09-21T06:08:38.097088+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:13.833258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"99'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.112539+0000","last_change":"2024-09-21T06:08:36.098530+0000","last_active":"2024-09-21T06:08:37.112539+0000","last_peered":"2024-09-21T06:08:37.112539+0000","last_clean":"2024-09-21T06:08:37.112539+0000","last_became_active":"2024-09-21T06:08:36.098150+0000","last_became_peered":"2024-09-21T06:08:36.098150+0000","last_unstale":"2024-09-21T06:08:37.112539+0000","last_undegraded":"2024-09-21T06:08:37.112539+0000","last_fullsized":"2024-09-21T06:08:37.112539+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:55.862448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.114088+0000","last_change":"2024-09-21T06:08:37.110080+0000","last_active":"2024-09-21T06:08:37.114088+0000","last_peered":"2024-09-21T06:08:37.114088+0000","last_clean":"2024-09-21T06:08:37.114088+0000","last_became_active":"2024-09-21T06:08:37.109702+0000","last_became_peered":"2024-09-21T06:08:37.109702+0000","last_unstale":"2024-09-21T06:08:37.114088+0000","last_undegraded":"2024-09-21T06:08:37.114088+0000","last_fullsized":"2024-09-21T06:08:37.114088+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:39.118789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"99'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.115234+0000","last_change":"2024-09-21T06:08:36.092070+0000","last_active":"2024-09-21T06:08:37.115234+0000","last_peered":"2024-09-21T06:08:37.115234+0000","last_clean":"2024-09-21T06:08:37.115234+0000","last_became_active":"2024-09-21T06:08:36.090747+0000","last_became_peered":"2024-09-21T06:08:36.090747+0000","last_unstale":"2024-09-21T06:08:37.115234+0000","last_undegraded":"2024-09-21T06:08:37.115234+0000","last_fullsized":"2024-09-21T06:08:37.115234+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:55.862448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098828+0000","last_change":"2024-09-21T06:08:37.085484+0000","last_active":"2024-09-21T06:08:38.098828+0000","last_peered":"2024-09-21T06:08:38.098828+0000","last_clean":"2024-09-21T06:08:38.098828+0000","last_became_active":"2024-09-21T06:08:37.085167+0000","last_became_peered":"2024-09-21T06:08:37.085167+0000","last_unstale":"2024-09-21T06:08:38.098828+0000","last_undegraded":"2024-09-21T06:08:38.098828+0000","last_fullsized":"2024-09-21T06:08:38.098828+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:33.956003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"99'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.114307+0000","last_change":"2024-09-21T06:08:36.085410+0000","last_active":"2024-09-21T06:08:37.114307+0000","last_peered":"2024-09-21T06:08:37.114307+0000","last_clean":"2024-09-21T06:08:37.114307+0000","last_became_active":"2024-09-21T06:08:36.085051+0000","last_became_peered":"2024-09-21T06:08:36.085051+0000","last_unstale":"2024-09-21T06:08:37.114307+0000","last_undegraded":"2024-09-21T06:08:37.114307+0000","last_fullsized":"2024-09-21T06:08:37.114307+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:43.773479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097582+0000","last_change":"2024-09-21T06:08:37.082292+0000","last_active":"2024-09-21T06:08:38.097582+0000","last_peered":"2024-09-21T06:08:38.097582+0000","last_clean":"2024-09-21T06:08:38.097582+0000","last_became_active":"2024-09-21T06:08:37.081859+0000","last_became_peered":"2024-09-21T06:08:37.081859+0000","last_unstale":"2024-09-21T06:08:38.097582+0000","last_undegraded":"2024-09-21T06:08:38.097582+0000","last_fullsized":"2024-09-21T06:08:38.097582+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:19.735552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"99'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.113848+0000","last_change":"2024-09-21T06:08:36.092630+0000","last_active":"2024-09-21T06:08:37.113848+0000","last_peered":"2024-09-21T06:08:37.113848+0000","last_clean":"2024-09-21T06:08:37.113848+0000","last_became_active":"2024-09-21T06:08:36.092361+0000","last_became_peered":"2024-09-21T06:08:36.092361+0000","last_unstale":"2024-09-21T06:08:37.113848+0000","last_undegraded":"2024-09-21T06:08:37.113848+0000","last_fullsized":"2024-09-21T06:08:37.113848+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:00.867561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.094024+0000","last_change":"2024-09-21T06:08:37.091778+0000","last_active":"2024-09-21T06:08:37.094024+0000","last_peered":"2024-09-21T06:08:37.094024+0000","last_clean":"2024-09-21T06:08:37.094024+0000","last_became_active":"2024-09-21T06:08:37.091438+0000","last_became_peered":"2024-09-21T06:08:37.091438+0000","last_unstale":"2024-09-21T06:08:37.094024+0000","last_undegraded":"2024-09-21T06:08:37.094024+0000","last_fullsized":"2024-09-21T06:08:37.094024+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:12.213858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"99'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.118714+0000","last_change":"2024-09-21T06:08:36.096536+0000","last_active":"2024-09-21T06:08:37.118714+0000","last_peered":"2024-09-21T06:08:37.118714+0000","last_clean":"2024-09-21T06:08:37.118714+0000","last_became_active":"2024-09-21T06:08:36.096220+0000","last_became_peered":"2024-09-21T06:08:36.096220+0000","last_unstale":"2024-09-21T06:08:37.118714+0000","last_undegraded":"2024-09-21T06:08:37.118714+0000","last_fullsized":"2024-09-21T06:08:37.118714+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:06.942640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096258+0000","last_change":"2024-09-21T06:08:37.096405+0000","last_active":"2024-09-21T06:08:38.096258+0000","last_peered":"2024-09-21T06:08:38.096258+0000","last_clean":"2024-09-21T06:08:38.096258+0000","last_became_active":"2024-09-21T06:08:37.095778+0000","last_became_peered":"2024-09-21T06:08:37.095778+0000","last_unstale":"2024-09-21T06:08:38.096258+0000","last_undegraded":"2024-09-21T06:08:38.096258+0000","last_fullsized":"2024-09-21T06:08:38.096258+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:12.316093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052184+0000","last_change":"2024-09-21T06:08:36.085333+0000","last_active":"2024-09-21T06:08:39.052184+0000","last_peered":"2024-09-21T06:08:39.052184+0000","last_clean":"2024-09-21T06:08:39.052184+0000","last_became_active":"2024-09-21T06:08:36.085016+0000","last_became_peered":"2024-09-21T06:08:36.085016+0000","last_unstale":"2024-09-21T06:08:39.052184+0000","last_undegraded":"2024-09-21T06:08:39.052184+0000","last_fullsized":"2024-09-21T06:08:39.052184+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:54.451399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097745+0000","last_change":"2024-09-21T06:08:37.094683+0000","last_active":"2024-09-21T06:08:38.097745+0000","last_peered":"2024-09-21T06:08:38.097745+0000","last_clean":"2024-09-21T06:08:38.097745+0000","last_became_active":"2024-09-21T06:08:37.090846+0000","last_became_peered":"2024-09-21T06:08:37.090846+0000","last_unstale":"2024-09-21T06:08:38.097745+0000","last_undegraded":"2024-09-21T06:08:38.097745+0000","last_fullsized":"2024-09-21T06:08:38.097745+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:56.865129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.096210+0000","last_change":"2024-09-21T06:08:36.098091+0000","last_active":"2024-09-21T06:08:37.096210+0000","last_peered":"2024-09-21T06:08:37.096210+0000","last_clean":"2024-09-21T06:08:37.096210+0000","last_became_active":"2024-09-21T06:08:36.097765+0000","last_became_peered":"2024-09-21T06:08:36.097765+0000","last_unstale":"2024-09-21T06:08:37.096210+0000","last_undegraded":"2024-09-21T06:08:37.096210+0000","last_fullsized":"2024-09-21T06:08:37.096210+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:14.038348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.106332+0000","last_change":"2024-09-21T06:08:37.102036+0000","last_active":"2024-09-21T06:08:37.106332+0000","last_peered":"2024-09-21T06:08:37.106332+0000","last_clean":"2024-09-21T06:08:37.106332+0000","last_became_active":"2024-09-21T06:08:37.101133+0000","last_became_peered":"2024-09-21T06:08:37.101133+0000","last_unstale":"2024-09-21T06:08:37.106332+0000","last_undegraded":"2024-09-21T06:08:37.106332+0000","last_fullsized":"2024-09-21T06:08:37.106332+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:37.179525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.098960+0000","last_change":"2024-09-21T06:08:36.095764+0000","last_active":"2024-09-21T06:08:37.098960+0000","last_peered":"2024-09-21T06:08:37.098960+0000","last_clean":"2024-09-21T06:08:37.098960+0000","last_became_active":"2024-09-21T06:08:36.095460+0000","last_became_peered":"2024-09-21T06:08:36.095460+0000","last_unstale":"2024-09-21T06:08:37.098960+0000","last_undegraded":"2024-09-21T06:08:37.098960+0000","last_fullsized":"2024-09-21T06:08:37.098960+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:42.891663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.099116+0000","last_change":"2024-09-21T06:08:37.083020+0000","last_active":"2024-09-21T06:08:38.099116+0000","last_peered":"2024-09-21T06:08:38.099116+0000","last_clean":"2024-09-21T06:08:38.099116+0000","last_became_active":"2024-09-21T06:08:37.082730+0000","last_became_peered":"2024-09-21T06:08:37.082730+0000","last_unstale":"2024-09-21T06:08:38.099116+0000","last_undegraded":"2024-09-21T06:08:38.099116+0000","last_fullsized":"2024-09-21T06:08:38.099116+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:54.634271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096796+0000","last_change":"2024-09-21T06:08:36.086708+0000","last_active":"2024-09-21T06:08:38.096796+0000","last_peered":"2024-09-21T06:08:38.096796+0000","last_clean":"2024-09-21T06:08:38.096796+0000","last_became_active":"2024-09-21T06:08:36.086359+0000","last_became_peered":"2024-09-21T06:08:36.086359+0000","last_unstale":"2024-09-21T06:08:38.096796+0000","last_undegraded":"2024-09-21T06:08:38.096796+0000","last_fullsized":"2024-09-21T06:08:38.096796+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:53.540272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052649+0000","last_change":"2024-09-21T06:08:37.111290+0000","last_active":"2024-09-21T06:08:39.052649+0000","last_peered":"2024-09-21T06:08:39.052649+0000","last_clean":"2024-09-21T06:08:39.052649+0000","last_became_active":"2024-09-21T06:08:37.110943+0000","last_became_peered":"2024-09-21T06:08:37.110943+0000","last_unstale":"2024-09-21T06:08:39.052649+0000","last_undegraded":"2024-09-21T06:08:39.052649+0000","last_fullsized":"2024-09-21T06:08:39.052649+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.909923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"99'3","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.109116+0000","last_change":"2024-09-21T06:08:36.093355+0000","last_active":"2024-09-21T06:08:37.109116+0000","last_peered":"2024-09-21T06:08:37.109116+0000","last_clean":"2024-09-21T06:08:37.109116+0000","last_became_active":"2024-09-21T06:08:36.093000+0000","last_became_peered":"2024-09-21T06:08:36.093000+0000","last_unstale":"2024-09-21T06:08:37.109116+0000","last_undegraded":"2024-09-21T06:08:37.109116+0000","last_fullsized":"2024-09-21T06:08:37.109116+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:53.540272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.106151+0000","last_change":"2024-09-21T06:08:37.104527+0000","last_active":"2024-09-21T06:08:37.106151+0000","last_peered":"2024-09-21T06:08:37.106151+0000","last_clean":"2024-09-21T06:08:37.106151+0000","last_became_active":"2024-09-21T06:08:37.104089+0000","last_became_peered":"2024-09-21T06:08:37.104089+0000","last_unstale":"2024-09-21T06:08:37.106151+0000","last_undegraded":"2024-09-21T06:08:37.106151+0000","last_fullsized":"2024-09-21T06:08:37.106151+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:58.527808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"99'6","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098249+0000","last_change":"2024-09-21T06:08:36.094885+0000","last_active":"2024-09-21T06:08:38.098249+0000","last_peered":"2024-09-21T06:08:38.098249+0000","last_clean":"2024-09-21T06:08:38.098249+0000","last_became_active":"2024-09-21T06:08:36.094591+0000","last_became_peered":"2024-09-21T06:08:36.094591+0000","last_unstale":"2024-09-21T06:08:38.098249+0000","last_undegraded":"2024-09-21T06:08:38.098249+0000","last_fullsized":"2024-09-21T06:08:38.098249+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:23.503527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096437+0000","last_change":"2024-09-21T06:08:37.098055+0000","last_active":"2024-09-21T06:08:38.096437+0000","last_peered":"2024-09-21T06:08:38.096437+0000","last_clean":"2024-09-21T06:08:38.096437+0000","last_became_active":"2024-09-21T06:08:37.097705+0000","last_became_peered":"2024-09-21T06:08:37.097705+0000","last_unstale":"2024-09-21T06:08:38.096437+0000","last_undegraded":"2024-09-21T06:08:38.096437+0000","last_fullsized":"2024-09-21T06:08:38.096437+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:39.123832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.112788+0000","last_change":"2024-09-21T06:08:36.098400+0000","last_active":"2024-09-21T06:08:37.112788+0000","last_peered":"2024-09-21T06:08:37.112788+0000","last_clean":"2024-09-21T06:08:37.112788+0000","last_became_active":"2024-09-21T06:08:36.098061+0000","last_became_peered":"2024-09-21T06:08:36.098061+0000","last_unstale":"2024-09-21T06:08:37.112788+0000","last_undegraded":"2024-09-21T06:08:37.112788+0000","last_fullsized":"2024-09-21T06:08:37.112788+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:07.664854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.117654+0000","last_change":"2024-09-21T06:08:37.114079+0000","last_active":"2024-09-21T06:08:37.117654+0000","last_peered":"2024-09-21T06:08:37.117654+0000","last_clean":"2024-09-21T06:08:37.117654+0000","last_became_active":"2024-09-21T06:08:37.109467+0000","last_became_peered":"2024-09-21T06:08:37.109467+0000","last_unstale":"2024-09-21T06:08:37.117654+0000","last_undegraded":"2024-09-21T06:08:37.117654+0000","last_fullsized":"2024-09-21T06:08:37.117654+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:12.148391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.090051+0000","last_change":"2024-09-21T06:08:36.099380+0000","last_active":"2024-09-21T06:08:37.090051+0000","last_peered":"2024-09-21T06:08:37.090051+0000","last_clean":"2024-09-21T06:08:37.090051+0000","last_became_active":"2024-09-21T06:08:36.099069+0000","last_became_peered":"2024-09-21T06:08:36.099069+0000","last_unstale":"2024-09-21T06:08:37.090051+0000","last_undegraded":"2024-09-21T06:08:37.090051+0000","last_fullsized":"2024-09-21T06:08:37.090051+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:15.875075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097312+0000","last_change":"2024-09-21T06:08:37.091273+0000","last_active":"2024-09-21T06:08:38.097312+0000","last_peered":"2024-09-21T06:08:38.097312+0000","last_clean":"2024-09-21T06:08:38.097312+0000","last_became_active":"2024-09-21T06:08:37.084101+0000","last_became_peered":"2024-09-21T06:08:37.084101+0000","last_unstale":"2024-09-21T06:08:38.097312+0000","last_undegraded":"2024-09-21T06:08:38.097312+0000","last_fullsized":"2024-09-21T06:08:38.097312+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:16.299027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097570+0000","last_change":"2024-09-21T06:08:36.087689+0000","last_active":"2024-09-21T06:08:38.097570+0000","last_peered":"2024-09-21T06:08:38.097570+0000","last_clean":"2024-09-21T06:08:38.097570+0000","last_became_active":"2024-09-21T06:08:36.087388+0000","last_became_peered":"2024-09-21T06:08:36.087388+0000","last_unstale":"2024-09-21T06:08:38.097570+0000","last_undegraded":"2024-09-21T06:08:38.097570+0000","last_fullsized":"2024-09-21T06:08:38.097570+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:47.602151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.106841+0000","last_change":"2024-09-21T06:08:37.099943+0000","last_active":"2024-09-21T06:08:37.106841+0000","last_peered":"2024-09-21T06:08:37.106841+0000","last_clean":"2024-09-21T06:08:37.106841+0000","last_became_active":"2024-09-21T06:08:37.099643+0000","last_became_peered":"2024-09-21T06:08:37.099643+0000","last_unstale":"2024-09-21T06:08:37.106841+0000","last_undegraded":"2024-09-21T06:08:37.106841+0000","last_fullsized":"2024-09-21T06:08:37.106841+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"99'8","reported_seq":21,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.115160+0000","last_change":"2024-09-21T06:08:36.098044+0000","last_active":"2024-09-21T06:08:37.115160+0000","last_peered":"2024-09-21T06:08:37.115160+0000","last_clean":"2024-09-21T06:08:37.115160+0000","last_became_active":"2024-09-21T06:08:36.097723+0000","last_became_peered":"2024-09-21T06:08:36.097723+0000","last_unstale":"2024-09-21T06:08:37.115160+0000","last_undegraded":"2024-09-21T06:08:37.115160+0000","last_fullsized":"2024-09-21T06:08:37.115160+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:06.103687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052883+0000","last_change":"2024-09-21T06:08:37.099722+0000","last_active":"2024-09-21T06:08:39.052883+0000","last_peered":"2024-09-21T06:08:39.052883+0000","last_clean":"2024-09-21T06:08:39.052883+0000","last_became_active":"2024-09-21T06:08:37.099357+0000","last_became_peered":"2024-09-21T06:08:37.099357+0000","last_unstale":"2024-09-21T06:08:39.052883+0000","last_undegraded":"2024-09-21T06:08:39.052883+0000","last_fullsized":"2024-09-21T06:08:39.052883+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:17.894972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098623+0000","last_change":"2024-09-21T06:08:36.091038+0000","last_active":"2024-09-21T06:08:38.098623+0000","last_peered":"2024-09-21T06:08:38.098623+0000","last_clean":"2024-09-21T06:08:38.098623+0000","last_became_active":"2024-09-21T06:08:36.090716+0000","last_became_peered":"2024-09-21T06:08:36.090716+0000","last_unstale":"2024-09-21T06:08:38.098623+0000","last_undegraded":"2024-09-21T06:08:38.098623+0000","last_fullsized":"2024-09-21T06:08:38.098623+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:42.804603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097893+0000","last_change":"2024-09-21T06:08:37.089104+0000","last_active":"2024-09-21T06:08:38.097893+0000","last_peered":"2024-09-21T06:08:38.097893+0000","last_clean":"2024-09-21T06:08:38.097893+0000","last_became_active":"2024-09-21T06:08:37.088741+0000","last_became_peered":"2024-09-21T06:08:37.088741+0000","last_unstale":"2024-09-21T06:08:38.097893+0000","last_undegraded":"2024-09-21T06:08:38.097893+0000","last_fullsized":"2024-09-21T06:08:38.097893+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:25.961882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052668+0000","last_change":"2024-09-21T06:08:36.088687+0000","last_active":"2024-09-21T06:08:39.052668+0000","last_peered":"2024-09-21T06:08:39.052668+0000","last_clean":"2024-09-21T06:08:39.052668+0000","last_became_active":"2024-09-21T06:08:36.087706+0000","last_became_peered":"2024-09-21T06:08:36.087706+0000","last_unstale":"2024-09-21T06:08:39.052668+0000","last_undegraded":"2024-09-21T06:08:39.052668+0000","last_fullsized":"2024-09-21T06:08:39.052668+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:43.773479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103385+0000","last_change":"2024-09-21T06:08:37.100290+0000","last_active":"2024-09-21T06:08:37.103385+0000","last_peered":"2024-09-21T06:08:37.103385+0000","last_clean":"2024-09-21T06:08:37.103385+0000","last_became_active":"2024-09-21T06:08:37.100029+0000","last_became_peered":"2024-09-21T06:08:37.100029+0000","last_unstale":"2024-09-21T06:08:37.103385+0000","last_undegraded":"2024-09-21T06:08:37.103385+0000","last_fullsized":"2024-09-21T06:08:37.103385+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:17.553277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.091292+0000","last_change":"2024-09-21T06:08:36.084755+0000","last_active":"2024-09-21T06:08:37.091292+0000","last_peered":"2024-09-21T06:08:37.091292+0000","last_clean":"2024-09-21T06:08:37.091292+0000","last_became_active":"2024-09-21T06:08:36.084379+0000","last_became_peered":"2024-09-21T06:08:36.084379+0000","last_unstale":"2024-09-21T06:08:37.091292+0000","last_undegraded":"2024-09-21T06:08:37.091292+0000","last_fullsized":"2024-09-21T06:08:37.091292+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:11.211176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.114120+0000","last_change":"2024-09-21T06:08:37.107882+0000","last_active":"2024-09-21T06:08:37.114120+0000","last_peered":"2024-09-21T06:08:37.114120+0000","last_clean":"2024-09-21T06:08:37.114120+0000","last_became_active":"2024-09-21T06:08:37.107578+0000","last_became_peered":"2024-09-21T06:08:37.107578+0000","last_unstale":"2024-09-21T06:08:37.114120+0000","last_undegraded":"2024-09-21T06:08:37.114120+0000","last_fullsized":"2024-09-21T06:08:37.114120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:26.479609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"99'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.116533+0000","last_change":"2024-09-21T06:08:36.091770+0000","last_active":"2024-09-21T06:08:37.116533+0000","last_peered":"2024-09-21T06:08:37.116533+0000","last_clean":"2024-09-21T06:08:37.116533+0000","last_became_active":"2024-09-21T06:08:36.091443+0000","last_became_peered":"2024-09-21T06:08:36.091443+0000","last_unstale":"2024-09-21T06:08:37.116533+0000","last_undegraded":"2024-09-21T06:08:37.116533+0000","last_fullsized":"2024-09-21T06:08:37.116533+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:33.653888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103078+0000","last_change":"2024-09-21T06:08:37.098615+0000","last_active":"2024-09-21T06:08:37.103078+0000","last_peered":"2024-09-21T06:08:37.103078+0000","last_clean":"2024-09-21T06:08:37.103078+0000","last_became_active":"2024-09-21T06:08:37.098314+0000","last_became_peered":"2024-09-21T06:08:37.098314+0000","last_unstale":"2024-09-21T06:08:37.103078+0000","last_undegraded":"2024-09-21T06:08:37.103078+0000","last_fullsized":"2024-09-21T06:08:37.103078+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:44.498607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097503+0000","last_change":"2024-09-21T06:08:36.095525+0000","last_active":"2024-09-21T06:08:38.097503+0000","last_peered":"2024-09-21T06:08:38.097503+0000","last_clean":"2024-09-21T06:08:38.097503+0000","last_became_active":"2024-09-21T06:08:36.095158+0000","last_became_peered":"2024-09-21T06:08:36.095158+0000","last_unstale":"2024-09-21T06:08:38.097503+0000","last_undegraded":"2024-09-21T06:08:38.097503+0000","last_fullsized":"2024-09-21T06:08:38.097503+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:26.521534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.121347+0000","last_change":"2024-09-21T06:08:37.119864+0000","last_active":"2024-09-21T06:08:37.121347+0000","last_peered":"2024-09-21T06:08:37.121347+0000","last_clean":"2024-09-21T06:08:37.121347+0000","last_became_active":"2024-09-21T06:08:37.116834+0000","last_became_peered":"2024-09-21T06:08:37.116834+0000","last_unstale":"2024-09-21T06:08:37.121347+0000","last_undegraded":"2024-09-21T06:08:37.121347+0000","last_fullsized":"2024-09-21T06:08:37.121347+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:05.816651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097355+0000","last_change":"2024-09-21T06:08:36.085965+0000","last_active":"2024-09-21T06:08:38.097355+0000","last_peered":"2024-09-21T06:08:38.097355+0000","last_clean":"2024-09-21T06:08:38.097355+0000","last_became_active":"2024-09-21T06:08:36.085620+0000","last_became_peered":"2024-09-21T06:08:36.085620+0000","last_unstale":"2024-09-21T06:08:38.097355+0000","last_undegraded":"2024-09-21T06:08:38.097355+0000","last_fullsized":"2024-09-21T06:08:38.097355+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:44.707072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052045+0000","last_change":"2024-09-21T06:08:37.108407+0000","last_active":"2024-09-21T06:08:39.052045+0000","last_peered":"2024-09-21T06:08:39.052045+0000","last_clean":"2024-09-21T06:08:39.052045+0000","last_became_active":"2024-09-21T06:08:37.108007+0000","last_became_peered":"2024-09-21T06:08:37.108007+0000","last_unstale":"2024-09-21T06:08:39.052045+0000","last_undegraded":"2024-09-21T06:08:39.052045+0000","last_fullsized":"2024-09-21T06:08:39.052045+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:59.461401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.084321+0000","last_change":"2024-09-21T06:08:36.090186+0000","last_active":"2024-09-21T06:08:37.084321+0000","last_peered":"2024-09-21T06:08:37.084321+0000","last_clean":"2024-09-21T06:08:37.084321+0000","last_became_active":"2024-09-21T06:08:36.089660+0000","last_became_peered":"2024-09-21T06:08:36.089660+0000","last_unstale":"2024-09-21T06:08:37.084321+0000","last_undegraded":"2024-09-21T06:08:37.084321+0000","last_fullsized":"2024-09-21T06:08:37.084321+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:54.449692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103336+0000","last_change":"2024-09-21T06:08:37.098446+0000","last_active":"2024-09-21T06:08:37.103336+0000","last_peered":"2024-09-21T06:08:37.103336+0000","last_clean":"2024-09-21T06:08:37.103336+0000","last_became_active":"2024-09-21T06:08:37.098108+0000","last_became_peered":"2024-09-21T06:08:37.098108+0000","last_unstale":"2024-09-21T06:08:37.103336+0000","last_undegraded":"2024-09-21T06:08:37.103336+0000","last_fullsized":"2024-09-21T06:08:37.103336+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:17.553277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097889+0000","last_change":"2024-09-21T06:08:36.083937+0000","last_active":"2024-09-21T06:08:38.097889+0000","last_peered":"2024-09-21T06:08:38.097889+0000","last_clean":"2024-09-21T06:08:38.097889+0000","last_became_active":"2024-09-21T06:08:36.083578+0000","last_became_peered":"2024-09-21T06:08:36.083578+0000","last_unstale":"2024-09-21T06:08:38.097889+0000","last_undegraded":"2024-09-21T06:08:38.097889+0000","last_fullsized":"2024-09-21T06:08:38.097889+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:27:01.973768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.116340+0000","last_change":"2024-09-21T06:08:37.111778+0000","last_active":"2024-09-21T06:08:37.116340+0000","last_peered":"2024-09-21T06:08:37.116340+0000","last_clean":"2024-09-21T06:08:37.116340+0000","last_became_active":"2024-09-21T06:08:37.109252+0000","last_became_peered":"2024-09-21T06:08:37.109252+0000","last_unstale":"2024-09-21T06:08:37.116340+0000","last_undegraded":"2024-09-21T06:08:37.116340+0000","last_fullsized":"2024-09-21T06:08:37.116340+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:59.578813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.096342+0000","last_change":"2024-09-21T06:08:36.091546+0000","last_active":"2024-09-21T06:08:37.096342+0000","last_peered":"2024-09-21T06:08:37.096342+0000","last_clean":"2024-09-21T06:08:37.096342+0000","last_became_active":"2024-09-21T06:08:36.091225+0000","last_became_peered":"2024-09-21T06:08:36.091225+0000","last_unstale":"2024-09-21T06:08:37.096342+0000","last_undegraded":"2024-09-21T06:08:37.096342+0000","last_fullsized":"2024-09-21T06:08:37.096342+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:53.631589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.113784+0000","last_change":"2024-09-21T06:08:37.107171+0000","last_active":"2024-09-21T06:08:37.113784+0000","last_peered":"2024-09-21T06:08:37.113784+0000","last_clean":"2024-09-21T06:08:37.113784+0000","last_became_active":"2024-09-21T06:08:37.106783+0000","last_became_peered":"2024-09-21T06:08:37.106783+0000","last_unstale":"2024-09-21T06:08:37.113784+0000","last_undegraded":"2024-09-21T06:08:37.113784+0000","last_fullsized":"2024-09-21T06:08:37.113784+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:14.293953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.090604+0000","last_change":"2024-09-21T06:08:36.082851+0000","last_active":"2024-09-21T06:08:37.090604+0000","last_peered":"2024-09-21T06:08:37.090604+0000","last_clean":"2024-09-21T06:08:37.090604+0000","last_became_active":"2024-09-21T06:08:36.082525+0000","last_became_peered":"2024-09-21T06:08:36.082525+0000","last_unstale":"2024-09-21T06:08:37.090604+0000","last_undegraded":"2024-09-21T06:08:37.090604+0000","last_fullsized":"2024-09-21T06:08:37.090604+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:53.540272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052810+0000","last_change":"2024-09-21T06:08:37.107249+0000","last_active":"2024-09-21T06:08:39.052810+0000","last_peered":"2024-09-21T06:08:39.052810+0000","last_clean":"2024-09-21T06:08:39.052810+0000","last_became_active":"2024-09-21T06:08:37.106856+0000","last_became_peered":"2024-09-21T06:08:37.106856+0000","last_unstale":"2024-09-21T06:08:39.052810+0000","last_undegraded":"2024-09-21T06:08:39.052810+0000","last_fullsized":"2024-09-21T06:08:39.052810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:43.890366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052601+0000","last_change":"2024-09-21T06:08:36.093456+0000","last_active":"2024-09-21T06:08:39.052601+0000","last_peered":"2024-09-21T06:08:39.052601+0000","last_clean":"2024-09-21T06:08:39.052601+0000","last_became_active":"2024-09-21T06:08:36.092896+0000","last_became_peered":"2024-09-21T06:08:36.092896+0000","last_unstale":"2024-09-21T06:08:39.052601+0000","last_undegraded":"2024-09-21T06:08:39.052601+0000","last_fullsized":"2024-09-21T06:08:39.052601+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:33.653888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.101986+0000","last_change":"2024-09-21T06:08:37.097085+0000","last_active":"2024-09-21T06:08:37.101986+0000","last_peered":"2024-09-21T06:08:37.101986+0000","last_clean":"2024-09-21T06:08:37.101986+0000","last_became_active":"2024-09-21T06:08:37.096839+0000","last_became_peered":"2024-09-21T06:08:37.096839+0000","last_unstale":"2024-09-21T06:08:37.101986+0000","last_undegraded":"2024-09-21T06:08:37.101986+0000","last_fullsized":"2024-09-21T06:08:37.101986+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:26.638909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.090079+0000","last_change":"2024-09-21T06:08:36.082423+0000","last_active":"2024-09-21T06:08:37.090079+0000","last_peered":"2024-09-21T06:08:37.090079+0000","last_clean":"2024-09-21T06:08:37.090079+0000","last_became_active":"2024-09-21T06:08:36.082058+0000","last_became_peered":"2024-09-21T06:08:36.082058+0000","last_unstale":"2024-09-21T06:08:37.090079+0000","last_undegraded":"2024-09-21T06:08:37.090079+0000","last_fullsized":"2024-09-21T06:08:37.090079+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:15.296346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098453+0000","last_change":"2024-09-21T06:08:37.094800+0000","last_active":"2024-09-21T06:08:38.098453+0000","last_peered":"2024-09-21T06:08:38.098453+0000","last_clean":"2024-09-21T06:08:38.098453+0000","last_became_active":"2024-09-21T06:08:37.094448+0000","last_became_peered":"2024-09-21T06:08:37.094448+0000","last_unstale":"2024-09-21T06:08:38.098453+0000","last_undegraded":"2024-09-21T06:08:38.098453+0000","last_fullsized":"2024-09-21T06:08:38.098453+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:42.918938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.051916+0000","last_change":"2024-09-21T06:08:36.098401+0000","last_active":"2024-09-21T06:08:39.051916+0000","last_peered":"2024-09-21T06:08:39.051916+0000","last_clean":"2024-09-21T06:08:39.051916+0000","last_became_active":"2024-09-21T06:08:36.098009+0000","last_became_peered":"2024-09-21T06:08:36.098009+0000","last_unstale":"2024-09-21T06:08:39.051916+0000","last_undegraded":"2024-09-21T06:08:39.051916+0000","last_fullsized":"2024-09-21T06:08:39.051916+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:54.449692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.108270+0000","last_change":"2024-09-21T06:08:37.105350+0000","last_active":"2024-09-21T06:08:37.108270+0000","last_peered":"2024-09-21T06:08:37.108270+0000","last_clean":"2024-09-21T06:08:37.108270+0000","last_became_active":"2024-09-21T06:08:37.105033+0000","last_became_peered":"2024-09-21T06:08:37.105033+0000","last_unstale":"2024-09-21T06:08:37.108270+0000","last_undegraded":"2024-09-21T06:08:37.108270+0000","last_fullsized":"2024-09-21T06:08:37.108270+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:33.767005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096380+0000","last_change":"2024-09-21T06:08:36.099582+0000","last_active":"2024-09-21T06:08:38.096380+0000","last_peered":"2024-09-21T06:08:38.096380+0000","last_clean":"2024-09-21T06:08:38.096380+0000","last_became_active":"2024-09-21T06:08:36.098731+0000","last_became_peered":"2024-09-21T06:08:36.098731+0000","last_unstale":"2024-09-21T06:08:38.096380+0000","last_undegraded":"2024-09-21T06:08:38.096380+0000","last_fullsized":"2024-09-21T06:08:38.096380+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:55.862448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052095+0000","last_change":"2024-09-21T06:08:37.107049+0000","last_active":"2024-09-21T06:08:39.052095+0000","last_peered":"2024-09-21T06:08:39.052095+0000","last_clean":"2024-09-21T06:08:39.052095+0000","last_became_active":"2024-09-21T06:08:37.106728+0000","last_became_peered":"2024-09-21T06:08:37.106728+0000","last_unstale":"2024-09-21T06:08:39.052095+0000","last_undegraded":"2024-09-21T06:08:39.052095+0000","last_fullsized":"2024-09-21T06:08:39.052095+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:48:40.635993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.088165+0000","last_change":"2024-09-21T06:08:36.100472+0000","last_active":"2024-09-21T06:08:37.088165+0000","last_peered":"2024-09-21T06:08:37.088165+0000","last_clean":"2024-09-21T06:08:37.088165+0000","last_became_active":"2024-09-21T06:08:36.099971+0000","last_became_peered":"2024-09-21T06:08:36.099971+0000","last_unstale":"2024-09-21T06:08:37.088165+0000","last_undegraded":"2024-09-21T06:08:37.088165+0000","last_fullsized":"2024-09-21T06:08:37.088165+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:36.352220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.115853+0000","last_change":"2024-09-21T06:08:37.109520+0000","last_active":"2024-09-21T06:08:37.115853+0000","last_peered":"2024-09-21T06:08:37.115853+0000","last_clean":"2024-09-21T06:08:37.115853+0000","last_became_active":"2024-09-21T06:08:37.109147+0000","last_became_peered":"2024-09-21T06:08:37.109147+0000","last_unstale":"2024-09-21T06:08:37.115853+0000","last_undegraded":"2024-09-21T06:08:37.115853+0000","last_fullsized":"2024-09-21T06:08:37.115853+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:11.963080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"99'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.117719+0000","last_change":"2024-09-21T06:08:36.097402+0000","last_active":"2024-09-21T06:08:37.117719+0000","last_peered":"2024-09-21T06:08:37.117719+0000","last_clean":"2024-09-21T06:08:37.117719+0000","last_became_active":"2024-09-21T06:08:36.097157+0000","last_became_peered":"2024-09-21T06:08:36.097157+0000","last_unstale":"2024-09-21T06:08:37.117719+0000","last_undegraded":"2024-09-21T06:08:37.117719+0000","last_fullsized":"2024-09-21T06:08:37.117719+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:15.296346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":256,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.082618+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:08:37.082618+0000","last_peered":"2024-09-21T06:08:37.082618+0000","last_clean":"2024-09-21T06:08:37.082618+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:08:37.082618+0000","last_undegraded":"2024-09-21T06:08:37.082618+0000","last_fullsized":"2024-09-21T06:08:37.082618+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509591,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123140,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248700,"statfs":{"total":96636764160,"available":96510668800,"internally_reserved":0,"allocated":1818624,"data_stored":1115202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130892,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240948,"statfs":{"total":96636764160,"available":96502730752,"internally_reserved":0,"allocated":1236992,"data_stored":531730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76744,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295096,"statfs":{"total":96636764160,"available":96558178304,"internally_reserved":0,"allocated":1232896,"data_stored":531148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509591,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140636,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231204,"statfs":{"total":96636764160,"available":96492752896,"internally_reserved":0,"allocated":1249280,"data_stored":532814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130896,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240944,"statfs":{"total":96636764160,"available":96502726656,"internally_reserved":0,"allocated":1232896,"data_stored":526356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153084,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218756,"statfs":{"total":96636764160,"available":96480006144,"internally_reserved":0,"allocated":1220608,"data_stored":524756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509590,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131460,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240380,"statfs":{"total":96636764160,"available":96502149120,"internally_reserved":0,"allocated":1814528,"data_stored":1115672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542295,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140592,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231248,"statfs":{"total":96636764160,"available":96492797952,"internally_reserved":0,"allocated":1204224,"data_stored":517891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:08:41.295 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:08:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.491+0000 7f5046400640 1 -- 172.21.15.50:0/3202975042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50401527b0 msgr2=0x7f5040074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.491+0000 7f5046400640 1 --2- 172.21.15.50:0/3202975042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50401527b0 0x7f5040074e30 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5040072e30 tx=0x7f502801c800 comp rx=0 tx=0).stop 2024-09-21T06:08:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.491+0000 7f5046400640 1 -- 172.21.15.50:0/3202975042 shutdown_connections 2024-09-21T06:08:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.491+0000 7f5046400640 1 --2- 172.21.15.50:0/3202975042 >> v2:172.21.15.88:3300/0 conn(0x7f50400b4de0 0x7f50400b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.491+0000 7f5046400640 1 --2- 172.21.15.50:0/3202975042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50401527b0 0x7f5040074e30 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.491+0000 7f5046400640 1 --2- 172.21.15.50:0/3202975042 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50400b5c70 0x7f504015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.491+0000 7f5046400640 1 -- 172.21.15.50:0/3202975042 >> 172.21.15.50:0/3202975042 conn(0x7f50400b2040 msgr2=0x7f50400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.491+0000 7f5046400640 1 -- 172.21.15.50:0/3202975042 shutdown_connections 2024-09-21T06:08:41.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.491+0000 7f5046400640 1 -- 172.21.15.50:0/3202975042 wait complete. 2024-09-21T06:08:41.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.491+0000 7f5046400640 1 Processor -- start 2024-09-21T06:08:41.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.491+0000 7f5046400640 1 -- start start 2024-09-21T06:08:41.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5046400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50400b4de0 0x7f50400aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5046400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50400b5c70 0x7f5040144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5046400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50401527b0 0x7f50401f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5046400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f504015f6b0 con 0x7f50400b5c70 2024-09-21T06:08:41.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5046400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f504015f530 con 0x7f50400b4de0 2024-09-21T06:08:41.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5046400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f504015f830 con 0x7f50401527b0 2024-09-21T06:08:41.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5036a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50400b5c70 0x7f5040144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5036a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50400b5c70 0x7f5040144b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50132/0 (socket says 172.21.15.50:50132) 2024-09-21T06:08:41.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5036a00640 1 -- 172.21.15.50:0/2789139548 learned_addr learned my addr 172.21.15.50:0/2789139548 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:41.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f503f400640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50400b4de0 0x7f50400aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f503fe00640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50401527b0 0x7f50401f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5036a00640 1 -- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50401527b0 msgr2=0x7f50401f9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5036a00640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50401527b0 0x7f50401f9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5036a00640 1 -- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50400b4de0 msgr2=0x7f50400aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5036a00640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50400b4de0 0x7f50400aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5036a00640 1 -- 172.21.15.50:0/2789139548 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50400af4d0 con 0x7f50400b5c70 2024-09-21T06:08:41.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5036a00640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50400b5c70 0x7f5040144b90 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f502801aeb0 tx=0x7f50280a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:41.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5037e00640 1 -- 172.21.15.50:0/2789139548 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50280bb070 con 0x7f50400b5c70 2024-09-21T06:08:41.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5037e00640 1 -- 172.21.15.50:0/2789139548 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5028004430 con 0x7f50400b5c70 2024-09-21T06:08:41.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5037e00640 1 -- 172.21.15.50:0/2789139548 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50280b2040 con 0x7f50400b5c70 2024-09-21T06:08:41.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5046400640 1 -- 172.21.15.50:0/2789139548 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50400af7b0 con 0x7f50400b5c70 2024-09-21T06:08:41.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5046400640 1 -- 172.21.15.50:0/2789139548 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50400cdb30 con 0x7f50400b5c70 2024-09-21T06:08:41.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.495+0000 7f5046400640 1 -- 172.21.15.50:0/2789139548 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50400af240 con 0x7f50400b5c70 2024-09-21T06:08:41.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.499+0000 7f5037e00640 1 -- 172.21.15.50:0/2789139548 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f5028005290 con 0x7f50400b5c70 2024-09-21T06:08:41.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.499+0000 7f5037e00640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f501c074880 0x7f501c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.499+0000 7f5037e00640 1 -- 172.21.15.50:0/2789139548 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f502812ca60 con 0x7f50400b5c70 2024-09-21T06:08:41.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.499+0000 7f503f400640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f501c074880 0x7f501c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.499+0000 7f503f400640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f501c074880 0x7f501c076d90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5030004630 tx=0x7f503000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:41.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.503+0000 7f5037e00640 1 -- 172.21.15.50:0/2789139548 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f50280164b0 con 0x7f50400b5c70 2024-09-21T06:08:41.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.675+0000 7f5046400640 1 -- 172.21.15.50:0/2789139548 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f504015fce0 con 0x7f501c074880 2024-09-21T06:08:41.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.683+0000 7f5037e00640 1 -- 172.21.15.50:0/2789139548 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252130 (secure 0 0 0) 0x7f504015fce0 con 0x7f501c074880 2024-09-21T06:08:41.683 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:41.685 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:08:41.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.687+0000 7f5046400640 1 -- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f501c074880 msgr2=0x7f501c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.687+0000 7f5046400640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f501c074880 0x7f501c076d90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5030004630 tx=0x7f503000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.687+0000 7f5046400640 1 -- 172.21.15.50:0/2789139548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50400b5c70 msgr2=0x7f5040144b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.687+0000 7f5046400640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50400b5c70 0x7f5040144b90 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f502801aeb0 tx=0x7f50280a5790 comp rx=0 tx=0).stop 2024-09-21T06:08:41.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.687+0000 7f5046400640 1 -- 172.21.15.50:0/2789139548 shutdown_connections 2024-09-21T06:08:41.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.687+0000 7f5046400640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50401527b0 0x7f50401f9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.687+0000 7f5046400640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f501c074880 0x7f501c076d90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.687+0000 7f5046400640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50400b5c70 0x7f5040144b90 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.687+0000 7f5046400640 1 --2- 172.21.15.50:0/2789139548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50400b4de0 0x7f50400aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.687+0000 7f5046400640 1 -- 172.21.15.50:0/2789139548 >> 172.21.15.50:0/2789139548 conn(0x7f50400b2040 msgr2=0x7f5040154e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:41.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.687+0000 7f5046400640 1 -- 172.21.15.50:0/2789139548 shutdown_connections 2024-09-21T06:08:41.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.687+0000 7f5046400640 1 -- 172.21.15.50:0/2789139548 wait complete. 2024-09-21T06:08:41.706 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-21T06:08:41.164171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1027444,"kb_used_data":10752,"kb_used_omap":12,"kb_used_meta":1016499,"kb_avail":753947276,"statfs":{"total":773094113280,"available":772042010624,"internally_reserved":0,"allocated":11010048,"data_stored":5395569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1040895536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001679"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.115029+0000","last_change":"2024-09-21T06:08:37.111644+0000","last_active":"2024-09-21T06:08:37.115029+0000","last_peered":"2024-09-21T06:08:37.115029+0000","last_clean":"2024-09-21T06:08:37.115029+0000","last_became_active":"2024-09-21T06:08:37.109736+0000","last_became_peered":"2024-09-21T06:08:37.109736+0000","last_unstale":"2024-09-21T06:08:37.115029+0000","last_undegraded":"2024-09-21T06:08:37.115029+0000","last_fullsized":"2024-09-21T06:08:37.115029+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:18.299738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.108217+0000","last_change":"2024-09-21T06:08:37.103889+0000","last_active":"2024-09-21T06:08:37.108217+0000","last_peered":"2024-09-21T06:08:37.108217+0000","last_clean":"2024-09-21T06:08:37.108217+0000","last_became_active":"2024-09-21T06:08:37.103130+0000","last_became_peered":"2024-09-21T06:08:37.103130+0000","last_unstale":"2024-09-21T06:08:37.108217+0000","last_undegraded":"2024-09-21T06:08:37.108217+0000","last_fullsized":"2024-09-21T06:08:37.108217+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:58.527808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.118326+0000","last_change":"2024-09-21T06:08:37.114293+0000","last_active":"2024-09-21T06:08:37.118326+0000","last_peered":"2024-09-21T06:08:37.118326+0000","last_clean":"2024-09-21T06:08:37.118326+0000","last_became_active":"2024-09-21T06:08:37.113795+0000","last_became_peered":"2024-09-21T06:08:37.113795+0000","last_unstale":"2024-09-21T06:08:37.118326+0000","last_undegraded":"2024-09-21T06:08:37.118326+0000","last_fullsized":"2024-09-21T06:08:37.118326+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:38.848043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096880+0000","last_change":"2024-09-21T06:08:37.101236+0000","last_active":"2024-09-21T06:08:38.096880+0000","last_peered":"2024-09-21T06:08:38.096880+0000","last_clean":"2024-09-21T06:08:38.096880+0000","last_became_active":"2024-09-21T06:08:37.090014+0000","last_became_peered":"2024-09-21T06:08:37.090014+0000","last_unstale":"2024-09-21T06:08:38.096880+0000","last_undegraded":"2024-09-21T06:08:38.096880+0000","last_fullsized":"2024-09-21T06:08:38.096880+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:18.697748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052184+0000","last_change":"2024-09-21T06:08:37.109964+0000","last_active":"2024-09-21T06:08:39.052184+0000","last_peered":"2024-09-21T06:08:39.052184+0000","last_clean":"2024-09-21T06:08:39.052184+0000","last_became_active":"2024-09-21T06:08:37.109637+0000","last_became_peered":"2024-09-21T06:08:37.109637+0000","last_unstale":"2024-09-21T06:08:39.052184+0000","last_undegraded":"2024-09-21T06:08:39.052184+0000","last_fullsized":"2024-09-21T06:08:39.052184+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:56.569180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103088+0000","last_change":"2024-09-21T06:08:37.098656+0000","last_active":"2024-09-21T06:08:37.103088+0000","last_peered":"2024-09-21T06:08:37.103088+0000","last_clean":"2024-09-21T06:08:37.103088+0000","last_became_active":"2024-09-21T06:08:37.098236+0000","last_became_peered":"2024-09-21T06:08:37.098236+0000","last_unstale":"2024-09-21T06:08:37.103088+0000","last_undegraded":"2024-09-21T06:08:37.103088+0000","last_fullsized":"2024-09-21T06:08:37.103088+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:54.634271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103076+0000","last_change":"2024-09-21T06:08:37.098517+0000","last_active":"2024-09-21T06:08:37.103076+0000","last_peered":"2024-09-21T06:08:37.103076+0000","last_clean":"2024-09-21T06:08:37.103076+0000","last_became_active":"2024-09-21T06:08:37.098152+0000","last_became_peered":"2024-09-21T06:08:37.098152+0000","last_unstale":"2024-09-21T06:08:37.103076+0000","last_undegraded":"2024-09-21T06:08:37.103076+0000","last_fullsized":"2024-09-21T06:08:37.103076+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:10.378585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.102604+0000","last_change":"2024-09-21T06:08:37.094106+0000","last_active":"2024-09-21T06:08:37.102604+0000","last_peered":"2024-09-21T06:08:37.102604+0000","last_clean":"2024-09-21T06:08:37.102604+0000","last_became_active":"2024-09-21T06:08:37.093804+0000","last_became_peered":"2024-09-21T06:08:37.093804+0000","last_unstale":"2024-09-21T06:08:37.102604+0000","last_undegraded":"2024-09-21T06:08:37.102604+0000","last_fullsized":"2024-09-21T06:08:37.102604+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:42.918938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098883+0000","last_change":"2024-09-21T06:08:37.094564+0000","last_active":"2024-09-21T06:08:38.098883+0000","last_peered":"2024-09-21T06:08:38.098883+0000","last_clean":"2024-09-21T06:08:38.098883+0000","last_became_active":"2024-09-21T06:08:37.092224+0000","last_became_peered":"2024-09-21T06:08:37.092224+0000","last_unstale":"2024-09-21T06:08:38.098883+0000","last_undegraded":"2024-09-21T06:08:38.098883+0000","last_fullsized":"2024-09-21T06:08:38.098883+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:10.378585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052811+0000","last_change":"2024-09-21T06:08:37.112644+0000","last_active":"2024-09-21T06:08:39.052811+0000","last_peered":"2024-09-21T06:08:39.052811+0000","last_clean":"2024-09-21T06:08:39.052811+0000","last_became_active":"2024-09-21T06:08:37.112365+0000","last_became_peered":"2024-09-21T06:08:37.112365+0000","last_unstale":"2024-09-21T06:08:39.052811+0000","last_undegraded":"2024-09-21T06:08:39.052811+0000","last_fullsized":"2024-09-21T06:08:39.052811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:57.092581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103357+0000","last_change":"2024-09-21T06:08:37.100042+0000","last_active":"2024-09-21T06:08:37.103357+0000","last_peered":"2024-09-21T06:08:37.103357+0000","last_clean":"2024-09-21T06:08:37.103357+0000","last_became_active":"2024-09-21T06:08:37.099653+0000","last_became_peered":"2024-09-21T06:08:37.099653+0000","last_unstale":"2024-09-21T06:08:37.103357+0000","last_undegraded":"2024-09-21T06:08:37.103357+0000","last_fullsized":"2024-09-21T06:08:37.103357+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:20.619423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098116+0000","last_change":"2024-09-21T06:08:37.111023+0000","last_active":"2024-09-21T06:08:38.098116+0000","last_peered":"2024-09-21T06:08:38.098116+0000","last_clean":"2024-09-21T06:08:38.098116+0000","last_became_active":"2024-09-21T06:08:37.110651+0000","last_became_peered":"2024-09-21T06:08:37.110651+0000","last_unstale":"2024-09-21T06:08:38.098116+0000","last_undegraded":"2024-09-21T06:08:38.098116+0000","last_fullsized":"2024-09-21T06:08:38.098116+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:54.542954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096807+0000","last_change":"2024-09-21T06:08:37.102848+0000","last_active":"2024-09-21T06:08:38.096807+0000","last_peered":"2024-09-21T06:08:38.096807+0000","last_clean":"2024-09-21T06:08:38.096807+0000","last_became_active":"2024-09-21T06:08:37.102527+0000","last_became_peered":"2024-09-21T06:08:37.102527+0000","last_unstale":"2024-09-21T06:08:38.096807+0000","last_undegraded":"2024-09-21T06:08:38.096807+0000","last_fullsized":"2024-09-21T06:08:38.096807+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:57.092581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.101954+0000","last_change":"2024-09-21T06:08:37.094115+0000","last_active":"2024-09-21T06:08:37.101954+0000","last_peered":"2024-09-21T06:08:37.101954+0000","last_clean":"2024-09-21T06:08:37.101954+0000","last_became_active":"2024-09-21T06:08:37.093812+0000","last_became_peered":"2024-09-21T06:08:37.093812+0000","last_unstale":"2024-09-21T06:08:37.101954+0000","last_undegraded":"2024-09-21T06:08:37.101954+0000","last_fullsized":"2024-09-21T06:08:37.101954+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:23.803134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096227+0000","last_change":"2024-09-21T06:08:37.113343+0000","last_active":"2024-09-21T06:08:38.096227+0000","last_peered":"2024-09-21T06:08:38.096227+0000","last_clean":"2024-09-21T06:08:38.096227+0000","last_became_active":"2024-09-21T06:08:37.112810+0000","last_became_peered":"2024-09-21T06:08:37.112810+0000","last_unstale":"2024-09-21T06:08:38.096227+0000","last_undegraded":"2024-09-21T06:08:38.096227+0000","last_fullsized":"2024-09-21T06:08:38.096227+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:58.527808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.106132+0000","last_change":"2024-09-21T06:08:37.104315+0000","last_active":"2024-09-21T06:08:37.106132+0000","last_peered":"2024-09-21T06:08:37.106132+0000","last_clean":"2024-09-21T06:08:37.106132+0000","last_became_active":"2024-09-21T06:08:37.103962+0000","last_became_peered":"2024-09-21T06:08:37.103962+0000","last_unstale":"2024-09-21T06:08:37.106132+0000","last_undegraded":"2024-09-21T06:08:37.106132+0000","last_fullsized":"2024-09-21T06:08:37.106132+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:37.179525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103046+0000","last_change":"2024-09-21T06:08:37.096111+0000","last_active":"2024-09-21T06:08:37.103046+0000","last_peered":"2024-09-21T06:08:37.103046+0000","last_clean":"2024-09-21T06:08:37.103046+0000","last_became_active":"2024-09-21T06:08:37.095527+0000","last_became_peered":"2024-09-21T06:08:37.095527+0000","last_unstale":"2024-09-21T06:08:37.103046+0000","last_undegraded":"2024-09-21T06:08:37.103046+0000","last_fullsized":"2024-09-21T06:08:37.103046+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:10.378585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096920+0000","last_change":"2024-09-21T06:08:37.093479+0000","last_active":"2024-09-21T06:08:38.096920+0000","last_peered":"2024-09-21T06:08:38.096920+0000","last_clean":"2024-09-21T06:08:38.096920+0000","last_became_active":"2024-09-21T06:08:37.093080+0000","last_became_peered":"2024-09-21T06:08:37.093080+0000","last_unstale":"2024-09-21T06:08:38.096920+0000","last_undegraded":"2024-09-21T06:08:38.096920+0000","last_fullsized":"2024-09-21T06:08:38.096920+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:05.612097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.119214+0000","last_change":"2024-09-21T06:08:37.118195+0000","last_active":"2024-09-21T06:08:37.119214+0000","last_peered":"2024-09-21T06:08:37.119214+0000","last_clean":"2024-09-21T06:08:37.119214+0000","last_became_active":"2024-09-21T06:08:37.117853+0000","last_became_peered":"2024-09-21T06:08:37.117853+0000","last_unstale":"2024-09-21T06:08:37.119214+0000","last_undegraded":"2024-09-21T06:08:37.119214+0000","last_fullsized":"2024-09-21T06:08:37.119214+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:45.882864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098168+0000","last_change":"2024-09-21T06:08:37.093053+0000","last_active":"2024-09-21T06:08:38.098168+0000","last_peered":"2024-09-21T06:08:38.098168+0000","last_clean":"2024-09-21T06:08:38.098168+0000","last_became_active":"2024-09-21T06:08:37.092756+0000","last_became_peered":"2024-09-21T06:08:37.092756+0000","last_unstale":"2024-09-21T06:08:38.098168+0000","last_undegraded":"2024-09-21T06:08:38.098168+0000","last_fullsized":"2024-09-21T06:08:38.098168+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:12.213858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103026+0000","last_change":"2024-09-21T06:08:37.092381+0000","last_active":"2024-09-21T06:08:37.103026+0000","last_peered":"2024-09-21T06:08:37.103026+0000","last_clean":"2024-09-21T06:08:37.103026+0000","last_became_active":"2024-09-21T06:08:37.092047+0000","last_became_peered":"2024-09-21T06:08:37.092047+0000","last_unstale":"2024-09-21T06:08:37.103026+0000","last_undegraded":"2024-09-21T06:08:37.103026+0000","last_fullsized":"2024-09-21T06:08:37.103026+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:40.148267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.114756+0000","last_change":"2024-09-21T06:08:37.110704+0000","last_active":"2024-09-21T06:08:37.114756+0000","last_peered":"2024-09-21T06:08:37.114756+0000","last_clean":"2024-09-21T06:08:37.114756+0000","last_became_active":"2024-09-21T06:08:37.110329+0000","last_became_peered":"2024-09-21T06:08:37.110329+0000","last_unstale":"2024-09-21T06:08:37.114756+0000","last_undegraded":"2024-09-21T06:08:37.114756+0000","last_fullsized":"2024-09-21T06:08:37.114756+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:12.316093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.102852+0000","last_change":"2024-09-21T06:08:37.097419+0000","last_active":"2024-09-21T06:08:37.102852+0000","last_peered":"2024-09-21T06:08:37.102852+0000","last_clean":"2024-09-21T06:08:37.102852+0000","last_became_active":"2024-09-21T06:08:37.097111+0000","last_became_peered":"2024-09-21T06:08:37.097111+0000","last_unstale":"2024-09-21T06:08:37.102852+0000","last_undegraded":"2024-09-21T06:08:37.102852+0000","last_fullsized":"2024-09-21T06:08:37.102852+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:20.619423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.098424+0000","last_change":"2024-09-21T06:08:37.096439+0000","last_active":"2024-09-21T06:08:37.098424+0000","last_peered":"2024-09-21T06:08:37.098424+0000","last_clean":"2024-09-21T06:08:37.098424+0000","last_became_active":"2024-09-21T06:08:37.095640+0000","last_became_peered":"2024-09-21T06:08:37.095640+0000","last_unstale":"2024-09-21T06:08:37.098424+0000","last_undegraded":"2024-09-21T06:08:37.098424+0000","last_fullsized":"2024-09-21T06:08:37.098424+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:33.956003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052905+0000","last_change":"2024-09-21T06:08:37.105728+0000","last_active":"2024-09-21T06:08:39.052905+0000","last_peered":"2024-09-21T06:08:39.052905+0000","last_clean":"2024-09-21T06:08:39.052905+0000","last_became_active":"2024-09-21T06:08:37.104902+0000","last_became_peered":"2024-09-21T06:08:37.104902+0000","last_unstale":"2024-09-21T06:08:39.052905+0000","last_undegraded":"2024-09-21T06:08:39.052905+0000","last_fullsized":"2024-09-21T06:08:39.052905+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:37.177983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.115664+0000","last_change":"2024-09-21T06:08:37.112467+0000","last_active":"2024-09-21T06:08:37.115664+0000","last_peered":"2024-09-21T06:08:37.115664+0000","last_clean":"2024-09-21T06:08:37.115664+0000","last_became_active":"2024-09-21T06:08:37.110782+0000","last_became_peered":"2024-09-21T06:08:37.110782+0000","last_unstale":"2024-09-21T06:08:37.115664+0000","last_undegraded":"2024-09-21T06:08:37.115664+0000","last_fullsized":"2024-09-21T06:08:37.115664+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:24.499024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.096128+0000","last_change":"2024-09-21T06:08:37.093012+0000","last_active":"2024-09-21T06:08:37.096128+0000","last_peered":"2024-09-21T06:08:37.096128+0000","last_clean":"2024-09-21T06:08:37.096128+0000","last_became_active":"2024-09-21T06:08:37.092656+0000","last_became_peered":"2024-09-21T06:08:37.092656+0000","last_unstale":"2024-09-21T06:08:37.096128+0000","last_undegraded":"2024-09-21T06:08:37.096128+0000","last_fullsized":"2024-09-21T06:08:37.096128+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:42.918938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052348+0000","last_change":"2024-09-21T06:08:37.115603+0000","last_active":"2024-09-21T06:08:39.052348+0000","last_peered":"2024-09-21T06:08:39.052348+0000","last_clean":"2024-09-21T06:08:39.052348+0000","last_became_active":"2024-09-21T06:08:37.115374+0000","last_became_peered":"2024-09-21T06:08:37.115374+0000","last_unstale":"2024-09-21T06:08:39.052348+0000","last_undegraded":"2024-09-21T06:08:39.052348+0000","last_fullsized":"2024-09-21T06:08:39.052348+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:07.241230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.118330+0000","last_change":"2024-09-21T06:08:37.115735+0000","last_active":"2024-09-21T06:08:37.118330+0000","last_peered":"2024-09-21T06:08:37.118330+0000","last_clean":"2024-09-21T06:08:37.118330+0000","last_became_active":"2024-09-21T06:08:37.115427+0000","last_became_peered":"2024-09-21T06:08:37.115427+0000","last_unstale":"2024-09-21T06:08:37.118330+0000","last_undegraded":"2024-09-21T06:08:37.118330+0000","last_fullsized":"2024-09-21T06:08:37.118330+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:39.123832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.118383+0000","last_change":"2024-09-21T06:08:37.117199+0000","last_active":"2024-09-21T06:08:37.118383+0000","last_peered":"2024-09-21T06:08:37.118383+0000","last_clean":"2024-09-21T06:08:37.118383+0000","last_became_active":"2024-09-21T06:08:37.116852+0000","last_became_peered":"2024-09-21T06:08:37.116852+0000","last_unstale":"2024-09-21T06:08:37.118383+0000","last_undegraded":"2024-09-21T06:08:37.118383+0000","last_fullsized":"2024-09-21T06:08:37.118383+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:39.736419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096340+0000","last_change":"2024-09-21T06:08:37.093396+0000","last_active":"2024-09-21T06:08:38.096340+0000","last_peered":"2024-09-21T06:08:38.096340+0000","last_clean":"2024-09-21T06:08:38.096340+0000","last_became_active":"2024-09-21T06:08:37.093067+0000","last_became_peered":"2024-09-21T06:08:37.093067+0000","last_unstale":"2024-09-21T06:08:38.096340+0000","last_undegraded":"2024-09-21T06:08:38.096340+0000","last_fullsized":"2024-09-21T06:08:38.096340+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:24.499024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.115213+0000","last_change":"2024-09-21T06:08:37.104312+0000","last_active":"2024-09-21T06:08:37.115213+0000","last_peered":"2024-09-21T06:08:37.115213+0000","last_clean":"2024-09-21T06:08:37.115213+0000","last_became_active":"2024-09-21T06:08:37.103982+0000","last_became_peered":"2024-09-21T06:08:37.103982+0000","last_unstale":"2024-09-21T06:08:37.115213+0000","last_undegraded":"2024-09-21T06:08:37.115213+0000","last_fullsized":"2024-09-21T06:08:37.115213+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:12.827691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.092207+0000","last_change":"2024-09-21T06:08:37.090139+0000","last_active":"2024-09-21T06:08:37.092207+0000","last_peered":"2024-09-21T06:08:37.092207+0000","last_clean":"2024-09-21T06:08:37.092207+0000","last_became_active":"2024-09-21T06:08:37.089814+0000","last_became_peered":"2024-09-21T06:08:37.089814+0000","last_unstale":"2024-09-21T06:08:37.092207+0000","last_undegraded":"2024-09-21T06:08:37.092207+0000","last_fullsized":"2024-09-21T06:08:37.092207+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:23.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"99'4","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.113493+0000","last_change":"2024-09-21T06:08:36.087950+0000","last_active":"2024-09-21T06:08:37.113493+0000","last_peered":"2024-09-21T06:08:37.113493+0000","last_clean":"2024-09-21T06:08:37.113493+0000","last_became_active":"2024-09-21T06:08:36.087621+0000","last_became_peered":"2024-09-21T06:08:36.087621+0000","last_unstale":"2024-09-21T06:08:37.113493+0000","last_undegraded":"2024-09-21T06:08:37.113493+0000","last_fullsized":"2024-09-21T06:08:37.113493+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:32.953321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097088+0000","last_change":"2024-09-21T06:08:37.095434+0000","last_active":"2024-09-21T06:08:38.097088+0000","last_peered":"2024-09-21T06:08:38.097088+0000","last_clean":"2024-09-21T06:08:38.097088+0000","last_became_active":"2024-09-21T06:08:37.094577+0000","last_became_peered":"2024-09-21T06:08:37.094577+0000","last_unstale":"2024-09-21T06:08:38.097088+0000","last_undegraded":"2024-09-21T06:08:38.097088+0000","last_fullsized":"2024-09-21T06:08:38.097088+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:13.833258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"99'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.112539+0000","last_change":"2024-09-21T06:08:36.098530+0000","last_active":"2024-09-21T06:08:37.112539+0000","last_peered":"2024-09-21T06:08:37.112539+0000","last_clean":"2024-09-21T06:08:37.112539+0000","last_became_active":"2024-09-21T06:08:36.098150+0000","last_became_peered":"2024-09-21T06:08:36.098150+0000","last_unstale":"2024-09-21T06:08:37.112539+0000","last_undegraded":"2024-09-21T06:08:37.112539+0000","last_fullsized":"2024-09-21T06:08:37.112539+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:55.862448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.114088+0000","last_change":"2024-09-21T06:08:37.110080+0000","last_active":"2024-09-21T06:08:37.114088+0000","last_peered":"2024-09-21T06:08:37.114088+0000","last_clean":"2024-09-21T06:08:37.114088+0000","last_became_active":"2024-09-21T06:08:37.109702+0000","last_became_peered":"2024-09-21T06:08:37.109702+0000","last_unstale":"2024-09-21T06:08:37.114088+0000","last_undegraded":"2024-09-21T06:08:37.114088+0000","last_fullsized":"2024-09-21T06:08:37.114088+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:39.118789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"99'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.115234+0000","last_change":"2024-09-21T06:08:36.092070+0000","last_active":"2024-09-21T06:08:37.115234+0000","last_peered":"2024-09-21T06:08:37.115234+0000","last_clean":"2024-09-21T06:08:37.115234+0000","last_became_active":"2024-09-21T06:08:36.090747+0000","last_became_peered":"2024-09-21T06:08:36.090747+0000","last_unstale":"2024-09-21T06:08:37.115234+0000","last_undegraded":"2024-09-21T06:08:37.115234+0000","last_fullsized":"2024-09-21T06:08:37.115234+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:55.862448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098828+0000","last_change":"2024-09-21T06:08:37.085484+0000","last_active":"2024-09-21T06:08:38.098828+0000","last_peered":"2024-09-21T06:08:38.098828+0000","last_clean":"2024-09-21T06:08:38.098828+0000","last_became_active":"2024-09-21T06:08:37.085167+0000","last_became_peered":"2024-09-21T06:08:37.085167+0000","last_unstale":"2024-09-21T06:08:38.098828+0000","last_undegraded":"2024-09-21T06:08:38.098828+0000","last_fullsized":"2024-09-21T06:08:38.098828+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:33.956003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"99'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.114307+0000","last_change":"2024-09-21T06:08:36.085410+0000","last_active":"2024-09-21T06:08:37.114307+0000","last_peered":"2024-09-21T06:08:37.114307+0000","last_clean":"2024-09-21T06:08:37.114307+0000","last_became_active":"2024-09-21T06:08:36.085051+0000","last_became_peered":"2024-09-21T06:08:36.085051+0000","last_unstale":"2024-09-21T06:08:37.114307+0000","last_undegraded":"2024-09-21T06:08:37.114307+0000","last_fullsized":"2024-09-21T06:08:37.114307+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:43.773479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097582+0000","last_change":"2024-09-21T06:08:37.082292+0000","last_active":"2024-09-21T06:08:38.097582+0000","last_peered":"2024-09-21T06:08:38.097582+0000","last_clean":"2024-09-21T06:08:38.097582+0000","last_became_active":"2024-09-21T06:08:37.081859+0000","last_became_peered":"2024-09-21T06:08:37.081859+0000","last_unstale":"2024-09-21T06:08:38.097582+0000","last_undegraded":"2024-09-21T06:08:38.097582+0000","last_fullsized":"2024-09-21T06:08:38.097582+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:19.735552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"99'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.113848+0000","last_change":"2024-09-21T06:08:36.092630+0000","last_active":"2024-09-21T06:08:37.113848+0000","last_peered":"2024-09-21T06:08:37.113848+0000","last_clean":"2024-09-21T06:08:37.113848+0000","last_became_active":"2024-09-21T06:08:36.092361+0000","last_became_peered":"2024-09-21T06:08:36.092361+0000","last_unstale":"2024-09-21T06:08:37.113848+0000","last_undegraded":"2024-09-21T06:08:37.113848+0000","last_fullsized":"2024-09-21T06:08:37.113848+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:00.867561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.094024+0000","last_change":"2024-09-21T06:08:37.091778+0000","last_active":"2024-09-21T06:08:37.094024+0000","last_peered":"2024-09-21T06:08:37.094024+0000","last_clean":"2024-09-21T06:08:37.094024+0000","last_became_active":"2024-09-21T06:08:37.091438+0000","last_became_peered":"2024-09-21T06:08:37.091438+0000","last_unstale":"2024-09-21T06:08:37.094024+0000","last_undegraded":"2024-09-21T06:08:37.094024+0000","last_fullsized":"2024-09-21T06:08:37.094024+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:12.213858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"99'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.118714+0000","last_change":"2024-09-21T06:08:36.096536+0000","last_active":"2024-09-21T06:08:37.118714+0000","last_peered":"2024-09-21T06:08:37.118714+0000","last_clean":"2024-09-21T06:08:37.118714+0000","last_became_active":"2024-09-21T06:08:36.096220+0000","last_became_peered":"2024-09-21T06:08:36.096220+0000","last_unstale":"2024-09-21T06:08:37.118714+0000","last_undegraded":"2024-09-21T06:08:37.118714+0000","last_fullsized":"2024-09-21T06:08:37.118714+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:06.942640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096258+0000","last_change":"2024-09-21T06:08:37.096405+0000","last_active":"2024-09-21T06:08:38.096258+0000","last_peered":"2024-09-21T06:08:38.096258+0000","last_clean":"2024-09-21T06:08:38.096258+0000","last_became_active":"2024-09-21T06:08:37.095778+0000","last_became_peered":"2024-09-21T06:08:37.095778+0000","last_unstale":"2024-09-21T06:08:38.096258+0000","last_undegraded":"2024-09-21T06:08:38.096258+0000","last_fullsized":"2024-09-21T06:08:38.096258+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:12.316093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052184+0000","last_change":"2024-09-21T06:08:36.085333+0000","last_active":"2024-09-21T06:08:39.052184+0000","last_peered":"2024-09-21T06:08:39.052184+0000","last_clean":"2024-09-21T06:08:39.052184+0000","last_became_active":"2024-09-21T06:08:36.085016+0000","last_became_peered":"2024-09-21T06:08:36.085016+0000","last_unstale":"2024-09-21T06:08:39.052184+0000","last_undegraded":"2024-09-21T06:08:39.052184+0000","last_fullsized":"2024-09-21T06:08:39.052184+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:54.451399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097745+0000","last_change":"2024-09-21T06:08:37.094683+0000","last_active":"2024-09-21T06:08:38.097745+0000","last_peered":"2024-09-21T06:08:38.097745+0000","last_clean":"2024-09-21T06:08:38.097745+0000","last_became_active":"2024-09-21T06:08:37.090846+0000","last_became_peered":"2024-09-21T06:08:37.090846+0000","last_unstale":"2024-09-21T06:08:38.097745+0000","last_undegraded":"2024-09-21T06:08:38.097745+0000","last_fullsized":"2024-09-21T06:08:38.097745+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:56.865129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.096210+0000","last_change":"2024-09-21T06:08:36.098091+0000","last_active":"2024-09-21T06:08:37.096210+0000","last_peered":"2024-09-21T06:08:37.096210+0000","last_clean":"2024-09-21T06:08:37.096210+0000","last_became_active":"2024-09-21T06:08:36.097765+0000","last_became_peered":"2024-09-21T06:08:36.097765+0000","last_unstale":"2024-09-21T06:08:37.096210+0000","last_undegraded":"2024-09-21T06:08:37.096210+0000","last_fullsized":"2024-09-21T06:08:37.096210+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:14.038348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.106332+0000","last_change":"2024-09-21T06:08:37.102036+0000","last_active":"2024-09-21T06:08:37.106332+0000","last_peered":"2024-09-21T06:08:37.106332+0000","last_clean":"2024-09-21T06:08:37.106332+0000","last_became_active":"2024-09-21T06:08:37.101133+0000","last_became_peered":"2024-09-21T06:08:37.101133+0000","last_unstale":"2024-09-21T06:08:37.106332+0000","last_undegraded":"2024-09-21T06:08:37.106332+0000","last_fullsized":"2024-09-21T06:08:37.106332+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:37.179525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.098960+0000","last_change":"2024-09-21T06:08:36.095764+0000","last_active":"2024-09-21T06:08:37.098960+0000","last_peered":"2024-09-21T06:08:37.098960+0000","last_clean":"2024-09-21T06:08:37.098960+0000","last_became_active":"2024-09-21T06:08:36.095460+0000","last_became_peered":"2024-09-21T06:08:36.095460+0000","last_unstale":"2024-09-21T06:08:37.098960+0000","last_undegraded":"2024-09-21T06:08:37.098960+0000","last_fullsized":"2024-09-21T06:08:37.098960+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:42.891663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.099116+0000","last_change":"2024-09-21T06:08:37.083020+0000","last_active":"2024-09-21T06:08:38.099116+0000","last_peered":"2024-09-21T06:08:38.099116+0000","last_clean":"2024-09-21T06:08:38.099116+0000","last_became_active":"2024-09-21T06:08:37.082730+0000","last_became_peered":"2024-09-21T06:08:37.082730+0000","last_unstale":"2024-09-21T06:08:38.099116+0000","last_undegraded":"2024-09-21T06:08:38.099116+0000","last_fullsized":"2024-09-21T06:08:38.099116+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:54.634271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096796+0000","last_change":"2024-09-21T06:08:36.086708+0000","last_active":"2024-09-21T06:08:38.096796+0000","last_peered":"2024-09-21T06:08:38.096796+0000","last_clean":"2024-09-21T06:08:38.096796+0000","last_became_active":"2024-09-21T06:08:36.086359+0000","last_became_peered":"2024-09-21T06:08:36.086359+0000","last_unstale":"2024-09-21T06:08:38.096796+0000","last_undegraded":"2024-09-21T06:08:38.096796+0000","last_fullsized":"2024-09-21T06:08:38.096796+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:53.540272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052649+0000","last_change":"2024-09-21T06:08:37.111290+0000","last_active":"2024-09-21T06:08:39.052649+0000","last_peered":"2024-09-21T06:08:39.052649+0000","last_clean":"2024-09-21T06:08:39.052649+0000","last_became_active":"2024-09-21T06:08:37.110943+0000","last_became_peered":"2024-09-21T06:08:37.110943+0000","last_unstale":"2024-09-21T06:08:39.052649+0000","last_undegraded":"2024-09-21T06:08:39.052649+0000","last_fullsized":"2024-09-21T06:08:39.052649+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.909923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"99'3","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.109116+0000","last_change":"2024-09-21T06:08:36.093355+0000","last_active":"2024-09-21T06:08:37.109116+0000","last_peered":"2024-09-21T06:08:37.109116+0000","last_clean":"2024-09-21T06:08:37.109116+0000","last_became_active":"2024-09-21T06:08:36.093000+0000","last_became_peered":"2024-09-21T06:08:36.093000+0000","last_unstale":"2024-09-21T06:08:37.109116+0000","last_undegraded":"2024-09-21T06:08:37.109116+0000","last_fullsized":"2024-09-21T06:08:37.109116+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:53.540272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.106151+0000","last_change":"2024-09-21T06:08:37.104527+0000","last_active":"2024-09-21T06:08:37.106151+0000","last_peered":"2024-09-21T06:08:37.106151+0000","last_clean":"2024-09-21T06:08:37.106151+0000","last_became_active":"2024-09-21T06:08:37.104089+0000","last_became_peered":"2024-09-21T06:08:37.104089+0000","last_unstale":"2024-09-21T06:08:37.106151+0000","last_undegraded":"2024-09-21T06:08:37.106151+0000","last_fullsized":"2024-09-21T06:08:37.106151+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:58.527808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"99'6","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098249+0000","last_change":"2024-09-21T06:08:36.094885+0000","last_active":"2024-09-21T06:08:38.098249+0000","last_peered":"2024-09-21T06:08:38.098249+0000","last_clean":"2024-09-21T06:08:38.098249+0000","last_became_active":"2024-09-21T06:08:36.094591+0000","last_became_peered":"2024-09-21T06:08:36.094591+0000","last_unstale":"2024-09-21T06:08:38.098249+0000","last_undegraded":"2024-09-21T06:08:38.098249+0000","last_fullsized":"2024-09-21T06:08:38.098249+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:23.503527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096437+0000","last_change":"2024-09-21T06:08:37.098055+0000","last_active":"2024-09-21T06:08:38.096437+0000","last_peered":"2024-09-21T06:08:38.096437+0000","last_clean":"2024-09-21T06:08:38.096437+0000","last_became_active":"2024-09-21T06:08:37.097705+0000","last_became_peered":"2024-09-21T06:08:37.097705+0000","last_unstale":"2024-09-21T06:08:38.096437+0000","last_undegraded":"2024-09-21T06:08:38.096437+0000","last_fullsized":"2024-09-21T06:08:38.096437+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:39.123832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.112788+0000","last_change":"2024-09-21T06:08:36.098400+0000","last_active":"2024-09-21T06:08:37.112788+0000","last_peered":"2024-09-21T06:08:37.112788+0000","last_clean":"2024-09-21T06:08:37.112788+0000","last_became_active":"2024-09-21T06:08:36.098061+0000","last_became_peered":"2024-09-21T06:08:36.098061+0000","last_unstale":"2024-09-21T06:08:37.112788+0000","last_undegraded":"2024-09-21T06:08:37.112788+0000","last_fullsized":"2024-09-21T06:08:37.112788+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:07.664854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.117654+0000","last_change":"2024-09-21T06:08:37.114079+0000","last_active":"2024-09-21T06:08:37.117654+0000","last_peered":"2024-09-21T06:08:37.117654+0000","last_clean":"2024-09-21T06:08:37.117654+0000","last_became_active":"2024-09-21T06:08:37.109467+0000","last_became_peered":"2024-09-21T06:08:37.109467+0000","last_unstale":"2024-09-21T06:08:37.117654+0000","last_undegraded":"2024-09-21T06:08:37.117654+0000","last_fullsized":"2024-09-21T06:08:37.117654+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:12.148391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.090051+0000","last_change":"2024-09-21T06:08:36.099380+0000","last_active":"2024-09-21T06:08:37.090051+0000","last_peered":"2024-09-21T06:08:37.090051+0000","last_clean":"2024-09-21T06:08:37.090051+0000","last_became_active":"2024-09-21T06:08:36.099069+0000","last_became_peered":"2024-09-21T06:08:36.099069+0000","last_unstale":"2024-09-21T06:08:37.090051+0000","last_undegraded":"2024-09-21T06:08:37.090051+0000","last_fullsized":"2024-09-21T06:08:37.090051+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:15.875075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097312+0000","last_change":"2024-09-21T06:08:37.091273+0000","last_active":"2024-09-21T06:08:38.097312+0000","last_peered":"2024-09-21T06:08:38.097312+0000","last_clean":"2024-09-21T06:08:38.097312+0000","last_became_active":"2024-09-21T06:08:37.084101+0000","last_became_peered":"2024-09-21T06:08:37.084101+0000","last_unstale":"2024-09-21T06:08:38.097312+0000","last_undegraded":"2024-09-21T06:08:38.097312+0000","last_fullsized":"2024-09-21T06:08:38.097312+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:16.299027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097570+0000","last_change":"2024-09-21T06:08:36.087689+0000","last_active":"2024-09-21T06:08:38.097570+0000","last_peered":"2024-09-21T06:08:38.097570+0000","last_clean":"2024-09-21T06:08:38.097570+0000","last_became_active":"2024-09-21T06:08:36.087388+0000","last_became_peered":"2024-09-21T06:08:36.087388+0000","last_unstale":"2024-09-21T06:08:38.097570+0000","last_undegraded":"2024-09-21T06:08:38.097570+0000","last_fullsized":"2024-09-21T06:08:38.097570+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:47.602151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.106841+0000","last_change":"2024-09-21T06:08:37.099943+0000","last_active":"2024-09-21T06:08:37.106841+0000","last_peered":"2024-09-21T06:08:37.106841+0000","last_clean":"2024-09-21T06:08:37.106841+0000","last_became_active":"2024-09-21T06:08:37.099643+0000","last_became_peered":"2024-09-21T06:08:37.099643+0000","last_unstale":"2024-09-21T06:08:37.106841+0000","last_undegraded":"2024-09-21T06:08:37.106841+0000","last_fullsized":"2024-09-21T06:08:37.106841+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"99'8","reported_seq":21,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.115160+0000","last_change":"2024-09-21T06:08:36.098044+0000","last_active":"2024-09-21T06:08:37.115160+0000","last_peered":"2024-09-21T06:08:37.115160+0000","last_clean":"2024-09-21T06:08:37.115160+0000","last_became_active":"2024-09-21T06:08:36.097723+0000","last_became_peered":"2024-09-21T06:08:36.097723+0000","last_unstale":"2024-09-21T06:08:37.115160+0000","last_undegraded":"2024-09-21T06:08:37.115160+0000","last_fullsized":"2024-09-21T06:08:37.115160+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:06.103687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052883+0000","last_change":"2024-09-21T06:08:37.099722+0000","last_active":"2024-09-21T06:08:39.052883+0000","last_peered":"2024-09-21T06:08:39.052883+0000","last_clean":"2024-09-21T06:08:39.052883+0000","last_became_active":"2024-09-21T06:08:37.099357+0000","last_became_peered":"2024-09-21T06:08:37.099357+0000","last_unstale":"2024-09-21T06:08:39.052883+0000","last_undegraded":"2024-09-21T06:08:39.052883+0000","last_fullsized":"2024-09-21T06:08:39.052883+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:17.894972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098623+0000","last_change":"2024-09-21T06:08:36.091038+0000","last_active":"2024-09-21T06:08:38.098623+0000","last_peered":"2024-09-21T06:08:38.098623+0000","last_clean":"2024-09-21T06:08:38.098623+0000","last_became_active":"2024-09-21T06:08:36.090716+0000","last_became_peered":"2024-09-21T06:08:36.090716+0000","last_unstale":"2024-09-21T06:08:38.098623+0000","last_undegraded":"2024-09-21T06:08:38.098623+0000","last_fullsized":"2024-09-21T06:08:38.098623+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:42.804603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097893+0000","last_change":"2024-09-21T06:08:37.089104+0000","last_active":"2024-09-21T06:08:38.097893+0000","last_peered":"2024-09-21T06:08:38.097893+0000","last_clean":"2024-09-21T06:08:38.097893+0000","last_became_active":"2024-09-21T06:08:37.088741+0000","last_became_peered":"2024-09-21T06:08:37.088741+0000","last_unstale":"2024-09-21T06:08:38.097893+0000","last_undegraded":"2024-09-21T06:08:38.097893+0000","last_fullsized":"2024-09-21T06:08:38.097893+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:25.961882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052668+0000","last_change":"2024-09-21T06:08:36.088687+0000","last_active":"2024-09-21T06:08:39.052668+0000","last_peered":"2024-09-21T06:08:39.052668+0000","last_clean":"2024-09-21T06:08:39.052668+0000","last_became_active":"2024-09-21T06:08:36.087706+0000","last_became_peered":"2024-09-21T06:08:36.087706+0000","last_unstale":"2024-09-21T06:08:39.052668+0000","last_undegraded":"2024-09-21T06:08:39.052668+0000","last_fullsized":"2024-09-21T06:08:39.052668+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:43.773479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103385+0000","last_change":"2024-09-21T06:08:37.100290+0000","last_active":"2024-09-21T06:08:37.103385+0000","last_peered":"2024-09-21T06:08:37.103385+0000","last_clean":"2024-09-21T06:08:37.103385+0000","last_became_active":"2024-09-21T06:08:37.100029+0000","last_became_peered":"2024-09-21T06:08:37.100029+0000","last_unstale":"2024-09-21T06:08:37.103385+0000","last_undegraded":"2024-09-21T06:08:37.103385+0000","last_fullsized":"2024-09-21T06:08:37.103385+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:17.553277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.091292+0000","last_change":"2024-09-21T06:08:36.084755+0000","last_active":"2024-09-21T06:08:37.091292+0000","last_peered":"2024-09-21T06:08:37.091292+0000","last_clean":"2024-09-21T06:08:37.091292+0000","last_became_active":"2024-09-21T06:08:36.084379+0000","last_became_peered":"2024-09-21T06:08:36.084379+0000","last_unstale":"2024-09-21T06:08:37.091292+0000","last_undegraded":"2024-09-21T06:08:37.091292+0000","last_fullsized":"2024-09-21T06:08:37.091292+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:11.211176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.114120+0000","last_change":"2024-09-21T06:08:37.107882+0000","last_active":"2024-09-21T06:08:37.114120+0000","last_peered":"2024-09-21T06:08:37.114120+0000","last_clean":"2024-09-21T06:08:37.114120+0000","last_became_active":"2024-09-21T06:08:37.107578+0000","last_became_peered":"2024-09-21T06:08:37.107578+0000","last_unstale":"2024-09-21T06:08:37.114120+0000","last_undegraded":"2024-09-21T06:08:37.114120+0000","last_fullsized":"2024-09-21T06:08:37.114120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:26.479609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"99'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.116533+0000","last_change":"2024-09-21T06:08:36.091770+0000","last_active":"2024-09-21T06:08:37.116533+0000","last_peered":"2024-09-21T06:08:37.116533+0000","last_clean":"2024-09-21T06:08:37.116533+0000","last_became_active":"2024-09-21T06:08:36.091443+0000","last_became_peered":"2024-09-21T06:08:36.091443+0000","last_unstale":"2024-09-21T06:08:37.116533+0000","last_undegraded":"2024-09-21T06:08:37.116533+0000","last_fullsized":"2024-09-21T06:08:37.116533+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:33.653888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103078+0000","last_change":"2024-09-21T06:08:37.098615+0000","last_active":"2024-09-21T06:08:37.103078+0000","last_peered":"2024-09-21T06:08:37.103078+0000","last_clean":"2024-09-21T06:08:37.103078+0000","last_became_active":"2024-09-21T06:08:37.098314+0000","last_became_peered":"2024-09-21T06:08:37.098314+0000","last_unstale":"2024-09-21T06:08:37.103078+0000","last_undegraded":"2024-09-21T06:08:37.103078+0000","last_fullsized":"2024-09-21T06:08:37.103078+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:44.498607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097503+0000","last_change":"2024-09-21T06:08:36.095525+0000","last_active":"2024-09-21T06:08:38.097503+0000","last_peered":"2024-09-21T06:08:38.097503+0000","last_clean":"2024-09-21T06:08:38.097503+0000","last_became_active":"2024-09-21T06:08:36.095158+0000","last_became_peered":"2024-09-21T06:08:36.095158+0000","last_unstale":"2024-09-21T06:08:38.097503+0000","last_undegraded":"2024-09-21T06:08:38.097503+0000","last_fullsized":"2024-09-21T06:08:38.097503+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:26.521534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.121347+0000","last_change":"2024-09-21T06:08:37.119864+0000","last_active":"2024-09-21T06:08:37.121347+0000","last_peered":"2024-09-21T06:08:37.121347+0000","last_clean":"2024-09-21T06:08:37.121347+0000","last_became_active":"2024-09-21T06:08:37.116834+0000","last_became_peered":"2024-09-21T06:08:37.116834+0000","last_unstale":"2024-09-21T06:08:37.121347+0000","last_undegraded":"2024-09-21T06:08:37.121347+0000","last_fullsized":"2024-09-21T06:08:37.121347+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:05.816651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097355+0000","last_change":"2024-09-21T06:08:36.085965+0000","last_active":"2024-09-21T06:08:38.097355+0000","last_peered":"2024-09-21T06:08:38.097355+0000","last_clean":"2024-09-21T06:08:38.097355+0000","last_became_active":"2024-09-21T06:08:36.085620+0000","last_became_peered":"2024-09-21T06:08:36.085620+0000","last_unstale":"2024-09-21T06:08:38.097355+0000","last_undegraded":"2024-09-21T06:08:38.097355+0000","last_fullsized":"2024-09-21T06:08:38.097355+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:44.707072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052045+0000","last_change":"2024-09-21T06:08:37.108407+0000","last_active":"2024-09-21T06:08:39.052045+0000","last_peered":"2024-09-21T06:08:39.052045+0000","last_clean":"2024-09-21T06:08:39.052045+0000","last_became_active":"2024-09-21T06:08:37.108007+0000","last_became_peered":"2024-09-21T06:08:37.108007+0000","last_unstale":"2024-09-21T06:08:39.052045+0000","last_undegraded":"2024-09-21T06:08:39.052045+0000","last_fullsized":"2024-09-21T06:08:39.052045+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:59.461401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.084321+0000","last_change":"2024-09-21T06:08:36.090186+0000","last_active":"2024-09-21T06:08:37.084321+0000","last_peered":"2024-09-21T06:08:37.084321+0000","last_clean":"2024-09-21T06:08:37.084321+0000","last_became_active":"2024-09-21T06:08:36.089660+0000","last_became_peered":"2024-09-21T06:08:36.089660+0000","last_unstale":"2024-09-21T06:08:37.084321+0000","last_undegraded":"2024-09-21T06:08:37.084321+0000","last_fullsized":"2024-09-21T06:08:37.084321+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:54.449692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.103336+0000","last_change":"2024-09-21T06:08:37.098446+0000","last_active":"2024-09-21T06:08:37.103336+0000","last_peered":"2024-09-21T06:08:37.103336+0000","last_clean":"2024-09-21T06:08:37.103336+0000","last_became_active":"2024-09-21T06:08:37.098108+0000","last_became_peered":"2024-09-21T06:08:37.098108+0000","last_unstale":"2024-09-21T06:08:37.103336+0000","last_undegraded":"2024-09-21T06:08:37.103336+0000","last_fullsized":"2024-09-21T06:08:37.103336+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:17.553277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.097889+0000","last_change":"2024-09-21T06:08:36.083937+0000","last_active":"2024-09-21T06:08:38.097889+0000","last_peered":"2024-09-21T06:08:38.097889+0000","last_clean":"2024-09-21T06:08:38.097889+0000","last_became_active":"2024-09-21T06:08:36.083578+0000","last_became_peered":"2024-09-21T06:08:36.083578+0000","last_unstale":"2024-09-21T06:08:38.097889+0000","last_undegraded":"2024-09-21T06:08:38.097889+0000","last_fullsized":"2024-09-21T06:08:38.097889+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:27:01.973768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.116340+0000","last_change":"2024-09-21T06:08:37.111778+0000","last_active":"2024-09-21T06:08:37.116340+0000","last_peered":"2024-09-21T06:08:37.116340+0000","last_clean":"2024-09-21T06:08:37.116340+0000","last_became_active":"2024-09-21T06:08:37.109252+0000","last_became_peered":"2024-09-21T06:08:37.109252+0000","last_unstale":"2024-09-21T06:08:37.116340+0000","last_undegraded":"2024-09-21T06:08:37.116340+0000","last_fullsized":"2024-09-21T06:08:37.116340+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:59.578813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.096342+0000","last_change":"2024-09-21T06:08:36.091546+0000","last_active":"2024-09-21T06:08:37.096342+0000","last_peered":"2024-09-21T06:08:37.096342+0000","last_clean":"2024-09-21T06:08:37.096342+0000","last_became_active":"2024-09-21T06:08:36.091225+0000","last_became_peered":"2024-09-21T06:08:36.091225+0000","last_unstale":"2024-09-21T06:08:37.096342+0000","last_undegraded":"2024-09-21T06:08:37.096342+0000","last_fullsized":"2024-09-21T06:08:37.096342+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:53.631589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.113784+0000","last_change":"2024-09-21T06:08:37.107171+0000","last_active":"2024-09-21T06:08:37.113784+0000","last_peered":"2024-09-21T06:08:37.113784+0000","last_clean":"2024-09-21T06:08:37.113784+0000","last_became_active":"2024-09-21T06:08:37.106783+0000","last_became_peered":"2024-09-21T06:08:37.106783+0000","last_unstale":"2024-09-21T06:08:37.113784+0000","last_undegraded":"2024-09-21T06:08:37.113784+0000","last_fullsized":"2024-09-21T06:08:37.113784+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:14.293953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.090604+0000","last_change":"2024-09-21T06:08:36.082851+0000","last_active":"2024-09-21T06:08:37.090604+0000","last_peered":"2024-09-21T06:08:37.090604+0000","last_clean":"2024-09-21T06:08:37.090604+0000","last_became_active":"2024-09-21T06:08:36.082525+0000","last_became_peered":"2024-09-21T06:08:36.082525+0000","last_unstale":"2024-09-21T06:08:37.090604+0000","last_undegraded":"2024-09-21T06:08:37.090604+0000","last_fullsized":"2024-09-21T06:08:37.090604+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:53.540272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052810+0000","last_change":"2024-09-21T06:08:37.107249+0000","last_active":"2024-09-21T06:08:39.052810+0000","last_peered":"2024-09-21T06:08:39.052810+0000","last_clean":"2024-09-21T06:08:39.052810+0000","last_became_active":"2024-09-21T06:08:37.106856+0000","last_became_peered":"2024-09-21T06:08:37.106856+0000","last_unstale":"2024-09-21T06:08:39.052810+0000","last_undegraded":"2024-09-21T06:08:39.052810+0000","last_fullsized":"2024-09-21T06:08:39.052810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:43.890366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052601+0000","last_change":"2024-09-21T06:08:36.093456+0000","last_active":"2024-09-21T06:08:39.052601+0000","last_peered":"2024-09-21T06:08:39.052601+0000","last_clean":"2024-09-21T06:08:39.052601+0000","last_became_active":"2024-09-21T06:08:36.092896+0000","last_became_peered":"2024-09-21T06:08:36.092896+0000","last_unstale":"2024-09-21T06:08:39.052601+0000","last_undegraded":"2024-09-21T06:08:39.052601+0000","last_fullsized":"2024-09-21T06:08:39.052601+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:33.653888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.101986+0000","last_change":"2024-09-21T06:08:37.097085+0000","last_active":"2024-09-21T06:08:37.101986+0000","last_peered":"2024-09-21T06:08:37.101986+0000","last_clean":"2024-09-21T06:08:37.101986+0000","last_became_active":"2024-09-21T06:08:37.096839+0000","last_became_peered":"2024-09-21T06:08:37.096839+0000","last_unstale":"2024-09-21T06:08:37.101986+0000","last_undegraded":"2024-09-21T06:08:37.101986+0000","last_fullsized":"2024-09-21T06:08:37.101986+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:26.638909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.090079+0000","last_change":"2024-09-21T06:08:36.082423+0000","last_active":"2024-09-21T06:08:37.090079+0000","last_peered":"2024-09-21T06:08:37.090079+0000","last_clean":"2024-09-21T06:08:37.090079+0000","last_became_active":"2024-09-21T06:08:36.082058+0000","last_became_peered":"2024-09-21T06:08:36.082058+0000","last_unstale":"2024-09-21T06:08:37.090079+0000","last_undegraded":"2024-09-21T06:08:37.090079+0000","last_fullsized":"2024-09-21T06:08:37.090079+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:15.296346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.098453+0000","last_change":"2024-09-21T06:08:37.094800+0000","last_active":"2024-09-21T06:08:38.098453+0000","last_peered":"2024-09-21T06:08:38.098453+0000","last_clean":"2024-09-21T06:08:38.098453+0000","last_became_active":"2024-09-21T06:08:37.094448+0000","last_became_peered":"2024-09-21T06:08:37.094448+0000","last_unstale":"2024-09-21T06:08:38.098453+0000","last_undegraded":"2024-09-21T06:08:38.098453+0000","last_fullsized":"2024-09-21T06:08:38.098453+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:42.918938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.051916+0000","last_change":"2024-09-21T06:08:36.098401+0000","last_active":"2024-09-21T06:08:39.051916+0000","last_peered":"2024-09-21T06:08:39.051916+0000","last_clean":"2024-09-21T06:08:39.051916+0000","last_became_active":"2024-09-21T06:08:36.098009+0000","last_became_peered":"2024-09-21T06:08:36.098009+0000","last_unstale":"2024-09-21T06:08:39.051916+0000","last_undegraded":"2024-09-21T06:08:39.051916+0000","last_fullsized":"2024-09-21T06:08:39.051916+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:54.449692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.108270+0000","last_change":"2024-09-21T06:08:37.105350+0000","last_active":"2024-09-21T06:08:37.108270+0000","last_peered":"2024-09-21T06:08:37.108270+0000","last_clean":"2024-09-21T06:08:37.108270+0000","last_became_active":"2024-09-21T06:08:37.105033+0000","last_became_peered":"2024-09-21T06:08:37.105033+0000","last_unstale":"2024-09-21T06:08:37.108270+0000","last_undegraded":"2024-09-21T06:08:37.108270+0000","last_fullsized":"2024-09-21T06:08:37.108270+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:33.767005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:38.096380+0000","last_change":"2024-09-21T06:08:36.099582+0000","last_active":"2024-09-21T06:08:38.096380+0000","last_peered":"2024-09-21T06:08:38.096380+0000","last_clean":"2024-09-21T06:08:38.096380+0000","last_became_active":"2024-09-21T06:08:36.098731+0000","last_became_peered":"2024-09-21T06:08:36.098731+0000","last_unstale":"2024-09-21T06:08:38.096380+0000","last_undegraded":"2024-09-21T06:08:38.096380+0000","last_fullsized":"2024-09-21T06:08:38.096380+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:55.862448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T06:08:39.052095+0000","last_change":"2024-09-21T06:08:37.107049+0000","last_active":"2024-09-21T06:08:39.052095+0000","last_peered":"2024-09-21T06:08:39.052095+0000","last_clean":"2024-09-21T06:08:39.052095+0000","last_became_active":"2024-09-21T06:08:37.106728+0000","last_became_peered":"2024-09-21T06:08:37.106728+0000","last_unstale":"2024-09-21T06:08:39.052095+0000","last_undegraded":"2024-09-21T06:08:39.052095+0000","last_fullsized":"2024-09-21T06:08:39.052095+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:48:40.635993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.088165+0000","last_change":"2024-09-21T06:08:36.100472+0000","last_active":"2024-09-21T06:08:37.088165+0000","last_peered":"2024-09-21T06:08:37.088165+0000","last_clean":"2024-09-21T06:08:37.088165+0000","last_became_active":"2024-09-21T06:08:36.099971+0000","last_became_peered":"2024-09-21T06:08:36.099971+0000","last_unstale":"2024-09-21T06:08:37.088165+0000","last_undegraded":"2024-09-21T06:08:37.088165+0000","last_fullsized":"2024-09-21T06:08:37.088165+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:36.352220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.115853+0000","last_change":"2024-09-21T06:08:37.109520+0000","last_active":"2024-09-21T06:08:37.115853+0000","last_peered":"2024-09-21T06:08:37.115853+0000","last_clean":"2024-09-21T06:08:37.115853+0000","last_became_active":"2024-09-21T06:08:37.109147+0000","last_became_peered":"2024-09-21T06:08:37.109147+0000","last_unstale":"2024-09-21T06:08:37.115853+0000","last_undegraded":"2024-09-21T06:08:37.115853+0000","last_fullsized":"2024-09-21T06:08:37.115853+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:36.062719+0000","last_clean_scrub_stamp":"2024-09-21T06:08:36.062719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:11.963080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"99'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.117719+0000","last_change":"2024-09-21T06:08:36.097402+0000","last_active":"2024-09-21T06:08:37.117719+0000","last_peered":"2024-09-21T06:08:37.117719+0000","last_clean":"2024-09-21T06:08:37.117719+0000","last_became_active":"2024-09-21T06:08:36.097157+0000","last_became_peered":"2024-09-21T06:08:36.097157+0000","last_unstale":"2024-09-21T06:08:37.117719+0000","last_undegraded":"2024-09-21T06:08:37.117719+0000","last_fullsized":"2024-09-21T06:08:37.117719+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:35.060037+0000","last_clean_scrub_stamp":"2024-09-21T06:08:35.060037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:15.296346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":256,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T06:08:37.082618+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:08:37.082618+0000","last_peered":"2024-09-21T06:08:37.082618+0000","last_clean":"2024-09-21T06:08:37.082618+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:08:37.082618+0000","last_undegraded":"2024-09-21T06:08:37.082618+0000","last_fullsized":"2024-09-21T06:08:37.082618+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509591,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123140,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248700,"statfs":{"total":96636764160,"available":96510668800,"internally_reserved":0,"allocated":1818624,"data_stored":1115202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130892,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240948,"statfs":{"total":96636764160,"available":96502730752,"internally_reserved":0,"allocated":1236992,"data_stored":531730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76744,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295096,"statfs":{"total":96636764160,"available":96558178304,"internally_reserved":0,"allocated":1232896,"data_stored":531148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509591,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140636,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231204,"statfs":{"total":96636764160,"available":96492752896,"internally_reserved":0,"allocated":1249280,"data_stored":532814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130896,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240944,"statfs":{"total":96636764160,"available":96502726656,"internally_reserved":0,"allocated":1232896,"data_stored":526356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153084,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218756,"statfs":{"total":96636764160,"available":96480006144,"internally_reserved":0,"allocated":1220608,"data_stored":524756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509590,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131460,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240380,"statfs":{"total":96636764160,"available":96502149120,"internally_reserved":0,"allocated":1814528,"data_stored":1115672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542295,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140592,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231248,"statfs":{"total":96636764160,"available":96492797952,"internally_reserved":0,"allocated":1204224,"data_stored":517891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:08:41.710 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:08:41.711 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:08:41.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.915+0000 7f0faae00640 1 -- 172.21.15.50:0/4109116838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa40b5a80 msgr2=0x7f0fa4074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.915+0000 7f0faae00640 1 --2- 172.21.15.50:0/4109116838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa40b5a80 0x7f0fa4074410 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009840 tx=0x7f0f8c01c820 comp rx=0 tx=0).stop 2024-09-21T06:08:41.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.915+0000 7f0faae00640 1 -- 172.21.15.50:0/4109116838 shutdown_connections 2024-09-21T06:08:41.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.915+0000 7f0faae00640 1 --2- 172.21.15.50:0/4109116838 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0fa4074950 0x7f0fa415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.915+0000 7f0faae00640 1 --2- 172.21.15.50:0/4109116838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa40b5a80 0x7f0fa4074410 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.915+0000 7f0faae00640 1 -- 172.21.15.50:0/4109116838 >> 172.21.15.50:0/4109116838 conn(0x7f0fa40708c0 msgr2=0x7f0fa4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:41.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.915+0000 7f0faae00640 1 -- 172.21.15.50:0/4109116838 shutdown_connections 2024-09-21T06:08:41.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.915+0000 7f0faae00640 1 -- 172.21.15.50:0/4109116838 wait complete. 2024-09-21T06:08:41.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.915+0000 7f0faae00640 1 Processor -- start 2024-09-21T06:08:41.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.915+0000 7f0faae00640 1 -- start start 2024-09-21T06:08:41.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0faae00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa4074950 0x7f0fa41435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0faae00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0fa40b5a80 0x7f0fa4143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0faae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fa4157030 0x7f0fa41f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0faae00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa416a310 con 0x7f0fa4074950 2024-09-21T06:08:41.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0faae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa416a190 con 0x7f0fa4157030 2024-09-21T06:08:41.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0faae00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa416a490 con 0x7f0fa40b5a80 2024-09-21T06:08:41.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa4074950 0x7f0fa41435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa4074950 0x7f0fa41435b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50154/0 (socket says 172.21.15.50:50154) 2024-09-21T06:08:41.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa3e00640 1 -- 172.21.15.50:0/101383120 learned_addr learned my addr 172.21.15.50:0/101383120 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:41.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0f9be00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0fa40b5a80 0x7f0fa4143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa8e00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fa4157030 0x7f0fa41f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa3e00640 1 -- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0fa40b5a80 msgr2=0x7f0fa4143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa3e00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0fa40b5a80 0x7f0fa4143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa3e00640 1 -- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fa4157030 msgr2=0x7f0fa41f86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa3e00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fa4157030 0x7f0fa41f86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa3e00640 1 -- 172.21.15.50:0/101383120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa414d880 con 0x7f0fa4074950 2024-09-21T06:08:41.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0f9be00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0fa40b5a80 0x7f0fa4143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:41.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa8e00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fa4157030 0x7f0fa41f86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:41.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa3e00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa4074950 0x7f0fa41435b0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0f9400c760 tx=0x7f0f9400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:41.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa0c00640 1 -- 172.21.15.50:0/101383120 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94016020 con 0x7f0fa4074950 2024-09-21T06:08:41.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa0c00640 1 -- 172.21.15.50:0/101383120 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f9400a950 con 0x7f0fa4074950 2024-09-21T06:08:41.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0fa0c00640 1 -- 172.21.15.50:0/101383120 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9400ada0 con 0x7f0fa4074950 2024-09-21T06:08:41.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0faae00640 1 -- 172.21.15.50:0/101383120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa414db60 con 0x7f0fa4074950 2024-09-21T06:08:41.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0faae00640 1 -- 172.21.15.50:0/101383120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa414e120 con 0x7f0fa4074950 2024-09-21T06:08:41.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.919+0000 7f0faae00640 1 -- 172.21.15.50:0/101383120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa415cf90 con 0x7f0fa4074950 2024-09-21T06:08:41.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.923+0000 7f0fa0c00640 1 -- 172.21.15.50:0/101383120 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f0f94019020 con 0x7f0fa4074950 2024-09-21T06:08:41.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.923+0000 7f0fa0c00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f80074880 0x7f0f80076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.923+0000 7f0fa0c00640 1 -- 172.21.15.50:0/101383120 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f0f94092e80 con 0x7f0fa4074950 2024-09-21T06:08:41.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.923+0000 7f0f9be00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f80074880 0x7f0f80076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.923+0000 7f0f9be00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f80074880 0x7f0f80076d90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0af000 tx=0x7f0f8c01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:41.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:41.927+0000 7f0fa0c00640 1 -- 172.21.15.50:0/101383120 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0f94020020 con 0x7f0fa4074950 2024-09-21T06:08:42.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.171+0000 7f0faae00640 1 -- 172.21.15.50:0/101383120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0fa40d2260 con 0x7f0fa4074950 2024-09-21T06:08:42.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0fa0c00640 1 -- 172.21.15.50:0/101383120 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v36) ==== 190+0+147 (secure 0 0 0) 0x7f0fa4074d60 con 0x7f0fa4074950 2024-09-21T06:08:42.174 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-09-21T06:08:42.174 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw== 2024-09-21T06:08:42.174 INFO:teuthology.orchestra.run.smithi050.stdout: caps mds = "allow" 2024-09-21T06:08:42.174 INFO:teuthology.orchestra.run.smithi050.stdout: caps mon = "allow r" 2024-09-21T06:08:42.174 INFO:teuthology.orchestra.run.smithi050.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:08:42.174 INFO:teuthology.orchestra.run.smithi050.stderr:updated caps for client.0 2024-09-21T06:08:42.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0faae00640 1 -- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f80074880 msgr2=0x7f0f80076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0faae00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f80074880 0x7f0f80076d90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0af000 tx=0x7f0f8c01b060 comp rx=0 tx=0).stop 2024-09-21T06:08:42.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0faae00640 1 -- 172.21.15.50:0/101383120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa4074950 msgr2=0x7f0fa41435b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0faae00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa4074950 0x7f0fa41435b0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0f9400c760 tx=0x7f0f9400cc20 comp rx=0 tx=0).stop 2024-09-21T06:08:42.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0faae00640 1 -- 172.21.15.50:0/101383120 shutdown_connections 2024-09-21T06:08:42.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0faae00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fa4157030 0x7f0fa41f86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0faae00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0fa40b5a80 0x7f0fa4143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0faae00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f80074880 0x7f0f80076d90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0faae00640 1 --2- 172.21.15.50:0/101383120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa4074950 0x7f0fa41435b0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0faae00640 1 -- 172.21.15.50:0/101383120 >> 172.21.15.50:0/101383120 conn(0x7f0fa40708c0 msgr2=0x7f0fa4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:42.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0faae00640 1 -- 172.21.15.50:0/101383120 shutdown_connections 2024-09-21T06:08:42.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.175+0000 7f0faae00640 1 -- 172.21.15.50:0/101383120 wait complete. 2024-09-21T06:08:42.197 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:08:42.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.355+0000 7f60e5600640 1 -- 172.21.15.50:0/2858187451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c4004810 msgr2=0x7f60c4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.355+0000 7f60e5600640 1 --2- 172.21.15.50:0/2858187451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c4004810 0x7f60c4004c20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009b60 tx=0x7f60cc01d240 comp rx=0 tx=0).stop 2024-09-21T06:08:42.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.355+0000 7f60e5600640 1 -- 172.21.15.50:0/2858187451 shutdown_connections 2024-09-21T06:08:42.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.355+0000 7f60e5600640 1 --2- 172.21.15.50:0/2858187451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c4004810 0x7f60c4004c20 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.355+0000 7f60e5600640 1 -- 172.21.15.50:0/2858187451 >> 172.21.15.50:0/2858187451 conn(0x7f60e00b2040 msgr2=0x7f60e00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:42.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.355+0000 7f60e5600640 1 -- 172.21.15.50:0/2858187451 shutdown_connections 2024-09-21T06:08:42.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.355+0000 7f60e5600640 1 -- 172.21.15.50:0/2858187451 wait complete. 2024-09-21T06:08:42.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60e5600640 1 Processor -- start 2024-09-21T06:08:42.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60e5600640 1 -- start start 2024-09-21T06:08:42.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60e5600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60c4004810 0x7f60e0149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:42.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60e5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e00b3cd0 0x7f60e00b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:42.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60e5600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f60e01fa7d0 0x7f60e01fabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:42.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60e5600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f60e00b5690 con 0x7f60e00b3cd0 2024-09-21T06:08:42.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60e5600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60e00b5510 con 0x7f60c4004810 2024-09-21T06:08:42.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60e5600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f60e015f180 con 0x7f60e01fa7d0 2024-09-21T06:08:42.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60de000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e00b3cd0 0x7f60e00b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:42.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60de000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e00b3cd0 0x7f60e00b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50178/0 (socket says 172.21.15.50:50178) 2024-09-21T06:08:42.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60de000640 1 -- 172.21.15.50:0/4182712759 learned_addr learned my addr 172.21.15.50:0/4182712759 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:42.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60dea00640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60c4004810 0x7f60e0149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:42.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.359+0000 7f60df400640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f60e01fa7d0 0x7f60e01fabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:42.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60de000640 1 -- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f60e01fa7d0 msgr2=0x7f60e01fabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60de000640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f60e01fa7d0 0x7f60e01fabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60de000640 1 -- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60c4004810 msgr2=0x7f60e0149ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60de000640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60c4004810 0x7f60e0149ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60de000640 1 -- 172.21.15.50:0/4182712759 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e00b4620 con 0x7f60e00b3cd0 2024-09-21T06:08:42.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60dea00640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60c4004810 0x7f60e0149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:42.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60de000640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e00b3cd0 0x7f60e00b40e0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f60cc01d070 tx=0x7f60cc005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:42.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60d6a00640 1 -- 172.21.15.50:0/4182712759 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0bd070 con 0x7f60e00b3cd0 2024-09-21T06:08:42.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60d6a00640 1 -- 172.21.15.50:0/4182712759 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60cc004120 con 0x7f60e00b3cd0 2024-09-21T06:08:42.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60d6a00640 1 -- 172.21.15.50:0/4182712759 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0b1040 con 0x7f60e00b3cd0 2024-09-21T06:08:42.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60e5600640 1 -- 172.21.15.50:0/4182712759 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e01f7d00 con 0x7f60e00b3cd0 2024-09-21T06:08:42.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60e5600640 1 -- 172.21.15.50:0/4182712759 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e01f6f40 con 0x7f60e00b3cd0 2024-09-21T06:08:42.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60e5600640 1 -- 172.21.15.50:0/4182712759 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e014a1c0 con 0x7f60e00b3cd0 2024-09-21T06:08:42.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.363+0000 7f60d6a00640 1 -- 172.21.15.50:0/4182712759 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f60cc0c0070 con 0x7f60e00b3cd0 2024-09-21T06:08:42.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.367+0000 7f60d6a00640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f60c0074880 0x7f60c0076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:42.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.367+0000 7f60d6a00640 1 -- 172.21.15.50:0/4182712759 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f60cc12d520 con 0x7f60e00b3cd0 2024-09-21T06:08:42.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.367+0000 7f60dea00640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f60c0074880 0x7f60c0076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:42.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.367+0000 7f60dea00640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f60c0074880 0x7f60c0076d90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f60d0006640 tx=0x7f60d0006cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:42.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.371+0000 7f60d6a00640 1 -- 172.21.15.50:0/4182712759 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f60e014a1c0 con 0x7f60e00b3cd0 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.579+0000 7f60e5600640 1 -- 172.21.15.50:0/4182712759 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60e015f670 con 0x7f60e00b3cd0 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.579+0000 7f60d6a00640 1 -- 172.21.15.50:0/4182712759 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5222 (secure 0 0 0) 0x7f60e00b4620 con 0x7f60e00b3cd0 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":65,"btime":"2024-09-21T06:08:40:104993+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2607826086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2607826086},{"type":"v1","addr":"172.21.15.50:6835","nonce":2607826086}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1771536088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1771536088},{"type":"v1","addr":"172.21.15.88:6837","nonce":1771536088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/433905290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":433905290},{"type":"v1","addr":"172.21.15.88:6836","nonce":433905290}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:08:36.508641+0000","modified":"2024-09-21T06:08:39.519558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1418616207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1418616207},{"type":"v1","addr":"172.21.15.50:6834","nonce":1418616207}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7851,"qdb_cluster":[7851]},"id":7}]} 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 65 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.583+0000 7f60e5600640 1 -- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f60c0074880 msgr2=0x7f60c0076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.583+0000 7f60e5600640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f60c0074880 0x7f60c0076d90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f60d0006640 tx=0x7f60d0006cf0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.583+0000 7f60e5600640 1 -- 172.21.15.50:0/4182712759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e00b3cd0 msgr2=0x7f60e00b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.583+0000 7f60e5600640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e00b3cd0 0x7f60e00b40e0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f60cc01d070 tx=0x7f60cc005520 comp rx=0 tx=0).stop 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.583+0000 7f60e5600640 1 -- 172.21.15.50:0/4182712759 shutdown_connections 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.583+0000 7f60e5600640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f60e01fa7d0 0x7f60e01fabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.583+0000 7f60e5600640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f60c0074880 0x7f60c0076d90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.583+0000 7f60e5600640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e00b3cd0 0x7f60e00b40e0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.583+0000 7f60e5600640 1 --2- 172.21.15.50:0/4182712759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60c4004810 0x7f60e0149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.583+0000 7f60e5600640 1 -- 172.21.15.50:0/4182712759 >> 172.21.15.50:0/4182712759 conn(0x7f60e00b2040 msgr2=0x7f60e015a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.583+0000 7f60e5600640 1 -- 172.21.15.50:0/4182712759 shutdown_connections 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:42.583+0000 7f60e5600640 1 -- 172.21.15.50:0/4182712759 wait complete. 2024-09-21T06:08:43.005 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:08:36.508641+0000', 'modified': '2024-09-21T06:08:39.519558+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7851}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7851': {'gid': 7851, 'name': 'c', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6834/1418616207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6832', 'nonce': 1418616207}, {'type': 'v1', 'addr': '172.21.15.50:6834', 'nonce': 1418616207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7851, 'qdb_cluster': [7851]} 2024-09-21T06:08:43.005 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:08:43.006 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T06:08:43.043 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-09-21T06:08:43.043+0000 7efdb9a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.171+0000 7fdfefe00640 1 -- 172.21.15.50:0/524280514 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdff01536f0 msgr2=0x7fdff015d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.171+0000 7fdfefe00640 1 --2- 172.21.15.50:0/524280514 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdff01536f0 0x7fdff015d480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009840 tx=0x7fdfe001c790 comp rx=0 tx=0).stop 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.171+0000 7fdfefe00640 1 -- 172.21.15.50:0/524280514 shutdown_connections 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.171+0000 7fdfefe00640 1 --2- 172.21.15.50:0/524280514 >> v2:172.21.15.88:3300/0 conn(0x7fdff0074950 0x7fdff00b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.171+0000 7fdfefe00640 1 --2- 172.21.15.50:0/524280514 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdff01536f0 0x7fdff015d480 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.171+0000 7fdfefe00640 1 -- 172.21.15.50:0/524280514 >> 172.21.15.50:0/524280514 conn(0x7fdff00b4560 msgr2=0x7fdff00d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.171+0000 7fdfefe00640 1 -- 172.21.15.50:0/524280514 shutdown_connections 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.171+0000 7fdfefe00640 1 -- 172.21.15.50:0/524280514 wait complete. 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.171+0000 7fdfefe00640 1 Processor -- start 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.171+0000 7fdfefe00640 1 -- start start 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfefe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdff01536f0 0x7fdff0145c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfefe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff0160060 0x7fdff0156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfefe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdff01610d0 0x7fdff0156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfefe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0169dc0 con 0x7fdff01536f0 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfefe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0169c40 con 0x7fdff0160060 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfefe00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fdff0169f40 con 0x7fdff01610d0 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfeea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdff01536f0 0x7fdff0145c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfeea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdff01536f0 0x7fdff0145c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50192/0 (socket says 172.21.15.50:50192) 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfeea00640 1 -- 172.21.15.50:0/1122745052 learned_addr learned my addr 172.21.15.50:0/1122745052 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfe6000640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff0160060 0x7fdff0156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfef400640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdff01610d0 0x7fdff0156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfeea00640 1 -- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdff01610d0 msgr2=0x7fdff0156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfeea00640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdff01610d0 0x7fdff0156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfeea00640 1 -- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff0160060 msgr2=0x7fdff0156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfeea00640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff0160060 0x7fdff0156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfeea00640 1 -- 172.21.15.50:0/1122745052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff0161840 con 0x7fdff01536f0 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfef400640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdff01610d0 0x7fdff0156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfe6000640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff0160060 0x7fdff0156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfeea00640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdff01536f0 0x7fdff0145c30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0098170 tx=0x7fdfe0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfe7e00640 1 -- 172.21.15.50:0/1122745052 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe00be070 con 0x7fdff01536f0 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfefe00640 1 -- 172.21.15.50:0/1122745052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff0161b20 con 0x7fdff01536f0 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfe7e00640 1 -- 172.21.15.50:0/1122745052 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfe00a8a80 con 0x7fdff01536f0 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfe7e00640 1 -- 172.21.15.50:0/1122745052 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0005040 con 0x7fdff01536f0 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.175+0000 7fdfefe00640 1 -- 172.21.15.50:0/1122745052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff01620e0 con 0x7fdff01536f0 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdfe7e00640 1 -- 172.21.15.50:0/1122745052 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7fdfe00051e0 con 0x7fdff01536f0 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdfe7e00640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdfcc074880 0x7fdfcc076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdfe7e00640 1 -- 172.21.15.50:0/1122745052 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fdfe012d310 con 0x7fdff01536f0 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdfe6000640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdfcc074880 0x7fdfcc076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdffd37a000 1 Processor -- start 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdffd37a000 1 -- start start 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdfe6000640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdfcc074880 0x7fdfcc076d90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8001400 tx=0x7fdfd8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdffd37a000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb8171b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdffd37a000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 0x55eb8171bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdffd37a000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb8171c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdffd37a000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55eb815f0410 con 0x55eb811da460 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdffd37a000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55eb8152db40 con 0x55eb815ca950 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdffd37a000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55eb8151cf70 con 0x55eb815b3ca0 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.179+0000 7fdfeea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb8171b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfeea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb8171b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50200/0 (socket says 172.21.15.50:50200) 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfeea00640 1 -- 172.21.15.50:0/2207550254 learned_addr learned my addr 172.21.15.50:0/2207550254 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfef400640 1 --2- 172.21.15.50:0/2207550254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb8171c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfe6000640 1 --2- 172.21.15.50:0/2207550254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 0x55eb8171bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfeea00640 1 -- 172.21.15.50:0/2207550254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 msgr2=0x55eb8171c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfeea00640 1 --2- 172.21.15.50:0/2207550254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb8171c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfeea00640 1 -- 172.21.15.50:0/2207550254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 msgr2=0x55eb8171bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfeea00640 1 --2- 172.21.15.50:0/2207550254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 0x55eb8171bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfeea00640 1 -- 172.21.15.50:0/2207550254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb814e45b0 con 0x55eb811da460 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfeea00640 1 --2- 172.21.15.50:0/2207550254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb8171b6a0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00f6880 tx=0x7fdfe00f8d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25700599257b116 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfdf400640 1 -- 172.21.15.50:0/2207550254 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0102da0 con 0x55eb811da460 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfdf400640 1 -- 172.21.15.50:0/2207550254 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfe00f9610 con 0x55eb811da460 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdfdf400640 1 -- 172.21.15.50:0/2207550254 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe01022a0 con 0x55eb811da460 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdffd37a000 1 -- 172.21.15.50:0/2207550254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 msgr2=0x55eb8171b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdffd37a000 1 --2- 172.21.15.50:0/2207550254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb8171b6a0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00f6880 tx=0x7fdfe00f8d80 comp rx=0 tx=0).stop 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdffd37a000 1 -- 172.21.15.50:0/2207550254 shutdown_connections 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdffd37a000 1 --2- 172.21.15.50:0/2207550254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb8171c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdffd37a000 1 --2- 172.21.15.50:0/2207550254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 0x55eb8171bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdffd37a000 1 --2- 172.21.15.50:0/2207550254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb8171b6a0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdffd37a000 1 -- 172.21.15.50:0/2207550254 >> 172.21.15.50:0/2207550254 conn(0x55eb816a0d90 msgr2=0x55eb816d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdffd37a000 1 -- 172.21.15.50:0/2207550254 shutdown_connections 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdffd37a000 1 -- 172.21.15.50:0/2207550254 wait complete. 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdffd37a000 1 -- start start 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.183+0000 7fdffd37a000 1 Processor -- start 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb81722920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb81722e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 0x55eb817233a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55eb814b75f0 con 0x55eb811da460 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55eb81520d80 con 0x55eb815ca950 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55eb814c3520 con 0x55eb815b3ca0 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 10 client.0 _ll_register_callbacks cb 0x55eb814c7830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfeea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb81722920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfeea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb81722920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50210/0 (socket says 172.21.15.50:50210) 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfeea00640 1 -- 172.21.15.50:0/1750192184 learned_addr learned my addr 172.21.15.50:0/1750192184 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfe6000640 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb81722e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfef400640 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 0x55eb817233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfeea00640 1 -- 172.21.15.50:0/1750192184 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 msgr2=0x55eb81722e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfeea00640 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb81722e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfeea00640 1 -- 172.21.15.50:0/1750192184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 msgr2=0x55eb817233a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfeea00640 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 0x55eb817233a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfeea00640 1 -- 172.21.15.50:0/1750192184 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb814bb190 con 0x55eb811da460 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfe6000640 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb81722e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfeea00640 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb81722920 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009840 tx=0x7fdfe01022d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d4cc1389f1a137 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfd3e00640 1 -- 172.21.15.50:0/1750192184 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe00fe0f0 con 0x55eb811da460 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfd3e00640 1 -- 172.21.15.50:0/1750192184 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fdfe0113b60 con 0x55eb811da460 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 1 -- 172.21.15.50:0/1750192184 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb814b5110 con 0x55eb811da460 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfd3e00640 1 -- 172.21.15.50:0/1750192184 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fdfe0103030 con 0x55eb811da460 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 10 client.7968 fetch_fsmap learned FSMap version 65 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 1 -- 172.21.15.50:0/1750192184 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55eb815a1f10 con 0x55eb811da460 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfd3e00640 1 -- 172.21.15.50:0/1750192184 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55eb814b5110 con 0x55eb811da460 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfd3e00640 1 -- 172.21.15.50:0/1750192184 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 65) ==== 3502+0+0 (secure 0 0 0) 0x7fdfe00f4f60 con 0x55eb811da460 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 10 client.7968 fetch_fsmap finished waiting for FSMap version 65 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 10 client.7968 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 20 client.7968 populate_metadata read hostname 'smithi050' 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.50:6832/1418616207,v1:172.21.15.50:6834/1418616207] conn(0x55eb811bf7d0 0x55eb8172cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 4 client.7968 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdffd37a000 1 -- 172.21.15.50:0/1750192184 --> [v2:172.21.15.50:6832/1418616207,v1:172.21.15.50:6834/1418616207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb8152fa90 con 0x55eb811bf7d0 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfe6000640 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.50:6832/1418616207,v1:172.21.15.50:6834/1418616207] conn(0x55eb811bf7d0 0x55eb8172cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfe6000640 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.50:6832/1418616207,v1:172.21.15.50:6834/1418616207] conn(0x55eb811bf7d0 0x55eb8172cd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146e19ef3f196457 server_cookie=478c2894e7494a0f in_seq=0 out_seq=0 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.187+0000 7fdfd3e00640 10 client.7968 ms_handle_connect on v2:172.21.15.50:6832/1418616207 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdfd3e00640 1 -- 172.21.15.50:0/1750192184 <== mds.0 v2:172.21.15.50:6832/1418616207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eb8152fa90 con 0x55eb811bf7d0 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdfd3e00640 10 client.7968 handle_command_reply: tid=0 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdffd37a000 1 client.7968 shutdown 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdffd37a000 1 -- 172.21.15.50:0/1750192184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 msgr2=0x55eb81722920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdffd37a000 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb81722920 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009840 tx=0x7fdfe01022d0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdffd37a000 1 -- 172.21.15.50:0/1750192184 shutdown_connections 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdffd37a000 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.50:6832/1418616207,v1:172.21.15.50:6834/1418616207] conn(0x55eb811bf7d0 0x55eb8172cd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdffd37a000 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 0x55eb817233a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdffd37a000 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb81722e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdffd37a000 1 --2- 172.21.15.50:0/1750192184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb81722920 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdffd37a000 1 -- 172.21.15.50:0/1750192184 >> 172.21.15.50:0/1750192184 conn(0x55eb816a0d90 msgr2=0x55eb816d7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdfd3e00640 0 client.7968 ms_handle_reset on v2:172.21.15.50:6832/1418616207 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdffd37a000 1 -- 172.21.15.50:0/1750192184 shutdown_connections 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdffd37a000 1 -- 172.21.15.50:0/1750192184 wait complete. 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.191+0000 7fdffd37a000 20 client.7968 trim_cache size 0 max 16384 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 Processor -- start 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 -- start start 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb8178d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb8178dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 0x55eb8178e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0205500 con 0x55eb811da460 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdff02044e0 con 0x55eb815ca950 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fdff0154c80 con 0x55eb815b3ca0 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfeea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb8178d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfeea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb8178d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50214/0 (socket says 172.21.15.50:50214) 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfeea00640 1 -- 172.21.15.50:0/3780578497 learned_addr learned my addr 172.21.15.50:0/3780578497 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfef400640 1 --2- 172.21.15.50:0/3780578497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 0x55eb8178e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfe6000640 1 --2- 172.21.15.50:0/3780578497 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb8178dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfeea00640 1 -- 172.21.15.50:0/3780578497 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 msgr2=0x55eb8178dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfeea00640 1 --2- 172.21.15.50:0/3780578497 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb8178dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfeea00640 1 -- 172.21.15.50:0/3780578497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 msgr2=0x55eb8178e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfeea00640 1 --2- 172.21.15.50:0/3780578497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 0x55eb8178e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfeea00640 1 -- 172.21.15.50:0/3780578497 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb81578db0 con 0x55eb811da460 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfeea00640 1 --2- 172.21.15.50:0/3780578497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb8178d840 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fdfe001a3c0 tx=0x7fdfe00a8750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca946a8f81719c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfdea00640 1 -- 172.21.15.50:0/3780578497 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0102340 con 0x55eb811da460 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfdea00640 1 -- 172.21.15.50:0/3780578497 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfe00fe0f0 con 0x55eb811da460 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdfdea00640 1 -- 172.21.15.50:0/3780578497 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe00f9610 con 0x55eb811da460 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 -- 172.21.15.50:0/3780578497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 msgr2=0x55eb8178d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 --2- 172.21.15.50:0/3780578497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb8178d840 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fdfe001a3c0 tx=0x7fdfe00a8750 comp rx=0 tx=0).stop 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 -- 172.21.15.50:0/3780578497 shutdown_connections 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 --2- 172.21.15.50:0/3780578497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb815ca950 0x55eb8178e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 --2- 172.21.15.50:0/3780578497 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb8178dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 --2- 172.21.15.50:0/3780578497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb811da460 0x55eb8178d840 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 -- 172.21.15.50:0/3780578497 >> 172.21.15.50:0/3780578497 conn(0x55eb816a0d90 msgr2=0x55eb816cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 -- 172.21.15.50:0/3780578497 shutdown_connections 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.211+0000 7fdffd37a000 1 -- 172.21.15.50:0/3780578497 wait complete. 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdffd37a000 1 -- start start 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdffd37a000 1 Processor -- start 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdffd37a000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb811da460 0x55eb817911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdffd37a000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb81791700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdffd37a000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb815ca950 0x55eb81791c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdffd37a000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55eb814c3520 con 0x55eb815ca950 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdffd37a000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdff01552d0 con 0x55eb811da460 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdffd37a000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55eb814b75f0 con 0x55eb815b3ca0 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdffd37a000 10 client.0 _ll_register_callbacks cb 0x7fdfd8007ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfef400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb815ca950 0x55eb81791c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfef400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb815ca950 0x55eb81791c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50226/0 (socket says 172.21.15.50:50226) 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfef400640 1 -- 172.21.15.50:0/2667251138 learned_addr learned my addr 172.21.15.50:0/2667251138 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfeea00640 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb811da460 0x55eb817911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfe6000640 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb81791700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfef400640 1 -- 172.21.15.50:0/2667251138 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 msgr2=0x55eb81791700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfef400640 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb81791700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfef400640 1 -- 172.21.15.50:0/2667251138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb811da460 msgr2=0x55eb817911c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfef400640 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb811da460 0x55eb817911c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfef400640 1 -- 172.21.15.50:0/2667251138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb814e3ab0 con 0x55eb815ca950 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfef400640 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb815ca950 0x55eb81791c40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8011740 tx=0x7fdfe8011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3490c9b74b5321 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfd3e00640 1 -- 172.21.15.50:0/2667251138 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8021060 con 0x55eb815ca950 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfd3e00640 1 -- 172.21.15.50:0/2667251138 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fdfe8010cc0 con 0x55eb815ca950 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdffd37a000 1 -- 172.21.15.50:0/2667251138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe00fe0f0 con 0x55eb815ca950 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdfd3e00640 1 -- 172.21.15.50:0/2667251138 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fdfe8027050 con 0x55eb815ca950 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdffd37a000 10 client.7974 fetch_fsmap learned FSMap version 65 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.215+0000 7fdffd37a000 1 -- 172.21.15.50:0/2667251138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55eb814c2070 con 0x55eb815ca950 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfd3e00640 1 -- 172.21.15.50:0/2667251138 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfe00fe0f0 con 0x55eb815ca950 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfd3e00640 1 -- 172.21.15.50:0/2667251138 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 65) ==== 3502+0+0 (secure 0 0 0) 0x7fdfe802a020 con 0x55eb815ca950 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 10 client.7974 fetch_fsmap finished waiting for FSMap version 65 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 10 client.7974 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 20 client.7974 populate_metadata read hostname 'smithi050' 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.50:6832/1418616207,v1:172.21.15.50:6834/1418616207] conn(0x55eb811ad830 0x55eb8174df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 4 client.7974 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 -- 172.21.15.50:0/2667251138 --> [v2:172.21.15.50:6832/1418616207,v1:172.21.15.50:6834/1418616207] -- command(tid 0: {"prefix": "status"}) -- 0x55eb8152b540 con 0x55eb811ad830 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfeea00640 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.50:6832/1418616207,v1:172.21.15.50:6834/1418616207] conn(0x55eb811ad830 0x55eb8174df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfeea00640 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.50:6832/1418616207,v1:172.21.15.50:6834/1418616207] conn(0x55eb811ad830 0x55eb8174df90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba5dacc54e7fcdf2 server_cookie=2b14a56f04a04e93 in_seq=0 out_seq=0 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfd3e00640 10 client.7974 ms_handle_connect on v2:172.21.15.50:6832/1418616207 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfd3e00640 1 -- 172.21.15.50:0/2667251138 <== mds.0 v2:172.21.15.50:6832/1418616207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55eb8152b540 con 0x55eb811ad830 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfd3e00640 10 client.7974 handle_command_reply: tid=0 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 client.7974 shutdown 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 -- 172.21.15.50:0/2667251138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb815ca950 msgr2=0x55eb81791c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb815ca950 0x55eb81791c40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8011740 tx=0x7fdfe8011c00 comp rx=0 tx=0).stop 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 -- 172.21.15.50:0/2667251138 shutdown_connections 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.50:6832/1418616207,v1:172.21.15.50:6834/1418616207] conn(0x55eb811ad830 0x55eb8174df90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb815ca950 0x55eb81791c40 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55eb815b3ca0 0x55eb81791700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfd3e00640 0 client.7974 ms_handle_reset on v2:172.21.15.50:6832/1418616207 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 --2- 172.21.15.50:0/2667251138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55eb811da460 0x55eb817911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 -- 172.21.15.50:0/2667251138 >> 172.21.15.50:0/2667251138 conn(0x55eb816a0d90 msgr2=0x55eb816d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 -- 172.21.15.50:0/2667251138 shutdown_connections 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 1 -- 172.21.15.50:0/2667251138 wait complete. 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdffd37a000 20 client.7974 trim_cache size 0 max 16384 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 7851, 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 6.1494095020000001, 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 65, 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 100, 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 100, 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 10.500767654000001 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfefe00640 1 -- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdfcc074880 msgr2=0x7fdfcc076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfefe00640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdfcc074880 0x7fdfcc076d90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8001400 tx=0x7fdfd8009040 comp rx=0 tx=0).stop 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfefe00640 1 -- 172.21.15.50:0/1122745052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdff01536f0 msgr2=0x7fdff0145c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfefe00640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdff01536f0 0x7fdff0145c30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0098170 tx=0x7fdfe0005db0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfefe00640 1 -- 172.21.15.50:0/1122745052 shutdown_connections 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfefe00640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdff01610d0 0x7fdff0156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfefe00640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff0160060 0x7fdff0156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfefe00640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdfcc074880 0x7fdfcc076d90 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfefe00640 1 --2- 172.21.15.50:0/1122745052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdff01536f0 0x7fdff0145c30 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfefe00640 1 -- 172.21.15.50:0/1122745052 >> 172.21.15.50:0/1122745052 conn(0x7fdff00b4560 msgr2=0x7fdff015e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfefe00640 1 -- 172.21.15.50:0/1122745052 shutdown_connections 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.219+0000 7fdfefe00640 1 -- 172.21.15.50:0/1122745052 wait complete. 2024-09-21T06:08:43.238 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.439+0000 7f1895000640 1 -- 172.21.15.50:0/270411955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18900d11b0 msgr2=0x7f189015ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.439+0000 7f1895000640 1 --2- 172.21.15.50:0/270411955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18900d11b0 0x7f189015ea90 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1890073a00 tx=0x7f187801c650 comp rx=0 tx=0).stop 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.439+0000 7f1895000640 1 -- 172.21.15.50:0/270411955 shutdown_connections 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.439+0000 7f1895000640 1 --2- 172.21.15.50:0/270411955 >> v2:172.21.15.88:3300/0 conn(0x7f189006f1e0 0x7f189006f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.439+0000 7f1895000640 1 --2- 172.21.15.50:0/270411955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18900d11b0 0x7f189015ea90 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.439+0000 7f1895000640 1 -- 172.21.15.50:0/270411955 >> 172.21.15.50:0/270411955 conn(0x7f189006d1f0 msgr2=0x7f189006d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.439+0000 7f1895000640 1 -- 172.21.15.50:0/270411955 shutdown_connections 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.439+0000 7f1895000640 1 -- 172.21.15.50:0/270411955 wait complete. 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f1895000640 1 Processor -- start 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f1895000640 1 -- start start 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f1895000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f189006f1e0 0x7f189015c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f1895000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890072550 0x7f18900aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f1895000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18900aeff0 0x7f1890140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f1895000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1890168b10 con 0x7f18900aeff0 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f1895000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1890168990 con 0x7f1890072550 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f1895000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1890168c90 con 0x7f189006f1e0 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f188fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18900aeff0 0x7f1890140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f188fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18900aeff0 0x7f1890140ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50252/0 (socket says 172.21.15.50:50252) 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f188fe00640 1 -- 172.21.15.50:0/1872505209 learned_addr learned my addr 172.21.15.50:0/1872505209 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f1887e00640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890072550 0x7f18900aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f188f400640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f189006f1e0 0x7f189015c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f188fe00640 1 -- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f189006f1e0 msgr2=0x7f189015c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f188fe00640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f189006f1e0 0x7f189015c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f188fe00640 1 -- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890072550 msgr2=0x7f18900aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f188fe00640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890072550 0x7f18900aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.443+0000 7f188fe00640 1 -- 172.21.15.50:0/1872505209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189015d3b0 con 0x7f18900aeff0 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.447+0000 7f188fe00640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18900aeff0 0x7f1890140ae0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f188800b9e0 tx=0x7f188800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.447+0000 7f1887400640 1 -- 172.21.15.50:0/1872505209 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888019070 con 0x7f18900aeff0 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.447+0000 7f1887400640 1 -- 172.21.15.50:0/1872505209 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f188800cc30 con 0x7f18900aeff0 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.447+0000 7f1895000640 1 -- 172.21.15.50:0/1872505209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18901f8a30 con 0x7f18900aeff0 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.447+0000 7f1887400640 1 -- 172.21.15.50:0/1872505209 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18880146a0 con 0x7f18900aeff0 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.447+0000 7f1895000640 1 -- 172.21.15.50:0/1872505209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18901f8ff0 con 0x7f18900aeff0 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.447+0000 7f1895000640 1 -- 172.21.15.50:0/1872505209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189006f5f0 con 0x7f18900aeff0 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.447+0000 7f1887400640 1 -- 172.21.15.50:0/1872505209 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f188801c070 con 0x7f18900aeff0 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.447+0000 7f1887400640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f186c074880 0x7f186c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.447+0000 7f1887400640 1 -- 172.21.15.50:0/1872505209 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f1888092d30 con 0x7f18900aeff0 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.451+0000 7f188f400640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f186c074880 0x7f186c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.451+0000 7f188f400640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f186c074880 0x7f186c076d90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1880006fd0 tx=0x7f1880008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.455+0000 7f1887400640 1 -- 172.21.15.50:0/1872505209 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f188800ce20 con 0x7f18900aeff0 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.667+0000 7f1895000640 1 -- 172.21.15.50:0/1872505209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1890156af0 con 0x7f18900aeff0 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.667+0000 7f1887400640 1 -- 172.21.15.50:0/1872505209 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) ==== 53+0+73 (secure 0 0 0) 0x7f188805f7b0 con 0x7f18900aeff0 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.671+0000 7f1895000640 1 -- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f186c074880 msgr2=0x7f186c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.671+0000 7f1895000640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f186c074880 0x7f186c076d90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1880006fd0 tx=0x7f1880008040 comp rx=0 tx=0).stop 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.671+0000 7f1895000640 1 -- 172.21.15.50:0/1872505209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18900aeff0 msgr2=0x7f1890140ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.671+0000 7f1895000640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18900aeff0 0x7f1890140ae0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f188800b9e0 tx=0x7f188800bea0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.671+0000 7f1895000640 1 -- 172.21.15.50:0/1872505209 shutdown_connections 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.671+0000 7f1895000640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f186c074880 0x7f186c076d90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.671+0000 7f1895000640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18900aeff0 0x7f1890140ae0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.671+0000 7f1895000640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890072550 0x7f18900aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.671+0000 7f1895000640 1 --2- 172.21.15.50:0/1872505209 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f189006f1e0 0x7f189015c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.671+0000 7f1895000640 1 -- 172.21.15.50:0/1872505209 >> 172.21.15.50:0/1872505209 conn(0x7f189006d1f0 msgr2=0x7f18900b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.671+0000 7f1895000640 1 -- 172.21.15.50:0/1872505209 shutdown_connections 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43.671+0000 7f1895000640 1 -- 172.21.15.50:0/1872505209 wait complete. 2024-09-21T06:08:43.689 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:08:43.689 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:08:43.689 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-21T06:08:43.690 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:08:43.690 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:08:43.690 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:08:43.690 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:08:43.690 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:08:43.690 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:08:43.690 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 41672sec preferred_lft 41672sec 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:8b:d0:0a:5d brd ff:ff:ff:ff:ff:ff 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi050.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:08:43.703 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:08:43.764 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:08:43.818 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:08:43.818 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T06:08:43.818 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:08:43.819 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:08:43.819 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:08:43.819 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T06:08:43.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:08:43.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:08:43.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:08:43.956 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T06:08:43.965 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:08:43.965 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:08:43.966 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:08:43.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:08:43.966 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-09-21T06:08:44.027 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:08:44.028 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:08:44.082 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:08:44.082 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:08:44.082 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:08:44.131 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:08:44.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:08:44.131 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:08:44.232 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:08:44.232 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:08:44.233 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:08:44.251 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:08:44.251 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:08:44.251 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:08:44.251 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:08:44.251 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:08:44.262 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:08:44.262 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:08:44.262 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:08:44.262 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:08:44.262 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:08:44.262 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T06:08:44.262 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T06:08:44.262 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T06:08:44.266 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:08:44.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:08:44.267 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:08:44.286 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:08:44.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:08:44.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:08:44.391 INFO:teuthology.orchestra.run.smithi050.stdout:client7983 2024-09-21T06:08:44.391 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:08:44.391 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:08:44.400 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T06:08:44.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.607+0000 7f198c200640 1 -- 172.21.15.50:0/2995951770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f196c003fe0 msgr2=0x7f196c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:44.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.607+0000 7f198c200640 1 --2- 172.21.15.50:0/2995951770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f196c003fe0 0x7f196c004810 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f197000c470 tx=0x7f197001f6c0 comp rx=0 tx=0).stop 2024-09-21T06:08:44.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.607+0000 7f198a200640 1 -- 172.21.15.50:0/2995951770 reap_dead start 2024-09-21T06:08:44.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.607+0000 7f198c200640 1 -- 172.21.15.50:0/2995951770 shutdown_connections 2024-09-21T06:08:44.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.607+0000 7f198c200640 1 -- 172.21.15.50:0/2995951770 >> 172.21.15.50:0/2995951770 conn(0x7f19840b2040 msgr2=0x7f19840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:44.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.607+0000 7f198c200640 1 -- 172.21.15.50:0/2995951770 shutdown_connections 2024-09-21T06:08:44.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.607+0000 7f198c200640 1 -- 172.21.15.50:0/2995951770 wait complete. 2024-09-21T06:08:44.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f198c200640 1 Processor -- start 2024-09-21T06:08:44.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f198c200640 1 -- start start 2024-09-21T06:08:44.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f198c200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f196c003fe0 0x7f19841f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:44.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f198c200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19840ccae0 0x7f198414fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:44.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f198c200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1984159720 0x7f1984159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:44.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f198c200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f198415cd20 con 0x7f19840ccae0 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f198c200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f198415cba0 con 0x7f196c003fe0 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f198c200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f198415cea0 con 0x7f1984159720 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1988e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19840ccae0 0x7f198414fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1988e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19840ccae0 0x7f198414fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50268/0 (socket says 172.21.15.50:50268) 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1988e00640 1 -- 172.21.15.50:0/4167019018 learned_addr learned my addr 172.21.15.50:0/4167019018 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f198a200640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1984159720 0x7f1984159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1989800640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f196c003fe0 0x7f19841f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1988e00640 1 -- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1984159720 msgr2=0x7f1984159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1988e00640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1984159720 0x7f1984159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1988e00640 1 -- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f196c003fe0 msgr2=0x7f19841f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1988e00640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f196c003fe0 0x7f19841f4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1988e00640 1 -- 172.21.15.50:0/4167019018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19841501a0 con 0x7f19840ccae0 2024-09-21T06:08:44.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1989800640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f196c003fe0 0x7f19841f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:44.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1988e00640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19840ccae0 0x7f198414fc60 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f19700263d0 tx=0x7f1970006fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:44.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1981600640 1 -- 172.21.15.50:0/4167019018 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700c0070 con 0x7f19840ccae0 2024-09-21T06:08:44.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1981600640 1 -- 172.21.15.50:0/4167019018 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1970006380 con 0x7f19840ccae0 2024-09-21T06:08:44.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f198c200640 1 -- 172.21.15.50:0/4167019018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198415a070 con 0x7f19840ccae0 2024-09-21T06:08:44.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f1981600640 1 -- 172.21.15.50:0/4167019018 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700b3070 con 0x7f19840ccae0 2024-09-21T06:08:44.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.611+0000 7f198c200640 1 -- 172.21.15.50:0/4167019018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19841e2910 con 0x7f19840ccae0 2024-09-21T06:08:44.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.615+0000 7f198c200640 1 -- 172.21.15.50:0/4167019018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19841e23a0 con 0x7f19840ccae0 2024-09-21T06:08:44.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.615+0000 7f1981600640 1 -- 172.21.15.50:0/4167019018 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f19700c3070 con 0x7f19840ccae0 2024-09-21T06:08:44.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.615+0000 7f1981600640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1964074880 0x7f1964076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:44.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.615+0000 7f1981600640 1 -- 172.21.15.50:0/4167019018 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f1970130350 con 0x7f19840ccae0 2024-09-21T06:08:44.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.615+0000 7f1989800640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1964074880 0x7f1964076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:44.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.615+0000 7f1989800640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1964074880 0x7f1964076d90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1978009bf0 tx=0x7f19780091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:44.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.619+0000 7f1981600640 1 -- 172.21.15.50:0/4167019018 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f198415a070 con 0x7f19840ccae0 2024-09-21T06:08:44.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:44.835+0000 7f198c200640 1 -- 172.21.15.50:0/4167019018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f198415d430 con 0x7f19840ccae0 2024-09-21T06:08:45.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.131+0000 7f1981600640 1 -- 172.21.15.50:0/4167019018 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v66) ==== 136+0+0 (secure 0 0 0) 0x7f19840b5730 con 0x7f19840ccae0 2024-09-21T06:08:45.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.135+0000 7f198c200640 1 -- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1964074880 msgr2=0x7f1964076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:45.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.135+0000 7f198c200640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1964074880 0x7f1964076d90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1978009bf0 tx=0x7f19780091e0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.135+0000 7f198c200640 1 -- 172.21.15.50:0/4167019018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19840ccae0 msgr2=0x7f198414fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:45.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.135+0000 7f198c200640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19840ccae0 0x7f198414fc60 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f19700263d0 tx=0x7f1970006fd0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.135+0000 7f198c200640 1 -- 172.21.15.50:0/4167019018 shutdown_connections 2024-09-21T06:08:45.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.135+0000 7f198c200640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1984159720 0x7f1984159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.135+0000 7f198c200640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1964074880 0x7f1964076d90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.135+0000 7f198c200640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19840ccae0 0x7f198414fc60 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.135+0000 7f198c200640 1 --2- 172.21.15.50:0/4167019018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f196c003fe0 0x7f19841f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.135+0000 7f198c200640 1 -- 172.21.15.50:0/4167019018 >> 172.21.15.50:0/4167019018 conn(0x7f19840b2040 msgr2=0x7f1984151db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:45.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.135+0000 7f198c200640 1 -- 172.21.15.50:0/4167019018 shutdown_connections 2024-09-21T06:08:45.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.135+0000 7f198c200640 1 -- 172.21.15.50:0/4167019018 wait complete. 2024-09-21T06:08:45.153 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:08:45.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.315+0000 7f9944c00640 1 -- 172.21.15.50:0/589113081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9924004da0 msgr2=0x7f9924005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:45.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.315+0000 7f9944c00640 1 --2- 172.21.15.50:0/589113081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9924004da0 0x7f9924005250 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f993800c560 tx=0x7f993801d240 comp rx=0 tx=0).stop 2024-09-21T06:08:45.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.315+0000 7f993e800640 1 -- 172.21.15.50:0/589113081 reap_dead start 2024-09-21T06:08:45.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.315+0000 7f9944c00640 1 -- 172.21.15.50:0/589113081 shutdown_connections 2024-09-21T06:08:45.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.315+0000 7f9944c00640 1 -- 172.21.15.50:0/589113081 >> 172.21.15.50:0/589113081 conn(0x7f99400b2040 msgr2=0x7f99400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:45.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.315+0000 7f9944c00640 1 -- 172.21.15.50:0/589113081 shutdown_connections 2024-09-21T06:08:45.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.315+0000 7f9944c00640 1 -- 172.21.15.50:0/589113081 wait complete. 2024-09-21T06:08:45.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.319+0000 7f9944c00640 1 Processor -- start 2024-09-21T06:08:45.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.319+0000 7f9944c00640 1 -- start start 2024-09-21T06:08:45.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.319+0000 7f9944c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9924004da0 0x7f99400abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.319+0000 7f9944c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99401f9260 0x7f99401f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.319+0000 7f9944c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99400ced80 0x7f99400cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.319+0000 7f9944c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f994015d040 con 0x7f99401f9260 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.319+0000 7f9944c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f994015cec0 con 0x7f99400ced80 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.319+0000 7f9944c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f994015d1c0 con 0x7f9924004da0 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.319+0000 7f9937e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99401f9260 0x7f99401f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9937e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99401f9260 0x7f99401f9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50286/0 (socket says 172.21.15.50:50286) 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9937e00640 1 -- 172.21.15.50:0/4009573302 learned_addr learned my addr 172.21.15.50:0/4009573302 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f993de00640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9924004da0 0x7f99400abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f993e800640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99400ced80 0x7f99400cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9937e00640 1 -- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9924004da0 msgr2=0x7f99400abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9937e00640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9924004da0 0x7f99400abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9937e00640 1 -- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99400ced80 msgr2=0x7f99400cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:45.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9937e00640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99400ced80 0x7f99400cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9937e00640 1 -- 172.21.15.50:0/4009573302 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99401f9bb0 con 0x7f99401f9260 2024-09-21T06:08:45.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f993e800640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99400ced80 0x7f99400cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:45.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9937e00640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99401f9260 0x7f99401f9670 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9928011ae0 tx=0x7f9928009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:45.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9936000640 1 -- 172.21.15.50:0/4009573302 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928004460 con 0x7f99401f9260 2024-09-21T06:08:45.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9936000640 1 -- 172.21.15.50:0/4009573302 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f992800fa00 con 0x7f99401f9260 2024-09-21T06:08:45.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9936000640 1 -- 172.21.15.50:0/4009573302 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992800fe10 con 0x7f99401f9260 2024-09-21T06:08:45.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9944c00640 1 -- 172.21.15.50:0/4009573302 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99400cf6d0 con 0x7f99401f9260 2024-09-21T06:08:45.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9944c00640 1 -- 172.21.15.50:0/4009573302 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99401f5cc0 con 0x7f99401f9260 2024-09-21T06:08:45.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9944c00640 1 -- 172.21.15.50:0/4009573302 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9940073cf0 con 0x7f99401f9260 2024-09-21T06:08:45.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.323+0000 7f9936000640 1 -- 172.21.15.50:0/4009573302 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f9928005020 con 0x7f99401f9260 2024-09-21T06:08:45.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.327+0000 7f9936000640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f991c074880 0x7f991c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:45.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.327+0000 7f9936000640 1 -- 172.21.15.50:0/4009573302 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f9928093ce0 con 0x7f99401f9260 2024-09-21T06:08:45.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.327+0000 7f993de00640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f991c074880 0x7f991c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:45.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.327+0000 7f993de00640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f991c074880 0x7f991c076d90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9930000c10 tx=0x7f9930006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:45.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.331+0000 7f9936000640 1 -- 172.21.15.50:0/4009573302 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f99400cf6d0 con 0x7f99401f9260 2024-09-21T06:08:45.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.527+0000 7f9944c00640 1 -- 172.21.15.50:0/4009573302 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f994015c7a0 con 0x7f991c074880 2024-09-21T06:08:45.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.527+0000 7f9936000640 1 -- 172.21.15.50:0/4009573302 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f994015c7a0 con 0x7f991c074880 2024-09-21T06:08:45.528 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:08:45.528 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:08:45.528 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:08:45.528 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:08:45.528 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:08:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.531+0000 7f9944c00640 1 -- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f991c074880 msgr2=0x7f991c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.531+0000 7f9944c00640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f991c074880 0x7f991c076d90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9930000c10 tx=0x7f9930006b20 comp rx=0 tx=0).stop 2024-09-21T06:08:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.531+0000 7f9944c00640 1 -- 172.21.15.50:0/4009573302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99401f9260 msgr2=0x7f99401f9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.531+0000 7f9944c00640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99401f9260 0x7f99401f9670 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9928011ae0 tx=0x7f9928009e10 comp rx=0 tx=0).stop 2024-09-21T06:08:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.531+0000 7f9944c00640 1 -- 172.21.15.50:0/4009573302 shutdown_connections 2024-09-21T06:08:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.531+0000 7f9944c00640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99400ced80 0x7f99400cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.531+0000 7f9944c00640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f991c074880 0x7f991c076d90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.531+0000 7f9944c00640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99401f9260 0x7f99401f9670 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.531+0000 7f9944c00640 1 --2- 172.21.15.50:0/4009573302 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9924004da0 0x7f99400abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.531+0000 7f9944c00640 1 -- 172.21.15.50:0/4009573302 >> 172.21.15.50:0/4009573302 conn(0x7f99400b2040 msgr2=0x7f9940152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:45.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.531+0000 7f9944c00640 1 -- 172.21.15.50:0/4009573302 shutdown_connections 2024-09-21T06:08:45.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.531+0000 7f9944c00640 1 -- 172.21.15.50:0/4009573302 wait complete. 2024-09-21T06:08:45.551 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:08:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.714+0000 7ff49f200640 1 -- 172.21.15.50:0/825403361 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980b4de0 msgr2=0x7ff4980b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.714+0000 7ff49f200640 1 --2- 172.21.15.50:0/825403361 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980b4de0 0x7ff4980b51f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff490009c80 tx=0x7ff49001cce0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.714+0000 7ff49f200640 1 -- 172.21.15.50:0/825403361 shutdown_connections 2024-09-21T06:08:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.714+0000 7ff49f200640 1 --2- 172.21.15.50:0/825403361 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980b4de0 0x7ff4980b51f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.714+0000 7ff49f200640 1 --2- 172.21.15.50:0/825403361 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff4981527b0 0x7ff49815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.714+0000 7ff49f200640 1 -- 172.21.15.50:0/825403361 >> 172.21.15.50:0/825403361 conn(0x7ff4980b2040 msgr2=0x7ff4980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.714+0000 7ff49f200640 1 -- 172.21.15.50:0/825403361 shutdown_connections 2024-09-21T06:08:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.714+0000 7ff49f200640 1 -- 172.21.15.50:0/825403361 wait complete. 2024-09-21T06:08:45.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.714+0000 7ff49f200640 1 Processor -- start 2024-09-21T06:08:45.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.714+0000 7ff49f200640 1 -- start start 2024-09-21T06:08:45.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff49f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4980b4de0 0x7ff49813d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:45.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff49f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980b5c70 0x7ff4981582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:45.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff49f200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff4981527b0 0x7ff4981f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:45.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff49f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff498168ab0 con 0x7ff4980b5c70 2024-09-21T06:08:45.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff49f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff498168930 con 0x7ff4980b4de0 2024-09-21T06:08:45.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff49f200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff498168c30 con 0x7ff4981527b0 2024-09-21T06:08:45.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff48fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980b5c70 0x7ff4981582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:45.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff48fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980b5c70 0x7ff4981582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50304/0 (socket says 172.21.15.50:50304) 2024-09-21T06:08:45.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff48fe00640 1 -- 172.21.15.50:0/4285596520 learned_addr learned my addr 172.21.15.50:0/4285596520 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:45.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff497e00640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4980b4de0 0x7ff49813d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:45.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff49d200640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff4981527b0 0x7ff4981f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:45.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff48fe00640 1 -- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff4981527b0 msgr2=0x7ff4981f5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:45.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff48fe00640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff4981527b0 0x7ff4981f5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff48fe00640 1 -- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4980b4de0 msgr2=0x7ff49813d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:45.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff48fe00640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4980b4de0 0x7ff49813d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff48fe00640 1 -- 172.21.15.50:0/4285596520 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4981572b0 con 0x7ff4980b5c70 2024-09-21T06:08:45.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff48fe00640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980b5c70 0x7ff4981582e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff48000e7c0 tx=0x7ff48000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:45.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff494c00640 1 -- 172.21.15.50:0/4285596520 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48000cc60 con 0x7ff4980b5c70 2024-09-21T06:08:45.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff494c00640 1 -- 172.21.15.50:0/4285596520 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff48000ce00 con 0x7ff4980b5c70 2024-09-21T06:08:45.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff494c00640 1 -- 172.21.15.50:0/4285596520 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480005280 con 0x7ff4980b5c70 2024-09-21T06:08:45.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff49f200640 1 -- 172.21.15.50:0/4285596520 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff498157520 con 0x7ff4980b5c70 2024-09-21T06:08:45.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff49f200640 1 -- 172.21.15.50:0/4285596520 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff498157ae0 con 0x7ff4980b5c70 2024-09-21T06:08:45.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.718+0000 7ff49f200640 1 -- 172.21.15.50:0/4285596520 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4980af780 con 0x7ff4980b5c70 2024-09-21T06:08:45.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.722+0000 7ff494c00640 1 -- 172.21.15.50:0/4285596520 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7ff480005420 con 0x7ff4980b5c70 2024-09-21T06:08:45.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.722+0000 7ff494c00640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff474074880 0x7ff474076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:45.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.722+0000 7ff494c00640 1 -- 172.21.15.50:0/4285596520 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7ff4800937a0 con 0x7ff4980b5c70 2024-09-21T06:08:45.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.722+0000 7ff497e00640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff474074880 0x7ff474076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:45.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.722+0000 7ff497e00640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff474074880 0x7ff474076d90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff4880017f0 tx=0x7ff488009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:45.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.726+0000 7ff494c00640 1 -- 172.21.15.50:0/4285596520 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff48005fbd0 con 0x7ff4980b5c70 2024-09-21T06:08:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 -- 172.21.15.50:0/4285596520 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff4980b51f0 con 0x7ff4980b5c70 2024-09-21T06:08:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff494c00640 1 -- 172.21.15.50:0/4285596520 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v49)=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7ff480064600 con 0x7ff4980b5c70 2024-09-21T06:08:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 -- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff474074880 msgr2=0x7ff474076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff474074880 0x7ff474076d90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff4880017f0 tx=0x7ff488009040 comp rx=0 tx=0).stop 2024-09-21T06:08:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 -- 172.21.15.50:0/4285596520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980b5c70 msgr2=0x7ff4981582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980b5c70 0x7ff4981582e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff48000e7c0 tx=0x7ff48000ec80 comp rx=0 tx=0).stop 2024-09-21T06:08:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 -- 172.21.15.50:0/4285596520 shutdown_connections 2024-09-21T06:08:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff4981527b0 0x7ff4981f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff474074880 0x7ff474076d90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980b5c70 0x7ff4981582e0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 --2- 172.21.15.50:0/4285596520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4980b4de0 0x7ff49813d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 -- 172.21.15.50:0/4285596520 >> 172.21.15.50:0/4285596520 conn(0x7ff4980b2040 msgr2=0x7ff49815b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 -- 172.21.15.50:0/4285596520 shutdown_connections 2024-09-21T06:08:45.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:45.906+0000 7ff49f200640 1 -- 172.21.15.50:0/4285596520 wait complete. 2024-09-21T06:08:45.928 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-21T06:08:46.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.090+0000 7f1a16800640 1 -- 172.21.15.50:0/2059191785 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a10074b20 msgr2=0x7f1a1015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.090+0000 7f1a16800640 1 --2- 172.21.15.50:0/2059191785 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a10074b20 0x7f1a1015cf90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009bf0 tx=0x7f1a0801cb00 comp rx=0 tx=0).stop 2024-09-21T06:08:46.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.090+0000 7f1a16800640 1 -- 172.21.15.50:0/2059191785 shutdown_connections 2024-09-21T06:08:46.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.090+0000 7f1a16800640 1 --2- 172.21.15.50:0/2059191785 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a10074b20 0x7f1a1015cf90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.090+0000 7f1a16800640 1 -- 172.21.15.50:0/2059191785 >> 172.21.15.50:0/2059191785 conn(0x7f1a100708c0 msgr2=0x7f1a10070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:46.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.090+0000 7f1a16800640 1 -- 172.21.15.50:0/2059191785 shutdown_connections 2024-09-21T06:08:46.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.090+0000 7f1a16800640 1 -- 172.21.15.50:0/2059191785 wait complete. 2024-09-21T06:08:46.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.090+0000 7f1a16800640 1 Processor -- start 2024-09-21T06:08:46.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.090+0000 7f1a16800640 1 -- start start 2024-09-21T06:08:46.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a16800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a100b5a80 0x7f1a1013bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a16800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a10157030 0x7f1a10152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a16800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a10150740 0x7f1a1006cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a16800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1016b680 con 0x7f1a10150740 2024-09-21T06:08:46.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a16800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1016b500 con 0x7f1a10157030 2024-09-21T06:08:46.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a16800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1016b800 con 0x7f1a100b5a80 2024-09-21T06:08:46.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a0fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a10150740 0x7f1a1006cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a0fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a10150740 0x7f1a1006cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50316/0 (socket says 172.21.15.50:50316) 2024-09-21T06:08:46.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a0fe00640 1 -- 172.21.15.50:0/1262325429 learned_addr learned my addr 172.21.15.50:0/1262325429 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:46.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a0f400640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a100b5a80 0x7f1a1013bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a07e00640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a10157030 0x7f1a10152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a0fe00640 1 -- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a100b5a80 msgr2=0x7f1a1013bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a0fe00640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a100b5a80 0x7f1a1013bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a0fe00640 1 -- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a10157030 msgr2=0x7f1a10152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a0fe00640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a10157030 0x7f1a10152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a0fe00640 1 -- 172.21.15.50:0/1262325429 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a101fd580 con 0x7f1a10150740 2024-09-21T06:08:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a0f400640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a100b5a80 0x7f1a1013bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a0fe00640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a10150740 0x7f1a1006cdf0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1a080b0040 tx=0x7f1a08005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a07400640 1 -- 172.21.15.50:0/1262325429 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080b2040 con 0x7f1a10150740 2024-09-21T06:08:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a07400640 1 -- 172.21.15.50:0/1262325429 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a080bb070 con 0x7f1a10150740 2024-09-21T06:08:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a07400640 1 -- 172.21.15.50:0/1262325429 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08004040 con 0x7f1a10150740 2024-09-21T06:08:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a16800640 1 -- 172.21.15.50:0/1262325429 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1013c450 con 0x7f1a10150740 2024-09-21T06:08:46.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a16800640 1 -- 172.21.15.50:0/1262325429 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a10153190 con 0x7f1a10150740 2024-09-21T06:08:46.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.094+0000 7f1a16800640 1 -- 172.21.15.50:0/1262325429 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a10151fa0 con 0x7f1a10150740 2024-09-21T06:08:46.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.098+0000 7f1a07400640 1 -- 172.21.15.50:0/1262325429 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f1a08004200 con 0x7f1a10150740 2024-09-21T06:08:46.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.098+0000 7f1a07400640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19ec0748b0 0x7f19ec076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.098+0000 7f1a07400640 1 -- 172.21.15.50:0/1262325429 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f1a0812d320 con 0x7f1a10150740 2024-09-21T06:08:46.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.098+0000 7f1a0f400640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19ec0748b0 0x7f19ec076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.098+0000 7f1a0f400640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19ec0748b0 0x7f19ec076dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1a00001130 tx=0x7f1a00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:46.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.102+0000 7f1a07400640 1 -- 172.21.15.50:0/1262325429 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1a080166f0 con 0x7f1a10150740 2024-09-21T06:08:46.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.278+0000 7f1a16800640 1 -- 172.21.15.50:0/1262325429 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f1a1015cf90 con 0x7f1a10150740 2024-09-21T06:08:46.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.278+0000 7f1a07400640 1 -- 172.21.15.50:0/1262325429 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v49) ==== 116+0+2 (secure 0 0 0) 0x7f1a080f9e30 con 0x7f1a10150740 2024-09-21T06:08:46.280 INFO:teuthology.orchestra.run.smithi050.stdout:4 2024-09-21T06:08:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.282+0000 7f1a16800640 1 -- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19ec0748b0 msgr2=0x7f19ec076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.282+0000 7f1a16800640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19ec0748b0 0x7f19ec076dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1a00001130 tx=0x7f1a00009040 comp rx=0 tx=0).stop 2024-09-21T06:08:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.282+0000 7f1a16800640 1 -- 172.21.15.50:0/1262325429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a10150740 msgr2=0x7f1a1006cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.282+0000 7f1a16800640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a10150740 0x7f1a1006cdf0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1a080b0040 tx=0x7f1a08005a60 comp rx=0 tx=0).stop 2024-09-21T06:08:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.282+0000 7f1a16800640 1 -- 172.21.15.50:0/1262325429 shutdown_connections 2024-09-21T06:08:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.282+0000 7f1a16800640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f19ec0748b0 0x7f19ec076dc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.282+0000 7f1a16800640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a10150740 0x7f1a1006cdf0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.282+0000 7f1a16800640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a10157030 0x7f1a10152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.282+0000 7f1a16800640 1 --2- 172.21.15.50:0/1262325429 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a100b5a80 0x7f1a1013bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.282+0000 7f1a16800640 1 -- 172.21.15.50:0/1262325429 >> 172.21.15.50:0/1262325429 conn(0x7f1a100708c0 msgr2=0x7f1a1015f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.282+0000 7f1a16800640 1 -- 172.21.15.50:0/1262325429 shutdown_connections 2024-09-21T06:08:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.282+0000 7f1a16800640 1 -- 172.21.15.50:0/1262325429 wait complete. 2024-09-21T06:08:46.303 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:08:46.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.466+0000 7f51c4e00640 1 -- 172.21.15.50:0/563138167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51a4004010 msgr2=0x7f51a4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.466+0000 7f51c4e00640 1 --2- 172.21.15.50:0/563138167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51a4004010 0x7f51a4004810 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f51a80016f0 tx=0x7f51a801c7b0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.466+0000 7f51be800640 1 -- 172.21.15.50:0/563138167 reap_dead start 2024-09-21T06:08:46.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.466+0000 7f51c4e00640 1 -- 172.21.15.50:0/563138167 shutdown_connections 2024-09-21T06:08:46.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.466+0000 7f51c4e00640 1 -- 172.21.15.50:0/563138167 >> 172.21.15.50:0/563138167 conn(0x7f51c00d06e0 msgr2=0x7f51c00d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:46.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.466+0000 7f51c4e00640 1 -- 172.21.15.50:0/563138167 shutdown_connections 2024-09-21T06:08:46.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.466+0000 7f51c4e00640 1 -- 172.21.15.50:0/563138167 wait complete. 2024-09-21T06:08:46.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.466+0000 7f51c4e00640 1 Processor -- start 2024-09-21T06:08:46.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51c4e00640 1 -- start start 2024-09-21T06:08:46.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51c4e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f51a4004010 0x7f51c01419e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51c4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51c0145d60 0x7f51c0146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51c4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51c01466b0 0x7f51c0144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51c4e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f51c00b3710 con 0x7f51c01466b0 2024-09-21T06:08:46.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51c4e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f51c006fd30 con 0x7f51c0145d60 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51c4e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f51c00b3890 con 0x7f51a4004010 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51be800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51c01466b0 0x7f51c0144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51be800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51c01466b0 0x7f51c0144240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50334/0 (socket says 172.21.15.50:50334) 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51be800640 1 -- 172.21.15.50:0/1825590048 learned_addr learned my addr 172.21.15.50:0/1825590048 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51bde00640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f51a4004010 0x7f51c01419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51be800640 1 -- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f51a4004010 msgr2=0x7f51c01419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51be800640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f51a4004010 0x7f51c01419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51b7e00640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51c0145d60 0x7f51c0146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51be800640 1 -- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51c0145d60 msgr2=0x7f51c0146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51be800640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51c0145d60 0x7f51c0146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51be800640 1 -- 172.21.15.50:0/1825590048 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c01437f0 con 0x7f51c01466b0 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51bde00640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f51a4004010 0x7f51c01419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51b7e00640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51c0145d60 0x7f51c0146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:46.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51be800640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51c01466b0 0x7f51c0144240 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f51b8000f90 tx=0x7f51b8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:46.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51b6000640 1 -- 172.21.15.50:0/1825590048 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8013920 con 0x7f51c01466b0 2024-09-21T06:08:46.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51b6000640 1 -- 172.21.15.50:0/1825590048 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51b801a070 con 0x7f51c01466b0 2024-09-21T06:08:46.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51b6000640 1 -- 172.21.15.50:0/1825590048 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8013e50 con 0x7f51c01466b0 2024-09-21T06:08:46.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51c4e00640 1 -- 172.21.15.50:0/1825590048 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c0143ad0 con 0x7f51c01466b0 2024-09-21T06:08:46.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51c4e00640 1 -- 172.21.15.50:0/1825590048 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c0146de0 con 0x7f51c01466b0 2024-09-21T06:08:46.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.470+0000 7f51c4e00640 1 -- 172.21.15.50:0/1825590048 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c0143ff0 con 0x7f51c01466b0 2024-09-21T06:08:46.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.474+0000 7f51b6000640 1 -- 172.21.15.50:0/1825590048 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f51b801f460 con 0x7f51c01466b0 2024-09-21T06:08:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.474+0000 7f51b6000640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f519c074a50 0x7f519c076f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.474+0000 7f51b6000640 1 -- 172.21.15.50:0/1825590048 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f51b8097280 con 0x7f51c01466b0 2024-09-21T06:08:46.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.474+0000 7f51bde00640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f519c074a50 0x7f519c076f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.474+0000 7f51bde00640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f519c074a50 0x7f519c076f60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009b50 tx=0x7f51b00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.478+0000 7f51b6000640 1 -- 172.21.15.50:0/1825590048 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f51c0143ff0 con 0x7f51c01466b0 2024-09-21T06:08:46.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.654+0000 7f51c4e00640 1 -- 172.21.15.50:0/1825590048 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f51c00743a0 con 0x7f51c01466b0 2024-09-21T06:08:46.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51b6000640 1 -- 172.21.15.50:0/1825590048 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v50)=0 v50) ==== 131+0+0 (secure 0 0 0) 0x7f51b8063d90 con 0x7f51c01466b0 2024-09-21T06:08:46.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51c4e00640 1 -- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f519c074a50 msgr2=0x7f519c076f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51c4e00640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f519c074a50 0x7f519c076f60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009b50 tx=0x7f51b00091e0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51c4e00640 1 -- 172.21.15.50:0/1825590048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51c01466b0 msgr2=0x7f51c0144240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51c4e00640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51c01466b0 0x7f51c0144240 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f51b8000f90 tx=0x7f51b8011a30 comp rx=0 tx=0).stop 2024-09-21T06:08:46.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51c4e00640 1 -- 172.21.15.50:0/1825590048 shutdown_connections 2024-09-21T06:08:46.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51c4e00640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f519c074a50 0x7f519c076f60 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51c4e00640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51c01466b0 0x7f51c0144240 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51c4e00640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51c0145d60 0x7f51c0146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51c4e00640 1 --2- 172.21.15.50:0/1825590048 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f51a4004010 0x7f51c01419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51c4e00640 1 -- 172.21.15.50:0/1825590048 >> 172.21.15.50:0/1825590048 conn(0x7f51c00d06e0 msgr2=0x7f51c00ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:46.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51c4e00640 1 -- 172.21.15.50:0/1825590048 shutdown_connections 2024-09-21T06:08:46.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.658+0000 7f51c4e00640 1 -- 172.21.15.50:0/1825590048 wait complete. 2024-09-21T06:08:46.682 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:46.683+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:46.682 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:46.683+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:46.683 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:46.683+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:46.683 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:46.683+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:46.683 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:46.683+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:46.684 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-21T06:08:46.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.850+0000 7f48b2c00640 1 -- 172.21.15.50:0/2308731082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ac0cefa0 msgr2=0x7f48ac0cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.850+0000 7f48b2c00640 1 --2- 172.21.15.50:0/2308731082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ac0cefa0 0x7f48ac0cf3b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f4894009900 tx=0x7f489401c8e0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.850+0000 7f48b2c00640 1 -- 172.21.15.50:0/2308731082 shutdown_connections 2024-09-21T06:08:46.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.850+0000 7f48b2c00640 1 --2- 172.21.15.50:0/2308731082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ac0cefa0 0x7f48ac0cf3b0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.850+0000 7f48b2c00640 1 --2- 172.21.15.50:0/2308731082 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48ac0749a0 0x7f48ac074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.850+0000 7f48b2c00640 1 -- 172.21.15.50:0/2308731082 >> 172.21.15.50:0/2308731082 conn(0x7f48ac0b2040 msgr2=0x7f48ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:46.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.850+0000 7f48b2c00640 1 -- 172.21.15.50:0/2308731082 shutdown_connections 2024-09-21T06:08:46.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.850+0000 7f48b2c00640 1 -- 172.21.15.50:0/2308731082 wait complete. 2024-09-21T06:08:46.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.850+0000 7f48b2c00640 1 Processor -- start 2024-09-21T06:08:46.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.850+0000 7f48b2c00640 1 -- start start 2024-09-21T06:08:46.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48b2c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48ac0749a0 0x7f48ac13cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48b2c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ac0cefa0 0x7f48ac15dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48b2c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48ac160330 0x7f48ac15e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48b2c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac16a060 con 0x7f48ac0cefa0 2024-09-21T06:08:46.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48b2c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac169ee0 con 0x7f48ac0749a0 2024-09-21T06:08:46.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48b2c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f48ac16a1e0 con 0x7f48ac160330 2024-09-21T06:08:46.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ac0cefa0 0x7f48ac15dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ac0cefa0 0x7f48ac15dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60692/0 (socket says 172.21.15.50:60692) 2024-09-21T06:08:46.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a3e00640 1 -- 172.21.15.50:0/3264700119 learned_addr learned my addr 172.21.15.50:0/3264700119 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:46.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a3e00640 1 -- 172.21.15.50:0/3264700119 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48ac160330 msgr2=0x7f48ac15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a3e00640 1 --2- 172.21.15.50:0/3264700119 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48ac160330 0x7f48ac15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a3e00640 1 -- 172.21.15.50:0/3264700119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48ac0749a0 msgr2=0x7f48ac13cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a3e00640 1 --2- 172.21.15.50:0/3264700119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48ac0749a0 0x7f48ac13cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a3e00640 1 -- 172.21.15.50:0/3264700119 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac13d150 con 0x7f48ac0cefa0 2024-09-21T06:08:46.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a3e00640 1 --2- 172.21.15.50:0/3264700119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ac0cefa0 0x7f48ac15dee0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f4894009850 tx=0x7f48940a5890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:46.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a8c00640 1 -- 172.21.15.50:0/3264700119 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48940b2040 con 0x7f48ac0cefa0 2024-09-21T06:08:46.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a8c00640 1 -- 172.21.15.50:0/3264700119 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4894004430 con 0x7f48ac0cefa0 2024-09-21T06:08:46.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48b2c00640 1 -- 172.21.15.50:0/3264700119 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ac0af7c0 con 0x7f48ac0cefa0 2024-09-21T06:08:46.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a8c00640 1 -- 172.21.15.50:0/3264700119 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894005260 con 0x7f48ac0cefa0 2024-09-21T06:08:46.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48b2c00640 1 -- 172.21.15.50:0/3264700119 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ac0aa600 con 0x7f48ac0cefa0 2024-09-21T06:08:46.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48b2c00640 1 -- 172.21.15.50:0/3264700119 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ac074db0 con 0x7f48ac0cefa0 2024-09-21T06:08:46.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.854+0000 7f48a8c00640 1 -- 172.21.15.50:0/3264700119 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f4894005400 con 0x7f48ac0cefa0 2024-09-21T06:08:46.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.858+0000 7f48a8c00640 1 --2- 172.21.15.50:0/3264700119 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4888074880 0x7f4888076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.858+0000 7f48a8c00640 1 -- 172.21.15.50:0/3264700119 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f489412c670 con 0x7f48ac0cefa0 2024-09-21T06:08:46.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.858+0000 7f48abe00640 1 --2- 172.21.15.50:0/3264700119 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4888074880 0x7f4888076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.858+0000 7f48abe00640 1 --2- 172.21.15.50:0/3264700119 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4888074880 0x7f4888076d90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f489c0058d0 tx=0x7f489c005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:46.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:46.862+0000 7f48a8c00640 1 -- 172.21.15.50:0/3264700119 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f48940bd020 con 0x7f48ac0cefa0 2024-09-21T06:08:47.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.066+0000 7f48b2c00640 1 -- 172.21.15.50:0/3264700119 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f48ac06b540 con 0x7f4888074880 2024-09-21T06:08:47.078 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:47.079+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:47.078 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:47.079+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:47.078 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:47.079+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:47.078 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:47.079+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:47.078 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:08:47.079+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:08:47.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.118+0000 7f48a8c00640 1 -- 172.21.15.50:0/3264700119 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f489c0096c0 con 0x7f4888074880 2024-09-21T06:08:47.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.122+0000 7f48b2c00640 1 -- 172.21.15.50:0/3264700119 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4888074880 msgr2=0x7f4888076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:47.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.122+0000 7f48b2c00640 1 --2- 172.21.15.50:0/3264700119 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4888074880 0x7f4888076d90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f489c0058d0 tx=0x7f489c005860 comp rx=0 tx=0).stop 2024-09-21T06:08:47.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.122+0000 7f48b2c00640 1 -- 172.21.15.50:0/3264700119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ac0cefa0 msgr2=0x7f48ac15dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:47.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.122+0000 7f48b2c00640 1 --2- 172.21.15.50:0/3264700119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ac0cefa0 0x7f48ac15dee0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f4894009850 tx=0x7f48940a5890 comp rx=0 tx=0).stop 2024-09-21T06:08:47.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.122+0000 7f48b2c00640 1 -- 172.21.15.50:0/3264700119 shutdown_connections 2024-09-21T06:08:47.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.122+0000 7f48b2c00640 1 --2- 172.21.15.50:0/3264700119 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f48ac160330 0x7f48ac15e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.122+0000 7f48b2c00640 1 --2- 172.21.15.50:0/3264700119 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4888074880 0x7f4888076d90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.122+0000 7f48b2c00640 1 --2- 172.21.15.50:0/3264700119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48ac0cefa0 0x7f48ac15dee0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.122+0000 7f48b2c00640 1 --2- 172.21.15.50:0/3264700119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48ac0749a0 0x7f48ac13cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.122+0000 7f48b2c00640 1 -- 172.21.15.50:0/3264700119 >> 172.21.15.50:0/3264700119 conn(0x7f48ac0b2040 msgr2=0x7f48ac15bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:47.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.122+0000 7f48b2c00640 1 -- 172.21.15.50:0/3264700119 shutdown_connections 2024-09-21T06:08:47.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.122+0000 7f48b2c00640 1 -- 172.21.15.50:0/3264700119 wait complete. 2024-09-21T06:08:47.141 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:08:47.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.302+0000 7f8650e00640 1 -- 172.21.15.50:0/3036242963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f864c0749a0 msgr2=0x7f864c074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:47.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.302+0000 7f8650e00640 1 --2- 172.21.15.50:0/3036242963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f864c0749a0 0x7f864c074db0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f864c0726f0 tx=0x7f863c01c8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.302+0000 7f8650e00640 1 -- 172.21.15.50:0/3036242963 shutdown_connections 2024-09-21T06:08:47.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.302+0000 7f8650e00640 1 --2- 172.21.15.50:0/3036242963 >> v2:172.21.15.88:3300/0 conn(0x7f864c0cefa0 0x7f864c0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.302+0000 7f8650e00640 1 --2- 172.21.15.50:0/3036242963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f864c0749a0 0x7f864c074db0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.302+0000 7f8650e00640 1 -- 172.21.15.50:0/3036242963 >> 172.21.15.50:0/3036242963 conn(0x7f864c0b2040 msgr2=0x7f864c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:47.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.302+0000 7f8650e00640 1 -- 172.21.15.50:0/3036242963 shutdown_connections 2024-09-21T06:08:47.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.302+0000 7f8650e00640 1 -- 172.21.15.50:0/3036242963 wait complete. 2024-09-21T06:08:47.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f8650e00640 1 Processor -- start 2024-09-21T06:08:47.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f8650e00640 1 -- start start 2024-09-21T06:08:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f8650e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f864c0749a0 0x7f864c159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f8650e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f864c0cefa0 0x7f864c141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f8650e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f864c160330 0x7f864c141880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f8650e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f864c16a060 con 0x7f864c160330 2024-09-21T06:08:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f8650e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f864c169ee0 con 0x7f864c0cefa0 2024-09-21T06:08:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f8650e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f864c16a1e0 con 0x7f864c0749a0 2024-09-21T06:08:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f864a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f864c160330 0x7f864c141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f864a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f864c160330 0x7f864c141880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60712/0 (socket says 172.21.15.50:60712) 2024-09-21T06:08:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f864a800640 1 -- 172.21.15.50:0/17950407 learned_addr learned my addr 172.21.15.50:0/17950407 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:08:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f8643e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f864c0cefa0 0x7f864c141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:47.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f8649e00640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f864c0749a0 0x7f864c159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:47.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f864a800640 1 -- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f864c0749a0 msgr2=0x7f864c159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:47.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f864a800640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f864c0749a0 0x7f864c159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f864a800640 1 -- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f864c0cefa0 msgr2=0x7f864c141340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:47.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f864a800640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f864c0cefa0 0x7f864c141340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f864a800640 1 -- 172.21.15.50:0/17950407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864c1f8b30 con 0x7f864c160330 2024-09-21T06:08:47.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f8643e00640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f864c0cefa0 0x7f864c141340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:47.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f864a800640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f864c160330 0x7f864c141880 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f864400c730 tx=0x7f864400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:47.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.306+0000 7f8642000640 1 -- 172.21.15.50:0/17950407 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644003b80 con 0x7f864c160330 2024-09-21T06:08:47.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.310+0000 7f8650e00640 1 -- 172.21.15.50:0/17950407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864c13d610 con 0x7f864c160330 2024-09-21T06:08:47.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.310+0000 7f8642000640 1 -- 172.21.15.50:0/17950407 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8644003d20 con 0x7f864c160330 2024-09-21T06:08:47.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.310+0000 7f8642000640 1 -- 172.21.15.50:0/17950407 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864400aab0 con 0x7f864c160330 2024-09-21T06:08:47.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.310+0000 7f8650e00640 1 -- 172.21.15.50:0/17950407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864c13dbd0 con 0x7f864c160330 2024-09-21T06:08:47.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.310+0000 7f8650e00640 1 -- 172.21.15.50:0/17950407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f864c074db0 con 0x7f864c160330 2024-09-21T06:08:47.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.310+0000 7f8642000640 1 -- 172.21.15.50:0/17950407 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 41) ==== 99274+0+0 (secure 0 0 0) 0x7f864400ac50 con 0x7f864c160330 2024-09-21T06:08:47.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.310+0000 7f8642000640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f86280748b0 0x7f8628076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:47.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.310+0000 7f8642000640 1 -- 172.21.15.50:0/17950407 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f8644092bf0 con 0x7f864c160330 2024-09-21T06:08:47.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.310+0000 7f8649e00640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f86280748b0 0x7f8628076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:47.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.314+0000 7f8649e00640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f86280748b0 0x7f8628076dc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f863c0af000 tx=0x7f863c01b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:47.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.314+0000 7f8642000640 1 -- 172.21.15.50:0/17950407 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f864405f030 con 0x7f864c160330 2024-09-21T06:08:47.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.522+0000 7f8650e00640 1 -- 172.21.15.50:0/17950407 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f864c0cece0 con 0x7f86280748b0 2024-09-21T06:08:47.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8642000640 1 -- 172.21.15.50:0/17950407 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f864c0cece0 con 0x7f86280748b0 2024-09-21T06:08:47.529 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/3a7dbaf5-83b5-45d5-9f5a-7111f9ec7ce2 2024-09-21T06:08:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8650e00640 1 -- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f86280748b0 msgr2=0x7f8628076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8650e00640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f86280748b0 0x7f8628076dc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f863c0af000 tx=0x7f863c01b100 comp rx=0 tx=0).stop 2024-09-21T06:08:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8650e00640 1 -- 172.21.15.50:0/17950407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f864c160330 msgr2=0x7f864c141880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8650e00640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f864c160330 0x7f864c141880 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f864400c730 tx=0x7f864400cbf0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8650e00640 1 -- 172.21.15.50:0/17950407 shutdown_connections 2024-09-21T06:08:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8650e00640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f86280748b0 0x7f8628076dc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8650e00640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f864c160330 0x7f864c141880 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8650e00640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f864c0cefa0 0x7f864c141340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8650e00640 1 --2- 172.21.15.50:0/17950407 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f864c0749a0 0x7f864c159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:47.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8650e00640 1 -- 172.21.15.50:0/17950407 >> 172.21.15.50:0/17950407 conn(0x7f864c0b2040 msgr2=0x7f864c06e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:47.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8650e00640 1 -- 172.21.15.50:0/17950407 shutdown_connections 2024-09-21T06:08:47.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:08:47.530+0000 7f8650e00640 1 -- 172.21.15.50:0/17950407 wait complete. 2024-09-21T06:08:47.551 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/3a7dbaf5-83b5-45d5-9f5a-7111f9ec7ce2 2024-09-21T06:08:47.552 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:08:47.552 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:08:47.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:08:47.561 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/3a7dbaf5-83b5-45d5-9f5a-7111f9ec7ce2/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:08:48.398 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:08:48.401 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:08:57.026 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:08:57.026 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:08:57.026 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.40232 s, 114 MB/s 2024-09-21T06:08:57.027 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:08:57.027 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:08:57.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:08:57.036 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/3a7dbaf5-83b5-45d5-9f5a-7111f9ec7ce2/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:09:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:09:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:09:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.23769 s, 116 MB/s 2024-09-21T06:09:06.332 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:09:06.332 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:06.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:06.342 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/3a7dbaf5-83b5-45d5-9f5a-7111f9ec7ce2/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:09:16.541 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:09:16.541 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:09:16.541 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.1437 s, 106 MB/s 2024-09-21T06:09:16.542 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-21T06:09:16.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.706+0000 7fcff1600640 1 -- 172.21.15.50:0/3469629092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfec15b2d0 msgr2=0x7fcfec15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.706+0000 7fcff1600640 1 --2- 172.21.15.50:0/3469629092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfec15b2d0 0x7fcfec15b6e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0b3040 tx=0x7fcfd401c8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.706+0000 7fcff1600640 1 -- 172.21.15.50:0/3469629092 shutdown_connections 2024-09-21T06:09:16.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.706+0000 7fcff1600640 1 --2- 172.21.15.50:0/3469629092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfec15b2d0 0x7fcfec15b6e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.706+0000 7fcff1600640 1 --2- 172.21.15.50:0/3469629092 >> v2:172.21.15.88:3300/0 conn(0x7fcfec15e8c0 0x7fcfec15ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.706+0000 7fcff1600640 1 -- 172.21.15.50:0/3469629092 >> 172.21.15.50:0/3469629092 conn(0x7fcfec0b2040 msgr2=0x7fcfec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:16.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.706+0000 7fcff1600640 1 -- 172.21.15.50:0/3469629092 shutdown_connections 2024-09-21T06:09:16.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.706+0000 7fcff1600640 1 -- 172.21.15.50:0/3469629092 wait complete. 2024-09-21T06:09:16.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcff1600640 1 Processor -- start 2024-09-21T06:09:16.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcff1600640 1 -- start start 2024-09-21T06:09:16.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcff1600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec15b2d0 0x7fcfec13c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcff1600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcfec15bc20 0x7fcfec06e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcff1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfec15e8c0 0x7fcfec06e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcff1600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec0b5550 con 0x7fcfec15e8c0 2024-09-21T06:09:16.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcff1600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec0b53d0 con 0x7fcfec15b2d0 2024-09-21T06:09:16.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcff1600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fcfec15d990 con 0x7fcfec15bc20 2024-09-21T06:09:16.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfeb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfec15e8c0 0x7fcfec06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfeb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfec15e8c0 0x7fcfec06e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55256/0 (socket says 172.21.15.50:55256) 2024-09-21T06:09:16.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfeb400640 1 -- 172.21.15.50:0/3535320819 learned_addr learned my addr 172.21.15.50:0/3535320819 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:16.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfe3e00640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcfec15bc20 0x7fcfec06e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfeaa00640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec15b2d0 0x7fcfec13c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfeb400640 1 -- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcfec15bc20 msgr2=0x7fcfec06e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfeb400640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcfec15bc20 0x7fcfec06e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfeb400640 1 -- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec15b2d0 msgr2=0x7fcfec13c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfeb400640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec15b2d0 0x7fcfec13c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfeb400640 1 -- 172.21.15.50:0/3535320819 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfec1fc750 con 0x7fcfec15e8c0 2024-09-21T06:09:16.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfeb400640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfec15e8c0 0x7fcfec06e710 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400c730 tx=0x7fcfe400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:16.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfe2a00640 1 -- 172.21.15.50:0/3535320819 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4003b80 con 0x7fcfec15e8c0 2024-09-21T06:09:16.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfe2a00640 1 -- 172.21.15.50:0/3535320819 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfe4003d20 con 0x7fcfec15e8c0 2024-09-21T06:09:16.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcff1600640 1 -- 172.21.15.50:0/3535320819 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfec1fca30 con 0x7fcfec15e8c0 2024-09-21T06:09:16.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcfe2a00640 1 -- 172.21.15.50:0/3535320819 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4005330 con 0x7fcfec15e8c0 2024-09-21T06:09:16.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.710+0000 7fcff1600640 1 -- 172.21.15.50:0/3535320819 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfec139d50 con 0x7fcfec15e8c0 2024-09-21T06:09:16.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.714+0000 7fcff1600640 1 -- 172.21.15.50:0/3535320819 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfec15b6e0 con 0x7fcfec15e8c0 2024-09-21T06:09:16.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.714+0000 7fcfe2a00640 1 -- 172.21.15.50:0/3535320819 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 42) ==== 99376+0+0 (secure 0 0 0) 0x7fcfe40056e0 con 0x7fcfec15e8c0 2024-09-21T06:09:16.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.714+0000 7fcfe2a00640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcfc80749a0 0x7fcfc8076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.714+0000 7fcfe2a00640 1 -- 172.21.15.50:0/3535320819 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fcfe4092e00 con 0x7fcfec15e8c0 2024-09-21T06:09:16.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.714+0000 7fcfeaa00640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcfc80749a0 0x7fcfc8076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.714+0000 7fcfeaa00640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcfc80749a0 0x7fcfc8076eb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009600 tx=0x7fcfdc0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:16.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.718+0000 7fcfe2a00640 1 -- 172.21.15.50:0/3535320819 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcfe405f2c0 con 0x7fcfec15e8c0 2024-09-21T06:09:16.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.926+0000 7fcff1600640 1 -- 172.21.15.50:0/3535320819 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fcfec13df20 con 0x7fcfc80749a0 2024-09-21T06:09:16.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcfe2a00640 1 -- 172.21.15.50:0/3535320819 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcfec13df20 con 0x7fcfc80749a0 2024-09-21T06:09:16.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcff1600640 1 -- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcfc80749a0 msgr2=0x7fcfc8076eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcff1600640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcfc80749a0 0x7fcfc8076eb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009600 tx=0x7fcfdc0091e0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcff1600640 1 -- 172.21.15.50:0/3535320819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfec15e8c0 msgr2=0x7fcfec06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcff1600640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfec15e8c0 0x7fcfec06e710 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400c730 tx=0x7fcfe400cbf0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcff1600640 1 -- 172.21.15.50:0/3535320819 shutdown_connections 2024-09-21T06:09:16.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcff1600640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcfc80749a0 0x7fcfc8076eb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcff1600640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfec15e8c0 0x7fcfec06e710 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcff1600640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcfec15bc20 0x7fcfec06e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcff1600640 1 --2- 172.21.15.50:0/3535320819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec15b2d0 0x7fcfec13c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcff1600640 1 -- 172.21.15.50:0/3535320819 >> 172.21.15.50:0/3535320819 conn(0x7fcfec0b2040 msgr2=0x7fcfec142890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:16.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcff1600640 1 -- 172.21.15.50:0/3535320819 shutdown_connections 2024-09-21T06:09:16.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:16.942+0000 7fcff1600640 1 -- 172.21.15.50:0/3535320819 wait complete. 2024-09-21T06:09:16.963 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:09:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.130+0000 7fd6efe00640 1 -- 172.21.15.50:0/3683641164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6d4003a60 msgr2=0x7fd6d4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.130+0000 7fd6efe00640 1 --2- 172.21.15.50:0/3683641164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6d4003a60 0x7fd6d4003ed0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009b60 tx=0x7fd6e001d270 comp rx=0 tx=0).stop 2024-09-21T06:09:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.130+0000 7fd6efe00640 1 -- 172.21.15.50:0/3683641164 shutdown_connections 2024-09-21T06:09:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.130+0000 7fd6efe00640 1 --2- 172.21.15.50:0/3683641164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6d4003a60 0x7fd6d4003ed0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.130+0000 7fd6efe00640 1 -- 172.21.15.50:0/3683641164 >> 172.21.15.50:0/3683641164 conn(0x7fd6f00b41f0 msgr2=0x7fd6f00b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.130+0000 7fd6efe00640 1 -- 172.21.15.50:0/3683641164 shutdown_connections 2024-09-21T06:09:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.130+0000 7fd6efe00640 1 -- 172.21.15.50:0/3683641164 wait complete. 2024-09-21T06:09:17.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.130+0000 7fd6efe00640 1 Processor -- start 2024-09-21T06:09:17.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.130+0000 7fd6efe00640 1 -- start start 2024-09-21T06:09:17.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6efe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6d4003a60 0x7fd6f009c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:17.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f0157f60 0x7fd6f009d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:17.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6efe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6f009d900 0x7fd6f0159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:17.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6efe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f00cea30 con 0x7fd6f0157f60 2024-09-21T06:09:17.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6efe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f00ce8b0 con 0x7fd6f009d900 2024-09-21T06:09:17.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6efe00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f00d1190 con 0x7fd6d4003a60 2024-09-21T06:09:17.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6ee000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f0157f60 0x7fd6f009d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:17.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6ee000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f0157f60 0x7fd6f009d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55278/0 (socket says 172.21.15.50:55278) 2024-09-21T06:09:17.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6ee000640 1 -- 172.21.15.50:0/2713235407 learned_addr learned my addr 172.21.15.50:0/2713235407 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:17.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6eea00640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6d4003a60 0x7fd6f009c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:17.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6ef400640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6f009d900 0x7fd6f0159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:17.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6ee000640 1 -- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6d4003a60 msgr2=0x7fd6f009c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:17.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6ee000640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6d4003a60 0x7fd6f009c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6ee000640 1 -- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6f009d900 msgr2=0x7fd6f0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:17.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6ee000640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6f009d900 0x7fd6f0159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6ee000640 1 -- 172.21.15.50:0/2713235407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f009b6f0 con 0x7fd6f0157f60 2024-09-21T06:09:17.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6ee000640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f0157f60 0x7fd6f009d3c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc006600 tx=0x7fd6dc0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:17.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6e7400640 1 -- 172.21.15.50:0/2713235407 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc015070 con 0x7fd6f0157f60 2024-09-21T06:09:17.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6e7400640 1 -- 172.21.15.50:0/2713235407 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6dc011900 con 0x7fd6f0157f60 2024-09-21T06:09:17.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6e7400640 1 -- 172.21.15.50:0/2713235407 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc011d50 con 0x7fd6f0157f60 2024-09-21T06:09:17.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6efe00640 1 -- 172.21.15.50:0/2713235407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f009b9d0 con 0x7fd6f0157f60 2024-09-21T06:09:17.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6efe00640 1 -- 172.21.15.50:0/2713235407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f009bf90 con 0x7fd6f0157f60 2024-09-21T06:09:17.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.134+0000 7fd6efe00640 1 -- 172.21.15.50:0/2713235407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f009b460 con 0x7fd6f0157f60 2024-09-21T06:09:17.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.138+0000 7fd6e7400640 1 -- 172.21.15.50:0/2713235407 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 42) ==== 99376+0+0 (secure 0 0 0) 0x7fd6dc018070 con 0x7fd6f0157f60 2024-09-21T06:09:17.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.138+0000 7fd6e7400640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd6d00749d0 0x7fd6d0076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:17.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.138+0000 7fd6e7400640 1 -- 172.21.15.50:0/2713235407 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fd6dc093bb0 con 0x7fd6f0157f60 2024-09-21T06:09:17.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.138+0000 7fd6eea00640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd6d00749d0 0x7fd6d0076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:17.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.138+0000 7fd6eea00640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd6d00749d0 0x7fd6d0076ee0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00ae000 tx=0x7fd6e001a010 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:17.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.142+0000 7fd6e7400640 1 -- 172.21.15.50:0/2713235407 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd6dc05ffe0 con 0x7fd6f0157f60 2024-09-21T06:09:17.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.350+0000 7fd6efe00640 1 -- 172.21.15.50:0/2713235407 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fd6f006b540 con 0x7fd6d00749d0 2024-09-21T06:09:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6e7400640 1 -- 172.21.15.50:0/2713235407 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fd6e0004200 con 0x7fd6d00749d0 2024-09-21T06:09:17.356 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/3a7dbaf5-83b5-45d5-9f5a-7111f9ec7ce2 2024-09-21T06:09:17.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6efe00640 1 -- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd6d00749d0 msgr2=0x7fd6d0076ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6efe00640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd6d00749d0 0x7fd6d0076ee0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00ae000 tx=0x7fd6e001a010 comp rx=0 tx=0).stop 2024-09-21T06:09:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6efe00640 1 -- 172.21.15.50:0/2713235407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f0157f60 msgr2=0x7fd6f009d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6efe00640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f0157f60 0x7fd6f009d3c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc006600 tx=0x7fd6dc0039d0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6efe00640 1 -- 172.21.15.50:0/2713235407 shutdown_connections 2024-09-21T06:09:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6efe00640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6f009d900 0x7fd6f0159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6efe00640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd6d00749d0 0x7fd6d0076ee0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6efe00640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f0157f60 0x7fd6f009d3c0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6efe00640 1 --2- 172.21.15.50:0/2713235407 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6d4003a60 0x7fd6f009c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6efe00640 1 -- 172.21.15.50:0/2713235407 >> 172.21.15.50:0/2713235407 conn(0x7fd6f00b41f0 msgr2=0x7fd6f015eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6efe00640 1 -- 172.21.15.50:0/2713235407 shutdown_connections 2024-09-21T06:09:17.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:17.358+0000 7fd6efe00640 1 -- 172.21.15.50:0/2713235407 wait complete. 2024-09-21T06:09:17.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:17.379 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/3a7dbaf5-83b5-45d5-9f5a-7111f9ec7ce2 2024-09-21T06:09:17.379 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:09:17.388 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/3a7dbaf5-83b5-45d5-9f5a-7111f9ec7ce2 2024-09-21T06:09:17.388 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="3055190016" 2024-09-21T06:09:17.388 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:17.389 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-21T06:09:18.477 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:09:18.480 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:09:19.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:19.390 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/3a7dbaf5-83b5-45d5-9f5a-7111f9ec7ce2 2024-09-21T06:09:19.390 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:09:19.402 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/3a7dbaf5-83b5-45d5-9f5a-7111f9ec7ce2 2024-09-21T06:09:19.402 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="3221225472" 2024-09-21T06:09:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:19.403 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-21T06:09:19.403 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-21T06:09:19.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.626+0000 7f3381800640 1 -- 172.21.15.50:0/4154959425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f337c0cefa0 msgr2=0x7f337c0cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:19.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.626+0000 7f3381800640 1 --2- 172.21.15.50:0/4154959425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f337c0cefa0 0x7f337c0cf3b0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f3364009900 tx=0x7f336401c8e0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.626+0000 7f3381800640 1 -- 172.21.15.50:0/4154959425 shutdown_connections 2024-09-21T06:09:19.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.626+0000 7f3381800640 1 --2- 172.21.15.50:0/4154959425 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f337c0b3340 0x7f337c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.626+0000 7f3381800640 1 --2- 172.21.15.50:0/4154959425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f337c0cefa0 0x7f337c0cf3b0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.626+0000 7f3381800640 1 -- 172.21.15.50:0/4154959425 >> 172.21.15.50:0/4154959425 conn(0x7f337c0b2040 msgr2=0x7f337c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:19.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.626+0000 7f3381800640 1 -- 172.21.15.50:0/4154959425 shutdown_connections 2024-09-21T06:09:19.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.626+0000 7f3381800640 1 -- 172.21.15.50:0/4154959425 wait complete. 2024-09-21T06:09:19.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.626+0000 7f3381800640 1 Processor -- start 2024-09-21T06:09:19.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.626+0000 7f3381800640 1 -- start start 2024-09-21T06:09:19.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3381800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f337c0749a0 0x7f337c13cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:19.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3381800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f337c0b3340 0x7f337c0b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:19.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3381800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f337c0cefa0 0x7f337c0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:19.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3381800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f337c169e10 con 0x7f337c0cefa0 2024-09-21T06:09:19.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3381800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f337c169c90 con 0x7f337c0749a0 2024-09-21T06:09:19.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3381800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f337c169f90 con 0x7f337c0b3340 2024-09-21T06:09:19.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f337b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f337c0cefa0 0x7f337c0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:19.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f337aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f337c0749a0 0x7f337c13cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:19.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3373e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f337c0b3340 0x7f337c0b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:19.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f337aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f337c0749a0 0x7f337c13cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:42910/0 (socket says 172.21.15.50:42910) 2024-09-21T06:09:19.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f337aa00640 1 -- 172.21.15.50:0/1503958314 learned_addr learned my addr 172.21.15.50:0/1503958314 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:19.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3373e00640 1 -- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f337c0749a0 msgr2=0x7f337c13cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:19.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3373e00640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f337c0749a0 0x7f337c13cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3373e00640 1 -- 172.21.15.50:0/1503958314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f337c0cefa0 msgr2=0x7f337c0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3373e00640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f337c0cefa0 0x7f337c0cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3373e00640 1 -- 172.21.15.50:0/1503958314 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c0b1cd0 con 0x7f337c0b3340 2024-09-21T06:09:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f337aa00640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f337c0749a0 0x7f337c13cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:09:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f337b400640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f337c0cefa0 0x7f337c0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:09:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3373e00640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f337c0b3340 0x7f337c0b0890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f33640ac000 tx=0x7f33640a5730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3372a00640 1 -- 172.21.15.50:0/1503958314 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33640b2040 con 0x7f337c0b3340 2024-09-21T06:09:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3372a00640 1 -- 172.21.15.50:0/1503958314 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33640042e0 con 0x7f337c0b3340 2024-09-21T06:09:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3372a00640 1 -- 172.21.15.50:0/1503958314 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364003860 con 0x7f337c0b3340 2024-09-21T06:09:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3381800640 1 -- 172.21.15.50:0/1503958314 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c1fc470 con 0x7f337c0b3340 2024-09-21T06:09:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3381800640 1 -- 172.21.15.50:0/1503958314 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c1fca30 con 0x7f337c0b3340 2024-09-21T06:09:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.630+0000 7f3381800640 1 -- 172.21.15.50:0/1503958314 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f337c0b3750 con 0x7f337c0b3340 2024-09-21T06:09:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.634+0000 7f3372a00640 1 -- 172.21.15.50:0/1503958314 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 42) ==== 99376+0+0 (secure 0 0 0) 0x7f3364003a00 con 0x7f337c0b3340 2024-09-21T06:09:19.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.634+0000 7f3372a00640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33580749a0 0x7f3358076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:19.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.634+0000 7f3372a00640 1 -- 172.21.15.50:0/1503958314 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f336412d230 con 0x7f337c0b3340 2024-09-21T06:09:19.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.634+0000 7f337aa00640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33580749a0 0x7f3358076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:19.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.634+0000 7f337aa00640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33580749a0 0x7f3358076eb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f336c001c60 tx=0x7f336c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:19.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.638+0000 7f3372a00640 1 -- 172.21.15.50:0/1503958314 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f33640164d0 con 0x7f337c0b3340 2024-09-21T06:09:19.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.850+0000 7f3381800640 1 -- 172.21.15.50:0/1503958314 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f337c06b540 con 0x7f33580749a0 2024-09-21T06:09:19.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.922+0000 7f3372a00640 1 -- 172.21.15.50:0/1503958314 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f336c00fc90 con 0x7f33580749a0 2024-09-21T06:09:19.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.926+0000 7f3381800640 1 -- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33580749a0 msgr2=0x7f3358076eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:19.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.926+0000 7f3381800640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33580749a0 0x7f3358076eb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f336c001c60 tx=0x7f336c009040 comp rx=0 tx=0).stop 2024-09-21T06:09:19.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.926+0000 7f3381800640 1 -- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f337c0b3340 msgr2=0x7f337c0b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:19.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.926+0000 7f3381800640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f337c0b3340 0x7f337c0b0890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f33640ac000 tx=0x7f33640a5730 comp rx=0 tx=0).stop 2024-09-21T06:09:19.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.926+0000 7f3381800640 1 -- 172.21.15.50:0/1503958314 shutdown_connections 2024-09-21T06:09:19.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.926+0000 7f3381800640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f33580749a0 0x7f3358076eb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.926+0000 7f3381800640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f337c0cefa0 0x7f337c0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.926+0000 7f3381800640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f337c0b3340 0x7f337c0b0890 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.926+0000 7f3381800640 1 --2- 172.21.15.50:0/1503958314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f337c0749a0 0x7f337c13cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.926+0000 7f3381800640 1 -- 172.21.15.50:0/1503958314 >> 172.21.15.50:0/1503958314 conn(0x7f337c0b2040 msgr2=0x7f337c13e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:19.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.926+0000 7f3381800640 1 -- 172.21.15.50:0/1503958314 shutdown_connections 2024-09-21T06:09:19.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:19.926+0000 7f3381800640 1 -- 172.21.15.50:0/1503958314 wait complete. 2024-09-21T06:09:19.927 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:19.927+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:19.927 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:19.927+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:19.927 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:19.927+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:19.927 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:19.927+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:19.928 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:19.927+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:19.946 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:19.947+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:19.946 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:19.947+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:19.946 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:19.947+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:19.946 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:19.947+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:19.946 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:19.947+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:19.948 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-21T06:09:20.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.114+0000 7f0c6f400640 1 -- 172.21.15.50:0/1740824751 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c680b54f0 msgr2=0x7f0c68155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:20.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.114+0000 7f0c6f400640 1 --2- 172.21.15.50:0/1740824751 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c680b54f0 0x7f0c68155b50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0c68073920 tx=0x7f0c5401c770 comp rx=0 tx=0).stop 2024-09-21T06:09:20.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.114+0000 7f0c6f400640 1 -- 172.21.15.50:0/1740824751 shutdown_connections 2024-09-21T06:09:20.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.114+0000 7f0c6f400640 1 --2- 172.21.15.50:0/1740824751 >> v2:172.21.15.88:3300/0 conn(0x7f0c68073a90 0x7f0c68073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.114+0000 7f0c6f400640 1 --2- 172.21.15.50:0/1740824751 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c680b54f0 0x7f0c68155b50 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.114+0000 7f0c6f400640 1 -- 172.21.15.50:0/1740824751 >> 172.21.15.50:0/1740824751 conn(0x7f0c680b2040 msgr2=0x7f0c680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:20.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.114+0000 7f0c6f400640 1 -- 172.21.15.50:0/1740824751 shutdown_connections 2024-09-21T06:09:20.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.114+0000 7f0c6f400640 1 -- 172.21.15.50:0/1740824751 wait complete. 2024-09-21T06:09:20.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c6f400640 1 Processor -- start 2024-09-21T06:09:20.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c6f400640 1 -- start start 2024-09-21T06:09:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c6f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c68072100 0x7f0c68154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c6f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c68073a90 0x7f0c6813ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c6f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c681501d0 0x7f0c680b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c6f400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68168e20 con 0x7f0c68073a90 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c6f400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68168ca0 con 0x7f0c68072100 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c6f400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0c68168fa0 con 0x7f0c681501d0 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c67e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c68073a90 0x7f0c6813ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c67e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c68073a90 0x7f0c6813ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55310/0 (socket says 172.21.15.50:55310) 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c67e00640 1 -- 172.21.15.50:0/2857896074 learned_addr learned my addr 172.21.15.50:0/2857896074 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c6ca00640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c68072100 0x7f0c68154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c6d400640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c681501d0 0x7f0c680b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c67e00640 1 -- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c681501d0 msgr2=0x7f0c680b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c67e00640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c681501d0 0x7f0c680b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c67e00640 1 -- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c68072100 msgr2=0x7f0c68154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:20.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c67e00640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c68072100 0x7f0c68154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.118+0000 7f0c67e00640 1 -- 172.21.15.50:0/2857896074 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c680afab0 con 0x7f0c68073a90 2024-09-21T06:09:20.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.122+0000 7f0c67e00640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c68073a90 0x7f0c6813ebe0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f0c5401cc50 tx=0x7f0c540a6760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:20.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.122+0000 7f0c64c00640 1 -- 172.21.15.50:0/2857896074 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c540be070 con 0x7f0c68073a90 2024-09-21T06:09:20.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.122+0000 7f0c64c00640 1 -- 172.21.15.50:0/2857896074 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c540a6c40 con 0x7f0c68073a90 2024-09-21T06:09:20.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.122+0000 7f0c64c00640 1 -- 172.21.15.50:0/2857896074 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c54005040 con 0x7f0c68073a90 2024-09-21T06:09:20.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.122+0000 7f0c6f400640 1 -- 172.21.15.50:0/2857896074 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c680aea10 con 0x7f0c68073a90 2024-09-21T06:09:20.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.122+0000 7f0c6f400640 1 -- 172.21.15.50:0/2857896074 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c680aefd0 con 0x7f0c68073a90 2024-09-21T06:09:20.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.122+0000 7f0c6f400640 1 -- 172.21.15.50:0/2857896074 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c68073ea0 con 0x7f0c68073a90 2024-09-21T06:09:20.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.122+0000 7f0c64c00640 1 -- 172.21.15.50:0/2857896074 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 42) ==== 99376+0+0 (secure 0 0 0) 0x7f0c540051e0 con 0x7f0c68073a90 2024-09-21T06:09:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.122+0000 7f0c64c00640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c480749a0 0x7f0c48076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.126+0000 7f0c64c00640 1 -- 172.21.15.50:0/2857896074 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f0c5412c8d0 con 0x7f0c68073a90 2024-09-21T06:09:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.126+0000 7f0c6ca00640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c480749a0 0x7f0c48076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.126+0000 7f0c6ca00640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c480749a0 0x7f0c48076eb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c007fd0 tx=0x7f0c5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:20.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.130+0000 7f0c64c00640 1 -- 172.21.15.50:0/2857896074 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0c540ac030 con 0x7f0c68073a90 2024-09-21T06:09:20.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.342+0000 7f0c6f400640 1 -- 172.21.15.50:0/2857896074 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f0c680ae7c0 con 0x7f0c480749a0 2024-09-21T06:09:20.929 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:20.927+0000 7f6539000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:20.929 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:20.927+0000 7f6539000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:20.929 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:20.927+0000 7f6539000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:20.929 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:20.927+0000 7f6539000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:20.929 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:20.927+0000 7f6539000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:20.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.962+0000 7f0c64c00640 1 -- 172.21.15.50:0/2857896074 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f0c680ae7c0 con 0x7f0c480749a0 2024-09-21T06:09:20.964 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:09:20.964 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:09:20.964 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "in-progress", 2024-09-21T06:09:20.964 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:09:20.964 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:09:20.964 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:09:20.964 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:09:20.964 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:20.964 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_report": { 2024-09-21T06:09:20.965 INFO:teuthology.orchestra.run.smithi050.stdout: "percentage cloned": "0.0%", 2024-09-21T06:09:20.965 INFO:teuthology.orchestra.run.smithi050.stdout: "amount cloned": "184/3.0G", 2024-09-21T06:09:20.965 INFO:teuthology.orchestra.run.smithi050.stdout: "files cloned": "3/4" 2024-09-21T06:09:20.965 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:20.965 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:20.965 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:09:20.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.966+0000 7f0c6f400640 1 -- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c480749a0 msgr2=0x7f0c48076eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:20.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.966+0000 7f0c6f400640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c480749a0 0x7f0c48076eb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c007fd0 tx=0x7f0c5c009040 comp rx=0 tx=0).stop 2024-09-21T06:09:20.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.966+0000 7f0c6f400640 1 -- 172.21.15.50:0/2857896074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c68073a90 msgr2=0x7f0c6813ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:20.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.966+0000 7f0c6f400640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c68073a90 0x7f0c6813ebe0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f0c5401cc50 tx=0x7f0c540a6760 comp rx=0 tx=0).stop 2024-09-21T06:09:20.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.966+0000 7f0c6f400640 1 -- 172.21.15.50:0/2857896074 shutdown_connections 2024-09-21T06:09:20.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.966+0000 7f0c6f400640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0c681501d0 0x7f0c680b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.966+0000 7f0c6f400640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c480749a0 0x7f0c48076eb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.966+0000 7f0c6f400640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c68073a90 0x7f0c6813ebe0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.966+0000 7f0c6f400640 1 --2- 172.21.15.50:0/2857896074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c68072100 0x7f0c68154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.966+0000 7f0c6f400640 1 -- 172.21.15.50:0/2857896074 >> 172.21.15.50:0/2857896074 conn(0x7f0c680b2040 msgr2=0x7f0c68159680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:20.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.966+0000 7f0c6f400640 1 -- 172.21.15.50:0/2857896074 shutdown_connections 2024-09-21T06:09:20.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:20.966+0000 7f0c6f400640 1 -- 172.21.15.50:0/2857896074 wait complete. 2024-09-21T06:09:21.000 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-21T06:09:21.000 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-21T06:09:21.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.258+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3851916049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf0068df0 msgr2=0x7f4bf0069200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:21.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.258+0000 7f4bf5c00640 1 --2- 172.21.15.50:0/3851916049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf0068df0 0x7f4bf0069200 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009c80 tx=0x7f4be801ccc0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.258+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3851916049 shutdown_connections 2024-09-21T06:09:21.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.258+0000 7f4bf5c00640 1 --2- 172.21.15.50:0/3851916049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf0068df0 0x7f4bf0069200 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.258+0000 7f4bf5c00640 1 --2- 172.21.15.50:0/3851916049 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4bf006de00 0x7f4bf006e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.258+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3851916049 >> 172.21.15.50:0/3851916049 conn(0x7f4bf0073380 msgr2=0x7f4bf0073790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:21.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.258+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3851916049 shutdown_connections 2024-09-21T06:09:21.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.258+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3851916049 wait complete. 2024-09-21T06:09:21.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.262+0000 7f4bf5c00640 1 Processor -- start 2024-09-21T06:09:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.262+0000 7f4bf5c00640 1 -- start start 2024-09-21T06:09:21.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.274+0000 7f4bf5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf0068df0 0x7f4bf0164740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.274+0000 7f4bf5c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf006de00 0x7f4bf0155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.274+0000 7f4bf5c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4bf0070460 0x7f4bf00b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.274+0000 7f4bf5c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf00d0940 con 0x7f4bf006de00 2024-09-21T06:09:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.274+0000 7f4bf5c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf00d07c0 con 0x7f4bf0068df0 2024-09-21T06:09:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.274+0000 7f4bf5c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4bf00d0ac0 con 0x7f4bf0070460 2024-09-21T06:09:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.278+0000 7f4bee000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf006de00 0x7f4bf0155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.278+0000 7f4bee000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf006de00 0x7f4bf0155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55330/0 (socket says 172.21.15.50:55330) 2024-09-21T06:09:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.278+0000 7f4bee000640 1 -- 172.21.15.50:0/3638674577 learned_addr learned my addr 172.21.15.50:0/3638674577 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.278+0000 7f4beea00640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf0068df0 0x7f4bf0164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.278+0000 7f4beea00640 1 -- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4bf0070460 msgr2=0x7f4bf00b4cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:09:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.278+0000 7f4beea00640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4bf0070460 0x7f4bf00b4cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.278+0000 7f4beea00640 1 -- 172.21.15.50:0/3638674577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf006de00 msgr2=0x7f4bf0155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.278+0000 7f4beea00640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf006de00 0x7f4bf0155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.278+0000 7f4beea00640 1 -- 172.21.15.50:0/3638674577 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf01654b0 con 0x7f4bf0068df0 2024-09-21T06:09:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.278+0000 7f4beea00640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf0068df0 0x7f4bf0164740 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009fd0 tx=0x7f4be000eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.282+0000 7f4be6a00640 1 -- 172.21.15.50:0/3638674577 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0004230 con 0x7f4bf0068df0 2024-09-21T06:09:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.286+0000 7f4be6a00640 1 -- 172.21.15.50:0/3638674577 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4be00043d0 con 0x7f4bf0068df0 2024-09-21T06:09:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.286+0000 7f4be6a00640 1 -- 172.21.15.50:0/3638674577 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be000cab0 con 0x7f4bf0068df0 2024-09-21T06:09:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.286+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3638674577 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf0165790 con 0x7f4bf0068df0 2024-09-21T06:09:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.286+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3638674577 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf01f9290 con 0x7f4bf0068df0 2024-09-21T06:09:21.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.286+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3638674577 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf0069200 con 0x7f4bf0068df0 2024-09-21T06:09:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.290+0000 7f4be6a00640 1 -- 172.21.15.50:0/3638674577 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 42) ==== 99376+0+0 (secure 0 0 0) 0x7f4be000cc50 con 0x7f4bf0068df0 2024-09-21T06:09:21.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.290+0000 7f4be6a00640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4bd0074a20 0x7f4bd0076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:21.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.290+0000 7f4be6a00640 1 -- 172.21.15.50:0/3638674577 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f4be00937e0 con 0x7f4bf0068df0 2024-09-21T06:09:21.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.290+0000 7f4bee000640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4bd0074a20 0x7f4bd0076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:21.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.294+0000 7f4bee000640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4bd0074a20 0x7f4bd0076f30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc001a40 tx=0x7f4bdc0014d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:21.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.294+0000 7f4be6a00640 1 -- 172.21.15.50:0/3638674577 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4be00602e0 con 0x7f4bf0068df0 2024-09-21T06:09:21.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.366+0000 7f4be6a00640 1 -- 172.21.15.50:0/3638674577 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 43) ==== 99529+0+0 (secure 0 0 0) 0x7f4be009b030 con 0x7f4bf0068df0 2024-09-21T06:09:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.658+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3638674577 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f4bf006b540 con 0x7f4bd0074a20 2024-09-21T06:09:21.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.738+0000 7f4be6a00640 1 -- 172.21.15.50:0/3638674577 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f4bdc00f8e0 con 0x7f4bd0074a20 2024-09-21T06:09:21.739 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "in-progress", 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_report": { 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: "percentage cloned": "0.0%", 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: "amount cloned": "184/3.0G", 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: "files cloned": "4/4" 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:21.740 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:09:21.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.742+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4bd0074a20 msgr2=0x7f4bd0076f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:21.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.742+0000 7f4bf5c00640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4bd0074a20 0x7f4bd0076f30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc001a40 tx=0x7f4bdc0014d0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.742+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf0068df0 msgr2=0x7f4bf0164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:21.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.742+0000 7f4bf5c00640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf0068df0 0x7f4bf0164740 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009fd0 tx=0x7f4be000eb50 comp rx=0 tx=0).stop 2024-09-21T06:09:21.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.742+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3638674577 shutdown_connections 2024-09-21T06:09:21.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.742+0000 7f4bf5c00640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4bf0070460 0x7f4bf00b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.742+0000 7f4bf5c00640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4bd0074a20 0x7f4bd0076f30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.742+0000 7f4bf5c00640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf006de00 0x7f4bf0155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.742+0000 7f4bf5c00640 1 --2- 172.21.15.50:0/3638674577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf0068df0 0x7f4bf0164740 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.742+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3638674577 >> 172.21.15.50:0/3638674577 conn(0x7f4bf0073380 msgr2=0x7f4bf00b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:21.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.742+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3638674577 shutdown_connections 2024-09-21T06:09:21.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:21.742+0000 7f4bf5c00640 1 -- 172.21.15.50:0/3638674577 wait complete. 2024-09-21T06:09:21.796 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 0.0% 2024-09-21T06:09:22.797 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-21T06:09:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.066+0000 7f7500a00640 1 -- 172.21.15.50:0/1208840455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74fc0cdbb0 msgr2=0x7f74fc15ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.066+0000 7f7500a00640 1 --2- 172.21.15.50:0/1208840455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74fc0cdbb0 0x7f74fc15ebf0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009c80 tx=0x7f74f401cda0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.066+0000 7f7500a00640 1 -- 172.21.15.50:0/1208840455 shutdown_connections 2024-09-21T06:09:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.066+0000 7f7500a00640 1 --2- 172.21.15.50:0/1208840455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74fc0cdbb0 0x7f74fc15ebf0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.066+0000 7f7500a00640 1 --2- 172.21.15.50:0/1208840455 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74fc15aa50 0x7f74fc0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.066+0000 7f7500a00640 1 -- 172.21.15.50:0/1208840455 >> 172.21.15.50:0/1208840455 conn(0x7f74fc0b2040 msgr2=0x7f74fc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:23.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.066+0000 7f7500a00640 1 -- 172.21.15.50:0/1208840455 shutdown_connections 2024-09-21T06:09:23.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.066+0000 7f7500a00640 1 -- 172.21.15.50:0/1208840455 wait complete. 2024-09-21T06:09:23.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f7500a00640 1 Processor -- start 2024-09-21T06:09:23.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f7500a00640 1 -- start start 2024-09-21T06:09:23.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f7500a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74fc0cd7a0 0x7f74fc15c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:23.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f7500a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74fc0cdbb0 0x7f74fc14e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:23.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f7500a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74fc15aa50 0x7f74fc155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:23.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f7500a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc070470 con 0x7f74fc0cdbb0 2024-09-21T06:09:23.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f7500a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc0702f0 con 0x7f74fc0cd7a0 2024-09-21T06:09:23.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f7500a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f74fc167e80 con 0x7f74fc15aa50 2024-09-21T06:09:23.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f74f9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74fc0cdbb0 0x7f74fc14e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:23.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f74f9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74fc0cdbb0 0x7f74fc14e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55354/0 (socket says 172.21.15.50:55354) 2024-09-21T06:09:23.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f74f9400640 1 -- 172.21.15.50:0/2086183571 learned_addr learned my addr 172.21.15.50:0/2086183571 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:23.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f74f9e00640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74fc0cd7a0 0x7f74fc15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:23.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f74f9400640 1 -- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74fc15aa50 msgr2=0x7f74fc155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:23.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f74f9400640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74fc15aa50 0x7f74fc155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f74f9400640 1 -- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74fc0cd7a0 msgr2=0x7f74fc15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:23.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f74f9400640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74fc0cd7a0 0x7f74fc15c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f74f9400640 1 -- 172.21.15.50:0/2086183571 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74fc1f68b0 con 0x7f74fc0cdbb0 2024-09-21T06:09:23.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.070+0000 7f74f9e00640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74fc0cd7a0 0x7f74fc15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:23.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.074+0000 7f74f9400640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74fc0cdbb0 0x7f74fc14e900 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f74e800c3c0 tx=0x7f74e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:23.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.074+0000 7f74f2000640 1 -- 172.21.15.50:0/2086183571 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8017070 con 0x7f74fc0cdbb0 2024-09-21T06:09:23.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.074+0000 7f74f2000640 1 -- 172.21.15.50:0/2086183571 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74e80053c0 con 0x7f74fc0cdbb0 2024-09-21T06:09:23.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.074+0000 7f74f2000640 1 -- 172.21.15.50:0/2086183571 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8003860 con 0x7f74fc0cdbb0 2024-09-21T06:09:23.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.074+0000 7f7500a00640 1 -- 172.21.15.50:0/2086183571 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74fc0af270 con 0x7f74fc0cdbb0 2024-09-21T06:09:23.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.074+0000 7f7500a00640 1 -- 172.21.15.50:0/2086183571 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74fc1f6e30 con 0x7f74fc0cdbb0 2024-09-21T06:09:23.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.074+0000 7f7500a00640 1 -- 172.21.15.50:0/2086183571 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74fc0aefc0 con 0x7f74fc0cdbb0 2024-09-21T06:09:23.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.078+0000 7f74f2000640 1 -- 172.21.15.50:0/2086183571 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 43) ==== 99529+0+0 (secure 0 0 0) 0x7f74e801a070 con 0x7f74fc0cdbb0 2024-09-21T06:09:23.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.078+0000 7f74f2000640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74dc074b50 0x7f74dc077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:23.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.078+0000 7f74f2000640 1 -- 172.21.15.50:0/2086183571 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f74e8092370 con 0x7f74fc0cdbb0 2024-09-21T06:09:23.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.082+0000 7f74f9e00640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74dc074b50 0x7f74dc077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:23.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.086+0000 7f74f2000640 1 -- 172.21.15.50:0/2086183571 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f74e805ee90 con 0x7f74fc0cdbb0 2024-09-21T06:09:23.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.086+0000 7f74f9e00640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74dc074b50 0x7f74dc077060 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00bbd0 tx=0x7f74ec00b5b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:23.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.454+0000 7f7500a00640 1 -- 172.21.15.50:0/2086183571 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f74fc06b540 con 0x7f74dc074b50 2024-09-21T06:09:23.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f74f2000640 1 -- 172.21.15.50:0/2086183571 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+281 (secure 0 0 0) 0x7f74ec00b7c0 con 0x7f74dc074b50 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "in-progress", 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_report": { 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout: "percentage cloned": "43.75%", 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout: "amount cloned": "1.3G/3.0G", 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout: "files cloned": "5/4" 2024-09-21T06:09:23.501 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:23.502 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:23.502 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:09:23.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f7500a00640 1 -- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74dc074b50 msgr2=0x7f74dc077060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:23.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f7500a00640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74dc074b50 0x7f74dc077060 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00bbd0 tx=0x7f74ec00b5b0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f7500a00640 1 -- 172.21.15.50:0/2086183571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74fc0cdbb0 msgr2=0x7f74fc14e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:23.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f7500a00640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74fc0cdbb0 0x7f74fc14e900 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f74e800c3c0 tx=0x7f74e800c880 comp rx=0 tx=0).stop 2024-09-21T06:09:23.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f7500a00640 1 -- 172.21.15.50:0/2086183571 shutdown_connections 2024-09-21T06:09:23.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f7500a00640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74fc15aa50 0x7f74fc155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f7500a00640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74dc074b50 0x7f74dc077060 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f7500a00640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74fc0cdbb0 0x7f74fc14e900 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f7500a00640 1 --2- 172.21.15.50:0/2086183571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74fc0cd7a0 0x7f74fc15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f7500a00640 1 -- 172.21.15.50:0/2086183571 >> 172.21.15.50:0/2086183571 conn(0x7f74fc0b2040 msgr2=0x7f74fc15d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:23.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f7500a00640 1 -- 172.21.15.50:0/2086183571 shutdown_connections 2024-09-21T06:09:23.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:23.502+0000 7f7500a00640 1 -- 172.21.15.50:0/2086183571 wait complete. 2024-09-21T06:09:23.537 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 43.75% 2024-09-21T06:09:24.538 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-21T06:09:24.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.790+0000 7fb99e400640 1 -- 172.21.15.50:0/3514258159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb998070530 msgr2=0x7fb998070940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:24.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.790+0000 7fb99e400640 1 --2- 172.21.15.50:0/3514258159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb998070530 0x7fb998070940 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb9980d24f0 tx=0x7fb98001c8e0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.790+0000 7fb99e400640 1 -- 172.21.15.50:0/3514258159 shutdown_connections 2024-09-21T06:09:24.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.790+0000 7fb99e400640 1 --2- 172.21.15.50:0/3514258159 >> v2:172.21.15.88:3300/0 conn(0x7fb9980b44a0 0x7fb9980b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.790+0000 7fb99e400640 1 --2- 172.21.15.50:0/3514258159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb998070530 0x7fb998070940 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.790+0000 7fb99e400640 1 --2- 172.21.15.50:0/3514258159 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb99806e330 0x7fb9980b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.790+0000 7fb99e400640 1 -- 172.21.15.50:0/3514258159 >> 172.21.15.50:0/3514258159 conn(0x7fb9980b3a20 msgr2=0x7fb9980d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:24.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.794+0000 7fb99e400640 1 -- 172.21.15.50:0/3514258159 shutdown_connections 2024-09-21T06:09:24.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.794+0000 7fb99e400640 1 -- 172.21.15.50:0/3514258159 wait complete. 2024-09-21T06:09:24.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.798+0000 7fb99e400640 1 Processor -- start 2024-09-21T06:09:24.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.798+0000 7fb99e400640 1 -- start start 2024-09-21T06:09:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb99e400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb99806e330 0x7fb998146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb99e400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9980b44a0 0x7fb998149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb99e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb998149dc0 0x7fb99815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb99e400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb998158fc0 con 0x7fb9980b44a0 2024-09-21T06:09:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb99e400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb998158e40 con 0x7fb998149dc0 2024-09-21T06:09:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb99e400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb998159140 con 0x7fb99806e330 2024-09-21T06:09:24.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb997e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb998149dc0 0x7fb99815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:24.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb997400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb99806e330 0x7fb998146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:24.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb997e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb998149dc0 0x7fb99815ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:42960/0 (socket says 172.21.15.50:42960) 2024-09-21T06:09:24.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb997e00640 1 -- 172.21.15.50:0/2520371586 learned_addr learned my addr 172.21.15.50:0/2520371586 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:24.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb997400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb99806e330 0x7fb998146d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:47686/0 (socket says 172.21.15.50:47686) 2024-09-21T06:09:24.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb98fe00640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9980b44a0 0x7fb998149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:24.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb98fe00640 1 -- 172.21.15.50:0/2520371586 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb99806e330 msgr2=0x7fb998146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:24.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb98fe00640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb99806e330 0x7fb998146d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb98fe00640 1 -- 172.21.15.50:0/2520371586 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb998149dc0 msgr2=0x7fb99815ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:24.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.802+0000 7fb98fe00640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb998149dc0 0x7fb99815ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.806+0000 7fb98fe00640 1 -- 172.21.15.50:0/2520371586 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb998201ab0 con 0x7fb9980b44a0 2024-09-21T06:09:24.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.806+0000 7fb98fe00640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9980b44a0 0x7fb998149880 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb9800ac000 tx=0x7fb9800a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:24.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.806+0000 7fb98f400640 1 -- 172.21.15.50:0/2520371586 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9800b2040 con 0x7fb9980b44a0 2024-09-21T06:09:24.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.806+0000 7fb98f400640 1 -- 172.21.15.50:0/2520371586 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb980005030 con 0x7fb9980b44a0 2024-09-21T06:09:24.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.806+0000 7fb98f400640 1 -- 172.21.15.50:0/2520371586 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9800044f0 con 0x7fb9980b44a0 2024-09-21T06:09:24.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.806+0000 7fb99e400640 1 -- 172.21.15.50:0/2520371586 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb998200cf0 con 0x7fb9980b44a0 2024-09-21T06:09:24.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.806+0000 7fb99e400640 1 -- 172.21.15.50:0/2520371586 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9982012b0 con 0x7fb9980b44a0 2024-09-21T06:09:24.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.810+0000 7fb99e400640 1 -- 172.21.15.50:0/2520371586 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb99814a3c0 con 0x7fb9980b44a0 2024-09-21T06:09:24.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.810+0000 7fb98f400640 1 -- 172.21.15.50:0/2520371586 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 43) ==== 99529+0+0 (secure 0 0 0) 0x7fb980005480 con 0x7fb9980b44a0 2024-09-21T06:09:24.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.810+0000 7fb98f400640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb974074b80 0x7fb974077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:24.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.810+0000 7fb98f400640 1 -- 172.21.15.50:0/2520371586 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fb98012c530 con 0x7fb9980b44a0 2024-09-21T06:09:24.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.814+0000 7fb997400640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb974074b80 0x7fb974077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:24.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.818+0000 7fb98f400640 1 -- 172.21.15.50:0/2520371586 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb9800bc060 con 0x7fb9980b44a0 2024-09-21T06:09:24.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:24.818+0000 7fb997400640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb974074b80 0x7fb974077090 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb988004630 tx=0x7fb98800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:25.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.198+0000 7fb99e400640 1 -- 172.21.15.50:0/2520371586 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fb99814a650 con 0x7fb974074b80 2024-09-21T06:09:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.302+0000 7fb98f400640 1 -- 172.21.15.50:0/2520371586 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7fb99814a650 con 0x7fb974074b80 2024-09-21T06:09:25.302 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "in-progress", 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_report": { 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: "percentage cloned": "66.667%", 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: "amount cloned": "2.0G/3.0G", 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: "files cloned": "5/4" 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:25.303 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:25.304 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:09:25.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.306+0000 7fb99e400640 1 -- 172.21.15.50:0/2520371586 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb974074b80 msgr2=0x7fb974077090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:25.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.306+0000 7fb99e400640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb974074b80 0x7fb974077090 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb988004630 tx=0x7fb98800b2e0 comp rx=0 tx=0).stop 2024-09-21T06:09:25.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.306+0000 7fb99e400640 1 -- 172.21.15.50:0/2520371586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9980b44a0 msgr2=0x7fb998149880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.306+0000 7fb99e400640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9980b44a0 0x7fb998149880 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb9800ac000 tx=0x7fb9800a57b0 comp rx=0 tx=0).stop 2024-09-21T06:09:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.306+0000 7fb99e400640 1 -- 172.21.15.50:0/2520371586 shutdown_connections 2024-09-21T06:09:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.306+0000 7fb99e400640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb998149dc0 0x7fb99815ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.306+0000 7fb99e400640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb974074b80 0x7fb974077090 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.306+0000 7fb99e400640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9980b44a0 0x7fb998149880 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.306+0000 7fb99e400640 1 --2- 172.21.15.50:0/2520371586 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb99806e330 0x7fb998146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.306+0000 7fb99e400640 1 -- 172.21.15.50:0/2520371586 >> 172.21.15.50:0/2520371586 conn(0x7fb9980b3a20 msgr2=0x7fb9980695f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.306+0000 7fb99e400640 1 -- 172.21.15.50:0/2520371586 shutdown_connections 2024-09-21T06:09:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:25.306+0000 7fb99e400640 1 -- 172.21.15.50:0/2520371586 wait complete. 2024-09-21T06:09:25.339 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 66.667% 2024-09-21T06:09:26.340 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-21T06:09:26.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.506+0000 7ff0dfc00640 1 -- 172.21.15.50:0/4208158222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d81527b0 msgr2=0x7ff0d8074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:26.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.506+0000 7ff0dfc00640 1 --2- 172.21.15.50:0/4208158222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d81527b0 0x7ff0d8074e30 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009900 tx=0x7ff0c401c820 comp rx=0 tx=0).stop 2024-09-21T06:09:26.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.506+0000 7ff0dfc00640 1 -- 172.21.15.50:0/4208158222 shutdown_connections 2024-09-21T06:09:26.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.506+0000 7ff0dfc00640 1 --2- 172.21.15.50:0/4208158222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d81527b0 0x7ff0d8074e30 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:26.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.506+0000 7ff0dfc00640 1 --2- 172.21.15.50:0/4208158222 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff0d80b5c70 0x7ff0d815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:26.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.506+0000 7ff0dfc00640 1 -- 172.21.15.50:0/4208158222 >> 172.21.15.50:0/4208158222 conn(0x7ff0d80b2040 msgr2=0x7ff0d80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:26.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.506+0000 7ff0dfc00640 1 -- 172.21.15.50:0/4208158222 shutdown_connections 2024-09-21T06:09:26.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.506+0000 7ff0dfc00640 1 -- 172.21.15.50:0/4208158222 wait complete. 2024-09-21T06:09:26.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0dfc00640 1 Processor -- start 2024-09-21T06:09:26.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0dfc00640 1 -- start start 2024-09-21T06:09:26.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0dfc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d80b4de0 0x7ff0d81fbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:26.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0dfc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d80b5c70 0x7ff0d81f9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:26.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0dfc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff0d81527b0 0x7ff0d813e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:26.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0dfc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d8168c50 con 0x7ff0d80b4de0 2024-09-21T06:09:26.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0dfc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d8168ad0 con 0x7ff0d80b5c70 2024-09-21T06:09:26.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0dfc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d8168dd0 con 0x7ff0d81527b0 2024-09-21T06:09:26.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0dd200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d80b4de0 0x7ff0d81fbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:26.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0dd200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d80b4de0 0x7ff0d81fbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55386/0 (socket says 172.21.15.50:55386) 2024-09-21T06:09:26.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0dd200640 1 -- 172.21.15.50:0/8525748 learned_addr learned my addr 172.21.15.50:0/8525748 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:26.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0d7e00640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d80b5c70 0x7ff0d81f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:26.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.510+0000 7ff0ddc00640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff0d81527b0 0x7ff0d813e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:26.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0dd200640 1 -- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff0d81527b0 msgr2=0x7ff0d813e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:26.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0dd200640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff0d81527b0 0x7ff0d813e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:26.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0dd200640 1 -- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d80b5c70 msgr2=0x7ff0d81f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:26.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0dd200640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d80b5c70 0x7ff0d81f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:26.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0dd200640 1 -- 172.21.15.50:0/8525748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d81531c0 con 0x7ff0d80b4de0 2024-09-21T06:09:26.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0dd200640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d80b4de0 0x7ff0d81fbbf0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00e7d0 tx=0x7ff0cc00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:26.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0d4c00640 1 -- 172.21.15.50:0/8525748 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc00ccc0 con 0x7ff0d80b4de0 2024-09-21T06:09:26.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0d4c00640 1 -- 172.21.15.50:0/8525748 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0cc010430 con 0x7ff0d80b4de0 2024-09-21T06:09:26.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0dfc00640 1 -- 172.21.15.50:0/8525748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d81fc440 con 0x7ff0d80b4de0 2024-09-21T06:09:26.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0d4c00640 1 -- 172.21.15.50:0/8525748 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc010860 con 0x7ff0d80b4de0 2024-09-21T06:09:26.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0dfc00640 1 -- 172.21.15.50:0/8525748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d81fca00 con 0x7ff0d80b4de0 2024-09-21T06:09:26.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0dfc00640 1 -- 172.21.15.50:0/8525748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d815bb60 con 0x7ff0d80b4de0 2024-09-21T06:09:26.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0d4c00640 1 -- 172.21.15.50:0/8525748 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 43) ==== 99529+0+0 (secure 0 0 0) 0x7ff0cc010a00 con 0x7ff0d80b4de0 2024-09-21T06:09:26.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.514+0000 7ff0d4c00640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff0b8074b80 0x7ff0b8077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:26.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.518+0000 7ff0d4c00640 1 -- 172.21.15.50:0/8525748 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7ff0cc093930 con 0x7ff0d80b4de0 2024-09-21T06:09:26.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.518+0000 7ff0d7e00640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff0b8074b80 0x7ff0b8077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:26.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.518+0000 7ff0d7e00640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff0b8074b80 0x7ff0b8077090 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40af000 tx=0x7ff0c401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:26.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.522+0000 7ff0d4c00640 1 -- 172.21.15.50:0/8525748 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff0cc05fdf0 con 0x7ff0d80b4de0 2024-09-21T06:09:26.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:26.730+0000 7ff0dfc00640 1 -- 172.21.15.50:0/8525748 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7ff0d806b540 con 0x7ff0b8074b80 2024-09-21T06:09:27.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.738+0000 7ff0d4c00640 1 -- 172.21.15.50:0/8525748 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7ff0c40a6a80 con 0x7ff0b8074b80 2024-09-21T06:09:27.739 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:09:27.740 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:09:27.740 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "complete" 2024-09-21T06:09:27.740 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:27.740 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:09:27.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.742+0000 7ff0dfc00640 1 -- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff0b8074b80 msgr2=0x7ff0b8077090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:27.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.742+0000 7ff0dfc00640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff0b8074b80 0x7ff0b8077090 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40af000 tx=0x7ff0c401b060 comp rx=0 tx=0).stop 2024-09-21T06:09:27.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.742+0000 7ff0dfc00640 1 -- 172.21.15.50:0/8525748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d80b4de0 msgr2=0x7ff0d81fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:27.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.742+0000 7ff0dfc00640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d80b4de0 0x7ff0d81fbbf0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00e7d0 tx=0x7ff0cc00ec90 comp rx=0 tx=0).stop 2024-09-21T06:09:27.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.742+0000 7ff0dfc00640 1 -- 172.21.15.50:0/8525748 shutdown_connections 2024-09-21T06:09:27.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.742+0000 7ff0dfc00640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff0d81527b0 0x7ff0d813e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:27.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.742+0000 7ff0dfc00640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d80b5c70 0x7ff0d81f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:27.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.742+0000 7ff0dfc00640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff0b8074b80 0x7ff0b8077090 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:27.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.742+0000 7ff0dfc00640 1 --2- 172.21.15.50:0/8525748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d80b4de0 0x7ff0d81fbbf0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:27.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.742+0000 7ff0dfc00640 1 -- 172.21.15.50:0/8525748 >> 172.21.15.50:0/8525748 conn(0x7ff0d80b2040 msgr2=0x7ff0d80af660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:27.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.742+0000 7ff0dfc00640 1 -- 172.21.15.50:0/8525748 shutdown_connections 2024-09-21T06:09:27.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.742+0000 7ff0dfc00640 1 -- 172.21.15.50:0/8525748 wait complete. 2024-09-21T06:09:27.767 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-21T06:09:27.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.934+0000 7ff62a800640 1 -- 172.21.15.50:0/245171509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6240b3340 msgr2=0x7ff6240b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:27.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.934+0000 7ff62a800640 1 --2- 172.21.15.50:0/245171509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6240b3340 0x7ff6240b3750 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009bc0 tx=0x7ff61c01cba0 comp rx=0 tx=0).stop 2024-09-21T06:09:27.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.934+0000 7ff62a800640 1 -- 172.21.15.50:0/245171509 shutdown_connections 2024-09-21T06:09:27.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.934+0000 7ff62a800640 1 --2- 172.21.15.50:0/245171509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6240b3340 0x7ff6240b3750 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:27.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.934+0000 7ff62a800640 1 --2- 172.21.15.50:0/245171509 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6240749a0 0x7ff624074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:27.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.934+0000 7ff62a800640 1 -- 172.21.15.50:0/245171509 >> 172.21.15.50:0/245171509 conn(0x7ff6240b2040 msgr2=0x7ff6240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:27.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.934+0000 7ff62a800640 1 -- 172.21.15.50:0/245171509 shutdown_connections 2024-09-21T06:09:27.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.934+0000 7ff62a800640 1 -- 172.21.15.50:0/245171509 wait complete. 2024-09-21T06:09:27.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff62a800640 1 Processor -- start 2024-09-21T06:09:27.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff62a800640 1 -- start start 2024-09-21T06:09:27.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff62a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6240749a0 0x7ff62406e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:27.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff62a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6240b3340 0x7ff62415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff62a800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6240cefa0 0x7ff624145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff62a800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff624169dc0 con 0x7ff6240b3340 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff62a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff624169c40 con 0x7ff6240749a0 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff62a800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff624169f40 con 0x7ff6240cefa0 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff61be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6240b3340 0x7ff62415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff61be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6240b3340 0x7ff62415ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52522/0 (socket says 172.21.15.50:52522) 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff61be00640 1 -- 172.21.15.50:0/2733662671 learned_addr learned my addr 172.21.15.50:0/2733662671 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff623e00640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6240cefa0 0x7ff624145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff61be00640 1 -- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6240cefa0 msgr2=0x7ff624145ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff623400640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6240749a0 0x7ff62406e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff61be00640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6240cefa0 0x7ff624145ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff61be00640 1 -- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6240749a0 msgr2=0x7ff62406e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:27.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff61be00640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6240749a0 0x7ff62406e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:27.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff61be00640 1 -- 172.21.15.50:0/2733662671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6241432a0 con 0x7ff6240b3340 2024-09-21T06:09:27.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff623e00640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6240cefa0 0x7ff624145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:09:27.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.938+0000 7ff623400640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6240749a0 0x7ff62406e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:09:27.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.942+0000 7ff61be00640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6240b3340 0x7ff62415ba60 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c3f0 tx=0x7ff60c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:27.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.942+0000 7ff61b400640 1 -- 172.21.15.50:0/2733662671 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c017070 con 0x7ff6240b3340 2024-09-21T06:09:27.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.942+0000 7ff61b400640 1 -- 172.21.15.50:0/2733662671 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff60c0053c0 con 0x7ff6240b3340 2024-09-21T06:09:27.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.942+0000 7ff61b400640 1 -- 172.21.15.50:0/2733662671 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c003860 con 0x7ff6240b3340 2024-09-21T06:09:27.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.942+0000 7ff62a800640 1 -- 172.21.15.50:0/2733662671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6241437a0 con 0x7ff6240b3340 2024-09-21T06:09:27.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.942+0000 7ff62a800640 1 -- 172.21.15.50:0/2733662671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6241454a0 con 0x7ff6240b3340 2024-09-21T06:09:27.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.942+0000 7ff62a800640 1 -- 172.21.15.50:0/2733662671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff624143510 con 0x7ff6240b3340 2024-09-21T06:09:27.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.942+0000 7ff61b400640 1 -- 172.21.15.50:0/2733662671 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 43) ==== 99529+0+0 (secure 0 0 0) 0x7ff60c01a070 con 0x7ff6240b3340 2024-09-21T06:09:27.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.946+0000 7ff61b400640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff600074b50 0x7ff600077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:27.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.946+0000 7ff61b400640 1 -- 172.21.15.50:0/2733662671 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7ff60c092410 con 0x7ff6240b3340 2024-09-21T06:09:27.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.946+0000 7ff623400640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff600074b50 0x7ff600077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:27.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.946+0000 7ff623400640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff600074b50 0x7ff600077060 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff614009ac0 tx=0x7ff614009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:27.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:27.950+0000 7ff61b400640 1 -- 172.21.15.50:0/2733662671 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff60c05ee90 con 0x7ff6240b3340 2024-09-21T06:09:28.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.166+0000 7ff62a800640 1 -- 172.21.15.50:0/2733662671 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7ff62406b540 con 0x7ff600074b50 2024-09-21T06:09:28.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.178+0000 7ff61b400640 1 -- 172.21.15.50:0/2733662671 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7ff614010a90 con 0x7ff600074b50 2024-09-21T06:09:28.179 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:09:28.179 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:09:28.179 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "complete" 2024-09-21T06:09:28.179 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:28.180 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:09:28.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.182+0000 7ff62a800640 1 -- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff600074b50 msgr2=0x7ff600077060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.182+0000 7ff62a800640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff600074b50 0x7ff600077060 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff614009ac0 tx=0x7ff614009260 comp rx=0 tx=0).stop 2024-09-21T06:09:28.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.182+0000 7ff62a800640 1 -- 172.21.15.50:0/2733662671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6240b3340 msgr2=0x7ff62415ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.182+0000 7ff62a800640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6240b3340 0x7ff62415ba60 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c3f0 tx=0x7ff60c00c8b0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.182+0000 7ff62a800640 1 -- 172.21.15.50:0/2733662671 shutdown_connections 2024-09-21T06:09:28.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.182+0000 7ff62a800640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff6240cefa0 0x7ff624145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.182+0000 7ff62a800640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff600074b50 0x7ff600077060 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.182+0000 7ff62a800640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6240b3340 0x7ff62415ba60 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.182+0000 7ff62a800640 1 --2- 172.21.15.50:0/2733662671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6240749a0 0x7ff62406e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.182+0000 7ff62a800640 1 -- 172.21.15.50:0/2733662671 >> 172.21.15.50:0/2733662671 conn(0x7ff6240b2040 msgr2=0x7ff6240cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:28.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.182+0000 7ff62a800640 1 -- 172.21.15.50:0/2733662671 shutdown_connections 2024-09-21T06:09:28.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.182+0000 7ff62a800640 1 -- 172.21.15.50:0/2733662671 wait complete. 2024-09-21T06:09:28.202 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-21T06:09:28.202 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:09:28.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.362+0000 7f0897e00640 1 -- 172.21.15.50:0/4024059680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08981536f0 msgr2=0x7f089815d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.362+0000 7f0897e00640 1 --2- 172.21.15.50:0/4024059680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08981536f0 0x7f089815d480 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0888009900 tx=0x7f088801c850 comp rx=0 tx=0).stop 2024-09-21T06:09:28.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.362+0000 7f0897e00640 1 -- 172.21.15.50:0/4024059680 shutdown_connections 2024-09-21T06:09:28.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.362+0000 7f0897e00640 1 --2- 172.21.15.50:0/4024059680 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0898074950 0x7f08980b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.362+0000 7f0897e00640 1 --2- 172.21.15.50:0/4024059680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08981536f0 0x7f089815d480 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.362+0000 7f0897e00640 1 -- 172.21.15.50:0/4024059680 >> 172.21.15.50:0/4024059680 conn(0x7f08980b4560 msgr2=0x7f08980d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:28.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.362+0000 7f0897e00640 1 -- 172.21.15.50:0/4024059680 shutdown_connections 2024-09-21T06:09:28.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.362+0000 7f0897e00640 1 -- 172.21.15.50:0/4024059680 wait complete. 2024-09-21T06:09:28.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.362+0000 7f0897e00640 1 Processor -- start 2024-09-21T06:09:28.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.362+0000 7f0897e00640 1 -- start start 2024-09-21T06:09:28.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08981536f0 0x7f089806cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0898160060 0x7f089814aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f089815f090 0x7f089815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0898169e10 con 0x7f089815f090 2024-09-21T06:09:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0898169c90 con 0x7f08981536f0 2024-09-21T06:09:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0898169f90 con 0x7f0898160060 2024-09-21T06:09:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f089815f090 0x7f089815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f089815f090 0x7f089815b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52546/0 (socket says 172.21.15.50:52546) 2024-09-21T06:09:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897400640 1 -- 172.21.15.50:0/2085589832 learned_addr learned my addr 172.21.15.50:0/2085589832 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f088e000640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0898160060 0x7f089814aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0896a00640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08981536f0 0x7f089806cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897400640 1 -- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0898160060 msgr2=0x7f089814aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897400640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0898160060 0x7f089814aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897400640 1 -- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08981536f0 msgr2=0x7f089806cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897400640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08981536f0 0x7f089806cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897400640 1 -- 172.21.15.50:0/2085589832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089814b800 con 0x7f089815f090 2024-09-21T06:09:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897400640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f089815f090 0x7f089815b4b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f089000ea80 tx=0x7f089000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f088fe00640 1 -- 172.21.15.50:0/2085589832 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089000cea0 con 0x7f089815f090 2024-09-21T06:09:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f088fe00640 1 -- 172.21.15.50:0/2085589832 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0890004b30 con 0x7f089815f090 2024-09-21T06:09:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897e00640 1 -- 172.21.15.50:0/2085589832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0898152fe0 con 0x7f089815f090 2024-09-21T06:09:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f088fe00640 1 -- 172.21.15.50:0/2085589832 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890005020 con 0x7f089815f090 2024-09-21T06:09:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897e00640 1 -- 172.21.15.50:0/2085589832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089814b3e0 con 0x7f089815f090 2024-09-21T06:09:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.366+0000 7f0897e00640 1 -- 172.21.15.50:0/2085589832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08980b5260 con 0x7f089815f090 2024-09-21T06:09:28.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.370+0000 7f088fe00640 1 -- 172.21.15.50:0/2085589832 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 43) ==== 99529+0+0 (secure 0 0 0) 0x7f08900051c0 con 0x7f089815f090 2024-09-21T06:09:28.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.370+0000 7f088fe00640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0874074b80 0x7f0874077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:28.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.370+0000 7f088fe00640 1 -- 172.21.15.50:0/2085589832 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f0890093bb0 con 0x7f089815f090 2024-09-21T06:09:28.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.370+0000 7f0896a00640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0874074b80 0x7f0874077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:28.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.370+0000 7f0896a00640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0874074b80 0x7f0874077090 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f08880ae000 tx=0x7f088801b090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:28.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.374+0000 7f088fe00640 1 -- 172.21.15.50:0/2085589832 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f089005fff0 con 0x7f089815f090 2024-09-21T06:09:28.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.574+0000 7f0897e00640 1 -- 172.21.15.50:0/2085589832 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f089806b540 con 0x7f0874074b80 2024-09-21T06:09:28.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f088fe00640 1 -- 172.21.15.50:0/2085589832 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f0888004040 con 0x7f0874074b80 2024-09-21T06:09:28.577 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:09:28.578 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:28.578 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:09:28.578 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:28.578 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:09:28.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f0897e00640 1 -- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0874074b80 msgr2=0x7f0874077090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f0897e00640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0874074b80 0x7f0874077090 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f08880ae000 tx=0x7f088801b090 comp rx=0 tx=0).stop 2024-09-21T06:09:28.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f0897e00640 1 -- 172.21.15.50:0/2085589832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f089815f090 msgr2=0x7f089815b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f0897e00640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f089815f090 0x7f089815b4b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f089000ea80 tx=0x7f089000ef40 comp rx=0 tx=0).stop 2024-09-21T06:09:28.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f0897e00640 1 -- 172.21.15.50:0/2085589832 shutdown_connections 2024-09-21T06:09:28.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f0897e00640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0874074b80 0x7f0874077090 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f0897e00640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f089815f090 0x7f089815b4b0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f0897e00640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0898160060 0x7f089814aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f0897e00640 1 --2- 172.21.15.50:0/2085589832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08981536f0 0x7f089806cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f0897e00640 1 -- 172.21.15.50:0/2085589832 >> 172.21.15.50:0/2085589832 conn(0x7f08980b4560 msgr2=0x7f0898071ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:28.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f0897e00640 1 -- 172.21.15.50:0/2085589832 shutdown_connections 2024-09-21T06:09:28.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.578+0000 7f0897e00640 1 -- 172.21.15.50:0/2085589832 wait complete. 2024-09-21T06:09:28.600 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-21T06:09:28.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.762+0000 7fef00e00640 1 -- 172.21.15.50:0/2757823387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b5c70 msgr2=0x7feefc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.762+0000 7fef00e00640 1 --2- 172.21.15.50:0/2757823387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b5c70 0x7feefc15bb60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feefc074b20 tx=0x7feeec01ca00 comp rx=0 tx=0).stop 2024-09-21T06:09:28.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.762+0000 7fef00e00640 1 -- 172.21.15.50:0/2757823387 shutdown_connections 2024-09-21T06:09:28.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.762+0000 7fef00e00640 1 --2- 172.21.15.50:0/2757823387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b5c70 0x7feefc15bb60 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.762+0000 7fef00e00640 1 -- 172.21.15.50:0/2757823387 >> 172.21.15.50:0/2757823387 conn(0x7feefc0b2040 msgr2=0x7feefc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:28.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.762+0000 7fef00e00640 1 -- 172.21.15.50:0/2757823387 shutdown_connections 2024-09-21T06:09:28.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.762+0000 7fef00e00640 1 -- 172.21.15.50:0/2757823387 wait complete. 2024-09-21T06:09:28.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7fef00e00640 1 Processor -- start 2024-09-21T06:09:28.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7fef00e00640 1 -- start start 2024-09-21T06:09:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7fef00e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feefc0b4de0 0x7feefc140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7fef00e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b5c70 0x7feefc153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7fef00e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feefc1527b0 0x7feefc1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7fef00e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feefc169b30 con 0x7feefc0b4de0 2024-09-21T06:09:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7fef00e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1699b0 con 0x7feefc0b5c70 2024-09-21T06:09:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7fef00e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7feefc169cb0 con 0x7feefc1527b0 2024-09-21T06:09:28.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7feef9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feefc0b4de0 0x7feefc140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:28.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7feef9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feefc0b4de0 0x7feefc140bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52562/0 (socket says 172.21.15.50:52562) 2024-09-21T06:09:28.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7feef9e00640 1 -- 172.21.15.50:0/1340710216 learned_addr learned my addr 172.21.15.50:0/1340710216 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:28.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7feef1400640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b5c70 0x7feefc153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:28.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7feefa800640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feefc1527b0 0x7feefc1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:28.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7feef9e00640 1 -- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feefc1527b0 msgr2=0x7feefc1541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7feef9e00640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feefc1527b0 0x7feefc1541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7feef9e00640 1 -- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b5c70 msgr2=0x7feefc153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7feef9e00640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b5c70 0x7feefc153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.766+0000 7feef9e00640 1 -- 172.21.15.50:0/1340710216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc141100 con 0x7feefc0b4de0 2024-09-21T06:09:28.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.770+0000 7feef9e00640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feefc0b4de0 0x7feefc140bc0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7feeec009b10 tx=0x7feeec009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:28.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.770+0000 7feef2a00640 1 -- 172.21.15.50:0/1340710216 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec0bb070 con 0x7feefc0b4de0 2024-09-21T06:09:28.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.770+0000 7feef2a00640 1 -- 172.21.15.50:0/1340710216 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeec004e00 con 0x7feefc0b4de0 2024-09-21T06:09:28.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.770+0000 7feef2a00640 1 -- 172.21.15.50:0/1340710216 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec0b2040 con 0x7feefc0b4de0 2024-09-21T06:09:28.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.770+0000 7fef00e00640 1 -- 172.21.15.50:0/1340710216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc141b20 con 0x7feefc0b4de0 2024-09-21T06:09:28.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.770+0000 7fef00e00640 1 -- 172.21.15.50:0/1340710216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc1420e0 con 0x7feefc0b4de0 2024-09-21T06:09:28.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.770+0000 7fef00e00640 1 -- 172.21.15.50:0/1340710216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feefc15bb60 con 0x7feefc0b4de0 2024-09-21T06:09:28.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.770+0000 7feef2a00640 1 -- 172.21.15.50:0/1340710216 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 43) ==== 99529+0+0 (secure 0 0 0) 0x7feeec0be070 con 0x7feefc0b4de0 2024-09-21T06:09:28.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.770+0000 7feef2a00640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feed8074b50 0x7feed8077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:28.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.770+0000 7feef2a00640 1 -- 172.21.15.50:0/1340710216 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7feeec12ca90 con 0x7feefc0b4de0 2024-09-21T06:09:28.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.774+0000 7feef1400640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feed8074b50 0x7feed8077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:28.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.774+0000 7feef1400640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feed8074b50 0x7feed8077060 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feee4001130 tx=0x7feee4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:28.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.778+0000 7feef2a00640 1 -- 172.21.15.50:0/1340710216 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7feeec0166b0 con 0x7feefc0b4de0 2024-09-21T06:09:28.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.974+0000 7fef00e00640 1 -- 172.21.15.50:0/1340710216 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feefc141660 con 0x7feed8074b50 2024-09-21T06:09:28.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.982+0000 7feef2a00640 1 -- 172.21.15.50:0/1340710216 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7feee4001c90 con 0x7feed8074b50 2024-09-21T06:09:28.982 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:28.982 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:09:28.982 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:28.982 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sv1" 2024-09-21T06:09:28.982 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:28.982 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:28.982 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ss1clone1" 2024-09-21T06:09:28.982 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:28.982 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:09:28.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.986+0000 7fef00e00640 1 -- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feed8074b50 msgr2=0x7feed8077060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.986+0000 7fef00e00640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feed8074b50 0x7feed8077060 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feee4001130 tx=0x7feee4009040 comp rx=0 tx=0).stop 2024-09-21T06:09:28.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.986+0000 7fef00e00640 1 -- 172.21.15.50:0/1340710216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feefc0b4de0 msgr2=0x7feefc140bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.986+0000 7fef00e00640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feefc0b4de0 0x7feefc140bc0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7feeec009b10 tx=0x7feeec009330 comp rx=0 tx=0).stop 2024-09-21T06:09:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.986+0000 7fef00e00640 1 -- 172.21.15.50:0/1340710216 shutdown_connections 2024-09-21T06:09:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.986+0000 7fef00e00640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feefc1527b0 0x7feefc1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.986+0000 7fef00e00640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b5c70 0x7feefc153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.986+0000 7fef00e00640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feed8074b50 0x7feed8077060 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.986+0000 7fef00e00640 1 --2- 172.21.15.50:0/1340710216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feefc0b4de0 0x7feefc140bc0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.986+0000 7fef00e00640 1 -- 172.21.15.50:0/1340710216 >> 172.21.15.50:0/1340710216 conn(0x7feefc0b2040 msgr2=0x7feefc1518f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.986+0000 7fef00e00640 1 -- 172.21.15.50:0/1340710216 shutdown_connections 2024-09-21T06:09:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:28.986+0000 7fef00e00640 1 -- 172.21.15.50:0/1340710216 wait complete. 2024-09-21T06:09:29.005 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-21T06:09:29.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.170+0000 7fc3cca00640 1 -- 172.21.15.50:0/2411565488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3ac003a60 msgr2=0x7fc3ac003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:29.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.170+0000 7fc3cca00640 1 --2- 172.21.15.50:0/2411565488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3ac003a60 0x7fc3ac003ed0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800c610 tx=0x7fc3b801cf60 comp rx=0 tx=0).stop 2024-09-21T06:09:29.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.170+0000 7fc3cca00640 1 -- 172.21.15.50:0/2411565488 shutdown_connections 2024-09-21T06:09:29.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.170+0000 7fc3cca00640 1 --2- 172.21.15.50:0/2411565488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3ac003a60 0x7fc3ac003ed0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.170+0000 7fc3cca00640 1 -- 172.21.15.50:0/2411565488 >> 172.21.15.50:0/2411565488 conn(0x7fc3c80708c0 msgr2=0x7fc3c8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:29.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.170+0000 7fc3cca00640 1 -- 172.21.15.50:0/2411565488 shutdown_connections 2024-09-21T06:09:29.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.170+0000 7fc3cca00640 1 -- 172.21.15.50:0/2411565488 wait complete. 2024-09-21T06:09:29.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.170+0000 7fc3cca00640 1 Processor -- start 2024-09-21T06:09:29.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.170+0000 7fc3cca00640 1 -- start start 2024-09-21T06:09:29.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3cca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3ac003a60 0x7fc3c814b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:29.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3cca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3c8154d30 0x7fc3c81501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:29.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3cca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c8155140 0x7fc3c81506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:29.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3cca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c8161270 con 0x7fc3c8155140 2024-09-21T06:09:29.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3cca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c815d9b0 con 0x7fc3c8154d30 2024-09-21T06:09:29.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3cca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c8071c00 con 0x7fc3ac003a60 2024-09-21T06:09:29.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3c6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c8155140 0x7fc3c81506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:29.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3c5e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3ac003a60 0x7fc3c814b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:29.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3c6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c8155140 0x7fc3c81506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52582/0 (socket says 172.21.15.50:52582) 2024-09-21T06:09:29.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3c6800640 1 -- 172.21.15.50:0/45162889 learned_addr learned my addr 172.21.15.50:0/45162889 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:29.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3c5e00640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3ac003a60 0x7fc3c814b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:41814/0 (socket says 172.21.15.50:41814) 2024-09-21T06:09:29.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3bd400640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3c8154d30 0x7fc3c81501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:29.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3c6800640 1 -- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3ac003a60 msgr2=0x7fc3c814b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:29.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3c6800640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3ac003a60 0x7fc3c814b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3c6800640 1 -- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3c8154d30 msgr2=0x7fc3c81501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:29.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3c6800640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3c8154d30 0x7fc3c81501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3c6800640 1 -- 172.21.15.50:0/45162889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c81fa2b0 con 0x7fc3c8155140 2024-09-21T06:09:29.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3c6800640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c8155140 0x7fc3c81506f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80cfb10 tx=0x7fc3c00019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:29.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3bea00640 1 -- 172.21.15.50:0/45162889 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0017070 con 0x7fc3c8155140 2024-09-21T06:09:29.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3bea00640 1 -- 172.21.15.50:0/45162889 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3c00108c0 con 0x7fc3c8155140 2024-09-21T06:09:29.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3bea00640 1 -- 172.21.15.50:0/45162889 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c00020d0 con 0x7fc3c8155140 2024-09-21T06:09:29.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3cca00640 1 -- 172.21.15.50:0/45162889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c81f9fd0 con 0x7fc3c8155140 2024-09-21T06:09:29.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3cca00640 1 -- 172.21.15.50:0/45162889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c8154810 con 0x7fc3c8155140 2024-09-21T06:09:29.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.174+0000 7fc3cca00640 1 -- 172.21.15.50:0/45162889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c80720f0 con 0x7fc3c8155140 2024-09-21T06:09:29.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.178+0000 7fc3bea00640 1 -- 172.21.15.50:0/45162889 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 43) ==== 99529+0+0 (secure 0 0 0) 0x7fc3c001a070 con 0x7fc3c8155140 2024-09-21T06:09:29.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.178+0000 7fc3bea00640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc3a4074b80 0x7fc3a4077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:29.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.178+0000 7fc3bea00640 1 -- 172.21.15.50:0/45162889 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fc3c0092ca0 con 0x7fc3c8155140 2024-09-21T06:09:29.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.178+0000 7fc3c5e00640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc3a4074b80 0x7fc3a4077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:29.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.178+0000 7fc3c5e00640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc3a4074b80 0x7fc3a4077090 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80aa000 tx=0x7fc3b8019da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:29.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.182+0000 7fc3bea00640 1 -- 172.21.15.50:0/45162889 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc3c005f160 con 0x7fc3c8155140 2024-09-21T06:09:29.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.394+0000 7fc3cca00640 1 -- 172.21.15.50:0/45162889 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3c8150c60 con 0x7fc3a4074b80 2024-09-21T06:09:29.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3bea00640 1 -- 172.21.15.50:0/45162889 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc3c8150c60 con 0x7fc3a4074b80 2024-09-21T06:09:29.409 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:29.409 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:09:29.410 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:29.410 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ss1" 2024-09-21T06:09:29.410 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:29.410 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:09:29.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3cca00640 1 -- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc3a4074b80 msgr2=0x7fc3a4077090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:29.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3cca00640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc3a4074b80 0x7fc3a4077090 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80aa000 tx=0x7fc3b8019da0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3cca00640 1 -- 172.21.15.50:0/45162889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c8155140 msgr2=0x7fc3c81506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:29.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3cca00640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c8155140 0x7fc3c81506f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80cfb10 tx=0x7fc3c00019c0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3cca00640 1 -- 172.21.15.50:0/45162889 shutdown_connections 2024-09-21T06:09:29.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3cca00640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc3a4074b80 0x7fc3a4077090 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3cca00640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c8155140 0x7fc3c81506f0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3cca00640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3c8154d30 0x7fc3c81501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3cca00640 1 --2- 172.21.15.50:0/45162889 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3ac003a60 0x7fc3c814b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3cca00640 1 -- 172.21.15.50:0/45162889 >> 172.21.15.50:0/45162889 conn(0x7fc3c80708c0 msgr2=0x7fc3c80b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:29.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3cca00640 1 -- 172.21.15.50:0/45162889 shutdown_connections 2024-09-21T06:09:29.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.410+0000 7fc3cca00640 1 -- 172.21.15.50:0/45162889 wait complete. 2024-09-21T06:09:29.432 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-21T06:09:29.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.598+0000 7fe006800640 1 -- 172.21.15.50:0/1752120834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0000b5c70 msgr2=0x7fe00015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:29.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.598+0000 7fe006800640 1 --2- 172.21.15.50:0/1752120834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0000b5c70 0x7fe00015bb60 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe000070710 tx=0x7fdff001c800 comp rx=0 tx=0).stop 2024-09-21T06:09:29.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.598+0000 7fe006800640 1 -- 172.21.15.50:0/1752120834 shutdown_connections 2024-09-21T06:09:29.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.598+0000 7fe006800640 1 --2- 172.21.15.50:0/1752120834 >> v2:172.21.15.88:3300/0 conn(0x7fe0001527b0 0x7fe000074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.598+0000 7fe006800640 1 --2- 172.21.15.50:0/1752120834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0000b5c70 0x7fe00015bb60 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.598+0000 7fe006800640 1 -- 172.21.15.50:0/1752120834 >> 172.21.15.50:0/1752120834 conn(0x7fe0000b2040 msgr2=0x7fe0000b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:29.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.598+0000 7fe006800640 1 -- 172.21.15.50:0/1752120834 shutdown_connections 2024-09-21T06:09:29.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.598+0000 7fe006800640 1 -- 172.21.15.50:0/1752120834 wait complete. 2024-09-21T06:09:29.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fe006800640 1 Processor -- start 2024-09-21T06:09:29.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fe006800640 1 -- start start 2024-09-21T06:09:29.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fe006800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe0000b4de0 0x7fe000158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:29.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fe006800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0000b5c70 0x7fe0001f9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:29.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fe006800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0001527b0 0x7fe0001f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:29.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fe006800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe000168b20 con 0x7fe0001527b0 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fe006800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe0001689a0 con 0x7fe0000b5c70 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fe006800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe000168ca0 con 0x7fe0000b4de0 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdfffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0001527b0 0x7fe0001f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdfffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0001527b0 0x7fe0001f9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52598/0 (socket says 172.21.15.50:52598) 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdfffe00640 1 -- 172.21.15.50:0/3839678622 learned_addr learned my addr 172.21.15.50:0/3839678622 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdfff400640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe0000b4de0 0x7fe000158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdff7e00640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0000b5c70 0x7fe0001f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdfffe00640 1 -- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe0000b4de0 msgr2=0x7fe000158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdfffe00640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe0000b4de0 0x7fe000158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdfffe00640 1 -- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0000b5c70 msgr2=0x7fe0001f9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdfffe00640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0000b5c70 0x7fe0001f9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdfffe00640 1 -- 172.21.15.50:0/3839678622 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe000156eb0 con 0x7fe0001527b0 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdff7e00640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0000b5c70 0x7fe0001f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:29.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdfffe00640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0001527b0 0x7fe0001f9c70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fdff800c6e0 tx=0x7fdff800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:29.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdff7400640 1 -- 172.21.15.50:0/3839678622 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8003b80 con 0x7fe0001527b0 2024-09-21T06:09:29.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdff7400640 1 -- 172.21.15.50:0/3839678622 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdff8003d20 con 0x7fe0001527b0 2024-09-21T06:09:29.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fe006800640 1 -- 172.21.15.50:0/3839678622 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe000157190 con 0x7fe0001527b0 2024-09-21T06:09:29.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fdff7400640 1 -- 172.21.15.50:0/3839678622 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8005330 con 0x7fe0001527b0 2024-09-21T06:09:29.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fe006800640 1 -- 172.21.15.50:0/3839678622 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe000157750 con 0x7fe0001527b0 2024-09-21T06:09:29.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.602+0000 7fe006800640 1 -- 172.21.15.50:0/3839678622 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe00015bb60 con 0x7fe0001527b0 2024-09-21T06:09:29.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.606+0000 7fdff7400640 1 -- 172.21.15.50:0/3839678622 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 43) ==== 99529+0+0 (secure 0 0 0) 0x7fdff80054d0 con 0x7fe0001527b0 2024-09-21T06:09:29.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.606+0000 7fdff7400640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdfdc074b50 0x7fdfdc077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:29.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.606+0000 7fdff7400640 1 -- 172.21.15.50:0/3839678622 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fdff8092d10 con 0x7fe0001527b0 2024-09-21T06:09:29.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.606+0000 7fdfff400640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdfdc074b50 0x7fdfdc077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:29.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.606+0000 7fdfff400640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdfdc074b50 0x7fdfdc077060 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdff00af000 tx=0x7fdff00a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:29.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.610+0000 7fdff7400640 1 -- 172.21.15.50:0/3839678622 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdff805f830 con 0x7fe0001527b0 2024-09-21T06:09:29.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.818+0000 7fe006800640 1 -- 172.21.15.50:0/3839678622 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe00006b540 con 0x7fdfdc074b50 2024-09-21T06:09:29.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fdff7400640 1 -- 172.21.15.50:0/3839678622 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdff00b3040 con 0x7fdfdc074b50 2024-09-21T06:09:29.850 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:29.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fe006800640 1 -- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdfdc074b50 msgr2=0x7fdfdc077060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:29.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fe006800640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdfdc074b50 0x7fdfdc077060 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdff00af000 tx=0x7fdff00a7040 comp rx=0 tx=0).stop 2024-09-21T06:09:29.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fe006800640 1 -- 172.21.15.50:0/3839678622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0001527b0 msgr2=0x7fe0001f9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:29.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fe006800640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0001527b0 0x7fe0001f9c70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fdff800c6e0 tx=0x7fdff800cba0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fe006800640 1 -- 172.21.15.50:0/3839678622 shutdown_connections 2024-09-21T06:09:29.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fe006800640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdfdc074b50 0x7fdfdc077060 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fe006800640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0001527b0 0x7fe0001f9c70 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fe006800640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0000b5c70 0x7fe0001f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fe006800640 1 --2- 172.21.15.50:0/3839678622 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe0000b4de0 0x7fe000158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fe006800640 1 -- 172.21.15.50:0/3839678622 >> 172.21.15.50:0/3839678622 conn(0x7fe0000b2040 msgr2=0x7fe0000ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:29.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fe006800640 1 -- 172.21.15.50:0/3839678622 shutdown_connections 2024-09-21T06:09:29.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:29.850+0000 7fe006800640 1 -- 172.21.15.50:0/3839678622 wait complete. 2024-09-21T06:09:29.873 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-21T06:09:30.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.034+0000 7fc754800640 1 -- 172.21.15.50:0/2156050689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc74c1527b0 msgr2=0x7fc74c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:30.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.034+0000 7fc754800640 1 --2- 172.21.15.50:0/2156050689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc74c1527b0 0x7fc74c15bb60 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0cca90 tx=0x7fc73c01c640 comp rx=0 tx=0).stop 2024-09-21T06:09:30.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.034+0000 7fc754800640 1 -- 172.21.15.50:0/2156050689 shutdown_connections 2024-09-21T06:09:30.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.034+0000 7fc754800640 1 --2- 172.21.15.50:0/2156050689 >> v2:172.21.15.88:3300/0 conn(0x7fc74c0b4de0 0x7fc74c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.034+0000 7fc754800640 1 --2- 172.21.15.50:0/2156050689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc74c1527b0 0x7fc74c15bb60 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.034+0000 7fc754800640 1 -- 172.21.15.50:0/2156050689 >> 172.21.15.50:0/2156050689 conn(0x7fc74c0b2040 msgr2=0x7fc74c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:30.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.034+0000 7fc754800640 1 -- 172.21.15.50:0/2156050689 shutdown_connections 2024-09-21T06:09:30.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.034+0000 7fc754800640 1 -- 172.21.15.50:0/2156050689 wait complete. 2024-09-21T06:09:30.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc754800640 1 Processor -- start 2024-09-21T06:09:30.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc754800640 1 -- start start 2024-09-21T06:09:30.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc754800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc74c0b4de0 0x7fc74c14e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:30.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc754800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc74c0b5c70 0x7fc74c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc754800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc74c1527b0 0x7fc74c0ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc754800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1689c0 con 0x7fc74c1527b0 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc754800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c168840 con 0x7fc74c0b5c70 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc754800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc74c168b40 con 0x7fc74c0b4de0 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc752800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc74c1527b0 0x7fc74c0ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc752800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc74c1527b0 0x7fc74c0ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52612/0 (socket says 172.21.15.50:52612) 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc752800640 1 -- 172.21.15.50:0/2044820769 learned_addr learned my addr 172.21.15.50:0/2044820769 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc751e00640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc74c0b4de0 0x7fc74c14e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc751400640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc74c0b5c70 0x7fc74c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc752800640 1 -- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc74c0b4de0 msgr2=0x7fc74c14e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc752800640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc74c0b4de0 0x7fc74c14e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc752800640 1 -- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc74c0b5c70 msgr2=0x7fc74c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc752800640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc74c0b5c70 0x7fc74c0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc752800640 1 -- 172.21.15.50:0/2044820769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c0cca70 con 0x7fc74c1527b0 2024-09-21T06:09:30.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc751400640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc74c0b5c70 0x7fc74c0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:30.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc751e00640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc74c0b4de0 0x7fc74c14e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:30.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc752800640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc74c1527b0 0x7fc74c0ce010 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fc74800b9e0 tx=0x7fc74800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:30.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc746000640 1 -- 172.21.15.50:0/2044820769 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748019070 con 0x7fc74c1527b0 2024-09-21T06:09:30.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc746000640 1 -- 172.21.15.50:0/2044820769 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc74800cbb0 con 0x7fc74c1527b0 2024-09-21T06:09:30.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc746000640 1 -- 172.21.15.50:0/2044820769 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748014440 con 0x7fc74c1527b0 2024-09-21T06:09:30.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc754800640 1 -- 172.21.15.50:0/2044820769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74c0ccf70 con 0x7fc74c1527b0 2024-09-21T06:09:30.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.038+0000 7fc754800640 1 -- 172.21.15.50:0/2044820769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc74c0cd530 con 0x7fc74c1527b0 2024-09-21T06:09:30.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.042+0000 7fc754800640 1 -- 172.21.15.50:0/2044820769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc74c0cccc0 con 0x7fc74c1527b0 2024-09-21T06:09:30.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.042+0000 7fc746000640 1 -- 172.21.15.50:0/2044820769 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 43) ==== 99529+0+0 (secure 0 0 0) 0x7fc74801c070 con 0x7fc74c1527b0 2024-09-21T06:09:30.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.042+0000 7fc746000640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc730074b80 0x7fc730077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:30.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.042+0000 7fc746000640 1 -- 172.21.15.50:0/2044820769 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fc748092ce0 con 0x7fc74c1527b0 2024-09-21T06:09:30.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.042+0000 7fc751e00640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc730074b80 0x7fc730077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:30.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.042+0000 7fc751e00640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc730074b80 0x7fc730077090 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc740009890 tx=0x7fc740008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:30.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.046+0000 7fc746000640 1 -- 172.21.15.50:0/2044820769 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc74805f7f0 con 0x7fc74c1527b0 2024-09-21T06:09:30.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.250+0000 7fc754800640 1 -- 172.21.15.50:0/2044820769 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fc74c0b51f0 con 0x7fc730074b80 2024-09-21T06:09:30.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc746000640 1 -- 172.21.15.50:0/2044820769 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc74c0b51f0 con 0x7fc730074b80 2024-09-21T06:09:30.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc754800640 1 -- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc730074b80 msgr2=0x7fc730077090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc754800640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc730074b80 0x7fc730077090 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc740009890 tx=0x7fc740008040 comp rx=0 tx=0).stop 2024-09-21T06:09:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc754800640 1 -- 172.21.15.50:0/2044820769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc74c1527b0 msgr2=0x7fc74c0ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc754800640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc74c1527b0 0x7fc74c0ce010 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fc74800b9e0 tx=0x7fc74800bea0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc754800640 1 -- 172.21.15.50:0/2044820769 shutdown_connections 2024-09-21T06:09:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc754800640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc730074b80 0x7fc730077090 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc754800640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc74c1527b0 0x7fc74c0ce010 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc754800640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc74c0b5c70 0x7fc74c0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc754800640 1 --2- 172.21.15.50:0/2044820769 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc74c0b4de0 0x7fc74c14e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc754800640 1 -- 172.21.15.50:0/2044820769 >> 172.21.15.50:0/2044820769 conn(0x7fc74c0b2040 msgr2=0x7fc74c159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc754800640 1 -- 172.21.15.50:0/2044820769 shutdown_connections 2024-09-21T06:09:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.274+0000 7fc754800640 1 -- 172.21.15.50:0/2044820769 wait complete. 2024-09-21T06:09:30.287 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:30.287+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:30.287 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:30.287+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:30.287 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:30.287+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:30.287 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:30.287+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:30.287 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:30.287+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:30.298 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-21T06:09:30.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.466+0000 7fae4a000640 1 -- 172.21.15.50:0/2210332731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae28003c40 msgr2=0x7fae28004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:30.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.466+0000 7fae4a000640 1 --2- 172.21.15.50:0/2210332731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae28003c40 0x7fae28004090 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fae34001860 tx=0x7fae3401c880 comp rx=0 tx=0).stop 2024-09-21T06:09:30.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.466+0000 7fae4a000640 1 -- 172.21.15.50:0/2210332731 shutdown_connections 2024-09-21T06:09:30.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.466+0000 7fae4a000640 1 --2- 172.21.15.50:0/2210332731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae28003c40 0x7fae28004090 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.466+0000 7fae4a000640 1 -- 172.21.15.50:0/2210332731 >> 172.21.15.50:0/2210332731 conn(0x7fae440708c0 msgr2=0x7fae44070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:30.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.466+0000 7fae4a000640 1 -- 172.21.15.50:0/2210332731 shutdown_connections 2024-09-21T06:09:30.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.466+0000 7fae4a000640 1 -- 172.21.15.50:0/2210332731 wait complete. 2024-09-21T06:09:30.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae4a000640 1 Processor -- start 2024-09-21T06:09:30.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae4a000640 1 -- start start 2024-09-21T06:09:30.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae4a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae28003c40 0x7fae4413e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:30.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae4a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae441fa210 0x7fae441576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:30.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae4a000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fae44157bf0 0x7fae441540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:30.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae4a000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fae4415d3d0 con 0x7fae28003c40 2024-09-21T06:09:30.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae4a000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae4415d250 con 0x7fae441fa210 2024-09-21T06:09:30.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae4a000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fae4415d550 con 0x7fae44157bf0 2024-09-21T06:09:30.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae43400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae28003c40 0x7fae4413e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:30.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae43400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae28003c40 0x7fae4413e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52626/0 (socket says 172.21.15.50:52626) 2024-09-21T06:09:30.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae43400640 1 -- 172.21.15.50:0/2311219512 learned_addr learned my addr 172.21.15.50:0/2311219512 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:30.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae43e00640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fae44157bf0 0x7fae441540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:30.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae3be00640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae441fa210 0x7fae441576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:30.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae43400640 1 -- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fae44157bf0 msgr2=0x7fae441540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:30.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae43400640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fae44157bf0 0x7fae441540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae43400640 1 -- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae441fa210 msgr2=0x7fae441576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:30.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae43400640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae441fa210 0x7fae441576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae43400640 1 -- 172.21.15.50:0/2311219512 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae44158000 con 0x7fae28003c40 2024-09-21T06:09:30.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae43400640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae28003c40 0x7fae4413e790 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fae3401af80 tx=0x7fae34003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:30.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae3b400640 1 -- 172.21.15.50:0/2311219512 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34001480 con 0x7fae28003c40 2024-09-21T06:09:30.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae3b400640 1 -- 172.21.15.50:0/2311219512 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae340060b0 con 0x7fae28003c40 2024-09-21T06:09:30.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae4a000640 1 -- 172.21.15.50:0/2311219512 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae44151a30 con 0x7fae28003c40 2024-09-21T06:09:30.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae3b400640 1 -- 172.21.15.50:0/2311219512 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34003d40 con 0x7fae28003c40 2024-09-21T06:09:30.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.470+0000 7fae4a000640 1 -- 172.21.15.50:0/2311219512 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae441faff0 con 0x7fae28003c40 2024-09-21T06:09:30.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.474+0000 7fae4a000640 1 -- 172.21.15.50:0/2311219512 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae44071c00 con 0x7fae28003c40 2024-09-21T06:09:30.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.474+0000 7fae3b400640 1 -- 172.21.15.50:0/2311219512 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 43) ==== 99529+0+0 (secure 0 0 0) 0x7fae340b7030 con 0x7fae28003c40 2024-09-21T06:09:30.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.474+0000 7fae3b400640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fae20074b50 0x7fae20077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:30.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.474+0000 7fae3b400640 1 -- 172.21.15.50:0/2311219512 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fae3412cfe0 con 0x7fae28003c40 2024-09-21T06:09:30.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.474+0000 7fae3be00640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fae20074b50 0x7fae20077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:30.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.478+0000 7fae3be00640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fae20074b50 0x7fae20077060 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fae2c000c10 tx=0x7fae2c000fd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:30.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.478+0000 7fae3b400640 1 -- 172.21.15.50:0/2311219512 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fae44158000 con 0x7fae28003c40 2024-09-21T06:09:30.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.682+0000 7fae4a000640 1 -- 172.21.15.50:0/2311219512 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fae4406b540 con 0x7fae20074b50 2024-09-21T06:09:30.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.698+0000 7fae3b400640 1 -- 172.21.15.50:0/2311219512 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fae2c003600 con 0x7fae20074b50 2024-09-21T06:09:30.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.702+0000 7fae4a000640 1 -- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fae20074b50 msgr2=0x7fae20077060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:30.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.702+0000 7fae4a000640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fae20074b50 0x7fae20077060 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fae2c000c10 tx=0x7fae2c000fd0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.702+0000 7fae4a000640 1 -- 172.21.15.50:0/2311219512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae28003c40 msgr2=0x7fae4413e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:30.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.702+0000 7fae4a000640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae28003c40 0x7fae4413e790 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fae3401af80 tx=0x7fae34003b50 comp rx=0 tx=0).stop 2024-09-21T06:09:30.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.702+0000 7fae4a000640 1 -- 172.21.15.50:0/2311219512 shutdown_connections 2024-09-21T06:09:30.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.702+0000 7fae4a000640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fae44157bf0 0x7fae441540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.702+0000 7fae4a000640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae441fa210 0x7fae441576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.702+0000 7fae4a000640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fae20074b50 0x7fae20077060 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.702+0000 7fae4a000640 1 --2- 172.21.15.50:0/2311219512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae28003c40 0x7fae4413e790 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:30.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.702+0000 7fae4a000640 1 -- 172.21.15.50:0/2311219512 >> 172.21.15.50:0/2311219512 conn(0x7fae440708c0 msgr2=0x7fae4414d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:30.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.702+0000 7fae4a000640 1 -- 172.21.15.50:0/2311219512 shutdown_connections 2024-09-21T06:09:30.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:30.702+0000 7fae4a000640 1 -- 172.21.15.50:0/2311219512 wait complete. 2024-09-21T06:09:30.711 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:30.711+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:30.711 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:30.711+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:30.711 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:30.711+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:30.711 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:30.711+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:30.711 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:09:30.711+0000 7f65a2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:30.725 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T06:09:31.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:09:31.019 INFO:teuthology.orchestra.run.smithi050.stdout:2 2024-09-21T06:09:31.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:09:31.026 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:09:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.190+0000 7efefe200640 1 -- 172.21.15.50:0/1648147018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef80b5c70 msgr2=0x7efef815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.190+0000 7efefe200640 1 --2- 172.21.15.50:0/1648147018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef80b5c70 0x7efef815bb60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7efef8070710 tx=0x7efee801c890 comp rx=0 tx=0).stop 2024-09-21T06:09:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.190+0000 7efefe200640 1 -- 172.21.15.50:0/1648147018 shutdown_connections 2024-09-21T06:09:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.190+0000 7efefe200640 1 --2- 172.21.15.50:0/1648147018 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef80b4de0 0x7efef80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.190+0000 7efefe200640 1 --2- 172.21.15.50:0/1648147018 >> v2:172.21.15.88:3300/0 conn(0x7efef81527b0 0x7efef8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.190+0000 7efefe200640 1 --2- 172.21.15.50:0/1648147018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef80b5c70 0x7efef815bb60 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.190+0000 7efefe200640 1 -- 172.21.15.50:0/1648147018 >> 172.21.15.50:0/1648147018 conn(0x7efef80b2040 msgr2=0x7efef80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.190+0000 7efefe200640 1 -- 172.21.15.50:0/1648147018 shutdown_connections 2024-09-21T06:09:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.190+0000 7efefe200640 1 -- 172.21.15.50:0/1648147018 wait complete. 2024-09-21T06:09:31.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.190+0000 7efefe200640 1 Processor -- start 2024-09-21T06:09:31.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.190+0000 7efefe200640 1 -- start start 2024-09-21T06:09:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efefe200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef80b4de0 0x7efef81512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efefe200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef80b5c70 0x7efef81fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efefe200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef81527b0 0x7efef81fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efefe200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efef815f6b0 con 0x7efef81527b0 2024-09-21T06:09:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efefe200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efef815f530 con 0x7efef80b5c70 2024-09-21T06:09:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efefe200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7efef815f830 con 0x7efef80b4de0 2024-09-21T06:09:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efef7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef81527b0 0x7efef81fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efef7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef81527b0 0x7efef81fc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52654/0 (socket says 172.21.15.50:52654) 2024-09-21T06:09:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efef7e00640 1 -- 172.21.15.50:0/2730387966 learned_addr learned my addr 172.21.15.50:0/2730387966 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efeeea00640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef80b5c70 0x7efef81fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efef7400640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef80b4de0 0x7efef81512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efef7e00640 1 -- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef80b4de0 msgr2=0x7efef81512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efef7e00640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef80b4de0 0x7efef81512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efef7e00640 1 -- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef80b5c70 msgr2=0x7efef81fbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efef7e00640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef80b5c70 0x7efef81fbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efef7e00640 1 -- 172.21.15.50:0/2730387966 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef8157270 con 0x7efef81527b0 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efef7e00640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef81527b0 0x7efef81fc1d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7efef000cae0 tx=0x7efef0007220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efeefe00640 1 -- 172.21.15.50:0/2730387966 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0007da0 con 0x7efef81527b0 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efeefe00640 1 -- 172.21.15.50:0/2730387966 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efef000f560 con 0x7efef81527b0 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efeefe00640 1 -- 172.21.15.50:0/2730387966 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef000f9b0 con 0x7efef81527b0 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efefe200640 1 -- 172.21.15.50:0/2730387966 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef8157510 con 0x7efef81527b0 2024-09-21T06:09:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efefe200640 1 -- 172.21.15.50:0/2730387966 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef8152c20 con 0x7efef81527b0 2024-09-21T06:09:31.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.194+0000 7efefe200640 1 -- 172.21.15.50:0/2730387966 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef815fdd0 con 0x7efef81527b0 2024-09-21T06:09:31.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.198+0000 7efeefe00640 1 -- 172.21.15.50:0/2730387966 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 43) ==== 99529+0+0 (secure 0 0 0) 0x7efef000fb50 con 0x7efef81527b0 2024-09-21T06:09:31.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.198+0000 7efeefe00640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efed4074b50 0x7efed4077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:31.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.198+0000 7efeefe00640 1 -- 172.21.15.50:0/2730387966 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7efef0093af0 con 0x7efef81527b0 2024-09-21T06:09:31.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.198+0000 7efef7400640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efed4074b50 0x7efed4077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:31.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.198+0000 7efef7400640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efed4074b50 0x7efed4077060 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7efee80af000 tx=0x7efee801b0d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:31.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.202+0000 7efeefe00640 1 -- 172.21.15.50:0/2730387966 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efef0014070 con 0x7efef81527b0 2024-09-21T06:09:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.378+0000 7efefe200640 1 -- 172.21.15.50:0/2730387966 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7efef815c530 con 0x7efef81527b0 2024-09-21T06:09:31.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.378+0000 7efeefe00640 1 -- 172.21.15.50:0/2730387966 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v50)=0 v50) ==== 131+0+0 (secure 0 0 0) 0x7efef0060570 con 0x7efef81527b0 2024-09-21T06:09:31.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.382+0000 7efefe200640 1 -- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efed4074b50 msgr2=0x7efed4077060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.382+0000 7efefe200640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efed4074b50 0x7efed4077060 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7efee80af000 tx=0x7efee801b0d0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.382+0000 7efefe200640 1 -- 172.21.15.50:0/2730387966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef81527b0 msgr2=0x7efef81fc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.382+0000 7efefe200640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef81527b0 0x7efef81fc1d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7efef000cae0 tx=0x7efef0007220 comp rx=0 tx=0).stop 2024-09-21T06:09:31.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.382+0000 7efefe200640 1 -- 172.21.15.50:0/2730387966 shutdown_connections 2024-09-21T06:09:31.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.382+0000 7efefe200640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efed4074b50 0x7efed4077060 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.382+0000 7efefe200640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef81527b0 0x7efef81fc1d0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.382+0000 7efefe200640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef80b5c70 0x7efef81fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.382+0000 7efefe200640 1 --2- 172.21.15.50:0/2730387966 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7efef80b4de0 0x7efef81512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.382+0000 7efefe200640 1 -- 172.21.15.50:0/2730387966 >> 172.21.15.50:0/2730387966 conn(0x7efef80b2040 msgr2=0x7efef80b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:31.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.382+0000 7efefe200640 1 -- 172.21.15.50:0/2730387966 shutdown_connections 2024-09-21T06:09:31.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.382+0000 7efefe200640 1 -- 172.21.15.50:0/2730387966 wait complete. 2024-09-21T06:09:31.401 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:09:31.401 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:09:31.474 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:09:31.559 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:09:31.559 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:09:31.560 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:09:31.560 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:31.569 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:09:31.569 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:31.620 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:09:31.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.822+0000 7f9497c00640 1 -- 172.21.15.50:0/3330810622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9478004d50 msgr2=0x7f9478005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.822+0000 7f9497c00640 1 --2- 172.21.15.50:0/3330810622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9478004d50 0x7f9478005160 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f94900719d0 tx=0x7f948801d2a0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.822+0000 7f9495c00640 1 -- 172.21.15.50:0/3330810622 reap_dead start 2024-09-21T06:09:31.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.822+0000 7f9497c00640 1 -- 172.21.15.50:0/3330810622 shutdown_connections 2024-09-21T06:09:31.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.822+0000 7f9497c00640 1 -- 172.21.15.50:0/3330810622 >> 172.21.15.50:0/3330810622 conn(0x7f94900b2040 msgr2=0x7f94900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:31.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.822+0000 7f9497c00640 1 -- 172.21.15.50:0/3330810622 shutdown_connections 2024-09-21T06:09:31.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.822+0000 7f9497c00640 1 -- 172.21.15.50:0/3330810622 wait complete. 2024-09-21T06:09:31.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f9497c00640 1 Processor -- start 2024-09-21T06:09:31.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f9497c00640 1 -- start start 2024-09-21T06:09:31.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f9497c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478004d50 0x7f949013cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:31.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f9497c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94900b3cc0 0x7f9490159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:31.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f9497c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f949013d580 0x7f949015a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:31.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f9497c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f94900cf470 con 0x7f94900b3cc0 2024-09-21T06:09:31.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f9497c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f94900cf2f0 con 0x7f9478004d50 2024-09-21T06:09:31.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f9497c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9490160a20 con 0x7f949013d580 2024-09-21T06:09:31.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f948fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94900b3cc0 0x7f9490159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f948fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94900b3cc0 0x7f9490159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52672/0 (socket says 172.21.15.50:52672) 2024-09-21T06:09:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f948fe00640 1 -- 172.21.15.50:0/545560868 learned_addr learned my addr 172.21.15.50:0/545560868 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f9495200640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478004d50 0x7f949013cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.826+0000 7f9495c00640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f949013d580 0x7f949015a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f948fe00640 1 -- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f949013d580 msgr2=0x7f949015a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f948fe00640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f949013d580 0x7f949015a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f948fe00640 1 -- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478004d50 msgr2=0x7f949013cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f948fe00640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478004d50 0x7f949013cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f948fe00640 1 -- 172.21.15.50:0/545560868 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94901f91e0 con 0x7f94900b3cc0 2024-09-21T06:09:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f9495200640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478004d50 0x7f949013cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f948fe00640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94900b3cc0 0x7f9490159b80 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f947c011b10 tx=0x7f947c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f948cc00640 1 -- 172.21.15.50:0/545560868 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c00fca0 con 0x7f94900b3cc0 2024-09-21T06:09:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f948cc00640 1 -- 172.21.15.50:0/545560868 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f947c00fe40 con 0x7f94900b3cc0 2024-09-21T06:09:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f9497c00640 1 -- 172.21.15.50:0/545560868 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94901f94c0 con 0x7f94900b3cc0 2024-09-21T06:09:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f948cc00640 1 -- 172.21.15.50:0/545560868 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c004280 con 0x7f94900b3cc0 2024-09-21T06:09:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f9497c00640 1 -- 172.21.15.50:0/545560868 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94901f9a80 con 0x7f94900b3cc0 2024-09-21T06:09:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f9497c00640 1 -- 172.21.15.50:0/545560868 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f949015b2d0 con 0x7f94900b3cc0 2024-09-21T06:09:31.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f948cc00640 1 -- 172.21.15.50:0/545560868 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f947c003600 con 0x7f94900b3cc0 2024-09-21T06:09:31.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.830+0000 7f948cc00640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9470074c70 0x7f9470077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:31.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.834+0000 7f948cc00640 1 -- 172.21.15.50:0/545560868 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f947c093c40 con 0x7f94900b3cc0 2024-09-21T06:09:31.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.834+0000 7f9495200640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9470074c70 0x7f9470077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:31.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.834+0000 7f9495200640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9470074c70 0x7f9470077180 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9484009c60 tx=0x7f94840091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:31.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:31.838+0000 7f948cc00640 1 -- 172.21.15.50:0/545560868 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f947c0606c0 con 0x7f94900b3cc0 2024-09-21T06:09:32.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.021+0000 7f9497c00640 1 -- 172.21.15.50:0/545560868 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f94901f8f60 con 0x7f94900b3cc0 2024-09-21T06:09:32.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.021+0000 7f948cc00640 1 -- 172.21.15.50:0/545560868 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) ==== 80+0+0 (secure 0 0 0) 0x7f94901f8f60 con 0x7f94900b3cc0 2024-09-21T06:09:32.024 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T06:09:32.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.025+0000 7f9497c00640 1 -- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9470074c70 msgr2=0x7f9470077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.025+0000 7f9497c00640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9470074c70 0x7f9470077180 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9484009c60 tx=0x7f94840091e0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.025+0000 7f9497c00640 1 -- 172.21.15.50:0/545560868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94900b3cc0 msgr2=0x7f9490159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.025+0000 7f9497c00640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94900b3cc0 0x7f9490159b80 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f947c011b10 tx=0x7f947c009e90 comp rx=0 tx=0).stop 2024-09-21T06:09:32.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.025+0000 7f9497c00640 1 -- 172.21.15.50:0/545560868 shutdown_connections 2024-09-21T06:09:32.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.025+0000 7f9497c00640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f949013d580 0x7f949015a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.025+0000 7f9497c00640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9470074c70 0x7f9470077180 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.025+0000 7f9497c00640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94900b3cc0 0x7f9490159b80 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.025+0000 7f9497c00640 1 --2- 172.21.15.50:0/545560868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478004d50 0x7f949013cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.025+0000 7f9497c00640 1 -- 172.21.15.50:0/545560868 >> 172.21.15.50:0/545560868 conn(0x7f94900b2040 msgr2=0x7f94900affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:32.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.025+0000 7f9497c00640 1 -- 172.21.15.50:0/545560868 shutdown_connections 2024-09-21T06:09:32.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.025+0000 7f9497c00640 1 -- 172.21.15.50:0/545560868 wait complete. 2024-09-21T06:09:32.046 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:09:32.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:32.047 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:09:32.101 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:09:32.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:32.101 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:09:32.127 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:09:32.127 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:09:32.127 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:32.138 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:09:32.138 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:09:32.138 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:09:32.138 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:32.185 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:09:32.186 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:09:32.186 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:09:32.186 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:32.203 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:09:32.204 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:09:32.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.501+0000 7feaf6a00640 1 -- 172.21.15.50:0/1692164235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf006cdf0 msgr2=0x7feaf006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.501+0000 7feaf6a00640 1 --2- 172.21.15.50:0/1692164235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf006cdf0 0x7feaf006d200 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feaf00d14d0 tx=0x7fead801c790 comp rx=0 tx=0).stop 2024-09-21T06:09:32.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.501+0000 7feaf6a00640 1 -- 172.21.15.50:0/1692164235 shutdown_connections 2024-09-21T06:09:32.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.501+0000 7feaf6a00640 1 --2- 172.21.15.50:0/1692164235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf006cdf0 0x7feaf006d200 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.501+0000 7feaf6a00640 1 --2- 172.21.15.50:0/1692164235 >> v2:172.21.15.50:3300/0 conn(0x7feaf0159130 0x7feaf015bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.501+0000 7feaf6a00640 1 -- 172.21.15.50:0/1692164235 >> 172.21.15.50:0/1692164235 conn(0x7feaf00b35f0 msgr2=0x7feaf00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:32.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.501+0000 7feaf6a00640 1 -- 172.21.15.50:0/1692164235 shutdown_connections 2024-09-21T06:09:32.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.501+0000 7feaf6a00640 1 -- 172.21.15.50:0/1692164235 wait complete. 2024-09-21T06:09:32.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.505+0000 7feaf6a00640 1 Processor -- start 2024-09-21T06:09:32.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.505+0000 7feaf6a00640 1 -- start start 2024-09-21T06:09:32.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaf6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf006cdf0 0x7feaf014e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:32.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaf6a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feaf006fc90 0x7feaf00cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:32.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaf6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaf0159130 0x7feaf00cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:32.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaf6a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feaf0069680 con 0x7feaf0159130 2024-09-21T06:09:32.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaf6a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feaf0069500 con 0x7feaf006cdf0 2024-09-21T06:09:32.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaf6a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7feaf0069800 con 0x7feaf006fc90 2024-09-21T06:09:32.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaefe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf006cdf0 0x7feaf014e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:32.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaefe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf006cdf0 0x7feaf014e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:45208/0 (socket says 172.21.15.50:45208) 2024-09-21T06:09:32.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaefe00640 1 -- 172.21.15.50:0/3612987431 learned_addr learned my addr 172.21.15.50:0/3612987431 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:32.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaf4a00640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaf0159130 0x7feaf00cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:32.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaef400640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feaf006fc90 0x7feaf00cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:32.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaefe00640 1 -- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feaf006fc90 msgr2=0x7feaf00cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaefe00640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feaf006fc90 0x7feaf00cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaefe00640 1 -- 172.21.15.50:0/3612987431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaf0159130 msgr2=0x7feaf00cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaefe00640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaf0159130 0x7feaf00cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaefe00640 1 -- 172.21.15.50:0/3612987431 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf0157b50 con 0x7feaf006cdf0 2024-09-21T06:09:32.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaefe00640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf006cdf0 0x7feaf014e3c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feae00058d0 tx=0x7feae000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:32.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feae7e00640 1 -- 172.21.15.50:0/3612987431 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0017070 con 0x7feaf006cdf0 2024-09-21T06:09:32.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feae7e00640 1 -- 172.21.15.50:0/3612987431 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feae0009d20 con 0x7feaf006cdf0 2024-09-21T06:09:32.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feae7e00640 1 -- 172.21.15.50:0/3612987431 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae00106c0 con 0x7feaf006cdf0 2024-09-21T06:09:32.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaf6a00640 1 -- 172.21.15.50:0/3612987431 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf0157e30 con 0x7feaf006cdf0 2024-09-21T06:09:32.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.509+0000 7feaf6a00640 1 -- 172.21.15.50:0/3612987431 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf01583f0 con 0x7feaf006cdf0 2024-09-21T06:09:32.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.513+0000 7feaf6a00640 1 -- 172.21.15.50:0/3612987431 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf0069cb0 con 0x7feaf006cdf0 2024-09-21T06:09:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.513+0000 7feae7e00640 1 -- 172.21.15.50:0/3612987431 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7feae001a070 con 0x7feaf006cdf0 2024-09-21T06:09:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.517+0000 7feae7e00640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feacc074cc0 0x7feacc0771d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:32.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.517+0000 7feae7e00640 1 -- 172.21.15.50:0/3612987431 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7feae00930d0 con 0x7feaf006cdf0 2024-09-21T06:09:32.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.517+0000 7feaef400640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feacc074cc0 0x7feacc0771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:32.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.517+0000 7feaef400640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feacc074cc0 0x7feacc0771d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fead80af000 tx=0x7fead801afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:32.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.525+0000 7feae7e00640 1 -- 172.21.15.50:0/3612987431 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7feae005f680 con 0x7feaf006cdf0 2024-09-21T06:09:32.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.753+0000 7feaf6a00640 1 -- 172.21.15.50:0/3612987431 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feaf00b3d40 con 0x7feaf006cdf0 2024-09-21T06:09:32.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.753+0000 7feae7e00640 1 -- 172.21.15.50:0/3612987431 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5222 (secure 0 0 0) 0x7feae00640b0 con 0x7feaf006cdf0 2024-09-21T06:09:32.755 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:32.755 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":66,"btime":"2024-09-21T06:08:45:128314+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2607826086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2607826086},{"type":"v1","addr":"172.21.15.50:6835","nonce":2607826086}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1771536088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1771536088},{"type":"v1","addr":"172.21.15.88:6837","nonce":1771536088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/433905290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":433905290},{"type":"v1","addr":"172.21.15.88:6836","nonce":433905290}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:08:36.508641+0000","modified":"2024-09-21T06:08:39.519558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1418616207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1418616207},{"type":"v1","addr":"172.21.15.50:6834","nonce":1418616207}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7851,"qdb_cluster":[7851]},"id":7}]} 2024-09-21T06:09:32.755 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 66 2024-09-21T06:09:32.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.757+0000 7feaf6a00640 1 -- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feacc074cc0 msgr2=0x7feacc0771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.757+0000 7feaf6a00640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feacc074cc0 0x7feacc0771d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fead80af000 tx=0x7fead801afd0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.757+0000 7feaf6a00640 1 -- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf006cdf0 msgr2=0x7feaf014e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.757+0000 7feaf6a00640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf006cdf0 0x7feaf014e3c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feae00058d0 tx=0x7feae000dde0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.757+0000 7feaf6a00640 1 -- 172.21.15.50:0/3612987431 shutdown_connections 2024-09-21T06:09:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.757+0000 7feaf6a00640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feacc074cc0 0x7feacc0771d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.757+0000 7feaf6a00640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaf0159130 0x7feaf00cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.757+0000 7feaf6a00640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feaf006fc90 0x7feaf00cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.757+0000 7feaf6a00640 1 --2- 172.21.15.50:0/3612987431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf006cdf0 0x7feaf014e3c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.757+0000 7feaf6a00640 1 -- 172.21.15.50:0/3612987431 >> 172.21.15.50:0/3612987431 conn(0x7feaf00b35f0 msgr2=0x7feaf01633d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.757+0000 7feaf6a00640 1 -- 172.21.15.50:0/3612987431 shutdown_connections 2024-09-21T06:09:32.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.757+0000 7feaf6a00640 1 -- 172.21.15.50:0/3612987431 wait complete. 2024-09-21T06:09:32.778 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:09:32.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.937+0000 7f0131400640 1 -- 172.21.15.50:0/2793552458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f012c1527b0 msgr2=0x7f012c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.937+0000 7f0131400640 1 --2- 172.21.15.50:0/2793552458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f012c1527b0 0x7f012c15bb60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f012c074b20 tx=0x7f011401c760 comp rx=0 tx=0).stop 2024-09-21T06:09:32.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.937+0000 7f0131400640 1 -- 172.21.15.50:0/2793552458 shutdown_connections 2024-09-21T06:09:32.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.937+0000 7f0131400640 1 --2- 172.21.15.50:0/2793552458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f012c1527b0 0x7f012c15bb60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.937+0000 7f0131400640 1 -- 172.21.15.50:0/2793552458 >> 172.21.15.50:0/2793552458 conn(0x7f012c0b2040 msgr2=0x7f012c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:32.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.937+0000 7f0131400640 1 -- 172.21.15.50:0/2793552458 shutdown_connections 2024-09-21T06:09:32.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.937+0000 7f0131400640 1 -- 172.21.15.50:0/2793552458 wait complete. 2024-09-21T06:09:32.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.937+0000 7f0131400640 1 Processor -- start 2024-09-21T06:09:32.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.937+0000 7f0131400640 1 -- start start 2024-09-21T06:09:32.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0131400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f012c0b4de0 0x7f012c0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0131400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f012c0b5c70 0x7f012c0ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0131400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f012c1527b0 0x7f012c0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0131400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f012c169bd0 con 0x7f012c0b5c70 2024-09-21T06:09:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0131400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f012c169a50 con 0x7f012c0b4de0 2024-09-21T06:09:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0131400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f012c169d50 con 0x7f012c1527b0 2024-09-21T06:09:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0121e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f012c0b5c70 0x7f012c0ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0121e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f012c0b5c70 0x7f012c0ab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52708/0 (socket says 172.21.15.50:52708) 2024-09-21T06:09:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0121e00640 1 -- 172.21.15.50:0/475513097 learned_addr learned my addr 172.21.15.50:0/475513097 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f012a800640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f012c0b4de0 0x7f012c0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f012b200640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f012c1527b0 0x7f012c0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0121e00640 1 -- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f012c1527b0 msgr2=0x7f012c0b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0121e00640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f012c1527b0 0x7f012c0b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0121e00640 1 -- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f012c0b4de0 msgr2=0x7f012c0af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0121e00640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f012c0b4de0 0x7f012c0af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0121e00640 1 -- 172.21.15.50:0/475513097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012c157e60 con 0x7f012c0b5c70 2024-09-21T06:09:32.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f012a800640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f012c0b4de0 0x7f012c0af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:32.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0121e00640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f012c0b5c70 0x7f012c0ab7b0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f0114009790 tx=0x7f0114005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:32.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0123400640 1 -- 172.21.15.50:0/475513097 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01140b2040 con 0x7f012c0b5c70 2024-09-21T06:09:32.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0123400640 1 -- 172.21.15.50:0/475513097 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01140bb070 con 0x7f012c0b5c70 2024-09-21T06:09:32.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0123400640 1 -- 172.21.15.50:0/475513097 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114003600 con 0x7f012c0b5c70 2024-09-21T06:09:32.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0131400640 1 -- 172.21.15.50:0/475513097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012c158140 con 0x7f012c0b5c70 2024-09-21T06:09:32.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0131400640 1 -- 172.21.15.50:0/475513097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f012c158700 con 0x7f012c0b5c70 2024-09-21T06:09:32.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.941+0000 7f0131400640 1 -- 172.21.15.50:0/475513097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f012c15bb60 con 0x7f012c0b5c70 2024-09-21T06:09:32.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.945+0000 7f0123400640 1 -- 172.21.15.50:0/475513097 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f01140037a0 con 0x7f012c0b5c70 2024-09-21T06:09:32.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.945+0000 7f0123400640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0108074c70 0x7f0108077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:32.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.945+0000 7f0123400640 1 -- 172.21.15.50:0/475513097 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f011412dfd0 con 0x7f012c0b5c70 2024-09-21T06:09:32.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.945+0000 7f012a800640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0108074c70 0x7f0108077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:32.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.945+0000 7f012a800640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0108074c70 0x7f0108077180 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f011c001130 tx=0x7f011c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:32.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:32.949+0000 7f0123400640 1 -- 172.21.15.50:0/475513097 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0114016410 con 0x7f012c0b5c70 2024-09-21T06:09:33.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.209+0000 7f0131400640 1 -- 172.21.15.50:0/475513097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f012c157bf0 con 0x7f012c0b5c70 2024-09-21T06:09:33.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.209+0000 7f0123400640 1 -- 172.21.15.50:0/475513097 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5222 (secure 0 0 0) 0x7f01140faad0 con 0x7f012c0b5c70 2024-09-21T06:09:33.211 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:33.211 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":66,"btime":"2024-09-21T06:08:45:128314+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2607826086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2607826086},{"type":"v1","addr":"172.21.15.50:6835","nonce":2607826086}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1771536088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1771536088},{"type":"v1","addr":"172.21.15.88:6837","nonce":1771536088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/433905290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":433905290},{"type":"v1","addr":"172.21.15.88:6836","nonce":433905290}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:08:36.508641+0000","modified":"2024-09-21T06:08:39.519558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1418616207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1418616207},{"type":"v1","addr":"172.21.15.50:6834","nonce":1418616207}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7851,"qdb_cluster":[7851]},"id":7}]} 2024-09-21T06:09:33.211 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 66 2024-09-21T06:09:33.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.213+0000 7f0131400640 1 -- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0108074c70 msgr2=0x7f0108077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:33.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.213+0000 7f0131400640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0108074c70 0x7f0108077180 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f011c001130 tx=0x7f011c009040 comp rx=0 tx=0).stop 2024-09-21T06:09:33.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.213+0000 7f0131400640 1 -- 172.21.15.50:0/475513097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f012c0b5c70 msgr2=0x7f012c0ab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:33.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.213+0000 7f0131400640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f012c0b5c70 0x7f012c0ab7b0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f0114009790 tx=0x7f0114005ca0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.213+0000 7f0131400640 1 -- 172.21.15.50:0/475513097 shutdown_connections 2024-09-21T06:09:33.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.213+0000 7f0131400640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f012c1527b0 0x7f012c0b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.213+0000 7f0131400640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0108074c70 0x7f0108077180 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.213+0000 7f0131400640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f012c0b5c70 0x7f012c0ab7b0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.213+0000 7f0131400640 1 --2- 172.21.15.50:0/475513097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f012c0b4de0 0x7f012c0af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.213+0000 7f0131400640 1 -- 172.21.15.50:0/475513097 >> 172.21.15.50:0/475513097 conn(0x7f012c0b2040 msgr2=0x7f012c1518f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:33.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.213+0000 7f0131400640 1 -- 172.21.15.50:0/475513097 shutdown_connections 2024-09-21T06:09:33.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.213+0000 7f0131400640 1 -- 172.21.15.50:0/475513097 wait complete. 2024-09-21T06:09:33.248 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:09:33.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.521+0000 7f207d200640 1 -- 172.21.15.50:0/3027858429 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20780b5040 msgr2=0x7f205c0037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:33.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.521+0000 7f207d200640 1 --2- 172.21.15.50:0/3027858429 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20780b5040 0x7f205c0037b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f20780754f0 tx=0x7f206001ccf0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.521+0000 7f2077e00640 1 -- 172.21.15.50:0/3027858429 reap_dead start 2024-09-21T06:09:33.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.521+0000 7f207d200640 1 -- 172.21.15.50:0/3027858429 shutdown_connections 2024-09-21T06:09:33.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.521+0000 7f207d200640 1 -- 172.21.15.50:0/3027858429 >> 172.21.15.50:0/3027858429 conn(0x7f207806cdf0 msgr2=0x7f207806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:33.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.521+0000 7f207d200640 1 -- 172.21.15.50:0/3027858429 shutdown_connections 2024-09-21T06:09:33.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.521+0000 7f207d200640 1 -- 172.21.15.50:0/3027858429 wait complete. 2024-09-21T06:09:33.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.525+0000 7f207d200640 1 Processor -- start 2024-09-21T06:09:33.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.525+0000 7f207d200640 1 -- start start 2024-09-21T06:09:33.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f207d200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20780b5040 0x7f2078208d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:33.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f207d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2078143cc0 0x7f2078209290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:33.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f207d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f207813d8d0 0x7f207813dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:33.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f2077400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20780b5040 0x7f2078208d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:33.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f207d200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2078166e00 con 0x7f207813d8d0 2024-09-21T06:09:33.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f2077400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20780b5040 0x7f2078208d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:41922/0 (socket says 172.21.15.50:41922) 2024-09-21T06:09:33.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f2077400640 1 -- 172.21.15.50:0/1077977017 learned_addr learned my addr 172.21.15.50:0/1077977017 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:33.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f207d200640 1 -- 172.21.15.50:0/1077977017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2078166c80 con 0x7f2078143cc0 2024-09-21T06:09:33.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f206fe00640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2078143cc0 0x7f2078209290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:33.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f207d200640 1 -- 172.21.15.50:0/1077977017 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2078166f80 con 0x7f20780b5040 2024-09-21T06:09:33.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f2077e00640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f207813d8d0 0x7f207813dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:33.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f2077400640 1 -- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2078143cc0 msgr2=0x7f2078209290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:33.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f2077400640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2078143cc0 0x7f2078209290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f2077400640 1 -- 172.21.15.50:0/1077977017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f207813d8d0 msgr2=0x7f207813dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:33.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f2077400640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f207813d8d0 0x7f207813dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.529+0000 7f2077400640 1 -- 172.21.15.50:0/1077977017 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2078142c20 con 0x7f20780b5040 2024-09-21T06:09:33.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.533+0000 7f2077400640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20780b5040 0x7f2078208d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f206800ea10 tx=0x7f206800eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:33.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.533+0000 7f206f400640 1 -- 172.21.15.50:0/1077977017 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206800cbc0 con 0x7f20780b5040 2024-09-21T06:09:33.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.533+0000 7f206f400640 1 -- 172.21.15.50:0/1077977017 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f206800cd60 con 0x7f20780b5040 2024-09-21T06:09:33.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.533+0000 7f206f400640 1 -- 172.21.15.50:0/1077977017 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068004280 con 0x7f20780b5040 2024-09-21T06:09:33.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.533+0000 7f207d200640 1 -- 172.21.15.50:0/1077977017 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2078142f00 con 0x7f20780b5040 2024-09-21T06:09:33.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.533+0000 7f207d200640 1 -- 172.21.15.50:0/1077977017 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f20781434c0 con 0x7f20780b5040 2024-09-21T06:09:33.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.533+0000 7f207d200640 1 -- 172.21.15.50:0/1077977017 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2078072030 con 0x7f20780b5040 2024-09-21T06:09:33.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.537+0000 7f206f400640 1 -- 172.21.15.50:0/1077977017 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f206803d740 con 0x7f20780b5040 2024-09-21T06:09:33.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.541+0000 7f206f400640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2054074c70 0x7f2054077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:33.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.541+0000 7f206f400640 1 -- 172.21.15.50:0/1077977017 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(101..101 src has 1..101) ==== 5821+0+0 (secure 0 0 0) 0x7f20680933e0 con 0x7f20780b5040 2024-09-21T06:09:33.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.541+0000 7f206fe00640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2054074c70 0x7f2054077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:33.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.541+0000 7f206fe00640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2054074c70 0x7f2054077180 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f20600ab000 tx=0x7f2060019ae0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:33.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.545+0000 7f206f400640 1 -- 172.21.15.50:0/1077977017 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f206805f890 con 0x7f20780b5040 2024-09-21T06:09:33.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.733+0000 7f207d200640 1 -- 172.21.15.50:0/1077977017 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f207806d330 con 0x7f20780b5040 2024-09-21T06:09:33.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.733+0000 7f206f400640 1 -- 172.21.15.50:0/1077977017 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16822 (secure 0 0 0) 0x7f20680642c0 con 0x7f20780b5040 2024-09-21T06:09:33.734 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:33.734 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":101,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:09:33.419272+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:08:34.249299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-21T06:08:35.470212+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:09:33.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.733+0000 7f207d200640 1 -- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2054074c70 msgr2=0x7f2054077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:33.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.733+0000 7f207d200640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2054074c70 0x7f2054077180 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f20600ab000 tx=0x7f2060019ae0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.733+0000 7f207d200640 1 -- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20780b5040 msgr2=0x7f2078208d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:33.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.733+0000 7f207d200640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20780b5040 0x7f2078208d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f206800ea10 tx=0x7f206800eed0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.733+0000 7f207d200640 1 -- 172.21.15.50:0/1077977017 shutdown_connections 2024-09-21T06:09:33.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.733+0000 7f207d200640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2054074c70 0x7f2054077180 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.733+0000 7f207d200640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f207813d8d0 0x7f207813dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.733+0000 7f207d200640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2078143cc0 0x7f2078209290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.733+0000 7f207d200640 1 --2- 172.21.15.50:0/1077977017 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20780b5040 0x7f2078208d50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.733+0000 7f207d200640 1 -- 172.21.15.50:0/1077977017 >> 172.21.15.50:0/1077977017 conn(0x7f207806cdf0 msgr2=0x7f207815d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:33.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.737+0000 7f207d200640 1 -- 172.21.15.50:0/1077977017 shutdown_connections 2024-09-21T06:09:33.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.737+0000 7f207d200640 1 -- 172.21.15.50:0/1077977017 wait complete. 2024-09-21T06:09:33.758 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:09:33.758 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:09:33.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.913+0000 7fd19fe00640 1 -- 172.21.15.50:0/1684185067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd198069500 msgr2=0x7fd19815dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:33.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.913+0000 7fd19fe00640 1 --2- 172.21.15.50:0/1684185067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd198069500 0x7fd19815dee0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd190009bc0 tx=0x7fd19001cb90 comp rx=0 tx=0).stop 2024-09-21T06:09:33.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.913+0000 7fd19fe00640 1 -- 172.21.15.50:0/1684185067 shutdown_connections 2024-09-21T06:09:33.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.913+0000 7fd19fe00640 1 --2- 172.21.15.50:0/1684185067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd198069500 0x7fd19815dee0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.913+0000 7fd19fe00640 1 --2- 172.21.15.50:0/1684185067 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1980d0e40 0x7fd19815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.913+0000 7fd19fe00640 1 -- 172.21.15.50:0/1684185067 >> 172.21.15.50:0/1684185067 conn(0x7fd1980b35f0 msgr2=0x7fd1980b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:33.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.913+0000 7fd19fe00640 1 -- 172.21.15.50:0/1684185067 shutdown_connections 2024-09-21T06:09:33.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.913+0000 7fd19fe00640 1 -- 172.21.15.50:0/1684185067 wait complete. 2024-09-21T06:09:33.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19fe00640 1 Processor -- start 2024-09-21T06:09:33.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19fe00640 1 -- start start 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1980d0e40 0x7fd19806ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19fe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1980ae8e0 0x7fd1980af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1980afd30 0x7fd19814c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd19816a120 con 0x7fd1980d0e40 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd198169fa0 con 0x7fd1980afd30 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19fe00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd19816a2a0 con 0x7fd1980ae8e0 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1980d0e40 0x7fd19806ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1980d0e40 0x7fd19806ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52746/0 (socket says 172.21.15.50:52746) 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19d400640 1 -- 172.21.15.50:0/1878919525 learned_addr learned my addr 172.21.15.50:0/1878919525 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19ca00640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1980ae8e0 0x7fd1980af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19de00640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1980afd30 0x7fd19814c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19d400640 1 -- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1980ae8e0 msgr2=0x7fd1980af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19d400640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1980ae8e0 0x7fd1980af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19d400640 1 -- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1980afd30 msgr2=0x7fd19814c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:33.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19d400640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1980afd30 0x7fd19814c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19d400640 1 -- 172.21.15.50:0/1878919525 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1980b4a70 con 0x7fd1980d0e40 2024-09-21T06:09:33.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19ca00640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1980ae8e0 0x7fd1980af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:09:33.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.917+0000 7fd19de00640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1980afd30 0x7fd19814c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:09:33.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.921+0000 7fd19d400640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1980d0e40 0x7fd19806ff30 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00e7c0 tx=0x7fd18c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:33.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.921+0000 7fd195600640 1 -- 172.21.15.50:0/1878919525 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00ccc0 con 0x7fd1980d0e40 2024-09-21T06:09:33.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.921+0000 7fd195600640 1 -- 172.21.15.50:0/1878919525 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd18c00ce80 con 0x7fd1980d0e40 2024-09-21T06:09:33.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.921+0000 7fd195600640 1 -- 172.21.15.50:0/1878919525 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c005280 con 0x7fd1980d0e40 2024-09-21T06:09:33.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.921+0000 7fd19fe00640 1 -- 172.21.15.50:0/1878919525 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1980b4d50 con 0x7fd1980d0e40 2024-09-21T06:09:33.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.921+0000 7fd19fe00640 1 -- 172.21.15.50:0/1878919525 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1980af2d0 con 0x7fd1980d0e40 2024-09-21T06:09:33.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.921+0000 7fd19fe00640 1 -- 172.21.15.50:0/1878919525 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1980d1250 con 0x7fd1980d0e40 2024-09-21T06:09:33.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.921+0000 7fd195600640 1 -- 172.21.15.50:0/1878919525 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fd18c005420 con 0x7fd1980d0e40 2024-09-21T06:09:33.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.925+0000 7fd195600640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd178074ca0 0x7fd1780771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:33.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.925+0000 7fd195600640 1 -- 172.21.15.50:0/1878919525 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5821+0+0 (secure 0 0 0) 0x7fd18c093a70 con 0x7fd1980d0e40 2024-09-21T06:09:33.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.925+0000 7fd19ca00640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd178074ca0 0x7fd1780771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:33.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.925+0000 7fd19ca00640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd178074ca0 0x7fd1780771b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd1840046c0 tx=0x7fd184009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:33.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:33.929+0000 7fd195600640 1 -- 172.21.15.50:0/1878919525 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd18c060580 con 0x7fd1980d0e40 2024-09-21T06:09:34.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 -- 172.21.15.50:0/1878919525 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1980b1070 con 0x7fd1980d0e40 2024-09-21T06:09:34.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd195600640 1 -- 172.21.15.50:0/1878919525 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5222 (secure 0 0 0) 0x7fd18c062090 con 0x7fd1980d0e40 2024-09-21T06:09:34.145 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:34.145 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":66,"btime":"2024-09-21T06:08:45:128314+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2607826086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2607826086},{"type":"v1","addr":"172.21.15.50:6835","nonce":2607826086}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1771536088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1771536088},{"type":"v1","addr":"172.21.15.88:6837","nonce":1771536088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/433905290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":433905290},{"type":"v1","addr":"172.21.15.88:6836","nonce":433905290}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:08:36.508641+0000","modified":"2024-09-21T06:08:39.519558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1418616207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1418616207},{"type":"v1","addr":"172.21.15.50:6834","nonce":1418616207}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7851,"qdb_cluster":[7851]},"id":7}]} 2024-09-21T06:09:34.145 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 66 2024-09-21T06:09:34.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 -- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd178074ca0 msgr2=0x7fd1780771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd178074ca0 0x7fd1780771b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd1840046c0 tx=0x7fd184009040 comp rx=0 tx=0).stop 2024-09-21T06:09:34.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 -- 172.21.15.50:0/1878919525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1980d0e40 msgr2=0x7fd19806ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1980d0e40 0x7fd19806ff30 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00e7c0 tx=0x7fd18c00ec80 comp rx=0 tx=0).stop 2024-09-21T06:09:34.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 -- 172.21.15.50:0/1878919525 shutdown_connections 2024-09-21T06:09:34.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1980afd30 0x7fd19814c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd1980ae8e0 0x7fd1980af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd178074ca0 0x7fd1780771b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 --2- 172.21.15.50:0/1878919525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1980d0e40 0x7fd19806ff30 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 -- 172.21.15.50:0/1878919525 >> 172.21.15.50:0/1878919525 conn(0x7fd1980b35f0 msgr2=0x7fd198068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:34.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 -- 172.21.15.50:0/1878919525 shutdown_connections 2024-09-21T06:09:34.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.145+0000 7fd19fe00640 1 -- 172.21.15.50:0/1878919525 wait complete. 2024-09-21T06:09:34.168 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:09:34.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.325+0000 7fdca8000640 1 -- 172.21.15.50:0/2078179166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca00b3340 msgr2=0x7fdca00b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.325+0000 7fdca8000640 1 --2- 172.21.15.50:0/2078179166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca00b3340 0x7fdca00b3750 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fdca00726f0 tx=0x7fdc9801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.325+0000 7fdca8000640 1 -- 172.21.15.50:0/2078179166 shutdown_connections 2024-09-21T06:09:34.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.325+0000 7fdca8000640 1 --2- 172.21.15.50:0/2078179166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca00b3340 0x7fdca00b3750 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.325+0000 7fdca8000640 1 --2- 172.21.15.50:0/2078179166 >> v2:172.21.15.88:3300/0 conn(0x7fdca00cefa0 0x7fdca00cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.325+0000 7fdca8000640 1 -- 172.21.15.50:0/2078179166 >> 172.21.15.50:0/2078179166 conn(0x7fdca00b2040 msgr2=0x7fdca00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:34.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.325+0000 7fdca8000640 1 -- 172.21.15.50:0/2078179166 shutdown_connections 2024-09-21T06:09:34.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.325+0000 7fdca8000640 1 -- 172.21.15.50:0/2078179166 wait complete. 2024-09-21T06:09:34.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca8000640 1 Processor -- start 2024-09-21T06:09:34.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca8000640 1 -- start start 2024-09-21T06:09:34.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca8000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdca00749a0 0x7fdca00ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:34.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca8000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca00b3340 0x7fdca013c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:34.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca8000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdca00cefa0 0x7fdca01f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:34.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca8000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0169d20 con 0x7fdca00b3340 2024-09-21T06:09:34.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca8000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0169ba0 con 0x7fdca00cefa0 2024-09-21T06:09:34.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca8000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fdca0169ea0 con 0x7fdca00749a0 2024-09-21T06:09:34.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca5600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdca00749a0 0x7fdca00ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:34.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca00b3340 0x7fdca013c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:34.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca6000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdca00cefa0 0x7fdca01f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:34.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca5600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdca00749a0 0x7fdca00ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:41938/0 (socket says 172.21.15.50:41938) 2024-09-21T06:09:34.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca5600640 1 -- 172.21.15.50:0/80787221 learned_addr learned my addr 172.21.15.50:0/80787221 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:34.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca00b3340 0x7fdca013c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52766/0 (socket says 172.21.15.50:52766) 2024-09-21T06:09:34.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca6000640 1 -- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdca00749a0 msgr2=0x7fdca00ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca6000640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdca00749a0 0x7fdca00ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca6000640 1 -- 172.21.15.50:0/80787221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca00b3340 msgr2=0x7fdca013c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca6000640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca00b3340 0x7fdca013c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca6000640 1 -- 172.21.15.50:0/80787221 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca00afaf0 con 0x7fdca00cefa0 2024-09-21T06:09:34.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca4c00640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca00b3340 0x7fdca013c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:09:34.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca6000640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdca00cefa0 0x7fdca01f92e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdc980ac000 tx=0x7fdc980a57f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:34.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdc9d600640 1 -- 172.21.15.50:0/80787221 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc980b2040 con 0x7fdca00cefa0 2024-09-21T06:09:34.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdc9d600640 1 -- 172.21.15.50:0/80787221 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc98004430 con 0x7fdca00cefa0 2024-09-21T06:09:34.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca8000640 1 -- 172.21.15.50:0/80787221 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca00af530 con 0x7fdca00cefa0 2024-09-21T06:09:34.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdc9d600640 1 -- 172.21.15.50:0/80787221 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98005290 con 0x7fdca00cefa0 2024-09-21T06:09:34.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.329+0000 7fdca8000640 1 -- 172.21.15.50:0/80787221 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca01f8ae0 con 0x7fdca00cefa0 2024-09-21T06:09:34.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.333+0000 7fdca8000640 1 -- 172.21.15.50:0/80787221 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca01fce40 con 0x7fdca00cefa0 2024-09-21T06:09:34.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.333+0000 7fdc9d600640 1 -- 172.21.15.50:0/80787221 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fdc98005430 con 0x7fdca00cefa0 2024-09-21T06:09:34.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.333+0000 7fdc9d600640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdc80074c70 0x7fdc80077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:34.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.333+0000 7fdc9d600640 1 -- 172.21.15.50:0/80787221 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5821+0+0 (secure 0 0 0) 0x7fdc9812d090 con 0x7fdca00cefa0 2024-09-21T06:09:34.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.333+0000 7fdca5600640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdc80074c70 0x7fdc80077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:34.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.333+0000 7fdca5600640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdc80074c70 0x7fdc80077180 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdc94001f00 tx=0x7fdc94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:34.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.337+0000 7fdc9d600640 1 -- 172.21.15.50:0/80787221 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdc980164b0 con 0x7fdca00cefa0 2024-09-21T06:09:34.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 -- 172.21.15.50:0/80787221 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdca00b3750 con 0x7fdca00cefa0 2024-09-21T06:09:34.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdc9d600640 1 -- 172.21.15.50:0/80787221 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5222 (secure 0 0 0) 0x7fdc980f94c0 con 0x7fdca00cefa0 2024-09-21T06:09:34.553 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:34.553 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":66,"btime":"2024-09-21T06:08:45:128314+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2607826086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2607826086},{"type":"v1","addr":"172.21.15.50:6835","nonce":2607826086}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1771536088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1771536088},{"type":"v1","addr":"172.21.15.88:6837","nonce":1771536088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/433905290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":433905290},{"type":"v1","addr":"172.21.15.88:6836","nonce":433905290}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:08:36.508641+0000","modified":"2024-09-21T06:08:39.519558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1418616207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1418616207},{"type":"v1","addr":"172.21.15.50:6834","nonce":1418616207}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7851,"qdb_cluster":[7851]},"id":7}]} 2024-09-21T06:09:34.553 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 66 2024-09-21T06:09:34.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 -- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdc80074c70 msgr2=0x7fdc80077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdc80074c70 0x7fdc80077180 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdc94001f00 tx=0x7fdc94009040 comp rx=0 tx=0).stop 2024-09-21T06:09:34.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 -- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdca00cefa0 msgr2=0x7fdca01f92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdca00cefa0 0x7fdca01f92e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdc980ac000 tx=0x7fdc980a57f0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 -- 172.21.15.50:0/80787221 shutdown_connections 2024-09-21T06:09:34.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdca00cefa0 0x7fdca01f92e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fdc80074c70 0x7fdc80077180 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca00b3340 0x7fdca013c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 --2- 172.21.15.50:0/80787221 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fdca00749a0 0x7fdca00ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 -- 172.21.15.50:0/80787221 >> 172.21.15.50:0/80787221 conn(0x7fdca00b2040 msgr2=0x7fdca00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:34.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 -- 172.21.15.50:0/80787221 shutdown_connections 2024-09-21T06:09:34.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.553+0000 7fdca8000640 1 -- 172.21.15.50:0/80787221 wait complete. 2024-09-21T06:09:34.575 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:09:34.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.733+0000 7fac85800640 1 -- 172.21.15.50:0/957203540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac64004810 msgr2=0x7fac64004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.733+0000 7fac85800640 1 --2- 172.21.15.50:0/957203540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac64004810 0x7fac64004c20 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fac800b3070 tx=0x7fac6c01c750 comp rx=0 tx=0).stop 2024-09-21T06:09:34.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.733+0000 7fac7f400640 1 -- 172.21.15.50:0/957203540 reap_dead start 2024-09-21T06:09:34.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.733+0000 7fac85800640 1 -- 172.21.15.50:0/957203540 shutdown_connections 2024-09-21T06:09:34.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.733+0000 7fac85800640 1 -- 172.21.15.50:0/957203540 >> 172.21.15.50:0/957203540 conn(0x7fac800b2040 msgr2=0x7fac800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:34.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.733+0000 7fac85800640 1 -- 172.21.15.50:0/957203540 shutdown_connections 2024-09-21T06:09:34.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.733+0000 7fac85800640 1 -- 172.21.15.50:0/957203540 wait complete. 2024-09-21T06:09:34.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac85800640 1 Processor -- start 2024-09-21T06:09:34.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac85800640 1 -- start start 2024-09-21T06:09:34.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac85800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac64004810 0x7fac80152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac85800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac8014fd60 0x7fac801e7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac85800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac80150170 0x7fac801e7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac85800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fac8015d070 con 0x7fac8014fd60 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac85800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fac8015cef0 con 0x7fac80150170 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac85800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fac8015d1f0 con 0x7fac64004810 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac7e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac8014fd60 0x7fac801e7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac7e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac8014fd60 0x7fac801e7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52784/0 (socket says 172.21.15.50:52784) 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac7e000640 1 -- 172.21.15.50:0/2106918569 learned_addr learned my addr 172.21.15.50:0/2106918569 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac7ea00640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac64004810 0x7fac80152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac7f400640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac80150170 0x7fac801e7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac7e000640 1 -- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac64004810 msgr2=0x7fac80152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac7e000640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac64004810 0x7fac80152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac7e000640 1 -- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac80150170 msgr2=0x7fac801e7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac7e000640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac80150170 0x7fac801e7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac7e000640 1 -- 172.21.15.50:0/2106918569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8015b330 con 0x7fac8014fd60 2024-09-21T06:09:34.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac7e000640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac8014fd60 0x7fac801e7740 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fac6c01b020 tx=0x7fac6c0013a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:34.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac76a00640 1 -- 172.21.15.50:0/2106918569 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c003810 con 0x7fac8014fd60 2024-09-21T06:09:34.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac76a00640 1 -- 172.21.15.50:0/2106918569 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac6c003aa0 con 0x7fac8014fd60 2024-09-21T06:09:34.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac76a00640 1 -- 172.21.15.50:0/2106918569 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c003e50 con 0x7fac8014fd60 2024-09-21T06:09:34.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac85800640 1 -- 172.21.15.50:0/2106918569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac801535c0 con 0x7fac8014fd60 2024-09-21T06:09:34.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.737+0000 7fac85800640 1 -- 172.21.15.50:0/2106918569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac80153b80 con 0x7fac8014fd60 2024-09-21T06:09:34.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.741+0000 7fac85800640 1 -- 172.21.15.50:0/2106918569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac80153330 con 0x7fac8014fd60 2024-09-21T06:09:34.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.741+0000 7fac76a00640 1 -- 172.21.15.50:0/2106918569 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fac6c0a55b0 con 0x7fac8014fd60 2024-09-21T06:09:34.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.741+0000 7fac76a00640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac60074e10 0x7fac60077320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:34.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.741+0000 7fac76a00640 1 -- 172.21.15.50:0/2106918569 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5821+0+0 (secure 0 0 0) 0x7fac6c12cda0 con 0x7fac8014fd60 2024-09-21T06:09:34.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.741+0000 7fac7ea00640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac60074e10 0x7fac60077320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:34.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.741+0000 7fac7ea00640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac60074e10 0x7fac60077320 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fac70009c60 tx=0x7fac700091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:34.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.745+0000 7fac76a00640 1 -- 172.21.15.50:0/2106918569 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fac6c0b3070 con 0x7fac8014fd60 2024-09-21T06:09:34.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.921+0000 7fac85800640 1 -- 172.21.15.50:0/2106918569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac800b51f0 con 0x7fac8014fd60 2024-09-21T06:09:34.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.921+0000 7fac76a00640 1 -- 172.21.15.50:0/2106918569 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16822 (secure 0 0 0) 0x7fac6c0fee20 con 0x7fac8014fd60 2024-09-21T06:09:34.924 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:34.924 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":101,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:09:33.419272+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:08:34.249299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-21T06:08:35.470212+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:09:34.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.925+0000 7fac85800640 1 -- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac60074e10 msgr2=0x7fac60077320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.925+0000 7fac85800640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac60074e10 0x7fac60077320 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fac70009c60 tx=0x7fac700091e0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.925+0000 7fac85800640 1 -- 172.21.15.50:0/2106918569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac8014fd60 msgr2=0x7fac801e7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.925+0000 7fac85800640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac8014fd60 0x7fac801e7740 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fac6c01b020 tx=0x7fac6c0013a0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.925+0000 7fac85800640 1 -- 172.21.15.50:0/2106918569 shutdown_connections 2024-09-21T06:09:34.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.925+0000 7fac85800640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac80150170 0x7fac801e7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.925+0000 7fac85800640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fac60074e10 0x7fac60077320 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.925+0000 7fac85800640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac8014fd60 0x7fac801e7740 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.925+0000 7fac85800640 1 --2- 172.21.15.50:0/2106918569 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac64004810 0x7fac80152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.925+0000 7fac85800640 1 -- 172.21.15.50:0/2106918569 >> 172.21.15.50:0/2106918569 conn(0x7fac800b2040 msgr2=0x7fac80151db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:34.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.925+0000 7fac85800640 1 -- 172.21.15.50:0/2106918569 shutdown_connections 2024-09-21T06:09:34.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:34.925+0000 7fac85800640 1 -- 172.21.15.50:0/2106918569 wait complete. 2024-09-21T06:09:34.947 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:09:35.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.105+0000 7fc561000640 1 -- 172.21.15.50:0/3264645345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc55c160a20 msgr2=0x7fc55c162e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:35.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.105+0000 7fc561000640 1 --2- 172.21.15.50:0/3264645345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc55c160a20 0x7fc55c162e40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc544009840 tx=0x7fc54401c820 comp rx=0 tx=0).stop 2024-09-21T06:09:35.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.109+0000 7fc561000640 1 -- 172.21.15.50:0/3264645345 shutdown_connections 2024-09-21T06:09:35.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.109+0000 7fc561000640 1 --2- 172.21.15.50:0/3264645345 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc55c163380 0x7fc55c1657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.109+0000 7fc561000640 1 --2- 172.21.15.50:0/3264645345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc55c160a20 0x7fc55c162e40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.109+0000 7fc561000640 1 -- 172.21.15.50:0/3264645345 >> 172.21.15.50:0/3264645345 conn(0x7fc55c0b2040 msgr2=0x7fc55c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:35.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.109+0000 7fc561000640 1 -- 172.21.15.50:0/3264645345 shutdown_connections 2024-09-21T06:09:35.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.109+0000 7fc561000640 1 -- 172.21.15.50:0/3264645345 wait complete. 2024-09-21T06:09:35.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.109+0000 7fc561000640 1 Processor -- start 2024-09-21T06:09:35.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.109+0000 7fc561000640 1 -- start start 2024-09-21T06:09:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.109+0000 7fc561000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc55c15e8c0 0x7fc55c158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.109+0000 7fc561000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc55c160a20 0x7fc55c13da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.109+0000 7fc561000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c163380 0x7fc55c12f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.109+0000 7fc561000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c168cf0 con 0x7fc55c163380 2024-09-21T06:09:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc561000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c168b70 con 0x7fc55c15e8c0 2024-09-21T06:09:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc561000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc55c168e70 con 0x7fc55c160a20 2024-09-21T06:09:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc55a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c163380 0x7fc55c12f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:35.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc55a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c163380 0x7fc55c12f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52794/0 (socket says 172.21.15.50:52794) 2024-09-21T06:09:35.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc55a800640 1 -- 172.21.15.50:0/488785489 learned_addr learned my addr 172.21.15.50:0/488785489 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:35.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc559e00640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc55c15e8c0 0x7fc55c158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:35.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc551400640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc55c160a20 0x7fc55c13da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc55a800640 1 -- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc55c160a20 msgr2=0x7fc55c13da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc55a800640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc55c160a20 0x7fc55c13da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc55a800640 1 -- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc55c15e8c0 msgr2=0x7fc55c158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc55a800640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc55c15e8c0 0x7fc55c158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc55a800640 1 -- 172.21.15.50:0/488785489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55c158cb0 con 0x7fc55c163380 2024-09-21T06:09:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc55a800640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c163380 0x7fc55c12f420 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fc55400eab0 tx=0x7fc554009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc552a00640 1 -- 172.21.15.50:0/488785489 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55400ce40 con 0x7fc55c163380 2024-09-21T06:09:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc552a00640 1 -- 172.21.15.50:0/488785489 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc554004c20 con 0x7fc55c163380 2024-09-21T06:09:35.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc561000640 1 -- 172.21.15.50:0/488785489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc55c161f00 con 0x7fc55c163380 2024-09-21T06:09:35.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc552a00640 1 -- 172.21.15.50:0/488785489 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc554005020 con 0x7fc55c163380 2024-09-21T06:09:35.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc561000640 1 -- 172.21.15.50:0/488785489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc55c1624c0 con 0x7fc55c163380 2024-09-21T06:09:35.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc561000640 1 -- 172.21.15.50:0/488785489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc55c161c70 con 0x7fc55c163380 2024-09-21T06:09:35.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.113+0000 7fc552a00640 1 -- 172.21.15.50:0/488785489 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fc5540051e0 con 0x7fc55c163380 2024-09-21T06:09:35.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.117+0000 7fc552a00640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc538074c70 0x7fc538077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:35.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.117+0000 7fc552a00640 1 -- 172.21.15.50:0/488785489 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5821+0+0 (secure 0 0 0) 0x7fc554093d40 con 0x7fc55c163380 2024-09-21T06:09:35.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.117+0000 7fc559e00640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc538074c70 0x7fc538077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:35.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.117+0000 7fc559e00640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc538074c70 0x7fc538077180 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc54c007fd0 tx=0x7fc54c007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:35.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.121+0000 7fc552a00640 1 -- 172.21.15.50:0/488785489 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc554060180 con 0x7fc55c163380 2024-09-21T06:09:35.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.333+0000 7fc561000640 1 -- 172.21.15.50:0/488785489 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc55c15ecd0 con 0x7fc55c163380 2024-09-21T06:09:35.335 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-09-21T06:09:35.333+0000 7f0d45000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:09:35.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.337+0000 7fc552a00640 1 -- 172.21.15.50:0/488785489 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) ==== 161+0+0 (secure 0 0 0) 0x7fc5540670d0 con 0x7fc55c163380 2024-09-21T06:09:35.337 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:09:35.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.341+0000 7fc561000640 1 -- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc538074c70 msgr2=0x7fc538077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:35.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.341+0000 7fc561000640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc538074c70 0x7fc538077180 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc54c007fd0 tx=0x7fc54c007ea0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.341+0000 7fc561000640 1 -- 172.21.15.50:0/488785489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c163380 msgr2=0x7fc55c12f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:35.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.341+0000 7fc561000640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c163380 0x7fc55c12f420 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fc55400eab0 tx=0x7fc554009f70 comp rx=0 tx=0).stop 2024-09-21T06:09:35.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.341+0000 7fc561000640 1 -- 172.21.15.50:0/488785489 shutdown_connections 2024-09-21T06:09:35.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.341+0000 7fc561000640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc538074c70 0x7fc538077180 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.341+0000 7fc561000640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c163380 0x7fc55c12f420 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.341+0000 7fc561000640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc55c160a20 0x7fc55c13da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.341+0000 7fc561000640 1 --2- 172.21.15.50:0/488785489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc55c15e8c0 0x7fc55c158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.341+0000 7fc561000640 1 -- 172.21.15.50:0/488785489 >> 172.21.15.50:0/488785489 conn(0x7fc55c0b2040 msgr2=0x7fc55c1592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:35.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.341+0000 7fc561000640 1 -- 172.21.15.50:0/488785489 shutdown_connections 2024-09-21T06:09:35.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.341+0000 7fc561000640 1 -- 172.21.15.50:0/488785489 wait complete. 2024-09-21T06:09:35.363 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:09:35.483 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T06:09:35.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.529+0000 7f39d0800640 1 -- 172.21.15.50:0/4118304344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c80b4de0 msgr2=0x7f39c80b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:35.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.529+0000 7f39d0800640 1 --2- 172.21.15.50:0/4118304344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c80b4de0 0x7f39c80b51f0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f39c8071cb0 tx=0x7f39c401ccc0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.529+0000 7f39d0800640 1 -- 172.21.15.50:0/4118304344 shutdown_connections 2024-09-21T06:09:35.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.529+0000 7f39d0800640 1 --2- 172.21.15.50:0/4118304344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c80b4de0 0x7f39c80b51f0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.529+0000 7f39d0800640 1 --2- 172.21.15.50:0/4118304344 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f39c81527b0 0x7f39c8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.529+0000 7f39d0800640 1 --2- 172.21.15.50:0/4118304344 >> v2:172.21.15.88:3300/0 conn(0x7f39c80b5c70 0x7f39c815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.529+0000 7f39d0800640 1 -- 172.21.15.50:0/4118304344 >> 172.21.15.50:0/4118304344 conn(0x7f39c80b2040 msgr2=0x7f39c80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:35.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.529+0000 7f39d0800640 1 -- 172.21.15.50:0/4118304344 shutdown_connections 2024-09-21T06:09:35.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.529+0000 7f39d0800640 1 -- 172.21.15.50:0/4118304344 wait complete. 2024-09-21T06:09:35.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39d0800640 1 Processor -- start 2024-09-21T06:09:35.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39d0800640 1 -- start start 2024-09-21T06:09:35.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39d0800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c80b4de0 0x7f39c813dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:35.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39d0800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f39c80b5c70 0x7f39c8157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:35.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39d0800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39c81527b0 0x7f39c8158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:35.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39d0800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f39c815f730 con 0x7f39c80b4de0 2024-09-21T06:09:35.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39d0800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39c815f5b0 con 0x7f39c81527b0 2024-09-21T06:09:35.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39d0800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f39c815f8b0 con 0x7f39c80b5c70 2024-09-21T06:09:35.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39cde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c80b4de0 0x7f39c813dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:35.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39cde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c80b4de0 0x7f39c813dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52834/0 (socket says 172.21.15.50:52834) 2024-09-21T06:09:35.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39cde00640 1 -- 172.21.15.50:0/3651758113 learned_addr learned my addr 172.21.15.50:0/3651758113 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:35.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39cd400640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f39c80b5c70 0x7f39c8157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:35.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39ce800640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39c81527b0 0x7f39c8158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:35.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39cde00640 1 -- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f39c80b5c70 msgr2=0x7f39c8157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:35.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39cde00640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f39c80b5c70 0x7f39c8157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39cde00640 1 -- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39c81527b0 msgr2=0x7f39c8158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:35.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39cde00640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39c81527b0 0x7f39c8158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39cde00640 1 -- 172.21.15.50:0/3651758113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c81561c0 con 0x7f39c80b4de0 2024-09-21T06:09:35.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39cde00640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c80b4de0 0x7f39c813dcd0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00d6d0 tx=0x7f39bc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:35.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39c2000640 1 -- 172.21.15.50:0/3651758113 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc017070 con 0x7f39c80b4de0 2024-09-21T06:09:35.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39c2000640 1 -- 172.21.15.50:0/3651758113 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39bc009b10 con 0x7f39c80b4de0 2024-09-21T06:09:35.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39c2000640 1 -- 172.21.15.50:0/3651758113 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc010460 con 0x7f39c80b4de0 2024-09-21T06:09:35.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39d0800640 1 -- 172.21.15.50:0/3651758113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c81530a0 con 0x7f39c80b4de0 2024-09-21T06:09:35.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.533+0000 7f39d0800640 1 -- 172.21.15.50:0/3651758113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c8158820 con 0x7f39c80b4de0 2024-09-21T06:09:35.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.537+0000 7f39d0800640 1 -- 172.21.15.50:0/3651758113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c815fdd0 con 0x7f39c80b4de0 2024-09-21T06:09:35.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.537+0000 7f39c2000640 1 -- 172.21.15.50:0/3651758113 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f39bc01a070 con 0x7f39c80b4de0 2024-09-21T06:09:35.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.537+0000 7f39c2000640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39ac074c70 0x7f39ac077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:35.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.537+0000 7f39c2000640 1 -- 172.21.15.50:0/3651758113 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5879+0+0 (secure 0 0 0) 0x7f39bc093090 con 0x7f39c80b4de0 2024-09-21T06:09:35.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.537+0000 7f39cd400640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39ac074c70 0x7f39ac077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:35.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.537+0000 7f39cd400640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39ac074c70 0x7f39ac077180 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f39c81510a0 tx=0x7f39b800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:35.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.541+0000 7f39c2000640 1 -- 172.21.15.50:0/3651758113 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f39bc05f5f0 con 0x7f39c80b4de0 2024-09-21T06:09:35.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:35.757+0000 7f39d0800640 1 -- 172.21.15.50:0/3651758113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f39c80b4ac0 con 0x7f39c80b4de0 2024-09-21T06:09:36.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.441+0000 7f39c2000640 1 -- 172.21.15.50:0/3651758113 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) ==== 104+0+0 (secure 0 0 0) 0x7f39bc066020 con 0x7f39c80b4de0 2024-09-21T06:09:36.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.445+0000 7f39d0800640 1 -- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39ac074c70 msgr2=0x7f39ac077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:36.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.445+0000 7f39d0800640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39ac074c70 0x7f39ac077180 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f39c81510a0 tx=0x7f39b800b2e0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.445+0000 7f39d0800640 1 -- 172.21.15.50:0/3651758113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c80b4de0 msgr2=0x7f39c813dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:36.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.445+0000 7f39d0800640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c80b4de0 0x7f39c813dcd0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00d6d0 tx=0x7f39bc00db90 comp rx=0 tx=0).stop 2024-09-21T06:09:36.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.445+0000 7f39d0800640 1 -- 172.21.15.50:0/3651758113 shutdown_connections 2024-09-21T06:09:36.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.445+0000 7f39d0800640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39c81527b0 0x7f39c8158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.445+0000 7f39d0800640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f39c80b5c70 0x7f39c8157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.445+0000 7f39d0800640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39ac074c70 0x7f39ac077180 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.445+0000 7f39d0800640 1 --2- 172.21.15.50:0/3651758113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c80b4de0 0x7f39c813dcd0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.445+0000 7f39d0800640 1 -- 172.21.15.50:0/3651758113 >> 172.21.15.50:0/3651758113 conn(0x7f39c80b2040 msgr2=0x7f39c806ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:36.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.445+0000 7f39d0800640 1 -- 172.21.15.50:0/3651758113 shutdown_connections 2024-09-21T06:09:36.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.445+0000 7f39d0800640 1 -- 172.21.15.50:0/3651758113 wait complete. 2024-09-21T06:09:36.473 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:09:36.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.641+0000 7f1c84c00640 1 -- 172.21.15.50:0/3230693933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c800b5c70 msgr2=0x7f1c8015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:36.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.641+0000 7f1c84c00640 1 --2- 172.21.15.50:0/3230693933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c800b5c70 0x7f1c8015bb60 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009900 tx=0x7f1c7001c820 comp rx=0 tx=0).stop 2024-09-21T06:09:36.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.641+0000 7f1c84c00640 1 -- 172.21.15.50:0/3230693933 shutdown_connections 2024-09-21T06:09:36.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.641+0000 7f1c84c00640 1 --2- 172.21.15.50:0/3230693933 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1c801527b0 0x7f1c80074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.641+0000 7f1c84c00640 1 --2- 172.21.15.50:0/3230693933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c800b5c70 0x7f1c8015bb60 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.641+0000 7f1c84c00640 1 -- 172.21.15.50:0/3230693933 >> 172.21.15.50:0/3230693933 conn(0x7f1c800b2040 msgr2=0x7f1c800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:36.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.641+0000 7f1c84c00640 1 -- 172.21.15.50:0/3230693933 shutdown_connections 2024-09-21T06:09:36.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.641+0000 7f1c84c00640 1 -- 172.21.15.50:0/3230693933 wait complete. 2024-09-21T06:09:36.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.641+0000 7f1c84c00640 1 Processor -- start 2024-09-21T06:09:36.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.641+0000 7f1c84c00640 1 -- start start 2024-09-21T06:09:36.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c84c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c800b4de0 0x7f1c8014f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:36.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c84c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1c800b5c70 0x7f1c800ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c84c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c801527b0 0x7f1c80153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c84c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c80168ca0 con 0x7f1c800b4de0 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c84c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1c80168b20 con 0x7f1c801527b0 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c84c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1c80168e20 con 0x7f1c800b5c70 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c7de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c800b4de0 0x7f1c8014f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c7de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c800b4de0 0x7f1c8014f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52848/0 (socket says 172.21.15.50:52848) 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c7de00640 1 -- 172.21.15.50:0/1865662577 learned_addr learned my addr 172.21.15.50:0/1865662577 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c75400640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1c800b5c70 0x7f1c800ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c7e800640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c801527b0 0x7f1c80153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c7de00640 1 -- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1c800b5c70 msgr2=0x7f1c800ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c7de00640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1c800b5c70 0x7f1c800ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c7de00640 1 -- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c801527b0 msgr2=0x7f1c80153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:36.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c7de00640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c801527b0 0x7f1c80153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c7de00640 1 -- 172.21.15.50:0/1865662577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c800cec20 con 0x7f1c800b4de0 2024-09-21T06:09:36.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c7de00640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c800b4de0 0x7f1c8014f910 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f1c700b0040 tx=0x7f1c700040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:36.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c76a00640 1 -- 172.21.15.50:0/1865662577 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c700b2040 con 0x7f1c800b4de0 2024-09-21T06:09:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c76a00640 1 -- 172.21.15.50:0/1865662577 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c700044b0 con 0x7f1c800b4de0 2024-09-21T06:09:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c76a00640 1 -- 172.21.15.50:0/1865662577 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70003860 con 0x7f1c800b4de0 2024-09-21T06:09:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c84c00640 1 -- 172.21.15.50:0/1865662577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c800ce660 con 0x7f1c800b4de0 2024-09-21T06:09:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c84c00640 1 -- 172.21.15.50:0/1865662577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c80152d80 con 0x7f1c800b4de0 2024-09-21T06:09:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.645+0000 7f1c84c00640 1 -- 172.21.15.50:0/1865662577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c8015bb60 con 0x7f1c800b4de0 2024-09-21T06:09:36.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.649+0000 7f1c76a00640 1 -- 172.21.15.50:0/1865662577 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f1c70003a00 con 0x7f1c800b4de0 2024-09-21T06:09:36.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.649+0000 7f1c76a00640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1c5c074ca0 0x7f1c5c0771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:36.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.649+0000 7f1c76a00640 1 -- 172.21.15.50:0/1865662577 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5879+0+0 (secure 0 0 0) 0x7f1c7012d450 con 0x7f1c800b4de0 2024-09-21T06:09:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.649+0000 7f1c75400640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1c5c074ca0 0x7f1c5c0771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.649+0000 7f1c75400640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1c5c074ca0 0x7f1c5c0771b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1c800b27c0 tx=0x7f1c6800a250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:36.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.653+0000 7f1c76a00640 1 -- 172.21.15.50:0/1865662577 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1c80075580 con 0x7f1c800b4de0 2024-09-21T06:09:36.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:36.833+0000 7f1c84c00640 1 -- 172.21.15.50:0/1865662577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1c8014fe50 con 0x7f1c800b4de0 2024-09-21T06:09:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.453+0000 7f1c76a00640 1 -- 172.21.15.50:0/1865662577 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7f1c700164d0 con 0x7f1c800b4de0 2024-09-21T06:09:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:09:37.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.457+0000 7f1c84c00640 1 -- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1c5c074ca0 msgr2=0x7f1c5c0771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:37.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.457+0000 7f1c84c00640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1c5c074ca0 0x7f1c5c0771b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1c800b27c0 tx=0x7f1c6800a250 comp rx=0 tx=0).stop 2024-09-21T06:09:37.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.457+0000 7f1c84c00640 1 -- 172.21.15.50:0/1865662577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c800b4de0 msgr2=0x7f1c8014f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:37.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.457+0000 7f1c84c00640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c800b4de0 0x7f1c8014f910 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f1c700b0040 tx=0x7f1c700040c0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.457+0000 7f1c84c00640 1 -- 172.21.15.50:0/1865662577 shutdown_connections 2024-09-21T06:09:37.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.457+0000 7f1c84c00640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c801527b0 0x7f1c80153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.457+0000 7f1c84c00640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1c800b5c70 0x7f1c800ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.457+0000 7f1c84c00640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1c5c074ca0 0x7f1c5c0771b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.457+0000 7f1c84c00640 1 --2- 172.21.15.50:0/1865662577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c800b4de0 0x7f1c8014f910 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.457+0000 7f1c84c00640 1 -- 172.21.15.50:0/1865662577 >> 172.21.15.50:0/1865662577 conn(0x7f1c800b2040 msgr2=0x7f1c80159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:37.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.461+0000 7f1c84c00640 1 -- 172.21.15.50:0/1865662577 shutdown_connections 2024-09-21T06:09:37.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.461+0000 7f1c84c00640 1 -- 172.21.15.50:0/1865662577 wait complete. 2024-09-21T06:09:37.486 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:09:37.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.653+0000 7f4298c00640 1 -- 172.21.15.50:0/1072613385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429415cc60 msgr2=0x7f429415d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:37.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.653+0000 7f4298c00640 1 --2- 172.21.15.50:0/1072613385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429415cc60 0x7f429415d070 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f428c009cb0 tx=0x7f428c01cd70 comp rx=0 tx=0).stop 2024-09-21T06:09:37.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.653+0000 7f4298c00640 1 -- 172.21.15.50:0/1072613385 shutdown_connections 2024-09-21T06:09:37.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.653+0000 7f4298c00640 1 --2- 172.21.15.50:0/1072613385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429415cc60 0x7f429415d070 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.653+0000 7f4298c00640 1 --2- 172.21.15.50:0/1072613385 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f42940738b0 0x7f4294075010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.653+0000 7f4298c00640 1 -- 172.21.15.50:0/1072613385 >> 172.21.15.50:0/1072613385 conn(0x7f42940b2040 msgr2=0x7f42940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:37.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.653+0000 7f4298c00640 1 -- 172.21.15.50:0/1072613385 shutdown_connections 2024-09-21T06:09:37.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.653+0000 7f4298c00640 1 -- 172.21.15.50:0/1072613385 wait complete. 2024-09-21T06:09:37.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4298c00640 1 Processor -- start 2024-09-21T06:09:37.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4298c00640 1 -- start start 2024-09-21T06:09:37.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4298c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42940738b0 0x7f42940ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:37.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4298c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42940b4de0 0x7f42941f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:37.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4298c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f429415cc60 0x7f42940cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:37.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4298c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4294168eb0 con 0x7f42940b4de0 2024-09-21T06:09:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4298c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4294168d30 con 0x7f42940738b0 2024-09-21T06:09:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4298c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4294169030 con 0x7f429415cc60 2024-09-21T06:09:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4291400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42940b4de0 0x7f42941f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4291400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42940b4de0 0x7f42941f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34474/0 (socket says 172.21.15.50:34474) 2024-09-21T06:09:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4291400640 1 -- 172.21.15.50:0/1782407763 learned_addr learned my addr 172.21.15.50:0/1782407763 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4292800640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f429415cc60 0x7f42940cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4291e00640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42940738b0 0x7f42940ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4291400640 1 -- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f429415cc60 msgr2=0x7f42940cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4291400640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f429415cc60 0x7f42940cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4291400640 1 -- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42940738b0 msgr2=0x7f42940ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4291400640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42940738b0 0x7f42940ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4291400640 1 -- 172.21.15.50:0/1782407763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42940cd740 con 0x7f42940b4de0 2024-09-21T06:09:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4291400640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42940b4de0 0x7f42941f6b70 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f428000c3c0 tx=0x7f428000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f428a000640 1 -- 172.21.15.50:0/1782407763 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280017070 con 0x7f42940b4de0 2024-09-21T06:09:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f428a000640 1 -- 172.21.15.50:0/1782407763 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42800053c0 con 0x7f42940b4de0 2024-09-21T06:09:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f428a000640 1 -- 172.21.15.50:0/1782407763 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280003860 con 0x7f42940b4de0 2024-09-21T06:09:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4298c00640 1 -- 172.21.15.50:0/1782407763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42940cda20 con 0x7f42940b4de0 2024-09-21T06:09:37.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4298c00640 1 -- 172.21.15.50:0/1782407763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42940cdfe0 con 0x7f42940b4de0 2024-09-21T06:09:37.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.657+0000 7f4298c00640 1 -- 172.21.15.50:0/1782407763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4294075010 con 0x7f42940b4de0 2024-09-21T06:09:37.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.661+0000 7f428a000640 1 -- 172.21.15.50:0/1782407763 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f428001a070 con 0x7f42940b4de0 2024-09-21T06:09:37.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.661+0000 7f428a000640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4274074c70 0x7f4274077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:37.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.661+0000 7f428a000640 1 -- 172.21.15.50:0/1782407763 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5441+0+0 (secure 0 0 0) 0x7f4280092890 con 0x7f42940b4de0 2024-09-21T06:09:37.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.661+0000 7f4291e00640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4274074c70 0x7f4274077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:37.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.665+0000 7f4291e00640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4274074c70 0x7f4274077180 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f42940b6600 tx=0x7f428400b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:37.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.665+0000 7f428a000640 1 -- 172.21.15.50:0/1782407763 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4280021020 con 0x7f42940b4de0 2024-09-21T06:09:37.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:37.841+0000 7f4298c00640 1 -- 172.21.15.50:0/1782407763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f42940b63c0 con 0x7f42940b4de0 2024-09-21T06:09:38.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.457+0000 7f428a000640 1 -- 172.21.15.50:0/1782407763 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7f428005ecc0 con 0x7f42940b4de0 2024-09-21T06:09:38.457 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-09-21T06:09:38.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.461+0000 7f4298c00640 1 -- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4274074c70 msgr2=0x7f4274077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.461+0000 7f4298c00640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4274074c70 0x7f4274077180 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f42940b6600 tx=0x7f428400b2e0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.461+0000 7f4298c00640 1 -- 172.21.15.50:0/1782407763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42940b4de0 msgr2=0x7f42941f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.461+0000 7f4298c00640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42940b4de0 0x7f42941f6b70 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f428000c3c0 tx=0x7f428000c880 comp rx=0 tx=0).stop 2024-09-21T06:09:38.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.461+0000 7f4298c00640 1 -- 172.21.15.50:0/1782407763 shutdown_connections 2024-09-21T06:09:38.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.461+0000 7f4298c00640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f429415cc60 0x7f42940cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.461+0000 7f4298c00640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4274074c70 0x7f4274077180 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.461+0000 7f4298c00640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42940b4de0 0x7f42941f6b70 secure :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f428000c3c0 tx=0x7f428000c880 comp rx=0 tx=0).stop 2024-09-21T06:09:38.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.461+0000 7f4298c00640 1 --2- 172.21.15.50:0/1782407763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42940738b0 0x7f42940ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.461+0000 7f4298c00640 1 -- 172.21.15.50:0/1782407763 >> 172.21.15.50:0/1782407763 conn(0x7f42940b2040 msgr2=0x7f42940b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:38.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.461+0000 7f4298c00640 1 -- 172.21.15.50:0/1782407763 shutdown_connections 2024-09-21T06:09:38.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.461+0000 7f4298c00640 1 -- 172.21.15.50:0/1782407763 wait complete. 2024-09-21T06:09:38.487 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.657+0000 7f3046a00640 1 -- 172.21.15.50:0/947370300 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30400b54f0 msgr2=0x7f304015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.657+0000 7f3046a00640 1 --2- 172.21.15.50:0/947370300 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30400b54f0 0x7f304015bb60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3038009bf0 tx=0x7f303801cbb0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.657+0000 7f3046a00640 1 -- 172.21.15.50:0/947370300 shutdown_connections 2024-09-21T06:09:38.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.657+0000 7f3046a00640 1 --2- 172.21.15.50:0/947370300 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30400b54f0 0x7f304015bb60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.657+0000 7f3046a00640 1 -- 172.21.15.50:0/947370300 >> 172.21.15.50:0/947370300 conn(0x7f30400b2040 msgr2=0x7f30400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:38.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.657+0000 7f3046a00640 1 -- 172.21.15.50:0/947370300 shutdown_connections 2024-09-21T06:09:38.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.657+0000 7f3046a00640 1 -- 172.21.15.50:0/947370300 wait complete. 2024-09-21T06:09:38.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f3046a00640 1 Processor -- start 2024-09-21T06:09:38.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f3046a00640 1 -- start start 2024-09-21T06:09:38.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f3046a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30400b5c70 0x7f304014ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:38.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f3046a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30401527b0 0x7f30401535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f3046a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30401f8140 0x7f3040153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f3046a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3040169bd0 con 0x7f30400b5c70 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f3046a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3040169a50 con 0x7f30401527b0 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f3046a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3040169d50 con 0x7f30401f8140 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f303fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30400b5c70 0x7f304014ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f303fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30400b5c70 0x7f304014ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34484/0 (socket says 172.21.15.50:34484) 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f303fe00640 1 -- 172.21.15.50:0/1031367747 learned_addr learned my addr 172.21.15.50:0/1031367747 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f3044a00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30401f8140 0x7f3040153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f3037e00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30401527b0 0x7f30401535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f303fe00640 1 -- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30401f8140 msgr2=0x7f3040153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f303fe00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30401f8140 0x7f3040153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f303fe00640 1 -- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30401527b0 msgr2=0x7f30401535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f303fe00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30401527b0 0x7f30401535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f303fe00640 1 -- 172.21.15.50:0/1031367747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304014ffd0 con 0x7f30400b5c70 2024-09-21T06:09:38.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f303fe00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30400b5c70 0x7f304014ef50 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f303000c430 tx=0x7f303000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:38.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f303cc00640 1 -- 172.21.15.50:0/1031367747 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030011850 con 0x7f30400b5c70 2024-09-21T06:09:38.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f3046a00640 1 -- 172.21.15.50:0/1031367747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30400ccaf0 con 0x7f30400b5c70 2024-09-21T06:09:38.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f303cc00640 1 -- 172.21.15.50:0/1031367747 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3030017070 con 0x7f30400b5c70 2024-09-21T06:09:38.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f303cc00640 1 -- 172.21.15.50:0/1031367747 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030011da0 con 0x7f30400b5c70 2024-09-21T06:09:38.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.661+0000 7f3046a00640 1 -- 172.21.15.50:0/1031367747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30400cd0b0 con 0x7f30400b5c70 2024-09-21T06:09:38.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.665+0000 7f3046a00640 1 -- 172.21.15.50:0/1031367747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304015bb60 con 0x7f30400b5c70 2024-09-21T06:09:38.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.665+0000 7f303cc00640 1 -- 172.21.15.50:0/1031367747 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f303001c460 con 0x7f30400b5c70 2024-09-21T06:09:38.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.665+0000 7f303cc00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f301c074c70 0x7f301c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:38.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.665+0000 7f303cc00640 1 -- 172.21.15.50:0/1031367747 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7f3030092ea0 con 0x7f30400b5c70 2024-09-21T06:09:38.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.665+0000 7f3037e00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f301c074c70 0x7f301c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:38.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.669+0000 7f3037e00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f301c074c70 0x7f301c077180 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3028000c10 tx=0x7f3028009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:38.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.669+0000 7f303cc00640 1 -- 172.21.15.50:0/1031367747 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f303005f250 con 0x7f30400b5c70 2024-09-21T06:09:38.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.845+0000 7f3046a00640 1 -- 172.21.15.50:0/1031367747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f30401597c0 con 0x7f30400b5c70 2024-09-21T06:09:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.849+0000 7f303cc00640 1 -- 172.21.15.50:0/1031367747 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v51) ==== 71+0+0 (secure 0 0 0) 0x7f3030063c80 con 0x7f30400b5c70 2024-09-21T06:09:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.853+0000 7f3046a00640 1 -- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f301c074c70 msgr2=0x7f301c077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.853+0000 7f3046a00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f301c074c70 0x7f301c077180 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3028000c10 tx=0x7f3028009040 comp rx=0 tx=0).stop 2024-09-21T06:09:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.853+0000 7f3046a00640 1 -- 172.21.15.50:0/1031367747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30400b5c70 msgr2=0x7f304014ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.853+0000 7f3046a00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30400b5c70 0x7f304014ef50 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f303000c430 tx=0x7f303000c8f0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.853+0000 7f3046a00640 1 -- 172.21.15.50:0/1031367747 shutdown_connections 2024-09-21T06:09:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.853+0000 7f3046a00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30401f8140 0x7f3040153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.853+0000 7f3046a00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30401527b0 0x7f30401535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.853+0000 7f3046a00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f301c074c70 0x7f301c077180 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.853+0000 7f3046a00640 1 --2- 172.21.15.50:0/1031367747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30400b5c70 0x7f304014ef50 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.853+0000 7f3046a00640 1 -- 172.21.15.50:0/1031367747 >> 172.21.15.50:0/1031367747 conn(0x7f30400b2040 msgr2=0x7f3040151550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.853+0000 7f3046a00640 1 -- 172.21.15.50:0/1031367747 shutdown_connections 2024-09-21T06:09:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:38.853+0000 7f3046a00640 1 -- 172.21.15.50:0/1031367747 wait complete. 2024-09-21T06:09:38.875 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-21T06:09:39.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.033+0000 7fd7d0200640 1 -- 172.21.15.50:0/1631444359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c80b5c70 msgr2=0x7fd7c815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:39.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.033+0000 7fd7d0200640 1 --2- 172.21.15.50:0/1631444359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c80b5c70 0x7fd7c815bb60 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009900 tx=0x7fd7bc01c6d0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.033+0000 7fd7d0200640 1 -- 172.21.15.50:0/1631444359 shutdown_connections 2024-09-21T06:09:39.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.033+0000 7fd7d0200640 1 --2- 172.21.15.50:0/1631444359 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7c80b4de0 0x7fd7c80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.033+0000 7fd7d0200640 1 --2- 172.21.15.50:0/1631444359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c80b5c70 0x7fd7c815bb60 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.033+0000 7fd7d0200640 1 -- 172.21.15.50:0/1631444359 >> 172.21.15.50:0/1631444359 conn(0x7fd7c80b2040 msgr2=0x7fd7c80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:39.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.033+0000 7fd7d0200640 1 -- 172.21.15.50:0/1631444359 shutdown_connections 2024-09-21T06:09:39.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.033+0000 7fd7d0200640 1 -- 172.21.15.50:0/1631444359 wait complete. 2024-09-21T06:09:39.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7d0200640 1 Processor -- start 2024-09-21T06:09:39.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7d0200640 1 -- start start 2024-09-21T06:09:39.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7d0200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7c80b4de0 0x7fd7c813c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:39.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7d0200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7c80b5c70 0x7fd7c80aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:39.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7d0200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c81527b0 0x7fd7c8151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:39.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7d0200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8168980 con 0x7fd7c81527b0 2024-09-21T06:09:39.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7d0200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8168800 con 0x7fd7c80b5c70 2024-09-21T06:09:39.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7d0200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c8168b00 con 0x7fd7c80b4de0 2024-09-21T06:09:39.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7ce200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c81527b0 0x7fd7c8151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7ce200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c81527b0 0x7fd7c8151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34512/0 (socket says 172.21.15.50:34512) 2024-09-21T06:09:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7ce200640 1 -- 172.21.15.50:0/2178093478 learned_addr learned my addr 172.21.15.50:0/2178093478 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7cce00640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7c80b5c70 0x7fd7c80aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7cd800640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7c80b4de0 0x7fd7c813c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7ce200640 1 -- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7c80b4de0 msgr2=0x7fd7c813c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7ce200640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7c80b4de0 0x7fd7c813c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7ce200640 1 -- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7c80b5c70 msgr2=0x7fd7c80aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7ce200640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7c80b5c70 0x7fd7c80aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7ce200640 1 -- 172.21.15.50:0/2178093478 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c813fcc0 con 0x7fd7c81527b0 2024-09-21T06:09:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7ce200640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c81527b0 0x7fd7c8151a40 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000cab0 tx=0x7fd7c000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7c5600640 1 -- 172.21.15.50:0/2178093478 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000ae20 con 0x7fd7c81527b0 2024-09-21T06:09:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7c5600640 1 -- 172.21.15.50:0/2178093478 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7c000f560 con 0x7fd7c81527b0 2024-09-21T06:09:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7d0200640 1 -- 172.21.15.50:0/2178093478 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c81f9c80 con 0x7fd7c81527b0 2024-09-21T06:09:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7c5600640 1 -- 172.21.15.50:0/2178093478 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000f9b0 con 0x7fd7c81527b0 2024-09-21T06:09:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.037+0000 7fd7d0200640 1 -- 172.21.15.50:0/2178093478 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c81fa240 con 0x7fd7c81527b0 2024-09-21T06:09:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.041+0000 7fd7d0200640 1 -- 172.21.15.50:0/2178093478 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c80af760 con 0x7fd7c81527b0 2024-09-21T06:09:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.041+0000 7fd7c5600640 1 -- 172.21.15.50:0/2178093478 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fd7c000fb50 con 0x7fd7c81527b0 2024-09-21T06:09:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.041+0000 7fd7c5600640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7a8074c70 0x7fd7a8077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.041+0000 7fd7c5600640 1 -- 172.21.15.50:0/2178093478 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7fd7c0093dc0 con 0x7fd7c81527b0 2024-09-21T06:09:39.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.041+0000 7fd7cd800640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7a8074c70 0x7fd7a8077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:39.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.045+0000 7fd7cd800640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7a8074c70 0x7fd7a8077180 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0af000 tx=0x7fd7bc01af10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:39.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.045+0000 7fd7c5600640 1 -- 172.21.15.50:0/2178093478 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd7c0014030 con 0x7fd7c81527b0 2024-09-21T06:09:39.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.285+0000 7fd7d0200640 1 -- 172.21.15.50:0/2178093478 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7fd7c813cd80 con 0x7fd7c81527b0 2024-09-21T06:09:39.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.465+0000 7fd7c5600640 1 -- 172.21.15.50:0/2178093478 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v587) ==== 178+0+0 (secure 0 0 0) 0x7fd7c0060300 con 0x7fd7c81527b0 2024-09-21T06:09:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.469+0000 7fd7d0200640 1 -- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7a8074c70 msgr2=0x7fd7a8077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.469+0000 7fd7d0200640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7a8074c70 0x7fd7a8077180 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0af000 tx=0x7fd7bc01af10 comp rx=0 tx=0).stop 2024-09-21T06:09:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.469+0000 7fd7d0200640 1 -- 172.21.15.50:0/2178093478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c81527b0 msgr2=0x7fd7c8151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.469+0000 7fd7d0200640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c81527b0 0x7fd7c8151a40 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000cab0 tx=0x7fd7c000cf70 comp rx=0 tx=0).stop 2024-09-21T06:09:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.469+0000 7fd7d0200640 1 -- 172.21.15.50:0/2178093478 shutdown_connections 2024-09-21T06:09:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.469+0000 7fd7d0200640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7a8074c70 0x7fd7a8077180 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.469+0000 7fd7d0200640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c81527b0 0x7fd7c8151a40 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.469+0000 7fd7d0200640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7c80b5c70 0x7fd7c80aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.469+0000 7fd7d0200640 1 --2- 172.21.15.50:0/2178093478 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7c80b4de0 0x7fd7c813c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.469+0000 7fd7d0200640 1 -- 172.21.15.50:0/2178093478 >> 172.21.15.50:0/2178093478 conn(0x7fd7c80b2040 msgr2=0x7fd7c8158190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.469+0000 7fd7d0200640 1 -- 172.21.15.50:0/2178093478 shutdown_connections 2024-09-21T06:09:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.469+0000 7fd7d0200640 1 -- 172.21.15.50:0/2178093478 wait complete. 2024-09-21T06:09:39.491 INFO:tasks.cephfs_test_runner:Test that the command "ceph fs clone status" prints progress stats ... ok 2024-09-21T06:09:39.491 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress bars, that printed for 3 ongoing clone jobs, are 2024-09-21T06:09:39.491 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-21T06:09:39.491 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:09:39.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.649+0000 7fa498800640 1 -- 172.21.15.50:0/163876570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa490160d50 msgr2=0x7fa490074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:39.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.649+0000 7fa498800640 1 --2- 172.21.15.50:0/163876570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa490160d50 0x7fa490074410 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa480009840 tx=0x7fa48001c820 comp rx=0 tx=0).stop 2024-09-21T06:09:39.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.649+0000 7fa498800640 1 -- 172.21.15.50:0/163876570 shutdown_connections 2024-09-21T06:09:39.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.649+0000 7fa498800640 1 --2- 172.21.15.50:0/163876570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa490160d50 0x7fa490074410 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.649+0000 7fa498800640 1 --2- 172.21.15.50:0/163876570 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa490157030 0x7fa4900b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.649+0000 7fa498800640 1 -- 172.21.15.50:0/163876570 >> 172.21.15.50:0/163876570 conn(0x7fa4900708c0 msgr2=0x7fa490070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:39.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.649+0000 7fa498800640 1 -- 172.21.15.50:0/163876570 shutdown_connections 2024-09-21T06:09:39.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.649+0000 7fa498800640 1 -- 172.21.15.50:0/163876570 wait complete. 2024-09-21T06:09:39.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa498800640 1 Processor -- start 2024-09-21T06:09:39.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa498800640 1 -- start start 2024-09-21T06:09:39.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa498800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa490157030 0x7fa4900b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:39.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa498800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa490160d50 0x7fa4900b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:39.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa498800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa490161160 0x7fa49014de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:39.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa498800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa49016a820 con 0x7fa490157030 2024-09-21T06:09:39.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa498800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa49016a6a0 con 0x7fa490160d50 2024-09-21T06:09:39.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa498800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa49016a9a0 con 0x7fa490161160 2024-09-21T06:09:39.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa495e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa490157030 0x7fa4900b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa495e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa490157030 0x7fa4900b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34542/0 (socket says 172.21.15.50:34542) 2024-09-21T06:09:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa495e00640 1 -- 172.21.15.50:0/3692515666 learned_addr learned my addr 172.21.15.50:0/3692515666 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa495400640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa490160d50 0x7fa4900b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa496800640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa490161160 0x7fa49014de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa495e00640 1 -- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa490161160 msgr2=0x7fa49014de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa495e00640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa490161160 0x7fa49014de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa495e00640 1 -- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa490160d50 msgr2=0x7fa4900b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa495e00640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa490160d50 0x7fa4900b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa495e00640 1 -- 172.21.15.50:0/3692515666 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4900b4b50 con 0x7fa490157030 2024-09-21T06:09:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa496800640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa490161160 0x7fa49014de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa495e00640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa490157030 0x7fa4900b2120 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa48400d750 tx=0x7fa48400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa48a000640 1 -- 172.21.15.50:0/3692515666 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484017070 con 0x7fa490157030 2024-09-21T06:09:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa48a000640 1 -- 172.21.15.50:0/3692515666 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa484009d20 con 0x7fa490157030 2024-09-21T06:09:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa48a000640 1 -- 172.21.15.50:0/3692515666 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48400de60 con 0x7fa490157030 2024-09-21T06:09:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa498800640 1 -- 172.21.15.50:0/3692515666 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4900b4df0 con 0x7fa490157030 2024-09-21T06:09:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.653+0000 7fa498800640 1 -- 172.21.15.50:0/3692515666 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49006d250 con 0x7fa490157030 2024-09-21T06:09:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.657+0000 7fa498800640 1 -- 172.21.15.50:0/3692515666 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4900b5030 con 0x7fa490157030 2024-09-21T06:09:39.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.657+0000 7fa48a000640 1 -- 172.21.15.50:0/3692515666 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fa484004280 con 0x7fa490157030 2024-09-21T06:09:39.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.657+0000 7fa48a000640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa474074c70 0x7fa474077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:39.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.657+0000 7fa48a000640 1 -- 172.21.15.50:0/3692515666 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7fa484093460 con 0x7fa490157030 2024-09-21T06:09:39.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.661+0000 7fa495400640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa474074c70 0x7fa474077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:39.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.661+0000 7fa495400640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa474074c70 0x7fa474077180 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa4800af000 tx=0x7fa48001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.665+0000 7fa48a000640 1 -- 172.21.15.50:0/3692515666 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa484020020 con 0x7fa490157030 2024-09-21T06:09:39.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 -- 172.21.15.50:0/3692515666 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4900d2260 con 0x7fa490157030 2024-09-21T06:09:39.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa48a000640 1 -- 172.21.15.50:0/3692515666 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+13119 (secure 0 0 0) 0x7fa48405fee0 con 0x7fa490157030 2024-09-21T06:09:39.841 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:39.842 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":104,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:09:38.460041+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6834/1418616207":"2024-09-22T06:09:35.340740+0000","172.21.15.50:6832/1418616207":"2024-09-22T06:09:35.340740+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:09:39.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 -- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa474074c70 msgr2=0x7fa474077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:39.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa474074c70 0x7fa474077180 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa4800af000 tx=0x7fa48001b060 comp rx=0 tx=0).stop 2024-09-21T06:09:39.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 -- 172.21.15.50:0/3692515666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa490157030 msgr2=0x7fa4900b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:39.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa490157030 0x7fa4900b2120 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa48400d750 tx=0x7fa48400dc10 comp rx=0 tx=0).stop 2024-09-21T06:09:39.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 -- 172.21.15.50:0/3692515666 shutdown_connections 2024-09-21T06:09:39.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa490161160 0x7fa49014de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa490160d50 0x7fa4900b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa474074c70 0x7fa474077180 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 --2- 172.21.15.50:0/3692515666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa490157030 0x7fa4900b2120 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:39.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 -- 172.21.15.50:0/3692515666 >> 172.21.15.50:0/3692515666 conn(0x7fa4900708c0 msgr2=0x7fa4900d1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:39.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 -- 172.21.15.50:0/3692515666 shutdown_connections 2024-09-21T06:09:39.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:39.841+0000 7fa498800640 1 -- 172.21.15.50:0/3692515666 wait complete. 2024-09-21T06:09:39.865 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:56:18.921197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:09:39.865 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:09:40.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.025+0000 7fa968400640 1 -- 172.21.15.50:0/3074526197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9600749a0 msgr2=0x7fa960074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.025+0000 7fa968400640 1 --2- 172.21.15.50:0/3074526197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9600749a0 0x7fa960074db0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fa96015a930 tx=0x7fa95401c8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.025+0000 7fa968400640 1 -- 172.21.15.50:0/3074526197 shutdown_connections 2024-09-21T06:09:40.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.025+0000 7fa968400640 1 --2- 172.21.15.50:0/3074526197 >> v2:172.21.15.88:3300/0 conn(0x7fa960160330 0x7fa960160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.025+0000 7fa968400640 1 --2- 172.21.15.50:0/3074526197 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa9600cefa0 0x7fa9600cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.025+0000 7fa968400640 1 --2- 172.21.15.50:0/3074526197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9600749a0 0x7fa960074db0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.025+0000 7fa968400640 1 -- 172.21.15.50:0/3074526197 >> 172.21.15.50:0/3074526197 conn(0x7fa9600b2040 msgr2=0x7fa9600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:40.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.025+0000 7fa968400640 1 -- 172.21.15.50:0/3074526197 shutdown_connections 2024-09-21T06:09:40.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.025+0000 7fa968400640 1 -- 172.21.15.50:0/3074526197 wait complete. 2024-09-21T06:09:40.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa968400640 1 Processor -- start 2024-09-21T06:09:40.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa968400640 1 -- start start 2024-09-21T06:09:40.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa968400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa9600749a0 0x7fa9601f9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:40.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa968400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9600cefa0 0x7fa96013f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:40.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa968400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa960160330 0x7fa96015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa968400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa9600b3ee0 con 0x7fa960160330 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa968400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9600b3d60 con 0x7fa9600cefa0 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa968400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa9600b4060 con 0x7fa9600749a0 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa966400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa960160330 0x7fa96015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa966400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa960160330 0x7fa96015f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34554/0 (socket says 172.21.15.50:34554) 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa966400640 1 -- 172.21.15.50:0/2893037077 learned_addr learned my addr 172.21.15.50:0/2893037077 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa965a00640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa9600749a0 0x7fa9601f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa965000640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9600cefa0 0x7fa96013f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa966400640 1 -- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa9600749a0 msgr2=0x7fa9601f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa966400640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa9600749a0 0x7fa9601f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa966400640 1 -- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9600cefa0 msgr2=0x7fa96013f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa966400640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9600cefa0 0x7fa96013f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.029+0000 7fa966400640 1 -- 172.21.15.50:0/2893037077 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9600b0420 con 0x7fa960160330 2024-09-21T06:09:40.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.033+0000 7fa966400640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa960160330 0x7fa96015f3d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fa95800eae0 tx=0x7fa958009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:40.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.033+0000 7fa95d600640 1 -- 172.21.15.50:0/2893037077 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95800ce40 con 0x7fa960160330 2024-09-21T06:09:40.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.033+0000 7fa95d600640 1 -- 172.21.15.50:0/2893037077 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa958004c20 con 0x7fa960160330 2024-09-21T06:09:40.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.033+0000 7fa95d600640 1 -- 172.21.15.50:0/2893037077 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958005020 con 0x7fa960160330 2024-09-21T06:09:40.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.033+0000 7fa968400640 1 -- 172.21.15.50:0/2893037077 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9600aefa0 con 0x7fa960160330 2024-09-21T06:09:40.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.033+0000 7fa968400640 1 -- 172.21.15.50:0/2893037077 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9600ad9f0 con 0x7fa960160330 2024-09-21T06:09:40.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.033+0000 7fa968400640 1 -- 172.21.15.50:0/2893037077 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9600adc50 con 0x7fa960160330 2024-09-21T06:09:40.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.033+0000 7fa95d600640 1 -- 172.21.15.50:0/2893037077 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fa9580051c0 con 0x7fa960160330 2024-09-21T06:09:40.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.037+0000 7fa95d600640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa940074ca0 0x7fa9400771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:40.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.037+0000 7fa95d600640 1 -- 172.21.15.50:0/2893037077 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7fa958093d30 con 0x7fa960160330 2024-09-21T06:09:40.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.037+0000 7fa965a00640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa940074ca0 0x7fa9400771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:40.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.037+0000 7fa965a00640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa940074ca0 0x7fa9400771b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa9540af000 tx=0x7fa95401b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:40.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.041+0000 7fa95d600640 1 -- 172.21.15.50:0/2893037077 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa9580601f0 con 0x7fa960160330 2024-09-21T06:09:40.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.217+0000 7fa968400640 1 -- 172.21.15.50:0/2893037077 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa9600cf3b0 con 0x7fa960160330 2024-09-21T06:09:40.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.217+0000 7fa95d600640 1 -- 172.21.15.50:0/2893037077 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7fa958064c20 con 0x7fa960160330 2024-09-21T06:09:40.218 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-09-21T06:09:40.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.221+0000 7fa968400640 1 -- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa940074ca0 msgr2=0x7fa9400771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.221+0000 7fa968400640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa940074ca0 0x7fa9400771b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa9540af000 tx=0x7fa95401b100 comp rx=0 tx=0).stop 2024-09-21T06:09:40.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.221+0000 7fa968400640 1 -- 172.21.15.50:0/2893037077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa960160330 msgr2=0x7fa96015f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.221+0000 7fa968400640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa960160330 0x7fa96015f3d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fa95800eae0 tx=0x7fa958009f70 comp rx=0 tx=0).stop 2024-09-21T06:09:40.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.221+0000 7fa968400640 1 -- 172.21.15.50:0/2893037077 shutdown_connections 2024-09-21T06:09:40.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.221+0000 7fa968400640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa940074ca0 0x7fa9400771b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.221+0000 7fa968400640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa960160330 0x7fa96015f3d0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.221+0000 7fa968400640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9600cefa0 0x7fa96013f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.221+0000 7fa968400640 1 --2- 172.21.15.50:0/2893037077 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa9600749a0 0x7fa9601f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.221+0000 7fa968400640 1 -- 172.21.15.50:0/2893037077 >> 172.21.15.50:0/2893037077 conn(0x7fa9600b2040 msgr2=0x7fa9601414c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:40.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.221+0000 7fa968400640 1 -- 172.21.15.50:0/2893037077 shutdown_connections 2024-09-21T06:09:40.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.221+0000 7fa968400640 1 -- 172.21.15.50:0/2893037077 wait complete. 2024-09-21T06:09:40.242 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-21T06:09:40.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.401+0000 7fee97a00640 1 -- 172.21.15.50:0/1735340935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee74003fa0 msgr2=0x7fee74004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.401+0000 7fee97a00640 1 --2- 172.21.15.50:0/1735340935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee74003fa0 0x7fee74004810 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fee90071fd0 tx=0x7fee7801ef70 comp rx=0 tx=0).stop 2024-09-21T06:09:40.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.401+0000 7fee95a00640 1 -- 172.21.15.50:0/1735340935 reap_dead start 2024-09-21T06:09:40.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.401+0000 7fee97a00640 1 -- 172.21.15.50:0/1735340935 shutdown_connections 2024-09-21T06:09:40.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.401+0000 7fee97a00640 1 -- 172.21.15.50:0/1735340935 >> 172.21.15.50:0/1735340935 conn(0x7fee900b2040 msgr2=0x7fee900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:40.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.401+0000 7fee97a00640 1 -- 172.21.15.50:0/1735340935 shutdown_connections 2024-09-21T06:09:40.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.401+0000 7fee97a00640 1 -- 172.21.15.50:0/1735340935 wait complete. 2024-09-21T06:09:40.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee97a00640 1 Processor -- start 2024-09-21T06:09:40.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee97a00640 1 -- start start 2024-09-21T06:09:40.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee97a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee74003fa0 0x7fee90142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:40.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee97a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee90151190 0x7fee9014d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:40.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee97a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee901f6910 0x7fee901f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:40.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee97a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fee9015d040 con 0x7fee90151190 2024-09-21T06:09:40.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee97a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee9015cec0 con 0x7fee901f6910 2024-09-21T06:09:40.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee97a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fee9015d1c0 con 0x7fee74003fa0 2024-09-21T06:09:40.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee87e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee90151190 0x7fee9014d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:40.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee87e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee90151190 0x7fee9014d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34578/0 (socket says 172.21.15.50:34578) 2024-09-21T06:09:40.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee87e00640 1 -- 172.21.15.50:0/3764220994 learned_addr learned my addr 172.21.15.50:0/3764220994 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:40.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee95a00640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee901f6910 0x7fee901f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:40.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee95000640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee74003fa0 0x7fee90142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:40.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee87e00640 1 -- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee74003fa0 msgr2=0x7fee90142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee87e00640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee74003fa0 0x7fee90142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee87e00640 1 -- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee901f6910 msgr2=0x7fee901f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee87e00640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee901f6910 0x7fee901f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee87e00640 1 -- 172.21.15.50:0/3764220994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee901f7260 con 0x7fee90151190 2024-09-21T06:09:40.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee87e00640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee90151190 0x7fee9014d250 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fee780012c0 tx=0x7fee780069d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:40.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee8d600640 1 -- 172.21.15.50:0/3764220994 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780061c0 con 0x7fee90151190 2024-09-21T06:09:40.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee8d600640 1 -- 172.21.15.50:0/3764220994 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee78006450 con 0x7fee90151190 2024-09-21T06:09:40.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee8d600640 1 -- 172.21.15.50:0/3764220994 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780a8d00 con 0x7fee90151190 2024-09-21T06:09:40.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee97a00640 1 -- 172.21.15.50:0/3764220994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee901433f0 con 0x7fee90151190 2024-09-21T06:09:40.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.405+0000 7fee97a00640 1 -- 172.21.15.50:0/3764220994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee901503d0 con 0x7fee90151190 2024-09-21T06:09:40.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.409+0000 7fee97a00640 1 -- 172.21.15.50:0/3764220994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee9014fe60 con 0x7fee90151190 2024-09-21T06:09:40.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.409+0000 7fee8d600640 1 -- 172.21.15.50:0/3764220994 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fee78003600 con 0x7fee90151190 2024-09-21T06:09:40.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.409+0000 7fee8d600640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fee6c074c70 0x7fee6c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:40.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.409+0000 7fee8d600640 1 -- 172.21.15.50:0/3764220994 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7fee781305c0 con 0x7fee90151190 2024-09-21T06:09:40.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.409+0000 7fee95000640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fee6c074c70 0x7fee6c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:40.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.413+0000 7fee95000640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fee6c074c70 0x7fee6c077180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fee80009c60 tx=0x7fee800091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:40.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.413+0000 7fee8d600640 1 -- 172.21.15.50:0/3764220994 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fee901f7260 con 0x7fee90151190 2024-09-21T06:09:40.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:40.661+0000 7fee97a00640 1 -- 172.21.15.50:0/3764220994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7fee900b51f0 con 0x7fee90151190 2024-09-21T06:09:41.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.473+0000 7fee8d600640 1 -- 172.21.15.50:0/3764220994 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v589) ==== 188+0+0 (secure 0 0 0) 0x7fee78018c80 con 0x7fee90151190 2024-09-21T06:09:41.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.477+0000 7fee97a00640 1 -- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fee6c074c70 msgr2=0x7fee6c077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:41.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.477+0000 7fee97a00640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fee6c074c70 0x7fee6c077180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fee80009c60 tx=0x7fee800091e0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.477+0000 7fee97a00640 1 -- 172.21.15.50:0/3764220994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee90151190 msgr2=0x7fee9014d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.477+0000 7fee97a00640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee90151190 0x7fee9014d250 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fee780012c0 tx=0x7fee780069d0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.477+0000 7fee97a00640 1 -- 172.21.15.50:0/3764220994 shutdown_connections 2024-09-21T06:09:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.477+0000 7fee97a00640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee901f6910 0x7fee901f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.477+0000 7fee97a00640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fee6c074c70 0x7fee6c077180 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.477+0000 7fee97a00640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee90151190 0x7fee9014d250 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.477+0000 7fee97a00640 1 --2- 172.21.15.50:0/3764220994 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee74003fa0 0x7fee90142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.477+0000 7fee97a00640 1 -- 172.21.15.50:0/3764220994 >> 172.21.15.50:0/3764220994 conn(0x7fee900b2040 msgr2=0x7fee900af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.477+0000 7fee97a00640 1 -- 172.21.15.50:0/3764220994 shutdown_connections 2024-09-21T06:09:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.477+0000 7fee97a00640 1 -- 172.21.15.50:0/3764220994 wait complete. 2024-09-21T06:09:41.503 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:09:41.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.665+0000 7f0191a00640 1 -- 172.21.15.50:0/115443205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018c157030 msgr2=0x7f018c0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:41.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.665+0000 7f0191a00640 1 --2- 172.21.15.50:0/115443205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018c157030 0x7f018c0b5a80 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f017c009840 tx=0x7f017c01c790 comp rx=0 tx=0).stop 2024-09-21T06:09:41.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.665+0000 7f0191a00640 1 -- 172.21.15.50:0/115443205 shutdown_connections 2024-09-21T06:09:41.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.665+0000 7f0191a00640 1 --2- 172.21.15.50:0/115443205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018c157030 0x7f018c0b5a80 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.665+0000 7f0191a00640 1 -- 172.21.15.50:0/115443205 >> 172.21.15.50:0/115443205 conn(0x7f018c0708c0 msgr2=0x7f018c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:41.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.665+0000 7f0191a00640 1 -- 172.21.15.50:0/115443205 shutdown_connections 2024-09-21T06:09:41.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.665+0000 7f0191a00640 1 -- 172.21.15.50:0/115443205 wait complete. 2024-09-21T06:09:41.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0191a00640 1 Processor -- start 2024-09-21T06:09:41.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0191a00640 1 -- start start 2024-09-21T06:09:41.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0191a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f018c157030 0x7f018c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:41.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0191a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018c160d50 0x7f018c15cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:41.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0191a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f018c135bb0 0x7f018c135fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:41.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0191a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f018c16b8a0 con 0x7f018c160d50 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0191a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f018c16b720 con 0x7f018c157030 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0191a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f018c16ba20 con 0x7f018c135bb0 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0182000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018c160d50 0x7f018c15cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0182000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018c160d50 0x7f018c15cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34596/0 (socket says 172.21.15.50:34596) 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0182000640 1 -- 172.21.15.50:0/3908452493 learned_addr learned my addr 172.21.15.50:0/3908452493 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f018aa00640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f018c157030 0x7f018c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f018b400640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f018c135bb0 0x7f018c135fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0182000640 1 -- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f018c135bb0 msgr2=0x7f018c135fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0182000640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f018c135bb0 0x7f018c135fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0182000640 1 -- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f018c157030 msgr2=0x7f018c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0182000640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f018c157030 0x7f018c0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f0182000640 1 -- 172.21.15.50:0/3908452493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018c15d490 con 0x7f018c160d50 2024-09-21T06:09:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.669+0000 7f018aa00640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f018c157030 0x7f018c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:41.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.673+0000 7f0182000640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018c160d50 0x7f018c15cf50 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f017400c400 tx=0x7f017400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:41.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.673+0000 7f0183400640 1 -- 172.21.15.50:0/3908452493 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174011790 con 0x7f018c160d50 2024-09-21T06:09:41.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.673+0000 7f0183400640 1 -- 172.21.15.50:0/3908452493 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0174017070 con 0x7f018c160d50 2024-09-21T06:09:41.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.673+0000 7f0183400640 1 -- 172.21.15.50:0/3908452493 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174011cc0 con 0x7f018c160d50 2024-09-21T06:09:41.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.673+0000 7f0191a00640 1 -- 172.21.15.50:0/3908452493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018c0b28a0 con 0x7f018c160d50 2024-09-21T06:09:41.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.673+0000 7f0191a00640 1 -- 172.21.15.50:0/3908452493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018c0b3080 con 0x7f018c160d50 2024-09-21T06:09:41.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.673+0000 7f0191a00640 1 -- 172.21.15.50:0/3908452493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f018c0b2b10 con 0x7f018c160d50 2024-09-21T06:09:41.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.673+0000 7f0183400640 1 -- 172.21.15.50:0/3908452493 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f017401c460 con 0x7f018c160d50 2024-09-21T06:09:41.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.677+0000 7f0183400640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0168074ca0 0x7f01680771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:41.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.677+0000 7f0183400640 1 -- 172.21.15.50:0/3908452493 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7f0174092f90 con 0x7f018c160d50 2024-09-21T06:09:41.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.677+0000 7f018aa00640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0168074ca0 0x7f01680771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:41.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.677+0000 7f018aa00640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0168074ca0 0x7f01680771b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f017c0ae000 tx=0x7f017c01afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:41.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.681+0000 7f0183400640 1 -- 172.21.15.50:0/3908452493 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f018c15d490 con 0x7f018c160d50 2024-09-21T06:09:41.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.857+0000 7f0191a00640 1 -- 172.21.15.50:0/3908452493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f018c0d2260 con 0x7f018c160d50 2024-09-21T06:09:41.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.857+0000 7f0183400640 1 -- 172.21.15.50:0/3908452493 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v51) ==== 59+0+2294 (secure 0 0 0) 0x7f017405fa10 con 0x7f018c160d50 2024-09-21T06:09:41.858 INFO:teuthology.orchestra.run.smithi050.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:09:41.858 INFO:teuthology.orchestra.run.smithi050.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:09:41.858 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:09:41.858 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_client 20/20 2024-09-21T06:09:41.858 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:09:41.858 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3735.149299 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4471.400731 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3640.131623 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3663.645087 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3762.055672 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3661.558528 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3661.456695 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3796.187852 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds 20/20 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_ms 2/2 2024-09-21T06:09:41.859 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_frag true 2024-09-21T06:09:41.860 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:09:41.860 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:09:41.860 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_verify_scatter true 2024-09-21T06:09:41.860 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:09:41.860 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:09:41.860 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:09:41.860 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced client_mount_timeout 600 2024-09-21T06:09:41.860 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_client 20/20 2024-09-21T06:09:41.860 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_ms 1/1 2024-09-21T06:09:41.860 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:09:41.860 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:09:41.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.861+0000 7f0191a00640 1 -- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0168074ca0 msgr2=0x7f01680771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:41.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.861+0000 7f0191a00640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0168074ca0 0x7f01680771b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f017c0ae000 tx=0x7f017c01afd0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.861+0000 7f0191a00640 1 -- 172.21.15.50:0/3908452493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018c160d50 msgr2=0x7f018c15cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:41.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.861+0000 7f0191a00640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018c160d50 0x7f018c15cf50 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f017400c400 tx=0x7f017400c8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.861+0000 7f0191a00640 1 -- 172.21.15.50:0/3908452493 shutdown_connections 2024-09-21T06:09:41.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.861+0000 7f0191a00640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f018c135bb0 0x7f018c135fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.861+0000 7f0191a00640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0168074ca0 0x7f01680771b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.861+0000 7f0191a00640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018c160d50 0x7f018c15cf50 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.861+0000 7f0191a00640 1 --2- 172.21.15.50:0/3908452493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f018c157030 0x7f018c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.861+0000 7f0191a00640 1 -- 172.21.15.50:0/3908452493 >> 172.21.15.50:0/3908452493 conn(0x7f018c0708c0 msgr2=0x7f018c075fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:41.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.861+0000 7f0191a00640 1 -- 172.21.15.50:0/3908452493 shutdown_connections 2024-09-21T06:09:41.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:41.861+0000 7f0191a00640 1 -- 172.21.15.50:0/3908452493 wait complete. 2024-09-21T06:09:41.881 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:09:42.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.041+0000 7f6d04400640 1 -- 172.21.15.50:0/3382318159 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cfc0b54f0 msgr2=0x7f6cfc155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.041+0000 7f6d04400640 1 --2- 172.21.15.50:0/3382318159 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cfc0b54f0 0x7f6cfc155b50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009870 tx=0x7f6ce801c6b0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.041+0000 7f6d04400640 1 -- 172.21.15.50:0/3382318159 shutdown_connections 2024-09-21T06:09:42.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.041+0000 7f6d04400640 1 --2- 172.21.15.50:0/3382318159 >> v2:172.21.15.88:3300/0 conn(0x7f6cfc073a90 0x7f6cfc073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.041+0000 7f6d04400640 1 --2- 172.21.15.50:0/3382318159 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cfc0b54f0 0x7f6cfc155b50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.041+0000 7f6d04400640 1 -- 172.21.15.50:0/3382318159 >> 172.21.15.50:0/3382318159 conn(0x7f6cfc0b2040 msgr2=0x7f6cfc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:42.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.041+0000 7f6d04400640 1 -- 172.21.15.50:0/3382318159 shutdown_connections 2024-09-21T06:09:42.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.041+0000 7f6d04400640 1 -- 172.21.15.50:0/3382318159 wait complete. 2024-09-21T06:09:42.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d04400640 1 Processor -- start 2024-09-21T06:09:42.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d04400640 1 -- start start 2024-09-21T06:09:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d04400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cfc072100 0x7f6cfc1f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d04400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cfc073a90 0x7f6cfc150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d04400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cfc150dc0 0x7f6cfc150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d04400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc168ee0 con 0x7f6cfc150dc0 2024-09-21T06:09:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d04400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc168d60 con 0x7f6cfc073a90 2024-09-21T06:09:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d04400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6cfc169060 con 0x7f6cfc072100 2024-09-21T06:09:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d02400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cfc150dc0 0x7f6cfc150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d02400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cfc150dc0 0x7f6cfc150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34598/0 (socket says 172.21.15.50:34598) 2024-09-21T06:09:42.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d02400640 1 -- 172.21.15.50:0/2774058715 learned_addr learned my addr 172.21.15.50:0/2774058715 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:42.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d01a00640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cfc072100 0x7f6cfc1f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:42.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d01000640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cfc073a90 0x7f6cfc150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:42.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d02400640 1 -- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cfc072100 msgr2=0x7f6cfc1f92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d02400640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cfc072100 0x7f6cfc1f92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d02400640 1 -- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cfc073a90 msgr2=0x7f6cfc150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d02400640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cfc073a90 0x7f6cfc150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d02400640 1 -- 172.21.15.50:0/2774058715 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cfc14db40 con 0x7f6cfc150dc0 2024-09-21T06:09:42.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d02400640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cfc150dc0 0x7f6cfc150670 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009fd0 tx=0x7f6cf400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:42.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6cf9600640 1 -- 172.21.15.50:0/2774058715 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf400cdc0 con 0x7f6cfc150dc0 2024-09-21T06:09:42.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6cf9600640 1 -- 172.21.15.50:0/2774058715 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cf4004b30 con 0x7f6cfc150dc0 2024-09-21T06:09:42.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d04400640 1 -- 172.21.15.50:0/2774058715 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cfc14f370 con 0x7f6cfc150dc0 2024-09-21T06:09:42.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6cf9600640 1 -- 172.21.15.50:0/2774058715 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4005020 con 0x7f6cfc150dc0 2024-09-21T06:09:42.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.045+0000 7f6d04400640 1 -- 172.21.15.50:0/2774058715 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cfc14f930 con 0x7f6cfc150dc0 2024-09-21T06:09:42.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.049+0000 7f6d04400640 1 -- 172.21.15.50:0/2774058715 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cfc073ea0 con 0x7f6cfc150dc0 2024-09-21T06:09:42.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.049+0000 7f6cf9600640 1 -- 172.21.15.50:0/2774058715 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f6cf40051c0 con 0x7f6cfc150dc0 2024-09-21T06:09:42.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.049+0000 7f6cf9600640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6cdc074c70 0x7f6cdc077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:42.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.049+0000 7f6cf9600640 1 -- 172.21.15.50:0/2774058715 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7f6cf4093bc0 con 0x7f6cfc150dc0 2024-09-21T06:09:42.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.049+0000 7f6d01a00640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6cdc074c70 0x7f6cdc077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:42.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.053+0000 7f6d01a00640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6cdc074c70 0x7f6cdc077180 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0007990 tx=0x7f6cf0001da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:42.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.053+0000 7f6cf9600640 1 -- 172.21.15.50:0/2774058715 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6cf4060080 con 0x7f6cfc150dc0 2024-09-21T06:09:42.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.233+0000 7f6d04400640 1 -- 172.21.15.50:0/2774058715 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6cfc0b3740 con 0x7f6cfc150dc0 2024-09-21T06:09:42.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6cf9600640 1 -- 172.21.15.50:0/2774058715 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7f6cf4064ab0 con 0x7f6cfc150dc0 2024-09-21T06:09:42.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6d04400640 1 -- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6cdc074c70 msgr2=0x7f6cdc077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6d04400640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6cdc074c70 0x7f6cdc077180 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0007990 tx=0x7f6cf0001da0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6d04400640 1 -- 172.21.15.50:0/2774058715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cfc150dc0 msgr2=0x7f6cfc150670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6d04400640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cfc150dc0 0x7f6cfc150670 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009fd0 tx=0x7f6cf400ef10 comp rx=0 tx=0).stop 2024-09-21T06:09:42.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6d04400640 1 -- 172.21.15.50:0/2774058715 shutdown_connections 2024-09-21T06:09:42.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6d04400640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6cdc074c70 0x7f6cdc077180 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6d04400640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cfc150dc0 0x7f6cfc150670 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6d04400640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cfc073a90 0x7f6cfc150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6d04400640 1 --2- 172.21.15.50:0/2774058715 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6cfc072100 0x7f6cfc1f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6d04400640 1 -- 172.21.15.50:0/2774058715 >> 172.21.15.50:0/2774058715 conn(0x7f6cfc0b2040 msgr2=0x7f6cfc15de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:42.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6d04400640 1 -- 172.21.15.50:0/2774058715 shutdown_connections 2024-09-21T06:09:42.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.237+0000 7f6d04400640 1 -- 172.21.15.50:0/2774058715 wait complete. 2024-09-21T06:09:42.259 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:09:42.260 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:09:42.354 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:09:42.455 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:09:42.456 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:09:42.456 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:42.465 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:09:42.465 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:09:42.465 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:42.474 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:09:42.474 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:09:42.474 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:42.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:09:42.520 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:09:42.521 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:42.530 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:09:42.530 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:09:42.530 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:09:42.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.721+0000 7fc48f600640 1 -- 172.21.15.50:0/2105824539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880b5c70 msgr2=0x7fc48815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.721+0000 7fc48f600640 1 --2- 172.21.15.50:0/2105824539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880b5c70 0x7fc48815bb60 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fc488072e30 tx=0x7fc47c01ea10 comp rx=0 tx=0).stop 2024-09-21T06:09:42.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.721+0000 7fc48f600640 1 -- 172.21.15.50:0/2105824539 shutdown_connections 2024-09-21T06:09:42.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.721+0000 7fc48f600640 1 --2- 172.21.15.50:0/2105824539 >> v2:172.21.15.88:3300/0 conn(0x7fc4880b4de0 0x7fc4880b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.721+0000 7fc48f600640 1 --2- 172.21.15.50:0/2105824539 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4881527b0 0x7fc488074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.721+0000 7fc48f600640 1 --2- 172.21.15.50:0/2105824539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880b5c70 0x7fc48815bb60 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.721+0000 7fc48f600640 1 -- 172.21.15.50:0/2105824539 >> 172.21.15.50:0/2105824539 conn(0x7fc4880b2040 msgr2=0x7fc4880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:42.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.721+0000 7fc48f600640 1 -- 172.21.15.50:0/2105824539 shutdown_connections 2024-09-21T06:09:42.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.721+0000 7fc48f600640 1 -- 172.21.15.50:0/2105824539 wait complete. 2024-09-21T06:09:42.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc48f600640 1 Processor -- start 2024-09-21T06:09:42.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc48f600640 1 -- start start 2024-09-21T06:09:42.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc48f600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4880b4de0 0x7fc488156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:42.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc48f600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880b5c70 0x7fc4881f9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:42.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc48f600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4881527b0 0x7fc4881449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:42.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc48f600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc48815f700 con 0x7fc4880b5c70 2024-09-21T06:09:42.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc48f600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc48815f580 con 0x7fc4880b4de0 2024-09-21T06:09:42.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc48f600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc48815f880 con 0x7fc4881527b0 2024-09-21T06:09:42.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc487e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880b5c70 0x7fc4881f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:42.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc487e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880b5c70 0x7fc4881f9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34604/0 (socket says 172.21.15.50:34604) 2024-09-21T06:09:42.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc487e00640 1 -- 172.21.15.50:0/2025554855 learned_addr learned my addr 172.21.15.50:0/2025554855 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:42.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc48cc00640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4880b4de0 0x7fc488156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:42.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc48d600640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4881527b0 0x7fc4881449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:42.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc487e00640 1 -- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4881527b0 msgr2=0x7fc4881449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc487e00640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4881527b0 0x7fc4881449f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc487e00640 1 -- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4880b4de0 msgr2=0x7fc488156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc487e00640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4880b4de0 0x7fc488156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc487e00640 1 -- 172.21.15.50:0/2025554855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4880cd570 con 0x7fc4880b5c70 2024-09-21T06:09:42.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.725+0000 7fc487e00640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880b5c70 0x7fc4881f9230 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc47400c970 tx=0x7fc47400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:42.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.729+0000 7fc484c00640 1 -- 172.21.15.50:0/2025554855 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474014070 con 0x7fc4880b5c70 2024-09-21T06:09:42.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.729+0000 7fc484c00640 1 -- 172.21.15.50:0/2025554855 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4740077a0 con 0x7fc4880b5c70 2024-09-21T06:09:42.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.729+0000 7fc484c00640 1 -- 172.21.15.50:0/2025554855 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474007bf0 con 0x7fc4880b5c70 2024-09-21T06:09:42.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.729+0000 7fc48f600640 1 -- 172.21.15.50:0/2025554855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc488156340 con 0x7fc4880b5c70 2024-09-21T06:09:42.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.729+0000 7fc48f600640 1 -- 172.21.15.50:0/2025554855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc488156900 con 0x7fc4880b5c70 2024-09-21T06:09:42.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.729+0000 7fc48f600640 1 -- 172.21.15.50:0/2025554855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc48815fdc0 con 0x7fc4880b5c70 2024-09-21T06:09:42.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.729+0000 7fc484c00640 1 -- 172.21.15.50:0/2025554855 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fc474017070 con 0x7fc4880b5c70 2024-09-21T06:09:42.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.729+0000 7fc484c00640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc468074c70 0x7fc468077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:42.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.729+0000 7fc484c00640 1 -- 172.21.15.50:0/2025554855 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7fc4740932e0 con 0x7fc4880b5c70 2024-09-21T06:09:42.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.729+0000 7fc48cc00640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc468074c70 0x7fc468077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:42.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.733+0000 7fc48cc00640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc468074c70 0x7fc468077180 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0b1000 tx=0x7fc47c01d250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:42.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.733+0000 7fc484c00640 1 -- 172.21.15.50:0/2025554855 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc47405f7a0 con 0x7fc4880b5c70 2024-09-21T06:09:42.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.949+0000 7fc48f600640 1 -- 172.21.15.50:0/2025554855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4880b5420 con 0x7fc4880b5c70 2024-09-21T06:09:42.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.949+0000 7fc484c00640 1 -- 172.21.15.50:0/2025554855 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+3908 (secure 0 0 0) 0x7fc4740641d0 con 0x7fc4880b5c70 2024-09-21T06:09:42.950 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:42.951 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":68,"btime":"2024-09-21T06:09:36:446213+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2607826086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2607826086},{"type":"v1","addr":"172.21.15.50:6835","nonce":2607826086}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1771536088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":1771536088},{"type":"v1","addr":"172.21.15.88:6837","nonce":1771536088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6941,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3072969958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3072969958},{"type":"v1","addr":"172.21.15.50:6834","nonce":3072969958}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/433905290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":433905290},{"type":"v1","addr":"172.21.15.88:6836","nonce":433905290}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[]} 2024-09-21T06:09:42.951 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 68 2024-09-21T06:09:42.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.953+0000 7fc48f600640 1 -- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc468074c70 msgr2=0x7fc468077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.953+0000 7fc48f600640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc468074c70 0x7fc468077180 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0b1000 tx=0x7fc47c01d250 comp rx=0 tx=0).stop 2024-09-21T06:09:42.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.953+0000 7fc48f600640 1 -- 172.21.15.50:0/2025554855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880b5c70 msgr2=0x7fc4881f9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.953+0000 7fc48f600640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880b5c70 0x7fc4881f9230 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc47400c970 tx=0x7fc47400ce30 comp rx=0 tx=0).stop 2024-09-21T06:09:42.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.953+0000 7fc48f600640 1 -- 172.21.15.50:0/2025554855 shutdown_connections 2024-09-21T06:09:42.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.953+0000 7fc48f600640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc4881527b0 0x7fc4881449f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.953+0000 7fc48f600640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc468074c70 0x7fc468077180 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.953+0000 7fc48f600640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880b5c70 0x7fc4881f9230 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.953+0000 7fc48f600640 1 --2- 172.21.15.50:0/2025554855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4880b4de0 0x7fc488156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.953+0000 7fc48f600640 1 -- 172.21.15.50:0/2025554855 >> 172.21.15.50:0/2025554855 conn(0x7fc4880b2040 msgr2=0x7fc4880b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:42.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.953+0000 7fc48f600640 1 -- 172.21.15.50:0/2025554855 shutdown_connections 2024-09-21T06:09:42.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:42.953+0000 7fc48f600640 1 -- 172.21.15.50:0/2025554855 wait complete. 2024-09-21T06:09:42.973 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:09:42.974 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:09:42.974 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:09:42.974 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:09:42.974 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:09:42.974 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:09:42.974 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:09:42.974 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:09:42.975 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:09:42.975 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:09:42.975 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:09:42.975 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:09:42.975 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:09:42.975 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:09:42.975 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:09:42.976 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:09:42.989 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:09:42.989 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:09:42.993 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:09:42.993 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:09:42.994 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:09:42.994 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:09:42.997 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:09:42.998 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:09:43.042 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:09:43.042 DEBUG:teuthology.parallel:result is None 2024-09-21T06:09:43.043 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:09:43.043 DEBUG:teuthology.parallel:result is None 2024-09-21T06:09:43.044 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:09:43.044 DEBUG:teuthology.parallel:result is None 2024-09-21T06:09:43.045 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:09:43.045 DEBUG:teuthology.parallel:result is None 2024-09-21T06:09:43.046 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:09:43.144 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-09-21T06:09:43.147 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T06:09:43.148 INFO:tasks.ceph.mds.b.smithi088.stdout:starting mds.b at 2024-09-21T06:09:43.151 INFO:tasks.ceph.mds.d.smithi088.stdout:starting mds.d at 2024-09-21T06:09:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.209+0000 7f2382600640 1 -- 172.21.15.50:0/2194838789 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f237c1527b0 msgr2=0x7f237c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.209+0000 7f2382600640 1 --2- 172.21.15.50:0/2194838789 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f237c1527b0 0x7f237c074e30 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f237c072e30 tx=0x7f236401c800 comp rx=0 tx=0).stop 2024-09-21T06:09:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.209+0000 7f2382600640 1 -- 172.21.15.50:0/2194838789 shutdown_connections 2024-09-21T06:09:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.209+0000 7f2382600640 1 --2- 172.21.15.50:0/2194838789 >> v2:172.21.15.88:3300/0 conn(0x7f237c0b4de0 0x7f237c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.209+0000 7f2382600640 1 --2- 172.21.15.50:0/2194838789 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f237c1527b0 0x7f237c074e30 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.209+0000 7f2382600640 1 --2- 172.21.15.50:0/2194838789 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f237c0b5c70 0x7f237c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.209+0000 7f2382600640 1 -- 172.21.15.50:0/2194838789 >> 172.21.15.50:0/2194838789 conn(0x7f237c0b2040 msgr2=0x7f237c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:43.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.209+0000 7f2382600640 1 -- 172.21.15.50:0/2194838789 shutdown_connections 2024-09-21T06:09:43.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.209+0000 7f2382600640 1 -- 172.21.15.50:0/2194838789 wait complete. 2024-09-21T06:09:43.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.213+0000 7f2382600640 1 Processor -- start 2024-09-21T06:09:43.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.213+0000 7f2382600640 1 -- start start 2024-09-21T06:09:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.213+0000 7f2382600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f237c0b4de0 0x7f237c154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.213+0000 7f2382600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f237c0b5c70 0x7f237c1432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.213+0000 7f2382600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f237c1527b0 0x7f237c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.213+0000 7f2382600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f237c15f6b0 con 0x7f237c0b5c70 2024-09-21T06:09:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.213+0000 7f2382600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f237c15f530 con 0x7f237c1527b0 2024-09-21T06:09:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.213+0000 7f2382600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f237c15f830 con 0x7f237c0b4de0 2024-09-21T06:09:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.213+0000 7f2373e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f237c0b5c70 0x7f237c1432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.213+0000 7f2373e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f237c0b5c70 0x7f237c1432c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34654/0 (socket says 172.21.15.50:34654) 2024-09-21T06:09:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.213+0000 7f2373e00640 1 -- 172.21.15.50:0/84319436 learned_addr learned my addr 172.21.15.50:0/84319436 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.213+0000 7f237b400640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f237c0b4de0 0x7f237c154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f237be00640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f237c1527b0 0x7f237c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2373e00640 1 -- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f237c0b4de0 msgr2=0x7f237c154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2373e00640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f237c0b4de0 0x7f237c154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2373e00640 1 -- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f237c1527b0 msgr2=0x7f237c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2373e00640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f237c1527b0 0x7f237c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2373e00640 1 -- 172.21.15.50:0/84319436 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c1fc120 con 0x7f237c0b5c70 2024-09-21T06:09:43.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2373e00640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f237c0b5c70 0x7f237c1432c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f23640b0040 tx=0x7f2364005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:43.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2373400640 1 -- 172.21.15.50:0/84319436 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23640b2040 con 0x7f237c0b5c70 2024-09-21T06:09:43.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2382600640 1 -- 172.21.15.50:0/84319436 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237c1fc3c0 con 0x7f237c0b5c70 2024-09-21T06:09:43.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2373400640 1 -- 172.21.15.50:0/84319436 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2364004330 con 0x7f237c0b5c70 2024-09-21T06:09:43.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2373400640 1 -- 172.21.15.50:0/84319436 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364005280 con 0x7f237c0b5c70 2024-09-21T06:09:43.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2382600640 1 -- 172.21.15.50:0/84319436 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237c1fc980 con 0x7f237c0b5c70 2024-09-21T06:09:43.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2382600640 1 -- 172.21.15.50:0/84319436 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237c06f0e0 con 0x7f237c0b5c70 2024-09-21T06:09:43.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.217+0000 7f2373400640 1 -- 172.21.15.50:0/84319436 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f2364005420 con 0x7f237c0b5c70 2024-09-21T06:09:43.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.221+0000 7f2373400640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2358074ca0 0x7f23580771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:43.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.221+0000 7f2373400640 1 -- 172.21.15.50:0/84319436 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7f236412cfe0 con 0x7f237c0b5c70 2024-09-21T06:09:43.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.221+0000 7f237b400640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2358074ca0 0x7f23580771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:43.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.221+0000 7f237b400640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2358074ca0 0x7f23580771b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f236c009b40 tx=0x7f236c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:43.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.225+0000 7f2373400640 1 -- 172.21.15.50:0/84319436 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f23640164b0 con 0x7f237c0b5c70 2024-09-21T06:09:43.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.413+0000 7f2382600640 1 -- 172.21.15.50:0/84319436 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f237c154a40 con 0x7f237c0b5c70 2024-09-21T06:09:43.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.485+0000 7f2373400640 1 -- 172.21.15.50:0/84319436 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v106) ==== 97+0+0 (secure 0 0 0) 0x7f23640f9a70 con 0x7f237c0b5c70 2024-09-21T06:09:43.488 INFO:teuthology.orchestra.run.smithi050.stderr: removed all blocklist entries 2024-09-21T06:09:43.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.489+0000 7f2382600640 1 -- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2358074ca0 msgr2=0x7f23580771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.489+0000 7f2382600640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2358074ca0 0x7f23580771b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f236c009b40 tx=0x7f236c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.489+0000 7f2382600640 1 -- 172.21.15.50:0/84319436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f237c0b5c70 msgr2=0x7f237c1432c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.489+0000 7f2382600640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f237c0b5c70 0x7f237c1432c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f23640b0040 tx=0x7f2364005e70 comp rx=0 tx=0).stop 2024-09-21T06:09:43.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.489+0000 7f2382600640 1 -- 172.21.15.50:0/84319436 shutdown_connections 2024-09-21T06:09:43.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.489+0000 7f2382600640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f237c1527b0 0x7f237c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.489+0000 7f2382600640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2358074ca0 0x7f23580771b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.489+0000 7f2382600640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f237c0b5c70 0x7f237c1432c0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.489+0000 7f2382600640 1 --2- 172.21.15.50:0/84319436 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f237c0b4de0 0x7f237c154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.489+0000 7f2382600640 1 -- 172.21.15.50:0/84319436 >> 172.21.15.50:0/84319436 conn(0x7f237c0b2040 msgr2=0x7f237c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:43.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.493+0000 7f2382600640 1 -- 172.21.15.50:0/84319436 shutdown_connections 2024-09-21T06:09:43.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.493+0000 7f2382600640 1 -- 172.21.15.50:0/84319436 wait complete. 2024-09-21T06:09:43.515 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:09:43.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.673+0000 7f5c46800640 1 -- 172.21.15.50:0/4135077322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c400b4de0 msgr2=0x7f5c400b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.673+0000 7f5c46800640 1 --2- 172.21.15.50:0/4135077322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c400b4de0 0x7f5c400b51f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5c400703e0 tx=0x7f5c3801ccc0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.673+0000 7f5c46800640 1 -- 172.21.15.50:0/4135077322 shutdown_connections 2024-09-21T06:09:43.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.673+0000 7f5c46800640 1 --2- 172.21.15.50:0/4135077322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c400b4de0 0x7f5c400b51f0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.673+0000 7f5c46800640 1 --2- 172.21.15.50:0/4135077322 >> v2:172.21.15.88:3300/0 conn(0x7f5c401527b0 0x7f5c40074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.673+0000 7f5c46800640 1 --2- 172.21.15.50:0/4135077322 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c400b5c70 0x7f5c4015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.673+0000 7f5c46800640 1 -- 172.21.15.50:0/4135077322 >> 172.21.15.50:0/4135077322 conn(0x7f5c400b2040 msgr2=0x7f5c400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:43.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.673+0000 7f5c46800640 1 -- 172.21.15.50:0/4135077322 shutdown_connections 2024-09-21T06:09:43.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.673+0000 7f5c46800640 1 -- 172.21.15.50:0/4135077322 wait complete. 2024-09-21T06:09:43.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c46800640 1 Processor -- start 2024-09-21T06:09:43.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c46800640 1 -- start start 2024-09-21T06:09:43.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c46800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c400b4de0 0x7f5c40150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:43.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c46800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c400b5c70 0x7f5c4014a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:43.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c46800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c401527b0 0x7f5c400b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:43.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c46800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4015f730 con 0x7f5c401527b0 2024-09-21T06:09:43.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c46800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4015f5b0 con 0x7f5c400b5c70 2024-09-21T06:09:43.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c46800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5c4015f8b0 con 0x7f5c400b4de0 2024-09-21T06:09:43.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c3f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c400b4de0 0x7f5c40150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:43.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c3fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c401527b0 0x7f5c400b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:43.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c3f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c400b4de0 0x7f5c40150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:46528/0 (socket says 172.21.15.50:46528) 2024-09-21T06:09:43.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c3f400640 1 -- 172.21.15.50:0/217969603 learned_addr learned my addr 172.21.15.50:0/217969603 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:43.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.677+0000 7f5c36a00640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c400b5c70 0x7f5c4014a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:43.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c3f400640 1 -- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c400b5c70 msgr2=0x7f5c4014a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c3f400640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c400b5c70 0x7f5c4014a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c3f400640 1 -- 172.21.15.50:0/217969603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c401527b0 msgr2=0x7f5c400b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c3f400640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c401527b0 0x7f5c400b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c3f400640 1 -- 172.21.15.50:0/217969603 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c401fcbe0 con 0x7f5c400b4de0 2024-09-21T06:09:43.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c3f400640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c400b4de0 0x7f5c40150c80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000e740 tx=0x7f5c3000ec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:43.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c37e00640 1 -- 172.21.15.50:0/217969603 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3000ccc0 con 0x7f5c400b4de0 2024-09-21T06:09:43.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c37e00640 1 -- 172.21.15.50:0/217969603 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c3000ce60 con 0x7f5c400b4de0 2024-09-21T06:09:43.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c46800640 1 -- 172.21.15.50:0/217969603 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c401fbe20 con 0x7f5c400b4de0 2024-09-21T06:09:43.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c37e00640 1 -- 172.21.15.50:0/217969603 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30005280 con 0x7f5c400b4de0 2024-09-21T06:09:43.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c46800640 1 -- 172.21.15.50:0/217969603 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c401fc3e0 con 0x7f5c400b4de0 2024-09-21T06:09:43.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c46800640 1 -- 172.21.15.50:0/217969603 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c4015fdd0 con 0x7f5c400b4de0 2024-09-21T06:09:43.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c37e00640 1 -- 172.21.15.50:0/217969603 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f5c30005420 con 0x7f5c400b4de0 2024-09-21T06:09:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.681+0000 7f5c37e00640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5c1c074c70 0x7f5c1c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.685+0000 7f5c37e00640 1 -- 172.21.15.50:0/217969603 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(108..108 src has 1..108) ==== 4941+0+0 (secure 0 0 0) 0x7f5c30093920 con 0x7f5c400b4de0 2024-09-21T06:09:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.685+0000 7f5c36a00640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5c1c074c70 0x7f5c1c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:43.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.685+0000 7f5c36a00640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5c1c074c70 0x7f5c1c077180 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5c400d0160 tx=0x7f5c280091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:43.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.689+0000 7f5c37e00640 1 -- 172.21.15.50:0/217969603 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5c3005fde0 con 0x7f5c400b4de0 2024-09-21T06:09:43.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.925+0000 7f5c46800640 1 -- 172.21.15.50:0/217969603 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5c4015c790 con 0x7f5c400b4de0 2024-09-21T06:09:43.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.925+0000 7f5c37e00640 1 -- 172.21.15.50:0/217969603 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v36) ==== 80+0+5304 (secure 0 0 0) 0x7f5c30064810 con 0x7f5c400b4de0 2024-09-21T06:09:43.927 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:43.927 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:09:43.927 INFO:teuthology.orchestra.run.smithi050.stdout: "auth_dump": [ 2024-09-21T06:09:43.927 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.927 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.a", 2024-09-21T06:09:43.927 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mSKq0ARAA2egaHGYWhRUEq6oRl1CIMg==", 2024-09-21T06:09:43.927 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.927 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:09:43.927 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:09:43.927 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:09:43.927 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.b", 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mwv0+DxAAY+eZPxv5uY9PnLl0asdr6w==", 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.928 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.c", 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5me6maCRAA4jvHgTLTKpb/mxzzG71SfQ==", 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.d", 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mzYnuFhAA/hEigjluXspzs/hyeWxBjQ==", 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:09:43.929 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.0", 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDIX+5mrZF/NhAA33M6ummG4r4yXU8/YOL5WA==", 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.1", 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDMX+5myVz5DxAAMYfTPep6ubD7Cxnfa9dMCg==", 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.2", 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDPX+5mA7ahIRAAf/osnY5l83p8ZWna0ErD2Q==", 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:09:43.931 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.3", 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDSX+5m+rDpNRAApGPZJIlAaA0DyjtPXPgoaQ==", 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.4", 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDYX+5mKL9SORAA21ITQUwUSoDUjXaD0Cwb3g==", 2024-09-21T06:09:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.5", 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDcX+5minRqFBAAMJOBQNvJFD4TNdKbPMLBig==", 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.933 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.6", 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDfX+5m6HadJhAAAv670Tvyi4IKMByFPiBlAw==", 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.7", 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDjX+5muPrrAhAAuRB7Muamiht+f7LvSmI6UQ==", 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:09:43.934 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.0", 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw==", 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow r", 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.1", 2024-09-21T06:09:43.935 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mo94GJBAAMBSb1NPb2L93vzvnAh59kw==", 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.2", 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5m4IrnHxAAFjER3ywuS+FlbuA7bPZVqw==", 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:09:43.936 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.3", 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mysQeKBAAq5c4bD3hbsU1H7JabaMjwQ==", 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.admin", 2024-09-21T06:09:43.937 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mcUXKHhAAV1Rs0OMNVPeAsRiAVt67jA==", 2024-09-21T06:09:43.938 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.938 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:09:43.938 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:09:43.938 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:09:43.938 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.938 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.938 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.938 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.938 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.x", 2024-09-21T06:09:43.938 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5m2F9WOBAAZuQTUyR7oTT01CRD1jo+/Q==", 2024-09-21T06:09:43.938 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.938 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.y", 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mEjC/MxAAHgwIu4CLDgFajwD2rL5WKA==", 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.942 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:43.943 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-21T06:09:43.943 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:09:43.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.929+0000 7f5c46800640 1 -- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5c1c074c70 msgr2=0x7f5c1c077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.929+0000 7f5c46800640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5c1c074c70 0x7f5c1c077180 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5c400d0160 tx=0x7f5c280091e0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.929+0000 7f5c46800640 1 -- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c400b4de0 msgr2=0x7f5c40150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.929+0000 7f5c46800640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c400b4de0 0x7f5c40150c80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000e740 tx=0x7f5c3000ec00 comp rx=0 tx=0).stop 2024-09-21T06:09:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.929+0000 7f5c46800640 1 -- 172.21.15.50:0/217969603 shutdown_connections 2024-09-21T06:09:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.929+0000 7f5c46800640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5c1c074c70 0x7f5c1c077180 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.929+0000 7f5c46800640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c401527b0 0x7f5c400b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.929+0000 7f5c46800640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c400b5c70 0x7f5c4014a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.929+0000 7f5c46800640 1 --2- 172.21.15.50:0/217969603 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c400b4de0 0x7f5c40150c80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.929+0000 7f5c46800640 1 -- 172.21.15.50:0/217969603 >> 172.21.15.50:0/217969603 conn(0x7f5c400b2040 msgr2=0x7f5c4015afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.929+0000 7f5c46800640 1 -- 172.21.15.50:0/217969603 shutdown_connections 2024-09-21T06:09:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:43.929+0000 7f5c46800640 1 -- 172.21.15.50:0/217969603 wait complete. 2024-09-21T06:09:43.949 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:09:44.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.113+0000 7f73b9a00640 1 -- 172.21.15.50:0/1734613683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7398003fa0 msgr2=0x7f7398004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:44.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.113+0000 7f73b9a00640 1 --2- 172.21.15.50:0/1734613683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7398003fa0 0x7f7398004810 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f73b4071fd0 tx=0x7f739c01d1e0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.113+0000 7f73b3400640 1 -- 172.21.15.50:0/1734613683 reap_dead start 2024-09-21T06:09:44.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.113+0000 7f73b9a00640 1 -- 172.21.15.50:0/1734613683 shutdown_connections 2024-09-21T06:09:44.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.113+0000 7f73b9a00640 1 -- 172.21.15.50:0/1734613683 >> 172.21.15.50:0/1734613683 conn(0x7f73b40b2040 msgr2=0x7f73b40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:44.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.113+0000 7f73b9a00640 1 -- 172.21.15.50:0/1734613683 shutdown_connections 2024-09-21T06:09:44.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.113+0000 7f73b9a00640 1 -- 172.21.15.50:0/1734613683 wait complete. 2024-09-21T06:09:44.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.113+0000 7f73b9a00640 1 Processor -- start 2024-09-21T06:09:44.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.113+0000 7f73b9a00640 1 -- start start 2024-09-21T06:09:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73b9a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7398003fa0 0x7f73b41612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73b9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73b4153c70 0x7f73b41f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73b9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73b41f6cb0 0x7f73b41f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73b9a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4167bc0 con 0x7f73b4153c70 2024-09-21T06:09:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73b9a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4167a40 con 0x7f73b41f6cb0 2024-09-21T06:09:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73b9a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f73b4167d40 con 0x7f7398003fa0 2024-09-21T06:09:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73aa000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73b4153c70 0x7f73b41f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73aa000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73b4153c70 0x7f73b41f6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34696/0 (socket says 172.21.15.50:34696) 2024-09-21T06:09:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73aa000640 1 -- 172.21.15.50:0/3029062028 learned_addr learned my addr 172.21.15.50:0/3029062028 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73b2a00640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7398003fa0 0x7f73b41612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73b3400640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73b41f6cb0 0x7f73b41f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:44.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73aa000640 1 -- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7398003fa0 msgr2=0x7f73b41612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73aa000640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7398003fa0 0x7f73b41612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73aa000640 1 -- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73b41f6cb0 msgr2=0x7f73b41f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73aa000640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73b41f6cb0 0x7f73b41f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73aa000640 1 -- 172.21.15.50:0/3029062028 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b4154730 con 0x7f73b4153c70 2024-09-21T06:09:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73aa000640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73b4153c70 0x7f73b41f6770 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f739c098c00 tx=0x7f739c009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73ab400640 1 -- 172.21.15.50:0/3029062028 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c0bb030 con 0x7f73b4153c70 2024-09-21T06:09:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73ab400640 1 -- 172.21.15.50:0/3029062028 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f739c004880 con 0x7f73b4153c70 2024-09-21T06:09:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73ab400640 1 -- 172.21.15.50:0/3029062028 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c004c90 con 0x7f73b4153c70 2024-09-21T06:09:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73b9a00640 1 -- 172.21.15.50:0/3029062028 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b4154a10 con 0x7f73b4153c70 2024-09-21T06:09:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73b9a00640 1 -- 172.21.15.50:0/3029062028 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b4154fd0 con 0x7f73b4153c70 2024-09-21T06:09:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.117+0000 7f73b9a00640 1 -- 172.21.15.50:0/3029062028 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b40b51f0 con 0x7f73b4153c70 2024-09-21T06:09:44.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.121+0000 7f73ab400640 1 -- 172.21.15.50:0/3029062028 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f739c0be030 con 0x7f73b4153c70 2024-09-21T06:09:44.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.121+0000 7f73ab400640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7390074ca0 0x7f73900771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:44.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.121+0000 7f73ab400640 1 -- 172.21.15.50:0/3029062028 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4941+0+0 (secure 0 0 0) 0x7f739c12e6a0 con 0x7f73b4153c70 2024-09-21T06:09:44.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.121+0000 7f73b2a00640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7390074ca0 0x7f73900771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:44.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.121+0000 7f73b2a00640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7390074ca0 0x7f73900771b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f73a4000c10 tx=0x7f73a4006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:44.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.125+0000 7f73ab400640 1 -- 172.21.15.50:0/3029062028 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f739c016f20 con 0x7f73b4153c70 2024-09-21T06:09:44.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.301+0000 7f73b9a00640 1 -- 172.21.15.50:0/3029062028 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f73b40b2570 con 0x7f73b4153c70 2024-09-21T06:09:44.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.301+0000 7f73ab400640 1 -- 172.21.15.50:0/3029062028 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) ==== 84+0+65 (secure 0 0 0) 0x7f739c0fb1b0 con 0x7f73b4153c70 2024-09-21T06:09:44.303 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:44.303 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:09:44.303 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:09:44.303 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-09-21T06:09:44.303 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-09-21T06:09:44.304 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:09:44.304 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.305+0000 7f73b9a00640 1 -- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7390074ca0 msgr2=0x7f73900771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.305+0000 7f73b9a00640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7390074ca0 0x7f73900771b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f73a4000c10 tx=0x7f73a4006b20 comp rx=0 tx=0).stop 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.305+0000 7f73b9a00640 1 -- 172.21.15.50:0/3029062028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73b4153c70 msgr2=0x7f73b41f6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.305+0000 7f73b9a00640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73b4153c70 0x7f73b41f6770 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f739c098c00 tx=0x7f739c009820 comp rx=0 tx=0).stop 2024-09-21T06:09:44.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.305+0000 7f73b9a00640 1 -- 172.21.15.50:0/3029062028 shutdown_connections 2024-09-21T06:09:44.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.305+0000 7f73b9a00640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73b41f6cb0 0x7f73b41f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.305+0000 7f73b9a00640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7390074ca0 0x7f73900771b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.305+0000 7f73b9a00640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73b4153c70 0x7f73b41f6770 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.305+0000 7f73b9a00640 1 --2- 172.21.15.50:0/3029062028 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7398003fa0 0x7f73b41612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.305+0000 7f73b9a00640 1 -- 172.21.15.50:0/3029062028 >> 172.21.15.50:0/3029062028 conn(0x7f73b40b2040 msgr2=0x7f73b40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:44.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.305+0000 7f73b9a00640 1 -- 172.21.15.50:0/3029062028 shutdown_connections 2024-09-21T06:09:44.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.305+0000 7f73b9a00640 1 -- 172.21.15.50:0/3029062028 wait complete. 2024-09-21T06:09:44.326 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:09:44.326 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:09:44.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.497+0000 7fb014000640 1 -- 172.21.15.50:0/429812741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c160d50 msgr2=0x7fb00c074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:44.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.497+0000 7fb014000640 1 --2- 172.21.15.50:0/429812741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c160d50 0x7fb00c074410 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0713e0 tx=0x7faff801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.497+0000 7fb014000640 1 -- 172.21.15.50:0/429812741 shutdown_connections 2024-09-21T06:09:44.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.497+0000 7fb014000640 1 --2- 172.21.15.50:0/429812741 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb00c161160 0x7fb00c074950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.497+0000 7fb014000640 1 --2- 172.21.15.50:0/429812741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c160d50 0x7fb00c074410 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.497+0000 7fb014000640 1 --2- 172.21.15.50:0/429812741 >> v2:172.21.15.88:3300/0 conn(0x7fb00c157030 0x7fb00c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.497+0000 7fb014000640 1 -- 172.21.15.50:0/429812741 >> 172.21.15.50:0/429812741 conn(0x7fb00c0708c0 msgr2=0x7fb00c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:44.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.497+0000 7fb014000640 1 -- 172.21.15.50:0/429812741 shutdown_connections 2024-09-21T06:09:44.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.497+0000 7fb014000640 1 -- 172.21.15.50:0/429812741 wait complete. 2024-09-21T06:09:44.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.497+0000 7fb014000640 1 Processor -- start 2024-09-21T06:09:44.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.497+0000 7fb014000640 1 -- start start 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb014000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c157030 0x7fb00c13cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb014000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb00c160d50 0x7fb00c152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb014000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb00c161160 0x7fb00c153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb014000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c072180 con 0x7fb00c157030 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb014000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c072000 con 0x7fb00c160d50 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb014000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb00c072300 con 0x7fb00c161160 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb011600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c157030 0x7fb00c13cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb011600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c157030 0x7fb00c13cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34720/0 (socket says 172.21.15.50:34720) 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb011600640 1 -- 172.21.15.50:0/291251885 learned_addr learned my addr 172.21.15.50:0/291251885 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb010c00640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb00c160d50 0x7fb00c152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb012000640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb00c161160 0x7fb00c153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb011600640 1 -- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb00c161160 msgr2=0x7fb00c153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb011600640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb00c161160 0x7fb00c153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb011600640 1 -- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb00c160d50 msgr2=0x7fb00c152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:44.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb011600640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb00c160d50 0x7fb00c152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb011600640 1 -- 172.21.15.50:0/291251885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c14d030 con 0x7fb00c157030 2024-09-21T06:09:44.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb010c00640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb00c160d50 0x7fb00c152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:44.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb011600640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c157030 0x7fb00c13cd70 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb00000e7f0 tx=0x7fb00000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:44.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb009600640 1 -- 172.21.15.50:0/291251885 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00000eea0 con 0x7fb00c157030 2024-09-21T06:09:44.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb009600640 1 -- 172.21.15.50:0/291251885 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb00000cdb0 con 0x7fb00c157030 2024-09-21T06:09:44.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb014000640 1 -- 172.21.15.50:0/291251885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c14d310 con 0x7fb00c157030 2024-09-21T06:09:44.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb009600640 1 -- 172.21.15.50:0/291251885 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000010690 con 0x7fb00c157030 2024-09-21T06:09:44.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.501+0000 7fb014000640 1 -- 172.21.15.50:0/291251885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c152350 con 0x7fb00c157030 2024-09-21T06:09:44.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.505+0000 7fb014000640 1 -- 172.21.15.50:0/291251885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c0727b0 con 0x7fb00c157030 2024-09-21T06:09:44.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.505+0000 7fb009600640 1 -- 172.21.15.50:0/291251885 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fb000010830 con 0x7fb00c157030 2024-09-21T06:09:44.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.505+0000 7fb009600640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fafec074c70 0x7fafec077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:44.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.505+0000 7fb009600640 1 -- 172.21.15.50:0/291251885 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4941+0+0 (secure 0 0 0) 0x7fb0000939c0 con 0x7fb00c157030 2024-09-21T06:09:44.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.505+0000 7fb010c00640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fafec074c70 0x7fafec077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:44.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.505+0000 7fb010c00640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fafec074c70 0x7fafec077180 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faff80af000 tx=0x7faff801b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:44.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.509+0000 7fb009600640 1 -- 172.21.15.50:0/291251885 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb00005fe80 con 0x7fb00c157030 2024-09-21T06:09:44.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:44.685+0000 7fb014000640 1 -- 172.21.15.50:0/291251885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb00c0d2260 con 0x7fb00c157030 2024-09-21T06:09:45.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.501+0000 7fb009600640 1 -- 172.21.15.50:0/291251885 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v109) ==== 138+0+0 (secure 0 0 0) 0x7fb0000648b0 con 0x7fb00c157030 2024-09-21T06:09:45.501 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-09-21T06:09:45.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.505+0000 7fb014000640 1 -- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fafec074c70 msgr2=0x7fafec077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:45.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.505+0000 7fb014000640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fafec074c70 0x7fafec077180 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faff80af000 tx=0x7faff801b100 comp rx=0 tx=0).stop 2024-09-21T06:09:45.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.505+0000 7fb014000640 1 -- 172.21.15.50:0/291251885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c157030 msgr2=0x7fb00c13cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:45.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.505+0000 7fb014000640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c157030 0x7fb00c13cd70 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb00000e7f0 tx=0x7fb00000ecb0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.505+0000 7fb014000640 1 -- 172.21.15.50:0/291251885 shutdown_connections 2024-09-21T06:09:45.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.505+0000 7fb014000640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb00c161160 0x7fb00c153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.505+0000 7fb014000640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb00c160d50 0x7fb00c152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.505+0000 7fb014000640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fafec074c70 0x7fafec077180 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.505+0000 7fb014000640 1 --2- 172.21.15.50:0/291251885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb00c157030 0x7fb00c13cd70 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.505+0000 7fb014000640 1 -- 172.21.15.50:0/291251885 >> 172.21.15.50:0/291251885 conn(0x7fb00c0708c0 msgr2=0x7fb00c0b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:45.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.505+0000 7fb014000640 1 -- 172.21.15.50:0/291251885 shutdown_connections 2024-09-21T06:09:45.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.505+0000 7fb014000640 1 -- 172.21.15.50:0/291251885 wait complete. 2024-09-21T06:09:45.536 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:09:45.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.705+0000 7fec30600640 1 -- 172.21.15.50:0/1926405196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec28157030 msgr2=0x7fec280b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:45.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.705+0000 7fec30600640 1 --2- 172.21.15.50:0/1926405196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec28157030 0x7fec280b5a80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fec28160830 tx=0x7fec1c01c8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.705+0000 7fec30600640 1 -- 172.21.15.50:0/1926405196 shutdown_connections 2024-09-21T06:09:45.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.705+0000 7fec30600640 1 --2- 172.21.15.50:0/1926405196 >> v2:172.21.15.88:3300/0 conn(0x7fec28161160 0x7fec28074950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.705+0000 7fec30600640 1 --2- 172.21.15.50:0/1926405196 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec28160d50 0x7fec28074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.705+0000 7fec30600640 1 --2- 172.21.15.50:0/1926405196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec28157030 0x7fec280b5a80 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.705+0000 7fec30600640 1 -- 172.21.15.50:0/1926405196 >> 172.21.15.50:0/1926405196 conn(0x7fec280708c0 msgr2=0x7fec28070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:45.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.705+0000 7fec30600640 1 -- 172.21.15.50:0/1926405196 shutdown_connections 2024-09-21T06:09:45.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.705+0000 7fec30600640 1 -- 172.21.15.50:0/1926405196 wait complete. 2024-09-21T06:09:45.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec30600640 1 Processor -- start 2024-09-21T06:09:45.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec30600640 1 -- start start 2024-09-21T06:09:45.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec30600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec28157030 0x7fec2815c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:45.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec30600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec28160d50 0x7fec28137210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:45.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec30600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec28161160 0x7fec28137750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:45.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec30600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fec28072290 con 0x7fec28160d50 2024-09-21T06:09:45.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec30600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec28072110 con 0x7fec28157030 2024-09-21T06:09:45.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec30600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fec28072410 con 0x7fec28161160 2024-09-21T06:09:45.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec2d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec28160d50 0x7fec28137210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:45.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec2d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec28160d50 0x7fec28137210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34732/0 (socket says 172.21.15.50:34732) 2024-09-21T06:09:45.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec2dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec28157030 0x7fec2815c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:45.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec2d200640 1 -- 172.21.15.50:0/2150314549 learned_addr learned my addr 172.21.15.50:0/2150314549 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec2e600640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec28161160 0x7fec28137750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec2d200640 1 -- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec28161160 msgr2=0x7fec28137750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec2d200640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec28161160 0x7fec28137750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec2d200640 1 -- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec28157030 msgr2=0x7fec2815c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec2d200640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec28157030 0x7fec2815c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec2d200640 1 -- 172.21.15.50:0/2150314549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec28136a50 con 0x7fec28160d50 2024-09-21T06:09:45.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec2e600640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec28161160 0x7fec28137750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec2d200640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec28160d50 0x7fec28137210 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fec1400d630 tx=0x7fec1400daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec25600640 1 -- 172.21.15.50:0/2150314549 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14017070 con 0x7fec28160d50 2024-09-21T06:09:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec25600640 1 -- 172.21.15.50:0/2150314549 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec14009a70 con 0x7fec28160d50 2024-09-21T06:09:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec25600640 1 -- 172.21.15.50:0/2150314549 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14010460 con 0x7fec28160d50 2024-09-21T06:09:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.709+0000 7fec30600640 1 -- 172.21.15.50:0/2150314549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2814bba0 con 0x7fec28160d50 2024-09-21T06:09:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.713+0000 7fec30600640 1 -- 172.21.15.50:0/2150314549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2814c160 con 0x7fec28160d50 2024-09-21T06:09:45.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.713+0000 7fec30600640 1 -- 172.21.15.50:0/2150314549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec280728e0 con 0x7fec28160d50 2024-09-21T06:09:45.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.713+0000 7fec25600640 1 -- 172.21.15.50:0/2150314549 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fec1401a070 con 0x7fec28160d50 2024-09-21T06:09:45.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.717+0000 7fec25600640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec08074c70 0x7fec08077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:45.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.717+0000 7fec25600640 1 -- 172.21.15.50:0/2150314549 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5311+0+0 (secure 0 0 0) 0x7fec140934f0 con 0x7fec28160d50 2024-09-21T06:09:45.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.717+0000 7fec2dc00640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec08074c70 0x7fec08077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:45.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.717+0000 7fec2dc00640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec08074c70 0x7fec08077180 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0af000 tx=0x7fec1c0a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:45.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.721+0000 7fec25600640 1 -- 172.21.15.50:0/2150314549 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fec1405fa30 con 0x7fec28160d50 2024-09-21T06:09:45.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:45.897+0000 7fec30600640 1 -- 172.21.15.50:0/2150314549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fec280b6d60 con 0x7fec28160d50 2024-09-21T06:09:46.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.505+0000 7fec25600640 1 -- 172.21.15.50:0/2150314549 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v110) ==== 170+0+0 (secure 0 0 0) 0x7fec14064460 con 0x7fec28160d50 2024-09-21T06:09:46.504 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-09-21T06:09:46.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.505+0000 7fec30600640 1 -- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec08074c70 msgr2=0x7fec08077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.505+0000 7fec30600640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec08074c70 0x7fec08077180 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0af000 tx=0x7fec1c0a7040 comp rx=0 tx=0).stop 2024-09-21T06:09:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.505+0000 7fec30600640 1 -- 172.21.15.50:0/2150314549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec28160d50 msgr2=0x7fec28137210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.505+0000 7fec30600640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec28160d50 0x7fec28137210 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fec1400d630 tx=0x7fec1400daf0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.509+0000 7fec30600640 1 -- 172.21.15.50:0/2150314549 shutdown_connections 2024-09-21T06:09:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.509+0000 7fec30600640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec28161160 0x7fec28137750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.509+0000 7fec30600640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fec08074c70 0x7fec08077180 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.509+0000 7fec30600640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec28160d50 0x7fec28137210 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.509+0000 7fec30600640 1 --2- 172.21.15.50:0/2150314549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec28157030 0x7fec2815c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.509+0000 7fec30600640 1 -- 172.21.15.50:0/2150314549 >> 172.21.15.50:0/2150314549 conn(0x7fec280708c0 msgr2=0x7fec280b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.509+0000 7fec30600640 1 -- 172.21.15.50:0/2150314549 shutdown_connections 2024-09-21T06:09:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.509+0000 7fec30600640 1 -- 172.21.15.50:0/2150314549 wait complete. 2024-09-21T06:09:46.550 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:09:46.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.713+0000 7f74e6c00640 1 -- 172.21.15.50:0/4188152082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74e00b4de0 msgr2=0x7f74e00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:46.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.713+0000 7f74e6c00640 1 --2- 172.21.15.50:0/4188152082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74e00b4de0 0x7f74e00b51f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009bc0 tx=0x7f74d801cae0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.713+0000 7f74e6c00640 1 -- 172.21.15.50:0/4188152082 shutdown_connections 2024-09-21T06:09:46.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.713+0000 7f74e6c00640 1 --2- 172.21.15.50:0/4188152082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74e00b4de0 0x7f74e00b51f0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.713+0000 7f74e6c00640 1 --2- 172.21.15.50:0/4188152082 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74e01527b0 0x7f74e015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.713+0000 7f74e6c00640 1 -- 172.21.15.50:0/4188152082 >> 172.21.15.50:0/4188152082 conn(0x7f74e00b2040 msgr2=0x7f74e00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:46.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.713+0000 7f74e6c00640 1 -- 172.21.15.50:0/4188152082 shutdown_connections 2024-09-21T06:09:46.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.713+0000 7f74e6c00640 1 -- 172.21.15.50:0/4188152082 wait complete. 2024-09-21T06:09:46.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74e6c00640 1 Processor -- start 2024-09-21T06:09:46.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74e6c00640 1 -- start start 2024-09-21T06:09:46.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74e6c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74e00b4de0 0x7f74e01fa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:46.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74e6c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74e00b5c70 0x7f74e01466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:46.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74e6c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74e01527b0 0x7f74e015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:46.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74e6c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74e0168a10 con 0x7f74e00b5c70 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74e6c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74e0168890 con 0x7f74e01527b0 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74e6c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f74e0168b90 con 0x7f74e00b4de0 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74d7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74e00b5c70 0x7f74e01466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74d7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74e00b5c70 0x7f74e01466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37790/0 (socket says 172.21.15.50:37790) 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74d7e00640 1 -- 172.21.15.50:0/2168423076 learned_addr learned my addr 172.21.15.50:0/2168423076 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74dfe00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74e00b4de0 0x7f74e01fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74e4c00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74e01527b0 0x7f74e015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74d7e00640 1 -- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74e00b4de0 msgr2=0x7f74e01fa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74d7e00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74e00b4de0 0x7f74e01fa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74d7e00640 1 -- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74e01527b0 msgr2=0x7f74e015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74d7e00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74e01527b0 0x7f74e015a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74d7e00640 1 -- 172.21.15.50:0/2168423076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e014c470 con 0x7f74e00b5c70 2024-09-21T06:09:46.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.717+0000 7f74d7e00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74e00b5c70 0x7f74e01466f0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f74c8005d10 tx=0x7f74c800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:46.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.721+0000 7f74dcc00640 1 -- 172.21.15.50:0/2168423076 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8017070 con 0x7f74e00b5c70 2024-09-21T06:09:46.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.721+0000 7f74dcc00640 1 -- 172.21.15.50:0/2168423076 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74c8009b20 con 0x7f74e00b5c70 2024-09-21T06:09:46.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.721+0000 7f74e6c00640 1 -- 172.21.15.50:0/2168423076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e00ccba0 con 0x7f74e00b5c70 2024-09-21T06:09:46.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.721+0000 7f74dcc00640 1 -- 172.21.15.50:0/2168423076 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8010460 con 0x7f74e00b5c70 2024-09-21T06:09:46.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.721+0000 7f74e6c00640 1 -- 172.21.15.50:0/2168423076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e0146c30 con 0x7f74e00b5c70 2024-09-21T06:09:46.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.721+0000 7f74e6c00640 1 -- 172.21.15.50:0/2168423076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e00b51f0 con 0x7f74e00b5c70 2024-09-21T06:09:46.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.721+0000 7f74dcc00640 1 -- 172.21.15.50:0/2168423076 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f74c801a070 con 0x7f74e00b5c70 2024-09-21T06:09:46.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.721+0000 7f74dcc00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74bc074ca0 0x7f74bc0771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:46.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.721+0000 7f74dcc00640 1 -- 172.21.15.50:0/2168423076 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5693+0+0 (secure 0 0 0) 0x7f74c80935d0 con 0x7f74e00b5c70 2024-09-21T06:09:46.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.725+0000 7f74dfe00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74bc074ca0 0x7f74bc0771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:46.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.725+0000 7f74dfe00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74bc074ca0 0x7f74bc0771b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f74d0007fd0 tx=0x7f74d0007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:46.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.725+0000 7f74dcc00640 1 -- 172.21.15.50:0/2168423076 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f74c805fa90 con 0x7f74e00b5c70 2024-09-21T06:09:46.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:46.941+0000 7f74e6c00640 1 -- 172.21.15.50:0/2168423076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f74e00cff10 con 0x7f74e00b5c70 2024-09-21T06:09:47.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.541+0000 7f74dcc00640 1 -- 172.21.15.50:0/2168423076 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v71) ==== 174+0+0 (secure 0 0 0) 0x7f74c80644c0 con 0x7f74e00b5c70 2024-09-21T06:09:47.544 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 16 and data pool 17 2024-09-21T06:09:47.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.561+0000 7f74e6c00640 1 -- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74bc074ca0 msgr2=0x7f74bc0771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:47.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.561+0000 7f74e6c00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74bc074ca0 0x7f74bc0771b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f74d0007fd0 tx=0x7f74d0007ea0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.561+0000 7f74e6c00640 1 -- 172.21.15.50:0/2168423076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74e00b5c70 msgr2=0x7f74e01466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:47.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.561+0000 7f74e6c00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74e00b5c70 0x7f74e01466f0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f74c8005d10 tx=0x7f74c800dbb0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.561+0000 7f74e6c00640 1 -- 172.21.15.50:0/2168423076 shutdown_connections 2024-09-21T06:09:47.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.561+0000 7f74e6c00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74e01527b0 0x7f74e015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.561+0000 7f74e6c00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f74bc074ca0 0x7f74bc0771b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.561+0000 7f74e6c00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74e00b5c70 0x7f74e01466f0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.561+0000 7f74e6c00640 1 --2- 172.21.15.50:0/2168423076 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f74e00b4de0 0x7f74e01fa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.561+0000 7f74e6c00640 1 -- 172.21.15.50:0/2168423076 >> 172.21.15.50:0/2168423076 conn(0x7f74e00b2040 msgr2=0x7f74e01518a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:47.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.561+0000 7f74e6c00640 1 -- 172.21.15.50:0/2168423076 shutdown_connections 2024-09-21T06:09:47.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.565+0000 7f74e6c00640 1 -- 172.21.15.50:0/2168423076 wait complete. 2024-09-21T06:09:47.595 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:09:47.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.757+0000 7fa99c000640 1 -- 172.21.15.50:0/2568942489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9940b54f0 msgr2=0x7fa99415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:47.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.757+0000 7fa99c000640 1 --2- 172.21.15.50:0/2568942489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9940b54f0 0x7fa99415bb60 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009bc0 tx=0x7fa98c01cae0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.757+0000 7fa99c000640 1 -- 172.21.15.50:0/2568942489 shutdown_connections 2024-09-21T06:09:47.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.757+0000 7fa99c000640 1 --2- 172.21.15.50:0/2568942489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9940b54f0 0x7fa99415bb60 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.757+0000 7fa99c000640 1 -- 172.21.15.50:0/2568942489 >> 172.21.15.50:0/2568942489 conn(0x7fa9940b2040 msgr2=0x7fa9940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:47.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.757+0000 7fa99c000640 1 -- 172.21.15.50:0/2568942489 shutdown_connections 2024-09-21T06:09:47.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.757+0000 7fa99c000640 1 -- 172.21.15.50:0/2568942489 wait complete. 2024-09-21T06:09:47.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.757+0000 7fa99c000640 1 Processor -- start 2024-09-21T06:09:47.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.757+0000 7fa99c000640 1 -- start start 2024-09-21T06:09:47.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99c000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9940b5c70 0x7fa9941f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:47.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99c000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa9941527b0 0x7fa9940b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:47.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99c000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa994158d10 0x7fa9940b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:47.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99c000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa994169bd0 con 0x7fa994158d10 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99c000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa994169a50 con 0x7fa9940b5c70 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99c000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa994169d50 con 0x7fa9941527b0 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa994158d10 0x7fa9940b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa994158d10 0x7fa9940b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37802/0 (socket says 172.21.15.50:37802) 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99a000640 1 -- 172.21.15.50:0/3939643878 learned_addr learned my addr 172.21.15.50:0/3939643878 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa999600640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9940b5c70 0x7fa9941f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa998c00640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa9941527b0 0x7fa9940b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99a000640 1 -- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa9941527b0 msgr2=0x7fa9940b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99a000640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa9941527b0 0x7fa9940b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99a000640 1 -- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9940b5c70 msgr2=0x7fa9941f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99a000640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9940b5c70 0x7fa9941f6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99a000640 1 -- 172.21.15.50:0/3939643878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9940ae480 con 0x7fa994158d10 2024-09-21T06:09:47.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99a000640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa994158d10 0x7fa9940b4960 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0196f0 tx=0x7fa98c0a5b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:47.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa991600640 1 -- 172.21.15.50:0/3939643878 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0bb070 con 0x7fa994158d10 2024-09-21T06:09:47.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa991600640 1 -- 172.21.15.50:0/3939643878 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa98c0a5e50 con 0x7fa994158d10 2024-09-21T06:09:47.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa991600640 1 -- 172.21.15.50:0/3939643878 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0b2040 con 0x7fa994158d10 2024-09-21T06:09:47.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99c000640 1 -- 172.21.15.50:0/3939643878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9940ae740 con 0x7fa994158d10 2024-09-21T06:09:47.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99c000640 1 -- 172.21.15.50:0/3939643878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9940aed00 con 0x7fa994158d10 2024-09-21T06:09:47.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.761+0000 7fa99c000640 1 -- 172.21.15.50:0/3939643878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa99415bb60 con 0x7fa994158d10 2024-09-21T06:09:47.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.765+0000 7fa991600640 1 -- 172.21.15.50:0/3939643878 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fa98c0040c0 con 0x7fa994158d10 2024-09-21T06:09:47.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.765+0000 7fa991600640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa974074cd0 0x7fa9740771e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:47.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.765+0000 7fa991600640 1 -- 172.21.15.50:0/3939643878 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5793+0+0 (secure 0 0 0) 0x7fa98c12ce80 con 0x7fa994158d10 2024-09-21T06:09:47.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.765+0000 7fa999600640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa974074cd0 0x7fa9740771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:47.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.765+0000 7fa999600640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa974074cd0 0x7fa9740771e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa988007fd0 tx=0x7fa988009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:47.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.769+0000 7fa991600640 1 -- 172.21.15.50:0/3939643878 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa98c016790 con 0x7fa994158d10 2024-09-21T06:09:47.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.949+0000 7fa99c000640 1 -- 172.21.15.50:0/3939643878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9940cc100 con 0x7fa994158d10 2024-09-21T06:09:47.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.949+0000 7fa991600640 1 -- 172.21.15.50:0/3939643878 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16785 (secure 0 0 0) 0x7fa98c0f92b0 con 0x7fa994158d10 2024-09-21T06:09:47.950 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:47.950 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":111,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:09:46.946876+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:44.704087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:45.912566+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:09:47.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.949+0000 7fa99c000640 1 -- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa974074cd0 msgr2=0x7fa9740771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:47.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.949+0000 7fa99c000640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa974074cd0 0x7fa9740771e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa988007fd0 tx=0x7fa988009040 comp rx=0 tx=0).stop 2024-09-21T06:09:47.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.949+0000 7fa99c000640 1 -- 172.21.15.50:0/3939643878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa994158d10 msgr2=0x7fa9940b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:47.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.949+0000 7fa99c000640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa994158d10 0x7fa9940b4960 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0196f0 tx=0x7fa98c0a5b10 comp rx=0 tx=0).stop 2024-09-21T06:09:47.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.953+0000 7fa99c000640 1 -- 172.21.15.50:0/3939643878 shutdown_connections 2024-09-21T06:09:47.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.953+0000 7fa99c000640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa974074cd0 0x7fa9740771e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.953+0000 7fa99c000640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa994158d10 0x7fa9940b4960 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.953+0000 7fa99c000640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa9941527b0 0x7fa9940b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.953+0000 7fa99c000640 1 --2- 172.21.15.50:0/3939643878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9940b5c70 0x7fa9941f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.953+0000 7fa99c000640 1 -- 172.21.15.50:0/3939643878 >> 172.21.15.50:0/3939643878 conn(0x7fa9940b2040 msgr2=0x7fa99415b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:47.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.953+0000 7fa99c000640 1 -- 172.21.15.50:0/3939643878 shutdown_connections 2024-09-21T06:09:47.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:47.953+0000 7fa99c000640 1 -- 172.21.15.50:0/3939643878 wait complete. 2024-09-21T06:09:47.973 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:09:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.129+0000 7fbdc9e00640 1 -- 172.21.15.50:0/115745033 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdc40b5a80 msgr2=0x7fbdc4074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.129+0000 7fbdc9e00640 1 --2- 172.21.15.50:0/115745033 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdc40b5a80 0x7fbdc4074410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009900 tx=0x7fbdac01c8e0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.129+0000 7fbdc9e00640 1 -- 172.21.15.50:0/115745033 shutdown_connections 2024-09-21T06:09:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.129+0000 7fbdc9e00640 1 --2- 172.21.15.50:0/115745033 >> v2:172.21.15.88:3300/0 conn(0x7fbdc4074950 0x7fbdc415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.129+0000 7fbdc9e00640 1 --2- 172.21.15.50:0/115745033 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdc40b5a80 0x7fbdc4074410 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.129+0000 7fbdc9e00640 1 -- 172.21.15.50:0/115745033 >> 172.21.15.50:0/115745033 conn(0x7fbdc40708c0 msgr2=0x7fbdc4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:48.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.129+0000 7fbdc9e00640 1 -- 172.21.15.50:0/115745033 shutdown_connections 2024-09-21T06:09:48.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.129+0000 7fbdc9e00640 1 -- 172.21.15.50:0/115745033 wait complete. 2024-09-21T06:09:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc9e00640 1 Processor -- start 2024-09-21T06:09:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc9e00640 1 -- start start 2024-09-21T06:09:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdc4074950 0x7fbdc4155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdc40b5a80 0x7fbdc4148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc9e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdc4157030 0x7fbdc415bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc9e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc416a310 con 0x7fbdc4074950 2024-09-21T06:09:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc9e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc416a190 con 0x7fbdc40b5a80 2024-09-21T06:09:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc9e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc416a490 con 0x7fbdc4157030 2024-09-21T06:09:48.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdc4074950 0x7fbdc4155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:48.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdc4074950 0x7fbdc4155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37818/0 (socket says 172.21.15.50:37818) 2024-09-21T06:09:48.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc2a00640 1 -- 172.21.15.50:0/278250672 learned_addr learned my addr 172.21.15.50:0/278250672 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:48.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdbbe00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdc40b5a80 0x7fbdc4148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:48.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc3400640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdc4157030 0x7fbdc415bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc2a00640 1 -- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdc4157030 msgr2=0x7fbdc415bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc2a00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdc4157030 0x7fbdc415bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc2a00640 1 -- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdc40b5a80 msgr2=0x7fbdc4148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.133+0000 7fbdc2a00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdc40b5a80 0x7fbdc4148b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.137+0000 7fbdc2a00640 1 -- 172.21.15.50:0/278250672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc414a130 con 0x7fbdc4074950 2024-09-21T06:09:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.137+0000 7fbdbbe00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdc40b5a80 0x7fbdc4148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:09:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.137+0000 7fbdc2a00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdc4074950 0x7fbdc4155d00 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400c3c0 tx=0x7fbdb400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.137+0000 7fbdbb400640 1 -- 172.21.15.50:0/278250672 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4017070 con 0x7fbdc4074950 2024-09-21T06:09:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.137+0000 7fbdc9e00640 1 -- 172.21.15.50:0/278250672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc414b380 con 0x7fbdc4074950 2024-09-21T06:09:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.137+0000 7fbdbb400640 1 -- 172.21.15.50:0/278250672 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdb40053c0 con 0x7fbdc4074950 2024-09-21T06:09:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.137+0000 7fbdbb400640 1 -- 172.21.15.50:0/278250672 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4003860 con 0x7fbdc4074950 2024-09-21T06:09:48.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.137+0000 7fbdc9e00640 1 -- 172.21.15.50:0/278250672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc415f8d0 con 0x7fbdc4074950 2024-09-21T06:09:48.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.137+0000 7fbdc9e00640 1 -- 172.21.15.50:0/278250672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc415cf90 con 0x7fbdc4074950 2024-09-21T06:09:48.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.137+0000 7fbdbb400640 1 -- 172.21.15.50:0/278250672 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fbdb401a070 con 0x7fbdc4074950 2024-09-21T06:09:48.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.137+0000 7fbdbb400640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbda0074ca0 0x7fbda00771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:48.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.141+0000 7fbdbb400640 1 -- 172.21.15.50:0/278250672 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5793+0+0 (secure 0 0 0) 0x7fbdb40923f0 con 0x7fbdc4074950 2024-09-21T06:09:48.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.141+0000 7fbdbbe00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbda0074ca0 0x7fbda00771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:48.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.141+0000 7fbdbbe00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbda0074ca0 0x7fbda00771b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbdac0af000 tx=0x7fbdac01b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:48.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.145+0000 7fbdbb400640 1 -- 172.21.15.50:0/278250672 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbdc4074d60 con 0x7fbdc4074950 2024-09-21T06:09:48.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.321+0000 7fbdc9e00640 1 -- 172.21.15.50:0/278250672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdc40d2260 con 0x7fbdc4074950 2024-09-21T06:09:48.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.321+0000 7fbdbb400640 1 -- 172.21.15.50:0/278250672 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16785 (secure 0 0 0) 0x7fbdb405e830 con 0x7fbdc4074950 2024-09-21T06:09:48.324 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:48.324 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":111,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:09:46.946876+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:44.704087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:45.912566+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:09:48.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.325+0000 7fbdc9e00640 1 -- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbda0074ca0 msgr2=0x7fbda00771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:48.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.325+0000 7fbdc9e00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbda0074ca0 0x7fbda00771b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbdac0af000 tx=0x7fbdac01b120 comp rx=0 tx=0).stop 2024-09-21T06:09:48.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.325+0000 7fbdc9e00640 1 -- 172.21.15.50:0/278250672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdc4074950 msgr2=0x7fbdc4155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:48.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.325+0000 7fbdc9e00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdc4074950 0x7fbdc4155d00 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400c3c0 tx=0x7fbdb400c7c0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.325+0000 7fbdc9e00640 1 -- 172.21.15.50:0/278250672 shutdown_connections 2024-09-21T06:09:48.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.325+0000 7fbdc9e00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbdc4157030 0x7fbdc415bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.325+0000 7fbdc9e00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdc40b5a80 0x7fbdc4148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.325+0000 7fbdc9e00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbda0074ca0 0x7fbda00771b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.325+0000 7fbdc9e00640 1 --2- 172.21.15.50:0/278250672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdc4074950 0x7fbdc4155d00 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.325+0000 7fbdc9e00640 1 -- 172.21.15.50:0/278250672 >> 172.21.15.50:0/278250672 conn(0x7fbdc40708c0 msgr2=0x7fbdc4150440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:48.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.325+0000 7fbdc9e00640 1 -- 172.21.15.50:0/278250672 shutdown_connections 2024-09-21T06:09:48.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.325+0000 7fbdc9e00640 1 -- 172.21.15.50:0/278250672 wait complete. 2024-09-21T06:09:48.347 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:09:48.493 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-09-21T06:09:48.493+0000 7fb6e1c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:09:48.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.509+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3278688678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc004810 msgr2=0x7f0ccc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:48.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.509+0000 7f0ce7e00640 1 --2- 172.21.15.50:0/3278688678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc004810 0x7f0ccc004c20 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009b10 tx=0x7f0cd001d230 comp rx=0 tx=0).stop 2024-09-21T06:09:48.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.509+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3278688678 shutdown_connections 2024-09-21T06:09:48.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.509+0000 7f0ce7e00640 1 --2- 172.21.15.50:0/3278688678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc004810 0x7f0ccc004c20 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.509+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3278688678 >> 172.21.15.50:0/3278688678 conn(0x7f0ce80b4560 msgr2=0x7f0ce80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:48.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.509+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3278688678 shutdown_connections 2024-09-21T06:09:48.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.509+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3278688678 wait complete. 2024-09-21T06:09:48.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0ce7e00640 1 Processor -- start 2024-09-21T06:09:48.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0ce7e00640 1 -- start start 2024-09-21T06:09:48.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc004810 0x7f0ce8151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:48.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce806e5b0 0x7f0ce806e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:48.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ce815b4b0 0x7f0ce815b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:48.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0ce7e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce80b5c50 con 0x7f0ccc004810 2024-09-21T06:09:48.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0ce7e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce80b5ad0 con 0x7f0ce806e5b0 2024-09-21T06:09:48.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0ce7e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0ce80b5dd0 con 0x7f0ce815b4b0 2024-09-21T06:09:48.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0ce6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc004810 0x7f0ce8151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:48.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0ce6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc004810 0x7f0ce8151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37838/0 (socket says 172.21.15.50:37838) 2024-09-21T06:09:48.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0ce6a00640 1 -- 172.21.15.50:0/3448298264 learned_addr learned my addr 172.21.15.50:0/3448298264 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:48.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0cdfe00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce806e5b0 0x7f0ce806e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:48.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.513+0000 7f0ce7400640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ce815b4b0 0x7f0ce815b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:48.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0ce6a00640 1 -- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ce815b4b0 msgr2=0x7f0ce815b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:48.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0ce6a00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ce815b4b0 0x7f0ce815b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0ce6a00640 1 -- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce806e5b0 msgr2=0x7f0ce806e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:48.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0ce6a00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce806e5b0 0x7f0ce806e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:48.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0ce6a00640 1 -- 172.21.15.50:0/3448298264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce815be00 con 0x7f0ccc004810 2024-09-21T06:09:48.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0ce6a00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc004810 0x7f0ce8151040 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8006690 tx=0x7f0cd8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:48.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0cdea00640 1 -- 172.21.15.50:0/3448298264 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8015070 con 0x7f0ccc004810 2024-09-21T06:09:48.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0cdea00640 1 -- 172.21.15.50:0/3448298264 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cd8011840 con 0x7f0ccc004810 2024-09-21T06:09:48.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0cdea00640 1 -- 172.21.15.50:0/3448298264 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8011c70 con 0x7f0ccc004810 2024-09-21T06:09:48.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3448298264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce81fc3b0 con 0x7f0ccc004810 2024-09-21T06:09:48.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3448298264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce81fc970 con 0x7f0ccc004810 2024-09-21T06:09:48.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3448298264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce8071be0 con 0x7f0ccc004810 2024-09-21T06:09:48.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.517+0000 7f0cdea00640 1 -- 172.21.15.50:0/3448298264 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f0cd8018070 con 0x7f0ccc004810 2024-09-21T06:09:48.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.521+0000 7f0cdea00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0cc4074c70 0x7f0cc4077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:48.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.521+0000 7f0cdea00640 1 -- 172.21.15.50:0/3448298264 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5793+0+0 (secure 0 0 0) 0x7f0cd8093eb0 con 0x7f0ccc004810 2024-09-21T06:09:48.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.521+0000 7f0cdfe00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0cc4074c70 0x7f0cc4077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:48.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.521+0000 7f0cdfe00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0cc4074c70 0x7f0cc4077180 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00ad000 tx=0x7f0cd0019fd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:48.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.521+0000 7f0cdea00640 1 -- 172.21.15.50:0/3448298264 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0ce815be00 con 0x7f0ccc004810 2024-09-21T06:09:48.537 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:09:48.541 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:09:48.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:48.737+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3448298264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0ce80d2fe0 con 0x7f0ccc004810 2024-09-21T06:09:49.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.549+0000 7f0cdea00640 1 -- 172.21.15.50:0/3448298264 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) ==== 118+0+0 (secure 0 0 0) 0x7f0cd8060360 con 0x7f0ccc004810 2024-09-21T06:09:49.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.553+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0cc4074c70 msgr2=0x7f0cc4077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:49.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.553+0000 7f0ce7e00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0cc4074c70 0x7f0cc4077180 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00ad000 tx=0x7f0cd0019fd0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.553+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3448298264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc004810 msgr2=0x7f0ce8151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:49.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.553+0000 7f0ce7e00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc004810 0x7f0ce8151040 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8006690 tx=0x7f0cd8001640 comp rx=0 tx=0).stop 2024-09-21T06:09:49.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.553+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3448298264 shutdown_connections 2024-09-21T06:09:49.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.553+0000 7f0ce7e00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ce815b4b0 0x7f0ce815b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.553+0000 7f0ce7e00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce806e5b0 0x7f0ce806e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.553+0000 7f0ce7e00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0cc4074c70 0x7f0cc4077180 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.553+0000 7f0ce7e00640 1 --2- 172.21.15.50:0/3448298264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc004810 0x7f0ce8151040 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.553+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3448298264 >> 172.21.15.50:0/3448298264 conn(0x7f0ce80b4560 msgr2=0x7f0ce80d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:49.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.553+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3448298264 shutdown_connections 2024-09-21T06:09:49.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.553+0000 7f0ce7e00640 1 -- 172.21.15.50:0/3448298264 wait complete. 2024-09-21T06:09:49.578 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:09:49.578 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:09:49.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.745+0000 7fc584200640 1 -- 172.21.15.50:0/11131235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c0b5c70 msgr2=0x7fc57c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:49.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.745+0000 7fc584200640 1 --2- 172.21.15.50:0/11131235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c0b5c70 0x7fc57c15bb60 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc570009900 tx=0x7fc57001c8b0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.745+0000 7fc584200640 1 -- 172.21.15.50:0/11131235 shutdown_connections 2024-09-21T06:09:49.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.745+0000 7fc584200640 1 --2- 172.21.15.50:0/11131235 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc57c0b4de0 0x7fc57c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.745+0000 7fc584200640 1 --2- 172.21.15.50:0/11131235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c0b5c70 0x7fc57c15bb60 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.745+0000 7fc584200640 1 -- 172.21.15.50:0/11131235 >> 172.21.15.50:0/11131235 conn(0x7fc57c0b2040 msgr2=0x7fc57c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:49.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.745+0000 7fc584200640 1 -- 172.21.15.50:0/11131235 shutdown_connections 2024-09-21T06:09:49.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.745+0000 7fc584200640 1 -- 172.21.15.50:0/11131235 wait complete. 2024-09-21T06:09:49.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.745+0000 7fc584200640 1 Processor -- start 2024-09-21T06:09:49.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.745+0000 7fc584200640 1 -- start start 2024-09-21T06:09:49.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc584200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc57c0b4de0 0x7fc57c14be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:49.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc584200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc57c0b5c70 0x7fc57c1fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:49.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc584200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c1527b0 0x7fc57c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:49.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc584200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c168970 con 0x7fc57c1527b0 2024-09-21T06:09:49.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc584200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c1687f0 con 0x7fc57c0b4de0 2024-09-21T06:09:49.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc584200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc57c168af0 con 0x7fc57c0b5c70 2024-09-21T06:09:49.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc582200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c1527b0 0x7fc57c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:49.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc582200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c1527b0 0x7fc57c0cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37848/0 (socket says 172.21.15.50:37848) 2024-09-21T06:09:49.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc582200640 1 -- 172.21.15.50:0/1354153845 learned_addr learned my addr 172.21.15.50:0/1354153845 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:49.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc581800640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc57c0b4de0 0x7fc57c14be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:49.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc580e00640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc57c0b5c70 0x7fc57c1fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:49.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc582200640 1 -- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc57c0b5c70 msgr2=0x7fc57c1fa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:49.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc582200640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc57c0b5c70 0x7fc57c1fa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc582200640 1 -- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc57c0b4de0 msgr2=0x7fc57c14be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:49.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc582200640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc57c0b4de0 0x7fc57c14be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc582200640 1 -- 172.21.15.50:0/1354153845 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57c14fce0 con 0x7fc57c1527b0 2024-09-21T06:09:49.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc581800640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc57c0b4de0 0x7fc57c14be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:49.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc582200640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c1527b0 0x7fc57c0cd4f0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc57400e940 tx=0x7fc57400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:49.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc579600640 1 -- 172.21.15.50:0/1354153845 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574004d30 con 0x7fc57c1527b0 2024-09-21T06:09:49.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc579600640 1 -- 172.21.15.50:0/1354153845 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc57400cac0 con 0x7fc57c1527b0 2024-09-21T06:09:49.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc584200640 1 -- 172.21.15.50:0/1354153845 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57c14ffc0 con 0x7fc57c1527b0 2024-09-21T06:09:49.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc579600640 1 -- 172.21.15.50:0/1354153845 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574005020 con 0x7fc57c1527b0 2024-09-21T06:09:49.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc584200640 1 -- 172.21.15.50:0/1354153845 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc57c1f9df0 con 0x7fc57c1527b0 2024-09-21T06:09:49.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.749+0000 7fc584200640 1 -- 172.21.15.50:0/1354153845 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc57c0b51f0 con 0x7fc57c1527b0 2024-09-21T06:09:49.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.753+0000 7fc579600640 1 -- 172.21.15.50:0/1354153845 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fc5740051c0 con 0x7fc57c1527b0 2024-09-21T06:09:49.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.753+0000 7fc579600640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc55c074c70 0x7fc55c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:49.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.753+0000 7fc579600640 1 -- 172.21.15.50:0/1354153845 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fc574093a80 con 0x7fc57c1527b0 2024-09-21T06:09:49.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.753+0000 7fc581800640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc55c074c70 0x7fc55c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:49.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.753+0000 7fc581800640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc55c074c70 0x7fc55c077180 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc5700af000 tx=0x7fc57001b0f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:49.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.757+0000 7fc579600640 1 -- 172.21.15.50:0/1354153845 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc574016030 con 0x7fc57c1527b0 2024-09-21T06:09:49.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:49.969+0000 7fc584200640 1 -- 172.21.15.50:0/1354153845 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc57c1505b0 con 0x7fc57c1527b0 2024-09-21T06:09:50.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc579600640 1 -- 172.21.15.50:0/1354153845 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) ==== 122+0+0 (secure 0 0 0) 0x7fc574060590 con 0x7fc57c1527b0 2024-09-21T06:09:50.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc584200640 1 -- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc55c074c70 msgr2=0x7fc55c077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:50.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc584200640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc55c074c70 0x7fc55c077180 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc5700af000 tx=0x7fc57001b0f0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc584200640 1 -- 172.21.15.50:0/1354153845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c1527b0 msgr2=0x7fc57c0cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:50.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc584200640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c1527b0 0x7fc57c0cd4f0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc57400e940 tx=0x7fc57400ee00 comp rx=0 tx=0).stop 2024-09-21T06:09:50.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc584200640 1 -- 172.21.15.50:0/1354153845 shutdown_connections 2024-09-21T06:09:50.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc584200640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc55c074c70 0x7fc55c077180 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc584200640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c1527b0 0x7fc57c0cd4f0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc584200640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc57c0b5c70 0x7fc57c1fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc584200640 1 --2- 172.21.15.50:0/1354153845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc57c0b4de0 0x7fc57c14be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc584200640 1 -- 172.21.15.50:0/1354153845 >> 172.21.15.50:0/1354153845 conn(0x7fc57c0b2040 msgr2=0x7fc57c15a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:50.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc584200640 1 -- 172.21.15.50:0/1354153845 shutdown_connections 2024-09-21T06:09:50.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.553+0000 7fc584200640 1 -- 172.21.15.50:0/1354153845 wait complete. 2024-09-21T06:09:50.580 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:09:50.580 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:09:50.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.745+0000 7f53c9a00640 1 -- 172.21.15.50:0/2364974071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c4074950 msgr2=0x7f53c415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:50.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.745+0000 7f53c9a00640 1 --2- 172.21.15.50:0/2364974071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c4074950 0x7f53c415cf90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f53c40b1740 tx=0x7f53bc01ccf0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.745+0000 7f53c9a00640 1 -- 172.21.15.50:0/2364974071 shutdown_connections 2024-09-21T06:09:50.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.745+0000 7f53c9a00640 1 --2- 172.21.15.50:0/2364974071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c4074950 0x7f53c415cf90 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.745+0000 7f53c9a00640 1 --2- 172.21.15.50:0/2364974071 >> v2:172.21.15.88:3300/0 conn(0x7f53c40b5a80 0x7f53c4074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.745+0000 7f53c9a00640 1 -- 172.21.15.50:0/2364974071 >> 172.21.15.50:0/2364974071 conn(0x7f53c40708c0 msgr2=0x7f53c4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:50.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.745+0000 7f53c9a00640 1 -- 172.21.15.50:0/2364974071 shutdown_connections 2024-09-21T06:09:50.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.745+0000 7f53c9a00640 1 -- 172.21.15.50:0/2364974071 wait complete. 2024-09-21T06:09:50.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.745+0000 7f53c9a00640 1 Processor -- start 2024-09-21T06:09:50.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.745+0000 7f53c9a00640 1 -- start start 2024-09-21T06:09:50.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c9a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f53c4074950 0x7f53c4166eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:50.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c40b5a80 0x7f53c413e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:50.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c4157030 0x7f53c413eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:50.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c9a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f53c416a310 con 0x7f53c4157030 2024-09-21T06:09:50.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c9a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f53c416a190 con 0x7f53c40b5a80 2024-09-21T06:09:50.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c9a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f53c416a490 con 0x7f53c4074950 2024-09-21T06:09:50.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c4157030 0x7f53c413eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:50.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c4157030 0x7f53c413eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37854/0 (socket says 172.21.15.50:37854) 2024-09-21T06:09:50.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c3400640 1 -- 172.21.15.50:0/2887938526 learned_addr learned my addr 172.21.15.50:0/2887938526 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:50.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c2000640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c40b5a80 0x7f53c413e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:50.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c3400640 1 -- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f53c4074950 msgr2=0x7f53c4166eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:50.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c3400640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f53c4074950 0x7f53c4166eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c3400640 1 -- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c40b5a80 msgr2=0x7f53c413e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:50.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c3400640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c40b5a80 0x7f53c413e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c3400640 1 -- 172.21.15.50:0/2887938526 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c4151550 con 0x7f53c4157030 2024-09-21T06:09:50.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c2000640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c40b5a80 0x7f53c413e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:09:50.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c3400640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c4157030 0x7f53c413eaa0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009bd0 tx=0x7f53bc005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:50.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53baa00640 1 -- 172.21.15.50:0/2887938526 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0b2040 con 0x7f53c4157030 2024-09-21T06:09:50.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53baa00640 1 -- 172.21.15.50:0/2887938526 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53bc004000 con 0x7f53c4157030 2024-09-21T06:09:50.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c9a00640 1 -- 172.21.15.50:0/2887938526 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c4151830 con 0x7f53c4157030 2024-09-21T06:09:50.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53baa00640 1 -- 172.21.15.50:0/2887938526 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc004410 con 0x7f53c4157030 2024-09-21T06:09:50.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c9a00640 1 -- 172.21.15.50:0/2887938526 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c414e590 con 0x7f53c4157030 2024-09-21T06:09:50.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.749+0000 7f53c9a00640 1 -- 172.21.15.50:0/2887938526 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c414cb00 con 0x7f53c4157030 2024-09-21T06:09:50.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.753+0000 7f53baa00640 1 -- 172.21.15.50:0/2887938526 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f53bc0045b0 con 0x7f53c4157030 2024-09-21T06:09:50.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.753+0000 7f53baa00640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53a4074c70 0x7f53a4077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:50.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.753+0000 7f53baa00640 1 -- 172.21.15.50:0/2887938526 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f53bc12e360 con 0x7f53c4157030 2024-09-21T06:09:50.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.753+0000 7f53c2a00640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53a4074c70 0x7f53a4077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:50.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.753+0000 7f53c2a00640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53a4074c70 0x7f53a4077180 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f53b4000f70 tx=0x7f53b40079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:50.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.757+0000 7f53baa00640 1 -- 172.21.15.50:0/2887938526 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f53bc016970 con 0x7f53c4157030 2024-09-21T06:09:50.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.969+0000 7f53c9a00640 1 -- 172.21.15.50:0/2887938526 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53c415cf90 con 0x7f53c4157030 2024-09-21T06:09:50.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.969+0000 7f53baa00640 1 -- 172.21.15.50:0/2887938526 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5225 (secure 0 0 0) 0x7f53bc0faef0 con 0x7f53c4157030 2024-09-21T06:09:50.971 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:50.971 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":74,"btime":"2024-09-21T06:09:50:555528+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4024495468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":4024495468},{"type":"v1","addr":"172.21.15.50:6835","nonce":4024495468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3620352888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3620352888},{"type":"v1","addr":"172.21.15.88:6835","nonce":3620352888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4286454792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4286454792},{"type":"v1","addr":"172.21.15.88:6837","nonce":4286454792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:09:46.946836+0000","modified":"2024-09-21T06:09:49.976058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6833/1093475697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1093475697},{"type":"v1","addr":"172.21.15.50:6833","nonce":1093475697}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8304,"qdb_cluster":[8304]},"id":8}]} 2024-09-21T06:09:50.971 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 74 2024-09-21T06:09:50.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.973+0000 7f53c9a00640 1 -- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53a4074c70 msgr2=0x7f53a4077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:50.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.973+0000 7f53c9a00640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53a4074c70 0x7f53a4077180 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f53b4000f70 tx=0x7f53b40079c0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.973+0000 7f53c9a00640 1 -- 172.21.15.50:0/2887938526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c4157030 msgr2=0x7f53c413eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:50.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.973+0000 7f53c9a00640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c4157030 0x7f53c413eaa0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009bd0 tx=0x7f53bc005ae0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.973+0000 7f53c9a00640 1 -- 172.21.15.50:0/2887938526 shutdown_connections 2024-09-21T06:09:50.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.973+0000 7f53c9a00640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53a4074c70 0x7f53a4077180 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.973+0000 7f53c9a00640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c4157030 0x7f53c413eaa0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.973+0000 7f53c9a00640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c40b5a80 0x7f53c413e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.973+0000 7f53c9a00640 1 --2- 172.21.15.50:0/2887938526 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f53c4074950 0x7f53c4166eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:50.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.973+0000 7f53c9a00640 1 -- 172.21.15.50:0/2887938526 >> 172.21.15.50:0/2887938526 conn(0x7f53c40708c0 msgr2=0x7f53c4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:50.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.973+0000 7f53c9a00640 1 -- 172.21.15.50:0/2887938526 shutdown_connections 2024-09-21T06:09:50.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:50.973+0000 7f53c9a00640 1 -- 172.21.15.50:0/2887938526 wait complete. 2024-09-21T06:09:50.993 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:09:51.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.149+0000 7ff749c00640 1 -- 172.21.15.50:0/2866436357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7440ce600 msgr2=0x7ff744068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.149+0000 7ff749c00640 1 --2- 172.21.15.50:0/2866436357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7440ce600 0x7ff744068df0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff7440cb7e0 tx=0x7ff73001c860 comp rx=0 tx=0).stop 2024-09-21T06:09:51.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.149+0000 7ff749c00640 1 -- 172.21.15.50:0/2866436357 shutdown_connections 2024-09-21T06:09:51.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.149+0000 7ff749c00640 1 --2- 172.21.15.50:0/2866436357 >> v2:172.21.15.88:3300/0 conn(0x7ff744069330 0x7ff7440b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.149+0000 7ff749c00640 1 --2- 172.21.15.50:0/2866436357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7440ce600 0x7ff744068df0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.149+0000 7ff749c00640 1 --2- 172.21.15.50:0/2866436357 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff744075160 0x7ff74415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.149+0000 7ff749c00640 1 -- 172.21.15.50:0/2866436357 >> 172.21.15.50:0/2866436357 conn(0x7ff7440b41f0 msgr2=0x7ff7440b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:51.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.149+0000 7ff749c00640 1 -- 172.21.15.50:0/2866436357 shutdown_connections 2024-09-21T06:09:51.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.149+0000 7ff749c00640 1 -- 172.21.15.50:0/2866436357 wait complete. 2024-09-21T06:09:51.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff749c00640 1 Processor -- start 2024-09-21T06:09:51.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff749c00640 1 -- start start 2024-09-21T06:09:51.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff749c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744069330 0x7ff74409a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff749c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff744075160 0x7ff7440d0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff749c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7440ce600 0x7ff744160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff749c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff744162dd0 con 0x7ff7440ce600 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff749c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff744162c50 con 0x7ff744069330 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff749c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff744162f50 con 0x7ff744075160 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff743e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744069330 0x7ff74409a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff749200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7440ce600 0x7ff744160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff743400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff744075160 0x7ff7440d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff743e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744069330 0x7ff74409a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:53816/0 (socket says 172.21.15.50:53816) 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff743e00640 1 -- 172.21.15.50:0/129108863 learned_addr learned my addr 172.21.15.50:0/129108863 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff743e00640 1 -- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff744075160 msgr2=0x7ff7440d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff743e00640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff744075160 0x7ff7440d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff743e00640 1 -- 172.21.15.50:0/129108863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7440ce600 msgr2=0x7ff744160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff743e00640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7440ce600 0x7ff744160e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff743e00640 1 -- 172.21.15.50:0/129108863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74409aef0 con 0x7ff744069330 2024-09-21T06:09:51.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.153+0000 7ff749200640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7440ce600 0x7ff744160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:09:51.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.157+0000 7ff743e00640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744069330 0x7ff74409a9b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff73400e7f0 tx=0x7ff73400ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:51.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.157+0000 7ff73be00640 1 -- 172.21.15.50:0/129108863 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73400cac0 con 0x7ff744069330 2024-09-21T06:09:51.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.157+0000 7ff73be00640 1 -- 172.21.15.50:0/129108863 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff73400cc80 con 0x7ff744069330 2024-09-21T06:09:51.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.157+0000 7ff73be00640 1 -- 172.21.15.50:0/129108863 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734005280 con 0x7ff744069330 2024-09-21T06:09:51.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.157+0000 7ff749c00640 1 -- 172.21.15.50:0/129108863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7440d12f0 con 0x7ff744069330 2024-09-21T06:09:51.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.157+0000 7ff749c00640 1 -- 172.21.15.50:0/129108863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7440d18b0 con 0x7ff744069330 2024-09-21T06:09:51.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.157+0000 7ff749c00640 1 -- 172.21.15.50:0/129108863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff744163470 con 0x7ff744069330 2024-09-21T06:09:51.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.157+0000 7ff73be00640 1 -- 172.21.15.50:0/129108863 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7ff734005420 con 0x7ff744069330 2024-09-21T06:09:51.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.157+0000 7ff73be00640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff724074c70 0x7ff724077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.157+0000 7ff73be00640 1 -- 172.21.15.50:0/129108863 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7ff734093e10 con 0x7ff744069330 2024-09-21T06:09:51.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.161+0000 7ff743400640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff724074c70 0x7ff724077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.161+0000 7ff743400640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff724074c70 0x7ff724077180 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff7300af000 tx=0x7ff7300a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:51.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.161+0000 7ff73be00640 1 -- 172.21.15.50:0/129108863 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff734060910 con 0x7ff744069330 2024-09-21T06:09:51.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.341+0000 7ff749c00640 1 -- 172.21.15.50:0/129108863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff744069920 con 0x7ff744069330 2024-09-21T06:09:51.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.341+0000 7ff73be00640 1 -- 172.21.15.50:0/129108863 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16772 (secure 0 0 0) 0x7ff734060210 con 0x7ff744069330 2024-09-21T06:09:51.342 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:51.343 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":112,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:09:48.542957+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:44.704087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:45.912566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:09:51.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.345+0000 7ff749c00640 1 -- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff724074c70 msgr2=0x7ff724077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.345+0000 7ff749c00640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff724074c70 0x7ff724077180 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff7300af000 tx=0x7ff7300a7040 comp rx=0 tx=0).stop 2024-09-21T06:09:51.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.345+0000 7ff749c00640 1 -- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744069330 msgr2=0x7ff74409a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.345+0000 7ff749c00640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744069330 0x7ff74409a9b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff73400e7f0 tx=0x7ff73400ebf0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.345+0000 7ff749c00640 1 -- 172.21.15.50:0/129108863 shutdown_connections 2024-09-21T06:09:51.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.345+0000 7ff749c00640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff724074c70 0x7ff724077180 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.345+0000 7ff749c00640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7440ce600 0x7ff744160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.345+0000 7ff749c00640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff744075160 0x7ff7440d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.345+0000 7ff749c00640 1 --2- 172.21.15.50:0/129108863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744069330 0x7ff74409a9b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.345+0000 7ff749c00640 1 -- 172.21.15.50:0/129108863 >> 172.21.15.50:0/129108863 conn(0x7ff7440b41f0 msgr2=0x7ff74406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:51.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.345+0000 7ff749c00640 1 -- 172.21.15.50:0/129108863 shutdown_connections 2024-09-21T06:09:51.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.345+0000 7ff749c00640 1 -- 172.21.15.50:0/129108863 wait complete. 2024-09-21T06:09:51.366 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:09:51.366 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:09:51.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.525+0000 7f4c2a200640 1 -- 172.21.15.50:0/2065967060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c08004810 msgr2=0x7f4c08004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.525+0000 7f4c2a200640 1 --2- 172.21.15.50:0/2065967060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c08004810 0x7f4c08004c20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009b30 tx=0x7f4c0c01d230 comp rx=0 tx=0).stop 2024-09-21T06:09:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.525+0000 7f4c2a200640 1 -- 172.21.15.50:0/2065967060 shutdown_connections 2024-09-21T06:09:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.525+0000 7f4c2a200640 1 --2- 172.21.15.50:0/2065967060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c08004810 0x7f4c08004c20 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.525+0000 7f4c2a200640 1 -- 172.21.15.50:0/2065967060 >> 172.21.15.50:0/2065967060 conn(0x7f4c240b2040 msgr2=0x7f4c240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.525+0000 7f4c2a200640 1 -- 172.21.15.50:0/2065967060 shutdown_connections 2024-09-21T06:09:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.525+0000 7f4c2a200640 1 -- 172.21.15.50:0/2065967060 wait complete. 2024-09-21T06:09:51.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.525+0000 7f4c2a200640 1 Processor -- start 2024-09-21T06:09:51.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c2a200640 1 -- start start 2024-09-21T06:09:51.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c2a200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c08004810 0x7f4c24144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c2a200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c241434c0 0x7f4c240ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c2a200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c240ae8e0 0x7f4c240aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c2a200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4c240753c0 con 0x7f4c241434c0 2024-09-21T06:09:51.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c2a200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24075240 con 0x7f4c08004810 2024-09-21T06:09:51.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c2a200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4c240cf3b0 con 0x7f4c240ae8e0 2024-09-21T06:09:51.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c1aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c241434c0 0x7f4c240ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c1aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c241434c0 0x7f4c240ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37910/0 (socket says 172.21.15.50:37910) 2024-09-21T06:09:51.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c1aa00640 1 -- 172.21.15.50:0/1035073397 learned_addr learned my addr 172.21.15.50:0/1035073397 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:51.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c23400640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c08004810 0x7f4c24144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c23e00640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c240ae8e0 0x7f4c240aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c1aa00640 1 -- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c240ae8e0 msgr2=0x7f4c240aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c1aa00640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c240ae8e0 0x7f4c240aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c1aa00640 1 -- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c08004810 msgr2=0x7f4c24144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c1aa00640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c08004810 0x7f4c24144510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c1aa00640 1 -- 172.21.15.50:0/1035073397 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c240af230 con 0x7f4c241434c0 2024-09-21T06:09:51.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c1aa00640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c241434c0 0x7f4c240ce700 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c01d060 tx=0x7f4c0c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:51.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c1be00640 1 -- 172.21.15.50:0/1035073397 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c0bd070 con 0x7f4c241434c0 2024-09-21T06:09:51.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c1be00640 1 -- 172.21.15.50:0/1035073397 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c0c004030 con 0x7f4c241434c0 2024-09-21T06:09:51.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c2a200640 1 -- 172.21.15.50:0/1035073397 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c241f5870 con 0x7f4c241434c0 2024-09-21T06:09:51.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c1be00640 1 -- 172.21.15.50:0/1035073397 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c0b1040 con 0x7f4c241434c0 2024-09-21T06:09:51.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.529+0000 7f4c2a200640 1 -- 172.21.15.50:0/1035073397 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c241f9650 con 0x7f4c241434c0 2024-09-21T06:09:51.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.533+0000 7f4c2a200640 1 -- 172.21.15.50:0/1035073397 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c240cf880 con 0x7f4c241434c0 2024-09-21T06:09:51.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.533+0000 7f4c1be00640 1 -- 172.21.15.50:0/1035073397 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f4c0c0c0070 con 0x7f4c241434c0 2024-09-21T06:09:51.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.533+0000 7f4c1be00640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c00074ca0 0x7f4c000771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.533+0000 7f4c1be00640 1 -- 172.21.15.50:0/1035073397 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f4c0c12d480 con 0x7f4c241434c0 2024-09-21T06:09:51.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.533+0000 7f4c23400640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c00074ca0 0x7f4c000771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.537+0000 7f4c23400640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c00074ca0 0x7f4c000771b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4c140013e0 tx=0x7f4c14006ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:51.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.537+0000 7f4c1be00640 1 -- 172.21.15.50:0/1035073397 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4c240af230 con 0x7f4c241434c0 2024-09-21T06:09:51.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.713+0000 7f4c2a200640 1 -- 172.21.15.50:0/1035073397 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4c2415a6e0 con 0x7f4c00074ca0 2024-09-21T06:09:51.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.721+0000 7f4c1be00640 1 -- 172.21.15.50:0/1035073397 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251905 (secure 0 0 0) 0x7f4c2415a6e0 con 0x7f4c00074ca0 2024-09-21T06:09:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:51.724 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:09:51.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.721+0000 7f4c2a200640 1 -- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c00074ca0 msgr2=0x7f4c000771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.721+0000 7f4c2a200640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c00074ca0 0x7f4c000771b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4c140013e0 tx=0x7f4c14006ab0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.721+0000 7f4c2a200640 1 -- 172.21.15.50:0/1035073397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c241434c0 msgr2=0x7f4c240ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.721+0000 7f4c2a200640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c241434c0 0x7f4c240ce700 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c01d060 tx=0x7f4c0c005520 comp rx=0 tx=0).stop 2024-09-21T06:09:51.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.721+0000 7f4c2a200640 1 -- 172.21.15.50:0/1035073397 shutdown_connections 2024-09-21T06:09:51.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.721+0000 7f4c2a200640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c240ae8e0 0x7f4c240aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.721+0000 7f4c2a200640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c00074ca0 0x7f4c000771b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.721+0000 7f4c2a200640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c241434c0 0x7f4c240ce700 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.721+0000 7f4c2a200640 1 --2- 172.21.15.50:0/1035073397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c08004810 0x7f4c24144510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.721+0000 7f4c2a200640 1 -- 172.21.15.50:0/1035073397 >> 172.21.15.50:0/1035073397 conn(0x7f4c240b2040 msgr2=0x7f4c2415a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:51.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.725+0000 7f4c2a200640 1 -- 172.21.15.50:0/1035073397 shutdown_connections 2024-09-21T06:09:51.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.725+0000 7f4c2a200640 1 -- 172.21.15.50:0/1035073397 wait complete. 2024-09-21T06:09:51.745 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-09-21T06:09:51.183805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1025864,"kb_used_data":11904,"kb_used_omap":12,"kb_used_meta":1013747,"kb_avail":753948856,"statfs":{"total":773094113280,"available":772043628544,"internally_reserved":0,"allocated":12189696,"data_stored":5938438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1038077488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.673149"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.572843+0000","last_change":"2024-09-21T06:09:47.570525+0000","last_active":"2024-09-21T06:09:47.572843+0000","last_peered":"2024-09-21T06:09:47.572843+0000","last_clean":"2024-09-21T06:09:47.572843+0000","last_became_active":"2024-09-21T06:09:47.567795+0000","last_became_peered":"2024-09-21T06:09:47.567795+0000","last_unstale":"2024-09-21T06:09:47.572843+0000","last_undegraded":"2024-09-21T06:09:47.572843+0000","last_fullsized":"2024-09-21T06:09:47.572843+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:18.342305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552384+0000","last_change":"2024-09-21T06:09:47.552824+0000","last_active":"2024-09-21T06:09:48.552384+0000","last_peered":"2024-09-21T06:09:48.552384+0000","last_clean":"2024-09-21T06:09:48.552384+0000","last_became_active":"2024-09-21T06:09:47.552490+0000","last_became_peered":"2024-09-21T06:09:47.552490+0000","last_unstale":"2024-09-21T06:09:48.552384+0000","last_undegraded":"2024-09-21T06:09:48.552384+0000","last_fullsized":"2024-09-21T06:09:48.552384+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.562410+0000","last_change":"2024-09-21T06:09:47.560626+0000","last_active":"2024-09-21T06:09:47.562410+0000","last_peered":"2024-09-21T06:09:47.562410+0000","last_clean":"2024-09-21T06:09:47.562410+0000","last_became_active":"2024-09-21T06:09:47.560221+0000","last_became_peered":"2024-09-21T06:09:47.560221+0000","last_unstale":"2024-09-21T06:09:47.562410+0000","last_undegraded":"2024-09-21T06:09:47.562410+0000","last_fullsized":"2024-09-21T06:09:47.562410+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561543+0000","last_change":"2024-09-21T06:09:47.551941+0000","last_active":"2024-09-21T06:09:47.561543+0000","last_peered":"2024-09-21T06:09:47.561543+0000","last_clean":"2024-09-21T06:09:47.561543+0000","last_became_active":"2024-09-21T06:09:47.551615+0000","last_became_peered":"2024-09-21T06:09:47.551615+0000","last_unstale":"2024-09-21T06:09:47.561543+0000","last_undegraded":"2024-09-21T06:09:47.561543+0000","last_fullsized":"2024-09-21T06:09:47.561543+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:54.401707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.544241+0000","last_change":"2024-09-21T06:09:47.541551+0000","last_active":"2024-09-21T06:09:47.544241+0000","last_peered":"2024-09-21T06:09:47.544241+0000","last_clean":"2024-09-21T06:09:47.544241+0000","last_became_active":"2024-09-21T06:09:47.541207+0000","last_became_peered":"2024-09-21T06:09:47.541207+0000","last_unstale":"2024-09-21T06:09:47.544241+0000","last_undegraded":"2024-09-21T06:09:47.544241+0000","last_fullsized":"2024-09-21T06:09:47.544241+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:32.688598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.523983+0000","last_change":"2024-09-21T06:09:46.523983+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.523983+0000","last_undegraded":"2024-09-21T06:09:46.523983+0000","last_fullsized":"2024-09-21T06:09:46.523983+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552250+0000","last_change":"2024-09-21T06:09:47.552984+0000","last_active":"2024-09-21T06:09:48.552250+0000","last_peered":"2024-09-21T06:09:48.552250+0000","last_clean":"2024-09-21T06:09:48.552250+0000","last_became_active":"2024-09-21T06:09:47.550957+0000","last_became_peered":"2024-09-21T06:09:47.550957+0000","last_unstale":"2024-09-21T06:09:48.552250+0000","last_undegraded":"2024-09-21T06:09:48.552250+0000","last_fullsized":"2024-09-21T06:09:48.552250+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:29.803361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.562370+0000","last_change":"2024-09-21T06:09:47.559238+0000","last_active":"2024-09-21T06:09:47.562370+0000","last_peered":"2024-09-21T06:09:47.562370+0000","last_clean":"2024-09-21T06:09:47.562370+0000","last_became_active":"2024-09-21T06:09:47.558875+0000","last_became_peered":"2024-09-21T06:09:47.558875+0000","last_unstale":"2024-09-21T06:09:47.562370+0000","last_undegraded":"2024-09-21T06:09:47.562370+0000","last_fullsized":"2024-09-21T06:09:47.562370+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561508+0000","last_change":"2024-09-21T06:09:47.550887+0000","last_active":"2024-09-21T06:09:47.561508+0000","last_peered":"2024-09-21T06:09:47.561508+0000","last_clean":"2024-09-21T06:09:47.561508+0000","last_became_active":"2024-09-21T06:09:47.550442+0000","last_became_peered":"2024-09-21T06:09:47.550442+0000","last_unstale":"2024-09-21T06:09:47.561508+0000","last_undegraded":"2024-09-21T06:09:47.561508+0000","last_fullsized":"2024-09-21T06:09:47.561508+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:56.405036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568432+0000","last_change":"2024-09-21T06:09:47.560905+0000","last_active":"2024-09-21T06:09:47.568432+0000","last_peered":"2024-09-21T06:09:47.568432+0000","last_clean":"2024-09-21T06:09:47.568432+0000","last_became_active":"2024-09-21T06:09:47.558757+0000","last_became_peered":"2024-09-21T06:09:47.558757+0000","last_unstale":"2024-09-21T06:09:47.568432+0000","last_undegraded":"2024-09-21T06:09:47.568432+0000","last_fullsized":"2024-09-21T06:09:47.568432+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:19.852862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553229+0000","last_change":"2024-09-21T06:09:47.557467+0000","last_active":"2024-09-21T06:09:48.553229+0000","last_peered":"2024-09-21T06:09:48.553229+0000","last_clean":"2024-09-21T06:09:48.553229+0000","last_became_active":"2024-09-21T06:09:47.554901+0000","last_became_peered":"2024-09-21T06:09:47.554901+0000","last_unstale":"2024-09-21T06:09:48.553229+0000","last_undegraded":"2024-09-21T06:09:48.553229+0000","last_fullsized":"2024-09-21T06:09:48.553229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:31.714893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565149+0000","last_change":"2024-09-21T06:09:47.556163+0000","last_active":"2024-09-21T06:09:47.565149+0000","last_peered":"2024-09-21T06:09:47.565149+0000","last_clean":"2024-09-21T06:09:47.565149+0000","last_became_active":"2024-09-21T06:09:47.555656+0000","last_became_peered":"2024-09-21T06:09:47.555656+0000","last_unstale":"2024-09-21T06:09:47.565149+0000","last_undegraded":"2024-09-21T06:09:47.565149+0000","last_fullsized":"2024-09-21T06:09:47.565149+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:44.553253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561358+0000","last_change":"2024-09-21T06:09:47.554101+0000","last_active":"2024-09-21T06:09:47.561358+0000","last_peered":"2024-09-21T06:09:47.561358+0000","last_clean":"2024-09-21T06:09:47.561358+0000","last_became_active":"2024-09-21T06:09:47.553743+0000","last_became_peered":"2024-09-21T06:09:47.553743+0000","last_unstale":"2024-09-21T06:09:47.561358+0000","last_undegraded":"2024-09-21T06:09:47.561358+0000","last_fullsized":"2024-09-21T06:09:47.561358+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:09.595527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561935+0000","last_change":"2024-09-21T06:09:47.554855+0000","last_active":"2024-09-21T06:09:47.561935+0000","last_peered":"2024-09-21T06:09:47.561935+0000","last_clean":"2024-09-21T06:09:47.561935+0000","last_became_active":"2024-09-21T06:09:47.554540+0000","last_became_peered":"2024-09-21T06:09:47.554540+0000","last_unstale":"2024-09-21T06:09:47.561935+0000","last_undegraded":"2024-09-21T06:09:47.561935+0000","last_fullsized":"2024-09-21T06:09:47.561935+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.523903+0000","last_change":"2024-09-21T06:09:46.523903+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.523903+0000","last_undegraded":"2024-09-21T06:09:46.523903+0000","last_fullsized":"2024-09-21T06:09:46.523903+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561644+0000","last_change":"2024-09-21T06:09:47.554644+0000","last_active":"2024-09-21T06:09:47.561644+0000","last_peered":"2024-09-21T06:09:47.561644+0000","last_clean":"2024-09-21T06:09:47.561644+0000","last_became_active":"2024-09-21T06:09:47.554247+0000","last_became_peered":"2024-09-21T06:09:47.554247+0000","last_unstale":"2024-09-21T06:09:47.561644+0000","last_undegraded":"2024-09-21T06:09:47.561644+0000","last_fullsized":"2024-09-21T06:09:47.561644+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:13.781170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552610+0000","last_change":"2024-09-21T06:09:47.557683+0000","last_active":"2024-09-21T06:09:48.552610+0000","last_peered":"2024-09-21T06:09:48.552610+0000","last_clean":"2024-09-21T06:09:48.552610+0000","last_became_active":"2024-09-21T06:09:47.557357+0000","last_became_peered":"2024-09-21T06:09:47.557357+0000","last_unstale":"2024-09-21T06:09:48.552610+0000","last_undegraded":"2024-09-21T06:09:48.552610+0000","last_fullsized":"2024-09-21T06:09:48.552610+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561742+0000","last_change":"2024-09-21T06:09:47.557188+0000","last_active":"2024-09-21T06:09:48.561742+0000","last_peered":"2024-09-21T06:09:48.561742+0000","last_clean":"2024-09-21T06:09:48.561742+0000","last_became_active":"2024-09-21T06:09:47.556870+0000","last_became_peered":"2024-09-21T06:09:47.556870+0000","last_unstale":"2024-09-21T06:09:48.561742+0000","last_undegraded":"2024-09-21T06:09:48.561742+0000","last_fullsized":"2024-09-21T06:09:48.561742+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:38.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564093+0000","last_change":"2024-09-21T06:09:47.562338+0000","last_active":"2024-09-21T06:09:47.564093+0000","last_peered":"2024-09-21T06:09:47.564093+0000","last_clean":"2024-09-21T06:09:47.564093+0000","last_became_active":"2024-09-21T06:09:47.561835+0000","last_became_peered":"2024-09-21T06:09:47.561835+0000","last_unstale":"2024-09-21T06:09:47.564093+0000","last_undegraded":"2024-09-21T06:09:47.564093+0000","last_fullsized":"2024-09-21T06:09:47.564093+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552835+0000","last_change":"2024-09-21T06:09:47.548087+0000","last_active":"2024-09-21T06:09:48.552835+0000","last_peered":"2024-09-21T06:09:48.552835+0000","last_clean":"2024-09-21T06:09:48.552835+0000","last_became_active":"2024-09-21T06:09:47.546530+0000","last_became_peered":"2024-09-21T06:09:47.546530+0000","last_unstale":"2024-09-21T06:09:48.552835+0000","last_undegraded":"2024-09-21T06:09:48.552835+0000","last_fullsized":"2024-09-21T06:09:48.552835+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552595+0000","last_change":"2024-09-21T06:09:47.549010+0000","last_active":"2024-09-21T06:09:48.552595+0000","last_peered":"2024-09-21T06:09:48.552595+0000","last_clean":"2024-09-21T06:09:48.552595+0000","last_became_active":"2024-09-21T06:09:47.548666+0000","last_became_peered":"2024-09-21T06:09:47.548666+0000","last_unstale":"2024-09-21T06:09:48.552595+0000","last_undegraded":"2024-09-21T06:09:48.552595+0000","last_fullsized":"2024-09-21T06:09:48.552595+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565056+0000","last_change":"2024-09-21T06:09:47.554127+0000","last_active":"2024-09-21T06:09:47.565056+0000","last_peered":"2024-09-21T06:09:47.565056+0000","last_clean":"2024-09-21T06:09:47.565056+0000","last_became_active":"2024-09-21T06:09:47.553809+0000","last_became_peered":"2024-09-21T06:09:47.553809+0000","last_unstale":"2024-09-21T06:09:47.565056+0000","last_undegraded":"2024-09-21T06:09:47.565056+0000","last_fullsized":"2024-09-21T06:09:47.565056+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553426+0000","last_change":"2024-09-21T06:09:47.546392+0000","last_active":"2024-09-21T06:09:48.553426+0000","last_peered":"2024-09-21T06:09:48.553426+0000","last_clean":"2024-09-21T06:09:48.553426+0000","last_became_active":"2024-09-21T06:09:47.545962+0000","last_became_peered":"2024-09-21T06:09:47.545962+0000","last_unstale":"2024-09-21T06:09:48.553426+0000","last_undegraded":"2024-09-21T06:09:48.553426+0000","last_fullsized":"2024-09-21T06:09:48.553426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:35.324859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565233+0000","last_change":"2024-09-21T06:09:47.555478+0000","last_active":"2024-09-21T06:09:47.565233+0000","last_peered":"2024-09-21T06:09:47.565233+0000","last_clean":"2024-09-21T06:09:47.565233+0000","last_became_active":"2024-09-21T06:09:47.555175+0000","last_became_peered":"2024-09-21T06:09:47.555175+0000","last_unstale":"2024-09-21T06:09:47.565233+0000","last_undegraded":"2024-09-21T06:09:47.565233+0000","last_fullsized":"2024-09-21T06:09:47.565233+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552852+0000","last_change":"2024-09-21T06:09:47.557085+0000","last_active":"2024-09-21T06:09:48.552852+0000","last_peered":"2024-09-21T06:09:48.552852+0000","last_clean":"2024-09-21T06:09:48.552852+0000","last_became_active":"2024-09-21T06:09:47.556727+0000","last_became_peered":"2024-09-21T06:09:47.556727+0000","last_unstale":"2024-09-21T06:09:48.552852+0000","last_undegraded":"2024-09-21T06:09:48.552852+0000","last_fullsized":"2024-09-21T06:09:48.552852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568402+0000","last_change":"2024-09-21T06:09:47.559094+0000","last_active":"2024-09-21T06:09:47.568402+0000","last_peered":"2024-09-21T06:09:47.568402+0000","last_clean":"2024-09-21T06:09:47.568402+0000","last_became_active":"2024-09-21T06:09:47.558738+0000","last_became_peered":"2024-09-21T06:09:47.558738+0000","last_unstale":"2024-09-21T06:09:47.568402+0000","last_undegraded":"2024-09-21T06:09:47.568402+0000","last_fullsized":"2024-09-21T06:09:47.568402+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:39.640902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.527681+0000","last_change":"2024-09-21T06:09:46.527681+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.527681+0000","last_undegraded":"2024-09-21T06:09:46.527681+0000","last_fullsized":"2024-09-21T06:09:46.527681+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565098+0000","last_change":"2024-09-21T06:09:47.552743+0000","last_active":"2024-09-21T06:09:47.565098+0000","last_peered":"2024-09-21T06:09:47.565098+0000","last_clean":"2024-09-21T06:09:47.565098+0000","last_became_active":"2024-09-21T06:09:47.552402+0000","last_became_peered":"2024-09-21T06:09:47.552402+0000","last_unstale":"2024-09-21T06:09:47.565098+0000","last_undegraded":"2024-09-21T06:09:47.565098+0000","last_fullsized":"2024-09-21T06:09:47.565098+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552061+0000","last_change":"2024-09-21T06:09:47.547136+0000","last_active":"2024-09-21T06:09:48.552061+0000","last_peered":"2024-09-21T06:09:48.552061+0000","last_clean":"2024-09-21T06:09:48.552061+0000","last_became_active":"2024-09-21T06:09:47.546771+0000","last_became_peered":"2024-09-21T06:09:47.546771+0000","last_unstale":"2024-09-21T06:09:48.552061+0000","last_undegraded":"2024-09-21T06:09:48.552061+0000","last_fullsized":"2024-09-21T06:09:48.552061+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:09.595527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553225+0000","last_change":"2024-09-21T06:09:47.550506+0000","last_active":"2024-09-21T06:09:48.553225+0000","last_peered":"2024-09-21T06:09:48.553225+0000","last_clean":"2024-09-21T06:09:48.553225+0000","last_became_active":"2024-09-21T06:09:47.550172+0000","last_became_peered":"2024-09-21T06:09:47.550172+0000","last_unstale":"2024-09-21T06:09:48.553225+0000","last_undegraded":"2024-09-21T06:09:48.553225+0000","last_fullsized":"2024-09-21T06:09:48.553225+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.338192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564838+0000","last_change":"2024-09-21T06:09:47.548514+0000","last_active":"2024-09-21T06:09:47.564838+0000","last_peered":"2024-09-21T06:09:47.564838+0000","last_clean":"2024-09-21T06:09:47.564838+0000","last_became_active":"2024-09-21T06:09:47.548034+0000","last_became_peered":"2024-09-21T06:09:47.548034+0000","last_unstale":"2024-09-21T06:09:47.564838+0000","last_undegraded":"2024-09-21T06:09:47.564838+0000","last_fullsized":"2024-09-21T06:09:47.564838+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:47.209532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.526758+0000","last_change":"2024-09-21T06:09:46.526758+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.526758+0000","last_undegraded":"2024-09-21T06:09:46.526758+0000","last_fullsized":"2024-09-21T06:09:46.526758+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.526087+0000","last_change":"2024-09-21T06:09:46.526087+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.526087+0000","last_undegraded":"2024-09-21T06:09:46.526087+0000","last_fullsized":"2024-09-21T06:09:46.526087+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553791+0000","last_change":"2024-09-21T06:09:46.528315+0000","last_active":"2024-09-21T06:09:48.553791+0000","last_peered":"2024-09-21T06:09:48.553791+0000","last_clean":"2024-09-21T06:09:48.553791+0000","last_became_active":"2024-09-21T06:09:46.527949+0000","last_became_peered":"2024-09-21T06:09:46.527949+0000","last_unstale":"2024-09-21T06:09:48.553791+0000","last_undegraded":"2024-09-21T06:09:48.553791+0000","last_fullsized":"2024-09-21T06:09:48.553791+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:23.941639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561877+0000","last_change":"2024-09-21T06:09:47.553802+0000","last_active":"2024-09-21T06:09:48.561877+0000","last_peered":"2024-09-21T06:09:48.561877+0000","last_clean":"2024-09-21T06:09:48.561877+0000","last_became_active":"2024-09-21T06:09:47.553503+0000","last_became_peered":"2024-09-21T06:09:47.553503+0000","last_unstale":"2024-09-21T06:09:48.561877+0000","last_undegraded":"2024-09-21T06:09:48.561877+0000","last_fullsized":"2024-09-21T06:09:48.561877+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:58.813046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569291+0000","last_change":"2024-09-21T06:09:46.531370+0000","last_active":"2024-09-21T06:09:47.569291+0000","last_peered":"2024-09-21T06:09:47.569291+0000","last_clean":"2024-09-21T06:09:47.569291+0000","last_became_active":"2024-09-21T06:09:46.530960+0000","last_became_peered":"2024-09-21T06:09:46.530960+0000","last_unstale":"2024-09-21T06:09:47.569291+0000","last_undegraded":"2024-09-21T06:09:47.569291+0000","last_fullsized":"2024-09-21T06:09:47.569291+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.528678+0000","last_change":"2024-09-21T06:09:46.528678+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.528678+0000","last_undegraded":"2024-09-21T06:09:46.528678+0000","last_fullsized":"2024-09-21T06:09:46.528678+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.577193+0000","last_change":"2024-09-21T06:09:46.544553+0000","last_active":"2024-09-21T06:09:47.577193+0000","last_peered":"2024-09-21T06:09:47.577193+0000","last_clean":"2024-09-21T06:09:47.577193+0000","last_became_active":"2024-09-21T06:09:46.544246+0000","last_became_peered":"2024-09-21T06:09:46.544246+0000","last_unstale":"2024-09-21T06:09:47.577193+0000","last_undegraded":"2024-09-21T06:09:47.577193+0000","last_fullsized":"2024-09-21T06:09:47.577193+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.718740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.570926+0000","last_change":"2024-09-21T06:09:47.569840+0000","last_active":"2024-09-21T06:09:47.570926+0000","last_peered":"2024-09-21T06:09:47.570926+0000","last_clean":"2024-09-21T06:09:47.570926+0000","last_became_active":"2024-09-21T06:09:47.569541+0000","last_became_peered":"2024-09-21T06:09:47.569541+0000","last_unstale":"2024-09-21T06:09:47.570926+0000","last_undegraded":"2024-09-21T06:09:47.570926+0000","last_fullsized":"2024-09-21T06:09:47.570926+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.577763+0000","last_change":"2024-09-21T06:09:46.543348+0000","last_active":"2024-09-21T06:09:47.577763+0000","last_peered":"2024-09-21T06:09:47.577763+0000","last_clean":"2024-09-21T06:09:47.577763+0000","last_became_active":"2024-09-21T06:09:46.543036+0000","last_became_peered":"2024-09-21T06:09:46.543036+0000","last_unstale":"2024-09-21T06:09:47.577763+0000","last_undegraded":"2024-09-21T06:09:47.577763+0000","last_fullsized":"2024-09-21T06:09:47.577763+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:17.178909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568248+0000","last_change":"2024-09-21T06:09:47.558350+0000","last_active":"2024-09-21T06:09:47.568248+0000","last_peered":"2024-09-21T06:09:47.568248+0000","last_clean":"2024-09-21T06:09:47.568248+0000","last_became_active":"2024-09-21T06:09:47.552686+0000","last_became_peered":"2024-09-21T06:09:47.552686+0000","last_unstale":"2024-09-21T06:09:47.568248+0000","last_undegraded":"2024-09-21T06:09:47.568248+0000","last_fullsized":"2024-09-21T06:09:47.568248+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:05.811070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553484+0000","last_change":"2024-09-21T06:09:46.535845+0000","last_active":"2024-09-21T06:09:48.553484+0000","last_peered":"2024-09-21T06:09:48.553484+0000","last_clean":"2024-09-21T06:09:48.553484+0000","last_became_active":"2024-09-21T06:09:46.535462+0000","last_became_peered":"2024-09-21T06:09:46.535462+0000","last_unstale":"2024-09-21T06:09:48.553484+0000","last_undegraded":"2024-09-21T06:09:48.553484+0000","last_fullsized":"2024-09-21T06:09:48.553484+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:32.585856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553096+0000","last_change":"2024-09-21T06:09:47.556065+0000","last_active":"2024-09-21T06:09:48.553096+0000","last_peered":"2024-09-21T06:09:48.553096+0000","last_clean":"2024-09-21T06:09:48.553096+0000","last_became_active":"2024-09-21T06:09:47.555742+0000","last_became_peered":"2024-09-21T06:09:47.555742+0000","last_unstale":"2024-09-21T06:09:48.553096+0000","last_undegraded":"2024-09-21T06:09:48.553096+0000","last_fullsized":"2024-09-21T06:09:48.553096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.575260+0000","last_change":"2024-09-21T06:09:46.549487+0000","last_active":"2024-09-21T06:09:47.575260+0000","last_peered":"2024-09-21T06:09:47.575260+0000","last_clean":"2024-09-21T06:09:47.575260+0000","last_became_active":"2024-09-21T06:09:46.548860+0000","last_became_peered":"2024-09-21T06:09:46.548860+0000","last_unstale":"2024-09-21T06:09:47.575260+0000","last_undegraded":"2024-09-21T06:09:47.575260+0000","last_fullsized":"2024-09-21T06:09:47.575260+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:45.648740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569391+0000","last_change":"2024-09-21T06:09:47.563169+0000","last_active":"2024-09-21T06:09:47.569391+0000","last_peered":"2024-09-21T06:09:47.569391+0000","last_clean":"2024-09-21T06:09:47.569391+0000","last_became_active":"2024-09-21T06:09:47.562868+0000","last_became_peered":"2024-09-21T06:09:47.562868+0000","last_unstale":"2024-09-21T06:09:47.569391+0000","last_undegraded":"2024-09-21T06:09:47.569391+0000","last_fullsized":"2024-09-21T06:09:47.569391+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:51.138002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551911+0000","last_change":"2024-09-21T06:09:46.545661+0000","last_active":"2024-09-21T06:09:48.551911+0000","last_peered":"2024-09-21T06:09:48.551911+0000","last_clean":"2024-09-21T06:09:48.551911+0000","last_became_active":"2024-09-21T06:09:46.545340+0000","last_became_peered":"2024-09-21T06:09:46.545340+0000","last_unstale":"2024-09-21T06:09:48.551911+0000","last_undegraded":"2024-09-21T06:09:48.551911+0000","last_fullsized":"2024-09-21T06:09:48.551911+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:47.880211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552001+0000","last_change":"2024-09-21T06:09:47.553054+0000","last_active":"2024-09-21T06:09:48.552001+0000","last_peered":"2024-09-21T06:09:48.552001+0000","last_clean":"2024-09-21T06:09:48.552001+0000","last_became_active":"2024-09-21T06:09:47.552742+0000","last_became_peered":"2024-09-21T06:09:47.552742+0000","last_unstale":"2024-09-21T06:09:48.552001+0000","last_undegraded":"2024-09-21T06:09:48.552001+0000","last_fullsized":"2024-09-21T06:09:48.552001+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:54.333587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.542901+0000","last_change":"2024-09-21T06:09:46.536718+0000","last_active":"2024-09-21T06:09:47.542901+0000","last_peered":"2024-09-21T06:09:47.542901+0000","last_clean":"2024-09-21T06:09:47.542901+0000","last_became_active":"2024-09-21T06:09:46.535034+0000","last_became_peered":"2024-09-21T06:09:46.535034+0000","last_unstale":"2024-09-21T06:09:47.542901+0000","last_undegraded":"2024-09-21T06:09:47.542901+0000","last_fullsized":"2024-09-21T06:09:47.542901+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:49.688976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553944+0000","last_change":"2024-09-21T06:09:47.539900+0000","last_active":"2024-09-21T06:09:48.553944+0000","last_peered":"2024-09-21T06:09:48.553944+0000","last_clean":"2024-09-21T06:09:48.553944+0000","last_became_active":"2024-09-21T06:09:47.539412+0000","last_became_peered":"2024-09-21T06:09:47.539412+0000","last_unstale":"2024-09-21T06:09:48.553944+0000","last_undegraded":"2024-09-21T06:09:48.553944+0000","last_fullsized":"2024-09-21T06:09:48.553944+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:52.206476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.532722+0000","last_change":"2024-09-21T06:09:46.540269+0000","last_active":"2024-09-21T06:09:47.532722+0000","last_peered":"2024-09-21T06:09:47.532722+0000","last_clean":"2024-09-21T06:09:47.532722+0000","last_became_active":"2024-09-21T06:09:46.539954+0000","last_became_peered":"2024-09-21T06:09:46.539954+0000","last_unstale":"2024-09-21T06:09:47.532722+0000","last_undegraded":"2024-09-21T06:09:47.532722+0000","last_fullsized":"2024-09-21T06:09:47.532722+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:13.428634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.562059+0000","last_change":"2024-09-21T06:09:47.566984+0000","last_active":"2024-09-21T06:09:48.562059+0000","last_peered":"2024-09-21T06:09:48.562059+0000","last_clean":"2024-09-21T06:09:48.562059+0000","last_became_active":"2024-09-21T06:09:47.566193+0000","last_became_peered":"2024-09-21T06:09:47.566193+0000","last_unstale":"2024-09-21T06:09:48.562059+0000","last_undegraded":"2024-09-21T06:09:48.562059+0000","last_fullsized":"2024-09-21T06:09:48.562059+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:28.501310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.535666+0000","last_change":"2024-09-21T06:09:46.536342+0000","last_active":"2024-09-21T06:09:47.535666+0000","last_peered":"2024-09-21T06:09:47.535666+0000","last_clean":"2024-09-21T06:09:47.535666+0000","last_became_active":"2024-09-21T06:09:46.536012+0000","last_became_peered":"2024-09-21T06:09:46.536012+0000","last_unstale":"2024-09-21T06:09:47.535666+0000","last_undegraded":"2024-09-21T06:09:47.535666+0000","last_fullsized":"2024-09-21T06:09:47.535666+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:57.383442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.552610+0000","last_change":"2024-09-21T06:09:47.543982+0000","last_active":"2024-09-21T06:09:47.552610+0000","last_peered":"2024-09-21T06:09:47.552610+0000","last_clean":"2024-09-21T06:09:47.552610+0000","last_became_active":"2024-09-21T06:09:47.543662+0000","last_became_peered":"2024-09-21T06:09:47.543662+0000","last_unstale":"2024-09-21T06:09:47.552610+0000","last_undegraded":"2024-09-21T06:09:47.552610+0000","last_fullsized":"2024-09-21T06:09:47.552610+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.580229+0000","last_change":"2024-09-21T06:09:46.548929+0000","last_active":"2024-09-21T06:09:47.580229+0000","last_peered":"2024-09-21T06:09:47.580229+0000","last_clean":"2024-09-21T06:09:47.580229+0000","last_became_active":"2024-09-21T06:09:46.548521+0000","last_became_peered":"2024-09-21T06:09:46.548521+0000","last_unstale":"2024-09-21T06:09:47.580229+0000","last_undegraded":"2024-09-21T06:09:47.580229+0000","last_fullsized":"2024-09-21T06:09:47.580229+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:27.494160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553855+0000","last_change":"2024-09-21T06:09:47.554124+0000","last_active":"2024-09-21T06:09:48.553855+0000","last_peered":"2024-09-21T06:09:48.553855+0000","last_clean":"2024-09-21T06:09:48.553855+0000","last_became_active":"2024-09-21T06:09:47.551484+0000","last_became_peered":"2024-09-21T06:09:47.551484+0000","last_unstale":"2024-09-21T06:09:48.553855+0000","last_undegraded":"2024-09-21T06:09:48.553855+0000","last_fullsized":"2024-09-21T06:09:48.553855+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:47.621204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":19,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.582014+0000","last_change":"2024-09-21T06:09:46.539077+0000","last_active":"2024-09-21T06:09:47.582014+0000","last_peered":"2024-09-21T06:09:47.582014+0000","last_clean":"2024-09-21T06:09:47.582014+0000","last_became_active":"2024-09-21T06:09:46.537287+0000","last_became_peered":"2024-09-21T06:09:46.537287+0000","last_unstale":"2024-09-21T06:09:47.582014+0000","last_undegraded":"2024-09-21T06:09:47.582014+0000","last_fullsized":"2024-09-21T06:09:47.582014+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:52.872003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.554303+0000","last_change":"2024-09-21T06:09:47.546084+0000","last_active":"2024-09-21T06:09:47.554303+0000","last_peered":"2024-09-21T06:09:47.554303+0000","last_clean":"2024-09-21T06:09:47.554303+0000","last_became_active":"2024-09-21T06:09:47.545677+0000","last_became_peered":"2024-09-21T06:09:47.545677+0000","last_unstale":"2024-09-21T06:09:47.554303+0000","last_undegraded":"2024-09-21T06:09:47.554303+0000","last_fullsized":"2024-09-21T06:09:47.554303+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T06:09:46.543756+0000","last_change":"2024-09-21T06:09:46.537690+0000","last_active":"2024-09-21T06:09:46.543756+0000","last_peered":"2024-09-21T06:09:46.543756+0000","last_clean":"2024-09-21T06:09:46.543756+0000","last_became_active":"2024-09-21T06:09:46.537157+0000","last_became_peered":"2024-09-21T06:09:46.537157+0000","last_unstale":"2024-09-21T06:09:46.543756+0000","last_undegraded":"2024-09-21T06:09:46.543756+0000","last_fullsized":"2024-09-21T06:09:46.543756+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:02.269032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564968+0000","last_change":"2024-09-21T06:09:47.550402+0000","last_active":"2024-09-21T06:09:47.564968+0000","last_peered":"2024-09-21T06:09:47.564968+0000","last_clean":"2024-09-21T06:09:47.564968+0000","last_became_active":"2024-09-21T06:09:47.550085+0000","last_became_peered":"2024-09-21T06:09:47.550085+0000","last_unstale":"2024-09-21T06:09:47.564968+0000","last_undegraded":"2024-09-21T06:09:47.564968+0000","last_fullsized":"2024-09-21T06:09:47.564968+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561520+0000","last_change":"2024-09-21T06:09:46.524911+0000","last_active":"2024-09-21T06:09:48.561520+0000","last_peered":"2024-09-21T06:09:48.561520+0000","last_clean":"2024-09-21T06:09:48.561520+0000","last_became_active":"2024-09-21T06:09:46.524561+0000","last_became_peered":"2024-09-21T06:09:46.524561+0000","last_unstale":"2024-09-21T06:09:48.561520+0000","last_undegraded":"2024-09-21T06:09:48.561520+0000","last_fullsized":"2024-09-21T06:09:48.561520+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:32.557867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.547211+0000","last_change":"2024-09-21T06:09:47.541745+0000","last_active":"2024-09-21T06:09:47.547211+0000","last_peered":"2024-09-21T06:09:47.547211+0000","last_clean":"2024-09-21T06:09:47.547211+0000","last_became_active":"2024-09-21T06:09:47.541184+0000","last_became_peered":"2024-09-21T06:09:47.541184+0000","last_unstale":"2024-09-21T06:09:47.547211+0000","last_undegraded":"2024-09-21T06:09:47.547211+0000","last_fullsized":"2024-09-21T06:09:47.547211+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:49.440767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":21,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.579446+0000","last_change":"2024-09-21T06:09:46.533914+0000","last_active":"2024-09-21T06:09:47.579446+0000","last_peered":"2024-09-21T06:09:47.579446+0000","last_clean":"2024-09-21T06:09:47.579446+0000","last_became_active":"2024-09-21T06:09:46.533541+0000","last_became_peered":"2024-09-21T06:09:46.533541+0000","last_unstale":"2024-09-21T06:09:47.579446+0000","last_undegraded":"2024-09-21T06:09:47.579446+0000","last_fullsized":"2024-09-21T06:09:47.579446+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:24.340748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569557+0000","last_change":"2024-09-21T06:09:47.557155+0000","last_active":"2024-09-21T06:09:47.569557+0000","last_peered":"2024-09-21T06:09:47.569557+0000","last_clean":"2024-09-21T06:09:47.569557+0000","last_became_active":"2024-09-21T06:09:47.556834+0000","last_became_peered":"2024-09-21T06:09:47.556834+0000","last_unstale":"2024-09-21T06:09:47.569557+0000","last_undegraded":"2024-09-21T06:09:47.569557+0000","last_fullsized":"2024-09-21T06:09:47.569557+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:21.313342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.575298+0000","last_change":"2024-09-21T06:09:46.546891+0000","last_active":"2024-09-21T06:09:47.575298+0000","last_peered":"2024-09-21T06:09:47.575298+0000","last_clean":"2024-09-21T06:09:47.575298+0000","last_became_active":"2024-09-21T06:09:46.546573+0000","last_became_peered":"2024-09-21T06:09:46.546573+0000","last_unstale":"2024-09-21T06:09:47.575298+0000","last_undegraded":"2024-09-21T06:09:47.575298+0000","last_fullsized":"2024-09-21T06:09:47.575298+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:17.766821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.524609+0000","last_change":"2024-09-21T06:09:46.524609+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.524609+0000","last_undegraded":"2024-09-21T06:09:46.524609+0000","last_fullsized":"2024-09-21T06:09:46.524609+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.575222+0000","last_change":"2024-09-21T06:09:46.529780+0000","last_active":"2024-09-21T06:09:47.575222+0000","last_peered":"2024-09-21T06:09:47.575222+0000","last_clean":"2024-09-21T06:09:47.575222+0000","last_became_active":"2024-09-21T06:09:46.528136+0000","last_became_peered":"2024-09-21T06:09:46.528136+0000","last_unstale":"2024-09-21T06:09:47.575222+0000","last_undegraded":"2024-09-21T06:09:47.575222+0000","last_fullsized":"2024-09-21T06:09:47.575222+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:17.381392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568420+0000","last_change":"2024-09-21T06:09:47.567084+0000","last_active":"2024-09-21T06:09:47.568420+0000","last_peered":"2024-09-21T06:09:47.568420+0000","last_clean":"2024-09-21T06:09:47.568420+0000","last_became_active":"2024-09-21T06:09:47.566717+0000","last_became_peered":"2024-09-21T06:09:47.566717+0000","last_unstale":"2024-09-21T06:09:47.568420+0000","last_undegraded":"2024-09-21T06:09:47.568420+0000","last_fullsized":"2024-09-21T06:09:47.568420+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561699+0000","last_change":"2024-09-21T06:09:46.524059+0000","last_active":"2024-09-21T06:09:48.561699+0000","last_peered":"2024-09-21T06:09:48.561699+0000","last_clean":"2024-09-21T06:09:48.561699+0000","last_became_active":"2024-09-21T06:09:46.523459+0000","last_became_peered":"2024-09-21T06:09:46.523459+0000","last_unstale":"2024-09-21T06:09:48.561699+0000","last_undegraded":"2024-09-21T06:09:48.561699+0000","last_fullsized":"2024-09-21T06:09:48.561699+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.562759+0000","last_change":"2024-09-21T06:09:47.561252+0000","last_active":"2024-09-21T06:09:47.562759+0000","last_peered":"2024-09-21T06:09:47.562759+0000","last_clean":"2024-09-21T06:09:47.562759+0000","last_became_active":"2024-09-21T06:09:47.560849+0000","last_became_peered":"2024-09-21T06:09:47.560849+0000","last_unstale":"2024-09-21T06:09:47.562759+0000","last_undegraded":"2024-09-21T06:09:47.562759+0000","last_fullsized":"2024-09-21T06:09:47.562759+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553549+0000","last_change":"2024-09-21T06:09:46.534239+0000","last_active":"2024-09-21T06:09:48.553549+0000","last_peered":"2024-09-21T06:09:48.553549+0000","last_clean":"2024-09-21T06:09:48.553549+0000","last_became_active":"2024-09-21T06:09:46.533715+0000","last_became_peered":"2024-09-21T06:09:46.533715+0000","last_unstale":"2024-09-21T06:09:48.553549+0000","last_undegraded":"2024-09-21T06:09:48.553549+0000","last_fullsized":"2024-09-21T06:09:48.553549+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:08:04.897143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.559568+0000","last_change":"2024-09-21T06:09:47.552337+0000","last_active":"2024-09-21T06:09:47.559568+0000","last_peered":"2024-09-21T06:09:47.559568+0000","last_clean":"2024-09-21T06:09:47.559568+0000","last_became_active":"2024-09-21T06:09:47.552009+0000","last_became_peered":"2024-09-21T06:09:47.552009+0000","last_unstale":"2024-09-21T06:09:47.559568+0000","last_undegraded":"2024-09-21T06:09:47.559568+0000","last_fullsized":"2024-09-21T06:09:47.559568+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:46.655889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T06:09:46.539682+0000","last_change":"2024-09-21T06:09:46.536200+0000","last_active":"2024-09-21T06:09:46.539682+0000","last_peered":"2024-09-21T06:09:46.539682+0000","last_clean":"2024-09-21T06:09:46.539682+0000","last_became_active":"2024-09-21T06:09:46.535894+0000","last_became_peered":"2024-09-21T06:09:46.535894+0000","last_unstale":"2024-09-21T06:09:46.539682+0000","last_undegraded":"2024-09-21T06:09:46.539682+0000","last_fullsized":"2024-09-21T06:09:46.539682+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:46.409661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561750+0000","last_change":"2024-09-21T06:09:47.552189+0000","last_active":"2024-09-21T06:09:47.561750+0000","last_peered":"2024-09-21T06:09:47.561750+0000","last_clean":"2024-09-21T06:09:47.561750+0000","last_became_active":"2024-09-21T06:09:47.548927+0000","last_became_peered":"2024-09-21T06:09:47.548927+0000","last_unstale":"2024-09-21T06:09:47.561750+0000","last_undegraded":"2024-09-21T06:09:47.561750+0000","last_fullsized":"2024-09-21T06:09:47.561750+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:33.335946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.532137+0000","last_change":"2024-09-21T06:09:46.526945+0000","last_active":"2024-09-21T06:09:47.532137+0000","last_peered":"2024-09-21T06:09:47.532137+0000","last_clean":"2024-09-21T06:09:47.532137+0000","last_became_active":"2024-09-21T06:09:46.526588+0000","last_became_peered":"2024-09-21T06:09:46.526588+0000","last_unstale":"2024-09-21T06:09:47.532137+0000","last_undegraded":"2024-09-21T06:09:47.532137+0000","last_fullsized":"2024-09-21T06:09:47.532137+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:14.014304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561635+0000","last_change":"2024-09-21T06:09:47.555659+0000","last_active":"2024-09-21T06:09:47.561635+0000","last_peered":"2024-09-21T06:09:47.561635+0000","last_clean":"2024-09-21T06:09:47.561635+0000","last_became_active":"2024-09-21T06:09:47.555358+0000","last_became_peered":"2024-09-21T06:09:47.555358+0000","last_unstale":"2024-09-21T06:09:47.561635+0000","last_undegraded":"2024-09-21T06:09:47.561635+0000","last_fullsized":"2024-09-21T06:09:47.561635+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T06:09:46.544242+0000","last_change":"2024-09-21T06:09:46.541688+0000","last_active":"2024-09-21T06:09:46.544242+0000","last_peered":"2024-09-21T06:09:46.544242+0000","last_clean":"2024-09-21T06:09:46.544242+0000","last_became_active":"2024-09-21T06:09:46.541322+0000","last_became_peered":"2024-09-21T06:09:46.541322+0000","last_unstale":"2024-09-21T06:09:46.544242+0000","last_undegraded":"2024-09-21T06:09:46.544242+0000","last_fullsized":"2024-09-21T06:09:46.544242+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:39.745209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565078+0000","last_change":"2024-09-21T06:09:47.551665+0000","last_active":"2024-09-21T06:09:47.565078+0000","last_peered":"2024-09-21T06:09:47.565078+0000","last_clean":"2024-09-21T06:09:47.565078+0000","last_became_active":"2024-09-21T06:09:47.551313+0000","last_became_peered":"2024-09-21T06:09:47.551313+0000","last_unstale":"2024-09-21T06:09:47.565078+0000","last_undegraded":"2024-09-21T06:09:47.565078+0000","last_fullsized":"2024-09-21T06:09:47.565078+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.543137+0000","last_change":"2024-09-21T06:09:46.545814+0000","last_active":"2024-09-21T06:09:47.543137+0000","last_peered":"2024-09-21T06:09:47.543137+0000","last_clean":"2024-09-21T06:09:47.543137+0000","last_became_active":"2024-09-21T06:09:46.545413+0000","last_became_peered":"2024-09-21T06:09:46.545413+0000","last_unstale":"2024-09-21T06:09:47.543137+0000","last_undegraded":"2024-09-21T06:09:47.543137+0000","last_fullsized":"2024-09-21T06:09:47.543137+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:13.029274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561834+0000","last_change":"2024-09-21T06:09:47.566967+0000","last_active":"2024-09-21T06:09:48.561834+0000","last_peered":"2024-09-21T06:09:48.561834+0000","last_clean":"2024-09-21T06:09:48.561834+0000","last_became_active":"2024-09-21T06:09:47.566211+0000","last_became_peered":"2024-09-21T06:09:47.566211+0000","last_unstale":"2024-09-21T06:09:48.561834+0000","last_undegraded":"2024-09-21T06:09:48.561834+0000","last_fullsized":"2024-09-21T06:09:48.561834+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:14.036424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.580510+0000","last_change":"2024-09-21T06:09:46.528896+0000","last_active":"2024-09-21T06:09:47.580510+0000","last_peered":"2024-09-21T06:09:47.580510+0000","last_clean":"2024-09-21T06:09:47.580510+0000","last_became_active":"2024-09-21T06:09:46.528590+0000","last_became_peered":"2024-09-21T06:09:46.528590+0000","last_unstale":"2024-09-21T06:09:47.580510+0000","last_undegraded":"2024-09-21T06:09:47.580510+0000","last_fullsized":"2024-09-21T06:09:47.580510+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564780+0000","last_change":"2024-09-21T06:09:47.556097+0000","last_active":"2024-09-21T06:09:47.564780+0000","last_peered":"2024-09-21T06:09:47.564780+0000","last_clean":"2024-09-21T06:09:47.564780+0000","last_became_active":"2024-09-21T06:09:47.555720+0000","last_became_peered":"2024-09-21T06:09:47.555720+0000","last_unstale":"2024-09-21T06:09:47.564780+0000","last_undegraded":"2024-09-21T06:09:47.564780+0000","last_fullsized":"2024-09-21T06:09:47.564780+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:14.289349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.563317+0000","last_change":"2024-09-21T06:09:46.525404+0000","last_active":"2024-09-21T06:09:48.563317+0000","last_peered":"2024-09-21T06:09:48.563317+0000","last_clean":"2024-09-21T06:09:48.563317+0000","last_became_active":"2024-09-21T06:09:46.525087+0000","last_became_peered":"2024-09-21T06:09:46.525087+0000","last_unstale":"2024-09-21T06:09:48.563317+0000","last_undegraded":"2024-09-21T06:09:48.563317+0000","last_fullsized":"2024-09-21T06:09:48.563317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:57.383442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.572681+0000","last_change":"2024-09-21T06:09:47.565828+0000","last_active":"2024-09-21T06:09:47.572681+0000","last_peered":"2024-09-21T06:09:47.572681+0000","last_clean":"2024-09-21T06:09:47.572681+0000","last_became_active":"2024-09-21T06:09:47.565467+0000","last_became_peered":"2024-09-21T06:09:47.565467+0000","last_unstale":"2024-09-21T06:09:47.572681+0000","last_undegraded":"2024-09-21T06:09:47.572681+0000","last_fullsized":"2024-09-21T06:09:47.572681+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:37.650305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.544753+0000","last_change":"2024-09-21T06:09:46.542691+0000","last_active":"2024-09-21T06:09:47.544753+0000","last_peered":"2024-09-21T06:09:47.544753+0000","last_clean":"2024-09-21T06:09:47.544753+0000","last_became_active":"2024-09-21T06:09:46.542293+0000","last_became_peered":"2024-09-21T06:09:46.542293+0000","last_unstale":"2024-09-21T06:09:47.544753+0000","last_undegraded":"2024-09-21T06:09:47.544753+0000","last_fullsized":"2024-09-21T06:09:47.544753+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:33.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561749+0000","last_change":"2024-09-21T06:09:47.576750+0000","last_active":"2024-09-21T06:09:48.561749+0000","last_peered":"2024-09-21T06:09:48.561749+0000","last_clean":"2024-09-21T06:09:48.561749+0000","last_became_active":"2024-09-21T06:09:47.576443+0000","last_became_peered":"2024-09-21T06:09:47.576443+0000","last_unstale":"2024-09-21T06:09:48.561749+0000","last_undegraded":"2024-09-21T06:09:48.561749+0000","last_fullsized":"2024-09-21T06:09:48.561749+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.542616+0000","last_change":"2024-09-21T06:09:46.547408+0000","last_active":"2024-09-21T06:09:47.542616+0000","last_peered":"2024-09-21T06:09:47.542616+0000","last_clean":"2024-09-21T06:09:47.542616+0000","last_became_active":"2024-09-21T06:09:46.547043+0000","last_became_peered":"2024-09-21T06:09:46.547043+0000","last_unstale":"2024-09-21T06:09:47.542616+0000","last_undegraded":"2024-09-21T06:09:47.542616+0000","last_fullsized":"2024-09-21T06:09:47.542616+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:08.588377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552144+0000","last_change":"2024-09-21T06:09:47.540415+0000","last_active":"2024-09-21T06:09:48.552144+0000","last_peered":"2024-09-21T06:09:48.552144+0000","last_clean":"2024-09-21T06:09:48.552144+0000","last_became_active":"2024-09-21T06:09:47.539926+0000","last_became_peered":"2024-09-21T06:09:47.539926+0000","last_unstale":"2024-09-21T06:09:48.552144+0000","last_undegraded":"2024-09-21T06:09:48.552144+0000","last_fullsized":"2024-09-21T06:09:48.552144+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553533+0000","last_change":"2024-09-21T06:09:46.529617+0000","last_active":"2024-09-21T06:09:48.553533+0000","last_peered":"2024-09-21T06:09:48.553533+0000","last_clean":"2024-09-21T06:09:48.553533+0000","last_became_active":"2024-09-21T06:09:46.528234+0000","last_became_peered":"2024-09-21T06:09:46.528234+0000","last_unstale":"2024-09-21T06:09:48.553533+0000","last_undegraded":"2024-09-21T06:09:48.553533+0000","last_fullsized":"2024-09-21T06:09:48.553533+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:19.814656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552277+0000","last_change":"2024-09-21T06:09:47.543597+0000","last_active":"2024-09-21T06:09:48.552277+0000","last_peered":"2024-09-21T06:09:48.552277+0000","last_clean":"2024-09-21T06:09:48.552277+0000","last_became_active":"2024-09-21T06:09:47.543282+0000","last_became_peered":"2024-09-21T06:09:47.543282+0000","last_unstale":"2024-09-21T06:09:48.552277+0000","last_undegraded":"2024-09-21T06:09:48.552277+0000","last_fullsized":"2024-09-21T06:09:48.552277+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552294+0000","last_change":"2024-09-21T06:09:46.535746+0000","last_active":"2024-09-21T06:09:48.552294+0000","last_peered":"2024-09-21T06:09:48.552294+0000","last_clean":"2024-09-21T06:09:48.552294+0000","last_became_active":"2024-09-21T06:09:46.535142+0000","last_became_peered":"2024-09-21T06:09:46.535142+0000","last_unstale":"2024-09-21T06:09:48.552294+0000","last_undegraded":"2024-09-21T06:09:48.552294+0000","last_fullsized":"2024-09-21T06:09:48.552294+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:56.819951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561609+0000","last_change":"2024-09-21T06:09:47.553873+0000","last_active":"2024-09-21T06:09:47.561609+0000","last_peered":"2024-09-21T06:09:47.561609+0000","last_clean":"2024-09-21T06:09:47.561609+0000","last_became_active":"2024-09-21T06:09:47.553515+0000","last_became_peered":"2024-09-21T06:09:47.553515+0000","last_unstale":"2024-09-21T06:09:47.561609+0000","last_undegraded":"2024-09-21T06:09:47.561609+0000","last_fullsized":"2024-09-21T06:09:47.561609+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:49.440767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.532370+0000","last_change":"2024-09-21T06:09:46.539038+0000","last_active":"2024-09-21T06:09:47.532370+0000","last_peered":"2024-09-21T06:09:47.532370+0000","last_clean":"2024-09-21T06:09:47.532370+0000","last_became_active":"2024-09-21T06:09:46.538692+0000","last_became_peered":"2024-09-21T06:09:46.538692+0000","last_unstale":"2024-09-21T06:09:47.532370+0000","last_undegraded":"2024-09-21T06:09:47.532370+0000","last_fullsized":"2024-09-21T06:09:47.532370+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:44.565196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569536+0000","last_change":"2024-09-21T06:09:47.564522+0000","last_active":"2024-09-21T06:09:47.569536+0000","last_peered":"2024-09-21T06:09:47.569536+0000","last_clean":"2024-09-21T06:09:47.569536+0000","last_became_active":"2024-09-21T06:09:47.564206+0000","last_became_peered":"2024-09-21T06:09:47.564206+0000","last_unstale":"2024-09-21T06:09:47.569536+0000","last_undegraded":"2024-09-21T06:09:47.569536+0000","last_fullsized":"2024-09-21T06:09:47.569536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:03.505698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.543377+0000","last_change":"2024-09-21T06:09:46.532492+0000","last_active":"2024-09-21T06:09:47.543377+0000","last_peered":"2024-09-21T06:09:47.543377+0000","last_clean":"2024-09-21T06:09:47.543377+0000","last_became_active":"2024-09-21T06:09:46.530409+0000","last_became_peered":"2024-09-21T06:09:46.530409+0000","last_unstale":"2024-09-21T06:09:47.543377+0000","last_undegraded":"2024-09-21T06:09:47.543377+0000","last_fullsized":"2024-09-21T06:09:47.543377+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:57.805896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.549238+0000","last_change":"2024-09-21T06:09:47.543465+0000","last_active":"2024-09-21T06:09:47.549238+0000","last_peered":"2024-09-21T06:09:47.549238+0000","last_clean":"2024-09-21T06:09:47.549238+0000","last_became_active":"2024-09-21T06:09:47.543121+0000","last_became_peered":"2024-09-21T06:09:47.543121+0000","last_unstale":"2024-09-21T06:09:47.549238+0000","last_undegraded":"2024-09-21T06:09:47.549238+0000","last_fullsized":"2024-09-21T06:09:47.549238+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552395+0000","last_change":"2024-09-21T06:09:46.534147+0000","last_active":"2024-09-21T06:09:48.552395+0000","last_peered":"2024-09-21T06:09:48.552395+0000","last_clean":"2024-09-21T06:09:48.552395+0000","last_became_active":"2024-09-21T06:09:46.533758+0000","last_became_peered":"2024-09-21T06:09:46.533758+0000","last_unstale":"2024-09-21T06:09:48.552395+0000","last_undegraded":"2024-09-21T06:09:48.552395+0000","last_fullsized":"2024-09-21T06:09:48.552395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:29.394474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":277,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.543464+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:09:47.543464+0000","last_peered":"2024-09-21T06:09:47.543464+0000","last_clean":"2024-09-21T06:09:47.543464+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:09:47.543464+0000","last_undegraded":"2024-09-21T06:09:47.543464+0000","last_fullsized":"2024-09-21T06:09:47.543464+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509605,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127456,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244384,"statfs":{"total":96636764160,"available":96506249216,"internally_reserved":0,"allocated":1974272,"data_stored":1186242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509605,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118560,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253280,"statfs":{"total":96636764160,"available":96515358720,"internally_reserved":0,"allocated":1388544,"data_stored":601712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509604,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82456,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289384,"statfs":{"total":96636764160,"available":96552329216,"internally_reserved":0,"allocated":1380352,"data_stored":595862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509605,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144920,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226920,"statfs":{"total":96636764160,"available":96488366080,"internally_reserved":0,"allocated":1372160,"data_stored":595258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509604,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135208,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236632,"statfs":{"total":96636764160,"available":96498311168,"internally_reserved":0,"allocated":1388544,"data_stored":596940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509606,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140760,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231080,"statfs":{"total":96636764160,"available":96492625920,"internally_reserved":0,"allocated":1372160,"data_stored":595258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509604,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131596,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240244,"statfs":{"total":96636764160,"available":96502009856,"internally_reserved":0,"allocated":1953792,"data_stored":1178767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542309,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144908,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226932,"statfs":{"total":96636764160,"available":96488378368,"internally_reserved":0,"allocated":1359872,"data_stored":588399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:09:51.751 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:09:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.945+0000 7f484d000640 1 -- 172.21.15.50:0/3414854144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f482c0051d0 msgr2=0x7f482c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.945+0000 7f484d000640 1 --2- 172.21.15.50:0/3414854144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f482c0051d0 0x7f482c00b9d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4840009b10 tx=0x7f484001d270 comp rx=0 tx=0).stop 2024-09-21T06:09:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.945+0000 7f484d000640 1 -- 172.21.15.50:0/3414854144 shutdown_connections 2024-09-21T06:09:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.945+0000 7f484d000640 1 --2- 172.21.15.50:0/3414854144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f482c0051d0 0x7f482c00b9d0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.945+0000 7f484d000640 1 -- 172.21.15.50:0/3414854144 >> 172.21.15.50:0/3414854144 conn(0x7f48480b2040 msgr2=0x7f48480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.945+0000 7f484d000640 1 -- 172.21.15.50:0/3414854144 shutdown_connections 2024-09-21T06:09:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.945+0000 7f484d000640 1 -- 172.21.15.50:0/3414854144 wait complete. 2024-09-21T06:09:51.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f484d000640 1 Processor -- start 2024-09-21T06:09:51.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f484d000640 1 -- start start 2024-09-21T06:09:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f484d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f482c0051d0 0x7f48481f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f484d000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4848161d20 0x7f4848164c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f484d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48481467b0 0x7f4848146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f484d000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4848075190 con 0x7f482c0051d0 2024-09-21T06:09:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f484d000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4848075010 con 0x7f48481467b0 2024-09-21T06:09:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f484d000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4848075310 con 0x7f4848161d20 2024-09-21T06:09:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f4845e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f482c0051d0 0x7f48481f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f4845e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f482c0051d0 0x7f48481f3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37922/0 (socket says 172.21.15.50:37922) 2024-09-21T06:09:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f4845e00640 1 -- 172.21.15.50:0/841995185 learned_addr learned my addr 172.21.15.50:0/841995185 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f4846800640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48481467b0 0x7f4848146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f483d400640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4848161d20 0x7f4848164c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f4845e00640 1 -- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4848161d20 msgr2=0x7f4848164c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f4845e00640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4848161d20 0x7f4848164c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f4845e00640 1 -- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48481467b0 msgr2=0x7f4848146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f4845e00640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48481467b0 0x7f4848146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f4845e00640 1 -- 172.21.15.50:0/841995185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48481f42d0 con 0x7f482c0051d0 2024-09-21T06:09:51.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f4846800640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48481467b0 0x7f4848146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:51.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f4845e00640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f482c0051d0 0x7f48481f3330 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4838001490 tx=0x7f483800e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f483ea00640 1 -- 172.21.15.50:0/841995185 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838012850 con 0x7f482c0051d0 2024-09-21T06:09:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f483ea00640 1 -- 172.21.15.50:0/841995185 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4838012b00 con 0x7f482c0051d0 2024-09-21T06:09:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f484d000640 1 -- 172.21.15.50:0/841995185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48481f3d10 con 0x7f482c0051d0 2024-09-21T06:09:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f483ea00640 1 -- 172.21.15.50:0/841995185 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483801a400 con 0x7f482c0051d0 2024-09-21T06:09:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.949+0000 7f484d000640 1 -- 172.21.15.50:0/841995185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4848161520 con 0x7f482c0051d0 2024-09-21T06:09:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.953+0000 7f484d000640 1 -- 172.21.15.50:0/841995185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48480cf650 con 0x7f482c0051d0 2024-09-21T06:09:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.953+0000 7f483ea00640 1 -- 172.21.15.50:0/841995185 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f483800d030 con 0x7f482c0051d0 2024-09-21T06:09:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.953+0000 7f483ea00640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4824074ca0 0x7f48240771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.953+0000 7f483ea00640 1 -- 172.21.15.50:0/841995185 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f4838096130 con 0x7f482c0051d0 2024-09-21T06:09:51.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.953+0000 7f483d400640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4824074ca0 0x7f48240771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.957+0000 7f483d400640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4824074ca0 0x7f48240771b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f48480cc500 tx=0x7f4830003a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:51.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:51.957+0000 7f483ea00640 1 -- 172.21.15.50:0/841995185 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4838016030 con 0x7f482c0051d0 2024-09-21T06:09:52.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.137+0000 7f484d000640 1 -- 172.21.15.50:0/841995185 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f484806b540 con 0x7f4824074ca0 2024-09-21T06:09:52.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.141+0000 7f483ea00640 1 -- 172.21.15.50:0/841995185 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251905 (secure 0 0 0) 0x7f4830006610 con 0x7f4824074ca0 2024-09-21T06:09:52.145 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:52.146 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:09:52.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.145+0000 7f484d000640 1 -- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4824074ca0 msgr2=0x7f48240771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.145+0000 7f484d000640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4824074ca0 0x7f48240771b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f48480cc500 tx=0x7f4830003a30 comp rx=0 tx=0).stop 2024-09-21T06:09:52.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.145+0000 7f484d000640 1 -- 172.21.15.50:0/841995185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f482c0051d0 msgr2=0x7f48481f3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.145+0000 7f484d000640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f482c0051d0 0x7f48481f3330 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4838001490 tx=0x7f483800e6b0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.145+0000 7f484d000640 1 -- 172.21.15.50:0/841995185 shutdown_connections 2024-09-21T06:09:52.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.145+0000 7f484d000640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48481467b0 0x7f4848146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.145+0000 7f484d000640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4848161d20 0x7f4848164c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.145+0000 7f484d000640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4824074ca0 0x7f48240771b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.145+0000 7f484d000640 1 --2- 172.21.15.50:0/841995185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f482c0051d0 0x7f48481f3330 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.145+0000 7f484d000640 1 -- 172.21.15.50:0/841995185 >> 172.21.15.50:0/841995185 conn(0x7f48480b2040 msgr2=0x7f484815b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:52.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.145+0000 7f484d000640 1 -- 172.21.15.50:0/841995185 shutdown_connections 2024-09-21T06:09:52.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.145+0000 7f484d000640 1 -- 172.21.15.50:0/841995185 wait complete. 2024-09-21T06:09:52.167 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-09-21T06:09:51.183805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1025864,"kb_used_data":11904,"kb_used_omap":12,"kb_used_meta":1013747,"kb_avail":753948856,"statfs":{"total":773094113280,"available":772043628544,"internally_reserved":0,"allocated":12189696,"data_stored":5938438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1038077488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.673149"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.572843+0000","last_change":"2024-09-21T06:09:47.570525+0000","last_active":"2024-09-21T06:09:47.572843+0000","last_peered":"2024-09-21T06:09:47.572843+0000","last_clean":"2024-09-21T06:09:47.572843+0000","last_became_active":"2024-09-21T06:09:47.567795+0000","last_became_peered":"2024-09-21T06:09:47.567795+0000","last_unstale":"2024-09-21T06:09:47.572843+0000","last_undegraded":"2024-09-21T06:09:47.572843+0000","last_fullsized":"2024-09-21T06:09:47.572843+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:18.342305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552384+0000","last_change":"2024-09-21T06:09:47.552824+0000","last_active":"2024-09-21T06:09:48.552384+0000","last_peered":"2024-09-21T06:09:48.552384+0000","last_clean":"2024-09-21T06:09:48.552384+0000","last_became_active":"2024-09-21T06:09:47.552490+0000","last_became_peered":"2024-09-21T06:09:47.552490+0000","last_unstale":"2024-09-21T06:09:48.552384+0000","last_undegraded":"2024-09-21T06:09:48.552384+0000","last_fullsized":"2024-09-21T06:09:48.552384+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.562410+0000","last_change":"2024-09-21T06:09:47.560626+0000","last_active":"2024-09-21T06:09:47.562410+0000","last_peered":"2024-09-21T06:09:47.562410+0000","last_clean":"2024-09-21T06:09:47.562410+0000","last_became_active":"2024-09-21T06:09:47.560221+0000","last_became_peered":"2024-09-21T06:09:47.560221+0000","last_unstale":"2024-09-21T06:09:47.562410+0000","last_undegraded":"2024-09-21T06:09:47.562410+0000","last_fullsized":"2024-09-21T06:09:47.562410+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561543+0000","last_change":"2024-09-21T06:09:47.551941+0000","last_active":"2024-09-21T06:09:47.561543+0000","last_peered":"2024-09-21T06:09:47.561543+0000","last_clean":"2024-09-21T06:09:47.561543+0000","last_became_active":"2024-09-21T06:09:47.551615+0000","last_became_peered":"2024-09-21T06:09:47.551615+0000","last_unstale":"2024-09-21T06:09:47.561543+0000","last_undegraded":"2024-09-21T06:09:47.561543+0000","last_fullsized":"2024-09-21T06:09:47.561543+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:54.401707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.544241+0000","last_change":"2024-09-21T06:09:47.541551+0000","last_active":"2024-09-21T06:09:47.544241+0000","last_peered":"2024-09-21T06:09:47.544241+0000","last_clean":"2024-09-21T06:09:47.544241+0000","last_became_active":"2024-09-21T06:09:47.541207+0000","last_became_peered":"2024-09-21T06:09:47.541207+0000","last_unstale":"2024-09-21T06:09:47.544241+0000","last_undegraded":"2024-09-21T06:09:47.544241+0000","last_fullsized":"2024-09-21T06:09:47.544241+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:32.688598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.523983+0000","last_change":"2024-09-21T06:09:46.523983+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.523983+0000","last_undegraded":"2024-09-21T06:09:46.523983+0000","last_fullsized":"2024-09-21T06:09:46.523983+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552250+0000","last_change":"2024-09-21T06:09:47.552984+0000","last_active":"2024-09-21T06:09:48.552250+0000","last_peered":"2024-09-21T06:09:48.552250+0000","last_clean":"2024-09-21T06:09:48.552250+0000","last_became_active":"2024-09-21T06:09:47.550957+0000","last_became_peered":"2024-09-21T06:09:47.550957+0000","last_unstale":"2024-09-21T06:09:48.552250+0000","last_undegraded":"2024-09-21T06:09:48.552250+0000","last_fullsized":"2024-09-21T06:09:48.552250+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:29.803361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.562370+0000","last_change":"2024-09-21T06:09:47.559238+0000","last_active":"2024-09-21T06:09:47.562370+0000","last_peered":"2024-09-21T06:09:47.562370+0000","last_clean":"2024-09-21T06:09:47.562370+0000","last_became_active":"2024-09-21T06:09:47.558875+0000","last_became_peered":"2024-09-21T06:09:47.558875+0000","last_unstale":"2024-09-21T06:09:47.562370+0000","last_undegraded":"2024-09-21T06:09:47.562370+0000","last_fullsized":"2024-09-21T06:09:47.562370+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561508+0000","last_change":"2024-09-21T06:09:47.550887+0000","last_active":"2024-09-21T06:09:47.561508+0000","last_peered":"2024-09-21T06:09:47.561508+0000","last_clean":"2024-09-21T06:09:47.561508+0000","last_became_active":"2024-09-21T06:09:47.550442+0000","last_became_peered":"2024-09-21T06:09:47.550442+0000","last_unstale":"2024-09-21T06:09:47.561508+0000","last_undegraded":"2024-09-21T06:09:47.561508+0000","last_fullsized":"2024-09-21T06:09:47.561508+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:56.405036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568432+0000","last_change":"2024-09-21T06:09:47.560905+0000","last_active":"2024-09-21T06:09:47.568432+0000","last_peered":"2024-09-21T06:09:47.568432+0000","last_clean":"2024-09-21T06:09:47.568432+0000","last_became_active":"2024-09-21T06:09:47.558757+0000","last_became_peered":"2024-09-21T06:09:47.558757+0000","last_unstale":"2024-09-21T06:09:47.568432+0000","last_undegraded":"2024-09-21T06:09:47.568432+0000","last_fullsized":"2024-09-21T06:09:47.568432+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:19.852862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553229+0000","last_change":"2024-09-21T06:09:47.557467+0000","last_active":"2024-09-21T06:09:48.553229+0000","last_peered":"2024-09-21T06:09:48.553229+0000","last_clean":"2024-09-21T06:09:48.553229+0000","last_became_active":"2024-09-21T06:09:47.554901+0000","last_became_peered":"2024-09-21T06:09:47.554901+0000","last_unstale":"2024-09-21T06:09:48.553229+0000","last_undegraded":"2024-09-21T06:09:48.553229+0000","last_fullsized":"2024-09-21T06:09:48.553229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:31.714893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565149+0000","last_change":"2024-09-21T06:09:47.556163+0000","last_active":"2024-09-21T06:09:47.565149+0000","last_peered":"2024-09-21T06:09:47.565149+0000","last_clean":"2024-09-21T06:09:47.565149+0000","last_became_active":"2024-09-21T06:09:47.555656+0000","last_became_peered":"2024-09-21T06:09:47.555656+0000","last_unstale":"2024-09-21T06:09:47.565149+0000","last_undegraded":"2024-09-21T06:09:47.565149+0000","last_fullsized":"2024-09-21T06:09:47.565149+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:44.553253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561358+0000","last_change":"2024-09-21T06:09:47.554101+0000","last_active":"2024-09-21T06:09:47.561358+0000","last_peered":"2024-09-21T06:09:47.561358+0000","last_clean":"2024-09-21T06:09:47.561358+0000","last_became_active":"2024-09-21T06:09:47.553743+0000","last_became_peered":"2024-09-21T06:09:47.553743+0000","last_unstale":"2024-09-21T06:09:47.561358+0000","last_undegraded":"2024-09-21T06:09:47.561358+0000","last_fullsized":"2024-09-21T06:09:47.561358+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:09.595527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561935+0000","last_change":"2024-09-21T06:09:47.554855+0000","last_active":"2024-09-21T06:09:47.561935+0000","last_peered":"2024-09-21T06:09:47.561935+0000","last_clean":"2024-09-21T06:09:47.561935+0000","last_became_active":"2024-09-21T06:09:47.554540+0000","last_became_peered":"2024-09-21T06:09:47.554540+0000","last_unstale":"2024-09-21T06:09:47.561935+0000","last_undegraded":"2024-09-21T06:09:47.561935+0000","last_fullsized":"2024-09-21T06:09:47.561935+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.523903+0000","last_change":"2024-09-21T06:09:46.523903+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.523903+0000","last_undegraded":"2024-09-21T06:09:46.523903+0000","last_fullsized":"2024-09-21T06:09:46.523903+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561644+0000","last_change":"2024-09-21T06:09:47.554644+0000","last_active":"2024-09-21T06:09:47.561644+0000","last_peered":"2024-09-21T06:09:47.561644+0000","last_clean":"2024-09-21T06:09:47.561644+0000","last_became_active":"2024-09-21T06:09:47.554247+0000","last_became_peered":"2024-09-21T06:09:47.554247+0000","last_unstale":"2024-09-21T06:09:47.561644+0000","last_undegraded":"2024-09-21T06:09:47.561644+0000","last_fullsized":"2024-09-21T06:09:47.561644+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:13.781170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552610+0000","last_change":"2024-09-21T06:09:47.557683+0000","last_active":"2024-09-21T06:09:48.552610+0000","last_peered":"2024-09-21T06:09:48.552610+0000","last_clean":"2024-09-21T06:09:48.552610+0000","last_became_active":"2024-09-21T06:09:47.557357+0000","last_became_peered":"2024-09-21T06:09:47.557357+0000","last_unstale":"2024-09-21T06:09:48.552610+0000","last_undegraded":"2024-09-21T06:09:48.552610+0000","last_fullsized":"2024-09-21T06:09:48.552610+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561742+0000","last_change":"2024-09-21T06:09:47.557188+0000","last_active":"2024-09-21T06:09:48.561742+0000","last_peered":"2024-09-21T06:09:48.561742+0000","last_clean":"2024-09-21T06:09:48.561742+0000","last_became_active":"2024-09-21T06:09:47.556870+0000","last_became_peered":"2024-09-21T06:09:47.556870+0000","last_unstale":"2024-09-21T06:09:48.561742+0000","last_undegraded":"2024-09-21T06:09:48.561742+0000","last_fullsized":"2024-09-21T06:09:48.561742+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:38.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564093+0000","last_change":"2024-09-21T06:09:47.562338+0000","last_active":"2024-09-21T06:09:47.564093+0000","last_peered":"2024-09-21T06:09:47.564093+0000","last_clean":"2024-09-21T06:09:47.564093+0000","last_became_active":"2024-09-21T06:09:47.561835+0000","last_became_peered":"2024-09-21T06:09:47.561835+0000","last_unstale":"2024-09-21T06:09:47.564093+0000","last_undegraded":"2024-09-21T06:09:47.564093+0000","last_fullsized":"2024-09-21T06:09:47.564093+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552835+0000","last_change":"2024-09-21T06:09:47.548087+0000","last_active":"2024-09-21T06:09:48.552835+0000","last_peered":"2024-09-21T06:09:48.552835+0000","last_clean":"2024-09-21T06:09:48.552835+0000","last_became_active":"2024-09-21T06:09:47.546530+0000","last_became_peered":"2024-09-21T06:09:47.546530+0000","last_unstale":"2024-09-21T06:09:48.552835+0000","last_undegraded":"2024-09-21T06:09:48.552835+0000","last_fullsized":"2024-09-21T06:09:48.552835+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552595+0000","last_change":"2024-09-21T06:09:47.549010+0000","last_active":"2024-09-21T06:09:48.552595+0000","last_peered":"2024-09-21T06:09:48.552595+0000","last_clean":"2024-09-21T06:09:48.552595+0000","last_became_active":"2024-09-21T06:09:47.548666+0000","last_became_peered":"2024-09-21T06:09:47.548666+0000","last_unstale":"2024-09-21T06:09:48.552595+0000","last_undegraded":"2024-09-21T06:09:48.552595+0000","last_fullsized":"2024-09-21T06:09:48.552595+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565056+0000","last_change":"2024-09-21T06:09:47.554127+0000","last_active":"2024-09-21T06:09:47.565056+0000","last_peered":"2024-09-21T06:09:47.565056+0000","last_clean":"2024-09-21T06:09:47.565056+0000","last_became_active":"2024-09-21T06:09:47.553809+0000","last_became_peered":"2024-09-21T06:09:47.553809+0000","last_unstale":"2024-09-21T06:09:47.565056+0000","last_undegraded":"2024-09-21T06:09:47.565056+0000","last_fullsized":"2024-09-21T06:09:47.565056+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553426+0000","last_change":"2024-09-21T06:09:47.546392+0000","last_active":"2024-09-21T06:09:48.553426+0000","last_peered":"2024-09-21T06:09:48.553426+0000","last_clean":"2024-09-21T06:09:48.553426+0000","last_became_active":"2024-09-21T06:09:47.545962+0000","last_became_peered":"2024-09-21T06:09:47.545962+0000","last_unstale":"2024-09-21T06:09:48.553426+0000","last_undegraded":"2024-09-21T06:09:48.553426+0000","last_fullsized":"2024-09-21T06:09:48.553426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:35.324859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565233+0000","last_change":"2024-09-21T06:09:47.555478+0000","last_active":"2024-09-21T06:09:47.565233+0000","last_peered":"2024-09-21T06:09:47.565233+0000","last_clean":"2024-09-21T06:09:47.565233+0000","last_became_active":"2024-09-21T06:09:47.555175+0000","last_became_peered":"2024-09-21T06:09:47.555175+0000","last_unstale":"2024-09-21T06:09:47.565233+0000","last_undegraded":"2024-09-21T06:09:47.565233+0000","last_fullsized":"2024-09-21T06:09:47.565233+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552852+0000","last_change":"2024-09-21T06:09:47.557085+0000","last_active":"2024-09-21T06:09:48.552852+0000","last_peered":"2024-09-21T06:09:48.552852+0000","last_clean":"2024-09-21T06:09:48.552852+0000","last_became_active":"2024-09-21T06:09:47.556727+0000","last_became_peered":"2024-09-21T06:09:47.556727+0000","last_unstale":"2024-09-21T06:09:48.552852+0000","last_undegraded":"2024-09-21T06:09:48.552852+0000","last_fullsized":"2024-09-21T06:09:48.552852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568402+0000","last_change":"2024-09-21T06:09:47.559094+0000","last_active":"2024-09-21T06:09:47.568402+0000","last_peered":"2024-09-21T06:09:47.568402+0000","last_clean":"2024-09-21T06:09:47.568402+0000","last_became_active":"2024-09-21T06:09:47.558738+0000","last_became_peered":"2024-09-21T06:09:47.558738+0000","last_unstale":"2024-09-21T06:09:47.568402+0000","last_undegraded":"2024-09-21T06:09:47.568402+0000","last_fullsized":"2024-09-21T06:09:47.568402+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:39.640902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.527681+0000","last_change":"2024-09-21T06:09:46.527681+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.527681+0000","last_undegraded":"2024-09-21T06:09:46.527681+0000","last_fullsized":"2024-09-21T06:09:46.527681+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565098+0000","last_change":"2024-09-21T06:09:47.552743+0000","last_active":"2024-09-21T06:09:47.565098+0000","last_peered":"2024-09-21T06:09:47.565098+0000","last_clean":"2024-09-21T06:09:47.565098+0000","last_became_active":"2024-09-21T06:09:47.552402+0000","last_became_peered":"2024-09-21T06:09:47.552402+0000","last_unstale":"2024-09-21T06:09:47.565098+0000","last_undegraded":"2024-09-21T06:09:47.565098+0000","last_fullsized":"2024-09-21T06:09:47.565098+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552061+0000","last_change":"2024-09-21T06:09:47.547136+0000","last_active":"2024-09-21T06:09:48.552061+0000","last_peered":"2024-09-21T06:09:48.552061+0000","last_clean":"2024-09-21T06:09:48.552061+0000","last_became_active":"2024-09-21T06:09:47.546771+0000","last_became_peered":"2024-09-21T06:09:47.546771+0000","last_unstale":"2024-09-21T06:09:48.552061+0000","last_undegraded":"2024-09-21T06:09:48.552061+0000","last_fullsized":"2024-09-21T06:09:48.552061+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:09.595527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553225+0000","last_change":"2024-09-21T06:09:47.550506+0000","last_active":"2024-09-21T06:09:48.553225+0000","last_peered":"2024-09-21T06:09:48.553225+0000","last_clean":"2024-09-21T06:09:48.553225+0000","last_became_active":"2024-09-21T06:09:47.550172+0000","last_became_peered":"2024-09-21T06:09:47.550172+0000","last_unstale":"2024-09-21T06:09:48.553225+0000","last_undegraded":"2024-09-21T06:09:48.553225+0000","last_fullsized":"2024-09-21T06:09:48.553225+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.338192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564838+0000","last_change":"2024-09-21T06:09:47.548514+0000","last_active":"2024-09-21T06:09:47.564838+0000","last_peered":"2024-09-21T06:09:47.564838+0000","last_clean":"2024-09-21T06:09:47.564838+0000","last_became_active":"2024-09-21T06:09:47.548034+0000","last_became_peered":"2024-09-21T06:09:47.548034+0000","last_unstale":"2024-09-21T06:09:47.564838+0000","last_undegraded":"2024-09-21T06:09:47.564838+0000","last_fullsized":"2024-09-21T06:09:47.564838+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:47.209532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.526758+0000","last_change":"2024-09-21T06:09:46.526758+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.526758+0000","last_undegraded":"2024-09-21T06:09:46.526758+0000","last_fullsized":"2024-09-21T06:09:46.526758+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.526087+0000","last_change":"2024-09-21T06:09:46.526087+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.526087+0000","last_undegraded":"2024-09-21T06:09:46.526087+0000","last_fullsized":"2024-09-21T06:09:46.526087+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553791+0000","last_change":"2024-09-21T06:09:46.528315+0000","last_active":"2024-09-21T06:09:48.553791+0000","last_peered":"2024-09-21T06:09:48.553791+0000","last_clean":"2024-09-21T06:09:48.553791+0000","last_became_active":"2024-09-21T06:09:46.527949+0000","last_became_peered":"2024-09-21T06:09:46.527949+0000","last_unstale":"2024-09-21T06:09:48.553791+0000","last_undegraded":"2024-09-21T06:09:48.553791+0000","last_fullsized":"2024-09-21T06:09:48.553791+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:23.941639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561877+0000","last_change":"2024-09-21T06:09:47.553802+0000","last_active":"2024-09-21T06:09:48.561877+0000","last_peered":"2024-09-21T06:09:48.561877+0000","last_clean":"2024-09-21T06:09:48.561877+0000","last_became_active":"2024-09-21T06:09:47.553503+0000","last_became_peered":"2024-09-21T06:09:47.553503+0000","last_unstale":"2024-09-21T06:09:48.561877+0000","last_undegraded":"2024-09-21T06:09:48.561877+0000","last_fullsized":"2024-09-21T06:09:48.561877+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:58.813046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569291+0000","last_change":"2024-09-21T06:09:46.531370+0000","last_active":"2024-09-21T06:09:47.569291+0000","last_peered":"2024-09-21T06:09:47.569291+0000","last_clean":"2024-09-21T06:09:47.569291+0000","last_became_active":"2024-09-21T06:09:46.530960+0000","last_became_peered":"2024-09-21T06:09:46.530960+0000","last_unstale":"2024-09-21T06:09:47.569291+0000","last_undegraded":"2024-09-21T06:09:47.569291+0000","last_fullsized":"2024-09-21T06:09:47.569291+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.528678+0000","last_change":"2024-09-21T06:09:46.528678+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.528678+0000","last_undegraded":"2024-09-21T06:09:46.528678+0000","last_fullsized":"2024-09-21T06:09:46.528678+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.577193+0000","last_change":"2024-09-21T06:09:46.544553+0000","last_active":"2024-09-21T06:09:47.577193+0000","last_peered":"2024-09-21T06:09:47.577193+0000","last_clean":"2024-09-21T06:09:47.577193+0000","last_became_active":"2024-09-21T06:09:46.544246+0000","last_became_peered":"2024-09-21T06:09:46.544246+0000","last_unstale":"2024-09-21T06:09:47.577193+0000","last_undegraded":"2024-09-21T06:09:47.577193+0000","last_fullsized":"2024-09-21T06:09:47.577193+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.718740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.570926+0000","last_change":"2024-09-21T06:09:47.569840+0000","last_active":"2024-09-21T06:09:47.570926+0000","last_peered":"2024-09-21T06:09:47.570926+0000","last_clean":"2024-09-21T06:09:47.570926+0000","last_became_active":"2024-09-21T06:09:47.569541+0000","last_became_peered":"2024-09-21T06:09:47.569541+0000","last_unstale":"2024-09-21T06:09:47.570926+0000","last_undegraded":"2024-09-21T06:09:47.570926+0000","last_fullsized":"2024-09-21T06:09:47.570926+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.577763+0000","last_change":"2024-09-21T06:09:46.543348+0000","last_active":"2024-09-21T06:09:47.577763+0000","last_peered":"2024-09-21T06:09:47.577763+0000","last_clean":"2024-09-21T06:09:47.577763+0000","last_became_active":"2024-09-21T06:09:46.543036+0000","last_became_peered":"2024-09-21T06:09:46.543036+0000","last_unstale":"2024-09-21T06:09:47.577763+0000","last_undegraded":"2024-09-21T06:09:47.577763+0000","last_fullsized":"2024-09-21T06:09:47.577763+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:17.178909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568248+0000","last_change":"2024-09-21T06:09:47.558350+0000","last_active":"2024-09-21T06:09:47.568248+0000","last_peered":"2024-09-21T06:09:47.568248+0000","last_clean":"2024-09-21T06:09:47.568248+0000","last_became_active":"2024-09-21T06:09:47.552686+0000","last_became_peered":"2024-09-21T06:09:47.552686+0000","last_unstale":"2024-09-21T06:09:47.568248+0000","last_undegraded":"2024-09-21T06:09:47.568248+0000","last_fullsized":"2024-09-21T06:09:47.568248+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:05.811070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553484+0000","last_change":"2024-09-21T06:09:46.535845+0000","last_active":"2024-09-21T06:09:48.553484+0000","last_peered":"2024-09-21T06:09:48.553484+0000","last_clean":"2024-09-21T06:09:48.553484+0000","last_became_active":"2024-09-21T06:09:46.535462+0000","last_became_peered":"2024-09-21T06:09:46.535462+0000","last_unstale":"2024-09-21T06:09:48.553484+0000","last_undegraded":"2024-09-21T06:09:48.553484+0000","last_fullsized":"2024-09-21T06:09:48.553484+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:32.585856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553096+0000","last_change":"2024-09-21T06:09:47.556065+0000","last_active":"2024-09-21T06:09:48.553096+0000","last_peered":"2024-09-21T06:09:48.553096+0000","last_clean":"2024-09-21T06:09:48.553096+0000","last_became_active":"2024-09-21T06:09:47.555742+0000","last_became_peered":"2024-09-21T06:09:47.555742+0000","last_unstale":"2024-09-21T06:09:48.553096+0000","last_undegraded":"2024-09-21T06:09:48.553096+0000","last_fullsized":"2024-09-21T06:09:48.553096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.575260+0000","last_change":"2024-09-21T06:09:46.549487+0000","last_active":"2024-09-21T06:09:47.575260+0000","last_peered":"2024-09-21T06:09:47.575260+0000","last_clean":"2024-09-21T06:09:47.575260+0000","last_became_active":"2024-09-21T06:09:46.548860+0000","last_became_peered":"2024-09-21T06:09:46.548860+0000","last_unstale":"2024-09-21T06:09:47.575260+0000","last_undegraded":"2024-09-21T06:09:47.575260+0000","last_fullsized":"2024-09-21T06:09:47.575260+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:45.648740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569391+0000","last_change":"2024-09-21T06:09:47.563169+0000","last_active":"2024-09-21T06:09:47.569391+0000","last_peered":"2024-09-21T06:09:47.569391+0000","last_clean":"2024-09-21T06:09:47.569391+0000","last_became_active":"2024-09-21T06:09:47.562868+0000","last_became_peered":"2024-09-21T06:09:47.562868+0000","last_unstale":"2024-09-21T06:09:47.569391+0000","last_undegraded":"2024-09-21T06:09:47.569391+0000","last_fullsized":"2024-09-21T06:09:47.569391+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:51.138002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551911+0000","last_change":"2024-09-21T06:09:46.545661+0000","last_active":"2024-09-21T06:09:48.551911+0000","last_peered":"2024-09-21T06:09:48.551911+0000","last_clean":"2024-09-21T06:09:48.551911+0000","last_became_active":"2024-09-21T06:09:46.545340+0000","last_became_peered":"2024-09-21T06:09:46.545340+0000","last_unstale":"2024-09-21T06:09:48.551911+0000","last_undegraded":"2024-09-21T06:09:48.551911+0000","last_fullsized":"2024-09-21T06:09:48.551911+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:47.880211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552001+0000","last_change":"2024-09-21T06:09:47.553054+0000","last_active":"2024-09-21T06:09:48.552001+0000","last_peered":"2024-09-21T06:09:48.552001+0000","last_clean":"2024-09-21T06:09:48.552001+0000","last_became_active":"2024-09-21T06:09:47.552742+0000","last_became_peered":"2024-09-21T06:09:47.552742+0000","last_unstale":"2024-09-21T06:09:48.552001+0000","last_undegraded":"2024-09-21T06:09:48.552001+0000","last_fullsized":"2024-09-21T06:09:48.552001+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:54.333587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.542901+0000","last_change":"2024-09-21T06:09:46.536718+0000","last_active":"2024-09-21T06:09:47.542901+0000","last_peered":"2024-09-21T06:09:47.542901+0000","last_clean":"2024-09-21T06:09:47.542901+0000","last_became_active":"2024-09-21T06:09:46.535034+0000","last_became_peered":"2024-09-21T06:09:46.535034+0000","last_unstale":"2024-09-21T06:09:47.542901+0000","last_undegraded":"2024-09-21T06:09:47.542901+0000","last_fullsized":"2024-09-21T06:09:47.542901+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:49.688976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553944+0000","last_change":"2024-09-21T06:09:47.539900+0000","last_active":"2024-09-21T06:09:48.553944+0000","last_peered":"2024-09-21T06:09:48.553944+0000","last_clean":"2024-09-21T06:09:48.553944+0000","last_became_active":"2024-09-21T06:09:47.539412+0000","last_became_peered":"2024-09-21T06:09:47.539412+0000","last_unstale":"2024-09-21T06:09:48.553944+0000","last_undegraded":"2024-09-21T06:09:48.553944+0000","last_fullsized":"2024-09-21T06:09:48.553944+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:52.206476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.532722+0000","last_change":"2024-09-21T06:09:46.540269+0000","last_active":"2024-09-21T06:09:47.532722+0000","last_peered":"2024-09-21T06:09:47.532722+0000","last_clean":"2024-09-21T06:09:47.532722+0000","last_became_active":"2024-09-21T06:09:46.539954+0000","last_became_peered":"2024-09-21T06:09:46.539954+0000","last_unstale":"2024-09-21T06:09:47.532722+0000","last_undegraded":"2024-09-21T06:09:47.532722+0000","last_fullsized":"2024-09-21T06:09:47.532722+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:13.428634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.562059+0000","last_change":"2024-09-21T06:09:47.566984+0000","last_active":"2024-09-21T06:09:48.562059+0000","last_peered":"2024-09-21T06:09:48.562059+0000","last_clean":"2024-09-21T06:09:48.562059+0000","last_became_active":"2024-09-21T06:09:47.566193+0000","last_became_peered":"2024-09-21T06:09:47.566193+0000","last_unstale":"2024-09-21T06:09:48.562059+0000","last_undegraded":"2024-09-21T06:09:48.562059+0000","last_fullsized":"2024-09-21T06:09:48.562059+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:28.501310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.535666+0000","last_change":"2024-09-21T06:09:46.536342+0000","last_active":"2024-09-21T06:09:47.535666+0000","last_peered":"2024-09-21T06:09:47.535666+0000","last_clean":"2024-09-21T06:09:47.535666+0000","last_became_active":"2024-09-21T06:09:46.536012+0000","last_became_peered":"2024-09-21T06:09:46.536012+0000","last_unstale":"2024-09-21T06:09:47.535666+0000","last_undegraded":"2024-09-21T06:09:47.535666+0000","last_fullsized":"2024-09-21T06:09:47.535666+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:57.383442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.552610+0000","last_change":"2024-09-21T06:09:47.543982+0000","last_active":"2024-09-21T06:09:47.552610+0000","last_peered":"2024-09-21T06:09:47.552610+0000","last_clean":"2024-09-21T06:09:47.552610+0000","last_became_active":"2024-09-21T06:09:47.543662+0000","last_became_peered":"2024-09-21T06:09:47.543662+0000","last_unstale":"2024-09-21T06:09:47.552610+0000","last_undegraded":"2024-09-21T06:09:47.552610+0000","last_fullsized":"2024-09-21T06:09:47.552610+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.580229+0000","last_change":"2024-09-21T06:09:46.548929+0000","last_active":"2024-09-21T06:09:47.580229+0000","last_peered":"2024-09-21T06:09:47.580229+0000","last_clean":"2024-09-21T06:09:47.580229+0000","last_became_active":"2024-09-21T06:09:46.548521+0000","last_became_peered":"2024-09-21T06:09:46.548521+0000","last_unstale":"2024-09-21T06:09:47.580229+0000","last_undegraded":"2024-09-21T06:09:47.580229+0000","last_fullsized":"2024-09-21T06:09:47.580229+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:27.494160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553855+0000","last_change":"2024-09-21T06:09:47.554124+0000","last_active":"2024-09-21T06:09:48.553855+0000","last_peered":"2024-09-21T06:09:48.553855+0000","last_clean":"2024-09-21T06:09:48.553855+0000","last_became_active":"2024-09-21T06:09:47.551484+0000","last_became_peered":"2024-09-21T06:09:47.551484+0000","last_unstale":"2024-09-21T06:09:48.553855+0000","last_undegraded":"2024-09-21T06:09:48.553855+0000","last_fullsized":"2024-09-21T06:09:48.553855+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:47.621204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":19,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.582014+0000","last_change":"2024-09-21T06:09:46.539077+0000","last_active":"2024-09-21T06:09:47.582014+0000","last_peered":"2024-09-21T06:09:47.582014+0000","last_clean":"2024-09-21T06:09:47.582014+0000","last_became_active":"2024-09-21T06:09:46.537287+0000","last_became_peered":"2024-09-21T06:09:46.537287+0000","last_unstale":"2024-09-21T06:09:47.582014+0000","last_undegraded":"2024-09-21T06:09:47.582014+0000","last_fullsized":"2024-09-21T06:09:47.582014+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:52.872003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.554303+0000","last_change":"2024-09-21T06:09:47.546084+0000","last_active":"2024-09-21T06:09:47.554303+0000","last_peered":"2024-09-21T06:09:47.554303+0000","last_clean":"2024-09-21T06:09:47.554303+0000","last_became_active":"2024-09-21T06:09:47.545677+0000","last_became_peered":"2024-09-21T06:09:47.545677+0000","last_unstale":"2024-09-21T06:09:47.554303+0000","last_undegraded":"2024-09-21T06:09:47.554303+0000","last_fullsized":"2024-09-21T06:09:47.554303+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T06:09:46.543756+0000","last_change":"2024-09-21T06:09:46.537690+0000","last_active":"2024-09-21T06:09:46.543756+0000","last_peered":"2024-09-21T06:09:46.543756+0000","last_clean":"2024-09-21T06:09:46.543756+0000","last_became_active":"2024-09-21T06:09:46.537157+0000","last_became_peered":"2024-09-21T06:09:46.537157+0000","last_unstale":"2024-09-21T06:09:46.543756+0000","last_undegraded":"2024-09-21T06:09:46.543756+0000","last_fullsized":"2024-09-21T06:09:46.543756+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:02.269032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564968+0000","last_change":"2024-09-21T06:09:47.550402+0000","last_active":"2024-09-21T06:09:47.564968+0000","last_peered":"2024-09-21T06:09:47.564968+0000","last_clean":"2024-09-21T06:09:47.564968+0000","last_became_active":"2024-09-21T06:09:47.550085+0000","last_became_peered":"2024-09-21T06:09:47.550085+0000","last_unstale":"2024-09-21T06:09:47.564968+0000","last_undegraded":"2024-09-21T06:09:47.564968+0000","last_fullsized":"2024-09-21T06:09:47.564968+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561520+0000","last_change":"2024-09-21T06:09:46.524911+0000","last_active":"2024-09-21T06:09:48.561520+0000","last_peered":"2024-09-21T06:09:48.561520+0000","last_clean":"2024-09-21T06:09:48.561520+0000","last_became_active":"2024-09-21T06:09:46.524561+0000","last_became_peered":"2024-09-21T06:09:46.524561+0000","last_unstale":"2024-09-21T06:09:48.561520+0000","last_undegraded":"2024-09-21T06:09:48.561520+0000","last_fullsized":"2024-09-21T06:09:48.561520+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:32.557867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.547211+0000","last_change":"2024-09-21T06:09:47.541745+0000","last_active":"2024-09-21T06:09:47.547211+0000","last_peered":"2024-09-21T06:09:47.547211+0000","last_clean":"2024-09-21T06:09:47.547211+0000","last_became_active":"2024-09-21T06:09:47.541184+0000","last_became_peered":"2024-09-21T06:09:47.541184+0000","last_unstale":"2024-09-21T06:09:47.547211+0000","last_undegraded":"2024-09-21T06:09:47.547211+0000","last_fullsized":"2024-09-21T06:09:47.547211+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:49.440767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":21,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.579446+0000","last_change":"2024-09-21T06:09:46.533914+0000","last_active":"2024-09-21T06:09:47.579446+0000","last_peered":"2024-09-21T06:09:47.579446+0000","last_clean":"2024-09-21T06:09:47.579446+0000","last_became_active":"2024-09-21T06:09:46.533541+0000","last_became_peered":"2024-09-21T06:09:46.533541+0000","last_unstale":"2024-09-21T06:09:47.579446+0000","last_undegraded":"2024-09-21T06:09:47.579446+0000","last_fullsized":"2024-09-21T06:09:47.579446+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:24.340748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569557+0000","last_change":"2024-09-21T06:09:47.557155+0000","last_active":"2024-09-21T06:09:47.569557+0000","last_peered":"2024-09-21T06:09:47.569557+0000","last_clean":"2024-09-21T06:09:47.569557+0000","last_became_active":"2024-09-21T06:09:47.556834+0000","last_became_peered":"2024-09-21T06:09:47.556834+0000","last_unstale":"2024-09-21T06:09:47.569557+0000","last_undegraded":"2024-09-21T06:09:47.569557+0000","last_fullsized":"2024-09-21T06:09:47.569557+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:21.313342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.575298+0000","last_change":"2024-09-21T06:09:46.546891+0000","last_active":"2024-09-21T06:09:47.575298+0000","last_peered":"2024-09-21T06:09:47.575298+0000","last_clean":"2024-09-21T06:09:47.575298+0000","last_became_active":"2024-09-21T06:09:46.546573+0000","last_became_peered":"2024-09-21T06:09:46.546573+0000","last_unstale":"2024-09-21T06:09:47.575298+0000","last_undegraded":"2024-09-21T06:09:47.575298+0000","last_fullsized":"2024-09-21T06:09:47.575298+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:17.766821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.524609+0000","last_change":"2024-09-21T06:09:46.524609+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.524609+0000","last_undegraded":"2024-09-21T06:09:46.524609+0000","last_fullsized":"2024-09-21T06:09:46.524609+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.575222+0000","last_change":"2024-09-21T06:09:46.529780+0000","last_active":"2024-09-21T06:09:47.575222+0000","last_peered":"2024-09-21T06:09:47.575222+0000","last_clean":"2024-09-21T06:09:47.575222+0000","last_became_active":"2024-09-21T06:09:46.528136+0000","last_became_peered":"2024-09-21T06:09:46.528136+0000","last_unstale":"2024-09-21T06:09:47.575222+0000","last_undegraded":"2024-09-21T06:09:47.575222+0000","last_fullsized":"2024-09-21T06:09:47.575222+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:17.381392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568420+0000","last_change":"2024-09-21T06:09:47.567084+0000","last_active":"2024-09-21T06:09:47.568420+0000","last_peered":"2024-09-21T06:09:47.568420+0000","last_clean":"2024-09-21T06:09:47.568420+0000","last_became_active":"2024-09-21T06:09:47.566717+0000","last_became_peered":"2024-09-21T06:09:47.566717+0000","last_unstale":"2024-09-21T06:09:47.568420+0000","last_undegraded":"2024-09-21T06:09:47.568420+0000","last_fullsized":"2024-09-21T06:09:47.568420+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561699+0000","last_change":"2024-09-21T06:09:46.524059+0000","last_active":"2024-09-21T06:09:48.561699+0000","last_peered":"2024-09-21T06:09:48.561699+0000","last_clean":"2024-09-21T06:09:48.561699+0000","last_became_active":"2024-09-21T06:09:46.523459+0000","last_became_peered":"2024-09-21T06:09:46.523459+0000","last_unstale":"2024-09-21T06:09:48.561699+0000","last_undegraded":"2024-09-21T06:09:48.561699+0000","last_fullsized":"2024-09-21T06:09:48.561699+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.562759+0000","last_change":"2024-09-21T06:09:47.561252+0000","last_active":"2024-09-21T06:09:47.562759+0000","last_peered":"2024-09-21T06:09:47.562759+0000","last_clean":"2024-09-21T06:09:47.562759+0000","last_became_active":"2024-09-21T06:09:47.560849+0000","last_became_peered":"2024-09-21T06:09:47.560849+0000","last_unstale":"2024-09-21T06:09:47.562759+0000","last_undegraded":"2024-09-21T06:09:47.562759+0000","last_fullsized":"2024-09-21T06:09:47.562759+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553549+0000","last_change":"2024-09-21T06:09:46.534239+0000","last_active":"2024-09-21T06:09:48.553549+0000","last_peered":"2024-09-21T06:09:48.553549+0000","last_clean":"2024-09-21T06:09:48.553549+0000","last_became_active":"2024-09-21T06:09:46.533715+0000","last_became_peered":"2024-09-21T06:09:46.533715+0000","last_unstale":"2024-09-21T06:09:48.553549+0000","last_undegraded":"2024-09-21T06:09:48.553549+0000","last_fullsized":"2024-09-21T06:09:48.553549+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:08:04.897143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.559568+0000","last_change":"2024-09-21T06:09:47.552337+0000","last_active":"2024-09-21T06:09:47.559568+0000","last_peered":"2024-09-21T06:09:47.559568+0000","last_clean":"2024-09-21T06:09:47.559568+0000","last_became_active":"2024-09-21T06:09:47.552009+0000","last_became_peered":"2024-09-21T06:09:47.552009+0000","last_unstale":"2024-09-21T06:09:47.559568+0000","last_undegraded":"2024-09-21T06:09:47.559568+0000","last_fullsized":"2024-09-21T06:09:47.559568+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:46.655889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T06:09:46.539682+0000","last_change":"2024-09-21T06:09:46.536200+0000","last_active":"2024-09-21T06:09:46.539682+0000","last_peered":"2024-09-21T06:09:46.539682+0000","last_clean":"2024-09-21T06:09:46.539682+0000","last_became_active":"2024-09-21T06:09:46.535894+0000","last_became_peered":"2024-09-21T06:09:46.535894+0000","last_unstale":"2024-09-21T06:09:46.539682+0000","last_undegraded":"2024-09-21T06:09:46.539682+0000","last_fullsized":"2024-09-21T06:09:46.539682+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:46.409661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561750+0000","last_change":"2024-09-21T06:09:47.552189+0000","last_active":"2024-09-21T06:09:47.561750+0000","last_peered":"2024-09-21T06:09:47.561750+0000","last_clean":"2024-09-21T06:09:47.561750+0000","last_became_active":"2024-09-21T06:09:47.548927+0000","last_became_peered":"2024-09-21T06:09:47.548927+0000","last_unstale":"2024-09-21T06:09:47.561750+0000","last_undegraded":"2024-09-21T06:09:47.561750+0000","last_fullsized":"2024-09-21T06:09:47.561750+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:33.335946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.532137+0000","last_change":"2024-09-21T06:09:46.526945+0000","last_active":"2024-09-21T06:09:47.532137+0000","last_peered":"2024-09-21T06:09:47.532137+0000","last_clean":"2024-09-21T06:09:47.532137+0000","last_became_active":"2024-09-21T06:09:46.526588+0000","last_became_peered":"2024-09-21T06:09:46.526588+0000","last_unstale":"2024-09-21T06:09:47.532137+0000","last_undegraded":"2024-09-21T06:09:47.532137+0000","last_fullsized":"2024-09-21T06:09:47.532137+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:14.014304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561635+0000","last_change":"2024-09-21T06:09:47.555659+0000","last_active":"2024-09-21T06:09:47.561635+0000","last_peered":"2024-09-21T06:09:47.561635+0000","last_clean":"2024-09-21T06:09:47.561635+0000","last_became_active":"2024-09-21T06:09:47.555358+0000","last_became_peered":"2024-09-21T06:09:47.555358+0000","last_unstale":"2024-09-21T06:09:47.561635+0000","last_undegraded":"2024-09-21T06:09:47.561635+0000","last_fullsized":"2024-09-21T06:09:47.561635+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T06:09:46.544242+0000","last_change":"2024-09-21T06:09:46.541688+0000","last_active":"2024-09-21T06:09:46.544242+0000","last_peered":"2024-09-21T06:09:46.544242+0000","last_clean":"2024-09-21T06:09:46.544242+0000","last_became_active":"2024-09-21T06:09:46.541322+0000","last_became_peered":"2024-09-21T06:09:46.541322+0000","last_unstale":"2024-09-21T06:09:46.544242+0000","last_undegraded":"2024-09-21T06:09:46.544242+0000","last_fullsized":"2024-09-21T06:09:46.544242+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:39.745209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565078+0000","last_change":"2024-09-21T06:09:47.551665+0000","last_active":"2024-09-21T06:09:47.565078+0000","last_peered":"2024-09-21T06:09:47.565078+0000","last_clean":"2024-09-21T06:09:47.565078+0000","last_became_active":"2024-09-21T06:09:47.551313+0000","last_became_peered":"2024-09-21T06:09:47.551313+0000","last_unstale":"2024-09-21T06:09:47.565078+0000","last_undegraded":"2024-09-21T06:09:47.565078+0000","last_fullsized":"2024-09-21T06:09:47.565078+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.543137+0000","last_change":"2024-09-21T06:09:46.545814+0000","last_active":"2024-09-21T06:09:47.543137+0000","last_peered":"2024-09-21T06:09:47.543137+0000","last_clean":"2024-09-21T06:09:47.543137+0000","last_became_active":"2024-09-21T06:09:46.545413+0000","last_became_peered":"2024-09-21T06:09:46.545413+0000","last_unstale":"2024-09-21T06:09:47.543137+0000","last_undegraded":"2024-09-21T06:09:47.543137+0000","last_fullsized":"2024-09-21T06:09:47.543137+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:13.029274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561834+0000","last_change":"2024-09-21T06:09:47.566967+0000","last_active":"2024-09-21T06:09:48.561834+0000","last_peered":"2024-09-21T06:09:48.561834+0000","last_clean":"2024-09-21T06:09:48.561834+0000","last_became_active":"2024-09-21T06:09:47.566211+0000","last_became_peered":"2024-09-21T06:09:47.566211+0000","last_unstale":"2024-09-21T06:09:48.561834+0000","last_undegraded":"2024-09-21T06:09:48.561834+0000","last_fullsized":"2024-09-21T06:09:48.561834+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:14.036424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.580510+0000","last_change":"2024-09-21T06:09:46.528896+0000","last_active":"2024-09-21T06:09:47.580510+0000","last_peered":"2024-09-21T06:09:47.580510+0000","last_clean":"2024-09-21T06:09:47.580510+0000","last_became_active":"2024-09-21T06:09:46.528590+0000","last_became_peered":"2024-09-21T06:09:46.528590+0000","last_unstale":"2024-09-21T06:09:47.580510+0000","last_undegraded":"2024-09-21T06:09:47.580510+0000","last_fullsized":"2024-09-21T06:09:47.580510+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564780+0000","last_change":"2024-09-21T06:09:47.556097+0000","last_active":"2024-09-21T06:09:47.564780+0000","last_peered":"2024-09-21T06:09:47.564780+0000","last_clean":"2024-09-21T06:09:47.564780+0000","last_became_active":"2024-09-21T06:09:47.555720+0000","last_became_peered":"2024-09-21T06:09:47.555720+0000","last_unstale":"2024-09-21T06:09:47.564780+0000","last_undegraded":"2024-09-21T06:09:47.564780+0000","last_fullsized":"2024-09-21T06:09:47.564780+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:14.289349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.563317+0000","last_change":"2024-09-21T06:09:46.525404+0000","last_active":"2024-09-21T06:09:48.563317+0000","last_peered":"2024-09-21T06:09:48.563317+0000","last_clean":"2024-09-21T06:09:48.563317+0000","last_became_active":"2024-09-21T06:09:46.525087+0000","last_became_peered":"2024-09-21T06:09:46.525087+0000","last_unstale":"2024-09-21T06:09:48.563317+0000","last_undegraded":"2024-09-21T06:09:48.563317+0000","last_fullsized":"2024-09-21T06:09:48.563317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:57.383442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.572681+0000","last_change":"2024-09-21T06:09:47.565828+0000","last_active":"2024-09-21T06:09:47.572681+0000","last_peered":"2024-09-21T06:09:47.572681+0000","last_clean":"2024-09-21T06:09:47.572681+0000","last_became_active":"2024-09-21T06:09:47.565467+0000","last_became_peered":"2024-09-21T06:09:47.565467+0000","last_unstale":"2024-09-21T06:09:47.572681+0000","last_undegraded":"2024-09-21T06:09:47.572681+0000","last_fullsized":"2024-09-21T06:09:47.572681+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:37.650305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.544753+0000","last_change":"2024-09-21T06:09:46.542691+0000","last_active":"2024-09-21T06:09:47.544753+0000","last_peered":"2024-09-21T06:09:47.544753+0000","last_clean":"2024-09-21T06:09:47.544753+0000","last_became_active":"2024-09-21T06:09:46.542293+0000","last_became_peered":"2024-09-21T06:09:46.542293+0000","last_unstale":"2024-09-21T06:09:47.544753+0000","last_undegraded":"2024-09-21T06:09:47.544753+0000","last_fullsized":"2024-09-21T06:09:47.544753+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:33.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561749+0000","last_change":"2024-09-21T06:09:47.576750+0000","last_active":"2024-09-21T06:09:48.561749+0000","last_peered":"2024-09-21T06:09:48.561749+0000","last_clean":"2024-09-21T06:09:48.561749+0000","last_became_active":"2024-09-21T06:09:47.576443+0000","last_became_peered":"2024-09-21T06:09:47.576443+0000","last_unstale":"2024-09-21T06:09:48.561749+0000","last_undegraded":"2024-09-21T06:09:48.561749+0000","last_fullsized":"2024-09-21T06:09:48.561749+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.542616+0000","last_change":"2024-09-21T06:09:46.547408+0000","last_active":"2024-09-21T06:09:47.542616+0000","last_peered":"2024-09-21T06:09:47.542616+0000","last_clean":"2024-09-21T06:09:47.542616+0000","last_became_active":"2024-09-21T06:09:46.547043+0000","last_became_peered":"2024-09-21T06:09:46.547043+0000","last_unstale":"2024-09-21T06:09:47.542616+0000","last_undegraded":"2024-09-21T06:09:47.542616+0000","last_fullsized":"2024-09-21T06:09:47.542616+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:08.588377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552144+0000","last_change":"2024-09-21T06:09:47.540415+0000","last_active":"2024-09-21T06:09:48.552144+0000","last_peered":"2024-09-21T06:09:48.552144+0000","last_clean":"2024-09-21T06:09:48.552144+0000","last_became_active":"2024-09-21T06:09:47.539926+0000","last_became_peered":"2024-09-21T06:09:47.539926+0000","last_unstale":"2024-09-21T06:09:48.552144+0000","last_undegraded":"2024-09-21T06:09:48.552144+0000","last_fullsized":"2024-09-21T06:09:48.552144+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553533+0000","last_change":"2024-09-21T06:09:46.529617+0000","last_active":"2024-09-21T06:09:48.553533+0000","last_peered":"2024-09-21T06:09:48.553533+0000","last_clean":"2024-09-21T06:09:48.553533+0000","last_became_active":"2024-09-21T06:09:46.528234+0000","last_became_peered":"2024-09-21T06:09:46.528234+0000","last_unstale":"2024-09-21T06:09:48.553533+0000","last_undegraded":"2024-09-21T06:09:48.553533+0000","last_fullsized":"2024-09-21T06:09:48.553533+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:19.814656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552277+0000","last_change":"2024-09-21T06:09:47.543597+0000","last_active":"2024-09-21T06:09:48.552277+0000","last_peered":"2024-09-21T06:09:48.552277+0000","last_clean":"2024-09-21T06:09:48.552277+0000","last_became_active":"2024-09-21T06:09:47.543282+0000","last_became_peered":"2024-09-21T06:09:47.543282+0000","last_unstale":"2024-09-21T06:09:48.552277+0000","last_undegraded":"2024-09-21T06:09:48.552277+0000","last_fullsized":"2024-09-21T06:09:48.552277+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552294+0000","last_change":"2024-09-21T06:09:46.535746+0000","last_active":"2024-09-21T06:09:48.552294+0000","last_peered":"2024-09-21T06:09:48.552294+0000","last_clean":"2024-09-21T06:09:48.552294+0000","last_became_active":"2024-09-21T06:09:46.535142+0000","last_became_peered":"2024-09-21T06:09:46.535142+0000","last_unstale":"2024-09-21T06:09:48.552294+0000","last_undegraded":"2024-09-21T06:09:48.552294+0000","last_fullsized":"2024-09-21T06:09:48.552294+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:56.819951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561609+0000","last_change":"2024-09-21T06:09:47.553873+0000","last_active":"2024-09-21T06:09:47.561609+0000","last_peered":"2024-09-21T06:09:47.561609+0000","last_clean":"2024-09-21T06:09:47.561609+0000","last_became_active":"2024-09-21T06:09:47.553515+0000","last_became_peered":"2024-09-21T06:09:47.553515+0000","last_unstale":"2024-09-21T06:09:47.561609+0000","last_undegraded":"2024-09-21T06:09:47.561609+0000","last_fullsized":"2024-09-21T06:09:47.561609+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:49.440767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.532370+0000","last_change":"2024-09-21T06:09:46.539038+0000","last_active":"2024-09-21T06:09:47.532370+0000","last_peered":"2024-09-21T06:09:47.532370+0000","last_clean":"2024-09-21T06:09:47.532370+0000","last_became_active":"2024-09-21T06:09:46.538692+0000","last_became_peered":"2024-09-21T06:09:46.538692+0000","last_unstale":"2024-09-21T06:09:47.532370+0000","last_undegraded":"2024-09-21T06:09:47.532370+0000","last_fullsized":"2024-09-21T06:09:47.532370+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:44.565196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569536+0000","last_change":"2024-09-21T06:09:47.564522+0000","last_active":"2024-09-21T06:09:47.569536+0000","last_peered":"2024-09-21T06:09:47.569536+0000","last_clean":"2024-09-21T06:09:47.569536+0000","last_became_active":"2024-09-21T06:09:47.564206+0000","last_became_peered":"2024-09-21T06:09:47.564206+0000","last_unstale":"2024-09-21T06:09:47.569536+0000","last_undegraded":"2024-09-21T06:09:47.569536+0000","last_fullsized":"2024-09-21T06:09:47.569536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:03.505698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.543377+0000","last_change":"2024-09-21T06:09:46.532492+0000","last_active":"2024-09-21T06:09:47.543377+0000","last_peered":"2024-09-21T06:09:47.543377+0000","last_clean":"2024-09-21T06:09:47.543377+0000","last_became_active":"2024-09-21T06:09:46.530409+0000","last_became_peered":"2024-09-21T06:09:46.530409+0000","last_unstale":"2024-09-21T06:09:47.543377+0000","last_undegraded":"2024-09-21T06:09:47.543377+0000","last_fullsized":"2024-09-21T06:09:47.543377+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:57.805896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.549238+0000","last_change":"2024-09-21T06:09:47.543465+0000","last_active":"2024-09-21T06:09:47.549238+0000","last_peered":"2024-09-21T06:09:47.549238+0000","last_clean":"2024-09-21T06:09:47.549238+0000","last_became_active":"2024-09-21T06:09:47.543121+0000","last_became_peered":"2024-09-21T06:09:47.543121+0000","last_unstale":"2024-09-21T06:09:47.549238+0000","last_undegraded":"2024-09-21T06:09:47.549238+0000","last_fullsized":"2024-09-21T06:09:47.549238+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552395+0000","last_change":"2024-09-21T06:09:46.534147+0000","last_active":"2024-09-21T06:09:48.552395+0000","last_peered":"2024-09-21T06:09:48.552395+0000","last_clean":"2024-09-21T06:09:48.552395+0000","last_became_active":"2024-09-21T06:09:46.533758+0000","last_became_peered":"2024-09-21T06:09:46.533758+0000","last_unstale":"2024-09-21T06:09:48.552395+0000","last_undegraded":"2024-09-21T06:09:48.552395+0000","last_fullsized":"2024-09-21T06:09:48.552395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:29.394474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":277,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.543464+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:09:47.543464+0000","last_peered":"2024-09-21T06:09:47.543464+0000","last_clean":"2024-09-21T06:09:47.543464+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:09:47.543464+0000","last_undegraded":"2024-09-21T06:09:47.543464+0000","last_fullsized":"2024-09-21T06:09:47.543464+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509605,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127456,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244384,"statfs":{"total":96636764160,"available":96506249216,"internally_reserved":0,"allocated":1974272,"data_stored":1186242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509605,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118560,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253280,"statfs":{"total":96636764160,"available":96515358720,"internally_reserved":0,"allocated":1388544,"data_stored":601712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509604,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82456,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289384,"statfs":{"total":96636764160,"available":96552329216,"internally_reserved":0,"allocated":1380352,"data_stored":595862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509605,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144920,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226920,"statfs":{"total":96636764160,"available":96488366080,"internally_reserved":0,"allocated":1372160,"data_stored":595258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509604,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135208,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236632,"statfs":{"total":96636764160,"available":96498311168,"internally_reserved":0,"allocated":1388544,"data_stored":596940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509606,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140760,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231080,"statfs":{"total":96636764160,"available":96492625920,"internally_reserved":0,"allocated":1372160,"data_stored":595258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509604,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131596,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240244,"statfs":{"total":96636764160,"available":96502009856,"internally_reserved":0,"allocated":1953792,"data_stored":1178767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542309,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144908,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226932,"statfs":{"total":96636764160,"available":96488378368,"internally_reserved":0,"allocated":1359872,"data_stored":588399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:09:52.171 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:09:52.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.369+0000 7f1c89800640 1 -- 172.21.15.50:0/3182103845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c840749a0 msgr2=0x7f1c84074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.369+0000 7f1c89800640 1 --2- 172.21.15.50:0/3182103845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c840749a0 0x7f1c84074db0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f1c840726f0 tx=0x7f1c7401c8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.369+0000 7f1c89800640 1 -- 172.21.15.50:0/3182103845 shutdown_connections 2024-09-21T06:09:52.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.369+0000 7f1c89800640 1 --2- 172.21.15.50:0/3182103845 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1c84160330 0x7f1c84160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.369+0000 7f1c89800640 1 --2- 172.21.15.50:0/3182103845 >> v2:172.21.15.88:3300/0 conn(0x7f1c840cefa0 0x7f1c840cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.369+0000 7f1c89800640 1 --2- 172.21.15.50:0/3182103845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c840749a0 0x7f1c84074db0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.369+0000 7f1c89800640 1 -- 172.21.15.50:0/3182103845 >> 172.21.15.50:0/3182103845 conn(0x7f1c840b2040 msgr2=0x7f1c840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:52.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.369+0000 7f1c89800640 1 -- 172.21.15.50:0/3182103845 shutdown_connections 2024-09-21T06:09:52.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.369+0000 7f1c89800640 1 -- 172.21.15.50:0/3182103845 wait complete. 2024-09-21T06:09:52.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c89800640 1 Processor -- start 2024-09-21T06:09:52.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c89800640 1 -- start start 2024-09-21T06:09:52.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c89800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c840749a0 0x7f1c8413d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:52.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c89800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1c840cefa0 0x7f1c840ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:52.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c89800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c84160330 0x7f1c841456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:52.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c89800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c840b3ee0 con 0x7f1c84160330 2024-09-21T06:09:52.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c89800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1c840b3d60 con 0x7f1c840749a0 2024-09-21T06:09:52.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c89800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1c840b4060 con 0x7f1c840cefa0 2024-09-21T06:09:52.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c83400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c84160330 0x7f1c841456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:52.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c83400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c84160330 0x7f1c841456d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37940/0 (socket says 172.21.15.50:37940) 2024-09-21T06:09:52.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c83400640 1 -- 172.21.15.50:0/3752986520 learned_addr learned my addr 172.21.15.50:0/3752986520 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:52.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c82a00640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c840749a0 0x7f1c8413d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:52.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c7be00640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1c840cefa0 0x7f1c840ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:52.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c83400640 1 -- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1c840cefa0 msgr2=0x7f1c840ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c83400640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1c840cefa0 0x7f1c840ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c83400640 1 -- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c840749a0 msgr2=0x7f1c8413d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c83400640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c840749a0 0x7f1c8413d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c83400640 1 -- 172.21.15.50:0/3752986520 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c841fa3e0 con 0x7f1c84160330 2024-09-21T06:09:52.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c83400640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c84160330 0x7f1c841456d0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00e970 tx=0x7f1c7c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:52.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c7aa00640 1 -- 172.21.15.50:0/3752986520 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c004d30 con 0x7f1c84160330 2024-09-21T06:09:52.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c7aa00640 1 -- 172.21.15.50:0/3752986520 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c7c00cb20 con 0x7f1c84160330 2024-09-21T06:09:52.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c89800640 1 -- 172.21.15.50:0/3752986520 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c841fa6c0 con 0x7f1c84160330 2024-09-21T06:09:52.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c7aa00640 1 -- 172.21.15.50:0/3752986520 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c005020 con 0x7f1c84160330 2024-09-21T06:09:52.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.373+0000 7f1c89800640 1 -- 172.21.15.50:0/3752986520 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c841fac80 con 0x7f1c84160330 2024-09-21T06:09:52.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.377+0000 7f1c89800640 1 -- 172.21.15.50:0/3752986520 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c841fb060 con 0x7f1c84160330 2024-09-21T06:09:52.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.377+0000 7f1c7aa00640 1 -- 172.21.15.50:0/3752986520 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f1c7c0051c0 con 0x7f1c84160330 2024-09-21T06:09:52.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.377+0000 7f1c7aa00640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1c60074c70 0x7f1c60077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:52.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.377+0000 7f1c7aa00640 1 -- 172.21.15.50:0/3752986520 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f1c7c093910 con 0x7f1c84160330 2024-09-21T06:09:52.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.381+0000 7f1c82a00640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1c60074c70 0x7f1c60077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:52.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.381+0000 7f1c82a00640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1c60074c70 0x7f1c60077180 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1c740af000 tx=0x7f1c7401b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.385+0000 7f1c7aa00640 1 -- 172.21.15.50:0/3752986520 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1c841fb060 con 0x7f1c84160330 2024-09-21T06:09:52.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.561+0000 7f1c89800640 1 -- 172.21.15.50:0/3752986520 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1c841fa190 con 0x7f1c60074c70 2024-09-21T06:09:52.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.565+0000 7f1c7aa00640 1 -- 172.21.15.50:0/3752986520 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251905 (secure 0 0 0) 0x7f1c740f9820 con 0x7f1c60074c70 2024-09-21T06:09:52.568 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:52.570 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:09:52.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.569+0000 7f1c89800640 1 -- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1c60074c70 msgr2=0x7f1c60077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.569+0000 7f1c89800640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1c60074c70 0x7f1c60077180 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1c740af000 tx=0x7f1c7401b100 comp rx=0 tx=0).stop 2024-09-21T06:09:52.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.569+0000 7f1c89800640 1 -- 172.21.15.50:0/3752986520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c84160330 msgr2=0x7f1c841456d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.569+0000 7f1c89800640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c84160330 0x7f1c841456d0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00e970 tx=0x7f1c7c00ee30 comp rx=0 tx=0).stop 2024-09-21T06:09:52.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.569+0000 7f1c89800640 1 -- 172.21.15.50:0/3752986520 shutdown_connections 2024-09-21T06:09:52.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.569+0000 7f1c89800640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1c60074c70 0x7f1c60077180 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.569+0000 7f1c89800640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c84160330 0x7f1c841456d0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.569+0000 7f1c89800640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1c840cefa0 0x7f1c840ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.569+0000 7f1c89800640 1 --2- 172.21.15.50:0/3752986520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c840749a0 0x7f1c8413d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.569+0000 7f1c89800640 1 -- 172.21.15.50:0/3752986520 >> 172.21.15.50:0/3752986520 conn(0x7f1c840b2040 msgr2=0x7f1c84141cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:52.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.569+0000 7f1c89800640 1 -- 172.21.15.50:0/3752986520 shutdown_connections 2024-09-21T06:09:52.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.569+0000 7f1c89800640 1 -- 172.21.15.50:0/3752986520 wait complete. 2024-09-21T06:09:52.590 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-09-21T06:09:51.183805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1025864,"kb_used_data":11904,"kb_used_omap":12,"kb_used_meta":1013747,"kb_avail":753948856,"statfs":{"total":773094113280,"available":772043628544,"internally_reserved":0,"allocated":12189696,"data_stored":5938438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1038077488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.673149"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.572843+0000","last_change":"2024-09-21T06:09:47.570525+0000","last_active":"2024-09-21T06:09:47.572843+0000","last_peered":"2024-09-21T06:09:47.572843+0000","last_clean":"2024-09-21T06:09:47.572843+0000","last_became_active":"2024-09-21T06:09:47.567795+0000","last_became_peered":"2024-09-21T06:09:47.567795+0000","last_unstale":"2024-09-21T06:09:47.572843+0000","last_undegraded":"2024-09-21T06:09:47.572843+0000","last_fullsized":"2024-09-21T06:09:47.572843+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:18.342305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552384+0000","last_change":"2024-09-21T06:09:47.552824+0000","last_active":"2024-09-21T06:09:48.552384+0000","last_peered":"2024-09-21T06:09:48.552384+0000","last_clean":"2024-09-21T06:09:48.552384+0000","last_became_active":"2024-09-21T06:09:47.552490+0000","last_became_peered":"2024-09-21T06:09:47.552490+0000","last_unstale":"2024-09-21T06:09:48.552384+0000","last_undegraded":"2024-09-21T06:09:48.552384+0000","last_fullsized":"2024-09-21T06:09:48.552384+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.562410+0000","last_change":"2024-09-21T06:09:47.560626+0000","last_active":"2024-09-21T06:09:47.562410+0000","last_peered":"2024-09-21T06:09:47.562410+0000","last_clean":"2024-09-21T06:09:47.562410+0000","last_became_active":"2024-09-21T06:09:47.560221+0000","last_became_peered":"2024-09-21T06:09:47.560221+0000","last_unstale":"2024-09-21T06:09:47.562410+0000","last_undegraded":"2024-09-21T06:09:47.562410+0000","last_fullsized":"2024-09-21T06:09:47.562410+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561543+0000","last_change":"2024-09-21T06:09:47.551941+0000","last_active":"2024-09-21T06:09:47.561543+0000","last_peered":"2024-09-21T06:09:47.561543+0000","last_clean":"2024-09-21T06:09:47.561543+0000","last_became_active":"2024-09-21T06:09:47.551615+0000","last_became_peered":"2024-09-21T06:09:47.551615+0000","last_unstale":"2024-09-21T06:09:47.561543+0000","last_undegraded":"2024-09-21T06:09:47.561543+0000","last_fullsized":"2024-09-21T06:09:47.561543+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:54.401707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.544241+0000","last_change":"2024-09-21T06:09:47.541551+0000","last_active":"2024-09-21T06:09:47.544241+0000","last_peered":"2024-09-21T06:09:47.544241+0000","last_clean":"2024-09-21T06:09:47.544241+0000","last_became_active":"2024-09-21T06:09:47.541207+0000","last_became_peered":"2024-09-21T06:09:47.541207+0000","last_unstale":"2024-09-21T06:09:47.544241+0000","last_undegraded":"2024-09-21T06:09:47.544241+0000","last_fullsized":"2024-09-21T06:09:47.544241+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:32.688598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.523983+0000","last_change":"2024-09-21T06:09:46.523983+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.523983+0000","last_undegraded":"2024-09-21T06:09:46.523983+0000","last_fullsized":"2024-09-21T06:09:46.523983+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552250+0000","last_change":"2024-09-21T06:09:47.552984+0000","last_active":"2024-09-21T06:09:48.552250+0000","last_peered":"2024-09-21T06:09:48.552250+0000","last_clean":"2024-09-21T06:09:48.552250+0000","last_became_active":"2024-09-21T06:09:47.550957+0000","last_became_peered":"2024-09-21T06:09:47.550957+0000","last_unstale":"2024-09-21T06:09:48.552250+0000","last_undegraded":"2024-09-21T06:09:48.552250+0000","last_fullsized":"2024-09-21T06:09:48.552250+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:29.803361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.562370+0000","last_change":"2024-09-21T06:09:47.559238+0000","last_active":"2024-09-21T06:09:47.562370+0000","last_peered":"2024-09-21T06:09:47.562370+0000","last_clean":"2024-09-21T06:09:47.562370+0000","last_became_active":"2024-09-21T06:09:47.558875+0000","last_became_peered":"2024-09-21T06:09:47.558875+0000","last_unstale":"2024-09-21T06:09:47.562370+0000","last_undegraded":"2024-09-21T06:09:47.562370+0000","last_fullsized":"2024-09-21T06:09:47.562370+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561508+0000","last_change":"2024-09-21T06:09:47.550887+0000","last_active":"2024-09-21T06:09:47.561508+0000","last_peered":"2024-09-21T06:09:47.561508+0000","last_clean":"2024-09-21T06:09:47.561508+0000","last_became_active":"2024-09-21T06:09:47.550442+0000","last_became_peered":"2024-09-21T06:09:47.550442+0000","last_unstale":"2024-09-21T06:09:47.561508+0000","last_undegraded":"2024-09-21T06:09:47.561508+0000","last_fullsized":"2024-09-21T06:09:47.561508+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:56.405036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568432+0000","last_change":"2024-09-21T06:09:47.560905+0000","last_active":"2024-09-21T06:09:47.568432+0000","last_peered":"2024-09-21T06:09:47.568432+0000","last_clean":"2024-09-21T06:09:47.568432+0000","last_became_active":"2024-09-21T06:09:47.558757+0000","last_became_peered":"2024-09-21T06:09:47.558757+0000","last_unstale":"2024-09-21T06:09:47.568432+0000","last_undegraded":"2024-09-21T06:09:47.568432+0000","last_fullsized":"2024-09-21T06:09:47.568432+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:19.852862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553229+0000","last_change":"2024-09-21T06:09:47.557467+0000","last_active":"2024-09-21T06:09:48.553229+0000","last_peered":"2024-09-21T06:09:48.553229+0000","last_clean":"2024-09-21T06:09:48.553229+0000","last_became_active":"2024-09-21T06:09:47.554901+0000","last_became_peered":"2024-09-21T06:09:47.554901+0000","last_unstale":"2024-09-21T06:09:48.553229+0000","last_undegraded":"2024-09-21T06:09:48.553229+0000","last_fullsized":"2024-09-21T06:09:48.553229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:31.714893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565149+0000","last_change":"2024-09-21T06:09:47.556163+0000","last_active":"2024-09-21T06:09:47.565149+0000","last_peered":"2024-09-21T06:09:47.565149+0000","last_clean":"2024-09-21T06:09:47.565149+0000","last_became_active":"2024-09-21T06:09:47.555656+0000","last_became_peered":"2024-09-21T06:09:47.555656+0000","last_unstale":"2024-09-21T06:09:47.565149+0000","last_undegraded":"2024-09-21T06:09:47.565149+0000","last_fullsized":"2024-09-21T06:09:47.565149+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:44.553253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561358+0000","last_change":"2024-09-21T06:09:47.554101+0000","last_active":"2024-09-21T06:09:47.561358+0000","last_peered":"2024-09-21T06:09:47.561358+0000","last_clean":"2024-09-21T06:09:47.561358+0000","last_became_active":"2024-09-21T06:09:47.553743+0000","last_became_peered":"2024-09-21T06:09:47.553743+0000","last_unstale":"2024-09-21T06:09:47.561358+0000","last_undegraded":"2024-09-21T06:09:47.561358+0000","last_fullsized":"2024-09-21T06:09:47.561358+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:09.595527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561935+0000","last_change":"2024-09-21T06:09:47.554855+0000","last_active":"2024-09-21T06:09:47.561935+0000","last_peered":"2024-09-21T06:09:47.561935+0000","last_clean":"2024-09-21T06:09:47.561935+0000","last_became_active":"2024-09-21T06:09:47.554540+0000","last_became_peered":"2024-09-21T06:09:47.554540+0000","last_unstale":"2024-09-21T06:09:47.561935+0000","last_undegraded":"2024-09-21T06:09:47.561935+0000","last_fullsized":"2024-09-21T06:09:47.561935+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.523903+0000","last_change":"2024-09-21T06:09:46.523903+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.523903+0000","last_undegraded":"2024-09-21T06:09:46.523903+0000","last_fullsized":"2024-09-21T06:09:46.523903+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561644+0000","last_change":"2024-09-21T06:09:47.554644+0000","last_active":"2024-09-21T06:09:47.561644+0000","last_peered":"2024-09-21T06:09:47.561644+0000","last_clean":"2024-09-21T06:09:47.561644+0000","last_became_active":"2024-09-21T06:09:47.554247+0000","last_became_peered":"2024-09-21T06:09:47.554247+0000","last_unstale":"2024-09-21T06:09:47.561644+0000","last_undegraded":"2024-09-21T06:09:47.561644+0000","last_fullsized":"2024-09-21T06:09:47.561644+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:13.781170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552610+0000","last_change":"2024-09-21T06:09:47.557683+0000","last_active":"2024-09-21T06:09:48.552610+0000","last_peered":"2024-09-21T06:09:48.552610+0000","last_clean":"2024-09-21T06:09:48.552610+0000","last_became_active":"2024-09-21T06:09:47.557357+0000","last_became_peered":"2024-09-21T06:09:47.557357+0000","last_unstale":"2024-09-21T06:09:48.552610+0000","last_undegraded":"2024-09-21T06:09:48.552610+0000","last_fullsized":"2024-09-21T06:09:48.552610+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561742+0000","last_change":"2024-09-21T06:09:47.557188+0000","last_active":"2024-09-21T06:09:48.561742+0000","last_peered":"2024-09-21T06:09:48.561742+0000","last_clean":"2024-09-21T06:09:48.561742+0000","last_became_active":"2024-09-21T06:09:47.556870+0000","last_became_peered":"2024-09-21T06:09:47.556870+0000","last_unstale":"2024-09-21T06:09:48.561742+0000","last_undegraded":"2024-09-21T06:09:48.561742+0000","last_fullsized":"2024-09-21T06:09:48.561742+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:38.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564093+0000","last_change":"2024-09-21T06:09:47.562338+0000","last_active":"2024-09-21T06:09:47.564093+0000","last_peered":"2024-09-21T06:09:47.564093+0000","last_clean":"2024-09-21T06:09:47.564093+0000","last_became_active":"2024-09-21T06:09:47.561835+0000","last_became_peered":"2024-09-21T06:09:47.561835+0000","last_unstale":"2024-09-21T06:09:47.564093+0000","last_undegraded":"2024-09-21T06:09:47.564093+0000","last_fullsized":"2024-09-21T06:09:47.564093+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552835+0000","last_change":"2024-09-21T06:09:47.548087+0000","last_active":"2024-09-21T06:09:48.552835+0000","last_peered":"2024-09-21T06:09:48.552835+0000","last_clean":"2024-09-21T06:09:48.552835+0000","last_became_active":"2024-09-21T06:09:47.546530+0000","last_became_peered":"2024-09-21T06:09:47.546530+0000","last_unstale":"2024-09-21T06:09:48.552835+0000","last_undegraded":"2024-09-21T06:09:48.552835+0000","last_fullsized":"2024-09-21T06:09:48.552835+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552595+0000","last_change":"2024-09-21T06:09:47.549010+0000","last_active":"2024-09-21T06:09:48.552595+0000","last_peered":"2024-09-21T06:09:48.552595+0000","last_clean":"2024-09-21T06:09:48.552595+0000","last_became_active":"2024-09-21T06:09:47.548666+0000","last_became_peered":"2024-09-21T06:09:47.548666+0000","last_unstale":"2024-09-21T06:09:48.552595+0000","last_undegraded":"2024-09-21T06:09:48.552595+0000","last_fullsized":"2024-09-21T06:09:48.552595+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565056+0000","last_change":"2024-09-21T06:09:47.554127+0000","last_active":"2024-09-21T06:09:47.565056+0000","last_peered":"2024-09-21T06:09:47.565056+0000","last_clean":"2024-09-21T06:09:47.565056+0000","last_became_active":"2024-09-21T06:09:47.553809+0000","last_became_peered":"2024-09-21T06:09:47.553809+0000","last_unstale":"2024-09-21T06:09:47.565056+0000","last_undegraded":"2024-09-21T06:09:47.565056+0000","last_fullsized":"2024-09-21T06:09:47.565056+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553426+0000","last_change":"2024-09-21T06:09:47.546392+0000","last_active":"2024-09-21T06:09:48.553426+0000","last_peered":"2024-09-21T06:09:48.553426+0000","last_clean":"2024-09-21T06:09:48.553426+0000","last_became_active":"2024-09-21T06:09:47.545962+0000","last_became_peered":"2024-09-21T06:09:47.545962+0000","last_unstale":"2024-09-21T06:09:48.553426+0000","last_undegraded":"2024-09-21T06:09:48.553426+0000","last_fullsized":"2024-09-21T06:09:48.553426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:35.324859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565233+0000","last_change":"2024-09-21T06:09:47.555478+0000","last_active":"2024-09-21T06:09:47.565233+0000","last_peered":"2024-09-21T06:09:47.565233+0000","last_clean":"2024-09-21T06:09:47.565233+0000","last_became_active":"2024-09-21T06:09:47.555175+0000","last_became_peered":"2024-09-21T06:09:47.555175+0000","last_unstale":"2024-09-21T06:09:47.565233+0000","last_undegraded":"2024-09-21T06:09:47.565233+0000","last_fullsized":"2024-09-21T06:09:47.565233+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552852+0000","last_change":"2024-09-21T06:09:47.557085+0000","last_active":"2024-09-21T06:09:48.552852+0000","last_peered":"2024-09-21T06:09:48.552852+0000","last_clean":"2024-09-21T06:09:48.552852+0000","last_became_active":"2024-09-21T06:09:47.556727+0000","last_became_peered":"2024-09-21T06:09:47.556727+0000","last_unstale":"2024-09-21T06:09:48.552852+0000","last_undegraded":"2024-09-21T06:09:48.552852+0000","last_fullsized":"2024-09-21T06:09:48.552852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568402+0000","last_change":"2024-09-21T06:09:47.559094+0000","last_active":"2024-09-21T06:09:47.568402+0000","last_peered":"2024-09-21T06:09:47.568402+0000","last_clean":"2024-09-21T06:09:47.568402+0000","last_became_active":"2024-09-21T06:09:47.558738+0000","last_became_peered":"2024-09-21T06:09:47.558738+0000","last_unstale":"2024-09-21T06:09:47.568402+0000","last_undegraded":"2024-09-21T06:09:47.568402+0000","last_fullsized":"2024-09-21T06:09:47.568402+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:39.640902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.527681+0000","last_change":"2024-09-21T06:09:46.527681+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.527681+0000","last_undegraded":"2024-09-21T06:09:46.527681+0000","last_fullsized":"2024-09-21T06:09:46.527681+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565098+0000","last_change":"2024-09-21T06:09:47.552743+0000","last_active":"2024-09-21T06:09:47.565098+0000","last_peered":"2024-09-21T06:09:47.565098+0000","last_clean":"2024-09-21T06:09:47.565098+0000","last_became_active":"2024-09-21T06:09:47.552402+0000","last_became_peered":"2024-09-21T06:09:47.552402+0000","last_unstale":"2024-09-21T06:09:47.565098+0000","last_undegraded":"2024-09-21T06:09:47.565098+0000","last_fullsized":"2024-09-21T06:09:47.565098+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552061+0000","last_change":"2024-09-21T06:09:47.547136+0000","last_active":"2024-09-21T06:09:48.552061+0000","last_peered":"2024-09-21T06:09:48.552061+0000","last_clean":"2024-09-21T06:09:48.552061+0000","last_became_active":"2024-09-21T06:09:47.546771+0000","last_became_peered":"2024-09-21T06:09:47.546771+0000","last_unstale":"2024-09-21T06:09:48.552061+0000","last_undegraded":"2024-09-21T06:09:48.552061+0000","last_fullsized":"2024-09-21T06:09:48.552061+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:09.595527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553225+0000","last_change":"2024-09-21T06:09:47.550506+0000","last_active":"2024-09-21T06:09:48.553225+0000","last_peered":"2024-09-21T06:09:48.553225+0000","last_clean":"2024-09-21T06:09:48.553225+0000","last_became_active":"2024-09-21T06:09:47.550172+0000","last_became_peered":"2024-09-21T06:09:47.550172+0000","last_unstale":"2024-09-21T06:09:48.553225+0000","last_undegraded":"2024-09-21T06:09:48.553225+0000","last_fullsized":"2024-09-21T06:09:48.553225+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.338192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564838+0000","last_change":"2024-09-21T06:09:47.548514+0000","last_active":"2024-09-21T06:09:47.564838+0000","last_peered":"2024-09-21T06:09:47.564838+0000","last_clean":"2024-09-21T06:09:47.564838+0000","last_became_active":"2024-09-21T06:09:47.548034+0000","last_became_peered":"2024-09-21T06:09:47.548034+0000","last_unstale":"2024-09-21T06:09:47.564838+0000","last_undegraded":"2024-09-21T06:09:47.564838+0000","last_fullsized":"2024-09-21T06:09:47.564838+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:47.209532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.526758+0000","last_change":"2024-09-21T06:09:46.526758+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.526758+0000","last_undegraded":"2024-09-21T06:09:46.526758+0000","last_fullsized":"2024-09-21T06:09:46.526758+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.526087+0000","last_change":"2024-09-21T06:09:46.526087+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.526087+0000","last_undegraded":"2024-09-21T06:09:46.526087+0000","last_fullsized":"2024-09-21T06:09:46.526087+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553791+0000","last_change":"2024-09-21T06:09:46.528315+0000","last_active":"2024-09-21T06:09:48.553791+0000","last_peered":"2024-09-21T06:09:48.553791+0000","last_clean":"2024-09-21T06:09:48.553791+0000","last_became_active":"2024-09-21T06:09:46.527949+0000","last_became_peered":"2024-09-21T06:09:46.527949+0000","last_unstale":"2024-09-21T06:09:48.553791+0000","last_undegraded":"2024-09-21T06:09:48.553791+0000","last_fullsized":"2024-09-21T06:09:48.553791+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:23.941639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561877+0000","last_change":"2024-09-21T06:09:47.553802+0000","last_active":"2024-09-21T06:09:48.561877+0000","last_peered":"2024-09-21T06:09:48.561877+0000","last_clean":"2024-09-21T06:09:48.561877+0000","last_became_active":"2024-09-21T06:09:47.553503+0000","last_became_peered":"2024-09-21T06:09:47.553503+0000","last_unstale":"2024-09-21T06:09:48.561877+0000","last_undegraded":"2024-09-21T06:09:48.561877+0000","last_fullsized":"2024-09-21T06:09:48.561877+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:58.813046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569291+0000","last_change":"2024-09-21T06:09:46.531370+0000","last_active":"2024-09-21T06:09:47.569291+0000","last_peered":"2024-09-21T06:09:47.569291+0000","last_clean":"2024-09-21T06:09:47.569291+0000","last_became_active":"2024-09-21T06:09:46.530960+0000","last_became_peered":"2024-09-21T06:09:46.530960+0000","last_unstale":"2024-09-21T06:09:47.569291+0000","last_undegraded":"2024-09-21T06:09:47.569291+0000","last_fullsized":"2024-09-21T06:09:47.569291+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.528678+0000","last_change":"2024-09-21T06:09:46.528678+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.528678+0000","last_undegraded":"2024-09-21T06:09:46.528678+0000","last_fullsized":"2024-09-21T06:09:46.528678+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.577193+0000","last_change":"2024-09-21T06:09:46.544553+0000","last_active":"2024-09-21T06:09:47.577193+0000","last_peered":"2024-09-21T06:09:47.577193+0000","last_clean":"2024-09-21T06:09:47.577193+0000","last_became_active":"2024-09-21T06:09:46.544246+0000","last_became_peered":"2024-09-21T06:09:46.544246+0000","last_unstale":"2024-09-21T06:09:47.577193+0000","last_undegraded":"2024-09-21T06:09:47.577193+0000","last_fullsized":"2024-09-21T06:09:47.577193+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.718740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.570926+0000","last_change":"2024-09-21T06:09:47.569840+0000","last_active":"2024-09-21T06:09:47.570926+0000","last_peered":"2024-09-21T06:09:47.570926+0000","last_clean":"2024-09-21T06:09:47.570926+0000","last_became_active":"2024-09-21T06:09:47.569541+0000","last_became_peered":"2024-09-21T06:09:47.569541+0000","last_unstale":"2024-09-21T06:09:47.570926+0000","last_undegraded":"2024-09-21T06:09:47.570926+0000","last_fullsized":"2024-09-21T06:09:47.570926+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.577763+0000","last_change":"2024-09-21T06:09:46.543348+0000","last_active":"2024-09-21T06:09:47.577763+0000","last_peered":"2024-09-21T06:09:47.577763+0000","last_clean":"2024-09-21T06:09:47.577763+0000","last_became_active":"2024-09-21T06:09:46.543036+0000","last_became_peered":"2024-09-21T06:09:46.543036+0000","last_unstale":"2024-09-21T06:09:47.577763+0000","last_undegraded":"2024-09-21T06:09:47.577763+0000","last_fullsized":"2024-09-21T06:09:47.577763+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:17.178909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568248+0000","last_change":"2024-09-21T06:09:47.558350+0000","last_active":"2024-09-21T06:09:47.568248+0000","last_peered":"2024-09-21T06:09:47.568248+0000","last_clean":"2024-09-21T06:09:47.568248+0000","last_became_active":"2024-09-21T06:09:47.552686+0000","last_became_peered":"2024-09-21T06:09:47.552686+0000","last_unstale":"2024-09-21T06:09:47.568248+0000","last_undegraded":"2024-09-21T06:09:47.568248+0000","last_fullsized":"2024-09-21T06:09:47.568248+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:05.811070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553484+0000","last_change":"2024-09-21T06:09:46.535845+0000","last_active":"2024-09-21T06:09:48.553484+0000","last_peered":"2024-09-21T06:09:48.553484+0000","last_clean":"2024-09-21T06:09:48.553484+0000","last_became_active":"2024-09-21T06:09:46.535462+0000","last_became_peered":"2024-09-21T06:09:46.535462+0000","last_unstale":"2024-09-21T06:09:48.553484+0000","last_undegraded":"2024-09-21T06:09:48.553484+0000","last_fullsized":"2024-09-21T06:09:48.553484+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:32.585856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553096+0000","last_change":"2024-09-21T06:09:47.556065+0000","last_active":"2024-09-21T06:09:48.553096+0000","last_peered":"2024-09-21T06:09:48.553096+0000","last_clean":"2024-09-21T06:09:48.553096+0000","last_became_active":"2024-09-21T06:09:47.555742+0000","last_became_peered":"2024-09-21T06:09:47.555742+0000","last_unstale":"2024-09-21T06:09:48.553096+0000","last_undegraded":"2024-09-21T06:09:48.553096+0000","last_fullsized":"2024-09-21T06:09:48.553096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.575260+0000","last_change":"2024-09-21T06:09:46.549487+0000","last_active":"2024-09-21T06:09:47.575260+0000","last_peered":"2024-09-21T06:09:47.575260+0000","last_clean":"2024-09-21T06:09:47.575260+0000","last_became_active":"2024-09-21T06:09:46.548860+0000","last_became_peered":"2024-09-21T06:09:46.548860+0000","last_unstale":"2024-09-21T06:09:47.575260+0000","last_undegraded":"2024-09-21T06:09:47.575260+0000","last_fullsized":"2024-09-21T06:09:47.575260+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:45.648740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569391+0000","last_change":"2024-09-21T06:09:47.563169+0000","last_active":"2024-09-21T06:09:47.569391+0000","last_peered":"2024-09-21T06:09:47.569391+0000","last_clean":"2024-09-21T06:09:47.569391+0000","last_became_active":"2024-09-21T06:09:47.562868+0000","last_became_peered":"2024-09-21T06:09:47.562868+0000","last_unstale":"2024-09-21T06:09:47.569391+0000","last_undegraded":"2024-09-21T06:09:47.569391+0000","last_fullsized":"2024-09-21T06:09:47.569391+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:51.138002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551911+0000","last_change":"2024-09-21T06:09:46.545661+0000","last_active":"2024-09-21T06:09:48.551911+0000","last_peered":"2024-09-21T06:09:48.551911+0000","last_clean":"2024-09-21T06:09:48.551911+0000","last_became_active":"2024-09-21T06:09:46.545340+0000","last_became_peered":"2024-09-21T06:09:46.545340+0000","last_unstale":"2024-09-21T06:09:48.551911+0000","last_undegraded":"2024-09-21T06:09:48.551911+0000","last_fullsized":"2024-09-21T06:09:48.551911+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:47.880211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552001+0000","last_change":"2024-09-21T06:09:47.553054+0000","last_active":"2024-09-21T06:09:48.552001+0000","last_peered":"2024-09-21T06:09:48.552001+0000","last_clean":"2024-09-21T06:09:48.552001+0000","last_became_active":"2024-09-21T06:09:47.552742+0000","last_became_peered":"2024-09-21T06:09:47.552742+0000","last_unstale":"2024-09-21T06:09:48.552001+0000","last_undegraded":"2024-09-21T06:09:48.552001+0000","last_fullsized":"2024-09-21T06:09:48.552001+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:54.333587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.542901+0000","last_change":"2024-09-21T06:09:46.536718+0000","last_active":"2024-09-21T06:09:47.542901+0000","last_peered":"2024-09-21T06:09:47.542901+0000","last_clean":"2024-09-21T06:09:47.542901+0000","last_became_active":"2024-09-21T06:09:46.535034+0000","last_became_peered":"2024-09-21T06:09:46.535034+0000","last_unstale":"2024-09-21T06:09:47.542901+0000","last_undegraded":"2024-09-21T06:09:47.542901+0000","last_fullsized":"2024-09-21T06:09:47.542901+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:49.688976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553944+0000","last_change":"2024-09-21T06:09:47.539900+0000","last_active":"2024-09-21T06:09:48.553944+0000","last_peered":"2024-09-21T06:09:48.553944+0000","last_clean":"2024-09-21T06:09:48.553944+0000","last_became_active":"2024-09-21T06:09:47.539412+0000","last_became_peered":"2024-09-21T06:09:47.539412+0000","last_unstale":"2024-09-21T06:09:48.553944+0000","last_undegraded":"2024-09-21T06:09:48.553944+0000","last_fullsized":"2024-09-21T06:09:48.553944+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:52.206476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.532722+0000","last_change":"2024-09-21T06:09:46.540269+0000","last_active":"2024-09-21T06:09:47.532722+0000","last_peered":"2024-09-21T06:09:47.532722+0000","last_clean":"2024-09-21T06:09:47.532722+0000","last_became_active":"2024-09-21T06:09:46.539954+0000","last_became_peered":"2024-09-21T06:09:46.539954+0000","last_unstale":"2024-09-21T06:09:47.532722+0000","last_undegraded":"2024-09-21T06:09:47.532722+0000","last_fullsized":"2024-09-21T06:09:47.532722+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:13.428634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.562059+0000","last_change":"2024-09-21T06:09:47.566984+0000","last_active":"2024-09-21T06:09:48.562059+0000","last_peered":"2024-09-21T06:09:48.562059+0000","last_clean":"2024-09-21T06:09:48.562059+0000","last_became_active":"2024-09-21T06:09:47.566193+0000","last_became_peered":"2024-09-21T06:09:47.566193+0000","last_unstale":"2024-09-21T06:09:48.562059+0000","last_undegraded":"2024-09-21T06:09:48.562059+0000","last_fullsized":"2024-09-21T06:09:48.562059+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:28.501310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.535666+0000","last_change":"2024-09-21T06:09:46.536342+0000","last_active":"2024-09-21T06:09:47.535666+0000","last_peered":"2024-09-21T06:09:47.535666+0000","last_clean":"2024-09-21T06:09:47.535666+0000","last_became_active":"2024-09-21T06:09:46.536012+0000","last_became_peered":"2024-09-21T06:09:46.536012+0000","last_unstale":"2024-09-21T06:09:47.535666+0000","last_undegraded":"2024-09-21T06:09:47.535666+0000","last_fullsized":"2024-09-21T06:09:47.535666+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:57.383442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.552610+0000","last_change":"2024-09-21T06:09:47.543982+0000","last_active":"2024-09-21T06:09:47.552610+0000","last_peered":"2024-09-21T06:09:47.552610+0000","last_clean":"2024-09-21T06:09:47.552610+0000","last_became_active":"2024-09-21T06:09:47.543662+0000","last_became_peered":"2024-09-21T06:09:47.543662+0000","last_unstale":"2024-09-21T06:09:47.552610+0000","last_undegraded":"2024-09-21T06:09:47.552610+0000","last_fullsized":"2024-09-21T06:09:47.552610+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.580229+0000","last_change":"2024-09-21T06:09:46.548929+0000","last_active":"2024-09-21T06:09:47.580229+0000","last_peered":"2024-09-21T06:09:47.580229+0000","last_clean":"2024-09-21T06:09:47.580229+0000","last_became_active":"2024-09-21T06:09:46.548521+0000","last_became_peered":"2024-09-21T06:09:46.548521+0000","last_unstale":"2024-09-21T06:09:47.580229+0000","last_undegraded":"2024-09-21T06:09:47.580229+0000","last_fullsized":"2024-09-21T06:09:47.580229+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:27.494160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553855+0000","last_change":"2024-09-21T06:09:47.554124+0000","last_active":"2024-09-21T06:09:48.553855+0000","last_peered":"2024-09-21T06:09:48.553855+0000","last_clean":"2024-09-21T06:09:48.553855+0000","last_became_active":"2024-09-21T06:09:47.551484+0000","last_became_peered":"2024-09-21T06:09:47.551484+0000","last_unstale":"2024-09-21T06:09:48.553855+0000","last_undegraded":"2024-09-21T06:09:48.553855+0000","last_fullsized":"2024-09-21T06:09:48.553855+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:47.621204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":19,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.582014+0000","last_change":"2024-09-21T06:09:46.539077+0000","last_active":"2024-09-21T06:09:47.582014+0000","last_peered":"2024-09-21T06:09:47.582014+0000","last_clean":"2024-09-21T06:09:47.582014+0000","last_became_active":"2024-09-21T06:09:46.537287+0000","last_became_peered":"2024-09-21T06:09:46.537287+0000","last_unstale":"2024-09-21T06:09:47.582014+0000","last_undegraded":"2024-09-21T06:09:47.582014+0000","last_fullsized":"2024-09-21T06:09:47.582014+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:52.872003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.554303+0000","last_change":"2024-09-21T06:09:47.546084+0000","last_active":"2024-09-21T06:09:47.554303+0000","last_peered":"2024-09-21T06:09:47.554303+0000","last_clean":"2024-09-21T06:09:47.554303+0000","last_became_active":"2024-09-21T06:09:47.545677+0000","last_became_peered":"2024-09-21T06:09:47.545677+0000","last_unstale":"2024-09-21T06:09:47.554303+0000","last_undegraded":"2024-09-21T06:09:47.554303+0000","last_fullsized":"2024-09-21T06:09:47.554303+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T06:09:46.543756+0000","last_change":"2024-09-21T06:09:46.537690+0000","last_active":"2024-09-21T06:09:46.543756+0000","last_peered":"2024-09-21T06:09:46.543756+0000","last_clean":"2024-09-21T06:09:46.543756+0000","last_became_active":"2024-09-21T06:09:46.537157+0000","last_became_peered":"2024-09-21T06:09:46.537157+0000","last_unstale":"2024-09-21T06:09:46.543756+0000","last_undegraded":"2024-09-21T06:09:46.543756+0000","last_fullsized":"2024-09-21T06:09:46.543756+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:02.269032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564968+0000","last_change":"2024-09-21T06:09:47.550402+0000","last_active":"2024-09-21T06:09:47.564968+0000","last_peered":"2024-09-21T06:09:47.564968+0000","last_clean":"2024-09-21T06:09:47.564968+0000","last_became_active":"2024-09-21T06:09:47.550085+0000","last_became_peered":"2024-09-21T06:09:47.550085+0000","last_unstale":"2024-09-21T06:09:47.564968+0000","last_undegraded":"2024-09-21T06:09:47.564968+0000","last_fullsized":"2024-09-21T06:09:47.564968+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561520+0000","last_change":"2024-09-21T06:09:46.524911+0000","last_active":"2024-09-21T06:09:48.561520+0000","last_peered":"2024-09-21T06:09:48.561520+0000","last_clean":"2024-09-21T06:09:48.561520+0000","last_became_active":"2024-09-21T06:09:46.524561+0000","last_became_peered":"2024-09-21T06:09:46.524561+0000","last_unstale":"2024-09-21T06:09:48.561520+0000","last_undegraded":"2024-09-21T06:09:48.561520+0000","last_fullsized":"2024-09-21T06:09:48.561520+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:32.557867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.547211+0000","last_change":"2024-09-21T06:09:47.541745+0000","last_active":"2024-09-21T06:09:47.547211+0000","last_peered":"2024-09-21T06:09:47.547211+0000","last_clean":"2024-09-21T06:09:47.547211+0000","last_became_active":"2024-09-21T06:09:47.541184+0000","last_became_peered":"2024-09-21T06:09:47.541184+0000","last_unstale":"2024-09-21T06:09:47.547211+0000","last_undegraded":"2024-09-21T06:09:47.547211+0000","last_fullsized":"2024-09-21T06:09:47.547211+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:49.440767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":21,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.579446+0000","last_change":"2024-09-21T06:09:46.533914+0000","last_active":"2024-09-21T06:09:47.579446+0000","last_peered":"2024-09-21T06:09:47.579446+0000","last_clean":"2024-09-21T06:09:47.579446+0000","last_became_active":"2024-09-21T06:09:46.533541+0000","last_became_peered":"2024-09-21T06:09:46.533541+0000","last_unstale":"2024-09-21T06:09:47.579446+0000","last_undegraded":"2024-09-21T06:09:47.579446+0000","last_fullsized":"2024-09-21T06:09:47.579446+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:24.340748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569557+0000","last_change":"2024-09-21T06:09:47.557155+0000","last_active":"2024-09-21T06:09:47.569557+0000","last_peered":"2024-09-21T06:09:47.569557+0000","last_clean":"2024-09-21T06:09:47.569557+0000","last_became_active":"2024-09-21T06:09:47.556834+0000","last_became_peered":"2024-09-21T06:09:47.556834+0000","last_unstale":"2024-09-21T06:09:47.569557+0000","last_undegraded":"2024-09-21T06:09:47.569557+0000","last_fullsized":"2024-09-21T06:09:47.569557+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:21.313342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.575298+0000","last_change":"2024-09-21T06:09:46.546891+0000","last_active":"2024-09-21T06:09:47.575298+0000","last_peered":"2024-09-21T06:09:47.575298+0000","last_clean":"2024-09-21T06:09:47.575298+0000","last_became_active":"2024-09-21T06:09:46.546573+0000","last_became_peered":"2024-09-21T06:09:46.546573+0000","last_unstale":"2024-09-21T06:09:47.575298+0000","last_undegraded":"2024-09-21T06:09:47.575298+0000","last_fullsized":"2024-09-21T06:09:47.575298+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:17.766821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-21T06:09:46.524609+0000","last_change":"2024-09-21T06:09:46.524609+0000","last_active":"2024-09-21T06:09:46.505939+0000","last_peered":"2024-09-21T06:09:46.505939+0000","last_clean":"2024-09-21T06:09:46.505939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T06:09:46.524609+0000","last_undegraded":"2024-09-21T06:09:46.524609+0000","last_fullsized":"2024-09-21T06:09:46.524609+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.575222+0000","last_change":"2024-09-21T06:09:46.529780+0000","last_active":"2024-09-21T06:09:47.575222+0000","last_peered":"2024-09-21T06:09:47.575222+0000","last_clean":"2024-09-21T06:09:47.575222+0000","last_became_active":"2024-09-21T06:09:46.528136+0000","last_became_peered":"2024-09-21T06:09:46.528136+0000","last_unstale":"2024-09-21T06:09:47.575222+0000","last_undegraded":"2024-09-21T06:09:47.575222+0000","last_fullsized":"2024-09-21T06:09:47.575222+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:17.381392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568420+0000","last_change":"2024-09-21T06:09:47.567084+0000","last_active":"2024-09-21T06:09:47.568420+0000","last_peered":"2024-09-21T06:09:47.568420+0000","last_clean":"2024-09-21T06:09:47.568420+0000","last_became_active":"2024-09-21T06:09:47.566717+0000","last_became_peered":"2024-09-21T06:09:47.566717+0000","last_unstale":"2024-09-21T06:09:47.568420+0000","last_undegraded":"2024-09-21T06:09:47.568420+0000","last_fullsized":"2024-09-21T06:09:47.568420+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561699+0000","last_change":"2024-09-21T06:09:46.524059+0000","last_active":"2024-09-21T06:09:48.561699+0000","last_peered":"2024-09-21T06:09:48.561699+0000","last_clean":"2024-09-21T06:09:48.561699+0000","last_became_active":"2024-09-21T06:09:46.523459+0000","last_became_peered":"2024-09-21T06:09:46.523459+0000","last_unstale":"2024-09-21T06:09:48.561699+0000","last_undegraded":"2024-09-21T06:09:48.561699+0000","last_fullsized":"2024-09-21T06:09:48.561699+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.562759+0000","last_change":"2024-09-21T06:09:47.561252+0000","last_active":"2024-09-21T06:09:47.562759+0000","last_peered":"2024-09-21T06:09:47.562759+0000","last_clean":"2024-09-21T06:09:47.562759+0000","last_became_active":"2024-09-21T06:09:47.560849+0000","last_became_peered":"2024-09-21T06:09:47.560849+0000","last_unstale":"2024-09-21T06:09:47.562759+0000","last_undegraded":"2024-09-21T06:09:47.562759+0000","last_fullsized":"2024-09-21T06:09:47.562759+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553549+0000","last_change":"2024-09-21T06:09:46.534239+0000","last_active":"2024-09-21T06:09:48.553549+0000","last_peered":"2024-09-21T06:09:48.553549+0000","last_clean":"2024-09-21T06:09:48.553549+0000","last_became_active":"2024-09-21T06:09:46.533715+0000","last_became_peered":"2024-09-21T06:09:46.533715+0000","last_unstale":"2024-09-21T06:09:48.553549+0000","last_undegraded":"2024-09-21T06:09:48.553549+0000","last_fullsized":"2024-09-21T06:09:48.553549+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:08:04.897143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.559568+0000","last_change":"2024-09-21T06:09:47.552337+0000","last_active":"2024-09-21T06:09:47.559568+0000","last_peered":"2024-09-21T06:09:47.559568+0000","last_clean":"2024-09-21T06:09:47.559568+0000","last_became_active":"2024-09-21T06:09:47.552009+0000","last_became_peered":"2024-09-21T06:09:47.552009+0000","last_unstale":"2024-09-21T06:09:47.559568+0000","last_undegraded":"2024-09-21T06:09:47.559568+0000","last_fullsized":"2024-09-21T06:09:47.559568+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:46.655889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T06:09:46.539682+0000","last_change":"2024-09-21T06:09:46.536200+0000","last_active":"2024-09-21T06:09:46.539682+0000","last_peered":"2024-09-21T06:09:46.539682+0000","last_clean":"2024-09-21T06:09:46.539682+0000","last_became_active":"2024-09-21T06:09:46.535894+0000","last_became_peered":"2024-09-21T06:09:46.535894+0000","last_unstale":"2024-09-21T06:09:46.539682+0000","last_undegraded":"2024-09-21T06:09:46.539682+0000","last_fullsized":"2024-09-21T06:09:46.539682+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:46.409661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561750+0000","last_change":"2024-09-21T06:09:47.552189+0000","last_active":"2024-09-21T06:09:47.561750+0000","last_peered":"2024-09-21T06:09:47.561750+0000","last_clean":"2024-09-21T06:09:47.561750+0000","last_became_active":"2024-09-21T06:09:47.548927+0000","last_became_peered":"2024-09-21T06:09:47.548927+0000","last_unstale":"2024-09-21T06:09:47.561750+0000","last_undegraded":"2024-09-21T06:09:47.561750+0000","last_fullsized":"2024-09-21T06:09:47.561750+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:33.335946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.532137+0000","last_change":"2024-09-21T06:09:46.526945+0000","last_active":"2024-09-21T06:09:47.532137+0000","last_peered":"2024-09-21T06:09:47.532137+0000","last_clean":"2024-09-21T06:09:47.532137+0000","last_became_active":"2024-09-21T06:09:46.526588+0000","last_became_peered":"2024-09-21T06:09:46.526588+0000","last_unstale":"2024-09-21T06:09:47.532137+0000","last_undegraded":"2024-09-21T06:09:47.532137+0000","last_fullsized":"2024-09-21T06:09:47.532137+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:14.014304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561635+0000","last_change":"2024-09-21T06:09:47.555659+0000","last_active":"2024-09-21T06:09:47.561635+0000","last_peered":"2024-09-21T06:09:47.561635+0000","last_clean":"2024-09-21T06:09:47.561635+0000","last_became_active":"2024-09-21T06:09:47.555358+0000","last_became_peered":"2024-09-21T06:09:47.555358+0000","last_unstale":"2024-09-21T06:09:47.561635+0000","last_undegraded":"2024-09-21T06:09:47.561635+0000","last_fullsized":"2024-09-21T06:09:47.561635+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T06:09:46.544242+0000","last_change":"2024-09-21T06:09:46.541688+0000","last_active":"2024-09-21T06:09:46.544242+0000","last_peered":"2024-09-21T06:09:46.544242+0000","last_clean":"2024-09-21T06:09:46.544242+0000","last_became_active":"2024-09-21T06:09:46.541322+0000","last_became_peered":"2024-09-21T06:09:46.541322+0000","last_unstale":"2024-09-21T06:09:46.544242+0000","last_undegraded":"2024-09-21T06:09:46.544242+0000","last_fullsized":"2024-09-21T06:09:46.544242+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:39.745209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565078+0000","last_change":"2024-09-21T06:09:47.551665+0000","last_active":"2024-09-21T06:09:47.565078+0000","last_peered":"2024-09-21T06:09:47.565078+0000","last_clean":"2024-09-21T06:09:47.565078+0000","last_became_active":"2024-09-21T06:09:47.551313+0000","last_became_peered":"2024-09-21T06:09:47.551313+0000","last_unstale":"2024-09-21T06:09:47.565078+0000","last_undegraded":"2024-09-21T06:09:47.565078+0000","last_fullsized":"2024-09-21T06:09:47.565078+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.543137+0000","last_change":"2024-09-21T06:09:46.545814+0000","last_active":"2024-09-21T06:09:47.543137+0000","last_peered":"2024-09-21T06:09:47.543137+0000","last_clean":"2024-09-21T06:09:47.543137+0000","last_became_active":"2024-09-21T06:09:46.545413+0000","last_became_peered":"2024-09-21T06:09:46.545413+0000","last_unstale":"2024-09-21T06:09:47.543137+0000","last_undegraded":"2024-09-21T06:09:47.543137+0000","last_fullsized":"2024-09-21T06:09:47.543137+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:13.029274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561834+0000","last_change":"2024-09-21T06:09:47.566967+0000","last_active":"2024-09-21T06:09:48.561834+0000","last_peered":"2024-09-21T06:09:48.561834+0000","last_clean":"2024-09-21T06:09:48.561834+0000","last_became_active":"2024-09-21T06:09:47.566211+0000","last_became_peered":"2024-09-21T06:09:47.566211+0000","last_unstale":"2024-09-21T06:09:48.561834+0000","last_undegraded":"2024-09-21T06:09:48.561834+0000","last_fullsized":"2024-09-21T06:09:48.561834+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:14.036424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.580510+0000","last_change":"2024-09-21T06:09:46.528896+0000","last_active":"2024-09-21T06:09:47.580510+0000","last_peered":"2024-09-21T06:09:47.580510+0000","last_clean":"2024-09-21T06:09:47.580510+0000","last_became_active":"2024-09-21T06:09:46.528590+0000","last_became_peered":"2024-09-21T06:09:46.528590+0000","last_unstale":"2024-09-21T06:09:47.580510+0000","last_undegraded":"2024-09-21T06:09:47.580510+0000","last_fullsized":"2024-09-21T06:09:47.580510+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564780+0000","last_change":"2024-09-21T06:09:47.556097+0000","last_active":"2024-09-21T06:09:47.564780+0000","last_peered":"2024-09-21T06:09:47.564780+0000","last_clean":"2024-09-21T06:09:47.564780+0000","last_became_active":"2024-09-21T06:09:47.555720+0000","last_became_peered":"2024-09-21T06:09:47.555720+0000","last_unstale":"2024-09-21T06:09:47.564780+0000","last_undegraded":"2024-09-21T06:09:47.564780+0000","last_fullsized":"2024-09-21T06:09:47.564780+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:14.289349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.563317+0000","last_change":"2024-09-21T06:09:46.525404+0000","last_active":"2024-09-21T06:09:48.563317+0000","last_peered":"2024-09-21T06:09:48.563317+0000","last_clean":"2024-09-21T06:09:48.563317+0000","last_became_active":"2024-09-21T06:09:46.525087+0000","last_became_peered":"2024-09-21T06:09:46.525087+0000","last_unstale":"2024-09-21T06:09:48.563317+0000","last_undegraded":"2024-09-21T06:09:48.563317+0000","last_fullsized":"2024-09-21T06:09:48.563317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:57.383442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.572681+0000","last_change":"2024-09-21T06:09:47.565828+0000","last_active":"2024-09-21T06:09:47.572681+0000","last_peered":"2024-09-21T06:09:47.572681+0000","last_clean":"2024-09-21T06:09:47.572681+0000","last_became_active":"2024-09-21T06:09:47.565467+0000","last_became_peered":"2024-09-21T06:09:47.565467+0000","last_unstale":"2024-09-21T06:09:47.572681+0000","last_undegraded":"2024-09-21T06:09:47.572681+0000","last_fullsized":"2024-09-21T06:09:47.572681+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:37.650305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.544753+0000","last_change":"2024-09-21T06:09:46.542691+0000","last_active":"2024-09-21T06:09:47.544753+0000","last_peered":"2024-09-21T06:09:47.544753+0000","last_clean":"2024-09-21T06:09:47.544753+0000","last_became_active":"2024-09-21T06:09:46.542293+0000","last_became_peered":"2024-09-21T06:09:46.542293+0000","last_unstale":"2024-09-21T06:09:47.544753+0000","last_undegraded":"2024-09-21T06:09:47.544753+0000","last_fullsized":"2024-09-21T06:09:47.544753+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:33.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561749+0000","last_change":"2024-09-21T06:09:47.576750+0000","last_active":"2024-09-21T06:09:48.561749+0000","last_peered":"2024-09-21T06:09:48.561749+0000","last_clean":"2024-09-21T06:09:48.561749+0000","last_became_active":"2024-09-21T06:09:47.576443+0000","last_became_peered":"2024-09-21T06:09:47.576443+0000","last_unstale":"2024-09-21T06:09:48.561749+0000","last_undegraded":"2024-09-21T06:09:48.561749+0000","last_fullsized":"2024-09-21T06:09:48.561749+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.542616+0000","last_change":"2024-09-21T06:09:46.547408+0000","last_active":"2024-09-21T06:09:47.542616+0000","last_peered":"2024-09-21T06:09:47.542616+0000","last_clean":"2024-09-21T06:09:47.542616+0000","last_became_active":"2024-09-21T06:09:46.547043+0000","last_became_peered":"2024-09-21T06:09:46.547043+0000","last_unstale":"2024-09-21T06:09:47.542616+0000","last_undegraded":"2024-09-21T06:09:47.542616+0000","last_fullsized":"2024-09-21T06:09:47.542616+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:08.588377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552144+0000","last_change":"2024-09-21T06:09:47.540415+0000","last_active":"2024-09-21T06:09:48.552144+0000","last_peered":"2024-09-21T06:09:48.552144+0000","last_clean":"2024-09-21T06:09:48.552144+0000","last_became_active":"2024-09-21T06:09:47.539926+0000","last_became_peered":"2024-09-21T06:09:47.539926+0000","last_unstale":"2024-09-21T06:09:48.552144+0000","last_undegraded":"2024-09-21T06:09:48.552144+0000","last_fullsized":"2024-09-21T06:09:48.552144+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553533+0000","last_change":"2024-09-21T06:09:46.529617+0000","last_active":"2024-09-21T06:09:48.553533+0000","last_peered":"2024-09-21T06:09:48.553533+0000","last_clean":"2024-09-21T06:09:48.553533+0000","last_became_active":"2024-09-21T06:09:46.528234+0000","last_became_peered":"2024-09-21T06:09:46.528234+0000","last_unstale":"2024-09-21T06:09:48.553533+0000","last_undegraded":"2024-09-21T06:09:48.553533+0000","last_fullsized":"2024-09-21T06:09:48.553533+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:19.814656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552277+0000","last_change":"2024-09-21T06:09:47.543597+0000","last_active":"2024-09-21T06:09:48.552277+0000","last_peered":"2024-09-21T06:09:48.552277+0000","last_clean":"2024-09-21T06:09:48.552277+0000","last_became_active":"2024-09-21T06:09:47.543282+0000","last_became_peered":"2024-09-21T06:09:47.543282+0000","last_unstale":"2024-09-21T06:09:48.552277+0000","last_undegraded":"2024-09-21T06:09:48.552277+0000","last_fullsized":"2024-09-21T06:09:48.552277+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552294+0000","last_change":"2024-09-21T06:09:46.535746+0000","last_active":"2024-09-21T06:09:48.552294+0000","last_peered":"2024-09-21T06:09:48.552294+0000","last_clean":"2024-09-21T06:09:48.552294+0000","last_became_active":"2024-09-21T06:09:46.535142+0000","last_became_peered":"2024-09-21T06:09:46.535142+0000","last_unstale":"2024-09-21T06:09:48.552294+0000","last_undegraded":"2024-09-21T06:09:48.552294+0000","last_fullsized":"2024-09-21T06:09:48.552294+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:56.819951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561609+0000","last_change":"2024-09-21T06:09:47.553873+0000","last_active":"2024-09-21T06:09:47.561609+0000","last_peered":"2024-09-21T06:09:47.561609+0000","last_clean":"2024-09-21T06:09:47.561609+0000","last_became_active":"2024-09-21T06:09:47.553515+0000","last_became_peered":"2024-09-21T06:09:47.553515+0000","last_unstale":"2024-09-21T06:09:47.561609+0000","last_undegraded":"2024-09-21T06:09:47.561609+0000","last_fullsized":"2024-09-21T06:09:47.561609+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:49.440767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.532370+0000","last_change":"2024-09-21T06:09:46.539038+0000","last_active":"2024-09-21T06:09:47.532370+0000","last_peered":"2024-09-21T06:09:47.532370+0000","last_clean":"2024-09-21T06:09:47.532370+0000","last_became_active":"2024-09-21T06:09:46.538692+0000","last_became_peered":"2024-09-21T06:09:46.538692+0000","last_unstale":"2024-09-21T06:09:47.532370+0000","last_undegraded":"2024-09-21T06:09:47.532370+0000","last_fullsized":"2024-09-21T06:09:47.532370+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:44.565196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569536+0000","last_change":"2024-09-21T06:09:47.564522+0000","last_active":"2024-09-21T06:09:47.569536+0000","last_peered":"2024-09-21T06:09:47.569536+0000","last_clean":"2024-09-21T06:09:47.569536+0000","last_became_active":"2024-09-21T06:09:47.564206+0000","last_became_peered":"2024-09-21T06:09:47.564206+0000","last_unstale":"2024-09-21T06:09:47.569536+0000","last_undegraded":"2024-09-21T06:09:47.569536+0000","last_fullsized":"2024-09-21T06:09:47.569536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:03.505698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.543377+0000","last_change":"2024-09-21T06:09:46.532492+0000","last_active":"2024-09-21T06:09:47.543377+0000","last_peered":"2024-09-21T06:09:47.543377+0000","last_clean":"2024-09-21T06:09:47.543377+0000","last_became_active":"2024-09-21T06:09:46.530409+0000","last_became_peered":"2024-09-21T06:09:46.530409+0000","last_unstale":"2024-09-21T06:09:47.543377+0000","last_undegraded":"2024-09-21T06:09:47.543377+0000","last_fullsized":"2024-09-21T06:09:47.543377+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:57.805896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.549238+0000","last_change":"2024-09-21T06:09:47.543465+0000","last_active":"2024-09-21T06:09:47.549238+0000","last_peered":"2024-09-21T06:09:47.549238+0000","last_clean":"2024-09-21T06:09:47.549238+0000","last_became_active":"2024-09-21T06:09:47.543121+0000","last_became_peered":"2024-09-21T06:09:47.543121+0000","last_unstale":"2024-09-21T06:09:47.549238+0000","last_undegraded":"2024-09-21T06:09:47.549238+0000","last_fullsized":"2024-09-21T06:09:47.549238+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552395+0000","last_change":"2024-09-21T06:09:46.534147+0000","last_active":"2024-09-21T06:09:48.552395+0000","last_peered":"2024-09-21T06:09:48.552395+0000","last_clean":"2024-09-21T06:09:48.552395+0000","last_became_active":"2024-09-21T06:09:46.533758+0000","last_became_peered":"2024-09-21T06:09:46.533758+0000","last_unstale":"2024-09-21T06:09:48.552395+0000","last_undegraded":"2024-09-21T06:09:48.552395+0000","last_fullsized":"2024-09-21T06:09:48.552395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:29.394474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":277,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.543464+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:09:47.543464+0000","last_peered":"2024-09-21T06:09:47.543464+0000","last_clean":"2024-09-21T06:09:47.543464+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:09:47.543464+0000","last_undegraded":"2024-09-21T06:09:47.543464+0000","last_fullsized":"2024-09-21T06:09:47.543464+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509605,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127456,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244384,"statfs":{"total":96636764160,"available":96506249216,"internally_reserved":0,"allocated":1974272,"data_stored":1186242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509605,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118560,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253280,"statfs":{"total":96636764160,"available":96515358720,"internally_reserved":0,"allocated":1388544,"data_stored":601712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509604,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82456,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289384,"statfs":{"total":96636764160,"available":96552329216,"internally_reserved":0,"allocated":1380352,"data_stored":595862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509605,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144920,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226920,"statfs":{"total":96636764160,"available":96488366080,"internally_reserved":0,"allocated":1372160,"data_stored":595258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509604,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135208,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236632,"statfs":{"total":96636764160,"available":96498311168,"internally_reserved":0,"allocated":1388544,"data_stored":596940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509606,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140760,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231080,"statfs":{"total":96636764160,"available":96492625920,"internally_reserved":0,"allocated":1372160,"data_stored":595258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509604,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131596,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240244,"statfs":{"total":96636764160,"available":96502009856,"internally_reserved":0,"allocated":1953792,"data_stored":1178767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542309,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144908,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226932,"statfs":{"total":96636764160,"available":96488378368,"internally_reserved":0,"allocated":1359872,"data_stored":588399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:09:52.594 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3a is not active+clean 2024-09-21T06:09:52.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:09:46.523983+0000', 'last_change': '2024-09-21T06:09:46.523983+0000', 'last_active': '2024-09-21T06:09:46.505939+0000', 'last_peered': '2024-09-21T06:09:46.505939+0000', 'last_clean': '2024-09-21T06:09:46.505939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:09:46.523983+0000', 'last_undegraded': '2024-09-21T06:09:46.523983+0000', 'last_fullsized': '2024-09-21T06:09:46.523983+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_clean_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T06:09:52.594 INFO:tasks.ceph.ceph_manager.ceph:PG 17.31 is not active+clean 2024-09-21T06:09:52.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:09:46.523903+0000', 'last_change': '2024-09-21T06:09:46.523903+0000', 'last_active': '2024-09-21T06:09:46.505939+0000', 'last_peered': '2024-09-21T06:09:46.505939+0000', 'last_clean': '2024-09-21T06:09:46.505939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:09:46.523903+0000', 'last_undegraded': '2024-09-21T06:09:46.523903+0000', 'last_fullsized': '2024-09-21T06:09:46.523903+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_clean_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T06:09:52.595 INFO:tasks.ceph.ceph_manager.ceph:PG 17.25 is not active+clean 2024-09-21T06:09:52.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:09:46.527681+0000', 'last_change': '2024-09-21T06:09:46.527681+0000', 'last_active': '2024-09-21T06:09:46.505939+0000', 'last_peered': '2024-09-21T06:09:46.505939+0000', 'last_clean': '2024-09-21T06:09:46.505939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:09:46.527681+0000', 'last_undegraded': '2024-09-21T06:09:46.527681+0000', 'last_fullsized': '2024-09-21T06:09:46.527681+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_clean_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T06:09:52.595 INFO:tasks.ceph.ceph_manager.ceph:PG 17.20 is not active+clean 2024-09-21T06:09:52.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:09:46.526758+0000', 'last_change': '2024-09-21T06:09:46.526758+0000', 'last_active': '2024-09-21T06:09:46.505939+0000', 'last_peered': '2024-09-21T06:09:46.505939+0000', 'last_clean': '2024-09-21T06:09:46.505939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:09:46.526758+0000', 'last_undegraded': '2024-09-21T06:09:46.526758+0000', 'last_fullsized': '2024-09-21T06:09:46.526758+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_clean_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T06:09:52.595 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1e is not active+clean 2024-09-21T06:09:52.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:09:46.526087+0000', 'last_change': '2024-09-21T06:09:46.526087+0000', 'last_active': '2024-09-21T06:09:46.505939+0000', 'last_peered': '2024-09-21T06:09:46.505939+0000', 'last_clean': '2024-09-21T06:09:46.505939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:09:46.526087+0000', 'last_undegraded': '2024-09-21T06:09:46.526087+0000', 'last_fullsized': '2024-09-21T06:09:46.526087+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_clean_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T06:09:52.595 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1c is not active+clean 2024-09-21T06:09:52.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:09:46.528678+0000', 'last_change': '2024-09-21T06:09:46.528678+0000', 'last_active': '2024-09-21T06:09:46.505939+0000', 'last_peered': '2024-09-21T06:09:46.505939+0000', 'last_clean': '2024-09-21T06:09:46.505939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:09:46.528678+0000', 'last_undegraded': '2024-09-21T06:09:46.528678+0000', 'last_fullsized': '2024-09-21T06:09:46.528678+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_clean_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T06:09:52.595 INFO:tasks.ceph.ceph_manager.ceph:PG 17.f is not active+clean 2024-09-21T06:09:52.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-21T06:09:46.524609+0000', 'last_change': '2024-09-21T06:09:46.524609+0000', 'last_active': '2024-09-21T06:09:46.505939+0000', 'last_peered': '2024-09-21T06:09:46.505939+0000', 'last_clean': '2024-09-21T06:09:46.505939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T06:09:46.524609+0000', 'last_undegraded': '2024-09-21T06:09:46.524609+0000', 'last_fullsized': '2024-09-21T06:09:46.524609+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'last_clean_scrub_stamp': '2024-09-21T06:09:46.505939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T06:09:52.596 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T06:09:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.789+0000 7f62a5600640 1 -- 172.21.15.50:0/3338088369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a0160060 msgr2=0x7f62a0074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.789+0000 7f62a5600640 1 --2- 172.21.15.50:0/3338088369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a0160060 0x7f62a0074240 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f6288009840 tx=0x7f628801c5b0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.789+0000 7f62a5600640 1 -- 172.21.15.50:0/3338088369 shutdown_connections 2024-09-21T06:09:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.789+0000 7f62a5600640 1 --2- 172.21.15.50:0/3338088369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a0160060 0x7f62a0074240 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.789+0000 7f62a5600640 1 -- 172.21.15.50:0/3338088369 >> 172.21.15.50:0/3338088369 conn(0x7f62a00b4560 msgr2=0x7f62a00d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.789+0000 7f62a5600640 1 -- 172.21.15.50:0/3338088369 shutdown_connections 2024-09-21T06:09:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.789+0000 7f62a5600640 1 -- 172.21.15.50:0/3338088369 wait complete. 2024-09-21T06:09:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.789+0000 7f62a5600640 1 Processor -- start 2024-09-21T06:09:52.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f62a5600640 1 -- start start 2024-09-21T06:09:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f62a5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a01536f0 0x7f62a0167b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f62a5600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a0160060 0x7f62a0151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f62a5600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62a0160470 0x7f62a00b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f62a5600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f62a016b530 con 0x7f62a01536f0 2024-09-21T06:09:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f62a5600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f62a016b3b0 con 0x7f62a0160060 2024-09-21T06:09:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f62a5600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f62a016b6b0 con 0x7f62a0160470 2024-09-21T06:09:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f629fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a01536f0 0x7f62a0167b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f629fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a01536f0 0x7f62a0167b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37958/0 (socket says 172.21.15.50:37958) 2024-09-21T06:09:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f629fe00640 1 -- 172.21.15.50:0/1039495507 learned_addr learned my addr 172.21.15.50:0/1039495507 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f6297400640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a0160060 0x7f62a0151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f62a4c00640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62a0160470 0x7f62a00b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f629fe00640 1 -- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62a0160470 msgr2=0x7f62a00b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f629fe00640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62a0160470 0x7f62a00b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f629fe00640 1 -- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a0160060 msgr2=0x7f62a0151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f629fe00640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a0160060 0x7f62a0151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f629fe00640 1 -- 172.21.15.50:0/1039495507 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a0161440 con 0x7f62a01536f0 2024-09-21T06:09:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f629fe00640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a01536f0 0x7f62a0167b10 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f629000c3c0 tx=0x7f629000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f629cc00640 1 -- 172.21.15.50:0/1039495507 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290017070 con 0x7f62a01536f0 2024-09-21T06:09:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f629cc00640 1 -- 172.21.15.50:0/1039495507 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62900053c0 con 0x7f62a01536f0 2024-09-21T06:09:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f629cc00640 1 -- 172.21.15.50:0/1039495507 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290003860 con 0x7f62a01536f0 2024-09-21T06:09:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f62a5600640 1 -- 172.21.15.50:0/1039495507 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a0161720 con 0x7f62a01536f0 2024-09-21T06:09:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.793+0000 7f62a5600640 1 -- 172.21.15.50:0/1039495507 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a0155e40 con 0x7f62a01536f0 2024-09-21T06:09:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.797+0000 7f62a5600640 1 -- 172.21.15.50:0/1039495507 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a0074240 con 0x7f62a01536f0 2024-09-21T06:09:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.797+0000 7f629cc00640 1 -- 172.21.15.50:0/1039495507 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f629001a070 con 0x7f62a01536f0 2024-09-21T06:09:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.797+0000 7f629cc00640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f627c074ca0 0x7f627c0771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.797+0000 7f629cc00640 1 -- 172.21.15.50:0/1039495507 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f6290092360 con 0x7f62a01536f0 2024-09-21T06:09:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.797+0000 7f6297400640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f627c074ca0 0x7f627c0771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.797+0000 7f6297400640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f627c074ca0 0x7f627c0771b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f62880ae000 tx=0x7f628801adf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:52.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:52.801+0000 7f629cc00640 1 -- 172.21.15.50:0/1039495507 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f629005e830 con 0x7f62a01536f0 2024-09-21T06:09:53.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.037+0000 7f62a5600640 1 -- 172.21.15.50:0/1039495507 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f62a015c440 con 0x7f62a01536f0 2024-09-21T06:09:53.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.037+0000 7f629cc00640 1 -- 172.21.15.50:0/1039495507 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1305 (secure 0 0 0) 0x7f6290063260 con 0x7f62a01536f0 2024-09-21T06:09:53.039 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:53.039 INFO:teuthology.orchestra.run.smithi050.stdout:{"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":829,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":112,"num_osds":8,"num_up_osds":8,"osd_up_since":1726898177,"num_in_osds":8,"osd_in_since":1726898167,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":90},{"state_name":"creating+peering","count":7}],"num_pgs":97,"num_pools":3,"num_objects":21,"data_bytes":592744,"bytes_used":1050484736,"bytes_avail":772043628544,"bytes_total":773094113280,"inactive_pgs_ratio":0.072164945304393768,"write_bytes_sec":1263,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":74,"btime":"2024-09-21T06:09:50:555528+0000","id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8304}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-09-21T06:09:35.179672+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:09:53.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.041+0000 7f62a5600640 1 -- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f627c074ca0 msgr2=0x7f627c0771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:53.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.041+0000 7f62a5600640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f627c074ca0 0x7f627c0771b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f62880ae000 tx=0x7f628801adf0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.041+0000 7f62a5600640 1 -- 172.21.15.50:0/1039495507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a01536f0 msgr2=0x7f62a0167b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:53.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.041+0000 7f62a5600640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a01536f0 0x7f62a0167b10 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f629000c3c0 tx=0x7f629000c880 comp rx=0 tx=0).stop 2024-09-21T06:09:53.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.041+0000 7f62a5600640 1 -- 172.21.15.50:0/1039495507 shutdown_connections 2024-09-21T06:09:53.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.041+0000 7f62a5600640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f62a0160470 0x7f62a00b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.041+0000 7f62a5600640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a0160060 0x7f62a0151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.041+0000 7f62a5600640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f627c074ca0 0x7f627c0771b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.041+0000 7f62a5600640 1 --2- 172.21.15.50:0/1039495507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a01536f0 0x7f62a0167b10 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.041+0000 7f62a5600640 1 -- 172.21.15.50:0/1039495507 >> 172.21.15.50:0/1039495507 conn(0x7f62a00b4560 msgr2=0x7f62a015f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:53.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.041+0000 7f62a5600640 1 -- 172.21.15.50:0/1039495507 shutdown_connections 2024-09-21T06:09:53.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.041+0000 7f62a5600640 1 -- 172.21.15.50:0/1039495507 wait complete. 2024-09-21T06:09:53.062 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T06:09:53.062 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:09:53.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.221+0000 7f85b3c00640 1 -- 172.21.15.50:0/73696679 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85ac0cdbb0 msgr2=0x7f85ac15ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:53.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.221+0000 7f85b3c00640 1 --2- 172.21.15.50:0/73696679 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85ac0cdbb0 0x7f85ac15ebf0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f85ac06f210 tx=0x7f85a401cd80 comp rx=0 tx=0).stop 2024-09-21T06:09:53.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.221+0000 7f85b3c00640 1 -- 172.21.15.50:0/73696679 shutdown_connections 2024-09-21T06:09:53.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.221+0000 7f85b3c00640 1 --2- 172.21.15.50:0/73696679 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85ac0cdbb0 0x7f85ac15ebf0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.221+0000 7f85b3c00640 1 --2- 172.21.15.50:0/73696679 >> v2:172.21.15.88:3300/0 conn(0x7f85ac15aa50 0x7f85ac0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.221+0000 7f85b3c00640 1 -- 172.21.15.50:0/73696679 >> 172.21.15.50:0/73696679 conn(0x7f85ac0b2040 msgr2=0x7f85ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:53.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.221+0000 7f85b3c00640 1 -- 172.21.15.50:0/73696679 shutdown_connections 2024-09-21T06:09:53.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.221+0000 7f85b3c00640 1 -- 172.21.15.50:0/73696679 wait complete. 2024-09-21T06:09:53.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.221+0000 7f85b3c00640 1 Processor -- start 2024-09-21T06:09:53.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.221+0000 7f85b3c00640 1 -- start start 2024-09-21T06:09:53.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b3c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ac0cd7a0 0x7f85ac1f9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:53.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b3c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85ac0cdbb0 0x7f85ac140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:53.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b3c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85ac15aa50 0x7f85ac157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:53.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b3c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac165e70 con 0x7f85ac15aa50 2024-09-21T06:09:53.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b3c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac0b5f40 con 0x7f85ac0cd7a0 2024-09-21T06:09:53.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b3c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f85ac165ff0 con 0x7f85ac0cdbb0 2024-09-21T06:09:53.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b1c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85ac15aa50 0x7f85ac157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:53.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b1c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85ac15aa50 0x7f85ac157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37984/0 (socket says 172.21.15.50:37984) 2024-09-21T06:09:53.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b1c00640 1 -- 172.21.15.50:0/268207013 learned_addr learned my addr 172.21.15.50:0/268207013 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:53.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b1200640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ac0cd7a0 0x7f85ac1f9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:53.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85abe00640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85ac0cdbb0 0x7f85ac140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:53.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b1c00640 1 -- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85ac0cdbb0 msgr2=0x7f85ac140b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:53.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b1c00640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85ac0cdbb0 0x7f85ac140b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b1c00640 1 -- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ac0cd7a0 msgr2=0x7f85ac1f9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:53.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b1c00640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ac0cd7a0 0x7f85ac1f9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b1c00640 1 -- 172.21.15.50:0/268207013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ac1410b0 con 0x7f85ac15aa50 2024-09-21T06:09:53.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b1c00640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85ac15aa50 0x7f85ac157700 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f85a4019990 tx=0x7f85a40092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:53.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85a8c00640 1 -- 172.21.15.50:0/268207013 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40bb070 con 0x7f85ac15aa50 2024-09-21T06:09:53.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85a8c00640 1 -- 172.21.15.50:0/268207013 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85a4004d40 con 0x7f85ac15aa50 2024-09-21T06:09:53.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b3c00640 1 -- 172.21.15.50:0/268207013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ac1556e0 con 0x7f85ac15aa50 2024-09-21T06:09:53.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85a8c00640 1 -- 172.21.15.50:0/268207013 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40b2040 con 0x7f85ac15aa50 2024-09-21T06:09:53.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b3c00640 1 -- 172.21.15.50:0/268207013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ac154390 con 0x7f85ac15aa50 2024-09-21T06:09:53.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.225+0000 7f85b3c00640 1 -- 172.21.15.50:0/268207013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ac154180 con 0x7f85ac15aa50 2024-09-21T06:09:53.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.229+0000 7f85a8c00640 1 -- 172.21.15.50:0/268207013 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f85a40be070 con 0x7f85ac15aa50 2024-09-21T06:09:53.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.229+0000 7f85a8c00640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f858c074ca0 0x7f858c0771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:53.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.229+0000 7f85a8c00640 1 -- 172.21.15.50:0/268207013 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f85a412ccd0 con 0x7f85ac15aa50 2024-09-21T06:09:53.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.229+0000 7f85b1200640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f858c074ca0 0x7f858c0771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:53.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.229+0000 7f85b1200640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f858c074ca0 0x7f858c0771b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009b40 tx=0x7f85a00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:53.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.233+0000 7f85a8c00640 1 -- 172.21.15.50:0/268207013 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f85a4016970 con 0x7f85ac15aa50 2024-09-21T06:09:53.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.405+0000 7f85b3c00640 1 -- 172.21.15.50:0/268207013 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85ac06b540 con 0x7f858c074ca0 2024-09-21T06:09:53.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85a8c00640 1 -- 172.21.15.50:0/268207013 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252492 (secure 0 0 0) 0x7f85a0011be0 con 0x7f858c074ca0 2024-09-21T06:09:53.415 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:53.416 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:09:53.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85b3c00640 1 -- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f858c074ca0 msgr2=0x7f858c0771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:53.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85b3c00640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f858c074ca0 0x7f858c0771b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009b40 tx=0x7f85a00091e0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85b3c00640 1 -- 172.21.15.50:0/268207013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85ac15aa50 msgr2=0x7f85ac157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85b3c00640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85ac15aa50 0x7f85ac157700 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f85a4019990 tx=0x7f85a40092c0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85b3c00640 1 -- 172.21.15.50:0/268207013 shutdown_connections 2024-09-21T06:09:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85b3c00640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f858c074ca0 0x7f858c0771b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85b3c00640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85ac15aa50 0x7f85ac157700 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85b3c00640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85ac0cdbb0 0x7f85ac140b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85b3c00640 1 --2- 172.21.15.50:0/268207013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ac0cd7a0 0x7f85ac1f9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85b3c00640 1 -- 172.21.15.50:0/268207013 >> 172.21.15.50:0/268207013 conn(0x7f85ac0b2040 msgr2=0x7f85ac15d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85b3c00640 1 -- 172.21.15.50:0/268207013 shutdown_connections 2024-09-21T06:09:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:53.413+0000 7f85b3c00640 1 -- 172.21.15.50:0/268207013 wait complete. 2024-09-21T06:09:53.436 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-09-21T06:09:53.184367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1025892,"kb_used_data":11936,"kb_used_omap":12,"kb_used_meta":1013747,"kb_avail":753948828,"statfs":{"total":773094113280,"available":772043599872,"internally_reserved":0,"allocated":12222464,"data_stored":5951805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1038077488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.672452"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.572843+0000","last_change":"2024-09-21T06:09:47.570525+0000","last_active":"2024-09-21T06:09:47.572843+0000","last_peered":"2024-09-21T06:09:47.572843+0000","last_clean":"2024-09-21T06:09:47.572843+0000","last_became_active":"2024-09-21T06:09:47.567795+0000","last_became_peered":"2024-09-21T06:09:47.567795+0000","last_unstale":"2024-09-21T06:09:47.572843+0000","last_undegraded":"2024-09-21T06:09:47.572843+0000","last_fullsized":"2024-09-21T06:09:47.572843+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:18.342305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552384+0000","last_change":"2024-09-21T06:09:47.552824+0000","last_active":"2024-09-21T06:09:48.552384+0000","last_peered":"2024-09-21T06:09:48.552384+0000","last_clean":"2024-09-21T06:09:48.552384+0000","last_became_active":"2024-09-21T06:09:47.552490+0000","last_became_peered":"2024-09-21T06:09:47.552490+0000","last_unstale":"2024-09-21T06:09:48.552384+0000","last_undegraded":"2024-09-21T06:09:48.552384+0000","last_fullsized":"2024-09-21T06:09:48.552384+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.562410+0000","last_change":"2024-09-21T06:09:47.560626+0000","last_active":"2024-09-21T06:09:47.562410+0000","last_peered":"2024-09-21T06:09:47.562410+0000","last_clean":"2024-09-21T06:09:47.562410+0000","last_became_active":"2024-09-21T06:09:47.560221+0000","last_became_peered":"2024-09-21T06:09:47.560221+0000","last_unstale":"2024-09-21T06:09:47.562410+0000","last_undegraded":"2024-09-21T06:09:47.562410+0000","last_fullsized":"2024-09-21T06:09:47.562410+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561543+0000","last_change":"2024-09-21T06:09:47.551941+0000","last_active":"2024-09-21T06:09:47.561543+0000","last_peered":"2024-09-21T06:09:47.561543+0000","last_clean":"2024-09-21T06:09:47.561543+0000","last_became_active":"2024-09-21T06:09:47.551615+0000","last_became_peered":"2024-09-21T06:09:47.551615+0000","last_unstale":"2024-09-21T06:09:47.561543+0000","last_undegraded":"2024-09-21T06:09:47.561543+0000","last_fullsized":"2024-09-21T06:09:47.561543+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:54.401707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.544241+0000","last_change":"2024-09-21T06:09:47.541551+0000","last_active":"2024-09-21T06:09:47.544241+0000","last_peered":"2024-09-21T06:09:47.544241+0000","last_clean":"2024-09-21T06:09:47.544241+0000","last_became_active":"2024-09-21T06:09:47.541207+0000","last_became_peered":"2024-09-21T06:09:47.541207+0000","last_unstale":"2024-09-21T06:09:47.544241+0000","last_undegraded":"2024-09-21T06:09:47.544241+0000","last_fullsized":"2024-09-21T06:09:47.544241+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:32.688598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535294+0000","last_change":"2024-09-21T06:09:47.559318+0000","last_active":"2024-09-21T06:09:49.535294+0000","last_peered":"2024-09-21T06:09:49.535294+0000","last_clean":"2024-09-21T06:09:49.535294+0000","last_became_active":"2024-09-21T06:09:47.556282+0000","last_became_peered":"2024-09-21T06:09:47.556282+0000","last_unstale":"2024-09-21T06:09:49.535294+0000","last_undegraded":"2024-09-21T06:09:49.535294+0000","last_fullsized":"2024-09-21T06:09:49.535294+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:34.368338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552250+0000","last_change":"2024-09-21T06:09:47.552984+0000","last_active":"2024-09-21T06:09:48.552250+0000","last_peered":"2024-09-21T06:09:48.552250+0000","last_clean":"2024-09-21T06:09:48.552250+0000","last_became_active":"2024-09-21T06:09:47.550957+0000","last_became_peered":"2024-09-21T06:09:47.550957+0000","last_unstale":"2024-09-21T06:09:48.552250+0000","last_undegraded":"2024-09-21T06:09:48.552250+0000","last_fullsized":"2024-09-21T06:09:48.552250+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:29.803361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.562370+0000","last_change":"2024-09-21T06:09:47.559238+0000","last_active":"2024-09-21T06:09:47.562370+0000","last_peered":"2024-09-21T06:09:47.562370+0000","last_clean":"2024-09-21T06:09:47.562370+0000","last_became_active":"2024-09-21T06:09:47.558875+0000","last_became_peered":"2024-09-21T06:09:47.558875+0000","last_unstale":"2024-09-21T06:09:47.562370+0000","last_undegraded":"2024-09-21T06:09:47.562370+0000","last_fullsized":"2024-09-21T06:09:47.562370+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561508+0000","last_change":"2024-09-21T06:09:47.550887+0000","last_active":"2024-09-21T06:09:47.561508+0000","last_peered":"2024-09-21T06:09:47.561508+0000","last_clean":"2024-09-21T06:09:47.561508+0000","last_became_active":"2024-09-21T06:09:47.550442+0000","last_became_peered":"2024-09-21T06:09:47.550442+0000","last_unstale":"2024-09-21T06:09:47.561508+0000","last_undegraded":"2024-09-21T06:09:47.561508+0000","last_fullsized":"2024-09-21T06:09:47.561508+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:56.405036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568432+0000","last_change":"2024-09-21T06:09:47.560905+0000","last_active":"2024-09-21T06:09:47.568432+0000","last_peered":"2024-09-21T06:09:47.568432+0000","last_clean":"2024-09-21T06:09:47.568432+0000","last_became_active":"2024-09-21T06:09:47.558757+0000","last_became_peered":"2024-09-21T06:09:47.558757+0000","last_unstale":"2024-09-21T06:09:47.568432+0000","last_undegraded":"2024-09-21T06:09:47.568432+0000","last_fullsized":"2024-09-21T06:09:47.568432+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:19.852862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553229+0000","last_change":"2024-09-21T06:09:47.557467+0000","last_active":"2024-09-21T06:09:48.553229+0000","last_peered":"2024-09-21T06:09:48.553229+0000","last_clean":"2024-09-21T06:09:48.553229+0000","last_became_active":"2024-09-21T06:09:47.554901+0000","last_became_peered":"2024-09-21T06:09:47.554901+0000","last_unstale":"2024-09-21T06:09:48.553229+0000","last_undegraded":"2024-09-21T06:09:48.553229+0000","last_fullsized":"2024-09-21T06:09:48.553229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:31.714893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565149+0000","last_change":"2024-09-21T06:09:47.556163+0000","last_active":"2024-09-21T06:09:47.565149+0000","last_peered":"2024-09-21T06:09:47.565149+0000","last_clean":"2024-09-21T06:09:47.565149+0000","last_became_active":"2024-09-21T06:09:47.555656+0000","last_became_peered":"2024-09-21T06:09:47.555656+0000","last_unstale":"2024-09-21T06:09:47.565149+0000","last_undegraded":"2024-09-21T06:09:47.565149+0000","last_fullsized":"2024-09-21T06:09:47.565149+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:44.553253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561358+0000","last_change":"2024-09-21T06:09:47.554101+0000","last_active":"2024-09-21T06:09:47.561358+0000","last_peered":"2024-09-21T06:09:47.561358+0000","last_clean":"2024-09-21T06:09:47.561358+0000","last_became_active":"2024-09-21T06:09:47.553743+0000","last_became_peered":"2024-09-21T06:09:47.553743+0000","last_unstale":"2024-09-21T06:09:47.561358+0000","last_undegraded":"2024-09-21T06:09:47.561358+0000","last_fullsized":"2024-09-21T06:09:47.561358+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:09.595527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561935+0000","last_change":"2024-09-21T06:09:47.554855+0000","last_active":"2024-09-21T06:09:47.561935+0000","last_peered":"2024-09-21T06:09:47.561935+0000","last_clean":"2024-09-21T06:09:47.561935+0000","last_became_active":"2024-09-21T06:09:47.554540+0000","last_became_peered":"2024-09-21T06:09:47.554540+0000","last_unstale":"2024-09-21T06:09:47.561935+0000","last_undegraded":"2024-09-21T06:09:47.561935+0000","last_fullsized":"2024-09-21T06:09:47.561935+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535197+0000","last_change":"2024-09-21T06:09:47.557875+0000","last_active":"2024-09-21T06:09:49.535197+0000","last_peered":"2024-09-21T06:09:49.535197+0000","last_clean":"2024-09-21T06:09:49.535197+0000","last_became_active":"2024-09-21T06:09:47.557565+0000","last_became_peered":"2024-09-21T06:09:47.557565+0000","last_unstale":"2024-09-21T06:09:49.535197+0000","last_undegraded":"2024-09-21T06:09:49.535197+0000","last_fullsized":"2024-09-21T06:09:49.535197+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:42.255816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561644+0000","last_change":"2024-09-21T06:09:47.554644+0000","last_active":"2024-09-21T06:09:47.561644+0000","last_peered":"2024-09-21T06:09:47.561644+0000","last_clean":"2024-09-21T06:09:47.561644+0000","last_became_active":"2024-09-21T06:09:47.554247+0000","last_became_peered":"2024-09-21T06:09:47.554247+0000","last_unstale":"2024-09-21T06:09:47.561644+0000","last_undegraded":"2024-09-21T06:09:47.561644+0000","last_fullsized":"2024-09-21T06:09:47.561644+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:13.781170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552610+0000","last_change":"2024-09-21T06:09:47.557683+0000","last_active":"2024-09-21T06:09:48.552610+0000","last_peered":"2024-09-21T06:09:48.552610+0000","last_clean":"2024-09-21T06:09:48.552610+0000","last_became_active":"2024-09-21T06:09:47.557357+0000","last_became_peered":"2024-09-21T06:09:47.557357+0000","last_unstale":"2024-09-21T06:09:48.552610+0000","last_undegraded":"2024-09-21T06:09:48.552610+0000","last_fullsized":"2024-09-21T06:09:48.552610+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561742+0000","last_change":"2024-09-21T06:09:47.557188+0000","last_active":"2024-09-21T06:09:48.561742+0000","last_peered":"2024-09-21T06:09:48.561742+0000","last_clean":"2024-09-21T06:09:48.561742+0000","last_became_active":"2024-09-21T06:09:47.556870+0000","last_became_peered":"2024-09-21T06:09:47.556870+0000","last_unstale":"2024-09-21T06:09:48.561742+0000","last_undegraded":"2024-09-21T06:09:48.561742+0000","last_fullsized":"2024-09-21T06:09:48.561742+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:38.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564093+0000","last_change":"2024-09-21T06:09:47.562338+0000","last_active":"2024-09-21T06:09:47.564093+0000","last_peered":"2024-09-21T06:09:47.564093+0000","last_clean":"2024-09-21T06:09:47.564093+0000","last_became_active":"2024-09-21T06:09:47.561835+0000","last_became_peered":"2024-09-21T06:09:47.561835+0000","last_unstale":"2024-09-21T06:09:47.564093+0000","last_undegraded":"2024-09-21T06:09:47.564093+0000","last_fullsized":"2024-09-21T06:09:47.564093+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552835+0000","last_change":"2024-09-21T06:09:47.548087+0000","last_active":"2024-09-21T06:09:48.552835+0000","last_peered":"2024-09-21T06:09:48.552835+0000","last_clean":"2024-09-21T06:09:48.552835+0000","last_became_active":"2024-09-21T06:09:47.546530+0000","last_became_peered":"2024-09-21T06:09:47.546530+0000","last_unstale":"2024-09-21T06:09:48.552835+0000","last_undegraded":"2024-09-21T06:09:48.552835+0000","last_fullsized":"2024-09-21T06:09:48.552835+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552595+0000","last_change":"2024-09-21T06:09:47.549010+0000","last_active":"2024-09-21T06:09:48.552595+0000","last_peered":"2024-09-21T06:09:48.552595+0000","last_clean":"2024-09-21T06:09:48.552595+0000","last_became_active":"2024-09-21T06:09:47.548666+0000","last_became_peered":"2024-09-21T06:09:47.548666+0000","last_unstale":"2024-09-21T06:09:48.552595+0000","last_undegraded":"2024-09-21T06:09:48.552595+0000","last_fullsized":"2024-09-21T06:09:48.552595+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565056+0000","last_change":"2024-09-21T06:09:47.554127+0000","last_active":"2024-09-21T06:09:47.565056+0000","last_peered":"2024-09-21T06:09:47.565056+0000","last_clean":"2024-09-21T06:09:47.565056+0000","last_became_active":"2024-09-21T06:09:47.553809+0000","last_became_peered":"2024-09-21T06:09:47.553809+0000","last_unstale":"2024-09-21T06:09:47.565056+0000","last_undegraded":"2024-09-21T06:09:47.565056+0000","last_fullsized":"2024-09-21T06:09:47.565056+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553426+0000","last_change":"2024-09-21T06:09:47.546392+0000","last_active":"2024-09-21T06:09:48.553426+0000","last_peered":"2024-09-21T06:09:48.553426+0000","last_clean":"2024-09-21T06:09:48.553426+0000","last_became_active":"2024-09-21T06:09:47.545962+0000","last_became_peered":"2024-09-21T06:09:47.545962+0000","last_unstale":"2024-09-21T06:09:48.553426+0000","last_undegraded":"2024-09-21T06:09:48.553426+0000","last_fullsized":"2024-09-21T06:09:48.553426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:35.324859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565233+0000","last_change":"2024-09-21T06:09:47.555478+0000","last_active":"2024-09-21T06:09:47.565233+0000","last_peered":"2024-09-21T06:09:47.565233+0000","last_clean":"2024-09-21T06:09:47.565233+0000","last_became_active":"2024-09-21T06:09:47.555175+0000","last_became_peered":"2024-09-21T06:09:47.555175+0000","last_unstale":"2024-09-21T06:09:47.565233+0000","last_undegraded":"2024-09-21T06:09:47.565233+0000","last_fullsized":"2024-09-21T06:09:47.565233+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552852+0000","last_change":"2024-09-21T06:09:47.557085+0000","last_active":"2024-09-21T06:09:48.552852+0000","last_peered":"2024-09-21T06:09:48.552852+0000","last_clean":"2024-09-21T06:09:48.552852+0000","last_became_active":"2024-09-21T06:09:47.556727+0000","last_became_peered":"2024-09-21T06:09:47.556727+0000","last_unstale":"2024-09-21T06:09:48.552852+0000","last_undegraded":"2024-09-21T06:09:48.552852+0000","last_fullsized":"2024-09-21T06:09:48.552852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568402+0000","last_change":"2024-09-21T06:09:47.559094+0000","last_active":"2024-09-21T06:09:47.568402+0000","last_peered":"2024-09-21T06:09:47.568402+0000","last_clean":"2024-09-21T06:09:47.568402+0000","last_became_active":"2024-09-21T06:09:47.558738+0000","last_became_peered":"2024-09-21T06:09:47.558738+0000","last_unstale":"2024-09-21T06:09:47.568402+0000","last_undegraded":"2024-09-21T06:09:47.568402+0000","last_fullsized":"2024-09-21T06:09:47.568402+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:39.640902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.536232+0000","last_change":"2024-09-21T06:09:47.557791+0000","last_active":"2024-09-21T06:09:49.536232+0000","last_peered":"2024-09-21T06:09:49.536232+0000","last_clean":"2024-09-21T06:09:49.536232+0000","last_became_active":"2024-09-21T06:09:47.557439+0000","last_became_peered":"2024-09-21T06:09:47.557439+0000","last_unstale":"2024-09-21T06:09:49.536232+0000","last_undegraded":"2024-09-21T06:09:49.536232+0000","last_fullsized":"2024-09-21T06:09:49.536232+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:17.327923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565098+0000","last_change":"2024-09-21T06:09:47.552743+0000","last_active":"2024-09-21T06:09:47.565098+0000","last_peered":"2024-09-21T06:09:47.565098+0000","last_clean":"2024-09-21T06:09:47.565098+0000","last_became_active":"2024-09-21T06:09:47.552402+0000","last_became_peered":"2024-09-21T06:09:47.552402+0000","last_unstale":"2024-09-21T06:09:47.565098+0000","last_undegraded":"2024-09-21T06:09:47.565098+0000","last_fullsized":"2024-09-21T06:09:47.565098+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552061+0000","last_change":"2024-09-21T06:09:47.547136+0000","last_active":"2024-09-21T06:09:48.552061+0000","last_peered":"2024-09-21T06:09:48.552061+0000","last_clean":"2024-09-21T06:09:48.552061+0000","last_became_active":"2024-09-21T06:09:47.546771+0000","last_became_peered":"2024-09-21T06:09:47.546771+0000","last_unstale":"2024-09-21T06:09:48.552061+0000","last_undegraded":"2024-09-21T06:09:48.552061+0000","last_fullsized":"2024-09-21T06:09:48.552061+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:09.595527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553225+0000","last_change":"2024-09-21T06:09:47.550506+0000","last_active":"2024-09-21T06:09:48.553225+0000","last_peered":"2024-09-21T06:09:48.553225+0000","last_clean":"2024-09-21T06:09:48.553225+0000","last_became_active":"2024-09-21T06:09:47.550172+0000","last_became_peered":"2024-09-21T06:09:47.550172+0000","last_unstale":"2024-09-21T06:09:48.553225+0000","last_undegraded":"2024-09-21T06:09:48.553225+0000","last_fullsized":"2024-09-21T06:09:48.553225+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.338192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564838+0000","last_change":"2024-09-21T06:09:47.548514+0000","last_active":"2024-09-21T06:09:47.564838+0000","last_peered":"2024-09-21T06:09:47.564838+0000","last_clean":"2024-09-21T06:09:47.564838+0000","last_became_active":"2024-09-21T06:09:47.548034+0000","last_became_peered":"2024-09-21T06:09:47.548034+0000","last_unstale":"2024-09-21T06:09:47.564838+0000","last_undegraded":"2024-09-21T06:09:47.564838+0000","last_fullsized":"2024-09-21T06:09:47.564838+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:47.209532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535081+0000","last_change":"2024-09-21T06:09:47.562416+0000","last_active":"2024-09-21T06:09:49.535081+0000","last_peered":"2024-09-21T06:09:49.535081+0000","last_clean":"2024-09-21T06:09:49.535081+0000","last_became_active":"2024-09-21T06:09:47.562061+0000","last_became_peered":"2024-09-21T06:09:47.562061+0000","last_unstale":"2024-09-21T06:09:49.535081+0000","last_undegraded":"2024-09-21T06:09:49.535081+0000","last_fullsized":"2024-09-21T06:09:49.535081+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:55.633603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.536401+0000","last_change":"2024-09-21T06:09:47.550815+0000","last_active":"2024-09-21T06:09:49.536401+0000","last_peered":"2024-09-21T06:09:49.536401+0000","last_clean":"2024-09-21T06:09:49.536401+0000","last_became_active":"2024-09-21T06:09:47.548731+0000","last_became_peered":"2024-09-21T06:09:47.548731+0000","last_unstale":"2024-09-21T06:09:49.536401+0000","last_undegraded":"2024-09-21T06:09:49.536401+0000","last_fullsized":"2024-09-21T06:09:49.536401+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:58.303903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553791+0000","last_change":"2024-09-21T06:09:46.528315+0000","last_active":"2024-09-21T06:09:48.553791+0000","last_peered":"2024-09-21T06:09:48.553791+0000","last_clean":"2024-09-21T06:09:48.553791+0000","last_became_active":"2024-09-21T06:09:46.527949+0000","last_became_peered":"2024-09-21T06:09:46.527949+0000","last_unstale":"2024-09-21T06:09:48.553791+0000","last_undegraded":"2024-09-21T06:09:48.553791+0000","last_fullsized":"2024-09-21T06:09:48.553791+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:23.941639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561877+0000","last_change":"2024-09-21T06:09:47.553802+0000","last_active":"2024-09-21T06:09:48.561877+0000","last_peered":"2024-09-21T06:09:48.561877+0000","last_clean":"2024-09-21T06:09:48.561877+0000","last_became_active":"2024-09-21T06:09:47.553503+0000","last_became_peered":"2024-09-21T06:09:47.553503+0000","last_unstale":"2024-09-21T06:09:48.561877+0000","last_undegraded":"2024-09-21T06:09:48.561877+0000","last_fullsized":"2024-09-21T06:09:48.561877+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:58.813046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569291+0000","last_change":"2024-09-21T06:09:46.531370+0000","last_active":"2024-09-21T06:09:47.569291+0000","last_peered":"2024-09-21T06:09:47.569291+0000","last_clean":"2024-09-21T06:09:47.569291+0000","last_became_active":"2024-09-21T06:09:46.530960+0000","last_became_peered":"2024-09-21T06:09:46.530960+0000","last_unstale":"2024-09-21T06:09:47.569291+0000","last_undegraded":"2024-09-21T06:09:47.569291+0000","last_fullsized":"2024-09-21T06:09:47.569291+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.536422+0000","last_change":"2024-09-21T06:09:47.557244+0000","last_active":"2024-09-21T06:09:49.536422+0000","last_peered":"2024-09-21T06:09:49.536422+0000","last_clean":"2024-09-21T06:09:49.536422+0000","last_became_active":"2024-09-21T06:09:47.556855+0000","last_became_peered":"2024-09-21T06:09:47.556855+0000","last_unstale":"2024-09-21T06:09:49.536422+0000","last_undegraded":"2024-09-21T06:09:49.536422+0000","last_fullsized":"2024-09-21T06:09:49.536422+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:00.301002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.577193+0000","last_change":"2024-09-21T06:09:46.544553+0000","last_active":"2024-09-21T06:09:47.577193+0000","last_peered":"2024-09-21T06:09:47.577193+0000","last_clean":"2024-09-21T06:09:47.577193+0000","last_became_active":"2024-09-21T06:09:46.544246+0000","last_became_peered":"2024-09-21T06:09:46.544246+0000","last_unstale":"2024-09-21T06:09:47.577193+0000","last_undegraded":"2024-09-21T06:09:47.577193+0000","last_fullsized":"2024-09-21T06:09:47.577193+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.718740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.570926+0000","last_change":"2024-09-21T06:09:47.569840+0000","last_active":"2024-09-21T06:09:47.570926+0000","last_peered":"2024-09-21T06:09:47.570926+0000","last_clean":"2024-09-21T06:09:47.570926+0000","last_became_active":"2024-09-21T06:09:47.569541+0000","last_became_peered":"2024-09-21T06:09:47.569541+0000","last_unstale":"2024-09-21T06:09:47.570926+0000","last_undegraded":"2024-09-21T06:09:47.570926+0000","last_fullsized":"2024-09-21T06:09:47.570926+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.577763+0000","last_change":"2024-09-21T06:09:46.543348+0000","last_active":"2024-09-21T06:09:47.577763+0000","last_peered":"2024-09-21T06:09:47.577763+0000","last_clean":"2024-09-21T06:09:47.577763+0000","last_became_active":"2024-09-21T06:09:46.543036+0000","last_became_peered":"2024-09-21T06:09:46.543036+0000","last_unstale":"2024-09-21T06:09:47.577763+0000","last_undegraded":"2024-09-21T06:09:47.577763+0000","last_fullsized":"2024-09-21T06:09:47.577763+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:17.178909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568248+0000","last_change":"2024-09-21T06:09:47.558350+0000","last_active":"2024-09-21T06:09:47.568248+0000","last_peered":"2024-09-21T06:09:47.568248+0000","last_clean":"2024-09-21T06:09:47.568248+0000","last_became_active":"2024-09-21T06:09:47.552686+0000","last_became_peered":"2024-09-21T06:09:47.552686+0000","last_unstale":"2024-09-21T06:09:47.568248+0000","last_undegraded":"2024-09-21T06:09:47.568248+0000","last_fullsized":"2024-09-21T06:09:47.568248+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:05.811070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553484+0000","last_change":"2024-09-21T06:09:46.535845+0000","last_active":"2024-09-21T06:09:48.553484+0000","last_peered":"2024-09-21T06:09:48.553484+0000","last_clean":"2024-09-21T06:09:48.553484+0000","last_became_active":"2024-09-21T06:09:46.535462+0000","last_became_peered":"2024-09-21T06:09:46.535462+0000","last_unstale":"2024-09-21T06:09:48.553484+0000","last_undegraded":"2024-09-21T06:09:48.553484+0000","last_fullsized":"2024-09-21T06:09:48.553484+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:32.585856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553096+0000","last_change":"2024-09-21T06:09:47.556065+0000","last_active":"2024-09-21T06:09:48.553096+0000","last_peered":"2024-09-21T06:09:48.553096+0000","last_clean":"2024-09-21T06:09:48.553096+0000","last_became_active":"2024-09-21T06:09:47.555742+0000","last_became_peered":"2024-09-21T06:09:47.555742+0000","last_unstale":"2024-09-21T06:09:48.553096+0000","last_undegraded":"2024-09-21T06:09:48.553096+0000","last_fullsized":"2024-09-21T06:09:48.553096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.575260+0000","last_change":"2024-09-21T06:09:46.549487+0000","last_active":"2024-09-21T06:09:47.575260+0000","last_peered":"2024-09-21T06:09:47.575260+0000","last_clean":"2024-09-21T06:09:47.575260+0000","last_became_active":"2024-09-21T06:09:46.548860+0000","last_became_peered":"2024-09-21T06:09:46.548860+0000","last_unstale":"2024-09-21T06:09:47.575260+0000","last_undegraded":"2024-09-21T06:09:47.575260+0000","last_fullsized":"2024-09-21T06:09:47.575260+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:45.648740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569391+0000","last_change":"2024-09-21T06:09:47.563169+0000","last_active":"2024-09-21T06:09:47.569391+0000","last_peered":"2024-09-21T06:09:47.569391+0000","last_clean":"2024-09-21T06:09:47.569391+0000","last_became_active":"2024-09-21T06:09:47.562868+0000","last_became_peered":"2024-09-21T06:09:47.562868+0000","last_unstale":"2024-09-21T06:09:47.569391+0000","last_undegraded":"2024-09-21T06:09:47.569391+0000","last_fullsized":"2024-09-21T06:09:47.569391+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:51.138002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551911+0000","last_change":"2024-09-21T06:09:46.545661+0000","last_active":"2024-09-21T06:09:48.551911+0000","last_peered":"2024-09-21T06:09:48.551911+0000","last_clean":"2024-09-21T06:09:48.551911+0000","last_became_active":"2024-09-21T06:09:46.545340+0000","last_became_peered":"2024-09-21T06:09:46.545340+0000","last_unstale":"2024-09-21T06:09:48.551911+0000","last_undegraded":"2024-09-21T06:09:48.551911+0000","last_fullsized":"2024-09-21T06:09:48.551911+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:47.880211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552001+0000","last_change":"2024-09-21T06:09:47.553054+0000","last_active":"2024-09-21T06:09:48.552001+0000","last_peered":"2024-09-21T06:09:48.552001+0000","last_clean":"2024-09-21T06:09:48.552001+0000","last_became_active":"2024-09-21T06:09:47.552742+0000","last_became_peered":"2024-09-21T06:09:47.552742+0000","last_unstale":"2024-09-21T06:09:48.552001+0000","last_undegraded":"2024-09-21T06:09:48.552001+0000","last_fullsized":"2024-09-21T06:09:48.552001+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:54.333587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.542901+0000","last_change":"2024-09-21T06:09:46.536718+0000","last_active":"2024-09-21T06:09:47.542901+0000","last_peered":"2024-09-21T06:09:47.542901+0000","last_clean":"2024-09-21T06:09:47.542901+0000","last_became_active":"2024-09-21T06:09:46.535034+0000","last_became_peered":"2024-09-21T06:09:46.535034+0000","last_unstale":"2024-09-21T06:09:47.542901+0000","last_undegraded":"2024-09-21T06:09:47.542901+0000","last_fullsized":"2024-09-21T06:09:47.542901+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:49.688976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553944+0000","last_change":"2024-09-21T06:09:47.539900+0000","last_active":"2024-09-21T06:09:48.553944+0000","last_peered":"2024-09-21T06:09:48.553944+0000","last_clean":"2024-09-21T06:09:48.553944+0000","last_became_active":"2024-09-21T06:09:47.539412+0000","last_became_peered":"2024-09-21T06:09:47.539412+0000","last_unstale":"2024-09-21T06:09:48.553944+0000","last_undegraded":"2024-09-21T06:09:48.553944+0000","last_fullsized":"2024-09-21T06:09:48.553944+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:52.206476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.532722+0000","last_change":"2024-09-21T06:09:46.540269+0000","last_active":"2024-09-21T06:09:47.532722+0000","last_peered":"2024-09-21T06:09:47.532722+0000","last_clean":"2024-09-21T06:09:47.532722+0000","last_became_active":"2024-09-21T06:09:46.539954+0000","last_became_peered":"2024-09-21T06:09:46.539954+0000","last_unstale":"2024-09-21T06:09:47.532722+0000","last_undegraded":"2024-09-21T06:09:47.532722+0000","last_fullsized":"2024-09-21T06:09:47.532722+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:13.428634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.562059+0000","last_change":"2024-09-21T06:09:47.566984+0000","last_active":"2024-09-21T06:09:48.562059+0000","last_peered":"2024-09-21T06:09:48.562059+0000","last_clean":"2024-09-21T06:09:48.562059+0000","last_became_active":"2024-09-21T06:09:47.566193+0000","last_became_peered":"2024-09-21T06:09:47.566193+0000","last_unstale":"2024-09-21T06:09:48.562059+0000","last_undegraded":"2024-09-21T06:09:48.562059+0000","last_fullsized":"2024-09-21T06:09:48.562059+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:28.501310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.535666+0000","last_change":"2024-09-21T06:09:46.536342+0000","last_active":"2024-09-21T06:09:47.535666+0000","last_peered":"2024-09-21T06:09:47.535666+0000","last_clean":"2024-09-21T06:09:47.535666+0000","last_became_active":"2024-09-21T06:09:46.536012+0000","last_became_peered":"2024-09-21T06:09:46.536012+0000","last_unstale":"2024-09-21T06:09:47.535666+0000","last_undegraded":"2024-09-21T06:09:47.535666+0000","last_fullsized":"2024-09-21T06:09:47.535666+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:57.383442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.552610+0000","last_change":"2024-09-21T06:09:47.543982+0000","last_active":"2024-09-21T06:09:47.552610+0000","last_peered":"2024-09-21T06:09:47.552610+0000","last_clean":"2024-09-21T06:09:47.552610+0000","last_became_active":"2024-09-21T06:09:47.543662+0000","last_became_peered":"2024-09-21T06:09:47.543662+0000","last_unstale":"2024-09-21T06:09:47.552610+0000","last_undegraded":"2024-09-21T06:09:47.552610+0000","last_fullsized":"2024-09-21T06:09:47.552610+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.580229+0000","last_change":"2024-09-21T06:09:46.548929+0000","last_active":"2024-09-21T06:09:47.580229+0000","last_peered":"2024-09-21T06:09:47.580229+0000","last_clean":"2024-09-21T06:09:47.580229+0000","last_became_active":"2024-09-21T06:09:46.548521+0000","last_became_peered":"2024-09-21T06:09:46.548521+0000","last_unstale":"2024-09-21T06:09:47.580229+0000","last_undegraded":"2024-09-21T06:09:47.580229+0000","last_fullsized":"2024-09-21T06:09:47.580229+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:27.494160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553855+0000","last_change":"2024-09-21T06:09:47.554124+0000","last_active":"2024-09-21T06:09:48.553855+0000","last_peered":"2024-09-21T06:09:48.553855+0000","last_clean":"2024-09-21T06:09:48.553855+0000","last_became_active":"2024-09-21T06:09:47.551484+0000","last_became_peered":"2024-09-21T06:09:47.551484+0000","last_unstale":"2024-09-21T06:09:48.553855+0000","last_undegraded":"2024-09-21T06:09:48.553855+0000","last_fullsized":"2024-09-21T06:09:48.553855+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:47.621204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":19,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.582014+0000","last_change":"2024-09-21T06:09:46.539077+0000","last_active":"2024-09-21T06:09:47.582014+0000","last_peered":"2024-09-21T06:09:47.582014+0000","last_clean":"2024-09-21T06:09:47.582014+0000","last_became_active":"2024-09-21T06:09:46.537287+0000","last_became_peered":"2024-09-21T06:09:46.537287+0000","last_unstale":"2024-09-21T06:09:47.582014+0000","last_undegraded":"2024-09-21T06:09:47.582014+0000","last_fullsized":"2024-09-21T06:09:47.582014+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:52.872003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.554303+0000","last_change":"2024-09-21T06:09:47.546084+0000","last_active":"2024-09-21T06:09:47.554303+0000","last_peered":"2024-09-21T06:09:47.554303+0000","last_clean":"2024-09-21T06:09:47.554303+0000","last_became_active":"2024-09-21T06:09:47.545677+0000","last_became_peered":"2024-09-21T06:09:47.545677+0000","last_unstale":"2024-09-21T06:09:47.554303+0000","last_undegraded":"2024-09-21T06:09:47.554303+0000","last_fullsized":"2024-09-21T06:09:47.554303+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535995+0000","last_change":"2024-09-21T06:09:46.537690+0000","last_active":"2024-09-21T06:09:49.535995+0000","last_peered":"2024-09-21T06:09:49.535995+0000","last_clean":"2024-09-21T06:09:49.535995+0000","last_became_active":"2024-09-21T06:09:46.537157+0000","last_became_peered":"2024-09-21T06:09:46.537157+0000","last_unstale":"2024-09-21T06:09:49.535995+0000","last_undegraded":"2024-09-21T06:09:49.535995+0000","last_fullsized":"2024-09-21T06:09:49.535995+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:11.151516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564968+0000","last_change":"2024-09-21T06:09:47.550402+0000","last_active":"2024-09-21T06:09:47.564968+0000","last_peered":"2024-09-21T06:09:47.564968+0000","last_clean":"2024-09-21T06:09:47.564968+0000","last_became_active":"2024-09-21T06:09:47.550085+0000","last_became_peered":"2024-09-21T06:09:47.550085+0000","last_unstale":"2024-09-21T06:09:47.564968+0000","last_undegraded":"2024-09-21T06:09:47.564968+0000","last_fullsized":"2024-09-21T06:09:47.564968+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561520+0000","last_change":"2024-09-21T06:09:46.524911+0000","last_active":"2024-09-21T06:09:48.561520+0000","last_peered":"2024-09-21T06:09:48.561520+0000","last_clean":"2024-09-21T06:09:48.561520+0000","last_became_active":"2024-09-21T06:09:46.524561+0000","last_became_peered":"2024-09-21T06:09:46.524561+0000","last_unstale":"2024-09-21T06:09:48.561520+0000","last_undegraded":"2024-09-21T06:09:48.561520+0000","last_fullsized":"2024-09-21T06:09:48.561520+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:32.557867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.547211+0000","last_change":"2024-09-21T06:09:47.541745+0000","last_active":"2024-09-21T06:09:47.547211+0000","last_peered":"2024-09-21T06:09:47.547211+0000","last_clean":"2024-09-21T06:09:47.547211+0000","last_became_active":"2024-09-21T06:09:47.541184+0000","last_became_peered":"2024-09-21T06:09:47.541184+0000","last_unstale":"2024-09-21T06:09:47.547211+0000","last_undegraded":"2024-09-21T06:09:47.547211+0000","last_fullsized":"2024-09-21T06:09:47.547211+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:49.440767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":21,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.579446+0000","last_change":"2024-09-21T06:09:46.533914+0000","last_active":"2024-09-21T06:09:47.579446+0000","last_peered":"2024-09-21T06:09:47.579446+0000","last_clean":"2024-09-21T06:09:47.579446+0000","last_became_active":"2024-09-21T06:09:46.533541+0000","last_became_peered":"2024-09-21T06:09:46.533541+0000","last_unstale":"2024-09-21T06:09:47.579446+0000","last_undegraded":"2024-09-21T06:09:47.579446+0000","last_fullsized":"2024-09-21T06:09:47.579446+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:24.340748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569557+0000","last_change":"2024-09-21T06:09:47.557155+0000","last_active":"2024-09-21T06:09:47.569557+0000","last_peered":"2024-09-21T06:09:47.569557+0000","last_clean":"2024-09-21T06:09:47.569557+0000","last_became_active":"2024-09-21T06:09:47.556834+0000","last_became_peered":"2024-09-21T06:09:47.556834+0000","last_unstale":"2024-09-21T06:09:47.569557+0000","last_undegraded":"2024-09-21T06:09:47.569557+0000","last_fullsized":"2024-09-21T06:09:47.569557+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:21.313342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.575298+0000","last_change":"2024-09-21T06:09:46.546891+0000","last_active":"2024-09-21T06:09:47.575298+0000","last_peered":"2024-09-21T06:09:47.575298+0000","last_clean":"2024-09-21T06:09:47.575298+0000","last_became_active":"2024-09-21T06:09:46.546573+0000","last_became_peered":"2024-09-21T06:09:46.546573+0000","last_unstale":"2024-09-21T06:09:47.575298+0000","last_undegraded":"2024-09-21T06:09:47.575298+0000","last_fullsized":"2024-09-21T06:09:47.575298+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:17.766821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.536371+0000","last_change":"2024-09-21T06:09:47.554707+0000","last_active":"2024-09-21T06:09:49.536371+0000","last_peered":"2024-09-21T06:09:49.536371+0000","last_clean":"2024-09-21T06:09:49.536371+0000","last_became_active":"2024-09-21T06:09:47.554388+0000","last_became_peered":"2024-09-21T06:09:47.554388+0000","last_unstale":"2024-09-21T06:09:49.536371+0000","last_undegraded":"2024-09-21T06:09:49.536371+0000","last_fullsized":"2024-09-21T06:09:49.536371+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:50.405611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.575222+0000","last_change":"2024-09-21T06:09:46.529780+0000","last_active":"2024-09-21T06:09:47.575222+0000","last_peered":"2024-09-21T06:09:47.575222+0000","last_clean":"2024-09-21T06:09:47.575222+0000","last_became_active":"2024-09-21T06:09:46.528136+0000","last_became_peered":"2024-09-21T06:09:46.528136+0000","last_unstale":"2024-09-21T06:09:47.575222+0000","last_undegraded":"2024-09-21T06:09:47.575222+0000","last_fullsized":"2024-09-21T06:09:47.575222+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:17.381392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.568420+0000","last_change":"2024-09-21T06:09:47.567084+0000","last_active":"2024-09-21T06:09:47.568420+0000","last_peered":"2024-09-21T06:09:47.568420+0000","last_clean":"2024-09-21T06:09:47.568420+0000","last_became_active":"2024-09-21T06:09:47.566717+0000","last_became_peered":"2024-09-21T06:09:47.566717+0000","last_unstale":"2024-09-21T06:09:47.568420+0000","last_undegraded":"2024-09-21T06:09:47.568420+0000","last_fullsized":"2024-09-21T06:09:47.568420+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561699+0000","last_change":"2024-09-21T06:09:46.524059+0000","last_active":"2024-09-21T06:09:48.561699+0000","last_peered":"2024-09-21T06:09:48.561699+0000","last_clean":"2024-09-21T06:09:48.561699+0000","last_became_active":"2024-09-21T06:09:46.523459+0000","last_became_peered":"2024-09-21T06:09:46.523459+0000","last_unstale":"2024-09-21T06:09:48.561699+0000","last_undegraded":"2024-09-21T06:09:48.561699+0000","last_fullsized":"2024-09-21T06:09:48.561699+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.562759+0000","last_change":"2024-09-21T06:09:47.561252+0000","last_active":"2024-09-21T06:09:47.562759+0000","last_peered":"2024-09-21T06:09:47.562759+0000","last_clean":"2024-09-21T06:09:47.562759+0000","last_became_active":"2024-09-21T06:09:47.560849+0000","last_became_peered":"2024-09-21T06:09:47.560849+0000","last_unstale":"2024-09-21T06:09:47.562759+0000","last_undegraded":"2024-09-21T06:09:47.562759+0000","last_fullsized":"2024-09-21T06:09:47.562759+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553549+0000","last_change":"2024-09-21T06:09:46.534239+0000","last_active":"2024-09-21T06:09:48.553549+0000","last_peered":"2024-09-21T06:09:48.553549+0000","last_clean":"2024-09-21T06:09:48.553549+0000","last_became_active":"2024-09-21T06:09:46.533715+0000","last_became_peered":"2024-09-21T06:09:46.533715+0000","last_unstale":"2024-09-21T06:09:48.553549+0000","last_undegraded":"2024-09-21T06:09:48.553549+0000","last_fullsized":"2024-09-21T06:09:48.553549+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:08:04.897143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.559568+0000","last_change":"2024-09-21T06:09:47.552337+0000","last_active":"2024-09-21T06:09:47.559568+0000","last_peered":"2024-09-21T06:09:47.559568+0000","last_clean":"2024-09-21T06:09:47.559568+0000","last_became_active":"2024-09-21T06:09:47.552009+0000","last_became_peered":"2024-09-21T06:09:47.552009+0000","last_unstale":"2024-09-21T06:09:47.559568+0000","last_undegraded":"2024-09-21T06:09:47.559568+0000","last_fullsized":"2024-09-21T06:09:47.559568+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:46.655889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.536021+0000","last_change":"2024-09-21T06:09:46.536200+0000","last_active":"2024-09-21T06:09:49.536021+0000","last_peered":"2024-09-21T06:09:49.536021+0000","last_clean":"2024-09-21T06:09:49.536021+0000","last_became_active":"2024-09-21T06:09:46.535894+0000","last_became_peered":"2024-09-21T06:09:46.535894+0000","last_unstale":"2024-09-21T06:09:49.536021+0000","last_undegraded":"2024-09-21T06:09:49.536021+0000","last_fullsized":"2024-09-21T06:09:49.536021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:20.365744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561750+0000","last_change":"2024-09-21T06:09:47.552189+0000","last_active":"2024-09-21T06:09:47.561750+0000","last_peered":"2024-09-21T06:09:47.561750+0000","last_clean":"2024-09-21T06:09:47.561750+0000","last_became_active":"2024-09-21T06:09:47.548927+0000","last_became_peered":"2024-09-21T06:09:47.548927+0000","last_unstale":"2024-09-21T06:09:47.561750+0000","last_undegraded":"2024-09-21T06:09:47.561750+0000","last_fullsized":"2024-09-21T06:09:47.561750+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:33.335946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.532137+0000","last_change":"2024-09-21T06:09:46.526945+0000","last_active":"2024-09-21T06:09:47.532137+0000","last_peered":"2024-09-21T06:09:47.532137+0000","last_clean":"2024-09-21T06:09:47.532137+0000","last_became_active":"2024-09-21T06:09:46.526588+0000","last_became_peered":"2024-09-21T06:09:46.526588+0000","last_unstale":"2024-09-21T06:09:47.532137+0000","last_undegraded":"2024-09-21T06:09:47.532137+0000","last_fullsized":"2024-09-21T06:09:47.532137+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:14.014304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561635+0000","last_change":"2024-09-21T06:09:47.555659+0000","last_active":"2024-09-21T06:09:47.561635+0000","last_peered":"2024-09-21T06:09:47.561635+0000","last_clean":"2024-09-21T06:09:47.561635+0000","last_became_active":"2024-09-21T06:09:47.555358+0000","last_became_peered":"2024-09-21T06:09:47.555358+0000","last_unstale":"2024-09-21T06:09:47.561635+0000","last_undegraded":"2024-09-21T06:09:47.561635+0000","last_fullsized":"2024-09-21T06:09:47.561635+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535295+0000","last_change":"2024-09-21T06:09:46.541688+0000","last_active":"2024-09-21T06:09:49.535295+0000","last_peered":"2024-09-21T06:09:49.535295+0000","last_clean":"2024-09-21T06:09:49.535295+0000","last_became_active":"2024-09-21T06:09:46.541322+0000","last_became_peered":"2024-09-21T06:09:46.541322+0000","last_unstale":"2024-09-21T06:09:49.535295+0000","last_undegraded":"2024-09-21T06:09:49.535295+0000","last_fullsized":"2024-09-21T06:09:49.535295+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:37.600477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.565078+0000","last_change":"2024-09-21T06:09:47.551665+0000","last_active":"2024-09-21T06:09:47.565078+0000","last_peered":"2024-09-21T06:09:47.565078+0000","last_clean":"2024-09-21T06:09:47.565078+0000","last_became_active":"2024-09-21T06:09:47.551313+0000","last_became_peered":"2024-09-21T06:09:47.551313+0000","last_unstale":"2024-09-21T06:09:47.565078+0000","last_undegraded":"2024-09-21T06:09:47.565078+0000","last_fullsized":"2024-09-21T06:09:47.565078+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.543137+0000","last_change":"2024-09-21T06:09:46.545814+0000","last_active":"2024-09-21T06:09:47.543137+0000","last_peered":"2024-09-21T06:09:47.543137+0000","last_clean":"2024-09-21T06:09:47.543137+0000","last_became_active":"2024-09-21T06:09:46.545413+0000","last_became_peered":"2024-09-21T06:09:46.545413+0000","last_unstale":"2024-09-21T06:09:47.543137+0000","last_undegraded":"2024-09-21T06:09:47.543137+0000","last_fullsized":"2024-09-21T06:09:47.543137+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:13.029274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561834+0000","last_change":"2024-09-21T06:09:47.566967+0000","last_active":"2024-09-21T06:09:48.561834+0000","last_peered":"2024-09-21T06:09:48.561834+0000","last_clean":"2024-09-21T06:09:48.561834+0000","last_became_active":"2024-09-21T06:09:47.566211+0000","last_became_peered":"2024-09-21T06:09:47.566211+0000","last_unstale":"2024-09-21T06:09:48.561834+0000","last_undegraded":"2024-09-21T06:09:48.561834+0000","last_fullsized":"2024-09-21T06:09:48.561834+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:14.036424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.580510+0000","last_change":"2024-09-21T06:09:46.528896+0000","last_active":"2024-09-21T06:09:47.580510+0000","last_peered":"2024-09-21T06:09:47.580510+0000","last_clean":"2024-09-21T06:09:47.580510+0000","last_became_active":"2024-09-21T06:09:46.528590+0000","last_became_peered":"2024-09-21T06:09:46.528590+0000","last_unstale":"2024-09-21T06:09:47.580510+0000","last_undegraded":"2024-09-21T06:09:47.580510+0000","last_fullsized":"2024-09-21T06:09:47.580510+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.564780+0000","last_change":"2024-09-21T06:09:47.556097+0000","last_active":"2024-09-21T06:09:47.564780+0000","last_peered":"2024-09-21T06:09:47.564780+0000","last_clean":"2024-09-21T06:09:47.564780+0000","last_became_active":"2024-09-21T06:09:47.555720+0000","last_became_peered":"2024-09-21T06:09:47.555720+0000","last_unstale":"2024-09-21T06:09:47.564780+0000","last_undegraded":"2024-09-21T06:09:47.564780+0000","last_fullsized":"2024-09-21T06:09:47.564780+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:14.289349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.563317+0000","last_change":"2024-09-21T06:09:46.525404+0000","last_active":"2024-09-21T06:09:48.563317+0000","last_peered":"2024-09-21T06:09:48.563317+0000","last_clean":"2024-09-21T06:09:48.563317+0000","last_became_active":"2024-09-21T06:09:46.525087+0000","last_became_peered":"2024-09-21T06:09:46.525087+0000","last_unstale":"2024-09-21T06:09:48.563317+0000","last_undegraded":"2024-09-21T06:09:48.563317+0000","last_fullsized":"2024-09-21T06:09:48.563317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:57.383442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.572681+0000","last_change":"2024-09-21T06:09:47.565828+0000","last_active":"2024-09-21T06:09:47.572681+0000","last_peered":"2024-09-21T06:09:47.572681+0000","last_clean":"2024-09-21T06:09:47.572681+0000","last_became_active":"2024-09-21T06:09:47.565467+0000","last_became_peered":"2024-09-21T06:09:47.565467+0000","last_unstale":"2024-09-21T06:09:47.572681+0000","last_undegraded":"2024-09-21T06:09:47.572681+0000","last_fullsized":"2024-09-21T06:09:47.572681+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:37.650305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.544753+0000","last_change":"2024-09-21T06:09:46.542691+0000","last_active":"2024-09-21T06:09:47.544753+0000","last_peered":"2024-09-21T06:09:47.544753+0000","last_clean":"2024-09-21T06:09:47.544753+0000","last_became_active":"2024-09-21T06:09:46.542293+0000","last_became_peered":"2024-09-21T06:09:46.542293+0000","last_unstale":"2024-09-21T06:09:47.544753+0000","last_undegraded":"2024-09-21T06:09:47.544753+0000","last_fullsized":"2024-09-21T06:09:47.544753+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:33.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561749+0000","last_change":"2024-09-21T06:09:47.576750+0000","last_active":"2024-09-21T06:09:48.561749+0000","last_peered":"2024-09-21T06:09:48.561749+0000","last_clean":"2024-09-21T06:09:48.561749+0000","last_became_active":"2024-09-21T06:09:47.576443+0000","last_became_peered":"2024-09-21T06:09:47.576443+0000","last_unstale":"2024-09-21T06:09:48.561749+0000","last_undegraded":"2024-09-21T06:09:48.561749+0000","last_fullsized":"2024-09-21T06:09:48.561749+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.542616+0000","last_change":"2024-09-21T06:09:46.547408+0000","last_active":"2024-09-21T06:09:47.542616+0000","last_peered":"2024-09-21T06:09:47.542616+0000","last_clean":"2024-09-21T06:09:47.542616+0000","last_became_active":"2024-09-21T06:09:46.547043+0000","last_became_peered":"2024-09-21T06:09:46.547043+0000","last_unstale":"2024-09-21T06:09:47.542616+0000","last_undegraded":"2024-09-21T06:09:47.542616+0000","last_fullsized":"2024-09-21T06:09:47.542616+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:08.588377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552144+0000","last_change":"2024-09-21T06:09:47.540415+0000","last_active":"2024-09-21T06:09:48.552144+0000","last_peered":"2024-09-21T06:09:48.552144+0000","last_clean":"2024-09-21T06:09:48.552144+0000","last_became_active":"2024-09-21T06:09:47.539926+0000","last_became_peered":"2024-09-21T06:09:47.539926+0000","last_unstale":"2024-09-21T06:09:48.552144+0000","last_undegraded":"2024-09-21T06:09:48.552144+0000","last_fullsized":"2024-09-21T06:09:48.552144+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553533+0000","last_change":"2024-09-21T06:09:46.529617+0000","last_active":"2024-09-21T06:09:48.553533+0000","last_peered":"2024-09-21T06:09:48.553533+0000","last_clean":"2024-09-21T06:09:48.553533+0000","last_became_active":"2024-09-21T06:09:46.528234+0000","last_became_peered":"2024-09-21T06:09:46.528234+0000","last_unstale":"2024-09-21T06:09:48.553533+0000","last_undegraded":"2024-09-21T06:09:48.553533+0000","last_fullsized":"2024-09-21T06:09:48.553533+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:19.814656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552277+0000","last_change":"2024-09-21T06:09:47.543597+0000","last_active":"2024-09-21T06:09:48.552277+0000","last_peered":"2024-09-21T06:09:48.552277+0000","last_clean":"2024-09-21T06:09:48.552277+0000","last_became_active":"2024-09-21T06:09:47.543282+0000","last_became_peered":"2024-09-21T06:09:47.543282+0000","last_unstale":"2024-09-21T06:09:48.552277+0000","last_undegraded":"2024-09-21T06:09:48.552277+0000","last_fullsized":"2024-09-21T06:09:48.552277+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552294+0000","last_change":"2024-09-21T06:09:46.535746+0000","last_active":"2024-09-21T06:09:48.552294+0000","last_peered":"2024-09-21T06:09:48.552294+0000","last_clean":"2024-09-21T06:09:48.552294+0000","last_became_active":"2024-09-21T06:09:46.535142+0000","last_became_peered":"2024-09-21T06:09:46.535142+0000","last_unstale":"2024-09-21T06:09:48.552294+0000","last_undegraded":"2024-09-21T06:09:48.552294+0000","last_fullsized":"2024-09-21T06:09:48.552294+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:56.819951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.561609+0000","last_change":"2024-09-21T06:09:47.553873+0000","last_active":"2024-09-21T06:09:47.561609+0000","last_peered":"2024-09-21T06:09:47.561609+0000","last_clean":"2024-09-21T06:09:47.561609+0000","last_became_active":"2024-09-21T06:09:47.553515+0000","last_became_peered":"2024-09-21T06:09:47.553515+0000","last_unstale":"2024-09-21T06:09:47.561609+0000","last_undegraded":"2024-09-21T06:09:47.561609+0000","last_fullsized":"2024-09-21T06:09:47.561609+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:49.440767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.532370+0000","last_change":"2024-09-21T06:09:46.539038+0000","last_active":"2024-09-21T06:09:47.532370+0000","last_peered":"2024-09-21T06:09:47.532370+0000","last_clean":"2024-09-21T06:09:47.532370+0000","last_became_active":"2024-09-21T06:09:46.538692+0000","last_became_peered":"2024-09-21T06:09:46.538692+0000","last_unstale":"2024-09-21T06:09:47.532370+0000","last_undegraded":"2024-09-21T06:09:47.532370+0000","last_fullsized":"2024-09-21T06:09:47.532370+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:44.565196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.569536+0000","last_change":"2024-09-21T06:09:47.564522+0000","last_active":"2024-09-21T06:09:47.569536+0000","last_peered":"2024-09-21T06:09:47.569536+0000","last_clean":"2024-09-21T06:09:47.569536+0000","last_became_active":"2024-09-21T06:09:47.564206+0000","last_became_peered":"2024-09-21T06:09:47.564206+0000","last_unstale":"2024-09-21T06:09:47.569536+0000","last_undegraded":"2024-09-21T06:09:47.569536+0000","last_fullsized":"2024-09-21T06:09:47.569536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:03.505698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.543377+0000","last_change":"2024-09-21T06:09:46.532492+0000","last_active":"2024-09-21T06:09:47.543377+0000","last_peered":"2024-09-21T06:09:47.543377+0000","last_clean":"2024-09-21T06:09:47.543377+0000","last_became_active":"2024-09-21T06:09:46.530409+0000","last_became_peered":"2024-09-21T06:09:46.530409+0000","last_unstale":"2024-09-21T06:09:47.543377+0000","last_undegraded":"2024-09-21T06:09:47.543377+0000","last_fullsized":"2024-09-21T06:09:47.543377+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:57.805896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.549238+0000","last_change":"2024-09-21T06:09:47.543465+0000","last_active":"2024-09-21T06:09:47.549238+0000","last_peered":"2024-09-21T06:09:47.549238+0000","last_clean":"2024-09-21T06:09:47.549238+0000","last_became_active":"2024-09-21T06:09:47.543121+0000","last_became_peered":"2024-09-21T06:09:47.543121+0000","last_unstale":"2024-09-21T06:09:47.549238+0000","last_undegraded":"2024-09-21T06:09:47.549238+0000","last_fullsized":"2024-09-21T06:09:47.549238+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552395+0000","last_change":"2024-09-21T06:09:46.534147+0000","last_active":"2024-09-21T06:09:48.552395+0000","last_peered":"2024-09-21T06:09:48.552395+0000","last_clean":"2024-09-21T06:09:48.552395+0000","last_became_active":"2024-09-21T06:09:46.533758+0000","last_became_peered":"2024-09-21T06:09:46.533758+0000","last_unstale":"2024-09-21T06:09:48.552395+0000","last_undegraded":"2024-09-21T06:09:48.552395+0000","last_fullsized":"2024-09-21T06:09:48.552395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:29.394474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":277,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-21T06:09:47.543464+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:09:47.543464+0000","last_peered":"2024-09-21T06:09:47.543464+0000","last_clean":"2024-09-21T06:09:47.543464+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:09:47.543464+0000","last_undegraded":"2024-09-21T06:09:47.543464+0000","last_fullsized":"2024-09-21T06:09:47.543464+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509605,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127456,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244384,"statfs":{"total":96636764160,"available":96506249216,"internally_reserved":0,"allocated":1974272,"data_stored":1186242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509605,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118560,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253280,"statfs":{"total":96636764160,"available":96515358720,"internally_reserved":0,"allocated":1388544,"data_stored":601712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509604,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82456,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289384,"statfs":{"total":96636764160,"available":96552329216,"internally_reserved":0,"allocated":1380352,"data_stored":595862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509605,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144920,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226920,"statfs":{"total":96636764160,"available":96488366080,"internally_reserved":0,"allocated":1372160,"data_stored":595258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509604,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135208,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236632,"statfs":{"total":96636764160,"available":96498311168,"internally_reserved":0,"allocated":1388544,"data_stored":596940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509606,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140760,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231080,"statfs":{"total":96636764160,"available":96492625920,"internally_reserved":0,"allocated":1372160,"data_stored":595258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509604,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131596,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240244,"statfs":{"total":96636764160,"available":96502009856,"internally_reserved":0,"allocated":1953792,"data_stored":1178767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542310,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144936,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226904,"statfs":{"total":96636764160,"available":96488349696,"internally_reserved":0,"allocated":1392640,"data_stored":601766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:09:56.441 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:09:56.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.613+0000 7fc8f4400640 1 -- 172.21.15.50:0/3728331668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b4de0 msgr2=0x7fc8ec0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:56.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.613+0000 7fc8f4400640 1 --2- 172.21.15.50:0/3728331668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b4de0 0x7fc8ec0b51f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009bc0 tx=0x7fc8e401cae0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.613+0000 7fc8f4400640 1 -- 172.21.15.50:0/3728331668 shutdown_connections 2024-09-21T06:09:56.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.613+0000 7fc8f4400640 1 --2- 172.21.15.50:0/3728331668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b4de0 0x7fc8ec0b51f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.613+0000 7fc8f4400640 1 --2- 172.21.15.50:0/3728331668 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8ec0b5c70 0x7fc8ec15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.613+0000 7fc8f4400640 1 -- 172.21.15.50:0/3728331668 >> 172.21.15.50:0/3728331668 conn(0x7fc8ec0b2040 msgr2=0x7fc8ec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:56.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.613+0000 7fc8f4400640 1 -- 172.21.15.50:0/3728331668 shutdown_connections 2024-09-21T06:09:56.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.613+0000 7fc8f4400640 1 -- 172.21.15.50:0/3728331668 wait complete. 2024-09-21T06:09:56.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.613+0000 7fc8f4400640 1 Processor -- start 2024-09-21T06:09:56.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.613+0000 7fc8f4400640 1 -- start start 2024-09-21T06:09:56.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f4400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b4de0 0x7fc8ec141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:56.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f4400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8ec0b5c70 0x7fc8ec0cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:56.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f4400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8ec1527b0 0x7fc8ec0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:56.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f4400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec168ab0 con 0x7fc8ec0b5c70 2024-09-21T06:09:56.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f4400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec168930 con 0x7fc8ec0b4de0 2024-09-21T06:09:56.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f4400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ec168c30 con 0x7fc8ec1527b0 2024-09-21T06:09:56.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8ec0b5c70 0x7fc8ec0cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:56.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8ec0b5c70 0x7fc8ec0cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37986/0 (socket says 172.21.15.50:37986) 2024-09-21T06:09:56.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f1000640 1 -- 172.21.15.50:0/2013109398 learned_addr learned my addr 172.21.15.50:0/2013109398 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:56.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f1a00640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b4de0 0x7fc8ec141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:56.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f2400640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8ec1527b0 0x7fc8ec0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:56.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f1000640 1 -- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8ec1527b0 msgr2=0x7fc8ec0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:56.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f1000640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8ec1527b0 0x7fc8ec0cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f1000640 1 -- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b4de0 msgr2=0x7fc8ec141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:56.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f1000640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b4de0 0x7fc8ec141c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f1000640 1 -- 172.21.15.50:0/2013109398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec1586c0 con 0x7fc8ec0b5c70 2024-09-21T06:09:56.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f1000640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8ec0b5c70 0x7fc8ec0cd360 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c3f0 tx=0x7fc8d800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:56.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8e9600640 1 -- 172.21.15.50:0/2013109398 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8011790 con 0x7fc8ec0b5c70 2024-09-21T06:09:56.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8e9600640 1 -- 172.21.15.50:0/2013109398 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8d8017070 con 0x7fc8ec0b5c70 2024-09-21T06:09:56.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8e9600640 1 -- 172.21.15.50:0/2013109398 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8011ce0 con 0x7fc8ec0b5c70 2024-09-21T06:09:56.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f4400640 1 -- 172.21.15.50:0/2013109398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec1589a0 con 0x7fc8ec0b5c70 2024-09-21T06:09:56.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f4400640 1 -- 172.21.15.50:0/2013109398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec0cce40 con 0x7fc8ec0b5c70 2024-09-21T06:09:56.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.617+0000 7fc8f4400640 1 -- 172.21.15.50:0/2013109398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ec0cc950 con 0x7fc8ec0b5c70 2024-09-21T06:09:56.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.621+0000 7fc8e9600640 1 -- 172.21.15.50:0/2013109398 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fc8d801c460 con 0x7fc8ec0b5c70 2024-09-21T06:09:56.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.621+0000 7fc8e9600640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc8cc074ca0 0x7fc8cc0771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:56.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.621+0000 7fc8e9600640 1 -- 172.21.15.50:0/2013109398 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fc8d8092d00 con 0x7fc8ec0b5c70 2024-09-21T06:09:56.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.621+0000 7fc8f1a00640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc8cc074ca0 0x7fc8cc0771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:56.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.621+0000 7fc8f1a00640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc8cc074ca0 0x7fc8cc0771b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009b40 tx=0x7fc8e00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:56.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.625+0000 7fc8e9600640 1 -- 172.21.15.50:0/2013109398 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc8ec0b5730 con 0x7fc8ec0b5c70 2024-09-21T06:09:56.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.801+0000 7fc8f4400640 1 -- 172.21.15.50:0/2013109398 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc8ec06b540 con 0x7fc8cc074ca0 2024-09-21T06:09:56.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.805+0000 7fc8e9600640 1 -- 172.21.15.50:0/2013109398 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252456 (secure 0 0 0) 0x7fc8e0011be0 con 0x7fc8cc074ca0 2024-09-21T06:09:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:56.811 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:09:56.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.809+0000 7fc8f4400640 1 -- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc8cc074ca0 msgr2=0x7fc8cc0771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:56.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.809+0000 7fc8f4400640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc8cc074ca0 0x7fc8cc0771b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009b40 tx=0x7fc8e00091e0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.809+0000 7fc8f4400640 1 -- 172.21.15.50:0/2013109398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8ec0b5c70 msgr2=0x7fc8ec0cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:56.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.809+0000 7fc8f4400640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8ec0b5c70 0x7fc8ec0cd360 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c3f0 tx=0x7fc8d800c8b0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.809+0000 7fc8f4400640 1 -- 172.21.15.50:0/2013109398 shutdown_connections 2024-09-21T06:09:56.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.809+0000 7fc8f4400640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8ec1527b0 0x7fc8ec0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.809+0000 7fc8f4400640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc8cc074ca0 0x7fc8cc0771b0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.809+0000 7fc8f4400640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8ec0b5c70 0x7fc8ec0cd360 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.809+0000 7fc8f4400640 1 --2- 172.21.15.50:0/2013109398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b4de0 0x7fc8ec141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.809+0000 7fc8f4400640 1 -- 172.21.15.50:0/2013109398 >> 172.21.15.50:0/2013109398 conn(0x7fc8ec0b2040 msgr2=0x7fc8ec154bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:56.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.809+0000 7fc8f4400640 1 -- 172.21.15.50:0/2013109398 shutdown_connections 2024-09-21T06:09:56.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:56.809+0000 7fc8f4400640 1 -- 172.21.15.50:0/2013109398 wait complete. 2024-09-21T06:09:56.831 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2024-09-21T06:09:55.184956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1025976,"kb_used_data":12040,"kb_used_omap":12,"kb_used_meta":1013747,"kb_avail":753948744,"statfs":{"total":773094113280,"available":772043513856,"internally_reserved":0,"allocated":12328960,"data_stored":5998692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1038077488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002107"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034597+0000","last_change":"2024-09-21T06:09:47.570525+0000","last_active":"2024-09-21T06:09:49.034597+0000","last_peered":"2024-09-21T06:09:49.034597+0000","last_clean":"2024-09-21T06:09:49.034597+0000","last_became_active":"2024-09-21T06:09:47.567795+0000","last_became_peered":"2024-09-21T06:09:47.567795+0000","last_unstale":"2024-09-21T06:09:49.034597+0000","last_undegraded":"2024-09-21T06:09:49.034597+0000","last_fullsized":"2024-09-21T06:09:49.034597+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:02.783698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552384+0000","last_change":"2024-09-21T06:09:47.552824+0000","last_active":"2024-09-21T06:09:48.552384+0000","last_peered":"2024-09-21T06:09:48.552384+0000","last_clean":"2024-09-21T06:09:48.552384+0000","last_became_active":"2024-09-21T06:09:47.552490+0000","last_became_peered":"2024-09-21T06:09:47.552490+0000","last_unstale":"2024-09-21T06:09:48.552384+0000","last_undegraded":"2024-09-21T06:09:48.552384+0000","last_fullsized":"2024-09-21T06:09:48.552384+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:03.345789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.099688+0000","last_change":"2024-09-21T06:09:47.560626+0000","last_active":"2024-09-21T06:09:49.099688+0000","last_peered":"2024-09-21T06:09:49.099688+0000","last_clean":"2024-09-21T06:09:49.099688+0000","last_became_active":"2024-09-21T06:09:47.560221+0000","last_became_peered":"2024-09-21T06:09:47.560221+0000","last_unstale":"2024-09-21T06:09:49.099688+0000","last_undegraded":"2024-09-21T06:09:49.099688+0000","last_fullsized":"2024-09-21T06:09:49.099688+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:33.335946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034445+0000","last_change":"2024-09-21T06:09:47.551941+0000","last_active":"2024-09-21T06:09:49.034445+0000","last_peered":"2024-09-21T06:09:49.034445+0000","last_clean":"2024-09-21T06:09:49.034445+0000","last_became_active":"2024-09-21T06:09:47.551615+0000","last_became_peered":"2024-09-21T06:09:47.551615+0000","last_unstale":"2024-09-21T06:09:49.034445+0000","last_undegraded":"2024-09-21T06:09:49.034445+0000","last_fullsized":"2024-09-21T06:09:49.034445+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:26.871173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535936+0000","last_change":"2024-09-21T06:09:47.541551+0000","last_active":"2024-09-21T06:09:49.535936+0000","last_peered":"2024-09-21T06:09:49.535936+0000","last_clean":"2024-09-21T06:09:49.535936+0000","last_became_active":"2024-09-21T06:09:47.541207+0000","last_became_peered":"2024-09-21T06:09:47.541207+0000","last_unstale":"2024-09-21T06:09:49.535936+0000","last_undegraded":"2024-09-21T06:09:49.535936+0000","last_fullsized":"2024-09-21T06:09:49.535936+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:42.444246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535294+0000","last_change":"2024-09-21T06:09:47.559318+0000","last_active":"2024-09-21T06:09:49.535294+0000","last_peered":"2024-09-21T06:09:49.535294+0000","last_clean":"2024-09-21T06:09:49.535294+0000","last_became_active":"2024-09-21T06:09:47.556282+0000","last_became_peered":"2024-09-21T06:09:47.556282+0000","last_unstale":"2024-09-21T06:09:49.535294+0000","last_undegraded":"2024-09-21T06:09:49.535294+0000","last_fullsized":"2024-09-21T06:09:49.535294+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:34.368338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552250+0000","last_change":"2024-09-21T06:09:47.552984+0000","last_active":"2024-09-21T06:09:48.552250+0000","last_peered":"2024-09-21T06:09:48.552250+0000","last_clean":"2024-09-21T06:09:48.552250+0000","last_became_active":"2024-09-21T06:09:47.550957+0000","last_became_peered":"2024-09-21T06:09:47.550957+0000","last_unstale":"2024-09-21T06:09:48.552250+0000","last_undegraded":"2024-09-21T06:09:48.552250+0000","last_fullsized":"2024-09-21T06:09:48.552250+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:29.803361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.099621+0000","last_change":"2024-09-21T06:09:47.559238+0000","last_active":"2024-09-21T06:09:49.099621+0000","last_peered":"2024-09-21T06:09:49.099621+0000","last_clean":"2024-09-21T06:09:49.099621+0000","last_became_active":"2024-09-21T06:09:47.558875+0000","last_became_peered":"2024-09-21T06:09:47.558875+0000","last_unstale":"2024-09-21T06:09:49.099621+0000","last_undegraded":"2024-09-21T06:09:49.099621+0000","last_fullsized":"2024-09-21T06:09:49.099621+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:56.405036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034753+0000","last_change":"2024-09-21T06:09:47.550887+0000","last_active":"2024-09-21T06:09:49.034753+0000","last_peered":"2024-09-21T06:09:49.034753+0000","last_clean":"2024-09-21T06:09:49.034753+0000","last_became_active":"2024-09-21T06:09:47.550442+0000","last_became_peered":"2024-09-21T06:09:47.550442+0000","last_unstale":"2024-09-21T06:09:49.034753+0000","last_undegraded":"2024-09-21T06:09:49.034753+0000","last_fullsized":"2024-09-21T06:09:49.034753+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:45.783960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034370+0000","last_change":"2024-09-21T06:09:47.560905+0000","last_active":"2024-09-21T06:09:49.034370+0000","last_peered":"2024-09-21T06:09:49.034370+0000","last_clean":"2024-09-21T06:09:49.034370+0000","last_became_active":"2024-09-21T06:09:47.558757+0000","last_became_peered":"2024-09-21T06:09:47.558757+0000","last_unstale":"2024-09-21T06:09:49.034370+0000","last_undegraded":"2024-09-21T06:09:49.034370+0000","last_fullsized":"2024-09-21T06:09:49.034370+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:28.823839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553229+0000","last_change":"2024-09-21T06:09:47.557467+0000","last_active":"2024-09-21T06:09:48.553229+0000","last_peered":"2024-09-21T06:09:48.553229+0000","last_clean":"2024-09-21T06:09:48.553229+0000","last_became_active":"2024-09-21T06:09:47.554901+0000","last_became_peered":"2024-09-21T06:09:47.554901+0000","last_unstale":"2024-09-21T06:09:48.553229+0000","last_undegraded":"2024-09-21T06:09:48.553229+0000","last_fullsized":"2024-09-21T06:09:48.553229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:31.714893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034452+0000","last_change":"2024-09-21T06:09:47.556163+0000","last_active":"2024-09-21T06:09:49.034452+0000","last_peered":"2024-09-21T06:09:49.034452+0000","last_clean":"2024-09-21T06:09:49.034452+0000","last_became_active":"2024-09-21T06:09:47.555656+0000","last_became_peered":"2024-09-21T06:09:47.555656+0000","last_unstale":"2024-09-21T06:09:49.034452+0000","last_undegraded":"2024-09-21T06:09:49.034452+0000","last_fullsized":"2024-09-21T06:09:49.034452+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:31.983551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.099690+0000","last_change":"2024-09-21T06:09:47.554101+0000","last_active":"2024-09-21T06:09:49.099690+0000","last_peered":"2024-09-21T06:09:49.099690+0000","last_clean":"2024-09-21T06:09:49.099690+0000","last_became_active":"2024-09-21T06:09:47.553743+0000","last_became_peered":"2024-09-21T06:09:47.553743+0000","last_unstale":"2024-09-21T06:09:49.099690+0000","last_undegraded":"2024-09-21T06:09:49.099690+0000","last_fullsized":"2024-09-21T06:09:49.099690+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:54.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.099243+0000","last_change":"2024-09-21T06:09:47.554855+0000","last_active":"2024-09-21T06:09:49.099243+0000","last_peered":"2024-09-21T06:09:49.099243+0000","last_clean":"2024-09-21T06:09:49.099243+0000","last_became_active":"2024-09-21T06:09:47.554540+0000","last_became_peered":"2024-09-21T06:09:47.554540+0000","last_unstale":"2024-09-21T06:09:49.099243+0000","last_undegraded":"2024-09-21T06:09:49.099243+0000","last_fullsized":"2024-09-21T06:09:49.099243+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:53.879153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535197+0000","last_change":"2024-09-21T06:09:47.557875+0000","last_active":"2024-09-21T06:09:49.535197+0000","last_peered":"2024-09-21T06:09:49.535197+0000","last_clean":"2024-09-21T06:09:49.535197+0000","last_became_active":"2024-09-21T06:09:47.557565+0000","last_became_peered":"2024-09-21T06:09:47.557565+0000","last_unstale":"2024-09-21T06:09:49.535197+0000","last_undegraded":"2024-09-21T06:09:49.535197+0000","last_fullsized":"2024-09-21T06:09:49.535197+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:42.255816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.033999+0000","last_change":"2024-09-21T06:09:47.554644+0000","last_active":"2024-09-21T06:09:49.033999+0000","last_peered":"2024-09-21T06:09:49.033999+0000","last_clean":"2024-09-21T06:09:49.033999+0000","last_became_active":"2024-09-21T06:09:47.554247+0000","last_became_peered":"2024-09-21T06:09:47.554247+0000","last_unstale":"2024-09-21T06:09:49.033999+0000","last_undegraded":"2024-09-21T06:09:49.033999+0000","last_fullsized":"2024-09-21T06:09:49.033999+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:50.610316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552610+0000","last_change":"2024-09-21T06:09:47.557683+0000","last_active":"2024-09-21T06:09:48.552610+0000","last_peered":"2024-09-21T06:09:48.552610+0000","last_clean":"2024-09-21T06:09:48.552610+0000","last_became_active":"2024-09-21T06:09:47.557357+0000","last_became_peered":"2024-09-21T06:09:47.557357+0000","last_unstale":"2024-09-21T06:09:48.552610+0000","last_undegraded":"2024-09-21T06:09:48.552610+0000","last_fullsized":"2024-09-21T06:09:48.552610+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.772775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561742+0000","last_change":"2024-09-21T06:09:47.557188+0000","last_active":"2024-09-21T06:09:48.561742+0000","last_peered":"2024-09-21T06:09:48.561742+0000","last_clean":"2024-09-21T06:09:48.561742+0000","last_became_active":"2024-09-21T06:09:47.556870+0000","last_became_peered":"2024-09-21T06:09:47.556870+0000","last_unstale":"2024-09-21T06:09:48.561742+0000","last_undegraded":"2024-09-21T06:09:48.561742+0000","last_fullsized":"2024-09-21T06:09:48.561742+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:38.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.099691+0000","last_change":"2024-09-21T06:09:47.562338+0000","last_active":"2024-09-21T06:09:49.099691+0000","last_peered":"2024-09-21T06:09:49.099691+0000","last_clean":"2024-09-21T06:09:49.099691+0000","last_became_active":"2024-09-21T06:09:47.561835+0000","last_became_peered":"2024-09-21T06:09:47.561835+0000","last_unstale":"2024-09-21T06:09:49.099691+0000","last_undegraded":"2024-09-21T06:09:49.099691+0000","last_fullsized":"2024-09-21T06:09:49.099691+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:03.841641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552835+0000","last_change":"2024-09-21T06:09:47.548087+0000","last_active":"2024-09-21T06:09:48.552835+0000","last_peered":"2024-09-21T06:09:48.552835+0000","last_clean":"2024-09-21T06:09:48.552835+0000","last_became_active":"2024-09-21T06:09:47.546530+0000","last_became_peered":"2024-09-21T06:09:47.546530+0000","last_unstale":"2024-09-21T06:09:48.552835+0000","last_undegraded":"2024-09-21T06:09:48.552835+0000","last_fullsized":"2024-09-21T06:09:48.552835+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:27.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552595+0000","last_change":"2024-09-21T06:09:47.549010+0000","last_active":"2024-09-21T06:09:48.552595+0000","last_peered":"2024-09-21T06:09:48.552595+0000","last_clean":"2024-09-21T06:09:48.552595+0000","last_became_active":"2024-09-21T06:09:47.548666+0000","last_became_peered":"2024-09-21T06:09:47.548666+0000","last_unstale":"2024-09-21T06:09:48.552595+0000","last_undegraded":"2024-09-21T06:09:48.552595+0000","last_fullsized":"2024-09-21T06:09:48.552595+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551377+0000","last_change":"2024-09-21T06:09:47.554127+0000","last_active":"2024-09-21T06:09:48.551377+0000","last_peered":"2024-09-21T06:09:48.551377+0000","last_clean":"2024-09-21T06:09:48.551377+0000","last_became_active":"2024-09-21T06:09:47.553809+0000","last_became_peered":"2024-09-21T06:09:47.553809+0000","last_unstale":"2024-09-21T06:09:48.551377+0000","last_undegraded":"2024-09-21T06:09:48.551377+0000","last_fullsized":"2024-09-21T06:09:48.551377+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:30.401119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553426+0000","last_change":"2024-09-21T06:09:47.546392+0000","last_active":"2024-09-21T06:09:48.553426+0000","last_peered":"2024-09-21T06:09:48.553426+0000","last_clean":"2024-09-21T06:09:48.553426+0000","last_became_active":"2024-09-21T06:09:47.545962+0000","last_became_peered":"2024-09-21T06:09:47.545962+0000","last_unstale":"2024-09-21T06:09:48.553426+0000","last_undegraded":"2024-09-21T06:09:48.553426+0000","last_fullsized":"2024-09-21T06:09:48.553426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:35.324859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.550989+0000","last_change":"2024-09-21T06:09:47.555478+0000","last_active":"2024-09-21T06:09:48.550989+0000","last_peered":"2024-09-21T06:09:48.550989+0000","last_clean":"2024-09-21T06:09:48.550989+0000","last_became_active":"2024-09-21T06:09:47.555175+0000","last_became_peered":"2024-09-21T06:09:47.555175+0000","last_unstale":"2024-09-21T06:09:48.550989+0000","last_undegraded":"2024-09-21T06:09:48.550989+0000","last_fullsized":"2024-09-21T06:09:48.550989+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:31.714893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552852+0000","last_change":"2024-09-21T06:09:47.557085+0000","last_active":"2024-09-21T06:09:48.552852+0000","last_peered":"2024-09-21T06:09:48.552852+0000","last_clean":"2024-09-21T06:09:48.552852+0000","last_became_active":"2024-09-21T06:09:47.556727+0000","last_became_peered":"2024-09-21T06:09:47.556727+0000","last_unstale":"2024-09-21T06:09:48.552852+0000","last_undegraded":"2024-09-21T06:09:48.552852+0000","last_fullsized":"2024-09-21T06:09:48.552852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:42.840231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034339+0000","last_change":"2024-09-21T06:09:47.559094+0000","last_active":"2024-09-21T06:09:49.034339+0000","last_peered":"2024-09-21T06:09:49.034339+0000","last_clean":"2024-09-21T06:09:49.034339+0000","last_became_active":"2024-09-21T06:09:47.558738+0000","last_became_peered":"2024-09-21T06:09:47.558738+0000","last_unstale":"2024-09-21T06:09:49.034339+0000","last_undegraded":"2024-09-21T06:09:49.034339+0000","last_fullsized":"2024-09-21T06:09:49.034339+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:13.816926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.536232+0000","last_change":"2024-09-21T06:09:47.557791+0000","last_active":"2024-09-21T06:09:49.536232+0000","last_peered":"2024-09-21T06:09:49.536232+0000","last_clean":"2024-09-21T06:09:49.536232+0000","last_became_active":"2024-09-21T06:09:47.557439+0000","last_became_peered":"2024-09-21T06:09:47.557439+0000","last_unstale":"2024-09-21T06:09:49.536232+0000","last_undegraded":"2024-09-21T06:09:49.536232+0000","last_fullsized":"2024-09-21T06:09:49.536232+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:17.327923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551657+0000","last_change":"2024-09-21T06:09:47.552743+0000","last_active":"2024-09-21T06:09:48.551657+0000","last_peered":"2024-09-21T06:09:48.551657+0000","last_clean":"2024-09-21T06:09:48.551657+0000","last_became_active":"2024-09-21T06:09:47.552402+0000","last_became_peered":"2024-09-21T06:09:47.552402+0000","last_unstale":"2024-09-21T06:09:48.551657+0000","last_undegraded":"2024-09-21T06:09:48.551657+0000","last_fullsized":"2024-09-21T06:09:48.551657+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:54.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552061+0000","last_change":"2024-09-21T06:09:47.547136+0000","last_active":"2024-09-21T06:09:48.552061+0000","last_peered":"2024-09-21T06:09:48.552061+0000","last_clean":"2024-09-21T06:09:48.552061+0000","last_became_active":"2024-09-21T06:09:47.546771+0000","last_became_peered":"2024-09-21T06:09:47.546771+0000","last_unstale":"2024-09-21T06:09:48.552061+0000","last_undegraded":"2024-09-21T06:09:48.552061+0000","last_fullsized":"2024-09-21T06:09:48.552061+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:09.595527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553225+0000","last_change":"2024-09-21T06:09:47.550506+0000","last_active":"2024-09-21T06:09:48.553225+0000","last_peered":"2024-09-21T06:09:48.553225+0000","last_clean":"2024-09-21T06:09:48.553225+0000","last_became_active":"2024-09-21T06:09:47.550172+0000","last_became_peered":"2024-09-21T06:09:47.550172+0000","last_unstale":"2024-09-21T06:09:48.553225+0000","last_undegraded":"2024-09-21T06:09:48.553225+0000","last_fullsized":"2024-09-21T06:09:48.553225+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.338192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034058+0000","last_change":"2024-09-21T06:09:47.548514+0000","last_active":"2024-09-21T06:09:49.034058+0000","last_peered":"2024-09-21T06:09:49.034058+0000","last_clean":"2024-09-21T06:09:49.034058+0000","last_became_active":"2024-09-21T06:09:47.548034+0000","last_became_peered":"2024-09-21T06:09:47.548034+0000","last_unstale":"2024-09-21T06:09:49.034058+0000","last_undegraded":"2024-09-21T06:09:49.034058+0000","last_fullsized":"2024-09-21T06:09:49.034058+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:40.762951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535081+0000","last_change":"2024-09-21T06:09:47.562416+0000","last_active":"2024-09-21T06:09:49.535081+0000","last_peered":"2024-09-21T06:09:49.535081+0000","last_clean":"2024-09-21T06:09:49.535081+0000","last_became_active":"2024-09-21T06:09:47.562061+0000","last_became_peered":"2024-09-21T06:09:47.562061+0000","last_unstale":"2024-09-21T06:09:49.535081+0000","last_undegraded":"2024-09-21T06:09:49.535081+0000","last_fullsized":"2024-09-21T06:09:49.535081+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:55.633603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.536401+0000","last_change":"2024-09-21T06:09:47.550815+0000","last_active":"2024-09-21T06:09:49.536401+0000","last_peered":"2024-09-21T06:09:49.536401+0000","last_clean":"2024-09-21T06:09:49.536401+0000","last_became_active":"2024-09-21T06:09:47.548731+0000","last_became_peered":"2024-09-21T06:09:47.548731+0000","last_unstale":"2024-09-21T06:09:49.536401+0000","last_undegraded":"2024-09-21T06:09:49.536401+0000","last_fullsized":"2024-09-21T06:09:49.536401+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:58.303903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553791+0000","last_change":"2024-09-21T06:09:46.528315+0000","last_active":"2024-09-21T06:09:48.553791+0000","last_peered":"2024-09-21T06:09:48.553791+0000","last_clean":"2024-09-21T06:09:48.553791+0000","last_became_active":"2024-09-21T06:09:46.527949+0000","last_became_peered":"2024-09-21T06:09:46.527949+0000","last_unstale":"2024-09-21T06:09:48.553791+0000","last_undegraded":"2024-09-21T06:09:48.553791+0000","last_fullsized":"2024-09-21T06:09:48.553791+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:23.941639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561877+0000","last_change":"2024-09-21T06:09:47.553802+0000","last_active":"2024-09-21T06:09:48.561877+0000","last_peered":"2024-09-21T06:09:48.561877+0000","last_clean":"2024-09-21T06:09:48.561877+0000","last_became_active":"2024-09-21T06:09:47.553503+0000","last_became_peered":"2024-09-21T06:09:47.553503+0000","last_unstale":"2024-09-21T06:09:48.561877+0000","last_undegraded":"2024-09-21T06:09:48.561877+0000","last_fullsized":"2024-09-21T06:09:48.561877+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:58.813046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.099399+0000","last_change":"2024-09-21T06:09:46.531370+0000","last_active":"2024-09-21T06:09:49.099399+0000","last_peered":"2024-09-21T06:09:49.099399+0000","last_clean":"2024-09-21T06:09:49.099399+0000","last_became_active":"2024-09-21T06:09:46.530960+0000","last_became_peered":"2024-09-21T06:09:46.530960+0000","last_unstale":"2024-09-21T06:09:49.099399+0000","last_undegraded":"2024-09-21T06:09:49.099399+0000","last_fullsized":"2024-09-21T06:09:49.099399+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.536422+0000","last_change":"2024-09-21T06:09:47.557244+0000","last_active":"2024-09-21T06:09:49.536422+0000","last_peered":"2024-09-21T06:09:49.536422+0000","last_clean":"2024-09-21T06:09:49.536422+0000","last_became_active":"2024-09-21T06:09:47.556855+0000","last_became_peered":"2024-09-21T06:09:47.556855+0000","last_unstale":"2024-09-21T06:09:49.536422+0000","last_undegraded":"2024-09-21T06:09:49.536422+0000","last_fullsized":"2024-09-21T06:09:49.536422+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:00.301002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535604+0000","last_change":"2024-09-21T06:09:46.544553+0000","last_active":"2024-09-21T06:09:49.535604+0000","last_peered":"2024-09-21T06:09:49.535604+0000","last_clean":"2024-09-21T06:09:49.535604+0000","last_became_active":"2024-09-21T06:09:46.544246+0000","last_became_peered":"2024-09-21T06:09:46.544246+0000","last_unstale":"2024-09-21T06:09:49.535604+0000","last_undegraded":"2024-09-21T06:09:49.535604+0000","last_fullsized":"2024-09-21T06:09:49.535604+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.718740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.099998+0000","last_change":"2024-09-21T06:09:47.569840+0000","last_active":"2024-09-21T06:09:49.099998+0000","last_peered":"2024-09-21T06:09:49.099998+0000","last_clean":"2024-09-21T06:09:49.099998+0000","last_became_active":"2024-09-21T06:09:47.569541+0000","last_became_peered":"2024-09-21T06:09:47.569541+0000","last_unstale":"2024-09-21T06:09:49.099998+0000","last_undegraded":"2024-09-21T06:09:49.099998+0000","last_fullsized":"2024-09-21T06:09:49.099998+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:05.811070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535544+0000","last_change":"2024-09-21T06:09:46.543348+0000","last_active":"2024-09-21T06:09:49.535544+0000","last_peered":"2024-09-21T06:09:49.535544+0000","last_clean":"2024-09-21T06:09:49.535544+0000","last_became_active":"2024-09-21T06:09:46.543036+0000","last_became_peered":"2024-09-21T06:09:46.543036+0000","last_unstale":"2024-09-21T06:09:49.535544+0000","last_undegraded":"2024-09-21T06:09:49.535544+0000","last_fullsized":"2024-09-21T06:09:49.535544+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:17.178909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034630+0000","last_change":"2024-09-21T06:09:47.558350+0000","last_active":"2024-09-21T06:09:49.034630+0000","last_peered":"2024-09-21T06:09:49.034630+0000","last_clean":"2024-09-21T06:09:49.034630+0000","last_became_active":"2024-09-21T06:09:47.552686+0000","last_became_peered":"2024-09-21T06:09:47.552686+0000","last_unstale":"2024-09-21T06:09:49.034630+0000","last_undegraded":"2024-09-21T06:09:49.034630+0000","last_fullsized":"2024-09-21T06:09:49.034630+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:25.118514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553484+0000","last_change":"2024-09-21T06:09:46.535845+0000","last_active":"2024-09-21T06:09:48.553484+0000","last_peered":"2024-09-21T06:09:48.553484+0000","last_clean":"2024-09-21T06:09:48.553484+0000","last_became_active":"2024-09-21T06:09:46.535462+0000","last_became_peered":"2024-09-21T06:09:46.535462+0000","last_unstale":"2024-09-21T06:09:48.553484+0000","last_undegraded":"2024-09-21T06:09:48.553484+0000","last_fullsized":"2024-09-21T06:09:48.553484+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:32.585856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553096+0000","last_change":"2024-09-21T06:09:47.556065+0000","last_active":"2024-09-21T06:09:48.553096+0000","last_peered":"2024-09-21T06:09:48.553096+0000","last_clean":"2024-09-21T06:09:48.553096+0000","last_became_active":"2024-09-21T06:09:47.555742+0000","last_became_peered":"2024-09-21T06:09:47.555742+0000","last_unstale":"2024-09-21T06:09:48.553096+0000","last_undegraded":"2024-09-21T06:09:48.553096+0000","last_fullsized":"2024-09-21T06:09:48.553096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:51.996277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551569+0000","last_change":"2024-09-21T06:09:46.549487+0000","last_active":"2024-09-21T06:09:48.551569+0000","last_peered":"2024-09-21T06:09:48.551569+0000","last_clean":"2024-09-21T06:09:48.551569+0000","last_became_active":"2024-09-21T06:09:46.548860+0000","last_became_peered":"2024-09-21T06:09:46.548860+0000","last_unstale":"2024-09-21T06:09:48.551569+0000","last_undegraded":"2024-09-21T06:09:48.551569+0000","last_fullsized":"2024-09-21T06:09:48.551569+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:45.648740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034000+0000","last_change":"2024-09-21T06:09:47.563169+0000","last_active":"2024-09-21T06:09:49.034000+0000","last_peered":"2024-09-21T06:09:49.034000+0000","last_clean":"2024-09-21T06:09:49.034000+0000","last_became_active":"2024-09-21T06:09:47.562868+0000","last_became_peered":"2024-09-21T06:09:47.562868+0000","last_unstale":"2024-09-21T06:09:49.034000+0000","last_undegraded":"2024-09-21T06:09:49.034000+0000","last_fullsized":"2024-09-21T06:09:49.034000+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:40.462715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551911+0000","last_change":"2024-09-21T06:09:46.545661+0000","last_active":"2024-09-21T06:09:48.551911+0000","last_peered":"2024-09-21T06:09:48.551911+0000","last_clean":"2024-09-21T06:09:48.551911+0000","last_became_active":"2024-09-21T06:09:46.545340+0000","last_became_peered":"2024-09-21T06:09:46.545340+0000","last_unstale":"2024-09-21T06:09:48.551911+0000","last_undegraded":"2024-09-21T06:09:48.551911+0000","last_fullsized":"2024-09-21T06:09:48.551911+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:47.880211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552001+0000","last_change":"2024-09-21T06:09:47.553054+0000","last_active":"2024-09-21T06:09:48.552001+0000","last_peered":"2024-09-21T06:09:48.552001+0000","last_clean":"2024-09-21T06:09:48.552001+0000","last_became_active":"2024-09-21T06:09:47.552742+0000","last_became_peered":"2024-09-21T06:09:47.552742+0000","last_unstale":"2024-09-21T06:09:48.552001+0000","last_undegraded":"2024-09-21T06:09:48.552001+0000","last_fullsized":"2024-09-21T06:09:48.552001+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:54.333587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034304+0000","last_change":"2024-09-21T06:09:46.536718+0000","last_active":"2024-09-21T06:09:49.034304+0000","last_peered":"2024-09-21T06:09:49.034304+0000","last_clean":"2024-09-21T06:09:49.034304+0000","last_became_active":"2024-09-21T06:09:46.535034+0000","last_became_peered":"2024-09-21T06:09:46.535034+0000","last_unstale":"2024-09-21T06:09:49.034304+0000","last_undegraded":"2024-09-21T06:09:49.034304+0000","last_fullsized":"2024-09-21T06:09:49.034304+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:49.688976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553944+0000","last_change":"2024-09-21T06:09:47.539900+0000","last_active":"2024-09-21T06:09:48.553944+0000","last_peered":"2024-09-21T06:09:48.553944+0000","last_clean":"2024-09-21T06:09:48.553944+0000","last_became_active":"2024-09-21T06:09:47.539412+0000","last_became_peered":"2024-09-21T06:09:47.539412+0000","last_unstale":"2024-09-21T06:09:48.553944+0000","last_undegraded":"2024-09-21T06:09:48.553944+0000","last_fullsized":"2024-09-21T06:09:48.553944+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:52.206476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535544+0000","last_change":"2024-09-21T06:09:46.540269+0000","last_active":"2024-09-21T06:09:49.535544+0000","last_peered":"2024-09-21T06:09:49.535544+0000","last_clean":"2024-09-21T06:09:49.535544+0000","last_became_active":"2024-09-21T06:09:46.539954+0000","last_became_peered":"2024-09-21T06:09:46.539954+0000","last_unstale":"2024-09-21T06:09:49.535544+0000","last_undegraded":"2024-09-21T06:09:49.535544+0000","last_fullsized":"2024-09-21T06:09:49.535544+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:13.428634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.562059+0000","last_change":"2024-09-21T06:09:47.566984+0000","last_active":"2024-09-21T06:09:48.562059+0000","last_peered":"2024-09-21T06:09:48.562059+0000","last_clean":"2024-09-21T06:09:48.562059+0000","last_became_active":"2024-09-21T06:09:47.566193+0000","last_became_peered":"2024-09-21T06:09:47.566193+0000","last_unstale":"2024-09-21T06:09:48.562059+0000","last_undegraded":"2024-09-21T06:09:48.562059+0000","last_fullsized":"2024-09-21T06:09:48.562059+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:28.501310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551374+0000","last_change":"2024-09-21T06:09:46.536342+0000","last_active":"2024-09-21T06:09:48.551374+0000","last_peered":"2024-09-21T06:09:48.551374+0000","last_clean":"2024-09-21T06:09:48.551374+0000","last_became_active":"2024-09-21T06:09:46.536012+0000","last_became_peered":"2024-09-21T06:09:46.536012+0000","last_unstale":"2024-09-21T06:09:48.551374+0000","last_undegraded":"2024-09-21T06:09:48.551374+0000","last_fullsized":"2024-09-21T06:09:48.551374+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:57.383442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551680+0000","last_change":"2024-09-21T06:09:47.543982+0000","last_active":"2024-09-21T06:09:48.551680+0000","last_peered":"2024-09-21T06:09:48.551680+0000","last_clean":"2024-09-21T06:09:48.551680+0000","last_became_active":"2024-09-21T06:09:47.543662+0000","last_became_peered":"2024-09-21T06:09:47.543662+0000","last_unstale":"2024-09-21T06:09:48.551680+0000","last_undegraded":"2024-09-21T06:09:48.551680+0000","last_fullsized":"2024-09-21T06:09:48.551680+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:03.841641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034762+0000","last_change":"2024-09-21T06:09:46.548929+0000","last_active":"2024-09-21T06:09:49.034762+0000","last_peered":"2024-09-21T06:09:49.034762+0000","last_clean":"2024-09-21T06:09:49.034762+0000","last_became_active":"2024-09-21T06:09:46.548521+0000","last_became_peered":"2024-09-21T06:09:46.548521+0000","last_unstale":"2024-09-21T06:09:49.034762+0000","last_undegraded":"2024-09-21T06:09:49.034762+0000","last_fullsized":"2024-09-21T06:09:49.034762+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:27.494160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553855+0000","last_change":"2024-09-21T06:09:47.554124+0000","last_active":"2024-09-21T06:09:48.553855+0000","last_peered":"2024-09-21T06:09:48.553855+0000","last_clean":"2024-09-21T06:09:48.553855+0000","last_became_active":"2024-09-21T06:09:47.551484+0000","last_became_peered":"2024-09-21T06:09:47.551484+0000","last_unstale":"2024-09-21T06:09:48.553855+0000","last_undegraded":"2024-09-21T06:09:48.553855+0000","last_fullsized":"2024-09-21T06:09:48.553855+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:47.621204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":21,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034758+0000","last_change":"2024-09-21T06:09:46.539077+0000","last_active":"2024-09-21T06:09:49.034758+0000","last_peered":"2024-09-21T06:09:49.034758+0000","last_clean":"2024-09-21T06:09:49.034758+0000","last_became_active":"2024-09-21T06:09:46.537287+0000","last_became_peered":"2024-09-21T06:09:46.537287+0000","last_unstale":"2024-09-21T06:09:49.034758+0000","last_undegraded":"2024-09-21T06:09:49.034758+0000","last_fullsized":"2024-09-21T06:09:49.034758+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:52.872003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551541+0000","last_change":"2024-09-21T06:09:47.546084+0000","last_active":"2024-09-21T06:09:48.551541+0000","last_peered":"2024-09-21T06:09:48.551541+0000","last_clean":"2024-09-21T06:09:48.551541+0000","last_became_active":"2024-09-21T06:09:47.545677+0000","last_became_peered":"2024-09-21T06:09:47.545677+0000","last_unstale":"2024-09-21T06:09:48.551541+0000","last_undegraded":"2024-09-21T06:09:48.551541+0000","last_fullsized":"2024-09-21T06:09:48.551541+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:50.691603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535995+0000","last_change":"2024-09-21T06:09:46.537690+0000","last_active":"2024-09-21T06:09:49.535995+0000","last_peered":"2024-09-21T06:09:49.535995+0000","last_clean":"2024-09-21T06:09:49.535995+0000","last_became_active":"2024-09-21T06:09:46.537157+0000","last_became_peered":"2024-09-21T06:09:46.537157+0000","last_unstale":"2024-09-21T06:09:49.535995+0000","last_undegraded":"2024-09-21T06:09:49.535995+0000","last_fullsized":"2024-09-21T06:09:49.535995+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:11.151516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551690+0000","last_change":"2024-09-21T06:09:47.550402+0000","last_active":"2024-09-21T06:09:48.551690+0000","last_peered":"2024-09-21T06:09:48.551690+0000","last_clean":"2024-09-21T06:09:48.551690+0000","last_became_active":"2024-09-21T06:09:47.550085+0000","last_became_peered":"2024-09-21T06:09:47.550085+0000","last_unstale":"2024-09-21T06:09:48.551690+0000","last_undegraded":"2024-09-21T06:09:48.551690+0000","last_fullsized":"2024-09-21T06:09:48.551690+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:48.971220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561520+0000","last_change":"2024-09-21T06:09:46.524911+0000","last_active":"2024-09-21T06:09:48.561520+0000","last_peered":"2024-09-21T06:09:48.561520+0000","last_clean":"2024-09-21T06:09:48.561520+0000","last_became_active":"2024-09-21T06:09:46.524561+0000","last_became_peered":"2024-09-21T06:09:46.524561+0000","last_unstale":"2024-09-21T06:09:48.561520+0000","last_undegraded":"2024-09-21T06:09:48.561520+0000","last_fullsized":"2024-09-21T06:09:48.561520+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:32.557867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551083+0000","last_change":"2024-09-21T06:09:47.541745+0000","last_active":"2024-09-21T06:09:48.551083+0000","last_peered":"2024-09-21T06:09:48.551083+0000","last_clean":"2024-09-21T06:09:48.551083+0000","last_became_active":"2024-09-21T06:09:47.541184+0000","last_became_peered":"2024-09-21T06:09:47.541184+0000","last_unstale":"2024-09-21T06:09:48.551083+0000","last_undegraded":"2024-09-21T06:09:48.551083+0000","last_fullsized":"2024-09-21T06:09:48.551083+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:27.860396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":23,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.534994+0000","last_change":"2024-09-21T06:09:46.533914+0000","last_active":"2024-09-21T06:09:49.534994+0000","last_peered":"2024-09-21T06:09:49.534994+0000","last_clean":"2024-09-21T06:09:49.534994+0000","last_became_active":"2024-09-21T06:09:46.533541+0000","last_became_peered":"2024-09-21T06:09:46.533541+0000","last_unstale":"2024-09-21T06:09:49.534994+0000","last_undegraded":"2024-09-21T06:09:49.534994+0000","last_fullsized":"2024-09-21T06:09:49.534994+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:24.340748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535819+0000","last_change":"2024-09-21T06:09:47.557155+0000","last_active":"2024-09-21T06:09:49.535819+0000","last_peered":"2024-09-21T06:09:49.535819+0000","last_clean":"2024-09-21T06:09:49.535819+0000","last_became_active":"2024-09-21T06:09:47.556834+0000","last_became_peered":"2024-09-21T06:09:47.556834+0000","last_unstale":"2024-09-21T06:09:49.535819+0000","last_undegraded":"2024-09-21T06:09:49.535819+0000","last_fullsized":"2024-09-21T06:09:49.535819+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:32.757916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.035043+0000","last_change":"2024-09-21T06:09:46.546891+0000","last_active":"2024-09-21T06:09:49.035043+0000","last_peered":"2024-09-21T06:09:49.035043+0000","last_clean":"2024-09-21T06:09:49.035043+0000","last_became_active":"2024-09-21T06:09:46.546573+0000","last_became_peered":"2024-09-21T06:09:46.546573+0000","last_unstale":"2024-09-21T06:09:49.035043+0000","last_undegraded":"2024-09-21T06:09:49.035043+0000","last_fullsized":"2024-09-21T06:09:49.035043+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:17.766821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.536371+0000","last_change":"2024-09-21T06:09:47.554707+0000","last_active":"2024-09-21T06:09:49.536371+0000","last_peered":"2024-09-21T06:09:49.536371+0000","last_clean":"2024-09-21T06:09:49.536371+0000","last_became_active":"2024-09-21T06:09:47.554388+0000","last_became_peered":"2024-09-21T06:09:47.554388+0000","last_unstale":"2024-09-21T06:09:49.536371+0000","last_undegraded":"2024-09-21T06:09:49.536371+0000","last_fullsized":"2024-09-21T06:09:49.536371+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:50.405611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535767+0000","last_change":"2024-09-21T06:09:46.529780+0000","last_active":"2024-09-21T06:09:49.535767+0000","last_peered":"2024-09-21T06:09:49.535767+0000","last_clean":"2024-09-21T06:09:49.535767+0000","last_became_active":"2024-09-21T06:09:46.528136+0000","last_became_peered":"2024-09-21T06:09:46.528136+0000","last_unstale":"2024-09-21T06:09:49.535767+0000","last_undegraded":"2024-09-21T06:09:49.535767+0000","last_fullsized":"2024-09-21T06:09:49.535767+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:17.381392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.099659+0000","last_change":"2024-09-21T06:09:47.567084+0000","last_active":"2024-09-21T06:09:49.099659+0000","last_peered":"2024-09-21T06:09:49.099659+0000","last_clean":"2024-09-21T06:09:49.099659+0000","last_became_active":"2024-09-21T06:09:47.566717+0000","last_became_peered":"2024-09-21T06:09:47.566717+0000","last_unstale":"2024-09-21T06:09:49.099659+0000","last_undegraded":"2024-09-21T06:09:49.099659+0000","last_fullsized":"2024-09-21T06:09:49.099659+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:54.401707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561699+0000","last_change":"2024-09-21T06:09:46.524059+0000","last_active":"2024-09-21T06:09:48.561699+0000","last_peered":"2024-09-21T06:09:48.561699+0000","last_clean":"2024-09-21T06:09:48.561699+0000","last_became_active":"2024-09-21T06:09:46.523459+0000","last_became_peered":"2024-09-21T06:09:46.523459+0000","last_unstale":"2024-09-21T06:09:48.561699+0000","last_undegraded":"2024-09-21T06:09:48.561699+0000","last_fullsized":"2024-09-21T06:09:48.561699+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.100030+0000","last_change":"2024-09-21T06:09:47.561252+0000","last_active":"2024-09-21T06:09:49.100030+0000","last_peered":"2024-09-21T06:09:49.100030+0000","last_clean":"2024-09-21T06:09:49.100030+0000","last_became_active":"2024-09-21T06:09:47.560849+0000","last_became_peered":"2024-09-21T06:09:47.560849+0000","last_unstale":"2024-09-21T06:09:49.100030+0000","last_undegraded":"2024-09-21T06:09:49.100030+0000","last_fullsized":"2024-09-21T06:09:49.100030+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:39.640902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553549+0000","last_change":"2024-09-21T06:09:46.534239+0000","last_active":"2024-09-21T06:09:48.553549+0000","last_peered":"2024-09-21T06:09:48.553549+0000","last_clean":"2024-09-21T06:09:48.553549+0000","last_became_active":"2024-09-21T06:09:46.533715+0000","last_became_peered":"2024-09-21T06:09:46.533715+0000","last_unstale":"2024-09-21T06:09:48.553549+0000","last_undegraded":"2024-09-21T06:09:48.553549+0000","last_fullsized":"2024-09-21T06:09:48.553549+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:08:04.897143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.099280+0000","last_change":"2024-09-21T06:09:47.552337+0000","last_active":"2024-09-21T06:09:49.099280+0000","last_peered":"2024-09-21T06:09:49.099280+0000","last_clean":"2024-09-21T06:09:49.099280+0000","last_became_active":"2024-09-21T06:09:47.552009+0000","last_became_peered":"2024-09-21T06:09:47.552009+0000","last_unstale":"2024-09-21T06:09:49.099280+0000","last_undegraded":"2024-09-21T06:09:49.099280+0000","last_fullsized":"2024-09-21T06:09:49.099280+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:35.324859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.536021+0000","last_change":"2024-09-21T06:09:46.536200+0000","last_active":"2024-09-21T06:09:49.536021+0000","last_peered":"2024-09-21T06:09:49.536021+0000","last_clean":"2024-09-21T06:09:49.536021+0000","last_became_active":"2024-09-21T06:09:46.535894+0000","last_became_peered":"2024-09-21T06:09:46.535894+0000","last_unstale":"2024-09-21T06:09:49.536021+0000","last_undegraded":"2024-09-21T06:09:49.536021+0000","last_fullsized":"2024-09-21T06:09:49.536021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:20.365744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034091+0000","last_change":"2024-09-21T06:09:47.552189+0000","last_active":"2024-09-21T06:09:49.034091+0000","last_peered":"2024-09-21T06:09:49.034091+0000","last_clean":"2024-09-21T06:09:49.034091+0000","last_became_active":"2024-09-21T06:09:47.548927+0000","last_became_peered":"2024-09-21T06:09:47.548927+0000","last_unstale":"2024-09-21T06:09:49.034091+0000","last_undegraded":"2024-09-21T06:09:49.034091+0000","last_fullsized":"2024-09-21T06:09:49.034091+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:36.277071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535060+0000","last_change":"2024-09-21T06:09:46.526945+0000","last_active":"2024-09-21T06:09:49.535060+0000","last_peered":"2024-09-21T06:09:49.535060+0000","last_clean":"2024-09-21T06:09:49.535060+0000","last_became_active":"2024-09-21T06:09:46.526588+0000","last_became_peered":"2024-09-21T06:09:46.526588+0000","last_unstale":"2024-09-21T06:09:49.535060+0000","last_undegraded":"2024-09-21T06:09:49.535060+0000","last_fullsized":"2024-09-21T06:09:49.535060+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:14.014304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.099274+0000","last_change":"2024-09-21T06:09:47.555659+0000","last_active":"2024-09-21T06:09:49.099274+0000","last_peered":"2024-09-21T06:09:49.099274+0000","last_clean":"2024-09-21T06:09:49.099274+0000","last_became_active":"2024-09-21T06:09:47.555358+0000","last_became_peered":"2024-09-21T06:09:47.555358+0000","last_unstale":"2024-09-21T06:09:49.099274+0000","last_undegraded":"2024-09-21T06:09:49.099274+0000","last_fullsized":"2024-09-21T06:09:49.099274+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:47.209532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535295+0000","last_change":"2024-09-21T06:09:46.541688+0000","last_active":"2024-09-21T06:09:49.535295+0000","last_peered":"2024-09-21T06:09:49.535295+0000","last_clean":"2024-09-21T06:09:49.535295+0000","last_became_active":"2024-09-21T06:09:46.541322+0000","last_became_peered":"2024-09-21T06:09:46.541322+0000","last_unstale":"2024-09-21T06:09:49.535295+0000","last_undegraded":"2024-09-21T06:09:49.535295+0000","last_fullsized":"2024-09-21T06:09:49.535295+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:37.600477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551290+0000","last_change":"2024-09-21T06:09:47.551665+0000","last_active":"2024-09-21T06:09:48.551290+0000","last_peered":"2024-09-21T06:09:48.551290+0000","last_clean":"2024-09-21T06:09:48.551290+0000","last_became_active":"2024-09-21T06:09:47.551313+0000","last_became_peered":"2024-09-21T06:09:47.551313+0000","last_unstale":"2024-09-21T06:09:48.551290+0000","last_undegraded":"2024-09-21T06:09:48.551290+0000","last_fullsized":"2024-09-21T06:09:48.551290+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:54.251777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034446+0000","last_change":"2024-09-21T06:09:46.545814+0000","last_active":"2024-09-21T06:09:49.034446+0000","last_peered":"2024-09-21T06:09:49.034446+0000","last_clean":"2024-09-21T06:09:49.034446+0000","last_became_active":"2024-09-21T06:09:46.545413+0000","last_became_peered":"2024-09-21T06:09:46.545413+0000","last_unstale":"2024-09-21T06:09:49.034446+0000","last_undegraded":"2024-09-21T06:09:49.034446+0000","last_fullsized":"2024-09-21T06:09:49.034446+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:13.029274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561834+0000","last_change":"2024-09-21T06:09:47.566967+0000","last_active":"2024-09-21T06:09:48.561834+0000","last_peered":"2024-09-21T06:09:48.561834+0000","last_clean":"2024-09-21T06:09:48.561834+0000","last_became_active":"2024-09-21T06:09:47.566211+0000","last_became_peered":"2024-09-21T06:09:47.566211+0000","last_unstale":"2024-09-21T06:09:48.561834+0000","last_undegraded":"2024-09-21T06:09:48.561834+0000","last_fullsized":"2024-09-21T06:09:48.561834+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:14.036424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551922+0000","last_change":"2024-09-21T06:09:46.528896+0000","last_active":"2024-09-21T06:09:48.551922+0000","last_peered":"2024-09-21T06:09:48.551922+0000","last_clean":"2024-09-21T06:09:48.551922+0000","last_became_active":"2024-09-21T06:09:46.528590+0000","last_became_peered":"2024-09-21T06:09:46.528590+0000","last_unstale":"2024-09-21T06:09:48.551922+0000","last_undegraded":"2024-09-21T06:09:48.551922+0000","last_fullsized":"2024-09-21T06:09:48.551922+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:14.043429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034983+0000","last_change":"2024-09-21T06:09:47.556097+0000","last_active":"2024-09-21T06:09:49.034983+0000","last_peered":"2024-09-21T06:09:49.034983+0000","last_clean":"2024-09-21T06:09:49.034983+0000","last_became_active":"2024-09-21T06:09:47.555720+0000","last_became_peered":"2024-09-21T06:09:47.555720+0000","last_unstale":"2024-09-21T06:09:49.034983+0000","last_undegraded":"2024-09-21T06:09:49.034983+0000","last_fullsized":"2024-09-21T06:09:49.034983+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:27.096555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.563317+0000","last_change":"2024-09-21T06:09:46.525404+0000","last_active":"2024-09-21T06:09:48.563317+0000","last_peered":"2024-09-21T06:09:48.563317+0000","last_clean":"2024-09-21T06:09:48.563317+0000","last_became_active":"2024-09-21T06:09:46.525087+0000","last_became_peered":"2024-09-21T06:09:46.525087+0000","last_unstale":"2024-09-21T06:09:48.563317+0000","last_undegraded":"2024-09-21T06:09:48.563317+0000","last_fullsized":"2024-09-21T06:09:48.563317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:57.383442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034738+0000","last_change":"2024-09-21T06:09:47.565828+0000","last_active":"2024-09-21T06:09:49.034738+0000","last_peered":"2024-09-21T06:09:49.034738+0000","last_clean":"2024-09-21T06:09:49.034738+0000","last_became_active":"2024-09-21T06:09:47.565467+0000","last_became_peered":"2024-09-21T06:09:47.565467+0000","last_unstale":"2024-09-21T06:09:49.034738+0000","last_undegraded":"2024-09-21T06:09:49.034738+0000","last_fullsized":"2024-09-21T06:09:49.034738+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:50.298161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034794+0000","last_change":"2024-09-21T06:09:46.542691+0000","last_active":"2024-09-21T06:09:49.034794+0000","last_peered":"2024-09-21T06:09:49.034794+0000","last_clean":"2024-09-21T06:09:49.034794+0000","last_became_active":"2024-09-21T06:09:46.542293+0000","last_became_peered":"2024-09-21T06:09:46.542293+0000","last_unstale":"2024-09-21T06:09:49.034794+0000","last_undegraded":"2024-09-21T06:09:49.034794+0000","last_fullsized":"2024-09-21T06:09:49.034794+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:33.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.561749+0000","last_change":"2024-09-21T06:09:47.576750+0000","last_active":"2024-09-21T06:09:48.561749+0000","last_peered":"2024-09-21T06:09:48.561749+0000","last_clean":"2024-09-21T06:09:48.561749+0000","last_became_active":"2024-09-21T06:09:47.576443+0000","last_became_peered":"2024-09-21T06:09:47.576443+0000","last_unstale":"2024-09-21T06:09:48.561749+0000","last_undegraded":"2024-09-21T06:09:48.561749+0000","last_fullsized":"2024-09-21T06:09:48.561749+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:49.257686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551844+0000","last_change":"2024-09-21T06:09:46.547408+0000","last_active":"2024-09-21T06:09:48.551844+0000","last_peered":"2024-09-21T06:09:48.551844+0000","last_clean":"2024-09-21T06:09:48.551844+0000","last_became_active":"2024-09-21T06:09:46.547043+0000","last_became_peered":"2024-09-21T06:09:46.547043+0000","last_unstale":"2024-09-21T06:09:48.551844+0000","last_undegraded":"2024-09-21T06:09:48.551844+0000","last_fullsized":"2024-09-21T06:09:48.551844+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:08.588377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552144+0000","last_change":"2024-09-21T06:09:47.540415+0000","last_active":"2024-09-21T06:09:48.552144+0000","last_peered":"2024-09-21T06:09:48.552144+0000","last_clean":"2024-09-21T06:09:48.552144+0000","last_became_active":"2024-09-21T06:09:47.539926+0000","last_became_peered":"2024-09-21T06:09:47.539926+0000","last_unstale":"2024-09-21T06:09:48.552144+0000","last_undegraded":"2024-09-21T06:09:48.552144+0000","last_fullsized":"2024-09-21T06:09:48.552144+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:06.724345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.553533+0000","last_change":"2024-09-21T06:09:46.529617+0000","last_active":"2024-09-21T06:09:48.553533+0000","last_peered":"2024-09-21T06:09:48.553533+0000","last_clean":"2024-09-21T06:09:48.553533+0000","last_became_active":"2024-09-21T06:09:46.528234+0000","last_became_peered":"2024-09-21T06:09:46.528234+0000","last_unstale":"2024-09-21T06:09:48.553533+0000","last_undegraded":"2024-09-21T06:09:48.553533+0000","last_fullsized":"2024-09-21T06:09:48.553533+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:19.814656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552277+0000","last_change":"2024-09-21T06:09:47.543597+0000","last_active":"2024-09-21T06:09:48.552277+0000","last_peered":"2024-09-21T06:09:48.552277+0000","last_clean":"2024-09-21T06:09:48.552277+0000","last_became_active":"2024-09-21T06:09:47.543282+0000","last_became_peered":"2024-09-21T06:09:47.543282+0000","last_unstale":"2024-09-21T06:09:48.552277+0000","last_undegraded":"2024-09-21T06:09:48.552277+0000","last_fullsized":"2024-09-21T06:09:48.552277+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:27.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552294+0000","last_change":"2024-09-21T06:09:46.535746+0000","last_active":"2024-09-21T06:09:48.552294+0000","last_peered":"2024-09-21T06:09:48.552294+0000","last_clean":"2024-09-21T06:09:48.552294+0000","last_became_active":"2024-09-21T06:09:46.535142+0000","last_became_peered":"2024-09-21T06:09:46.535142+0000","last_unstale":"2024-09-21T06:09:48.552294+0000","last_undegraded":"2024-09-21T06:09:48.552294+0000","last_fullsized":"2024-09-21T06:09:48.552294+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:56.819951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.099780+0000","last_change":"2024-09-21T06:09:47.553873+0000","last_active":"2024-09-21T06:09:49.099780+0000","last_peered":"2024-09-21T06:09:49.099780+0000","last_clean":"2024-09-21T06:09:49.099780+0000","last_became_active":"2024-09-21T06:09:47.553515+0000","last_became_peered":"2024-09-21T06:09:47.553515+0000","last_unstale":"2024-09-21T06:09:49.099780+0000","last_undegraded":"2024-09-21T06:09:49.099780+0000","last_fullsized":"2024-09-21T06:09:49.099780+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:44.553253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.535163+0000","last_change":"2024-09-21T06:09:46.539038+0000","last_active":"2024-09-21T06:09:49.535163+0000","last_peered":"2024-09-21T06:09:49.535163+0000","last_clean":"2024-09-21T06:09:49.535163+0000","last_became_active":"2024-09-21T06:09:46.538692+0000","last_became_peered":"2024-09-21T06:09:46.538692+0000","last_unstale":"2024-09-21T06:09:49.535163+0000","last_undegraded":"2024-09-21T06:09:49.535163+0000","last_fullsized":"2024-09-21T06:09:49.535163+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:44.565196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034151+0000","last_change":"2024-09-21T06:09:47.564522+0000","last_active":"2024-09-21T06:09:49.034151+0000","last_peered":"2024-09-21T06:09:49.034151+0000","last_clean":"2024-09-21T06:09:49.034151+0000","last_became_active":"2024-09-21T06:09:47.564206+0000","last_became_peered":"2024-09-21T06:09:47.564206+0000","last_unstale":"2024-09-21T06:09:49.034151+0000","last_undegraded":"2024-09-21T06:09:49.034151+0000","last_fullsized":"2024-09-21T06:09:49.034151+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:06.251922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.034533+0000","last_change":"2024-09-21T06:09:46.532492+0000","last_active":"2024-09-21T06:09:49.034533+0000","last_peered":"2024-09-21T06:09:49.034533+0000","last_clean":"2024-09-21T06:09:49.034533+0000","last_became_active":"2024-09-21T06:09:46.530409+0000","last_became_peered":"2024-09-21T06:09:46.530409+0000","last_unstale":"2024-09-21T06:09:49.034533+0000","last_undegraded":"2024-09-21T06:09:49.034533+0000","last_fullsized":"2024-09-21T06:09:49.034533+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:57.805896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.551023+0000","last_change":"2024-09-21T06:09:47.543465+0000","last_active":"2024-09-21T06:09:48.551023+0000","last_peered":"2024-09-21T06:09:48.551023+0000","last_clean":"2024-09-21T06:09:48.551023+0000","last_became_active":"2024-09-21T06:09:47.543121+0000","last_became_peered":"2024-09-21T06:09:47.543121+0000","last_unstale":"2024-09-21T06:09:48.551023+0000","last_undegraded":"2024-09-21T06:09:48.551023+0000","last_fullsized":"2024-09-21T06:09:48.551023+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:46.505939+0000","last_clean_scrub_stamp":"2024-09-21T06:09:46.505939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:35.324859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:48.552395+0000","last_change":"2024-09-21T06:09:46.534147+0000","last_active":"2024-09-21T06:09:48.552395+0000","last_peered":"2024-09-21T06:09:48.552395+0000","last_clean":"2024-09-21T06:09:48.552395+0000","last_became_active":"2024-09-21T06:09:46.533758+0000","last_became_peered":"2024-09-21T06:09:46.533758+0000","last_unstale":"2024-09-21T06:09:48.552395+0000","last_undegraded":"2024-09-21T06:09:48.552395+0000","last_fullsized":"2024-09-21T06:09:48.552395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:45.498790+0000","last_clean_scrub_stamp":"2024-09-21T06:09:45.498790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:29.394474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":279,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T06:09:49.099958+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:09:49.099958+0000","last_peered":"2024-09-21T06:09:49.099958+0000","last_clean":"2024-09-21T06:09:49.099958+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:09:49.099958+0000","last_undegraded":"2024-09-21T06:09:49.099958+0000","last_fullsized":"2024-09-21T06:09:49.099958+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509606,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127472,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244368,"statfs":{"total":96636764160,"available":96506232832,"internally_reserved":0,"allocated":1986560,"data_stored":1192626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509606,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118560,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253280,"statfs":{"total":96636764160,"available":96515358720,"internally_reserved":0,"allocated":1388544,"data_stored":601712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509605,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82472,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289368,"statfs":{"total":96636764160,"available":96552312832,"internally_reserved":0,"allocated":1392640,"data_stored":602246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509606,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144924,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226916,"statfs":{"total":96636764160,"available":96488361984,"internally_reserved":0,"allocated":1384448,"data_stored":601642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509605,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135212,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236628,"statfs":{"total":96636764160,"available":96498307072,"internally_reserved":0,"allocated":1400832,"data_stored":603324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509607,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140764,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231076,"statfs":{"total":96636764160,"available":96492621824,"internally_reserved":0,"allocated":1384448,"data_stored":601642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509605,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131636,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240204,"statfs":{"total":96636764160,"available":96501968896,"internally_reserved":0,"allocated":1998848,"data_stored":1193734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542310,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144936,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226904,"statfs":{"total":96636764160,"available":96488349696,"internally_reserved":0,"allocated":1392640,"data_stored":601766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:09:56.836 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:09:56.836 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:09:57.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.045+0000 7f54a3200640 1 -- 172.21.15.50:0/3489393950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c06cdf0 msgr2=0x7f549c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.045+0000 7f54a3200640 1 --2- 172.21.15.50:0/3489393950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c06cdf0 0x7f549c06d200 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f549c0b6bf0 tx=0x7f548401c830 comp rx=0 tx=0).stop 2024-09-21T06:09:57.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.045+0000 7f54a3200640 1 -- 172.21.15.50:0/3489393950 shutdown_connections 2024-09-21T06:09:57.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.045+0000 7f54a3200640 1 --2- 172.21.15.50:0/3489393950 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f549c06fc90 0x7f549c0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.045+0000 7f54a3200640 1 --2- 172.21.15.50:0/3489393950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c06cdf0 0x7f549c06d200 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.045+0000 7f54a3200640 1 --2- 172.21.15.50:0/3489393950 >> v2:172.21.15.88:3300/0 conn(0x7f549c159130 0x7f549c15bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.045+0000 7f54a3200640 1 -- 172.21.15.50:0/3489393950 >> 172.21.15.50:0/3489393950 conn(0x7f549c0b35f0 msgr2=0x7f549c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:57.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.045+0000 7f54a3200640 1 -- 172.21.15.50:0/3489393950 shutdown_connections 2024-09-21T06:09:57.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.045+0000 7f54a3200640 1 -- 172.21.15.50:0/3489393950 wait complete. 2024-09-21T06:09:57.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.045+0000 7f54a3200640 1 Processor -- start 2024-09-21T06:09:57.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.045+0000 7f54a3200640 1 -- start start 2024-09-21T06:09:57.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a3200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f549c06cdf0 0x7f549c150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:57.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a3200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f549c06fc90 0x7f549c1535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:57.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a3200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c159130 0x7f549c153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a3200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f549c15fa10 con 0x7f549c159130 2024-09-21T06:09:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a3200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f549c15f890 con 0x7f549c06cdf0 2024-09-21T06:09:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a3200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f549c15fb90 con 0x7f549c06fc90 2024-09-21T06:09:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c159130 0x7f549c153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f549be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f549c06cdf0 0x7f549c150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c159130 0x7f549c153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52454/0 (socket says 172.21.15.50:52454) 2024-09-21T06:09:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f549be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f549c06cdf0 0x7f549c150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:55046/0 (socket says 172.21.15.50:55046) 2024-09-21T06:09:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a1200640 1 -- 172.21.15.50:0/3839455515 learned_addr learned my addr 172.21.15.50:0/3839455515 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f5493e00640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f549c06fc90 0x7f549c1535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a1200640 1 -- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f549c06fc90 msgr2=0x7f549c1535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a1200640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f549c06fc90 0x7f549c1535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a1200640 1 -- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f549c06cdf0 msgr2=0x7f549c150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a1200640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f549c06cdf0 0x7f549c150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a1200640 1 -- 172.21.15.50:0/3839455515 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c156790 con 0x7f549c159130 2024-09-21T06:09:57.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a1200640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c159130 0x7f549c153b10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f549400e940 tx=0x7f549400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:57.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f5498c00640 1 -- 172.21.15.50:0/3839455515 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494004d30 con 0x7f549c159130 2024-09-21T06:09:57.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f5498c00640 1 -- 172.21.15.50:0/3839455515 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f549400cb20 con 0x7f549c159130 2024-09-21T06:09:57.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f5498c00640 1 -- 172.21.15.50:0/3839455515 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494005020 con 0x7f549c159130 2024-09-21T06:09:57.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a3200640 1 -- 172.21.15.50:0/3839455515 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c156a70 con 0x7f549c159130 2024-09-21T06:09:57.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a3200640 1 -- 172.21.15.50:0/3839455515 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c159770 con 0x7f549c159130 2024-09-21T06:09:57.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.049+0000 7f54a3200640 1 -- 172.21.15.50:0/3839455515 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f549c1599a0 con 0x7f549c159130 2024-09-21T06:09:57.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.053+0000 7f5498c00640 1 -- 172.21.15.50:0/3839455515 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f54940051c0 con 0x7f549c159130 2024-09-21T06:09:57.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.053+0000 7f5498c00640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5478074c70 0x7f5478077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:57.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.053+0000 7f5498c00640 1 -- 172.21.15.50:0/3839455515 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f5494093920 con 0x7f549c159130 2024-09-21T06:09:57.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.053+0000 7f549be00640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5478074c70 0x7f5478077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:57.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.053+0000 7f549be00640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5478074c70 0x7f5478077180 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f548c00bad0 tx=0x7f548c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:57.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.057+0000 7f5498c00640 1 -- 172.21.15.50:0/3839455515 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f549c1599a0 con 0x7f549c159130 2024-09-21T06:09:57.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.297+0000 7f54a3200640 1 -- 172.21.15.50:0/3839455515 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f549c1fcf10 con 0x7f549c159130 2024-09-21T06:09:57.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f5498c00640 1 -- 172.21.15.50:0/3839455515 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v37) ==== 190+0+147 (secure 0 0 0) 0x7f5494016030 con 0x7f549c159130 2024-09-21T06:09:57.300 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-09-21T06:09:57.300 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw== 2024-09-21T06:09:57.300 INFO:teuthology.orchestra.run.smithi050.stdout: caps mds = "allow" 2024-09-21T06:09:57.300 INFO:teuthology.orchestra.run.smithi050.stdout: caps mon = "allow r" 2024-09-21T06:09:57.300 INFO:teuthology.orchestra.run.smithi050.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:09:57.300 INFO:teuthology.orchestra.run.smithi050.stderr:updated caps for client.0 2024-09-21T06:09:57.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f54a3200640 1 -- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5478074c70 msgr2=0x7f5478077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f54a3200640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5478074c70 0x7f5478077180 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f548c00bad0 tx=0x7f548c00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f54a3200640 1 -- 172.21.15.50:0/3839455515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c159130 msgr2=0x7f549c153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f54a3200640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c159130 0x7f549c153b10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f549400e940 tx=0x7f549400ee00 comp rx=0 tx=0).stop 2024-09-21T06:09:57.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f54a3200640 1 -- 172.21.15.50:0/3839455515 shutdown_connections 2024-09-21T06:09:57.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f54a3200640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5478074c70 0x7f5478077180 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f54a3200640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c159130 0x7f549c153b10 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f54a3200640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f549c06fc90 0x7f549c1535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f54a3200640 1 --2- 172.21.15.50:0/3839455515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f549c06cdf0 0x7f549c150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f54a3200640 1 -- 172.21.15.50:0/3839455515 >> 172.21.15.50:0/3839455515 conn(0x7f549c0b35f0 msgr2=0x7f549c0b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:57.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f54a3200640 1 -- 172.21.15.50:0/3839455515 shutdown_connections 2024-09-21T06:09:57.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.301+0000 7f54a3200640 1 -- 172.21.15.50:0/3839455515 wait complete. 2024-09-21T06:09:57.323 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:09:57.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.481+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2627485901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8dc004810 msgr2=0x7fb8dc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.481+0000 7fb8fdc00640 1 --2- 172.21.15.50:0/2627485901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8dc004810 0x7fb8dc004c20 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8072810 tx=0x7fb8e001d1f0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.481+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2627485901 shutdown_connections 2024-09-21T06:09:57.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.481+0000 7fb8fdc00640 1 --2- 172.21.15.50:0/2627485901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8dc004810 0x7fb8dc004c20 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.481+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2627485901 >> 172.21.15.50:0/2627485901 conn(0x7fb8f80b2c40 msgr2=0x7fb8f80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:57.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.481+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2627485901 shutdown_connections 2024-09-21T06:09:57.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.481+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2627485901 wait complete. 2024-09-21T06:09:57.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8fdc00640 1 Processor -- start 2024-09-21T06:09:57.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8fdc00640 1 -- start start 2024-09-21T06:09:57.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8fdc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8dc004810 0x7fb8f8157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:57.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8fdc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f814c5a0 0x7fb8f81f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:57.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8fdc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f81f8a30 0x7fb8f8159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8fdc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f806ea60 con 0x7fb8f814c5a0 2024-09-21T06:09:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8fdc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f80b3d50 con 0x7fb8f81f8a30 2024-09-21T06:09:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8fdc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f806ebe0 con 0x7fb8dc004810 2024-09-21T06:09:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f814c5a0 0x7fb8f81f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f814c5a0 0x7fb8f81f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52468/0 (socket says 172.21.15.50:52468) 2024-09-21T06:09:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8efe00640 1 -- 172.21.15.50:0/2645611712 learned_addr learned my addr 172.21.15.50:0/2645611712 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8f6a00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8dc004810 0x7fb8f8157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8f7400640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f81f8a30 0x7fb8f8159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8efe00640 1 -- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8dc004810 msgr2=0x7fb8f8157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8efe00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8dc004810 0x7fb8f8157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8efe00640 1 -- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f81f8a30 msgr2=0x7fb8f8159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8efe00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f81f8a30 0x7fb8f8159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8efe00640 1 -- 172.21.15.50:0/2645611712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f8158130 con 0x7fb8f814c5a0 2024-09-21T06:09:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8f6a00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8dc004810 0x7fb8f8157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8f7400640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f81f8a30 0x7fb8f8159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.485+0000 7fb8efe00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f814c5a0 0x7fb8f81f84f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb8e001d020 tx=0x7fb8e00050a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:57.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.489+0000 7fb8eea00640 1 -- 172.21.15.50:0/2645611712 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00bb030 con 0x7fb8f814c5a0 2024-09-21T06:09:57.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.489+0000 7fb8eea00640 1 -- 172.21.15.50:0/2645611712 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8e0004900 con 0x7fb8f814c5a0 2024-09-21T06:09:57.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.489+0000 7fb8eea00640 1 -- 172.21.15.50:0/2645611712 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0004d30 con 0x7fb8f814c5a0 2024-09-21T06:09:57.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.489+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2645611712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f81f8e40 con 0x7fb8f814c5a0 2024-09-21T06:09:57.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.489+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2645611712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f80afa20 con 0x7fb8f814c5a0 2024-09-21T06:09:57.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.489+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2645611712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f80af4b0 con 0x7fb8f814c5a0 2024-09-21T06:09:57.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.489+0000 7fb8eea00640 1 -- 172.21.15.50:0/2645611712 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7fb8e00be030 con 0x7fb8f814c5a0 2024-09-21T06:09:57.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.489+0000 7fb8eea00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb8d4074ca0 0x7fb8d40771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:57.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.489+0000 7fb8eea00640 1 -- 172.21.15.50:0/2645611712 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fb8e012e6e0 con 0x7fb8f814c5a0 2024-09-21T06:09:57.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.489+0000 7fb8f6a00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb8d4074ca0 0x7fb8d40771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:57.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.493+0000 7fb8f6a00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb8d4074ca0 0x7fb8d40771b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb8f806bb90 tx=0x7fb8e8006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:57.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.493+0000 7fb8eea00640 1 -- 172.21.15.50:0/2645611712 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb8e0016f00 con 0x7fb8f814c5a0 2024-09-21T06:09:57.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.713+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2645611712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8f8073ea0 con 0x7fb8f814c5a0 2024-09-21T06:09:57.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.713+0000 7fb8eea00640 1 -- 172.21.15.50:0/2645611712 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5225 (secure 0 0 0) 0x7fb8e00fb170 con 0x7fb8f814c5a0 2024-09-21T06:09:57.714 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:09:57.714 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":74,"btime":"2024-09-21T06:09:50:555528+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4024495468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":4024495468},{"type":"v1","addr":"172.21.15.50:6835","nonce":4024495468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3620352888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3620352888},{"type":"v1","addr":"172.21.15.88:6835","nonce":3620352888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4286454792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4286454792},{"type":"v1","addr":"172.21.15.88:6837","nonce":4286454792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:09:46.946836+0000","modified":"2024-09-21T06:09:49.976058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6833/1093475697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1093475697},{"type":"v1","addr":"172.21.15.50:6833","nonce":1093475697}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8304,"qdb_cluster":[8304]},"id":8}]} 2024-09-21T06:09:57.714 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 74 2024-09-21T06:09:57.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.713+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb8d4074ca0 msgr2=0x7fb8d40771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.713+0000 7fb8fdc00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb8d4074ca0 0x7fb8d40771b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb8f806bb90 tx=0x7fb8e8006b70 comp rx=0 tx=0).stop 2024-09-21T06:09:57.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.713+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2645611712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f814c5a0 msgr2=0x7fb8f81f84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.713+0000 7fb8fdc00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f814c5a0 0x7fb8f81f84f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb8e001d020 tx=0x7fb8e00050a0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.717+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2645611712 shutdown_connections 2024-09-21T06:09:57.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.717+0000 7fb8fdc00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f81f8a30 0x7fb8f8159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.717+0000 7fb8fdc00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb8d4074ca0 0x7fb8d40771b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.717+0000 7fb8fdc00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f814c5a0 0x7fb8f81f84f0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.717+0000 7fb8fdc00640 1 --2- 172.21.15.50:0/2645611712 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8dc004810 0x7fb8f8157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.717+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2645611712 >> 172.21.15.50:0/2645611712 conn(0x7fb8f80b2c40 msgr2=0x7fb8f815e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:57.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.717+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2645611712 shutdown_connections 2024-09-21T06:09:57.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:57.717+0000 7fb8fdc00640 1 -- 172.21.15.50:0/2645611712 wait complete. 2024-09-21T06:09:58.140 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:09:46.946836+0000', 'modified': '2024-09-21T06:09:49.976058+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8304}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8304': {'gid': 8304, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6833/1093475697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6832', 'nonce': 1093475697}, {'type': 'v1', 'addr': '172.21.15.50:6833', 'nonce': 1093475697}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8304, 'qdb_cluster': [8304]} 2024-09-21T06:09:58.140 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:09:58.140 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T06:09:58.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.305+0000 7f41de200640 1 -- 172.21.15.50:0/294118243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41d815f6b0 msgr2=0x7f41d815fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.305+0000 7f41de200640 1 --2- 172.21.15.50:0/294118243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41d815f6b0 0x7f41d815fac0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f41d80b5f20 tx=0x7f41d001ccd0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.305+0000 7f41de200640 1 -- 172.21.15.50:0/294118243 shutdown_connections 2024-09-21T06:09:58.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.305+0000 7f41de200640 1 --2- 172.21.15.50:0/294118243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41d815f6b0 0x7f41d815fac0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.305+0000 7f41de200640 1 --2- 172.21.15.50:0/294118243 >> v2:172.21.15.88:3300/0 conn(0x7f41d8074300 0x7f41d815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.305+0000 7f41de200640 1 --2- 172.21.15.50:0/294118243 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f41d815c5d0 0x7f41d8073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.305+0000 7f41de200640 1 -- 172.21.15.50:0/294118243 >> 172.21.15.50:0/294118243 conn(0x7f41d80b2c40 msgr2=0x7f41d80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:58.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.305+0000 7f41de200640 1 -- 172.21.15.50:0/294118243 shutdown_connections 2024-09-21T06:09:58.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.305+0000 7f41de200640 1 -- 172.21.15.50:0/294118243 wait complete. 2024-09-21T06:09:58.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.305+0000 7f41de200640 1 Processor -- start 2024-09-21T06:09:58.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.305+0000 7f41de200640 1 -- start start 2024-09-21T06:09:58.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41de200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d8074300 0x7f41d80afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41de200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41d815c5d0 0x7f41d8154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41de200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f41d815f6b0 0x7f41d8155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41de200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f41d806f3f0 con 0x7f41d815c5d0 2024-09-21T06:09:58.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41de200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f41d806f270 con 0x7f41d8074300 2024-09-21T06:09:58.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41de200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f41d806f570 con 0x7f41d815f6b0 2024-09-21T06:09:58.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41cea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41d815c5d0 0x7f41d8154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41cea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41d815c5d0 0x7f41d8154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52476/0 (socket says 172.21.15.50:52476) 2024-09-21T06:09:58.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41d7400640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d8074300 0x7f41d80afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41cea00640 1 -- 172.21.15.50:0/21145757 learned_addr learned my addr 172.21.15.50:0/21145757 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:58.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41d7e00640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f41d815f6b0 0x7f41d8155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41d7400640 1 -- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f41d815f6b0 msgr2=0x7f41d8155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41d7400640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f41d815f6b0 0x7f41d8155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41d7400640 1 -- 172.21.15.50:0/21145757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41d815c5d0 msgr2=0x7f41d8154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41d7400640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41d815c5d0 0x7f41d8154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41d7400640 1 -- 172.21.15.50:0/21145757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d81511f0 con 0x7f41d8074300 2024-09-21T06:09:58.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41cea00640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41d815c5d0 0x7f41d8154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:09:58.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41d7400640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d8074300 0x7f41d80afe10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f41c800b440 tx=0x7f41c800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:58.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41cfe00640 1 -- 172.21.15.50:0/21145757 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8015020 con 0x7f41d8074300 2024-09-21T06:09:58.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41cfe00640 1 -- 172.21.15.50:0/21145757 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41c8011920 con 0x7f41d8074300 2024-09-21T06:09:58.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41de200640 1 -- 172.21.15.50:0/21145757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d81514d0 con 0x7f41d8074300 2024-09-21T06:09:58.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41cfe00640 1 -- 172.21.15.50:0/21145757 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8011d70 con 0x7f41d8074300 2024-09-21T06:09:58.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.309+0000 7f41de200640 1 -- 172.21.15.50:0/21145757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d80ab210 con 0x7f41d8074300 2024-09-21T06:09:58.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.313+0000 7f41cfe00640 1 -- 172.21.15.50:0/21145757 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f41c8018020 con 0x7f41d8074300 2024-09-21T06:09:58.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.313+0000 7f41e58f9000 1 Processor -- start 2024-09-21T06:09:58.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.313+0000 7f41cfe00640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f41b4074c70 0x7f41b4077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.313+0000 7f41e58f9000 1 -- start start 2024-09-21T06:09:58.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.313+0000 7f41cfe00640 1 -- 172.21.15.50:0/21145757 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f41c8092e60 con 0x7f41d8074300 2024-09-21T06:09:58.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.313+0000 7f41d7e00640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f41b4074c70 0x7f41b4077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41d7e00640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f41b4074c70 0x7f41b4077180 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f41d00af000 tx=0x7f41d001b570 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:58.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32864460 0x55fc32da56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c54950 0x55fc32da5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 0x55fc32da6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fc32c7a410 con 0x55fc32c3dca0 2024-09-21T06:09:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fc32bb7b40 con 0x55fc32864460 2024-09-21T06:09:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55fc32ba6f70 con 0x55fc32c54950 2024-09-21T06:09:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41d7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 0x55fc32da6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41d7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 0x55fc32da6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52478/0 (socket says 172.21.15.50:52478) 2024-09-21T06:09:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41d7e00640 1 -- 172.21.15.50:0/1095823188 learned_addr learned my addr 172.21.15.50:0/1095823188 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41d7400640 1 --2- 172.21.15.50:0/1095823188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32864460 0x55fc32da56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41cea00640 1 --2- 172.21.15.50:0/1095823188 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c54950 0x55fc32da5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41d7e00640 1 -- 172.21.15.50:0/1095823188 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c54950 msgr2=0x55fc32da5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:09:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41d7e00640 1 --2- 172.21.15.50:0/1095823188 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c54950 0x55fc32da5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41d7e00640 1 -- 172.21.15.50:0/1095823188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32864460 msgr2=0x55fc32da56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:09:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41d7e00640 1 --2- 172.21.15.50:0/1095823188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32864460 0x55fc32da56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41d7e00640 1 -- 172.21.15.50:0/1095823188 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc32b6e5b0 con 0x55fc32c3dca0 2024-09-21T06:09:58.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41d7e00640 1 --2- 172.21.15.50:0/1095823188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 0x55fc32da6120 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f41d001a960 tx=0x7f41d00b6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333646a213d608a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:58.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41c7400640 1 -- 172.21.15.50:0/1095823188 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d00b2040 con 0x55fc32c3dca0 2024-09-21T06:09:58.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41c7400640 1 -- 172.21.15.50:0/1095823188 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41d00c8070 con 0x55fc32c3dca0 2024-09-21T06:09:58.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41c7400640 1 -- 172.21.15.50:0/1095823188 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0004670 con 0x55fc32c3dca0 2024-09-21T06:09:58.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 -- 172.21.15.50:0/1095823188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 msgr2=0x55fc32da6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1095823188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 0x55fc32da6120 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f41d001a960 tx=0x7f41d00b6000 comp rx=0 tx=0).stop 2024-09-21T06:09:58.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 -- 172.21.15.50:0/1095823188 shutdown_connections 2024-09-21T06:09:58.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1095823188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 0x55fc32da6120 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1095823188 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c54950 0x55fc32da5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1095823188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32864460 0x55fc32da56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 -- 172.21.15.50:0/1095823188 >> 172.21.15.50:0/1095823188 conn(0x55fc32d2ad90 msgr2=0x55fc32d5a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:58.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 -- 172.21.15.50:0/1095823188 shutdown_connections 2024-09-21T06:09:58.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.317+0000 7f41e58f9000 1 -- 172.21.15.50:0/1095823188 wait complete. 2024-09-21T06:09:58.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 1 -- start start 2024-09-21T06:09:58.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 1 Processor -- start 2024-09-21T06:09:58.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32864460 0x55fc32dac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 0x55fc32dace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c54950 0x55fc32dad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fc32b415f0 con 0x55fc32c3dca0 2024-09-21T06:09:58.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fc32baad80 con 0x55fc32c54950 2024-09-21T06:09:58.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55fc32b4d520 con 0x55fc32864460 2024-09-21T06:09:58.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 10 client.0 _ll_register_callbacks cb 0x55fc32b51830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:09:58.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41cea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 0x55fc32dace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41cea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 0x55fc32dace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52492/0 (socket says 172.21.15.50:52492) 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41cea00640 1 -- 172.21.15.50:0/1068204914 learned_addr learned my addr 172.21.15.50:0/1068204914 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41d7400640 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32864460 0x55fc32dac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41d7e00640 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c54950 0x55fc32dad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41cea00640 1 -- 172.21.15.50:0/1068204914 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32864460 msgr2=0x55fc32dac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41cea00640 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32864460 0x55fc32dac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41cea00640 1 -- 172.21.15.50:0/1068204914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c54950 msgr2=0x55fc32dad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41cea00640 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c54950 0x55fc32dad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41cea00640 1 -- 172.21.15.50:0/1068204914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc32b45190 con 0x55fc32c3dca0 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41cea00640 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 0x55fc32dace60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f41c0012bf0 tx=0x7f41c00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b180bfccb15fd9e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41bbe00640 1 -- 172.21.15.50:0/1068204914 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c001f060 con 0x55fc32c3dca0 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41bbe00640 1 -- 172.21.15.50:0/1068204914 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f41c000e7c0 con 0x55fc32c3dca0 2024-09-21T06:09:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 1 -- 172.21.15.50:0/1068204914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc32b3f110 con 0x55fc32c3dca0 2024-09-21T06:09:58.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41bbe00640 1 -- 172.21.15.50:0/1068204914 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f41c0025050 con 0x55fc32c3dca0 2024-09-21T06:09:58.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 10 client.8454 fetch_fsmap learned FSMap version 74 2024-09-21T06:09:58.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 1 -- 172.21.15.50:0/1068204914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55fc32c2bf10 con 0x55fc32c3dca0 2024-09-21T06:09:58.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41bbe00640 1 -- 172.21.15.50:0/1068204914 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55fc32b3f110 con 0x55fc32c3dca0 2024-09-21T06:09:58.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41bbe00640 1 -- 172.21.15.50:0/1068204914 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 74) ==== 3502+0+0 (secure 0 0 0) 0x7f41c0028020 con 0x55fc32c3dca0 2024-09-21T06:09:58.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 10 client.8454 fetch_fsmap finished waiting for FSMap version 74 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 10 client.8454 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 20 client.8454 populate_metadata read hostname 'smithi050' 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.50:6832/1093475697,v1:172.21.15.50:6833/1093475697] conn(0x55fc328497d0 0x55fc32db6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 4 client.8454 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.321+0000 7f41e58f9000 1 -- 172.21.15.50:0/1068204914 --> [v2:172.21.15.50:6832/1093475697,v1:172.21.15.50:6833/1093475697] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc32bb9a90 con 0x55fc328497d0 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41d7400640 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.50:6832/1093475697,v1:172.21.15.50:6833/1093475697] conn(0x55fc328497d0 0x55fc32db6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41d7400640 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.50:6832/1093475697,v1:172.21.15.50:6833/1093475697] conn(0x55fc328497d0 0x55fc32db6d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e266ee581dc317a server_cookie=85e391512726c94b in_seq=0 out_seq=0 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41bbe00640 10 client.8454 ms_handle_connect on v2:172.21.15.50:6832/1093475697 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41bbe00640 1 -- 172.21.15.50:0/1068204914 <== mds.0 v2:172.21.15.50:6832/1093475697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc32bb9a90 con 0x55fc328497d0 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41bbe00640 10 client.8454 handle_command_reply: tid=0 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41e58f9000 1 client.8454 shutdown 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41e58f9000 1 -- 172.21.15.50:0/1068204914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 msgr2=0x55fc32dace60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 0x55fc32dace60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f41c0012bf0 tx=0x7f41c00107a0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41e58f9000 1 -- 172.21.15.50:0/1068204914 shutdown_connections 2024-09-21T06:09:58.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.50:6832/1093475697,v1:172.21.15.50:6833/1093475697] conn(0x55fc328497d0 0x55fc32db6d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c54950 0x55fc32dad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32c3dca0 0x55fc32dace60 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41bbe00640 0 client.8454 ms_handle_reset on v2:172.21.15.50:6832/1093475697 2024-09-21T06:09:58.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1068204914 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32864460 0x55fc32dac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41e58f9000 1 -- 172.21.15.50:0/1068204914 >> 172.21.15.50:0/1068204914 conn(0x55fc32d2ad90 msgr2=0x55fc32d61440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:58.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41e58f9000 1 -- 172.21.15.50:0/1068204914 shutdown_connections 2024-09-21T06:09:58.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41e58f9000 1 -- 172.21.15.50:0/1068204914 wait complete. 2024-09-21T06:09:58.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.325+0000 7f41e58f9000 20 client.8454 trim_cache size 0 max 16384 2024-09-21T06:09:58.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.345+0000 7f41e58f9000 1 Processor -- start 2024-09-21T06:09:58.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.345+0000 7f41e58f9000 1 -- start start 2024-09-21T06:09:58.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.345+0000 7f41e58f9000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 0x55fc32e17620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.345+0000 7f41e58f9000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c3dca0 0x55fc32e17b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.345+0000 7f41e58f9000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c54950 0x55fc32e180a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.345+0000 7f41e58f9000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f41d82052a0 con 0x55fc32864460 2024-09-21T06:09:58.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.345+0000 7f41e58f9000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f41d82042c0 con 0x55fc32c54950 2024-09-21T06:09:58.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.345+0000 7f41e58f9000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f41d81528c0 con 0x55fc32c3dca0 2024-09-21T06:09:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.345+0000 7f41d7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 0x55fc32e17620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41d7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 0x55fc32e17620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52502/0 (socket says 172.21.15.50:52502) 2024-09-21T06:09:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41d7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c54950 0x55fc32e180a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41d7400640 1 -- 172.21.15.50:0/1503383254 learned_addr learned my addr 172.21.15.50:0/1503383254 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41cea00640 1 --2- 172.21.15.50:0/1503383254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c3dca0 0x55fc32e17b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41d7400640 1 -- 172.21.15.50:0/1503383254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c3dca0 msgr2=0x55fc32e17b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:09:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41d7400640 1 --2- 172.21.15.50:0/1503383254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c3dca0 0x55fc32e17b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41d7400640 1 -- 172.21.15.50:0/1503383254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c54950 msgr2=0x55fc32e180a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:09:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41d7400640 1 --2- 172.21.15.50:0/1503383254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c54950 0x55fc32e180a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41d7400640 1 -- 172.21.15.50:0/1503383254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc32c02db0 con 0x55fc32864460 2024-09-21T06:09:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41d7e00640 1 --2- 172.21.15.50:0/1503383254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c54950 0x55fc32e180a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:09:58.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41d7400640 1 --2- 172.21.15.50:0/1503383254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 0x55fc32e17620 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f41c800b410 tx=0x7f41c8068040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668d0a0fd0ec886a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:58.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41c6a00640 1 -- 172.21.15.50:0/1503383254 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8068c00 con 0x55fc32864460 2024-09-21T06:09:58.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41c6a00640 1 -- 172.21.15.50:0/1503383254 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41c80660b0 con 0x55fc32864460 2024-09-21T06:09:58.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41c6a00640 1 -- 172.21.15.50:0/1503383254 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c805e7a0 con 0x55fc32864460 2024-09-21T06:09:58.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41e58f9000 1 -- 172.21.15.50:0/1503383254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 msgr2=0x55fc32e17620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1503383254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 0x55fc32e17620 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f41c800b410 tx=0x7f41c8068040 comp rx=0 tx=0).stop 2024-09-21T06:09:58.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41e58f9000 1 -- 172.21.15.50:0/1503383254 shutdown_connections 2024-09-21T06:09:58.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1503383254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c54950 0x55fc32e180a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1503383254 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c3dca0 0x55fc32e17b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41e58f9000 1 --2- 172.21.15.50:0/1503383254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 0x55fc32e17620 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41e58f9000 1 -- 172.21.15.50:0/1503383254 >> 172.21.15.50:0/1503383254 conn(0x55fc32d2ad90 msgr2=0x55fc32da3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:58.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41e58f9000 1 -- 172.21.15.50:0/1503383254 shutdown_connections 2024-09-21T06:09:58.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41e58f9000 1 -- 172.21.15.50:0/1503383254 wait complete. 2024-09-21T06:09:58.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41e58f9000 1 -- start start 2024-09-21T06:09:58.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.349+0000 7f41e58f9000 1 Processor -- start 2024-09-21T06:09:58.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 0x55fc32e1b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c3dca0 0x55fc32e1b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c54950 0x55fc32e1be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fc32b72ee0 con 0x55fc32864460 2024-09-21T06:09:58.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fc32baf3b0 con 0x55fc32c3dca0 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x55fc32bb6910 con 0x55fc32c54950 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 10 client.0 _ll_register_callbacks cb 0x7f41c000aab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41d7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 0x55fc32e1b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41d7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 0x55fc32e1b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52514/0 (socket says 172.21.15.50:52514) 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41d7400640 1 -- 172.21.15.50:0/733282597 learned_addr learned my addr 172.21.15.50:0/733282597 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41cea00640 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c3dca0 0x55fc32e1b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41d7e00640 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c54950 0x55fc32e1be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41d7400640 1 -- 172.21.15.50:0/733282597 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c54950 msgr2=0x55fc32e1be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41d7400640 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c54950 0x55fc32e1be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41d7400640 1 -- 172.21.15.50:0/733282597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c3dca0 msgr2=0x55fc32e1b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41d7400640 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c3dca0 0x55fc32e1b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41d7400640 1 -- 172.21.15.50:0/733282597 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc32b6dab0 con 0x55fc32864460 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41d7400640 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 0x55fc32e1b390 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f41c805f020 tx=0x7f41c805b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a38a26bcf8ee9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41bbe00640 1 -- 172.21.15.50:0/733282597 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c805e790 con 0x55fc32864460 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41bbe00640 1 -- 172.21.15.50:0/733282597 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f41c8080970 con 0x55fc32864460 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 -- 172.21.15.50:0/733282597 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c80660b0 con 0x55fc32864460 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41bbe00640 1 -- 172.21.15.50:0/733282597 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f41c806d050 con 0x55fc32864460 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 10 client.8460 fetch_fsmap learned FSMap version 74 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 -- 172.21.15.50:0/733282597 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55fc32b4c070 con 0x55fc32864460 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41bbe00640 1 -- 172.21.15.50:0/733282597 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41c80660b0 con 0x55fc32864460 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41bbe00640 1 -- 172.21.15.50:0/733282597 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 74) ==== 3502+0+0 (secure 0 0 0) 0x7f41c8070020 con 0x55fc32864460 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 10 client.8460 fetch_fsmap finished waiting for FSMap version 74 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 10 client.8460 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 20 client.8460 populate_metadata read hostname 'smithi050' 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.50:6832/1093475697,v1:172.21.15.50:6833/1093475697] conn(0x55fc32837830 0x55fc32dd7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 4 client.8460 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:09:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 -- 172.21.15.50:0/733282597 --> [v2:172.21.15.50:6832/1093475697,v1:172.21.15.50:6833/1093475697] -- command(tid 0: {"prefix": "status"}) -- 0x55fc32bb5540 con 0x55fc32837830 2024-09-21T06:09:58.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41cea00640 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.50:6832/1093475697,v1:172.21.15.50:6833/1093475697] conn(0x55fc32837830 0x55fc32dd7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41cea00640 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.50:6832/1093475697,v1:172.21.15.50:6833/1093475697] conn(0x55fc32837830 0x55fc32dd7670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7f446ee6d05e0f server_cookie=dc1d43c01a24896b in_seq=0 out_seq=0 2024-09-21T06:09:58.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41bbe00640 10 client.8460 ms_handle_connect on v2:172.21.15.50:6832/1093475697 2024-09-21T06:09:58.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41bbe00640 1 -- 172.21.15.50:0/733282597 <== mds.0 v2:172.21.15.50:6832/1093475697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x55fc32bb5540 con 0x55fc32837830 2024-09-21T06:09:58.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41bbe00640 10 client.8460 handle_command_reply: tid=0 2024-09-21T06:09:58.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 client.8460 shutdown 2024-09-21T06:09:58.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 -- 172.21.15.50:0/733282597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 msgr2=0x55fc32e1b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 0x55fc32e1b390 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f41c805f020 tx=0x7f41c805b7c0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 -- 172.21.15.50:0/733282597 shutdown_connections 2024-09-21T06:09:58.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.50:6832/1093475697,v1:172.21.15.50:6833/1093475697] conn(0x55fc32837830 0x55fc32dd7670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x55fc32c54950 0x55fc32e1be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fc32c3dca0 0x55fc32e1b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 --2- 172.21.15.50:0/733282597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc32864460 0x55fc32e1b390 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 -- 172.21.15.50:0/733282597 >> 172.21.15.50:0/733282597 conn(0x55fc32d2ad90 msgr2=0x55fc32da3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:58.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41bbe00640 0 client.8460 ms_handle_reset on v2:172.21.15.50:6832/1093475697 2024-09-21T06:09:58.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.353+0000 7f41e58f9000 1 -- 172.21.15.50:0/733282597 shutdown_connections 2024-09-21T06:09:58.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41e58f9000 1 -- 172.21.15.50:0/733282597 wait complete. 2024-09-21T06:09:58.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41e58f9000 20 client.8460 trim_cache size 0 max 16384 2024-09-21T06:09:58.359 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:09:58.359 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:09:58.359 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-21T06:09:58.359 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 8304, 2024-09-21T06:09:58.359 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-21T06:09:58.360 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-21T06:09:58.360 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-21T06:09:58.360 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 10.815335251, 2024-09-21T06:09:58.360 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 74, 2024-09-21T06:09:58.360 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 112, 2024-09-21T06:09:58.360 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 112, 2024-09-21T06:09:58.360 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 15.20735898 2024-09-21T06:09:58.360 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:09:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41de200640 1 -- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f41b4074c70 msgr2=0x7f41b4077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41de200640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f41b4074c70 0x7f41b4077180 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f41d00af000 tx=0x7f41d001b570 comp rx=0 tx=0).stop 2024-09-21T06:09:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41de200640 1 -- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d8074300 msgr2=0x7f41d80afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41de200640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d8074300 0x7f41d80afe10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f41c800b440 tx=0x7f41c800b840 comp rx=0 tx=0).stop 2024-09-21T06:09:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41de200640 1 -- 172.21.15.50:0/21145757 shutdown_connections 2024-09-21T06:09:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41de200640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f41d815f6b0 0x7f41d8155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41de200640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f41b4074c70 0x7f41b4077180 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41de200640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41d815c5d0 0x7f41d8154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41de200640 1 --2- 172.21.15.50:0/21145757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d8074300 0x7f41d80afe10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41de200640 1 -- 172.21.15.50:0/21145757 >> 172.21.15.50:0/21145757 conn(0x7f41d80b2c40 msgr2=0x7f41d80b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41de200640 1 -- 172.21.15.50:0/21145757 shutdown_connections 2024-09-21T06:09:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.357+0000 7f41de200640 1 -- 172.21.15.50:0/21145757 wait complete. 2024-09-21T06:09:58.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:09:58.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.537+0000 7f6852800640 1 -- 172.21.15.50:0/151193346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f684c06cdf0 msgr2=0x7f684c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.537+0000 7f6852800640 1 --2- 172.21.15.50:0/151193346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f684c06cdf0 0x7f684c06d200 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6834009840 tx=0x7f683401c790 comp rx=0 tx=0).stop 2024-09-21T06:09:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.537+0000 7f6852800640 1 -- 172.21.15.50:0/151193346 shutdown_connections 2024-09-21T06:09:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.537+0000 7f6852800640 1 --2- 172.21.15.50:0/151193346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f684c06cdf0 0x7f684c06d200 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.537+0000 7f6852800640 1 --2- 172.21.15.50:0/151193346 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f684c159130 0x7f684c15bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.537+0000 7f6852800640 1 -- 172.21.15.50:0/151193346 >> 172.21.15.50:0/151193346 conn(0x7f684c0b35f0 msgr2=0x7f684c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.537+0000 7f6852800640 1 -- 172.21.15.50:0/151193346 shutdown_connections 2024-09-21T06:09:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.537+0000 7f6852800640 1 -- 172.21.15.50:0/151193346 wait complete. 2024-09-21T06:09:58.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.537+0000 7f6852800640 1 Processor -- start 2024-09-21T06:09:58.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.537+0000 7f6852800640 1 -- start start 2024-09-21T06:09:58.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6852800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f684c06cdf0 0x7f684c164b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6852800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f684c06fc90 0x7f684c15f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6852800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f684c159130 0x7f684c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6852800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f684c168df0 con 0x7f684c159130 2024-09-21T06:09:58.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6852800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f684c168c70 con 0x7f684c06cdf0 2024-09-21T06:09:58.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6852800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f684c168f70 con 0x7f684c06fc90 2024-09-21T06:09:58.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f684be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f684c159130 0x7f684c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f684be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f684c159130 0x7f684c15f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52524/0 (socket says 172.21.15.50:52524) 2024-09-21T06:09:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f684be00640 1 -- 172.21.15.50:0/143707088 learned_addr learned my addr 172.21.15.50:0/143707088 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f684b400640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f684c06cdf0 0x7f684c164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6843e00640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f684c06fc90 0x7f684c15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f684be00640 1 -- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f684c06fc90 msgr2=0x7f684c15f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f684be00640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f684c06fc90 0x7f684c15f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f684be00640 1 -- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f684c06cdf0 msgr2=0x7f684c164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f684be00640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f684c06cdf0 0x7f684c164b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f684be00640 1 -- 172.21.15.50:0/143707088 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c1f9380 con 0x7f684c159130 2024-09-21T06:09:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6843e00640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f684c06fc90 0x7f684c15f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f684be00640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f684c159130 0x7f684c15f650 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f684400cb40 tx=0x7f684400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6843400640 1 -- 172.21.15.50:0/143707088 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844016020 con 0x7f684c159130 2024-09-21T06:09:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6843400640 1 -- 172.21.15.50:0/143707088 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f684400ab40 con 0x7f684c159130 2024-09-21T06:09:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6852800640 1 -- 172.21.15.50:0/143707088 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684c1f9660 con 0x7f684c159130 2024-09-21T06:09:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6843400640 1 -- 172.21.15.50:0/143707088 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844012420 con 0x7f684c159130 2024-09-21T06:09:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6852800640 1 -- 172.21.15.50:0/143707088 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684c15b610 con 0x7f684c159130 2024-09-21T06:09:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.541+0000 7f6852800640 1 -- 172.21.15.50:0/143707088 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f684c06d200 con 0x7f684c159130 2024-09-21T06:09:58.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.545+0000 7f6843400640 1 -- 172.21.15.50:0/143707088 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f6844019020 con 0x7f684c159130 2024-09-21T06:09:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.545+0000 7f6843400640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6828074c70 0x7f6828077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.545+0000 7f6843400640 1 -- 172.21.15.50:0/143707088 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f68440932a0 con 0x7f684c159130 2024-09-21T06:09:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.545+0000 7f684b400640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6828074c70 0x7f6828077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:58.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.545+0000 7f684b400640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6828074c70 0x7f6828077180 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f683c009b40 tx=0x7f683c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.549+0000 7f6843400640 1 -- 172.21.15.50:0/143707088 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f684409c070 con 0x7f684c159130 2024-09-21T06:09:58.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.761+0000 7f6852800640 1 -- 172.21.15.50:0/143707088 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f684c165270 con 0x7f684c159130 2024-09-21T06:09:58.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.761+0000 7f6843400640 1 -- 172.21.15.50:0/143707088 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f6844020020 con 0x7f684c159130 2024-09-21T06:09:58.764 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:09:58.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.765+0000 7f6852800640 1 -- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6828074c70 msgr2=0x7f6828077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.765+0000 7f6852800640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6828074c70 0x7f6828077180 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f683c009b40 tx=0x7f683c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.765+0000 7f6852800640 1 -- 172.21.15.50:0/143707088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f684c159130 msgr2=0x7f684c15f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:58.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.765+0000 7f6852800640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f684c159130 0x7f684c15f650 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f684400cb40 tx=0x7f684400cc30 comp rx=0 tx=0).stop 2024-09-21T06:09:58.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.765+0000 7f6852800640 1 -- 172.21.15.50:0/143707088 shutdown_connections 2024-09-21T06:09:58.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.765+0000 7f6852800640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6828074c70 0x7f6828077180 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.765+0000 7f6852800640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f684c159130 0x7f684c15f650 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.765+0000 7f6852800640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f684c06fc90 0x7f684c15f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.765+0000 7f6852800640 1 --2- 172.21.15.50:0/143707088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f684c06cdf0 0x7f684c164b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:58.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.765+0000 7f6852800640 1 -- 172.21.15.50:0/143707088 >> 172.21.15.50:0/143707088 conn(0x7f684c0b35f0 msgr2=0x7f684c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:58.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.765+0000 7f6852800640 1 -- 172.21.15.50:0/143707088 shutdown_connections 2024-09-21T06:09:58.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:58.765+0000 7f6852800640 1 -- 172.21.15.50:0/143707088 wait complete. 2024-09-21T06:09:58.785 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:09:58.785 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:09:58.785 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-21T06:09:58.785 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:09:58.785 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:09:58.785 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:09:58.786 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:09:58.786 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:09:58.786 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:09:58.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:58.786 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-21T06:09:58.795 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:09:58.796 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:09:58.796 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:09:58.796 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:09:58.796 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T06:09:58.796 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:09:58.796 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:09:58.796 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:09:58.796 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-21T06:09:58.796 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:09:58.796 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:09:58.796 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-21T06:09:58.796 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 41597sec preferred_lft 41597sec 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:8b:d0:0a:5d brd ff:ff:ff:ff:ff:ff 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:09:58.797 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T06:09:58.798 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:09:58.798 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:09:58.798 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T06:09:58.798 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:09:58.798 INFO:teuthology.orchestra.run.smithi050.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:09:58.798 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:09:58.798 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T06:09:58.798 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:09:58.798 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:09:58.798 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:09:58.798 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T06:09:58.798 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:09:58.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:58.799 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:09:58.857 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-21T06:09:58.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:58.860 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-21T06:09:58.916 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-21T06:09:58.917 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:09:58.917 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-21T06:09:58.917 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T06:09:58.917 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-21T06:09:58.917 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:09:58.917 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:09:58.917 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T06:09:58.917 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:09:58.917 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:09:58.917 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:09:58.917 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T06:09:59.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:09:59.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:09:59.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:59.063 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T06:09:59.072 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:09:59.072 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:09:59.073 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:09:59.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:59.073 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-09-21T06:09:59.136 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:09:59.136 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:09:59.188 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:09:59.188 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:09:59.188 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:09:59.239 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:09:59.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:59.239 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:09:59.347 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:09:59.347 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:09:59.347 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:09:59.365 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:09:59.366 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:09:59.366 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:09:59.366 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:09:59.366 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:09:59.377 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:09:59.377 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:09:59.377 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:09:59.377 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:09:59.377 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:09:59.377 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T06:09:59.377 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T06:09:59.377 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T06:09:59.381 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:09:59.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:59.381 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:09:59.401 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:09:59.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:09:59.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:09:59.508 INFO:teuthology.orchestra.run.smithi050.stdout:client7885 2024-09-21T06:09:59.508 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:09:59.508 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:59.517 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T06:09:59.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.729+0000 7f63e1c00640 1 -- 172.21.15.50:0/741145469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63dc0b5c70 msgr2=0x7f63dc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:59.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.729+0000 7f63e1c00640 1 --2- 172.21.15.50:0/741145469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63dc0b5c70 0x7f63dc15bb60 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009bc0 tx=0x7f63cc01cb70 comp rx=0 tx=0).stop 2024-09-21T06:09:59.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.729+0000 7f63e1c00640 1 -- 172.21.15.50:0/741145469 shutdown_connections 2024-09-21T06:09:59.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.729+0000 7f63e1c00640 1 --2- 172.21.15.50:0/741145469 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f63dc0b4de0 0x7f63dc0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.729+0000 7f63e1c00640 1 --2- 172.21.15.50:0/741145469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63dc0b5c70 0x7f63dc15bb60 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.729+0000 7f63e1c00640 1 -- 172.21.15.50:0/741145469 >> 172.21.15.50:0/741145469 conn(0x7f63dc0b2040 msgr2=0x7f63dc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:59.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.729+0000 7f63e1c00640 1 -- 172.21.15.50:0/741145469 shutdown_connections 2024-09-21T06:09:59.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.729+0000 7f63e1c00640 1 -- 172.21.15.50:0/741145469 wait complete. 2024-09-21T06:09:59.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.729+0000 7f63e1c00640 1 Processor -- start 2024-09-21T06:09:59.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.729+0000 7f63e1c00640 1 -- start start 2024-09-21T06:09:59.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63e1c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63dc0b4de0 0x7f63dc1498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:59.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63e1c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f63dc0b5c70 0x7f63dc15af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:59.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63e1c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc1527b0 0x7f63dc14b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:59.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63e1c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc168970 con 0x7f63dc0b4de0 2024-09-21T06:09:59.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63e1c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1687f0 con 0x7f63dc1527b0 2024-09-21T06:09:59.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63e1c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f63dc168af0 con 0x7f63dc0b5c70 2024-09-21T06:09:59.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63dc0b4de0 0x7f63dc1498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:59.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63dc0b4de0 0x7f63dc1498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52542/0 (socket says 172.21.15.50:52542) 2024-09-21T06:09:59.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63daa00640 1 -- 172.21.15.50:0/1960625181 learned_addr learned my addr 172.21.15.50:0/1960625181 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:09:59.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63db400640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc1527b0 0x7f63dc14b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:59.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63d2000640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f63dc0b5c70 0x7f63dc15af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:59.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63daa00640 1 -- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f63dc0b5c70 msgr2=0x7f63dc15af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:59.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63daa00640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f63dc0b5c70 0x7f63dc15af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63daa00640 1 -- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc1527b0 msgr2=0x7f63dc14b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:59.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63daa00640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc1527b0 0x7f63dc14b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63daa00640 1 -- 172.21.15.50:0/1960625181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc15aa50 con 0x7f63dc0b4de0 2024-09-21T06:09:59.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63daa00640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63dc0b4de0 0x7f63dc1498b0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f63cc01c5e0 tx=0x7f63cc009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:59.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63d3400640 1 -- 172.21.15.50:0/1960625181 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc0b2040 con 0x7f63dc0b4de0 2024-09-21T06:09:59.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63d3400640 1 -- 172.21.15.50:0/1960625181 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63cc004040 con 0x7f63dc0b4de0 2024-09-21T06:09:59.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63e1c00640 1 -- 172.21.15.50:0/1960625181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63dc159a00 con 0x7f63dc0b4de0 2024-09-21T06:09:59.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63d3400640 1 -- 172.21.15.50:0/1960625181 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc004410 con 0x7f63dc0b4de0 2024-09-21T06:09:59.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.733+0000 7f63e1c00640 1 -- 172.21.15.50:0/1960625181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63dc159fc0 con 0x7f63dc0b4de0 2024-09-21T06:09:59.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.737+0000 7f63e1c00640 1 -- 172.21.15.50:0/1960625181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63dc0b51f0 con 0x7f63dc0b4de0 2024-09-21T06:09:59.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.737+0000 7f63d3400640 1 -- 172.21.15.50:0/1960625181 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f63cc0bd030 con 0x7f63dc0b4de0 2024-09-21T06:09:59.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.737+0000 7f63d3400640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63b8074c70 0x7f63b8077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:59.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.737+0000 7f63d3400640 1 -- 172.21.15.50:0/1960625181 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f63cc12d150 con 0x7f63dc0b4de0 2024-09-21T06:09:59.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.737+0000 7f63d2000640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63b8074c70 0x7f63b8077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:59.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.737+0000 7f63d2000640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63b8074c70 0x7f63b8077180 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f63c4007fd0 tx=0x7f63c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:59.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.741+0000 7f63d3400640 1 -- 172.21.15.50:0/1960625181 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f63cc0c4060 con 0x7f63dc0b4de0 2024-09-21T06:09:59.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:09:59.957+0000 7f63e1c00640 1 -- 172.21.15.50:0/1960625181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f63dc149df0 con 0x7f63dc0b4de0 2024-09-21T06:10:00.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63d3400640 1 -- 172.21.15.50:0/1960625181 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v75) ==== 136+0+0 (secure 0 0 0) 0x7f63cc016790 con 0x7f63dc0b4de0 2024-09-21T06:10:00.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63e1c00640 1 -- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63b8074c70 msgr2=0x7f63b8077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:00.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63e1c00640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63b8074c70 0x7f63b8077180 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f63c4007fd0 tx=0x7f63c4009040 comp rx=0 tx=0).stop 2024-09-21T06:10:00.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63e1c00640 1 -- 172.21.15.50:0/1960625181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63dc0b4de0 msgr2=0x7f63dc1498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:00.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63e1c00640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63dc0b4de0 0x7f63dc1498b0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f63cc01c5e0 tx=0x7f63cc009280 comp rx=0 tx=0).stop 2024-09-21T06:10:00.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63e1c00640 1 -- 172.21.15.50:0/1960625181 shutdown_connections 2024-09-21T06:10:00.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63e1c00640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc1527b0 0x7f63dc14b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63e1c00640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f63dc0b5c70 0x7f63dc15af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63e1c00640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63b8074c70 0x7f63b8077180 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63e1c00640 1 --2- 172.21.15.50:0/1960625181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63dc0b4de0 0x7f63dc1498b0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63e1c00640 1 -- 172.21.15.50:0/1960625181 >> 172.21.15.50:0/1960625181 conn(0x7f63dc0b2040 msgr2=0x7f63dc150aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:00.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63e1c00640 1 -- 172.21.15.50:0/1960625181 shutdown_connections 2024-09-21T06:10:00.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.601+0000 7f63e1c00640 1 -- 172.21.15.50:0/1960625181 wait complete. 2024-09-21T06:10:00.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:10:00.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.789+0000 7f3fa0400640 1 -- 172.21.15.50:0/2327654760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f980b4de0 msgr2=0x7f3f980b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:00.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.789+0000 7f3fa0400640 1 --2- 172.21.15.50:0/2327654760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f980b4de0 0x7f3f980b51f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009c80 tx=0x7f3f9001cce0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.789+0000 7f3fa0400640 1 -- 172.21.15.50:0/2327654760 shutdown_connections 2024-09-21T06:10:00.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.789+0000 7f3fa0400640 1 --2- 172.21.15.50:0/2327654760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f980b4de0 0x7f3f980b51f0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.789+0000 7f3fa0400640 1 --2- 172.21.15.50:0/2327654760 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f980b5c70 0x7f3f9815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.789+0000 7f3fa0400640 1 -- 172.21.15.50:0/2327654760 >> 172.21.15.50:0/2327654760 conn(0x7f3f980b2040 msgr2=0x7f3f980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:00.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.789+0000 7f3fa0400640 1 -- 172.21.15.50:0/2327654760 shutdown_connections 2024-09-21T06:10:00.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.789+0000 7f3fa0400640 1 -- 172.21.15.50:0/2327654760 wait complete. 2024-09-21T06:10:00.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3fa0400640 1 Processor -- start 2024-09-21T06:10:00.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3fa0400640 1 -- start start 2024-09-21T06:10:00.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3fa0400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f980b4de0 0x7f3f98141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:00.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3fa0400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f980b5c70 0x7f3f980cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:00.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3fa0400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f981527b0 0x7f3f980cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:00.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3fa0400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3f98168ab0 con 0x7f3f980b4de0 2024-09-21T06:10:00.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3fa0400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3f98168930 con 0x7f3f981527b0 2024-09-21T06:10:00.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3fa0400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3f98168c30 con 0x7f3f980b5c70 2024-09-21T06:10:00.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3f9da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f980b4de0 0x7f3f98141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:00.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3f9da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f980b4de0 0x7f3f98141c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52560/0 (socket says 172.21.15.50:52560) 2024-09-21T06:10:00.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3f9da00640 1 -- 172.21.15.50:0/3880102232 learned_addr learned my addr 172.21.15.50:0/3880102232 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:00.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3f9d000640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f980b5c70 0x7f3f980cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:00.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3f9e400640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f981527b0 0x7f3f980cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:00.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3f9da00640 1 -- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f980b5c70 msgr2=0x7f3f980cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:00.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3f9da00640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f980b5c70 0x7f3f980cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3f9da00640 1 -- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f981527b0 msgr2=0x7f3f980cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:00.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3f9da00640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f981527b0 0x7f3f980cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3f9da00640 1 -- 172.21.15.50:0/3880102232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f981586c0 con 0x7f3f980b4de0 2024-09-21T06:10:00.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.793+0000 7f3f9da00640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f980b4de0 0x7f3f98141c80 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00d780 tx=0x7f3f8c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:00.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.797+0000 7f3f95600640 1 -- 172.21.15.50:0/3880102232 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c017070 con 0x7f3f980b4de0 2024-09-21T06:10:00.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.797+0000 7f3f95600640 1 -- 172.21.15.50:0/3880102232 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f8c00de90 con 0x7f3f980b4de0 2024-09-21T06:10:00.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.797+0000 7f3fa0400640 1 -- 172.21.15.50:0/3880102232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f981589a0 con 0x7f3f980b4de0 2024-09-21T06:10:00.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.797+0000 7f3f95600640 1 -- 172.21.15.50:0/3880102232 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c004020 con 0x7f3f980b4de0 2024-09-21T06:10:00.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.797+0000 7f3fa0400640 1 -- 172.21.15.50:0/3880102232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f980cce40 con 0x7f3f980b4de0 2024-09-21T06:10:00.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.797+0000 7f3fa0400640 1 -- 172.21.15.50:0/3880102232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f980cc950 con 0x7f3f980b4de0 2024-09-21T06:10:00.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.797+0000 7f3f95600640 1 -- 172.21.15.50:0/3880102232 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f3f8c0041c0 con 0x7f3f980b4de0 2024-09-21T06:10:00.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.801+0000 7f3f95600640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f78074ca0 0x7f3f780771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:00.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.801+0000 7f3f95600640 1 -- 172.21.15.50:0/3880102232 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f3f8c093320 con 0x7f3f980b4de0 2024-09-21T06:10:00.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.801+0000 7f3f9d000640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f78074ca0 0x7f3f780771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:00.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.801+0000 7f3f9d000640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f78074ca0 0x7f3f780771b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3f980cec50 tx=0x7f3f84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:00.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:00.805+0000 7f3f95600640 1 -- 172.21.15.50:0/3880102232 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3f8c020020 con 0x7f3f980b4de0 2024-09-21T06:10:00.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.001+0000 7f3fa0400640 1 -- 172.21.15.50:0/3880102232 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f3f9806b540 con 0x7f3f78074ca0 2024-09-21T06:10:01.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3f95600640 1 -- 172.21.15.50:0/3880102232 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f3f84001d20 con 0x7f3f78074ca0 2024-09-21T06:10:01.000 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:10:01.000 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:10:01.001 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:10:01.001 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:01.001 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:10:01.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3fa0400640 1 -- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f78074ca0 msgr2=0x7f3f780771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3fa0400640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f78074ca0 0x7f3f780771b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3f980cec50 tx=0x7f3f84009040 comp rx=0 tx=0).stop 2024-09-21T06:10:01.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3fa0400640 1 -- 172.21.15.50:0/3880102232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f980b4de0 msgr2=0x7f3f98141c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3fa0400640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f980b4de0 0x7f3f98141c80 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00d780 tx=0x7f3f8c00dc40 comp rx=0 tx=0).stop 2024-09-21T06:10:01.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3fa0400640 1 -- 172.21.15.50:0/3880102232 shutdown_connections 2024-09-21T06:10:01.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3fa0400640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f981527b0 0x7f3f980cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3fa0400640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f980b5c70 0x7f3f980cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3fa0400640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f78074ca0 0x7f3f780771b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3fa0400640 1 --2- 172.21.15.50:0/3880102232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f980b4de0 0x7f3f98141c80 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3fa0400640 1 -- 172.21.15.50:0/3880102232 >> 172.21.15.50:0/3880102232 conn(0x7f3f980b2040 msgr2=0x7f3f98154bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:01.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3fa0400640 1 -- 172.21.15.50:0/3880102232 shutdown_connections 2024-09-21T06:10:01.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.005+0000 7f3fa0400640 1 -- 172.21.15.50:0/3880102232 wait complete. 2024-09-21T06:10:01.023 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:10:01.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.189+0000 7f818fc00640 1 -- 172.21.15.50:0/1145285764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8188157030 msgr2=0x7f81880b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.189+0000 7f818fc00640 1 --2- 172.21.15.50:0/1145285764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8188157030 0x7f81880b5a80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f817c009840 tx=0x7f817c01c790 comp rx=0 tx=0).stop 2024-09-21T06:10:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.189+0000 7f818fc00640 1 -- 172.21.15.50:0/1145285764 shutdown_connections 2024-09-21T06:10:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.189+0000 7f818fc00640 1 --2- 172.21.15.50:0/1145285764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8188157030 0x7f81880b5a80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.189+0000 7f818fc00640 1 -- 172.21.15.50:0/1145285764 >> 172.21.15.50:0/1145285764 conn(0x7f81880708c0 msgr2=0x7f8188070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.189+0000 7f818fc00640 1 -- 172.21.15.50:0/1145285764 shutdown_connections 2024-09-21T06:10:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.189+0000 7f818fc00640 1 -- 172.21.15.50:0/1145285764 wait complete. 2024-09-21T06:10:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.189+0000 7f818fc00640 1 Processor -- start 2024-09-21T06:10:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.189+0000 7f818fc00640 1 -- start start 2024-09-21T06:10:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818fc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8188157030 0x7f81880b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818fc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8188160d50 0x7f8188146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818fc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8188154930 0x7f81880b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818fc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f818816b8c0 con 0x7f8188154930 2024-09-21T06:10:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818fc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f818816b740 con 0x7f8188157030 2024-09-21T06:10:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818fc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f818816ba40 con 0x7f8188160d50 2024-09-21T06:10:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8188154930 0x7f81880b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8188154930 0x7f81880b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52566/0 (socket says 172.21.15.50:52566) 2024-09-21T06:10:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818dc00640 1 -- 172.21.15.50:0/724050283 learned_addr learned my addr 172.21.15.50:0/724050283 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818d200640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8188157030 0x7f81880b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f8187e00640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8188160d50 0x7f8188146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:01.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818dc00640 1 -- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8188160d50 msgr2=0x7f8188146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818dc00640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8188160d50 0x7f8188146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818dc00640 1 -- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8188157030 msgr2=0x7f81880b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818dc00640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8188157030 0x7f81880b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818dc00640 1 -- 172.21.15.50:0/724050283 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81880b3870 con 0x7f8188154930 2024-09-21T06:10:01.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818dc00640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8188154930 0x7f81880b25d0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f818000c740 tx=0x7f818000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:01.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f8184c00640 1 -- 172.21.15.50:0/724050283 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180003b80 con 0x7f8188154930 2024-09-21T06:10:01.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818fc00640 1 -- 172.21.15.50:0/724050283 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818806e390 con 0x7f8188154930 2024-09-21T06:10:01.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f8184c00640 1 -- 172.21.15.50:0/724050283 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f818000ce50 con 0x7f8188154930 2024-09-21T06:10:01.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f8184c00640 1 -- 172.21.15.50:0/724050283 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818000a850 con 0x7f8188154930 2024-09-21T06:10:01.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818fc00640 1 -- 172.21.15.50:0/724050283 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81880b1dd0 con 0x7f8188154930 2024-09-21T06:10:01.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.193+0000 7f818fc00640 1 -- 172.21.15.50:0/724050283 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f818815dc90 con 0x7f8188154930 2024-09-21T06:10:01.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.197+0000 7f8184c00640 1 -- 172.21.15.50:0/724050283 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f818000a9f0 con 0x7f8188154930 2024-09-21T06:10:01.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.197+0000 7f8184c00640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8168074c70 0x7f8168077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:01.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.197+0000 7f8184c00640 1 -- 172.21.15.50:0/724050283 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f8180092e10 con 0x7f8188154930 2024-09-21T06:10:01.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.197+0000 7f818d200640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8168074c70 0x7f8168077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:01.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.197+0000 7f818d200640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8168074c70 0x7f8168077180 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f817c0af000 tx=0x7f817c01afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:01.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.201+0000 7f8184c00640 1 -- 172.21.15.50:0/724050283 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f818005f240 con 0x7f8188154930 2024-09-21T06:10:01.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.377+0000 7f818fc00640 1 -- 172.21.15.50:0/724050283 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f818815f620 con 0x7f8188154930 2024-09-21T06:10:01.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f8184c00640 1 -- 172.21.15.50:0/724050283 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7f8180063c70 con 0x7f8188154930 2024-09-21T06:10:01.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f818fc00640 1 -- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8168074c70 msgr2=0x7f8168077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f818fc00640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8168074c70 0x7f8168077180 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f817c0af000 tx=0x7f817c01afd0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f818fc00640 1 -- 172.21.15.50:0/724050283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8188154930 msgr2=0x7f81880b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f818fc00640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8188154930 0x7f81880b25d0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f818000c740 tx=0x7f818000cc00 comp rx=0 tx=0).stop 2024-09-21T06:10:01.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f818fc00640 1 -- 172.21.15.50:0/724050283 shutdown_connections 2024-09-21T06:10:01.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f818fc00640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8168074c70 0x7f8168077180 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f818fc00640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8188154930 0x7f81880b25d0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f818fc00640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8188160d50 0x7f8188146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f818fc00640 1 --2- 172.21.15.50:0/724050283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8188157030 0x7f81880b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f818fc00640 1 -- 172.21.15.50:0/724050283 >> 172.21.15.50:0/724050283 conn(0x7f81880708c0 msgr2=0x7f818815cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:01.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f818fc00640 1 -- 172.21.15.50:0/724050283 shutdown_connections 2024-09-21T06:10:01.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.381+0000 7f818fc00640 1 -- 172.21.15.50:0/724050283 wait complete. 2024-09-21T06:10:01.399 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-21T06:10:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.561+0000 7f78aa400640 1 -- 172.21.15.50:0/3056864739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a40d0e40 msgr2=0x7f78a415d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.561+0000 7f78aa400640 1 --2- 172.21.15.50:0/3056864739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a40d0e40 0x7f78a415d600 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f78a40cfcf0 tx=0x7f789401c840 comp rx=0 tx=0).stop 2024-09-21T06:10:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.561+0000 7f78aa400640 1 -- 172.21.15.50:0/3056864739 shutdown_connections 2024-09-21T06:10:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.561+0000 7f78aa400640 1 --2- 172.21.15.50:0/3056864739 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f78a4069330 0x7f78a415dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.561+0000 7f78aa400640 1 --2- 172.21.15.50:0/3056864739 >> v2:172.21.15.88:3300/0 conn(0x7f78a40d1250 0x7f78a4068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.561+0000 7f78aa400640 1 --2- 172.21.15.50:0/3056864739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a40d0e40 0x7f78a415d600 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.561+0000 7f78aa400640 1 -- 172.21.15.50:0/3056864739 >> 172.21.15.50:0/3056864739 conn(0x7f78a40b35f0 msgr2=0x7f78a40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.561+0000 7f78aa400640 1 -- 172.21.15.50:0/3056864739 shutdown_connections 2024-09-21T06:10:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.561+0000 7f78aa400640 1 -- 172.21.15.50:0/3056864739 wait complete. 2024-09-21T06:10:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78aa400640 1 Processor -- start 2024-09-21T06:10:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78aa400640 1 -- start start 2024-09-21T06:10:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78aa400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a4069330 0x7f78a41f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78aa400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78a40d0e40 0x7f78a40afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78aa400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f78a41fc2b0 0x7f78a40ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78aa400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78a406d990 con 0x7f78a4069330 2024-09-21T06:10:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78aa400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78a406d810 con 0x7f78a40d0e40 2024-09-21T06:10:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78aa400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f78a406db10 con 0x7f78a41fc2b0 2024-09-21T06:10:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a4069330 0x7f78a41f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a4069330 0x7f78a41f98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52594/0 (socket says 172.21.15.50:52594) 2024-09-21T06:10:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a3400640 1 -- 172.21.15.50:0/1995111095 learned_addr learned my addr 172.21.15.50:0/1995111095 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a3e00640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f78a41fc2b0 0x7f78a40ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a2a00640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78a40d0e40 0x7f78a40afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a3400640 1 -- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f78a41fc2b0 msgr2=0x7f78a40ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a3400640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f78a41fc2b0 0x7f78a40ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a3400640 1 -- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78a40d0e40 msgr2=0x7f78a40afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a3400640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78a40d0e40 0x7f78a40afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a3400640 1 -- 172.21.15.50:0/1995111095 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a41fce90 con 0x7f78a4069330 2024-09-21T06:10:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a3e00640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f78a41fc2b0 0x7f78a40ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a2a00640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78a40d0e40 0x7f78a40afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f78a3400640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a4069330 0x7f78a41f98b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f78940b0040 tx=0x7f78940a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f789b400640 1 -- 172.21.15.50:0/1995111095 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78940b2040 con 0x7f78a4069330 2024-09-21T06:10:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.565+0000 7f789b400640 1 -- 172.21.15.50:0/1995111095 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7894004430 con 0x7f78a4069330 2024-09-21T06:10:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.569+0000 7f789b400640 1 -- 172.21.15.50:0/1995111095 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894005280 con 0x7f78a4069330 2024-09-21T06:10:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.569+0000 7f78aa400640 1 -- 172.21.15.50:0/1995111095 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a41f8dd0 con 0x7f78a4069330 2024-09-21T06:10:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.569+0000 7f78aa400640 1 -- 172.21.15.50:0/1995111095 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a41fd170 con 0x7f78a4069330 2024-09-21T06:10:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.569+0000 7f78aa400640 1 -- 172.21.15.50:0/1995111095 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a40d1250 con 0x7f78a4069330 2024-09-21T06:10:01.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.569+0000 7f789b400640 1 -- 172.21.15.50:0/1995111095 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f7894005420 con 0x7f78a4069330 2024-09-21T06:10:01.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.569+0000 7f789b400640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7884074c70 0x7f7884077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.573+0000 7f789b400640 1 -- 172.21.15.50:0/1995111095 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f789412d3f0 con 0x7f78a4069330 2024-09-21T06:10:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.573+0000 7f78a2a00640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7884074c70 0x7f7884077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.573+0000 7f78a2a00640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7884074c70 0x7f7884077180 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7890005fd0 tx=0x7f7890005de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:01.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.577+0000 7f789b400640 1 -- 172.21.15.50:0/1995111095 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7894016490 con 0x7f78a4069330 2024-09-21T06:10:01.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.753+0000 7f78aa400640 1 -- 172.21.15.50:0/1995111095 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f78a40b1070 con 0x7f78a4069330 2024-09-21T06:10:01.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.753+0000 7f789b400640 1 -- 172.21.15.50:0/1995111095 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v52) ==== 116+0+2 (secure 0 0 0) 0x7f78940f9820 con 0x7f78a4069330 2024-09-21T06:10:01.751 INFO:teuthology.orchestra.run.smithi050.stdout:4 2024-09-21T06:10:01.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.757+0000 7f78aa400640 1 -- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7884074c70 msgr2=0x7f7884077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.757+0000 7f78aa400640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7884074c70 0x7f7884077180 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7890005fd0 tx=0x7f7890005de0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.757+0000 7f78aa400640 1 -- 172.21.15.50:0/1995111095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a4069330 msgr2=0x7f78a41f98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.757+0000 7f78aa400640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a4069330 0x7f78a41f98b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f78940b0040 tx=0x7f78940a57b0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.757+0000 7f78aa400640 1 -- 172.21.15.50:0/1995111095 shutdown_connections 2024-09-21T06:10:01.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.757+0000 7f78aa400640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f78a41fc2b0 0x7f78a40ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.757+0000 7f78aa400640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78a40d0e40 0x7f78a40afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.757+0000 7f78aa400640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7884074c70 0x7f7884077180 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.757+0000 7f78aa400640 1 --2- 172.21.15.50:0/1995111095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a4069330 0x7f78a41f98b0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.757+0000 7f78aa400640 1 -- 172.21.15.50:0/1995111095 >> 172.21.15.50:0/1995111095 conn(0x7f78a40b35f0 msgr2=0x7f78a415eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:01.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.757+0000 7f78aa400640 1 -- 172.21.15.50:0/1995111095 shutdown_connections 2024-09-21T06:10:01.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.757+0000 7f78aa400640 1 -- 172.21.15.50:0/1995111095 wait complete. 2024-09-21T06:10:01.774 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:10:01.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.941+0000 7f6406c00640 1 -- 172.21.15.50:0/4152556040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e40039f0 msgr2=0x7f63e4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.941+0000 7f6406c00640 1 --2- 172.21.15.50:0/4152556040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e40039f0 0x7f63e4003e20 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009b40 tx=0x7f63f001d240 comp rx=0 tx=0).stop 2024-09-21T06:10:01.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.941+0000 7f6404c00640 1 -- 172.21.15.50:0/4152556040 reap_dead start 2024-09-21T06:10:01.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.941+0000 7f6406c00640 1 -- 172.21.15.50:0/4152556040 shutdown_connections 2024-09-21T06:10:01.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.941+0000 7f6406c00640 1 -- 172.21.15.50:0/4152556040 >> 172.21.15.50:0/4152556040 conn(0x7f64000b2040 msgr2=0x7f64000b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:01.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.941+0000 7f6406c00640 1 -- 172.21.15.50:0/4152556040 shutdown_connections 2024-09-21T06:10:01.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.941+0000 7f6406c00640 1 -- 172.21.15.50:0/4152556040 wait complete. 2024-09-21T06:10:01.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6406c00640 1 Processor -- start 2024-09-21T06:10:01.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6406c00640 1 -- start start 2024-09-21T06:10:01.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6406c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e40039f0 0x7f64000ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:01.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6406c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f64000aee50 0x7f6400154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:01.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6406c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6400154b40 0x7f6400154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:01.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6406c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f64000b4590 con 0x7f6400154b40 2024-09-21T06:10:01.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6406c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64000b4410 con 0x7f63e40039f0 2024-09-21T06:10:01.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6406c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f64000b4710 con 0x7f64000aee50 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6404c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6400154b40 0x7f6400154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6404c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6400154b40 0x7f6400154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52604/0 (socket says 172.21.15.50:52604) 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6404c00640 1 -- 172.21.15.50:0/484199034 learned_addr learned my addr 172.21.15.50:0/484199034 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f63ffe00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e40039f0 0x7f64000ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f63f7e00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f64000aee50 0x7f6400154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6404c00640 1 -- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f64000aee50 msgr2=0x7f6400154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6404c00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f64000aee50 0x7f6400154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6404c00640 1 -- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e40039f0 msgr2=0x7f64000ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6404c00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e40039f0 0x7f64000ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6404c00640 1 -- 172.21.15.50:0/484199034 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64000b1e70 con 0x7f6400154b40 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f63ffe00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e40039f0 0x7f64000ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6404c00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6400154b40 0x7f6400154f50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f63f8000f90 tx=0x7f63f8006420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f63fcc00640 1 -- 172.21.15.50:0/484199034 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8010890 con 0x7f6400154b40 2024-09-21T06:10:01.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f63fcc00640 1 -- 172.21.15.50:0/484199034 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63f8017070 con 0x7f6400154b40 2024-09-21T06:10:01.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f63fcc00640 1 -- 172.21.15.50:0/484199034 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8010de0 con 0x7f6400154b40 2024-09-21T06:10:01.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6406c00640 1 -- 172.21.15.50:0/484199034 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64000af260 con 0x7f6400154b40 2024-09-21T06:10:01.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.945+0000 7f6406c00640 1 -- 172.21.15.50:0/484199034 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64000cfa10 con 0x7f6400154b40 2024-09-21T06:10:01.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.949+0000 7f6406c00640 1 -- 172.21.15.50:0/484199034 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64000cf4a0 con 0x7f6400154b40 2024-09-21T06:10:01.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.949+0000 7f63fcc00640 1 -- 172.21.15.50:0/484199034 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f63f801c460 con 0x7f6400154b40 2024-09-21T06:10:01.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.949+0000 7f63fcc00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63dc074ca0 0x7f63dc0771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:01.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.949+0000 7f63fcc00640 1 -- 172.21.15.50:0/484199034 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f63f8093a00 con 0x7f6400154b40 2024-09-21T06:10:01.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.949+0000 7f63ffe00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63dc074ca0 0x7f63dc0771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:01.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.953+0000 7f63ffe00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63dc074ca0 0x7f63dc0771b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f63f00ad000 tx=0x7f63f0019fe0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:01.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:01.953+0000 7f63fcc00640 1 -- 172.21.15.50:0/484199034 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f64000af260 con 0x7f6400154b40 2024-09-21T06:10:02.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.133+0000 7f6406c00640 1 -- 172.21.15.50:0/484199034 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f64000b51f0 con 0x7f6400154b40 2024-09-21T06:10:02.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.137+0000 7f63fcc00640 1 -- 172.21.15.50:0/484199034 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v53)=0 v53) ==== 131+0+0 (secure 0 0 0) 0x7f63f8060480 con 0x7f6400154b40 2024-09-21T06:10:02.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.141+0000 7f6406c00640 1 -- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63dc074ca0 msgr2=0x7f63dc0771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.141+0000 7f6406c00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63dc074ca0 0x7f63dc0771b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f63f00ad000 tx=0x7f63f0019fe0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.141+0000 7f6406c00640 1 -- 172.21.15.50:0/484199034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6400154b40 msgr2=0x7f6400154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.141+0000 7f6406c00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6400154b40 0x7f6400154f50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f63f8000f90 tx=0x7f63f8006420 comp rx=0 tx=0).stop 2024-09-21T06:10:02.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.141+0000 7f6406c00640 1 -- 172.21.15.50:0/484199034 shutdown_connections 2024-09-21T06:10:02.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.141+0000 7f6406c00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f63dc074ca0 0x7f63dc0771b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.141+0000 7f6406c00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6400154b40 0x7f6400154f50 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.141+0000 7f6406c00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f64000aee50 0x7f6400154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.141+0000 7f6406c00640 1 --2- 172.21.15.50:0/484199034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e40039f0 0x7f64000ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.141+0000 7f6406c00640 1 -- 172.21.15.50:0/484199034 >> 172.21.15.50:0/484199034 conn(0x7f64000b2040 msgr2=0x7f64000b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:02.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.141+0000 7f6406c00640 1 -- 172.21.15.50:0/484199034 shutdown_connections 2024-09-21T06:10:02.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.141+0000 7f6406c00640 1 -- 172.21.15.50:0/484199034 wait complete. 2024-09-21T06:10:02.157 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-21T06:10:02.161 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:02.158+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:02.161 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:02.158+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:02.161 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:02.158+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:02.161 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:02.158+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:02.161 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:02.158+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:02.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.325+0000 7f8a30a00640 1 -- 172.21.15.50:0/3307597307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a2c0b4de0 msgr2=0x7f8a2c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.325+0000 7f8a30a00640 1 --2- 172.21.15.50:0/3307597307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a2c0b4de0 0x7f8a2c0b51f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009c80 tx=0x7f8a2401cd70 comp rx=0 tx=0).stop 2024-09-21T06:10:02.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.325+0000 7f8a30a00640 1 -- 172.21.15.50:0/3307597307 shutdown_connections 2024-09-21T06:10:02.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.325+0000 7f8a30a00640 1 --2- 172.21.15.50:0/3307597307 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a2c0b4de0 0x7f8a2c0b51f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.325+0000 7f8a30a00640 1 --2- 172.21.15.50:0/3307597307 >> v2:172.21.15.88:3300/0 conn(0x7f8a2c1527b0 0x7f8a2c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.325+0000 7f8a30a00640 1 -- 172.21.15.50:0/3307597307 >> 172.21.15.50:0/3307597307 conn(0x7f8a2c0b2040 msgr2=0x7f8a2c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:02.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.325+0000 7f8a30a00640 1 -- 172.21.15.50:0/3307597307 shutdown_connections 2024-09-21T06:10:02.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.325+0000 7f8a30a00640 1 -- 172.21.15.50:0/3307597307 wait complete. 2024-09-21T06:10:02.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.325+0000 7f8a30a00640 1 Processor -- start 2024-09-21T06:10:02.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.325+0000 7f8a30a00640 1 -- start start 2024-09-21T06:10:02.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a30a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a2c0b4de0 0x7f8a2c156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:02.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a30a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a2c0b5c70 0x7f8a2c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:02.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a30a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a2c1527b0 0x7f8a2c0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:02.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a30a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1689d0 con 0x7f8a2c0b5c70 2024-09-21T06:10:02.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a30a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c168850 con 0x7f8a2c0b4de0 2024-09-21T06:10:02.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a30a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2c168b50 con 0x7f8a2c1527b0 2024-09-21T06:10:02.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a2c0b5c70 0x7f8a2c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:02.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a2a800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a2c1527b0 0x7f8a2c0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:02.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a2c0b5c70 0x7f8a2c0cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52614/0 (socket says 172.21.15.50:52614) 2024-09-21T06:10:02.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a23e00640 1 -- 172.21.15.50:0/1149978970 learned_addr learned my addr 172.21.15.50:0/1149978970 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:02.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a29e00640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a2c0b4de0 0x7f8a2c156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:02.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a23e00640 1 -- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a2c1527b0 msgr2=0x7f8a2c0cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a23e00640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a2c1527b0 0x7f8a2c0cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a23e00640 1 -- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a2c0b4de0 msgr2=0x7f8a2c156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a23e00640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a2c0b4de0 0x7f8a2c156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a23e00640 1 -- 172.21.15.50:0/1149978970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c1f8710 con 0x7f8a2c0b5c70 2024-09-21T06:10:02.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a23e00640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a2c0b5c70 0x7f8a2c0cc950 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400e7a0 tx=0x7f8a1400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:02.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a22000640 1 -- 172.21.15.50:0/1149978970 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400ccc0 con 0x7f8a2c0b5c70 2024-09-21T06:10:02.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a22000640 1 -- 172.21.15.50:0/1149978970 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a14010430 con 0x7f8a2c0b5c70 2024-09-21T06:10:02.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a22000640 1 -- 172.21.15.50:0/1149978970 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14010860 con 0x7f8a2c0b5c70 2024-09-21T06:10:02.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a30a00640 1 -- 172.21.15.50:0/1149978970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2c1566c0 con 0x7f8a2c0b5c70 2024-09-21T06:10:02.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a30a00640 1 -- 172.21.15.50:0/1149978970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a2c0cdda0 con 0x7f8a2c0b5c70 2024-09-21T06:10:02.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.329+0000 7f8a30a00640 1 -- 172.21.15.50:0/1149978970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a2c0b51f0 con 0x7f8a2c0b5c70 2024-09-21T06:10:02.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.333+0000 7f8a22000640 1 -- 172.21.15.50:0/1149978970 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f8a14010a00 con 0x7f8a2c0b5c70 2024-09-21T06:10:02.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.333+0000 7f8a22000640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a08074ca0 0x7f8a080771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:02.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.333+0000 7f8a22000640 1 -- 172.21.15.50:0/1149978970 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f8a14093a70 con 0x7f8a2c0b5c70 2024-09-21T06:10:02.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.333+0000 7f8a29e00640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a08074ca0 0x7f8a080771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:02.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.333+0000 7f8a29e00640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a08074ca0 0x7f8a080771b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c007fd0 tx=0x7f8a1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:02.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.337+0000 7f8a22000640 1 -- 172.21.15.50:0/1149978970 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8a1405ff40 con 0x7f8a2c0b5c70 2024-09-21T06:10:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.541+0000 7f8a30a00640 1 -- 172.21.15.50:0/1149978970 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f8a2c06b540 con 0x7f8a08074ca0 2024-09-21T06:10:02.548 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:02.546+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:02.548 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:02.546+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:02.549 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:02.546+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:02.549 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:02.546+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:02.549 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:02.546+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:02.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a22000640 1 -- 172.21.15.50:0/1149978970 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8a1c001410 con 0x7f8a08074ca0 2024-09-21T06:10:02.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a30a00640 1 -- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a08074ca0 msgr2=0x7f8a080771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a30a00640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a08074ca0 0x7f8a080771b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c007fd0 tx=0x7f8a1c009040 comp rx=0 tx=0).stop 2024-09-21T06:10:02.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a30a00640 1 -- 172.21.15.50:0/1149978970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a2c0b5c70 msgr2=0x7f8a2c0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a30a00640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a2c0b5c70 0x7f8a2c0cc950 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400e7a0 tx=0x7f8a1400ec60 comp rx=0 tx=0).stop 2024-09-21T06:10:02.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a30a00640 1 -- 172.21.15.50:0/1149978970 shutdown_connections 2024-09-21T06:10:02.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a30a00640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a2c1527b0 0x7f8a2c0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a30a00640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a08074ca0 0x7f8a080771b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a30a00640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a2c0b5c70 0x7f8a2c0cc950 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a30a00640 1 --2- 172.21.15.50:0/1149978970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a2c0b4de0 0x7f8a2c156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a30a00640 1 -- 172.21.15.50:0/1149978970 >> 172.21.15.50:0/1149978970 conn(0x7f8a2c0b2040 msgr2=0x7f8a2c0b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:02.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a30a00640 1 -- 172.21.15.50:0/1149978970 shutdown_connections 2024-09-21T06:10:02.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.589+0000 7f8a30a00640 1 -- 172.21.15.50:0/1149978970 wait complete. 2024-09-21T06:10:02.607 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:10:02.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.773+0000 7f0f64c00640 1 -- 172.21.15.50:0/1707091645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f44003a60 msgr2=0x7f0f44003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.773+0000 7f0f64c00640 1 --2- 172.21.15.50:0/1707091645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f44003a60 0x7f0f44003ed0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0f500016f0 tx=0x7f0f5001c720 comp rx=0 tx=0).stop 2024-09-21T06:10:02.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.773+0000 7f0f64c00640 1 -- 172.21.15.50:0/1707091645 shutdown_connections 2024-09-21T06:10:02.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.773+0000 7f0f64c00640 1 --2- 172.21.15.50:0/1707091645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f44003a60 0x7f0f44003ed0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.773+0000 7f0f64c00640 1 -- 172.21.15.50:0/1707091645 >> 172.21.15.50:0/1707091645 conn(0x7f0f600b2040 msgr2=0x7f0f600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:02.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.773+0000 7f0f64c00640 1 -- 172.21.15.50:0/1707091645 shutdown_connections 2024-09-21T06:10:02.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.773+0000 7f0f64c00640 1 -- 172.21.15.50:0/1707091645 wait complete. 2024-09-21T06:10:02.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.773+0000 7f0f64c00640 1 Processor -- start 2024-09-21T06:10:02.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.773+0000 7f0f64c00640 1 -- start start 2024-09-21T06:10:02.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f64c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f44003a60 0x7f0f600ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:02.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f64c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f6014b0f0 0x7f0f60147a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f64c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f60157f00 0x7f0f60158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f64c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6015f620 con 0x7f0f6014b0f0 2024-09-21T06:10:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f64c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6015f4a0 con 0x7f0f60157f00 2024-09-21T06:10:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f64c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6015f7a0 con 0x7f0f44003a60 2024-09-21T06:10:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f55400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f6014b0f0 0x7f0f60147a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f55400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f6014b0f0 0x7f0f60147a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52630/0 (socket says 172.21.15.50:52630) 2024-09-21T06:10:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f55400640 1 -- 172.21.15.50:0/1939268688 learned_addr learned my addr 172.21.15.50:0/1939268688 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f5e800640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f60157f00 0x7f0f60158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:02.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f5de00640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f44003a60 0x7f0f600ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:02.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f55400640 1 -- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f44003a60 msgr2=0x7f0f600ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f55400640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f44003a60 0x7f0f600ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f55400640 1 -- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f60157f00 msgr2=0x7f0f60158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f55400640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f60157f00 0x7f0f60158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f55400640 1 -- 172.21.15.50:0/1939268688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f60158850 con 0x7f0f6014b0f0 2024-09-21T06:10:02.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f55400640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f6014b0f0 0x7f0f60147a20 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0f48000fc0 tx=0x7f0f48001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:02.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f56a00640 1 -- 172.21.15.50:0/1939268688 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48010890 con 0x7f0f6014b0f0 2024-09-21T06:10:02.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f56a00640 1 -- 172.21.15.50:0/1939268688 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f48016070 con 0x7f0f6014b0f0 2024-09-21T06:10:02.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f64c00640 1 -- 172.21.15.50:0/1939268688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f600af170 con 0x7f0f6014b0f0 2024-09-21T06:10:02.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f56a00640 1 -- 172.21.15.50:0/1939268688 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48010dc0 con 0x7f0f6014b0f0 2024-09-21T06:10:02.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f64c00640 1 -- 172.21.15.50:0/1939268688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f60154270 con 0x7f0f6014b0f0 2024-09-21T06:10:02.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.777+0000 7f0f64c00640 1 -- 172.21.15.50:0/1939268688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6015fc90 con 0x7f0f6014b0f0 2024-09-21T06:10:02.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.781+0000 7f0f56a00640 1 -- 172.21.15.50:0/1939268688 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f0f4801b460 con 0x7f0f6014b0f0 2024-09-21T06:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.781+0000 7f0f56a00640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f3c074c70 0x7f0f3c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.781+0000 7f0f56a00640 1 -- 172.21.15.50:0/1939268688 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f0f480935d0 con 0x7f0f6014b0f0 2024-09-21T06:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.781+0000 7f0f5de00640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f3c074c70 0x7f0f3c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.781+0000 7f0f5de00640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f3c074c70 0x7f0f3c077180 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0f500016c0 tx=0x7f0f50019510 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.785+0000 7f0f56a00640 1 -- 172.21.15.50:0/1939268688 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0f60158850 con 0x7f0f6014b0f0 2024-09-21T06:10:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.989+0000 7f0f64c00640 1 -- 172.21.15.50:0/1939268688 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f0f6006b540 con 0x7f0f3c074c70 2024-09-21T06:10:02.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.993+0000 7f0f56a00640 1 -- 172.21.15.50:0/1939268688 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f0f500ae040 con 0x7f0f3c074c70 2024-09-21T06:10:02.991 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/09ac0379-02f7-4c55-a6aa-7355563232b1 2024-09-21T06:10:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.997+0000 7f0f64c00640 1 -- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f3c074c70 msgr2=0x7f0f3c077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.997+0000 7f0f64c00640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f3c074c70 0x7f0f3c077180 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0f500016c0 tx=0x7f0f50019510 comp rx=0 tx=0).stop 2024-09-21T06:10:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.997+0000 7f0f64c00640 1 -- 172.21.15.50:0/1939268688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f6014b0f0 msgr2=0x7f0f60147a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.997+0000 7f0f64c00640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f6014b0f0 0x7f0f60147a20 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0f48000fc0 tx=0x7f0f48001830 comp rx=0 tx=0).stop 2024-09-21T06:10:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.997+0000 7f0f64c00640 1 -- 172.21.15.50:0/1939268688 shutdown_connections 2024-09-21T06:10:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.997+0000 7f0f64c00640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f60157f00 0x7f0f60158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.997+0000 7f0f64c00640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f3c074c70 0x7f0f3c077180 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.997+0000 7f0f64c00640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f6014b0f0 0x7f0f60147a20 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.997+0000 7f0f64c00640 1 --2- 172.21.15.50:0/1939268688 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f44003a60 0x7f0f600ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.997+0000 7f0f64c00640 1 -- 172.21.15.50:0/1939268688 >> 172.21.15.50:0/1939268688 conn(0x7f0f600b2040 msgr2=0x7f0f6006e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.997+0000 7f0f64c00640 1 -- 172.21.15.50:0/1939268688 shutdown_connections 2024-09-21T06:10:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:02.997+0000 7f0f64c00640 1 -- 172.21.15.50:0/1939268688 wait complete. 2024-09-21T06:10:03.013 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:10:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.177+0000 7f7e88a00640 1 -- 172.21.15.50:0/3678917666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e841654a0 msgr2=0x7f7e841678c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.177+0000 7f7e88a00640 1 --2- 172.21.15.50:0/3678917666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e841654a0 0x7f7e841678c0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f7e840cfc40 tx=0x7f7e7c01cba0 comp rx=0 tx=0).stop 2024-09-21T06:10:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.177+0000 7f7e88a00640 1 -- 172.21.15.50:0/3678917666 shutdown_connections 2024-09-21T06:10:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.177+0000 7f7e88a00640 1 --2- 172.21.15.50:0/3678917666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e841654a0 0x7f7e841678c0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.177+0000 7f7e88a00640 1 -- 172.21.15.50:0/3678917666 >> 172.21.15.50:0/3678917666 conn(0x7f7e840b2040 msgr2=0x7f7e840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.177+0000 7f7e88a00640 1 -- 172.21.15.50:0/3678917666 shutdown_connections 2024-09-21T06:10:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.177+0000 7f7e88a00640 1 -- 172.21.15.50:0/3678917666 wait complete. 2024-09-21T06:10:03.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e88a00640 1 Processor -- start 2024-09-21T06:10:03.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e88a00640 1 -- start start 2024-09-21T06:10:03.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e88a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e840749a0 0x7f7e84149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:03.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e88a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e840cefa0 0x7f7e84149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:03.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e88a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e841654a0 0x7f7e841f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:03.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e88a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8416ae30 con 0x7f7e840cefa0 2024-09-21T06:10:03.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e88a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8416acb0 con 0x7f7e841654a0 2024-09-21T06:10:03.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e88a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8416afb0 con 0x7f7e840749a0 2024-09-21T06:10:03.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e7be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e840cefa0 0x7f7e84149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:03.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e7be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e840cefa0 0x7f7e84149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52644/0 (socket says 172.21.15.50:52644) 2024-09-21T06:10:03.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e7be00640 1 -- 172.21.15.50:0/655639910 learned_addr learned my addr 172.21.15.50:0/655639910 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:03.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e82800640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e841654a0 0x7f7e841f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:03.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e81e00640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e840749a0 0x7f7e84149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:03.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e7be00640 1 -- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e840749a0 msgr2=0x7f7e84149a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:03.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e7be00640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e840749a0 0x7f7e84149a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:03.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e7be00640 1 -- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e841654a0 msgr2=0x7f7e841f9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:03.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e7be00640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e841654a0 0x7f7e841f9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:03.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.181+0000 7f7e7be00640 1 -- 172.21.15.50:0/655639910 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8413eed0 con 0x7f7e840cefa0 2024-09-21T06:10:03.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.185+0000 7f7e7be00640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e840cefa0 0x7f7e84149f50 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00c3f0 tx=0x7f7e6c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:03.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.185+0000 7f7e7a000640 1 -- 172.21.15.50:0/655639910 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c017070 con 0x7f7e840cefa0 2024-09-21T06:10:03.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.185+0000 7f7e7a000640 1 -- 172.21.15.50:0/655639910 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e6c0053c0 con 0x7f7e840cefa0 2024-09-21T06:10:03.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.185+0000 7f7e7a000640 1 -- 172.21.15.50:0/655639910 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c003860 con 0x7f7e840cefa0 2024-09-21T06:10:03.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.185+0000 7f7e88a00640 1 -- 172.21.15.50:0/655639910 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e841422f0 con 0x7f7e840cefa0 2024-09-21T06:10:03.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.185+0000 7f7e88a00640 1 -- 172.21.15.50:0/655639910 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8413e630 con 0x7f7e840cefa0 2024-09-21T06:10:03.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.185+0000 7f7e88a00640 1 -- 172.21.15.50:0/655639910 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e84074db0 con 0x7f7e840cefa0 2024-09-21T06:10:03.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.185+0000 7f7e7a000640 1 -- 172.21.15.50:0/655639910 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 44) ==== 99631+0+0 (secure 0 0 0) 0x7f7e6c01a070 con 0x7f7e840cefa0 2024-09-21T06:10:03.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.185+0000 7f7e7a000640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7e60074c70 0x7f7e60077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:03.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.189+0000 7f7e7a000640 1 -- 172.21.15.50:0/655639910 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f7e6c0923c0 con 0x7f7e840cefa0 2024-09-21T06:10:03.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.189+0000 7f7e81e00640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7e60074c70 0x7f7e60077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:03.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.189+0000 7f7e81e00640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7e60074c70 0x7f7e60077180 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7e74007d70 tx=0x7f7e74007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:03.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.193+0000 7f7e7a000640 1 -- 172.21.15.50:0/655639910 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7e6c05e880 con 0x7f7e840cefa0 2024-09-21T06:10:03.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.397+0000 7f7e88a00640 1 -- 172.21.15.50:0/655639910 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f7e8406ed80 con 0x7f7e60074c70 2024-09-21T06:10:03.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.401+0000 7f7e7a000640 1 -- 172.21.15.50:0/655639910 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f7e8406ed80 con 0x7f7e60074c70 2024-09-21T06:10:03.399 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/09ac0379-02f7-4c55-a6aa-7355563232b1 2024-09-21T06:10:03.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.405+0000 7f7e88a00640 1 -- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7e60074c70 msgr2=0x7f7e60077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.405+0000 7f7e88a00640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7e60074c70 0x7f7e60077180 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7e74007d70 tx=0x7f7e74007d00 comp rx=0 tx=0).stop 2024-09-21T06:10:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.405+0000 7f7e88a00640 1 -- 172.21.15.50:0/655639910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e840cefa0 msgr2=0x7f7e84149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.405+0000 7f7e88a00640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e840cefa0 0x7f7e84149f50 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00c3f0 tx=0x7f7e6c00c8b0 comp rx=0 tx=0).stop 2024-09-21T06:10:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.405+0000 7f7e88a00640 1 -- 172.21.15.50:0/655639910 shutdown_connections 2024-09-21T06:10:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.405+0000 7f7e88a00640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e841654a0 0x7f7e841f9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.405+0000 7f7e88a00640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7e60074c70 0x7f7e60077180 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.405+0000 7f7e88a00640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e840cefa0 0x7f7e84149f50 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.405+0000 7f7e88a00640 1 --2- 172.21.15.50:0/655639910 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7e840749a0 0x7f7e84149a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.405+0000 7f7e88a00640 1 -- 172.21.15.50:0/655639910 >> 172.21.15.50:0/655639910 conn(0x7f7e840b2040 msgr2=0x7f7e840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.405+0000 7f7e88a00640 1 -- 172.21.15.50:0/655639910 shutdown_connections 2024-09-21T06:10:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:03.405+0000 7f7e88a00640 1 -- 172.21.15.50:0/655639910 wait complete. 2024-09-21T06:10:03.421 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/09ac0379-02f7-4c55-a6aa-7355563232b1 2024-09-21T06:10:03.422 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:10:03.422 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:10:03.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:10:03.430 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/09ac0379-02f7-4c55-a6aa-7355563232b1/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:10:12.996 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:10:12.996 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:10:12.996 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.50693 s, 113 MB/s 2024-09-21T06:10:12.997 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:10:12.998 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:10:13.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:10:13.007 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/09ac0379-02f7-4c55-a6aa-7355563232b1/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:10:18.593 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:10:18.598 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:10:22.710 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:10:22.710 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:10:22.710 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.65314 s, 111 MB/s 2024-09-21T06:10:22.711 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:10:22.711 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:10:22.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:10:22.720 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/09ac0379-02f7-4c55-a6aa-7355563232b1/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:10:31.870 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:10:31.873 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:10:31.873 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.09792 s, 118 MB/s 2024-09-21T06:10:31.874 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-21T06:10:32.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.044+0000 7f4a72800640 1 -- 172.21.15.50:0/4126627331 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a6c074b20 msgr2=0x7f4a6c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.044+0000 7f4a72800640 1 --2- 172.21.15.50:0/4126627331 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a6c074b20 0x7f4a6c15cf90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009bf0 tx=0x7f4a6401cbe0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.044+0000 7f4a72800640 1 -- 172.21.15.50:0/4126627331 shutdown_connections 2024-09-21T06:10:32.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.044+0000 7f4a72800640 1 --2- 172.21.15.50:0/4126627331 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a6c074b20 0x7f4a6c15cf90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.044+0000 7f4a72800640 1 -- 172.21.15.50:0/4126627331 >> 172.21.15.50:0/4126627331 conn(0x7f4a6c0708c0 msgr2=0x7f4a6c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:32.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.044+0000 7f4a72800640 1 -- 172.21.15.50:0/4126627331 shutdown_connections 2024-09-21T06:10:32.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.044+0000 7f4a72800640 1 -- 172.21.15.50:0/4126627331 wait complete. 2024-09-21T06:10:32.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.044+0000 7f4a72800640 1 Processor -- start 2024-09-21T06:10:32.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.044+0000 7f4a72800640 1 -- start start 2024-09-21T06:10:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a72800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a6c0b5a80 0x7f4a6c1f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a72800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a6c157030 0x7f4a6c1386d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a72800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a6c15c060 0x7f4a6c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a72800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c16b5a0 con 0x7f4a6c0b5a80 2024-09-21T06:10:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a72800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c16b420 con 0x7f4a6c15c060 2024-09-21T06:10:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a72800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6c16b720 con 0x7f4a6c157030 2024-09-21T06:10:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a6b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a6c0b5a80 0x7f4a6c1f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a6b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a6c0b5a80 0x7f4a6c1f9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54426/0 (socket says 172.21.15.50:54426) 2024-09-21T06:10:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a6b400640 1 -- 172.21.15.50:0/3714819814 learned_addr learned my addr 172.21.15.50:0/3714819814 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a6be00640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a6c15c060 0x7f4a6c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a62a00640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a6c157030 0x7f4a6c1386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a6b400640 1 -- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a6c157030 msgr2=0x7f4a6c1386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a6b400640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a6c157030 0x7f4a6c1386d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a6b400640 1 -- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a6c15c060 msgr2=0x7f4a6c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a6b400640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a6c15c060 0x7f4a6c15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a6b400640 1 -- 172.21.15.50:0/3714819814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6c15c9b0 con 0x7f4a6c0b5a80 2024-09-21T06:10:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a6be00640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a6c15c060 0x7f4a6c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a62a00640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a6c157030 0x7f4a6c1386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a6b400640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a6c0b5a80 0x7f4a6c1f9a40 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00c410 tx=0x7f4a5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a63e00640 1 -- 172.21.15.50:0/3714819814 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c011790 con 0x7f4a6c0b5a80 2024-09-21T06:10:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a63e00640 1 -- 172.21.15.50:0/3714819814 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a5c017070 con 0x7f4a6c0b5a80 2024-09-21T06:10:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a63e00640 1 -- 172.21.15.50:0/3714819814 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c011d40 con 0x7f4a6c0b5a80 2024-09-21T06:10:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a72800640 1 -- 172.21.15.50:0/3714819814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6c1f9f80 con 0x7f4a6c0b5a80 2024-09-21T06:10:32.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a72800640 1 -- 172.21.15.50:0/3714819814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a6c1fc8c0 con 0x7f4a6c0b5a80 2024-09-21T06:10:32.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.048+0000 7f4a72800640 1 -- 172.21.15.50:0/3714819814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a6c15cf90 con 0x7f4a6c0b5a80 2024-09-21T06:10:32.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.052+0000 7f4a63e00640 1 -- 172.21.15.50:0/3714819814 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 45) ==== 99733+0+0 (secure 0 0 0) 0x7f4a5c01c460 con 0x7f4a6c0b5a80 2024-09-21T06:10:32.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.052+0000 7f4a63e00640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a48074d90 0x7f4a480772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:32.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.052+0000 7f4a63e00640 1 -- 172.21.15.50:0/3714819814 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f4a5c093a70 con 0x7f4a6c0b5a80 2024-09-21T06:10:32.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.052+0000 7f4a62a00640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a48074d90 0x7f4a480772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:32.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.052+0000 7f4a62a00640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a48074d90 0x7f4a480772a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c14d740 tx=0x7f4a54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:32.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.056+0000 7f4a63e00640 1 -- 172.21.15.50:0/3714819814 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4a6c15c9b0 con 0x7f4a6c0b5a80 2024-09-21T06:10:32.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.268+0000 7f4a72800640 1 -- 172.21.15.50:0/3714819814 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f4a6c0b36d0 con 0x7f4a48074d90 2024-09-21T06:10:32.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a63e00640 1 -- 172.21.15.50:0/3714819814 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4a54001300 con 0x7f4a48074d90 2024-09-21T06:10:32.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a72800640 1 -- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a48074d90 msgr2=0x7f4a480772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a72800640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a48074d90 0x7f4a480772a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c14d740 tx=0x7f4a54009040 comp rx=0 tx=0).stop 2024-09-21T06:10:32.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a72800640 1 -- 172.21.15.50:0/3714819814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a6c0b5a80 msgr2=0x7f4a6c1f9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a72800640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a6c0b5a80 0x7f4a6c1f9a40 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00c410 tx=0x7f4a5c00c8d0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a72800640 1 -- 172.21.15.50:0/3714819814 shutdown_connections 2024-09-21T06:10:32.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a72800640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a6c15c060 0x7f4a6c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a72800640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a6c157030 0x7f4a6c1386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a72800640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a48074d90 0x7f4a480772a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a72800640 1 --2- 172.21.15.50:0/3714819814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a6c0b5a80 0x7f4a6c1f9a40 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a72800640 1 -- 172.21.15.50:0/3714819814 >> 172.21.15.50:0/3714819814 conn(0x7f4a6c0708c0 msgr2=0x7f4a6c150dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:32.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a72800640 1 -- 172.21.15.50:0/3714819814 shutdown_connections 2024-09-21T06:10:32.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.284+0000 7f4a72800640 1 -- 172.21.15.50:0/3714819814 wait complete. 2024-09-21T06:10:32.303 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:10:32.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.468+0000 7fa252800640 1 -- 172.21.15.50:0/1568123733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa24c06efa0 msgr2=0x7fa24c06f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.468+0000 7fa252800640 1 --2- 172.21.15.50:0/1568123733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa24c06efa0 0x7fa24c06f3b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa244009bc0 tx=0x7fa24401cb70 comp rx=0 tx=0).stop 2024-09-21T06:10:32.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.468+0000 7fa252800640 1 -- 172.21.15.50:0/1568123733 shutdown_connections 2024-09-21T06:10:32.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.468+0000 7fa252800640 1 --2- 172.21.15.50:0/1568123733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa24c06efa0 0x7fa24c06f3b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.468+0000 7fa252800640 1 --2- 172.21.15.50:0/1568123733 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa24c0744d0 0x7fa24c06ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.468+0000 7fa252800640 1 -- 172.21.15.50:0/1568123733 >> 172.21.15.50:0/1568123733 conn(0x7fa24c0b2c40 msgr2=0x7fa24c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:32.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.468+0000 7fa252800640 1 -- 172.21.15.50:0/1568123733 shutdown_connections 2024-09-21T06:10:32.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.468+0000 7fa252800640 1 -- 172.21.15.50:0/1568123733 wait complete. 2024-09-21T06:10:32.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.468+0000 7fa252800640 1 Processor -- start 2024-09-21T06:10:32.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.468+0000 7fa252800640 1 -- start start 2024-09-21T06:10:32.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa252800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa24c06efa0 0x7fa24c1fc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:32.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa252800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa24c15c5d0 0x7fa24c1403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:32.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa252800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa24c0aa4d0 0x7fa24c153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:32.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa252800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c168d20 con 0x7fa24c15c5d0 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa252800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0b3d30 con 0x7fa24c06efa0 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa252800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa24c168ea0 con 0x7fa24c0aa4d0 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa24aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa24c15c5d0 0x7fa24c1403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa24aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa24c15c5d0 0x7fa24c1403a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54434/0 (socket says 172.21.15.50:54434) 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa24be00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa24c0aa4d0 0x7fa24c153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa24aa00640 1 -- 172.21.15.50:0/2059630176 learned_addr learned my addr 172.21.15.50:0/2059630176 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa24b400640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa24c06efa0 0x7fa24c1fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa24be00640 1 -- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa24c06efa0 msgr2=0x7fa24c1fc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa24be00640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa24c06efa0 0x7fa24c1fc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa24be00640 1 -- 172.21.15.50:0/2059630176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa24c15c5d0 msgr2=0x7fa24c1403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa24be00640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa24c15c5d0 0x7fa24c1403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa24be00640 1 -- 172.21.15.50:0/2059630176 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c1fc850 con 0x7fa24c0aa4d0 2024-09-21T06:10:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa24be00640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa24c0aa4d0 0x7fa24c153720 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa2440ac000 tx=0x7fa244005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:32.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa243400640 1 -- 172.21.15.50:0/2059630176 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2440b2040 con 0x7fa24c0aa4d0 2024-09-21T06:10:32.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa243400640 1 -- 172.21.15.50:0/2059630176 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa244004130 con 0x7fa24c0aa4d0 2024-09-21T06:10:32.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa252800640 1 -- 172.21.15.50:0/2059630176 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c1fcb30 con 0x7fa24c0aa4d0 2024-09-21T06:10:32.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa243400640 1 -- 172.21.15.50:0/2059630176 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244004540 con 0x7fa24c0aa4d0 2024-09-21T06:10:32.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa252800640 1 -- 172.21.15.50:0/2059630176 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c1fc560 con 0x7fa24c0aa4d0 2024-09-21T06:10:32.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.472+0000 7fa252800640 1 -- 172.21.15.50:0/2059630176 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24c06f3b0 con 0x7fa24c0aa4d0 2024-09-21T06:10:32.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.476+0000 7fa243400640 1 -- 172.21.15.50:0/2059630176 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 45) ==== 99733+0+0 (secure 0 0 0) 0x7fa2440bd060 con 0x7fa24c0aa4d0 2024-09-21T06:10:32.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.476+0000 7fa243400640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa22c074d90 0x7fa22c0772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:32.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.476+0000 7fa243400640 1 -- 172.21.15.50:0/2059630176 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fa24412ca10 con 0x7fa24c0aa4d0 2024-09-21T06:10:32.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.476+0000 7fa24b400640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa22c074d90 0x7fa22c0772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:32.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.476+0000 7fa24b400640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa22c074d90 0x7fa22c0772a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa23c001870 tx=0x7fa23c001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:32.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.480+0000 7fa243400640 1 -- 172.21.15.50:0/2059630176 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa2440f1290 con 0x7fa24c0aa4d0 2024-09-21T06:10:32.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.680+0000 7fa252800640 1 -- 172.21.15.50:0/2059630176 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fa24c0745d0 con 0x7fa22c074d90 2024-09-21T06:10:32.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.688+0000 7fa243400640 1 -- 172.21.15.50:0/2059630176 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fa24c0745d0 con 0x7fa22c074d90 2024-09-21T06:10:32.687 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/09ac0379-02f7-4c55-a6aa-7355563232b1 2024-09-21T06:10:32.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.692+0000 7fa252800640 1 -- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa22c074d90 msgr2=0x7fa22c0772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.692+0000 7fa252800640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa22c074d90 0x7fa22c0772a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa23c001870 tx=0x7fa23c001380 comp rx=0 tx=0).stop 2024-09-21T06:10:32.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.692+0000 7fa252800640 1 -- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa24c0aa4d0 msgr2=0x7fa24c153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.692+0000 7fa252800640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa24c0aa4d0 0x7fa24c153720 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa2440ac000 tx=0x7fa244005c40 comp rx=0 tx=0).stop 2024-09-21T06:10:32.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.692+0000 7fa252800640 1 -- 172.21.15.50:0/2059630176 shutdown_connections 2024-09-21T06:10:32.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.692+0000 7fa252800640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa24c0aa4d0 0x7fa24c153720 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.692+0000 7fa252800640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa22c074d90 0x7fa22c0772a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.692+0000 7fa252800640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa24c15c5d0 0x7fa24c1403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.692+0000 7fa252800640 1 --2- 172.21.15.50:0/2059630176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa24c06efa0 0x7fa24c1fc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.692+0000 7fa252800640 1 -- 172.21.15.50:0/2059630176 >> 172.21.15.50:0/2059630176 conn(0x7fa24c0b2c40 msgr2=0x7fa24c0b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:32.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.692+0000 7fa252800640 1 -- 172.21.15.50:0/2059630176 shutdown_connections 2024-09-21T06:10:32.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:32.692+0000 7fa252800640 1 -- 172.21.15.50:0/2059630176 wait complete. 2024-09-21T06:10:32.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:10:32.711 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/09ac0379-02f7-4c55-a6aa-7355563232b1 2024-09-21T06:10:32.711 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:10:32.720 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/09ac0379-02f7-4c55-a6aa-7355563232b1 2024-09-21T06:10:32.720 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="3055190016" 2024-09-21T06:10:32.720 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:10:32.721 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-21T06:10:34.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:10:34.723 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/09ac0379-02f7-4c55-a6aa-7355563232b1 2024-09-21T06:10:34.723 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:10:34.733 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/09ac0379-02f7-4c55-a6aa-7355563232b1 2024-09-21T06:10:34.733 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="3221225472" 2024-09-21T06:10:34.733 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:10:34.734 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-21T06:10:34.734 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone87300 2024-09-21T06:10:34.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.960+0000 7f4eae000640 1 -- 172.21.15.50:0/3211007479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8157030 msgr2=0x7f4ea80b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:34.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.960+0000 7f4eae000640 1 --2- 172.21.15.50:0/3211007479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8157030 0x7f4ea80b5a80 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009bc0 tx=0x7f4e9801cba0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.960+0000 7f4eae000640 1 -- 172.21.15.50:0/3211007479 shutdown_connections 2024-09-21T06:10:34.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.960+0000 7f4eae000640 1 --2- 172.21.15.50:0/3211007479 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ea8160d50 0x7f4ea8074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.960+0000 7f4eae000640 1 --2- 172.21.15.50:0/3211007479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8157030 0x7f4ea80b5a80 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.960+0000 7f4eae000640 1 -- 172.21.15.50:0/3211007479 >> 172.21.15.50:0/3211007479 conn(0x7f4ea80708c0 msgr2=0x7f4ea8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:34.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.960+0000 7f4eae000640 1 -- 172.21.15.50:0/3211007479 shutdown_connections 2024-09-21T06:10:34.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.960+0000 7f4eae000640 1 -- 172.21.15.50:0/3211007479 wait complete. 2024-09-21T06:10:34.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.960+0000 7f4eae000640 1 Processor -- start 2024-09-21T06:10:34.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.960+0000 7f4eae000640 1 -- start start 2024-09-21T06:10:34.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4eae000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ea8157030 0x7f4ea8152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4eae000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8160d50 0x7f4ea814ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4eae000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea8161160 0x7f4ea80b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4eae000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea816a790 con 0x7f4ea8160d50 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4eae000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea816a610 con 0x7f4ea8161160 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4eae000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea816a910 con 0x7f4ea8157030 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8160d50 0x7f4ea814ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8160d50 0x7f4ea814ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54446/0 (socket says 172.21.15.50:54446) 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9ea00640 1 -- 172.21.15.50:0/1445669935 learned_addr learned my addr 172.21.15.50:0/1445669935 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4ea7400640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ea8157030 0x7f4ea8152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4ea7e00640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea8161160 0x7f4ea80b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9ea00640 1 -- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ea8157030 msgr2=0x7f4ea8152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9ea00640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ea8157030 0x7f4ea8152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9ea00640 1 -- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea8161160 msgr2=0x7f4ea80b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9ea00640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea8161160 0x7f4ea80b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9ea00640 1 -- 172.21.15.50:0/1445669935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea81fa750 con 0x7f4ea8160d50 2024-09-21T06:10:34.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4ea7e00640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea8161160 0x7f4ea80b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:34.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9ea00640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8160d50 0x7f4ea814ea40 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000d720 tx=0x7f4e9000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:34.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9fe00640 1 -- 172.21.15.50:0/1445669935 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90017070 con 0x7f4ea8160d50 2024-09-21T06:10:34.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9fe00640 1 -- 172.21.15.50:0/1445669935 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e90009b10 con 0x7f4ea8160d50 2024-09-21T06:10:34.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9fe00640 1 -- 172.21.15.50:0/1445669935 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90010460 con 0x7f4ea8160d50 2024-09-21T06:10:34.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4eae000640 1 -- 172.21.15.50:0/1445669935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea81faa30 con 0x7f4ea8160d50 2024-09-21T06:10:34.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4eae000640 1 -- 172.21.15.50:0/1445669935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea81faff0 con 0x7f4ea8160d50 2024-09-21T06:10:34.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4eae000640 1 -- 172.21.15.50:0/1445669935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea80d2260 con 0x7f4ea8160d50 2024-09-21T06:10:34.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.964+0000 7f4e9fe00640 1 -- 172.21.15.50:0/1445669935 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 45) ==== 99733+0+0 (secure 0 0 0) 0x7f4e9001a070 con 0x7f4ea8160d50 2024-09-21T06:10:34.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.968+0000 7f4e9fe00640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4e84074d90 0x7f4e840772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:34.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.968+0000 7f4e9fe00640 1 -- 172.21.15.50:0/1445669935 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f4e90093190 con 0x7f4ea8160d50 2024-09-21T06:10:34.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.968+0000 7f4ea7400640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4e84074d90 0x7f4e840772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:34.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.968+0000 7f4ea7400640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4e84074d90 0x7f4e840772a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4e980af000 tx=0x7f4e9801b3e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:34.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:34.972+0000 7f4e9fe00640 1 -- 172.21.15.50:0/1445669935 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4e9005f650 con 0x7f4ea8160d50 2024-09-21T06:10:35.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.180+0000 7f4eae000640 1 -- 172.21.15.50:0/1445669935 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone87300", "target": ["mon-mgr", ""]}) -- 0x7f4ea806b540 con 0x7f4e84074d90 2024-09-21T06:10:35.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4e9fe00640 1 -- 172.21.15.50:0/1445669935 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e98004a50 con 0x7f4e84074d90 2024-09-21T06:10:35.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4eae000640 1 -- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4e84074d90 msgr2=0x7f4e840772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:35.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4eae000640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4e84074d90 0x7f4e840772a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4e980af000 tx=0x7f4e9801b3e0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4eae000640 1 -- 172.21.15.50:0/1445669935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8160d50 msgr2=0x7f4ea814ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:35.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4eae000640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8160d50 0x7f4ea814ea40 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000d720 tx=0x7f4e9000dbe0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4eae000640 1 -- 172.21.15.50:0/1445669935 shutdown_connections 2024-09-21T06:10:35.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4eae000640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea8161160 0x7f4ea80b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4eae000640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4e84074d90 0x7f4e840772a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4eae000640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8160d50 0x7f4ea814ea40 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4eae000640 1 --2- 172.21.15.50:0/1445669935 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4ea8157030 0x7f4ea8152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4eae000640 1 -- 172.21.15.50:0/1445669935 >> 172.21.15.50:0/1445669935 conn(0x7f4ea80708c0 msgr2=0x7f4ea815f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:35.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4eae000640 1 -- 172.21.15.50:0/1445669935 shutdown_connections 2024-09-21T06:10:35.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.248+0000 7f4eae000640 1 -- 172.21.15.50:0/1445669935 wait complete. 2024-09-21T06:10:35.252 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:35.253+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:35.252 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:35.253+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:35.252 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:35.253+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:35.252 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:35.253+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:35.253 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:35.253+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:35.267 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone73151 2024-09-21T06:10:35.271 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:35.273+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:35.271 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:35.273+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:35.271 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:35.273+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:35.272 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:35.273+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:35.272 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:35.273+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:35.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.440+0000 7fef97200640 1 -- 172.21.15.50:0/2525632677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef98155a00 msgr2=0x7fef9815f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:35.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.440+0000 7fef97200640 1 --2- 172.21.15.50:0/2525632677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef98155a00 0x7fef9815f070 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fef98074530 tx=0x7fef8801c890 comp rx=0 tx=0).stop 2024-09-21T06:10:35.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.440+0000 7fef97200640 1 -- 172.21.15.50:0/2525632677 shutdown_connections 2024-09-21T06:10:35.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.440+0000 7fef97200640 1 --2- 172.21.15.50:0/2525632677 >> v2:172.21.15.88:3300/0 conn(0x7fef980b2300 0x7fef980b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.440+0000 7fef97200640 1 --2- 172.21.15.50:0/2525632677 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef9815f5b0 0x7fef980b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.440+0000 7fef97200640 1 --2- 172.21.15.50:0/2525632677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef98155a00 0x7fef9815f070 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.440+0000 7fef97200640 1 -- 172.21.15.50:0/2525632677 >> 172.21.15.50:0/2525632677 conn(0x7fef9806cdf0 msgr2=0x7fef9806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:35.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.440+0000 7fef97200640 1 -- 172.21.15.50:0/2525632677 shutdown_connections 2024-09-21T06:10:35.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.440+0000 7fef97200640 1 -- 172.21.15.50:0/2525632677 wait complete. 2024-09-21T06:10:35.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef97200640 1 Processor -- start 2024-09-21T06:10:35.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef97200640 1 -- start start 2024-09-21T06:10:35.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef97200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef980b2300 0x7fef98146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:35.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef97200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef98155a00 0x7fef98153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:35.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef97200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef9815f5b0 0x7fef981540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:35.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef97200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef98072640 con 0x7fef9815f5b0 2024-09-21T06:10:35.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef97200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fef980724c0 con 0x7fef980b2300 2024-09-21T06:10:35.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef97200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fef980727c0 con 0x7fef98155a00 2024-09-21T06:10:35.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef96800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef9815f5b0 0x7fef981540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:35.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef96800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef9815f5b0 0x7fef981540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54458/0 (socket says 172.21.15.50:54458) 2024-09-21T06:10:35.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef96800640 1 -- 172.21.15.50:0/728492475 learned_addr learned my addr 172.21.15.50:0/728492475 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:35.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef95e00640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef980b2300 0x7fef98146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:35.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef8d400640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef98155a00 0x7fef98153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:35.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef96800640 1 -- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef98155a00 msgr2=0x7fef98153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:35.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef96800640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef98155a00 0x7fef98153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef96800640 1 -- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef980b2300 msgr2=0x7fef98146470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:35.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef96800640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef980b2300 0x7fef98146470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef96800640 1 -- 172.21.15.50:0/728492475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef98152c20 con 0x7fef9815f5b0 2024-09-21T06:10:35.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef96800640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef9815f5b0 0x7fef981540d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fef9000ea00 tx=0x7fef9000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:35.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef8ea00640 1 -- 172.21.15.50:0/728492475 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9000cde0 con 0x7fef9815f5b0 2024-09-21T06:10:35.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef8ea00640 1 -- 172.21.15.50:0/728492475 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef90004c20 con 0x7fef9815f5b0 2024-09-21T06:10:35.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef97200640 1 -- 172.21.15.50:0/728492475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef981f8890 con 0x7fef9815f5b0 2024-09-21T06:10:35.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef8ea00640 1 -- 172.21.15.50:0/728492475 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90005020 con 0x7fef9815f5b0 2024-09-21T06:10:35.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef97200640 1 -- 172.21.15.50:0/728492475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef981f8e50 con 0x7fef9815f5b0 2024-09-21T06:10:35.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.444+0000 7fef97200640 1 -- 172.21.15.50:0/728492475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef98072d20 con 0x7fef9815f5b0 2024-09-21T06:10:35.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.448+0000 7fef8ea00640 1 -- 172.21.15.50:0/728492475 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 45) ==== 99733+0+0 (secure 0 0 0) 0x7fef900051c0 con 0x7fef9815f5b0 2024-09-21T06:10:35.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.448+0000 7fef8ea00640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef74074d90 0x7fef740772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:35.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.448+0000 7fef8ea00640 1 -- 172.21.15.50:0/728492475 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fef90093eb0 con 0x7fef9815f5b0 2024-09-21T06:10:35.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.448+0000 7fef95e00640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef74074d90 0x7fef740772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:35.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.448+0000 7fef95e00640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef74074d90 0x7fef740772a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fef880af000 tx=0x7fef880a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:35.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.452+0000 7fef8ea00640 1 -- 172.21.15.50:0/728492475 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fef900602e0 con 0x7fef9815f5b0 2024-09-21T06:10:35.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.668+0000 7fef97200640 1 -- 172.21.15.50:0/728492475 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone73151", "target": ["mon-mgr", ""]}) -- 0x7fef9806b540 con 0x7fef74074d90 2024-09-21T06:10:35.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:35.724+0000 7fef8ea00640 1 -- 172.21.15.50:0/728492475 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 46) ==== 99835+0+0 (secure 0 0 0) 0x7fef9005c550 con 0x7fef9815f5b0 2024-09-21T06:10:36.251 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:36.253+0000 7f6560600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:36.251 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:36.253+0000 7f6560600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:36.251 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:36.253+0000 7f6560600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:36.251 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:36.253+0000 7f6560600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:36.251 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:36.253+0000 7f6560600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:36.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.364+0000 7fef8ea00640 1 -- 172.21.15.50:0/728492475 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fef880b3040 con 0x7fef74074d90 2024-09-21T06:10:36.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.368+0000 7fef97200640 1 -- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef74074d90 msgr2=0x7fef740772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:36.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.368+0000 7fef97200640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef74074d90 0x7fef740772a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fef880af000 tx=0x7fef880a7040 comp rx=0 tx=0).stop 2024-09-21T06:10:36.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.368+0000 7fef97200640 1 -- 172.21.15.50:0/728492475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef9815f5b0 msgr2=0x7fef981540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:36.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.368+0000 7fef97200640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef9815f5b0 0x7fef981540d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fef9000ea00 tx=0x7fef9000eec0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.368+0000 7fef97200640 1 -- 172.21.15.50:0/728492475 shutdown_connections 2024-09-21T06:10:36.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.368+0000 7fef97200640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fef74074d90 0x7fef740772a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.368+0000 7fef97200640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef9815f5b0 0x7fef981540d0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.368+0000 7fef97200640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fef98155a00 0x7fef98153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.368+0000 7fef97200640 1 --2- 172.21.15.50:0/728492475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef980b2300 0x7fef98146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.368+0000 7fef97200640 1 -- 172.21.15.50:0/728492475 >> 172.21.15.50:0/728492475 conn(0x7fef9806cdf0 msgr2=0x7fef9815e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:36.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.368+0000 7fef97200640 1 -- 172.21.15.50:0/728492475 shutdown_connections 2024-09-21T06:10:36.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.368+0000 7fef97200640 1 -- 172.21.15.50:0/728492475 wait complete. 2024-09-21T06:10:36.408 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone29872 2024-09-21T06:10:36.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.696+0000 7f6d4f400640 1 -- 172.21.15.50:0/1361044269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d480b3340 msgr2=0x7f6d480b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.696+0000 7f6d4f400640 1 --2- 172.21.15.50:0/1361044269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d480b3340 0x7f6d480b3750 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f6d480726f0 tx=0x7f6d3801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.696+0000 7f6d4f400640 1 -- 172.21.15.50:0/1361044269 shutdown_connections 2024-09-21T06:10:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.696+0000 7f6d4f400640 1 --2- 172.21.15.50:0/1361044269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d480b3340 0x7f6d480b3750 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.696+0000 7f6d4f400640 1 --2- 172.21.15.50:0/1361044269 >> v2:172.21.15.88:3300/0 conn(0x7f6d480cefa0 0x7f6d480cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.696+0000 7f6d4f400640 1 -- 172.21.15.50:0/1361044269 >> 172.21.15.50:0/1361044269 conn(0x7f6d480b2040 msgr2=0x7f6d480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.696+0000 7f6d4f400640 1 -- 172.21.15.50:0/1361044269 shutdown_connections 2024-09-21T06:10:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.696+0000 7f6d4f400640 1 -- 172.21.15.50:0/1361044269 wait complete. 2024-09-21T06:10:36.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d4f400640 1 Processor -- start 2024-09-21T06:10:36.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d4f400640 1 -- start start 2024-09-21T06:10:36.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d4f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d480749a0 0x7f6d480afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:36.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d4f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d480b3340 0x7f6d48159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:36.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d4f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d480cefa0 0x7f6d481f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d4f400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48169d50 con 0x7f6d480b3340 2024-09-21T06:10:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d4f400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48169bd0 con 0x7f6d480749a0 2024-09-21T06:10:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d4f400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6d48169ed0 con 0x7f6d480cefa0 2024-09-21T06:10:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d47e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d480b3340 0x7f6d48159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d47e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d480b3340 0x7f6d48159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40828/0 (socket says 172.21.15.50:40828) 2024-09-21T06:10:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d47e00640 1 -- 172.21.15.50:0/236676796 learned_addr learned my addr 172.21.15.50:0/236676796 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d47e00640 1 -- 172.21.15.50:0/236676796 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d480cefa0 msgr2=0x7f6d481f93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d47e00640 1 --2- 172.21.15.50:0/236676796 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d480cefa0 0x7f6d481f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d47e00640 1 -- 172.21.15.50:0/236676796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d480749a0 msgr2=0x7f6d480afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:36.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.700+0000 7f6d47e00640 1 --2- 172.21.15.50:0/236676796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d480749a0 0x7f6d480afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.704+0000 7f6d47e00640 1 -- 172.21.15.50:0/236676796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d48141460 con 0x7f6d480b3340 2024-09-21T06:10:36.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.704+0000 7f6d47e00640 1 --2- 172.21.15.50:0/236676796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d480b3340 0x7f6d48159d60 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400b3f0 tx=0x7f6d3400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:36.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.704+0000 7f6d44c00640 1 -- 172.21.15.50:0/236676796 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34015020 con 0x7f6d480b3340 2024-09-21T06:10:36.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.704+0000 7f6d4f400640 1 -- 172.21.15.50:0/236676796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d48143360 con 0x7f6d480b3340 2024-09-21T06:10:36.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.704+0000 7f6d44c00640 1 -- 172.21.15.50:0/236676796 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d34011920 con 0x7f6d480b3340 2024-09-21T06:10:36.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.704+0000 7f6d44c00640 1 -- 172.21.15.50:0/236676796 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34011d70 con 0x7f6d480b3340 2024-09-21T06:10:36.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.704+0000 7f6d4f400640 1 -- 172.21.15.50:0/236676796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d481f8ea0 con 0x7f6d480b3340 2024-09-21T06:10:36.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.704+0000 7f6d4f400640 1 -- 172.21.15.50:0/236676796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d481411d0 con 0x7f6d480b3340 2024-09-21T06:10:36.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.704+0000 7f6d44c00640 1 -- 172.21.15.50:0/236676796 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 46) ==== 99835+0+0 (secure 0 0 0) 0x7f6d34018020 con 0x7f6d480b3340 2024-09-21T06:10:36.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.708+0000 7f6d44c00640 1 --2- 172.21.15.50:0/236676796 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d28074ee0 0x7f6d280773f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:36.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.708+0000 7f6d44c00640 1 -- 172.21.15.50:0/236676796 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f6d34093090 con 0x7f6d480b3340 2024-09-21T06:10:36.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.708+0000 7f6d4ca00640 1 --2- 172.21.15.50:0/236676796 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d28074ee0 0x7f6d280773f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:36.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.712+0000 7f6d4ca00640 1 --2- 172.21.15.50:0/236676796 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d28074ee0 0x7f6d280773f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6d40007fd0 tx=0x7f6d40007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:36.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:36.712+0000 7f6d44c00640 1 -- 172.21.15.50:0/236676796 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6d3401f020 con 0x7f6d480b3340 2024-09-21T06:10:37.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.048+0000 7f6d4f400640 1 -- 172.21.15.50:0/236676796 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone29872", "target": ["mon-mgr", ""]}) -- 0x7f6d4806b540 con 0x7f6d28074ee0 2024-09-21T06:10:37.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.584+0000 7f6d44c00640 1 -- 172.21.15.50:0/236676796 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6d40003980 con 0x7f6d28074ee0 2024-09-21T06:10:37.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.596+0000 7f6d4f400640 1 -- 172.21.15.50:0/236676796 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d28074ee0 msgr2=0x7f6d280773f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:37.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.596+0000 7f6d4f400640 1 --2- 172.21.15.50:0/236676796 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d28074ee0 0x7f6d280773f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6d40007fd0 tx=0x7f6d40007d40 comp rx=0 tx=0).stop 2024-09-21T06:10:37.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.596+0000 7f6d4f400640 1 -- 172.21.15.50:0/236676796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d480b3340 msgr2=0x7f6d48159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:37.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.596+0000 7f6d4f400640 1 --2- 172.21.15.50:0/236676796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d480b3340 0x7f6d48159d60 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400b3f0 tx=0x7f6d3400b8b0 comp rx=0 tx=0).stop 2024-09-21T06:10:37.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.596+0000 7f6d4f400640 1 -- 172.21.15.50:0/236676796 shutdown_connections 2024-09-21T06:10:37.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.596+0000 7f6d4f400640 1 --2- 172.21.15.50:0/236676796 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d480cefa0 0x7f6d481f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:37.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.596+0000 7f6d4f400640 1 --2- 172.21.15.50:0/236676796 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d28074ee0 0x7f6d280773f0 secure :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6d40007fd0 tx=0x7f6d40007d40 comp rx=0 tx=0).stop 2024-09-21T06:10:37.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.596+0000 7f6d4f400640 1 --2- 172.21.15.50:0/236676796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d480b3340 0x7f6d48159d60 secure :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400b3f0 tx=0x7f6d3400b8b0 comp rx=0 tx=0).stop 2024-09-21T06:10:37.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.596+0000 7f6d4f400640 1 --2- 172.21.15.50:0/236676796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d480749a0 0x7f6d480afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:37.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.596+0000 7f6d4f400640 1 -- 172.21.15.50:0/236676796 >> 172.21.15.50:0/236676796 conn(0x7f6d480b2040 msgr2=0x7f6d4813d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:37.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.596+0000 7f6d4f400640 1 -- 172.21.15.50:0/236676796 shutdown_connections 2024-09-21T06:10:37.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:37.596+0000 7f6d4f400640 1 -- 172.21.15.50:0/236676796 wait complete. 2024-09-21T06:10:37.633 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:37.637+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:37.633 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:37.637+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:37.633 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:37.637+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:37.633 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:37.637+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:37.633 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:37.637+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:38.646 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone87300 2024-09-21T06:10:38.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.924+0000 7f2206a00640 1 -- 172.21.15.50:0/604951714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2200159130 msgr2=0x7f220015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:38.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.924+0000 7f2206a00640 1 --2- 172.21.15.50:0/604951714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2200159130 0x7f220015bb90 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f22000ce170 tx=0x7f21f001c850 comp rx=0 tx=0).stop 2024-09-21T06:10:38.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.924+0000 7f2206a00640 1 -- 172.21.15.50:0/604951714 shutdown_connections 2024-09-21T06:10:38.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.924+0000 7f2206a00640 1 --2- 172.21.15.50:0/604951714 >> v2:172.21.15.88:3300/0 conn(0x7f220006fc90 0x7f22000700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:38.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.924+0000 7f2206a00640 1 --2- 172.21.15.50:0/604951714 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f220006cdf0 0x7f220006d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:38.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.924+0000 7f2206a00640 1 --2- 172.21.15.50:0/604951714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2200159130 0x7f220015bb90 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:38.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.924+0000 7f2206a00640 1 -- 172.21.15.50:0/604951714 >> 172.21.15.50:0/604951714 conn(0x7f22000b35f0 msgr2=0x7f22000b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:38.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.928+0000 7f2206a00640 1 -- 172.21.15.50:0/604951714 shutdown_connections 2024-09-21T06:10:38.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.928+0000 7f2206a00640 1 -- 172.21.15.50:0/604951714 wait complete. 2024-09-21T06:10:38.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.928+0000 7f2206a00640 1 Processor -- start 2024-09-21T06:10:38.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.928+0000 7f2206a00640 1 -- start start 2024-09-21T06:10:38.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2206a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f220006cdf0 0x7f22001f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:38.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2206a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f220006fc90 0x7f220014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:38.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2206a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2200159130 0x7f220014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:38.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2206a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f220015fad0 con 0x7f2200159130 2024-09-21T06:10:38.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2206a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f220015f950 con 0x7f220006fc90 2024-09-21T06:10:38.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2206a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f220015fc50 con 0x7f220006cdf0 2024-09-21T06:10:38.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2204a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2200159130 0x7f220014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:38.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2204a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2200159130 0x7f220014f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40842/0 (socket says 172.21.15.50:40842) 2024-09-21T06:10:38.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2204a00640 1 -- 172.21.15.50:0/410025674 learned_addr learned my addr 172.21.15.50:0/410025674 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:38.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f21ffe00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f220006cdf0 0x7f22001f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:38.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2204a00640 1 -- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f220006cdf0 msgr2=0x7f22001f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:38.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2204a00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f220006cdf0 0x7f22001f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:38.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2204a00640 1 -- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f220006fc90 msgr2=0x7f220014e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:38.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f21ff400640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f220006fc90 0x7f220014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:38.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2204a00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f220006fc90 0x7f220014e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:38.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2204a00640 1 -- 172.21.15.50:0/410025674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2200151f30 con 0x7f2200159130 2024-09-21T06:10:38.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f21ff400640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f220006fc90 0x7f220014e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:38.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2204a00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2200159130 0x7f220014f360 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f21f800b9a0 tx=0x7f21f800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:38.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f21f7e00640 1 -- 172.21.15.50:0/410025674 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8019070 con 0x7f2200159130 2024-09-21T06:10:38.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f21f7e00640 1 -- 172.21.15.50:0/410025674 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21f800ccb0 con 0x7f2200159130 2024-09-21T06:10:38.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f21f7e00640 1 -- 172.21.15.50:0/410025674 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f80146a0 con 0x7f2200159130 2024-09-21T06:10:38.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2206a00640 1 -- 172.21.15.50:0/410025674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2200151970 con 0x7f2200159130 2024-09-21T06:10:38.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.932+0000 7f2206a00640 1 -- 172.21.15.50:0/410025674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22001562d0 con 0x7f2200159130 2024-09-21T06:10:38.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.936+0000 7f2206a00640 1 -- 172.21.15.50:0/410025674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22000700a0 con 0x7f2200159130 2024-09-21T06:10:38.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.936+0000 7f21f7e00640 1 -- 172.21.15.50:0/410025674 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 47) ==== 99886+0+0 (secure 0 0 0) 0x7f21f801c070 con 0x7f2200159130 2024-09-21T06:10:38.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.940+0000 7f21f7e00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21e0074f40 0x7f21e0077450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:38.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.940+0000 7f21f7e00640 1 -- 172.21.15.50:0/410025674 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f21f8092e90 con 0x7f2200159130 2024-09-21T06:10:38.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.944+0000 7f21ffe00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21e0074f40 0x7f21e0077450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:38.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.956+0000 7f21f7e00640 1 -- 172.21.15.50:0/410025674 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f21f805fa40 con 0x7f2200159130 2024-09-21T06:10:38.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:38.964+0000 7f21ffe00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21e0074f40 0x7f21e0077450 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f21f00af000 tx=0x7f21f001b090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:39.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.296+0000 7f2206a00640 1 -- 172.21.15.50:0/410025674 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone87300", "target": ["mon-mgr", ""]}) -- 0x7f220006b540 con 0x7f21e0074f40 2024-09-21T06:10:39.302 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:39.305+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:39.302 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:39.305+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:39.302 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:39.305+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:39.302 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:39.305+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:39.302 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:39.305+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:39.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.788+0000 7f21f7e00640 1 -- 172.21.15.50:0/410025674 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 48) ==== 99937+0+0 (secure 0 0 0) 0x7f21f805b5b0 con 0x7f2200159130 2024-09-21T06:10:39.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.924+0000 7f21f7e00640 1 -- 172.21.15.50:0/410025674 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f21f0004020 con 0x7f21e0074f40 2024-09-21T06:10:39.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.928+0000 7f2206a00640 1 -- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21e0074f40 msgr2=0x7f21e0077450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:39.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.928+0000 7f2206a00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21e0074f40 0x7f21e0077450 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f21f00af000 tx=0x7f21f001b090 comp rx=0 tx=0).stop 2024-09-21T06:10:39.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.928+0000 7f2206a00640 1 -- 172.21.15.50:0/410025674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2200159130 msgr2=0x7f220014f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:39.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.928+0000 7f2206a00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2200159130 0x7f220014f360 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f21f800b9a0 tx=0x7f21f800be60 comp rx=0 tx=0).stop 2024-09-21T06:10:39.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.928+0000 7f2206a00640 1 -- 172.21.15.50:0/410025674 shutdown_connections 2024-09-21T06:10:39.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.928+0000 7f2206a00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f21e0074f40 0x7f21e0077450 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:39.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.928+0000 7f2206a00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2200159130 0x7f220014f360 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:39.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.928+0000 7f2206a00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f220006fc90 0x7f220014e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:39.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.928+0000 7f2206a00640 1 --2- 172.21.15.50:0/410025674 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f220006cdf0 0x7f22001f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:39.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.928+0000 7f2206a00640 1 -- 172.21.15.50:0/410025674 >> 172.21.15.50:0/410025674 conn(0x7f22000b35f0 msgr2=0x7f2200076060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:39.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.928+0000 7f2206a00640 1 -- 172.21.15.50:0/410025674 shutdown_connections 2024-09-21T06:10:39.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:39.928+0000 7f2206a00640 1 -- 172.21.15.50:0/410025674 wait complete. 2024-09-21T06:10:39.966 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone73151 2024-09-21T06:10:40.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.236+0000 7fd915400640 1 -- 172.21.15.50:0/1978283448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f4005390 msgr2=0x7fd8f400ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:40.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.236+0000 7fd915400640 1 --2- 172.21.15.50:0/1978283448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f4005390 0x7fd8f400ba50 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd90800c640 tx=0x7fd90801fa00 comp rx=0 tx=0).stop 2024-09-21T06:10:40.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.236+0000 7fd915400640 1 -- 172.21.15.50:0/1978283448 shutdown_connections 2024-09-21T06:10:40.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.236+0000 7fd915400640 1 --2- 172.21.15.50:0/1978283448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f4005390 0x7fd8f400ba50 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.236+0000 7fd915400640 1 -- 172.21.15.50:0/1978283448 >> 172.21.15.50:0/1978283448 conn(0x7fd9100708c0 msgr2=0x7fd910070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:40.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.236+0000 7fd915400640 1 -- 172.21.15.50:0/1978283448 shutdown_connections 2024-09-21T06:10:40.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.236+0000 7fd915400640 1 -- 172.21.15.50:0/1978283448 wait complete. 2024-09-21T06:10:40.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.236+0000 7fd915400640 1 Processor -- start 2024-09-21T06:10:40.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.236+0000 7fd915400640 1 -- start start 2024-09-21T06:10:40.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.240+0000 7fd915400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f4005390 0x7fd910151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:40.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.240+0000 7fd915400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd910150a90 0x7fd9101f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:40.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.240+0000 7fd915400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd91014edc0 0x7fd91014f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:40.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.240+0000 7fd915400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd9100b5e10 con 0x7fd910150a90 2024-09-21T06:10:40.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.240+0000 7fd915400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9100b5c90 con 0x7fd91014edc0 2024-09-21T06:10:40.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.240+0000 7fd915400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd9100b5f90 con 0x7fd8f4005390 2024-09-21T06:10:40.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.240+0000 7fd90de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd910150a90 0x7fd9101f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:40.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.240+0000 7fd90de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd910150a90 0x7fd9101f6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40858/0 (socket says 172.21.15.50:40858) 2024-09-21T06:10:40.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.240+0000 7fd90de00640 1 -- 172.21.15.50:0/1209222362 learned_addr learned my addr 172.21.15.50:0/1209222362 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:40.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.240+0000 7fd90e800640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f4005390 0x7fd910151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:40.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.244+0000 7fd90de00640 1 -- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f4005390 msgr2=0x7fd910151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:40.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.244+0000 7fd90de00640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f4005390 0x7fd910151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.244+0000 7fd90de00640 1 -- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd91014edc0 msgr2=0x7fd91014f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:40.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.244+0000 7fd90de00640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd91014edc0 0x7fd91014f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.244+0000 7fd90de00640 1 -- 172.21.15.50:0/1209222362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91014f710 con 0x7fd910150a90 2024-09-21T06:10:40.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.244+0000 7fd90de00640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd910150a90 0x7fd9101f6450 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00d740 tx=0x7fd8fc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:40.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.244+0000 7fd906a00640 1 -- 172.21.15.50:0/1209222362 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc006270 con 0x7fd910150a90 2024-09-21T06:10:40.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.244+0000 7fd915400640 1 -- 172.21.15.50:0/1209222362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd910150ea0 con 0x7fd910150a90 2024-09-21T06:10:40.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.244+0000 7fd915400640 1 -- 172.21.15.50:0/1209222362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91014cca0 con 0x7fd910150a90 2024-09-21T06:10:40.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.244+0000 7fd906a00640 1 -- 172.21.15.50:0/1209222362 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8fc010450 con 0x7fd910150a90 2024-09-21T06:10:40.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.244+0000 7fd906a00640 1 -- 172.21.15.50:0/1209222362 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc0108a0 con 0x7fd910150a90 2024-09-21T06:10:40.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.244+0000 7fd8fbe00640 1 -- 172.21.15.50:0/1209222362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e80052a0 con 0x7fd910150a90 2024-09-21T06:10:40.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.248+0000 7fd906a00640 1 -- 172.21.15.50:0/1209222362 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 48) ==== 99937+0+0 (secure 0 0 0) 0x7fd8fc010a40 con 0x7fd910150a90 2024-09-21T06:10:40.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.248+0000 7fd906a00640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8f0074fd0 0x7fd8f00774e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:40.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.248+0000 7fd906a00640 1 -- 172.21.15.50:0/1209222362 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fd8fc0937d0 con 0x7fd910150a90 2024-09-21T06:10:40.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.248+0000 7fd90e800640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8f0074fd0 0x7fd8f00774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:40.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.256+0000 7fd90e800640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8f0074fd0 0x7fd8f00774e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd9000057d0 tx=0x7fd900005ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.256+0000 7fd906a00640 1 -- 172.21.15.50:0/1209222362 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd91014f710 con 0x7fd910150a90 2024-09-21T06:10:40.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:40.596+0000 7fd8fbe00640 1 -- 172.21.15.50:0/1209222362 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone73151", "target": ["mon-mgr", ""]}) -- 0x7fd8e8000bd0 con 0x7fd8f0074fd0 2024-09-21T06:10:40.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.000+0000 7fd906a00640 1 -- 172.21.15.50:0/1209222362 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd8e8000bd0 con 0x7fd8f0074fd0 2024-09-21T06:10:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.004+0000 7fd8fbe00640 1 -- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8f0074fd0 msgr2=0x7fd8f00774e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.004+0000 7fd8fbe00640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8f0074fd0 0x7fd8f00774e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd9000057d0 tx=0x7fd900005ed0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.004+0000 7fd8fbe00640 1 -- 172.21.15.50:0/1209222362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd910150a90 msgr2=0x7fd9101f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.004+0000 7fd8fbe00640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd910150a90 0x7fd9101f6450 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00d740 tx=0x7fd8fc00dc00 comp rx=0 tx=0).stop 2024-09-21T06:10:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.004+0000 7fd8fbe00640 1 -- 172.21.15.50:0/1209222362 shutdown_connections 2024-09-21T06:10:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.004+0000 7fd8fbe00640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd91014edc0 0x7fd91014f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.004+0000 7fd8fbe00640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8f0074fd0 0x7fd8f00774e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.004+0000 7fd8fbe00640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd910150a90 0x7fd9101f6450 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.004+0000 7fd8fbe00640 1 --2- 172.21.15.50:0/1209222362 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f4005390 0x7fd910151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.004+0000 7fd8fbe00640 1 -- 172.21.15.50:0/1209222362 >> 172.21.15.50:0/1209222362 conn(0x7fd9100708c0 msgr2=0x7fd91015f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:41.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.004+0000 7fd8fbe00640 1 -- 172.21.15.50:0/1209222362 shutdown_connections 2024-09-21T06:10:41.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.004+0000 7fd8fbe00640 1 -- 172.21.15.50:0/1209222362 wait complete. 2024-09-21T06:10:41.025 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone29872 2024-09-21T06:10:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.264+0000 7fa059400640 1 -- 172.21.15.50:0/2893265223 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0540b4fd0 msgr2=0x7fa0540b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.264+0000 7fa059400640 1 --2- 172.21.15.50:0/2893265223 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0540b4fd0 0x7fa0540b53e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009900 tx=0x7fa03c01c670 comp rx=0 tx=0).stop 2024-09-21T06:10:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.264+0000 7fa059400640 1 -- 172.21.15.50:0/2893265223 shutdown_connections 2024-09-21T06:10:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.264+0000 7fa059400640 1 --2- 172.21.15.50:0/2893265223 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0540b4fd0 0x7fa0540b53e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.264+0000 7fa059400640 1 --2- 172.21.15.50:0/2893265223 >> v2:172.21.15.88:3300/0 conn(0x7fa05415cb90 0x7fa054070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.264+0000 7fa059400640 1 -- 172.21.15.50:0/2893265223 >> 172.21.15.50:0/2893265223 conn(0x7fa05406d3f0 msgr2=0x7fa0540b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.264+0000 7fa059400640 1 -- 172.21.15.50:0/2893265223 shutdown_connections 2024-09-21T06:10:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.264+0000 7fa059400640 1 -- 172.21.15.50:0/2893265223 wait complete. 2024-09-21T06:10:41.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.268+0000 7fa059400640 1 Processor -- start 2024-09-21T06:10:41.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.268+0000 7fa059400640 1 -- start start 2024-09-21T06:10:41.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.268+0000 7fa059400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0540af520 0x7fa054144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:41.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.268+0000 7fa059400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0540b4fd0 0x7fa054149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:41.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.268+0000 7fa059400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa05415cb90 0x7fa054155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:41.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.268+0000 7fa059400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa0540cfc30 con 0x7fa0540af520 2024-09-21T06:10:41.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.268+0000 7fa059400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0540cfab0 con 0x7fa0540b4fd0 2024-09-21T06:10:41.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.268+0000 7fa059400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa0540cfdb0 con 0x7fa05415cb90 2024-09-21T06:10:41.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.268+0000 7fa053e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0540af520 0x7fa054144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:41.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.268+0000 7fa053e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0540af520 0x7fa054144f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40874/0 (socket says 172.21.15.50:40874) 2024-09-21T06:10:41.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.268+0000 7fa053e00640 1 -- 172.21.15.50:0/1618738157 learned_addr learned my addr 172.21.15.50:0/1618738157 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa058a00640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa05415cb90 0x7fa054155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa053e00640 1 -- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa05415cb90 msgr2=0x7fa054155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa053e00640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa05415cb90 0x7fa054155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa053e00640 1 -- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0540b4fd0 msgr2=0x7fa054149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa053e00640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0540b4fd0 0x7fa054149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa053e00640 1 -- 172.21.15.50:0/1618738157 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa054157340 con 0x7fa0540af520 2024-09-21T06:10:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa053e00640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0540af520 0x7fa054144f90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa04400b410 tx=0x7fa04400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa050c00640 1 -- 172.21.15.50:0/1618738157 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044015020 con 0x7fa0540af520 2024-09-21T06:10:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa059400640 1 -- 172.21.15.50:0/1618738157 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa054157620 con 0x7fa0540af520 2024-09-21T06:10:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa050c00640 1 -- 172.21.15.50:0/1618738157 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa044011920 con 0x7fa0540af520 2024-09-21T06:10:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa050c00640 1 -- 172.21.15.50:0/1618738157 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044011d70 con 0x7fa0540af520 2024-09-21T06:10:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa059400640 1 -- 172.21.15.50:0/1618738157 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa054157be0 con 0x7fa0540af520 2024-09-21T06:10:41.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.272+0000 7fa059400640 1 -- 172.21.15.50:0/1618738157 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0540b53e0 con 0x7fa0540af520 2024-09-21T06:10:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.276+0000 7fa050c00640 1 -- 172.21.15.50:0/1618738157 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 48) ==== 99937+0+0 (secure 0 0 0) 0x7fa044018020 con 0x7fa0540af520 2024-09-21T06:10:41.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.276+0000 7fa050c00640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa030075000 0x7fa030077510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:41.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.276+0000 7fa050c00640 1 -- 172.21.15.50:0/1618738157 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fa044093210 con 0x7fa0540af520 2024-09-21T06:10:41.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.280+0000 7fa050c00640 1 -- 172.21.15.50:0/1618738157 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa04401f020 con 0x7fa0540af520 2024-09-21T06:10:41.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.284+0000 7fa04be00640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa030075000 0x7fa030077510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:41.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.296+0000 7fa04be00640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa030075000 0x7fa030077510 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0af000 tx=0x7fa03c019410 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:41.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.636+0000 7fa059400640 1 -- 172.21.15.50:0/1618738157 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone29872", "target": ["mon-mgr", ""]}) -- 0x7fa05406b540 con 0x7fa030075000 2024-09-21T06:10:41.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.768+0000 7fa050c00640 1 -- 172.21.15.50:0/1618738157 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa03c0a84b0 con 0x7fa030075000 2024-09-21T06:10:41.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.772+0000 7fa059400640 1 -- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa030075000 msgr2=0x7fa030077510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.772+0000 7fa059400640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa030075000 0x7fa030077510 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0af000 tx=0x7fa03c019410 comp rx=0 tx=0).stop 2024-09-21T06:10:41.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.772+0000 7fa059400640 1 -- 172.21.15.50:0/1618738157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0540af520 msgr2=0x7fa054144f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.772+0000 7fa059400640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0540af520 0x7fa054144f90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa04400b410 tx=0x7fa04400b8d0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.772+0000 7fa059400640 1 -- 172.21.15.50:0/1618738157 shutdown_connections 2024-09-21T06:10:41.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.772+0000 7fa059400640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa05415cb90 0x7fa054155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.772+0000 7fa059400640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0540b4fd0 0x7fa054149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.772+0000 7fa059400640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa030075000 0x7fa030077510 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.772+0000 7fa059400640 1 --2- 172.21.15.50:0/1618738157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0540af520 0x7fa054144f90 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.772+0000 7fa059400640 1 -- 172.21.15.50:0/1618738157 >> 172.21.15.50:0/1618738157 conn(0x7fa05406d3f0 msgr2=0x7fa05406e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:41.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.772+0000 7fa059400640 1 -- 172.21.15.50:0/1618738157 shutdown_connections 2024-09-21T06:10:41.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.772+0000 7fa059400640 1 -- 172.21.15.50:0/1618738157 wait complete. 2024-09-21T06:10:41.790 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87300 2024-09-21T06:10:41.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.956+0000 7f82a9000640 1 -- 172.21.15.50:0/729625264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a41527b0 msgr2=0x7f82a4074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.956+0000 7f82a9000640 1 --2- 172.21.15.50:0/729625264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a41527b0 0x7f82a4074e30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f8290009840 tx=0x7f829001c760 comp rx=0 tx=0).stop 2024-09-21T06:10:41.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.956+0000 7f82a9000640 1 -- 172.21.15.50:0/729625264 shutdown_connections 2024-09-21T06:10:41.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.956+0000 7f82a9000640 1 --2- 172.21.15.50:0/729625264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a41527b0 0x7f82a4074e30 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.956+0000 7f82a9000640 1 --2- 172.21.15.50:0/729625264 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f82a40b5c70 0x7f82a415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.956+0000 7f82a9000640 1 -- 172.21.15.50:0/729625264 >> 172.21.15.50:0/729625264 conn(0x7f82a40b2040 msgr2=0x7f82a40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:41.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.956+0000 7f82a9000640 1 -- 172.21.15.50:0/729625264 shutdown_connections 2024-09-21T06:10:41.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.956+0000 7f82a9000640 1 -- 172.21.15.50:0/729625264 wait complete. 2024-09-21T06:10:41.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a9000640 1 Processor -- start 2024-09-21T06:10:41.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a9000640 1 -- start start 2024-09-21T06:10:41.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a9000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a40b4de0 0x7f82a4155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:41.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a9000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82a40b5c70 0x7f82a4150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:41.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a9000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f82a41527b0 0x7f82a41581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:41.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a9000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4168bb0 con 0x7f82a40b4de0 2024-09-21T06:10:41.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a9000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4168a30 con 0x7f82a40b5c70 2024-09-21T06:10:41.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a9000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f82a4168d30 con 0x7f82a41527b0 2024-09-21T06:10:41.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a40b4de0 0x7f82a4155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:41.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a40b4de0 0x7f82a4155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40902/0 (socket says 172.21.15.50:40902) 2024-09-21T06:10:41.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a1e00640 1 -- 172.21.15.50:0/1998772257 learned_addr learned my addr 172.21.15.50:0/1998772257 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:41.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a1400640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82a40b5c70 0x7f82a4150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:41.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a2800640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f82a41527b0 0x7f82a41581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:41.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a1e00640 1 -- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f82a41527b0 msgr2=0x7f82a41581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a1e00640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f82a41527b0 0x7f82a41581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a1e00640 1 -- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82a40b5c70 msgr2=0x7f82a4150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a1e00640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82a40b5c70 0x7f82a4150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a1e00640 1 -- 172.21.15.50:0/1998772257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a4155b00 con 0x7f82a40b4de0 2024-09-21T06:10:41.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a1400640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82a40b5c70 0x7f82a4150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:41.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a2800640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f82a41527b0 0x7f82a41581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:41.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f82a1e00640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a40b4de0 0x7f82a4155040 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f829400e7d0 tx=0x7f829400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:41.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.960+0000 7f829a000640 1 -- 172.21.15.50:0/1998772257 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829400cc60 con 0x7f82a40b4de0 2024-09-21T06:10:41.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.964+0000 7f829a000640 1 -- 172.21.15.50:0/1998772257 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8294010430 con 0x7f82a40b4de0 2024-09-21T06:10:41.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.964+0000 7f829a000640 1 -- 172.21.15.50:0/1998772257 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294010860 con 0x7f82a40b4de0 2024-09-21T06:10:41.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.964+0000 7f82a9000640 1 -- 172.21.15.50:0/1998772257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a415d9e0 con 0x7f82a40b4de0 2024-09-21T06:10:41.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.964+0000 7f82a9000640 1 -- 172.21.15.50:0/1998772257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a415dfa0 con 0x7f82a40b4de0 2024-09-21T06:10:41.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.964+0000 7f82a9000640 1 -- 172.21.15.50:0/1998772257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a415bb60 con 0x7f82a40b4de0 2024-09-21T06:10:41.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.964+0000 7f829a000640 1 -- 172.21.15.50:0/1998772257 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 49) ==== 99988+0+0 (secure 0 0 0) 0x7f8294010a00 con 0x7f82a40b4de0 2024-09-21T06:10:41.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.964+0000 7f829a000640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8284075090 0x7f82840775a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:41.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.964+0000 7f829a000640 1 -- 172.21.15.50:0/1998772257 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f8294093c40 con 0x7f82a40b4de0 2024-09-21T06:10:41.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.964+0000 7f82a1400640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8284075090 0x7f82840775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:41.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.968+0000 7f82a1400640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8284075090 0x7f82840775a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f82900af000 tx=0x7f829001afa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:41.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:41.968+0000 7f829a000640 1 -- 172.21.15.50:0/1998772257 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f82940606c0 con 0x7f82a40b4de0 2024-09-21T06:10:42.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:42.184+0000 7f82a9000640 1 -- 172.21.15.50:0/1998772257 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87300", "target": ["mon-mgr", ""]}) -- 0x7f82a406b540 con 0x7f8284075090 2024-09-21T06:10:43.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f829a000640 1 -- 172.21.15.50:0/1998772257 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f8290004040 con 0x7f8284075090 2024-09-21T06:10:43.172 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "canceled", 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout: "failure": { 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout: "errno": "4", 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout: "error_msg": "user interrupted clone operation" 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:43.173 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:10:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f82a9000640 1 -- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8284075090 msgr2=0x7f82840775a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f82a9000640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8284075090 0x7f82840775a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f82900af000 tx=0x7f829001afa0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f82a9000640 1 -- 172.21.15.50:0/1998772257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a40b4de0 msgr2=0x7f82a4155040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:43.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f82a9000640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a40b4de0 0x7f82a4155040 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f829400e7d0 tx=0x7f829400ec90 comp rx=0 tx=0).stop 2024-09-21T06:10:43.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f82a9000640 1 -- 172.21.15.50:0/1998772257 shutdown_connections 2024-09-21T06:10:43.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f82a9000640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f82a41527b0 0x7f82a41581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f82a9000640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82a40b5c70 0x7f82a4150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f82a9000640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8284075090 0x7f82840775a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f82a9000640 1 --2- 172.21.15.50:0/1998772257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82a40b4de0 0x7f82a4155040 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f82a9000640 1 -- 172.21.15.50:0/1998772257 >> 172.21.15.50:0/1998772257 conn(0x7f82a40b2040 msgr2=0x7f82a40b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:43.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f82a9000640 1 -- 172.21.15.50:0/1998772257 shutdown_connections 2024-09-21T06:10:43.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.176+0000 7f82a9000640 1 -- 172.21.15.50:0/1998772257 wait complete. 2024-09-21T06:10:43.199 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-21T06:10:43.200 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone73151 2024-09-21T06:10:43.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.368+0000 7f7010800640 1 -- 172.21.15.50:0/2761886337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70081527b0 msgr2=0x7f700815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:43.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.368+0000 7f7010800640 1 --2- 172.21.15.50:0/2761886337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70081527b0 0x7f700815bb60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7008074b20 tx=0x7f6ff801c760 comp rx=0 tx=0).stop 2024-09-21T06:10:43.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.368+0000 7f7010800640 1 -- 172.21.15.50:0/2761886337 shutdown_connections 2024-09-21T06:10:43.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.368+0000 7f7010800640 1 --2- 172.21.15.50:0/2761886337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70081527b0 0x7f700815bb60 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.368+0000 7f7010800640 1 -- 172.21.15.50:0/2761886337 >> 172.21.15.50:0/2761886337 conn(0x7f70080b2040 msgr2=0x7f70080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:43.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.368+0000 7f7010800640 1 -- 172.21.15.50:0/2761886337 shutdown_connections 2024-09-21T06:10:43.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.368+0000 7f7010800640 1 -- 172.21.15.50:0/2761886337 wait complete. 2024-09-21T06:10:43.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.372+0000 7f7010800640 1 Processor -- start 2024-09-21T06:10:43.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.372+0000 7f7010800640 1 -- start start 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.372+0000 7f7010800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f70080b4de0 0x7f700813b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.372+0000 7f7010800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70080b5c70 0x7f70081f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.372+0000 7f7010800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081527b0 0x7f70081f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.372+0000 7f7010800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7008169b30 con 0x7f70081527b0 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.372+0000 7f7010800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f70081699b0 con 0x7f70080b5c70 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.372+0000 7f7010800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7008169cb0 con 0x7f70080b4de0 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.372+0000 7f700e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081527b0 0x7f70081f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.372+0000 7f700e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081527b0 0x7f70081f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40916/0 (socket says 172.21.15.50:40916) 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.372+0000 7f700e800640 1 -- 172.21.15.50:0/2102154345 learned_addr learned my addr 172.21.15.50:0/2102154345 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f700d400640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70080b5c70 0x7f70081f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f700de00640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f70080b4de0 0x7f700813b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f700e800640 1 -- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f70080b4de0 msgr2=0x7f700813b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:43.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f700e800640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f70080b4de0 0x7f700813b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f700e800640 1 -- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70080b5c70 msgr2=0x7f70081f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:43.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f700e800640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70080b5c70 0x7f70081f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f700e800640 1 -- 172.21.15.50:0/2102154345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70080aed30 con 0x7f70081527b0 2024-09-21T06:10:43.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f700e800640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081527b0 0x7f70081f6cf0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f700400caa0 tx=0x7f700400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f7002000640 1 -- 172.21.15.50:0/2102154345 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004016020 con 0x7f70081527b0 2024-09-21T06:10:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f7002000640 1 -- 172.21.15.50:0/2102154345 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f700400a930 con 0x7f70081527b0 2024-09-21T06:10:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f7010800640 1 -- 172.21.15.50:0/2102154345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70080af010 con 0x7f70081527b0 2024-09-21T06:10:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f7002000640 1 -- 172.21.15.50:0/2102154345 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700400adc0 con 0x7f70081527b0 2024-09-21T06:10:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f7010800640 1 -- 172.21.15.50:0/2102154345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70080af5d0 con 0x7f70081527b0 2024-09-21T06:10:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f7010800640 1 -- 172.21.15.50:0/2102154345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70081f72c0 con 0x7f70081527b0 2024-09-21T06:10:43.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.376+0000 7f7002000640 1 -- 172.21.15.50:0/2102154345 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 49) ==== 99988+0+0 (secure 0 0 0) 0x7f7004019020 con 0x7f70081527b0 2024-09-21T06:10:43.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.380+0000 7f7002000640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6fec075060 0x7f6fec077570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:43.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.380+0000 7f7002000640 1 -- 172.21.15.50:0/2102154345 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f7004093660 con 0x7f70081527b0 2024-09-21T06:10:43.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.380+0000 7f700de00640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6fec075060 0x7f6fec077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:43.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.380+0000 7f700de00640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6fec075060 0x7f6fec077570 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc000c10 tx=0x7f6ffc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:43.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.384+0000 7f7002000640 1 -- 172.21.15.50:0/2102154345 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7004020020 con 0x7f70081527b0 2024-09-21T06:10:43.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:43.600+0000 7f7010800640 1 -- 172.21.15.50:0/2102154345 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone73151", "target": ["mon-mgr", ""]}) -- 0x7f700806b540 con 0x7f6fec075060 2024-09-21T06:10:46.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.744+0000 7f7002000640 1 -- 172.21.15.50:0/2102154345 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f6ffc0050d0 con 0x7f6fec075060 2024-09-21T06:10:46.741 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "canceled", 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout: "failure": { 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout: "errno": "4", 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout: "error_msg": "user interrupted clone operation" 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:46.742 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:10:46.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.748+0000 7f7010800640 1 -- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6fec075060 msgr2=0x7f6fec077570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:46.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.748+0000 7f7010800640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6fec075060 0x7f6fec077570 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc000c10 tx=0x7f6ffc009040 comp rx=0 tx=0).stop 2024-09-21T06:10:46.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.748+0000 7f7010800640 1 -- 172.21.15.50:0/2102154345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081527b0 msgr2=0x7f70081f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:46.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.748+0000 7f7010800640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081527b0 0x7f70081f6cf0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f700400caa0 tx=0x7f700400cb90 comp rx=0 tx=0).stop 2024-09-21T06:10:46.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.748+0000 7f7010800640 1 -- 172.21.15.50:0/2102154345 shutdown_connections 2024-09-21T06:10:46.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.748+0000 7f7010800640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6fec075060 0x7f6fec077570 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.748+0000 7f7010800640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081527b0 0x7f70081f6cf0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.748+0000 7f7010800640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70080b5c70 0x7f70081f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.748+0000 7f7010800640 1 --2- 172.21.15.50:0/2102154345 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f70080b4de0 0x7f700813b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.748+0000 7f7010800640 1 -- 172.21.15.50:0/2102154345 >> 172.21.15.50:0/2102154345 conn(0x7f70080b2040 msgr2=0x7f70080b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:46.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.748+0000 7f7010800640 1 -- 172.21.15.50:0/2102154345 shutdown_connections 2024-09-21T06:10:46.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.748+0000 7f7010800640 1 -- 172.21.15.50:0/2102154345 wait complete. 2024-09-21T06:10:46.769 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-21T06:10:46.769 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone29872 2024-09-21T06:10:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.940+0000 7f43aa800640 1 -- 172.21.15.50:0/2648460509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a41536f0 msgr2=0x7f43a415d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.940+0000 7f43aa800640 1 --2- 172.21.15.50:0/2648460509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a41536f0 0x7f43a415d480 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4398009bc0 tx=0x7f439801cb10 comp rx=0 tx=0).stop 2024-09-21T06:10:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.940+0000 7f43aa800640 1 -- 172.21.15.50:0/2648460509 shutdown_connections 2024-09-21T06:10:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.940+0000 7f43aa800640 1 --2- 172.21.15.50:0/2648460509 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43a4160060 0x7f43a4074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.940+0000 7f43aa800640 1 --2- 172.21.15.50:0/2648460509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a41536f0 0x7f43a415d480 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.940+0000 7f43aa800640 1 -- 172.21.15.50:0/2648460509 >> 172.21.15.50:0/2648460509 conn(0x7f43a40b4560 msgr2=0x7f43a40d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:46.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.940+0000 7f43aa800640 1 -- 172.21.15.50:0/2648460509 shutdown_connections 2024-09-21T06:10:46.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.940+0000 7f43aa800640 1 -- 172.21.15.50:0/2648460509 wait complete. 2024-09-21T06:10:46.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43aa800640 1 Processor -- start 2024-09-21T06:10:46.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43aa800640 1 -- start start 2024-09-21T06:10:46.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43aa800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43a41536f0 0x7f43a4153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:46.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43aa800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a4160060 0x7f43a414c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:46.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43aa800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a4160470 0x7f43a41417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:46.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43aa800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43a416a490 con 0x7f43a4160060 2024-09-21T06:10:46.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43aa800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43a416a310 con 0x7f43a4160470 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43aa800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f43a416a610 con 0x7f43a41536f0 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a8a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a4160060 0x7f43a414c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a8a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a4160060 0x7f43a414c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53122/0 (socket says 172.21.15.50:53122) 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a8a00640 1 -- 172.21.15.50:0/4111825440 learned_addr learned my addr 172.21.15.50:0/4111825440 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a9400640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43a41536f0 0x7f43a4153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a9e00640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a4160470 0x7f43a41417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a8a00640 1 -- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43a41536f0 msgr2=0x7f43a4153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a8a00640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43a41536f0 0x7f43a4153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a8a00640 1 -- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a4160470 msgr2=0x7f43a41417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a8a00640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a4160470 0x7f43a41417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a8a00640 1 -- 172.21.15.50:0/4111825440 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a41fb5e0 con 0x7f43a4160060 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a9400640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43a41536f0 0x7f43a4153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a8a00640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a4160060 0x7f43a414c180 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f439000e7c0 tx=0x7f439000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a1600640 1 -- 172.21.15.50:0/4111825440 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439000cc60 con 0x7f43a4160060 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a1600640 1 -- 172.21.15.50:0/4111825440 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f439000ce20 con 0x7f43a4160060 2024-09-21T06:10:46.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43a1600640 1 -- 172.21.15.50:0/4111825440 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390005280 con 0x7f43a4160060 2024-09-21T06:10:46.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43aa800640 1 -- 172.21.15.50:0/4111825440 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a41fad40 con 0x7f43a4160060 2024-09-21T06:10:46.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.944+0000 7f43aa800640 1 -- 172.21.15.50:0/4111825440 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a41fb300 con 0x7f43a4160060 2024-09-21T06:10:46.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.948+0000 7f43aa800640 1 -- 172.21.15.50:0/4111825440 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a415d480 con 0x7f43a4160060 2024-09-21T06:10:46.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.948+0000 7f43a1600640 1 -- 172.21.15.50:0/4111825440 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 49) ==== 99988+0+0 (secure 0 0 0) 0x7f4390005420 con 0x7f43a4160060 2024-09-21T06:10:46.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.948+0000 7f43a1600640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4384075090 0x7f43840775a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:46.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.948+0000 7f43a1600640 1 -- 172.21.15.50:0/4111825440 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f4390093c10 con 0x7f43a4160060 2024-09-21T06:10:46.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.948+0000 7f43a9400640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4384075090 0x7f43840775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:46.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.952+0000 7f43a9400640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4384075090 0x7f43840775a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f43980af000 tx=0x7f439801b350 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:46.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:46.952+0000 7f43a1600640 1 -- 172.21.15.50:0/4111825440 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4390060690 con 0x7f43a4160060 2024-09-21T06:10:47.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:47.164+0000 7f43aa800640 1 -- 172.21.15.50:0/4111825440 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone29872", "target": ["mon-mgr", ""]}) -- 0x7f43a406b540 con 0x7f4384075090 2024-09-21T06:10:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.168+0000 7f43a1600640 1 -- 172.21.15.50:0/4111825440 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f4398003980 con 0x7f4384075090 2024-09-21T06:10:48.167 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "canceled", 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout: "failure": { 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout: "errno": "4", 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout: "error_msg": "user interrupted clone operation" 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:48.168 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:10:48.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.172+0000 7f43aa800640 1 -- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4384075090 msgr2=0x7f43840775a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.172+0000 7f43aa800640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4384075090 0x7f43840775a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f43980af000 tx=0x7f439801b350 comp rx=0 tx=0).stop 2024-09-21T06:10:48.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.172+0000 7f43aa800640 1 -- 172.21.15.50:0/4111825440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a4160060 msgr2=0x7f43a414c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.172+0000 7f43aa800640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a4160060 0x7f43a414c180 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f439000e7c0 tx=0x7f439000ec80 comp rx=0 tx=0).stop 2024-09-21T06:10:48.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.172+0000 7f43aa800640 1 -- 172.21.15.50:0/4111825440 shutdown_connections 2024-09-21T06:10:48.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.172+0000 7f43aa800640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a4160470 0x7f43a41417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.172+0000 7f43aa800640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4384075090 0x7f43840775a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.172+0000 7f43aa800640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a4160060 0x7f43a414c180 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.172+0000 7f43aa800640 1 --2- 172.21.15.50:0/4111825440 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43a41536f0 0x7f43a4153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.172+0000 7f43aa800640 1 -- 172.21.15.50:0/4111825440 >> 172.21.15.50:0/4111825440 conn(0x7f43a40b4560 msgr2=0x7f43a40715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:48.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.172+0000 7f43aa800640 1 -- 172.21.15.50:0/4111825440 shutdown_connections 2024-09-21T06:10:48.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.172+0000 7f43aa800640 1 -- 172.21.15.50:0/4111825440 wait complete. 2024-09-21T06:10:48.193 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-21T06:10:48.193 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:10:48.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.356+0000 7f3dbcc00640 1 -- 172.21.15.50:0/3108675290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db8155ad0 msgr2=0x7f3db806e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.356+0000 7f3dbcc00640 1 --2- 172.21.15.50:0/3108675290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db8155ad0 0x7f3db806e520 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009840 tx=0x7f3da801c790 comp rx=0 tx=0).stop 2024-09-21T06:10:48.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.356+0000 7f3dbcc00640 1 -- 172.21.15.50:0/3108675290 shutdown_connections 2024-09-21T06:10:48.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.356+0000 7f3dbcc00640 1 --2- 172.21.15.50:0/3108675290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db8155ad0 0x7f3db806e520 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.356+0000 7f3dbcc00640 1 -- 172.21.15.50:0/3108675290 >> 172.21.15.50:0/3108675290 conn(0x7f3db80708c0 msgr2=0x7f3db8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:48.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.356+0000 7f3dbcc00640 1 -- 172.21.15.50:0/3108675290 shutdown_connections 2024-09-21T06:10:48.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.356+0000 7f3dbcc00640 1 -- 172.21.15.50:0/3108675290 wait complete. 2024-09-21T06:10:48.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3dbcc00640 1 Processor -- start 2024-09-21T06:10:48.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3dbcc00640 1 -- start start 2024-09-21T06:10:48.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3dbcc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db80d29c0 0x7f3db8151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:48.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3dbcc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3db8155ad0 0x7f3db814ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:48.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3dbcc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db815f180 0x7f3db814b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:48.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3dbcc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3db816db30 con 0x7f3db815f180 2024-09-21T06:10:48.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3dbcc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3db816d9b0 con 0x7f3db80d29c0 2024-09-21T06:10:48.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3dbcc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3db816dcb0 con 0x7f3db8155ad0 2024-09-21T06:10:48.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3db6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db815f180 0x7f3db814b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:48.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3db6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db815f180 0x7f3db814b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53134/0 (socket says 172.21.15.50:53134) 2024-09-21T06:10:48.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3db6800640 1 -- 172.21.15.50:0/70704805 learned_addr learned my addr 172.21.15.50:0/70704805 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:48.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3db5e00640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db80d29c0 0x7f3db8151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:48.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3dafe00640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3db8155ad0 0x7f3db814ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:48.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3db6800640 1 -- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3db8155ad0 msgr2=0x7f3db814ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3db6800640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3db8155ad0 0x7f3db814ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3db6800640 1 -- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db80d29c0 msgr2=0x7f3db8151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3db6800640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db80d29c0 0x7f3db8151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3db6800640 1 -- 172.21.15.50:0/70704805 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db8156210 con 0x7f3db815f180 2024-09-21T06:10:48.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.360+0000 7f3dafe00640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3db8155ad0 0x7f3db814ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:48.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.364+0000 7f3db6800640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db815f180 0x7f3db814b190 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b9e0 tx=0x7f3db000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:48.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.364+0000 7f3dae000640 1 -- 172.21.15.50:0/70704805 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0003b80 con 0x7f3db815f180 2024-09-21T06:10:48.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.364+0000 7f3dae000640 1 -- 172.21.15.50:0/70704805 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3db0003e10 con 0x7f3db815f180 2024-09-21T06:10:48.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.364+0000 7f3dae000640 1 -- 172.21.15.50:0/70704805 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db000cb80 con 0x7f3db815f180 2024-09-21T06:10:48.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.364+0000 7f3dbcc00640 1 -- 172.21.15.50:0/70704805 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db8146420 con 0x7f3db815f180 2024-09-21T06:10:48.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.364+0000 7f3dbcc00640 1 -- 172.21.15.50:0/70704805 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db81469e0 con 0x7f3db815f180 2024-09-21T06:10:48.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.364+0000 7f3dbcc00640 1 -- 172.21.15.50:0/70704805 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db81563e0 con 0x7f3db815f180 2024-09-21T06:10:48.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.364+0000 7f3dae000640 1 -- 172.21.15.50:0/70704805 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 49) ==== 99988+0+0 (secure 0 0 0) 0x7f3db0006d10 con 0x7f3db815f180 2024-09-21T06:10:48.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.368+0000 7f3dae000640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d94075090 0x7f3d940775a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:48.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.368+0000 7f3dae000640 1 -- 172.21.15.50:0/70704805 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f3db0092d40 con 0x7f3db815f180 2024-09-21T06:10:48.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.368+0000 7f3db5e00640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d94075090 0x7f3d940775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:48.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.368+0000 7f3db5e00640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d94075090 0x7f3d940775a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3da80ae000 tx=0x7f3da801afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:48.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.372+0000 7f3dae000640 1 -- 172.21.15.50:0/70704805 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3db005f850 con 0x7f3db815f180 2024-09-21T06:10:48.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.616+0000 7f3dbcc00640 1 -- 172.21.15.50:0/70704805 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3db80b56c0 con 0x7f3db815f180 2024-09-21T06:10:48.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.616+0000 7f3dae000640 1 -- 172.21.15.50:0/70704805 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2374 (secure 0 0 0) 0x7f3db005f150 con 0x7f3db815f180 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:10:48.615 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 885, 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:10:48.616 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 114, 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:48.617 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 1720, 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 7072889440, 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 15175782400, 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 757918330880, 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 327041204, 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 320798151, 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 80, 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 91 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 75, 2024-09-21T06:10:48.618 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T06:10:00:602285+0000", 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 8, 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 8, 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "a", 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 8304 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 20, 2024-09-21T06:10:48.620 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:10:45.200168+0000", 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: "services": { 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: "daemons": { 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: "summary": "", 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: "a": { 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: "start_epoch": 0, 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: "start_stamp": "0.000000", 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 0, 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": {}, 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: "task_status": {} 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:48.621 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:48.622 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:48.622 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": {} 2024-09-21T06:10:48.622 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:10:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.620+0000 7f3dbcc00640 1 -- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d94075090 msgr2=0x7f3d940775a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.620+0000 7f3dbcc00640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d94075090 0x7f3d940775a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3da80ae000 tx=0x7f3da801afd0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.620+0000 7f3dbcc00640 1 -- 172.21.15.50:0/70704805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db815f180 msgr2=0x7f3db814b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.620+0000 7f3dbcc00640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db815f180 0x7f3db814b190 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b9e0 tx=0x7f3db000bea0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.620+0000 7f3dbcc00640 1 -- 172.21.15.50:0/70704805 shutdown_connections 2024-09-21T06:10:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.620+0000 7f3dbcc00640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3d94075090 0x7f3d940775a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.620+0000 7f3dbcc00640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db815f180 0x7f3db814b190 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.620+0000 7f3dbcc00640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3db8155ad0 0x7f3db814ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.620+0000 7f3dbcc00640 1 --2- 172.21.15.50:0/70704805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db80d29c0 0x7f3db8151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.620+0000 7f3dbcc00640 1 -- 172.21.15.50:0/70704805 >> 172.21.15.50:0/70704805 conn(0x7f3db80708c0 msgr2=0x7f3db814dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.620+0000 7f3dbcc00640 1 -- 172.21.15.50:0/70704805 shutdown_connections 2024-09-21T06:10:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.620+0000 7f3dbcc00640 1 -- 172.21.15.50:0/70704805 wait complete. 2024-09-21T06:10:48.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:10:48.637 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone87300/09ac0379-02f7-4c55-a6aa-7355563232b1 2024-09-21T06:10:48.637 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:10:48.647 INFO:teuthology.orchestra.run.smithi050.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone87300/09ac0379-02f7-4c55-a6aa-7355563232b1': No such file or directory 2024-09-21T06:10:48.648 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T06:10:48.648 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:10:48.673 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:10:48.696 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:10:48.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.852+0000 7fb465800640 1 -- 172.21.15.50:0/1474377015 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4600b4de0 msgr2=0x7fb4600b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.852+0000 7fb465800640 1 --2- 172.21.15.50:0/1474377015 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4600b4de0 0x7fb4600b51f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb458009bc0 tx=0x7fb45801cb70 comp rx=0 tx=0).stop 2024-09-21T06:10:48.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.852+0000 7fb465800640 1 -- 172.21.15.50:0/1474377015 shutdown_connections 2024-09-21T06:10:48.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.852+0000 7fb465800640 1 --2- 172.21.15.50:0/1474377015 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4600b4de0 0x7fb4600b51f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.852+0000 7fb465800640 1 --2- 172.21.15.50:0/1474377015 >> v2:172.21.15.88:3300/0 conn(0x7fb4601527b0 0x7fb46015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.852+0000 7fb465800640 1 -- 172.21.15.50:0/1474377015 >> 172.21.15.50:0/1474377015 conn(0x7fb4600b2040 msgr2=0x7fb4600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:48.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.852+0000 7fb465800640 1 -- 172.21.15.50:0/1474377015 shutdown_connections 2024-09-21T06:10:48.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.852+0000 7fb465800640 1 -- 172.21.15.50:0/1474377015 wait complete. 2024-09-21T06:10:48.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb465800640 1 Processor -- start 2024-09-21T06:10:48.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb465800640 1 -- start start 2024-09-21T06:10:48.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb465800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4600b4de0 0x7fb4601f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:48.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb465800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4600b5c70 0x7fb460154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:48.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb465800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4601527b0 0x7fb4601fc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:48.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb465800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb460168970 con 0x7fb4600b4de0 2024-09-21T06:10:48.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb465800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601687f0 con 0x7fb4600b5c70 2024-09-21T06:10:48.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb465800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb460168af0 con 0x7fb4601527b0 2024-09-21T06:10:48.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb45ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4600b4de0 0x7fb4601f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:48.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb45ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4600b4de0 0x7fb4601f9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53136/0 (socket says 172.21.15.50:53136) 2024-09-21T06:10:48.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb45ea00640 1 -- 172.21.15.50:0/2409361774 learned_addr learned my addr 172.21.15.50:0/2409361774 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:48.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb45f400640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4601527b0 0x7fb4601fc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:48.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb457e00640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4600b5c70 0x7fb460154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:48.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb45ea00640 1 -- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4601527b0 msgr2=0x7fb4601fc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb45ea00640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4601527b0 0x7fb4601fc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb45ea00640 1 -- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4600b5c70 msgr2=0x7fb460154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb45ea00640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4600b5c70 0x7fb460154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.856+0000 7fb45ea00640 1 -- 172.21.15.50:0/2409361774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb460153a40 con 0x7fb4600b4de0 2024-09-21T06:10:48.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.860+0000 7fb45ea00640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4600b4de0 0x7fb4601f9670 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fb45000c730 tx=0x7fb45000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:48.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.860+0000 7fb456a00640 1 -- 172.21.15.50:0/2409361774 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450016020 con 0x7fb4600b4de0 2024-09-21T06:10:48.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.860+0000 7fb456a00640 1 -- 172.21.15.50:0/2409361774 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb45000a8f0 con 0x7fb4600b4de0 2024-09-21T06:10:48.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.860+0000 7fb465800640 1 -- 172.21.15.50:0/2409361774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb460153d20 con 0x7fb4600b4de0 2024-09-21T06:10:48.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.860+0000 7fb456a00640 1 -- 172.21.15.50:0/2409361774 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45000ad40 con 0x7fb4600b4de0 2024-09-21T06:10:48.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.860+0000 7fb465800640 1 -- 172.21.15.50:0/2409361774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4601f9150 con 0x7fb4600b4de0 2024-09-21T06:10:48.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.860+0000 7fb465800640 1 -- 172.21.15.50:0/2409361774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4600b51f0 con 0x7fb4600b4de0 2024-09-21T06:10:48.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.860+0000 7fb456a00640 1 -- 172.21.15.50:0/2409361774 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 49) ==== 99988+0+0 (secure 0 0 0) 0x7fb450019020 con 0x7fb4600b4de0 2024-09-21T06:10:48.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.860+0000 7fb456a00640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb43c075060 0x7fb43c077570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:48.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.860+0000 7fb456a00640 1 -- 172.21.15.50:0/2409361774 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fb450093090 con 0x7fb4600b4de0 2024-09-21T06:10:48.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.864+0000 7fb457e00640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb43c075060 0x7fb43c077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:48.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.864+0000 7fb457e00640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb43c075060 0x7fb43c077570 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb4600b2500 tx=0x7fb4480091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:48.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:48.868+0000 7fb456a00640 1 -- 172.21.15.50:0/2409361774 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb450020020 con 0x7fb4600b4de0 2024-09-21T06:10:49.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.064+0000 7fb465800640 1 -- 172.21.15.50:0/2409361774 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fb46006b540 con 0x7fb43c075060 2024-09-21T06:10:49.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb456a00640 1 -- 172.21.15.50:0/2409361774 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fb448011bb0 con 0x7fb43c075060 2024-09-21T06:10:49.064 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:10:49.064 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:10:49.064 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:10:49.064 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:49.064 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:10:49.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb465800640 1 -- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb43c075060 msgr2=0x7fb43c077570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:49.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb465800640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb43c075060 0x7fb43c077570 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb4600b2500 tx=0x7fb4480091e0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb465800640 1 -- 172.21.15.50:0/2409361774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4600b4de0 msgr2=0x7fb4601f9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:49.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb465800640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4600b4de0 0x7fb4601f9670 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fb45000c730 tx=0x7fb45000cbf0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb465800640 1 -- 172.21.15.50:0/2409361774 shutdown_connections 2024-09-21T06:10:49.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb465800640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4601527b0 0x7fb4601fc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb465800640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4600b5c70 0x7fb460154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb465800640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb43c075060 0x7fb43c077570 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb465800640 1 --2- 172.21.15.50:0/2409361774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4600b4de0 0x7fb4601f9670 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb465800640 1 -- 172.21.15.50:0/2409361774 >> 172.21.15.50:0/2409361774 conn(0x7fb4600b2040 msgr2=0x7fb46014fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:49.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb465800640 1 -- 172.21.15.50:0/2409361774 shutdown_connections 2024-09-21T06:10:49.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.068+0000 7fb465800640 1 -- 172.21.15.50:0/2409361774 wait complete. 2024-09-21T06:10:49.086 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-21T06:10:49.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.248+0000 7f4331a00640 1 -- 172.21.15.50:0/3511317418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f432c0cefa0 msgr2=0x7f432c0cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:49.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.248+0000 7f4331a00640 1 --2- 172.21.15.50:0/3511317418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f432c0cefa0 0x7f432c0cf3b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4318009900 tx=0x7f431801c8e0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.248+0000 7f4331a00640 1 -- 172.21.15.50:0/3511317418 shutdown_connections 2024-09-21T06:10:49.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.248+0000 7f4331a00640 1 --2- 172.21.15.50:0/3511317418 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432c0b3340 0x7f432c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.248+0000 7f4331a00640 1 --2- 172.21.15.50:0/3511317418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f432c0cefa0 0x7f432c0cf3b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.248+0000 7f4331a00640 1 -- 172.21.15.50:0/3511317418 >> 172.21.15.50:0/3511317418 conn(0x7f432c0b2040 msgr2=0x7f432c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:49.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.248+0000 7f4331a00640 1 -- 172.21.15.50:0/3511317418 shutdown_connections 2024-09-21T06:10:49.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.248+0000 7f4331a00640 1 -- 172.21.15.50:0/3511317418 wait complete. 2024-09-21T06:10:49.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f4331a00640 1 Processor -- start 2024-09-21T06:10:49.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f4331a00640 1 -- start start 2024-09-21T06:10:49.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f4331a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c0749a0 0x7f432c1fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:49.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f4331a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432c0b3340 0x7f432c15f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:49.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f4331a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f432c0cefa0 0x7f432c1f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:49.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f4331a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f432c169cc0 con 0x7f432c0749a0 2024-09-21T06:10:49.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f4331a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f432c169b40 con 0x7f432c0cefa0 2024-09-21T06:10:49.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f4331a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f432c169e40 con 0x7f432c0b3340 2024-09-21T06:10:49.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f432aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c0749a0 0x7f432c1fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:49.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f432aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c0749a0 0x7f432c1fceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53144/0 (socket says 172.21.15.50:53144) 2024-09-21T06:10:49.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f432aa00640 1 -- 172.21.15.50:0/2020385326 learned_addr learned my addr 172.21.15.50:0/2020385326 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:49.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f432a000640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432c0b3340 0x7f432c15f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:49.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f432b400640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f432c0cefa0 0x7f432c1f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:49.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f432aa00640 1 -- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432c0b3340 msgr2=0x7f432c15f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:49.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f432aa00640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432c0b3340 0x7f432c15f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f432aa00640 1 -- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f432c0cefa0 msgr2=0x7f432c1f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:49.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f432aa00640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f432c0cefa0 0x7f432c1f97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.252+0000 7f432aa00640 1 -- 172.21.15.50:0/2020385326 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c1462f0 con 0x7f432c0749a0 2024-09-21T06:10:49.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.256+0000 7f432aa00640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c0749a0 0x7f432c1fceb0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c730 tx=0x7f431c00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:49.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.256+0000 7f4322a00640 1 -- 172.21.15.50:0/2020385326 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c0052d0 con 0x7f432c0749a0 2024-09-21T06:10:49.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.256+0000 7f4322a00640 1 -- 172.21.15.50:0/2020385326 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f431c001d20 con 0x7f432c0749a0 2024-09-21T06:10:49.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.256+0000 7f4331a00640 1 -- 172.21.15.50:0/2020385326 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c1fd3f0 con 0x7f432c0749a0 2024-09-21T06:10:49.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.256+0000 7f4322a00640 1 -- 172.21.15.50:0/2020385326 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c003820 con 0x7f432c0749a0 2024-09-21T06:10:49.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.256+0000 7f4331a00640 1 -- 172.21.15.50:0/2020385326 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c13ea10 con 0x7f432c0749a0 2024-09-21T06:10:49.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.256+0000 7f4331a00640 1 -- 172.21.15.50:0/2020385326 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432c0b3750 con 0x7f432c0749a0 2024-09-21T06:10:49.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.256+0000 7f4322a00640 1 -- 172.21.15.50:0/2020385326 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 49) ==== 99988+0+0 (secure 0 0 0) 0x7f431c0039c0 con 0x7f432c0749a0 2024-09-21T06:10:49.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.256+0000 7f4322a00640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f430c075060 0x7f430c077570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:49.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.256+0000 7f4322a00640 1 -- 172.21.15.50:0/2020385326 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f431c092b70 con 0x7f432c0749a0 2024-09-21T06:10:49.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.260+0000 7f432a000640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f430c075060 0x7f430c077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:49.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.260+0000 7f432a000640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f430c075060 0x7f430c077570 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f43180af000 tx=0x7f431801b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:49.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.264+0000 7f4322a00640 1 -- 172.21.15.50:0/2020385326 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f432c1fd3f0 con 0x7f432c0749a0 2024-09-21T06:10:49.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.464+0000 7f4331a00640 1 -- 172.21.15.50:0/2020385326 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f432c06b540 con 0x7f430c075060 2024-09-21T06:10:49.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.472+0000 7f4322a00640 1 -- 172.21.15.50:0/2020385326 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+185 (secure 0 0 0) 0x7f43180b3040 con 0x7f430c075060 2024-09-21T06:10:49.470 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:10:49.470 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:10:49.470 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:10:49.471 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sv1" 2024-09-21T06:10:49.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:49.471 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:10:49.471 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone73151" 2024-09-21T06:10:49.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:49.471 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:10:49.471 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone29872" 2024-09-21T06:10:49.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:10:49.471 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:10:49.471 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone87300" 2024-09-21T06:10:49.471 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:49.471 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:10:49.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.476+0000 7f4331a00640 1 -- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f430c075060 msgr2=0x7f430c077570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:49.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.476+0000 7f4331a00640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f430c075060 0x7f430c077570 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f43180af000 tx=0x7f431801b120 comp rx=0 tx=0).stop 2024-09-21T06:10:49.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.476+0000 7f4331a00640 1 -- 172.21.15.50:0/2020385326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c0749a0 msgr2=0x7f432c1fceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:49.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.476+0000 7f4331a00640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c0749a0 0x7f432c1fceb0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c730 tx=0x7f431c00cb30 comp rx=0 tx=0).stop 2024-09-21T06:10:49.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.476+0000 7f4331a00640 1 -- 172.21.15.50:0/2020385326 shutdown_connections 2024-09-21T06:10:49.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.476+0000 7f4331a00640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f432c0cefa0 0x7f432c1f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.476+0000 7f4331a00640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f432c0b3340 0x7f432c15f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.476+0000 7f4331a00640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f430c075060 0x7f430c077570 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.476+0000 7f4331a00640 1 --2- 172.21.15.50:0/2020385326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f432c0749a0 0x7f432c1fceb0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.476+0000 7f4331a00640 1 -- 172.21.15.50:0/2020385326 >> 172.21.15.50:0/2020385326 conn(0x7f432c0b2040 msgr2=0x7f432c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:49.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.476+0000 7f4331a00640 1 -- 172.21.15.50:0/2020385326 shutdown_connections 2024-09-21T06:10:49.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.476+0000 7f4331a00640 1 -- 172.21.15.50:0/2020385326 wait complete. 2024-09-21T06:10:49.493 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-21T06:10:49.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.660+0000 7f6d5f400640 1 -- 172.21.15.50:0/2245174380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d580b5c70 msgr2=0x7f6d5815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:49.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.660+0000 7f6d5f400640 1 --2- 172.21.15.50:0/2245174380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d580b5c70 0x7f6d5815bb60 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6d58072e30 tx=0x7f6d4801c800 comp rx=0 tx=0).stop 2024-09-21T06:10:49.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.660+0000 7f6d5f400640 1 -- 172.21.15.50:0/2245174380 shutdown_connections 2024-09-21T06:10:49.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.660+0000 7f6d5f400640 1 --2- 172.21.15.50:0/2245174380 >> v2:172.21.15.88:3300/0 conn(0x7f6d580b4de0 0x7f6d580b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.660+0000 7f6d5f400640 1 --2- 172.21.15.50:0/2245174380 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d581527b0 0x7f6d58074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.660+0000 7f6d5f400640 1 --2- 172.21.15.50:0/2245174380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d580b5c70 0x7f6d5815bb60 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.660+0000 7f6d5f400640 1 -- 172.21.15.50:0/2245174380 >> 172.21.15.50:0/2245174380 conn(0x7f6d580b2040 msgr2=0x7f6d580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:49.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.660+0000 7f6d5f400640 1 -- 172.21.15.50:0/2245174380 shutdown_connections 2024-09-21T06:10:49.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.660+0000 7f6d5f400640 1 -- 172.21.15.50:0/2245174380 wait complete. 2024-09-21T06:10:49.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.660+0000 7f6d5f400640 1 Processor -- start 2024-09-21T06:10:49.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.660+0000 7f6d5f400640 1 -- start start 2024-09-21T06:10:49.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d5f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d580b4de0 0x7f6d5813ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:49.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d5f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d580b5c70 0x7f6d58150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:49.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d5f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d581527b0 0x7f6d58150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:49.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d5f400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5815f660 con 0x7f6d580b5c70 2024-09-21T06:10:49.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d5f400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5815f4e0 con 0x7f6d581527b0 2024-09-21T06:10:49.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d5f400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5815f7e0 con 0x7f6d580b4de0 2024-09-21T06:10:49.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d4fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d580b5c70 0x7f6d58150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:49.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d4fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d580b5c70 0x7f6d58150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53156/0 (socket says 172.21.15.50:53156) 2024-09-21T06:10:49.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d4fe00640 1 -- 172.21.15.50:0/441220820 learned_addr learned my addr 172.21.15.50:0/441220820 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:49.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d5d400640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d581527b0 0x7f6d58150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:49.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d5ca00640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d580b4de0 0x7f6d5813ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:49.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d4fe00640 1 -- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d580b4de0 msgr2=0x7f6d5813ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:49.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d4fe00640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d580b4de0 0x7f6d5813ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d4fe00640 1 -- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d581527b0 msgr2=0x7f6d58150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:49.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d4fe00640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d581527b0 0x7f6d58150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d4fe00640 1 -- 172.21.15.50:0/441220820 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d58155890 con 0x7f6d580b5c70 2024-09-21T06:10:49.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d5d400640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d581527b0 0x7f6d58150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:49.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d4fe00640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d580b5c70 0x7f6d58150020 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6d40005d10 tx=0x7f6d4000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:49.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d55600640 1 -- 172.21.15.50:0/441220820 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40017070 con 0x7f6d580b5c70 2024-09-21T06:10:49.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d55600640 1 -- 172.21.15.50:0/441220820 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d40009b20 con 0x7f6d580b5c70 2024-09-21T06:10:49.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d5f400640 1 -- 172.21.15.50:0/441220820 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d581585b0 con 0x7f6d580b5c70 2024-09-21T06:10:49.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d55600640 1 -- 172.21.15.50:0/441220820 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40010460 con 0x7f6d580b5c70 2024-09-21T06:10:49.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d5f400640 1 -- 172.21.15.50:0/441220820 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d58156310 con 0x7f6d580b5c70 2024-09-21T06:10:49.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.664+0000 7f6d5f400640 1 -- 172.21.15.50:0/441220820 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5815fd20 con 0x7f6d580b5c70 2024-09-21T06:10:49.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.668+0000 7f6d55600640 1 -- 172.21.15.50:0/441220820 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 49) ==== 99988+0+0 (secure 0 0 0) 0x7f6d4001a070 con 0x7f6d580b5c70 2024-09-21T06:10:49.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.668+0000 7f6d55600640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d34075060 0x7f6d34077570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:49.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.668+0000 7f6d55600640 1 -- 172.21.15.50:0/441220820 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f6d400933d0 con 0x7f6d580b5c70 2024-09-21T06:10:49.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.668+0000 7f6d5ca00640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d34075060 0x7f6d34077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:49.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.668+0000 7f6d5ca00640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d34075060 0x7f6d34077570 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6d480af000 tx=0x7f6d4801b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:49.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.672+0000 7f6d55600640 1 -- 172.21.15.50:0/441220820 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6d4005fe50 con 0x7f6d580b5c70 2024-09-21T06:10:49.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.876+0000 7f6d5f400640 1 -- 172.21.15.50:0/441220820 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d5806b540 con 0x7f6d34075060 2024-09-21T06:10:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d55600640 1 -- 172.21.15.50:0/441220820 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6d48004040 con 0x7f6d34075060 2024-09-21T06:10:49.892 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:10:49.892 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:10:49.892 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:10:49.892 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ss1" 2024-09-21T06:10:49.892 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:10:49.892 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:10:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d5f400640 1 -- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d34075060 msgr2=0x7f6d34077570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d5f400640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d34075060 0x7f6d34077570 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6d480af000 tx=0x7f6d4801b040 comp rx=0 tx=0).stop 2024-09-21T06:10:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d5f400640 1 -- 172.21.15.50:0/441220820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d580b5c70 msgr2=0x7f6d58150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d5f400640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d580b5c70 0x7f6d58150020 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6d40005d10 tx=0x7f6d4000dbb0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d5f400640 1 -- 172.21.15.50:0/441220820 shutdown_connections 2024-09-21T06:10:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d5f400640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d581527b0 0x7f6d58150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d5f400640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d34075060 0x7f6d34077570 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d5f400640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d580b5c70 0x7f6d58150020 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d5f400640 1 --2- 172.21.15.50:0/441220820 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6d580b4de0 0x7f6d5813ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d5f400640 1 -- 172.21.15.50:0/441220820 >> 172.21.15.50:0/441220820 conn(0x7f6d580b2040 msgr2=0x7f6d5815b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d5f400640 1 -- 172.21.15.50:0/441220820 shutdown_connections 2024-09-21T06:10:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:49.896+0000 7f6d5f400640 1 -- 172.21.15.50:0/441220820 wait complete. 2024-09-21T06:10:49.915 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-21T06:10:50.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.084+0000 7fd0c4400640 1 -- 172.21.15.50:0/99103347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0bc0b5c70 msgr2=0x7fd0bc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.084+0000 7fd0c4400640 1 --2- 172.21.15.50:0/99103347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0bc0b5c70 0x7fd0bc15bb60 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc072e30 tx=0x7fd0b001c800 comp rx=0 tx=0).stop 2024-09-21T06:10:50.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.084+0000 7fd0c4400640 1 -- 172.21.15.50:0/99103347 shutdown_connections 2024-09-21T06:10:50.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.084+0000 7fd0c4400640 1 --2- 172.21.15.50:0/99103347 >> v2:172.21.15.88:3300/0 conn(0x7fd0bc0b4de0 0x7fd0bc0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.084+0000 7fd0c4400640 1 --2- 172.21.15.50:0/99103347 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0bc1527b0 0x7fd0bc074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.084+0000 7fd0c4400640 1 --2- 172.21.15.50:0/99103347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0bc0b5c70 0x7fd0bc15bb60 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.084+0000 7fd0c4400640 1 -- 172.21.15.50:0/99103347 >> 172.21.15.50:0/99103347 conn(0x7fd0bc0b2040 msgr2=0x7fd0bc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:50.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.084+0000 7fd0c4400640 1 -- 172.21.15.50:0/99103347 shutdown_connections 2024-09-21T06:10:50.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.084+0000 7fd0c4400640 1 -- 172.21.15.50:0/99103347 wait complete. 2024-09-21T06:10:50.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.084+0000 7fd0c4400640 1 Processor -- start 2024-09-21T06:10:50.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.084+0000 7fd0c4400640 1 -- start start 2024-09-21T06:10:50.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c4400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0bc0b4de0 0x7fd0bc1409b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c4400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0bc0b5c70 0x7fd0bc159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c4400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0bc1527b0 0x7fd0bc159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c4400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc15f700 con 0x7fd0bc0b5c70 2024-09-21T06:10:50.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c4400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc15f580 con 0x7fd0bc0b4de0 2024-09-21T06:10:50.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c4400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd0bc15f880 con 0x7fd0bc1527b0 2024-09-21T06:10:50.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0bc0b5c70 0x7fd0bc159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0bc0b5c70 0x7fd0bc159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53164/0 (socket says 172.21.15.50:53164) 2024-09-21T06:10:50.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c1000640 1 -- 172.21.15.50:0/1470888567 learned_addr learned my addr 172.21.15.50:0/1470888567 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:50.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c1a00640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0bc0b4de0 0x7fd0bc1409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c1000640 1 -- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0bc1527b0 msgr2=0x7fd0bc159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c2400640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0bc1527b0 0x7fd0bc159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c1000640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0bc1527b0 0x7fd0bc159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c1000640 1 -- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0bc0b4de0 msgr2=0x7fd0bc1409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c1000640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0bc0b4de0 0x7fd0bc1409b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c1000640 1 -- 172.21.15.50:0/1470888567 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc0cf9e0 con 0x7fd0bc0b5c70 2024-09-21T06:10:50.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c1a00640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0bc0b4de0 0x7fd0bc1409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:50.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c2400640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0bc1527b0 0x7fd0bc159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:50.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c1000640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0bc0b5c70 0x7fd0bc159720 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800c970 tx=0x7fd0a800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:50.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0b9600640 1 -- 172.21.15.50:0/1470888567 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8014070 con 0x7fd0bc0b5c70 2024-09-21T06:10:50.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0b9600640 1 -- 172.21.15.50:0/1470888567 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0a80077a0 con 0x7fd0bc0b5c70 2024-09-21T06:10:50.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c4400640 1 -- 172.21.15.50:0/1470888567 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0bc0cfcc0 con 0x7fd0bc0b5c70 2024-09-21T06:10:50.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0b9600640 1 -- 172.21.15.50:0/1470888567 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8007bf0 con 0x7fd0bc0b5c70 2024-09-21T06:10:50.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c4400640 1 -- 172.21.15.50:0/1470888567 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0bc14ef50 con 0x7fd0bc0b5c70 2024-09-21T06:10:50.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.088+0000 7fd0c4400640 1 -- 172.21.15.50:0/1470888567 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0bc15fdc0 con 0x7fd0bc0b5c70 2024-09-21T06:10:50.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.092+0000 7fd0b9600640 1 -- 172.21.15.50:0/1470888567 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 49) ==== 99988+0+0 (secure 0 0 0) 0x7fd0a8017070 con 0x7fd0bc0b5c70 2024-09-21T06:10:50.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.092+0000 7fd0b9600640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd09c075060 0x7fd09c077570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.092+0000 7fd0b9600640 1 -- 172.21.15.50:0/1470888567 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fd0a8093560 con 0x7fd0bc0b5c70 2024-09-21T06:10:50.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.092+0000 7fd0c1a00640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd09c075060 0x7fd09c077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.092+0000 7fd0c1a00640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd09c075060 0x7fd09c077570 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00af000 tx=0x7fd0b001b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:50.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.096+0000 7fd0b9600640 1 -- 172.21.15.50:0/1470888567 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd0a805fff0 con 0x7fd0bc0b5c70 2024-09-21T06:10:50.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.304+0000 7fd0c4400640 1 -- 172.21.15.50:0/1470888567 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd0bc06b540 con 0x7fd09c075060 2024-09-21T06:10:50.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0b9600640 1 -- 172.21.15.50:0/1470888567 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd0b00040c0 con 0x7fd09c075060 2024-09-21T06:10:50.332 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:10:50.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0c4400640 1 -- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd09c075060 msgr2=0x7fd09c077570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0c4400640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd09c075060 0x7fd09c077570 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00af000 tx=0x7fd0b001b040 comp rx=0 tx=0).stop 2024-09-21T06:10:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0c4400640 1 -- 172.21.15.50:0/1470888567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0bc0b5c70 msgr2=0x7fd0bc159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0c4400640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0bc0b5c70 0x7fd0bc159720 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800c970 tx=0x7fd0a800ce30 comp rx=0 tx=0).stop 2024-09-21T06:10:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0c4400640 1 -- 172.21.15.50:0/1470888567 shutdown_connections 2024-09-21T06:10:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0c4400640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0bc1527b0 0x7fd0bc159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0c4400640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd09c075060 0x7fd09c077570 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0c4400640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0bc0b5c70 0x7fd0bc159720 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0c4400640 1 --2- 172.21.15.50:0/1470888567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0bc0b4de0 0x7fd0bc1409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0c4400640 1 -- 172.21.15.50:0/1470888567 >> 172.21.15.50:0/1470888567 conn(0x7fd0bc0b2040 msgr2=0x7fd0bc06e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0c4400640 1 -- 172.21.15.50:0/1470888567 shutdown_connections 2024-09-21T06:10:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.336+0000 7fd0c4400640 1 -- 172.21.15.50:0/1470888567 wait complete. 2024-09-21T06:10:50.355 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-21T06:10:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.520+0000 7f4015000640 1 -- 172.21.15.50:0/2197497584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40101527b0 msgr2=0x7f4010074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.520+0000 7f4015000640 1 --2- 172.21.15.50:0/2197497584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40101527b0 0x7f4010074e30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009900 tx=0x7f3ff801c820 comp rx=0 tx=0).stop 2024-09-21T06:10:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.520+0000 7f4015000640 1 -- 172.21.15.50:0/2197497584 shutdown_connections 2024-09-21T06:10:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.520+0000 7f4015000640 1 --2- 172.21.15.50:0/2197497584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40101527b0 0x7f4010074e30 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.520+0000 7f4015000640 1 --2- 172.21.15.50:0/2197497584 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f40100b5c70 0x7f401015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.520+0000 7f4015000640 1 -- 172.21.15.50:0/2197497584 >> 172.21.15.50:0/2197497584 conn(0x7f40100b2040 msgr2=0x7f40100b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:50.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.520+0000 7f4015000640 1 -- 172.21.15.50:0/2197497584 shutdown_connections 2024-09-21T06:10:50.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.520+0000 7f4015000640 1 -- 172.21.15.50:0/2197497584 wait complete. 2024-09-21T06:10:50.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.520+0000 7f4015000640 1 Processor -- start 2024-09-21T06:10:50.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.520+0000 7f4015000640 1 -- start start 2024-09-21T06:10:50.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4015000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40100b4de0 0x7f40101563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4015000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f40100b5c70 0x7f40101fc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4015000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40101527b0 0x7f40100b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4015000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4010168c50 con 0x7f40100b4de0 2024-09-21T06:10:50.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4015000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4010168ad0 con 0x7f40101527b0 2024-09-21T06:10:50.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4015000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4010168dd0 con 0x7f40100b5c70 2024-09-21T06:10:50.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f400de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40100b4de0 0x7f40101563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f400de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40100b4de0 0x7f40101563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53166/0 (socket says 172.21.15.50:53166) 2024-09-21T06:10:50.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f400de00640 1 -- 172.21.15.50:0/912839197 learned_addr learned my addr 172.21.15.50:0/912839197 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:50.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f400e800640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40101527b0 0x7f40100b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4005400640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f40100b5c70 0x7f40101fc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f400de00640 1 -- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f40100b5c70 msgr2=0x7f40101fc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f400de00640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f40100b5c70 0x7f40101fc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f400de00640 1 -- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40101527b0 msgr2=0x7f40100b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f400de00640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40101527b0 0x7f40100b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f400de00640 1 -- 172.21.15.50:0/912839197 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40100ad260 con 0x7f40100b4de0 2024-09-21T06:10:50.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f400de00640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40100b4de0 0x7f40101563c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f400000e7d0 tx=0x7f400000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:50.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4006a00640 1 -- 172.21.15.50:0/912839197 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400000ccc0 con 0x7f40100b4de0 2024-09-21T06:10:50.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4006a00640 1 -- 172.21.15.50:0/912839197 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4000010430 con 0x7f40100b4de0 2024-09-21T06:10:50.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4015000640 1 -- 172.21.15.50:0/912839197 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40100ad540 con 0x7f40100b4de0 2024-09-21T06:10:50.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4006a00640 1 -- 172.21.15.50:0/912839197 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000010860 con 0x7f40100b4de0 2024-09-21T06:10:50.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4015000640 1 -- 172.21.15.50:0/912839197 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40101fbe90 con 0x7f40100b4de0 2024-09-21T06:10:50.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.524+0000 7f4015000640 1 -- 172.21.15.50:0/912839197 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401015bb60 con 0x7f40100b4de0 2024-09-21T06:10:50.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.528+0000 7f4006a00640 1 -- 172.21.15.50:0/912839197 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 49) ==== 99988+0+0 (secure 0 0 0) 0x7f4000010a00 con 0x7f40100b4de0 2024-09-21T06:10:50.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.528+0000 7f4006a00640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3fec075060 0x7f3fec077570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.528+0000 7f4006a00640 1 -- 172.21.15.50:0/912839197 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f4000093c00 con 0x7f40100b4de0 2024-09-21T06:10:50.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.528+0000 7f4005400640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3fec075060 0x7f3fec077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.528+0000 7f4005400640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3fec075060 0x7f3fec077570 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80af000 tx=0x7f3ff801b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:50.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.532+0000 7f4006a00640 1 -- 172.21.15.50:0/912839197 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f40000600c0 con 0x7f40100b4de0 2024-09-21T06:10:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.736+0000 7f4015000640 1 -- 172.21.15.50:0/912839197 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f401006b540 con 0x7f3fec075060 2024-09-21T06:10:50.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.760+0000 7f4006a00640 1 -- 172.21.15.50:0/912839197 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3ff80a6a80 con 0x7f3fec075060 2024-09-21T06:10:50.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.764+0000 7f4015000640 1 -- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3fec075060 msgr2=0x7f3fec077570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.764+0000 7f4015000640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3fec075060 0x7f3fec077570 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80af000 tx=0x7f3ff801b060 comp rx=0 tx=0).stop 2024-09-21T06:10:50.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.764+0000 7f4015000640 1 -- 172.21.15.50:0/912839197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40100b4de0 msgr2=0x7f40101563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.764+0000 7f4015000640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40100b4de0 0x7f40101563c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f400000e7d0 tx=0x7f400000ec90 comp rx=0 tx=0).stop 2024-09-21T06:10:50.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.764+0000 7f4015000640 1 -- 172.21.15.50:0/912839197 shutdown_connections 2024-09-21T06:10:50.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.764+0000 7f4015000640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40101527b0 0x7f40100b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.764+0000 7f4015000640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f40100b5c70 0x7f40101fc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.764+0000 7f4015000640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3fec075060 0x7f3fec077570 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.764+0000 7f4015000640 1 --2- 172.21.15.50:0/912839197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40100b4de0 0x7f40101563c0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.764+0000 7f4015000640 1 -- 172.21.15.50:0/912839197 >> 172.21.15.50:0/912839197 conn(0x7f40100b2040 msgr2=0x7f4010152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:50.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.764+0000 7f4015000640 1 -- 172.21.15.50:0/912839197 shutdown_connections 2024-09-21T06:10:50.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.764+0000 7f4015000640 1 -- 172.21.15.50:0/912839197 wait complete. 2024-09-21T06:10:50.774 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:50.777+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:50.774 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:50.777+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:50.774 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:50.777+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:50.774 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:50.777+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:50.774 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:50.777+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:50.783 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone73151 2024-09-21T06:10:50.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.948+0000 7f44d6200640 1 -- 172.21.15.50:0/3108292962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44d0074c30 msgr2=0x7f44d0075040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.948+0000 7f44d6200640 1 --2- 172.21.15.50:0/3108292962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44d0074c30 0x7f44d0075040 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009840 tx=0x7f44bc01c760 comp rx=0 tx=0).stop 2024-09-21T06:10:50.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.948+0000 7f44d6200640 1 -- 172.21.15.50:0/3108292962 shutdown_connections 2024-09-21T06:10:50.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.948+0000 7f44d6200640 1 --2- 172.21.15.50:0/3108292962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44d0074c30 0x7f44d0075040 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.948+0000 7f44d6200640 1 --2- 172.21.15.50:0/3108292962 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44d00746e0 0x7f44d015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.948+0000 7f44d6200640 1 -- 172.21.15.50:0/3108292962 >> 172.21.15.50:0/3108292962 conn(0x7f44d00b41f0 msgr2=0x7f44d00b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:50.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.948+0000 7f44d6200640 1 -- 172.21.15.50:0/3108292962 shutdown_connections 2024-09-21T06:10:50.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.948+0000 7f44d6200640 1 -- 172.21.15.50:0/3108292962 wait complete. 2024-09-21T06:10:50.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.952+0000 7f44d6200640 1 Processor -- start 2024-09-21T06:10:50.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.952+0000 7f44d6200640 1 -- start start 2024-09-21T06:10:50.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.952+0000 7f44d6200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44d00746e0 0x7f44d0099550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.952+0000 7f44d6200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44d0074c30 0x7f44d00708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.952+0000 7f44d6200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44d00d1190 0x7f44d0151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.952+0000 7f44d6200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44d016a310 con 0x7f44d00d1190 2024-09-21T06:10:50.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.952+0000 7f44d6200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44d016a190 con 0x7f44d00746e0 2024-09-21T06:10:50.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.952+0000 7f44d6200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f44d016a490 con 0x7f44d0074c30 2024-09-21T06:10:50.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.952+0000 7f44d5800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44d00d1190 0x7f44d0151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.952+0000 7f44d5800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44d00d1190 0x7f44d0151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53192/0 (socket says 172.21.15.50:53192) 2024-09-21T06:10:50.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.952+0000 7f44d5800640 1 -- 172.21.15.50:0/4260009691 learned_addr learned my addr 172.21.15.50:0/4260009691 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:50.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.952+0000 7f44d4e00640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44d00746e0 0x7f44d0099550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44cfe00640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44d0074c30 0x7f44d00708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44d5800640 1 -- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44d0074c30 msgr2=0x7f44d00708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44d5800640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44d0074c30 0x7f44d00708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44d5800640 1 -- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44d00746e0 msgr2=0x7f44d0099550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44d5800640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44d00746e0 0x7f44d0099550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44d5800640 1 -- 172.21.15.50:0/4260009691 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d0159460 con 0x7f44d00d1190 2024-09-21T06:10:50.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44d5800640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44d00d1190 0x7f44d0151090 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f44c800cb10 tx=0x7f44c800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:50.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44ccc00640 1 -- 172.21.15.50:0/4260009691 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8016020 con 0x7f44d00d1190 2024-09-21T06:10:50.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44ccc00640 1 -- 172.21.15.50:0/4260009691 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44c800ce50 con 0x7f44d00d1190 2024-09-21T06:10:50.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44ccc00640 1 -- 172.21.15.50:0/4260009691 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c800ade0 con 0x7f44d00d1190 2024-09-21T06:10:50.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44d6200640 1 -- 172.21.15.50:0/4260009691 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d01f71b0 con 0x7f44d00d1190 2024-09-21T06:10:50.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44d6200640 1 -- 172.21.15.50:0/4260009691 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d01fb1f0 con 0x7f44d00d1190 2024-09-21T06:10:50.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44d6200640 1 -- 172.21.15.50:0/4260009691 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d015c920 con 0x7f44d00d1190 2024-09-21T06:10:50.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.956+0000 7f44ccc00640 1 -- 172.21.15.50:0/4260009691 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 49) ==== 99988+0+0 (secure 0 0 0) 0x7f44c8019020 con 0x7f44d00d1190 2024-09-21T06:10:50.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.960+0000 7f44ccc00640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44b0075060 0x7f44b0077570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.960+0000 7f44ccc00640 1 -- 172.21.15.50:0/4260009691 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f44c8092fa0 con 0x7f44d00d1190 2024-09-21T06:10:50.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.960+0000 7f44d4e00640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44b0075060 0x7f44b0077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.960+0000 7f44d4e00640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44b0075060 0x7f44b0077570 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009600 tx=0x7f44c40091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:50.964+0000 7f44ccc00640 1 -- 172.21.15.50:0/4260009691 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f44c8020030 con 0x7f44d00d1190 2024-09-21T06:10:51.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.168+0000 7f44d6200640 1 -- 172.21.15.50:0/4260009691 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone73151", "target": ["mon-mgr", ""]}) -- 0x7f44d006b540 con 0x7f44b0075060 2024-09-21T06:10:51.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.464+0000 7f44ccc00640 1 -- 172.21.15.50:0/4260009691 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f44c4011cd0 con 0x7f44b0075060 2024-09-21T06:10:51.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.468+0000 7f44d6200640 1 -- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44b0075060 msgr2=0x7f44b0077570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.468+0000 7f44d6200640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44b0075060 0x7f44b0077570 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009600 tx=0x7f44c40091e0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.468+0000 7f44d6200640 1 -- 172.21.15.50:0/4260009691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44d00d1190 msgr2=0x7f44d0151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.468+0000 7f44d6200640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44d00d1190 0x7f44d0151090 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f44c800cb10 tx=0x7f44c800cc00 comp rx=0 tx=0).stop 2024-09-21T06:10:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.468+0000 7f44d6200640 1 -- 172.21.15.50:0/4260009691 shutdown_connections 2024-09-21T06:10:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.468+0000 7f44d6200640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44b0075060 0x7f44b0077570 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.468+0000 7f44d6200640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44d00d1190 0x7f44d0151090 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.468+0000 7f44d6200640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44d0074c30 0x7f44d00708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.468+0000 7f44d6200640 1 --2- 172.21.15.50:0/4260009691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44d00746e0 0x7f44d0099550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.468+0000 7f44d6200640 1 -- 172.21.15.50:0/4260009691 >> 172.21.15.50:0/4260009691 conn(0x7f44d00b41f0 msgr2=0x7f44d0155830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.468+0000 7f44d6200640 1 -- 172.21.15.50:0/4260009691 shutdown_connections 2024-09-21T06:10:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.468+0000 7f44d6200640 1 -- 172.21.15.50:0/4260009691 wait complete. 2024-09-21T06:10:51.485 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone29872 2024-09-21T06:10:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.656+0000 7f43c4200640 1 -- 172.21.15.50:0/2847018984 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43bc157030 msgr2=0x7f43bc0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.656+0000 7f43c4200640 1 --2- 172.21.15.50:0/2847018984 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43bc157030 0x7f43bc0b5a80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009840 tx=0x7f43b001c820 comp rx=0 tx=0).stop 2024-09-21T06:10:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.656+0000 7f43c4200640 1 -- 172.21.15.50:0/2847018984 shutdown_connections 2024-09-21T06:10:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.656+0000 7f43c4200640 1 --2- 172.21.15.50:0/2847018984 >> v2:172.21.15.88:3300/0 conn(0x7f43bc074b20 0x7f43bc15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.656+0000 7f43c4200640 1 --2- 172.21.15.50:0/2847018984 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43bc157030 0x7f43bc0b5a80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.656+0000 7f43c4200640 1 -- 172.21.15.50:0/2847018984 >> 172.21.15.50:0/2847018984 conn(0x7f43bc0708c0 msgr2=0x7f43bc070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.656+0000 7f43c4200640 1 -- 172.21.15.50:0/2847018984 shutdown_connections 2024-09-21T06:10:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.656+0000 7f43c4200640 1 -- 172.21.15.50:0/2847018984 wait complete. 2024-09-21T06:10:51.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.656+0000 7f43c4200640 1 Processor -- start 2024-09-21T06:10:51.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.656+0000 7f43c4200640 1 -- start start 2024-09-21T06:10:51.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c4200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc157030 0x7f43bc138d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c4200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43bc160d50 0x7f43bc155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c4200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43bc150e60 0x7f43bc155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c4200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc16a780 con 0x7f43bc160d50 2024-09-21T06:10:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c4200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc16a600 con 0x7f43bc157030 2024-09-21T06:10:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c4200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f43bc16a900 con 0x7f43bc150e60 2024-09-21T06:10:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c0e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43bc160d50 0x7f43bc155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c0e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43bc160d50 0x7f43bc155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53208/0 (socket says 172.21.15.50:53208) 2024-09-21T06:10:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c0e00640 1 -- 172.21.15.50:0/3863347972 learned_addr learned my addr 172.21.15.50:0/3863347972 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c1800640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc157030 0x7f43bc138d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c0e00640 1 -- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43bc150e60 msgr2=0x7f43bc155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c0e00640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43bc150e60 0x7f43bc155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c0e00640 1 -- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc157030 msgr2=0x7f43bc138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c0e00640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc157030 0x7f43bc138d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c0e00640 1 -- 172.21.15.50:0/3863347972 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43bc154580 con 0x7f43bc160d50 2024-09-21T06:10:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c1800640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc157030 0x7f43bc138d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c0e00640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43bc160d50 0x7f43bc155340 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c390 tx=0x7f43a800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43b9600640 1 -- 172.21.15.50:0/3863347972 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8011790 con 0x7f43bc160d50 2024-09-21T06:10:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c4200640 1 -- 172.21.15.50:0/3863347972 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43bc154860 con 0x7f43bc160d50 2024-09-21T06:10:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43b9600640 1 -- 172.21.15.50:0/3863347972 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43a8017070 con 0x7f43bc160d50 2024-09-21T06:10:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43b9600640 1 -- 172.21.15.50:0/3863347972 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8011cc0 con 0x7f43bc160d50 2024-09-21T06:10:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c4200640 1 -- 172.21.15.50:0/3863347972 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43bc139290 con 0x7f43bc160d50 2024-09-21T06:10:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.660+0000 7f43c4200640 1 -- 172.21.15.50:0/3863347972 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43bc0b45e0 con 0x7f43bc160d50 2024-09-21T06:10:51.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.664+0000 7f43b9600640 1 -- 172.21.15.50:0/3863347972 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 49) ==== 99988+0+0 (secure 0 0 0) 0x7f43a801c460 con 0x7f43bc160d50 2024-09-21T06:10:51.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.664+0000 7f43b9600640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f439c075090 0x7f439c0775a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.664+0000 7f43b9600640 1 -- 172.21.15.50:0/3863347972 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f43a8093000 con 0x7f43bc160d50 2024-09-21T06:10:51.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.664+0000 7f43c1800640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f439c075090 0x7f439c0775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.664+0000 7f43c1800640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f439c075090 0x7f439c0775a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f43b00af000 tx=0x7f43b001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:51.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.668+0000 7f43b9600640 1 -- 172.21.15.50:0/3863347972 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f43a805fa00 con 0x7f43bc160d50 2024-09-21T06:10:51.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.824+0000 7f43b9600640 1 -- 172.21.15.50:0/3863347972 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 50) ==== 100039+0+0 (secure 0 0 0) 0x7f43a805b570 con 0x7f43bc160d50 2024-09-21T06:10:51.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:51.876+0000 7f43c4200640 1 -- 172.21.15.50:0/3863347972 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone29872", "target": ["mon-mgr", ""]}) -- 0x7f43bc14b4f0 con 0x7f439c075090 2024-09-21T06:10:58.178 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:58.181+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:58.179 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:58.181+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:58.179 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:58.181+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:58.179 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:58.181+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:58.179 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:10:58.181+0000 7f65a3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:10:58.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.204+0000 7f43b9600640 1 -- 172.21.15.50:0/3863347972 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f43b00a66d0 con 0x7f439c075090 2024-09-21T06:10:58.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.208+0000 7f43c4200640 1 -- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f439c075090 msgr2=0x7f439c0775a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:58.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.208+0000 7f43c4200640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f439c075090 0x7f439c0775a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f43b00af000 tx=0x7f43b001b060 comp rx=0 tx=0).stop 2024-09-21T06:10:58.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.208+0000 7f43c4200640 1 -- 172.21.15.50:0/3863347972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43bc160d50 msgr2=0x7f43bc155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:58.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.208+0000 7f43c4200640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43bc160d50 0x7f43bc155340 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c390 tx=0x7f43a800c850 comp rx=0 tx=0).stop 2024-09-21T06:10:58.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.208+0000 7f43c4200640 1 -- 172.21.15.50:0/3863347972 shutdown_connections 2024-09-21T06:10:58.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.208+0000 7f43c4200640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f43bc150e60 0x7f43bc155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.208+0000 7f43c4200640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f439c075090 0x7f439c0775a0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.208+0000 7f43c4200640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43bc160d50 0x7f43bc155340 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.208+0000 7f43c4200640 1 --2- 172.21.15.50:0/3863347972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc157030 0x7f43bc138d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.208+0000 7f43c4200640 1 -- 172.21.15.50:0/3863347972 >> 172.21.15.50:0/3863347972 conn(0x7f43bc0708c0 msgr2=0x7f43bc074410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:58.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.208+0000 7f43c4200640 1 -- 172.21.15.50:0/3863347972 shutdown_connections 2024-09-21T06:10:58.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.208+0000 7f43c4200640 1 -- 172.21.15.50:0/3863347972 wait complete. 2024-09-21T06:10:58.233 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone87300 2024-09-21T06:10:58.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.400+0000 7f5c05200640 1 -- 172.21.15.50:0/3202949233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c000b3340 msgr2=0x7f5c000b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:58.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.400+0000 7f5c05200640 1 --2- 172.21.15.50:0/3202949233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c000b3340 0x7f5c000b3750 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5c000b3040 tx=0x7f5bf801cd80 comp rx=0 tx=0).stop 2024-09-21T06:10:58.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.400+0000 7f5c05200640 1 -- 172.21.15.50:0/3202949233 shutdown_connections 2024-09-21T06:10:58.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 --2- 172.21.15.50:0/3202949233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c000b3340 0x7f5c000b3750 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 --2- 172.21.15.50:0/3202949233 >> v2:172.21.15.88:3300/0 conn(0x7f5c000749a0 0x7f5c00074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 -- 172.21.15.50:0/3202949233 >> 172.21.15.50:0/3202949233 conn(0x7f5c000b2040 msgr2=0x7f5c000b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:58.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 -- 172.21.15.50:0/3202949233 shutdown_connections 2024-09-21T06:10:58.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 -- 172.21.15.50:0/3202949233 wait complete. 2024-09-21T06:10:58.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 Processor -- start 2024-09-21T06:10:58.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 -- start start 2024-09-21T06:10:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c000749a0 0x7f5c00137f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c000b3340 0x7f5c000cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c000cefa0 0x7f5c00157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5c00169cd0 con 0x7f5c000b3340 2024-09-21T06:10:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c00169b50 con 0x7f5c000749a0 2024-09-21T06:10:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.404+0000 7f5c05200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5c00169e50 con 0x7f5c000cefa0 2024-09-21T06:10:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c000b3340 0x7f5c000cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c000b3340 0x7f5c000cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40546/0 (socket says 172.21.15.50:40546) 2024-09-21T06:10:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf7e00640 1 -- 172.21.15.50:0/2041229466 learned_addr learned my addr 172.21.15.50:0/2041229466 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bfde00640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c000749a0 0x7f5c00137f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:58.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bfe800640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c000cefa0 0x7f5c00157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:58.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf7e00640 1 -- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c000cefa0 msgr2=0x7f5c00157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:58.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf7e00640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c000cefa0 0x7f5c00157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf7e00640 1 -- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c000749a0 msgr2=0x7f5c00137f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:58.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf7e00640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c000749a0 0x7f5c00137f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf7e00640 1 -- 172.21.15.50:0/2041229466 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c001fc090 con 0x7f5c000b3340 2024-09-21T06:10:58.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf7e00640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c000b3340 0x7f5c000cd810 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5be800c3c0 tx=0x7f5be800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf6000640 1 -- 172.21.15.50:0/2041229466 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8017070 con 0x7f5c000b3340 2024-09-21T06:10:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf6000640 1 -- 172.21.15.50:0/2041229466 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5be80053c0 con 0x7f5c000b3340 2024-09-21T06:10:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5c05200640 1 -- 172.21.15.50:0/2041229466 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c00138470 con 0x7f5c000b3340 2024-09-21T06:10:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf6000640 1 -- 172.21.15.50:0/2041229466 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8003860 con 0x7f5c000b3340 2024-09-21T06:10:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5c05200640 1 -- 172.21.15.50:0/2041229466 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c001fc840 con 0x7f5c000b3340 2024-09-21T06:10:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5c05200640 1 -- 172.21.15.50:0/2041229466 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c00074db0 con 0x7f5c000b3340 2024-09-21T06:10:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.408+0000 7f5bf6000640 1 -- 172.21.15.50:0/2041229466 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 50) ==== 100039+0+0 (secure 0 0 0) 0x7f5be801a070 con 0x7f5c000b3340 2024-09-21T06:10:58.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.412+0000 7f5bf6000640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5bdc075120 0x7f5bdc077630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:58.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.412+0000 7f5bf6000640 1 -- 172.21.15.50:0/2041229466 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f5be8092660 con 0x7f5c000b3340 2024-09-21T06:10:58.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.412+0000 7f5bfde00640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5bdc075120 0x7f5bdc077630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:58.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.412+0000 7f5bfde00640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5bdc075120 0x7f5bdc077630 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000bb50 tx=0x7f5bf000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:58.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.412+0000 7f5bf6000640 1 -- 172.21.15.50:0/2041229466 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5be8021080 con 0x7f5c000b3340 2024-09-21T06:10:58.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.620+0000 7f5c05200640 1 -- 172.21.15.50:0/2041229466 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone87300", "target": ["mon-mgr", ""]}) -- 0x7f5c0006b540 con 0x7f5bdc075120 2024-09-21T06:10:58.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5bf6000640 1 -- 172.21.15.50:0/2041229466 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5bf0015070 con 0x7f5bdc075120 2024-09-21T06:10:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5c05200640 1 -- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5bdc075120 msgr2=0x7f5bdc077630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5c05200640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5bdc075120 0x7f5bdc077630 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000bb50 tx=0x7f5bf000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5c05200640 1 -- 172.21.15.50:0/2041229466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c000b3340 msgr2=0x7f5c000cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5c05200640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c000b3340 0x7f5c000cd810 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5be800c3c0 tx=0x7f5be800c880 comp rx=0 tx=0).stop 2024-09-21T06:10:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5c05200640 1 -- 172.21.15.50:0/2041229466 shutdown_connections 2024-09-21T06:10:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5c05200640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5c000cefa0 0x7f5c00157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5c05200640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5bdc075120 0x7f5bdc077630 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5c05200640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c000b3340 0x7f5c000cd810 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5c05200640 1 --2- 172.21.15.50:0/2041229466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c000749a0 0x7f5c00137f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5c05200640 1 -- 172.21.15.50:0/2041229466 >> 172.21.15.50:0/2041229466 conn(0x7f5c000b2040 msgr2=0x7f5c0015a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5c05200640 1 -- 172.21.15.50:0/2041229466 shutdown_connections 2024-09-21T06:10:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.644+0000 7f5c05200640 1 -- 172.21.15.50:0/2041229466 wait complete. 2024-09-21T06:10:58.665 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T06:10:58.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:10:58.721 INFO:teuthology.orchestra.run.smithi050.stdout:2 2024-09-21T06:10:58.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:10:58.729 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:10:58.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.892+0000 7fb809000640 1 -- 172.21.15.50:0/2011606026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8040749a0 msgr2=0x7fb804074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:58.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.892+0000 7fb809000640 1 --2- 172.21.15.50:0/2011606026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8040749a0 0x7fb804074db0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fb80415a930 tx=0x7fb7f401c8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.896+0000 7fb809000640 1 -- 172.21.15.50:0/2011606026 shutdown_connections 2024-09-21T06:10:58.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.896+0000 7fb809000640 1 --2- 172.21.15.50:0/2011606026 >> v2:172.21.15.88:3300/0 conn(0x7fb804160330 0x7fb804160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.896+0000 7fb809000640 1 --2- 172.21.15.50:0/2011606026 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8040cefa0 0x7fb8040cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.896+0000 7fb809000640 1 --2- 172.21.15.50:0/2011606026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8040749a0 0x7fb804074db0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.896+0000 7fb809000640 1 -- 172.21.15.50:0/2011606026 >> 172.21.15.50:0/2011606026 conn(0x7fb8040b2040 msgr2=0x7fb8040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:58.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.896+0000 7fb809000640 1 -- 172.21.15.50:0/2011606026 shutdown_connections 2024-09-21T06:10:58.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.896+0000 7fb809000640 1 -- 172.21.15.50:0/2011606026 wait complete. 2024-09-21T06:10:58.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.896+0000 7fb809000640 1 Processor -- start 2024-09-21T06:10:58.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.896+0000 7fb809000640 1 -- start start 2024-09-21T06:10:58.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.896+0000 7fb809000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8040749a0 0x7fb8041f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:58.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.896+0000 7fb809000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8040cefa0 0x7fb80415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:58.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb809000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb804160330 0x7fb80415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:58.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb809000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb8040b3ee0 con 0x7fb8040749a0 2024-09-21T06:10:58.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb809000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb8040b3d60 con 0x7fb8040cefa0 2024-09-21T06:10:58.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb809000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb8040b4060 con 0x7fb804160330 2024-09-21T06:10:58.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb801e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8040749a0 0x7fb8041f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb801e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8040749a0 0x7fb8041f8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40570/0 (socket says 172.21.15.50:40570) 2024-09-21T06:10:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb801e00640 1 -- 172.21.15.50:0/152730233 learned_addr learned my addr 172.21.15.50:0/152730233 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb7fbe00640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8040cefa0 0x7fb80415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb802800640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb804160330 0x7fb80415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb801e00640 1 -- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb804160330 msgr2=0x7fb80415dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb801e00640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb804160330 0x7fb80415dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb801e00640 1 -- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8040cefa0 msgr2=0x7fb80415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb801e00640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8040cefa0 0x7fb80415bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb801e00640 1 -- 172.21.15.50:0/152730233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8041f99f0 con 0x7fb8040749a0 2024-09-21T06:10:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb7fbe00640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8040cefa0 0x7fb80415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb801e00640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8040749a0 0x7fb8041f8f50 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40b0040 tx=0x7fb7f4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:58.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb7fa000640 1 -- 172.21.15.50:0/152730233 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40b2040 con 0x7fb8040749a0 2024-09-21T06:10:58.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb7fa000640 1 -- 172.21.15.50:0/152730233 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7f4005030 con 0x7fb8040749a0 2024-09-21T06:10:58.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb809000640 1 -- 172.21.15.50:0/152730233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8041f9cd0 con 0x7fb8040749a0 2024-09-21T06:10:58.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb7fa000640 1 -- 172.21.15.50:0/152730233 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4005480 con 0x7fb8040749a0 2024-09-21T06:10:58.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb809000640 1 -- 172.21.15.50:0/152730233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8041fa590 con 0x7fb8040749a0 2024-09-21T06:10:58.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb809000640 1 -- 172.21.15.50:0/152730233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8040cf3b0 con 0x7fb8040749a0 2024-09-21T06:10:58.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.900+0000 7fb7fa000640 1 -- 172.21.15.50:0/152730233 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 50) ==== 100039+0+0 (secure 0 0 0) 0x7fb7f40046d0 con 0x7fb8040749a0 2024-09-21T06:10:58.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.904+0000 7fb7fa000640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7e00750f0 0x7fb7e0077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:58.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.904+0000 7fb7fa000640 1 -- 172.21.15.50:0/152730233 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7fb7f412d090 con 0x7fb8040749a0 2024-09-21T06:10:58.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.904+0000 7fb7fbe00640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7e00750f0 0x7fb7e0077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:58.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.904+0000 7fb7fbe00640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7e00750f0 0x7fb7e0077600 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb80415f7b0 tx=0x7fb7ec0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:58.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:58.908+0000 7fb7fa000640 1 -- 172.21.15.50:0/152730233 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb7f40164b0 con 0x7fb8040749a0 2024-09-21T06:10:59.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.084+0000 7fb809000640 1 -- 172.21.15.50:0/152730233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fb8040b0190 con 0x7fb8040749a0 2024-09-21T06:10:59.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.084+0000 7fb7fa000640 1 -- 172.21.15.50:0/152730233 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v53)=0 v53) ==== 131+0+0 (secure 0 0 0) 0x7fb7f40f9c20 con 0x7fb8040749a0 2024-09-21T06:10:59.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.088+0000 7fb809000640 1 -- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7e00750f0 msgr2=0x7fb7e0077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:59.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.088+0000 7fb809000640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7e00750f0 0x7fb7e0077600 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb80415f7b0 tx=0x7fb7ec0091e0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.088+0000 7fb809000640 1 -- 172.21.15.50:0/152730233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8040749a0 msgr2=0x7fb8041f8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:59.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.088+0000 7fb809000640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8040749a0 0x7fb8041f8f50 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40b0040 tx=0x7fb7f4004600 comp rx=0 tx=0).stop 2024-09-21T06:10:59.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.088+0000 7fb809000640 1 -- 172.21.15.50:0/152730233 shutdown_connections 2024-09-21T06:10:59.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.088+0000 7fb809000640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb804160330 0x7fb80415dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.088+0000 7fb809000640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8040cefa0 0x7fb80415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.088+0000 7fb809000640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7e00750f0 0x7fb7e0077600 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.088+0000 7fb809000640 1 --2- 172.21.15.50:0/152730233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8040749a0 0x7fb8041f8f50 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.088+0000 7fb809000640 1 -- 172.21.15.50:0/152730233 >> 172.21.15.50:0/152730233 conn(0x7fb8040b2040 msgr2=0x7fb8040b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:59.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.088+0000 7fb809000640 1 -- 172.21.15.50:0/152730233 shutdown_connections 2024-09-21T06:10:59.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.088+0000 7fb809000640 1 -- 172.21.15.50:0/152730233 wait complete. 2024-09-21T06:10:59.105 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:10:59.105 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:10:59.210 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:10:59.303 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:10:59.304 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:10:59.304 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:10:59.304 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:10:59.313 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:10:59.313 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:10:59.368 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:10:59.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.576+0000 7f687de00640 1 -- 172.21.15.50:0/2538048176 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f687806cdf0 msgr2=0x7f687806d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:59.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.576+0000 7f687de00640 1 --2- 172.21.15.50:0/2538048176 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f687806cdf0 0x7f687806d200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6860009870 tx=0x7f686001c7d0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.576+0000 7f687de00640 1 -- 172.21.15.50:0/2538048176 shutdown_connections 2024-09-21T06:10:59.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.576+0000 7f687de00640 1 --2- 172.21.15.50:0/2538048176 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f687806cdf0 0x7f687806d200 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.576+0000 7f687de00640 1 -- 172.21.15.50:0/2538048176 >> 172.21.15.50:0/2538048176 conn(0x7f68780b35f0 msgr2=0x7f68780b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:59.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.576+0000 7f687de00640 1 -- 172.21.15.50:0/2538048176 shutdown_connections 2024-09-21T06:10:59.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.576+0000 7f687de00640 1 -- 172.21.15.50:0/2538048176 wait complete. 2024-09-21T06:10:59.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f687de00640 1 Processor -- start 2024-09-21T06:10:59.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f687de00640 1 -- start start 2024-09-21T06:10:59.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f687de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687806cdf0 0x7f687815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:59.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f687de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f687806fc90 0x7f68781fa130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:59.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f687de00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6878159130 0x7f6878156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:59.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f687de00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f687816a5d0 con 0x7f687806cdf0 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f687de00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f687816a450 con 0x7f687806fc90 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f687de00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f687816a750 con 0x7f6878159130 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f6876a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687806cdf0 0x7f687815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f6876a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687806cdf0 0x7f687815cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40580/0 (socket says 172.21.15.50:40580) 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f6876a00640 1 -- 172.21.15.50:0/465881933 learned_addr learned my addr 172.21.15.50:0/465881933 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f686fe00640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f687806fc90 0x7f68781fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f6877400640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6878159130 0x7f6878156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f6876a00640 1 -- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6878159130 msgr2=0x7f6878156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f6876a00640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6878159130 0x7f6878156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f6876a00640 1 -- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f687806fc90 msgr2=0x7f68781fa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f6876a00640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f687806fc90 0x7f68781fa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f6876a00640 1 -- 172.21.15.50:0/465881933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6878069be0 con 0x7f687806cdf0 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.580+0000 7f6876a00640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687806cdf0 0x7f687815cd30 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f686800c400 tx=0x7f686800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:59.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.584+0000 7f686f400640 1 -- 172.21.15.50:0/465881933 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868017070 con 0x7f687806cdf0 2024-09-21T06:10:59.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.584+0000 7f686f400640 1 -- 172.21.15.50:0/465881933 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6868005400 con 0x7f687806cdf0 2024-09-21T06:10:59.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.584+0000 7f686f400640 1 -- 172.21.15.50:0/465881933 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868003860 con 0x7f687806cdf0 2024-09-21T06:10:59.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.584+0000 7f687de00640 1 -- 172.21.15.50:0/465881933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687815ffa0 con 0x7f687806cdf0 2024-09-21T06:10:59.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.584+0000 7f687de00640 1 -- 172.21.15.50:0/465881933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68781f9970 con 0x7f687806cdf0 2024-09-21T06:10:59.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.584+0000 7f687de00640 1 -- 172.21.15.50:0/465881933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f687815e980 con 0x7f687806cdf0 2024-09-21T06:10:59.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.584+0000 7f686f400640 1 -- 172.21.15.50:0/465881933 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 50) ==== 100039+0+0 (secure 0 0 0) 0x7f686801a070 con 0x7f687806cdf0 2024-09-21T06:10:59.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.584+0000 7f686f400640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f68540750f0 0x7f6854077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:59.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.584+0000 7f686f400640 1 -- 172.21.15.50:0/465881933 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f68680926a0 con 0x7f687806cdf0 2024-09-21T06:10:59.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.584+0000 7f686fe00640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f68540750f0 0x7f6854077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:59.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.588+0000 7f686fe00640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f68540750f0 0x7f6854077600 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f68600af000 tx=0x7f68600b2040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:59.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.588+0000 7f686f400640 1 -- 172.21.15.50:0/465881933 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f686805ead0 con 0x7f687806cdf0 2024-09-21T06:10:59.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.776+0000 7f687de00640 1 -- 172.21.15.50:0/465881933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f68781fd6b0 con 0x7f687806cdf0 2024-09-21T06:10:59.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.776+0000 7f686f400640 1 -- 172.21.15.50:0/465881933 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v115) ==== 80+0+0 (secure 0 0 0) 0x7f6868063500 con 0x7f687806cdf0 2024-09-21T06:10:59.775 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T06:10:59.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.780+0000 7f687de00640 1 -- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f68540750f0 msgr2=0x7f6854077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:59.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.780+0000 7f687de00640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f68540750f0 0x7f6854077600 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f68600af000 tx=0x7f68600b2040 comp rx=0 tx=0).stop 2024-09-21T06:10:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.780+0000 7f687de00640 1 -- 172.21.15.50:0/465881933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687806cdf0 msgr2=0x7f687815cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.780+0000 7f687de00640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687806cdf0 0x7f687815cd30 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f686800c400 tx=0x7f686800c8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.780+0000 7f687de00640 1 -- 172.21.15.50:0/465881933 shutdown_connections 2024-09-21T06:10:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.780+0000 7f687de00640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6878159130 0x7f6878156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.780+0000 7f687de00640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f687806fc90 0x7f68781fa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.780+0000 7f687de00640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f68540750f0 0x7f6854077600 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.780+0000 7f687de00640 1 --2- 172.21.15.50:0/465881933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687806cdf0 0x7f687815cd30 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.780+0000 7f687de00640 1 -- 172.21.15.50:0/465881933 >> 172.21.15.50:0/465881933 conn(0x7f68780b35f0 msgr2=0x7f687815bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.780+0000 7f687de00640 1 -- 172.21.15.50:0/465881933 shutdown_connections 2024-09-21T06:10:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:10:59.780+0000 7f687de00640 1 -- 172.21.15.50:0/465881933 wait complete. 2024-09-21T06:10:59.798 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:10:59.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:10:59.798 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:10:59.843 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:10:59.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:10:59.844 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:10:59.879 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:10:59.879 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:10:59.879 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:10:59.891 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:10:59.891 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:10:59.892 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:10:59.892 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:10:59.933 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:10:59.934 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:10:59.934 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:10:59.934 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:10:59.945 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:10:59.945 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:11:00.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.136+0000 7eff1f400640 1 -- 172.21.15.50:0/1988881554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff18160d50 msgr2=0x7eff18074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.136+0000 7eff1f400640 1 --2- 172.21.15.50:0/1988881554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff18160d50 0x7eff18074410 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7eff18160830 tx=0x7eff0001c8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.136+0000 7eff1f400640 1 -- 172.21.15.50:0/1988881554 shutdown_connections 2024-09-21T06:11:00.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.136+0000 7eff1f400640 1 --2- 172.21.15.50:0/1988881554 >> v2:172.21.15.88:3300/0 conn(0x7eff18161160 0x7eff18074950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.136+0000 7eff1f400640 1 --2- 172.21.15.50:0/1988881554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff18160d50 0x7eff18074410 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.136+0000 7eff1f400640 1 --2- 172.21.15.50:0/1988881554 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7eff18157030 0x7eff180b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.136+0000 7eff1f400640 1 -- 172.21.15.50:0/1988881554 >> 172.21.15.50:0/1988881554 conn(0x7eff180708c0 msgr2=0x7eff18070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:00.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.136+0000 7eff1f400640 1 -- 172.21.15.50:0/1988881554 shutdown_connections 2024-09-21T06:11:00.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.136+0000 7eff1f400640 1 -- 172.21.15.50:0/1988881554 wait complete. 2024-09-21T06:11:00.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1f400640 1 Processor -- start 2024-09-21T06:11:00.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1f400640 1 -- start start 2024-09-21T06:11:00.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7eff18157030 0x7eff1814cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff18160d50 0x7eff181f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff18161160 0x7eff1815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1f400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff18072290 con 0x7eff18160d50 2024-09-21T06:11:00.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1f400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7eff18072110 con 0x7eff18161160 2024-09-21T06:11:00.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1f400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7eff18072410 con 0x7eff18157030 2024-09-21T06:11:00.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff0fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff18160d50 0x7eff181f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff0fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff18160d50 0x7eff181f76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40606/0 (socket says 172.21.15.50:40606) 2024-09-21T06:11:00.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff0fe00640 1 -- 172.21.15.50:0/1864012886 learned_addr learned my addr 172.21.15.50:0/1864012886 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1ca00640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7eff18157030 0x7eff1814cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1d400640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff18161160 0x7eff1815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff0fe00640 1 -- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7eff18157030 msgr2=0x7eff1814cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff0fe00640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7eff18157030 0x7eff1814cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff0fe00640 1 -- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff18161160 msgr2=0x7eff1815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff0fe00640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff18161160 0x7eff1815b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff0fe00640 1 -- 172.21.15.50:0/1864012886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1815a890 con 0x7eff18160d50 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1d400640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff18161160 0x7eff1815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff0fe00640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff18160d50 0x7eff181f76e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7eff000b0040 tx=0x7eff00005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff15600640 1 -- 172.21.15.50:0/1864012886 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff000b2040 con 0x7eff18160d50 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff15600640 1 -- 172.21.15.50:0/1864012886 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff00004240 con 0x7eff18160d50 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1f400640 1 -- 172.21.15.50:0/1864012886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1815c500 con 0x7eff18160d50 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff15600640 1 -- 172.21.15.50:0/1864012886 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00005030 con 0x7eff18160d50 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.140+0000 7eff1f400640 1 -- 172.21.15.50:0/1864012886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1815ab10 con 0x7eff18160d50 2024-09-21T06:11:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.144+0000 7eff1f400640 1 -- 172.21.15.50:0/1864012886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff1815ca20 con 0x7eff18160d50 2024-09-21T06:11:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.144+0000 7eff15600640 1 -- 172.21.15.50:0/1864012886 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7eff000051f0 con 0x7eff18160d50 2024-09-21T06:11:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.144+0000 7eff15600640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efef4075180 0x7efef4077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.144+0000 7eff15600640 1 -- 172.21.15.50:0/1864012886 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7eff0012ccb0 con 0x7eff18160d50 2024-09-21T06:11:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.144+0000 7eff1ca00640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efef4075180 0x7efef4077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.144+0000 7eff1ca00640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efef4075180 0x7efef4077690 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7eff08009ac0 tx=0x7eff080091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.148+0000 7eff15600640 1 -- 172.21.15.50:0/1864012886 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7eff1815a890 con 0x7eff18160d50 2024-09-21T06:11:00.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.364+0000 7eff1f400640 1 -- 172.21.15.50:0/1864012886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff180728c0 con 0x7eff18160d50 2024-09-21T06:11:00.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.364+0000 7eff15600640 1 -- 172.21.15.50:0/1864012886 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5225 (secure 0 0 0) 0x7eff000164b0 con 0x7eff18160d50 2024-09-21T06:11:00.361 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:00.361 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":75,"btime":"2024-09-21T06:10:00:602285+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4024495468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":4024495468},{"type":"v1","addr":"172.21.15.50:6835","nonce":4024495468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3620352888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3620352888},{"type":"v1","addr":"172.21.15.88:6835","nonce":3620352888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4286454792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4286454792},{"type":"v1","addr":"172.21.15.88:6837","nonce":4286454792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:09:46.946836+0000","modified":"2024-09-21T06:09:49.976058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6833/1093475697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1093475697},{"type":"v1","addr":"172.21.15.50:6833","nonce":1093475697}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8304,"qdb_cluster":[8304]},"id":8}]} 2024-09-21T06:11:00.362 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 75 2024-09-21T06:11:00.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.364+0000 7eff1f400640 1 -- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efef4075180 msgr2=0x7efef4077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.364+0000 7eff1f400640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efef4075180 0x7efef4077690 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7eff08009ac0 tx=0x7eff080091e0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.364+0000 7eff1f400640 1 -- 172.21.15.50:0/1864012886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff18160d50 msgr2=0x7eff181f76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.364+0000 7eff1f400640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff18160d50 0x7eff181f76e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7eff000b0040 tx=0x7eff00005de0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.364+0000 7eff1f400640 1 -- 172.21.15.50:0/1864012886 shutdown_connections 2024-09-21T06:11:00.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.364+0000 7eff1f400640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff18161160 0x7eff1815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.364+0000 7eff1f400640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7efef4075180 0x7efef4077690 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.364+0000 7eff1f400640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff18160d50 0x7eff181f76e0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.364+0000 7eff1f400640 1 --2- 172.21.15.50:0/1864012886 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7eff18157030 0x7eff1814cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.364+0000 7eff1f400640 1 -- 172.21.15.50:0/1864012886 >> 172.21.15.50:0/1864012886 conn(0x7eff180708c0 msgr2=0x7eff180d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:00.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.368+0000 7eff1f400640 1 -- 172.21.15.50:0/1864012886 shutdown_connections 2024-09-21T06:11:00.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.368+0000 7eff1f400640 1 -- 172.21.15.50:0/1864012886 wait complete. 2024-09-21T06:11:00.385 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:11:00.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.548+0000 7f7c54000640 1 -- 172.21.15.50:0/1192292891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c34004810 msgr2=0x7f7c34004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.548+0000 7f7c54000640 1 --2- 172.21.15.50:0/1192292891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c34004810 0x7f7c34004c40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009b60 tx=0x7f7c3801d270 comp rx=0 tx=0).stop 2024-09-21T06:11:00.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.548+0000 7f7c54000640 1 -- 172.21.15.50:0/1192292891 shutdown_connections 2024-09-21T06:11:00.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.548+0000 7f7c54000640 1 --2- 172.21.15.50:0/1192292891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c34004810 0x7f7c34004c40 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.548+0000 7f7c54000640 1 -- 172.21.15.50:0/1192292891 >> 172.21.15.50:0/1192292891 conn(0x7f7c4c0b2040 msgr2=0x7f7c4c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:00.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.548+0000 7f7c54000640 1 -- 172.21.15.50:0/1192292891 shutdown_connections 2024-09-21T06:11:00.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.548+0000 7f7c54000640 1 -- 172.21.15.50:0/1192292891 wait complete. 2024-09-21T06:11:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.548+0000 7f7c54000640 1 Processor -- start 2024-09-21T06:11:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.548+0000 7f7c54000640 1 -- start start 2024-09-21T06:11:00.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c54000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c34004810 0x7f7c4c14ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c54000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7c4c0af0b0 0x7f7c4c14e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c54000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c4c14ee90 0x7f7c4c1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c54000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c0751c0 con 0x7f7c34004810 2024-09-21T06:11:00.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c54000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c075040 con 0x7f7c4c14ee90 2024-09-21T06:11:00.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c54000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4c075340 con 0x7f7c4c0af0b0 2024-09-21T06:11:00.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c51600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c34004810 0x7f7c4c14ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c51600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c34004810 0x7f7c4c14ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40614/0 (socket says 172.21.15.50:40614) 2024-09-21T06:11:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c51600640 1 -- 172.21.15.50:0/1071270961 learned_addr learned my addr 172.21.15.50:0/1071270961 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c52000640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c4c14ee90 0x7f7c4c1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c50c00640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7c4c0af0b0 0x7f7c4c14e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c51600640 1 -- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7c4c0af0b0 msgr2=0x7f7c4c14e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c51600640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7c4c0af0b0 0x7f7c4c14e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c51600640 1 -- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c4c14ee90 msgr2=0x7f7c4c1531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c51600640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c4c14ee90 0x7f7c4c1531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c51600640 1 -- 172.21.15.50:0/1071270961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c14f2a0 con 0x7f7c34004810 2024-09-21T06:11:00.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c51600640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c34004810 0x7f7c4c14ad10 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7c40005e80 tx=0x7f7c40011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:00.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c49600640 1 -- 172.21.15.50:0/1071270961 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40011e50 con 0x7f7c34004810 2024-09-21T06:11:00.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c49600640 1 -- 172.21.15.50:0/1071270961 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c40018020 con 0x7f7c34004810 2024-09-21T06:11:00.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c54000640 1 -- 172.21.15.50:0/1071270961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4c1f96f0 con 0x7f7c34004810 2024-09-21T06:11:00.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c49600640 1 -- 172.21.15.50:0/1071270961 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40003600 con 0x7f7c34004810 2024-09-21T06:11:00.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c54000640 1 -- 172.21.15.50:0/1071270961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4c1f9cb0 con 0x7f7c34004810 2024-09-21T06:11:00.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.552+0000 7f7c54000640 1 -- 172.21.15.50:0/1071270961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c4c0b5500 con 0x7f7c34004810 2024-09-21T06:11:00.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.556+0000 7f7c49600640 1 -- 172.21.15.50:0/1071270961 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f7c400037a0 con 0x7f7c34004810 2024-09-21T06:11:00.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.556+0000 7f7c49600640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7c2c0751b0 0x7f7c2c0776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.556+0000 7f7c49600640 1 -- 172.21.15.50:0/1071270961 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f7c40096870 con 0x7f7c34004810 2024-09-21T06:11:00.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.556+0000 7f7c50c00640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7c2c0751b0 0x7f7c2c0776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.556+0000 7f7c50c00640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7c2c0751b0 0x7f7c2c0776c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7c380ae000 tx=0x7f7c3801a010 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:00.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.560+0000 7f7c49600640 1 -- 172.21.15.50:0/1071270961 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7c40022020 con 0x7f7c34004810 2024-09-21T06:11:00.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.948+0000 7f7c54000640 1 -- 172.21.15.50:0/1071270961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c4c06ec00 con 0x7f7c34004810 2024-09-21T06:11:00.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.948+0000 7f7c49600640 1 -- 172.21.15.50:0/1071270961 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5225 (secure 0 0 0) 0x7f7c4c14f2a0 con 0x7f7c34004810 2024-09-21T06:11:00.945 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:00.945 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":75,"btime":"2024-09-21T06:10:00:602285+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4024495468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":4024495468},{"type":"v1","addr":"172.21.15.50:6835","nonce":4024495468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3620352888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3620352888},{"type":"v1","addr":"172.21.15.88:6835","nonce":3620352888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4286454792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4286454792},{"type":"v1","addr":"172.21.15.88:6837","nonce":4286454792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:09:46.946836+0000","modified":"2024-09-21T06:09:49.976058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6833/1093475697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1093475697},{"type":"v1","addr":"172.21.15.50:6833","nonce":1093475697}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8304,"qdb_cluster":[8304]},"id":8}]} 2024-09-21T06:11:00.945 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 75 2024-09-21T06:11:00.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.952+0000 7f7c54000640 1 -- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7c2c0751b0 msgr2=0x7f7c2c0776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.952+0000 7f7c54000640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7c2c0751b0 0x7f7c2c0776c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7c380ae000 tx=0x7f7c3801a010 comp rx=0 tx=0).stop 2024-09-21T06:11:00.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.952+0000 7f7c54000640 1 -- 172.21.15.50:0/1071270961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c34004810 msgr2=0x7f7c4c14ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.952+0000 7f7c54000640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c34004810 0x7f7c4c14ad10 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7c40005e80 tx=0x7f7c40011740 comp rx=0 tx=0).stop 2024-09-21T06:11:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.952+0000 7f7c54000640 1 -- 172.21.15.50:0/1071270961 shutdown_connections 2024-09-21T06:11:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.952+0000 7f7c54000640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c4c14ee90 0x7f7c4c1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.952+0000 7f7c54000640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7c4c0af0b0 0x7f7c4c14e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.952+0000 7f7c54000640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7c2c0751b0 0x7f7c2c0776c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.952+0000 7f7c54000640 1 --2- 172.21.15.50:0/1071270961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c34004810 0x7f7c4c14ad10 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.952+0000 7f7c54000640 1 -- 172.21.15.50:0/1071270961 >> 172.21.15.50:0/1071270961 conn(0x7f7c4c0b2040 msgr2=0x7f7c4c14feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:00.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.952+0000 7f7c54000640 1 -- 172.21.15.50:0/1071270961 shutdown_connections 2024-09-21T06:11:00.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:00.952+0000 7f7c54000640 1 -- 172.21.15.50:0/1071270961 wait complete. 2024-09-21T06:11:00.986 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:11:01.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.180+0000 7f083e000640 1 -- 172.21.15.50:0/3056945891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380b5c70 msgr2=0x7f083815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.180+0000 7f083e000640 1 --2- 172.21.15.50:0/3056945891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380b5c70 0x7f083815bb60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0828009840 tx=0x7f082801c7f0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.180+0000 7f083e000640 1 -- 172.21.15.50:0/3056945891 shutdown_connections 2024-09-21T06:11:01.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.180+0000 7f083e000640 1 --2- 172.21.15.50:0/3056945891 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380b4de0 0x7f08380b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.180+0000 7f083e000640 1 --2- 172.21.15.50:0/3056945891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380b5c70 0x7f083815bb60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.180+0000 7f083e000640 1 -- 172.21.15.50:0/3056945891 >> 172.21.15.50:0/3056945891 conn(0x7f08380b2040 msgr2=0x7f08380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:01.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.180+0000 7f083e000640 1 -- 172.21.15.50:0/3056945891 shutdown_connections 2024-09-21T06:11:01.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.180+0000 7f083e000640 1 -- 172.21.15.50:0/3056945891 wait complete. 2024-09-21T06:11:01.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f083e000640 1 Processor -- start 2024-09-21T06:11:01.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f083e000640 1 -- start start 2024-09-21T06:11:01.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f083e000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380b4de0 0x7f08380af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:01.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f083e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380b5c70 0x7f0838159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:01.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f083e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08381527b0 0x7f0838159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:01.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f083e000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0838168970 con 0x7f08381527b0 2024-09-21T06:11:01.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f083e000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f08381687f0 con 0x7f08380b5c70 2024-09-21T06:11:01.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f083e000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0838168af0 con 0x7f08380b4de0 2024-09-21T06:11:01.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f0837e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08381527b0 0x7f0838159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:01.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f082ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380b5c70 0x7f0838159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:01.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f0837400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380b4de0 0x7f08380af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:01.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f0837400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380b4de0 0x7f08380af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:53438/0 (socket says 172.21.15.50:53438) 2024-09-21T06:11:01.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f0837e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08381527b0 0x7f0838159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40622/0 (socket says 172.21.15.50:40622) 2024-09-21T06:11:01.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f0837400640 1 -- 172.21.15.50:0/4036016497 learned_addr learned my addr 172.21.15.50:0/4036016497 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:01.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f082ea00640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380b5c70 0x7f0838159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:38556/0 (socket says 172.21.15.50:38556) 2024-09-21T06:11:01.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f0837400640 1 -- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380b5c70 msgr2=0x7f0838159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f0837400640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380b5c70 0x7f0838159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f0837400640 1 -- 172.21.15.50:0/4036016497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08381527b0 msgr2=0x7f0838159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f0837400640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08381527b0 0x7f0838159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f0837400640 1 -- 172.21.15.50:0/4036016497 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08381f7960 con 0x7f08380b4de0 2024-09-21T06:11:01.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.184+0000 7f0837400640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380b4de0 0x7f08380af160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0828000c10 tx=0x7f0828005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:01.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.188+0000 7f082fe00640 1 -- 172.21.15.50:0/4036016497 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08280b2040 con 0x7f08380b4de0 2024-09-21T06:11:01.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.188+0000 7f082fe00640 1 -- 172.21.15.50:0/4036016497 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0828004330 con 0x7f08380b4de0 2024-09-21T06:11:01.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.188+0000 7f082fe00640 1 -- 172.21.15.50:0/4036016497 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828003820 con 0x7f08380b4de0 2024-09-21T06:11:01.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.188+0000 7f083e000640 1 -- 172.21.15.50:0/4036016497 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08381f7c00 con 0x7f08380b4de0 2024-09-21T06:11:01.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.188+0000 7f083e000640 1 -- 172.21.15.50:0/4036016497 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f08381f81c0 con 0x7f08380b4de0 2024-09-21T06:11:01.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.188+0000 7f083e000640 1 -- 172.21.15.50:0/4036016497 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08380b51f0 con 0x7f08380b4de0 2024-09-21T06:11:01.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.188+0000 7f082fe00640 1 -- 172.21.15.50:0/4036016497 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f08280bd060 con 0x7f08380b4de0 2024-09-21T06:11:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.188+0000 7f082fe00640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f08140751b0 0x7f08140776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.188+0000 7f082fe00640 1 -- 172.21.15.50:0/4036016497 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f082812c610 con 0x7f08380b4de0 2024-09-21T06:11:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.188+0000 7f082ea00640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f08140751b0 0x7f08140776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.192+0000 7f082ea00640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f08140751b0 0x7f08140776c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f083815b0a0 tx=0x7f0820010040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:01.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.196+0000 7f082fe00640 1 -- 172.21.15.50:0/4036016497 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0828016410 con 0x7f08380b4de0 2024-09-21T06:11:01.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.372+0000 7f083e000640 1 -- 172.21.15.50:0/4036016497 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08381660e0 con 0x7f08380b4de0 2024-09-21T06:11:01.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f082fe00640 1 -- 172.21.15.50:0/4036016497 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16976 (secure 0 0 0) 0x7f08280f9090 con 0x7f08380b4de0 2024-09-21T06:11:01.373 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:01.373 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":115,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:10:53.826836+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:44.704087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:45.912566+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1","mappings":[{"from":0,"to":5}]},{"pgid":"17.7","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:11:01.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f083e000640 1 -- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f08140751b0 msgr2=0x7f08140776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f083e000640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f08140751b0 0x7f08140776c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f083815b0a0 tx=0x7f0820010040 comp rx=0 tx=0).stop 2024-09-21T06:11:01.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f083e000640 1 -- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380b4de0 msgr2=0x7f08380af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f083e000640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380b4de0 0x7f08380af160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0828000c10 tx=0x7f0828005db0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f083e000640 1 -- 172.21.15.50:0/4036016497 shutdown_connections 2024-09-21T06:11:01.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f083e000640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f08140751b0 0x7f08140776c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f083e000640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08381527b0 0x7f0838159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f083e000640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08380b5c70 0x7f0838159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f083e000640 1 --2- 172.21.15.50:0/4036016497 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f08380b4de0 0x7f08380af160 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f083e000640 1 -- 172.21.15.50:0/4036016497 >> 172.21.15.50:0/4036016497 conn(0x7f08380b2040 msgr2=0x7f083815b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:01.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f083e000640 1 -- 172.21.15.50:0/4036016497 shutdown_connections 2024-09-21T06:11:01.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.376+0000 7f083e000640 1 -- 172.21.15.50:0/4036016497 wait complete. 2024-09-21T06:11:01.396 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:11:01.396 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:11:01.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.556+0000 7fe527e00640 1 -- 172.21.15.50:0/394555000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5200b5a80 msgr2=0x7fe520074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.556+0000 7fe527e00640 1 --2- 172.21.15.50:0/394555000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5200b5a80 0x7fe520074410 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fe5200b6a50 tx=0x7fe50c01c770 comp rx=0 tx=0).stop 2024-09-21T06:11:01.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.556+0000 7fe527e00640 1 -- 172.21.15.50:0/394555000 shutdown_connections 2024-09-21T06:11:01.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.556+0000 7fe527e00640 1 --2- 172.21.15.50:0/394555000 >> v2:172.21.15.88:3300/0 conn(0x7fe520074950 0x7fe52015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.556+0000 7fe527e00640 1 --2- 172.21.15.50:0/394555000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5200b5a80 0x7fe520074410 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.556+0000 7fe527e00640 1 -- 172.21.15.50:0/394555000 >> 172.21.15.50:0/394555000 conn(0x7fe5200708c0 msgr2=0x7fe520070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:01.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.556+0000 7fe527e00640 1 -- 172.21.15.50:0/394555000 shutdown_connections 2024-09-21T06:11:01.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.556+0000 7fe527e00640 1 -- 172.21.15.50:0/394555000 wait complete. 2024-09-21T06:11:01.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe527e00640 1 Processor -- start 2024-09-21T06:11:01.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe527e00640 1 -- start start 2024-09-21T06:11:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe527e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe520074950 0x7fe52013ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe527e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5200b5a80 0x7fe5200b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe527e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe520157030 0x7fe520167100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe527e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe52016a310 con 0x7fe520074950 2024-09-21T06:11:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe527e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe52016a190 con 0x7fe5200b5a80 2024-09-21T06:11:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe527e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe52016a490 con 0x7fe520157030 2024-09-21T06:11:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe525400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe520074950 0x7fe52013ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe525400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe520074950 0x7fe52013ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40626/0 (socket says 172.21.15.50:40626) 2024-09-21T06:11:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe525400640 1 -- 172.21.15.50:0/2835079618 learned_addr learned my addr 172.21.15.50:0/2835079618 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe524a00640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5200b5a80 0x7fe5200b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.560+0000 7fe525e00640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe520157030 0x7fe520167100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe525400640 1 -- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe520157030 msgr2=0x7fe520167100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe525400640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe520157030 0x7fe520167100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe525400640 1 -- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5200b5a80 msgr2=0x7fe5200b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe525400640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5200b5a80 0x7fe5200b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe525400640 1 -- 172.21.15.50:0/2835079618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5200b4790 con 0x7fe520074950 2024-09-21T06:11:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe525400640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe520074950 0x7fe52013ca20 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c430 tx=0x7fe51400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:01.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe51d600640 1 -- 172.21.15.50:0/2835079618 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514011790 con 0x7fe520074950 2024-09-21T06:11:01.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe51d600640 1 -- 172.21.15.50:0/2835079618 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe514017070 con 0x7fe520074950 2024-09-21T06:11:01.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe527e00640 1 -- 172.21.15.50:0/2835079618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5200b4a70 con 0x7fe520074950 2024-09-21T06:11:01.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe51d600640 1 -- 172.21.15.50:0/2835079618 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514011cc0 con 0x7fe520074950 2024-09-21T06:11:01.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe527e00640 1 -- 172.21.15.50:0/2835079618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52013f4d0 con 0x7fe520074950 2024-09-21T06:11:01.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe527e00640 1 -- 172.21.15.50:0/2835079618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe52013cfb0 con 0x7fe520074950 2024-09-21T06:11:01.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.564+0000 7fe51d600640 1 -- 172.21.15.50:0/2835079618 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7fe51401c460 con 0x7fe520074950 2024-09-21T06:11:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.568+0000 7fe51d600640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe5000751b0 0x7fe5000776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.568+0000 7fe51d600640 1 -- 172.21.15.50:0/2835079618 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7fe514092e90 con 0x7fe520074950 2024-09-21T06:11:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.568+0000 7fe524a00640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe5000751b0 0x7fe5000776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.568+0000 7fe524a00640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe5000751b0 0x7fe5000776c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0af000 tx=0x7fe50c01afb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.572+0000 7fe51d600640 1 -- 172.21.15.50:0/2835079618 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe520074d60 con 0x7fe520074950 2024-09-21T06:11:01.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 -- 172.21.15.50:0/2835079618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe52015cf90 con 0x7fe520074950 2024-09-21T06:11:01.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe51d600640 1 -- 172.21.15.50:0/2835079618 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5225 (secure 0 0 0) 0x7fe51405f360 con 0x7fe520074950 2024-09-21T06:11:01.782 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:01.782 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":75,"btime":"2024-09-21T06:10:00:602285+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4024495468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":4024495468},{"type":"v1","addr":"172.21.15.50:6835","nonce":4024495468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3620352888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3620352888},{"type":"v1","addr":"172.21.15.88:6835","nonce":3620352888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4286454792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4286454792},{"type":"v1","addr":"172.21.15.88:6837","nonce":4286454792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:09:46.946836+0000","modified":"2024-09-21T06:09:49.976058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6833/1093475697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1093475697},{"type":"v1","addr":"172.21.15.50:6833","nonce":1093475697}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8304,"qdb_cluster":[8304]},"id":8}]} 2024-09-21T06:11:01.783 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 75 2024-09-21T06:11:01.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 -- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe5000751b0 msgr2=0x7fe5000776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe5000751b0 0x7fe5000776c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0af000 tx=0x7fe50c01afb0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 -- 172.21.15.50:0/2835079618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe520074950 msgr2=0x7fe52013ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe520074950 0x7fe52013ca20 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c430 tx=0x7fe51400c8f0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 -- 172.21.15.50:0/2835079618 shutdown_connections 2024-09-21T06:11:01.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe520157030 0x7fe520167100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5200b5a80 0x7fe5200b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe5000751b0 0x7fe5000776c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 --2- 172.21.15.50:0/2835079618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe520074950 0x7fe52013ca20 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 -- 172.21.15.50:0/2835079618 >> 172.21.15.50:0/2835079618 conn(0x7fe5200708c0 msgr2=0x7fe520070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:01.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 -- 172.21.15.50:0/2835079618 shutdown_connections 2024-09-21T06:11:01.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.784+0000 7fe527e00640 1 -- 172.21.15.50:0/2835079618 wait complete. 2024-09-21T06:11:01.804 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:11:01.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.968+0000 7f4c42400640 1 -- 172.21.15.50:0/4148275733 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c3c157030 msgr2=0x7f4c3c0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.968+0000 7f4c42400640 1 --2- 172.21.15.50:0/4148275733 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c3c157030 0x7f4c3c0b5a80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009900 tx=0x7f4c2c01c8e0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.968+0000 7f4c42400640 1 -- 172.21.15.50:0/4148275733 shutdown_connections 2024-09-21T06:11:01.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.968+0000 7f4c42400640 1 --2- 172.21.15.50:0/4148275733 >> v2:172.21.15.88:3300/0 conn(0x7f4c3c160d50 0x7f4c3c074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.968+0000 7f4c42400640 1 --2- 172.21.15.50:0/4148275733 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c3c157030 0x7f4c3c0b5a80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.968+0000 7f4c42400640 1 -- 172.21.15.50:0/4148275733 >> 172.21.15.50:0/4148275733 conn(0x7f4c3c0708c0 msgr2=0x7f4c3c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:01.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.968+0000 7f4c42400640 1 -- 172.21.15.50:0/4148275733 shutdown_connections 2024-09-21T06:11:01.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.968+0000 7f4c42400640 1 -- 172.21.15.50:0/4148275733 wait complete. 2024-09-21T06:11:01.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.968+0000 7f4c42400640 1 Processor -- start 2024-09-21T06:11:01.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.968+0000 7f4c42400640 1 -- start start 2024-09-21T06:11:01.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c42400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c3c157030 0x7f4c3c14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:01.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c42400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c3c160d50 0x7f4c3c14ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:01.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c42400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c3c161160 0x7f4c3c1fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:01.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c42400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c16a820 con 0x7f4c3c161160 2024-09-21T06:11:01.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c42400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c16a6a0 con 0x7f4c3c157030 2024-09-21T06:11:01.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c42400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3c16a9a0 con 0x7f4c3c160d50 2024-09-21T06:11:01.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c3be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c3c161160 0x7f4c3c1fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:01.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c3b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c3c157030 0x7f4c3c14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:01.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c33e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c3c160d50 0x7f4c3c14ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:01.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c3b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c3c157030 0x7f4c3c14f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:38594/0 (socket says 172.21.15.50:38594) 2024-09-21T06:11:01.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c3be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c3c161160 0x7f4c3c1fcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40634/0 (socket says 172.21.15.50:40634) 2024-09-21T06:11:01.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c3b400640 1 -- 172.21.15.50:0/3168921552 learned_addr learned my addr 172.21.15.50:0/3168921552 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:01.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c3b400640 1 -- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c3c160d50 msgr2=0x7f4c3c14ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c3b400640 1 --2- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c3c160d50 0x7f4c3c14ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c3b400640 1 -- 172.21.15.50:0/3168921552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c3c161160 msgr2=0x7f4c3c1fcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c3b400640 1 --2- 172.21.15.50:0/3168921552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c3c161160 0x7f4c3c1fcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c3b400640 1 -- 172.21.15.50:0/3168921552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c14e450 con 0x7f4c3c157030 2024-09-21T06:11:01.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c3b400640 1 --2- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c3c157030 0x7f4c3c14f4f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0b0040 tx=0x7f4c2c005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:01.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c33400640 1 -- 172.21.15.50:0/3168921552 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0b2040 con 0x7f4c3c157030 2024-09-21T06:11:01.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c33400640 1 -- 172.21.15.50:0/3168921552 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c2c004320 con 0x7f4c3c157030 2024-09-21T06:11:01.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c33400640 1 -- 172.21.15.50:0/3168921552 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c005290 con 0x7f4c3c157030 2024-09-21T06:11:01.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c42400640 1 -- 172.21.15.50:0/3168921552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3c14e730 con 0x7f4c3c157030 2024-09-21T06:11:01.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c42400640 1 -- 172.21.15.50:0/3168921552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3c14ecf0 con 0x7f4c3c157030 2024-09-21T06:11:01.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.972+0000 7f4c42400640 1 -- 172.21.15.50:0/3168921552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c3c0d2260 con 0x7f4c3c157030 2024-09-21T06:11:01.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.976+0000 7f4c33400640 1 -- 172.21.15.50:0/3168921552 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f4c2c005430 con 0x7f4c3c157030 2024-09-21T06:11:01.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.976+0000 7f4c33400640 1 --2- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c18075180 0x7f4c18077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:01.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.976+0000 7f4c33400640 1 -- 172.21.15.50:0/3168921552 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f4c2c12c700 con 0x7f4c3c157030 2024-09-21T06:11:01.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.976+0000 7f4c33e00640 1 --2- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c18075180 0x7f4c18077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:01.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.976+0000 7f4c33e00640 1 --2- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c18075180 0x7f4c18077690 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c160050 tx=0x7f4c240091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:01.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:01.980+0000 7f4c33400640 1 -- 172.21.15.50:0/3168921552 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4c2c0164d0 con 0x7f4c3c157030 2024-09-21T06:11:02.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 -- 172.21.15.50:0/3168921552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c3c152a50 con 0x7f4c3c157030 2024-09-21T06:11:02.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c33400640 1 -- 172.21.15.50:0/3168921552 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5225 (secure 0 0 0) 0x7f4c2c0f8cb0 con 0x7f4c3c157030 2024-09-21T06:11:02.192 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:02.192 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":75,"btime":"2024-09-21T06:10:00:602285+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4024495468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":4024495468},{"type":"v1","addr":"172.21.15.50:6835","nonce":4024495468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3620352888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3620352888},{"type":"v1","addr":"172.21.15.88:6835","nonce":3620352888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4286454792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4286454792},{"type":"v1","addr":"172.21.15.88:6837","nonce":4286454792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:09:46.946836+0000","modified":"2024-09-21T06:09:49.976058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6833/1093475697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1093475697},{"type":"v1","addr":"172.21.15.50:6833","nonce":1093475697}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8304,"qdb_cluster":[8304]},"id":8}]} 2024-09-21T06:11:02.194 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 75 2024-09-21T06:11:02.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 -- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c18075180 msgr2=0x7f4c18077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 --2- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c18075180 0x7f4c18077690 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c160050 tx=0x7f4c240091e0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 -- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c3c157030 msgr2=0x7f4c3c14f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 --2- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c3c157030 0x7f4c3c14f4f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0b0040 tx=0x7f4c2c005760 comp rx=0 tx=0).stop 2024-09-21T06:11:02.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 -- 172.21.15.50:0/3168921552 shutdown_connections 2024-09-21T06:11:02.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 --2- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c18075180 0x7f4c18077690 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 --2- 172.21.15.50:0/3168921552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c3c161160 0x7f4c3c1fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 --2- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c3c160d50 0x7f4c3c14ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 --2- 172.21.15.50:0/3168921552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c3c157030 0x7f4c3c14f4f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 -- 172.21.15.50:0/3168921552 >> 172.21.15.50:0/3168921552 conn(0x7f4c3c0708c0 msgr2=0x7f4c3c14c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:02.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 -- 172.21.15.50:0/3168921552 shutdown_connections 2024-09-21T06:11:02.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.192+0000 7f4c42400640 1 -- 172.21.15.50:0/3168921552 wait complete. 2024-09-21T06:11:02.211 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:11:02.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.372+0000 7f68e8e00640 1 -- 172.21.15.50:0/2168017392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68e41527b0 msgr2=0x7f68e4074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.372+0000 7f68e8e00640 1 --2- 172.21.15.50:0/2168017392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68e41527b0 0x7f68e4074e30 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009900 tx=0x7f68cc01c820 comp rx=0 tx=0).stop 2024-09-21T06:11:02.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.372+0000 7f68e8e00640 1 -- 172.21.15.50:0/2168017392 shutdown_connections 2024-09-21T06:11:02.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.372+0000 7f68e8e00640 1 --2- 172.21.15.50:0/2168017392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68e41527b0 0x7f68e4074e30 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.372+0000 7f68e8e00640 1 --2- 172.21.15.50:0/2168017392 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f68e40b5c70 0x7f68e415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.372+0000 7f68e8e00640 1 -- 172.21.15.50:0/2168017392 >> 172.21.15.50:0/2168017392 conn(0x7f68e40b2040 msgr2=0x7f68e40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:02.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.372+0000 7f68e8e00640 1 -- 172.21.15.50:0/2168017392 shutdown_connections 2024-09-21T06:11:02.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.372+0000 7f68e8e00640 1 -- 172.21.15.50:0/2168017392 wait complete. 2024-09-21T06:11:02.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e8e00640 1 Processor -- start 2024-09-21T06:11:02.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e8e00640 1 -- start start 2024-09-21T06:11:02.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68e40b4de0 0x7f68e414faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:02.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e40b5c70 0x7f68e41561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:02.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e8e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f68e41527b0 0x7f68e40b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:02.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e8e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4168bb0 con 0x7f68e40b4de0 2024-09-21T06:11:02.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e8e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4168a30 con 0x7f68e40b5c70 2024-09-21T06:11:02.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e8e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f68e4168d30 con 0x7f68e41527b0 2024-09-21T06:11:02.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68e40b4de0 0x7f68e414faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:02.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68e40b4de0 0x7f68e414faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40662/0 (socket says 172.21.15.50:40662) 2024-09-21T06:11:02.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e1e00640 1 -- 172.21.15.50:0/1864027149 learned_addr learned my addr 172.21.15.50:0/1864027149 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:02.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68d9400640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e40b5c70 0x7f68e41561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:02.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e2800640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f68e41527b0 0x7f68e40b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:02.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e1e00640 1 -- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f68e41527b0 msgr2=0x7f68e40b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e1e00640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f68e41527b0 0x7f68e40b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e1e00640 1 -- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e40b5c70 msgr2=0x7f68e41561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e1e00640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e40b5c70 0x7f68e41561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e1e00640 1 -- 172.21.15.50:0/1864027149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e4156d80 con 0x7f68e40b4de0 2024-09-21T06:11:02.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e1e00640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68e40b4de0 0x7f68e414faa0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f68d400d720 tx=0x7f68d400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:02.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68daa00640 1 -- 172.21.15.50:0/1864027149 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4017070 con 0x7f68e40b4de0 2024-09-21T06:11:02.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68daa00640 1 -- 172.21.15.50:0/1864027149 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68d4009d20 con 0x7f68e40b4de0 2024-09-21T06:11:02.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e8e00640 1 -- 172.21.15.50:0/1864027149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e41f8cd0 con 0x7f68e40b4de0 2024-09-21T06:11:02.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68daa00640 1 -- 172.21.15.50:0/1864027149 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4005280 con 0x7f68e40b4de0 2024-09-21T06:11:02.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.376+0000 7f68e8e00640 1 -- 172.21.15.50:0/1864027149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e41f9290 con 0x7f68e40b4de0 2024-09-21T06:11:02.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.380+0000 7f68e8e00640 1 -- 172.21.15.50:0/1864027149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e415bb60 con 0x7f68e40b4de0 2024-09-21T06:11:02.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.380+0000 7f68daa00640 1 -- 172.21.15.50:0/1864027149 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f68d4005420 con 0x7f68e40b4de0 2024-09-21T06:11:02.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.380+0000 7f68daa00640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f68c0075180 0x7f68c0077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:02.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.380+0000 7f68daa00640 1 -- 172.21.15.50:0/1864027149 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f68d40932d0 con 0x7f68e40b4de0 2024-09-21T06:11:02.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.380+0000 7f68d9400640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f68c0075180 0x7f68c0077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:02.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.384+0000 7f68d9400640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f68c0075180 0x7f68c0077690 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0af000 tx=0x7f68cc01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:02.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.384+0000 7f68daa00640 1 -- 172.21.15.50:0/1864027149 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f68d405f790 con 0x7f68e40b4de0 2024-09-21T06:11:02.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.568+0000 7f68e8e00640 1 -- 172.21.15.50:0/1864027149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68e4153d40 con 0x7f68e40b4de0 2024-09-21T06:11:02.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68daa00640 1 -- 172.21.15.50:0/1864027149 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16976 (secure 0 0 0) 0x7f68d40641c0 con 0x7f68e40b4de0 2024-09-21T06:11:02.568 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:02.568 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":115,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:10:53.826836+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:44.704087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:45.912566+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1","mappings":[{"from":0,"to":5}]},{"pgid":"17.7","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:11:02.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68e8e00640 1 -- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f68c0075180 msgr2=0x7f68c0077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68e8e00640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f68c0075180 0x7f68c0077690 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0af000 tx=0x7f68cc01b060 comp rx=0 tx=0).stop 2024-09-21T06:11:02.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68e8e00640 1 -- 172.21.15.50:0/1864027149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68e40b4de0 msgr2=0x7f68e414faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68e8e00640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68e40b4de0 0x7f68e414faa0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f68d400d720 tx=0x7f68d400dbe0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68e8e00640 1 -- 172.21.15.50:0/1864027149 shutdown_connections 2024-09-21T06:11:02.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68e8e00640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f68e41527b0 0x7f68e40b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68e8e00640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e40b5c70 0x7f68e41561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68e8e00640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f68c0075180 0x7f68c0077690 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68e8e00640 1 --2- 172.21.15.50:0/1864027149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68e40b4de0 0x7f68e414faa0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68e8e00640 1 -- 172.21.15.50:0/1864027149 >> 172.21.15.50:0/1864027149 conn(0x7f68e40b2040 msgr2=0x7f68e40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:02.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68e8e00640 1 -- 172.21.15.50:0/1864027149 shutdown_connections 2024-09-21T06:11:02.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.572+0000 7f68e8e00640 1 -- 172.21.15.50:0/1864027149 wait complete. 2024-09-21T06:11:02.591 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:11:02.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.756+0000 7f0e1a800640 1 -- 172.21.15.50:0/2185121310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e140749a0 msgr2=0x7f0e14074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.756+0000 7f0e1a800640 1 --2- 172.21.15.50:0/2185121310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e140749a0 0x7f0e14074db0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f0e140cfc90 tx=0x7f0e0401c8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.756+0000 7f0e1a800640 1 -- 172.21.15.50:0/2185121310 shutdown_connections 2024-09-21T06:11:02.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.756+0000 7f0e1a800640 1 --2- 172.21.15.50:0/2185121310 >> v2:172.21.15.88:3300/0 conn(0x7f0e140b3340 0x7f0e140b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.756+0000 7f0e1a800640 1 --2- 172.21.15.50:0/2185121310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e140749a0 0x7f0e14074db0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.756+0000 7f0e1a800640 1 -- 172.21.15.50:0/2185121310 >> 172.21.15.50:0/2185121310 conn(0x7f0e140b2040 msgr2=0x7f0e140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:02.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.756+0000 7f0e1a800640 1 -- 172.21.15.50:0/2185121310 shutdown_connections 2024-09-21T06:11:02.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.756+0000 7f0e1a800640 1 -- 172.21.15.50:0/2185121310 wait complete. 2024-09-21T06:11:02.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.756+0000 7f0e1a800640 1 Processor -- start 2024-09-21T06:11:02.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.756+0000 7f0e1a800640 1 -- start start 2024-09-21T06:11:02.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e1a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e140749a0 0x7f0e1413b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:02.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e1a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e140b3340 0x7f0e1415f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:02.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e1a800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0e140cefa0 0x7f0e1413d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:02.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e1a800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14169d20 con 0x7f0e140749a0 2024-09-21T06:11:02.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e1a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14169ba0 con 0x7f0e140b3340 2024-09-21T06:11:02.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e1a800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0e14169ea0 con 0x7f0e140cefa0 2024-09-21T06:11:02.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e13400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e140749a0 0x7f0e1413b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:02.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e13400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e140749a0 0x7f0e1413b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40676/0 (socket says 172.21.15.50:40676) 2024-09-21T06:11:02.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e13400640 1 -- 172.21.15.50:0/2456084036 learned_addr learned my addr 172.21.15.50:0/2456084036 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:02.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e0aa00640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e140b3340 0x7f0e1415f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:02.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e13e00640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0e140cefa0 0x7f0e1413d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:02.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e13400640 1 -- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0e140cefa0 msgr2=0x7f0e1413d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e13400640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0e140cefa0 0x7f0e1413d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e13400640 1 -- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e140b3340 msgr2=0x7f0e1415f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e13400640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e140b3340 0x7f0e1415f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e13400640 1 -- 172.21.15.50:0/2456084036 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1413be40 con 0x7f0e140749a0 2024-09-21T06:11:02.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e0aa00640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e140b3340 0x7f0e1415f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:02.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e13400640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e140749a0 0x7f0e1413b900 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f0e0401af70 tx=0x7f0e040a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:02.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e0be00640 1 -- 172.21.15.50:0/2456084036 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e040bb070 con 0x7f0e140749a0 2024-09-21T06:11:02.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e0be00640 1 -- 172.21.15.50:0/2456084036 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e04004430 con 0x7f0e140749a0 2024-09-21T06:11:02.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e1a800640 1 -- 172.21.15.50:0/2456084036 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e141f8d70 con 0x7f0e140749a0 2024-09-21T06:11:02.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e0be00640 1 -- 172.21.15.50:0/2456084036 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e040b2040 con 0x7f0e140749a0 2024-09-21T06:11:02.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e1a800640 1 -- 172.21.15.50:0/2456084036 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e141f9330 con 0x7f0e140749a0 2024-09-21T06:11:02.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.760+0000 7f0e1a800640 1 -- 172.21.15.50:0/2456084036 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e14074db0 con 0x7f0e140749a0 2024-09-21T06:11:02.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.764+0000 7f0e0be00640 1 -- 172.21.15.50:0/2456084036 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f0e040be070 con 0x7f0e140749a0 2024-09-21T06:11:02.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.764+0000 7f0e0be00640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0df0075180 0x7f0df0077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:02.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.764+0000 7f0e0be00640 1 -- 172.21.15.50:0/2456084036 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f0e0412ca60 con 0x7f0e140749a0 2024-09-21T06:11:02.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.764+0000 7f0e0aa00640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0df0075180 0x7f0df0077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:02.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.764+0000 7f0e0aa00640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0df0075180 0x7f0df0077690 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0e1406e180 tx=0x7f0dfc009570 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:02.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.768+0000 7f0e0be00640 1 -- 172.21.15.50:0/2456084036 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0e040164b0 con 0x7f0e140749a0 2024-09-21T06:11:02.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.980+0000 7f0e1a800640 1 -- 172.21.15.50:0/2456084036 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0e14160cb0 con 0x7f0e140749a0 2024-09-21T06:11:02.977 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-09-21T06:11:02.980+0000 7f0d45000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:11:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.980+0000 7f0e0be00640 1 -- 172.21.15.50:0/2456084036 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) ==== 161+0+0 (secure 0 0 0) 0x7f0e040f94e0 con 0x7f0e140749a0 2024-09-21T06:11:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:11:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.984+0000 7f0e1a800640 1 -- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0df0075180 msgr2=0x7f0df0077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.984+0000 7f0e1a800640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0df0075180 0x7f0df0077690 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0e1406e180 tx=0x7f0dfc009570 comp rx=0 tx=0).stop 2024-09-21T06:11:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.984+0000 7f0e1a800640 1 -- 172.21.15.50:0/2456084036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e140749a0 msgr2=0x7f0e1413b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.984+0000 7f0e1a800640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e140749a0 0x7f0e1413b900 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f0e0401af70 tx=0x7f0e040a57b0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.984+0000 7f0e1a800640 1 -- 172.21.15.50:0/2456084036 shutdown_connections 2024-09-21T06:11:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.984+0000 7f0e1a800640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0e140cefa0 0x7f0e1413d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.984+0000 7f0e1a800640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e140b3340 0x7f0e1415f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.984+0000 7f0e1a800640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0df0075180 0x7f0df0077690 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.984+0000 7f0e1a800640 1 --2- 172.21.15.50:0/2456084036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e140749a0 0x7f0e1413b900 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.984+0000 7f0e1a800640 1 -- 172.21.15.50:0/2456084036 >> 172.21.15.50:0/2456084036 conn(0x7f0e140b2040 msgr2=0x7f0e140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.984+0000 7f0e1a800640 1 -- 172.21.15.50:0/2456084036 shutdown_connections 2024-09-21T06:11:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:02.984+0000 7f0e1a800640 1 -- 172.21.15.50:0/2456084036 wait complete. 2024-09-21T06:11:03.014 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:11:03.133 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-09-21T06:11:03.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.180+0000 7f3c4bc00640 1 -- 172.21.15.50:0/690322995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c440d0e40 msgr2=0x7f3c4415d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:03.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.180+0000 7f3c4bc00640 1 --2- 172.21.15.50:0/690322995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c440d0e40 0x7f3c4415d600 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009840 tx=0x7f3c3801c820 comp rx=0 tx=0).stop 2024-09-21T06:11:03.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.180+0000 7f3c4bc00640 1 -- 172.21.15.50:0/690322995 shutdown_connections 2024-09-21T06:11:03.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.180+0000 7f3c4bc00640 1 --2- 172.21.15.50:0/690322995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c440d0e40 0x7f3c4415d600 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:03.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.180+0000 7f3c4bc00640 1 -- 172.21.15.50:0/690322995 >> 172.21.15.50:0/690322995 conn(0x7f3c440b35f0 msgr2=0x7f3c440b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:03.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.180+0000 7f3c4bc00640 1 -- 172.21.15.50:0/690322995 shutdown_connections 2024-09-21T06:11:03.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.180+0000 7f3c4bc00640 1 -- 172.21.15.50:0/690322995 wait complete. 2024-09-21T06:11:03.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c4bc00640 1 Processor -- start 2024-09-21T06:11:03.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c4bc00640 1 -- start start 2024-09-21T06:11:03.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c4bc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c440d0e40 0x7f3c4414c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:03.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c4bc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c4414cbb0 0x7f3c44070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:03.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c4bc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c4414d100 0x7f3c44070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:03.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c4bc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4416b480 con 0x7f3c4414d100 2024-09-21T06:11:03.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c4bc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4416b300 con 0x7f3c4414cbb0 2024-09-21T06:11:03.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c4bc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3c4416b600 con 0x7f3c440d0e40 2024-09-21T06:11:03.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c49c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c4414d100 0x7f3c44070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:03.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c49c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c4414d100 0x7f3c44070e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40698/0 (socket says 172.21.15.50:40698) 2024-09-21T06:11:03.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c49c00640 1 -- 172.21.15.50:0/644266549 learned_addr learned my addr 172.21.15.50:0/644266549 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:03.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c43e00640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c4414cbb0 0x7f3c44070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:03.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c49200640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c440d0e40 0x7f3c4414c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:03.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c49c00640 1 -- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c440d0e40 msgr2=0x7f3c4414c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:03.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c49c00640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c440d0e40 0x7f3c4414c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:03.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c49c00640 1 -- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c4414cbb0 msgr2=0x7f3c44070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:03.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c49c00640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c4414cbb0 0x7f3c44070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:03.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c49c00640 1 -- 172.21.15.50:0/644266549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c44154db0 con 0x7f3c4414d100 2024-09-21T06:11:03.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c49c00640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c4414d100 0x7f3c44070e90 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00ca80 tx=0x7f3c3c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:03.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c40c00640 1 -- 172.21.15.50:0/644266549 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c016020 con 0x7f3c4414d100 2024-09-21T06:11:03.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c40c00640 1 -- 172.21.15.50:0/644266549 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c3c00a930 con 0x7f3c4414d100 2024-09-21T06:11:03.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.184+0000 7f3c4bc00640 1 -- 172.21.15.50:0/644266549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4414dda0 con 0x7f3c4414d100 2024-09-21T06:11:03.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.188+0000 7f3c40c00640 1 -- 172.21.15.50:0/644266549 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c00ad80 con 0x7f3c4414d100 2024-09-21T06:11:03.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.188+0000 7f3c4bc00640 1 -- 172.21.15.50:0/644266549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c44150700 con 0x7f3c4414d100 2024-09-21T06:11:03.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.188+0000 7f3c4bc00640 1 -- 172.21.15.50:0/644266549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c44155290 con 0x7f3c4414d100 2024-09-21T06:11:03.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.188+0000 7f3c40c00640 1 -- 172.21.15.50:0/644266549 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f3c3c019020 con 0x7f3c4414d100 2024-09-21T06:11:03.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.188+0000 7f3c40c00640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3c24075180 0x7f3c24077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:03.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.192+0000 7f3c40c00640 1 -- 172.21.15.50:0/644266549 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5965+0+0 (secure 0 0 0) 0x7f3c3c0932d0 con 0x7f3c4414d100 2024-09-21T06:11:03.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.192+0000 7f3c49200640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3c24075180 0x7f3c24077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:03.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.192+0000 7f3c49200640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3c24075180 0x7f3c24077690 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3c380af000 tx=0x7f3c3801b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:03.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.196+0000 7f3c40c00640 1 -- 172.21.15.50:0/644266549 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3c3c020020 con 0x7f3c4414d100 2024-09-21T06:11:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.408+0000 7f3c4bc00640 1 -- 172.21.15.50:0/644266549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3c440d1250 con 0x7f3c4414d100 2024-09-21T06:11:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.992+0000 7f3c40c00640 1 -- 172.21.15.50:0/644266549 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) ==== 104+0+0 (secure 0 0 0) 0x7f3c3c05f810 con 0x7f3c4414d100 2024-09-21T06:11:03.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.992+0000 7f3c4bc00640 1 -- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3c24075180 msgr2=0x7f3c24077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:03.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.992+0000 7f3c4bc00640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3c24075180 0x7f3c24077690 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3c380af000 tx=0x7f3c3801b060 comp rx=0 tx=0).stop 2024-09-21T06:11:03.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.992+0000 7f3c4bc00640 1 -- 172.21.15.50:0/644266549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c4414d100 msgr2=0x7f3c44070e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:03.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.992+0000 7f3c4bc00640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c4414d100 0x7f3c44070e90 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00ca80 tx=0x7f3c3c00cb70 comp rx=0 tx=0).stop 2024-09-21T06:11:03.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.992+0000 7f3c4bc00640 1 -- 172.21.15.50:0/644266549 shutdown_connections 2024-09-21T06:11:03.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.992+0000 7f3c4bc00640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3c24075180 0x7f3c24077690 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:03.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.992+0000 7f3c4bc00640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c4414d100 0x7f3c44070e90 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:03.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.992+0000 7f3c4bc00640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c4414cbb0 0x7f3c44070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:03.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.992+0000 7f3c4bc00640 1 --2- 172.21.15.50:0/644266549 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3c440d0e40 0x7f3c4414c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:03.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.992+0000 7f3c4bc00640 1 -- 172.21.15.50:0/644266549 >> 172.21.15.50:0/644266549 conn(0x7f3c440b35f0 msgr2=0x7f3c4406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:03.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.996+0000 7f3c4bc00640 1 -- 172.21.15.50:0/644266549 shutdown_connections 2024-09-21T06:11:03.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:03.996+0000 7f3c4bc00640 1 -- 172.21.15.50:0/644266549 wait complete. 2024-09-21T06:11:04.014 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:11:04.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.180+0000 7fc15c200640 1 -- 172.21.15.50:0/2897819156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1540b3340 msgr2=0x7fc1540b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:04.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.180+0000 7fc15c200640 1 --2- 172.21.15.50:0/2897819156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1540b3340 0x7fc1540b3750 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009c80 tx=0x7fc14c01cda0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.180+0000 7fc15c200640 1 -- 172.21.15.50:0/2897819156 shutdown_connections 2024-09-21T06:11:04.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.180+0000 7fc15c200640 1 --2- 172.21.15.50:0/2897819156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1540b3340 0x7fc1540b3750 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.180+0000 7fc15c200640 1 --2- 172.21.15.50:0/2897819156 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1540749a0 0x7fc154074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.180+0000 7fc15c200640 1 -- 172.21.15.50:0/2897819156 >> 172.21.15.50:0/2897819156 conn(0x7fc1540b2040 msgr2=0x7fc1540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:04.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.180+0000 7fc15c200640 1 -- 172.21.15.50:0/2897819156 shutdown_connections 2024-09-21T06:11:04.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.180+0000 7fc15c200640 1 -- 172.21.15.50:0/2897819156 wait complete. 2024-09-21T06:11:04.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc15c200640 1 Processor -- start 2024-09-21T06:11:04.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc15c200640 1 -- start start 2024-09-21T06:11:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc15c200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1540749a0 0x7fc1541f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc15c200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1540b3340 0x7fc154140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc15c200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1540cefa0 0x7fc15413f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc15c200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc154169dc0 con 0x7fc1540b3340 2024-09-21T06:11:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc15c200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc154169c40 con 0x7fc1540749a0 2024-09-21T06:11:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc15c200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc154169f40 con 0x7fc1540cefa0 2024-09-21T06:11:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc158e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1540b3340 0x7fc154140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc158e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1540b3340 0x7fc154140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40720/0 (socket says 172.21.15.50:40720) 2024-09-21T06:11:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc158e00640 1 -- 172.21.15.50:0/2288316509 learned_addr learned my addr 172.21.15.50:0/2288316509 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc15a200640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1540cefa0 0x7fc15413f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc159800640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1540749a0 0x7fc1541f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc158e00640 1 -- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1540cefa0 msgr2=0x7fc15413f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc158e00640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1540cefa0 0x7fc15413f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc158e00640 1 -- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1540749a0 msgr2=0x7fc1541f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc158e00640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1540749a0 0x7fc1541f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc158e00640 1 -- 172.21.15.50:0/2288316509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15415f0b0 con 0x7fc1540b3340 2024-09-21T06:11:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc159800640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1540749a0 0x7fc1541f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc158e00640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1540b3340 0x7fc154140290 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc14000c3c0 tx=0x7fc14000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc151600640 1 -- 172.21.15.50:0/2288316509 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140017070 con 0x7fc1540b3340 2024-09-21T06:11:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc151600640 1 -- 172.21.15.50:0/2288316509 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1400053c0 con 0x7fc1540b3340 2024-09-21T06:11:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc15c200640 1 -- 172.21.15.50:0/2288316509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15415f5e0 con 0x7fc1540b3340 2024-09-21T06:11:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc151600640 1 -- 172.21.15.50:0/2288316509 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140003860 con 0x7fc1540b3340 2024-09-21T06:11:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.184+0000 7fc15c200640 1 -- 172.21.15.50:0/2288316509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15415fba0 con 0x7fc1540b3340 2024-09-21T06:11:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.188+0000 7fc15c200640 1 -- 172.21.15.50:0/2288316509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc15406f0e0 con 0x7fc1540b3340 2024-09-21T06:11:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.188+0000 7fc151600640 1 -- 172.21.15.50:0/2288316509 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7fc14001a070 con 0x7fc1540b3340 2024-09-21T06:11:04.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.188+0000 7fc151600640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc134075180 0x7fc134077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:04.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.188+0000 7fc151600640 1 -- 172.21.15.50:0/2288316509 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5965+0+0 (secure 0 0 0) 0x7fc140092470 con 0x7fc1540b3340 2024-09-21T06:11:04.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.188+0000 7fc159800640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc134075180 0x7fc134077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:04.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.192+0000 7fc159800640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc134075180 0x7fc134077690 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc148009b90 tx=0x7fc1480091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:04.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.192+0000 7fc151600640 1 -- 172.21.15.50:0/2288316509 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc14005e930 con 0x7fc1540b3340 2024-09-21T06:11:04.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.372+0000 7fc15c200640 1 -- 172.21.15.50:0/2288316509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc154074db0 con 0x7fc1540b3340 2024-09-21T06:11:04.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.992+0000 7fc151600640 1 -- 172.21.15.50:0/2288316509 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v118) ==== 181+0+0 (secure 0 0 0) 0x7fc140063360 con 0x7fc1540b3340 2024-09-21T06:11:04.991 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:11:04.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.996+0000 7fc15c200640 1 -- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc134075180 msgr2=0x7fc134077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:04.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.996+0000 7fc15c200640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc134075180 0x7fc134077690 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc148009b90 tx=0x7fc1480091e0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.996+0000 7fc15c200640 1 -- 172.21.15.50:0/2288316509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1540b3340 msgr2=0x7fc154140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:04.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.996+0000 7fc15c200640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1540b3340 0x7fc154140290 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc14000c3c0 tx=0x7fc14000c880 comp rx=0 tx=0).stop 2024-09-21T06:11:04.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.996+0000 7fc15c200640 1 -- 172.21.15.50:0/2288316509 shutdown_connections 2024-09-21T06:11:04.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.996+0000 7fc15c200640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc1540cefa0 0x7fc15413f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.996+0000 7fc15c200640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc134075180 0x7fc134077690 secure :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc148009b90 tx=0x7fc1480091e0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.996+0000 7fc15c200640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1540b3340 0x7fc154140290 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.996+0000 7fc15c200640 1 --2- 172.21.15.50:0/2288316509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1540749a0 0x7fc1541f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:04.996+0000 7fc15c200640 1 -- 172.21.15.50:0/2288316509 >> 172.21.15.50:0/2288316509 conn(0x7fc1540b2040 msgr2=0x7fc1540af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:04.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.000+0000 7fc15c200640 1 -- 172.21.15.50:0/2288316509 shutdown_connections 2024-09-21T06:11:04.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.000+0000 7fc15c200640 1 -- 172.21.15.50:0/2288316509 wait complete. 2024-09-21T06:11:05.021 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:11:05.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.192+0000 7f9960200640 1 -- 172.21.15.50:0/1670660309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f995806f170 msgr2=0x7f995806f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:05.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.192+0000 7f9960200640 1 --2- 172.21.15.50:0/1670660309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f995806f170 0x7f995806f580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9950009bc0 tx=0x7f995001cb70 comp rx=0 tx=0).stop 2024-09-21T06:11:05.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.192+0000 7f9960200640 1 -- 172.21.15.50:0/1670660309 shutdown_connections 2024-09-21T06:11:05.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.192+0000 7f9960200640 1 --2- 172.21.15.50:0/1670660309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f995806f170 0x7f995806f580 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:05.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.192+0000 7f9960200640 1 -- 172.21.15.50:0/1670660309 >> 172.21.15.50:0/1670660309 conn(0x7f99580b2c40 msgr2=0x7f99580b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:05.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.192+0000 7f9960200640 1 -- 172.21.15.50:0/1670660309 shutdown_connections 2024-09-21T06:11:05.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.192+0000 7f9960200640 1 -- 172.21.15.50:0/1670660309 wait complete. 2024-09-21T06:11:05.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.192+0000 7f9960200640 1 Processor -- start 2024-09-21T06:11:05.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.192+0000 7f9960200640 1 -- start start 2024-09-21T06:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f9960200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f995806f170 0x7f995815ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f9960200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f995815c5d0 0x7f995814fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f9960200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99581502e0 0x7f99580ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f9960200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9958169f60 con 0x7f99581502e0 2024-09-21T06:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f9960200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9958169de0 con 0x7f995815c5d0 2024-09-21T06:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f9960200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f995816a0e0 con 0x7f995806f170 2024-09-21T06:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f995e200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99581502e0 0x7f99580ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f995e200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99581502e0 0x7f99580ac220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40730/0 (socket says 172.21.15.50:40730) 2024-09-21T06:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f995e200640 1 -- 172.21.15.50:0/1656056752 learned_addr learned my addr 172.21.15.50:0/1656056752 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f995ce00640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f995815c5d0 0x7f995814fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f995d800640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f995806f170 0x7f995815ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:05.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f995e200640 1 -- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f995806f170 msgr2=0x7f995815ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:05.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f995e200640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f995806f170 0x7f995815ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:05.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f995e200640 1 -- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f995815c5d0 msgr2=0x7f995814fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:05.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f995e200640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f995815c5d0 0x7f995814fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:05.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f995e200640 1 -- 172.21.15.50:0/1656056752 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99580ab1c0 con 0x7f99581502e0 2024-09-21T06:11:05.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f995e200640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99581502e0 0x7f99580ac220 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f9950019780 tx=0x7f9950009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:05.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f9955600640 1 -- 172.21.15.50:0/1656056752 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99500bb070 con 0x7f99581502e0 2024-09-21T06:11:05.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f9955600640 1 -- 172.21.15.50:0/1656056752 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9950004d40 con 0x7f99581502e0 2024-09-21T06:11:05.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f9955600640 1 -- 172.21.15.50:0/1656056752 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99500b2040 con 0x7f99581502e0 2024-09-21T06:11:05.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f9960200640 1 -- 172.21.15.50:0/1656056752 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99580ab4a0 con 0x7f99581502e0 2024-09-21T06:11:05.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f9960200640 1 -- 172.21.15.50:0/1656056752 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99580cdfe0 con 0x7f99581502e0 2024-09-21T06:11:05.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.196+0000 7f9960200640 1 -- 172.21.15.50:0/1656056752 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99580cdaf0 con 0x7f99581502e0 2024-09-21T06:11:05.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.200+0000 7f9955600640 1 -- 172.21.15.50:0/1656056752 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f9950004260 con 0x7f99581502e0 2024-09-21T06:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.200+0000 7f9955600640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9938075180 0x7f9938077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.200+0000 7f9955600640 1 -- 172.21.15.50:0/1656056752 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5499+0+0 (secure 0 0 0) 0x7f995012cf00 con 0x7f99581502e0 2024-09-21T06:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.200+0000 7f995d800640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9938075180 0x7f9938077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:05.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.200+0000 7f995d800640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9938075180 0x7f9938077690 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f994c007fd0 tx=0x7f994c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.204+0000 7f9955600640 1 -- 172.21.15.50:0/1656056752 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f99500f2290 con 0x7f99581502e0 2024-09-21T06:11:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:05.384+0000 7f9960200640 1 -- 172.21.15.50:0/1656056752 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f99580aaf30 con 0x7f99581502e0 2024-09-21T06:11:05.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.000+0000 7f9955600640 1 -- 172.21.15.50:0/1656056752 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v119) ==== 169+0+0 (secure 0 0 0) 0x7f9950016790 con 0x7f99581502e0 2024-09-21T06:11:05.997 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-09-21T06:11:06.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.004+0000 7f9960200640 1 -- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9938075180 msgr2=0x7f9938077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:06.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.004+0000 7f9960200640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9938075180 0x7f9938077690 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f994c007fd0 tx=0x7f994c009040 comp rx=0 tx=0).stop 2024-09-21T06:11:06.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.004+0000 7f9960200640 1 -- 172.21.15.50:0/1656056752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99581502e0 msgr2=0x7f99580ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:06.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.004+0000 7f9960200640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99581502e0 0x7f99580ac220 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f9950019780 tx=0x7f9950009260 comp rx=0 tx=0).stop 2024-09-21T06:11:06.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.004+0000 7f9960200640 1 -- 172.21.15.50:0/1656056752 shutdown_connections 2024-09-21T06:11:06.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.004+0000 7f9960200640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9938075180 0x7f9938077690 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.004+0000 7f9960200640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99581502e0 0x7f99580ac220 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.004+0000 7f9960200640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f995815c5d0 0x7f995814fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.004+0000 7f9960200640 1 --2- 172.21.15.50:0/1656056752 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f995806f170 0x7f995815ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.004+0000 7f9960200640 1 -- 172.21.15.50:0/1656056752 >> 172.21.15.50:0/1656056752 conn(0x7f99580b2c40 msgr2=0x7f99580b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:06.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.004+0000 7f9960200640 1 -- 172.21.15.50:0/1656056752 shutdown_connections 2024-09-21T06:11:06.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.004+0000 7f9960200640 1 -- 172.21.15.50:0/1656056752 wait complete. 2024-09-21T06:11:06.027 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:11:06.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.188+0000 7fb562400640 1 -- 172.21.15.50:0/152134783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb55c074c30 msgr2=0x7fb55c075040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:06.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.192+0000 7fb562400640 1 --2- 172.21.15.50:0/152134783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb55c074c30 0x7fb55c075040 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb55c075760 tx=0x7fb54801c780 comp rx=0 tx=0).stop 2024-09-21T06:11:06.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.192+0000 7fb562400640 1 -- 172.21.15.50:0/152134783 shutdown_connections 2024-09-21T06:11:06.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.192+0000 7fb562400640 1 --2- 172.21.15.50:0/152134783 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb55c0d1190 0x7fb55c0d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.192+0000 7fb562400640 1 --2- 172.21.15.50:0/152134783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb55c074c30 0x7fb55c075040 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.192+0000 7fb562400640 1 --2- 172.21.15.50:0/152134783 >> v2:172.21.15.88:3300/0 conn(0x7fb55c0746e0 0x7fb55c15c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.192+0000 7fb562400640 1 -- 172.21.15.50:0/152134783 >> 172.21.15.50:0/152134783 conn(0x7fb55c0b41f0 msgr2=0x7fb55c0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:06.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.192+0000 7fb562400640 1 -- 172.21.15.50:0/152134783 shutdown_connections 2024-09-21T06:11:06.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.192+0000 7fb562400640 1 -- 172.21.15.50:0/152134783 wait complete. 2024-09-21T06:11:06.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.192+0000 7fb562400640 1 Processor -- start 2024-09-21T06:11:06.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.192+0000 7fb562400640 1 -- start start 2024-09-21T06:11:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb562400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb55c0746e0 0x7fb55c0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb562400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb55c074c30 0x7fb55c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb562400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb55c0d1190 0x7fb55c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb562400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c0ce620 con 0x7fb55c0746e0 2024-09-21T06:11:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb562400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c0ce4a0 con 0x7fb55c0d1190 2024-09-21T06:11:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb562400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb55c0ce7a0 con 0x7fb55c074c30 2024-09-21T06:11:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb561000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb55c0746e0 0x7fb55c0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb561000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb55c0746e0 0x7fb55c0703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40740/0 (socket says 172.21.15.50:40740) 2024-09-21T06:11:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb561000640 1 -- 172.21.15.50:0/2560132366 learned_addr learned my addr 172.21.15.50:0/2560132366 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb55be00640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb55c074c30 0x7fb55c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb561a00640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb55c0d1190 0x7fb55c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb561000640 1 -- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb55c074c30 msgr2=0x7fb55c070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb561000640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb55c074c30 0x7fb55c070920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb561000640 1 -- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb55c0d1190 msgr2=0x7fb55c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb561000640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb55c0d1190 0x7fb55c0b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb561000640 1 -- 172.21.15.50:0/2560132366 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55c0c9f10 con 0x7fb55c0746e0 2024-09-21T06:11:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb561000640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb55c0746e0 0x7fb55c0703e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb55000d700 tx=0x7fb55000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb558c00640 1 -- 172.21.15.50:0/2560132366 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550017070 con 0x7fb55c0746e0 2024-09-21T06:11:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb558c00640 1 -- 172.21.15.50:0/2560132366 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb550009b10 con 0x7fb55c0746e0 2024-09-21T06:11:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb562400640 1 -- 172.21.15.50:0/2560132366 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55c0ca4d0 con 0x7fb55c0746e0 2024-09-21T06:11:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb558c00640 1 -- 172.21.15.50:0/2560132366 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550010460 con 0x7fb55c0746e0 2024-09-21T06:11:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb562400640 1 -- 172.21.15.50:0/2560132366 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb55c0cef00 con 0x7fb55c0746e0 2024-09-21T06:11:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb562400640 1 -- 172.21.15.50:0/2560132366 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb55c0ca2c0 con 0x7fb55c0746e0 2024-09-21T06:11:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.196+0000 7fb558c00640 1 -- 172.21.15.50:0/2560132366 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7fb55001a070 con 0x7fb55c0746e0 2024-09-21T06:11:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.200+0000 7fb558c00640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb53c075180 0x7fb53c077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.200+0000 7fb558c00640 1 -- 172.21.15.50:0/2560132366 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7fb550093240 con 0x7fb55c0746e0 2024-09-21T06:11:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.200+0000 7fb55be00640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb53c075180 0x7fb53c077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.200+0000 7fb55be00640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb53c075180 0x7fb53c077690 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb5480af000 tx=0x7fb54801b020 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.204+0000 7fb558c00640 1 -- 172.21.15.50:0/2560132366 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb55005f7a0 con 0x7fb55c0746e0 2024-09-21T06:11:06.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.388+0000 7fb562400640 1 -- 172.21.15.50:0/2560132366 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fb55c075040 con 0x7fb55c0746e0 2024-09-21T06:11:06.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb558c00640 1 -- 172.21.15.50:0/2560132366 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v54) ==== 71+0+0 (secure 0 0 0) 0x7fb5500641d0 con 0x7fb55c0746e0 2024-09-21T06:11:06.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb562400640 1 -- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb53c075180 msgr2=0x7fb53c077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:06.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb562400640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb53c075180 0x7fb53c077690 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb5480af000 tx=0x7fb54801b020 comp rx=0 tx=0).stop 2024-09-21T06:11:06.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb562400640 1 -- 172.21.15.50:0/2560132366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb55c0746e0 msgr2=0x7fb55c0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:06.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb562400640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb55c0746e0 0x7fb55c0703e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb55000d700 tx=0x7fb55000dbc0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb562400640 1 -- 172.21.15.50:0/2560132366 shutdown_connections 2024-09-21T06:11:06.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb562400640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb55c0d1190 0x7fb55c0b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb562400640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb55c074c30 0x7fb55c070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb562400640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb53c075180 0x7fb53c077690 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb562400640 1 --2- 172.21.15.50:0/2560132366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb55c0746e0 0x7fb55c0703e0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb562400640 1 -- 172.21.15.50:0/2560132366 >> 172.21.15.50:0/2560132366 conn(0x7fb55c0b41f0 msgr2=0x7fb55c158470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:06.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb562400640 1 -- 172.21.15.50:0/2560132366 shutdown_connections 2024-09-21T06:11:06.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.392+0000 7fb562400640 1 -- 172.21.15.50:0/2560132366 wait complete. 2024-09-21T06:11:06.410 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-21T06:11:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.572+0000 7faa44800640 1 -- 172.21.15.50:0/2552155484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa3c0cf1b0 msgr2=0x7faa3c0cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.572+0000 7faa44800640 1 --2- 172.21.15.50:0/2552155484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa3c0cf1b0 0x7faa3c0cf5c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7faa3c1464a0 tx=0x7faa3801cba0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.572+0000 7faa44800640 1 -- 172.21.15.50:0/2552155484 shutdown_connections 2024-09-21T06:11:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.572+0000 7faa44800640 1 --2- 172.21.15.50:0/2552155484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa3c0cf1b0 0x7faa3c0cf5c0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.572+0000 7faa44800640 1 --2- 172.21.15.50:0/2552155484 >> v2:172.21.15.88:3300/0 conn(0x7faa3c1602a0 0x7faa3c1606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.572+0000 7faa44800640 1 -- 172.21.15.50:0/2552155484 >> 172.21.15.50:0/2552155484 conn(0x7faa3c0b2c40 msgr2=0x7faa3c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.572+0000 7faa44800640 1 -- 172.21.15.50:0/2552155484 shutdown_connections 2024-09-21T06:11:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.572+0000 7faa44800640 1 -- 172.21.15.50:0/2552155484 wait complete. 2024-09-21T06:11:06.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa44800640 1 Processor -- start 2024-09-21T06:11:06.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa44800640 1 -- start start 2024-09-21T06:11:06.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa44800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa3c0cd2e0 0x7faa3c151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:06.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa44800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faa3c0cf1b0 0x7faa3c159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:06.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa44800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa3c1602a0 0x7faa3c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:06.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa44800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c169e30 con 0x7faa3c1602a0 2024-09-21T06:11:06.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa44800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c169cb0 con 0x7faa3c0cd2e0 2024-09-21T06:11:06.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa44800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7faa3c169fb0 con 0x7faa3c0cf1b0 2024-09-21T06:11:06.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa42800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa3c1602a0 0x7faa3c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:06.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa42800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa3c1602a0 0x7faa3c157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40758/0 (socket says 172.21.15.50:40758) 2024-09-21T06:11:06.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa42800640 1 -- 172.21.15.50:0/3152468257 learned_addr learned my addr 172.21.15.50:0/3152468257 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:06.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa41e00640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa3c0cd2e0 0x7faa3c151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:06.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa41400640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faa3c0cf1b0 0x7faa3c159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:06.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa42800640 1 -- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faa3c0cf1b0 msgr2=0x7faa3c159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:06.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa42800640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faa3c0cf1b0 0x7faa3c159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa42800640 1 -- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa3c0cd2e0 msgr2=0x7faa3c151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:06.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa42800640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa3c0cd2e0 0x7faa3c151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa42800640 1 -- 172.21.15.50:0/3152468257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3c1572e0 con 0x7faa3c1602a0 2024-09-21T06:11:06.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa42800640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa3c1602a0 0x7faa3c157cf0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7faa38009bd0 tx=0x7faa38003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:06.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa36000640 1 -- 172.21.15.50:0/3152468257 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa380bc070 con 0x7faa3c1602a0 2024-09-21T06:11:06.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa36000640 1 -- 172.21.15.50:0/3152468257 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa380042f0 con 0x7faa3c1602a0 2024-09-21T06:11:06.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa44800640 1 -- 172.21.15.50:0/3152468257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa3c1549a0 con 0x7faa3c1602a0 2024-09-21T06:11:06.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa36000640 1 -- 172.21.15.50:0/3152468257 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa380b2040 con 0x7faa3c1602a0 2024-09-21T06:11:06.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.576+0000 7faa44800640 1 -- 172.21.15.50:0/3152468257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa3c151e60 con 0x7faa3c1602a0 2024-09-21T06:11:06.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.580+0000 7faa44800640 1 -- 172.21.15.50:0/3152468257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa3c156dd0 con 0x7faa3c1602a0 2024-09-21T06:11:06.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.580+0000 7faa36000640 1 -- 172.21.15.50:0/3152468257 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7faa380bf070 con 0x7faa3c1602a0 2024-09-21T06:11:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.580+0000 7faa36000640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faa20075180 0x7faa20077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.580+0000 7faa36000640 1 -- 172.21.15.50:0/3152468257 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7faa3812cf10 con 0x7faa3c1602a0 2024-09-21T06:11:06.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.580+0000 7faa41e00640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faa20075180 0x7faa20077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:06.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.584+0000 7faa41e00640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faa20075180 0x7faa20077690 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7faa300098e0 tx=0x7faa30009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.584+0000 7faa36000640 1 -- 172.21.15.50:0/3152468257 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7faa3c160bf0 con 0x7faa3c1602a0 2024-09-21T06:11:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:06.832+0000 7faa44800640 1 -- 172.21.15.50:0/3152468257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7faa3c0cf5c0 con 0x7faa3c1602a0 2024-09-21T06:11:06.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.000+0000 7faa36000640 1 -- 172.21.15.50:0/3152468257 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v652) ==== 185+0+0 (secure 0 0 0) 0x7faa380f9470 con 0x7faa3c1602a0 2024-09-21T06:11:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.004+0000 7faa44800640 1 -- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faa20075180 msgr2=0x7faa20077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.004+0000 7faa44800640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faa20075180 0x7faa20077690 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7faa300098e0 tx=0x7faa30009060 comp rx=0 tx=0).stop 2024-09-21T06:11:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.004+0000 7faa44800640 1 -- 172.21.15.50:0/3152468257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa3c1602a0 msgr2=0x7faa3c157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.004+0000 7faa44800640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa3c1602a0 0x7faa3c157cf0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7faa38009bd0 tx=0x7faa38003f80 comp rx=0 tx=0).stop 2024-09-21T06:11:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.004+0000 7faa44800640 1 -- 172.21.15.50:0/3152468257 shutdown_connections 2024-09-21T06:11:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.004+0000 7faa44800640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faa20075180 0x7faa20077690 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.004+0000 7faa44800640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa3c1602a0 0x7faa3c157cf0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.004+0000 7faa44800640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7faa3c0cf1b0 0x7faa3c159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.004+0000 7faa44800640 1 --2- 172.21.15.50:0/3152468257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa3c0cd2e0 0x7faa3c151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.004+0000 7faa44800640 1 -- 172.21.15.50:0/3152468257 >> 172.21.15.50:0/3152468257 conn(0x7faa3c0b2c40 msgr2=0x7faa3c0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.004+0000 7faa44800640 1 -- 172.21.15.50:0/3152468257 shutdown_connections 2024-09-21T06:11:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.004+0000 7faa44800640 1 -- 172.21.15.50:0/3152468257 wait complete. 2024-09-21T06:11:07.022 INFO:tasks.cephfs_test_runner:Test that progress bars, that printed for 3 ongoing clone jobs, are ... ok 2024-09-21T06:11:07.022 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the progress bar that is printed for 1 ongoing clone job is 2024-09-21T06:11:07.023 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-21T06:11:07.023 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:11:07.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.184+0000 7f40f3600640 1 -- 172.21.15.50:0/2402303857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40ec15b2d0 msgr2=0x7f40ec15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.184+0000 7f40f3600640 1 --2- 172.21.15.50:0/2402303857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40ec15b2d0 0x7f40ec15b6e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009840 tx=0x7f40d801c820 comp rx=0 tx=0).stop 2024-09-21T06:11:07.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.184+0000 7f40f3600640 1 -- 172.21.15.50:0/2402303857 shutdown_connections 2024-09-21T06:11:07.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.184+0000 7f40f3600640 1 --2- 172.21.15.50:0/2402303857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40ec15b2d0 0x7f40ec15b6e0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.184+0000 7f40f3600640 1 --2- 172.21.15.50:0/2402303857 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f40ec15e8c0 0x7f40ec15ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.184+0000 7f40f3600640 1 -- 172.21.15.50:0/2402303857 >> 172.21.15.50:0/2402303857 conn(0x7f40ec0b2040 msgr2=0x7f40ec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:07.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.184+0000 7f40f3600640 1 -- 172.21.15.50:0/2402303857 shutdown_connections 2024-09-21T06:11:07.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.184+0000 7f40f3600640 1 -- 172.21.15.50:0/2402303857 wait complete. 2024-09-21T06:11:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f3600640 1 Processor -- start 2024-09-21T06:11:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f3600640 1 -- start start 2024-09-21T06:11:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f3600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40ec15b2d0 0x7f40ec1584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f3600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f40ec15bc20 0x7f40ec160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f3600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40ec15e8c0 0x7f40ec13cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f3600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec15e1e0 con 0x7f40ec15b2d0 2024-09-21T06:11:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f3600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec15e060 con 0x7f40ec15e8c0 2024-09-21T06:11:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f3600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f40ec15e360 con 0x7f40ec15bc20 2024-09-21T06:11:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40ec15b2d0 0x7f40ec1584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40ec15b2d0 0x7f40ec1584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45072/0 (socket says 172.21.15.50:45072) 2024-09-21T06:11:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f0c00640 1 -- 172.21.15.50:0/1218047336 learned_addr learned my addr 172.21.15.50:0/1218047336 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40ebe00640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f40ec15bc20 0x7f40ec160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f1600640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40ec15e8c0 0x7f40ec13cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f0c00640 1 -- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f40ec15bc20 msgr2=0x7f40ec160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f0c00640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f40ec15bc20 0x7f40ec160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f0c00640 1 -- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40ec15e8c0 msgr2=0x7f40ec13cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f0c00640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40ec15e8c0 0x7f40ec13cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f0c00640 1 -- 172.21.15.50:0/1218047336 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ec158c80 con 0x7f40ec15b2d0 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f1600640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40ec15e8c0 0x7f40ec13cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f0c00640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40ec15b2d0 0x7f40ec1584c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c9f0 tx=0x7f40e000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40e8c00640 1 -- 172.21.15.50:0/1218047336 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0007bb0 con 0x7f40ec15b2d0 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40e8c00640 1 -- 172.21.15.50:0/1218047336 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40e0007e40 con 0x7f40ec15b2d0 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f3600640 1 -- 172.21.15.50:0/1218047336 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ec15fa90 con 0x7f40ec15b2d0 2024-09-21T06:11:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40e8c00640 1 -- 172.21.15.50:0/1218047336 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e000f690 con 0x7f40ec15b2d0 2024-09-21T06:11:07.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.188+0000 7f40f3600640 1 -- 172.21.15.50:0/1218047336 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ec160050 con 0x7f40ec15b2d0 2024-09-21T06:11:07.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.192+0000 7f40f3600640 1 -- 172.21.15.50:0/1218047336 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ec15b6e0 con 0x7f40ec15b2d0 2024-09-21T06:11:07.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.192+0000 7f40e8c00640 1 -- 172.21.15.50:0/1218047336 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f40e000f830 con 0x7f40ec15b2d0 2024-09-21T06:11:07.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.192+0000 7f40e8c00640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f40cc0751b0 0x7f40cc0776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.192+0000 7f40e8c00640 1 -- 172.21.15.50:0/1218047336 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f40e0093ae0 con 0x7f40ec15b2d0 2024-09-21T06:11:07.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.192+0000 7f40ebe00640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f40cc0751b0 0x7f40cc0776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.192+0000 7f40ebe00640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f40cc0751b0 0x7f40cc0776c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f40d80af000 tx=0x7f40d801b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:07.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.196+0000 7f40e8c00640 1 -- 172.21.15.50:0/1218047336 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f40e0014030 con 0x7f40ec15b2d0 2024-09-21T06:11:07.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.376+0000 7f40f3600640 1 -- 172.21.15.50:0/1218047336 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40ec15d720 con 0x7f40ec15b2d0 2024-09-21T06:11:07.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.376+0000 7f40e8c00640 1 -- 172.21.15.50:0/1218047336 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+13127 (secure 0 0 0) 0x7f40e0099360 con 0x7f40ec15b2d0 2024-09-21T06:11:07.374 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:07.374 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":119,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:11:05.997245+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6833/1093475697":"2024-09-22T06:11:02.982425+0000","172.21.15.50:6832/1093475697":"2024-09-22T06:11:02.982425+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:11:07.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.380+0000 7f40f3600640 1 -- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f40cc0751b0 msgr2=0x7f40cc0776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.380+0000 7f40f3600640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f40cc0751b0 0x7f40cc0776c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f40d80af000 tx=0x7f40d801b060 comp rx=0 tx=0).stop 2024-09-21T06:11:07.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.380+0000 7f40f3600640 1 -- 172.21.15.50:0/1218047336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40ec15b2d0 msgr2=0x7f40ec1584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.380+0000 7f40f3600640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40ec15b2d0 0x7f40ec1584c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c9f0 tx=0x7f40e000ceb0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.380+0000 7f40f3600640 1 -- 172.21.15.50:0/1218047336 shutdown_connections 2024-09-21T06:11:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.380+0000 7f40f3600640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40ec15e8c0 0x7f40ec13cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.380+0000 7f40f3600640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f40ec15bc20 0x7f40ec160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.380+0000 7f40f3600640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f40cc0751b0 0x7f40cc0776c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.380+0000 7f40f3600640 1 --2- 172.21.15.50:0/1218047336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40ec15b2d0 0x7f40ec1584c0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.380+0000 7f40f3600640 1 -- 172.21.15.50:0/1218047336 >> 172.21.15.50:0/1218047336 conn(0x7f40ec0b2040 msgr2=0x7f40ec0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.380+0000 7f40f3600640 1 -- 172.21.15.50:0/1218047336 shutdown_connections 2024-09-21T06:11:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.380+0000 7f40f3600640 1 -- 172.21.15.50:0/1218047336 wait complete. 2024-09-21T06:11:07.397 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:56:18.921197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:11:07.398 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:11:07.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.564+0000 7fe7e8400640 1 -- 172.21.15.50:0/3379971666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7e00b4de0 msgr2=0x7fe7e00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.564+0000 7fe7e8400640 1 --2- 172.21.15.50:0/3379971666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7e00b4de0 0x7fe7e00b51f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0070710 tx=0x7fe7d801cac0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.564+0000 7fe7e8400640 1 -- 172.21.15.50:0/3379971666 shutdown_connections 2024-09-21T06:11:07.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.564+0000 7fe7e8400640 1 --2- 172.21.15.50:0/3379971666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7e00b4de0 0x7fe7e00b51f0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.564+0000 7fe7e8400640 1 --2- 172.21.15.50:0/3379971666 >> v2:172.21.15.88:3300/0 conn(0x7fe7e01527b0 0x7fe7e015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.564+0000 7fe7e8400640 1 -- 172.21.15.50:0/3379971666 >> 172.21.15.50:0/3379971666 conn(0x7fe7e00b2040 msgr2=0x7fe7e00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:07.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.564+0000 7fe7e8400640 1 -- 172.21.15.50:0/3379971666 shutdown_connections 2024-09-21T06:11:07.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.564+0000 7fe7e8400640 1 -- 172.21.15.50:0/3379971666 wait complete. 2024-09-21T06:11:07.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e8400640 1 Processor -- start 2024-09-21T06:11:07.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e8400640 1 -- start start 2024-09-21T06:11:07.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e8400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7e00b4de0 0x7fe7e014c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e8400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7e00b5c70 0x7fe7e01fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e8400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe7e01527b0 0x7fe7e01f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e8400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e01688b0 con 0x7fe7e00b4de0 2024-09-21T06:11:07.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e8400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0168730 con 0x7fe7e00b5c70 2024-09-21T06:11:07.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e8400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e0168a30 con 0x7fe7e01527b0 2024-09-21T06:11:07.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7e00b4de0 0x7fe7e014c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7e00b4de0 0x7fe7e014c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45094/0 (socket says 172.21.15.50:45094) 2024-09-21T06:11:07.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e5a00640 1 -- 172.21.15.50:0/2784978072 learned_addr learned my addr 172.21.15.50:0/2784978072 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:07.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e5000640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7e00b5c70 0x7fe7e01fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e6400640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe7e01527b0 0x7fe7e01f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e5a00640 1 -- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe7e01527b0 msgr2=0x7fe7e01f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e5a00640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe7e01527b0 0x7fe7e01f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e5a00640 1 -- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7e00b5c70 msgr2=0x7fe7e01fa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e5a00640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7e00b5c70 0x7fe7e01fa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e5a00640 1 -- 172.21.15.50:0/2784978072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e01575e0 con 0x7fe7e00b4de0 2024-09-21T06:11:07.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e5a00640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7e00b4de0 0x7fe7e014c590 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400c400 tx=0x7fe7d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:07.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7dd600640 1 -- 172.21.15.50:0/2784978072 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4017070 con 0x7fe7e00b4de0 2024-09-21T06:11:07.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7dd600640 1 -- 172.21.15.50:0/2784978072 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7d4003600 con 0x7fe7e00b4de0 2024-09-21T06:11:07.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7dd600640 1 -- 172.21.15.50:0/2784978072 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4003a50 con 0x7fe7e00b4de0 2024-09-21T06:11:07.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e8400640 1 -- 172.21.15.50:0/2784978072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e01578c0 con 0x7fe7e00b4de0 2024-09-21T06:11:07.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.568+0000 7fe7e8400640 1 -- 172.21.15.50:0/2784978072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e0157e80 con 0x7fe7e00b4de0 2024-09-21T06:11:07.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.572+0000 7fe7e8400640 1 -- 172.21.15.50:0/2784978072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e00b51f0 con 0x7fe7e00b4de0 2024-09-21T06:11:07.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.572+0000 7fe7dd600640 1 -- 172.21.15.50:0/2784978072 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7fe7d401a070 con 0x7fe7e00b4de0 2024-09-21T06:11:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.572+0000 7fe7dd600640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe7c0075180 0x7fe7c0077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.572+0000 7fe7dd600640 1 -- 172.21.15.50:0/2784978072 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7fe7d4092580 con 0x7fe7e00b4de0 2024-09-21T06:11:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.572+0000 7fe7e5000640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe7c0075180 0x7fe7c0077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.572+0000 7fe7e5000640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe7c0075180 0x7fe7c0077690 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe7e015c290 tx=0x7fe7cc0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.576+0000 7fe7dd600640 1 -- 172.21.15.50:0/2784978072 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe7e00b5730 con 0x7fe7e00b4de0 2024-09-21T06:11:07.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.756+0000 7fe7e8400640 1 -- 172.21.15.50:0/2784978072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe7e015bfe0 con 0x7fe7e00b4de0 2024-09-21T06:11:07.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7dd600640 1 -- 172.21.15.50:0/2784978072 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7fe7d405e9b0 con 0x7fe7e00b4de0 2024-09-21T06:11:07.756 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-09-21T06:11:07.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7e8400640 1 -- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe7c0075180 msgr2=0x7fe7c0077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7e8400640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe7c0075180 0x7fe7c0077690 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe7e015c290 tx=0x7fe7cc0091e0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7e8400640 1 -- 172.21.15.50:0/2784978072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7e00b4de0 msgr2=0x7fe7e014c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7e8400640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7e00b4de0 0x7fe7e014c590 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400c400 tx=0x7fe7d400c8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7e8400640 1 -- 172.21.15.50:0/2784978072 shutdown_connections 2024-09-21T06:11:07.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7e8400640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe7e01527b0 0x7fe7e01f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7e8400640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7e00b5c70 0x7fe7e01fa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7e8400640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe7c0075180 0x7fe7c0077690 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7e8400640 1 --2- 172.21.15.50:0/2784978072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7e00b4de0 0x7fe7e014c590 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7e8400640 1 -- 172.21.15.50:0/2784978072 >> 172.21.15.50:0/2784978072 conn(0x7fe7e00b2040 msgr2=0x7fe7e00b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:07.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7e8400640 1 -- 172.21.15.50:0/2784978072 shutdown_connections 2024-09-21T06:11:07.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.760+0000 7fe7e8400640 1 -- 172.21.15.50:0/2784978072 wait complete. 2024-09-21T06:11:07.779 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-21T06:11:07.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.940+0000 7f3710800640 1 -- 172.21.15.50:0/3045011942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37080b54f0 msgr2=0x7f370815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.940+0000 7f3710800640 1 --2- 172.21.15.50:0/3045011942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37080b54f0 0x7f370815bb60 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3704009bc0 tx=0x7f370401cae0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.940+0000 7f3710800640 1 -- 172.21.15.50:0/3045011942 shutdown_connections 2024-09-21T06:11:07.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.940+0000 7f3710800640 1 --2- 172.21.15.50:0/3045011942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37080b54f0 0x7f370815bb60 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.940+0000 7f3710800640 1 -- 172.21.15.50:0/3045011942 >> 172.21.15.50:0/3045011942 conn(0x7f37080b2040 msgr2=0x7f37080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:07.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.940+0000 7f3710800640 1 -- 172.21.15.50:0/3045011942 shutdown_connections 2024-09-21T06:11:07.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.940+0000 7f3710800640 1 -- 172.21.15.50:0/3045011942 wait complete. 2024-09-21T06:11:07.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f3710800640 1 Processor -- start 2024-09-21T06:11:07.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f3710800640 1 -- start start 2024-09-21T06:11:07.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f3710800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37080b5c70 0x7f370814a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f3710800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f37081527b0 0x7f370814bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f3710800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f370814ae20 0x7f37080ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f3710800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f37080b45e0 con 0x7f37080b5c70 2024-09-21T06:11:07.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f3710800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f37080b4460 con 0x7f370814ae20 2024-09-21T06:11:07.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f3710800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f37080b4760 con 0x7f37081527b0 2024-09-21T06:11:07.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f370de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37080b5c70 0x7f370814a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f370de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37080b5c70 0x7f370814a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45110/0 (socket says 172.21.15.50:45110) 2024-09-21T06:11:07.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f370de00640 1 -- 172.21.15.50:0/549364680 learned_addr learned my addr 172.21.15.50:0/549364680 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:07.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f370e800640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f370814ae20 0x7f37080ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f370d400640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f37081527b0 0x7f370814bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f370de00640 1 -- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f37081527b0 msgr2=0x7f370814bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f370de00640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f37081527b0 0x7f370814bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f370de00640 1 -- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f370814ae20 msgr2=0x7f37080ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f370de00640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f370814ae20 0x7f37080ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.944+0000 7f370de00640 1 -- 172.21.15.50:0/549364680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3708157210 con 0x7f37080b5c70 2024-09-21T06:11:07.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.948+0000 7f370de00640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37080b5c70 0x7f370814a730 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00c490 tx=0x7f36fc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:07.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.948+0000 7f3702000640 1 -- 172.21.15.50:0/549364680 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc011790 con 0x7f37080b5c70 2024-09-21T06:11:07.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.948+0000 7f3702000640 1 -- 172.21.15.50:0/549364680 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36fc017070 con 0x7f37080b5c70 2024-09-21T06:11:07.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.948+0000 7f3702000640 1 -- 172.21.15.50:0/549364680 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc011ce0 con 0x7f37080b5c70 2024-09-21T06:11:07.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.948+0000 7f3710800640 1 -- 172.21.15.50:0/549364680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37081fbd00 con 0x7f37080b5c70 2024-09-21T06:11:07.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.948+0000 7f3710800640 1 -- 172.21.15.50:0/549364680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37081fc2c0 con 0x7f37080b5c70 2024-09-21T06:11:07.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.948+0000 7f3710800640 1 -- 172.21.15.50:0/549364680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f370814d6b0 con 0x7f37080b5c70 2024-09-21T06:11:07.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.948+0000 7f3702000640 1 -- 172.21.15.50:0/549364680 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f36fc01c460 con 0x7f37080b5c70 2024-09-21T06:11:07.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.948+0000 7f3702000640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f36ec0751b0 0x7f36ec0776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.952+0000 7f3702000640 1 -- 172.21.15.50:0/549364680 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f36fc092eb0 con 0x7f37080b5c70 2024-09-21T06:11:07.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.952+0000 7f370d400640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f36ec0751b0 0x7f36ec0776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.952+0000 7f370d400640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f36ec0751b0 0x7f36ec0776c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f37080b64d0 tx=0x7f36f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:07.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:07.956+0000 7f3702000640 1 -- 172.21.15.50:0/549364680 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f36fc05f430 con 0x7f37080b5c70 2024-09-21T06:11:08.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:08.196+0000 7f3710800640 1 -- 172.21.15.50:0/549364680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7f370815bb60 con 0x7f37080b5c70 2024-09-21T06:11:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.008+0000 7f3702000640 1 -- 172.21.15.50:0/549364680 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v654) ==== 189+0+0 (secure 0 0 0) 0x7f36fc063e60 con 0x7f37080b5c70 2024-09-21T06:11:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.012+0000 7f3710800640 1 -- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f36ec0751b0 msgr2=0x7f36ec0776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.012+0000 7f3710800640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f36ec0751b0 0x7f36ec0776c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f37080b64d0 tx=0x7f36f8009040 comp rx=0 tx=0).stop 2024-09-21T06:11:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.012+0000 7f3710800640 1 -- 172.21.15.50:0/549364680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37080b5c70 msgr2=0x7f370814a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.012+0000 7f3710800640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37080b5c70 0x7f370814a730 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00c490 tx=0x7f36fc00c950 comp rx=0 tx=0).stop 2024-09-21T06:11:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.012+0000 7f3710800640 1 -- 172.21.15.50:0/549364680 shutdown_connections 2024-09-21T06:11:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.012+0000 7f3710800640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f370814ae20 0x7f37080ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.012+0000 7f3710800640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f37081527b0 0x7f370814bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.012+0000 7f3710800640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f36ec0751b0 0x7f36ec0776c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.012+0000 7f3710800640 1 --2- 172.21.15.50:0/549364680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37080b5c70 0x7f370814a730 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.012+0000 7f3710800640 1 -- 172.21.15.50:0/549364680 >> 172.21.15.50:0/549364680 conn(0x7f37080b2040 msgr2=0x7f370815b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.012+0000 7f3710800640 1 -- 172.21.15.50:0/549364680 shutdown_connections 2024-09-21T06:11:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.012+0000 7f3710800640 1 -- 172.21.15.50:0/549364680 wait complete. 2024-09-21T06:11:09.034 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:11:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.200+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2332924969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc01527b0 msgr2=0x7f2bc015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.200+0000 7f2bc7a00640 1 --2- 172.21.15.50:0/2332924969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc01527b0 0x7f2bc015bb60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0074b20 tx=0x7f2bac01c680 comp rx=0 tx=0).stop 2024-09-21T06:11:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.200+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2332924969 shutdown_connections 2024-09-21T06:11:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.200+0000 7f2bc7a00640 1 --2- 172.21.15.50:0/2332924969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc01527b0 0x7f2bc015bb60 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.200+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2332924969 >> 172.21.15.50:0/2332924969 conn(0x7f2bc00b2040 msgr2=0x7f2bc00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:09.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.200+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2332924969 shutdown_connections 2024-09-21T06:11:09.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.200+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2332924969 wait complete. 2024-09-21T06:11:09.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.200+0000 7f2bc7a00640 1 Processor -- start 2024-09-21T06:11:09.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.200+0000 7f2bc7a00640 1 -- start start 2024-09-21T06:11:09.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc7a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc00b4de0 0x7f2bc01509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc7a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2bc00b5c70 0x7f2bc0142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc7a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc01527b0 0x7f2bc00af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc7a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0169bb0 con 0x7f2bc01527b0 2024-09-21T06:11:09.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc7a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0169a30 con 0x7f2bc00b4de0 2024-09-21T06:11:09.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc7a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc0169d30 con 0x7f2bc00b5c70 2024-09-21T06:11:09.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc01527b0 0x7f2bc00af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc01527b0 0x7f2bc00af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45124/0 (socket says 172.21.15.50:45124) 2024-09-21T06:11:09.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc5a00640 1 -- 172.21.15.50:0/2632168147 learned_addr learned my addr 172.21.15.50:0/2632168147 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:09.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bbfe00640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2bc00b5c70 0x7f2bc0142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc5000640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc00b4de0 0x7f2bc01509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc5a00640 1 -- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2bc00b5c70 msgr2=0x7f2bc0142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc5a00640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2bc00b5c70 0x7f2bc0142b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc5a00640 1 -- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc00b4de0 msgr2=0x7f2bc01509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc5a00640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc00b4de0 0x7f2bc01509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc5a00640 1 -- 172.21.15.50:0/2632168147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc00acdb0 con 0x7f2bc01527b0 2024-09-21T06:11:09.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc5a00640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc01527b0 0x7f2bc00af080 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c730 tx=0x7f2bb800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:09.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bbcc00640 1 -- 172.21.15.50:0/2632168147 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8003b80 con 0x7f2bc01527b0 2024-09-21T06:11:09.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bbcc00640 1 -- 172.21.15.50:0/2632168147 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bb8003d60 con 0x7f2bc01527b0 2024-09-21T06:11:09.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bbcc00640 1 -- 172.21.15.50:0/2632168147 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8005330 con 0x7f2bc01527b0 2024-09-21T06:11:09.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2632168147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc00ad220 con 0x7f2bc01527b0 2024-09-21T06:11:09.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2632168147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc00ad7e0 con 0x7f2bc01527b0 2024-09-21T06:11:09.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.204+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2632168147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc00ada20 con 0x7f2bc01527b0 2024-09-21T06:11:09.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.208+0000 7f2bbcc00640 1 -- 172.21.15.50:0/2632168147 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f2bb80056e0 con 0x7f2bc01527b0 2024-09-21T06:11:09.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.208+0000 7f2bbcc00640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ba00751b0 0x7f2ba00776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.208+0000 7f2bbcc00640 1 -- 172.21.15.50:0/2632168147 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f2bb80931a0 con 0x7f2bc01527b0 2024-09-21T06:11:09.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.208+0000 7f2bc5000640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ba00751b0 0x7f2ba00776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.208+0000 7f2bc5000640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ba00751b0 0x7f2ba00776c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4000fe0 tx=0x7f2bb4001730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.212+0000 7f2bbcc00640 1 -- 172.21.15.50:0/2632168147 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2bc00ada20 con 0x7f2bc01527b0 2024-09-21T06:11:09.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2632168147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f2bc015bb60 con 0x7f2bc01527b0 2024-09-21T06:11:09.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bbcc00640 1 -- 172.21.15.50:0/2632168147 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v54) ==== 59+0+2294 (secure 0 0 0) 0x7f2bb805f5d0 con 0x7f2bc01527b0 2024-09-21T06:11:09.388 INFO:teuthology.orchestra.run.smithi050.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:11:09.388 INFO:teuthology.orchestra.run.smithi050.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:11:09.388 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:11:09.388 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_client 20/20 2024-09-21T06:11:09.388 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:11:09.388 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:11:09.388 INFO:teuthology.orchestra.run.smithi050.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3735.149299 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4471.400731 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3640.131623 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3663.645087 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3762.055672 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3661.558528 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3661.456695 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3796.187852 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds 20/20 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_ms 2/2 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_frag true 2024-09-21T06:11:09.389 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:11:09.390 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:11:09.390 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_verify_scatter true 2024-09-21T06:11:09.390 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:11:09.390 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:11:09.390 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:11:09.390 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced client_mount_timeout 600 2024-09-21T06:11:09.390 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_client 20/20 2024-09-21T06:11:09.390 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_ms 1/1 2024-09-21T06:11:09.390 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:11:09.390 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:11:09.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ba00751b0 msgr2=0x7f2ba00776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ba00751b0 0x7f2ba00776c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4000fe0 tx=0x7f2bb4001730 comp rx=0 tx=0).stop 2024-09-21T06:11:09.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2632168147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc01527b0 msgr2=0x7f2bc00af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc01527b0 0x7f2bc00af080 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c730 tx=0x7f2bb800cbf0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2632168147 shutdown_connections 2024-09-21T06:11:09.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ba00751b0 0x7f2ba00776c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc01527b0 0x7f2bc00af080 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2bc00b5c70 0x7f2bc0142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 --2- 172.21.15.50:0/2632168147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc00b4de0 0x7f2bc01509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2632168147 >> 172.21.15.50:0/2632168147 conn(0x7f2bc00b2040 msgr2=0x7f2bc00ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:09.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2632168147 shutdown_connections 2024-09-21T06:11:09.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.392+0000 7f2bc7a00640 1 -- 172.21.15.50:0/2632168147 wait complete. 2024-09-21T06:11:09.410 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:11:09.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.576+0000 7f61f6600640 1 -- 172.21.15.50:0/4202232394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d4003a10 msgr2=0x7f61d4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.576+0000 7f61f6600640 1 --2- 172.21.15.50:0/4202232394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d4003a10 0x7f61d4003e80 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009b60 tx=0x7f61e001d280 comp rx=0 tx=0).stop 2024-09-21T06:11:09.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.576+0000 7f61efe00640 1 -- 172.21.15.50:0/4202232394 reap_dead start 2024-09-21T06:11:09.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.576+0000 7f61f6600640 1 -- 172.21.15.50:0/4202232394 shutdown_connections 2024-09-21T06:11:09.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.576+0000 7f61f6600640 1 -- 172.21.15.50:0/4202232394 >> 172.21.15.50:0/4202232394 conn(0x7f61f00b2040 msgr2=0x7f61f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:09.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.576+0000 7f61f6600640 1 -- 172.21.15.50:0/4202232394 shutdown_connections 2024-09-21T06:11:09.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.576+0000 7f61f6600640 1 -- 172.21.15.50:0/4202232394 wait complete. 2024-09-21T06:11:09.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.576+0000 7f61f6600640 1 Processor -- start 2024-09-21T06:11:09.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.576+0000 7f61f6600640 1 -- start start 2024-09-21T06:11:09.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61f6600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d4003a10 0x7f61f00af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61f6600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f61f0150ce0 0x7f61f00aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61f6600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61f00ab2b0 0x7f61f01fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61f6600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0166b40 con 0x7f61f00ab2b0 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61f6600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01669c0 con 0x7f61d4003a10 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61f6600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f61f0166cc0 con 0x7f61f0150ce0 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61f00ab2b0 0x7f61f01fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61f00ab2b0 0x7f61f01fc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45136/0 (socket says 172.21.15.50:45136) 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61efe00640 1 -- 172.21.15.50:0/1492752970 learned_addr learned my addr 172.21.15.50:0/1492752970 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61e7e00640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f61f0150ce0 0x7f61f00aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61ef400640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d4003a10 0x7f61f00af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61efe00640 1 -- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f61f0150ce0 msgr2=0x7f61f00aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61efe00640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f61f0150ce0 0x7f61f00aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61efe00640 1 -- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d4003a10 msgr2=0x7f61f00af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61efe00640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d4003a10 0x7f61f00af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61efe00640 1 -- 172.21.15.50:0/1492752970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f0152c20 con 0x7f61f00ab2b0 2024-09-21T06:11:09.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61e7e00640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f61f0150ce0 0x7f61f00aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:09.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61ef400640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d4003a10 0x7f61f00af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:09.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61efe00640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61f00ab2b0 0x7f61f01fc4b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f61e8002240 tx=0x7f61e800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:09.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61e7400640 1 -- 172.21.15.50:0/1492752970 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e800c990 con 0x7f61f00ab2b0 2024-09-21T06:11:09.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61e7400640 1 -- 172.21.15.50:0/1492752970 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61e8018070 con 0x7f61f00ab2b0 2024-09-21T06:11:09.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61e7400640 1 -- 172.21.15.50:0/1492752970 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8011460 con 0x7f61f00ab2b0 2024-09-21T06:11:09.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61f6600640 1 -- 172.21.15.50:0/1492752970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f0154bd0 con 0x7f61f00ab2b0 2024-09-21T06:11:09.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61f6600640 1 -- 172.21.15.50:0/1492752970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f0158ef0 con 0x7f61f00ab2b0 2024-09-21T06:11:09.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.580+0000 7f61f6600640 1 -- 172.21.15.50:0/1492752970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f015bb60 con 0x7f61f00ab2b0 2024-09-21T06:11:09.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.584+0000 7f61e7400640 1 -- 172.21.15.50:0/1492752970 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f61e8011810 con 0x7f61f00ab2b0 2024-09-21T06:11:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.584+0000 7f61e7400640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f61cc075180 0x7f61cc077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.584+0000 7f61e7400640 1 -- 172.21.15.50:0/1492752970 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f61e80944a0 con 0x7f61f00ab2b0 2024-09-21T06:11:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.584+0000 7f61ef400640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f61cc075180 0x7f61cc077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.584+0000 7f61ef400640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f61cc075180 0x7f61cc077690 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f61e00ae000 tx=0x7f61e001a020 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:09.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.588+0000 7f61e7400640 1 -- 172.21.15.50:0/1492752970 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f61e8061030 con 0x7f61f00ab2b0 2024-09-21T06:11:09.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.764+0000 7f61f6600640 1 -- 172.21.15.50:0/1492752970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f61f0159160 con 0x7f61f00ab2b0 2024-09-21T06:11:09.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.768+0000 7f61e7400640 1 -- 172.21.15.50:0/1492752970 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v55)=0 v55) ==== 122+0+0 (secure 0 0 0) 0x7f61e8060930 con 0x7f61f00ab2b0 2024-09-21T06:11:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.772+0000 7f61f6600640 1 -- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f61cc075180 msgr2=0x7f61cc077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.772+0000 7f61f6600640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f61cc075180 0x7f61cc077690 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f61e00ae000 tx=0x7f61e001a020 comp rx=0 tx=0).stop 2024-09-21T06:11:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.772+0000 7f61f6600640 1 -- 172.21.15.50:0/1492752970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61f00ab2b0 msgr2=0x7f61f01fc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.772+0000 7f61f6600640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61f00ab2b0 0x7f61f01fc4b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f61e8002240 tx=0x7f61e800ecf0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.772+0000 7f61f6600640 1 -- 172.21.15.50:0/1492752970 shutdown_connections 2024-09-21T06:11:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.772+0000 7f61f6600640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f61cc075180 0x7f61cc077690 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.772+0000 7f61f6600640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61f00ab2b0 0x7f61f01fc4b0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.772+0000 7f61f6600640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f61f0150ce0 0x7f61f00aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.772+0000 7f61f6600640 1 --2- 172.21.15.50:0/1492752970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d4003a10 0x7f61f00af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.772+0000 7f61f6600640 1 -- 172.21.15.50:0/1492752970 >> 172.21.15.50:0/1492752970 conn(0x7f61f00b2040 msgr2=0x7f61f00cf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.772+0000 7f61f6600640 1 -- 172.21.15.50:0/1492752970 shutdown_connections 2024-09-21T06:11:09.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:09.772+0000 7f61f6600640 1 -- 172.21.15.50:0/1492752970 wait complete. 2024-09-21T06:11:09.789 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:11:09.789 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:11:09.866 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:11:09.952 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:11:09.952 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:11:09.952 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:11:09.961 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:11:09.961 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:11:09.961 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:11:09.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:11:09.970 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:11:09.970 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:11:10.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:11:10.015 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:11:10.015 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:11:10.024 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:11:10.025 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:11:10.025 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:11:10.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.220+0000 7f59aac00640 1 -- 172.21.15.50:0/1579423471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59a40b4de0 msgr2=0x7f59a40b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:10.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.220+0000 7f59aac00640 1 --2- 172.21.15.50:0/1579423471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59a40b4de0 0x7f59a40b51f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f59a40703e0 tx=0x7f599c01ccc0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.220+0000 7f59aac00640 1 -- 172.21.15.50:0/1579423471 shutdown_connections 2024-09-21T06:11:10.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.220+0000 7f59aac00640 1 --2- 172.21.15.50:0/1579423471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59a40b4de0 0x7f59a40b51f0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.220+0000 7f59aac00640 1 --2- 172.21.15.50:0/1579423471 >> v2:172.21.15.88:3300/0 conn(0x7f59a41527b0 0x7f59a4074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.220+0000 7f59aac00640 1 --2- 172.21.15.50:0/1579423471 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f59a40b5c70 0x7f59a415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.220+0000 7f59aac00640 1 -- 172.21.15.50:0/1579423471 >> 172.21.15.50:0/1579423471 conn(0x7f59a40b2040 msgr2=0x7f59a40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:10.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.220+0000 7f59aac00640 1 -- 172.21.15.50:0/1579423471 shutdown_connections 2024-09-21T06:11:10.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.220+0000 7f59aac00640 1 -- 172.21.15.50:0/1579423471 wait complete. 2024-09-21T06:11:10.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59aac00640 1 Processor -- start 2024-09-21T06:11:10.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59aac00640 1 -- start start 2024-09-21T06:11:10.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59aac00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59a40b4de0 0x7f59a40cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:10.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59aac00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59a40b5c70 0x7f59a41517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:10.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59aac00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f59a41527b0 0x7f59a4158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:10.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59aac00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f59a415f730 con 0x7f59a40b5c70 2024-09-21T06:11:10.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59aac00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f59a415f5b0 con 0x7f59a40b4de0 2024-09-21T06:11:10.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59aac00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f59a415f8b0 con 0x7f59a41527b0 2024-09-21T06:11:10.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f599b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59a40b5c70 0x7f59a41517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:10.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f599b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59a40b5c70 0x7f59a41517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45146/0 (socket says 172.21.15.50:45146) 2024-09-21T06:11:10.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f599b400640 1 -- 172.21.15.50:0/3336554445 learned_addr learned my addr 172.21.15.50:0/3336554445 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:10.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59a8c00640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f59a41527b0 0x7f59a4158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:10.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59a3e00640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59a40b4de0 0x7f59a40cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:10.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f599b400640 1 -- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f59a41527b0 msgr2=0x7f59a4158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:10.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f599b400640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f59a41527b0 0x7f59a4158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f599b400640 1 -- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59a40b4de0 msgr2=0x7f59a40cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:10.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f599b400640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59a40b4de0 0x7f59a40cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f599b400640 1 -- 172.21.15.50:0/3336554445 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a406f030 con 0x7f59a40b5c70 2024-09-21T06:11:10.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59a3e00640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59a40b4de0 0x7f59a40cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:10.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f599b400640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59a40b5c70 0x7f59a41517a0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f598c005d10 tx=0x7f598c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:10.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59a0c00640 1 -- 172.21.15.50:0/3336554445 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c017070 con 0x7f59a40b5c70 2024-09-21T06:11:10.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59a0c00640 1 -- 172.21.15.50:0/3336554445 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f598c009b20 con 0x7f59a40b5c70 2024-09-21T06:11:10.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59aac00640 1 -- 172.21.15.50:0/3336554445 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a41fc230 con 0x7f59a40b5c70 2024-09-21T06:11:10.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59a0c00640 1 -- 172.21.15.50:0/3336554445 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c010460 con 0x7f59a40b5c70 2024-09-21T06:11:10.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.224+0000 7f59aac00640 1 -- 172.21.15.50:0/3336554445 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a41fc7f0 con 0x7f59a40b5c70 2024-09-21T06:11:10.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.228+0000 7f59aac00640 1 -- 172.21.15.50:0/3336554445 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a415fdd0 con 0x7f59a40b5c70 2024-09-21T06:11:10.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.228+0000 7f59a0c00640 1 -- 172.21.15.50:0/3336554445 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f598c01a070 con 0x7f59a40b5c70 2024-09-21T06:11:10.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.228+0000 7f59a0c00640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f59800751b0 0x7f59800776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:10.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.228+0000 7f59a0c00640 1 -- 172.21.15.50:0/3336554445 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f598c0937c0 con 0x7f59a40b5c70 2024-09-21T06:11:10.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.228+0000 7f59a3e00640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f59800751b0 0x7f59800776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:10.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.232+0000 7f59a3e00640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f59800751b0 0x7f59800776c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5994005fd0 tx=0x7f5994005e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:10.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.232+0000 7f59a0c00640 1 -- 172.21.15.50:0/3336554445 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f598c05fc70 con 0x7f59a40b5c70 2024-09-21T06:11:10.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.444+0000 7f59aac00640 1 -- 172.21.15.50:0/3336554445 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59a415a6b0 con 0x7f59a40b5c70 2024-09-21T06:11:10.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.444+0000 7f59a0c00640 1 -- 172.21.15.50:0/3336554445 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+3911 (secure 0 0 0) 0x7f598c0646a0 con 0x7f59a40b5c70 2024-09-21T06:11:10.443 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:10.444 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":77,"btime":"2024-09-21T06:11:03:985390+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4024495468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":4024495468},{"type":"v1","addr":"172.21.15.50:6835","nonce":4024495468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3620352888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3620352888},{"type":"v1","addr":"172.21.15.88:6835","nonce":3620352888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4286454792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4286454792},{"type":"v1","addr":"172.21.15.88:6837","nonce":4286454792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/3375080867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3375080867},{"type":"v1","addr":"172.21.15.50:6833","nonce":3375080867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[]} 2024-09-21T06:11:10.444 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 77 2024-09-21T06:11:10.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.448+0000 7f59aac00640 1 -- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f59800751b0 msgr2=0x7f59800776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:10.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.448+0000 7f59aac00640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f59800751b0 0x7f59800776c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5994005fd0 tx=0x7f5994005e80 comp rx=0 tx=0).stop 2024-09-21T06:11:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.448+0000 7f59aac00640 1 -- 172.21.15.50:0/3336554445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59a40b5c70 msgr2=0x7f59a41517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.448+0000 7f59aac00640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59a40b5c70 0x7f59a41517a0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f598c005d10 tx=0x7f598c00dbb0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.448+0000 7f59aac00640 1 -- 172.21.15.50:0/3336554445 shutdown_connections 2024-09-21T06:11:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.448+0000 7f59aac00640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f59a41527b0 0x7f59a4158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.448+0000 7f59aac00640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f59800751b0 0x7f59800776c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.448+0000 7f59aac00640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59a40b5c70 0x7f59a41517a0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.448+0000 7f59aac00640 1 --2- 172.21.15.50:0/3336554445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59a40b4de0 0x7f59a40cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.448+0000 7f59aac00640 1 -- 172.21.15.50:0/3336554445 >> 172.21.15.50:0/3336554445 conn(0x7f59a40b2040 msgr2=0x7f59a406e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.448+0000 7f59aac00640 1 -- 172.21.15.50:0/3336554445 shutdown_connections 2024-09-21T06:11:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.448+0000 7f59aac00640 1 -- 172.21.15.50:0/3336554445 wait complete. 2024-09-21T06:11:10.466 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:11:10.467 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:11:10.467 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:11:10.467 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:11:10.467 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:11:10.467 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:11:10.467 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:11:10.468 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:11:10.468 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:11:10.468 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:11:10.468 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:11:10.468 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:11:10.468 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:11:10.468 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:11:10.469 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:11:10.469 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:11:10.482 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:11:10.482 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:11:10.486 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:11:10.486 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:11:10.487 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:11:10.487 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:11:10.490 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:11:10.490 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:11:10.531 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:11:10.532 DEBUG:teuthology.parallel:result is None 2024-09-21T06:11:10.533 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:11:10.533 DEBUG:teuthology.parallel:result is None 2024-09-21T06:11:10.537 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:11:10.537 DEBUG:teuthology.parallel:result is None 2024-09-21T06:11:10.539 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:11:10.539 DEBUG:teuthology.parallel:result is None 2024-09-21T06:11:10.540 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:11:10.636 INFO:tasks.ceph.mds.d.smithi088.stdout:starting mds.d at 2024-09-21T06:11:10.637 INFO:tasks.ceph.mds.b.smithi088.stdout:starting mds.b at 2024-09-21T06:11:10.637 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T06:11:10.643 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-09-21T06:11:10.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.708+0000 7f0fbee00640 1 -- 172.21.15.50:0/465500326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fb8155b50 msgr2=0x7f0fb80741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:10.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.708+0000 7f0fbee00640 1 --2- 172.21.15.50:0/465500326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fb8155b50 0x7f0fb80741a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009840 tx=0x7f0fb001c790 comp rx=0 tx=0).stop 2024-09-21T06:11:10.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.708+0000 7f0fbee00640 1 -- 172.21.15.50:0/465500326 shutdown_connections 2024-09-21T06:11:10.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.708+0000 7f0fbee00640 1 --2- 172.21.15.50:0/465500326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fb8155b50 0x7f0fb80741a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.708+0000 7f0fbee00640 1 -- 172.21.15.50:0/465500326 >> 172.21.15.50:0/465500326 conn(0x7f0fb80b2040 msgr2=0x7f0fb80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:10.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.708+0000 7f0fbee00640 1 -- 172.21.15.50:0/465500326 shutdown_connections 2024-09-21T06:11:10.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.708+0000 7f0fbee00640 1 -- 172.21.15.50:0/465500326 wait complete. 2024-09-21T06:11:10.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0fbee00640 1 Processor -- start 2024-09-21T06:11:10.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0fbee00640 1 -- start start 2024-09-21T06:11:10.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0fbee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fb8072100 0x7f0fb81fcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0fbee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fb8155b50 0x7f0fb814d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0fbee00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0fb815ac10 0x7f0fb814eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0fbee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8169e30 con 0x7f0fb8155b50 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0fbee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8169cb0 con 0x7f0fb8072100 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0fbee00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb8169fb0 con 0x7f0fb815ac10 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0faf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fb8155b50 0x7f0fb814d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0faf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fb8155b50 0x7f0fb814d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45218/0 (socket says 172.21.15.50:45218) 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0faf400640 1 -- 172.21.15.50:0/543463226 learned_addr learned my addr 172.21.15.50:0/543463226 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0fb7e00640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fb8072100 0x7f0fb81fcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0fbce00640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0fb815ac10 0x7f0fb814eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0faf400640 1 -- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0fb815ac10 msgr2=0x7f0fb814eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0faf400640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0fb815ac10 0x7f0fb814eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0faf400640 1 -- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fb8072100 msgr2=0x7f0fb81fcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:10.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0faf400640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fb8072100 0x7f0fb81fcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0faf400640 1 -- 172.21.15.50:0/543463226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb81fcfe0 con 0x7f0fb8155b50 2024-09-21T06:11:10.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.712+0000 7f0faf400640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fb8155b50 0x7f0fb814d8f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000c3c0 tx=0x7f0fa000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:10.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.716+0000 7f0fb4c00640 1 -- 172.21.15.50:0/543463226 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0011790 con 0x7f0fb8155b50 2024-09-21T06:11:10.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.716+0000 7f0fb4c00640 1 -- 172.21.15.50:0/543463226 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fa0017070 con 0x7f0fb8155b50 2024-09-21T06:11:10.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.716+0000 7f0fbee00640 1 -- 172.21.15.50:0/543463226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb815b1b0 con 0x7f0fb8155b50 2024-09-21T06:11:10.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.716+0000 7f0fb4c00640 1 -- 172.21.15.50:0/543463226 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0011ce0 con 0x7f0fb8155b50 2024-09-21T06:11:10.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.716+0000 7f0fbee00640 1 -- 172.21.15.50:0/543463226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb81f90d0 con 0x7f0fb8155b50 2024-09-21T06:11:10.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.716+0000 7f0fbee00640 1 -- 172.21.15.50:0/543463226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb80b4de0 con 0x7f0fb8155b50 2024-09-21T06:11:10.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.716+0000 7f0fb4c00640 1 -- 172.21.15.50:0/543463226 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f0fa001c460 con 0x7f0fb8155b50 2024-09-21T06:11:10.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.716+0000 7f0fb4c00640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f94075180 0x7f0f94077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:10.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.716+0000 7f0fb4c00640 1 -- 172.21.15.50:0/543463226 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f0fa0092de0 con 0x7f0fb8155b50 2024-09-21T06:11:10.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.716+0000 7f0fb7e00640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f94075180 0x7f0f94077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:10.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.720+0000 7f0fb7e00640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f94075180 0x7f0f94077690 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8000f90 tx=0x7f0fa8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:10.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.724+0000 7f0fb4c00640 1 -- 172.21.15.50:0/543463226 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0fb81fcfe0 con 0x7f0fb8155b50 2024-09-21T06:11:10.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:10.912+0000 7f0fbee00640 1 -- 172.21.15.50:0/543463226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0fb80b3740 con 0x7f0fb8155b50 2024-09-21T06:11:11.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.036+0000 7f0fb4c00640 1 -- 172.21.15.50:0/543463226 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v121) ==== 97+0+0 (secure 0 0 0) 0x7f0fa005f2a0 con 0x7f0fb8155b50 2024-09-21T06:11:11.034 INFO:teuthology.orchestra.run.smithi050.stderr: removed all blocklist entries 2024-09-21T06:11:11.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.040+0000 7f0fbee00640 1 -- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f94075180 msgr2=0x7f0f94077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.040+0000 7f0fbee00640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f94075180 0x7f0f94077690 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8000f90 tx=0x7f0fa8009040 comp rx=0 tx=0).stop 2024-09-21T06:11:11.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.040+0000 7f0fbee00640 1 -- 172.21.15.50:0/543463226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fb8155b50 msgr2=0x7f0fb814d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.040+0000 7f0fbee00640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fb8155b50 0x7f0fb814d8f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000c3c0 tx=0x7f0fa000c880 comp rx=0 tx=0).stop 2024-09-21T06:11:11.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.040+0000 7f0fbee00640 1 -- 172.21.15.50:0/543463226 shutdown_connections 2024-09-21T06:11:11.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.040+0000 7f0fbee00640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0fb815ac10 0x7f0fb814eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.040+0000 7f0fbee00640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f94075180 0x7f0f94077690 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.040+0000 7f0fbee00640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fb8155b50 0x7f0fb814d8f0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.040+0000 7f0fbee00640 1 --2- 172.21.15.50:0/543463226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fb8072100 0x7f0fb81fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.040+0000 7f0fbee00640 1 -- 172.21.15.50:0/543463226 >> 172.21.15.50:0/543463226 conn(0x7f0fb80b2040 msgr2=0x7f0fb8154d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:11.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.040+0000 7f0fbee00640 1 -- 172.21.15.50:0/543463226 shutdown_connections 2024-09-21T06:11:11.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.040+0000 7f0fbee00640 1 -- 172.21.15.50:0/543463226 wait complete. 2024-09-21T06:11:11.060 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:11:11.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.220+0000 7f5a8d400640 1 -- 172.21.15.50:0/2029912739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a880b5c70 msgr2=0x7f5a8815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.220+0000 7f5a8d400640 1 --2- 172.21.15.50:0/2029912739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a880b5c70 0x7f5a8815bb60 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f5a88074b20 tx=0x7f5a7801c760 comp rx=0 tx=0).stop 2024-09-21T06:11:11.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.220+0000 7f5a8d400640 1 -- 172.21.15.50:0/2029912739 shutdown_connections 2024-09-21T06:11:11.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.220+0000 7f5a8d400640 1 --2- 172.21.15.50:0/2029912739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a880b5c70 0x7f5a8815bb60 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.220+0000 7f5a8d400640 1 -- 172.21.15.50:0/2029912739 >> 172.21.15.50:0/2029912739 conn(0x7f5a880b2040 msgr2=0x7f5a880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:11.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.220+0000 7f5a8d400640 1 -- 172.21.15.50:0/2029912739 shutdown_connections 2024-09-21T06:11:11.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.220+0000 7f5a8d400640 1 -- 172.21.15.50:0/2029912739 wait complete. 2024-09-21T06:11:11.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a8d400640 1 Processor -- start 2024-09-21T06:11:11.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a8d400640 1 -- start start 2024-09-21T06:11:11.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a8d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a880b4de0 0x7f5a8813bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a8d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880b5c70 0x7f5a88151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a8d400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a881527b0 0x7f5a881fc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a8d400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88169bd0 con 0x7f5a880b4de0 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a8d400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88169a50 con 0x7f5a880b5c70 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a8d400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5a88169d50 con 0x7f5a881527b0 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a86800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a880b4de0 0x7f5a8813bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a86800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a880b4de0 0x7f5a8813bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45242/0 (socket says 172.21.15.50:45242) 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a86800640 1 -- 172.21.15.50:0/3583477510 learned_addr learned my addr 172.21.15.50:0/3583477510 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a7fe00640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880b5c70 0x7f5a88151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a87200640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a881527b0 0x7f5a881fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a86800640 1 -- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a881527b0 msgr2=0x7f5a881fc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a86800640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a881527b0 0x7f5a881fc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a86800640 1 -- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880b5c70 msgr2=0x7f5a88151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a86800640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880b5c70 0x7f5a88151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a86800640 1 -- 172.21.15.50:0/3583477510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a881fc860 con 0x7f5a880b4de0 2024-09-21T06:11:11.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a7fe00640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880b5c70 0x7f5a88151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:11.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a87200640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a881527b0 0x7f5a881fc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:11.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a86800640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a880b4de0 0x7f5a8813bc20 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f5a78000c10 tx=0x7f5a78004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:11.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a7ea00640 1 -- 172.21.15.50:0/3583477510 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a780bb070 con 0x7f5a880b4de0 2024-09-21T06:11:11.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a7ea00640 1 -- 172.21.15.50:0/3583477510 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a78005030 con 0x7f5a880b4de0 2024-09-21T06:11:11.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a7ea00640 1 -- 172.21.15.50:0/3583477510 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a780b2040 con 0x7f5a880b4de0 2024-09-21T06:11:11.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.224+0000 7f5a8d400640 1 -- 172.21.15.50:0/3583477510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a881501e0 con 0x7f5a880b4de0 2024-09-21T06:11:11.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.228+0000 7f5a8d400640 1 -- 172.21.15.50:0/3583477510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a881fbb20 con 0x7f5a880b4de0 2024-09-21T06:11:11.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.228+0000 7f5a8d400640 1 -- 172.21.15.50:0/3583477510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a88150700 con 0x7f5a880b4de0 2024-09-21T06:11:11.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.228+0000 7f5a7ea00640 1 -- 172.21.15.50:0/3583477510 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f5a780be070 con 0x7f5a880b4de0 2024-09-21T06:11:11.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.228+0000 7f5a7ea00640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a64075180 0x7f5a64077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:11.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.228+0000 7f5a7ea00640 1 -- 172.21.15.50:0/3583477510 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4941+0+0 (secure 0 0 0) 0x7f5a7812ce70 con 0x7f5a880b4de0 2024-09-21T06:11:11.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.232+0000 7f5a7fe00640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a64075180 0x7f5a64077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:11.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.232+0000 7f5a7fe00640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a64075180 0x7f5a64077690 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5a700046c0 tx=0x7f5a70009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:11.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.236+0000 7f5a7ea00640 1 -- 172.21.15.50:0/3583477510 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5a88150700 con 0x7f5a880b4de0 2024-09-21T06:11:11.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.472+0000 7f5a8d400640 1 -- 172.21.15.50:0/3583477510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5a8815bb60 con 0x7f5a880b4de0 2024-09-21T06:11:11.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.472+0000 7f5a7ea00640 1 -- 172.21.15.50:0/3583477510 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v37) ==== 80+0+5304 (secure 0 0 0) 0x7f5a78016410 con 0x7f5a880b4de0 2024-09-21T06:11:11.470 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:11.470 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:11:11.470 INFO:teuthology.orchestra.run.smithi050.stdout: "auth_dump": [ 2024-09-21T06:11:11.470 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.470 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.a", 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mSKq0ARAA2egaHGYWhRUEq6oRl1CIMg==", 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.b", 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mwv0+DxAAY+eZPxv5uY9PnLl0asdr6w==", 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:11:11.471 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.c", 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5me6maCRAA4jvHgTLTKpb/mxzzG71SfQ==", 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.472 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.d", 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mzYnuFhAA/hEigjluXspzs/hyeWxBjQ==", 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.0", 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDIX+5mrZF/NhAA33M6ummG4r4yXU8/YOL5WA==", 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.473 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.1", 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDMX+5myVz5DxAAMYfTPep6ubD7Cxnfa9dMCg==", 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.474 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.2", 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDPX+5mA7ahIRAAf/osnY5l83p8ZWna0ErD2Q==", 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.3", 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDSX+5m+rDpNRAApGPZJIlAaA0DyjtPXPgoaQ==", 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:11:11.475 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.4", 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDYX+5mKL9SORAA21ITQUwUSoDUjXaD0Cwb3g==", 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.5", 2024-09-21T06:11:11.476 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDcX+5minRqFBAAMJOBQNvJFD4TNdKbPMLBig==", 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.6", 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDfX+5m6HadJhAAAv670Tvyi4IKMByFPiBlAw==", 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.477 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.7", 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDjX+5muPrrAhAAuRB7Muamiht+f7LvSmI6UQ==", 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.0", 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw==", 2024-09-21T06:11:11.478 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow r", 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.1", 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mo94GJBAAMBSb1NPb2L93vzvnAh59kw==", 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:11:11.479 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.2", 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5m4IrnHxAAFjER3ywuS+FlbuA7bPZVqw==", 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.3", 2024-09-21T06:11:11.480 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mysQeKBAAq5c4bD3hbsU1H7JabaMjwQ==", 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.admin", 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mcUXKHhAAV1Rs0OMNVPeAsRiAVt67jA==", 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:11:11.481 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.x", 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5m2F9WOBAAZuQTUyR7oTT01CRD1jo+/Q==", 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:11:11.482 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.483 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.y", 2024-09-21T06:11:11.483 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mEjC/MxAAHgwIu4CLDgFajwD2rL5WKA==", 2024-09-21T06:11:11.483 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:11:11.483 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:11:11.483 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:11:11.483 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:11:11.483 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.483 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.483 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-21T06:11:11.483 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:11:11.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.476+0000 7f5a8d400640 1 -- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a64075180 msgr2=0x7f5a64077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.476+0000 7f5a8d400640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a64075180 0x7f5a64077690 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5a700046c0 tx=0x7f5a70009040 comp rx=0 tx=0).stop 2024-09-21T06:11:11.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.476+0000 7f5a8d400640 1 -- 172.21.15.50:0/3583477510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a880b4de0 msgr2=0x7f5a8813bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.476+0000 7f5a8d400640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a880b4de0 0x7f5a8813bc20 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f5a78000c10 tx=0x7f5a78004040 comp rx=0 tx=0).stop 2024-09-21T06:11:11.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.476+0000 7f5a8d400640 1 -- 172.21.15.50:0/3583477510 shutdown_connections 2024-09-21T06:11:11.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.476+0000 7f5a8d400640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a881527b0 0x7f5a881fc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.476+0000 7f5a8d400640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880b5c70 0x7f5a88151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.476+0000 7f5a8d400640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a64075180 0x7f5a64077690 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.476+0000 7f5a8d400640 1 --2- 172.21.15.50:0/3583477510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a880b4de0 0x7f5a8813bc20 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.476+0000 7f5a8d400640 1 -- 172.21.15.50:0/3583477510 >> 172.21.15.50:0/3583477510 conn(0x7f5a880b2040 msgr2=0x7f5a88159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:11.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.476+0000 7f5a8d400640 1 -- 172.21.15.50:0/3583477510 shutdown_connections 2024-09-21T06:11:11.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.476+0000 7f5a8d400640 1 -- 172.21.15.50:0/3583477510 wait complete. 2024-09-21T06:11:11.493 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:11:11.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.656+0000 7f0d59400640 1 -- 172.21.15.50:0/2134264630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d541602a0 msgr2=0x7f0d541606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.656+0000 7f0d59400640 1 --2- 172.21.15.50:0/2134264630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d541602a0 0x7f0d541606b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009840 tx=0x7f0d3c01c820 comp rx=0 tx=0).stop 2024-09-21T06:11:11.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.656+0000 7f0d59400640 1 -- 172.21.15.50:0/2134264630 shutdown_connections 2024-09-21T06:11:11.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.656+0000 7f0d59400640 1 --2- 172.21.15.50:0/2134264630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d541602a0 0x7f0d541606b0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.656+0000 7f0d59400640 1 -- 172.21.15.50:0/2134264630 >> 172.21.15.50:0/2134264630 conn(0x7f0d540b2c40 msgr2=0x7f0d540b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:11.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.656+0000 7f0d59400640 1 -- 172.21.15.50:0/2134264630 shutdown_connections 2024-09-21T06:11:11.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.656+0000 7f0d59400640 1 -- 172.21.15.50:0/2134264630 wait complete. 2024-09-21T06:11:11.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.656+0000 7f0d59400640 1 Processor -- start 2024-09-21T06:11:11.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.656+0000 7f0d59400640 1 -- start start 2024-09-21T06:11:11.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d59400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d540cd2e0 0x7f0d540b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:11.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d59400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0d540cf1b0 0x7f0d54151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:11.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d59400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d541602a0 0x7f0d54151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:11.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d59400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5416ade0 con 0x7f0d541602a0 2024-09-21T06:11:11.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d59400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5416ac60 con 0x7f0d540cd2e0 2024-09-21T06:11:11.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d59400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5416af60 con 0x7f0d540cf1b0 2024-09-21T06:11:11.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d53200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d541602a0 0x7f0d54151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:11.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d53200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d541602a0 0x7f0d54151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45268/0 (socket says 172.21.15.50:45268) 2024-09-21T06:11:11.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d53200640 1 -- 172.21.15.50:0/4057461485 learned_addr learned my addr 172.21.15.50:0/4057461485 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:11.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d52800640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d540cd2e0 0x7f0d540b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:11.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d49e00640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0d540cf1b0 0x7f0d54151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:11.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d53200640 1 -- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0d540cf1b0 msgr2=0x7f0d54151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d53200640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0d540cf1b0 0x7f0d54151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d53200640 1 -- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d540cd2e0 msgr2=0x7f0d540b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d53200640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d540cd2e0 0x7f0d540b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d53200640 1 -- 172.21.15.50:0/4057461485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d541f9530 con 0x7f0d541602a0 2024-09-21T06:11:11.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d53200640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d541602a0 0x7f0d54151b90 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c730 tx=0x7f0d4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:11.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d4b400640 1 -- 172.21.15.50:0/4057461485 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c003b80 con 0x7f0d541602a0 2024-09-21T06:11:11.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d4b400640 1 -- 172.21.15.50:0/4057461485 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d4c003d40 con 0x7f0d541602a0 2024-09-21T06:11:11.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d59400640 1 -- 172.21.15.50:0/4057461485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d541f89b0 con 0x7f0d541602a0 2024-09-21T06:11:11.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d4b400640 1 -- 172.21.15.50:0/4057461485 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0052d0 con 0x7f0d541602a0 2024-09-21T06:11:11.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d59400640 1 -- 172.21.15.50:0/4057461485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d541f8f70 con 0x7f0d541602a0 2024-09-21T06:11:11.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.660+0000 7f0d59400640 1 -- 172.21.15.50:0/4057461485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d541f8720 con 0x7f0d541602a0 2024-09-21T06:11:11.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.664+0000 7f0d4b400640 1 -- 172.21.15.50:0/4057461485 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f0d4c005680 con 0x7f0d541602a0 2024-09-21T06:11:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.664+0000 7f0d4b400640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0d30075180 0x7f0d30077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.664+0000 7f0d4b400640 1 -- 172.21.15.50:0/4057461485 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4941+0+0 (secure 0 0 0) 0x7f0d4c093120 con 0x7f0d541602a0 2024-09-21T06:11:11.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.664+0000 7f0d52800640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0d30075180 0x7f0d30077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:11.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.664+0000 7f0d52800640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0d30075180 0x7f0d30077690 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0d44007d70 tx=0x7f0d44007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:11.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.668+0000 7f0d4b400640 1 -- 172.21.15.50:0/4057461485 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0d4c05f550 con 0x7f0d541602a0 2024-09-21T06:11:11.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d59400640 1 -- 172.21.15.50:0/4057461485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0d541606b0 con 0x7f0d541602a0 2024-09-21T06:11:11.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d4b400640 1 -- 172.21.15.50:0/4057461485 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v123) ==== 84+0+65 (secure 0 0 0) 0x7f0d4c063f80 con 0x7f0d541602a0 2024-09-21T06:11:11.845 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:11.845 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:11:11.846 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:11.846 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-09-21T06:11:11.846 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-09-21T06:11:11.846 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:11.846 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:11:11.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d59400640 1 -- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0d30075180 msgr2=0x7f0d30077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d59400640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0d30075180 0x7f0d30077690 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0d44007d70 tx=0x7f0d44007d00 comp rx=0 tx=0).stop 2024-09-21T06:11:11.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d59400640 1 -- 172.21.15.50:0/4057461485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d541602a0 msgr2=0x7f0d54151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d59400640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d541602a0 0x7f0d54151b90 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c730 tx=0x7f0d4c00cbf0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d59400640 1 -- 172.21.15.50:0/4057461485 shutdown_connections 2024-09-21T06:11:11.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d59400640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0d30075180 0x7f0d30077690 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d59400640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d541602a0 0x7f0d54151b90 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d59400640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0d540cf1b0 0x7f0d54151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d59400640 1 --2- 172.21.15.50:0/4057461485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d540cd2e0 0x7f0d540b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d59400640 1 -- 172.21.15.50:0/4057461485 >> 172.21.15.50:0/4057461485 conn(0x7f0d540b2c40 msgr2=0x7f0d540d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:11.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.848+0000 7f0d59400640 1 -- 172.21.15.50:0/4057461485 shutdown_connections 2024-09-21T06:11:11.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:11.852+0000 7f0d59400640 1 -- 172.21.15.50:0/4057461485 wait complete. 2024-09-21T06:11:11.868 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:11:11.868 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:11:12.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.036+0000 7f2d8a800640 1 -- 172.21.15.50:0/1078326151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d840749a0 msgr2=0x7f2d84074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:12.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.036+0000 7f2d8a800640 1 --2- 172.21.15.50:0/1078326151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d840749a0 0x7f2d84074db0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2d840cfc90 tx=0x7f2d7401c8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:12.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.036+0000 7f2d8a800640 1 -- 172.21.15.50:0/1078326151 shutdown_connections 2024-09-21T06:11:12.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.036+0000 7f2d8a800640 1 --2- 172.21.15.50:0/1078326151 >> v2:172.21.15.88:3300/0 conn(0x7f2d840b3340 0x7f2d840b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:12.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.036+0000 7f2d8a800640 1 --2- 172.21.15.50:0/1078326151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d840749a0 0x7f2d84074db0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:12.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.036+0000 7f2d8a800640 1 -- 172.21.15.50:0/1078326151 >> 172.21.15.50:0/1078326151 conn(0x7f2d840b2040 msgr2=0x7f2d840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:12.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.036+0000 7f2d8a800640 1 -- 172.21.15.50:0/1078326151 shutdown_connections 2024-09-21T06:11:12.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.036+0000 7f2d8a800640 1 -- 172.21.15.50:0/1078326151 wait complete. 2024-09-21T06:11:12.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.036+0000 7f2d8a800640 1 Processor -- start 2024-09-21T06:11:12.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.036+0000 7f2d8a800640 1 -- start start 2024-09-21T06:11:12.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d8a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d840749a0 0x7f2d840aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:12.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d8a800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2d840b3340 0x7f2d84144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:12.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d8a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d840cefa0 0x7f2d84145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:12.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d8a800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84169d20 con 0x7f2d840cefa0 2024-09-21T06:11:12.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d8a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84169ba0 con 0x7f2d840749a0 2024-09-21T06:11:12.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d8a800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2d84169ea0 con 0x7f2d840b3340 2024-09-21T06:11:12.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d83e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d840cefa0 0x7f2d84145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:12.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d83e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d840cefa0 0x7f2d84145300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45286/0 (socket says 172.21.15.50:45286) 2024-09-21T06:11:12.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d83e00640 1 -- 172.21.15.50:0/1005983991 learned_addr learned my addr 172.21.15.50:0/1005983991 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:12.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d7be00640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2d840b3340 0x7f2d84144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:12.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d83400640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d840749a0 0x7f2d840aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:12.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d83e00640 1 -- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2d840b3340 msgr2=0x7f2d84144dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:12.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d83e00640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2d840b3340 0x7f2d84144dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:12.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d83e00640 1 -- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d840749a0 msgr2=0x7f2d840aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:12.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d83e00640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d840749a0 0x7f2d840aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:12.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d83e00640 1 -- 172.21.15.50:0/1005983991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d84073700 con 0x7f2d840cefa0 2024-09-21T06:11:12.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d83e00640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d840cefa0 0x7f2d84145300 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00e940 tx=0x7f2d7c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:12.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d7b400640 1 -- 172.21.15.50:0/1005983991 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c004d30 con 0x7f2d840cefa0 2024-09-21T06:11:12.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d7b400640 1 -- 172.21.15.50:0/1005983991 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d7c00cb20 con 0x7f2d840cefa0 2024-09-21T06:11:12.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d7b400640 1 -- 172.21.15.50:0/1005983991 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c005020 con 0x7f2d840cefa0 2024-09-21T06:11:12.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d8a800640 1 -- 172.21.15.50:0/1005983991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d84142540 con 0x7f2d840cefa0 2024-09-21T06:11:12.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d8a800640 1 -- 172.21.15.50:0/1005983991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8412c820 con 0x7f2d840cefa0 2024-09-21T06:11:12.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.040+0000 7f2d8a800640 1 -- 172.21.15.50:0/1005983991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d84074db0 con 0x7f2d840cefa0 2024-09-21T06:11:12.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.044+0000 7f2d7b400640 1 -- 172.21.15.50:0/1005983991 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f2d7c0051c0 con 0x7f2d840cefa0 2024-09-21T06:11:12.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.044+0000 7f2d7b400640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2d60075180 0x7f2d60077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:12.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.044+0000 7f2d7b400640 1 -- 172.21.15.50:0/1005983991 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4941+0+0 (secure 0 0 0) 0x7f2d7c093b50 con 0x7f2d840cefa0 2024-09-21T06:11:12.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.044+0000 7f2d83400640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2d60075180 0x7f2d60077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:12.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.048+0000 7f2d83400640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2d60075180 0x7f2d60077690 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2d740af000 tx=0x7f2d740a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:12.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.048+0000 7f2d7b400640 1 -- 172.21.15.50:0/1005983991 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2d7c016030 con 0x7f2d840cefa0 2024-09-21T06:11:12.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:12.228+0000 7f2d8a800640 1 -- 172.21.15.50:0/1005983991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2d840b27a0 con 0x7f2d840cefa0 2024-09-21T06:11:13.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.052+0000 7f2d7b400640 1 -- 172.21.15.50:0/1005983991 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v124) ==== 138+0+0 (secure 0 0 0) 0x7f2d7c060010 con 0x7f2d840cefa0 2024-09-21T06:11:13.050 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-09-21T06:11:13.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.056+0000 7f2d8a800640 1 -- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2d60075180 msgr2=0x7f2d60077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.056+0000 7f2d8a800640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2d60075180 0x7f2d60077690 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2d740af000 tx=0x7f2d740a7040 comp rx=0 tx=0).stop 2024-09-21T06:11:13.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.056+0000 7f2d8a800640 1 -- 172.21.15.50:0/1005983991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d840cefa0 msgr2=0x7f2d84145300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.056+0000 7f2d8a800640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d840cefa0 0x7f2d84145300 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00e940 tx=0x7f2d7c00ee00 comp rx=0 tx=0).stop 2024-09-21T06:11:13.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.056+0000 7f2d8a800640 1 -- 172.21.15.50:0/1005983991 shutdown_connections 2024-09-21T06:11:13.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.056+0000 7f2d8a800640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2d60075180 0x7f2d60077690 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.056+0000 7f2d8a800640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d840cefa0 0x7f2d84145300 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.056+0000 7f2d8a800640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2d840b3340 0x7f2d84144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.056+0000 7f2d8a800640 1 --2- 172.21.15.50:0/1005983991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d840749a0 0x7f2d840aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.056+0000 7f2d8a800640 1 -- 172.21.15.50:0/1005983991 >> 172.21.15.50:0/1005983991 conn(0x7f2d840b2040 msgr2=0x7f2d8415ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:13.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.056+0000 7f2d8a800640 1 -- 172.21.15.50:0/1005983991 shutdown_connections 2024-09-21T06:11:13.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.056+0000 7f2d8a800640 1 -- 172.21.15.50:0/1005983991 wait complete. 2024-09-21T06:11:13.082 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:11:13.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.256+0000 7f7991800640 1 -- 172.21.15.50:0/1456015484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798c1527b0 msgr2=0x7f798c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.256+0000 7f7991800640 1 --2- 172.21.15.50:0/1456015484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798c1527b0 0x7f798c15bb60 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7974009900 tx=0x7f797401c8b0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.256+0000 7f7991800640 1 -- 172.21.15.50:0/1456015484 shutdown_connections 2024-09-21T06:11:13.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.256+0000 7f7991800640 1 --2- 172.21.15.50:0/1456015484 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f798c0b4de0 0x7f798c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.256+0000 7f7991800640 1 --2- 172.21.15.50:0/1456015484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798c1527b0 0x7f798c15bb60 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.256+0000 7f7991800640 1 -- 172.21.15.50:0/1456015484 >> 172.21.15.50:0/1456015484 conn(0x7f798c0b2040 msgr2=0x7f798c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:13.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.256+0000 7f7991800640 1 -- 172.21.15.50:0/1456015484 shutdown_connections 2024-09-21T06:11:13.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.256+0000 7f7991800640 1 -- 172.21.15.50:0/1456015484 wait complete. 2024-09-21T06:11:13.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7991800640 1 Processor -- start 2024-09-21T06:11:13.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7991800640 1 -- start start 2024-09-21T06:11:13.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7991800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f798c0b4de0 0x7f798c0cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:13.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7991800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798c0b5c70 0x7f798c1fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:13.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7991800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f798c1527b0 0x7f798c1fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:13.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7991800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f798c168980 con 0x7f798c0b5c70 2024-09-21T06:11:13.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7991800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f798c168800 con 0x7f798c0b4de0 2024-09-21T06:11:13.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7991800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f798c168b00 con 0x7f798c1527b0 2024-09-21T06:11:13.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7982000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798c0b5c70 0x7f798c1fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7982000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798c0b5c70 0x7f798c1fbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45310/0 (socket says 172.21.15.50:45310) 2024-09-21T06:11:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7982000640 1 -- 172.21.15.50:0/757469076 learned_addr learned my addr 172.21.15.50:0/757469076 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f798b400640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f798c1527b0 0x7f798c1fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f798aa00640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f798c0b4de0 0x7f798c0cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7982000640 1 -- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f798c1527b0 msgr2=0x7f798c1fc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7982000640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f798c1527b0 0x7f798c1fc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7982000640 1 -- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f798c0b4de0 msgr2=0x7f798c0cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7982000640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f798c0b4de0 0x7f798c0cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7982000640 1 -- 172.21.15.50:0/757469076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c1f8800 con 0x7f798c0b5c70 2024-09-21T06:11:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7982000640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798c0b5c70 0x7f798c1fbf90 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7974009850 tx=0x7f7974005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:13.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7983400640 1 -- 172.21.15.50:0/757469076 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79740b2040 con 0x7f798c0b5c70 2024-09-21T06:11:13.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7983400640 1 -- 172.21.15.50:0/757469076 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7974003600 con 0x7f798c0b5c70 2024-09-21T06:11:13.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7991800640 1 -- 172.21.15.50:0/757469076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798c142810 con 0x7f798c0b5c70 2024-09-21T06:11:13.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7983400640 1 -- 172.21.15.50:0/757469076 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974003a30 con 0x7f798c0b5c70 2024-09-21T06:11:13.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.260+0000 7f7991800640 1 -- 172.21.15.50:0/757469076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798c142dd0 con 0x7f798c0b5c70 2024-09-21T06:11:13.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.264+0000 7f7991800640 1 -- 172.21.15.50:0/757469076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f798c0b51f0 con 0x7f798c0b5c70 2024-09-21T06:11:13.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.264+0000 7f7983400640 1 -- 172.21.15.50:0/757469076 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f79740bf020 con 0x7f798c0b5c70 2024-09-21T06:11:13.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.264+0000 7f7983400640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7968075180 0x7f7968077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:13.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.268+0000 7f7983400640 1 -- 172.21.15.50:0/757469076 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5311+0+0 (secure 0 0 0) 0x7f797412ddf0 con 0x7f798c0b5c70 2024-09-21T06:11:13.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.268+0000 7f798aa00640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7968075180 0x7f7968077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:13.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.268+0000 7f798aa00640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7968075180 0x7f7968077690 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f797c007fd0 tx=0x7f797c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:13.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.268+0000 7f7983400640 1 -- 172.21.15.50:0/757469076 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f79740c6020 con 0x7f798c0b5c70 2024-09-21T06:11:13.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:13.448+0000 7f7991800640 1 -- 172.21.15.50:0/757469076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f798c0b0d80 con 0x7f798c0b5c70 2024-09-21T06:11:14.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.060+0000 7f7983400640 1 -- 172.21.15.50:0/757469076 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v125) ==== 170+0+0 (secure 0 0 0) 0x7f79740164d0 con 0x7f798c0b5c70 2024-09-21T06:11:14.058 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-09-21T06:11:14.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.064+0000 7f7991800640 1 -- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7968075180 msgr2=0x7f7968077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.064+0000 7f7991800640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7968075180 0x7f7968077690 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f797c007fd0 tx=0x7f797c009040 comp rx=0 tx=0).stop 2024-09-21T06:11:14.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.064+0000 7f7991800640 1 -- 172.21.15.50:0/757469076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798c0b5c70 msgr2=0x7f798c1fbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.064+0000 7f7991800640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798c0b5c70 0x7f798c1fbf90 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7974009850 tx=0x7f7974005760 comp rx=0 tx=0).stop 2024-09-21T06:11:14.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.064+0000 7f7991800640 1 -- 172.21.15.50:0/757469076 shutdown_connections 2024-09-21T06:11:14.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.064+0000 7f7991800640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f798c1527b0 0x7f798c1fc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.064+0000 7f7991800640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7968075180 0x7f7968077690 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.064+0000 7f7991800640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798c0b5c70 0x7f798c1fbf90 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.064+0000 7f7991800640 1 --2- 172.21.15.50:0/757469076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f798c0b4de0 0x7f798c0cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.064+0000 7f7991800640 1 -- 172.21.15.50:0/757469076 >> 172.21.15.50:0/757469076 conn(0x7f798c0b2040 msgr2=0x7f798c151a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:14.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.064+0000 7f7991800640 1 -- 172.21.15.50:0/757469076 shutdown_connections 2024-09-21T06:11:14.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.064+0000 7f7991800640 1 -- 172.21.15.50:0/757469076 wait complete. 2024-09-21T06:11:14.103 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:11:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.268+0000 7fe268c00640 1 -- 172.21.15.50:0/221017564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2641527b0 msgr2=0x7fe264074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.268+0000 7fe268c00640 1 --2- 172.21.15.50:0/221017564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2641527b0 0x7fe264074e30 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe264071cb0 tx=0x7fe25001c800 comp rx=0 tx=0).stop 2024-09-21T06:11:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.268+0000 7fe268c00640 1 -- 172.21.15.50:0/221017564 shutdown_connections 2024-09-21T06:11:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.268+0000 7fe268c00640 1 --2- 172.21.15.50:0/221017564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2641527b0 0x7fe264074e30 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.268+0000 7fe268c00640 1 --2- 172.21.15.50:0/221017564 >> v2:172.21.15.88:3300/0 conn(0x7fe2640b5c70 0x7fe26415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.268+0000 7fe268c00640 1 -- 172.21.15.50:0/221017564 >> 172.21.15.50:0/221017564 conn(0x7fe2640b2040 msgr2=0x7fe2640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.268+0000 7fe268c00640 1 -- 172.21.15.50:0/221017564 shutdown_connections 2024-09-21T06:11:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.268+0000 7fe268c00640 1 -- 172.21.15.50:0/221017564 wait complete. 2024-09-21T06:11:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe268c00640 1 Processor -- start 2024-09-21T06:11:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe268c00640 1 -- start start 2024-09-21T06:11:14.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe268c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2640b4de0 0x7fe2641f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:14.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe268c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2640b5c70 0x7fe2641fbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:14.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe268c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe2641527b0 0x7fe264158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:14.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe268c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe264168a80 con 0x7fe2640b4de0 2024-09-21T06:11:14.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe268c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe264168900 con 0x7fe2640b5c70 2024-09-21T06:11:14.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe268c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe264168c00 con 0x7fe2641527b0 2024-09-21T06:11:14.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe261e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2640b4de0 0x7fe2641f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:14.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe261e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2640b4de0 0x7fe2641f9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45318/0 (socket says 172.21.15.50:45318) 2024-09-21T06:11:14.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe261e00640 1 -- 172.21.15.50:0/168592221 learned_addr learned my addr 172.21.15.50:0/168592221 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:14.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe261400640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2640b5c70 0x7fe2641fbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:14.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe262800640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe2641527b0 0x7fe264158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:14.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe261e00640 1 -- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe2641527b0 msgr2=0x7fe264158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe261e00640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe2641527b0 0x7fe264158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe261e00640 1 -- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2640b5c70 msgr2=0x7fe2641fbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe261e00640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2640b5c70 0x7fe2641fbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe261e00640 1 -- 172.21.15.50:0/168592221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26415c8b0 con 0x7fe2640b4de0 2024-09-21T06:11:14.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.272+0000 7fe261400640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2640b5c70 0x7fe2641fbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:14.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.276+0000 7fe261e00640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2640b4de0 0x7fe2641f9cc0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe25400d950 tx=0x7fe25400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:14.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.276+0000 7fe25a000640 1 -- 172.21.15.50:0/168592221 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254017070 con 0x7fe2640b4de0 2024-09-21T06:11:14.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.276+0000 7fe25a000640 1 -- 172.21.15.50:0/168592221 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe254009d20 con 0x7fe2640b4de0 2024-09-21T06:11:14.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.276+0000 7fe25a000640 1 -- 172.21.15.50:0/168592221 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2540106c0 con 0x7fe2640b4de0 2024-09-21T06:11:14.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.276+0000 7fe268c00640 1 -- 172.21.15.50:0/168592221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe264159150 con 0x7fe2640b4de0 2024-09-21T06:11:14.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.276+0000 7fe268c00640 1 -- 172.21.15.50:0/168592221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2641fc9a0 con 0x7fe2640b4de0 2024-09-21T06:11:14.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.276+0000 7fe268c00640 1 -- 172.21.15.50:0/168592221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26415bb60 con 0x7fe2640b4de0 2024-09-21T06:11:14.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.276+0000 7fe25a000640 1 -- 172.21.15.50:0/168592221 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7fe25401a070 con 0x7fe2640b4de0 2024-09-21T06:11:14.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.276+0000 7fe25a000640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe244075180 0x7fe244077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:14.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.280+0000 7fe25a000640 1 -- 172.21.15.50:0/168592221 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5693+0+0 (secure 0 0 0) 0x7fe254093880 con 0x7fe2640b4de0 2024-09-21T06:11:14.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.280+0000 7fe261400640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe244075180 0x7fe244077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:14.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.280+0000 7fe261400640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe244075180 0x7fe244077690 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe2500af000 tx=0x7fe25001b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:14.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.280+0000 7fe25a000640 1 -- 172.21.15.50:0/168592221 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe254060380 con 0x7fe2640b4de0 2024-09-21T06:11:14.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:14.492+0000 7fe268c00640 1 -- 172.21.15.50:0/168592221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe2640b24f0 con 0x7fe2640b4de0 2024-09-21T06:11:15.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.064+0000 7fe25a000640 1 -- 172.21.15.50:0/168592221 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v80) ==== 174+0+0 (secure 0 0 0) 0x7fe25405fc80 con 0x7fe2640b4de0 2024-09-21T06:11:15.060 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 18 and data pool 19 2024-09-21T06:11:15.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.068+0000 7fe268c00640 1 -- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe244075180 msgr2=0x7fe244077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.068+0000 7fe268c00640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe244075180 0x7fe244077690 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe2500af000 tx=0x7fe25001b040 comp rx=0 tx=0).stop 2024-09-21T06:11:15.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.068+0000 7fe268c00640 1 -- 172.21.15.50:0/168592221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2640b4de0 msgr2=0x7fe2641f9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.068+0000 7fe268c00640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2640b4de0 0x7fe2641f9cc0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe25400d950 tx=0x7fe25400de10 comp rx=0 tx=0).stop 2024-09-21T06:11:15.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.068+0000 7fe268c00640 1 -- 172.21.15.50:0/168592221 shutdown_connections 2024-09-21T06:11:15.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.068+0000 7fe268c00640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe2641527b0 0x7fe264158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.068+0000 7fe268c00640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2640b5c70 0x7fe2641fbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.068+0000 7fe268c00640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe244075180 0x7fe244077690 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.068+0000 7fe268c00640 1 --2- 172.21.15.50:0/168592221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2640b4de0 0x7fe2641f9cc0 secure :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe25400d950 tx=0x7fe25400de10 comp rx=0 tx=0).stop 2024-09-21T06:11:15.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.068+0000 7fe268c00640 1 -- 172.21.15.50:0/168592221 >> 172.21.15.50:0/168592221 conn(0x7fe2640b2040 msgr2=0x7fe2640b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:15.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.068+0000 7fe268c00640 1 -- 172.21.15.50:0/168592221 shutdown_connections 2024-09-21T06:11:15.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.068+0000 7fe268c00640 1 -- 172.21.15.50:0/168592221 wait complete. 2024-09-21T06:11:15.120 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:11:15.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.284+0000 7f1a2b200640 1 -- 172.21.15.50:0/3902006090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a2c155a00 msgr2=0x7f1a2c15f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.284+0000 7f1a2b200640 1 --2- 172.21.15.50:0/3902006090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a2c155a00 0x7f1a2c15f070 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009900 tx=0x7f1a1c01c7e0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.284+0000 7f1a2b200640 1 -- 172.21.15.50:0/3902006090 shutdown_connections 2024-09-21T06:11:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.284+0000 7f1a2b200640 1 --2- 172.21.15.50:0/3902006090 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a2c0b2300 0x7f1a2c0b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.284+0000 7f1a2b200640 1 --2- 172.21.15.50:0/3902006090 >> v2:172.21.15.88:3300/0 conn(0x7f1a2c15f5b0 0x7f1a2c0b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.284+0000 7f1a2b200640 1 --2- 172.21.15.50:0/3902006090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a2c155a00 0x7f1a2c15f070 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.284+0000 7f1a2b200640 1 -- 172.21.15.50:0/3902006090 >> 172.21.15.50:0/3902006090 conn(0x7f1a2c06cdf0 msgr2=0x7f1a2c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.284+0000 7f1a2b200640 1 -- 172.21.15.50:0/3902006090 shutdown_connections 2024-09-21T06:11:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.284+0000 7f1a2b200640 1 -- 172.21.15.50:0/3902006090 wait complete. 2024-09-21T06:11:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.284+0000 7f1a2b200640 1 Processor -- start 2024-09-21T06:11:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.284+0000 7f1a2b200640 1 -- start start 2024-09-21T06:11:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a2b200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a2c0b2300 0x7f1a2c0aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a2b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a2c155a00 0x7f1a2c0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a2b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a2c15f5b0 0x7f1a2c0b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a2b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c072660 con 0x7f1a2c155a00 2024-09-21T06:11:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a2b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c0724e0 con 0x7f1a2c15f5b0 2024-09-21T06:11:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a2b200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1a2c0727e0 con 0x7f1a2c0b2300 2024-09-21T06:11:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a21400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a2c155a00 0x7f1a2c0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a21400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a2c155a00 0x7f1a2c0cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45336/0 (socket says 172.21.15.50:45336) 2024-09-21T06:11:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a21400640 1 -- 172.21.15.50:0/1662513035 learned_addr learned my addr 172.21.15.50:0/1662513035 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a29e00640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a2c0b2300 0x7f1a2c0aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a2a800640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a2c15f5b0 0x7f1a2c0b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a21400640 1 -- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a2c0b2300 msgr2=0x7f1a2c0aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a21400640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a2c0b2300 0x7f1a2c0aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a21400640 1 -- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a2c15f5b0 msgr2=0x7f1a2c0b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a21400640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a2c15f5b0 0x7f1a2c0b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a21400640 1 -- 172.21.15.50:0/1662513035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2c0ce500 con 0x7f1a2c155a00 2024-09-21T06:11:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a29e00640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a2c0b2300 0x7f1a2c0aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a2a800640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a2c15f5b0 0x7f1a2c0b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a21400640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a2c155a00 0x7f1a2c0cd8b0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400d950 tx=0x7f1a1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a22a00640 1 -- 172.21.15.50:0/1662513035 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14017070 con 0x7f1a2c155a00 2024-09-21T06:11:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a22a00640 1 -- 172.21.15.50:0/1662513035 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a14009d20 con 0x7f1a2c155a00 2024-09-21T06:11:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a2b200640 1 -- 172.21.15.50:0/1662513035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2c0ce7e0 con 0x7f1a2c155a00 2024-09-21T06:11:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a22a00640 1 -- 172.21.15.50:0/1662513035 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a140106c0 con 0x7f1a2c155a00 2024-09-21T06:11:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a2b200640 1 -- 172.21.15.50:0/1662513035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a2c0ceda0 con 0x7f1a2c155a00 2024-09-21T06:11:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.288+0000 7f1a2b200640 1 -- 172.21.15.50:0/1662513035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a2c0ce270 con 0x7f1a2c155a00 2024-09-21T06:11:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.292+0000 7f1a22a00640 1 -- 172.21.15.50:0/1662513035 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f1a1401a070 con 0x7f1a2c155a00 2024-09-21T06:11:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.292+0000 7f1a22a00640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1a08075180 0x7f1a08077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.292+0000 7f1a22a00640 1 -- 172.21.15.50:0/1662513035 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5793+0+0 (secure 0 0 0) 0x7f1a140934c0 con 0x7f1a2c155a00 2024-09-21T06:11:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.292+0000 7f1a29e00640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1a08075180 0x7f1a08077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.292+0000 7f1a29e00640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1a08075180 0x7f1a08077690 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0af000 tx=0x7f1a1c01b020 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.296+0000 7f1a22a00640 1 -- 172.21.15.50:0/1662513035 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1a1405ff40 con 0x7f1a2c155a00 2024-09-21T06:11:15.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.476+0000 7f1a2b200640 1 -- 172.21.15.50:0/1662513035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a2c072d60 con 0x7f1a2c155a00 2024-09-21T06:11:15.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.476+0000 7f1a22a00640 1 -- 172.21.15.50:0/1662513035 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16837 (secure 0 0 0) 0x7f1a1405f840 con 0x7f1a2c155a00 2024-09-21T06:11:15.475 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:15.476 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":126,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:11:14.496785+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:11:12.241790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T06:11:13.462910+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:11:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.480+0000 7f1a2b200640 1 -- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1a08075180 msgr2=0x7f1a08077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.480+0000 7f1a2b200640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1a08075180 0x7f1a08077690 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0af000 tx=0x7f1a1c01b020 comp rx=0 tx=0).stop 2024-09-21T06:11:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.480+0000 7f1a2b200640 1 -- 172.21.15.50:0/1662513035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a2c155a00 msgr2=0x7f1a2c0cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.480+0000 7f1a2b200640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a2c155a00 0x7f1a2c0cd8b0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400d950 tx=0x7f1a1400de10 comp rx=0 tx=0).stop 2024-09-21T06:11:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.480+0000 7f1a2b200640 1 -- 172.21.15.50:0/1662513035 shutdown_connections 2024-09-21T06:11:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.480+0000 7f1a2b200640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a2c15f5b0 0x7f1a2c0b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.480+0000 7f1a2b200640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1a08075180 0x7f1a08077690 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.480+0000 7f1a2b200640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a2c155a00 0x7f1a2c0cd8b0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.480+0000 7f1a2b200640 1 --2- 172.21.15.50:0/1662513035 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1a2c0b2300 0x7f1a2c0aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.480+0000 7f1a2b200640 1 -- 172.21.15.50:0/1662513035 >> 172.21.15.50:0/1662513035 conn(0x7f1a2c06cdf0 msgr2=0x7f1a2c1533f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.480+0000 7f1a2b200640 1 -- 172.21.15.50:0/1662513035 shutdown_connections 2024-09-21T06:11:15.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.480+0000 7f1a2b200640 1 -- 172.21.15.50:0/1662513035 wait complete. 2024-09-21T06:11:15.498 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:11:15.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.660+0000 7f570dc00640 1 -- 172.21.15.50:0/3996230250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57080cefa0 msgr2=0x7f57080cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.660+0000 7f570dc00640 1 --2- 172.21.15.50:0/3996230250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57080cefa0 0x7f57080cf3b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009840 tx=0x7f56f001c820 comp rx=0 tx=0).stop 2024-09-21T06:11:15.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.660+0000 7f570dc00640 1 -- 172.21.15.50:0/3996230250 shutdown_connections 2024-09-21T06:11:15.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.660+0000 7f570dc00640 1 --2- 172.21.15.50:0/3996230250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57080cefa0 0x7f57080cf3b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.660+0000 7f570dc00640 1 --2- 172.21.15.50:0/3996230250 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57080749a0 0x7f5708074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.660+0000 7f570dc00640 1 -- 172.21.15.50:0/3996230250 >> 172.21.15.50:0/3996230250 conn(0x7f57080b2040 msgr2=0x7f57080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:15.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.660+0000 7f570dc00640 1 -- 172.21.15.50:0/3996230250 shutdown_connections 2024-09-21T06:11:15.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.660+0000 7f570dc00640 1 -- 172.21.15.50:0/3996230250 wait complete. 2024-09-21T06:11:15.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.660+0000 7f570dc00640 1 Processor -- start 2024-09-21T06:11:15.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.660+0000 7f570dc00640 1 -- start start 2024-09-21T06:11:15.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f570dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57080749a0 0x7f570813ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:15.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f570dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57080cefa0 0x7f5708158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:15.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f570dc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5708160330 0x7f570806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:15.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f570dc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f570816a060 con 0x7f57080cefa0 2024-09-21T06:11:15.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f570dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5708169ee0 con 0x7f57080749a0 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f570dc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f570816a1e0 con 0x7f5708160330 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f56ffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57080cefa0 0x7f5708158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f56ffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57080cefa0 0x7f5708158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45348/0 (socket says 172.21.15.50:45348) 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f56ffe00640 1 -- 172.21.15.50:0/2899043271 learned_addr learned my addr 172.21.15.50:0/2899043271 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f5706a00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57080749a0 0x7f570813ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f5707400640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5708160330 0x7f570806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f56ffe00640 1 -- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5708160330 msgr2=0x7f570806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f56ffe00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5708160330 0x7f570806e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f56ffe00640 1 -- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57080749a0 msgr2=0x7f570813ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f56ffe00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57080749a0 0x7f570813ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f56ffe00640 1 -- 172.21.15.50:0/2899043271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570813b190 con 0x7f57080cefa0 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f5707400640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5708160330 0x7f570806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f56ffe00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57080cefa0 0x7f5708158a30 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009790 tx=0x7f56f0005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:15.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f56fea00640 1 -- 172.21.15.50:0/2899043271 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f00b2040 con 0x7f57080cefa0 2024-09-21T06:11:15.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f570dc00640 1 -- 172.21.15.50:0/2899043271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5708158f70 con 0x7f57080cefa0 2024-09-21T06:11:15.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f56fea00640 1 -- 172.21.15.50:0/2899043271 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56f0004330 con 0x7f57080cefa0 2024-09-21T06:11:15.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f56fea00640 1 -- 172.21.15.50:0/2899043271 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0003800 con 0x7f57080cefa0 2024-09-21T06:11:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f570dc00640 1 -- 172.21.15.50:0/2899043271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f570813b740 con 0x7f57080cefa0 2024-09-21T06:11:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.664+0000 7f570dc00640 1 -- 172.21.15.50:0/2899043271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5708074db0 con 0x7f57080cefa0 2024-09-21T06:11:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.668+0000 7f56fea00640 1 -- 172.21.15.50:0/2899043271 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f56f00bd060 con 0x7f57080cefa0 2024-09-21T06:11:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.668+0000 7f56fea00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f56e4075180 0x7f56e4077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.668+0000 7f56fea00640 1 -- 172.21.15.50:0/2899043271 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5793+0+0 (secure 0 0 0) 0x7f56f012cc10 con 0x7f57080cefa0 2024-09-21T06:11:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.668+0000 7f5706a00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f56e4075180 0x7f56e4077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:15.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.668+0000 7f5706a00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f56e4075180 0x7f56e4077690 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f56f80096c0 tx=0x7f56f8009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:15.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.672+0000 7f56fea00640 1 -- 172.21.15.50:0/2899043271 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f56f0016410 con 0x7f57080cefa0 2024-09-21T06:11:15.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.852+0000 7f570dc00640 1 -- 172.21.15.50:0/2899043271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f570815afc0 con 0x7f57080cefa0 2024-09-21T06:11:15.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.852+0000 7f56fea00640 1 -- 172.21.15.50:0/2899043271 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16837 (secure 0 0 0) 0x7f56f00f96a0 con 0x7f57080cefa0 2024-09-21T06:11:15.849 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:15.850 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":126,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:11:14.496785+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:11:12.241790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T06:11:13.462910+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:11:15.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.856+0000 7f570dc00640 1 -- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f56e4075180 msgr2=0x7f56e4077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.856+0000 7f570dc00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f56e4075180 0x7f56e4077690 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f56f80096c0 tx=0x7f56f8009260 comp rx=0 tx=0).stop 2024-09-21T06:11:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.856+0000 7f570dc00640 1 -- 172.21.15.50:0/2899043271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57080cefa0 msgr2=0x7f5708158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.856+0000 7f570dc00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57080cefa0 0x7f5708158a30 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009790 tx=0x7f56f0005d20 comp rx=0 tx=0).stop 2024-09-21T06:11:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.856+0000 7f570dc00640 1 -- 172.21.15.50:0/2899043271 shutdown_connections 2024-09-21T06:11:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.856+0000 7f570dc00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5708160330 0x7f570806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.856+0000 7f570dc00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f56e4075180 0x7f56e4077690 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.856+0000 7f570dc00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57080cefa0 0x7f5708158a30 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.856+0000 7f570dc00640 1 --2- 172.21.15.50:0/2899043271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57080749a0 0x7f570813ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.856+0000 7f570dc00640 1 -- 172.21.15.50:0/2899043271 >> 172.21.15.50:0/2899043271 conn(0x7f57080b2040 msgr2=0x7f57080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.856+0000 7f570dc00640 1 -- 172.21.15.50:0/2899043271 shutdown_connections 2024-09-21T06:11:15.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:15.856+0000 7f570dc00640 1 -- 172.21.15.50:0/2899043271 wait complete. 2024-09-21T06:11:15.873 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:11:16.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.036+0000 7fa6fb600640 1 -- 172.21.15.50:0/2651422047 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f4155b50 msgr2=0x7fa6f40741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:16.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.036+0000 7fa6fb600640 1 --2- 172.21.15.50:0/2651422047 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f4155b50 0x7fa6f40741a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009bf0 tx=0x7fa6ec01ca70 comp rx=0 tx=0).stop 2024-09-21T06:11:16.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.036+0000 7fa6fb600640 1 -- 172.21.15.50:0/2651422047 shutdown_connections 2024-09-21T06:11:16.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.036+0000 7fa6fb600640 1 --2- 172.21.15.50:0/2651422047 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f4155b50 0x7fa6f40741a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.036+0000 7fa6fb600640 1 -- 172.21.15.50:0/2651422047 >> 172.21.15.50:0/2651422047 conn(0x7fa6f40b2040 msgr2=0x7fa6f40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:16.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.036+0000 7fa6fb600640 1 -- 172.21.15.50:0/2651422047 shutdown_connections 2024-09-21T06:11:16.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.036+0000 7fa6fb600640 1 -- 172.21.15.50:0/2651422047 wait complete. 2024-09-21T06:11:16.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6fb600640 1 Processor -- start 2024-09-21T06:11:16.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6fb600640 1 -- start start 2024-09-21T06:11:16.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6fb600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6f4072100 0x7fa6f40af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:16.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6fb600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f4155b50 0x7fa6f40ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:16.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6fb600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f40aacc0 0x7fa6f414bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:16.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6fb600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4169e70 con 0x7fa6f4072100 2024-09-21T06:11:16.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6fb600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4169cf0 con 0x7fa6f4155b50 2024-09-21T06:11:16.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6fb600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f4169ff0 con 0x7fa6f40aacc0 2024-09-21T06:11:16.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6f8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6f4072100 0x7fa6f40af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:16.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6f8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6f4072100 0x7fa6f40af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45352/0 (socket says 172.21.15.50:45352) 2024-09-21T06:11:16.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6f8c00640 1 -- 172.21.15.50:0/2582134637 learned_addr learned my addr 172.21.15.50:0/2582134637 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:16.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6f9600640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f40aacc0 0x7fa6f414bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:16.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6f3e00640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f4155b50 0x7fa6f40ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:16.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6f8c00640 1 -- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f40aacc0 msgr2=0x7fa6f414bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:16.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6f8c00640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f40aacc0 0x7fa6f414bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6f8c00640 1 -- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f4155b50 msgr2=0x7fa6f40ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:16.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6f8c00640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f4155b50 0x7fa6f40ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.040+0000 7fa6f8c00640 1 -- 172.21.15.50:0/2582134637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f40b57d0 con 0x7fa6f4072100 2024-09-21T06:11:16.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.044+0000 7fa6f8c00640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6f4072100 0x7fa6f40af1a0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c400 tx=0x7fa6e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:16.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.044+0000 7fa6f0c00640 1 -- 172.21.15.50:0/2582134637 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8017070 con 0x7fa6f4072100 2024-09-21T06:11:16.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.044+0000 7fa6f0c00640 1 -- 172.21.15.50:0/2582134637 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6e8005400 con 0x7fa6f4072100 2024-09-21T06:11:16.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.044+0000 7fa6f0c00640 1 -- 172.21.15.50:0/2582134637 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8003860 con 0x7fa6f4072100 2024-09-21T06:11:16.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.044+0000 7fa6fb600640 1 -- 172.21.15.50:0/2582134637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f40ab4a0 con 0x7fa6f4072100 2024-09-21T06:11:16.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.044+0000 7fa6fb600640 1 -- 172.21.15.50:0/2582134637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f414e350 con 0x7fa6f4072100 2024-09-21T06:11:16.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.044+0000 7fa6fb600640 1 -- 172.21.15.50:0/2582134637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f40b4de0 con 0x7fa6f4072100 2024-09-21T06:11:16.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.044+0000 7fa6f0c00640 1 -- 172.21.15.50:0/2582134637 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7fa6e801a070 con 0x7fa6f4072100 2024-09-21T06:11:16.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.044+0000 7fa6f0c00640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6d4075180 0x7fa6d4077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:16.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.044+0000 7fa6f0c00640 1 -- 172.21.15.50:0/2582134637 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5793+0+0 (secure 0 0 0) 0x7fa6e8092630 con 0x7fa6f4072100 2024-09-21T06:11:16.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.048+0000 7fa6f3e00640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6d4075180 0x7fa6d4077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:16.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.048+0000 7fa6f3e00640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6d4075180 0x7fa6d4077690 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa6f415b3d0 tx=0x7fa6e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:16.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.052+0000 7fa6f0c00640 1 -- 172.21.15.50:0/2582134637 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa6f40ab4a0 con 0x7fa6f4072100 2024-09-21T06:11:16.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:16.264+0000 7fa6fb600640 1 -- 172.21.15.50:0/2582134637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa6f4152960 con 0x7fa6f4072100 2024-09-21T06:11:17.041 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-09-21T06:11:17.044+0000 7f4739a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:11:17.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.096+0000 7fa6f0c00640 1 -- 172.21.15.50:0/2582134637 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) ==== 118+0+0 (secure 0 0 0) 0x7fa6e805ea60 con 0x7fa6f4072100 2024-09-21T06:11:17.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.100+0000 7fa6fb600640 1 -- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6d4075180 msgr2=0x7fa6d4077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.100+0000 7fa6fb600640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6d4075180 0x7fa6d4077690 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa6f415b3d0 tx=0x7fa6e0009040 comp rx=0 tx=0).stop 2024-09-21T06:11:17.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.100+0000 7fa6fb600640 1 -- 172.21.15.50:0/2582134637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6f4072100 msgr2=0x7fa6f40af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.100+0000 7fa6fb600640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6f4072100 0x7fa6f40af1a0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c400 tx=0x7fa6e800c8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.100+0000 7fa6fb600640 1 -- 172.21.15.50:0/2582134637 shutdown_connections 2024-09-21T06:11:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.100+0000 7fa6fb600640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa6f40aacc0 0x7fa6f414bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.100+0000 7fa6fb600640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f4155b50 0x7fa6f40ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.100+0000 7fa6fb600640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa6d4075180 0x7fa6d4077690 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.100+0000 7fa6fb600640 1 --2- 172.21.15.50:0/2582134637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6f4072100 0x7fa6f40af1a0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.100+0000 7fa6fb600640 1 -- 172.21.15.50:0/2582134637 >> 172.21.15.50:0/2582134637 conn(0x7fa6f40b2040 msgr2=0x7fa6f40b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.100+0000 7fa6fb600640 1 -- 172.21.15.50:0/2582134637 shutdown_connections 2024-09-21T06:11:17.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.104+0000 7fa6fb600640 1 -- 172.21.15.50:0/2582134637 wait complete. 2024-09-21T06:11:17.124 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:11:17.124 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:11:17.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.292+0000 7f4452400640 1 -- 172.21.15.50:0/1421771107 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f444c160060 msgr2=0x7f444c074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.292+0000 7f4452400640 1 --2- 172.21.15.50:0/1421771107 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f444c160060 0x7f444c074240 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f444c0cf890 tx=0x7f443401c830 comp rx=0 tx=0).stop 2024-09-21T06:11:17.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.292+0000 7f4452400640 1 -- 172.21.15.50:0/1421771107 shutdown_connections 2024-09-21T06:11:17.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.292+0000 7f4452400640 1 --2- 172.21.15.50:0/1421771107 >> v2:172.21.15.88:3300/0 conn(0x7f444c160470 0x7f444c074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.292+0000 7f4452400640 1 --2- 172.21.15.50:0/1421771107 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f444c160060 0x7f444c074240 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.292+0000 7f4452400640 1 -- 172.21.15.50:0/1421771107 >> 172.21.15.50:0/1421771107 conn(0x7f444c0b4560 msgr2=0x7f444c0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:17.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.292+0000 7f4452400640 1 -- 172.21.15.50:0/1421771107 shutdown_connections 2024-09-21T06:11:17.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.292+0000 7f4452400640 1 -- 172.21.15.50:0/1421771107 wait complete. 2024-09-21T06:11:17.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4452400640 1 Processor -- start 2024-09-21T06:11:17.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4452400640 1 -- start start 2024-09-21T06:11:17.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4452400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f444c1536f0 0x7f444c149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:17.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4452400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f444c160060 0x7f444c153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:17.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4452400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f444c160470 0x7f444c0d1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:17.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4452400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f444c1696e0 con 0x7f444c1536f0 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4452400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f444c169560 con 0x7f444c160470 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4452400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f444c169860 con 0x7f444c160060 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4451000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f444c1536f0 0x7f444c149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4451000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f444c1536f0 0x7f444c149f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60990/0 (socket says 172.21.15.50:60990) 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4451000640 1 -- 172.21.15.50:0/1844027314 learned_addr learned my addr 172.21.15.50:0/1844027314 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4443e00640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f444c160060 0x7f444c153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.296+0000 7f4451a00640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f444c160470 0x7f444c0d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4451000640 1 -- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f444c160060 msgr2=0x7f444c153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4451000640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f444c160060 0x7f444c153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4451000640 1 -- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f444c160470 msgr2=0x7f444c0d1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4451000640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f444c160470 0x7f444c0d1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4451000640 1 -- 172.21.15.50:0/1844027314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c14a4d0 con 0x7f444c1536f0 2024-09-21T06:11:17.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4451a00640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f444c160470 0x7f444c0d1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:17.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4451000640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f444c1536f0 0x7f444c149f90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f443c00b440 tx=0x7f443c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:17.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4449600640 1 -- 172.21.15.50:0/1844027314 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c015020 con 0x7f444c1536f0 2024-09-21T06:11:17.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4449600640 1 -- 172.21.15.50:0/1844027314 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f443c011980 con 0x7f444c1536f0 2024-09-21T06:11:17.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4452400640 1 -- 172.21.15.50:0/1844027314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444c1540d0 con 0x7f444c1536f0 2024-09-21T06:11:17.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4449600640 1 -- 172.21.15.50:0/1844027314 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c011dd0 con 0x7f444c1536f0 2024-09-21T06:11:17.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4452400640 1 -- 172.21.15.50:0/1844027314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f444c1548c0 con 0x7f444c1536f0 2024-09-21T06:11:17.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4452400640 1 -- 172.21.15.50:0/1844027314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444c15d480 con 0x7f444c1536f0 2024-09-21T06:11:17.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.300+0000 7f4449600640 1 -- 172.21.15.50:0/1844027314 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f443c018020 con 0x7f444c1536f0 2024-09-21T06:11:17.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.304+0000 7f4449600640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44280751b0 0x7f44280776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:17.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.304+0000 7f4449600640 1 -- 172.21.15.50:0/1844027314 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f443c092ff0 con 0x7f444c1536f0 2024-09-21T06:11:17.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.304+0000 7f4443e00640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44280751b0 0x7f44280776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:17.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.304+0000 7f4443e00640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44280751b0 0x7f44280776c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f44340af000 tx=0x7f443401b070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:17.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.308+0000 7f4449600640 1 -- 172.21.15.50:0/1844027314 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f443c01f030 con 0x7f444c1536f0 2024-09-21T06:11:17.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:17.520+0000 7f4452400640 1 -- 172.21.15.50:0/1844027314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f444c0b2d40 con 0x7f444c1536f0 2024-09-21T06:11:18.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.104+0000 7f4449600640 1 -- 172.21.15.50:0/1844027314 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) ==== 122+0+0 (secure 0 0 0) 0x7f443c00d040 con 0x7f444c1536f0 2024-09-21T06:11:18.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.108+0000 7f4452400640 1 -- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44280751b0 msgr2=0x7f44280776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.108+0000 7f4452400640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44280751b0 0x7f44280776c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f44340af000 tx=0x7f443401b070 comp rx=0 tx=0).stop 2024-09-21T06:11:18.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.108+0000 7f4452400640 1 -- 172.21.15.50:0/1844027314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f444c1536f0 msgr2=0x7f444c149f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.108+0000 7f4452400640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f444c1536f0 0x7f444c149f90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f443c00b440 tx=0x7f443c00b900 comp rx=0 tx=0).stop 2024-09-21T06:11:18.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.108+0000 7f4452400640 1 -- 172.21.15.50:0/1844027314 shutdown_connections 2024-09-21T06:11:18.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.108+0000 7f4452400640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f444c160470 0x7f444c0d1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.108+0000 7f4452400640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f444c160060 0x7f444c153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.108+0000 7f4452400640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44280751b0 0x7f44280776c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.108+0000 7f4452400640 1 --2- 172.21.15.50:0/1844027314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f444c1536f0 0x7f444c149f90 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.108+0000 7f4452400640 1 -- 172.21.15.50:0/1844027314 >> 172.21.15.50:0/1844027314 conn(0x7f444c0b4560 msgr2=0x7f444c15dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:18.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.108+0000 7f4452400640 1 -- 172.21.15.50:0/1844027314 shutdown_connections 2024-09-21T06:11:18.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.108+0000 7f4452400640 1 -- 172.21.15.50:0/1844027314 wait complete. 2024-09-21T06:11:18.129 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:11:18.129 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:11:18.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.296+0000 7f2710a00640 1 -- 172.21.15.50:0/2091333106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f270c0749a0 msgr2=0x7f270c074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.296+0000 7f2710a00640 1 --2- 172.21.15.50:0/2091333106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f270c0749a0 0x7f270c074db0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f270c0b4ba0 tx=0x7f26fc01c800 comp rx=0 tx=0).stop 2024-09-21T06:11:18.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.296+0000 7f2710a00640 1 -- 172.21.15.50:0/2091333106 shutdown_connections 2024-09-21T06:11:18.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.296+0000 7f2710a00640 1 --2- 172.21.15.50:0/2091333106 >> v2:172.21.15.88:3300/0 conn(0x7f270c0b3340 0x7f270c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.296+0000 7f2710a00640 1 --2- 172.21.15.50:0/2091333106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f270c0749a0 0x7f270c074db0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.296+0000 7f2710a00640 1 -- 172.21.15.50:0/2091333106 >> 172.21.15.50:0/2091333106 conn(0x7f270c0b2040 msgr2=0x7f270c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:18.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.296+0000 7f2710a00640 1 -- 172.21.15.50:0/2091333106 shutdown_connections 2024-09-21T06:11:18.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.296+0000 7f2710a00640 1 -- 172.21.15.50:0/2091333106 wait complete. 2024-09-21T06:11:18.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2710a00640 1 Processor -- start 2024-09-21T06:11:18.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2710a00640 1 -- start start 2024-09-21T06:11:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2710a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f270c0749a0 0x7f270c13ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2710a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f270c0b3340 0x7f270c161460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2710a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f270c0cefa0 0x7f270c165570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2710a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f270c169c30 con 0x7f270c0b3340 2024-09-21T06:11:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2710a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f270c169ab0 con 0x7f270c0cefa0 2024-09-21T06:11:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2710a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f270c169db0 con 0x7f270c0749a0 2024-09-21T06:11:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2701400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f270c0b3340 0x7f270c161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2701400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f270c0b3340 0x7f270c161460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32774/0 (socket says 172.21.15.50:32774) 2024-09-21T06:11:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2701400640 1 -- 172.21.15.50:0/138639154 learned_addr learned my addr 172.21.15.50:0/138639154 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f270a800640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f270c0cefa0 0x7f270c165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2709e00640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f270c0749a0 0x7f270c13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2701400640 1 -- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f270c0749a0 msgr2=0x7f270c13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2701400640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f270c0749a0 0x7f270c13ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2701400640 1 -- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f270c0cefa0 msgr2=0x7f270c165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2701400640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f270c0cefa0 0x7f270c165570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2701400640 1 -- 172.21.15.50:0/138639154 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270c1460b0 con 0x7f270c0b3340 2024-09-21T06:11:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2709e00640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f270c0749a0 0x7f270c13ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2701400640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f270c0b3340 0x7f270c161460 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f26f400c400 tx=0x7f26f400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:18.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2702a00640 1 -- 172.21.15.50:0/138639154 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4017070 con 0x7f270c0b3340 2024-09-21T06:11:18.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2702a00640 1 -- 172.21.15.50:0/138639154 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26f4003600 con 0x7f270c0b3340 2024-09-21T06:11:18.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2710a00640 1 -- 172.21.15.50:0/138639154 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270c1647b0 con 0x7f270c0b3340 2024-09-21T06:11:18.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2702a00640 1 -- 172.21.15.50:0/138639154 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4003a50 con 0x7f270c0b3340 2024-09-21T06:11:18.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.300+0000 7f2710a00640 1 -- 172.21.15.50:0/138639154 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270c164d70 con 0x7f270c0b3340 2024-09-21T06:11:18.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.304+0000 7f2710a00640 1 -- 172.21.15.50:0/138639154 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270c074db0 con 0x7f270c0b3340 2024-09-21T06:11:18.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.304+0000 7f2702a00640 1 -- 172.21.15.50:0/138639154 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f26f401a070 con 0x7f270c0b3340 2024-09-21T06:11:18.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.304+0000 7f2702a00640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f26e8075180 0x7f26e8077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:18.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.304+0000 7f2702a00640 1 -- 172.21.15.50:0/138639154 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f26f40924c0 con 0x7f270c0b3340 2024-09-21T06:11:18.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.304+0000 7f2709e00640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f26e8075180 0x7f26e8077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:18.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.304+0000 7f2709e00640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f26e8075180 0x7f26e8077690 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0af000 tx=0x7f26fc01b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:18.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.308+0000 7f2702a00640 1 -- 172.21.15.50:0/138639154 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f270c1430e0 con 0x7f270c0b3340 2024-09-21T06:11:18.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.519+0000 7f2710a00640 1 -- 172.21.15.50:0/138639154 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f270c0b2450 con 0x7f270c0b3340 2024-09-21T06:11:18.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.519+0000 7f2702a00640 1 -- 172.21.15.50:0/138639154 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5219 (secure 0 0 0) 0x7f26f405f070 con 0x7f270c0b3340 2024-09-21T06:11:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:18.519 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":84,"btime":"2024-09-21T06:11:18:101077+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2643543096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2643543096},{"type":"v1","addr":"172.21.15.88:6836","nonce":2643543096}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3618269169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3618269169},{"type":"v1","addr":"172.21.15.88:6837","nonce":3618269169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/66879433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":66879433},{"type":"v1","addr":"172.21.15.50:6835","nonce":66879433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:11:14.496744+0000","modified":"2024-09-21T06:11:17.524950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8832},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8832":{"gid":8832,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6833/1864490990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1864490990},{"type":"v1","addr":"172.21.15.50:6833","nonce":1864490990}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8832,"qdb_cluster":[8832]},"id":9}]} 2024-09-21T06:11:18.519 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 84 2024-09-21T06:11:18.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.523+0000 7f2710a00640 1 -- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f26e8075180 msgr2=0x7f26e8077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.523+0000 7f2710a00640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f26e8075180 0x7f26e8077690 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0af000 tx=0x7f26fc01b040 comp rx=0 tx=0).stop 2024-09-21T06:11:18.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.523+0000 7f2710a00640 1 -- 172.21.15.50:0/138639154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f270c0b3340 msgr2=0x7f270c161460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.523+0000 7f2710a00640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f270c0b3340 0x7f270c161460 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f26f400c400 tx=0x7f26f400c8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.523+0000 7f2710a00640 1 -- 172.21.15.50:0/138639154 shutdown_connections 2024-09-21T06:11:18.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.523+0000 7f2710a00640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f270c0cefa0 0x7f270c165570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.523+0000 7f2710a00640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f26e8075180 0x7f26e8077690 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.523+0000 7f2710a00640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f270c0b3340 0x7f270c161460 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.523+0000 7f2710a00640 1 --2- 172.21.15.50:0/138639154 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f270c0749a0 0x7f270c13ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.523+0000 7f2710a00640 1 -- 172.21.15.50:0/138639154 >> 172.21.15.50:0/138639154 conn(0x7f270c0b2040 msgr2=0x7f270c15bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.523+0000 7f2710a00640 1 -- 172.21.15.50:0/138639154 shutdown_connections 2024-09-21T06:11:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.523+0000 7f2710a00640 1 -- 172.21.15.50:0/138639154 wait complete. 2024-09-21T06:11:18.542 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:11:18.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.703+0000 7fca38e00640 1 -- 172.21.15.50:0/137274012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca340b5c70 msgr2=0x7fca3415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.703+0000 7fca38e00640 1 --2- 172.21.15.50:0/137274012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca340b5c70 0x7fca3415bb60 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fca34070710 tx=0x7fca2401c800 comp rx=0 tx=0).stop 2024-09-21T06:11:18.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.703+0000 7fca38e00640 1 -- 172.21.15.50:0/137274012 shutdown_connections 2024-09-21T06:11:18.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.703+0000 7fca38e00640 1 --2- 172.21.15.50:0/137274012 >> v2:172.21.15.88:3300/0 conn(0x7fca341527b0 0x7fca34074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.703+0000 7fca38e00640 1 --2- 172.21.15.50:0/137274012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca340b5c70 0x7fca3415bb60 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.703+0000 7fca38e00640 1 -- 172.21.15.50:0/137274012 >> 172.21.15.50:0/137274012 conn(0x7fca340b2040 msgr2=0x7fca340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:18.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.703+0000 7fca38e00640 1 -- 172.21.15.50:0/137274012 shutdown_connections 2024-09-21T06:11:18.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.703+0000 7fca38e00640 1 -- 172.21.15.50:0/137274012 wait complete. 2024-09-21T06:11:18.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.703+0000 7fca38e00640 1 Processor -- start 2024-09-21T06:11:18.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.703+0000 7fca38e00640 1 -- start start 2024-09-21T06:11:18.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca38e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca340b4de0 0x7fca34152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:18.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca38e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca340b5c70 0x7fca34140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:18.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca38e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fca341527b0 0x7fca341fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:18.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca38e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fca34168b20 con 0x7fca340b5c70 2024-09-21T06:11:18.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca38e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca341689a0 con 0x7fca340b4de0 2024-09-21T06:11:18.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca38e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fca34168ca0 con 0x7fca341527b0 2024-09-21T06:11:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca340b5c70 0x7fca34140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca2be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca340b5c70 0x7fca34140dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32804/0 (socket says 172.21.15.50:32804) 2024-09-21T06:11:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca2be00640 1 -- 172.21.15.50:0/3614179854 learned_addr learned my addr 172.21.15.50:0/3614179854 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca31e00640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca340b4de0 0x7fca34152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca32800640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fca341527b0 0x7fca341fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca2be00640 1 -- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fca341527b0 msgr2=0x7fca341fc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca2be00640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fca341527b0 0x7fca341fc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca2be00640 1 -- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca340b4de0 msgr2=0x7fca34152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca2be00640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca340b4de0 0x7fca34152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca2be00640 1 -- 172.21.15.50:0/3614179854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca34153500 con 0x7fca340b5c70 2024-09-21T06:11:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca2be00640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca340b5c70 0x7fca34140dc0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00d750 tx=0x7fca1c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:18.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca2a000640 1 -- 172.21.15.50:0/3614179854 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c017070 con 0x7fca340b5c70 2024-09-21T06:11:18.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca2a000640 1 -- 172.21.15.50:0/3614179854 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca1c009cc0 con 0x7fca340b5c70 2024-09-21T06:11:18.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca38e00640 1 -- 172.21.15.50:0/3614179854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca341537e0 con 0x7fca340b5c70 2024-09-21T06:11:18.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca2a000640 1 -- 172.21.15.50:0/3614179854 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c005280 con 0x7fca340b5c70 2024-09-21T06:11:18.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca38e00640 1 -- 172.21.15.50:0/3614179854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca34153da0 con 0x7fca340b5c70 2024-09-21T06:11:18.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.707+0000 7fca38e00640 1 -- 172.21.15.50:0/3614179854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca3415bb60 con 0x7fca340b5c70 2024-09-21T06:11:18.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.711+0000 7fca2a000640 1 -- 172.21.15.50:0/3614179854 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7fca1c005420 con 0x7fca340b5c70 2024-09-21T06:11:18.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.711+0000 7fca2a000640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca10075180 0x7fca10077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:18.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.711+0000 7fca2a000640 1 -- 172.21.15.50:0/3614179854 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fca1c093330 con 0x7fca340b5c70 2024-09-21T06:11:18.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.711+0000 7fca31e00640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca10075180 0x7fca10077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:18.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.711+0000 7fca31e00640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca10075180 0x7fca10077690 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fca240af000 tx=0x7fca2401b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:18.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.715+0000 7fca2a000640 1 -- 172.21.15.50:0/3614179854 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fca1c00de60 con 0x7fca340b5c70 2024-09-21T06:11:18.767 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:11:18.770 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:11:18.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.895+0000 7fca38e00640 1 -- 172.21.15.50:0/3614179854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca34165f10 con 0x7fca340b5c70 2024-09-21T06:11:18.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.895+0000 7fca2a000640 1 -- 172.21.15.50:0/3614179854 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16824 (secure 0 0 0) 0x7fca1c05fe40 con 0x7fca340b5c70 2024-09-21T06:11:18.895 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:18.895 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":127,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:11:16.065681+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:11:12.241790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T06:11:13.462910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:11:18.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.899+0000 7fca38e00640 1 -- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca10075180 msgr2=0x7fca10077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.899+0000 7fca38e00640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca10075180 0x7fca10077690 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fca240af000 tx=0x7fca2401b040 comp rx=0 tx=0).stop 2024-09-21T06:11:18.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.899+0000 7fca38e00640 1 -- 172.21.15.50:0/3614179854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca340b5c70 msgr2=0x7fca34140dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.899+0000 7fca38e00640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca340b5c70 0x7fca34140dc0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00d750 tx=0x7fca1c00dc10 comp rx=0 tx=0).stop 2024-09-21T06:11:18.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.899+0000 7fca38e00640 1 -- 172.21.15.50:0/3614179854 shutdown_connections 2024-09-21T06:11:18.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.899+0000 7fca38e00640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fca341527b0 0x7fca341fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.899+0000 7fca38e00640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fca10075180 0x7fca10077690 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.899+0000 7fca38e00640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca340b5c70 0x7fca34140dc0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.899+0000 7fca38e00640 1 --2- 172.21.15.50:0/3614179854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca340b4de0 0x7fca34152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.899+0000 7fca38e00640 1 -- 172.21.15.50:0/3614179854 >> 172.21.15.50:0/3614179854 conn(0x7fca340b2040 msgr2=0x7fca34151d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:18.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.899+0000 7fca38e00640 1 -- 172.21.15.50:0/3614179854 shutdown_connections 2024-09-21T06:11:18.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:18.899+0000 7fca38e00640 1 -- 172.21.15.50:0/3614179854 wait complete. 2024-09-21T06:11:18.918 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:11:18.918 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:11:19.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.075+0000 7f7f94400640 1 -- 172.21.15.50:0/714012276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f8c0b5a80 msgr2=0x7f7f8c074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.075+0000 7f7f94400640 1 --2- 172.21.15.50:0/714012276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f8c0b5a80 0x7f7f8c074410 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009840 tx=0x7f7f7801c790 comp rx=0 tx=0).stop 2024-09-21T06:11:19.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.075+0000 7f7f94400640 1 -- 172.21.15.50:0/714012276 shutdown_connections 2024-09-21T06:11:19.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.075+0000 7f7f94400640 1 --2- 172.21.15.50:0/714012276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f8c0b5a80 0x7f7f8c074410 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.075+0000 7f7f94400640 1 -- 172.21.15.50:0/714012276 >> 172.21.15.50:0/714012276 conn(0x7f7f8c0708c0 msgr2=0x7f7f8c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:19.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.079+0000 7f7f94400640 1 -- 172.21.15.50:0/714012276 shutdown_connections 2024-09-21T06:11:19.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.079+0000 7f7f94400640 1 -- 172.21.15.50:0/714012276 wait complete. 2024-09-21T06:11:19.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.079+0000 7f7f94400640 1 Processor -- start 2024-09-21T06:11:19.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.079+0000 7f7f94400640 1 -- start start 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.079+0000 7f7f94400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f8c074950 0x7f7f8c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.079+0000 7f7f94400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7f8c0b5a80 0x7f7f8c153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.079+0000 7f7f94400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f8c157030 0x7f7f8c153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.079+0000 7f7f94400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c16b5f0 con 0x7f7f8c074950 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.079+0000 7f7f94400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c16b470 con 0x7f7f8c157030 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.079+0000 7f7f94400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7f8c16b770 con 0x7f7f8c0b5a80 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.079+0000 7f7f91a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f8c074950 0x7f7f8c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f91a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f8c074950 0x7f7f8c147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32812/0 (socket says 172.21.15.50:32812) 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f91a00640 1 -- 172.21.15.50:0/2892613981 learned_addr learned my addr 172.21.15.50:0/2892613981 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f92400640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f8c157030 0x7f7f8c153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f91000640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7f8c0b5a80 0x7f7f8c153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f91a00640 1 -- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7f8c0b5a80 msgr2=0x7f7f8c153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f91a00640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7f8c0b5a80 0x7f7f8c153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f91a00640 1 -- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f8c157030 msgr2=0x7f7f8c153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f91a00640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f8c157030 0x7f7f8c153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f91a00640 1 -- 172.21.15.50:0/2892613981 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8c14ee20 con 0x7f7f8c074950 2024-09-21T06:11:19.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f91a00640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f8c074950 0x7f7f8c147e40 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000c3f0 tx=0x7f7f8000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:19.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f89600640 1 -- 172.21.15.50:0/2892613981 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80017070 con 0x7f7f8c074950 2024-09-21T06:11:19.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f89600640 1 -- 172.21.15.50:0/2892613981 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f80003600 con 0x7f7f8c074950 2024-09-21T06:11:19.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f94400640 1 -- 172.21.15.50:0/2892613981 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8c14f100 con 0x7f7f8c074950 2024-09-21T06:11:19.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f89600640 1 -- 172.21.15.50:0/2892613981 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80003a50 con 0x7f7f8c074950 2024-09-21T06:11:19.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f94400640 1 -- 172.21.15.50:0/2892613981 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f8c14f6c0 con 0x7f7f8c074950 2024-09-21T06:11:19.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f94400640 1 -- 172.21.15.50:0/2892613981 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f8c074410 con 0x7f7f8c074950 2024-09-21T06:11:19.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.083+0000 7f7f89600640 1 -- 172.21.15.50:0/2892613981 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f7f8001a070 con 0x7f7f8c074950 2024-09-21T06:11:19.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.087+0000 7f7f89600640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7f6c075180 0x7f7f6c077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.087+0000 7f7f89600640 1 -- 172.21.15.50:0/2892613981 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f7f80092920 con 0x7f7f8c074950 2024-09-21T06:11:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.087+0000 7f7f91000640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7f6c075180 0x7f7f6c077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.087+0000 7f7f91000640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7f6c075180 0x7f7f6c077690 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7f780af000 tx=0x7f7f7801afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.087+0000 7f7f89600640 1 -- 172.21.15.50:0/2892613981 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7f80021020 con 0x7f7f8c074950 2024-09-21T06:11:19.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.263+0000 7f7f94400640 1 -- 172.21.15.50:0/2892613981 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f8c06b540 con 0x7f7f6c075180 2024-09-21T06:11:19.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.267+0000 7f7f89600640 1 -- 172.21.15.50:0/2892613981 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252494 (secure 0 0 0) 0x7f7f78004050 con 0x7f7f6c075180 2024-09-21T06:11:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:19.270 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:11:19.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.271+0000 7f7f94400640 1 -- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7f6c075180 msgr2=0x7f7f6c077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.271+0000 7f7f94400640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7f6c075180 0x7f7f6c077690 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7f780af000 tx=0x7f7f7801afd0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.271+0000 7f7f94400640 1 -- 172.21.15.50:0/2892613981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f8c074950 msgr2=0x7f7f8c147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.271+0000 7f7f94400640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f8c074950 0x7f7f8c147e40 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000c3f0 tx=0x7f7f8000c8b0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.271+0000 7f7f94400640 1 -- 172.21.15.50:0/2892613981 shutdown_connections 2024-09-21T06:11:19.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.271+0000 7f7f94400640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f8c157030 0x7f7f8c153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.271+0000 7f7f94400640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7f8c0b5a80 0x7f7f8c153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.271+0000 7f7f94400640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7f6c075180 0x7f7f6c077690 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.271+0000 7f7f94400640 1 --2- 172.21.15.50:0/2892613981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f8c074950 0x7f7f8c147e40 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.271+0000 7f7f94400640 1 -- 172.21.15.50:0/2892613981 >> 172.21.15.50:0/2892613981 conn(0x7f7f8c0708c0 msgr2=0x7f7f8c0b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:19.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.271+0000 7f7f94400640 1 -- 172.21.15.50:0/2892613981 shutdown_connections 2024-09-21T06:11:19.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.271+0000 7f7f94400640 1 -- 172.21.15.50:0/2892613981 wait complete. 2024-09-21T06:11:19.290 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":571,"stamp":"2024-09-21T06:11:19.208570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1067732,"kb_used_data":13532,"kb_used_omap":12,"kb_used_meta":1054003,"kb_avail":753906988,"statfs":{"total":773094113280,"available":772000755712,"internally_reserved":0,"allocated":13856768,"data_stored":6705884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1079299632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001594"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070700+0000","last_change":"2024-09-21T06:11:15.109497+0000","last_active":"2024-09-21T06:11:16.070700+0000","last_peered":"2024-09-21T06:11:16.070700+0000","last_clean":"2024-09-21T06:11:16.070700+0000","last_became_active":"2024-09-21T06:11:15.106405+0000","last_became_peered":"2024-09-21T06:11:15.106405+0000","last_unstale":"2024-09-21T06:11:16.070700+0000","last_undegraded":"2024-09-21T06:11:16.070700+0000","last_fullsized":"2024-09-21T06:11:16.070700+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:16.807470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734144+0000","last_change":"2024-09-21T06:11:15.088957+0000","last_active":"2024-09-21T06:11:16.734144+0000","last_peered":"2024-09-21T06:11:16.734144+0000","last_clean":"2024-09-21T06:11:16.734144+0000","last_became_active":"2024-09-21T06:11:15.088604+0000","last_became_peered":"2024-09-21T06:11:15.088604+0000","last_unstale":"2024-09-21T06:11:16.734144+0000","last_undegraded":"2024-09-21T06:11:16.734144+0000","last_fullsized":"2024-09-21T06:11:16.734144+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:38.255991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733755+0000","last_change":"2024-09-21T06:11:15.078587+0000","last_active":"2024-09-21T06:11:16.733755+0000","last_peered":"2024-09-21T06:11:16.733755+0000","last_clean":"2024-09-21T06:11:16.733755+0000","last_became_active":"2024-09-21T06:11:15.077576+0000","last_became_peered":"2024-09-21T06:11:15.077576+0000","last_unstale":"2024-09-21T06:11:16.733755+0000","last_undegraded":"2024-09-21T06:11:16.733755+0000","last_fullsized":"2024-09-21T06:11:16.733755+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:46.660796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070212+0000","last_change":"2024-09-21T06:11:15.107883+0000","last_active":"2024-09-21T06:11:16.070212+0000","last_peered":"2024-09-21T06:11:16.070212+0000","last_clean":"2024-09-21T06:11:16.070212+0000","last_became_active":"2024-09-21T06:11:15.106250+0000","last_became_peered":"2024-09-21T06:11:15.106250+0000","last_unstale":"2024-09-21T06:11:16.070212+0000","last_undegraded":"2024-09-21T06:11:16.070212+0000","last_fullsized":"2024-09-21T06:11:16.070212+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:13.724960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734109+0000","last_change":"2024-09-21T06:11:15.094376+0000","last_active":"2024-09-21T06:11:16.734109+0000","last_peered":"2024-09-21T06:11:16.734109+0000","last_clean":"2024-09-21T06:11:16.734109+0000","last_became_active":"2024-09-21T06:11:15.093830+0000","last_became_peered":"2024-09-21T06:11:15.093830+0000","last_unstale":"2024-09-21T06:11:16.734109+0000","last_undegraded":"2024-09-21T06:11:16.734109+0000","last_fullsized":"2024-09-21T06:11:16.734109+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:50.552668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075515+0000","last_change":"2024-09-21T06:11:15.103206+0000","last_active":"2024-09-21T06:11:16.075515+0000","last_peered":"2024-09-21T06:11:16.075515+0000","last_clean":"2024-09-21T06:11:16.075515+0000","last_became_active":"2024-09-21T06:11:15.102740+0000","last_became_peered":"2024-09-21T06:11:15.102740+0000","last_unstale":"2024-09-21T06:11:16.075515+0000","last_undegraded":"2024-09-21T06:11:16.075515+0000","last_fullsized":"2024-09-21T06:11:16.075515+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:44.695527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075497+0000","last_change":"2024-09-21T06:11:15.081361+0000","last_active":"2024-09-21T06:11:16.075497+0000","last_peered":"2024-09-21T06:11:16.075497+0000","last_clean":"2024-09-21T06:11:16.075497+0000","last_became_active":"2024-09-21T06:11:15.081064+0000","last_became_peered":"2024-09-21T06:11:15.081064+0000","last_unstale":"2024-09-21T06:11:16.075497+0000","last_undegraded":"2024-09-21T06:11:16.075497+0000","last_fullsized":"2024-09-21T06:11:16.075497+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:27.232804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733070+0000","last_change":"2024-09-21T06:11:15.093701+0000","last_active":"2024-09-21T06:11:16.733070+0000","last_peered":"2024-09-21T06:11:16.733070+0000","last_clean":"2024-09-21T06:11:16.733070+0000","last_became_active":"2024-09-21T06:11:15.093044+0000","last_became_peered":"2024-09-21T06:11:15.093044+0000","last_unstale":"2024-09-21T06:11:16.733070+0000","last_undegraded":"2024-09-21T06:11:16.733070+0000","last_fullsized":"2024-09-21T06:11:16.733070+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:28.621442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734386+0000","last_change":"2024-09-21T06:11:15.079906+0000","last_active":"2024-09-21T06:11:16.734386+0000","last_peered":"2024-09-21T06:11:16.734386+0000","last_clean":"2024-09-21T06:11:16.734386+0000","last_became_active":"2024-09-21T06:11:15.079581+0000","last_became_peered":"2024-09-21T06:11:15.079581+0000","last_unstale":"2024-09-21T06:11:16.734386+0000","last_undegraded":"2024-09-21T06:11:16.734386+0000","last_fullsized":"2024-09-21T06:11:16.734386+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:08.330721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734631+0000","last_change":"2024-09-21T06:11:15.090615+0000","last_active":"2024-09-21T06:11:16.734631+0000","last_peered":"2024-09-21T06:11:16.734631+0000","last_clean":"2024-09-21T06:11:16.734631+0000","last_became_active":"2024-09-21T06:11:15.090250+0000","last_became_peered":"2024-09-21T06:11:15.090250+0000","last_unstale":"2024-09-21T06:11:16.734631+0000","last_undegraded":"2024-09-21T06:11:16.734631+0000","last_fullsized":"2024-09-21T06:11:16.734631+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:33.123828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733954+0000","last_change":"2024-09-21T06:11:15.087419+0000","last_active":"2024-09-21T06:11:16.733954+0000","last_peered":"2024-09-21T06:11:16.733954+0000","last_clean":"2024-09-21T06:11:16.733954+0000","last_became_active":"2024-09-21T06:11:15.087068+0000","last_became_peered":"2024-09-21T06:11:15.087068+0000","last_unstale":"2024-09-21T06:11:16.733954+0000","last_undegraded":"2024-09-21T06:11:16.733954+0000","last_fullsized":"2024-09-21T06:11:16.733954+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:59.549881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733687+0000","last_change":"2024-09-21T06:11:15.079038+0000","last_active":"2024-09-21T06:11:16.733687+0000","last_peered":"2024-09-21T06:11:16.733687+0000","last_clean":"2024-09-21T06:11:16.733687+0000","last_became_active":"2024-09-21T06:11:15.078716+0000","last_became_peered":"2024-09-21T06:11:15.078716+0000","last_unstale":"2024-09-21T06:11:16.733687+0000","last_undegraded":"2024-09-21T06:11:16.733687+0000","last_fullsized":"2024-09-21T06:11:16.733687+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:16.763840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070018+0000","last_change":"2024-09-21T06:11:15.113803+0000","last_active":"2024-09-21T06:11:16.070018+0000","last_peered":"2024-09-21T06:11:16.070018+0000","last_clean":"2024-09-21T06:11:16.070018+0000","last_became_active":"2024-09-21T06:11:15.112722+0000","last_became_peered":"2024-09-21T06:11:15.112722+0000","last_unstale":"2024-09-21T06:11:16.070018+0000","last_undegraded":"2024-09-21T06:11:16.070018+0000","last_fullsized":"2024-09-21T06:11:16.070018+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:42.600363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733567+0000","last_change":"2024-09-21T06:11:15.110297+0000","last_active":"2024-09-21T06:11:16.733567+0000","last_peered":"2024-09-21T06:11:16.733567+0000","last_clean":"2024-09-21T06:11:16.733567+0000","last_became_active":"2024-09-21T06:11:15.109972+0000","last_became_peered":"2024-09-21T06:11:15.109972+0000","last_unstale":"2024-09-21T06:11:16.733567+0000","last_undegraded":"2024-09-21T06:11:16.733567+0000","last_fullsized":"2024-09-21T06:11:16.733567+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:23.201040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.069748+0000","last_change":"2024-09-21T06:11:15.098055+0000","last_active":"2024-09-21T06:11:16.069748+0000","last_peered":"2024-09-21T06:11:16.069748+0000","last_clean":"2024-09-21T06:11:16.069748+0000","last_became_active":"2024-09-21T06:11:15.097782+0000","last_became_peered":"2024-09-21T06:11:15.097782+0000","last_unstale":"2024-09-21T06:11:16.069748+0000","last_undegraded":"2024-09-21T06:11:16.069748+0000","last_fullsized":"2024-09-21T06:11:16.069748+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:15.938620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092127+0000","last_change":"2024-09-21T06:11:15.110173+0000","last_active":"2024-09-21T06:11:16.092127+0000","last_peered":"2024-09-21T06:11:16.092127+0000","last_clean":"2024-09-21T06:11:16.092127+0000","last_became_active":"2024-09-21T06:11:15.109787+0000","last_became_peered":"2024-09-21T06:11:15.109787+0000","last_unstale":"2024-09-21T06:11:16.092127+0000","last_undegraded":"2024-09-21T06:11:16.092127+0000","last_fullsized":"2024-09-21T06:11:16.092127+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:52.776766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.076098+0000","last_change":"2024-09-21T06:11:15.087445+0000","last_active":"2024-09-21T06:11:16.076098+0000","last_peered":"2024-09-21T06:11:16.076098+0000","last_clean":"2024-09-21T06:11:16.076098+0000","last_became_active":"2024-09-21T06:11:15.086239+0000","last_became_peered":"2024-09-21T06:11:15.086239+0000","last_unstale":"2024-09-21T06:11:16.076098+0000","last_undegraded":"2024-09-21T06:11:16.076098+0000","last_fullsized":"2024-09-21T06:11:16.076098+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:42.413437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092895+0000","last_change":"2024-09-21T06:11:15.098012+0000","last_active":"2024-09-21T06:11:16.092895+0000","last_peered":"2024-09-21T06:11:16.092895+0000","last_clean":"2024-09-21T06:11:16.092895+0000","last_became_active":"2024-09-21T06:11:15.097676+0000","last_became_peered":"2024-09-21T06:11:15.097676+0000","last_unstale":"2024-09-21T06:11:16.092895+0000","last_undegraded":"2024-09-21T06:11:16.092895+0000","last_fullsized":"2024-09-21T06:11:16.092895+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:14.095774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734260+0000","last_change":"2024-09-21T06:11:15.080470+0000","last_active":"2024-09-21T06:11:16.734260+0000","last_peered":"2024-09-21T06:11:16.734260+0000","last_clean":"2024-09-21T06:11:16.734260+0000","last_became_active":"2024-09-21T06:11:15.080110+0000","last_became_peered":"2024-09-21T06:11:15.080110+0000","last_unstale":"2024-09-21T06:11:16.734260+0000","last_undegraded":"2024-09-21T06:11:16.734260+0000","last_fullsized":"2024-09-21T06:11:16.734260+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:45.033650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092599+0000","last_change":"2024-09-21T06:11:15.094974+0000","last_active":"2024-09-21T06:11:16.092599+0000","last_peered":"2024-09-21T06:11:16.092599+0000","last_clean":"2024-09-21T06:11:16.092599+0000","last_became_active":"2024-09-21T06:11:15.094667+0000","last_became_peered":"2024-09-21T06:11:15.094667+0000","last_unstale":"2024-09-21T06:11:16.092599+0000","last_undegraded":"2024-09-21T06:11:16.092599+0000","last_fullsized":"2024-09-21T06:11:16.092599+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:26.818828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734327+0000","last_change":"2024-09-21T06:11:15.093205+0000","last_active":"2024-09-21T06:11:16.734327+0000","last_peered":"2024-09-21T06:11:16.734327+0000","last_clean":"2024-09-21T06:11:16.734327+0000","last_became_active":"2024-09-21T06:11:15.092864+0000","last_became_peered":"2024-09-21T06:11:15.092864+0000","last_unstale":"2024-09-21T06:11:16.734327+0000","last_undegraded":"2024-09-21T06:11:16.734327+0000","last_fullsized":"2024-09-21T06:11:16.734327+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:56.164663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733600+0000","last_change":"2024-09-21T06:11:15.088356+0000","last_active":"2024-09-21T06:11:16.733600+0000","last_peered":"2024-09-21T06:11:16.733600+0000","last_clean":"2024-09-21T06:11:16.733600+0000","last_became_active":"2024-09-21T06:11:15.087655+0000","last_became_peered":"2024-09-21T06:11:15.087655+0000","last_unstale":"2024-09-21T06:11:16.733600+0000","last_undegraded":"2024-09-21T06:11:16.733600+0000","last_fullsized":"2024-09-21T06:11:16.733600+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:02.985581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733206+0000","last_change":"2024-09-21T06:11:15.076156+0000","last_active":"2024-09-21T06:11:16.733206+0000","last_peered":"2024-09-21T06:11:16.733206+0000","last_clean":"2024-09-21T06:11:16.733206+0000","last_became_active":"2024-09-21T06:11:15.075829+0000","last_became_peered":"2024-09-21T06:11:15.075829+0000","last_unstale":"2024-09-21T06:11:16.733206+0000","last_undegraded":"2024-09-21T06:11:16.733206+0000","last_fullsized":"2024-09-21T06:11:16.733206+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:59.603828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734388+0000","last_change":"2024-09-21T06:11:15.075031+0000","last_active":"2024-09-21T06:11:16.734388+0000","last_peered":"2024-09-21T06:11:16.734388+0000","last_clean":"2024-09-21T06:11:16.734388+0000","last_became_active":"2024-09-21T06:11:15.074707+0000","last_became_peered":"2024-09-21T06:11:15.074707+0000","last_unstale":"2024-09-21T06:11:16.734388+0000","last_undegraded":"2024-09-21T06:11:16.734388+0000","last_fullsized":"2024-09-21T06:11:16.734388+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:24.911773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734183+0000","last_change":"2024-09-21T06:11:15.093790+0000","last_active":"2024-09-21T06:11:16.734183+0000","last_peered":"2024-09-21T06:11:16.734183+0000","last_clean":"2024-09-21T06:11:16.734183+0000","last_became_active":"2024-09-21T06:11:15.093457+0000","last_became_peered":"2024-09-21T06:11:15.093457+0000","last_unstale":"2024-09-21T06:11:16.734183+0000","last_undegraded":"2024-09-21T06:11:16.734183+0000","last_fullsized":"2024-09-21T06:11:16.734183+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:48.446640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092920+0000","last_change":"2024-09-21T06:11:15.111543+0000","last_active":"2024-09-21T06:11:16.092920+0000","last_peered":"2024-09-21T06:11:16.092920+0000","last_clean":"2024-09-21T06:11:16.092920+0000","last_became_active":"2024-09-21T06:11:15.109730+0000","last_became_peered":"2024-09-21T06:11:15.109730+0000","last_unstale":"2024-09-21T06:11:16.092920+0000","last_undegraded":"2024-09-21T06:11:16.092920+0000","last_fullsized":"2024-09-21T06:11:16.092920+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:06:07.695357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070209+0000","last_change":"2024-09-21T06:11:15.113166+0000","last_active":"2024-09-21T06:11:16.070209+0000","last_peered":"2024-09-21T06:11:16.070209+0000","last_clean":"2024-09-21T06:11:16.070209+0000","last_became_active":"2024-09-21T06:11:15.112798+0000","last_became_peered":"2024-09-21T06:11:15.112798+0000","last_unstale":"2024-09-21T06:11:16.070209+0000","last_undegraded":"2024-09-21T06:11:16.070209+0000","last_fullsized":"2024-09-21T06:11:16.070209+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:16.990551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.073876+0000","last_change":"2024-09-21T06:11:15.095407+0000","last_active":"2024-09-21T06:11:16.073876+0000","last_peered":"2024-09-21T06:11:16.073876+0000","last_clean":"2024-09-21T06:11:16.073876+0000","last_became_active":"2024-09-21T06:11:15.095080+0000","last_became_peered":"2024-09-21T06:11:15.095080+0000","last_unstale":"2024-09-21T06:11:16.073876+0000","last_undegraded":"2024-09-21T06:11:16.073876+0000","last_fullsized":"2024-09-21T06:11:16.073876+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:55.410180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070136+0000","last_change":"2024-09-21T06:11:15.108159+0000","last_active":"2024-09-21T06:11:16.070136+0000","last_peered":"2024-09-21T06:11:16.070136+0000","last_clean":"2024-09-21T06:11:16.070136+0000","last_became_active":"2024-09-21T06:11:15.097350+0000","last_became_peered":"2024-09-21T06:11:15.097350+0000","last_unstale":"2024-09-21T06:11:16.070136+0000","last_undegraded":"2024-09-21T06:11:16.070136+0000","last_fullsized":"2024-09-21T06:11:16.070136+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:14.205673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.069878+0000","last_change":"2024-09-21T06:11:15.109650+0000","last_active":"2024-09-21T06:11:16.069878+0000","last_peered":"2024-09-21T06:11:16.069878+0000","last_clean":"2024-09-21T06:11:16.069878+0000","last_became_active":"2024-09-21T06:11:15.109283+0000","last_became_peered":"2024-09-21T06:11:15.109283+0000","last_unstale":"2024-09-21T06:11:16.069878+0000","last_undegraded":"2024-09-21T06:11:16.069878+0000","last_fullsized":"2024-09-21T06:11:16.069878+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:01.114801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734037+0000","last_change":"2024-09-21T06:11:15.111180+0000","last_active":"2024-09-21T06:11:16.734037+0000","last_peered":"2024-09-21T06:11:16.734037+0000","last_clean":"2024-09-21T06:11:16.734037+0000","last_became_active":"2024-09-21T06:11:15.110858+0000","last_became_peered":"2024-09-21T06:11:15.110858+0000","last_unstale":"2024-09-21T06:11:16.734037+0000","last_undegraded":"2024-09-21T06:11:16.734037+0000","last_fullsized":"2024-09-21T06:11:16.734037+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:38.734624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075301+0000","last_change":"2024-09-21T06:11:15.097185+0000","last_active":"2024-09-21T06:11:16.075301+0000","last_peered":"2024-09-21T06:11:16.075301+0000","last_clean":"2024-09-21T06:11:16.075301+0000","last_became_active":"2024-09-21T06:11:15.096830+0000","last_became_peered":"2024-09-21T06:11:15.096830+0000","last_unstale":"2024-09-21T06:11:16.075301+0000","last_undegraded":"2024-09-21T06:11:16.075301+0000","last_fullsized":"2024-09-21T06:11:16.075301+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:01.693132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.093118+0000","last_change":"2024-09-21T06:11:15.093486+0000","last_active":"2024-09-21T06:11:16.093118+0000","last_peered":"2024-09-21T06:11:16.093118+0000","last_clean":"2024-09-21T06:11:16.093118+0000","last_became_active":"2024-09-21T06:11:15.093103+0000","last_became_peered":"2024-09-21T06:11:15.093103+0000","last_unstale":"2024-09-21T06:11:16.093118+0000","last_undegraded":"2024-09-21T06:11:16.093118+0000","last_fullsized":"2024-09-21T06:11:16.093118+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:38.734624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"126'4","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733680+0000","last_change":"2024-09-21T06:11:14.092934+0000","last_active":"2024-09-21T06:11:16.733680+0000","last_peered":"2024-09-21T06:11:16.733680+0000","last_clean":"2024-09-21T06:11:16.733680+0000","last_became_active":"2024-09-21T06:11:14.092573+0000","last_became_peered":"2024-09-21T06:11:14.092573+0000","last_unstale":"2024-09-21T06:11:16.733680+0000","last_undegraded":"2024-09-21T06:11:16.733680+0000","last_fullsized":"2024-09-21T06:11:16.733680+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:43.692987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734665+0000","last_change":"2024-09-21T06:11:15.099589+0000","last_active":"2024-09-21T06:11:16.734665+0000","last_peered":"2024-09-21T06:11:16.734665+0000","last_clean":"2024-09-21T06:11:16.734665+0000","last_became_active":"2024-09-21T06:11:15.099253+0000","last_became_peered":"2024-09-21T06:11:15.099253+0000","last_unstale":"2024-09-21T06:11:16.734665+0000","last_undegraded":"2024-09-21T06:11:16.734665+0000","last_fullsized":"2024-09-21T06:11:16.734665+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:27.232804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.074041+0000","last_change":"2024-09-21T06:11:14.082687+0000","last_active":"2024-09-21T06:11:16.074041+0000","last_peered":"2024-09-21T06:11:16.074041+0000","last_clean":"2024-09-21T06:11:16.074041+0000","last_became_active":"2024-09-21T06:11:14.082250+0000","last_became_peered":"2024-09-21T06:11:14.082250+0000","last_unstale":"2024-09-21T06:11:16.074041+0000","last_undegraded":"2024-09-21T06:11:16.074041+0000","last_fullsized":"2024-09-21T06:11:16.074041+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:33.271588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734122+0000","last_change":"2024-09-21T06:11:15.097931+0000","last_active":"2024-09-21T06:11:16.734122+0000","last_peered":"2024-09-21T06:11:16.734122+0000","last_clean":"2024-09-21T06:11:16.734122+0000","last_became_active":"2024-09-21T06:11:15.096894+0000","last_became_peered":"2024-09-21T06:11:15.096894+0000","last_unstale":"2024-09-21T06:11:16.734122+0000","last_undegraded":"2024-09-21T06:11:16.734122+0000","last_fullsized":"2024-09-21T06:11:16.734122+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:10.140285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734159+0000","last_change":"2024-09-21T06:11:14.089797+0000","last_active":"2024-09-21T06:11:16.734159+0000","last_peered":"2024-09-21T06:11:16.734159+0000","last_clean":"2024-09-21T06:11:16.734159+0000","last_became_active":"2024-09-21T06:11:14.089503+0000","last_became_peered":"2024-09-21T06:11:14.089503+0000","last_unstale":"2024-09-21T06:11:16.734159+0000","last_undegraded":"2024-09-21T06:11:16.734159+0000","last_fullsized":"2024-09-21T06:11:16.734159+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:04.628772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733461+0000","last_change":"2024-09-21T06:11:15.096053+0000","last_active":"2024-09-21T06:11:16.733461+0000","last_peered":"2024-09-21T06:11:16.733461+0000","last_clean":"2024-09-21T06:11:16.733461+0000","last_became_active":"2024-09-21T06:11:15.095486+0000","last_became_peered":"2024-09-21T06:11:15.095486+0000","last_unstale":"2024-09-21T06:11:16.733461+0000","last_undegraded":"2024-09-21T06:11:16.733461+0000","last_fullsized":"2024-09-21T06:11:16.733461+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:59.008882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733987+0000","last_change":"2024-09-21T06:11:14.081889+0000","last_active":"2024-09-21T06:11:16.733987+0000","last_peered":"2024-09-21T06:11:16.733987+0000","last_clean":"2024-09-21T06:11:16.733987+0000","last_became_active":"2024-09-21T06:11:14.081204+0000","last_became_peered":"2024-09-21T06:11:14.081204+0000","last_unstale":"2024-09-21T06:11:16.733987+0000","last_undegraded":"2024-09-21T06:11:16.733987+0000","last_fullsized":"2024-09-21T06:11:16.733987+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:20.460183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092017+0000","last_change":"2024-09-21T06:11:15.102737+0000","last_active":"2024-09-21T06:11:16.092017+0000","last_peered":"2024-09-21T06:11:16.092017+0000","last_clean":"2024-09-21T06:11:16.092017+0000","last_became_active":"2024-09-21T06:11:15.101542+0000","last_became_peered":"2024-09-21T06:11:15.101542+0000","last_unstale":"2024-09-21T06:11:16.092017+0000","last_undegraded":"2024-09-21T06:11:16.092017+0000","last_fullsized":"2024-09-21T06:11:16.092017+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:48.919648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733922+0000","last_change":"2024-09-21T06:11:14.077079+0000","last_active":"2024-09-21T06:11:16.733922+0000","last_peered":"2024-09-21T06:11:16.733922+0000","last_clean":"2024-09-21T06:11:16.733922+0000","last_became_active":"2024-09-21T06:11:14.076767+0000","last_became_peered":"2024-09-21T06:11:14.076767+0000","last_unstale":"2024-09-21T06:11:16.733922+0000","last_undegraded":"2024-09-21T06:11:16.733922+0000","last_fullsized":"2024-09-21T06:11:16.733922+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:01.324687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075665+0000","last_change":"2024-09-21T06:11:15.086370+0000","last_active":"2024-09-21T06:11:16.075665+0000","last_peered":"2024-09-21T06:11:16.075665+0000","last_clean":"2024-09-21T06:11:16.075665+0000","last_became_active":"2024-09-21T06:11:15.086012+0000","last_became_peered":"2024-09-21T06:11:15.086012+0000","last_unstale":"2024-09-21T06:11:16.075665+0000","last_undegraded":"2024-09-21T06:11:16.075665+0000","last_fullsized":"2024-09-21T06:11:16.075665+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:51.193220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.091850+0000","last_change":"2024-09-21T06:11:14.080257+0000","last_active":"2024-09-21T06:11:16.091850+0000","last_peered":"2024-09-21T06:11:16.091850+0000","last_clean":"2024-09-21T06:11:16.091850+0000","last_became_active":"2024-09-21T06:11:14.079840+0000","last_became_peered":"2024-09-21T06:11:14.079840+0000","last_unstale":"2024-09-21T06:11:16.091850+0000","last_undegraded":"2024-09-21T06:11:16.091850+0000","last_fullsized":"2024-09-21T06:11:16.091850+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:00.690591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733439+0000","last_change":"2024-09-21T06:11:15.093389+0000","last_active":"2024-09-21T06:11:16.733439+0000","last_peered":"2024-09-21T06:11:16.733439+0000","last_clean":"2024-09-21T06:11:16.733439+0000","last_became_active":"2024-09-21T06:11:15.092636+0000","last_became_peered":"2024-09-21T06:11:15.092636+0000","last_unstale":"2024-09-21T06:11:16.733439+0000","last_undegraded":"2024-09-21T06:11:16.733439+0000","last_fullsized":"2024-09-21T06:11:16.733439+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:01.030644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075254+0000","last_change":"2024-09-21T06:11:14.085843+0000","last_active":"2024-09-21T06:11:16.075254+0000","last_peered":"2024-09-21T06:11:16.075254+0000","last_clean":"2024-09-21T06:11:16.075254+0000","last_became_active":"2024-09-21T06:11:14.085512+0000","last_became_peered":"2024-09-21T06:11:14.085512+0000","last_unstale":"2024-09-21T06:11:16.075254+0000","last_undegraded":"2024-09-21T06:11:16.075254+0000","last_fullsized":"2024-09-21T06:11:16.075254+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:29.330942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733248+0000","last_change":"2024-09-21T06:11:15.103046+0000","last_active":"2024-09-21T06:11:16.733248+0000","last_peered":"2024-09-21T06:11:16.733248+0000","last_clean":"2024-09-21T06:11:16.733248+0000","last_became_active":"2024-09-21T06:11:15.102737+0000","last_became_peered":"2024-09-21T06:11:15.102737+0000","last_unstale":"2024-09-21T06:11:16.733248+0000","last_undegraded":"2024-09-21T06:11:16.733248+0000","last_fullsized":"2024-09-21T06:11:16.733248+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:12.172196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733371+0000","last_change":"2024-09-21T06:11:14.084434+0000","last_active":"2024-09-21T06:11:16.733371+0000","last_peered":"2024-09-21T06:11:16.733371+0000","last_clean":"2024-09-21T06:11:16.733371+0000","last_became_active":"2024-09-21T06:11:14.084117+0000","last_became_peered":"2024-09-21T06:11:14.084117+0000","last_unstale":"2024-09-21T06:11:16.733371+0000","last_undegraded":"2024-09-21T06:11:16.733371+0000","last_fullsized":"2024-09-21T06:11:16.733371+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:33.655342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070608+0000","last_change":"2024-09-21T06:11:15.101334+0000","last_active":"2024-09-21T06:11:16.070608+0000","last_peered":"2024-09-21T06:11:16.070608+0000","last_clean":"2024-09-21T06:11:16.070608+0000","last_became_active":"2024-09-21T06:11:15.100932+0000","last_became_peered":"2024-09-21T06:11:15.100932+0000","last_unstale":"2024-09-21T06:11:16.070608+0000","last_undegraded":"2024-09-21T06:11:16.070608+0000","last_fullsized":"2024-09-21T06:11:16.070608+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:54.767335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.074303+0000","last_change":"2024-09-21T06:11:14.093351+0000","last_active":"2024-09-21T06:11:16.074303+0000","last_peered":"2024-09-21T06:11:16.074303+0000","last_clean":"2024-09-21T06:11:16.074303+0000","last_became_active":"2024-09-21T06:11:14.091566+0000","last_became_peered":"2024-09-21T06:11:14.091566+0000","last_unstale":"2024-09-21T06:11:16.074303+0000","last_undegraded":"2024-09-21T06:11:16.074303+0000","last_fullsized":"2024-09-21T06:11:16.074303+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:56.320019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.074236+0000","last_change":"2024-09-21T06:11:15.098035+0000","last_active":"2024-09-21T06:11:16.074236+0000","last_peered":"2024-09-21T06:11:16.074236+0000","last_clean":"2024-09-21T06:11:16.074236+0000","last_became_active":"2024-09-21T06:11:15.097720+0000","last_became_peered":"2024-09-21T06:11:15.097720+0000","last_unstale":"2024-09-21T06:11:16.074236+0000","last_undegraded":"2024-09-21T06:11:16.074236+0000","last_fullsized":"2024-09-21T06:11:16.074236+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:21.801561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092991+0000","last_change":"2024-09-21T06:11:14.082838+0000","last_active":"2024-09-21T06:11:16.092991+0000","last_peered":"2024-09-21T06:11:16.092991+0000","last_clean":"2024-09-21T06:11:16.092991+0000","last_became_active":"2024-09-21T06:11:14.082507+0000","last_became_peered":"2024-09-21T06:11:14.082507+0000","last_unstale":"2024-09-21T06:11:16.092991+0000","last_undegraded":"2024-09-21T06:11:16.092991+0000","last_fullsized":"2024-09-21T06:11:16.092991+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:07.328181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734366+0000","last_change":"2024-09-21T06:11:15.097547+0000","last_active":"2024-09-21T06:11:16.734366+0000","last_peered":"2024-09-21T06:11:16.734366+0000","last_clean":"2024-09-21T06:11:16.734366+0000","last_became_active":"2024-09-21T06:11:15.096519+0000","last_became_peered":"2024-09-21T06:11:15.096519+0000","last_unstale":"2024-09-21T06:11:16.734366+0000","last_undegraded":"2024-09-21T06:11:16.734366+0000","last_fullsized":"2024-09-21T06:11:16.734366+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:26.255775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"126'3","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.073922+0000","last_change":"2024-09-21T06:11:14.079129+0000","last_active":"2024-09-21T06:11:16.073922+0000","last_peered":"2024-09-21T06:11:16.073922+0000","last_clean":"2024-09-21T06:11:16.073922+0000","last_became_active":"2024-09-21T06:11:14.078794+0000","last_became_peered":"2024-09-21T06:11:14.078794+0000","last_unstale":"2024-09-21T06:11:16.073922+0000","last_undegraded":"2024-09-21T06:11:16.073922+0000","last_fullsized":"2024-09-21T06:11:16.073922+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:36.142770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733806+0000","last_change":"2024-09-21T06:11:15.092606+0000","last_active":"2024-09-21T06:11:16.733806+0000","last_peered":"2024-09-21T06:11:16.733806+0000","last_clean":"2024-09-21T06:11:16.733806+0000","last_became_active":"2024-09-21T06:11:15.091721+0000","last_became_peered":"2024-09-21T06:11:15.091721+0000","last_unstale":"2024-09-21T06:11:16.733806+0000","last_undegraded":"2024-09-21T06:11:16.733806+0000","last_fullsized":"2024-09-21T06:11:16.733806+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:06:07.695357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"126'6","reported_seq":21,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733870+0000","last_change":"2024-09-21T06:11:14.085929+0000","last_active":"2024-09-21T06:11:16.733870+0000","last_peered":"2024-09-21T06:11:16.733870+0000","last_clean":"2024-09-21T06:11:16.733870+0000","last_became_active":"2024-09-21T06:11:14.085601+0000","last_became_peered":"2024-09-21T06:11:14.085601+0000","last_unstale":"2024-09-21T06:11:16.733870+0000","last_undegraded":"2024-09-21T06:11:16.733870+0000","last_fullsized":"2024-09-21T06:11:16.733870+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:34.285189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070290+0000","last_change":"2024-09-21T06:11:15.099518+0000","last_active":"2024-09-21T06:11:16.070290+0000","last_peered":"2024-09-21T06:11:16.070290+0000","last_clean":"2024-09-21T06:11:16.070290+0000","last_became_active":"2024-09-21T06:11:15.097037+0000","last_became_peered":"2024-09-21T06:11:15.097037+0000","last_unstale":"2024-09-21T06:11:16.070290+0000","last_undegraded":"2024-09-21T06:11:16.070290+0000","last_fullsized":"2024-09-21T06:11:16.070290+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:34.274129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.069952+0000","last_change":"2024-09-21T06:11:14.094742+0000","last_active":"2024-09-21T06:11:16.069952+0000","last_peered":"2024-09-21T06:11:16.069952+0000","last_clean":"2024-09-21T06:11:16.069952+0000","last_became_active":"2024-09-21T06:11:14.094296+0000","last_became_peered":"2024-09-21T06:11:14.094296+0000","last_unstale":"2024-09-21T06:11:16.069952+0000","last_undegraded":"2024-09-21T06:11:16.069952+0000","last_fullsized":"2024-09-21T06:11:16.069952+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:38.795110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733284+0000","last_change":"2024-09-21T06:11:15.091269+0000","last_active":"2024-09-21T06:11:16.733284+0000","last_peered":"2024-09-21T06:11:16.733284+0000","last_clean":"2024-09-21T06:11:16.733284+0000","last_became_active":"2024-09-21T06:11:15.090867+0000","last_became_peered":"2024-09-21T06:11:15.090867+0000","last_unstale":"2024-09-21T06:11:16.733284+0000","last_undegraded":"2024-09-21T06:11:16.733284+0000","last_fullsized":"2024-09-21T06:11:16.733284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:55.196646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733380+0000","last_change":"2024-09-21T06:11:14.088722+0000","last_active":"2024-09-21T06:11:16.733380+0000","last_peered":"2024-09-21T06:11:16.733380+0000","last_clean":"2024-09-21T06:11:16.733380+0000","last_became_active":"2024-09-21T06:11:14.087894+0000","last_became_peered":"2024-09-21T06:11:14.087894+0000","last_unstale":"2024-09-21T06:11:16.733380+0000","last_undegraded":"2024-09-21T06:11:16.733380+0000","last_fullsized":"2024-09-21T06:11:16.733380+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:07.215003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733175+0000","last_change":"2024-09-21T06:11:15.079806+0000","last_active":"2024-09-21T06:11:16.733175+0000","last_peered":"2024-09-21T06:11:16.733175+0000","last_clean":"2024-09-21T06:11:16.733175+0000","last_became_active":"2024-09-21T06:11:15.079416+0000","last_became_peered":"2024-09-21T06:11:15.079416+0000","last_unstale":"2024-09-21T06:11:16.733175+0000","last_undegraded":"2024-09-21T06:11:16.733175+0000","last_fullsized":"2024-09-21T06:11:16.733175+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:33.035953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"126'8","reported_seq":23,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733518+0000","last_change":"2024-09-21T06:11:14.088130+0000","last_active":"2024-09-21T06:11:16.733518+0000","last_peered":"2024-09-21T06:11:16.733518+0000","last_clean":"2024-09-21T06:11:16.733518+0000","last_became_active":"2024-09-21T06:11:14.087804+0000","last_became_peered":"2024-09-21T06:11:14.087804+0000","last_unstale":"2024-09-21T06:11:16.733518+0000","last_undegraded":"2024-09-21T06:11:16.733518+0000","last_fullsized":"2024-09-21T06:11:16.733518+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:38.035381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734386+0000","last_change":"2024-09-21T06:11:15.110004+0000","last_active":"2024-09-21T06:11:16.734386+0000","last_peered":"2024-09-21T06:11:16.734386+0000","last_clean":"2024-09-21T06:11:16.734386+0000","last_became_active":"2024-09-21T06:11:15.109328+0000","last_became_peered":"2024-09-21T06:11:15.109328+0000","last_unstale":"2024-09-21T06:11:16.734386+0000","last_undegraded":"2024-09-21T06:11:16.734386+0000","last_fullsized":"2024-09-21T06:11:16.734386+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:01.693132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.074310+0000","last_change":"2024-09-21T06:11:14.086968+0000","last_active":"2024-09-21T06:11:16.074310+0000","last_peered":"2024-09-21T06:11:16.074310+0000","last_clean":"2024-09-21T06:11:16.074310+0000","last_became_active":"2024-09-21T06:11:14.086556+0000","last_became_peered":"2024-09-21T06:11:14.086556+0000","last_unstale":"2024-09-21T06:11:16.074310+0000","last_undegraded":"2024-09-21T06:11:16.074310+0000","last_fullsized":"2024-09-21T06:11:16.074310+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:15.804930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.074253+0000","last_change":"2024-09-21T06:11:15.076272+0000","last_active":"2024-09-21T06:11:16.074253+0000","last_peered":"2024-09-21T06:11:16.074253+0000","last_clean":"2024-09-21T06:11:16.074253+0000","last_became_active":"2024-09-21T06:11:15.075852+0000","last_became_peered":"2024-09-21T06:11:15.075852+0000","last_unstale":"2024-09-21T06:11:16.074253+0000","last_undegraded":"2024-09-21T06:11:16.074253+0000","last_fullsized":"2024-09-21T06:11:16.074253+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:57.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734425+0000","last_change":"2024-09-21T06:11:14.084620+0000","last_active":"2024-09-21T06:11:16.734425+0000","last_peered":"2024-09-21T06:11:16.734425+0000","last_clean":"2024-09-21T06:11:16.734425+0000","last_became_active":"2024-09-21T06:11:14.084113+0000","last_became_peered":"2024-09-21T06:11:14.084113+0000","last_unstale":"2024-09-21T06:11:16.734425+0000","last_undegraded":"2024-09-21T06:11:16.734425+0000","last_fullsized":"2024-09-21T06:11:16.734425+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:00.095854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092632+0000","last_change":"2024-09-21T06:11:15.098426+0000","last_active":"2024-09-21T06:11:16.092632+0000","last_peered":"2024-09-21T06:11:16.092632+0000","last_clean":"2024-09-21T06:11:16.092632+0000","last_became_active":"2024-09-21T06:11:15.098093+0000","last_became_peered":"2024-09-21T06:11:15.098093+0000","last_unstale":"2024-09-21T06:11:16.092632+0000","last_undegraded":"2024-09-21T06:11:16.092632+0000","last_fullsized":"2024-09-21T06:11:16.092632+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:59.008882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075864+0000","last_change":"2024-09-21T06:11:14.079722+0000","last_active":"2024-09-21T06:11:16.075864+0000","last_peered":"2024-09-21T06:11:16.075864+0000","last_clean":"2024-09-21T06:11:16.075864+0000","last_became_active":"2024-09-21T06:11:14.079366+0000","last_became_peered":"2024-09-21T06:11:14.079366+0000","last_unstale":"2024-09-21T06:11:16.075864+0000","last_undegraded":"2024-09-21T06:11:16.075864+0000","last_fullsized":"2024-09-21T06:11:16.075864+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:53.643799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734318+0000","last_change":"2024-09-21T06:11:15.098241+0000","last_active":"2024-09-21T06:11:16.734318+0000","last_peered":"2024-09-21T06:11:16.734318+0000","last_clean":"2024-09-21T06:11:16.734318+0000","last_became_active":"2024-09-21T06:11:15.097725+0000","last_became_peered":"2024-09-21T06:11:15.097725+0000","last_unstale":"2024-09-21T06:11:16.734318+0000","last_undegraded":"2024-09-21T06:11:16.734318+0000","last_fullsized":"2024-09-21T06:11:16.734318+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:55.906006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.073770+0000","last_change":"2024-09-21T06:11:14.082559+0000","last_active":"2024-09-21T06:11:16.073770+0000","last_peered":"2024-09-21T06:11:16.073770+0000","last_clean":"2024-09-21T06:11:16.073770+0000","last_became_active":"2024-09-21T06:11:14.082206+0000","last_became_peered":"2024-09-21T06:11:14.082206+0000","last_unstale":"2024-09-21T06:11:16.073770+0000","last_undegraded":"2024-09-21T06:11:16.073770+0000","last_fullsized":"2024-09-21T06:11:16.073770+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:29.893033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.069919+0000","last_change":"2024-09-21T06:11:15.105157+0000","last_active":"2024-09-21T06:11:16.069919+0000","last_peered":"2024-09-21T06:11:16.069919+0000","last_clean":"2024-09-21T06:11:16.069919+0000","last_became_active":"2024-09-21T06:11:15.104741+0000","last_became_peered":"2024-09-21T06:11:15.104741+0000","last_unstale":"2024-09-21T06:11:16.069919+0000","last_undegraded":"2024-09-21T06:11:16.069919+0000","last_fullsized":"2024-09-21T06:11:16.069919+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:25.940376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.073601+0000","last_change":"2024-09-21T06:11:14.093595+0000","last_active":"2024-09-21T06:11:16.073601+0000","last_peered":"2024-09-21T06:11:16.073601+0000","last_clean":"2024-09-21T06:11:16.073601+0000","last_became_active":"2024-09-21T06:11:14.092250+0000","last_became_peered":"2024-09-21T06:11:14.092250+0000","last_unstale":"2024-09-21T06:11:16.073601+0000","last_undegraded":"2024-09-21T06:11:16.073601+0000","last_fullsized":"2024-09-21T06:11:16.073601+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:53.764795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075686+0000","last_change":"2024-09-21T06:11:15.082685+0000","last_active":"2024-09-21T06:11:16.075686+0000","last_peered":"2024-09-21T06:11:16.075686+0000","last_clean":"2024-09-21T06:11:16.075686+0000","last_became_active":"2024-09-21T06:11:15.082393+0000","last_became_peered":"2024-09-21T06:11:15.082393+0000","last_unstale":"2024-09-21T06:11:16.075686+0000","last_undegraded":"2024-09-21T06:11:16.075686+0000","last_fullsized":"2024-09-21T06:11:16.075686+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:05.123399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092124+0000","last_change":"2024-09-21T06:11:14.086360+0000","last_active":"2024-09-21T06:11:16.092124+0000","last_peered":"2024-09-21T06:11:16.092124+0000","last_clean":"2024-09-21T06:11:16.092124+0000","last_became_active":"2024-09-21T06:11:14.086037+0000","last_became_peered":"2024-09-21T06:11:14.086037+0000","last_unstale":"2024-09-21T06:11:16.092124+0000","last_undegraded":"2024-09-21T06:11:16.092124+0000","last_fullsized":"2024-09-21T06:11:16.092124+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:26.230264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733233+0000","last_change":"2024-09-21T06:11:15.103917+0000","last_active":"2024-09-21T06:11:16.733233+0000","last_peered":"2024-09-21T06:11:16.733233+0000","last_clean":"2024-09-21T06:11:16.733233+0000","last_became_active":"2024-09-21T06:11:15.097800+0000","last_became_peered":"2024-09-21T06:11:15.097800+0000","last_unstale":"2024-09-21T06:11:16.733233+0000","last_undegraded":"2024-09-21T06:11:16.733233+0000","last_fullsized":"2024-09-21T06:11:16.733233+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:07.217818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.091808+0000","last_change":"2024-09-21T06:11:14.076227+0000","last_active":"2024-09-21T06:11:16.091808+0000","last_peered":"2024-09-21T06:11:16.091808+0000","last_clean":"2024-09-21T06:11:16.091808+0000","last_became_active":"2024-09-21T06:11:14.075950+0000","last_became_peered":"2024-09-21T06:11:14.075950+0000","last_unstale":"2024-09-21T06:11:16.091808+0000","last_undegraded":"2024-09-21T06:11:16.091808+0000","last_fullsized":"2024-09-21T06:11:16.091808+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:15.761300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733662+0000","last_change":"2024-09-21T06:11:15.111827+0000","last_active":"2024-09-21T06:11:16.733662+0000","last_peered":"2024-09-21T06:11:16.733662+0000","last_clean":"2024-09-21T06:11:16.733662+0000","last_became_active":"2024-09-21T06:11:15.111460+0000","last_became_peered":"2024-09-21T06:11:15.111460+0000","last_unstale":"2024-09-21T06:11:16.733662+0000","last_undegraded":"2024-09-21T06:11:16.733662+0000","last_fullsized":"2024-09-21T06:11:16.733662+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:14.095774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733770+0000","last_change":"2024-09-21T06:11:14.083452+0000","last_active":"2024-09-21T06:11:16.733770+0000","last_peered":"2024-09-21T06:11:16.733770+0000","last_clean":"2024-09-21T06:11:16.733770+0000","last_became_active":"2024-09-21T06:11:14.083117+0000","last_became_peered":"2024-09-21T06:11:14.083117+0000","last_unstale":"2024-09-21T06:11:16.733770+0000","last_undegraded":"2024-09-21T06:11:16.733770+0000","last_fullsized":"2024-09-21T06:11:16.733770+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:25.253235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.076077+0000","last_change":"2024-09-21T06:11:15.082815+0000","last_active":"2024-09-21T06:11:16.076077+0000","last_peered":"2024-09-21T06:11:16.076077+0000","last_clean":"2024-09-21T06:11:16.076077+0000","last_became_active":"2024-09-21T06:11:15.082445+0000","last_became_peered":"2024-09-21T06:11:15.082445+0000","last_unstale":"2024-09-21T06:11:16.076077+0000","last_undegraded":"2024-09-21T06:11:16.076077+0000","last_fullsized":"2024-09-21T06:11:16.076077+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:00.080102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070733+0000","last_change":"2024-09-21T06:11:14.096024+0000","last_active":"2024-09-21T06:11:16.070733+0000","last_peered":"2024-09-21T06:11:16.070733+0000","last_clean":"2024-09-21T06:11:16.070733+0000","last_became_active":"2024-09-21T06:11:14.095727+0000","last_became_peered":"2024-09-21T06:11:14.095727+0000","last_unstale":"2024-09-21T06:11:16.070733+0000","last_undegraded":"2024-09-21T06:11:16.070733+0000","last_fullsized":"2024-09-21T06:11:16.070733+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:25.153625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070622+0000","last_change":"2024-09-21T06:11:15.109848+0000","last_active":"2024-09-21T06:11:16.070622+0000","last_peered":"2024-09-21T06:11:16.070622+0000","last_clean":"2024-09-21T06:11:16.070622+0000","last_became_active":"2024-09-21T06:11:15.109489+0000","last_became_peered":"2024-09-21T06:11:15.109489+0000","last_unstale":"2024-09-21T06:11:16.070622+0000","last_undegraded":"2024-09-21T06:11:16.070622+0000","last_fullsized":"2024-09-21T06:11:16.070622+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:30.895573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"127'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734242+0000","last_change":"2024-09-21T06:11:14.092255+0000","last_active":"2024-09-21T06:11:16.734242+0000","last_peered":"2024-09-21T06:11:16.734242+0000","last_clean":"2024-09-21T06:11:16.734242+0000","last_became_active":"2024-09-21T06:11:14.091547+0000","last_became_peered":"2024-09-21T06:11:14.091547+0000","last_unstale":"2024-09-21T06:11:16.734242+0000","last_undegraded":"2024-09-21T06:11:16.734242+0000","last_fullsized":"2024-09-21T06:11:16.734242+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:47.693414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734620+0000","last_change":"2024-09-21T06:11:15.089194+0000","last_active":"2024-09-21T06:11:16.734620+0000","last_peered":"2024-09-21T06:11:16.734620+0000","last_clean":"2024-09-21T06:11:16.734620+0000","last_became_active":"2024-09-21T06:11:15.088212+0000","last_became_peered":"2024-09-21T06:11:15.088212+0000","last_unstale":"2024-09-21T06:11:16.734620+0000","last_undegraded":"2024-09-21T06:11:16.734620+0000","last_fullsized":"2024-09-21T06:11:16.734620+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:15.209294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734425+0000","last_change":"2024-09-21T06:11:14.084372+0000","last_active":"2024-09-21T06:11:16.734425+0000","last_peered":"2024-09-21T06:11:16.734425+0000","last_clean":"2024-09-21T06:11:16.734425+0000","last_became_active":"2024-09-21T06:11:14.084139+0000","last_became_peered":"2024-09-21T06:11:14.084139+0000","last_unstale":"2024-09-21T06:11:16.734425+0000","last_undegraded":"2024-09-21T06:11:16.734425+0000","last_fullsized":"2024-09-21T06:11:16.734425+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:07.303289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075883+0000","last_change":"2024-09-21T06:11:15.103627+0000","last_active":"2024-09-21T06:11:16.075883+0000","last_peered":"2024-09-21T06:11:16.075883+0000","last_clean":"2024-09-21T06:11:16.075883+0000","last_became_active":"2024-09-21T06:11:15.103283+0000","last_became_peered":"2024-09-21T06:11:15.103283+0000","last_unstale":"2024-09-21T06:11:16.075883+0000","last_undegraded":"2024-09-21T06:11:16.075883+0000","last_fullsized":"2024-09-21T06:11:16.075883+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:12.379746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733646+0000","last_change":"2024-09-21T06:11:14.084166+0000","last_active":"2024-09-21T06:11:16.733646+0000","last_peered":"2024-09-21T06:11:16.733646+0000","last_clean":"2024-09-21T06:11:16.733646+0000","last_became_active":"2024-09-21T06:11:14.083578+0000","last_became_peered":"2024-09-21T06:11:14.083578+0000","last_unstale":"2024-09-21T06:11:16.733646+0000","last_undegraded":"2024-09-21T06:11:16.733646+0000","last_fullsized":"2024-09-21T06:11:16.733646+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:59.922768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734040+0000","last_change":"2024-09-21T06:11:15.100569+0000","last_active":"2024-09-21T06:11:16.734040+0000","last_peered":"2024-09-21T06:11:16.734040+0000","last_clean":"2024-09-21T06:11:16.734040+0000","last_became_active":"2024-09-21T06:11:15.100250+0000","last_became_peered":"2024-09-21T06:11:15.100250+0000","last_unstale":"2024-09-21T06:11:16.734040+0000","last_undegraded":"2024-09-21T06:11:16.734040+0000","last_fullsized":"2024-09-21T06:11:16.734040+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:21.873054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733774+0000","last_change":"2024-09-21T06:11:14.087191+0000","last_active":"2024-09-21T06:11:16.733774+0000","last_peered":"2024-09-21T06:11:16.733774+0000","last_clean":"2024-09-21T06:11:16.733774+0000","last_became_active":"2024-09-21T06:11:14.086955+0000","last_became_peered":"2024-09-21T06:11:14.086955+0000","last_unstale":"2024-09-21T06:11:16.733774+0000","last_undegraded":"2024-09-21T06:11:16.733774+0000","last_fullsized":"2024-09-21T06:11:16.733774+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:20.335280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.073827+0000","last_change":"2024-09-21T06:11:15.114119+0000","last_active":"2024-09-21T06:11:16.073827+0000","last_peered":"2024-09-21T06:11:16.073827+0000","last_clean":"2024-09-21T06:11:16.073827+0000","last_became_active":"2024-09-21T06:11:15.113749+0000","last_became_peered":"2024-09-21T06:11:15.113749+0000","last_unstale":"2024-09-21T06:11:16.073827+0000","last_undegraded":"2024-09-21T06:11:16.073827+0000","last_fullsized":"2024-09-21T06:11:16.073827+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:02.874643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733778+0000","last_change":"2024-09-21T06:11:14.091971+0000","last_active":"2024-09-21T06:11:16.733778+0000","last_peered":"2024-09-21T06:11:16.733778+0000","last_clean":"2024-09-21T06:11:16.733778+0000","last_became_active":"2024-09-21T06:11:14.091587+0000","last_became_peered":"2024-09-21T06:11:14.091587+0000","last_unstale":"2024-09-21T06:11:16.733778+0000","last_undegraded":"2024-09-21T06:11:16.733778+0000","last_fullsized":"2024-09-21T06:11:16.733778+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:40.752652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092195+0000","last_change":"2024-09-21T06:11:15.094559+0000","last_active":"2024-09-21T06:11:16.092195+0000","last_peered":"2024-09-21T06:11:16.092195+0000","last_clean":"2024-09-21T06:11:16.092195+0000","last_became_active":"2024-09-21T06:11:15.092399+0000","last_became_peered":"2024-09-21T06:11:15.092399+0000","last_unstale":"2024-09-21T06:11:16.092195+0000","last_undegraded":"2024-09-21T06:11:16.092195+0000","last_fullsized":"2024-09-21T06:11:16.092195+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:01:02.518244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.091827+0000","last_change":"2024-09-21T06:11:14.086180+0000","last_active":"2024-09-21T06:11:16.091827+0000","last_peered":"2024-09-21T06:11:16.091827+0000","last_clean":"2024-09-21T06:11:16.091827+0000","last_became_active":"2024-09-21T06:11:14.085840+0000","last_became_peered":"2024-09-21T06:11:14.085840+0000","last_unstale":"2024-09-21T06:11:16.091827+0000","last_undegraded":"2024-09-21T06:11:16.091827+0000","last_fullsized":"2024-09-21T06:11:16.091827+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:45.658256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733405+0000","last_change":"2024-09-21T06:11:15.088407+0000","last_active":"2024-09-21T06:11:16.733405+0000","last_peered":"2024-09-21T06:11:16.733405+0000","last_clean":"2024-09-21T06:11:16.733405+0000","last_became_active":"2024-09-21T06:11:15.088041+0000","last_became_peered":"2024-09-21T06:11:15.088041+0000","last_unstale":"2024-09-21T06:11:16.733405+0000","last_undegraded":"2024-09-21T06:11:16.733405+0000","last_fullsized":"2024-09-21T06:11:16.733405+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:30.659562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734072+0000","last_change":"2024-09-21T06:11:14.082021+0000","last_active":"2024-09-21T06:11:16.734072+0000","last_peered":"2024-09-21T06:11:16.734072+0000","last_clean":"2024-09-21T06:11:16.734072+0000","last_became_active":"2024-09-21T06:11:14.081692+0000","last_became_peered":"2024-09-21T06:11:14.081692+0000","last_unstale":"2024-09-21T06:11:16.734072+0000","last_undegraded":"2024-09-21T06:11:16.734072+0000","last_fullsized":"2024-09-21T06:11:16.734072+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:41.871307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733335+0000","last_change":"2024-09-21T06:11:15.112024+0000","last_active":"2024-09-21T06:11:16.733335+0000","last_peered":"2024-09-21T06:11:16.733335+0000","last_clean":"2024-09-21T06:11:16.733335+0000","last_became_active":"2024-09-21T06:11:15.111665+0000","last_became_peered":"2024-09-21T06:11:15.111665+0000","last_unstale":"2024-09-21T06:11:16.733335+0000","last_undegraded":"2024-09-21T06:11:16.733335+0000","last_fullsized":"2024-09-21T06:11:16.733335+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:26.818828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734335+0000","last_change":"2024-09-21T06:11:14.093273+0000","last_active":"2024-09-21T06:11:16.734335+0000","last_peered":"2024-09-21T06:11:16.734335+0000","last_clean":"2024-09-21T06:11:16.734335+0000","last_became_active":"2024-09-21T06:11:14.092704+0000","last_became_peered":"2024-09-21T06:11:14.092704+0000","last_unstale":"2024-09-21T06:11:16.734335+0000","last_undegraded":"2024-09-21T06:11:16.734335+0000","last_fullsized":"2024-09-21T06:11:16.734335+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:04.842818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":306,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733401+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:11:16.733401+0000","last_peered":"2024-09-21T06:11:16.733401+0000","last_clean":"2024-09-21T06:11:16.733401+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:11:16.733401+0000","last_undegraded":"2024-09-21T06:11:16.733401+0000","last_fullsized":"2024-09-21T06:11:16.733401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509623,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131836,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240004,"statfs":{"total":96636764160,"available":96501764096,"internally_reserved":0,"allocated":2195456,"data_stored":1283457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509623,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140964,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230876,"statfs":{"total":96636764160,"available":96492417024,"internally_reserved":0,"allocated":1589248,"data_stored":692451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509622,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86824,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285016,"statfs":{"total":96636764160,"available":96547856384,"internally_reserved":0,"allocated":1585152,"data_stored":690951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509623,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153436,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218404,"statfs":{"total":96636764160,"available":96479645696,"internally_reserved":0,"allocated":1581056,"data_stored":690909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509622,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122920,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248920,"statfs":{"total":96636764160,"available":96510894080,"internally_reserved":0,"allocated":1585152,"data_stored":690999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509624,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140960,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230880,"statfs":{"total":96636764160,"available":96492421120,"internally_reserved":0,"allocated":1585152,"data_stored":691455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509622,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123484,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248356,"statfs":{"total":96636764160,"available":96510316544,"internally_reserved":0,"allocated":2170880,"data_stored":1281207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542327,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167308,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204532,"statfs":{"total":96636764160,"available":96465440768,"internally_reserved":0,"allocated":1564672,"data_stored":684455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:11:19.295 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:11:19.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.491+0000 7f05ee400640 1 -- 172.21.15.50:0/3060345230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05cc004010 msgr2=0x7f05cc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.491+0000 7f05ee400640 1 --2- 172.21.15.50:0/3060345230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05cc004010 0x7f05cc004810 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f05d00016f0 tx=0x7f05d001c780 comp rx=0 tx=0).stop 2024-09-21T06:11:19.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.491+0000 7f05eda00640 1 -- 172.21.15.50:0/3060345230 reap_dead start 2024-09-21T06:11:19.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.491+0000 7f05ee400640 1 -- 172.21.15.50:0/3060345230 shutdown_connections 2024-09-21T06:11:19.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.491+0000 7f05ee400640 1 -- 172.21.15.50:0/3060345230 >> 172.21.15.50:0/3060345230 conn(0x7f05e806cdf0 msgr2=0x7f05e806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:19.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.491+0000 7f05ee400640 1 -- 172.21.15.50:0/3060345230 shutdown_connections 2024-09-21T06:11:19.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.491+0000 7f05ee400640 1 -- 172.21.15.50:0/3060345230 wait complete. 2024-09-21T06:11:19.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05ee400640 1 Processor -- start 2024-09-21T06:11:19.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05ee400640 1 -- start start 2024-09-21T06:11:19.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05ee400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05cc004010 0x7f05e80af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:19.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05ee400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05e80ab8e0 0x7f05e80ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:19.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05ee400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05e80acbc0 0x7f05e81462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:19.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05ee400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f05e81676f0 con 0x7f05e80ab8e0 2024-09-21T06:11:19.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05ee400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05e8167570 con 0x7f05cc004010 2024-09-21T06:11:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05ee400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f05e8167870 con 0x7f05e80acbc0 2024-09-21T06:11:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05dfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05e80ab8e0 0x7f05e80ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05dfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05e80ab8e0 0x7f05e80ac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32828/0 (socket says 172.21.15.50:32828) 2024-09-21T06:11:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05dfe00640 1 -- 172.21.15.50:0/971731867 learned_addr learned my addr 172.21.15.50:0/971731867 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05ed000640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05cc004010 0x7f05e80af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05eda00640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05e80acbc0 0x7f05e81462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05dfe00640 1 -- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05e80acbc0 msgr2=0x7f05e81462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05dfe00640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05e80acbc0 0x7f05e81462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05dfe00640 1 -- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05cc004010 msgr2=0x7f05e80af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05dfe00640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05cc004010 0x7f05e80af070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05dfe00640 1 -- 172.21.15.50:0/971731867 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e80acfd0 con 0x7f05e80ab8e0 2024-09-21T06:11:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05ed000640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05cc004010 0x7f05e80af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:19.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05dfe00640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05e80ab8e0 0x7f05e80ac680 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f05d001b050 tx=0x7f05d00013a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:19.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05e5600640 1 -- 172.21.15.50:0/971731867 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0003600 con 0x7f05e80ab8e0 2024-09-21T06:11:19.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05e5600640 1 -- 172.21.15.50:0/971731867 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05d0003890 con 0x7f05e80ab8e0 2024-09-21T06:11:19.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05e5600640 1 -- 172.21.15.50:0/971731867 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0003c80 con 0x7f05e80ab8e0 2024-09-21T06:11:19.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05ee400640 1 -- 172.21.15.50:0/971731867 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e81f8ad0 con 0x7f05e80ab8e0 2024-09-21T06:11:19.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.495+0000 7f05ee400640 1 -- 172.21.15.50:0/971731867 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e81f9090 con 0x7f05e80ab8e0 2024-09-21T06:11:19.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.499+0000 7f05ee400640 1 -- 172.21.15.50:0/971731867 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e815f070 con 0x7f05e80ab8e0 2024-09-21T06:11:19.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.499+0000 7f05e5600640 1 -- 172.21.15.50:0/971731867 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f05d00a5580 con 0x7f05e80ab8e0 2024-09-21T06:11:19.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.499+0000 7f05e5600640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f05c4075180 0x7f05c4077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:19.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.499+0000 7f05e5600640 1 -- 172.21.15.50:0/971731867 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f05d012cba0 con 0x7f05e80ab8e0 2024-09-21T06:11:19.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.499+0000 7f05ed000640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f05c4075180 0x7f05c4077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:19.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.503+0000 7f05ed000640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f05c4075180 0x7f05c4077690 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f05d8000fc0 tx=0x7f05d80067b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:19.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.503+0000 7f05e5600640 1 -- 172.21.15.50:0/971731867 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f05e80acfd0 con 0x7f05e80ab8e0 2024-09-21T06:11:19.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.675+0000 7f05ee400640 1 -- 172.21.15.50:0/971731867 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05e806b540 con 0x7f05c4075180 2024-09-21T06:11:19.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.683+0000 7f05e5600640 1 -- 172.21.15.50:0/971731867 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252494 (secure 0 0 0) 0x7f05d8006d00 con 0x7f05c4075180 2024-09-21T06:11:19.682 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:19.684 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:11:19.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.687+0000 7f05ee400640 1 -- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f05c4075180 msgr2=0x7f05c4077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.687+0000 7f05ee400640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f05c4075180 0x7f05c4077690 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f05d8000fc0 tx=0x7f05d80067b0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.687+0000 7f05ee400640 1 -- 172.21.15.50:0/971731867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05e80ab8e0 msgr2=0x7f05e80ac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.687+0000 7f05ee400640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05e80ab8e0 0x7f05e80ac680 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f05d001b050 tx=0x7f05d00013a0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.687+0000 7f05ee400640 1 -- 172.21.15.50:0/971731867 shutdown_connections 2024-09-21T06:11:19.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.687+0000 7f05ee400640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05e80acbc0 0x7f05e81462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.687+0000 7f05ee400640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f05c4075180 0x7f05c4077690 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.687+0000 7f05ee400640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05e80ab8e0 0x7f05e80ac680 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.687+0000 7f05ee400640 1 --2- 172.21.15.50:0/971731867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05cc004010 0x7f05e80af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.687+0000 7f05ee400640 1 -- 172.21.15.50:0/971731867 >> 172.21.15.50:0/971731867 conn(0x7f05e806cdf0 msgr2=0x7f05e815d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:19.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.687+0000 7f05ee400640 1 -- 172.21.15.50:0/971731867 shutdown_connections 2024-09-21T06:11:19.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.687+0000 7f05ee400640 1 -- 172.21.15.50:0/971731867 wait complete. 2024-09-21T06:11:19.705 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":571,"stamp":"2024-09-21T06:11:19.208570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1067732,"kb_used_data":13532,"kb_used_omap":12,"kb_used_meta":1054003,"kb_avail":753906988,"statfs":{"total":773094113280,"available":772000755712,"internally_reserved":0,"allocated":13856768,"data_stored":6705884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1079299632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001594"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070700+0000","last_change":"2024-09-21T06:11:15.109497+0000","last_active":"2024-09-21T06:11:16.070700+0000","last_peered":"2024-09-21T06:11:16.070700+0000","last_clean":"2024-09-21T06:11:16.070700+0000","last_became_active":"2024-09-21T06:11:15.106405+0000","last_became_peered":"2024-09-21T06:11:15.106405+0000","last_unstale":"2024-09-21T06:11:16.070700+0000","last_undegraded":"2024-09-21T06:11:16.070700+0000","last_fullsized":"2024-09-21T06:11:16.070700+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:16.807470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734144+0000","last_change":"2024-09-21T06:11:15.088957+0000","last_active":"2024-09-21T06:11:16.734144+0000","last_peered":"2024-09-21T06:11:16.734144+0000","last_clean":"2024-09-21T06:11:16.734144+0000","last_became_active":"2024-09-21T06:11:15.088604+0000","last_became_peered":"2024-09-21T06:11:15.088604+0000","last_unstale":"2024-09-21T06:11:16.734144+0000","last_undegraded":"2024-09-21T06:11:16.734144+0000","last_fullsized":"2024-09-21T06:11:16.734144+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:38.255991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733755+0000","last_change":"2024-09-21T06:11:15.078587+0000","last_active":"2024-09-21T06:11:16.733755+0000","last_peered":"2024-09-21T06:11:16.733755+0000","last_clean":"2024-09-21T06:11:16.733755+0000","last_became_active":"2024-09-21T06:11:15.077576+0000","last_became_peered":"2024-09-21T06:11:15.077576+0000","last_unstale":"2024-09-21T06:11:16.733755+0000","last_undegraded":"2024-09-21T06:11:16.733755+0000","last_fullsized":"2024-09-21T06:11:16.733755+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:46.660796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070212+0000","last_change":"2024-09-21T06:11:15.107883+0000","last_active":"2024-09-21T06:11:16.070212+0000","last_peered":"2024-09-21T06:11:16.070212+0000","last_clean":"2024-09-21T06:11:16.070212+0000","last_became_active":"2024-09-21T06:11:15.106250+0000","last_became_peered":"2024-09-21T06:11:15.106250+0000","last_unstale":"2024-09-21T06:11:16.070212+0000","last_undegraded":"2024-09-21T06:11:16.070212+0000","last_fullsized":"2024-09-21T06:11:16.070212+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:13.724960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734109+0000","last_change":"2024-09-21T06:11:15.094376+0000","last_active":"2024-09-21T06:11:16.734109+0000","last_peered":"2024-09-21T06:11:16.734109+0000","last_clean":"2024-09-21T06:11:16.734109+0000","last_became_active":"2024-09-21T06:11:15.093830+0000","last_became_peered":"2024-09-21T06:11:15.093830+0000","last_unstale":"2024-09-21T06:11:16.734109+0000","last_undegraded":"2024-09-21T06:11:16.734109+0000","last_fullsized":"2024-09-21T06:11:16.734109+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:50.552668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075515+0000","last_change":"2024-09-21T06:11:15.103206+0000","last_active":"2024-09-21T06:11:16.075515+0000","last_peered":"2024-09-21T06:11:16.075515+0000","last_clean":"2024-09-21T06:11:16.075515+0000","last_became_active":"2024-09-21T06:11:15.102740+0000","last_became_peered":"2024-09-21T06:11:15.102740+0000","last_unstale":"2024-09-21T06:11:16.075515+0000","last_undegraded":"2024-09-21T06:11:16.075515+0000","last_fullsized":"2024-09-21T06:11:16.075515+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:44.695527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075497+0000","last_change":"2024-09-21T06:11:15.081361+0000","last_active":"2024-09-21T06:11:16.075497+0000","last_peered":"2024-09-21T06:11:16.075497+0000","last_clean":"2024-09-21T06:11:16.075497+0000","last_became_active":"2024-09-21T06:11:15.081064+0000","last_became_peered":"2024-09-21T06:11:15.081064+0000","last_unstale":"2024-09-21T06:11:16.075497+0000","last_undegraded":"2024-09-21T06:11:16.075497+0000","last_fullsized":"2024-09-21T06:11:16.075497+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:27.232804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733070+0000","last_change":"2024-09-21T06:11:15.093701+0000","last_active":"2024-09-21T06:11:16.733070+0000","last_peered":"2024-09-21T06:11:16.733070+0000","last_clean":"2024-09-21T06:11:16.733070+0000","last_became_active":"2024-09-21T06:11:15.093044+0000","last_became_peered":"2024-09-21T06:11:15.093044+0000","last_unstale":"2024-09-21T06:11:16.733070+0000","last_undegraded":"2024-09-21T06:11:16.733070+0000","last_fullsized":"2024-09-21T06:11:16.733070+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:28.621442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734386+0000","last_change":"2024-09-21T06:11:15.079906+0000","last_active":"2024-09-21T06:11:16.734386+0000","last_peered":"2024-09-21T06:11:16.734386+0000","last_clean":"2024-09-21T06:11:16.734386+0000","last_became_active":"2024-09-21T06:11:15.079581+0000","last_became_peered":"2024-09-21T06:11:15.079581+0000","last_unstale":"2024-09-21T06:11:16.734386+0000","last_undegraded":"2024-09-21T06:11:16.734386+0000","last_fullsized":"2024-09-21T06:11:16.734386+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:08.330721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734631+0000","last_change":"2024-09-21T06:11:15.090615+0000","last_active":"2024-09-21T06:11:16.734631+0000","last_peered":"2024-09-21T06:11:16.734631+0000","last_clean":"2024-09-21T06:11:16.734631+0000","last_became_active":"2024-09-21T06:11:15.090250+0000","last_became_peered":"2024-09-21T06:11:15.090250+0000","last_unstale":"2024-09-21T06:11:16.734631+0000","last_undegraded":"2024-09-21T06:11:16.734631+0000","last_fullsized":"2024-09-21T06:11:16.734631+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:33.123828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733954+0000","last_change":"2024-09-21T06:11:15.087419+0000","last_active":"2024-09-21T06:11:16.733954+0000","last_peered":"2024-09-21T06:11:16.733954+0000","last_clean":"2024-09-21T06:11:16.733954+0000","last_became_active":"2024-09-21T06:11:15.087068+0000","last_became_peered":"2024-09-21T06:11:15.087068+0000","last_unstale":"2024-09-21T06:11:16.733954+0000","last_undegraded":"2024-09-21T06:11:16.733954+0000","last_fullsized":"2024-09-21T06:11:16.733954+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:59.549881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733687+0000","last_change":"2024-09-21T06:11:15.079038+0000","last_active":"2024-09-21T06:11:16.733687+0000","last_peered":"2024-09-21T06:11:16.733687+0000","last_clean":"2024-09-21T06:11:16.733687+0000","last_became_active":"2024-09-21T06:11:15.078716+0000","last_became_peered":"2024-09-21T06:11:15.078716+0000","last_unstale":"2024-09-21T06:11:16.733687+0000","last_undegraded":"2024-09-21T06:11:16.733687+0000","last_fullsized":"2024-09-21T06:11:16.733687+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:16.763840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070018+0000","last_change":"2024-09-21T06:11:15.113803+0000","last_active":"2024-09-21T06:11:16.070018+0000","last_peered":"2024-09-21T06:11:16.070018+0000","last_clean":"2024-09-21T06:11:16.070018+0000","last_became_active":"2024-09-21T06:11:15.112722+0000","last_became_peered":"2024-09-21T06:11:15.112722+0000","last_unstale":"2024-09-21T06:11:16.070018+0000","last_undegraded":"2024-09-21T06:11:16.070018+0000","last_fullsized":"2024-09-21T06:11:16.070018+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:42.600363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733567+0000","last_change":"2024-09-21T06:11:15.110297+0000","last_active":"2024-09-21T06:11:16.733567+0000","last_peered":"2024-09-21T06:11:16.733567+0000","last_clean":"2024-09-21T06:11:16.733567+0000","last_became_active":"2024-09-21T06:11:15.109972+0000","last_became_peered":"2024-09-21T06:11:15.109972+0000","last_unstale":"2024-09-21T06:11:16.733567+0000","last_undegraded":"2024-09-21T06:11:16.733567+0000","last_fullsized":"2024-09-21T06:11:16.733567+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:23.201040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.069748+0000","last_change":"2024-09-21T06:11:15.098055+0000","last_active":"2024-09-21T06:11:16.069748+0000","last_peered":"2024-09-21T06:11:16.069748+0000","last_clean":"2024-09-21T06:11:16.069748+0000","last_became_active":"2024-09-21T06:11:15.097782+0000","last_became_peered":"2024-09-21T06:11:15.097782+0000","last_unstale":"2024-09-21T06:11:16.069748+0000","last_undegraded":"2024-09-21T06:11:16.069748+0000","last_fullsized":"2024-09-21T06:11:16.069748+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:15.938620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092127+0000","last_change":"2024-09-21T06:11:15.110173+0000","last_active":"2024-09-21T06:11:16.092127+0000","last_peered":"2024-09-21T06:11:16.092127+0000","last_clean":"2024-09-21T06:11:16.092127+0000","last_became_active":"2024-09-21T06:11:15.109787+0000","last_became_peered":"2024-09-21T06:11:15.109787+0000","last_unstale":"2024-09-21T06:11:16.092127+0000","last_undegraded":"2024-09-21T06:11:16.092127+0000","last_fullsized":"2024-09-21T06:11:16.092127+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:52.776766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.076098+0000","last_change":"2024-09-21T06:11:15.087445+0000","last_active":"2024-09-21T06:11:16.076098+0000","last_peered":"2024-09-21T06:11:16.076098+0000","last_clean":"2024-09-21T06:11:16.076098+0000","last_became_active":"2024-09-21T06:11:15.086239+0000","last_became_peered":"2024-09-21T06:11:15.086239+0000","last_unstale":"2024-09-21T06:11:16.076098+0000","last_undegraded":"2024-09-21T06:11:16.076098+0000","last_fullsized":"2024-09-21T06:11:16.076098+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:42.413437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092895+0000","last_change":"2024-09-21T06:11:15.098012+0000","last_active":"2024-09-21T06:11:16.092895+0000","last_peered":"2024-09-21T06:11:16.092895+0000","last_clean":"2024-09-21T06:11:16.092895+0000","last_became_active":"2024-09-21T06:11:15.097676+0000","last_became_peered":"2024-09-21T06:11:15.097676+0000","last_unstale":"2024-09-21T06:11:16.092895+0000","last_undegraded":"2024-09-21T06:11:16.092895+0000","last_fullsized":"2024-09-21T06:11:16.092895+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:14.095774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734260+0000","last_change":"2024-09-21T06:11:15.080470+0000","last_active":"2024-09-21T06:11:16.734260+0000","last_peered":"2024-09-21T06:11:16.734260+0000","last_clean":"2024-09-21T06:11:16.734260+0000","last_became_active":"2024-09-21T06:11:15.080110+0000","last_became_peered":"2024-09-21T06:11:15.080110+0000","last_unstale":"2024-09-21T06:11:16.734260+0000","last_undegraded":"2024-09-21T06:11:16.734260+0000","last_fullsized":"2024-09-21T06:11:16.734260+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:45.033650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092599+0000","last_change":"2024-09-21T06:11:15.094974+0000","last_active":"2024-09-21T06:11:16.092599+0000","last_peered":"2024-09-21T06:11:16.092599+0000","last_clean":"2024-09-21T06:11:16.092599+0000","last_became_active":"2024-09-21T06:11:15.094667+0000","last_became_peered":"2024-09-21T06:11:15.094667+0000","last_unstale":"2024-09-21T06:11:16.092599+0000","last_undegraded":"2024-09-21T06:11:16.092599+0000","last_fullsized":"2024-09-21T06:11:16.092599+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:26.818828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734327+0000","last_change":"2024-09-21T06:11:15.093205+0000","last_active":"2024-09-21T06:11:16.734327+0000","last_peered":"2024-09-21T06:11:16.734327+0000","last_clean":"2024-09-21T06:11:16.734327+0000","last_became_active":"2024-09-21T06:11:15.092864+0000","last_became_peered":"2024-09-21T06:11:15.092864+0000","last_unstale":"2024-09-21T06:11:16.734327+0000","last_undegraded":"2024-09-21T06:11:16.734327+0000","last_fullsized":"2024-09-21T06:11:16.734327+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:56.164663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733600+0000","last_change":"2024-09-21T06:11:15.088356+0000","last_active":"2024-09-21T06:11:16.733600+0000","last_peered":"2024-09-21T06:11:16.733600+0000","last_clean":"2024-09-21T06:11:16.733600+0000","last_became_active":"2024-09-21T06:11:15.087655+0000","last_became_peered":"2024-09-21T06:11:15.087655+0000","last_unstale":"2024-09-21T06:11:16.733600+0000","last_undegraded":"2024-09-21T06:11:16.733600+0000","last_fullsized":"2024-09-21T06:11:16.733600+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:02.985581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733206+0000","last_change":"2024-09-21T06:11:15.076156+0000","last_active":"2024-09-21T06:11:16.733206+0000","last_peered":"2024-09-21T06:11:16.733206+0000","last_clean":"2024-09-21T06:11:16.733206+0000","last_became_active":"2024-09-21T06:11:15.075829+0000","last_became_peered":"2024-09-21T06:11:15.075829+0000","last_unstale":"2024-09-21T06:11:16.733206+0000","last_undegraded":"2024-09-21T06:11:16.733206+0000","last_fullsized":"2024-09-21T06:11:16.733206+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:59.603828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734388+0000","last_change":"2024-09-21T06:11:15.075031+0000","last_active":"2024-09-21T06:11:16.734388+0000","last_peered":"2024-09-21T06:11:16.734388+0000","last_clean":"2024-09-21T06:11:16.734388+0000","last_became_active":"2024-09-21T06:11:15.074707+0000","last_became_peered":"2024-09-21T06:11:15.074707+0000","last_unstale":"2024-09-21T06:11:16.734388+0000","last_undegraded":"2024-09-21T06:11:16.734388+0000","last_fullsized":"2024-09-21T06:11:16.734388+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:24.911773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734183+0000","last_change":"2024-09-21T06:11:15.093790+0000","last_active":"2024-09-21T06:11:16.734183+0000","last_peered":"2024-09-21T06:11:16.734183+0000","last_clean":"2024-09-21T06:11:16.734183+0000","last_became_active":"2024-09-21T06:11:15.093457+0000","last_became_peered":"2024-09-21T06:11:15.093457+0000","last_unstale":"2024-09-21T06:11:16.734183+0000","last_undegraded":"2024-09-21T06:11:16.734183+0000","last_fullsized":"2024-09-21T06:11:16.734183+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:48.446640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092920+0000","last_change":"2024-09-21T06:11:15.111543+0000","last_active":"2024-09-21T06:11:16.092920+0000","last_peered":"2024-09-21T06:11:16.092920+0000","last_clean":"2024-09-21T06:11:16.092920+0000","last_became_active":"2024-09-21T06:11:15.109730+0000","last_became_peered":"2024-09-21T06:11:15.109730+0000","last_unstale":"2024-09-21T06:11:16.092920+0000","last_undegraded":"2024-09-21T06:11:16.092920+0000","last_fullsized":"2024-09-21T06:11:16.092920+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:06:07.695357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070209+0000","last_change":"2024-09-21T06:11:15.113166+0000","last_active":"2024-09-21T06:11:16.070209+0000","last_peered":"2024-09-21T06:11:16.070209+0000","last_clean":"2024-09-21T06:11:16.070209+0000","last_became_active":"2024-09-21T06:11:15.112798+0000","last_became_peered":"2024-09-21T06:11:15.112798+0000","last_unstale":"2024-09-21T06:11:16.070209+0000","last_undegraded":"2024-09-21T06:11:16.070209+0000","last_fullsized":"2024-09-21T06:11:16.070209+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:16.990551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.073876+0000","last_change":"2024-09-21T06:11:15.095407+0000","last_active":"2024-09-21T06:11:16.073876+0000","last_peered":"2024-09-21T06:11:16.073876+0000","last_clean":"2024-09-21T06:11:16.073876+0000","last_became_active":"2024-09-21T06:11:15.095080+0000","last_became_peered":"2024-09-21T06:11:15.095080+0000","last_unstale":"2024-09-21T06:11:16.073876+0000","last_undegraded":"2024-09-21T06:11:16.073876+0000","last_fullsized":"2024-09-21T06:11:16.073876+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:55.410180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070136+0000","last_change":"2024-09-21T06:11:15.108159+0000","last_active":"2024-09-21T06:11:16.070136+0000","last_peered":"2024-09-21T06:11:16.070136+0000","last_clean":"2024-09-21T06:11:16.070136+0000","last_became_active":"2024-09-21T06:11:15.097350+0000","last_became_peered":"2024-09-21T06:11:15.097350+0000","last_unstale":"2024-09-21T06:11:16.070136+0000","last_undegraded":"2024-09-21T06:11:16.070136+0000","last_fullsized":"2024-09-21T06:11:16.070136+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:14.205673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.069878+0000","last_change":"2024-09-21T06:11:15.109650+0000","last_active":"2024-09-21T06:11:16.069878+0000","last_peered":"2024-09-21T06:11:16.069878+0000","last_clean":"2024-09-21T06:11:16.069878+0000","last_became_active":"2024-09-21T06:11:15.109283+0000","last_became_peered":"2024-09-21T06:11:15.109283+0000","last_unstale":"2024-09-21T06:11:16.069878+0000","last_undegraded":"2024-09-21T06:11:16.069878+0000","last_fullsized":"2024-09-21T06:11:16.069878+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:01.114801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734037+0000","last_change":"2024-09-21T06:11:15.111180+0000","last_active":"2024-09-21T06:11:16.734037+0000","last_peered":"2024-09-21T06:11:16.734037+0000","last_clean":"2024-09-21T06:11:16.734037+0000","last_became_active":"2024-09-21T06:11:15.110858+0000","last_became_peered":"2024-09-21T06:11:15.110858+0000","last_unstale":"2024-09-21T06:11:16.734037+0000","last_undegraded":"2024-09-21T06:11:16.734037+0000","last_fullsized":"2024-09-21T06:11:16.734037+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:38.734624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075301+0000","last_change":"2024-09-21T06:11:15.097185+0000","last_active":"2024-09-21T06:11:16.075301+0000","last_peered":"2024-09-21T06:11:16.075301+0000","last_clean":"2024-09-21T06:11:16.075301+0000","last_became_active":"2024-09-21T06:11:15.096830+0000","last_became_peered":"2024-09-21T06:11:15.096830+0000","last_unstale":"2024-09-21T06:11:16.075301+0000","last_undegraded":"2024-09-21T06:11:16.075301+0000","last_fullsized":"2024-09-21T06:11:16.075301+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:01.693132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.093118+0000","last_change":"2024-09-21T06:11:15.093486+0000","last_active":"2024-09-21T06:11:16.093118+0000","last_peered":"2024-09-21T06:11:16.093118+0000","last_clean":"2024-09-21T06:11:16.093118+0000","last_became_active":"2024-09-21T06:11:15.093103+0000","last_became_peered":"2024-09-21T06:11:15.093103+0000","last_unstale":"2024-09-21T06:11:16.093118+0000","last_undegraded":"2024-09-21T06:11:16.093118+0000","last_fullsized":"2024-09-21T06:11:16.093118+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:38.734624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"126'4","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733680+0000","last_change":"2024-09-21T06:11:14.092934+0000","last_active":"2024-09-21T06:11:16.733680+0000","last_peered":"2024-09-21T06:11:16.733680+0000","last_clean":"2024-09-21T06:11:16.733680+0000","last_became_active":"2024-09-21T06:11:14.092573+0000","last_became_peered":"2024-09-21T06:11:14.092573+0000","last_unstale":"2024-09-21T06:11:16.733680+0000","last_undegraded":"2024-09-21T06:11:16.733680+0000","last_fullsized":"2024-09-21T06:11:16.733680+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:43.692987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734665+0000","last_change":"2024-09-21T06:11:15.099589+0000","last_active":"2024-09-21T06:11:16.734665+0000","last_peered":"2024-09-21T06:11:16.734665+0000","last_clean":"2024-09-21T06:11:16.734665+0000","last_became_active":"2024-09-21T06:11:15.099253+0000","last_became_peered":"2024-09-21T06:11:15.099253+0000","last_unstale":"2024-09-21T06:11:16.734665+0000","last_undegraded":"2024-09-21T06:11:16.734665+0000","last_fullsized":"2024-09-21T06:11:16.734665+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:27.232804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.074041+0000","last_change":"2024-09-21T06:11:14.082687+0000","last_active":"2024-09-21T06:11:16.074041+0000","last_peered":"2024-09-21T06:11:16.074041+0000","last_clean":"2024-09-21T06:11:16.074041+0000","last_became_active":"2024-09-21T06:11:14.082250+0000","last_became_peered":"2024-09-21T06:11:14.082250+0000","last_unstale":"2024-09-21T06:11:16.074041+0000","last_undegraded":"2024-09-21T06:11:16.074041+0000","last_fullsized":"2024-09-21T06:11:16.074041+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:33.271588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734122+0000","last_change":"2024-09-21T06:11:15.097931+0000","last_active":"2024-09-21T06:11:16.734122+0000","last_peered":"2024-09-21T06:11:16.734122+0000","last_clean":"2024-09-21T06:11:16.734122+0000","last_became_active":"2024-09-21T06:11:15.096894+0000","last_became_peered":"2024-09-21T06:11:15.096894+0000","last_unstale":"2024-09-21T06:11:16.734122+0000","last_undegraded":"2024-09-21T06:11:16.734122+0000","last_fullsized":"2024-09-21T06:11:16.734122+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:10.140285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734159+0000","last_change":"2024-09-21T06:11:14.089797+0000","last_active":"2024-09-21T06:11:16.734159+0000","last_peered":"2024-09-21T06:11:16.734159+0000","last_clean":"2024-09-21T06:11:16.734159+0000","last_became_active":"2024-09-21T06:11:14.089503+0000","last_became_peered":"2024-09-21T06:11:14.089503+0000","last_unstale":"2024-09-21T06:11:16.734159+0000","last_undegraded":"2024-09-21T06:11:16.734159+0000","last_fullsized":"2024-09-21T06:11:16.734159+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:04.628772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733461+0000","last_change":"2024-09-21T06:11:15.096053+0000","last_active":"2024-09-21T06:11:16.733461+0000","last_peered":"2024-09-21T06:11:16.733461+0000","last_clean":"2024-09-21T06:11:16.733461+0000","last_became_active":"2024-09-21T06:11:15.095486+0000","last_became_peered":"2024-09-21T06:11:15.095486+0000","last_unstale":"2024-09-21T06:11:16.733461+0000","last_undegraded":"2024-09-21T06:11:16.733461+0000","last_fullsized":"2024-09-21T06:11:16.733461+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:59.008882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733987+0000","last_change":"2024-09-21T06:11:14.081889+0000","last_active":"2024-09-21T06:11:16.733987+0000","last_peered":"2024-09-21T06:11:16.733987+0000","last_clean":"2024-09-21T06:11:16.733987+0000","last_became_active":"2024-09-21T06:11:14.081204+0000","last_became_peered":"2024-09-21T06:11:14.081204+0000","last_unstale":"2024-09-21T06:11:16.733987+0000","last_undegraded":"2024-09-21T06:11:16.733987+0000","last_fullsized":"2024-09-21T06:11:16.733987+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:20.460183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092017+0000","last_change":"2024-09-21T06:11:15.102737+0000","last_active":"2024-09-21T06:11:16.092017+0000","last_peered":"2024-09-21T06:11:16.092017+0000","last_clean":"2024-09-21T06:11:16.092017+0000","last_became_active":"2024-09-21T06:11:15.101542+0000","last_became_peered":"2024-09-21T06:11:15.101542+0000","last_unstale":"2024-09-21T06:11:16.092017+0000","last_undegraded":"2024-09-21T06:11:16.092017+0000","last_fullsized":"2024-09-21T06:11:16.092017+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:48.919648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733922+0000","last_change":"2024-09-21T06:11:14.077079+0000","last_active":"2024-09-21T06:11:16.733922+0000","last_peered":"2024-09-21T06:11:16.733922+0000","last_clean":"2024-09-21T06:11:16.733922+0000","last_became_active":"2024-09-21T06:11:14.076767+0000","last_became_peered":"2024-09-21T06:11:14.076767+0000","last_unstale":"2024-09-21T06:11:16.733922+0000","last_undegraded":"2024-09-21T06:11:16.733922+0000","last_fullsized":"2024-09-21T06:11:16.733922+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:01.324687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075665+0000","last_change":"2024-09-21T06:11:15.086370+0000","last_active":"2024-09-21T06:11:16.075665+0000","last_peered":"2024-09-21T06:11:16.075665+0000","last_clean":"2024-09-21T06:11:16.075665+0000","last_became_active":"2024-09-21T06:11:15.086012+0000","last_became_peered":"2024-09-21T06:11:15.086012+0000","last_unstale":"2024-09-21T06:11:16.075665+0000","last_undegraded":"2024-09-21T06:11:16.075665+0000","last_fullsized":"2024-09-21T06:11:16.075665+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:51.193220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.091850+0000","last_change":"2024-09-21T06:11:14.080257+0000","last_active":"2024-09-21T06:11:16.091850+0000","last_peered":"2024-09-21T06:11:16.091850+0000","last_clean":"2024-09-21T06:11:16.091850+0000","last_became_active":"2024-09-21T06:11:14.079840+0000","last_became_peered":"2024-09-21T06:11:14.079840+0000","last_unstale":"2024-09-21T06:11:16.091850+0000","last_undegraded":"2024-09-21T06:11:16.091850+0000","last_fullsized":"2024-09-21T06:11:16.091850+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:00.690591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733439+0000","last_change":"2024-09-21T06:11:15.093389+0000","last_active":"2024-09-21T06:11:16.733439+0000","last_peered":"2024-09-21T06:11:16.733439+0000","last_clean":"2024-09-21T06:11:16.733439+0000","last_became_active":"2024-09-21T06:11:15.092636+0000","last_became_peered":"2024-09-21T06:11:15.092636+0000","last_unstale":"2024-09-21T06:11:16.733439+0000","last_undegraded":"2024-09-21T06:11:16.733439+0000","last_fullsized":"2024-09-21T06:11:16.733439+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:01.030644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075254+0000","last_change":"2024-09-21T06:11:14.085843+0000","last_active":"2024-09-21T06:11:16.075254+0000","last_peered":"2024-09-21T06:11:16.075254+0000","last_clean":"2024-09-21T06:11:16.075254+0000","last_became_active":"2024-09-21T06:11:14.085512+0000","last_became_peered":"2024-09-21T06:11:14.085512+0000","last_unstale":"2024-09-21T06:11:16.075254+0000","last_undegraded":"2024-09-21T06:11:16.075254+0000","last_fullsized":"2024-09-21T06:11:16.075254+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:29.330942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733248+0000","last_change":"2024-09-21T06:11:15.103046+0000","last_active":"2024-09-21T06:11:16.733248+0000","last_peered":"2024-09-21T06:11:16.733248+0000","last_clean":"2024-09-21T06:11:16.733248+0000","last_became_active":"2024-09-21T06:11:15.102737+0000","last_became_peered":"2024-09-21T06:11:15.102737+0000","last_unstale":"2024-09-21T06:11:16.733248+0000","last_undegraded":"2024-09-21T06:11:16.733248+0000","last_fullsized":"2024-09-21T06:11:16.733248+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:12.172196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733371+0000","last_change":"2024-09-21T06:11:14.084434+0000","last_active":"2024-09-21T06:11:16.733371+0000","last_peered":"2024-09-21T06:11:16.733371+0000","last_clean":"2024-09-21T06:11:16.733371+0000","last_became_active":"2024-09-21T06:11:14.084117+0000","last_became_peered":"2024-09-21T06:11:14.084117+0000","last_unstale":"2024-09-21T06:11:16.733371+0000","last_undegraded":"2024-09-21T06:11:16.733371+0000","last_fullsized":"2024-09-21T06:11:16.733371+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:33.655342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070608+0000","last_change":"2024-09-21T06:11:15.101334+0000","last_active":"2024-09-21T06:11:16.070608+0000","last_peered":"2024-09-21T06:11:16.070608+0000","last_clean":"2024-09-21T06:11:16.070608+0000","last_became_active":"2024-09-21T06:11:15.100932+0000","last_became_peered":"2024-09-21T06:11:15.100932+0000","last_unstale":"2024-09-21T06:11:16.070608+0000","last_undegraded":"2024-09-21T06:11:16.070608+0000","last_fullsized":"2024-09-21T06:11:16.070608+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:54.767335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.074303+0000","last_change":"2024-09-21T06:11:14.093351+0000","last_active":"2024-09-21T06:11:16.074303+0000","last_peered":"2024-09-21T06:11:16.074303+0000","last_clean":"2024-09-21T06:11:16.074303+0000","last_became_active":"2024-09-21T06:11:14.091566+0000","last_became_peered":"2024-09-21T06:11:14.091566+0000","last_unstale":"2024-09-21T06:11:16.074303+0000","last_undegraded":"2024-09-21T06:11:16.074303+0000","last_fullsized":"2024-09-21T06:11:16.074303+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:56.320019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.074236+0000","last_change":"2024-09-21T06:11:15.098035+0000","last_active":"2024-09-21T06:11:16.074236+0000","last_peered":"2024-09-21T06:11:16.074236+0000","last_clean":"2024-09-21T06:11:16.074236+0000","last_became_active":"2024-09-21T06:11:15.097720+0000","last_became_peered":"2024-09-21T06:11:15.097720+0000","last_unstale":"2024-09-21T06:11:16.074236+0000","last_undegraded":"2024-09-21T06:11:16.074236+0000","last_fullsized":"2024-09-21T06:11:16.074236+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:21.801561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092991+0000","last_change":"2024-09-21T06:11:14.082838+0000","last_active":"2024-09-21T06:11:16.092991+0000","last_peered":"2024-09-21T06:11:16.092991+0000","last_clean":"2024-09-21T06:11:16.092991+0000","last_became_active":"2024-09-21T06:11:14.082507+0000","last_became_peered":"2024-09-21T06:11:14.082507+0000","last_unstale":"2024-09-21T06:11:16.092991+0000","last_undegraded":"2024-09-21T06:11:16.092991+0000","last_fullsized":"2024-09-21T06:11:16.092991+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:07.328181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734366+0000","last_change":"2024-09-21T06:11:15.097547+0000","last_active":"2024-09-21T06:11:16.734366+0000","last_peered":"2024-09-21T06:11:16.734366+0000","last_clean":"2024-09-21T06:11:16.734366+0000","last_became_active":"2024-09-21T06:11:15.096519+0000","last_became_peered":"2024-09-21T06:11:15.096519+0000","last_unstale":"2024-09-21T06:11:16.734366+0000","last_undegraded":"2024-09-21T06:11:16.734366+0000","last_fullsized":"2024-09-21T06:11:16.734366+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:26.255775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"126'3","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.073922+0000","last_change":"2024-09-21T06:11:14.079129+0000","last_active":"2024-09-21T06:11:16.073922+0000","last_peered":"2024-09-21T06:11:16.073922+0000","last_clean":"2024-09-21T06:11:16.073922+0000","last_became_active":"2024-09-21T06:11:14.078794+0000","last_became_peered":"2024-09-21T06:11:14.078794+0000","last_unstale":"2024-09-21T06:11:16.073922+0000","last_undegraded":"2024-09-21T06:11:16.073922+0000","last_fullsized":"2024-09-21T06:11:16.073922+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:36.142770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733806+0000","last_change":"2024-09-21T06:11:15.092606+0000","last_active":"2024-09-21T06:11:16.733806+0000","last_peered":"2024-09-21T06:11:16.733806+0000","last_clean":"2024-09-21T06:11:16.733806+0000","last_became_active":"2024-09-21T06:11:15.091721+0000","last_became_peered":"2024-09-21T06:11:15.091721+0000","last_unstale":"2024-09-21T06:11:16.733806+0000","last_undegraded":"2024-09-21T06:11:16.733806+0000","last_fullsized":"2024-09-21T06:11:16.733806+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:06:07.695357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"126'6","reported_seq":21,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733870+0000","last_change":"2024-09-21T06:11:14.085929+0000","last_active":"2024-09-21T06:11:16.733870+0000","last_peered":"2024-09-21T06:11:16.733870+0000","last_clean":"2024-09-21T06:11:16.733870+0000","last_became_active":"2024-09-21T06:11:14.085601+0000","last_became_peered":"2024-09-21T06:11:14.085601+0000","last_unstale":"2024-09-21T06:11:16.733870+0000","last_undegraded":"2024-09-21T06:11:16.733870+0000","last_fullsized":"2024-09-21T06:11:16.733870+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:34.285189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070290+0000","last_change":"2024-09-21T06:11:15.099518+0000","last_active":"2024-09-21T06:11:16.070290+0000","last_peered":"2024-09-21T06:11:16.070290+0000","last_clean":"2024-09-21T06:11:16.070290+0000","last_became_active":"2024-09-21T06:11:15.097037+0000","last_became_peered":"2024-09-21T06:11:15.097037+0000","last_unstale":"2024-09-21T06:11:16.070290+0000","last_undegraded":"2024-09-21T06:11:16.070290+0000","last_fullsized":"2024-09-21T06:11:16.070290+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:34.274129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.069952+0000","last_change":"2024-09-21T06:11:14.094742+0000","last_active":"2024-09-21T06:11:16.069952+0000","last_peered":"2024-09-21T06:11:16.069952+0000","last_clean":"2024-09-21T06:11:16.069952+0000","last_became_active":"2024-09-21T06:11:14.094296+0000","last_became_peered":"2024-09-21T06:11:14.094296+0000","last_unstale":"2024-09-21T06:11:16.069952+0000","last_undegraded":"2024-09-21T06:11:16.069952+0000","last_fullsized":"2024-09-21T06:11:16.069952+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:38.795110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733284+0000","last_change":"2024-09-21T06:11:15.091269+0000","last_active":"2024-09-21T06:11:16.733284+0000","last_peered":"2024-09-21T06:11:16.733284+0000","last_clean":"2024-09-21T06:11:16.733284+0000","last_became_active":"2024-09-21T06:11:15.090867+0000","last_became_peered":"2024-09-21T06:11:15.090867+0000","last_unstale":"2024-09-21T06:11:16.733284+0000","last_undegraded":"2024-09-21T06:11:16.733284+0000","last_fullsized":"2024-09-21T06:11:16.733284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:55.196646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733380+0000","last_change":"2024-09-21T06:11:14.088722+0000","last_active":"2024-09-21T06:11:16.733380+0000","last_peered":"2024-09-21T06:11:16.733380+0000","last_clean":"2024-09-21T06:11:16.733380+0000","last_became_active":"2024-09-21T06:11:14.087894+0000","last_became_peered":"2024-09-21T06:11:14.087894+0000","last_unstale":"2024-09-21T06:11:16.733380+0000","last_undegraded":"2024-09-21T06:11:16.733380+0000","last_fullsized":"2024-09-21T06:11:16.733380+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:07.215003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733175+0000","last_change":"2024-09-21T06:11:15.079806+0000","last_active":"2024-09-21T06:11:16.733175+0000","last_peered":"2024-09-21T06:11:16.733175+0000","last_clean":"2024-09-21T06:11:16.733175+0000","last_became_active":"2024-09-21T06:11:15.079416+0000","last_became_peered":"2024-09-21T06:11:15.079416+0000","last_unstale":"2024-09-21T06:11:16.733175+0000","last_undegraded":"2024-09-21T06:11:16.733175+0000","last_fullsized":"2024-09-21T06:11:16.733175+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:33.035953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"126'8","reported_seq":23,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733518+0000","last_change":"2024-09-21T06:11:14.088130+0000","last_active":"2024-09-21T06:11:16.733518+0000","last_peered":"2024-09-21T06:11:16.733518+0000","last_clean":"2024-09-21T06:11:16.733518+0000","last_became_active":"2024-09-21T06:11:14.087804+0000","last_became_peered":"2024-09-21T06:11:14.087804+0000","last_unstale":"2024-09-21T06:11:16.733518+0000","last_undegraded":"2024-09-21T06:11:16.733518+0000","last_fullsized":"2024-09-21T06:11:16.733518+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:38.035381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734386+0000","last_change":"2024-09-21T06:11:15.110004+0000","last_active":"2024-09-21T06:11:16.734386+0000","last_peered":"2024-09-21T06:11:16.734386+0000","last_clean":"2024-09-21T06:11:16.734386+0000","last_became_active":"2024-09-21T06:11:15.109328+0000","last_became_peered":"2024-09-21T06:11:15.109328+0000","last_unstale":"2024-09-21T06:11:16.734386+0000","last_undegraded":"2024-09-21T06:11:16.734386+0000","last_fullsized":"2024-09-21T06:11:16.734386+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:01.693132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.074310+0000","last_change":"2024-09-21T06:11:14.086968+0000","last_active":"2024-09-21T06:11:16.074310+0000","last_peered":"2024-09-21T06:11:16.074310+0000","last_clean":"2024-09-21T06:11:16.074310+0000","last_became_active":"2024-09-21T06:11:14.086556+0000","last_became_peered":"2024-09-21T06:11:14.086556+0000","last_unstale":"2024-09-21T06:11:16.074310+0000","last_undegraded":"2024-09-21T06:11:16.074310+0000","last_fullsized":"2024-09-21T06:11:16.074310+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:15.804930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.074253+0000","last_change":"2024-09-21T06:11:15.076272+0000","last_active":"2024-09-21T06:11:16.074253+0000","last_peered":"2024-09-21T06:11:16.074253+0000","last_clean":"2024-09-21T06:11:16.074253+0000","last_became_active":"2024-09-21T06:11:15.075852+0000","last_became_peered":"2024-09-21T06:11:15.075852+0000","last_unstale":"2024-09-21T06:11:16.074253+0000","last_undegraded":"2024-09-21T06:11:16.074253+0000","last_fullsized":"2024-09-21T06:11:16.074253+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:57.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734425+0000","last_change":"2024-09-21T06:11:14.084620+0000","last_active":"2024-09-21T06:11:16.734425+0000","last_peered":"2024-09-21T06:11:16.734425+0000","last_clean":"2024-09-21T06:11:16.734425+0000","last_became_active":"2024-09-21T06:11:14.084113+0000","last_became_peered":"2024-09-21T06:11:14.084113+0000","last_unstale":"2024-09-21T06:11:16.734425+0000","last_undegraded":"2024-09-21T06:11:16.734425+0000","last_fullsized":"2024-09-21T06:11:16.734425+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:00.095854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092632+0000","last_change":"2024-09-21T06:11:15.098426+0000","last_active":"2024-09-21T06:11:16.092632+0000","last_peered":"2024-09-21T06:11:16.092632+0000","last_clean":"2024-09-21T06:11:16.092632+0000","last_became_active":"2024-09-21T06:11:15.098093+0000","last_became_peered":"2024-09-21T06:11:15.098093+0000","last_unstale":"2024-09-21T06:11:16.092632+0000","last_undegraded":"2024-09-21T06:11:16.092632+0000","last_fullsized":"2024-09-21T06:11:16.092632+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:59.008882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075864+0000","last_change":"2024-09-21T06:11:14.079722+0000","last_active":"2024-09-21T06:11:16.075864+0000","last_peered":"2024-09-21T06:11:16.075864+0000","last_clean":"2024-09-21T06:11:16.075864+0000","last_became_active":"2024-09-21T06:11:14.079366+0000","last_became_peered":"2024-09-21T06:11:14.079366+0000","last_unstale":"2024-09-21T06:11:16.075864+0000","last_undegraded":"2024-09-21T06:11:16.075864+0000","last_fullsized":"2024-09-21T06:11:16.075864+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:53.643799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734318+0000","last_change":"2024-09-21T06:11:15.098241+0000","last_active":"2024-09-21T06:11:16.734318+0000","last_peered":"2024-09-21T06:11:16.734318+0000","last_clean":"2024-09-21T06:11:16.734318+0000","last_became_active":"2024-09-21T06:11:15.097725+0000","last_became_peered":"2024-09-21T06:11:15.097725+0000","last_unstale":"2024-09-21T06:11:16.734318+0000","last_undegraded":"2024-09-21T06:11:16.734318+0000","last_fullsized":"2024-09-21T06:11:16.734318+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:55.906006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.073770+0000","last_change":"2024-09-21T06:11:14.082559+0000","last_active":"2024-09-21T06:11:16.073770+0000","last_peered":"2024-09-21T06:11:16.073770+0000","last_clean":"2024-09-21T06:11:16.073770+0000","last_became_active":"2024-09-21T06:11:14.082206+0000","last_became_peered":"2024-09-21T06:11:14.082206+0000","last_unstale":"2024-09-21T06:11:16.073770+0000","last_undegraded":"2024-09-21T06:11:16.073770+0000","last_fullsized":"2024-09-21T06:11:16.073770+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:29.893033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.069919+0000","last_change":"2024-09-21T06:11:15.105157+0000","last_active":"2024-09-21T06:11:16.069919+0000","last_peered":"2024-09-21T06:11:16.069919+0000","last_clean":"2024-09-21T06:11:16.069919+0000","last_became_active":"2024-09-21T06:11:15.104741+0000","last_became_peered":"2024-09-21T06:11:15.104741+0000","last_unstale":"2024-09-21T06:11:16.069919+0000","last_undegraded":"2024-09-21T06:11:16.069919+0000","last_fullsized":"2024-09-21T06:11:16.069919+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:25.940376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.073601+0000","last_change":"2024-09-21T06:11:14.093595+0000","last_active":"2024-09-21T06:11:16.073601+0000","last_peered":"2024-09-21T06:11:16.073601+0000","last_clean":"2024-09-21T06:11:16.073601+0000","last_became_active":"2024-09-21T06:11:14.092250+0000","last_became_peered":"2024-09-21T06:11:14.092250+0000","last_unstale":"2024-09-21T06:11:16.073601+0000","last_undegraded":"2024-09-21T06:11:16.073601+0000","last_fullsized":"2024-09-21T06:11:16.073601+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:53.764795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075686+0000","last_change":"2024-09-21T06:11:15.082685+0000","last_active":"2024-09-21T06:11:16.075686+0000","last_peered":"2024-09-21T06:11:16.075686+0000","last_clean":"2024-09-21T06:11:16.075686+0000","last_became_active":"2024-09-21T06:11:15.082393+0000","last_became_peered":"2024-09-21T06:11:15.082393+0000","last_unstale":"2024-09-21T06:11:16.075686+0000","last_undegraded":"2024-09-21T06:11:16.075686+0000","last_fullsized":"2024-09-21T06:11:16.075686+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:05.123399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092124+0000","last_change":"2024-09-21T06:11:14.086360+0000","last_active":"2024-09-21T06:11:16.092124+0000","last_peered":"2024-09-21T06:11:16.092124+0000","last_clean":"2024-09-21T06:11:16.092124+0000","last_became_active":"2024-09-21T06:11:14.086037+0000","last_became_peered":"2024-09-21T06:11:14.086037+0000","last_unstale":"2024-09-21T06:11:16.092124+0000","last_undegraded":"2024-09-21T06:11:16.092124+0000","last_fullsized":"2024-09-21T06:11:16.092124+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:26.230264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733233+0000","last_change":"2024-09-21T06:11:15.103917+0000","last_active":"2024-09-21T06:11:16.733233+0000","last_peered":"2024-09-21T06:11:16.733233+0000","last_clean":"2024-09-21T06:11:16.733233+0000","last_became_active":"2024-09-21T06:11:15.097800+0000","last_became_peered":"2024-09-21T06:11:15.097800+0000","last_unstale":"2024-09-21T06:11:16.733233+0000","last_undegraded":"2024-09-21T06:11:16.733233+0000","last_fullsized":"2024-09-21T06:11:16.733233+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:07.217818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.091808+0000","last_change":"2024-09-21T06:11:14.076227+0000","last_active":"2024-09-21T06:11:16.091808+0000","last_peered":"2024-09-21T06:11:16.091808+0000","last_clean":"2024-09-21T06:11:16.091808+0000","last_became_active":"2024-09-21T06:11:14.075950+0000","last_became_peered":"2024-09-21T06:11:14.075950+0000","last_unstale":"2024-09-21T06:11:16.091808+0000","last_undegraded":"2024-09-21T06:11:16.091808+0000","last_fullsized":"2024-09-21T06:11:16.091808+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:15.761300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733662+0000","last_change":"2024-09-21T06:11:15.111827+0000","last_active":"2024-09-21T06:11:16.733662+0000","last_peered":"2024-09-21T06:11:16.733662+0000","last_clean":"2024-09-21T06:11:16.733662+0000","last_became_active":"2024-09-21T06:11:15.111460+0000","last_became_peered":"2024-09-21T06:11:15.111460+0000","last_unstale":"2024-09-21T06:11:16.733662+0000","last_undegraded":"2024-09-21T06:11:16.733662+0000","last_fullsized":"2024-09-21T06:11:16.733662+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:14.095774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733770+0000","last_change":"2024-09-21T06:11:14.083452+0000","last_active":"2024-09-21T06:11:16.733770+0000","last_peered":"2024-09-21T06:11:16.733770+0000","last_clean":"2024-09-21T06:11:16.733770+0000","last_became_active":"2024-09-21T06:11:14.083117+0000","last_became_peered":"2024-09-21T06:11:14.083117+0000","last_unstale":"2024-09-21T06:11:16.733770+0000","last_undegraded":"2024-09-21T06:11:16.733770+0000","last_fullsized":"2024-09-21T06:11:16.733770+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:25.253235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.076077+0000","last_change":"2024-09-21T06:11:15.082815+0000","last_active":"2024-09-21T06:11:16.076077+0000","last_peered":"2024-09-21T06:11:16.076077+0000","last_clean":"2024-09-21T06:11:16.076077+0000","last_became_active":"2024-09-21T06:11:15.082445+0000","last_became_peered":"2024-09-21T06:11:15.082445+0000","last_unstale":"2024-09-21T06:11:16.076077+0000","last_undegraded":"2024-09-21T06:11:16.076077+0000","last_fullsized":"2024-09-21T06:11:16.076077+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:00.080102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070733+0000","last_change":"2024-09-21T06:11:14.096024+0000","last_active":"2024-09-21T06:11:16.070733+0000","last_peered":"2024-09-21T06:11:16.070733+0000","last_clean":"2024-09-21T06:11:16.070733+0000","last_became_active":"2024-09-21T06:11:14.095727+0000","last_became_peered":"2024-09-21T06:11:14.095727+0000","last_unstale":"2024-09-21T06:11:16.070733+0000","last_undegraded":"2024-09-21T06:11:16.070733+0000","last_fullsized":"2024-09-21T06:11:16.070733+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:25.153625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.070622+0000","last_change":"2024-09-21T06:11:15.109848+0000","last_active":"2024-09-21T06:11:16.070622+0000","last_peered":"2024-09-21T06:11:16.070622+0000","last_clean":"2024-09-21T06:11:16.070622+0000","last_became_active":"2024-09-21T06:11:15.109489+0000","last_became_peered":"2024-09-21T06:11:15.109489+0000","last_unstale":"2024-09-21T06:11:16.070622+0000","last_undegraded":"2024-09-21T06:11:16.070622+0000","last_fullsized":"2024-09-21T06:11:16.070622+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:30.895573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"127'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734242+0000","last_change":"2024-09-21T06:11:14.092255+0000","last_active":"2024-09-21T06:11:16.734242+0000","last_peered":"2024-09-21T06:11:16.734242+0000","last_clean":"2024-09-21T06:11:16.734242+0000","last_became_active":"2024-09-21T06:11:14.091547+0000","last_became_peered":"2024-09-21T06:11:14.091547+0000","last_unstale":"2024-09-21T06:11:16.734242+0000","last_undegraded":"2024-09-21T06:11:16.734242+0000","last_fullsized":"2024-09-21T06:11:16.734242+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:47.693414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734620+0000","last_change":"2024-09-21T06:11:15.089194+0000","last_active":"2024-09-21T06:11:16.734620+0000","last_peered":"2024-09-21T06:11:16.734620+0000","last_clean":"2024-09-21T06:11:16.734620+0000","last_became_active":"2024-09-21T06:11:15.088212+0000","last_became_peered":"2024-09-21T06:11:15.088212+0000","last_unstale":"2024-09-21T06:11:16.734620+0000","last_undegraded":"2024-09-21T06:11:16.734620+0000","last_fullsized":"2024-09-21T06:11:16.734620+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:15.209294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734425+0000","last_change":"2024-09-21T06:11:14.084372+0000","last_active":"2024-09-21T06:11:16.734425+0000","last_peered":"2024-09-21T06:11:16.734425+0000","last_clean":"2024-09-21T06:11:16.734425+0000","last_became_active":"2024-09-21T06:11:14.084139+0000","last_became_peered":"2024-09-21T06:11:14.084139+0000","last_unstale":"2024-09-21T06:11:16.734425+0000","last_undegraded":"2024-09-21T06:11:16.734425+0000","last_fullsized":"2024-09-21T06:11:16.734425+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:07.303289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.075883+0000","last_change":"2024-09-21T06:11:15.103627+0000","last_active":"2024-09-21T06:11:16.075883+0000","last_peered":"2024-09-21T06:11:16.075883+0000","last_clean":"2024-09-21T06:11:16.075883+0000","last_became_active":"2024-09-21T06:11:15.103283+0000","last_became_peered":"2024-09-21T06:11:15.103283+0000","last_unstale":"2024-09-21T06:11:16.075883+0000","last_undegraded":"2024-09-21T06:11:16.075883+0000","last_fullsized":"2024-09-21T06:11:16.075883+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:12.379746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733646+0000","last_change":"2024-09-21T06:11:14.084166+0000","last_active":"2024-09-21T06:11:16.733646+0000","last_peered":"2024-09-21T06:11:16.733646+0000","last_clean":"2024-09-21T06:11:16.733646+0000","last_became_active":"2024-09-21T06:11:14.083578+0000","last_became_peered":"2024-09-21T06:11:14.083578+0000","last_unstale":"2024-09-21T06:11:16.733646+0000","last_undegraded":"2024-09-21T06:11:16.733646+0000","last_fullsized":"2024-09-21T06:11:16.733646+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:59.922768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734040+0000","last_change":"2024-09-21T06:11:15.100569+0000","last_active":"2024-09-21T06:11:16.734040+0000","last_peered":"2024-09-21T06:11:16.734040+0000","last_clean":"2024-09-21T06:11:16.734040+0000","last_became_active":"2024-09-21T06:11:15.100250+0000","last_became_peered":"2024-09-21T06:11:15.100250+0000","last_unstale":"2024-09-21T06:11:16.734040+0000","last_undegraded":"2024-09-21T06:11:16.734040+0000","last_fullsized":"2024-09-21T06:11:16.734040+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:21.873054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733774+0000","last_change":"2024-09-21T06:11:14.087191+0000","last_active":"2024-09-21T06:11:16.733774+0000","last_peered":"2024-09-21T06:11:16.733774+0000","last_clean":"2024-09-21T06:11:16.733774+0000","last_became_active":"2024-09-21T06:11:14.086955+0000","last_became_peered":"2024-09-21T06:11:14.086955+0000","last_unstale":"2024-09-21T06:11:16.733774+0000","last_undegraded":"2024-09-21T06:11:16.733774+0000","last_fullsized":"2024-09-21T06:11:16.733774+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:20.335280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.073827+0000","last_change":"2024-09-21T06:11:15.114119+0000","last_active":"2024-09-21T06:11:16.073827+0000","last_peered":"2024-09-21T06:11:16.073827+0000","last_clean":"2024-09-21T06:11:16.073827+0000","last_became_active":"2024-09-21T06:11:15.113749+0000","last_became_peered":"2024-09-21T06:11:15.113749+0000","last_unstale":"2024-09-21T06:11:16.073827+0000","last_undegraded":"2024-09-21T06:11:16.073827+0000","last_fullsized":"2024-09-21T06:11:16.073827+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:02.874643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733778+0000","last_change":"2024-09-21T06:11:14.091971+0000","last_active":"2024-09-21T06:11:16.733778+0000","last_peered":"2024-09-21T06:11:16.733778+0000","last_clean":"2024-09-21T06:11:16.733778+0000","last_became_active":"2024-09-21T06:11:14.091587+0000","last_became_peered":"2024-09-21T06:11:14.091587+0000","last_unstale":"2024-09-21T06:11:16.733778+0000","last_undegraded":"2024-09-21T06:11:16.733778+0000","last_fullsized":"2024-09-21T06:11:16.733778+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:40.752652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.092195+0000","last_change":"2024-09-21T06:11:15.094559+0000","last_active":"2024-09-21T06:11:16.092195+0000","last_peered":"2024-09-21T06:11:16.092195+0000","last_clean":"2024-09-21T06:11:16.092195+0000","last_became_active":"2024-09-21T06:11:15.092399+0000","last_became_peered":"2024-09-21T06:11:15.092399+0000","last_unstale":"2024-09-21T06:11:16.092195+0000","last_undegraded":"2024-09-21T06:11:16.092195+0000","last_fullsized":"2024-09-21T06:11:16.092195+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:01:02.518244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.091827+0000","last_change":"2024-09-21T06:11:14.086180+0000","last_active":"2024-09-21T06:11:16.091827+0000","last_peered":"2024-09-21T06:11:16.091827+0000","last_clean":"2024-09-21T06:11:16.091827+0000","last_became_active":"2024-09-21T06:11:14.085840+0000","last_became_peered":"2024-09-21T06:11:14.085840+0000","last_unstale":"2024-09-21T06:11:16.091827+0000","last_undegraded":"2024-09-21T06:11:16.091827+0000","last_fullsized":"2024-09-21T06:11:16.091827+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:45.658256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733405+0000","last_change":"2024-09-21T06:11:15.088407+0000","last_active":"2024-09-21T06:11:16.733405+0000","last_peered":"2024-09-21T06:11:16.733405+0000","last_clean":"2024-09-21T06:11:16.733405+0000","last_became_active":"2024-09-21T06:11:15.088041+0000","last_became_peered":"2024-09-21T06:11:15.088041+0000","last_unstale":"2024-09-21T06:11:16.733405+0000","last_undegraded":"2024-09-21T06:11:16.733405+0000","last_fullsized":"2024-09-21T06:11:16.733405+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:30.659562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734072+0000","last_change":"2024-09-21T06:11:14.082021+0000","last_active":"2024-09-21T06:11:16.734072+0000","last_peered":"2024-09-21T06:11:16.734072+0000","last_clean":"2024-09-21T06:11:16.734072+0000","last_became_active":"2024-09-21T06:11:14.081692+0000","last_became_peered":"2024-09-21T06:11:14.081692+0000","last_unstale":"2024-09-21T06:11:16.734072+0000","last_undegraded":"2024-09-21T06:11:16.734072+0000","last_fullsized":"2024-09-21T06:11:16.734072+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:41.871307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733335+0000","last_change":"2024-09-21T06:11:15.112024+0000","last_active":"2024-09-21T06:11:16.733335+0000","last_peered":"2024-09-21T06:11:16.733335+0000","last_clean":"2024-09-21T06:11:16.733335+0000","last_became_active":"2024-09-21T06:11:15.111665+0000","last_became_peered":"2024-09-21T06:11:15.111665+0000","last_unstale":"2024-09-21T06:11:16.733335+0000","last_undegraded":"2024-09-21T06:11:16.733335+0000","last_fullsized":"2024-09-21T06:11:16.733335+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:14.055724+0000","last_clean_scrub_stamp":"2024-09-21T06:11:14.055724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:26.818828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.734335+0000","last_change":"2024-09-21T06:11:14.093273+0000","last_active":"2024-09-21T06:11:16.734335+0000","last_peered":"2024-09-21T06:11:16.734335+0000","last_clean":"2024-09-21T06:11:16.734335+0000","last_became_active":"2024-09-21T06:11:14.092704+0000","last_became_peered":"2024-09-21T06:11:14.092704+0000","last_unstale":"2024-09-21T06:11:16.734335+0000","last_undegraded":"2024-09-21T06:11:16.734335+0000","last_fullsized":"2024-09-21T06:11:16.734335+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:11:13.053183+0000","last_clean_scrub_stamp":"2024-09-21T06:11:13.053183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:04.842818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":306,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-21T06:11:16.733401+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:11:16.733401+0000","last_peered":"2024-09-21T06:11:16.733401+0000","last_clean":"2024-09-21T06:11:16.733401+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:11:16.733401+0000","last_undegraded":"2024-09-21T06:11:16.733401+0000","last_fullsized":"2024-09-21T06:11:16.733401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509623,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131836,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240004,"statfs":{"total":96636764160,"available":96501764096,"internally_reserved":0,"allocated":2195456,"data_stored":1283457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509623,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140964,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230876,"statfs":{"total":96636764160,"available":96492417024,"internally_reserved":0,"allocated":1589248,"data_stored":692451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509622,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86824,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285016,"statfs":{"total":96636764160,"available":96547856384,"internally_reserved":0,"allocated":1585152,"data_stored":690951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509623,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153436,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218404,"statfs":{"total":96636764160,"available":96479645696,"internally_reserved":0,"allocated":1581056,"data_stored":690909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509622,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122920,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248920,"statfs":{"total":96636764160,"available":96510894080,"internally_reserved":0,"allocated":1585152,"data_stored":690999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509624,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140960,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230880,"statfs":{"total":96636764160,"available":96492421120,"internally_reserved":0,"allocated":1585152,"data_stored":691455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509622,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123484,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248356,"statfs":{"total":96636764160,"available":96510316544,"internally_reserved":0,"allocated":2170880,"data_stored":1281207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542327,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167308,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204532,"statfs":{"total":96636764160,"available":96465440768,"internally_reserved":0,"allocated":1564672,"data_stored":684455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:11:19.709 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:11:19.709 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:11:19.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.915+0000 7f7317a00640 1 -- 172.21.15.50:0/3537248683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73100749a0 msgr2=0x7f7310074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.915+0000 7f7317a00640 1 --2- 172.21.15.50:0/3537248683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73100749a0 0x7f7310074db0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f7300009840 tx=0x7f730001c820 comp rx=0 tx=0).stop 2024-09-21T06:11:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.915+0000 7f7317a00640 1 -- 172.21.15.50:0/3537248683 shutdown_connections 2024-09-21T06:11:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.915+0000 7f7317a00640 1 --2- 172.21.15.50:0/3537248683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73100749a0 0x7f7310074db0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.915+0000 7f7317a00640 1 -- 172.21.15.50:0/3537248683 >> 172.21.15.50:0/3537248683 conn(0x7f73100b2040 msgr2=0x7f73100b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.915+0000 7f7317a00640 1 -- 172.21.15.50:0/3537248683 shutdown_connections 2024-09-21T06:11:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.915+0000 7f7317a00640 1 -- 172.21.15.50:0/3537248683 wait complete. 2024-09-21T06:11:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.915+0000 7f7317a00640 1 Processor -- start 2024-09-21T06:11:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.915+0000 7f7317a00640 1 -- start start 2024-09-21T06:11:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7317a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73100749a0 0x7f73101f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7317a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73100b3340 0x7f73101fa280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7317a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f73100cefa0 0x7f73101f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7317a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f731016b350 con 0x7f73100749a0 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7317a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f731016b1d0 con 0x7f73100b3340 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7317a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f731016b4d0 con 0x7f73100cefa0 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7315000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73100749a0 0x7f73101f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7315000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73100749a0 0x7f73101f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32848/0 (socket says 172.21.15.50:32848) 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7315000640 1 -- 172.21.15.50:0/3152703830 learned_addr learned my addr 172.21.15.50:0/3152703830 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7315000640 1 -- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f73100cefa0 msgr2=0x7f73101f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7315000640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f73100cefa0 0x7f73101f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7315000640 1 -- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73100b3340 msgr2=0x7f73101fa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7307e00640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73100b3340 0x7f73101fa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7315000640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73100b3340 0x7f73101fa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7315000640 1 -- 172.21.15.50:0/3152703830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73101faab0 con 0x7f73100749a0 2024-09-21T06:11:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7307e00640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73100b3340 0x7f73101fa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7315000640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73100749a0 0x7f73101f6c70 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f7300009790 tx=0x7f7300004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f730d600640 1 -- 172.21.15.50:0/3152703830 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73000bb070 con 0x7f73100749a0 2024-09-21T06:11:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f730d600640 1 -- 172.21.15.50:0/3152703830 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7300005030 con 0x7f73100749a0 2024-09-21T06:11:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f730d600640 1 -- 172.21.15.50:0/3152703830 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73000b2040 con 0x7f73100749a0 2024-09-21T06:11:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7317a00640 1 -- 172.21.15.50:0/3152703830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73101fad90 con 0x7f73100749a0 2024-09-21T06:11:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7317a00640 1 -- 172.21.15.50:0/3152703830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7310140f40 con 0x7f73100749a0 2024-09-21T06:11:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.919+0000 7f7317a00640 1 -- 172.21.15.50:0/3152703830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7310074db0 con 0x7f73100749a0 2024-09-21T06:11:19.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.923+0000 7f730d600640 1 -- 172.21.15.50:0/3152703830 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f7300005440 con 0x7f73100749a0 2024-09-21T06:11:19.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.923+0000 7f730d600640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f72ec075180 0x7f72ec077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:19.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.923+0000 7f730d600640 1 -- 172.21.15.50:0/3152703830 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f730012cc70 con 0x7f73100749a0 2024-09-21T06:11:19.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.923+0000 7f7307e00640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f72ec075180 0x7f72ec077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:19.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.923+0000 7f7307e00640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f72ec075180 0x7f72ec077690 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7310157e60 tx=0x7f72f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:19.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:19.927+0000 7f730d600640 1 -- 172.21.15.50:0/3152703830 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f73000f2290 con 0x7f73100749a0 2024-09-21T06:11:20.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.171+0000 7f7317a00640 1 -- 172.21.15.50:0/3152703830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f73100b27d0 con 0x7f73100749a0 2024-09-21T06:11:20.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.171+0000 7f730d600640 1 -- 172.21.15.50:0/3152703830 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v38) ==== 190+0+147 (secure 0 0 0) 0x7f7300016410 con 0x7f73100749a0 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw== 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi050.stdout: caps mds = "allow" 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi050.stdout: caps mon = "allow r" 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi050.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi050.stderr:updated caps for client.0 2024-09-21T06:11:20.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.175+0000 7f7317a00640 1 -- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f72ec075180 msgr2=0x7f72ec077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:20.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.175+0000 7f7317a00640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f72ec075180 0x7f72ec077690 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7310157e60 tx=0x7f72f8008040 comp rx=0 tx=0).stop 2024-09-21T06:11:20.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.175+0000 7f7317a00640 1 -- 172.21.15.50:0/3152703830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73100749a0 msgr2=0x7f73101f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:20.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.175+0000 7f7317a00640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73100749a0 0x7f73101f6c70 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f7300009790 tx=0x7f7300004040 comp rx=0 tx=0).stop 2024-09-21T06:11:20.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.175+0000 7f7317a00640 1 -- 172.21.15.50:0/3152703830 shutdown_connections 2024-09-21T06:11:20.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.175+0000 7f7317a00640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f73100cefa0 0x7f73101f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.175+0000 7f7317a00640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73100b3340 0x7f73101fa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.175+0000 7f7317a00640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f72ec075180 0x7f72ec077690 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.175+0000 7f7317a00640 1 --2- 172.21.15.50:0/3152703830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73100749a0 0x7f73101f6c70 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.175+0000 7f7317a00640 1 -- 172.21.15.50:0/3152703830 >> 172.21.15.50:0/3152703830 conn(0x7f73100b2040 msgr2=0x7f731015bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.175+0000 7f7317a00640 1 -- 172.21.15.50:0/3152703830 shutdown_connections 2024-09-21T06:11:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.175+0000 7f7317a00640 1 -- 172.21.15.50:0/3152703830 wait complete. 2024-09-21T06:11:20.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:11:20.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.355+0000 7f3482000640 1 -- 172.21.15.50:0/589580050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347c160060 msgr2=0x7f347c074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:20.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.355+0000 7f3482000640 1 --2- 172.21.15.50:0/589580050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347c160060 0x7f347c074240 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f3464009900 tx=0x7f346401c850 comp rx=0 tx=0).stop 2024-09-21T06:11:20.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.355+0000 7f3482000640 1 -- 172.21.15.50:0/589580050 shutdown_connections 2024-09-21T06:11:20.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.355+0000 7f3482000640 1 --2- 172.21.15.50:0/589580050 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f347c160470 0x7f347c074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.355+0000 7f3482000640 1 --2- 172.21.15.50:0/589580050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347c160060 0x7f347c074240 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.355+0000 7f3482000640 1 -- 172.21.15.50:0/589580050 >> 172.21.15.50:0/589580050 conn(0x7f347c0b4560 msgr2=0x7f347c0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:20.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.355+0000 7f3482000640 1 -- 172.21.15.50:0/589580050 shutdown_connections 2024-09-21T06:11:20.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.355+0000 7f3482000640 1 -- 172.21.15.50:0/589580050 wait complete. 2024-09-21T06:11:20.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3482000640 1 Processor -- start 2024-09-21T06:11:20.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3482000640 1 -- start start 2024-09-21T06:11:20.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3482000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f347c1536f0 0x7f347c162140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:20.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3482000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f347c160060 0x7f347c141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3482000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347c160470 0x7f347c0715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3482000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f347c1696e0 con 0x7f347c160470 2024-09-21T06:11:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3482000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f347c169560 con 0x7f347c1536f0 2024-09-21T06:11:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3482000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f347c169860 con 0x7f347c160060 2024-09-21T06:11:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3481600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347c160470 0x7f347c0715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3481600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347c160470 0x7f347c0715f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32870/0 (socket says 172.21.15.50:32870) 2024-09-21T06:11:20.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3481600640 1 -- 172.21.15.50:0/3902288347 learned_addr learned my addr 172.21.15.50:0/3902288347 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3473e00640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f347c160060 0x7f347c141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3480c00640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f347c1536f0 0x7f347c162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3481600640 1 -- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f347c160060 msgr2=0x7f347c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3481600640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f347c160060 0x7f347c141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3481600640 1 -- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f347c1536f0 msgr2=0x7f347c162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3481600640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f347c1536f0 0x7f347c162140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3481600640 1 -- 172.21.15.50:0/3902288347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c154200 con 0x7f347c160470 2024-09-21T06:11:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3481600640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347c160470 0x7f347c0715f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f347400e940 tx=0x7f347400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:20.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3479600640 1 -- 172.21.15.50:0/3902288347 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474004d30 con 0x7f347c160470 2024-09-21T06:11:20.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3479600640 1 -- 172.21.15.50:0/3902288347 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f347400cb20 con 0x7f347c160470 2024-09-21T06:11:20.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3479600640 1 -- 172.21.15.50:0/3902288347 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474005020 con 0x7f347c160470 2024-09-21T06:11:20.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3482000640 1 -- 172.21.15.50:0/3902288347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347c1544e0 con 0x7f347c160470 2024-09-21T06:11:20.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.359+0000 7f3482000640 1 -- 172.21.15.50:0/3902288347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347c154aa0 con 0x7f347c160470 2024-09-21T06:11:20.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.363+0000 7f3482000640 1 -- 172.21.15.50:0/3902288347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347c15d480 con 0x7f347c160470 2024-09-21T06:11:20.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.363+0000 7f3479600640 1 -- 172.21.15.50:0/3902288347 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f34740051c0 con 0x7f347c160470 2024-09-21T06:11:20.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.363+0000 7f3479600640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3458075180 0x7f3458077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:20.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.363+0000 7f3479600640 1 -- 172.21.15.50:0/3902288347 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f3474093af0 con 0x7f347c160470 2024-09-21T06:11:20.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.363+0000 7f3480c00640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3458075180 0x7f3458077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:20.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.367+0000 7f3480c00640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3458075180 0x7f3458077690 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f346c007fd0 tx=0x7f346c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:20.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.367+0000 7f3479600640 1 -- 172.21.15.50:0/3902288347 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f347409e020 con 0x7f347c160470 2024-09-21T06:11:20.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 -- 172.21.15.50:0/3902288347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f347c06de60 con 0x7f347c160470 2024-09-21T06:11:20.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3479600640 1 -- 172.21.15.50:0/3902288347 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5219 (secure 0 0 0) 0x7f3474016030 con 0x7f347c160470 2024-09-21T06:11:20.580 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:20.580 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":84,"btime":"2024-09-21T06:11:18:101077+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2643543096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2643543096},{"type":"v1","addr":"172.21.15.88:6836","nonce":2643543096}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3618269169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3618269169},{"type":"v1","addr":"172.21.15.88:6837","nonce":3618269169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/66879433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":66879433},{"type":"v1","addr":"172.21.15.50:6835","nonce":66879433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:11:14.496744+0000","modified":"2024-09-21T06:11:17.524950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8832},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8832":{"gid":8832,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6833/1864490990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1864490990},{"type":"v1","addr":"172.21.15.50:6833","nonce":1864490990}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8832,"qdb_cluster":[8832]},"id":9}]} 2024-09-21T06:11:20.580 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 84 2024-09-21T06:11:20.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 -- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3458075180 msgr2=0x7f3458077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:20.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3458075180 0x7f3458077690 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f346c007fd0 tx=0x7f346c009040 comp rx=0 tx=0).stop 2024-09-21T06:11:20.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 -- 172.21.15.50:0/3902288347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347c160470 msgr2=0x7f347c0715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:20.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347c160470 0x7f347c0715f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f347400e940 tx=0x7f347400ee00 comp rx=0 tx=0).stop 2024-09-21T06:11:20.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 -- 172.21.15.50:0/3902288347 shutdown_connections 2024-09-21T06:11:20.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3458075180 0x7f3458077690 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f347c160470 0x7f347c0715f0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f347c160060 0x7f347c141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 --2- 172.21.15.50:0/3902288347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f347c1536f0 0x7f347c162140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 -- 172.21.15.50:0/3902288347 >> 172.21.15.50:0/3902288347 conn(0x7f347c0b4560 msgr2=0x7f347c152990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:20.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 -- 172.21.15.50:0/3902288347 shutdown_connections 2024-09-21T06:11:20.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:20.583+0000 7f3482000640 1 -- 172.21.15.50:0/3902288347 wait complete. 2024-09-21T06:11:21.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:11:14.496744+0000', 'modified': '2024-09-21T06:11:17.524950+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8832}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8832': {'gid': 8832, 'name': 'c', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6833/1864490990', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6832', 'nonce': 1864490990}, {'type': 'v1', 'addr': '172.21.15.50:6833', 'nonce': 1864490990}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8832, 'qdb_cluster': [8832]} 2024-09-21T06:11:21.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:11:21.006 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T06:11:21.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.175+0000 7f6fc6000640 1 -- 172.21.15.50:0/1565780406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fc015c5d0 msgr2=0x7f6fc0073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.175+0000 7f6fc6000640 1 --2- 172.21.15.50:0/1565780406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fc015c5d0 0x7f6fc0073dc0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009c80 tx=0x7f6fb001cd70 comp rx=0 tx=0).stop 2024-09-21T06:11:21.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.175+0000 7f6fc6000640 1 -- 172.21.15.50:0/1565780406 shutdown_connections 2024-09-21T06:11:21.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.175+0000 7f6fc6000640 1 --2- 172.21.15.50:0/1565780406 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6fc0074300 0x7f6fc015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.175+0000 7f6fc6000640 1 --2- 172.21.15.50:0/1565780406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fc015c5d0 0x7f6fc0073dc0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.175+0000 7f6fc6000640 1 -- 172.21.15.50:0/1565780406 >> 172.21.15.50:0/1565780406 conn(0x7f6fc00b2c40 msgr2=0x7f6fc00b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.175+0000 7f6fc6000640 1 -- 172.21.15.50:0/1565780406 shutdown_connections 2024-09-21T06:11:21.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.175+0000 7f6fc6000640 1 -- 172.21.15.50:0/1565780406 wait complete. 2024-09-21T06:11:21.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fc6000640 1 Processor -- start 2024-09-21T06:11:21.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fc6000640 1 -- start start 2024-09-21T06:11:21.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fc6000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc0074300 0x7f6fc01f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fc6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fc015c5d0 0x7f6fc0150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fc6000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6fc015f6b0 0x7f6fc0150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fc6000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc006f980 con 0x7f6fc015c5d0 2024-09-21T06:11:21.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fc6000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc00b3d50 con 0x7f6fc0074300 2024-09-21T06:11:21.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fc6000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc006fb00 con 0x7f6fc015f6b0 2024-09-21T06:11:21.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fb6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fc015c5d0 0x7f6fc0150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fb6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fc015c5d0 0x7f6fc0150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32878/0 (socket says 172.21.15.50:32878) 2024-09-21T06:11:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fb6a00640 1 -- 172.21.15.50:0/1363412727 learned_addr learned my addr 172.21.15.50:0/1363412727 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fbf400640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc0074300 0x7f6fc01f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fbfe00640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6fc015f6b0 0x7f6fc0150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fb6a00640 1 -- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6fc015f6b0 msgr2=0x7f6fc0150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fb6a00640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6fc015f6b0 0x7f6fc0150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fb6a00640 1 -- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc0074300 msgr2=0x7f6fc01f8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fb6a00640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc0074300 0x7f6fc01f8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fb6a00640 1 -- 172.21.15.50:0/1363412727 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc015ec50 con 0x7f6fc015c5d0 2024-09-21T06:11:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fb6a00640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fc015c5d0 0x7f6fc0150970 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800c970 tx=0x7f6fa800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fb7e00640 1 -- 172.21.15.50:0/1363412727 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8007bb0 con 0x7f6fc015c5d0 2024-09-21T06:11:21.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fb7e00640 1 -- 172.21.15.50:0/1363412727 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fa8007e40 con 0x7f6fc015c5d0 2024-09-21T06:11:21.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fb7e00640 1 -- 172.21.15.50:0/1363412727 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800f6b0 con 0x7f6fc015c5d0 2024-09-21T06:11:21.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fc6000640 1 -- 172.21.15.50:0/1363412727 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc0153ca0 con 0x7f6fc015c5d0 2024-09-21T06:11:21.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.179+0000 7f6fc6000640 1 -- 172.21.15.50:0/1363412727 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc0154260 con 0x7f6fc015c5d0 2024-09-21T06:11:21.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.183+0000 7f6fb7e00640 1 -- 172.21.15.50:0/1363412727 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f6fa800f850 con 0x7f6fc015c5d0 2024-09-21T06:11:21.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.183+0000 7f6fb7e00640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f9c075180 0x7f6f9c077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.183+0000 7f6fb7e00640 1 -- 172.21.15.50:0/1363412727 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f6fa8093990 con 0x7f6fc015c5d0 2024-09-21T06:11:21.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.183+0000 7f6fcd7f2000 1 Processor -- start 2024-09-21T06:11:21.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.183+0000 7f6fbf400640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f9c075180 0x7f6f9c077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.183+0000 7f6fcd7f2000 1 -- start start 2024-09-21T06:11:21.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fbf400640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f9c075180 0x7f6f9c077690 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00af000 tx=0x7f6fb001b5b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x5619025ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56190249e950 0x5619025efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 0x5619025f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5619024c4410 con 0x5619020ae460 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561902401b40 con 0x56190249e950 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x5619023f0f70 con 0x561902487ca0 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fbf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x5619025ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fbf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x5619025ef6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32890/0 (socket says 172.21.15.50:32890) 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fbf400640 1 -- 172.21.15.50:0/218432279 learned_addr learned my addr 172.21.15.50:0/218432279 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fbfe00640 1 --2- 172.21.15.50:0/218432279 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 0x5619025f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fb6a00640 1 --2- 172.21.15.50:0/218432279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56190249e950 0x5619025efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fbf400640 1 -- 172.21.15.50:0/218432279 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 msgr2=0x5619025f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fbf400640 1 --2- 172.21.15.50:0/218432279 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 0x5619025f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fbf400640 1 -- 172.21.15.50:0/218432279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56190249e950 msgr2=0x5619025efbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fbf400640 1 --2- 172.21.15.50:0/218432279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56190249e950 0x5619025efbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fbf400640 1 -- 172.21.15.50:0/218432279 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619023b85b0 con 0x5619020ae460 2024-09-21T06:11:21.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fb6a00640 1 --2- 172.21.15.50:0/218432279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56190249e950 0x5619025efbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:21.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fbf400640 1 --2- 172.21.15.50:0/218432279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x5619025ef6a0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00b0040 tx=0x7f6fb00b6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ded7c2a4d58dab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6faf400640 1 -- 172.21.15.50:0/218432279 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00b2040 con 0x5619020ae460 2024-09-21T06:11:21.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6faf400640 1 -- 172.21.15.50:0/218432279 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fb00c8070 con 0x5619020ae460 2024-09-21T06:11:21.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6faf400640 1 -- 172.21.15.50:0/218432279 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00042a0 con 0x5619020ae460 2024-09-21T06:11:21.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/218432279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 msgr2=0x5619025ef6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/218432279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x5619025ef6a0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00b0040 tx=0x7f6fb00b6000 comp rx=0 tx=0).stop 2024-09-21T06:11:21.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/218432279 shutdown_connections 2024-09-21T06:11:21.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/218432279 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 0x5619025f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/218432279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56190249e950 0x5619025efbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/218432279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x5619025ef6a0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/218432279 >> 172.21.15.50:0/218432279 conn(0x561902574d90 msgr2=0x5619025a4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/218432279 shutdown_connections 2024-09-21T06:11:21.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.187+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/218432279 wait complete. 2024-09-21T06:11:21.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 1 -- start start 2024-09-21T06:11:21.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 1 Processor -- start 2024-09-21T06:11:21.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x5619025f6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 0x5619025f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56190249e950 0x5619025f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56190238b5f0 con 0x5619020ae460 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5619023f4d80 con 0x56190249e950 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x561902397520 con 0x561902487ca0 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 10 client.0 _ll_register_callbacks cb 0x56190239b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fbf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x5619025f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fbf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x5619025f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32892/0 (socket says 172.21.15.50:32892) 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fbf400640 1 -- 172.21.15.50:0/1564916871 learned_addr learned my addr 172.21.15.50:0/1564916871 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fb6a00640 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 0x5619025f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fbfe00640 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56190249e950 0x5619025f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fbf400640 1 -- 172.21.15.50:0/1564916871 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 msgr2=0x5619025f6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fbf400640 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 0x5619025f6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fbf400640 1 -- 172.21.15.50:0/1564916871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56190249e950 msgr2=0x5619025f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fbf400640 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56190249e950 0x5619025f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fbf400640 1 -- 172.21.15.50:0/1564916871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190238f190 con 0x5619020ae460 2024-09-21T06:11:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fbf400640 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x5619025f6920 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6fb001d610 tx=0x7f6fb0004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7876f987b4194f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fa3e00640 1 -- 172.21.15.50:0/1564916871 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00b2040 con 0x5619020ae460 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fa3e00640 1 -- 172.21.15.50:0/1564916871 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f6fb0004780 con 0x5619020ae460 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/1564916871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561902389110 con 0x5619020ae460 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fa3e00640 1 -- 172.21.15.50:0/1564916871 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f6fb00dc050 con 0x5619020ae460 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 10 client.8961 fetch_fsmap learned FSMap version 84 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/1564916871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x561902475f10 con 0x5619020ae460 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fa3e00640 1 -- 172.21.15.50:0/1564916871 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x561902389110 con 0x5619020ae460 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fa3e00640 1 -- 172.21.15.50:0/1564916871 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7f6fb00df020 con 0x5619020ae460 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 10 client.8961 fetch_fsmap finished waiting for FSMap version 84 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 10 client.8961 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.191+0000 7f6fcd7f2000 20 client.8961 populate_metadata read hostname 'smithi050' 2024-09-21T06:11:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.50:6832/1864490990,v1:172.21.15.50:6833/1864490990] conn(0x5619020937d0 0x561902600d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 4 client.8961 mds_command: new command op to 8832 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:11:21.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/1564916871 --> [v2:172.21.15.50:6832/1864490990,v1:172.21.15.50:6833/1864490990] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561902403a90 con 0x5619020937d0 2024-09-21T06:11:21.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fb6a00640 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.50:6832/1864490990,v1:172.21.15.50:6833/1864490990] conn(0x5619020937d0 0x561902600d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fb6a00640 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.50:6832/1864490990,v1:172.21.15.50:6833/1864490990] conn(0x5619020937d0 0x561902600d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef893ee078dddce server_cookie=b16755a2a8b3aba2 in_seq=0 out_seq=0 2024-09-21T06:11:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fa3e00640 10 client.8961 ms_handle_connect on v2:172.21.15.50:6832/1864490990 2024-09-21T06:11:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fa3e00640 1 -- 172.21.15.50:0/1564916871 <== mds.0 v2:172.21.15.50:6832/1864490990 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561902403a90 con 0x5619020937d0 2024-09-21T06:11:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fa3e00640 10 client.8961 handle_command_reply: tid=0 2024-09-21T06:11:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 client.8961 shutdown 2024-09-21T06:11:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/1564916871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 msgr2=0x5619025f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x5619025f6920 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6fb001d610 tx=0x7f6fb0004590 comp rx=0 tx=0).stop 2024-09-21T06:11:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/1564916871 shutdown_connections 2024-09-21T06:11:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56190249e950 0x5619025f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 0x5619025f6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.50:6832/1864490990,v1:172.21.15.50:6833/1864490990] conn(0x5619020937d0 0x561902600d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/1564916871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x5619025f6920 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fa3e00640 0 client.8961 ms_handle_reset on v2:172.21.15.50:6832/1864490990 2024-09-21T06:11:21.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/1564916871 >> 172.21.15.50:0/1564916871 conn(0x561902574d90 msgr2=0x5619025ab440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/1564916871 shutdown_connections 2024-09-21T06:11:21.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/1564916871 wait complete. 2024-09-21T06:11:21.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.195+0000 7f6fcd7f2000 20 client.8961 trim_cache size 0 max 16384 2024-09-21T06:11:21.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fcd7f2000 1 Processor -- start 2024-09-21T06:11:21.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fcd7f2000 1 -- start start 2024-09-21T06:11:21.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fcd7f2000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619020ae460 0x561902661620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fcd7f2000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 0x561902661b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fcd7f2000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56190249e950 0x5619026620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fcd7f2000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc0205860 con 0x56190249e950 2024-09-21T06:11:21.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fcd7f2000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc0204a60 con 0x5619020ae460 2024-09-21T06:11:21.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fcd7f2000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc01fca40 con 0x561902487ca0 2024-09-21T06:11:21.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fbfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56190249e950 0x5619026620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fbfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56190249e950 0x5619026620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32896/0 (socket says 172.21.15.50:32896) 2024-09-21T06:11:21.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fbfe00640 1 -- 172.21.15.50:0/2643898113 learned_addr learned my addr 172.21.15.50:0/2643898113 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:21.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fbfe00640 1 -- 172.21.15.50:0/2643898113 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 msgr2=0x561902661b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fbfe00640 1 --2- 172.21.15.50:0/2643898113 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 0x561902661b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fbf400640 1 --2- 172.21.15.50:0/2643898113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619020ae460 0x561902661620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fbfe00640 1 -- 172.21.15.50:0/2643898113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619020ae460 msgr2=0x561902661620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fbfe00640 1 --2- 172.21.15.50:0/2643898113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619020ae460 0x561902661620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fbfe00640 1 -- 172.21.15.50:0/2643898113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190244cdb0 con 0x56190249e950 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fbf400640 1 --2- 172.21.15.50:0/2643898113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619020ae460 0x561902661620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.215+0000 7f6fbfe00640 1 --2- 172.21.15.50:0/2643898113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56190249e950 0x5619026620a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009b80 tx=0x7f6fb8014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccae6c2c6bc96cce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6faea00640 1 -- 172.21.15.50:0/2643898113 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8022070 con 0x56190249e950 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6faea00640 1 -- 172.21.15.50:0/2643898113 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fb801d440 con 0x56190249e950 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6faea00640 1 -- 172.21.15.50:0/2643898113 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb801d890 con 0x56190249e950 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2643898113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56190249e950 msgr2=0x5619026620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/2643898113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56190249e950 0x5619026620a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009b80 tx=0x7f6fb8014780 comp rx=0 tx=0).stop 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2643898113 shutdown_connections 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/2643898113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56190249e950 0x5619026620a0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/2643898113 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x561902487ca0 0x561902661b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/2643898113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5619020ae460 0x561902661620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2643898113 >> 172.21.15.50:0/2643898113 conn(0x561902574d90 msgr2=0x5619025ed620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2643898113 shutdown_connections 2024-09-21T06:11:21.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2643898113 wait complete. 2024-09-21T06:11:21.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 -- start start 2024-09-21T06:11:21.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 Processor -- start 2024-09-21T06:11:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x561902621220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561902487ca0 0x561902621760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x56190249e950 0x561902621ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5619023f93b0 con 0x5619020ae460 2024-09-21T06:11:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5619024b0750 con 0x561902487ca0 2024-09-21T06:11:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x561902475a80 con 0x56190249e950 2024-09-21T06:11:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fcd7f2000 10 client.0 _ll_register_callbacks cb 0x7f6fb80092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fbf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x561902621220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fbf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x561902621220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32908/0 (socket says 172.21.15.50:32908) 2024-09-21T06:11:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.219+0000 7f6fbf400640 1 -- 172.21.15.50:0/2603924485 learned_addr learned my addr 172.21.15.50:0/2603924485 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fb6a00640 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561902487ca0 0x561902621760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fbfe00640 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x56190249e950 0x561902621ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fbf400640 1 -- 172.21.15.50:0/2603924485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x56190249e950 msgr2=0x561902621ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fbf400640 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x56190249e950 0x561902621ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fbf400640 1 -- 172.21.15.50:0/2603924485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561902487ca0 msgr2=0x561902621760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fbf400640 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561902487ca0 0x561902621760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fbf400640 1 -- 172.21.15.50:0/2603924485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619023b7ab0 con 0x5619020ae460 2024-09-21T06:11:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fbfe00640 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x56190249e950 0x561902621ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fbf400640 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x561902621220 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f6fb001cfe0 tx=0x7f6fb00094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2138122b4bb6372 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:11:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fa3e00640 1 -- 172.21.15.50:0/2603924485 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00b2070 con 0x5619020ae460 2024-09-21T06:11:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fa3e00640 1 -- 172.21.15.50:0/2603924485 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f6fb00b9750 con 0x5619020ae460 2024-09-21T06:11:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2603924485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb801d440 con 0x5619020ae460 2024-09-21T06:11:21.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fa3e00640 1 -- 172.21.15.50:0/2603924485 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f6fb00e8050 con 0x5619020ae460 2024-09-21T06:11:21.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 10 client.8967 fetch_fsmap learned FSMap version 84 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2603924485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x561902396070 con 0x5619020ae460 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fa3e00640 1 -- 172.21.15.50:0/2603924485 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fb801d440 con 0x5619020ae460 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fa3e00640 1 -- 172.21.15.50:0/2603924485 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7f6fb00eb020 con 0x5619020ae460 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 10 client.8967 fetch_fsmap finished waiting for FSMap version 84 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 10 client.8967 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 20 client.8967 populate_metadata read hostname 'smithi050' 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.50:6832/1864490990,v1:172.21.15.50:6833/1864490990] conn(0x561902081830 0x561902622600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 4 client.8967 mds_command: new command op to 8832 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2603924485 --> [v2:172.21.15.50:6832/1864490990,v1:172.21.15.50:6833/1864490990] -- command(tid 0: {"prefix": "status"}) -- 0x5619023ff540 con 0x561902081830 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fb6a00640 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.50:6832/1864490990,v1:172.21.15.50:6833/1864490990] conn(0x561902081830 0x561902622600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fb6a00640 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.50:6832/1864490990,v1:172.21.15.50:6833/1864490990] conn(0x561902081830 0x561902622600 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e9afbd167a9c293 server_cookie=3e3509aa9f480f8a in_seq=0 out_seq=0 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fa3e00640 10 client.8967 ms_handle_connect on v2:172.21.15.50:6832/1864490990 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fa3e00640 1 -- 172.21.15.50:0/2603924485 <== mds.0 v2:172.21.15.50:6832/1864490990 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5619023ff540 con 0x561902081830 2024-09-21T06:11:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fa3e00640 10 client.8967 handle_command_reply: tid=0 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 client.8967 shutdown 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2603924485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 msgr2=0x561902621220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x561902621220 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f6fb001cfe0 tx=0x7f6fb00094a0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2603924485 shutdown_connections 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x56190249e950 0x561902621ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561902487ca0 0x561902621760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.50:6832/1864490990,v1:172.21.15.50:6833/1864490990] conn(0x561902081830 0x561902622600 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 --2- 172.21.15.50:0/2603924485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619020ae460 0x561902621220 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2603924485 >> 172.21.15.50:0/2603924485 conn(0x561902574d90 msgr2=0x5619025edfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fa3e00640 0 client.8967 ms_handle_reset on v2:172.21.15.50:6832/1864490990 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2603924485 shutdown_connections 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 1 -- 172.21.15.50:0/2603924485 wait complete. 2024-09-21T06:11:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fcd7f2000 20 client.8967 trim_cache size 0 max 16384 2024-09-21T06:11:21.226 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:11:21.226 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:11:21.226 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-21T06:11:21.226 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 8832, 2024-09-21T06:11:21.226 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-21T06:11:21.226 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-21T06:11:21.226 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-21T06:11:21.226 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 6.1611235520000003, 2024-09-21T06:11:21.226 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 84, 2024-09-21T06:11:21.226 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 127, 2024-09-21T06:11:21.226 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 127, 2024-09-21T06:11:21.226 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 10.580700731 2024-09-21T06:11:21.227 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:11:21.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fc6000640 1 -- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f9c075180 msgr2=0x7f6f9c077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fc6000640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f9c075180 0x7f6f9c077690 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00af000 tx=0x7f6fb001b5b0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fc6000640 1 -- 172.21.15.50:0/1363412727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fc015c5d0 msgr2=0x7f6fc0150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.223+0000 7f6fc6000640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fc015c5d0 0x7f6fc0150970 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800c970 tx=0x7f6fa800ce30 comp rx=0 tx=0).stop 2024-09-21T06:11:21.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.227+0000 7f6fc6000640 1 -- 172.21.15.50:0/1363412727 shutdown_connections 2024-09-21T06:11:21.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.227+0000 7f6fc6000640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6fc015f6b0 0x7f6fc0150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.227+0000 7f6fc6000640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f9c075180 0x7f6f9c077690 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.227+0000 7f6fc6000640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fc015c5d0 0x7f6fc0150970 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.227+0000 7f6fc6000640 1 --2- 172.21.15.50:0/1363412727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc0074300 0x7f6fc01f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.227+0000 7f6fc6000640 1 -- 172.21.15.50:0/1363412727 >> 172.21.15.50:0/1363412727 conn(0x7f6fc00b2c40 msgr2=0x7f6fc014fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.227+0000 7f6fc6000640 1 -- 172.21.15.50:0/1363412727 shutdown_connections 2024-09-21T06:11:21.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.227+0000 7f6fc6000640 1 -- 172.21.15.50:0/1363412727 wait complete. 2024-09-21T06:11:21.243 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:11:21.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.403+0000 7fec0e400640 1 -- 172.21.15.50:0/2164132084 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec080d01c0 msgr2=0x7fec08167420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.403+0000 7fec0e400640 1 --2- 172.21.15.50:0/2164132084 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec080d01c0 0x7fec08167420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fec00009bf0 tx=0x7fec0001cb00 comp rx=0 tx=0).stop 2024-09-21T06:11:21.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.403+0000 7fec0e400640 1 -- 172.21.15.50:0/2164132084 shutdown_connections 2024-09-21T06:11:21.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.403+0000 7fec0e400640 1 --2- 172.21.15.50:0/2164132084 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec080d01c0 0x7fec08167420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.403+0000 7fec0e400640 1 -- 172.21.15.50:0/2164132084 >> 172.21.15.50:0/2164132084 conn(0x7fec080b2c40 msgr2=0x7fec080b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.403+0000 7fec0e400640 1 -- 172.21.15.50:0/2164132084 shutdown_connections 2024-09-21T06:11:21.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.403+0000 7fec0e400640 1 -- 172.21.15.50:0/2164132084 wait complete. 2024-09-21T06:11:21.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec0e400640 1 Processor -- start 2024-09-21T06:11:21.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec0e400640 1 -- start start 2024-09-21T06:11:21.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec0e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec080cd2e0 0x7fec0813f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec0e400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec080d01c0 0x7fec0815be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec0e400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec081602a0 0x7fec08166750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec0e400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fec0816b0c0 con 0x7fec081602a0 2024-09-21T06:11:21.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec0e400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec0816af40 con 0x7fec080cd2e0 2024-09-21T06:11:21.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec0e400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fec0816b240 con 0x7fec080d01c0 2024-09-21T06:11:21.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec07e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec081602a0 0x7fec08166750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec07e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec081602a0 0x7fec08166750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32910/0 (socket says 172.21.15.50:32910) 2024-09-21T06:11:21.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec07e00640 1 -- 172.21.15.50:0/2249221740 learned_addr learned my addr 172.21.15.50:0/2249221740 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec07400640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec080cd2e0 0x7fec0813f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec06a00640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec080d01c0 0x7fec0815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec07e00640 1 -- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec080d01c0 msgr2=0x7fec0815be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec07e00640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec080d01c0 0x7fec0815be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec07e00640 1 -- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec080cd2e0 msgr2=0x7fec0813f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec07e00640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec080cd2e0 0x7fec0813f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec07e00640 1 -- 172.21.15.50:0/2249221740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec081f9420 con 0x7fec081602a0 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.407+0000 7fec06a00640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec080d01c0 0x7fec0815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.411+0000 7fec07e00640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec081602a0 0x7fec08166750 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fec0001a730 tx=0x7fec00009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.411+0000 7febff400640 1 -- 172.21.15.50:0/2249221740 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec000b2040 con 0x7fec081602a0 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.411+0000 7febff400640 1 -- 172.21.15.50:0/2249221740 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec00004d40 con 0x7fec081602a0 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.411+0000 7fec0e400640 1 -- 172.21.15.50:0/2249221740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec081f9700 con 0x7fec081602a0 2024-09-21T06:11:21.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.411+0000 7febff400640 1 -- 172.21.15.50:0/2249221740 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00005280 con 0x7fec081602a0 2024-09-21T06:11:21.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.411+0000 7fec0e400640 1 -- 172.21.15.50:0/2249221740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec081f9cc0 con 0x7fec081602a0 2024-09-21T06:11:21.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.411+0000 7fec0e400640 1 -- 172.21.15.50:0/2249221740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0815c970 con 0x7fec081602a0 2024-09-21T06:11:21.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.411+0000 7febff400640 1 -- 172.21.15.50:0/2249221740 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7fec000be060 con 0x7fec081602a0 2024-09-21T06:11:21.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.411+0000 7febff400640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7febe8075180 0x7febe8077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.411+0000 7febff400640 1 -- 172.21.15.50:0/2249221740 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fec0012cbc0 con 0x7fec081602a0 2024-09-21T06:11:21.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.415+0000 7fec07400640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7febe8075180 0x7febe8077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.415+0000 7fec07400640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7febe8075180 0x7febe8077690 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7febf8007fd0 tx=0x7febf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.415+0000 7febff400640 1 -- 172.21.15.50:0/2249221740 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fec000166f0 con 0x7fec081602a0 2024-09-21T06:11:21.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.631+0000 7fec0e400640 1 -- 172.21.15.50:0/2249221740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fec08070e80 con 0x7fec081602a0 2024-09-21T06:11:21.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.631+0000 7febff400640 1 -- 172.21.15.50:0/2249221740 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) ==== 53+0+73 (secure 0 0 0) 0x7fec000f96d0 con 0x7fec081602a0 2024-09-21T06:11:21.630 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:11:21.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.635+0000 7fec0e400640 1 -- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7febe8075180 msgr2=0x7febe8077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.635+0000 7fec0e400640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7febe8075180 0x7febe8077690 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7febf8007fd0 tx=0x7febf8009040 comp rx=0 tx=0).stop 2024-09-21T06:11:21.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.635+0000 7fec0e400640 1 -- 172.21.15.50:0/2249221740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec081602a0 msgr2=0x7fec08166750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.635+0000 7fec0e400640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec081602a0 0x7fec08166750 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fec0001a730 tx=0x7fec00009280 comp rx=0 tx=0).stop 2024-09-21T06:11:21.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.635+0000 7fec0e400640 1 -- 172.21.15.50:0/2249221740 shutdown_connections 2024-09-21T06:11:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.635+0000 7fec0e400640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7febe8075180 0x7febe8077690 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.635+0000 7fec0e400640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec081602a0 0x7fec08166750 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.635+0000 7fec0e400640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fec080d01c0 0x7fec0815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.635+0000 7fec0e400640 1 --2- 172.21.15.50:0/2249221740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec080cd2e0 0x7fec0813f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.635+0000 7fec0e400640 1 -- 172.21.15.50:0/2249221740 >> 172.21.15.50:0/2249221740 conn(0x7fec080b2c40 msgr2=0x7fec0815f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.635+0000 7fec0e400640 1 -- 172.21.15.50:0/2249221740 shutdown_connections 2024-09-21T06:11:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21.635+0000 7fec0e400640 1 -- 172.21.15.50:0/2249221740 wait complete. 2024-09-21T06:11:21.653 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:11:21.653 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:11:21.653 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-21T06:11:21.653 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:11:21.653 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:11:21.653 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:11:21.653 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:11:21.653 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:11:21.653 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:11:21.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:21.654 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-21T06:11:21.664 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:11:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:11:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:11:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:11:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T06:11:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:11:21.664 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:11:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:11:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 41514sec preferred_lft 41514sec 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-21T06:11:21.665 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:8b:d0:0a:5d brd ff:ff:ff:ff:ff:ff 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:11:21.666 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:11:21.667 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:11:21.667 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T06:11:21.667 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:11:21.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:21.667 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:11:21.725 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-21T06:11:21.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:21.727 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-21T06:11:21.784 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-21T06:11:21.784 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:11:21.784 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-21T06:11:21.784 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T06:11:21.784 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-21T06:11:21.784 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:11:21.785 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:11:21.785 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T06:11:21.785 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:11:21.785 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:11:21.785 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:11:21.785 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T06:11:21.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:11:21.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:21.927 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:11:21.936 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:11:21.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:21.937 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-09-21T06:11:22.000 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:11:22.000 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:11:22.051 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:11:22.051 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:11:22.051 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:11:22.103 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:11:22.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:22.103 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:11:22.208 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:11:22.208 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:11:22.208 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:11:22.226 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:11:22.227 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:11:22.227 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:11:22.227 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:11:22.227 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:11:22.237 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:11:22.237 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:11:22.237 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:11:22.237 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:11:22.237 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:11:22.237 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T06:11:22.237 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T06:11:22.237 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T06:11:22.241 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:11:22.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:22.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:11:22.261 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:11:22.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:11:22.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:11:22.367 INFO:teuthology.orchestra.run.smithi050.stdout:client7493 2024-09-21T06:11:22.367 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:11:22.367 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:11:22.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T06:11:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.587+0000 7f6ffca00640 1 -- 172.21.15.50:0/1765981103 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ff8074950 msgr2=0x7f6ff815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.587+0000 7f6ffca00640 1 --2- 172.21.15.50:0/1765981103 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ff8074950 0x7f6ff815cf90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009840 tx=0x7f6ff001c820 comp rx=0 tx=0).stop 2024-09-21T06:11:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.587+0000 7f6ffca00640 1 -- 172.21.15.50:0/1765981103 shutdown_connections 2024-09-21T06:11:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.587+0000 7f6ffca00640 1 --2- 172.21.15.50:0/1765981103 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ff8074950 0x7f6ff815cf90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.587+0000 7f6ffca00640 1 --2- 172.21.15.50:0/1765981103 >> v2:172.21.15.88:3300/0 conn(0x7f6ff80b5a80 0x7f6ff8074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.587+0000 7f6ffca00640 1 -- 172.21.15.50:0/1765981103 >> 172.21.15.50:0/1765981103 conn(0x7f6ff80708c0 msgr2=0x7f6ff8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:22.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.587+0000 7f6ffca00640 1 -- 172.21.15.50:0/1765981103 shutdown_connections 2024-09-21T06:11:22.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.587+0000 7f6ffca00640 1 -- 172.21.15.50:0/1765981103 wait complete. 2024-09-21T06:11:22.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.587+0000 7f6ffca00640 1 Processor -- start 2024-09-21T06:11:22.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.587+0000 7f6ffca00640 1 -- start start 2024-09-21T06:11:22.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6ffca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff8074950 0x7f6ff81fc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:22.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6ffca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff80b5a80 0x7f6ff8155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:22.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6ffca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ff8157030 0x7f6ff81f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:22.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6ffca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff816a310 con 0x7f6ff80b5a80 2024-09-21T06:11:22.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6ffca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff816a190 con 0x7f6ff8074950 2024-09-21T06:11:22.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6ffca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff816a490 con 0x7f6ff8157030 2024-09-21T06:11:22.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6fed400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff80b5a80 0x7f6ff8155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:22.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6fed400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff80b5a80 0x7f6ff8155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32922/0 (socket says 172.21.15.50:32922) 2024-09-21T06:11:22.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6fed400640 1 -- 172.21.15.50:0/2445083357 learned_addr learned my addr 172.21.15.50:0/2445083357 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:22.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6ff5e00640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff8074950 0x7f6ff81fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:22.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6ff6800640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ff8157030 0x7f6ff81f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6fed400640 1 -- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ff8157030 msgr2=0x7f6ff81f9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6fed400640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ff8157030 0x7f6ff81f9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6fed400640 1 -- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff8074950 msgr2=0x7f6ff81fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6fed400640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff8074950 0x7f6ff81fc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6fed400640 1 -- 172.21.15.50:0/2445083357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff81fd260 con 0x7f6ff80b5a80 2024-09-21T06:11:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6fed400640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff80b5a80 0x7f6ff8155030 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000e9a0 tx=0x7f6fe000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6feea00640 1 -- 172.21.15.50:0/2445083357 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe000cd80 con 0x7f6ff80b5a80 2024-09-21T06:11:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6ffca00640 1 -- 172.21.15.50:0/2445083357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff81fd540 con 0x7f6ff80b5a80 2024-09-21T06:11:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6feea00640 1 -- 172.21.15.50:0/2445083357 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fe0004020 con 0x7f6ff80b5a80 2024-09-21T06:11:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6feea00640 1 -- 172.21.15.50:0/2445083357 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0004450 con 0x7f6ff80b5a80 2024-09-21T06:11:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6ffca00640 1 -- 172.21.15.50:0/2445083357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff81f9480 con 0x7f6ff80b5a80 2024-09-21T06:11:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.591+0000 7f6ffca00640 1 -- 172.21.15.50:0/2445083357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff81fcfd0 con 0x7f6ff80b5a80 2024-09-21T06:11:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.595+0000 7f6feea00640 1 -- 172.21.15.50:0/2445083357 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f6fe0003600 con 0x7f6ff80b5a80 2024-09-21T06:11:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.595+0000 7f6feea00640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6fd40751b0 0x7f6fd40776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.595+0000 7f6feea00640 1 -- 172.21.15.50:0/2445083357 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f6fe0093bc0 con 0x7f6ff80b5a80 2024-09-21T06:11:22.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.595+0000 7f6ff5e00640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6fd40751b0 0x7f6fd40776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:22.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.595+0000 7f6ff5e00640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6fd40751b0 0x7f6fd40776c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8001780 tx=0x7f6fe8001e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.599+0000 7f6feea00640 1 -- 172.21.15.50:0/2445083357 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6fe005ff70 con 0x7f6ff80b5a80 2024-09-21T06:11:22.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:22.815+0000 7f6ffca00640 1 -- 172.21.15.50:0/2445083357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f6ff815cf90 con 0x7f6ff80b5a80 2024-09-21T06:11:23.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6feea00640 1 -- 172.21.15.50:0/2445083357 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v85) ==== 136+0+0 (secure 0 0 0) 0x7f6fe00649a0 con 0x7f6ff80b5a80 2024-09-21T06:11:23.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6ffca00640 1 -- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6fd40751b0 msgr2=0x7f6fd40776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6ffca00640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6fd40751b0 0x7f6fd40776c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8001780 tx=0x7f6fe8001e80 comp rx=0 tx=0).stop 2024-09-21T06:11:23.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6ffca00640 1 -- 172.21.15.50:0/2445083357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff80b5a80 msgr2=0x7f6ff8155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6ffca00640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff80b5a80 0x7f6ff8155030 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000e9a0 tx=0x7f6fe000ee60 comp rx=0 tx=0).stop 2024-09-21T06:11:23.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6ffca00640 1 -- 172.21.15.50:0/2445083357 shutdown_connections 2024-09-21T06:11:23.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6ffca00640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ff8157030 0x7f6ff81f9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6ffca00640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6fd40751b0 0x7f6fd40776c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6ffca00640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff80b5a80 0x7f6ff8155030 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6ffca00640 1 --2- 172.21.15.50:0/2445083357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff8074950 0x7f6ff81fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6ffca00640 1 -- 172.21.15.50:0/2445083357 >> 172.21.15.50:0/2445083357 conn(0x7f6ff80708c0 msgr2=0x7f6ff8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:23.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6ffca00640 1 -- 172.21.15.50:0/2445083357 shutdown_connections 2024-09-21T06:11:23.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.127+0000 7f6ffca00640 1 -- 172.21.15.50:0/2445083357 wait complete. 2024-09-21T06:11:23.148 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:11:23.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.307+0000 7fafaf800640 1 -- 172.21.15.50:0/1681098882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafa80b54f0 msgr2=0x7fafa8155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.307+0000 7fafaf800640 1 --2- 172.21.15.50:0/1681098882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafa80b54f0 0x7fafa8155b50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fafa8073920 tx=0x7faf9401c770 comp rx=0 tx=0).stop 2024-09-21T06:11:23.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.307+0000 7fafaf800640 1 -- 172.21.15.50:0/1681098882 shutdown_connections 2024-09-21T06:11:23.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.307+0000 7fafaf800640 1 --2- 172.21.15.50:0/1681098882 >> v2:172.21.15.88:3300/0 conn(0x7fafa8073a90 0x7fafa8073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.307+0000 7fafaf800640 1 --2- 172.21.15.50:0/1681098882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafa80b54f0 0x7fafa8155b50 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.307+0000 7fafaf800640 1 -- 172.21.15.50:0/1681098882 >> 172.21.15.50:0/1681098882 conn(0x7fafa80b2040 msgr2=0x7fafa80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:23.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.307+0000 7fafaf800640 1 -- 172.21.15.50:0/1681098882 shutdown_connections 2024-09-21T06:11:23.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.307+0000 7fafaf800640 1 -- 172.21.15.50:0/1681098882 wait complete. 2024-09-21T06:11:23.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafaf800640 1 Processor -- start 2024-09-21T06:11:23.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafaf800640 1 -- start start 2024-09-21T06:11:23.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafaf800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fafa8072100 0x7fafa80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafaf800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafa8073a90 0x7fafa80aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafaf800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafa81f8670 0x7fafa80b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafaf800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8168ec0 con 0x7fafa8073a90 2024-09-21T06:11:23.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafaf800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8168d40 con 0x7fafa81f8670 2024-09-21T06:11:23.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafaf800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fafa8169040 con 0x7fafa8072100 2024-09-21T06:11:23.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafa7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafa8073a90 0x7fafa80aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafa7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafa8073a90 0x7fafa80aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32944/0 (socket says 172.21.15.50:32944) 2024-09-21T06:11:23.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafa7e00640 1 -- 172.21.15.50:0/2392923372 learned_addr learned my addr 172.21.15.50:0/2392923372 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:23.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafad800640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafa81f8670 0x7fafa80b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7faface00640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fafa8072100 0x7fafa80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafa7e00640 1 -- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fafa8072100 msgr2=0x7fafa80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafa7e00640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fafa8072100 0x7fafa80ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafa7e00640 1 -- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafa81f8670 msgr2=0x7fafa80b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafa7e00640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafa81f8670 0x7fafa80b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.311+0000 7fafa7e00640 1 -- 172.21.15.50:0/2392923372 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa80b1900 con 0x7fafa8073a90 2024-09-21T06:11:23.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.315+0000 7fafa7e00640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafa8073a90 0x7fafa80aee20 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7faf94019380 tx=0x7faf940a6b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:23.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.315+0000 7fafa4c00640 1 -- 172.21.15.50:0/2392923372 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf940a6d00 con 0x7fafa8073a90 2024-09-21T06:11:23.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.315+0000 7fafa4c00640 1 -- 172.21.15.50:0/2392923372 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf940a6ea0 con 0x7fafa8073a90 2024-09-21T06:11:23.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.315+0000 7fafa4c00640 1 -- 172.21.15.50:0/2392923372 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94004290 con 0x7fafa8073a90 2024-09-21T06:11:23.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.315+0000 7fafaf800640 1 -- 172.21.15.50:0/2392923372 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa8165860 con 0x7fafa8073a90 2024-09-21T06:11:23.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.315+0000 7fafaf800640 1 -- 172.21.15.50:0/2392923372 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa8165e20 con 0x7fafa8073a90 2024-09-21T06:11:23.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.315+0000 7fafaf800640 1 -- 172.21.15.50:0/2392923372 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa8073ea0 con 0x7fafa8073a90 2024-09-21T06:11:23.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.315+0000 7fafa4c00640 1 -- 172.21.15.50:0/2392923372 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7faf94005070 con 0x7fafa8073a90 2024-09-21T06:11:23.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.315+0000 7fafa4c00640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf88075180 0x7faf88077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.319+0000 7fafa4c00640 1 -- 172.21.15.50:0/2392923372 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7faf9412cf00 con 0x7fafa8073a90 2024-09-21T06:11:23.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.319+0000 7faface00640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf88075180 0x7faf88077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.319+0000 7faface00640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf88075180 0x7faf88077690 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7faf9c000f70 tx=0x7faf9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:23.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.323+0000 7fafa4c00640 1 -- 172.21.15.50:0/2392923372 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7faf940bf030 con 0x7fafa8073a90 2024-09-21T06:11:23.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.519+0000 7fafaf800640 1 -- 172.21.15.50:0/2392923372 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fafa806b540 con 0x7faf88075180 2024-09-21T06:11:23.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.523+0000 7fafa4c00640 1 -- 172.21.15.50:0/2392923372 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7faf9c0050d0 con 0x7faf88075180 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:11:23.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.523+0000 7fafaf800640 1 -- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf88075180 msgr2=0x7faf88077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.523+0000 7fafaf800640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf88075180 0x7faf88077690 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7faf9c000f70 tx=0x7faf9c009040 comp rx=0 tx=0).stop 2024-09-21T06:11:23.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.523+0000 7fafaf800640 1 -- 172.21.15.50:0/2392923372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafa8073a90 msgr2=0x7fafa80aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.523+0000 7fafaf800640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafa8073a90 0x7fafa80aee20 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7faf94019380 tx=0x7faf940a6b10 comp rx=0 tx=0).stop 2024-09-21T06:11:23.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.527+0000 7fafaf800640 1 -- 172.21.15.50:0/2392923372 shutdown_connections 2024-09-21T06:11:23.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.527+0000 7fafaf800640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafa81f8670 0x7fafa80b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.527+0000 7fafaf800640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7faf88075180 0x7faf88077690 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.527+0000 7fafaf800640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafa8073a90 0x7fafa80aee20 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.527+0000 7fafaf800640 1 --2- 172.21.15.50:0/2392923372 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fafa8072100 0x7fafa80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.527+0000 7fafaf800640 1 -- 172.21.15.50:0/2392923372 >> 172.21.15.50:0/2392923372 conn(0x7fafa80b2040 msgr2=0x7fafa80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:23.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.527+0000 7fafaf800640 1 -- 172.21.15.50:0/2392923372 shutdown_connections 2024-09-21T06:11:23.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.527+0000 7fafaf800640 1 -- 172.21.15.50:0/2392923372 wait complete. 2024-09-21T06:11:23.544 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:11:23.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.707+0000 7f161f200640 1 -- 172.21.15.50:0/2858062636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1604005360 msgr2=0x7f160400b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.707+0000 7f161f200640 1 --2- 172.21.15.50:0/2858062636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1604005360 0x7f160400b9f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f1620074e10 tx=0x7f161801caf0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.707+0000 7f161f200640 1 -- 172.21.15.50:0/2858062636 shutdown_connections 2024-09-21T06:11:23.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.707+0000 7f161f200640 1 --2- 172.21.15.50:0/2858062636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1604005360 0x7f160400b9f0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.707+0000 7f161f200640 1 -- 172.21.15.50:0/2858062636 >> 172.21.15.50:0/2858062636 conn(0x7f162006cdf0 msgr2=0x7f162006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:23.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.707+0000 7f161f200640 1 -- 172.21.15.50:0/2858062636 shutdown_connections 2024-09-21T06:11:23.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.707+0000 7f161f200640 1 -- 172.21.15.50:0/2858062636 wait complete. 2024-09-21T06:11:23.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.711+0000 7f161f200640 1 Processor -- start 2024-09-21T06:11:23.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.711+0000 7f161f200640 1 -- start start 2024-09-21T06:11:23.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.711+0000 7f161f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1604005360 0x7f1620151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.711+0000 7f161f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16200cf2f0 0x7f16200d0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.711+0000 7f161f200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f16200d0670 0x7f1620156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.711+0000 7f161f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f16200b2560 con 0x7f16200cf2f0 2024-09-21T06:11:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.711+0000 7f161f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f16200b23e0 con 0x7f1604005360 2024-09-21T06:11:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.711+0000 7f161f200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f16200b54c0 con 0x7f16200d0670 2024-09-21T06:11:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.711+0000 7f1617e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16200cf2f0 0x7f16200d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.711+0000 7f1617e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16200cf2f0 0x7f16200d0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32954/0 (socket says 172.21.15.50:32954) 2024-09-21T06:11:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.711+0000 7f1617e00640 1 -- 172.21.15.50:0/1945501402 learned_addr learned my addr 172.21.15.50:0/1945501402 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f161de00640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1604005360 0x7f1620151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f161e800640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f16200d0670 0x7f1620156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f1617e00640 1 -- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f16200d0670 msgr2=0x7f1620156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f1617e00640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f16200d0670 0x7f1620156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f1617e00640 1 -- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1604005360 msgr2=0x7f1620151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f1617e00640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1604005360 0x7f1620151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f1617e00640 1 -- 172.21.15.50:0/1945501402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16200d0a80 con 0x7f16200cf2f0 2024-09-21T06:11:23.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f161de00640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1604005360 0x7f1620151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:23.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f1617e00640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16200cf2f0 0x7f16200d0130 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f16080118b0 tx=0x7f1608011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:23.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f1616000640 1 -- 172.21.15.50:0/1945501402 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608004440 con 0x7f16200cf2f0 2024-09-21T06:11:23.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f1616000640 1 -- 172.21.15.50:0/1945501402 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f160800f9a0 con 0x7f16200cf2f0 2024-09-21T06:11:23.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f161f200640 1 -- 172.21.15.50:0/1945501402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16201582e0 con 0x7f16200cf2f0 2024-09-21T06:11:23.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f1616000640 1 -- 172.21.15.50:0/1945501402 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160800fd90 con 0x7f16200cf2f0 2024-09-21T06:11:23.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f161f200640 1 -- 172.21.15.50:0/1945501402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16200aec60 con 0x7f16200cf2f0 2024-09-21T06:11:23.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f161f200640 1 -- 172.21.15.50:0/1945501402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16200cfe80 con 0x7f16200cf2f0 2024-09-21T06:11:23.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.715+0000 7f1616000640 1 -- 172.21.15.50:0/1945501402 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f1608003600 con 0x7f16200cf2f0 2024-09-21T06:11:23.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.719+0000 7f1616000640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15fc0751b0 0x7f15fc0776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.719+0000 7f1616000640 1 -- 172.21.15.50:0/1945501402 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f1608094270 con 0x7f16200cf2f0 2024-09-21T06:11:23.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.719+0000 7f161de00640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15fc0751b0 0x7f15fc0776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.719+0000 7f161de00640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15fc0751b0 0x7f15fc0776c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1610001ec0 tx=0x7f1610006980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:23.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.723+0000 7f1616000640 1 -- 172.21.15.50:0/1945501402 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f16200d0a80 con 0x7f16200cf2f0 2024-09-21T06:11:23.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 -- 172.21.15.50:0/1945501402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f162015f380 con 0x7f16200cf2f0 2024-09-21T06:11:23.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f1616000640 1 -- 172.21.15.50:0/1945501402 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v55)=0 v55) ==== 122+0+0 (secure 0 0 0) 0x7f1608060e00 con 0x7f16200cf2f0 2024-09-21T06:11:23.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 -- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15fc0751b0 msgr2=0x7f15fc0776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15fc0751b0 0x7f15fc0776c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1610001ec0 tx=0x7f1610006980 comp rx=0 tx=0).stop 2024-09-21T06:11:23.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 -- 172.21.15.50:0/1945501402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16200cf2f0 msgr2=0x7f16200d0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16200cf2f0 0x7f16200d0130 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f16080118b0 tx=0x7f1608011d70 comp rx=0 tx=0).stop 2024-09-21T06:11:23.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 -- 172.21.15.50:0/1945501402 shutdown_connections 2024-09-21T06:11:23.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f16200d0670 0x7f1620156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15fc0751b0 0x7f15fc0776c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16200cf2f0 0x7f16200d0130 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 --2- 172.21.15.50:0/1945501402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1604005360 0x7f1620151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 -- 172.21.15.50:0/1945501402 >> 172.21.15.50:0/1945501402 conn(0x7f162006cdf0 msgr2=0x7f162015e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:23.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 -- 172.21.15.50:0/1945501402 shutdown_connections 2024-09-21T06:11:23.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:23.899+0000 7f161f200640 1 -- 172.21.15.50:0/1945501402 wait complete. 2024-09-21T06:11:23.920 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-21T06:11:24.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.083+0000 7f589ce00640 1 -- 172.21.15.50:0/1665252205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58980b4de0 msgr2=0x7f58980b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.083+0000 7f589ce00640 1 --2- 172.21.15.50:0/1665252205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58980b4de0 0x7f58980b51f0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f5898071cb0 tx=0x7f589001cac0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.083+0000 7f589ce00640 1 -- 172.21.15.50:0/1665252205 shutdown_connections 2024-09-21T06:11:24.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.083+0000 7f589ce00640 1 --2- 172.21.15.50:0/1665252205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58980b4de0 0x7f58980b51f0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.083+0000 7f589ce00640 1 --2- 172.21.15.50:0/1665252205 >> v2:172.21.15.88:3300/0 conn(0x7f58980b5c70 0x7f589815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.083+0000 7f589ce00640 1 -- 172.21.15.50:0/1665252205 >> 172.21.15.50:0/1665252205 conn(0x7f58980b2040 msgr2=0x7f58980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:24.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.083+0000 7f589ce00640 1 -- 172.21.15.50:0/1665252205 shutdown_connections 2024-09-21T06:11:24.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.083+0000 7f589ce00640 1 -- 172.21.15.50:0/1665252205 wait complete. 2024-09-21T06:11:24.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.083+0000 7f589ce00640 1 Processor -- start 2024-09-21T06:11:24.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.083+0000 7f589ce00640 1 -- start start 2024-09-21T06:11:24.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f589ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58980b4de0 0x7f58981fc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:24.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f589ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58980b5c70 0x7f58981543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:24.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f589ce00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f58981527b0 0x7f58980ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:24.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f589ce00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5898168920 con 0x7f58980b5c70 2024-09-21T06:11:24.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f589ce00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f58981687a0 con 0x7f58980b4de0 2024-09-21T06:11:24.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f589ce00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5898168aa0 con 0x7f58981527b0 2024-09-21T06:11:24.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f588d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58980b5c70 0x7f58981543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:24.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f588d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58980b5c70 0x7f58981543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32966/0 (socket says 172.21.15.50:32966) 2024-09-21T06:11:24.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f588d400640 1 -- 172.21.15.50:0/2923546861 learned_addr learned my addr 172.21.15.50:0/2923546861 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:24.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f5895e00640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58980b4de0 0x7f58981fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f5896800640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f58981527b0 0x7f58980ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f588d400640 1 -- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f58981527b0 msgr2=0x7f58980ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f588d400640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f58981527b0 0x7f58980ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f588d400640 1 -- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58980b4de0 msgr2=0x7f58981fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f588d400640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58980b4de0 0x7f58981fc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f588d400640 1 -- 172.21.15.50:0/2923546861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589815b690 con 0x7f58980b5c70 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f5895e00640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58980b4de0 0x7f58981fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f5896800640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f58981527b0 0x7f58980ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f588d400640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58980b5c70 0x7f58981543c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f588000c400 tx=0x7f588000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f588ea00640 1 -- 172.21.15.50:0/2923546861 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5880017070 con 0x7f58980b5c70 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f588ea00640 1 -- 172.21.15.50:0/2923546861 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5880003600 con 0x7f58980b5c70 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f588ea00640 1 -- 172.21.15.50:0/2923546861 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5880003a50 con 0x7f58980b5c70 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f589ce00640 1 -- 172.21.15.50:0/2923546861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58981f8d20 con 0x7f58980b5c70 2024-09-21T06:11:24.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f589ce00640 1 -- 172.21.15.50:0/2923546861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58981f92e0 con 0x7f58980b5c70 2024-09-21T06:11:24.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.087+0000 7f589ce00640 1 -- 172.21.15.50:0/2923546861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58980b51f0 con 0x7f58980b5c70 2024-09-21T06:11:24.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.091+0000 7f588ea00640 1 -- 172.21.15.50:0/2923546861 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f588001a070 con 0x7f58980b5c70 2024-09-21T06:11:24.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.091+0000 7f588ea00640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5874075180 0x7f5874077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:24.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.091+0000 7f588ea00640 1 -- 172.21.15.50:0/2923546861 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f58800924d0 con 0x7f58980b5c70 2024-09-21T06:11:24.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.091+0000 7f5895e00640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5874075180 0x7f5874077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:24.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.095+0000 7f5895e00640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5874075180 0x7f5874077690 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5888005fd0 tx=0x7f5888005dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:24.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.095+0000 7f588ea00640 1 -- 172.21.15.50:0/2923546861 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f58980b5730 con 0x7f58980b5c70 2024-09-21T06:11:24.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 -- 172.21.15.50:0/2923546861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f5898151b70 con 0x7f58980b5c70 2024-09-21T06:11:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f588ea00640 1 -- 172.21.15.50:0/2923546861 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v55) ==== 116+0+2 (secure 0 0 0) 0x7f588005f080 con 0x7f58980b5c70 2024-09-21T06:11:24.272 INFO:teuthology.orchestra.run.smithi050.stdout:4 2024-09-21T06:11:24.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 -- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5874075180 msgr2=0x7f5874077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5874075180 0x7f5874077690 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5888005fd0 tx=0x7f5888005dc0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 -- 172.21.15.50:0/2923546861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58980b5c70 msgr2=0x7f58981543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58980b5c70 0x7f58981543c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f588000c400 tx=0x7f588000c8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 -- 172.21.15.50:0/2923546861 shutdown_connections 2024-09-21T06:11:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f58981527b0 0x7f58980ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5874075180 0x7f5874077690 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58980b5c70 0x7f58981543c0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 --2- 172.21.15.50:0/2923546861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58980b4de0 0x7f58981fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 -- 172.21.15.50:0/2923546861 >> 172.21.15.50:0/2923546861 conn(0x7f58980b2040 msgr2=0x7f58980b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 -- 172.21.15.50:0/2923546861 shutdown_connections 2024-09-21T06:11:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.275+0000 7f589ce00640 1 -- 172.21.15.50:0/2923546861 wait complete. 2024-09-21T06:11:24.295 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:11:24.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.459+0000 7fc87da00640 1 -- 172.21.15.50:0/2085839958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8781527b0 msgr2=0x7fc878074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.459+0000 7fc87da00640 1 --2- 172.21.15.50:0/2085839958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8781527b0 0x7fc878074e30 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fc860009840 tx=0x7fc86001c760 comp rx=0 tx=0).stop 2024-09-21T06:11:24.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.459+0000 7fc87da00640 1 -- 172.21.15.50:0/2085839958 shutdown_connections 2024-09-21T06:11:24.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.459+0000 7fc87da00640 1 --2- 172.21.15.50:0/2085839958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8781527b0 0x7fc878074e30 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.459+0000 7fc87da00640 1 --2- 172.21.15.50:0/2085839958 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8780b5c70 0x7fc87815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.459+0000 7fc87da00640 1 -- 172.21.15.50:0/2085839958 >> 172.21.15.50:0/2085839958 conn(0x7fc8780b2040 msgr2=0x7fc8780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.459+0000 7fc87da00640 1 -- 172.21.15.50:0/2085839958 shutdown_connections 2024-09-21T06:11:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.459+0000 7fc87da00640 1 -- 172.21.15.50:0/2085839958 wait complete. 2024-09-21T06:11:24.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.459+0000 7fc87da00640 1 Processor -- start 2024-09-21T06:11:24.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.459+0000 7fc87da00640 1 -- start start 2024-09-21T06:11:24.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc87da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8780b4de0 0x7fc878152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:24.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc87da00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8780b5c70 0x7fc8781499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:24.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc87da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8781527b0 0x7fc8781f9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:24.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc87da00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc878168c00 con 0x7fc8780b4de0 2024-09-21T06:11:24.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc87da00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc878168a80 con 0x7fc8781527b0 2024-09-21T06:11:24.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc87da00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc878168d80 con 0x7fc8780b5c70 2024-09-21T06:11:24.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc876a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8780b4de0 0x7fc878152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:24.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc876a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8780b4de0 0x7fc878152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32982/0 (socket says 172.21.15.50:32982) 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc876a00640 1 -- 172.21.15.50:0/57299513 learned_addr learned my addr 172.21.15.50:0/57299513 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc86fe00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8780b5c70 0x7fc8781499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc877400640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8781527b0 0x7fc8781f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc876a00640 1 -- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8780b5c70 msgr2=0x7fc8781499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc876a00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8780b5c70 0x7fc8781499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc876a00640 1 -- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8781527b0 msgr2=0x7fc8781f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc876a00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8781527b0 0x7fc8781f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc876a00640 1 -- 172.21.15.50:0/57299513 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc878158ca0 con 0x7fc8780b4de0 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc86fe00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8780b5c70 0x7fc8781499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc876a00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8780b4de0 0x7fc878152c20 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fc86800e7f0 tx=0x7fc86800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc86ea00640 1 -- 172.21.15.50:0/57299513 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86800ccc0 con 0x7fc8780b4de0 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc86ea00640 1 -- 172.21.15.50:0/57299513 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc86800ce80 con 0x7fc8780b4de0 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc87da00640 1 -- 172.21.15.50:0/57299513 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc878158f80 con 0x7fc8780b4de0 2024-09-21T06:11:24.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc86ea00640 1 -- 172.21.15.50:0/57299513 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868005280 con 0x7fc8780b4de0 2024-09-21T06:11:24.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc87da00640 1 -- 172.21.15.50:0/57299513 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8781f8c80 con 0x7fc8780b4de0 2024-09-21T06:11:24.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.463+0000 7fc87da00640 1 -- 172.21.15.50:0/57299513 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc87815bb60 con 0x7fc8780b4de0 2024-09-21T06:11:24.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.467+0000 7fc86ea00640 1 -- 172.21.15.50:0/57299513 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7fc868005420 con 0x7fc8780b4de0 2024-09-21T06:11:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.467+0000 7fc86ea00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc854075180 0x7fc854077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.467+0000 7fc86ea00640 1 -- 172.21.15.50:0/57299513 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fc868093d10 con 0x7fc8780b4de0 2024-09-21T06:11:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.467+0000 7fc86fe00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc854075180 0x7fc854077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.467+0000 7fc86fe00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc854075180 0x7fc854077690 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc8600af000 tx=0x7fc86001afa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:24.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.471+0000 7fc86ea00640 1 -- 172.21.15.50:0/57299513 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc868060790 con 0x7fc8780b4de0 2024-09-21T06:11:24.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.647+0000 7fc87da00640 1 -- 172.21.15.50:0/57299513 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fc8781660e0 con 0x7fc8780b4de0 2024-09-21T06:11:24.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc86ea00640 1 -- 172.21.15.50:0/57299513 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v56)=0 v56) ==== 131+0+0 (secure 0 0 0) 0x7fc868060090 con 0x7fc8780b4de0 2024-09-21T06:11:24.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc87da00640 1 -- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc854075180 msgr2=0x7fc854077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc87da00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc854075180 0x7fc854077690 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc8600af000 tx=0x7fc86001afa0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc87da00640 1 -- 172.21.15.50:0/57299513 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8780b4de0 msgr2=0x7fc878152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc87da00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8780b4de0 0x7fc878152c20 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fc86800e7f0 tx=0x7fc86800ecb0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc87da00640 1 -- 172.21.15.50:0/57299513 shutdown_connections 2024-09-21T06:11:24.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc87da00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8781527b0 0x7fc8781f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc87da00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc8780b5c70 0x7fc8781499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc87da00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc854075180 0x7fc854077690 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc87da00640 1 --2- 172.21.15.50:0/57299513 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8780b4de0 0x7fc878152c20 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc87da00640 1 -- 172.21.15.50:0/57299513 >> 172.21.15.50:0/57299513 conn(0x7fc8780b2040 msgr2=0x7fc8780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:24.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc87da00640 1 -- 172.21.15.50:0/57299513 shutdown_connections 2024-09-21T06:11:24.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.655+0000 7fc87da00640 1 -- 172.21.15.50:0/57299513 wait complete. 2024-09-21T06:11:24.675 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-21T06:11:24.679 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:24.680+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:24.679 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:24.680+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:24.679 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:24.680+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:24.679 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:24.680+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:24.679 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:24.680+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:24.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.839+0000 7f0d8aa00640 1 -- 172.21.15.50:0/3930544851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d68004000 msgr2=0x7f0d68004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.839+0000 7f0d8aa00640 1 --2- 172.21.15.50:0/3930544851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d68004000 0x7f0d68004810 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f0d84071fd0 tx=0x7f0d6c01d1e0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.839+0000 7f0d88a00640 1 -- 172.21.15.50:0/3930544851 reap_dead start 2024-09-21T06:11:24.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.839+0000 7f0d8aa00640 1 -- 172.21.15.50:0/3930544851 shutdown_connections 2024-09-21T06:11:24.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.839+0000 7f0d8aa00640 1 -- 172.21.15.50:0/3930544851 >> 172.21.15.50:0/3930544851 conn(0x7f0d840b2040 msgr2=0x7f0d840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:24.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.839+0000 7f0d8aa00640 1 -- 172.21.15.50:0/3930544851 shutdown_connections 2024-09-21T06:11:24.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.839+0000 7f0d8aa00640 1 -- 172.21.15.50:0/3930544851 wait complete. 2024-09-21T06:11:24.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d8aa00640 1 Processor -- start 2024-09-21T06:11:24.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d8aa00640 1 -- start start 2024-09-21T06:11:24.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d8aa00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0d68004000 0x7f0d84140690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:24.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d8aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84158660 0x7f0d84158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:24.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d8aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d84155010 0x7f0d84155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:24.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d8aa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0d84166b10 con 0x7f0d84158660 2024-09-21T06:11:24.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d8aa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d84166990 con 0x7f0d84155010 2024-09-21T06:11:24.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d8aa00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0d84166c90 con 0x7f0d68004000 2024-09-21T06:11:24.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d7be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84158660 0x7f0d84158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:24.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d7be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84158660 0x7f0d84158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32996/0 (socket says 172.21.15.50:32996) 2024-09-21T06:11:24.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d83e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0d68004000 0x7f0d84140690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:24.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d7be00640 1 -- 172.21.15.50:0/2690038125 learned_addr learned my addr 172.21.15.50:0/2690038125 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:24.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d88a00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d84155010 0x7f0d84155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:24.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d7be00640 1 -- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0d68004000 msgr2=0x7f0d84140690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d7be00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0d68004000 0x7f0d84140690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d7be00640 1 -- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d84155010 msgr2=0x7f0d84155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:24.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d7be00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d84155010 0x7f0d84155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:24.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.843+0000 7f0d7be00640 1 -- 172.21.15.50:0/2690038125 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d84155960 con 0x7f0d84158660 2024-09-21T06:11:24.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.847+0000 7f0d7be00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84158660 0x7f0d84158a70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c01d010 tx=0x7f0d6c009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:24.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.847+0000 7f0d80c00640 1 -- 172.21.15.50:0/2690038125 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0bb030 con 0x7f0d84158660 2024-09-21T06:11:24.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.847+0000 7f0d80c00640 1 -- 172.21.15.50:0/2690038125 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d6c0047c0 con 0x7f0d84158660 2024-09-21T06:11:24.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.847+0000 7f0d80c00640 1 -- 172.21.15.50:0/2690038125 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c004bd0 con 0x7f0d84158660 2024-09-21T06:11:24.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.847+0000 7f0d8aa00640 1 -- 172.21.15.50:0/2690038125 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d841593b0 con 0x7f0d84158660 2024-09-21T06:11:24.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.847+0000 7f0d8aa00640 1 -- 172.21.15.50:0/2690038125 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d841fc600 con 0x7f0d84158660 2024-09-21T06:11:24.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.847+0000 7f0d8aa00640 1 -- 172.21.15.50:0/2690038125 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d84159010 con 0x7f0d84158660 2024-09-21T06:11:24.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.847+0000 7f0d80c00640 1 -- 172.21.15.50:0/2690038125 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f0d6c0be030 con 0x7f0d84158660 2024-09-21T06:11:24.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.851+0000 7f0d80c00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0d600751b0 0x7f0d600776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:24.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.851+0000 7f0d80c00640 1 -- 172.21.15.50:0/2690038125 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f0d6c12e520 con 0x7f0d84158660 2024-09-21T06:11:24.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.851+0000 7f0d83e00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0d600751b0 0x7f0d600776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:24.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.851+0000 7f0d83e00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0d600751b0 0x7f0d600776c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009ce0 tx=0x7f0d740091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:24.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:24.855+0000 7f0d80c00640 1 -- 172.21.15.50:0/2690038125 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0d6c016f20 con 0x7f0d84158660 2024-09-21T06:11:25.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.055+0000 7f0d8aa00640 1 -- 172.21.15.50:0/2690038125 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f0d8415bb40 con 0x7f0d600751b0 2024-09-21T06:11:25.064 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:25.064+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:25.064 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:25.064+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:25.064 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:25.064+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:25.064 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:25.064+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:25.064 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:25.064+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:25.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.107+0000 7f0d80c00640 1 -- 172.21.15.50:0/2690038125 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d8415bb40 con 0x7f0d600751b0 2024-09-21T06:11:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.111+0000 7f0d8aa00640 1 -- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0d600751b0 msgr2=0x7f0d600776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.111+0000 7f0d8aa00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0d600751b0 0x7f0d600776c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009ce0 tx=0x7f0d740091e0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.111+0000 7f0d8aa00640 1 -- 172.21.15.50:0/2690038125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84158660 msgr2=0x7f0d84158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.111+0000 7f0d8aa00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84158660 0x7f0d84158a70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c01d010 tx=0x7f0d6c009820 comp rx=0 tx=0).stop 2024-09-21T06:11:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.111+0000 7f0d8aa00640 1 -- 172.21.15.50:0/2690038125 shutdown_connections 2024-09-21T06:11:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.111+0000 7f0d8aa00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d84155010 0x7f0d84155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.111+0000 7f0d8aa00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0d600751b0 0x7f0d600776c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.111+0000 7f0d8aa00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84158660 0x7f0d84158a70 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.111+0000 7f0d8aa00640 1 --2- 172.21.15.50:0/2690038125 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0d68004000 0x7f0d84140690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.111+0000 7f0d8aa00640 1 -- 172.21.15.50:0/2690038125 >> 172.21.15.50:0/2690038125 conn(0x7f0d840b2040 msgr2=0x7f0d84159fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.111+0000 7f0d8aa00640 1 -- 172.21.15.50:0/2690038125 shutdown_connections 2024-09-21T06:11:25.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.111+0000 7f0d8aa00640 1 -- 172.21.15.50:0/2690038125 wait complete. 2024-09-21T06:11:25.128 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:11:25.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.291+0000 7f461be00640 1 -- 172.21.15.50:0/2122501113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45fc0039f0 msgr2=0x7f45fc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.291+0000 7f461be00640 1 --2- 172.21.15.50:0/2122501113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45fc0039f0 0x7f45fc003e20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4608006f80 tx=0x7f460801ced0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.291+0000 7f4619e00640 1 -- 172.21.15.50:0/2122501113 reap_dead start 2024-09-21T06:11:25.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.291+0000 7f461be00640 1 -- 172.21.15.50:0/2122501113 shutdown_connections 2024-09-21T06:11:25.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.291+0000 7f461be00640 1 -- 172.21.15.50:0/2122501113 >> 172.21.15.50:0/2122501113 conn(0x7f46140b2c40 msgr2=0x7f46140b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:25.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.291+0000 7f461be00640 1 -- 172.21.15.50:0/2122501113 shutdown_connections 2024-09-21T06:11:25.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.291+0000 7f461be00640 1 -- 172.21.15.50:0/2122501113 wait complete. 2024-09-21T06:11:25.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.295+0000 7f461be00640 1 Processor -- start 2024-09-21T06:11:25.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.295+0000 7f461be00640 1 -- start start 2024-09-21T06:11:25.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.295+0000 7f461be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45fc0039f0 0x7f46141ec820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:25.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.295+0000 7f461be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4614147690 0x7f4614146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:25.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.295+0000 7f461be00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4614164d30 0x7f4614165140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:25.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.295+0000 7f461be00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f461406fac0 con 0x7f4614147690 2024-09-21T06:11:25.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.295+0000 7f461be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46140746e0 con 0x7f45fc0039f0 2024-09-21T06:11:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.295+0000 7f461be00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f461406fc40 con 0x7f4614164d30 2024-09-21T06:11:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.295+0000 7f4618a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4614147690 0x7f4614146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.295+0000 7f4618a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4614147690 0x7f4614146e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33014/0 (socket says 172.21.15.50:33014) 2024-09-21T06:11:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.295+0000 7f4618a00640 1 -- 172.21.15.50:0/2224842632 learned_addr learned my addr 172.21.15.50:0/2224842632 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f4619400640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45fc0039f0 0x7f46141ec820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f4619e00640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4614164d30 0x7f4614165140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f4618a00640 1 -- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4614164d30 msgr2=0x7f4614165140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f4618a00640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4614164d30 0x7f4614165140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f4618a00640 1 -- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45fc0039f0 msgr2=0x7f46141ec820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f4618a00640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45fc0039f0 0x7f46141ec820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f4618a00640 1 -- 172.21.15.50:0/2224842632 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4614166040 con 0x7f4614147690 2024-09-21T06:11:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f4618a00640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4614147690 0x7f4614146e60 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f46000117d0 tx=0x7f4600011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f4611600640 1 -- 172.21.15.50:0/2224842632 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600004460 con 0x7f4614147690 2024-09-21T06:11:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f4611600640 1 -- 172.21.15.50:0/2224842632 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4600016070 con 0x7f4614147690 2024-09-21T06:11:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f461be00640 1 -- 172.21.15.50:0/2224842632 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4614166320 con 0x7f4614147690 2024-09-21T06:11:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f4611600640 1 -- 172.21.15.50:0/2224842632 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460000fc50 con 0x7f4614147690 2024-09-21T06:11:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f461be00640 1 -- 172.21.15.50:0/2224842632 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4614164810 con 0x7f4614147690 2024-09-21T06:11:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f461be00640 1 -- 172.21.15.50:0/2224842632 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461406ec70 con 0x7f4614147690 2024-09-21T06:11:25.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.299+0000 7f4611600640 1 -- 172.21.15.50:0/2224842632 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f4600018030 con 0x7f4614147690 2024-09-21T06:11:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.303+0000 7f4611600640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45f4075180 0x7f45f4077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.303+0000 7f4611600640 1 -- 172.21.15.50:0/2224842632 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f4600094110 con 0x7f4614147690 2024-09-21T06:11:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.303+0000 7f4619400640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45f4075180 0x7f45f4077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:25.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.303+0000 7f4619400640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45f4075180 0x7f45f4077690 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f46080ac000 tx=0x7f4608019d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:25.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.303+0000 7f4611600640 1 -- 172.21.15.50:0/2224842632 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4600060650 con 0x7f4614147690 2024-09-21T06:11:25.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.507+0000 7f461be00640 1 -- 172.21.15.50:0/2224842632 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f4614165680 con 0x7f45f4075180 2024-09-21T06:11:25.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.515+0000 7f4611600640 1 -- 172.21.15.50:0/2224842632 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f4608001540 con 0x7f45f4075180 2024-09-21T06:11:25.513 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/7de9ef65-c5ec-4ec4-a5d9-579e05105f30 2024-09-21T06:11:25.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.515+0000 7f461be00640 1 -- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45f4075180 msgr2=0x7f45f4077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.515+0000 7f461be00640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45f4075180 0x7f45f4077690 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f46080ac000 tx=0x7f4608019d10 comp rx=0 tx=0).stop 2024-09-21T06:11:25.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.515+0000 7f461be00640 1 -- 172.21.15.50:0/2224842632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4614147690 msgr2=0x7f4614146e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.515+0000 7f461be00640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4614147690 0x7f4614146e60 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f46000117d0 tx=0x7f4600011c90 comp rx=0 tx=0).stop 2024-09-21T06:11:25.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.515+0000 7f461be00640 1 -- 172.21.15.50:0/2224842632 shutdown_connections 2024-09-21T06:11:25.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.515+0000 7f461be00640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4614164d30 0x7f4614165140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.515+0000 7f461be00640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45f4075180 0x7f45f4077690 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.515+0000 7f461be00640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4614147690 0x7f4614146e60 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.519+0000 7f461be00640 1 --2- 172.21.15.50:0/2224842632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45fc0039f0 0x7f46141ec820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.519+0000 7f461be00640 1 -- 172.21.15.50:0/2224842632 >> 172.21.15.50:0/2224842632 conn(0x7f46140b2c40 msgr2=0x7f46140b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:25.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.519+0000 7f461be00640 1 -- 172.21.15.50:0/2224842632 shutdown_connections 2024-09-21T06:11:25.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.519+0000 7f461be00640 1 -- 172.21.15.50:0/2224842632 wait complete. 2024-09-21T06:11:25.537 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:11:25.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.699+0000 7f8746800640 1 -- 172.21.15.50:0/699245733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87400b54f0 msgr2=0x7f874015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.699+0000 7f8746800640 1 --2- 172.21.15.50:0/699245733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87400b54f0 0x7f874015bb90 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f8740074b20 tx=0x7f873801cb70 comp rx=0 tx=0).stop 2024-09-21T06:11:25.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.699+0000 7f8746800640 1 -- 172.21.15.50:0/699245733 shutdown_connections 2024-09-21T06:11:25.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.703+0000 7f8746800640 1 --2- 172.21.15.50:0/699245733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87400b54f0 0x7f874015bb90 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.703+0000 7f8746800640 1 -- 172.21.15.50:0/699245733 >> 172.21.15.50:0/699245733 conn(0x7f87400b2040 msgr2=0x7f87400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:25.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.703+0000 7f8746800640 1 -- 172.21.15.50:0/699245733 shutdown_connections 2024-09-21T06:11:25.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.703+0000 7f8746800640 1 -- 172.21.15.50:0/699245733 wait complete. 2024-09-21T06:11:25.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.703+0000 7f8746800640 1 Processor -- start 2024-09-21T06:11:25.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.703+0000 7f8746800640 1 -- start start 2024-09-21T06:11:25.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.703+0000 7f8746800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87400b5c70 0x7f8740149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:25.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.703+0000 7f8746800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87401527b0 0x7f874015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:25.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.703+0000 7f8746800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8740148b60 0x7f8740158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:25.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.703+0000 7f8746800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8740169af0 con 0x7f87400b5c70 2024-09-21T06:11:25.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.703+0000 7f8746800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8740169970 con 0x7f8740148b60 2024-09-21T06:11:25.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.703+0000 7f8746800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8740169c70 con 0x7f87401527b0 2024-09-21T06:11:25.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f873f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87400b5c70 0x7f8740149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:25.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f873f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87400b5c70 0x7f8740149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33030/0 (socket says 172.21.15.50:33030) 2024-09-21T06:11:25.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f873f400640 1 -- 172.21.15.50:0/4260427097 learned_addr learned my addr 172.21.15.50:0/4260427097 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:25.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f873fe00640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8740148b60 0x7f8740158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:25.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f8736a00640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87401527b0 0x7f874015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:25.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f873f400640 1 -- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87401527b0 msgr2=0x7f874015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f873f400640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87401527b0 0x7f874015a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f873f400640 1 -- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8740148b60 msgr2=0x7f8740158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f873f400640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8740148b60 0x7f8740158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f873f400640 1 -- 172.21.15.50:0/4260427097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874014c5b0 con 0x7f87400b5c70 2024-09-21T06:11:25.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f873f400640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87400b5c70 0x7f8740149040 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f873000c3f0 tx=0x7f873000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:25.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f8737e00640 1 -- 172.21.15.50:0/4260427097 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730017070 con 0x7f87400b5c70 2024-09-21T06:11:25.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f8737e00640 1 -- 172.21.15.50:0/4260427097 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87300053c0 con 0x7f87400b5c70 2024-09-21T06:11:25.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f8737e00640 1 -- 172.21.15.50:0/4260427097 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730003860 con 0x7f87400b5c70 2024-09-21T06:11:25.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f8746800640 1 -- 172.21.15.50:0/4260427097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874014c890 con 0x7f87400b5c70 2024-09-21T06:11:25.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f8746800640 1 -- 172.21.15.50:0/4260427097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f874015a670 con 0x7f87400b5c70 2024-09-21T06:11:25.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f8746800640 1 -- 172.21.15.50:0/4260427097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f874015b1c0 con 0x7f87400b5c70 2024-09-21T06:11:25.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.707+0000 7f8737e00640 1 -- 172.21.15.50:0/4260427097 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 51) ==== 100090+0+0 (secure 0 0 0) 0x7f873001a070 con 0x7f87400b5c70 2024-09-21T06:11:25.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.711+0000 7f8737e00640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f871c075180 0x7f871c077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:25.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.711+0000 7f8737e00640 1 -- 172.21.15.50:0/4260427097 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f8730092580 con 0x7f87400b5c70 2024-09-21T06:11:25.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.711+0000 7f8736a00640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f871c075180 0x7f871c077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:25.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.711+0000 7f8736a00640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f871c075180 0x7f871c077690 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f874014a550 tx=0x7f8728009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:25.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.715+0000 7f8737e00640 1 -- 172.21.15.50:0/4260427097 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f873005ea50 con 0x7f87400b5c70 2024-09-21T06:11:25.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.919+0000 7f8746800640 1 -- 172.21.15.50:0/4260427097 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f874006b540 con 0x7f871c075180 2024-09-21T06:11:25.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.923+0000 7f8737e00640 1 -- 172.21.15.50:0/4260427097 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f8728015430 con 0x7f871c075180 2024-09-21T06:11:25.922 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/7de9ef65-c5ec-4ec4-a5d9-579e05105f30 2024-09-21T06:11:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.927+0000 7f8746800640 1 -- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f871c075180 msgr2=0x7f871c077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.927+0000 7f8746800640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f871c075180 0x7f871c077690 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f874014a550 tx=0x7f8728009040 comp rx=0 tx=0).stop 2024-09-21T06:11:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.927+0000 7f8746800640 1 -- 172.21.15.50:0/4260427097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87400b5c70 msgr2=0x7f8740149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.927+0000 7f8746800640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87400b5c70 0x7f8740149040 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f873000c3f0 tx=0x7f873000c8b0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.927+0000 7f8746800640 1 -- 172.21.15.50:0/4260427097 shutdown_connections 2024-09-21T06:11:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.927+0000 7f8746800640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8740148b60 0x7f8740158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.927+0000 7f8746800640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f87401527b0 0x7f874015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.927+0000 7f8746800640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f871c075180 0x7f871c077690 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.927+0000 7f8746800640 1 --2- 172.21.15.50:0/4260427097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87400b5c70 0x7f8740149040 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.927+0000 7f8746800640 1 -- 172.21.15.50:0/4260427097 >> 172.21.15.50:0/4260427097 conn(0x7f87400b2040 msgr2=0x7f87400cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.927+0000 7f8746800640 1 -- 172.21.15.50:0/4260427097 shutdown_connections 2024-09-21T06:11:25.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:25.927+0000 7f8746800640 1 -- 172.21.15.50:0/4260427097 wait complete. 2024-09-21T06:11:25.945 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/7de9ef65-c5ec-4ec4-a5d9-579e05105f30 2024-09-21T06:11:25.946 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:11:25.946 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:11:25.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:25.955 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/7de9ef65-c5ec-4ec4-a5d9-579e05105f30/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:11:36.032 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:11:36.032 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:11:36.032 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.018 s, 107 MB/s 2024-09-21T06:11:36.033 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:11:36.033 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:11:36.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:36.042 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/7de9ef65-c5ec-4ec4-a5d9-579e05105f30/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:11:45.415 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:11:45.415 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:11:45.415 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.32307 s, 115 MB/s 2024-09-21T06:11:45.416 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:11:45.416 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:11:45.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:45.425 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/7de9ef65-c5ec-4ec4-a5d9-579e05105f30/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:11:48.820 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:11:48.823 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:11:55.140 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:11:55.140 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:11:55.140 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.66371 s, 111 MB/s 2024-09-21T06:11:55.141 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-21T06:11:55.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.307+0000 7fb25a600640 1 -- 172.21.15.50:0/2661469936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2541527b0 msgr2=0x7fb254074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:55.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.307+0000 7fb25a600640 1 --2- 172.21.15.50:0/2661469936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2541527b0 0x7fb254074e30 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb254071cb0 tx=0x7fb24001c890 comp rx=0 tx=0).stop 2024-09-21T06:11:55.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.307+0000 7fb25a600640 1 -- 172.21.15.50:0/2661469936 shutdown_connections 2024-09-21T06:11:55.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.307+0000 7fb25a600640 1 --2- 172.21.15.50:0/2661469936 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2540b4de0 0x7fb2540b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.307+0000 7fb25a600640 1 --2- 172.21.15.50:0/2661469936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2541527b0 0x7fb254074e30 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.307+0000 7fb25a600640 1 --2- 172.21.15.50:0/2661469936 >> v2:172.21.15.88:3300/0 conn(0x7fb2540b5c70 0x7fb25415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.307+0000 7fb25a600640 1 -- 172.21.15.50:0/2661469936 >> 172.21.15.50:0/2661469936 conn(0x7fb2540b2040 msgr2=0x7fb2540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:55.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.307+0000 7fb25a600640 1 -- 172.21.15.50:0/2661469936 shutdown_connections 2024-09-21T06:11:55.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.307+0000 7fb25a600640 1 -- 172.21.15.50:0/2661469936 wait complete. 2024-09-21T06:11:55.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.307+0000 7fb25a600640 1 Processor -- start 2024-09-21T06:11:55.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.307+0000 7fb25a600640 1 -- start start 2024-09-21T06:11:55.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb25a600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2540b4de0 0x7fb2541f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb25a600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2540b5c70 0x7fb25413e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb25a600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2541527b0 0x7fb2541fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb25a600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb25415f5f0 con 0x7fb2540b5c70 2024-09-21T06:11:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb25a600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb25415f470 con 0x7fb2540b4de0 2024-09-21T06:11:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb25a600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb25415f770 con 0x7fb2541527b0 2024-09-21T06:11:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb252a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2540b5c70 0x7fb25413e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb253400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2540b4de0 0x7fb2541f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb252a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2540b5c70 0x7fb25413e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58042/0 (socket says 172.21.15.50:58042) 2024-09-21T06:11:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb252a00640 1 -- 172.21.15.50:0/1986851781 learned_addr learned my addr 172.21.15.50:0/1986851781 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb253e00640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2541527b0 0x7fb2541fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb252a00640 1 -- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2541527b0 msgr2=0x7fb2541fbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:55.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb252a00640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2541527b0 0x7fb2541fbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb252a00640 1 -- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2540b4de0 msgr2=0x7fb2541f92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:55.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb252a00640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2540b4de0 0x7fb2541f92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb252a00640 1 -- 172.21.15.50:0/1986851781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2541f9810 con 0x7fb2540b5c70 2024-09-21T06:11:55.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb253400640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2540b4de0 0x7fb2541f92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:55.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb252a00640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2540b5c70 0x7fb25413e270 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb24001af40 tx=0x7fb2400040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:55.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb24b400640 1 -- 172.21.15.50:0/1986851781 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2400b2040 con 0x7fb2540b5c70 2024-09-21T06:11:55.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb24b400640 1 -- 172.21.15.50:0/1986851781 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb240003600 con 0x7fb2540b5c70 2024-09-21T06:11:55.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb24b400640 1 -- 172.21.15.50:0/1986851781 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240003a10 con 0x7fb2540b5c70 2024-09-21T06:11:55.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb25a600640 1 -- 172.21.15.50:0/1986851781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb254154090 con 0x7fb2540b5c70 2024-09-21T06:11:55.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb25a600640 1 -- 172.21.15.50:0/1986851781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2540ce980 con 0x7fb2540b5c70 2024-09-21T06:11:55.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.311+0000 7fb25a600640 1 -- 172.21.15.50:0/1986851781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2540ccd10 con 0x7fb2540b5c70 2024-09-21T06:11:55.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.315+0000 7fb24b400640 1 -- 172.21.15.50:0/1986851781 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 52) ==== 100192+0+0 (secure 0 0 0) 0x7fb2400bf060 con 0x7fb2540b5c70 2024-09-21T06:11:55.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.315+0000 7fb24b400640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb2340752d0 0x7fb2340777e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:55.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.315+0000 7fb24b400640 1 -- 172.21.15.50:0/1986851781 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fb24012cee0 con 0x7fb2540b5c70 2024-09-21T06:11:55.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.315+0000 7fb253400640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb2340752d0 0x7fb2340777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:55.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.315+0000 7fb253400640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb2340752d0 0x7fb2340777e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb24400af10 tx=0x7fb24400a250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:55.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.319+0000 7fb24b400640 1 -- 172.21.15.50:0/1986851781 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb2541f9810 con 0x7fb2540b5c70 2024-09-21T06:11:55.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.531+0000 7fb25a600640 1 -- 172.21.15.50:0/1986851781 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fb2540ccae0 con 0x7fb2340752d0 2024-09-21T06:11:55.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.543+0000 7fb24b400640 1 -- 172.21.15.50:0/1986851781 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb24400ab60 con 0x7fb2340752d0 2024-09-21T06:11:55.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.547+0000 7fb25a600640 1 -- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb2340752d0 msgr2=0x7fb2340777e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:55.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.547+0000 7fb25a600640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb2340752d0 0x7fb2340777e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb24400af10 tx=0x7fb24400a250 comp rx=0 tx=0).stop 2024-09-21T06:11:55.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.547+0000 7fb25a600640 1 -- 172.21.15.50:0/1986851781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2540b5c70 msgr2=0x7fb25413e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:55.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.547+0000 7fb25a600640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2540b5c70 0x7fb25413e270 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb24001af40 tx=0x7fb2400040c0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.547+0000 7fb25a600640 1 -- 172.21.15.50:0/1986851781 shutdown_connections 2024-09-21T06:11:55.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.547+0000 7fb25a600640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb2541527b0 0x7fb2541fbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.547+0000 7fb25a600640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb2340752d0 0x7fb2340777e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.547+0000 7fb25a600640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2540b5c70 0x7fb25413e270 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.547+0000 7fb25a600640 1 --2- 172.21.15.50:0/1986851781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2540b4de0 0x7fb2541f92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.547+0000 7fb25a600640 1 -- 172.21.15.50:0/1986851781 >> 172.21.15.50:0/1986851781 conn(0x7fb2540b2040 msgr2=0x7fb2540af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:55.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.547+0000 7fb25a600640 1 -- 172.21.15.50:0/1986851781 shutdown_connections 2024-09-21T06:11:55.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.547+0000 7fb25a600640 1 -- 172.21.15.50:0/1986851781 wait complete. 2024-09-21T06:11:55.565 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:11:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.731+0000 7ff5dee00640 1 -- 172.21.15.50:0/1408296000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5d80b5c70 msgr2=0x7ff5d815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.731+0000 7ff5dee00640 1 --2- 172.21.15.50:0/1408296000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5d80b5c70 0x7ff5d815bb60 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009840 tx=0x7ff5c801c7f0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.731+0000 7ff5dee00640 1 -- 172.21.15.50:0/1408296000 shutdown_connections 2024-09-21T06:11:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.731+0000 7ff5dee00640 1 --2- 172.21.15.50:0/1408296000 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff5d80b4de0 0x7ff5d80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.731+0000 7ff5dee00640 1 --2- 172.21.15.50:0/1408296000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5d80b5c70 0x7ff5d815bb60 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.731+0000 7ff5dee00640 1 -- 172.21.15.50:0/1408296000 >> 172.21.15.50:0/1408296000 conn(0x7ff5d80b2040 msgr2=0x7ff5d80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.731+0000 7ff5dee00640 1 -- 172.21.15.50:0/1408296000 shutdown_connections 2024-09-21T06:11:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.731+0000 7ff5dee00640 1 -- 172.21.15.50:0/1408296000 wait complete. 2024-09-21T06:11:55.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.731+0000 7ff5dee00640 1 Processor -- start 2024-09-21T06:11:55.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.731+0000 7ff5dee00640 1 -- start start 2024-09-21T06:11:55.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5dee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5d80b4de0 0x7ff5d80cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:55.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5dee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5d80b5c70 0x7ff5d8150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:55.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5dee00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff5d81527b0 0x7ff5d80cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:55.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5dee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d8168970 con 0x7ff5d80b4de0 2024-09-21T06:11:55.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5dee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d81687f0 con 0x7ff5d80b5c70 2024-09-21T06:11:55.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5dee00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d8168af0 con 0x7ff5d81527b0 2024-09-21T06:11:55.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5d7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5d80b4de0 0x7ff5d80cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:55.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5d7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5d80b4de0 0x7ff5d80cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58066/0 (socket says 172.21.15.50:58066) 2024-09-21T06:11:55.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5d7e00640 1 -- 172.21.15.50:0/151120993 learned_addr learned my addr 172.21.15.50:0/151120993 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:55.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5dce00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff5d81527b0 0x7ff5d80cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:55.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5cfe00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5d80b5c70 0x7ff5d8150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:55.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5d7e00640 1 -- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff5d81527b0 msgr2=0x7ff5d80cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:55.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5d7e00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff5d81527b0 0x7ff5d80cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5d7e00640 1 -- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5d80b5c70 msgr2=0x7ff5d8150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:55.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5d7e00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5d80b5c70 0x7ff5d8150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5d7e00640 1 -- 172.21.15.50:0/151120993 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d80ad920 con 0x7ff5d80b4de0 2024-09-21T06:11:55.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5d7e00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5d80b4de0 0x7ff5d80cc330 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8000c10 tx=0x7ff5c8005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:55.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5d4c00640 1 -- 172.21.15.50:0/151120993 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80bb070 con 0x7ff5d80b4de0 2024-09-21T06:11:55.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5d4c00640 1 -- 172.21.15.50:0/151120993 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5c8004330 con 0x7ff5d80b4de0 2024-09-21T06:11:55.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5d4c00640 1 -- 172.21.15.50:0/151120993 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80b2040 con 0x7ff5d80b4de0 2024-09-21T06:11:55.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5dee00640 1 -- 172.21.15.50:0/151120993 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d80af2f0 con 0x7ff5d80b4de0 2024-09-21T06:11:55.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5dee00640 1 -- 172.21.15.50:0/151120993 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d80adc00 con 0x7ff5d80b4de0 2024-09-21T06:11:55.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.735+0000 7ff5dee00640 1 -- 172.21.15.50:0/151120993 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d80af0e0 con 0x7ff5d80b4de0 2024-09-21T06:11:55.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.739+0000 7ff5d4c00640 1 -- 172.21.15.50:0/151120993 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 52) ==== 100192+0+0 (secure 0 0 0) 0x7ff5c8003800 con 0x7ff5d80b4de0 2024-09-21T06:11:55.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.739+0000 7ff5d4c00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5b40752d0 0x7ff5b40777e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:55.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.739+0000 7ff5d4c00640 1 -- 172.21.15.50:0/151120993 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7ff5c812cbb0 con 0x7ff5d80b4de0 2024-09-21T06:11:55.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.739+0000 7ff5cfe00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5b40752d0 0x7ff5b40777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:55.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.739+0000 7ff5cfe00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5b40752d0 0x7ff5b40777e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80b0980 tx=0x7ff5c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:55.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.743+0000 7ff5d4c00640 1 -- 172.21.15.50:0/151120993 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff5c8016410 con 0x7ff5d80b4de0 2024-09-21T06:11:55.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.955+0000 7ff5dee00640 1 -- 172.21.15.50:0/151120993 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7ff5d806b540 con 0x7ff5b40752d0 2024-09-21T06:11:55.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.959+0000 7ff5d4c00640 1 -- 172.21.15.50:0/151120993 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7ff5c0015430 con 0x7ff5b40752d0 2024-09-21T06:11:55.959 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/7de9ef65-c5ec-4ec4-a5d9-579e05105f30 2024-09-21T06:11:55.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.963+0000 7ff5dee00640 1 -- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5b40752d0 msgr2=0x7ff5b40777e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:55.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.963+0000 7ff5dee00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5b40752d0 0x7ff5b40777e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80b0980 tx=0x7ff5c0009040 comp rx=0 tx=0).stop 2024-09-21T06:11:55.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.963+0000 7ff5dee00640 1 -- 172.21.15.50:0/151120993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5d80b4de0 msgr2=0x7ff5d80cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:55.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.963+0000 7ff5dee00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5d80b4de0 0x7ff5d80cc330 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8000c10 tx=0x7ff5c8005db0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.963+0000 7ff5dee00640 1 -- 172.21.15.50:0/151120993 shutdown_connections 2024-09-21T06:11:55.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.963+0000 7ff5dee00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff5d81527b0 0x7ff5d80cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.963+0000 7ff5dee00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5d80b5c70 0x7ff5d8150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.963+0000 7ff5dee00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff5b40752d0 0x7ff5b40777e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.963+0000 7ff5dee00640 1 --2- 172.21.15.50:0/151120993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5d80b4de0 0x7ff5d80cc330 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:55.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.963+0000 7ff5dee00640 1 -- 172.21.15.50:0/151120993 >> 172.21.15.50:0/151120993 conn(0x7ff5d80b2040 msgr2=0x7ff5d81581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:55.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.963+0000 7ff5dee00640 1 -- 172.21.15.50:0/151120993 shutdown_connections 2024-09-21T06:11:55.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:55.963+0000 7ff5dee00640 1 -- 172.21.15.50:0/151120993 wait complete. 2024-09-21T06:11:55.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:55.981 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/7de9ef65-c5ec-4ec4-a5d9-579e05105f30 2024-09-21T06:11:55.981 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:11:55.990 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/7de9ef65-c5ec-4ec4-a5d9-579e05105f30 2024-09-21T06:11:55.990 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="3221225472" 2024-09-21T06:11:55.990 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:11:55.991 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-21T06:11:55.991 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-21T06:11:56.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.207+0000 7fb74d200640 1 -- 172.21.15.50:0/2850683019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb748160470 msgr2=0x7fb748074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:56.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.207+0000 7fb74d200640 1 --2- 172.21.15.50:0/2850683019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb748160470 0x7fb748074780 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fb7480ced40 tx=0x7fb74001caf0 comp rx=0 tx=0).stop 2024-09-21T06:11:56.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.207+0000 7fb74d200640 1 -- 172.21.15.50:0/2850683019 shutdown_connections 2024-09-21T06:11:56.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.207+0000 7fb74d200640 1 --2- 172.21.15.50:0/2850683019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb748160470 0x7fb748074780 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:56.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.207+0000 7fb74d200640 1 --2- 172.21.15.50:0/2850683019 >> v2:172.21.15.88:3300/0 conn(0x7fb748160060 0x7fb748074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:56.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.207+0000 7fb74d200640 1 -- 172.21.15.50:0/2850683019 >> 172.21.15.50:0/2850683019 conn(0x7fb7480b4560 msgr2=0x7fb7480d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:56.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.207+0000 7fb74d200640 1 -- 172.21.15.50:0/2850683019 shutdown_connections 2024-09-21T06:11:56.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.207+0000 7fb74d200640 1 -- 172.21.15.50:0/2850683019 wait complete. 2024-09-21T06:11:56.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.207+0000 7fb74d200640 1 Processor -- start 2024-09-21T06:11:56.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.207+0000 7fb74d200640 1 -- start start 2024-09-21T06:11:56.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb74d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7481536f0 0x7fb748149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:56.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb74d200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb748160060 0x7fb7480d1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:56.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb74d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb748160470 0x7fb7481f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:56.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb74d200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb7481696e0 con 0x7fb7481536f0 2024-09-21T06:11:56.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb74d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb748169560 con 0x7fb748160470 2024-09-21T06:11:56.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb74d200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb748169860 con 0x7fb748160060 2024-09-21T06:11:56.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb747400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7481536f0 0x7fb748149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:56.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb747400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7481536f0 0x7fb748149ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58078/0 (socket says 172.21.15.50:58078) 2024-09-21T06:11:56.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb747400640 1 -- 172.21.15.50:0/3250802974 learned_addr learned my addr 172.21.15.50:0/3250802974 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:56.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb747400640 1 -- 172.21.15.50:0/3250802974 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb748160060 msgr2=0x7fb7480d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:56.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb747400640 1 --2- 172.21.15.50:0/3250802974 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb748160060 0x7fb7480d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:56.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb747400640 1 -- 172.21.15.50:0/3250802974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb748160470 msgr2=0x7fb7481f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:56.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb747400640 1 --2- 172.21.15.50:0/3250802974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb748160470 0x7fb7481f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:56.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb747400640 1 -- 172.21.15.50:0/3250802974 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7481f8310 con 0x7fb7481536f0 2024-09-21T06:11:56.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb747400640 1 --2- 172.21.15.50:0/3250802974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7481536f0 0x7fb748149ad0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fb73800c400 tx=0x7fb73800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:56.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb73fe00640 1 -- 172.21.15.50:0/3250802974 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738017070 con 0x7fb7481536f0 2024-09-21T06:11:56.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb73fe00640 1 -- 172.21.15.50:0/3250802974 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb738003600 con 0x7fb7481536f0 2024-09-21T06:11:56.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb73fe00640 1 -- 172.21.15.50:0/3250802974 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738003a50 con 0x7fb7481536f0 2024-09-21T06:11:56.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb74d200640 1 -- 172.21.15.50:0/3250802974 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74814a010 con 0x7fb7481536f0 2024-09-21T06:11:56.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb74d200640 1 -- 172.21.15.50:0/3250802974 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb748152b80 con 0x7fb7481536f0 2024-09-21T06:11:56.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb74d200640 1 -- 172.21.15.50:0/3250802974 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb74815d480 con 0x7fb7481536f0 2024-09-21T06:11:56.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.211+0000 7fb73fe00640 1 -- 172.21.15.50:0/3250802974 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 52) ==== 100192+0+0 (secure 0 0 0) 0x7fb73801a070 con 0x7fb7481536f0 2024-09-21T06:11:56.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.215+0000 7fb73fe00640 1 --2- 172.21.15.50:0/3250802974 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7240752d0 0x7fb7240777e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:56.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.215+0000 7fb73fe00640 1 -- 172.21.15.50:0/3250802974 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fb738092660 con 0x7fb7481536f0 2024-09-21T06:11:56.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.215+0000 7fb73ea00640 1 --2- 172.21.15.50:0/3250802974 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7240752d0 0x7fb7240777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:56.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.215+0000 7fb73ea00640 1 --2- 172.21.15.50:0/3250802974 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7240752d0 0x7fb7240777e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb7300058d0 tx=0x7fb730005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:56.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.219+0000 7fb73fe00640 1 -- 172.21.15.50:0/3250802974 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb73805f0e0 con 0x7fb7481536f0 2024-09-21T06:11:56.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.427+0000 7fb74d200640 1 -- 172.21.15.50:0/3250802974 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fb74806b540 con 0x7fb7240752d0 2024-09-21T06:11:56.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.495+0000 7fb73fe00640 1 -- 172.21.15.50:0/3250802974 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb73000e9e0 con 0x7fb7240752d0 2024-09-21T06:11:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.499+0000 7fb74d200640 1 -- 172.21.15.50:0/3250802974 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7240752d0 msgr2=0x7fb7240777e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.499+0000 7fb74d200640 1 --2- 172.21.15.50:0/3250802974 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7240752d0 0x7fb7240777e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb7300058d0 tx=0x7fb730005860 comp rx=0 tx=0).stop 2024-09-21T06:11:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.499+0000 7fb74d200640 1 -- 172.21.15.50:0/3250802974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7481536f0 msgr2=0x7fb748149ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.499+0000 7fb74d200640 1 --2- 172.21.15.50:0/3250802974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7481536f0 0x7fb748149ad0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fb73800c400 tx=0x7fb73800c8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:56.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.499+0000 7fb74d200640 1 -- 172.21.15.50:0/3250802974 shutdown_connections 2024-09-21T06:11:56.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.499+0000 7fb74d200640 1 --2- 172.21.15.50:0/3250802974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb748160470 0x7fb7481f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:56.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.499+0000 7fb74d200640 1 --2- 172.21.15.50:0/3250802974 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb748160060 0x7fb7480d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:56.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.499+0000 7fb74d200640 1 --2- 172.21.15.50:0/3250802974 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb7240752d0 0x7fb7240777e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:56.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.499+0000 7fb74d200640 1 --2- 172.21.15.50:0/3250802974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7481536f0 0x7fb748149ad0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:56.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.499+0000 7fb74d200640 1 -- 172.21.15.50:0/3250802974 >> 172.21.15.50:0/3250802974 conn(0x7fb7480b4560 msgr2=0x7fb7480b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:56.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.499+0000 7fb74d200640 1 -- 172.21.15.50:0/3250802974 shutdown_connections 2024-09-21T06:11:56.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:56.499+0000 7fb74d200640 1 -- 172.21.15.50:0/3250802974 wait complete. 2024-09-21T06:11:56.502 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:56.504+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:56.502 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:56.504+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:56.502 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:56.504+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:56.502 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:56.504+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:56.502 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:56.504+0000 7f65a7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:56.520 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:56.520+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:56.521 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:56.520+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:56.521 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:56.520+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:56.521 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:56.520+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:56.521 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:56.520+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:57.505 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:57.508+0000 7f6524600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:57.505 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:57.508+0000 7f6524600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:57.505 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:57.508+0000 7f6524600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:57.505 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:57.508+0000 7f6524600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:57.506 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:11:57.508+0000 7f6524600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:11:57.519 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-21T06:11:57.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.747+0000 7f6c6de00640 1 -- 172.21.15.50:0/1121731945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c4c0053b0 msgr2=0x7f6c4c00dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:57.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.747+0000 7f6c6de00640 1 --2- 172.21.15.50:0/1121731945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c4c0053b0 0x7f6c4c00dc40 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f6c68071ec0 tx=0x7f6c6001d260 comp rx=0 tx=0).stop 2024-09-21T06:11:57.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.747+0000 7f6c6de00640 1 -- 172.21.15.50:0/1121731945 shutdown_connections 2024-09-21T06:11:57.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.747+0000 7f6c6de00640 1 --2- 172.21.15.50:0/1121731945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c4c0053b0 0x7f6c4c00dc40 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:57.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.747+0000 7f6c6de00640 1 -- 172.21.15.50:0/1121731945 >> 172.21.15.50:0/1121731945 conn(0x7f6c680b35f0 msgr2=0x7f6c680b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:57.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.747+0000 7f6c6de00640 1 -- 172.21.15.50:0/1121731945 shutdown_connections 2024-09-21T06:11:57.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.747+0000 7f6c6de00640 1 -- 172.21.15.50:0/1121731945 wait complete. 2024-09-21T06:11:57.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.751+0000 7f6c6de00640 1 Processor -- start 2024-09-21T06:11:57.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.751+0000 7f6c6de00640 1 -- start start 2024-09-21T06:11:57.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c6de00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6c4c0053b0 0x7f6c68152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:57.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c6de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c681552e0 0x7f6c68152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:57.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c6de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c680d0370 0x7f6c680d0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:57.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c6de00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6806d5e0 con 0x7f6c680d0370 2024-09-21T06:11:57.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c6de00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6806d460 con 0x7f6c681552e0 2024-09-21T06:11:57.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c6de00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6c6806fc90 con 0x7f6c4c0053b0 2024-09-21T06:11:57.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c66a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6c4c0053b0 0x7f6c68152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:57.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c66a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6c4c0053b0 0x7f6c68152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:39960/0 (socket says 172.21.15.50:39960) 2024-09-21T06:11:57.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c66a00640 1 -- 172.21.15.50:0/969310396 learned_addr learned my addr 172.21.15.50:0/969310396 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:57.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c67400640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c680d0370 0x7f6c680d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:57.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c66000640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c681552e0 0x7f6c68152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:57.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c66a00640 1 -- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c681552e0 msgr2=0x7f6c68152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:57.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c66a00640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c681552e0 0x7f6c68152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:57.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c66a00640 1 -- 172.21.15.50:0/969310396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c680d0370 msgr2=0x7f6c680d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:57.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c66a00640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c680d0370 0x7f6c680d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:57.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c66a00640 1 -- 172.21.15.50:0/969310396 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c68160f50 con 0x7f6c4c0053b0 2024-09-21T06:11:57.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c66a00640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6c4c0053b0 0x7f6c68152d70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6c58010a40 tx=0x7f6c58010f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:57.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c5f400640 1 -- 172.21.15.50:0/969310396 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800eda0 con 0x7f6c4c0053b0 2024-09-21T06:11:57.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c5f400640 1 -- 172.21.15.50:0/969310396 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c58004030 con 0x7f6c4c0053b0 2024-09-21T06:11:57.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c5f400640 1 -- 172.21.15.50:0/969310396 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58004480 con 0x7f6c4c0053b0 2024-09-21T06:11:57.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.755+0000 7f6c6de00640 1 -- 172.21.15.50:0/969310396 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c680cf5b0 con 0x7f6c4c0053b0 2024-09-21T06:11:57.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.759+0000 7f6c6de00640 1 -- 172.21.15.50:0/969310396 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c680cfb70 con 0x7f6c4c0053b0 2024-09-21T06:11:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.759+0000 7f6c5cc00640 1 -- 172.21.15.50:0/969310396 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c68069600 con 0x7f6c4c0053b0 2024-09-21T06:11:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.759+0000 7f6c5f400640 1 -- 172.21.15.50:0/969310396 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 52) ==== 100192+0+0 (secure 0 0 0) 0x7f6c5803d820 con 0x7f6c4c0053b0 2024-09-21T06:11:57.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.763+0000 7f6c5f400640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6c480752a0 0x7f6c480777b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:57.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.763+0000 7f6c5f400640 1 -- 172.21.15.50:0/969310396 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f6c58094300 con 0x7f6c4c0053b0 2024-09-21T06:11:57.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.767+0000 7f6c66000640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6c480752a0 0x7f6c480777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:57.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.767+0000 7f6c5f400640 1 -- 172.21.15.50:0/969310396 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6c58060e90 con 0x7f6c4c0053b0 2024-09-21T06:11:57.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:57.771+0000 7f6c66000640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6c480752a0 0x7f6c480777b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6c68166390 tx=0x7f6c540060e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:58.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.147+0000 7f6c5cc00640 1 -- 172.21.15.50:0/969310396 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f6c6806b540 con 0x7f6c480752a0 2024-09-21T06:11:58.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.291+0000 7f6c5f400640 1 -- 172.21.15.50:0/969310396 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6c540062d0 con 0x7f6c480752a0 2024-09-21T06:11:58.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.295+0000 7f6c5cc00640 1 -- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6c480752a0 msgr2=0x7f6c480777b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:58.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.295+0000 7f6c5cc00640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6c480752a0 0x7f6c480777b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6c68166390 tx=0x7f6c540060e0 comp rx=0 tx=0).stop 2024-09-21T06:11:58.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.295+0000 7f6c5cc00640 1 -- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6c4c0053b0 msgr2=0x7f6c68152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:58.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.295+0000 7f6c5cc00640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6c4c0053b0 0x7f6c68152d70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6c58010a40 tx=0x7f6c58010f00 comp rx=0 tx=0).stop 2024-09-21T06:11:58.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.295+0000 7f6c5cc00640 1 -- 172.21.15.50:0/969310396 shutdown_connections 2024-09-21T06:11:58.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.295+0000 7f6c5cc00640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6c480752a0 0x7f6c480777b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:58.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.295+0000 7f6c5cc00640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c680d0370 0x7f6c680d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:58.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.295+0000 7f6c5cc00640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c681552e0 0x7f6c68152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:58.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.295+0000 7f6c5cc00640 1 --2- 172.21.15.50:0/969310396 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6c4c0053b0 0x7f6c68152d70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:58.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.295+0000 7f6c5cc00640 1 -- 172.21.15.50:0/969310396 >> 172.21.15.50:0/969310396 conn(0x7f6c680b35f0 msgr2=0x7f6c680b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:58.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.295+0000 7f6c5cc00640 1 -- 172.21.15.50:0/969310396 shutdown_connections 2024-09-21T06:11:58.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.295+0000 7f6c5cc00640 1 -- 172.21.15.50:0/969310396 wait complete. 2024-09-21T06:11:58.312 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-21T06:11:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.479+0000 7fefff200640 1 -- 172.21.15.50:0/886934733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff80744d0 msgr2=0x7feff806ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.479+0000 7fefff200640 1 --2- 172.21.15.50:0/886934733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff80744d0 0x7feff806ea60 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009840 tx=0x7fefe401c710 comp rx=0 tx=0).stop 2024-09-21T06:11:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.479+0000 7fefff200640 1 -- 172.21.15.50:0/886934733 shutdown_connections 2024-09-21T06:11:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.479+0000 7fefff200640 1 --2- 172.21.15.50:0/886934733 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff806efa0 0x7feff806f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.479+0000 7fefff200640 1 --2- 172.21.15.50:0/886934733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff80744d0 0x7feff806ea60 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.479+0000 7fefff200640 1 -- 172.21.15.50:0/886934733 >> 172.21.15.50:0/886934733 conn(0x7feff80b2c40 msgr2=0x7feff80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.479+0000 7fefff200640 1 -- 172.21.15.50:0/886934733 shutdown_connections 2024-09-21T06:11:58.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.479+0000 7fefff200640 1 -- 172.21.15.50:0/886934733 wait complete. 2024-09-21T06:11:58.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefff200640 1 Processor -- start 2024-09-21T06:11:58.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefff200640 1 -- start start 2024-09-21T06:11:58.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefff200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff806efa0 0x7feff80ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:58.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefff200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff815c5d0 0x7feff81511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:58.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefff200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff80cf1f0 0x7feff80cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:58.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefff200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feff8168c90 con 0x7feff806efa0 2024-09-21T06:11:58.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefff200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feff8168b10 con 0x7feff815c5d0 2024-09-21T06:11:58.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefff200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7feff8168e10 con 0x7feff80cf1f0 2024-09-21T06:11:58.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7feff7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff806efa0 0x7feff80ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:58.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7feff7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff806efa0 0x7feff80ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43260/0 (socket says 172.21.15.50:43260) 2024-09-21T06:11:58.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7feff7e00640 1 -- 172.21.15.50:0/1871664245 learned_addr learned my addr 172.21.15.50:0/1871664245 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7feffd200640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff80cf1f0 0x7feff80cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7feff7400640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff815c5d0 0x7feff81511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7feff7e00640 1 -- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff80cf1f0 msgr2=0x7feff80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7feff7e00640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff80cf1f0 0x7feff80cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7feff7e00640 1 -- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff815c5d0 msgr2=0x7feff81511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7feff7e00640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff815c5d0 0x7feff81511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7feff7e00640 1 -- 172.21.15.50:0/1871664245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff80afc00 con 0x7feff806efa0 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7feff7e00640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff806efa0 0x7feff80ae1d0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fefe800c500 tx=0x7fefe800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefefe00640 1 -- 172.21.15.50:0/1871664245 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8016020 con 0x7feff806efa0 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefefe00640 1 -- 172.21.15.50:0/1871664245 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefe8012940 con 0x7feff806efa0 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefefe00640 1 -- 172.21.15.50:0/1871664245 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8012d90 con 0x7feff806efa0 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefff200640 1 -- 172.21.15.50:0/1871664245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff80afee0 con 0x7feff806efa0 2024-09-21T06:11:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefff200640 1 -- 172.21.15.50:0/1871664245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff80cdc90 con 0x7feff806efa0 2024-09-21T06:11:58.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.483+0000 7fefff200640 1 -- 172.21.15.50:0/1871664245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff80b0400 con 0x7feff806efa0 2024-09-21T06:11:58.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.487+0000 7fefefe00640 1 -- 172.21.15.50:0/1871664245 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 53) ==== 100294+0+0 (secure 0 0 0) 0x7fefe8019020 con 0x7feff806efa0 2024-09-21T06:11:58.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.487+0000 7fefefe00640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fefd80753c0 0x7fefd80778d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:58.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.487+0000 7fefefe00640 1 -- 172.21.15.50:0/1871664245 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fefe8093400 con 0x7feff806efa0 2024-09-21T06:11:58.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.487+0000 7feff7400640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fefd80753c0 0x7fefd80778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:58.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.487+0000 7feff7400640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fefd80753c0 0x7fefd80778d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fefe40af000 tx=0x7fefe401af50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:58.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.491+0000 7fefefe00640 1 -- 172.21.15.50:0/1871664245 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7feff80b0400 con 0x7feff806efa0 2024-09-21T06:11:58.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:11:58.707+0000 7fefff200640 1 -- 172.21.15.50:0/1871664245 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7feff806b540 con 0x7fefd80753c0 2024-09-21T06:12:00.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.123+0000 7fefefe00640 1 -- 172.21.15.50:0/1871664245 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 54) ==== 100345+0+0 (secure 0 0 0) 0x7fefe805b9f0 con 0x7feff806efa0 2024-09-21T06:12:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefefe00640 1 -- 172.21.15.50:0/1871664245 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fefe40040c0 con 0x7fefd80753c0 2024-09-21T06:12:00.660 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:12:00.660 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:12:00.660 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "canceled", 2024-09-21T06:12:00.660 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:12:00.660 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:12:00.660 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:12:00.661 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:12:00.661 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:00.661 INFO:teuthology.orchestra.run.smithi050.stdout: "failure": { 2024-09-21T06:12:00.661 INFO:teuthology.orchestra.run.smithi050.stdout: "errno": "4", 2024-09-21T06:12:00.661 INFO:teuthology.orchestra.run.smithi050.stdout: "error_msg": "user interrupted clone operation" 2024-09-21T06:12:00.661 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:00.661 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:00.662 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:12:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefff200640 1 -- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fefd80753c0 msgr2=0x7fefd80778d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefff200640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fefd80753c0 0x7fefd80778d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fefe40af000 tx=0x7fefe401af50 comp rx=0 tx=0).stop 2024-09-21T06:12:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefff200640 1 -- 172.21.15.50:0/1871664245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff806efa0 msgr2=0x7feff80ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefff200640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff806efa0 0x7feff80ae1d0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fefe800c500 tx=0x7fefe800c900 comp rx=0 tx=0).stop 2024-09-21T06:12:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefff200640 1 -- 172.21.15.50:0/1871664245 shutdown_connections 2024-09-21T06:12:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefff200640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7feff80cf1f0 0x7feff80cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefff200640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff815c5d0 0x7feff81511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefff200640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fefd80753c0 0x7fefd80778d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefff200640 1 --2- 172.21.15.50:0/1871664245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff806efa0 0x7feff80ae1d0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefff200640 1 -- 172.21.15.50:0/1871664245 >> 172.21.15.50:0/1871664245 conn(0x7feff80b2c40 msgr2=0x7feff8075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefff200640 1 -- 172.21.15.50:0/1871664245 shutdown_connections 2024-09-21T06:12:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.663+0000 7fefff200640 1 -- 172.21.15.50:0/1871664245 wait complete. 2024-09-21T06:12:00.682 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-21T06:12:00.683 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:12:00.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.843+0000 7f294ec00640 1 -- 172.21.15.50:0/2769158265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2948159130 msgr2=0x7f294815bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:00.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.843+0000 7f294ec00640 1 --2- 172.21.15.50:0/2769158265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2948159130 0x7f294815bb90 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f2938009900 tx=0x7f293801c7c0 comp rx=0 tx=0).stop 2024-09-21T06:12:00.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.843+0000 7f294ec00640 1 -- 172.21.15.50:0/2769158265 shutdown_connections 2024-09-21T06:12:00.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.843+0000 7f294ec00640 1 --2- 172.21.15.50:0/2769158265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2948159130 0x7f294815bb90 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:00.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.843+0000 7f294ec00640 1 -- 172.21.15.50:0/2769158265 >> 172.21.15.50:0/2769158265 conn(0x7f29480b35f0 msgr2=0x7f29480b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:00.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.843+0000 7f294ec00640 1 -- 172.21.15.50:0/2769158265 shutdown_connections 2024-09-21T06:12:00.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.843+0000 7f294ec00640 1 -- 172.21.15.50:0/2769158265 wait complete. 2024-09-21T06:12:00.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294ec00640 1 Processor -- start 2024-09-21T06:12:00.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294ec00640 1 -- start start 2024-09-21T06:12:00.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294ec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f294806cdf0 0x7f2948153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:00.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294ec00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f29480703a0 0x7f29480b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:00.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294ec00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2948159130 0x7f29480b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:00.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294ec00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2948169d40 con 0x7f2948159130 2024-09-21T06:12:00.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294ec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f294806d580 con 0x7f294806cdf0 2024-09-21T06:12:00.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294ec00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2948169ec0 con 0x7f29480703a0 2024-09-21T06:12:00.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2948159130 0x7f29480b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:00.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2948159130 0x7f29480b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43284/0 (socket says 172.21.15.50:43284) 2024-09-21T06:12:00.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294cc00640 1 -- 172.21.15.50:0/3174648664 learned_addr learned my addr 172.21.15.50:0/3174648664 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:00.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f2947e00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f294806cdf0 0x7f2948153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:00.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f293f400640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f29480703a0 0x7f29480b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:00.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294cc00640 1 -- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f29480703a0 msgr2=0x7f29480b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:00.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294cc00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f29480703a0 0x7f29480b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:00.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294cc00640 1 -- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f294806cdf0 msgr2=0x7f2948153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:00.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294cc00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f294806cdf0 0x7f2948153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:00.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294cc00640 1 -- 172.21.15.50:0/3174648664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2948159a90 con 0x7f2948159130 2024-09-21T06:12:00.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f293f400640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f29480703a0 0x7f29480b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:00.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294cc00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2948159130 0x7f29480b2bd0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f294000c760 tx=0x7f294000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:00.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f2944c00640 1 -- 172.21.15.50:0/3174648664 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940003b80 con 0x7f2948159130 2024-09-21T06:12:00.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f2944c00640 1 -- 172.21.15.50:0/3174648664 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2940003d20 con 0x7f2948159130 2024-09-21T06:12:00.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f2944c00640 1 -- 172.21.15.50:0/3174648664 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294000aab0 con 0x7f2948159130 2024-09-21T06:12:00.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294ec00640 1 -- 172.21.15.50:0/3174648664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294815b750 con 0x7f2948159130 2024-09-21T06:12:00.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.847+0000 7f294ec00640 1 -- 172.21.15.50:0/3174648664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29480d0540 con 0x7f2948159130 2024-09-21T06:12:00.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.851+0000 7f294ec00640 1 -- 172.21.15.50:0/3174648664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294815b260 con 0x7f2948159130 2024-09-21T06:12:00.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.851+0000 7f2944c00640 1 -- 172.21.15.50:0/3174648664 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 54) ==== 100345+0+0 (secure 0 0 0) 0x7f294000ac50 con 0x7f2948159130 2024-09-21T06:12:00.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.851+0000 7f2944c00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2924075480 0x7f2924077990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:00.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.851+0000 7f2944c00640 1 -- 172.21.15.50:0/3174648664 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f2940093100 con 0x7f2948159130 2024-09-21T06:12:00.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.851+0000 7f2947e00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2924075480 0x7f2924077990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:00.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.851+0000 7f2947e00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2924075480 0x7f2924077990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f29380af000 tx=0x7f293801b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:00.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:00.855+0000 7f2944c00640 1 -- 172.21.15.50:0/3174648664 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f294005fc10 con 0x7f2948159130 2024-09-21T06:12:01.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 -- 172.21.15.50:0/3174648664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2948159800 con 0x7f2948159130 2024-09-21T06:12:01.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f2944c00640 1 -- 172.21.15.50:0/3174648664 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1917 (secure 0 0 0) 0x7f294005f510 con 0x7f2948159130 2024-09-21T06:12:01.096 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:01.096 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:12:01.096 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:12:01.096 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:12:01.096 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:12:01.096 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:12:01.097 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:12:01.097 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:01.097 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:12:01.097 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:12:01.097 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:12:01.097 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:12:01.097 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:12:01.097 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:12:01.097 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:12:01.097 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:12:01.098 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:12:01.098 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:12:01.098 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:12:01.098 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 957, 2024-09-21T06:12:01.098 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:12:01.098 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:12:01.098 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:12:01.098 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:12:01.098 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:01.098 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:12:01.099 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 127, 2024-09-21T06:12:01.099 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:12:01.099 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:12:01.099 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:12:01.099 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:12:01.099 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:12:01.099 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:12:01.099 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:01.099 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:12:01.099 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:12:01.100 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:01.100 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:12:01.100 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:12:01.100 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:01.100 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:12:01.100 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:12:01.100 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:12:01.100 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 864, 2024-09-21T06:12:01.100 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 3490564848, 2024-09-21T06:12:01.100 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 7653568512, 2024-09-21T06:12:01.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 765440544768, 2024-09-21T06:12:01.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:12:01.101 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 25158966, 2024-09-21T06:12:01.101 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 159005739, 2024-09-21T06:12:01.101 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 6, 2024-09-21T06:12:01.101 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 43 2024-09-21T06:12:01.101 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:01.101 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:12:01.101 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 85, 2024-09-21T06:12:01.101 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T06:11:23:123431+0000", 2024-09-21T06:12:01.102 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 9, 2024-09-21T06:12:01.102 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:12:01.102 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:12:01.102 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:12:01.102 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:12:01.102 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:01.102 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 9, 2024-09-21T06:12:01.102 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:12:01.102 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "c", 2024-09-21T06:12:01.102 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:12:01.103 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 8832 2024-09-21T06:12:01.103 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:01.103 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:12:01.103 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:12:01.103 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:01.103 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:12:01.103 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:12:01.103 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:12:01.103 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:12:01.103 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:12:01.104 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:12:01.104 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:12:01.104 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:12:01.104 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:12:01.104 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:01.104 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:12:01.104 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 21, 2024-09-21T06:12:01.104 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:10:55.202999+0000", 2024-09-21T06:12:01.104 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:12:01.104 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:01.104 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": {} 2024-09-21T06:12:01.105 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:12:01.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 -- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2924075480 msgr2=0x7f2924077990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:01.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2924075480 0x7f2924077990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f29380af000 tx=0x7f293801b000 comp rx=0 tx=0).stop 2024-09-21T06:12:01.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 -- 172.21.15.50:0/3174648664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2948159130 msgr2=0x7f29480b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:01.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2948159130 0x7f29480b2bd0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f294000c760 tx=0x7f294000cc20 comp rx=0 tx=0).stop 2024-09-21T06:12:01.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 -- 172.21.15.50:0/3174648664 shutdown_connections 2024-09-21T06:12:01.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2924075480 0x7f2924077990 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2948159130 0x7f29480b2bd0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f29480703a0 0x7f29480b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 --2- 172.21.15.50:0/3174648664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f294806cdf0 0x7f2948153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 -- 172.21.15.50:0/3174648664 >> 172.21.15.50:0/3174648664 conn(0x7f29480b35f0 msgr2=0x7f2948068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:01.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 -- 172.21.15.50:0/3174648664 shutdown_connections 2024-09-21T06:12:01.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.099+0000 7f294ec00640 1 -- 172.21.15.50:0/3174648664 wait complete. 2024-09-21T06:12:01.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:01.118 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/ss1clone1/7de9ef65-c5ec-4ec4-a5d9-579e05105f30 2024-09-21T06:12:01.118 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:12:01.128 INFO:teuthology.orchestra.run.smithi050.stderr:ls: cannot access 'volumes/_nogroup/ss1clone1/7de9ef65-c5ec-4ec4-a5d9-579e05105f30': No such file or directory 2024-09-21T06:12:01.129 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T06:12:01.129 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:12:01.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.331+0000 7f0567a00640 1 -- 172.21.15.50:0/4231967838 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05600b4de0 msgr2=0x7f05600b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:01.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.331+0000 7f0567a00640 1 --2- 172.21.15.50:0/4231967838 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05600b4de0 0x7f05600b51f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0558009c80 tx=0x7f055801cd70 comp rx=0 tx=0).stop 2024-09-21T06:12:01.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.331+0000 7f0567a00640 1 -- 172.21.15.50:0/4231967838 shutdown_connections 2024-09-21T06:12:01.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.331+0000 7f0567a00640 1 --2- 172.21.15.50:0/4231967838 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05600b4de0 0x7f05600b51f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.331+0000 7f0567a00640 1 --2- 172.21.15.50:0/4231967838 >> v2:172.21.15.88:3300/0 conn(0x7f05601527b0 0x7f056015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.331+0000 7f0567a00640 1 -- 172.21.15.50:0/4231967838 >> 172.21.15.50:0/4231967838 conn(0x7f05600b2040 msgr2=0x7f05600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:01.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.331+0000 7f0567a00640 1 -- 172.21.15.50:0/4231967838 shutdown_connections 2024-09-21T06:12:01.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.331+0000 7f0567a00640 1 -- 172.21.15.50:0/4231967838 wait complete. 2024-09-21T06:12:01.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0567a00640 1 Processor -- start 2024-09-21T06:12:01.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0567a00640 1 -- start start 2024-09-21T06:12:01.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0567a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05600b4de0 0x7f0560156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:01.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0567a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05600b5c70 0x7f05600cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:01.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0567a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05601527b0 0x7f05600cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:01.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0567a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f05601689d0 con 0x7f05600b4de0 2024-09-21T06:12:01.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0567a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0560168850 con 0x7f05601527b0 2024-09-21T06:12:01.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0567a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0560168b50 con 0x7f05600b5c70 2024-09-21T06:12:01.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0565000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05600b4de0 0x7f0560156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:01.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0565000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05600b4de0 0x7f0560156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43292/0 (socket says 172.21.15.50:43292) 2024-09-21T06:12:01.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0565000640 1 -- 172.21.15.50:0/2994454784 learned_addr learned my addr 172.21.15.50:0/2994454784 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:01.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0565a00640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05601527b0 0x7f05600cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:01.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0557e00640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05600b5c70 0x7f05600cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:01.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0565000640 1 -- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05600b5c70 msgr2=0x7f05600cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:01.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0565000640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05600b5c70 0x7f05600cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0565000640 1 -- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05601527b0 msgr2=0x7f05600cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0565000640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05601527b0 0x7f05600cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0565000640 1 -- 172.21.15.50:0/2994454784 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05601f8710 con 0x7f05600b4de0 2024-09-21T06:12:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0565a00640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05601527b0 0x7f05600cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0565000640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05600b4de0 0x7f0560156180 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f055000c3c0 tx=0x7f055000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f055d600640 1 -- 172.21.15.50:0/2994454784 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550017070 con 0x7f05600b4de0 2024-09-21T06:12:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f055d600640 1 -- 172.21.15.50:0/2994454784 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05500053c0 con 0x7f05600b4de0 2024-09-21T06:12:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0567a00640 1 -- 172.21.15.50:0/2994454784 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05601566c0 con 0x7f05600b4de0 2024-09-21T06:12:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f055d600640 1 -- 172.21.15.50:0/2994454784 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550003860 con 0x7f05600b4de0 2024-09-21T06:12:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.335+0000 7f0567a00640 1 -- 172.21.15.50:0/2994454784 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05600cdda0 con 0x7f05600b4de0 2024-09-21T06:12:01.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.339+0000 7f0567a00640 1 -- 172.21.15.50:0/2994454784 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05600b51f0 con 0x7f05600b4de0 2024-09-21T06:12:01.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.339+0000 7f055d600640 1 -- 172.21.15.50:0/2994454784 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 54) ==== 100345+0+0 (secure 0 0 0) 0x7f055001a070 con 0x7f05600b4de0 2024-09-21T06:12:01.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.339+0000 7f055d600640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f053c075480 0x7f053c077990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:01.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.343+0000 7f055d600640 1 -- 172.21.15.50:0/2994454784 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f0550092870 con 0x7f05600b4de0 2024-09-21T06:12:01.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.343+0000 7f0557e00640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f053c075480 0x7f053c077990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:01.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.343+0000 7f0557e00640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f053c075480 0x7f053c077990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f05601513b0 tx=0x7f05480091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:01.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.343+0000 7f055d600640 1 -- 172.21.15.50:0/2994454784 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0550021080 con 0x7f05600b4de0 2024-09-21T06:12:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.543+0000 7f0567a00640 1 -- 172.21.15.50:0/2994454784 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f056006b540 con 0x7f053c075480 2024-09-21T06:12:01.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.547+0000 7f055d600640 1 -- 172.21.15.50:0/2994454784 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f0548011b50 con 0x7f053c075480 2024-09-21T06:12:01.545 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:12:01.545 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:01.545 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:12:01.545 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:01.545 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:12:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.547+0000 7f0567a00640 1 -- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f053c075480 msgr2=0x7f053c077990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.547+0000 7f0567a00640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f053c075480 0x7f053c077990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f05601513b0 tx=0x7f05480091e0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.547+0000 7f0567a00640 1 -- 172.21.15.50:0/2994454784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05600b4de0 msgr2=0x7f0560156180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.547+0000 7f0567a00640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05600b4de0 0x7f0560156180 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f055000c3c0 tx=0x7f055000c880 comp rx=0 tx=0).stop 2024-09-21T06:12:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.547+0000 7f0567a00640 1 -- 172.21.15.50:0/2994454784 shutdown_connections 2024-09-21T06:12:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.547+0000 7f0567a00640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05601527b0 0x7f05600cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.547+0000 7f0567a00640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f05600b5c70 0x7f05600cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.547+0000 7f0567a00640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f053c075480 0x7f053c077990 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.547+0000 7f0567a00640 1 --2- 172.21.15.50:0/2994454784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05600b4de0 0x7f0560156180 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.551+0000 7f0567a00640 1 -- 172.21.15.50:0/2994454784 >> 172.21.15.50:0/2994454784 conn(0x7f05600b2040 msgr2=0x7f05600b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.551+0000 7f0567a00640 1 -- 172.21.15.50:0/2994454784 shutdown_connections 2024-09-21T06:12:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.551+0000 7f0567a00640 1 -- 172.21.15.50:0/2994454784 wait complete. 2024-09-21T06:12:01.567 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-21T06:12:01.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.731+0000 7f0edf400640 1 -- 172.21.15.50:0/1010693334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ebc004810 msgr2=0x7f0ebc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:01.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.731+0000 7f0edf400640 1 --2- 172.21.15.50:0/1010693334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ebc004810 0x7f0ebc004c20 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0009b40 tx=0x7f0ec001d240 comp rx=0 tx=0).stop 2024-09-21T06:12:01.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.731+0000 7f0edf400640 1 -- 172.21.15.50:0/1010693334 shutdown_connections 2024-09-21T06:12:01.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.731+0000 7f0edf400640 1 --2- 172.21.15.50:0/1010693334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ebc004810 0x7f0ebc004c20 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.731+0000 7f0edf400640 1 -- 172.21.15.50:0/1010693334 >> 172.21.15.50:0/1010693334 conn(0x7f0ed80b2040 msgr2=0x7f0ed80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:01.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.731+0000 7f0edf400640 1 -- 172.21.15.50:0/1010693334 shutdown_connections 2024-09-21T06:12:01.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.731+0000 7f0edf400640 1 -- 172.21.15.50:0/1010693334 wait complete. 2024-09-21T06:12:01.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edf400640 1 Processor -- start 2024-09-21T06:12:01.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edf400640 1 -- start start 2024-09-21T06:12:01.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ebc004810 0x7f0ed8150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:01.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ed81e84a0 0x7f0ed80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:01.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edf400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ed81e88b0 0x7f0ed80aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:01.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edf400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed80b5580 con 0x7f0ebc004810 2024-09-21T06:12:01.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edf400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed80b5400 con 0x7f0ed81e84a0 2024-09-21T06:12:01.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edf400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed80b5700 con 0x7f0ed81e88b0 2024-09-21T06:12:01.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ebc004810 0x7f0ed8150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:01.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ebc004810 0x7f0ed8150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43316/0 (socket says 172.21.15.50:43316) 2024-09-21T06:12:01.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edca00640 1 -- 172.21.15.50:0/3171821403 learned_addr learned my addr 172.21.15.50:0/3171821403 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:01.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0ecfe00640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ed81e84a0 0x7f0ed80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:01.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edd400640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ed81e88b0 0x7f0ed80aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:01.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edca00640 1 -- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ed81e88b0 msgr2=0x7f0ed80aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:01.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edca00640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ed81e88b0 0x7f0ed80aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edca00640 1 -- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ed81e84a0 msgr2=0x7f0ed80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:01.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edca00640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ed81e84a0 0x7f0ed80ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edca00640 1 -- 172.21.15.50:0/3171821403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed81e9010 con 0x7f0ebc004810 2024-09-21T06:12:01.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edca00640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ebc004810 0x7f0ed8150f70 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8003ee0 tx=0x7f0ec8001740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:01.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0ed5600640 1 -- 172.21.15.50:0/3171821403 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8015070 con 0x7f0ebc004810 2024-09-21T06:12:01.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0ed5600640 1 -- 172.21.15.50:0/3171821403 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ec8011840 con 0x7f0ebc004810 2024-09-21T06:12:01.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0ed5600640 1 -- 172.21.15.50:0/3171821403 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8011c90 con 0x7f0ebc004810 2024-09-21T06:12:01.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edf400640 1 -- 172.21.15.50:0/3171821403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed8153f30 con 0x7f0ebc004810 2024-09-21T06:12:01.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.735+0000 7f0edf400640 1 -- 172.21.15.50:0/3171821403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed81544f0 con 0x7f0ebc004810 2024-09-21T06:12:01.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.739+0000 7f0edf400640 1 -- 172.21.15.50:0/3171821403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed815f500 con 0x7f0ebc004810 2024-09-21T06:12:01.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.739+0000 7f0ed5600640 1 -- 172.21.15.50:0/3171821403 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 54) ==== 100345+0+0 (secure 0 0 0) 0x7f0ec8018070 con 0x7f0ebc004810 2024-09-21T06:12:01.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.739+0000 7f0ed5600640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0eb4075480 0x7f0eb4077990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:01.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.739+0000 7f0ed5600640 1 -- 172.21.15.50:0/3171821403 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f0ec8094260 con 0x7f0ebc004810 2024-09-21T06:12:01.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.739+0000 7f0ecfe00640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0eb4075480 0x7f0eb4077990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:01.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.743+0000 7f0ecfe00640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0eb4075480 0x7f0eb4077990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00ad000 tx=0x7f0ec0019fe0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:01.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.743+0000 7f0ed5600640 1 -- 172.21.15.50:0/3171821403 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0ec80607a0 con 0x7f0ebc004810 2024-09-21T06:12:01.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.943+0000 7f0edf400640 1 -- 172.21.15.50:0/3171821403 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ed806b540 con 0x7f0eb4075480 2024-09-21T06:12:01.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.951+0000 7f0ed5600640 1 -- 172.21.15.50:0/3171821403 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f0ec0005360 con 0x7f0eb4075480 2024-09-21T06:12:01.950 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:01.950 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:12:01.950 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:01.950 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sv1" 2024-09-21T06:12:01.950 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:01.950 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:01.950 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ss1clone1" 2024-09-21T06:12:01.950 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:01.951 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:12:01.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.951+0000 7f0edf400640 1 -- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0eb4075480 msgr2=0x7f0eb4077990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:01.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.951+0000 7f0edf400640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0eb4075480 0x7f0eb4077990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00ad000 tx=0x7f0ec0019fe0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.951+0000 7f0edf400640 1 -- 172.21.15.50:0/3171821403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ebc004810 msgr2=0x7f0ed8150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:01.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.951+0000 7f0edf400640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ebc004810 0x7f0ed8150f70 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8003ee0 tx=0x7f0ec8001740 comp rx=0 tx=0).stop 2024-09-21T06:12:01.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.951+0000 7f0edf400640 1 -- 172.21.15.50:0/3171821403 shutdown_connections 2024-09-21T06:12:01.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.951+0000 7f0edf400640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ed81e88b0 0x7f0ed80aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.951+0000 7f0edf400640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ed81e84a0 0x7f0ed80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.951+0000 7f0edf400640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0eb4075480 0x7f0eb4077990 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.951+0000 7f0edf400640 1 --2- 172.21.15.50:0/3171821403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ebc004810 0x7f0ed8150f70 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:01.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.951+0000 7f0edf400640 1 -- 172.21.15.50:0/3171821403 >> 172.21.15.50:0/3171821403 conn(0x7f0ed80b2040 msgr2=0x7f0ed8159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:01.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.951+0000 7f0edf400640 1 -- 172.21.15.50:0/3171821403 shutdown_connections 2024-09-21T06:12:01.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:01.955+0000 7f0edf400640 1 -- 172.21.15.50:0/3171821403 wait complete. 2024-09-21T06:12:01.971 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-21T06:12:02.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.135+0000 7f5057200640 1 -- 172.21.15.50:0/2140625591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50580b5040 msgr2=0x7f50580cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.135+0000 7f5057200640 1 --2- 172.21.15.50:0/2140625591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50580b5040 0x7f50580cfe40 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f5048009840 tx=0x7f504801c7f0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.135+0000 7f5057200640 1 -- 172.21.15.50:0/2140625591 shutdown_connections 2024-09-21T06:12:02.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.135+0000 7f5057200640 1 --2- 172.21.15.50:0/2140625591 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50580d0380 0x7f50580b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.135+0000 7f5057200640 1 --2- 172.21.15.50:0/2140625591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50580b5040 0x7f50580cfe40 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.135+0000 7f5057200640 1 -- 172.21.15.50:0/2140625591 >> 172.21.15.50:0/2140625591 conn(0x7f505806cdf0 msgr2=0x7f505806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:02.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.135+0000 7f5057200640 1 -- 172.21.15.50:0/2140625591 shutdown_connections 2024-09-21T06:12:02.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.135+0000 7f5057200640 1 -- 172.21.15.50:0/2140625591 wait complete. 2024-09-21T06:12:02.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f5057200640 1 Processor -- start 2024-09-21T06:12:02.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f5057200640 1 -- start start 2024-09-21T06:12:02.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f5057200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50580b3670 0x7f5058154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f5057200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50580b5040 0x7f5058140a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f5057200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50580d0380 0x7f50581fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f5057200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5058167d50 con 0x7f50580b5040 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f5057200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5058167bd0 con 0x7f50580b3670 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f5057200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5058167ed0 con 0x7f50580d0380 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f504fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50580b5040 0x7f5058140a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f504fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50580b5040 0x7f5058140a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43336/0 (socket says 172.21.15.50:43336) 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f504fe00640 1 -- 172.21.15.50:0/2356328942 learned_addr learned my addr 172.21.15.50:0/2356328942 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f5055e00640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50580b3670 0x7f5058154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f5056800640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50580d0380 0x7f50581fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f504fe00640 1 -- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50580d0380 msgr2=0x7f50581fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f504fe00640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50580d0380 0x7f50581fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f504fe00640 1 -- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50580b3670 msgr2=0x7f5058154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f504fe00640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50580b3670 0x7f5058154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f504fe00640 1 -- 172.21.15.50:0/2356328942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50580abaf0 con 0x7f50580b5040 2024-09-21T06:12:02.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.139+0000 7f5055e00640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50580b3670 0x7f5058154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:02.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.143+0000 7f504fe00640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50580b5040 0x7f5058140a20 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f5040005d10 tx=0x7f504000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:02.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.143+0000 7f504e000640 1 -- 172.21.15.50:0/2356328942 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504000de50 con 0x7f50580b5040 2024-09-21T06:12:02.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.143+0000 7f504e000640 1 -- 172.21.15.50:0/2356328942 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5040005110 con 0x7f50580b5040 2024-09-21T06:12:02.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.143+0000 7f504e000640 1 -- 172.21.15.50:0/2356328942 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040005560 con 0x7f50580b5040 2024-09-21T06:12:02.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.143+0000 7f5057200640 1 -- 172.21.15.50:0/2356328942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50580ad4b0 con 0x7f50580b5040 2024-09-21T06:12:02.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.143+0000 7f5057200640 1 -- 172.21.15.50:0/2356328942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50580ada70 con 0x7f50580b5040 2024-09-21T06:12:02.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.143+0000 7f5057200640 1 -- 172.21.15.50:0/2356328942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50580cfe40 con 0x7f50580b5040 2024-09-21T06:12:02.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.143+0000 7f504e000640 1 -- 172.21.15.50:0/2356328942 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 54) ==== 100345+0+0 (secure 0 0 0) 0x7f50400047b0 con 0x7f50580b5040 2024-09-21T06:12:02.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.147+0000 7f504e000640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50340755f0 0x7f5034077b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.147+0000 7f504e000640 1 -- 172.21.15.50:0/2356328942 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f50400939f0 con 0x7f50580b5040 2024-09-21T06:12:02.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.147+0000 7f5055e00640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50340755f0 0x7f5034077b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.147+0000 7f5055e00640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50340755f0 0x7f5034077b00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f50480af000 tx=0x7f504801b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:02.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.151+0000 7f504e000640 1 -- 172.21.15.50:0/2356328942 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5040060470 con 0x7f50580b5040 2024-09-21T06:12:02.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.355+0000 7f5057200640 1 -- 172.21.15.50:0/2356328942 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f505806b540 con 0x7f50340755f0 2024-09-21T06:12:02.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.371+0000 7f504e000640 1 -- 172.21.15.50:0/2356328942 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f50480041c0 con 0x7f50340755f0 2024-09-21T06:12:02.370 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:02.370 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:12:02.370 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:02.370 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ss1" 2024-09-21T06:12:02.370 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:02.370 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:12:02.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.375+0000 7f5057200640 1 -- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50340755f0 msgr2=0x7f5034077b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.375+0000 7f5057200640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50340755f0 0x7f5034077b00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f50480af000 tx=0x7f504801b030 comp rx=0 tx=0).stop 2024-09-21T06:12:02.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.375+0000 7f5057200640 1 -- 172.21.15.50:0/2356328942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50580b5040 msgr2=0x7f5058140a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.375+0000 7f5057200640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50580b5040 0x7f5058140a20 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f5040005d10 tx=0x7f504000a620 comp rx=0 tx=0).stop 2024-09-21T06:12:02.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.375+0000 7f5057200640 1 -- 172.21.15.50:0/2356328942 shutdown_connections 2024-09-21T06:12:02.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.375+0000 7f5057200640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f50580d0380 0x7f50581fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.375+0000 7f5057200640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f50340755f0 0x7f5034077b00 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.375+0000 7f5057200640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50580b5040 0x7f5058140a20 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.375+0000 7f5057200640 1 --2- 172.21.15.50:0/2356328942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50580b3670 0x7f5058154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.375+0000 7f5057200640 1 -- 172.21.15.50:0/2356328942 >> 172.21.15.50:0/2356328942 conn(0x7f505806cdf0 msgr2=0x7f5058165110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:02.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.375+0000 7f5057200640 1 -- 172.21.15.50:0/2356328942 shutdown_connections 2024-09-21T06:12:02.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.375+0000 7f5057200640 1 -- 172.21.15.50:0/2356328942 wait complete. 2024-09-21T06:12:02.392 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-21T06:12:02.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.559+0000 7fc364e00640 1 -- 172.21.15.50:0/4057033378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3600b4de0 msgr2=0x7fc3600b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.559+0000 7fc364e00640 1 --2- 172.21.15.50:0/4057033378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3600b4de0 0x7fc3600b51f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc358009bc0 tx=0x7fc35801cae0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.559+0000 7fc364e00640 1 -- 172.21.15.50:0/4057033378 shutdown_connections 2024-09-21T06:12:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.559+0000 7fc364e00640 1 --2- 172.21.15.50:0/4057033378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3600b4de0 0x7fc3600b51f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.559+0000 7fc364e00640 1 --2- 172.21.15.50:0/4057033378 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3600b5c70 0x7fc36015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.559+0000 7fc364e00640 1 -- 172.21.15.50:0/4057033378 >> 172.21.15.50:0/4057033378 conn(0x7fc3600b2040 msgr2=0x7fc3600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.559+0000 7fc364e00640 1 -- 172.21.15.50:0/4057033378 shutdown_connections 2024-09-21T06:12:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.559+0000 7fc364e00640 1 -- 172.21.15.50:0/4057033378 wait complete. 2024-09-21T06:12:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc364e00640 1 Processor -- start 2024-09-21T06:12:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc364e00640 1 -- start start 2024-09-21T06:12:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc364e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b4de0 0x7fc360154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc364e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3600b5c70 0x7fc3600aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc364e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3601527b0 0x7fc3600cead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc364e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc360168a00 con 0x7fc3600b4de0 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc364e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc360168880 con 0x7fc3601527b0 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc364e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc360168b80 con 0x7fc3600b5c70 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc35de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b4de0 0x7fc360154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc35de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b4de0 0x7fc360154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43342/0 (socket says 172.21.15.50:43342) 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc35de00640 1 -- 172.21.15.50:0/967348555 learned_addr learned my addr 172.21.15.50:0/967348555 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc355400640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3600b5c70 0x7fc3600aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc35e800640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3601527b0 0x7fc3600cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc35de00640 1 -- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3600b5c70 msgr2=0x7fc3600aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc35de00640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3600b5c70 0x7fc3600aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc35de00640 1 -- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3601527b0 msgr2=0x7fc3600cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc35de00640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3601527b0 0x7fc3600cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc35de00640 1 -- 172.21.15.50:0/967348555 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc360155310 con 0x7fc3600b4de0 2024-09-21T06:12:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc355400640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3600b5c70 0x7fc3600aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc35de00640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b4de0 0x7fc360154dd0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fc35000e7c0 tx=0x7fc35000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc356a00640 1 -- 172.21.15.50:0/967348555 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35000ccc0 con 0x7fc3600b4de0 2024-09-21T06:12:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc356a00640 1 -- 172.21.15.50:0/967348555 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc35000ce80 con 0x7fc3600b4de0 2024-09-21T06:12:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc364e00640 1 -- 172.21.15.50:0/967348555 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc360155820 con 0x7fc3600b4de0 2024-09-21T06:12:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc356a00640 1 -- 172.21.15.50:0/967348555 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350005280 con 0x7fc3600b4de0 2024-09-21T06:12:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.563+0000 7fc364e00640 1 -- 172.21.15.50:0/967348555 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3601f9f80 con 0x7fc3600b4de0 2024-09-21T06:12:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.567+0000 7fc364e00640 1 -- 172.21.15.50:0/967348555 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3600b51f0 con 0x7fc3600b4de0 2024-09-21T06:12:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.567+0000 7fc356a00640 1 -- 172.21.15.50:0/967348555 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 54) ==== 100345+0+0 (secure 0 0 0) 0x7fc350005420 con 0x7fc3600b4de0 2024-09-21T06:12:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.567+0000 7fc356a00640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc33c075450 0x7fc33c077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.567+0000 7fc356a00640 1 -- 172.21.15.50:0/967348555 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fc350093d50 con 0x7fc3600b4de0 2024-09-21T06:12:02.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.567+0000 7fc355400640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc33c075450 0x7fc33c077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.571+0000 7fc355400640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc33c075450 0x7fc33c077960 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc348001400 tx=0x7fc348009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:02.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.571+0000 7fc356a00640 1 -- 172.21.15.50:0/967348555 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc3500607d0 con 0x7fc3600b4de0 2024-09-21T06:12:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.783+0000 7fc364e00640 1 -- 172.21.15.50:0/967348555 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc36006b540 con 0x7fc33c075450 2024-09-21T06:12:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.815+0000 7fc356a00640 1 -- 172.21.15.50:0/967348555 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc348007ab0 con 0x7fc33c075450 2024-09-21T06:12:02.814 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:02.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.819+0000 7fc364e00640 1 -- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc33c075450 msgr2=0x7fc33c077960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.819+0000 7fc364e00640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc33c075450 0x7fc33c077960 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc348001400 tx=0x7fc348009040 comp rx=0 tx=0).stop 2024-09-21T06:12:02.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.819+0000 7fc364e00640 1 -- 172.21.15.50:0/967348555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b4de0 msgr2=0x7fc360154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.819+0000 7fc364e00640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b4de0 0x7fc360154dd0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fc35000e7c0 tx=0x7fc35000ec80 comp rx=0 tx=0).stop 2024-09-21T06:12:02.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.819+0000 7fc364e00640 1 -- 172.21.15.50:0/967348555 shutdown_connections 2024-09-21T06:12:02.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.819+0000 7fc364e00640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3601527b0 0x7fc3600cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.819+0000 7fc364e00640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc3600b5c70 0x7fc3600aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.819+0000 7fc364e00640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc33c075450 0x7fc33c077960 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.819+0000 7fc364e00640 1 --2- 172.21.15.50:0/967348555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b4de0 0x7fc360154dd0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.819+0000 7fc364e00640 1 -- 172.21.15.50:0/967348555 >> 172.21.15.50:0/967348555 conn(0x7fc3600b2040 msgr2=0x7fc3600b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:02.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.819+0000 7fc364e00640 1 -- 172.21.15.50:0/967348555 shutdown_connections 2024-09-21T06:12:02.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:02.819+0000 7fc364e00640 1 -- 172.21.15.50:0/967348555 wait complete. 2024-09-21T06:12:02.836 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-21T06:12:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.003+0000 7f44e8400640 1 -- 172.21.15.50:0/1396510691 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44e0160c20 msgr2=0x7f44e0068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.003+0000 7f44e8400640 1 --2- 172.21.15.50:0/1396510691 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44e0160c20 0x7f44e0068df0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009900 tx=0x7f44d401c8b0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.003+0000 7f44e8400640 1 -- 172.21.15.50:0/1396510691 shutdown_connections 2024-09-21T06:12:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.003+0000 7f44e8400640 1 --2- 172.21.15.50:0/1396510691 >> v2:172.21.15.88:3300/0 conn(0x7f44e00d0d60 0x7f44e006d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.003+0000 7f44e8400640 1 --2- 172.21.15.50:0/1396510691 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44e0160c20 0x7f44e0068df0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.003+0000 7f44e8400640 1 -- 172.21.15.50:0/1396510691 >> 172.21.15.50:0/1396510691 conn(0x7f44e00b35f0 msgr2=0x7f44e00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.003+0000 7f44e8400640 1 -- 172.21.15.50:0/1396510691 shutdown_connections 2024-09-21T06:12:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.003+0000 7f44e8400640 1 -- 172.21.15.50:0/1396510691 wait complete. 2024-09-21T06:12:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.003+0000 7f44e8400640 1 Processor -- start 2024-09-21T06:12:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.003+0000 7f44e8400640 1 -- start start 2024-09-21T06:12:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e8400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44e0069330 0x7f44e0161440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e8400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44e00d0d60 0x7f44e0152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e8400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44e0160c20 0x7f44e0070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e8400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44e0169e30 con 0x7f44e0160c20 2024-09-21T06:12:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e8400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44e0169cb0 con 0x7f44e0069330 2024-09-21T06:12:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e8400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f44e0169fb0 con 0x7f44e00d0d60 2024-09-21T06:12:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e6400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44e0160c20 0x7f44e0070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e5a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44e0069330 0x7f44e0161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e5000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44e00d0d60 0x7f44e0152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e5a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44e0069330 0x7f44e0161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:59296/0 (socket says 172.21.15.50:59296) 2024-09-21T06:12:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e5a00640 1 -- 172.21.15.50:0/2861793059 learned_addr learned my addr 172.21.15.50:0/2861793059 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e5a00640 1 -- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44e00d0d60 msgr2=0x7f44e0152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e5a00640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44e00d0d60 0x7f44e0152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e5a00640 1 -- 172.21.15.50:0/2861793059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44e0160c20 msgr2=0x7f44e0070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e5a00640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44e0160c20 0x7f44e0070af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e5a00640 1 -- 172.21.15.50:0/2861793059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e01fcc30 con 0x7f44e0069330 2024-09-21T06:12:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e6400640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44e0160c20 0x7f44e0070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e5a00640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44e0069330 0x7f44e0161440 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009850 tx=0x7f44d4005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44dd600640 1 -- 172.21.15.50:0/2861793059 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d40b2040 con 0x7f44e0069330 2024-09-21T06:12:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44dd600640 1 -- 172.21.15.50:0/2861793059 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44d4005320 con 0x7f44e0069330 2024-09-21T06:12:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e8400640 1 -- 172.21.15.50:0/2861793059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e01fcf10 con 0x7f44e0069330 2024-09-21T06:12:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44dd600640 1 -- 172.21.15.50:0/2861793059 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d40046d0 con 0x7f44e0069330 2024-09-21T06:12:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e8400640 1 -- 172.21.15.50:0/2861793059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e01fd4d0 con 0x7f44e0069330 2024-09-21T06:12:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.007+0000 7f44e8400640 1 -- 172.21.15.50:0/2861793059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e0068df0 con 0x7f44e0069330 2024-09-21T06:12:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.011+0000 7f44dd600640 1 -- 172.21.15.50:0/2861793059 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 54) ==== 100345+0+0 (secure 0 0 0) 0x7f44d40bd060 con 0x7f44e0069330 2024-09-21T06:12:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.011+0000 7f44dd600640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44c0075450 0x7f44c0077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.011+0000 7f44dd600640 1 -- 172.21.15.50:0/2861793059 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f44d412d480 con 0x7f44e0069330 2024-09-21T06:12:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.011+0000 7f44e5000640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44c0075450 0x7f44c0077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:03.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.011+0000 7f44e5000640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44c0075450 0x7f44c0077960 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f44e00b3ab0 tx=0x7f44cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:03.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.015+0000 7f44dd600640 1 -- 172.21.15.50:0/2861793059 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f44d40164d0 con 0x7f44e0069330 2024-09-21T06:12:03.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.219+0000 7f44e8400640 1 -- 172.21.15.50:0/2861793059 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f44e006b540 con 0x7f44c0075450 2024-09-21T06:12:03.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.239+0000 7f44dd600640 1 -- 172.21.15.50:0/2861793059 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f44cc001d20 con 0x7f44c0075450 2024-09-21T06:12:03.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.243+0000 7f44e8400640 1 -- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44c0075450 msgr2=0x7f44c0077960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:03.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.243+0000 7f44e8400640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44c0075450 0x7f44c0077960 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f44e00b3ab0 tx=0x7f44cc009040 comp rx=0 tx=0).stop 2024-09-21T06:12:03.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.243+0000 7f44e8400640 1 -- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44e0069330 msgr2=0x7f44e0161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:03.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.243+0000 7f44e8400640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44e0069330 0x7f44e0161440 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009850 tx=0x7f44d4005de0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.243+0000 7f44e8400640 1 -- 172.21.15.50:0/2861793059 shutdown_connections 2024-09-21T06:12:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.243+0000 7f44e8400640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f44c0075450 0x7f44c0077960 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.243+0000 7f44e8400640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44e0160c20 0x7f44e0070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.243+0000 7f44e8400640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f44e00d0d60 0x7f44e0152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.243+0000 7f44e8400640 1 --2- 172.21.15.50:0/2861793059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44e0069330 0x7f44e0161440 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.243+0000 7f44e8400640 1 -- 172.21.15.50:0/2861793059 >> 172.21.15.50:0/2861793059 conn(0x7f44e00b35f0 msgr2=0x7f44e015e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.243+0000 7f44e8400640 1 -- 172.21.15.50:0/2861793059 shutdown_connections 2024-09-21T06:12:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.243+0000 7f44e8400640 1 -- 172.21.15.50:0/2861793059 wait complete. 2024-09-21T06:12:03.251 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:03.251+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:03.251 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:03.251+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:03.251 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:03.251+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:03.251 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:03.251+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:03.251 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:03.251+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:03.263 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-21T06:12:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.435+0000 7f30c5200640 1 -- 172.21.15.50:0/63291937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c00b4ce0 msgr2=0x7f30c015d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.435+0000 7f30c5200640 1 --2- 172.21.15.50:0/63291937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c00b4ce0 0x7f30c015d890 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009bc0 tx=0x7f30c015e5e0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.435+0000 7f30c5200640 1 -- 172.21.15.50:0/63291937 shutdown_connections 2024-09-21T06:12:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.435+0000 7f30c5200640 1 --2- 172.21.15.50:0/63291937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c00b4ce0 0x7f30c015d890 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.435+0000 7f30c5200640 1 -- 172.21.15.50:0/63291937 >> 172.21.15.50:0/63291937 conn(0x7f30c00b2040 msgr2=0x7f30c00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.435+0000 7f30c5200640 1 -- 172.21.15.50:0/63291937 shutdown_connections 2024-09-21T06:12:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.435+0000 7f30c5200640 1 -- 172.21.15.50:0/63291937 wait complete. 2024-09-21T06:12:03.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.435+0000 7f30c5200640 1 Processor -- start 2024-09-21T06:12:03.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.435+0000 7f30c5200640 1 -- start start 2024-09-21T06:12:03.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30c5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c00b4ce0 0x7f30c0156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:03.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30c5200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30c00cf900 0x7f30c01f9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:03.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30c5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c01576e0 0x7f30c0157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:03.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30c5200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01694e0 con 0x7f30c00b4ce0 2024-09-21T06:12:03.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30c5200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f30c0169360 con 0x7f30c01576e0 2024-09-21T06:12:03.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30c5200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f30c0169660 con 0x7f30c00cf900 2024-09-21T06:12:03.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30bde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c00b4ce0 0x7f30c0156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:03.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30bde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c00b4ce0 0x7f30c0156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43360/0 (socket says 172.21.15.50:43360) 2024-09-21T06:12:03.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30bde00640 1 -- 172.21.15.50:0/1441404207 learned_addr learned my addr 172.21.15.50:0/1441404207 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:03.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30b7e00640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30c00cf900 0x7f30c01f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:03.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30be800640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c01576e0 0x7f30c0157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:03.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30bde00640 1 -- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30c00cf900 msgr2=0x7f30c01f9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:03.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30bde00640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30c00cf900 0x7f30c01f9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30bde00640 1 -- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c01576e0 msgr2=0x7f30c0157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:03.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30bde00640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c01576e0 0x7f30c0157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30bde00640 1 -- 172.21.15.50:0/1441404207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c01f9690 con 0x7f30c00b4ce0 2024-09-21T06:12:03.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30be800640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c01576e0 0x7f30c0157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:03.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30b7e00640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30c00cf900 0x7f30c01f9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:03.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30bde00640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c00b4ce0 0x7f30c0156840 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f30b000c410 tx=0x7f30b000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:03.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30b6000640 1 -- 172.21.15.50:0/1441404207 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0011790 con 0x7f30c00b4ce0 2024-09-21T06:12:03.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30b6000640 1 -- 172.21.15.50:0/1441404207 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30b0017070 con 0x7f30c00b4ce0 2024-09-21T06:12:03.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30b6000640 1 -- 172.21.15.50:0/1441404207 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0011d20 con 0x7f30c00b4ce0 2024-09-21T06:12:03.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30c5200640 1 -- 172.21.15.50:0/1441404207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c00af9c0 con 0x7f30c00b4ce0 2024-09-21T06:12:03.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30c5200640 1 -- 172.21.15.50:0/1441404207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c0155d60 con 0x7f30c00b4ce0 2024-09-21T06:12:03.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.439+0000 7f30c5200640 1 -- 172.21.15.50:0/1441404207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c015e8c0 con 0x7f30c00b4ce0 2024-09-21T06:12:03.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.443+0000 7f30b6000640 1 -- 172.21.15.50:0/1441404207 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f30b001c460 con 0x7f30c00b4ce0 2024-09-21T06:12:03.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.443+0000 7f30b6000640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f309c0754e0 0x7f309c0779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:03.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.443+0000 7f30b6000640 1 -- 172.21.15.50:0/1441404207 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f30b00932b0 con 0x7f30c00b4ce0 2024-09-21T06:12:03.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.443+0000 7f30b7e00640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f309c0754e0 0x7f309c0779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:03.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.443+0000 7f30b7e00640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f309c0754e0 0x7f309c0779f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f30c00b4b60 tx=0x7f30a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:03.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.447+0000 7f30b6000640 1 -- 172.21.15.50:0/1441404207 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f30c01f9690 con 0x7f30c00b4ce0 2024-09-21T06:12:03.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.659+0000 7f30c5200640 1 -- 172.21.15.50:0/1441404207 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f30c006b540 con 0x7f309c0754e0 2024-09-21T06:12:03.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30b6000640 1 -- 172.21.15.50:0/1441404207 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f30a80050d0 con 0x7f309c0754e0 2024-09-21T06:12:03.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30c5200640 1 -- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f309c0754e0 msgr2=0x7f309c0779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:03.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30c5200640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f309c0754e0 0x7f309c0779f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f30c00b4b60 tx=0x7f30a8009040 comp rx=0 tx=0).stop 2024-09-21T06:12:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30c5200640 1 -- 172.21.15.50:0/1441404207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c00b4ce0 msgr2=0x7f30c0156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30c5200640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c00b4ce0 0x7f30c0156840 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f30b000c410 tx=0x7f30b000c8d0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30c5200640 1 -- 172.21.15.50:0/1441404207 shutdown_connections 2024-09-21T06:12:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30c5200640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c01576e0 0x7f30c0157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30c5200640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f30c00cf900 0x7f30c01f9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30c5200640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f309c0754e0 0x7f309c0779f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30c5200640 1 --2- 172.21.15.50:0/1441404207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c00b4ce0 0x7f30c0156840 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30c5200640 1 -- 172.21.15.50:0/1441404207 >> 172.21.15.50:0/1441404207 conn(0x7f30c00b2040 msgr2=0x7f30c00b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30c5200640 1 -- 172.21.15.50:0/1441404207 shutdown_connections 2024-09-21T06:12:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:03.979+0000 7f30c5200640 1 -- 172.21.15.50:0/1441404207 wait complete. 2024-09-21T06:12:03.999 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T06:12:04.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:12:04.054 INFO:teuthology.orchestra.run.smithi050.stdout:3 2024-09-21T06:12:04.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:12:09.062 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T06:12:09.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:12:09.121 INFO:teuthology.orchestra.run.smithi050.stdout:2 2024-09-21T06:12:09.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:12:09.129 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:12:09.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.299+0000 7fdefa600640 1 -- 172.21.15.50:0/2193058547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fded8003c40 msgr2=0x7fded8004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:09.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.299+0000 7fdefa600640 1 --2- 172.21.15.50:0/2193058547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fded8003c40 0x7fded8004090 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fdee4001860 tx=0x7fdee401c890 comp rx=0 tx=0).stop 2024-09-21T06:12:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.299+0000 7fdefa600640 1 -- 172.21.15.50:0/2193058547 shutdown_connections 2024-09-21T06:12:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.299+0000 7fdefa600640 1 --2- 172.21.15.50:0/2193058547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fded8003c40 0x7fded8004090 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.299+0000 7fdefa600640 1 -- 172.21.15.50:0/2193058547 >> 172.21.15.50:0/2193058547 conn(0x7fdef40b2040 msgr2=0x7fdef40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.299+0000 7fdefa600640 1 -- 172.21.15.50:0/2193058547 shutdown_connections 2024-09-21T06:12:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.299+0000 7fdefa600640 1 -- 172.21.15.50:0/2193058547 wait complete. 2024-09-21T06:12:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.299+0000 7fdefa600640 1 Processor -- start 2024-09-21T06:12:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.299+0000 7fdefa600640 1 -- start start 2024-09-21T06:12:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdefa600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fded8003c40 0x7fdef4155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdefa600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdef4154380 0x7fdef414c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdefa600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdef41561a0 0x7fdef41fc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdefa600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdef415f560 con 0x7fdef4154380 2024-09-21T06:12:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdefa600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdef415f3e0 con 0x7fdef41561a0 2024-09-21T06:12:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdefa600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fdef415f6e0 con 0x7fded8003c40 2024-09-21T06:12:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdeebe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdef4154380 0x7fdef414c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdeebe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdef4154380 0x7fdef414c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42612/0 (socket says 172.21.15.50:42612) 2024-09-21T06:12:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdeebe00640 1 -- 172.21.15.50:0/2382836155 learned_addr learned my addr 172.21.15.50:0/2382836155 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdef3400640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fded8003c40 0x7fdef4155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdef3e00640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdef41561a0 0x7fdef41fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdeebe00640 1 -- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fded8003c40 msgr2=0x7fdef4155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdeebe00640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fded8003c40 0x7fdef4155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdeebe00640 1 -- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdef41561a0 msgr2=0x7fdef41fc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdeebe00640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdef41561a0 0x7fdef41fc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdeebe00640 1 -- 172.21.15.50:0/2382836155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef41532e0 con 0x7fdef4154380 2024-09-21T06:12:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdef3e00640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdef41561a0 0x7fdef41fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdeebe00640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdef4154380 0x7fdef414c850 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fdedc006600 tx=0x7fdedc0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdeeb400640 1 -- 172.21.15.50:0/2382836155 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc015070 con 0x7fdef4154380 2024-09-21T06:12:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdeeb400640 1 -- 172.21.15.50:0/2382836155 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdedc011920 con 0x7fdef4154380 2024-09-21T06:12:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdeeb400640 1 -- 172.21.15.50:0/2382836155 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc011d70 con 0x7fdef4154380 2024-09-21T06:12:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdefa600640 1 -- 172.21.15.50:0/2382836155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef41535c0 con 0x7fdef4154380 2024-09-21T06:12:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdefa600640 1 -- 172.21.15.50:0/2382836155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef4153b80 con 0x7fdef4154380 2024-09-21T06:12:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.303+0000 7fdefa600640 1 -- 172.21.15.50:0/2382836155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef415fc40 con 0x7fdef4154380 2024-09-21T06:12:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.307+0000 7fdeeb400640 1 -- 172.21.15.50:0/2382836155 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fdedc018070 con 0x7fdef4154380 2024-09-21T06:12:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.307+0000 7fdeeb400640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fded0075510 0x7fded0077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.307+0000 7fdeeb400640 1 -- 172.21.15.50:0/2382836155 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5849+0+0 (secure 0 0 0) 0x7fdedc0940f0 con 0x7fdef4154380 2024-09-21T06:12:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.307+0000 7fdef3400640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fded0075510 0x7fded0077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.307+0000 7fdef3400640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fded0075510 0x7fded0077a20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdee4001830 tx=0x7fdee40196d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.311+0000 7fdeeb400640 1 -- 172.21.15.50:0/2382836155 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdedc060cb0 con 0x7fdef4154380 2024-09-21T06:12:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.491+0000 7fdefa600640 1 -- 172.21.15.50:0/2382836155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fdef40b49d0 con 0x7fdef4154380 2024-09-21T06:12:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.491+0000 7fdeeb400640 1 -- 172.21.15.50:0/2382836155 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v56)=0 v56) ==== 131+0+0 (secure 0 0 0) 0x7fdedc0605b0 con 0x7fdef4154380 2024-09-21T06:12:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.495+0000 7fdefa600640 1 -- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fded0075510 msgr2=0x7fded0077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.495+0000 7fdefa600640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fded0075510 0x7fded0077a20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdee4001830 tx=0x7fdee40196d0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.495+0000 7fdefa600640 1 -- 172.21.15.50:0/2382836155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdef4154380 msgr2=0x7fdef414c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.495+0000 7fdefa600640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdef4154380 0x7fdef414c850 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fdedc006600 tx=0x7fdedc0039d0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.495+0000 7fdefa600640 1 -- 172.21.15.50:0/2382836155 shutdown_connections 2024-09-21T06:12:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.495+0000 7fdefa600640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdef41561a0 0x7fdef41fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.495+0000 7fdefa600640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fded0075510 0x7fded0077a20 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.495+0000 7fdefa600640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdef4154380 0x7fdef414c850 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.495+0000 7fdefa600640 1 --2- 172.21.15.50:0/2382836155 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fded8003c40 0x7fdef4155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.495+0000 7fdefa600640 1 -- 172.21.15.50:0/2382836155 >> 172.21.15.50:0/2382836155 conn(0x7fdef40b2040 msgr2=0x7fdef406e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.495+0000 7fdefa600640 1 -- 172.21.15.50:0/2382836155 shutdown_connections 2024-09-21T06:12:09.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.495+0000 7fdefa600640 1 -- 172.21.15.50:0/2382836155 wait complete. 2024-09-21T06:12:09.513 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:12:09.513 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:12:09.601 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:12:09.712 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:12:09.712 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:12:09.712 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:12:09.713 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:09.722 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:12:09.723 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:09.775 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:12:09.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.979+0000 7f856f400640 1 -- 172.21.15.50:0/321936885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85680b4de0 msgr2=0x7f85680b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:09.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.979+0000 7f856f400640 1 --2- 172.21.15.50:0/321936885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85680b4de0 0x7f85680b51f0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f8560009c80 tx=0x7f856001cb00 comp rx=0 tx=0).stop 2024-09-21T06:12:09.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.979+0000 7f856f400640 1 -- 172.21.15.50:0/321936885 shutdown_connections 2024-09-21T06:12:09.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.979+0000 7f856f400640 1 --2- 172.21.15.50:0/321936885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85680b4de0 0x7f85680b51f0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.979+0000 7f856f400640 1 --2- 172.21.15.50:0/321936885 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85680b5c70 0x7f856815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.979+0000 7f856f400640 1 -- 172.21.15.50:0/321936885 >> 172.21.15.50:0/321936885 conn(0x7f85680b2040 msgr2=0x7f85680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:09.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.979+0000 7f856f400640 1 -- 172.21.15.50:0/321936885 shutdown_connections 2024-09-21T06:12:09.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.979+0000 7f856f400640 1 -- 172.21.15.50:0/321936885 wait complete. 2024-09-21T06:12:09.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f856f400640 1 Processor -- start 2024-09-21T06:12:09.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f856f400640 1 -- start start 2024-09-21T06:12:09.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f856f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85680b4de0 0x7f856813ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:09.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f856f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85680b5c70 0x7f85680ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:09.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f856f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85681527b0 0x7f85680aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:09.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f856f400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8568168ab0 con 0x7f85680b5c70 2024-09-21T06:12:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f856f400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8568168930 con 0x7f85681527b0 2024-09-21T06:12:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f856f400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8568168c30 con 0x7f85680b4de0 2024-09-21T06:12:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f8567e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85680b5c70 0x7f85680ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f8567e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85680b5c70 0x7f85680ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42628/0 (socket says 172.21.15.50:42628) 2024-09-21T06:12:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f8567e00640 1 -- 172.21.15.50:0/4115528847 learned_addr learned my addr 172.21.15.50:0/4115528847 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f856ca00640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85680b4de0 0x7f856813ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.983+0000 7f856d400640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85681527b0 0x7f85680aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f8567e00640 1 -- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85680b4de0 msgr2=0x7f856813ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f8567e00640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85680b4de0 0x7f856813ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f8567e00640 1 -- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85681527b0 msgr2=0x7f85680aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f8567e00640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85681527b0 0x7f85680aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f8567e00640 1 -- 172.21.15.50:0/4115528847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85680aad70 con 0x7f85680b5c70 2024-09-21T06:12:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f8567e00640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85680b5c70 0x7f85680ad420 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f855400b3c0 tx=0x7f855400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f8564c00640 1 -- 172.21.15.50:0/4115528847 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554011830 con 0x7f85680b5c70 2024-09-21T06:12:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f8564c00640 1 -- 172.21.15.50:0/4115528847 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8554016070 con 0x7f85680b5c70 2024-09-21T06:12:09.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f856f400640 1 -- 172.21.15.50:0/4115528847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85680ab050 con 0x7f85680b5c70 2024-09-21T06:12:09.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f8564c00640 1 -- 172.21.15.50:0/4115528847 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554011d80 con 0x7f85680b5c70 2024-09-21T06:12:09.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f856f400640 1 -- 172.21.15.50:0/4115528847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85680ab610 con 0x7f85680b5c70 2024-09-21T06:12:09.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f856f400640 1 -- 172.21.15.50:0/4115528847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85680b51f0 con 0x7f85680b5c70 2024-09-21T06:12:09.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.987+0000 7f8564c00640 1 -- 172.21.15.50:0/4115528847 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f855401b460 con 0x7f85680b5c70 2024-09-21T06:12:09.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.991+0000 7f8564c00640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f85480754e0 0x7f85480779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:09.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.991+0000 7f8564c00640 1 -- 172.21.15.50:0/4115528847 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5849+0+0 (secure 0 0 0) 0x7f8554092d90 con 0x7f85680b5c70 2024-09-21T06:12:09.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.991+0000 7f856ca00640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f85480754e0 0x7f85480779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:09.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.991+0000 7f856ca00640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f85480754e0 0x7f85480779f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f855c00ab40 tx=0x7f855c00a250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:09.995+0000 7f8564c00640 1 -- 172.21.15.50:0/4115528847 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8554020020 con 0x7f85680b5c70 2024-09-21T06:12:10.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.183+0000 7f856f400640 1 -- 172.21.15.50:0/4115528847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f85680ace90 con 0x7f85680b5c70 2024-09-21T06:12:10.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.183+0000 7f8564c00640 1 -- 172.21.15.50:0/4115528847 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v128) ==== 80+0+0 (secure 0 0 0) 0x7f855405f1c0 con 0x7f85680b5c70 2024-09-21T06:12:10.183 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T06:12:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.187+0000 7f856f400640 1 -- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f85480754e0 msgr2=0x7f85480779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.187+0000 7f856f400640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f85480754e0 0x7f85480779f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f855c00ab40 tx=0x7f855c00a250 comp rx=0 tx=0).stop 2024-09-21T06:12:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.187+0000 7f856f400640 1 -- 172.21.15.50:0/4115528847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85680b5c70 msgr2=0x7f85680ad420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.187+0000 7f856f400640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85680b5c70 0x7f85680ad420 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f855400b3c0 tx=0x7f855400b880 comp rx=0 tx=0).stop 2024-09-21T06:12:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.187+0000 7f856f400640 1 -- 172.21.15.50:0/4115528847 shutdown_connections 2024-09-21T06:12:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.187+0000 7f856f400640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85681527b0 0x7f85680aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.187+0000 7f856f400640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f85480754e0 0x7f85480779f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.187+0000 7f856f400640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85680b5c70 0x7f85680ad420 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.187+0000 7f856f400640 1 --2- 172.21.15.50:0/4115528847 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f85680b4de0 0x7f856813ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.187+0000 7f856f400640 1 -- 172.21.15.50:0/4115528847 >> 172.21.15.50:0/4115528847 conn(0x7f85680b2040 msgr2=0x7f85680b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.187+0000 7f856f400640 1 -- 172.21.15.50:0/4115528847 shutdown_connections 2024-09-21T06:12:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.187+0000 7f856f400640 1 -- 172.21.15.50:0/4115528847 wait complete. 2024-09-21T06:12:10.206 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:12:10.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:10.206 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:12:10.254 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:12:10.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:10.254 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:12:10.289 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:12:10.289 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:12:10.289 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:10.298 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:12:10.299 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:12:10.299 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:12:10.299 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:10.338 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:12:10.339 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:12:10.339 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:12:10.339 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:10.348 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:12:10.349 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:12:10.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.543+0000 7f31f3e00640 1 -- 172.21.15.50:0/1652135627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31ec0cefa0 msgr2=0x7f31ec0cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:10.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.543+0000 7f31f3e00640 1 --2- 172.21.15.50:0/1652135627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31ec0cefa0 0x7f31ec0cf3b0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0cfc90 tx=0x7f31d801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.543+0000 7f31f3e00640 1 -- 172.21.15.50:0/1652135627 shutdown_connections 2024-09-21T06:12:10.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.543+0000 7f31f3e00640 1 --2- 172.21.15.50:0/1652135627 >> v2:172.21.15.88:3300/0 conn(0x7f31ec0b3340 0x7f31ec0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.543+0000 7f31f3e00640 1 --2- 172.21.15.50:0/1652135627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31ec0cefa0 0x7f31ec0cf3b0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.543+0000 7f31f3e00640 1 -- 172.21.15.50:0/1652135627 >> 172.21.15.50:0/1652135627 conn(0x7f31ec0b2040 msgr2=0x7f31ec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:10.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.543+0000 7f31f3e00640 1 -- 172.21.15.50:0/1652135627 shutdown_connections 2024-09-21T06:12:10.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.543+0000 7f31f3e00640 1 -- 172.21.15.50:0/1652135627 wait complete. 2024-09-21T06:12:10.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f3e00640 1 Processor -- start 2024-09-21T06:12:10.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f3e00640 1 -- start start 2024-09-21T06:12:10.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31ec0749a0 0x7f31ec0aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:10.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f3e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f31ec0b3340 0x7f31ec13c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:10.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ec0cefa0 0x7f31ec1f9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:10.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec169cd0 con 0x7f31ec0749a0 2024-09-21T06:12:10.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec169b50 con 0x7f31ec0cefa0 2024-09-21T06:12:10.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f3e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f31ec169e50 con 0x7f31ec0b3340 2024-09-21T06:12:10.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31ec0749a0 0x7f31ec0aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:10.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31ec0749a0 0x7f31ec0aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42646/0 (socket says 172.21.15.50:42646) 2024-09-21T06:12:10.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f1400640 1 -- 172.21.15.50:0/3219807452 learned_addr learned my addr 172.21.15.50:0/3219807452 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:10.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f0a00640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f31ec0b3340 0x7f31ec13c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:10.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f1e00640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ec0cefa0 0x7f31ec1f9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:10.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f1400640 1 -- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f31ec0b3340 msgr2=0x7f31ec13c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:10.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f1400640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f31ec0b3340 0x7f31ec13c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f1400640 1 -- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ec0cefa0 msgr2=0x7f31ec1f9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:10.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f1400640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ec0cefa0 0x7f31ec1f9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f1400640 1 -- 172.21.15.50:0/3219807452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ec0af710 con 0x7f31ec0749a0 2024-09-21T06:12:10.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31f1400640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31ec0749a0 0x7f31ec0aa4d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f31e0001ec0 tx=0x7f31e000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:10.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.547+0000 7f31e9600640 1 -- 172.21.15.50:0/3219807452 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0017070 con 0x7f31ec0749a0 2024-09-21T06:12:10.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.551+0000 7f31e9600640 1 -- 172.21.15.50:0/3219807452 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31e0003600 con 0x7f31ec0749a0 2024-09-21T06:12:10.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.551+0000 7f31e9600640 1 -- 172.21.15.50:0/3219807452 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0003a50 con 0x7f31ec0749a0 2024-09-21T06:12:10.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.551+0000 7f31f3e00640 1 -- 172.21.15.50:0/3219807452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ec0ab500 con 0x7f31ec0749a0 2024-09-21T06:12:10.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.551+0000 7f31f3e00640 1 -- 172.21.15.50:0/3219807452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ec13d100 con 0x7f31ec0749a0 2024-09-21T06:12:10.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.551+0000 7f31f3e00640 1 -- 172.21.15.50:0/3219807452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ec0b3750 con 0x7f31ec0749a0 2024-09-21T06:12:10.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.551+0000 7f31e9600640 1 -- 172.21.15.50:0/3219807452 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f31e001a070 con 0x7f31ec0749a0 2024-09-21T06:12:10.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.551+0000 7f31e9600640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f31cc0754e0 0x7f31cc0779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:10.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.551+0000 7f31e9600640 1 -- 172.21.15.50:0/3219807452 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5849+0+0 (secure 0 0 0) 0x7f31e00926a0 con 0x7f31ec0749a0 2024-09-21T06:12:10.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.551+0000 7f31f0a00640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f31cc0754e0 0x7f31cc0779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:10.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.555+0000 7f31f0a00640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f31cc0754e0 0x7f31cc0779f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f31d80af000 tx=0x7f31d801b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:10.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.555+0000 7f31e9600640 1 -- 172.21.15.50:0/3219807452 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f31e005eb70 con 0x7f31ec0749a0 2024-09-21T06:12:10.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.787+0000 7f31f3e00640 1 -- 172.21.15.50:0/3219807452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31ec1ef9e0 con 0x7f31ec0749a0 2024-09-21T06:12:10.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.787+0000 7f31e9600640 1 -- 172.21.15.50:0/3219807452 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5219 (secure 0 0 0) 0x7f31e00635a0 con 0x7f31ec0749a0 2024-09-21T06:12:10.787 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:10.787 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":85,"btime":"2024-09-21T06:11:23:123431+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2643543096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2643543096},{"type":"v1","addr":"172.21.15.88:6836","nonce":2643543096}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3618269169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3618269169},{"type":"v1","addr":"172.21.15.88:6837","nonce":3618269169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/66879433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":66879433},{"type":"v1","addr":"172.21.15.50:6835","nonce":66879433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:11:14.496744+0000","modified":"2024-09-21T06:11:17.524950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8832},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8832":{"gid":8832,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6833/1864490990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1864490990},{"type":"v1","addr":"172.21.15.50:6833","nonce":1864490990}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8832,"qdb_cluster":[8832]},"id":9}]} 2024-09-21T06:12:10.787 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 85 2024-09-21T06:12:10.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.791+0000 7f31f3e00640 1 -- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f31cc0754e0 msgr2=0x7f31cc0779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:10.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.791+0000 7f31f3e00640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f31cc0754e0 0x7f31cc0779f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f31d80af000 tx=0x7f31d801b100 comp rx=0 tx=0).stop 2024-09-21T06:12:10.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.791+0000 7f31f3e00640 1 -- 172.21.15.50:0/3219807452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31ec0749a0 msgr2=0x7f31ec0aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:10.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.791+0000 7f31f3e00640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31ec0749a0 0x7f31ec0aa4d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f31e0001ec0 tx=0x7f31e000c890 comp rx=0 tx=0).stop 2024-09-21T06:12:10.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.791+0000 7f31f3e00640 1 -- 172.21.15.50:0/3219807452 shutdown_connections 2024-09-21T06:12:10.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.791+0000 7f31f3e00640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ec0cefa0 0x7f31ec1f9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.791+0000 7f31f3e00640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f31ec0b3340 0x7f31ec13c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.791+0000 7f31f3e00640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f31cc0754e0 0x7f31cc0779f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.791+0000 7f31f3e00640 1 --2- 172.21.15.50:0/3219807452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31ec0749a0 0x7f31ec0aa4d0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.791+0000 7f31f3e00640 1 -- 172.21.15.50:0/3219807452 >> 172.21.15.50:0/3219807452 conn(0x7f31ec0b2040 msgr2=0x7f31ec06e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:10.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.791+0000 7f31f3e00640 1 -- 172.21.15.50:0/3219807452 shutdown_connections 2024-09-21T06:12:10.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.791+0000 7f31f3e00640 1 -- 172.21.15.50:0/3219807452 wait complete. 2024-09-21T06:12:10.810 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:12:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.971+0000 7f9dc9c00640 1 -- 172.21.15.50:0/2839350997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dc40cefa0 msgr2=0x7f9dc40cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.971+0000 7f9dc9c00640 1 --2- 172.21.15.50:0/2839350997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dc40cefa0 0x7f9dc40cf3b0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40cfc90 tx=0x7f9dac01c8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.971+0000 7f9dc9c00640 1 -- 172.21.15.50:0/2839350997 shutdown_connections 2024-09-21T06:12:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.971+0000 7f9dc9c00640 1 --2- 172.21.15.50:0/2839350997 >> v2:172.21.15.88:3300/0 conn(0x7f9dc40b3340 0x7f9dc40b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.971+0000 7f9dc9c00640 1 --2- 172.21.15.50:0/2839350997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dc40cefa0 0x7f9dc40cf3b0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.971+0000 7f9dc9c00640 1 -- 172.21.15.50:0/2839350997 >> 172.21.15.50:0/2839350997 conn(0x7f9dc40b2040 msgr2=0x7f9dc40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.971+0000 7f9dc9c00640 1 -- 172.21.15.50:0/2839350997 shutdown_connections 2024-09-21T06:12:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.971+0000 7f9dc9c00640 1 -- 172.21.15.50:0/2839350997 wait complete. 2024-09-21T06:12:10.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.971+0000 7f9dc9c00640 1 Processor -- start 2024-09-21T06:12:10.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.971+0000 7f9dc9c00640 1 -- start start 2024-09-21T06:12:10.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc9c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9dc40749a0 0x7f9dc413c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:10.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc9c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9dc40b3340 0x7f9dc40cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:10.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc9c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dc40cefa0 0x7f9dc40ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:10.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc9c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4169d20 con 0x7f9dc40cefa0 2024-09-21T06:12:10.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc9c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4169ba0 con 0x7f9dc40b3340 2024-09-21T06:12:10.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc9c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc4169ea0 con 0x7f9dc40749a0 2024-09-21T06:12:10.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dc40cefa0 0x7f9dc40ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:10.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dc40cefa0 0x7f9dc40ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42656/0 (socket says 172.21.15.50:42656) 2024-09-21T06:12:10.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc3400640 1 -- 172.21.15.50:0/4264489605 learned_addr learned my addr 172.21.15.50:0/4264489605 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:10.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dbbe00640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9dc40b3340 0x7f9dc40cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:10.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc2a00640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9dc40749a0 0x7f9dc413c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:10.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc3400640 1 -- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9dc40749a0 msgr2=0x7f9dc413c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:10.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc3400640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9dc40749a0 0x7f9dc413c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc3400640 1 -- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9dc40b3340 msgr2=0x7f9dc40cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:10.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc3400640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9dc40b3340 0x7f9dc40cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:10.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc3400640 1 -- 172.21.15.50:0/4264489605 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc414d8b0 con 0x7f9dc40cefa0 2024-09-21T06:12:10.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dbbe00640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9dc40b3340 0x7f9dc40cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:10.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc3400640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dc40cefa0 0x7f9dc40ce320 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00cae0 tx=0x7f9dbc00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:10.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dbaa00640 1 -- 172.21.15.50:0/4264489605 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc00ae30 con 0x7f9dc40cefa0 2024-09-21T06:12:10.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dbaa00640 1 -- 172.21.15.50:0/4264489605 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dbc00f560 con 0x7f9dc40cefa0 2024-09-21T06:12:10.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc9c00640 1 -- 172.21.15.50:0/4264489605 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc413d4e0 con 0x7f9dc40cefa0 2024-09-21T06:12:10.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dbaa00640 1 -- 172.21.15.50:0/4264489605 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc00f9b0 con 0x7f9dc40cefa0 2024-09-21T06:12:10.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc9c00640 1 -- 172.21.15.50:0/4264489605 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc40b02d0 con 0x7f9dc40cefa0 2024-09-21T06:12:10.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.975+0000 7f9dc9c00640 1 -- 172.21.15.50:0/4264489605 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc40b1cc0 con 0x7f9dc40cefa0 2024-09-21T06:12:10.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.979+0000 7f9dbaa00640 1 -- 172.21.15.50:0/4264489605 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f9dbc004b30 con 0x7f9dc40cefa0 2024-09-21T06:12:10.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.979+0000 7f9dbaa00640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9da00754e0 0x7f9da00779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:10.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.979+0000 7f9dbaa00640 1 -- 172.21.15.50:0/4264489605 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5849+0+0 (secure 0 0 0) 0x7f9dbc0940f0 con 0x7f9dc40cefa0 2024-09-21T06:12:10.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.979+0000 7f9dc2a00640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9da00754e0 0x7f9da00779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:10.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.983+0000 7f9dc2a00640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9da00754e0 0x7f9da00779f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9db400b810 tx=0x7f9db4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:10.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:10.983+0000 7f9dbaa00640 1 -- 172.21.15.50:0/4264489605 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9dbc014030 con 0x7f9dc40cefa0 2024-09-21T06:12:11.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.203+0000 7f9dc9c00640 1 -- 172.21.15.50:0/4264489605 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9dc40b3750 con 0x7f9dc40cefa0 2024-09-21T06:12:11.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.203+0000 7f9dbaa00640 1 -- 172.21.15.50:0/4264489605 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5219 (secure 0 0 0) 0x7f9dbc004e20 con 0x7f9dc40cefa0 2024-09-21T06:12:11.201 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:11.201 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":85,"btime":"2024-09-21T06:11:23:123431+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2643543096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2643543096},{"type":"v1","addr":"172.21.15.88:6836","nonce":2643543096}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3618269169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3618269169},{"type":"v1","addr":"172.21.15.88:6837","nonce":3618269169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/66879433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":66879433},{"type":"v1","addr":"172.21.15.50:6835","nonce":66879433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:11:14.496744+0000","modified":"2024-09-21T06:11:17.524950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8832},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8832":{"gid":8832,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6833/1864490990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1864490990},{"type":"v1","addr":"172.21.15.50:6833","nonce":1864490990}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8832,"qdb_cluster":[8832]},"id":9}]} 2024-09-21T06:12:11.202 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 85 2024-09-21T06:12:11.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.207+0000 7f9dc9c00640 1 -- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9da00754e0 msgr2=0x7f9da00779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.207+0000 7f9dc9c00640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9da00754e0 0x7f9da00779f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9db400b810 tx=0x7f9db4008040 comp rx=0 tx=0).stop 2024-09-21T06:12:11.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.207+0000 7f9dc9c00640 1 -- 172.21.15.50:0/4264489605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dc40cefa0 msgr2=0x7f9dc40ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.207+0000 7f9dc9c00640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dc40cefa0 0x7f9dc40ce320 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00cae0 tx=0x7f9dbc00a250 comp rx=0 tx=0).stop 2024-09-21T06:12:11.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.207+0000 7f9dc9c00640 1 -- 172.21.15.50:0/4264489605 shutdown_connections 2024-09-21T06:12:11.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.207+0000 7f9dc9c00640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9da00754e0 0x7f9da00779f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.207+0000 7f9dc9c00640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dc40cefa0 0x7f9dc40ce320 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.207+0000 7f9dc9c00640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9dc40b3340 0x7f9dc40cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.207+0000 7f9dc9c00640 1 --2- 172.21.15.50:0/4264489605 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9dc40749a0 0x7f9dc413c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.207+0000 7f9dc9c00640 1 -- 172.21.15.50:0/4264489605 >> 172.21.15.50:0/4264489605 conn(0x7f9dc40b2040 msgr2=0x7f9dc4074db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:11.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.207+0000 7f9dc9c00640 1 -- 172.21.15.50:0/4264489605 shutdown_connections 2024-09-21T06:12:11.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.207+0000 7f9dc9c00640 1 -- 172.21.15.50:0/4264489605 wait complete. 2024-09-21T06:12:11.224 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:12:11.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.383+0000 7f7a1e000640 1 -- 172.21.15.50:0/585988792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a181527b0 msgr2=0x7f7a18074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.383+0000 7f7a1e000640 1 --2- 172.21.15.50:0/585988792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a181527b0 0x7f7a18074e30 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7a18071cb0 tx=0x7f7a0001c890 comp rx=0 tx=0).stop 2024-09-21T06:12:11.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.387+0000 7f7a1e000640 1 -- 172.21.15.50:0/585988792 shutdown_connections 2024-09-21T06:12:11.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.387+0000 7f7a1e000640 1 --2- 172.21.15.50:0/585988792 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a180b4de0 0x7f7a180b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.387+0000 7f7a1e000640 1 --2- 172.21.15.50:0/585988792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a181527b0 0x7f7a18074e30 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.387+0000 7f7a1e000640 1 --2- 172.21.15.50:0/585988792 >> v2:172.21.15.88:3300/0 conn(0x7f7a180b5c70 0x7f7a1815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.387+0000 7f7a1e000640 1 -- 172.21.15.50:0/585988792 >> 172.21.15.50:0/585988792 conn(0x7f7a180b2040 msgr2=0x7f7a180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:11.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.387+0000 7f7a1e000640 1 -- 172.21.15.50:0/585988792 shutdown_connections 2024-09-21T06:12:11.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.387+0000 7f7a1e000640 1 -- 172.21.15.50:0/585988792 wait complete. 2024-09-21T06:12:11.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.387+0000 7f7a1e000640 1 Processor -- start 2024-09-21T06:12:11.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.387+0000 7f7a1e000640 1 -- start start 2024-09-21T06:12:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a1e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a180b4de0 0x7f7a180cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a1e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a180b5c70 0x7f7a1814c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a1e000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a181527b0 0x7f7a18150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a1e000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1815f6b0 con 0x7f7a180b5c70 2024-09-21T06:12:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a1e000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1815f530 con 0x7f7a180b4de0 2024-09-21T06:12:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a1e000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1815f830 con 0x7f7a181527b0 2024-09-21T06:12:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a180b5c70 0x7f7a1814c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a180b5c70 0x7f7a1814c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42670/0 (socket says 172.21.15.50:42670) 2024-09-21T06:12:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a0fe00640 1 -- 172.21.15.50:0/3109999517 learned_addr learned my addr 172.21.15.50:0/3109999517 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a17400640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a180b4de0 0x7f7a180cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a17e00640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a181527b0 0x7f7a18150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a0fe00640 1 -- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a181527b0 msgr2=0x7f7a18150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a0fe00640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a181527b0 0x7f7a18150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a0fe00640 1 -- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a180b4de0 msgr2=0x7f7a180cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a0fe00640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a180b4de0 0x7f7a180cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a0fe00640 1 -- 172.21.15.50:0/3109999517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a181581c0 con 0x7f7a180b5c70 2024-09-21T06:12:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a17e00640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a181527b0 0x7f7a18150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a0fe00640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a180b5c70 0x7f7a1814c870 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7a0001af40 tx=0x7f7a00004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:11.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a0f400640 1 -- 172.21.15.50:0/3109999517 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a000b2040 con 0x7f7a180b5c70 2024-09-21T06:12:11.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a0f400640 1 -- 172.21.15.50:0/3109999517 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a00005030 con 0x7f7a180b5c70 2024-09-21T06:12:11.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a0f400640 1 -- 172.21.15.50:0/3109999517 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00005460 con 0x7f7a180b5c70 2024-09-21T06:12:11.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a1e000640 1 -- 172.21.15.50:0/3109999517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a181fa750 con 0x7f7a180b5c70 2024-09-21T06:12:11.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a1e000640 1 -- 172.21.15.50:0/3109999517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a181faa30 con 0x7f7a180b5c70 2024-09-21T06:12:11.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.391+0000 7f7a1e000640 1 -- 172.21.15.50:0/3109999517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1815fd50 con 0x7f7a180b5c70 2024-09-21T06:12:11.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.395+0000 7f7a0f400640 1 -- 172.21.15.50:0/3109999517 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f7a000be060 con 0x7f7a180b5c70 2024-09-21T06:12:11.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.395+0000 7f7a0f400640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79f40754e0 0x7f79f40779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.395+0000 7f7a0f400640 1 -- 172.21.15.50:0/3109999517 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5849+0+0 (secure 0 0 0) 0x7f7a0012c930 con 0x7f7a180b5c70 2024-09-21T06:12:11.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.395+0000 7f7a17400640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79f40754e0 0x7f79f40779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.395+0000 7f7a17400640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79f40754e0 0x7f79f40779f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800bb50 tx=0x7f7a0800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:11.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.395+0000 7f7a0f400640 1 -- 172.21.15.50:0/3109999517 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7a000164b0 con 0x7f7a180b5c70 2024-09-21T06:12:11.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 -- 172.21.15.50:0/3109999517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a18151b50 con 0x7f7a180b5c70 2024-09-21T06:12:11.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a0f400640 1 -- 172.21.15.50:0/3109999517 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16990 (secure 0 0 0) 0x7f7a000f94c0 con 0x7f7a180b5c70 2024-09-21T06:12:11.577 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:11.577 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":129,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:12:11.504704+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:11:12.241790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T06:11:13.462910+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":128,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.21","mappings":[{"from":2,"to":0}]},{"pgid":"19.32","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:12:11.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 -- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79f40754e0 msgr2=0x7f79f40779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79f40754e0 0x7f79f40779f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800bb50 tx=0x7f7a0800b2e0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 -- 172.21.15.50:0/3109999517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a180b5c70 msgr2=0x7f7a1814c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a180b5c70 0x7f7a1814c870 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7a0001af40 tx=0x7f7a00004040 comp rx=0 tx=0).stop 2024-09-21T06:12:11.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 -- 172.21.15.50:0/3109999517 shutdown_connections 2024-09-21T06:12:11.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f7a181527b0 0x7f7a18150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f79f40754e0 0x7f79f40779f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a180b5c70 0x7f7a1814c870 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 --2- 172.21.15.50:0/3109999517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a180b4de0 0x7f7a180cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 -- 172.21.15.50:0/3109999517 >> 172.21.15.50:0/3109999517 conn(0x7f7a180b2040 msgr2=0x7f7a180b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:11.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 -- 172.21.15.50:0/3109999517 shutdown_connections 2024-09-21T06:12:11.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.579+0000 7f7a1e000640 1 -- 172.21.15.50:0/3109999517 wait complete. 2024-09-21T06:12:11.600 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:12:11.600 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:12:11.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.763+0000 7f75c6800640 1 -- 172.21.15.50:0/1645302151 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75c00b5c70 msgr2=0x7f75c015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.763+0000 7f75c6800640 1 --2- 172.21.15.50:0/1645302151 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75c00b5c70 0x7f75c015bb60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009bc0 tx=0x7f75b001cae0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.763+0000 7f75c6800640 1 -- 172.21.15.50:0/1645302151 shutdown_connections 2024-09-21T06:12:11.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.763+0000 7f75c6800640 1 --2- 172.21.15.50:0/1645302151 >> v2:172.21.15.88:3300/0 conn(0x7f75c00b4de0 0x7f75c00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.763+0000 7f75c6800640 1 --2- 172.21.15.50:0/1645302151 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75c00b5c70 0x7f75c015bb60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.763+0000 7f75c6800640 1 -- 172.21.15.50:0/1645302151 >> 172.21.15.50:0/1645302151 conn(0x7f75c00b2040 msgr2=0x7f75c00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:11.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.763+0000 7f75c6800640 1 -- 172.21.15.50:0/1645302151 shutdown_connections 2024-09-21T06:12:11.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.763+0000 7f75c6800640 1 -- 172.21.15.50:0/1645302151 wait complete. 2024-09-21T06:12:11.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.763+0000 7f75c6800640 1 Processor -- start 2024-09-21T06:12:11.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.763+0000 7f75c6800640 1 -- start start 2024-09-21T06:12:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75c6800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75c00b4de0 0x7f75c00b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75c6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75c00b5c70 0x7f75c0141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75c6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c01527b0 0x7f75c0158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75c6800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f75c0168ab0 con 0x7f75c00b5c70 2024-09-21T06:12:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75c6800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f75c0168930 con 0x7f75c01527b0 2024-09-21T06:12:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75c6800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f75c0168c30 con 0x7f75c00b4de0 2024-09-21T06:12:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75b6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75c00b5c70 0x7f75c0141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75bf400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75c00b4de0 0x7f75c00b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75b6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75c00b5c70 0x7f75c0141da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42680/0 (socket says 172.21.15.50:42680) 2024-09-21T06:12:11.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75b6a00640 1 -- 172.21.15.50:0/1294669931 learned_addr learned my addr 172.21.15.50:0/1294669931 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75bfe00640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c01527b0 0x7f75c0158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75bf400640 1 -- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c01527b0 msgr2=0x7f75c0158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75bf400640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c01527b0 0x7f75c0158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75bf400640 1 -- 172.21.15.50:0/1294669931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75c00b5c70 msgr2=0x7f75c0141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75bf400640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75c00b5c70 0x7f75c0141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75bf400640 1 -- 172.21.15.50:0/1294669931 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c01579b0 con 0x7f75c00b4de0 2024-09-21T06:12:11.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75bf400640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75c00b4de0 0x7f75c00b4890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f75c0074b20 tx=0x7f75b0009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75b7e00640 1 -- 172.21.15.50:0/1294669931 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b00bb070 con 0x7f75c00b4de0 2024-09-21T06:12:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75b7e00640 1 -- 172.21.15.50:0/1294669931 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75b00a5e50 con 0x7f75c00b4de0 2024-09-21T06:12:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75b7e00640 1 -- 172.21.15.50:0/1294669931 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b00b2040 con 0x7f75c00b4de0 2024-09-21T06:12:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75c6800640 1 -- 172.21.15.50:0/1294669931 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c0157c90 con 0x7f75c00b4de0 2024-09-21T06:12:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75c6800640 1 -- 172.21.15.50:0/1294669931 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c0158250 con 0x7f75c00b4de0 2024-09-21T06:12:11.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.767+0000 7f75c6800640 1 -- 172.21.15.50:0/1294669931 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75c00b51f0 con 0x7f75c00b4de0 2024-09-21T06:12:11.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.771+0000 7f75b7e00640 1 -- 172.21.15.50:0/1294669931 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f75b0004000 con 0x7f75c00b4de0 2024-09-21T06:12:11.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.771+0000 7f75b7e00640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f759c075510 0x7f759c077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.771+0000 7f75b7e00640 1 -- 172.21.15.50:0/1294669931 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(129..129 src has 1..129) ==== 5879+0+0 (secure 0 0 0) 0x7f75b012d500 con 0x7f75c00b4de0 2024-09-21T06:12:11.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.771+0000 7f75b6a00640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f759c075510 0x7f759c077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.771+0000 7f75b6a00640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f759c075510 0x7f759c077a20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f75c00b4820 tx=0x7f75a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:11.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.775+0000 7f75b7e00640 1 -- 172.21.15.50:0/1294669931 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f75b00c4020 con 0x7f75c00b4de0 2024-09-21T06:12:11.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.987+0000 7f75c6800640 1 -- 172.21.15.50:0/1294669931 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75c015bb00 con 0x7f75c00b4de0 2024-09-21T06:12:11.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.987+0000 7f75b7e00640 1 -- 172.21.15.50:0/1294669931 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5219 (secure 0 0 0) 0x7f75b0016790 con 0x7f75c00b4de0 2024-09-21T06:12:11.987 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:11.987 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":85,"btime":"2024-09-21T06:11:23:123431+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2643543096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2643543096},{"type":"v1","addr":"172.21.15.88:6836","nonce":2643543096}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3618269169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3618269169},{"type":"v1","addr":"172.21.15.88:6837","nonce":3618269169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/66879433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":66879433},{"type":"v1","addr":"172.21.15.50:6835","nonce":66879433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:11:14.496744+0000","modified":"2024-09-21T06:11:17.524950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8832},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8832":{"gid":8832,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6833/1864490990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1864490990},{"type":"v1","addr":"172.21.15.50:6833","nonce":1864490990}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8832,"qdb_cluster":[8832]},"id":9}]} 2024-09-21T06:12:11.987 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 85 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.991+0000 7f75c6800640 1 -- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f759c075510 msgr2=0x7f759c077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.991+0000 7f75c6800640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f759c075510 0x7f759c077a20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f75c00b4820 tx=0x7f75a8009040 comp rx=0 tx=0).stop 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.991+0000 7f75c6800640 1 -- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75c00b4de0 msgr2=0x7f75c00b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.991+0000 7f75c6800640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75c00b4de0 0x7f75c00b4890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f75c0074b20 tx=0x7f75b0009280 comp rx=0 tx=0).stop 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.991+0000 7f75c6800640 1 -- 172.21.15.50:0/1294669931 shutdown_connections 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.991+0000 7f75c6800640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c01527b0 0x7f75c0158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.991+0000 7f75c6800640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f759c075510 0x7f759c077a20 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.991+0000 7f75c6800640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75c00b5c70 0x7f75c0141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.991+0000 7f75c6800640 1 --2- 172.21.15.50:0/1294669931 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f75c00b4de0 0x7f75c00b4890 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.991+0000 7f75c6800640 1 -- 172.21.15.50:0/1294669931 >> 172.21.15.50:0/1294669931 conn(0x7f75c00b2040 msgr2=0x7f75c00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:11.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.991+0000 7f75c6800640 1 -- 172.21.15.50:0/1294669931 shutdown_connections 2024-09-21T06:12:11.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:11.991+0000 7f75c6800640 1 -- 172.21.15.50:0/1294669931 wait complete. 2024-09-21T06:12:12.009 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:12:12.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.171+0000 7fac06000640 1 -- 172.21.15.50:0/3647492384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac000b5c70 msgr2=0x7fac0015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.171+0000 7fac06000640 1 --2- 172.21.15.50:0/3647492384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac000b5c70 0x7fac0015bb60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009900 tx=0x7fabf001c820 comp rx=0 tx=0).stop 2024-09-21T06:12:12.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.171+0000 7fac06000640 1 -- 172.21.15.50:0/3647492384 shutdown_connections 2024-09-21T06:12:12.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.171+0000 7fac06000640 1 --2- 172.21.15.50:0/3647492384 >> v2:172.21.15.88:3300/0 conn(0x7fac001527b0 0x7fac00074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.171+0000 7fac06000640 1 --2- 172.21.15.50:0/3647492384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac000b5c70 0x7fac0015bb60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.171+0000 7fac06000640 1 -- 172.21.15.50:0/3647492384 >> 172.21.15.50:0/3647492384 conn(0x7fac000b2040 msgr2=0x7fac000b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:12.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.171+0000 7fac06000640 1 -- 172.21.15.50:0/3647492384 shutdown_connections 2024-09-21T06:12:12.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.171+0000 7fac06000640 1 -- 172.21.15.50:0/3647492384 wait complete. 2024-09-21T06:12:12.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fac06000640 1 Processor -- start 2024-09-21T06:12:12.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fac06000640 1 -- start start 2024-09-21T06:12:12.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fac06000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac000b4de0 0x7fac001f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fac06000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac000b5c70 0x7fac001589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fac06000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac001527b0 0x7fac00158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fac06000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fac00168c50 con 0x7fac000b5c70 2024-09-21T06:12:12.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fac06000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fac00168ad0 con 0x7fac000b4de0 2024-09-21T06:12:12.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fac06000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fac00168dd0 con 0x7fac001527b0 2024-09-21T06:12:12.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac000b5c70 0x7fac001589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabff400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac000b4de0 0x7fac001f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabffe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac001527b0 0x7fac00158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac000b5c70 0x7fac001589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42682/0 (socket says 172.21.15.50:42682) 2024-09-21T06:12:12.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabf7e00640 1 -- 172.21.15.50:0/3660581988 learned_addr learned my addr 172.21.15.50:0/3660581988 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:12.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabff400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac000b4de0 0x7fac001f93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:44676/0 (socket says 172.21.15.50:44676) 2024-09-21T06:12:12.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabf7e00640 1 -- 172.21.15.50:0/3660581988 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac001527b0 msgr2=0x7fac00158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabf7e00640 1 --2- 172.21.15.50:0/3660581988 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac001527b0 0x7fac00158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabf7e00640 1 -- 172.21.15.50:0/3660581988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac000b4de0 msgr2=0x7fac001f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabf7e00640 1 --2- 172.21.15.50:0/3660581988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac000b4de0 0x7fac001f93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabf7e00640 1 -- 172.21.15.50:0/3660581988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac001f6e50 con 0x7fac000b5c70 2024-09-21T06:12:12.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabf7e00640 1 --2- 172.21.15.50:0/3660581988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac000b5c70 0x7fac001589b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fabe800e9f0 tx=0x7fabe800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:12.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabf7400640 1 -- 172.21.15.50:0/3660581988 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe800cd80 con 0x7fac000b5c70 2024-09-21T06:12:12.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabf7400640 1 -- 172.21.15.50:0/3660581988 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabe8004020 con 0x7fac000b5c70 2024-09-21T06:12:12.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fabf7400640 1 -- 172.21.15.50:0/3660581988 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8004450 con 0x7fac000b5c70 2024-09-21T06:12:12.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fac06000640 1 -- 172.21.15.50:0/3660581988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac00156360 con 0x7fac000b5c70 2024-09-21T06:12:12.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.175+0000 7fac06000640 1 -- 172.21.15.50:0/3660581988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac00156920 con 0x7fac000b5c70 2024-09-21T06:12:12.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.179+0000 7fac06000640 1 -- 172.21.15.50:0/3660581988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac001f70c0 con 0x7fac000b5c70 2024-09-21T06:12:12.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.179+0000 7fabf7400640 1 -- 172.21.15.50:0/3660581988 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fabe80047b0 con 0x7fac000b5c70 2024-09-21T06:12:12.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.179+0000 7fabf7400640 1 --2- 172.21.15.50:0/3660581988 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabdc0754e0 0x7fabdc0779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.179+0000 7fabf7400640 1 -- 172.21.15.50:0/3660581988 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5879+0+0 (secure 0 0 0) 0x7fabe8093ba0 con 0x7fac000b5c70 2024-09-21T06:12:12.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.179+0000 7fabff400640 1 --2- 172.21.15.50:0/3660581988 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabdc0754e0 0x7fabdc0779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.183+0000 7fabff400640 1 --2- 172.21.15.50:0/3660581988 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabdc0754e0 0x7fabdc0779f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fabf00af000 tx=0x7fabf001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:12.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.183+0000 7fabf7400640 1 -- 172.21.15.50:0/3660581988 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fabe8060160 con 0x7fac000b5c70 2024-09-21T06:12:12.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.395+0000 7fac06000640 1 -- 172.21.15.50:0/3660581988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac0015bb60 con 0x7fac000b5c70 2024-09-21T06:12:12.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.395+0000 7fabf7400640 1 -- 172.21.15.50:0/3660581988 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5219 (secure 0 0 0) 0x7fabe8064b90 con 0x7fac000b5c70 2024-09-21T06:12:12.394 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:12.394 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":85,"btime":"2024-09-21T06:11:23:123431+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2643543096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2643543096},{"type":"v1","addr":"172.21.15.88:6836","nonce":2643543096}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3618269169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3618269169},{"type":"v1","addr":"172.21.15.88:6837","nonce":3618269169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/66879433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":66879433},{"type":"v1","addr":"172.21.15.50:6835","nonce":66879433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:11:14.496744+0000","modified":"2024-09-21T06:11:17.524950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8832},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8832":{"gid":8832,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6833/1864490990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1864490990},{"type":"v1","addr":"172.21.15.50:6833","nonce":1864490990}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8832,"qdb_cluster":[8832]},"id":9}]} 2024-09-21T06:12:12.394 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 85 2024-09-21T06:12:12.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.399+0000 7fac06000640 1 -- 172.21.15.50:0/3660581988 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabdc0754e0 msgr2=0x7fabdc0779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.399+0000 7fac06000640 1 --2- 172.21.15.50:0/3660581988 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabdc0754e0 0x7fabdc0779f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fabf00af000 tx=0x7fabf001b060 comp rx=0 tx=0).stop 2024-09-21T06:12:12.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.399+0000 7fac06000640 1 -- 172.21.15.50:0/3660581988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac000b5c70 msgr2=0x7fac001589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.399+0000 7fac06000640 1 --2- 172.21.15.50:0/3660581988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac000b5c70 0x7fac001589b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fabe800e9f0 tx=0x7fabe800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.399+0000 7fac06000640 1 -- 172.21.15.50:0/3660581988 shutdown_connections 2024-09-21T06:12:12.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.399+0000 7fac06000640 1 --2- 172.21.15.50:0/3660581988 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fac001527b0 0x7fac00158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.399+0000 7fac06000640 1 --2- 172.21.15.50:0/3660581988 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fabdc0754e0 0x7fabdc0779f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.399+0000 7fac06000640 1 --2- 172.21.15.50:0/3660581988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac000b5c70 0x7fac001589b0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.399+0000 7fac06000640 1 --2- 172.21.15.50:0/3660581988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac000b4de0 0x7fac001f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.399+0000 7fac06000640 1 -- 172.21.15.50:0/3660581988 >> 172.21.15.50:0/3660581988 conn(0x7fac000b2040 msgr2=0x7fac0015a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:12.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.399+0000 7fac06000640 1 -- 172.21.15.50:0/3660581988 shutdown_connections 2024-09-21T06:12:12.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.399+0000 7fac06000640 1 -- 172.21.15.50:0/3660581988 wait complete. 2024-09-21T06:12:12.416 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:12:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.583+0000 7f0f9c200640 1 -- 172.21.15.50:0/682739339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f94074b20 msgr2=0x7f0f9415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.583+0000 7f0f9c200640 1 --2- 172.21.15.50:0/682739339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f94074b20 0x7f0f9415cf90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009bc0 tx=0x7f0f8c01cb10 comp rx=0 tx=0).stop 2024-09-21T06:12:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.583+0000 7f0f9c200640 1 -- 172.21.15.50:0/682739339 shutdown_connections 2024-09-21T06:12:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.583+0000 7f0f9c200640 1 --2- 172.21.15.50:0/682739339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f94074b20 0x7f0f9415cf90 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.583+0000 7f0f9c200640 1 -- 172.21.15.50:0/682739339 >> 172.21.15.50:0/682739339 conn(0x7f0f940708c0 msgr2=0x7f0f94070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.583+0000 7f0f9c200640 1 -- 172.21.15.50:0/682739339 shutdown_connections 2024-09-21T06:12:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.583+0000 7f0f9c200640 1 -- 172.21.15.50:0/682739339 wait complete. 2024-09-21T06:12:12.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.583+0000 7f0f9c200640 1 Processor -- start 2024-09-21T06:12:12.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.583+0000 7f0f9c200640 1 -- start start 2024-09-21T06:12:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f9c200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f940b5a80 0x7f0f9413fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f9c200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f94157030 0x7f0f94161e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f9c200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f941fc520 0x7f0f941623d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f9c200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9416b4e0 con 0x7f0f940b5a80 2024-09-21T06:12:12.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f9c200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9416b360 con 0x7f0f94157030 2024-09-21T06:12:12.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f9c200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9416b660 con 0x7f0f941fc520 2024-09-21T06:12:12.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f99800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f940b5a80 0x7f0f9413fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f99800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f940b5a80 0x7f0f9413fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42686/0 (socket says 172.21.15.50:42686) 2024-09-21T06:12:12.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f99800640 1 -- 172.21.15.50:0/1474649289 learned_addr learned my addr 172.21.15.50:0/1474649289 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f99800640 1 -- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f941fc520 msgr2=0x7f0f941623d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f99800640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f941fc520 0x7f0f941623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f99800640 1 -- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f94157030 msgr2=0x7f0f94161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f98e00640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f94157030 0x7f0f94161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f99800640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f94157030 0x7f0f94161e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f99800640 1 -- 172.21.15.50:0/1474649289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f94152290 con 0x7f0f940b5a80 2024-09-21T06:12:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f98e00640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f94157030 0x7f0f94161e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f99800640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f940b5a80 0x7f0f9413fd30 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800c400 tx=0x7f0f8800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f91600640 1 -- 172.21.15.50:0/1474649289 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88011790 con 0x7f0f940b5a80 2024-09-21T06:12:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f91600640 1 -- 172.21.15.50:0/1474649289 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f88017070 con 0x7f0f940b5a80 2024-09-21T06:12:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f91600640 1 -- 172.21.15.50:0/1474649289 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88011ca0 con 0x7f0f940b5a80 2024-09-21T06:12:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f9c200640 1 -- 172.21.15.50:0/1474649289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f941fcc90 con 0x7f0f940b5a80 2024-09-21T06:12:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f9c200640 1 -- 172.21.15.50:0/1474649289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f941fd250 con 0x7f0f940b5a80 2024-09-21T06:12:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.587+0000 7f0f9c200640 1 -- 172.21.15.50:0/1474649289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f941fd450 con 0x7f0f940b5a80 2024-09-21T06:12:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.591+0000 7f0f91600640 1 -- 172.21.15.50:0/1474649289 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f0f8801c460 con 0x7f0f940b5a80 2024-09-21T06:12:12.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.591+0000 7f0f91600640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f740754e0 0x7f0f740779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.591+0000 7f0f91600640 1 -- 172.21.15.50:0/1474649289 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5879+0+0 (secure 0 0 0) 0x7f0f88093100 con 0x7f0f940b5a80 2024-09-21T06:12:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.591+0000 7f0f98e00640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f740754e0 0x7f0f740779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.591+0000 7f0f98e00640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f740754e0 0x7f0f740779f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0f94168740 tx=0x7f0f80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.595+0000 7f0f91600640 1 -- 172.21.15.50:0/1474649289 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0f8805fcb0 con 0x7f0f940b5a80 2024-09-21T06:12:12.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.771+0000 7f0f9c200640 1 -- 172.21.15.50:0/1474649289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f940d2260 con 0x7f0f940b5a80 2024-09-21T06:12:12.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.771+0000 7f0f91600640 1 -- 172.21.15.50:0/1474649289 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16990 (secure 0 0 0) 0x7f0f8805f5b0 con 0x7f0f940b5a80 2024-09-21T06:12:12.771 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:12.771 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":130,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:12:12.515632+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:11:12.241790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T06:11:13.462910+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":128,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.21","mappings":[{"from":2,"to":0}]},{"pgid":"19.32","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:12:12.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.775+0000 7f0f9c200640 1 -- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f740754e0 msgr2=0x7f0f740779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.775+0000 7f0f9c200640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f740754e0 0x7f0f740779f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0f94168740 tx=0x7f0f80009040 comp rx=0 tx=0).stop 2024-09-21T06:12:12.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.775+0000 7f0f9c200640 1 -- 172.21.15.50:0/1474649289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f940b5a80 msgr2=0x7f0f9413fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.775+0000 7f0f9c200640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f940b5a80 0x7f0f9413fd30 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800c400 tx=0x7f0f8800c8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.775+0000 7f0f9c200640 1 -- 172.21.15.50:0/1474649289 shutdown_connections 2024-09-21T06:12:12.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.775+0000 7f0f9c200640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0f941fc520 0x7f0f941623d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.775+0000 7f0f9c200640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f94157030 0x7f0f94161e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.775+0000 7f0f9c200640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0f740754e0 0x7f0f740779f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.775+0000 7f0f9c200640 1 --2- 172.21.15.50:0/1474649289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f940b5a80 0x7f0f9413fd30 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.775+0000 7f0f9c200640 1 -- 172.21.15.50:0/1474649289 >> 172.21.15.50:0/1474649289 conn(0x7f0f940708c0 msgr2=0x7f0f94070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:12.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.775+0000 7f0f9c200640 1 -- 172.21.15.50:0/1474649289 shutdown_connections 2024-09-21T06:12:12.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.775+0000 7f0f9c200640 1 -- 172.21.15.50:0/1474649289 wait complete. 2024-09-21T06:12:12.794 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:12:12.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.955+0000 7f6da5e00640 1 -- 172.21.15.50:0/1563166780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da01602a0 msgr2=0x7f6da01606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.955+0000 7f6da5e00640 1 --2- 172.21.15.50:0/1563166780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da01602a0 0x7f6da01606b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f6da0160da0 tx=0x7f6d8801c8a0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.955+0000 7f6da5e00640 1 -- 172.21.15.50:0/1563166780 shutdown_connections 2024-09-21T06:12:12.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.955+0000 7f6da5e00640 1 --2- 172.21.15.50:0/1563166780 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6da015cd20 0x7f6da015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.955+0000 7f6da5e00640 1 --2- 172.21.15.50:0/1563166780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da01602a0 0x7f6da01606b0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.955+0000 7f6da5e00640 1 --2- 172.21.15.50:0/1563166780 >> v2:172.21.15.88:3300/0 conn(0x7f6da00cd2e0 0x7f6da0070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.955+0000 7f6da5e00640 1 -- 172.21.15.50:0/1563166780 >> 172.21.15.50:0/1563166780 conn(0x7f6da00b2c40 msgr2=0x7f6da00b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:12.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.955+0000 7f6da5e00640 1 -- 172.21.15.50:0/1563166780 shutdown_connections 2024-09-21T06:12:12.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.955+0000 7f6da5e00640 1 -- 172.21.15.50:0/1563166780 wait complete. 2024-09-21T06:12:12.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6da5e00640 1 Processor -- start 2024-09-21T06:12:12.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6da5e00640 1 -- start start 2024-09-21T06:12:12.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6da5e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6da00cd2e0 0x7f6da0166a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6da5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da015cd20 0x7f6da0164b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6da5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6da01602a0 0x7f6da0200590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6da5e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6da00cffa0 con 0x7f6da015cd20 2024-09-21T06:12:12.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6da5e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6da00cfe20 con 0x7f6da01602a0 2024-09-21T06:12:12.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6da5e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6da00d0120 con 0x7f6da00cd2e0 2024-09-21T06:12:12.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d97e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da015cd20 0x7f6da0164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d97e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da015cd20 0x7f6da0164b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42706/0 (socket says 172.21.15.50:42706) 2024-09-21T06:12:12.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d97e00640 1 -- 172.21.15.50:0/2008459817 learned_addr learned my addr 172.21.15.50:0/2008459817 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:12.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d9ea00640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6da00cd2e0 0x7f6da0166a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d9f400640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6da01602a0 0x7f6da0200590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d97e00640 1 -- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6da00cd2e0 msgr2=0x7f6da0166a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d97e00640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6da00cd2e0 0x7f6da0166a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d97e00640 1 -- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6da01602a0 msgr2=0x7f6da0200590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d97e00640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6da01602a0 0x7f6da0200590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d97e00640 1 -- 172.21.15.50:0/2008459817 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da0165080 con 0x7f6da015cd20 2024-09-21T06:12:12.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d97e00640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da015cd20 0x7f6da0164b40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f6d880b0040 tx=0x7f6d880a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:12.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d97400640 1 -- 172.21.15.50:0/2008459817 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d880b2040 con 0x7f6da015cd20 2024-09-21T06:12:12.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d97400640 1 -- 172.21.15.50:0/2008459817 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d880044b0 con 0x7f6da015cd20 2024-09-21T06:12:12.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6da5e00640 1 -- 172.21.15.50:0/2008459817 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da014f8a0 con 0x7f6da015cd20 2024-09-21T06:12:12.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6d97400640 1 -- 172.21.15.50:0/2008459817 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88003860 con 0x7f6da015cd20 2024-09-21T06:12:12.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.959+0000 7f6da5e00640 1 -- 172.21.15.50:0/2008459817 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da00b4c10 con 0x7f6da015cd20 2024-09-21T06:12:12.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.963+0000 7f6da5e00640 1 -- 172.21.15.50:0/2008459817 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da014f610 con 0x7f6da015cd20 2024-09-21T06:12:12.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.963+0000 7f6d97400640 1 -- 172.21.15.50:0/2008459817 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f6d880bd060 con 0x7f6da015cd20 2024-09-21T06:12:12.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.963+0000 7f6d97400640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d7c0754e0 0x7f6d7c0779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.963+0000 7f6d97400640 1 -- 172.21.15.50:0/2008459817 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5879+0+0 (secure 0 0 0) 0x7f6d8812c890 con 0x7f6da015cd20 2024-09-21T06:12:12.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.963+0000 7f6d9ea00640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d7c0754e0 0x7f6d7c0779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.967+0000 7f6d9ea00640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d7c0754e0 0x7f6d7c0779f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000ab40 tx=0x7f6d9000a250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:12.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:12.967+0000 7f6d97400640 1 -- 172.21.15.50:0/2008459817 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6d88016490 con 0x7f6da015cd20 2024-09-21T06:12:13.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.187+0000 7f6da5e00640 1 -- 172.21.15.50:0/2008459817 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6da015d130 con 0x7f6da015cd20 2024-09-21T06:12:13.184 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-09-21T06:12:13.187+0000 7f0d45000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:12:13.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.187+0000 7f6d97400640 1 -- 172.21.15.50:0/2008459817 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) ==== 161+0+0 (secure 0 0 0) 0x7f6d880f8ed0 con 0x7f6da015cd20 2024-09-21T06:12:13.186 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:12:13.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.191+0000 7f6da5e00640 1 -- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d7c0754e0 msgr2=0x7f6d7c0779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:13.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.191+0000 7f6da5e00640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d7c0754e0 0x7f6d7c0779f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000ab40 tx=0x7f6d9000a250 comp rx=0 tx=0).stop 2024-09-21T06:12:13.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.191+0000 7f6da5e00640 1 -- 172.21.15.50:0/2008459817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da015cd20 msgr2=0x7f6da0164b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:13.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.191+0000 7f6da5e00640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da015cd20 0x7f6da0164b40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f6d880b0040 tx=0x7f6d880a5770 comp rx=0 tx=0).stop 2024-09-21T06:12:13.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.191+0000 7f6da5e00640 1 -- 172.21.15.50:0/2008459817 shutdown_connections 2024-09-21T06:12:13.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.191+0000 7f6da5e00640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6da01602a0 0x7f6da0200590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:13.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.191+0000 7f6da5e00640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d7c0754e0 0x7f6d7c0779f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:13.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.191+0000 7f6da5e00640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da015cd20 0x7f6da0164b40 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:13.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.191+0000 7f6da5e00640 1 --2- 172.21.15.50:0/2008459817 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6da00cd2e0 0x7f6da0166a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:13.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.191+0000 7f6da5e00640 1 -- 172.21.15.50:0/2008459817 >> 172.21.15.50:0/2008459817 conn(0x7f6da00b2c40 msgr2=0x7f6da0070340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:13.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.191+0000 7f6da5e00640 1 -- 172.21.15.50:0/2008459817 shutdown_connections 2024-09-21T06:12:13.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.191+0000 7f6da5e00640 1 -- 172.21.15.50:0/2008459817 wait complete. 2024-09-21T06:12:13.218 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:12:13.336 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T06:12:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.387+0000 7f107fc00640 1 -- 172.21.15.50:0/2028618934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10780cdd80 msgr2=0x7f1078165660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.387+0000 7f107fc00640 1 --2- 172.21.15.50:0/2028618934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10780cdd80 0x7f1078165660 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f1070009c80 tx=0x7f107001cda0 comp rx=0 tx=0).stop 2024-09-21T06:12:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.387+0000 7f107fc00640 1 -- 172.21.15.50:0/2028618934 shutdown_connections 2024-09-21T06:12:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.387+0000 7f107fc00640 1 --2- 172.21.15.50:0/2028618934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10780cdd80 0x7f1078165660 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.387+0000 7f107fc00640 1 --2- 172.21.15.50:0/2028618934 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f10780ce4c0 0x7f107815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.387+0000 7f107fc00640 1 -- 172.21.15.50:0/2028618934 >> 172.21.15.50:0/2028618934 conn(0x7f10780b2040 msgr2=0x7f10780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.387+0000 7f107fc00640 1 -- 172.21.15.50:0/2028618934 shutdown_connections 2024-09-21T06:12:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.387+0000 7f107fc00640 1 -- 172.21.15.50:0/2028618934 wait complete. 2024-09-21T06:12:13.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107fc00640 1 Processor -- start 2024-09-21T06:12:13.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107fc00640 1 -- start start 2024-09-21T06:12:13.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107fc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10780cd260 0x7f10780c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:13.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107fc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f10780cdd80 0x7f1078072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107fc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10780ce4c0 0x7f10781517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107fc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1078168bd0 con 0x7f10780cd260 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107fc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1078168a50 con 0x7f10780ce4c0 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107fc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1078168d50 con 0x7f10780cdd80 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10780cd260 0x7f10780c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10780cd260 0x7f10780c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42740/0 (socket says 172.21.15.50:42740) 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107d200640 1 -- 172.21.15.50:0/3502326644 learned_addr learned my addr 172.21.15.50:0/3502326644 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f1077e00640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f10780cdd80 0x7f1078072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107dc00640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10780ce4c0 0x7f10781517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107d200640 1 -- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f10780cdd80 msgr2=0x7f1078072ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107d200640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f10780cdd80 0x7f1078072ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107d200640 1 -- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10780ce4c0 msgr2=0x7f10781517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107d200640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10780ce4c0 0x7f10781517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:13.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107d200640 1 -- 172.21.15.50:0/3502326644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10780744a0 con 0x7f10780cd260 2024-09-21T06:12:13.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107d200640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10780cd260 0x7f10780c52b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f106c00d750 tx=0x7f106c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:13.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f1074c00640 1 -- 172.21.15.50:0/3502326644 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c017070 con 0x7f10780cd260 2024-09-21T06:12:13.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f1074c00640 1 -- 172.21.15.50:0/3502326644 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f106c009d20 con 0x7f10780cd260 2024-09-21T06:12:13.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107fc00640 1 -- 172.21.15.50:0/3502326644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1078074780 con 0x7f10780cd260 2024-09-21T06:12:13.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f1074c00640 1 -- 172.21.15.50:0/3502326644 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c005280 con 0x7f10780cd260 2024-09-21T06:12:13.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.391+0000 7f107fc00640 1 -- 172.21.15.50:0/3502326644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1078152150 con 0x7f10780cd260 2024-09-21T06:12:13.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.395+0000 7f107fc00640 1 -- 172.21.15.50:0/3502326644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107815aa50 con 0x7f10780cd260 2024-09-21T06:12:13.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.395+0000 7f1074c00640 1 -- 172.21.15.50:0/3502326644 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f106c005420 con 0x7f10780cd260 2024-09-21T06:12:13.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.395+0000 7f1074c00640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10580754e0 0x7f10580779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:13.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.395+0000 7f1074c00640 1 -- 172.21.15.50:0/3502326644 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5965+0+0 (secure 0 0 0) 0x7f106c093570 con 0x7f10780cd260 2024-09-21T06:12:13.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.395+0000 7f1077e00640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10580754e0 0x7f10580779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:13.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.395+0000 7f1077e00640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10580754e0 0x7f10580779f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f10780c69e0 tx=0x7f1064009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:13.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.399+0000 7f1074c00640 1 -- 172.21.15.50:0/3502326644 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f106c00de60 con 0x7f10780cd260 2024-09-21T06:12:13.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:13.619+0000 7f107fc00640 1 -- 172.21.15.50:0/3502326644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f10781fbf80 con 0x7f10780cd260 2024-09-21T06:12:14.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.195+0000 7f1074c00640 1 -- 172.21.15.50:0/3502326644 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) ==== 104+0+0 (secure 0 0 0) 0x7f106c05f9a0 con 0x7f10780cd260 2024-09-21T06:12:14.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.199+0000 7f107fc00640 1 -- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10580754e0 msgr2=0x7f10580779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.199+0000 7f107fc00640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10580754e0 0x7f10580779f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f10780c69e0 tx=0x7f1064009040 comp rx=0 tx=0).stop 2024-09-21T06:12:14.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.199+0000 7f107fc00640 1 -- 172.21.15.50:0/3502326644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10780cd260 msgr2=0x7f10780c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.199+0000 7f107fc00640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10780cd260 0x7f10780c52b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f106c00d750 tx=0x7f106c00dc10 comp rx=0 tx=0).stop 2024-09-21T06:12:14.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.199+0000 7f107fc00640 1 -- 172.21.15.50:0/3502326644 shutdown_connections 2024-09-21T06:12:14.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.199+0000 7f107fc00640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10780ce4c0 0x7f10781517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.199+0000 7f107fc00640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f10780cdd80 0x7f1078072ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.199+0000 7f107fc00640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10580754e0 0x7f10580779f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.199+0000 7f107fc00640 1 --2- 172.21.15.50:0/3502326644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10780cd260 0x7f10780c52b0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.199+0000 7f107fc00640 1 -- 172.21.15.50:0/3502326644 >> 172.21.15.50:0/3502326644 conn(0x7f10780b2040 msgr2=0x7f1078164f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:14.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.199+0000 7f107fc00640 1 -- 172.21.15.50:0/3502326644 shutdown_connections 2024-09-21T06:12:14.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.199+0000 7f107fc00640 1 -- 172.21.15.50:0/3502326644 wait complete. 2024-09-21T06:12:14.221 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:12:14.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.387+0000 7fb059400640 1 -- 172.21.15.50:0/2740863475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb05406efa0 msgr2=0x7fb05406f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.387+0000 7fb059400640 1 --2- 172.21.15.50:0/2740863475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb05406efa0 0x7fb05406f3b0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb05406ae60 tx=0x7fb04c01cd50 comp rx=0 tx=0).stop 2024-09-21T06:12:14.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.387+0000 7fb059400640 1 -- 172.21.15.50:0/2740863475 shutdown_connections 2024-09-21T06:12:14.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.387+0000 7fb059400640 1 --2- 172.21.15.50:0/2740863475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb05406efa0 0x7fb05406f3b0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.387+0000 7fb059400640 1 --2- 172.21.15.50:0/2740863475 >> v2:172.21.15.88:3300/0 conn(0x7fb0540744d0 0x7fb05406ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.387+0000 7fb059400640 1 -- 172.21.15.50:0/2740863475 >> 172.21.15.50:0/2740863475 conn(0x7fb0540b2c40 msgr2=0x7fb0540b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:14.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.387+0000 7fb059400640 1 -- 172.21.15.50:0/2740863475 shutdown_connections 2024-09-21T06:12:14.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.387+0000 7fb059400640 1 -- 172.21.15.50:0/2740863475 wait complete. 2024-09-21T06:12:14.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb059400640 1 Processor -- start 2024-09-21T06:12:14.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb059400640 1 -- start start 2024-09-21T06:12:14.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb059400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb05406efa0 0x7fb05415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:14.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb059400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb05415c5d0 0x7fb054073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:14.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb059400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0541552b0 0x7fb054073790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:14.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb059400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb054168d20 con 0x7fb05415c5d0 2024-09-21T06:12:14.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb059400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb054168ea0 con 0x7fb05406efa0 2024-09-21T06:12:14.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb059400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb054169020 con 0x7fb0541552b0 2024-09-21T06:12:14.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb051e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb05415c5d0 0x7fb054073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:14.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb051e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb05415c5d0 0x7fb054073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42758/0 (socket says 172.21.15.50:42758) 2024-09-21T06:12:14.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb051e00640 1 -- 172.21.15.50:0/1307308069 learned_addr learned my addr 172.21.15.50:0/1307308069 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:14.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb052800640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb05406efa0 0x7fb05415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:14.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb053200640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0541552b0 0x7fb054073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:14.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb051e00640 1 -- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0541552b0 msgr2=0x7fb054073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb051e00640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0541552b0 0x7fb054073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb051e00640 1 -- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb05406efa0 msgr2=0x7fb05415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb051e00640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb05406efa0 0x7fb05415a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb051e00640 1 -- 172.21.15.50:0/1307308069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05415b330 con 0x7fb05415c5d0 2024-09-21T06:12:14.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.391+0000 7fb051e00640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb05415c5d0 0x7fb054073250 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fb04000e790 tx=0x7fb04000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:14.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.395+0000 7fb04aa00640 1 -- 172.21.15.50:0/1307308069 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04000eea0 con 0x7fb05415c5d0 2024-09-21T06:12:14.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.395+0000 7fb04aa00640 1 -- 172.21.15.50:0/1307308069 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb04000cc60 con 0x7fb05415c5d0 2024-09-21T06:12:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.395+0000 7fb04aa00640 1 -- 172.21.15.50:0/1307308069 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040005280 con 0x7fb05415c5d0 2024-09-21T06:12:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.395+0000 7fb059400640 1 -- 172.21.15.50:0/1307308069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0541547d0 con 0x7fb05415c5d0 2024-09-21T06:12:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.395+0000 7fb059400640 1 -- 172.21.15.50:0/1307308069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb05415ae40 con 0x7fb05415c5d0 2024-09-21T06:12:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.395+0000 7fb059400640 1 -- 172.21.15.50:0/1307308069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb05415b0a0 con 0x7fb05415c5d0 2024-09-21T06:12:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.395+0000 7fb04aa00640 1 -- 172.21.15.50:0/1307308069 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fb040005420 con 0x7fb05415c5d0 2024-09-21T06:12:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.395+0000 7fb04aa00640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0340754e0 0x7fb0340779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.395+0000 7fb04aa00640 1 -- 172.21.15.50:0/1307308069 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5965+0+0 (secure 0 0 0) 0x7fb040093b40 con 0x7fb05415c5d0 2024-09-21T06:12:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.395+0000 7fb052800640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0340754e0 0x7fb0340779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.399+0000 7fb052800640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0340754e0 0x7fb0340779f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb044007fd0 tx=0x7fb044009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:14.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.403+0000 7fb04aa00640 1 -- 172.21.15.50:0/1307308069 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb040060080 con 0x7fb05415c5d0 2024-09-21T06:12:14.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:14.579+0000 7fb059400640 1 -- 172.21.15.50:0/1307308069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb05406f3b0 con 0x7fb05415c5d0 2024-09-21T06:12:15.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.531+0000 7fb04aa00640 1 -- 172.21.15.50:0/1307308069 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) ==== 181+0+0 (secure 0 0 0) 0x7fb040064ab0 con 0x7fb05415c5d0 2024-09-21T06:12:15.530 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:12:15.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.535+0000 7fb059400640 1 -- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0340754e0 msgr2=0x7fb0340779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:15.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.535+0000 7fb059400640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0340754e0 0x7fb0340779f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb044007fd0 tx=0x7fb044009040 comp rx=0 tx=0).stop 2024-09-21T06:12:15.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.535+0000 7fb059400640 1 -- 172.21.15.50:0/1307308069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb05415c5d0 msgr2=0x7fb054073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:15.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.535+0000 7fb059400640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb05415c5d0 0x7fb054073250 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fb04000e790 tx=0x7fb04000ec50 comp rx=0 tx=0).stop 2024-09-21T06:12:15.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.535+0000 7fb059400640 1 -- 172.21.15.50:0/1307308069 shutdown_connections 2024-09-21T06:12:15.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.535+0000 7fb059400640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0541552b0 0x7fb054073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:15.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.535+0000 7fb059400640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0340754e0 0x7fb0340779f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:15.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.535+0000 7fb059400640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb05415c5d0 0x7fb054073250 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:15.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.535+0000 7fb059400640 1 --2- 172.21.15.50:0/1307308069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb05406efa0 0x7fb05415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:15.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.535+0000 7fb059400640 1 -- 172.21.15.50:0/1307308069 >> 172.21.15.50:0/1307308069 conn(0x7fb0540b2c40 msgr2=0x7fb05415d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:15.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.535+0000 7fb059400640 1 -- 172.21.15.50:0/1307308069 shutdown_connections 2024-09-21T06:12:15.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.535+0000 7fb059400640 1 -- 172.21.15.50:0/1307308069 wait complete. 2024-09-21T06:12:15.560 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:12:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.731+0000 7f4a49400640 1 -- 172.21.15.50:0/3930809553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a28003a60 msgr2=0x7f4a28003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.731+0000 7f4a49400640 1 --2- 172.21.15.50:0/3930809553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a28003a60 0x7f4a28003ed0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f4a340016f0 tx=0x7f4a3401c720 comp rx=0 tx=0).stop 2024-09-21T06:12:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.731+0000 7f4a49400640 1 -- 172.21.15.50:0/3930809553 shutdown_connections 2024-09-21T06:12:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.731+0000 7f4a49400640 1 --2- 172.21.15.50:0/3930809553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a28003a60 0x7f4a28003ed0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.731+0000 7f4a49400640 1 -- 172.21.15.50:0/3930809553 >> 172.21.15.50:0/3930809553 conn(0x7f4a440b2040 msgr2=0x7f4a440b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.731+0000 7f4a49400640 1 -- 172.21.15.50:0/3930809553 shutdown_connections 2024-09-21T06:12:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.731+0000 7f4a49400640 1 -- 172.21.15.50:0/3930809553 wait complete. 2024-09-21T06:12:15.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a49400640 1 Processor -- start 2024-09-21T06:12:15.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a49400640 1 -- start start 2024-09-21T06:12:15.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a49400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a28003a60 0x7f4a44151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:15.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a49400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a4414af60 0x7f4a4414a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:15.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a49400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4414c910 0x7f4a4414cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:15.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a49400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4415d180 con 0x7f4a4414c910 2024-09-21T06:12:15.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a49400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4415d000 con 0x7f4a4414af60 2024-09-21T06:12:15.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a49400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4415d300 con 0x7f4a28003a60 2024-09-21T06:12:15.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a43200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4414c910 0x7f4a4414cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:15.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a43200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4414c910 0x7f4a4414cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42770/0 (socket says 172.21.15.50:42770) 2024-09-21T06:12:15.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a43200640 1 -- 172.21.15.50:0/1534625476 learned_addr learned my addr 172.21.15.50:0/1534625476 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:15.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a42800640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a28003a60 0x7f4a44151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:15.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a39e00640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a4414af60 0x7f4a4414a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:15.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a43200640 1 -- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a28003a60 msgr2=0x7f4a44151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:15.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a43200640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a28003a60 0x7f4a44151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:15.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a43200640 1 -- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a4414af60 msgr2=0x7f4a4414a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:15.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a43200640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a4414af60 0x7f4a4414a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:15.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a43200640 1 -- 172.21.15.50:0/1534625476 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4414d260 con 0x7f4a4414c910 2024-09-21T06:12:15.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a43200640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4414c910 0x7f4a4414cd20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4a44074950 tx=0x7f4a3c010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:15.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a3b400640 1 -- 172.21.15.50:0/1534625476 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c017070 con 0x7f4a4414c910 2024-09-21T06:12:15.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a3b400640 1 -- 172.21.15.50:0/1534625476 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a3c015070 con 0x7f4a4414c910 2024-09-21T06:12:15.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a49400640 1 -- 172.21.15.50:0/1534625476 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a44157a10 con 0x7f4a4414c910 2024-09-21T06:12:15.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a3b400640 1 -- 172.21.15.50:0/1534625476 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00ebb0 con 0x7f4a4414c910 2024-09-21T06:12:15.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.735+0000 7f4a49400640 1 -- 172.21.15.50:0/1534625476 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4414f400 con 0x7f4a4414c910 2024-09-21T06:12:15.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.739+0000 7f4a49400640 1 -- 172.21.15.50:0/1534625476 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a44151890 con 0x7f4a4414c910 2024-09-21T06:12:15.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.739+0000 7f4a3b400640 1 -- 172.21.15.50:0/1534625476 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f4a3c00ed50 con 0x7f4a4414c910 2024-09-21T06:12:15.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.739+0000 7f4a3b400640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a200754e0 0x7f4a200779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:15.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.739+0000 7f4a3b400640 1 -- 172.21.15.50:0/1534625476 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5499+0+0 (secure 0 0 0) 0x7f4a3c093c30 con 0x7f4a4414c910 2024-09-21T06:12:15.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.739+0000 7f4a42800640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a200754e0 0x7f4a200779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:15.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.743+0000 7f4a42800640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a200754e0 0x7f4a200779f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4a340016c0 tx=0x7f4a34019510 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:15.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.743+0000 7f4a3b400640 1 -- 172.21.15.50:0/1534625476 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4a4414d260 con 0x7f4a4414c910 2024-09-21T06:12:15.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:15.923+0000 7f4a49400640 1 -- 172.21.15.50:0/1534625476 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4a440b28b0 con 0x7f4a4414c910 2024-09-21T06:12:16.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.535+0000 7f4a3b400640 1 -- 172.21.15.50:0/1534625476 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) ==== 169+0+0 (secure 0 0 0) 0x7f4a3c060060 con 0x7f4a4414c910 2024-09-21T06:12:16.532 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-09-21T06:12:16.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.535+0000 7f4a49400640 1 -- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a200754e0 msgr2=0x7f4a200779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.535+0000 7f4a49400640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a200754e0 0x7f4a200779f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4a340016c0 tx=0x7f4a34019510 comp rx=0 tx=0).stop 2024-09-21T06:12:16.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.535+0000 7f4a49400640 1 -- 172.21.15.50:0/1534625476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4414c910 msgr2=0x7f4a4414cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.535+0000 7f4a49400640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4414c910 0x7f4a4414cd20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4a44074950 tx=0x7f4a3c010e50 comp rx=0 tx=0).stop 2024-09-21T06:12:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.539+0000 7f4a49400640 1 -- 172.21.15.50:0/1534625476 shutdown_connections 2024-09-21T06:12:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.539+0000 7f4a49400640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a200754e0 0x7f4a200779f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.539+0000 7f4a49400640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4414c910 0x7f4a4414cd20 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.539+0000 7f4a49400640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a4414af60 0x7f4a4414a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.539+0000 7f4a49400640 1 --2- 172.21.15.50:0/1534625476 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a28003a60 0x7f4a44151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.539+0000 7f4a49400640 1 -- 172.21.15.50:0/1534625476 >> 172.21.15.50:0/1534625476 conn(0x7f4a440b2040 msgr2=0x7f4a4415ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:16.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.539+0000 7f4a49400640 1 -- 172.21.15.50:0/1534625476 shutdown_connections 2024-09-21T06:12:16.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.539+0000 7f4a49400640 1 -- 172.21.15.50:0/1534625476 wait complete. 2024-09-21T06:12:16.565 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:12:16.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.743+0000 7fe50a000640 1 -- 172.21.15.50:0/2031682505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5040b3130 msgr2=0x7fe5040b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.743+0000 7fe50a000640 1 --2- 172.21.15.50:0/2031682505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5040b3130 0x7fe5040b3540 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009c80 tx=0x7fe4fc01cd70 comp rx=0 tx=0).stop 2024-09-21T06:12:16.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.743+0000 7fe50a000640 1 -- 172.21.15.50:0/2031682505 shutdown_connections 2024-09-21T06:12:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.743+0000 7fe50a000640 1 --2- 172.21.15.50:0/2031682505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5040b3130 0x7fe5040b3540 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.743+0000 7fe50a000640 1 --2- 172.21.15.50:0/2031682505 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe5040b1dc0 0x7fe5040b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.743+0000 7fe50a000640 1 -- 172.21.15.50:0/2031682505 >> 172.21.15.50:0/2031682505 conn(0x7fe50406cdf0 msgr2=0x7fe50406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.743+0000 7fe50a000640 1 -- 172.21.15.50:0/2031682505 shutdown_connections 2024-09-21T06:12:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.743+0000 7fe50a000640 1 -- 172.21.15.50:0/2031682505 wait complete. 2024-09-21T06:12:16.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.743+0000 7fe50a000640 1 Processor -- start 2024-09-21T06:12:16.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.743+0000 7fe50a000640 1 -- start start 2024-09-21T06:12:16.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.743+0000 7fe50a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5040b1dc0 0x7fe504148060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.743+0000 7fe50a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5040b3130 0x7fe504140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe50a000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe504155a00 0x7fe5041f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe50a000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe504168da0 con 0x7fe5040b1dc0 2024-09-21T06:12:16.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe50a000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe504168c20 con 0x7fe5040b3130 2024-09-21T06:12:16.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe50a000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe504168f20 con 0x7fe504155a00 2024-09-21T06:12:16.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe508c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5040b1dc0 0x7fe504148060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe508c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5040b1dc0 0x7fe504148060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47212/0 (socket says 172.21.15.50:47212) 2024-09-21T06:12:16.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe508c00640 1 -- 172.21.15.50:0/4221239938 learned_addr learned my addr 172.21.15.50:0/4221239938 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe509600640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe504155a00 0x7fe5041f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe503e00640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5040b3130 0x7fe504140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe508c00640 1 -- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe504155a00 msgr2=0x7fe5041f9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe508c00640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe504155a00 0x7fe5041f9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe508c00640 1 -- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5040b3130 msgr2=0x7fe504140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe508c00640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5040b3130 0x7fe504140060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe508c00640 1 -- 172.21.15.50:0/4221239938 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5041fa910 con 0x7fe5040b1dc0 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe503e00640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5040b3130 0x7fe504140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe508c00640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5040b1dc0 0x7fe504148060 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800c3c0 tx=0x7fe4f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe500c00640 1 -- 172.21.15.50:0/4221239938 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8017070 con 0x7fe5040b1dc0 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe500c00640 1 -- 172.21.15.50:0/4221239938 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4f80053c0 con 0x7fe5040b1dc0 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe500c00640 1 -- 172.21.15.50:0/4221239938 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8003860 con 0x7fe5040b1dc0 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe50a000640 1 -- 172.21.15.50:0/4221239938 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50414aab0 con 0x7fe5040b1dc0 2024-09-21T06:12:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe50a000640 1 -- 172.21.15.50:0/4221239938 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5041405a0 con 0x7fe5040b1dc0 2024-09-21T06:12:16.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe50a000640 1 -- 172.21.15.50:0/4221239938 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5040b21d0 con 0x7fe5040b1dc0 2024-09-21T06:12:16.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.747+0000 7fe500c00640 1 -- 172.21.15.50:0/4221239938 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fe4f801a070 con 0x7fe5040b1dc0 2024-09-21T06:12:16.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.751+0000 7fe500c00640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe4e4075510 0x7fe4e4077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.751+0000 7fe500c00640 1 -- 172.21.15.50:0/4221239938 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5027+0+0 (secure 0 0 0) 0x7fe4f80925c0 con 0x7fe5040b1dc0 2024-09-21T06:12:16.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.751+0000 7fe503e00640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe4e4075510 0x7fe4e4077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.751+0000 7fe503e00640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe4e4075510 0x7fe4e4077a20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe5041f99d0 tx=0x7fe4f00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:16.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.755+0000 7fe500c00640 1 -- 172.21.15.50:0/4221239938 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe5040b2710 con 0x7fe5040b1dc0 2024-09-21T06:12:16.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.931+0000 7fe50a000640 1 -- 172.21.15.50:0/4221239938 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fe50406b060 con 0x7fe5040b1dc0 2024-09-21T06:12:16.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe500c00640 1 -- 172.21.15.50:0/4221239938 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v57) ==== 71+0+0 (secure 0 0 0) 0x7fe4f805ea90 con 0x7fe5040b1dc0 2024-09-21T06:12:16.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe50a000640 1 -- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe4e4075510 msgr2=0x7fe4e4077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe50a000640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe4e4075510 0x7fe4e4077a20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe5041f99d0 tx=0x7fe4f00091e0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe50a000640 1 -- 172.21.15.50:0/4221239938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5040b1dc0 msgr2=0x7fe504148060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe50a000640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5040b1dc0 0x7fe504148060 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800c3c0 tx=0x7fe4f800c880 comp rx=0 tx=0).stop 2024-09-21T06:12:16.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe50a000640 1 -- 172.21.15.50:0/4221239938 shutdown_connections 2024-09-21T06:12:16.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe50a000640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe504155a00 0x7fe5041f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe50a000640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5040b3130 0x7fe504140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe50a000640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe4e4075510 0x7fe4e4077a20 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe50a000640 1 --2- 172.21.15.50:0/4221239938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5040b1dc0 0x7fe504148060 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe50a000640 1 -- 172.21.15.50:0/4221239938 >> 172.21.15.50:0/4221239938 conn(0x7fe50406cdf0 msgr2=0x7fe50415e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:16.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe50a000640 1 -- 172.21.15.50:0/4221239938 shutdown_connections 2024-09-21T06:12:16.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:16.939+0000 7fe50a000640 1 -- 172.21.15.50:0/4221239938 wait complete. 2024-09-21T06:12:16.959 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-21T06:12:17.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.119+0000 7fee06600640 1 -- 172.21.15.50:0/3459232662 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee000b3340 msgr2=0x7fee000b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.119+0000 7fee06600640 1 --2- 172.21.15.50:0/3459232662 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee000b3340 0x7fee000b3750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009bc0 tx=0x7fedf801cba0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.119+0000 7fee06600640 1 -- 172.21.15.50:0/3459232662 shutdown_connections 2024-09-21T06:12:17.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.119+0000 7fee06600640 1 --2- 172.21.15.50:0/3459232662 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee000b3340 0x7fee000b3750 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.119+0000 7fee06600640 1 --2- 172.21.15.50:0/3459232662 >> v2:172.21.15.88:3300/0 conn(0x7fee000cefa0 0x7fee000cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.119+0000 7fee06600640 1 -- 172.21.15.50:0/3459232662 >> 172.21.15.50:0/3459232662 conn(0x7fee000b2040 msgr2=0x7fee000b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:17.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.119+0000 7fee06600640 1 -- 172.21.15.50:0/3459232662 shutdown_connections 2024-09-21T06:12:17.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.119+0000 7fee06600640 1 -- 172.21.15.50:0/3459232662 wait complete. 2024-09-21T06:12:17.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fee06600640 1 Processor -- start 2024-09-21T06:12:17.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fee06600640 1 -- start start 2024-09-21T06:12:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fee06600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee000749a0 0x7fee000aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fee06600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee000b3340 0x7fee0015f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fee06600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee000cefa0 0x7fee0015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fee06600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fee00169e60 con 0x7fee000749a0 2024-09-21T06:12:17.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fee06600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee00169ce0 con 0x7fee000b3340 2024-09-21T06:12:17.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fee06600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fee00169fe0 con 0x7fee000cefa0 2024-09-21T06:12:17.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fedff400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee000749a0 0x7fee000aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fedff400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee000749a0 0x7fee000aaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47222/0 (socket says 172.21.15.50:47222) 2024-09-21T06:12:17.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fedff400640 1 -- 172.21.15.50:0/1828102384 learned_addr learned my addr 172.21.15.50:0/1828102384 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:17.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fedffe00640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee000cefa0 0x7fee0015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.123+0000 7fedf7e00640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee000b3340 0x7fee0015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fedff400640 1 -- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee000cefa0 msgr2=0x7fee0015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fedff400640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee000cefa0 0x7fee0015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fedff400640 1 -- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee000b3340 msgr2=0x7fee0015f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fedff400640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee000b3340 0x7fee0015f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fedff400640 1 -- 172.21.15.50:0/1828102384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee000afbe0 con 0x7fee000749a0 2024-09-21T06:12:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fedffe00640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee000cefa0 0x7fee0015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fedff400640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee000749a0 0x7fee000aaee0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fedf000c3f0 tx=0x7fedf000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fedf7400640 1 -- 172.21.15.50:0/1828102384 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0017070 con 0x7fee000749a0 2024-09-21T06:12:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fedf7400640 1 -- 172.21.15.50:0/1828102384 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedf00053c0 con 0x7fee000749a0 2024-09-21T06:12:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fedf7400640 1 -- 172.21.15.50:0/1828102384 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0003860 con 0x7fee000749a0 2024-09-21T06:12:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fee06600640 1 -- 172.21.15.50:0/1828102384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee000af900 con 0x7fee000749a0 2024-09-21T06:12:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fee06600640 1 -- 172.21.15.50:0/1828102384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0013dda0 con 0x7fee000749a0 2024-09-21T06:12:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fee06600640 1 -- 172.21.15.50:0/1828102384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee000b3750 con 0x7fee000749a0 2024-09-21T06:12:17.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.127+0000 7fedf7400640 1 -- 172.21.15.50:0/1828102384 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fedf001a070 con 0x7fee000749a0 2024-09-21T06:12:17.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.131+0000 7fedf7400640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feddc075510 0x7feddc077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.131+0000 7fedf7400640 1 -- 172.21.15.50:0/1828102384 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5027+0+0 (secure 0 0 0) 0x7fedf00929e0 con 0x7fee000749a0 2024-09-21T06:12:17.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.131+0000 7fedf7e00640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feddc075510 0x7feddc077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.131+0000 7fedf7e00640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feddc075510 0x7feddc077a20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fee000b05b0 tx=0x7fede8009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:17.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.131+0000 7fedf7400640 1 -- 172.21.15.50:0/1828102384 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fedf0021020 con 0x7fee000749a0 2024-09-21T06:12:17.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.375+0000 7fee06600640 1 -- 172.21.15.50:0/1828102384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7fee000ab420 con 0x7fee000749a0 2024-09-21T06:12:17.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.551+0000 7fedf7400640 1 -- 172.21.15.50:0/1828102384 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v706) ==== 186+0+0 (secure 0 0 0) 0x7fedf005ef20 con 0x7fee000749a0 2024-09-21T06:12:17.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.555+0000 7fee06600640 1 -- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feddc075510 msgr2=0x7feddc077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.555+0000 7fee06600640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feddc075510 0x7feddc077a20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fee000b05b0 tx=0x7fede8009260 comp rx=0 tx=0).stop 2024-09-21T06:12:17.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.555+0000 7fee06600640 1 -- 172.21.15.50:0/1828102384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee000749a0 msgr2=0x7fee000aaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.555+0000 7fee06600640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee000749a0 0x7fee000aaee0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fedf000c3f0 tx=0x7fedf000c8b0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.555+0000 7fee06600640 1 -- 172.21.15.50:0/1828102384 shutdown_connections 2024-09-21T06:12:17.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.555+0000 7fee06600640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fee000cefa0 0x7fee0015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.555+0000 7fee06600640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee000b3340 0x7fee0015f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.555+0000 7fee06600640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7feddc075510 0x7feddc077a20 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.555+0000 7fee06600640 1 --2- 172.21.15.50:0/1828102384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee000749a0 0x7fee000aaee0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.555+0000 7fee06600640 1 -- 172.21.15.50:0/1828102384 >> 172.21.15.50:0/1828102384 conn(0x7fee000b2040 msgr2=0x7fee0015b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:17.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.555+0000 7fee06600640 1 -- 172.21.15.50:0/1828102384 shutdown_connections 2024-09-21T06:12:17.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.555+0000 7fee06600640 1 -- 172.21.15.50:0/1828102384 wait complete. 2024-09-21T06:12:17.572 INFO:tasks.cephfs_test_runner:Test that the progress bar that is printed for 1 ongoing clone job is ... ok 2024-09-21T06:12:17.572 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that both the progress bars, that are printed for all 7 clone 2024-09-21T06:12:17.572 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-21T06:12:17.573 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:12:17.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.735+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3186708746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b4c003fe0 msgr2=0x7f3b4c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.735+0000 7f3b6dc00640 1 --2- 172.21.15.50:0/3186708746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b4c003fe0 0x7f3b4c004810 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009b40 tx=0x7f3b5001d230 comp rx=0 tx=0).stop 2024-09-21T06:12:17.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.735+0000 7f3b67400640 1 -- 172.21.15.50:0/3186708746 reap_dead start 2024-09-21T06:12:17.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.735+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3186708746 shutdown_connections 2024-09-21T06:12:17.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.735+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3186708746 >> 172.21.15.50:0/3186708746 conn(0x7f3b680b2040 msgr2=0x7f3b680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:17.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.735+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3186708746 shutdown_connections 2024-09-21T06:12:17.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.735+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3186708746 wait complete. 2024-09-21T06:12:17.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b6dc00640 1 Processor -- start 2024-09-21T06:12:17.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b6dc00640 1 -- start start 2024-09-21T06:12:17.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b6dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b4c003fe0 0x7f3b681f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b6dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b680c5320 0x7f3b680ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b6dc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b680ae8e0 0x7f3b680aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b6dc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68160fd0 con 0x7f3b4c003fe0 2024-09-21T06:12:17.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b6dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68160e50 con 0x7f3b680c5320 2024-09-21T06:12:17.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b6dc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3b68161150 con 0x7f3b680ae8e0 2024-09-21T06:12:17.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b66a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b4c003fe0 0x7f3b681f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b66a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b4c003fe0 0x7f3b681f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47236/0 (socket says 172.21.15.50:47236) 2024-09-21T06:12:17.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b66a00640 1 -- 172.21.15.50:0/3213160251 learned_addr learned my addr 172.21.15.50:0/3213160251 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b5fe00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b680c5320 0x7f3b680ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b67400640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b680ae8e0 0x7f3b680aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b66a00640 1 -- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b680ae8e0 msgr2=0x7f3b680aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b66a00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b680ae8e0 0x7f3b680aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b66a00640 1 -- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b680c5320 msgr2=0x7f3b680ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b66a00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b680c5320 0x7f3b680ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b66a00640 1 -- 172.21.15.50:0/3213160251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b680af230 con 0x7f3b4c003fe0 2024-09-21T06:12:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b5fe00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b680c5320 0x7f3b680ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:17.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b66a00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b4c003fe0 0x7f3b681f5dc0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f3b58011ae0 tx=0x7f3b58011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:17.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b5ea00640 1 -- 172.21.15.50:0/3213160251 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5800fc10 con 0x7f3b4c003fe0 2024-09-21T06:12:17.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b5ea00640 1 -- 172.21.15.50:0/3213160251 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b5800fdb0 con 0x7f3b4c003fe0 2024-09-21T06:12:17.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3213160251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b68144a90 con 0x7f3b4c003fe0 2024-09-21T06:12:17.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b5ea00640 1 -- 172.21.15.50:0/3213160251 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58019670 con 0x7f3b4c003fe0 2024-09-21T06:12:17.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3213160251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b68144d70 con 0x7f3b4c003fe0 2024-09-21T06:12:17.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.739+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3213160251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b680cf6c0 con 0x7f3b4c003fe0 2024-09-21T06:12:17.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.743+0000 7f3b5ea00640 1 -- 172.21.15.50:0/3213160251 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f3b58019810 con 0x7f3b4c003fe0 2024-09-21T06:12:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.743+0000 7f3b5ea00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b440754e0 0x7f3b440779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.743+0000 7f3b5ea00640 1 -- 172.21.15.50:0/3213160251 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5027+0+0 (secure 0 0 0) 0x7f3b58093f80 con 0x7f3b4c003fe0 2024-09-21T06:12:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.743+0000 7f3b5fe00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b440754e0 0x7f3b440779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.743+0000 7f3b5fe00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b440754e0 0x7f3b440779f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3b500ad000 tx=0x7f3b50019fd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:17.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.747+0000 7f3b5ea00640 1 -- 172.21.15.50:0/3213160251 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3b680af230 con 0x7f3b4c003fe0 2024-09-21T06:12:17.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.923+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3213160251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b6806b250 con 0x7f3b4c003fe0 2024-09-21T06:12:17.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b5ea00640 1 -- 172.21.15.50:0/3213160251 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+13127 (secure 0 0 0) 0x7f3b58060a90 con 0x7f3b4c003fe0 2024-09-21T06:12:17.924 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:17.924 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":134,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:12:16.535898+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6833/1864490990":"2024-09-22T06:12:13.190244+0000","172.21.15.50:6832/1864490990":"2024-09-22T06:12:13.190244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:12:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b440754e0 msgr2=0x7f3b440779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b6dc00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b440754e0 0x7f3b440779f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3b500ad000 tx=0x7f3b50019fd0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3213160251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b4c003fe0 msgr2=0x7f3b681f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b6dc00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b4c003fe0 0x7f3b681f5dc0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f3b58011ae0 tx=0x7f3b58011ee0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3213160251 shutdown_connections 2024-09-21T06:12:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b6dc00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b680ae8e0 0x7f3b680aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b6dc00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b680c5320 0x7f3b680ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b6dc00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b440754e0 0x7f3b440779f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b6dc00640 1 --2- 172.21.15.50:0/3213160251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b4c003fe0 0x7f3b681f5dc0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3213160251 >> 172.21.15.50:0/3213160251 conn(0x7f3b680b2040 msgr2=0x7f3b6815bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:17.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3213160251 shutdown_connections 2024-09-21T06:12:17.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:17.927+0000 7f3b6dc00640 1 -- 172.21.15.50:0/3213160251 wait complete. 2024-09-21T06:12:17.947 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:56:18.921197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:12:17.947 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:12:18.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.111+0000 7f02bbe00640 1 -- 172.21.15.50:0/1050723215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02bc1536f0 msgr2=0x7f02bc15d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:18.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.111+0000 7f02bbe00640 1 --2- 172.21.15.50:0/1050723215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02bc1536f0 0x7f02bc15d480 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009840 tx=0x7f02ac01c790 comp rx=0 tx=0).stop 2024-09-21T06:12:18.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.111+0000 7f02bbe00640 1 -- 172.21.15.50:0/1050723215 shutdown_connections 2024-09-21T06:12:18.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.111+0000 7f02bbe00640 1 --2- 172.21.15.50:0/1050723215 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f02bc074950 0x7f02bc0b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.111+0000 7f02bbe00640 1 --2- 172.21.15.50:0/1050723215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02bc1536f0 0x7f02bc15d480 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.111+0000 7f02bbe00640 1 -- 172.21.15.50:0/1050723215 >> 172.21.15.50:0/1050723215 conn(0x7f02bc0b4560 msgr2=0x7f02bc0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:18.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.111+0000 7f02bbe00640 1 -- 172.21.15.50:0/1050723215 shutdown_connections 2024-09-21T06:12:18.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.111+0000 7f02bbe00640 1 -- 172.21.15.50:0/1050723215 wait complete. 2024-09-21T06:12:18.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02bbe00640 1 Processor -- start 2024-09-21T06:12:18.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02bbe00640 1 -- start start 2024-09-21T06:12:18.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02bbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02bc1536f0 0x7f02bc14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:18.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02bbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02bc160060 0x7f02bc13da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:18.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02bbe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f02bc1fba60 0x7f02bc14c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:18.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02bbe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc169dc0 con 0x7f02bc1536f0 2024-09-21T06:12:18.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02bbe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc169c40 con 0x7f02bc160060 2024-09-21T06:12:18.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02bbe00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f02bc169f40 con 0x7f02bc1fba60 2024-09-21T06:12:18.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02baa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02bc1536f0 0x7f02bc14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:18.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02baa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02bc1536f0 0x7f02bc14afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47254/0 (socket says 172.21.15.50:47254) 2024-09-21T06:12:18.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02baa00640 1 -- 172.21.15.50:0/3928610153 learned_addr learned my addr 172.21.15.50:0/3928610153 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:18.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02bb400640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f02bc1fba60 0x7f02bc14c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:18.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02b2000640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02bc160060 0x7f02bc13da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02baa00640 1 -- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f02bc1fba60 msgr2=0x7f02bc14c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02baa00640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f02bc1fba60 0x7f02bc14c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02baa00640 1 -- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02bc160060 msgr2=0x7f02bc13da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02baa00640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02bc160060 0x7f02bc13da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02baa00640 1 -- 172.21.15.50:0/3928610153 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02bc0d1160 con 0x7f02bc1536f0 2024-09-21T06:12:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02baa00640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02bc1536f0 0x7f02bc14afe0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f02ac098170 tx=0x7f02ac0a85a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02b3400640 1 -- 172.21.15.50:0/3928610153 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac0be070 con 0x7f02bc1536f0 2024-09-21T06:12:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02b3400640 1 -- 172.21.15.50:0/3928610153 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02ac0a8c90 con 0x7f02bc1536f0 2024-09-21T06:12:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02b3400640 1 -- 172.21.15.50:0/3928610153 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac0a8e60 con 0x7f02bc1536f0 2024-09-21T06:12:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02bbe00640 1 -- 172.21.15.50:0/3928610153 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02bc1fc210 con 0x7f02bc1536f0 2024-09-21T06:12:18.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.115+0000 7f02bbe00640 1 -- 172.21.15.50:0/3928610153 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02bc1fc7d0 con 0x7f02bc1536f0 2024-09-21T06:12:18.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.119+0000 7f02bbe00640 1 -- 172.21.15.50:0/3928610153 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02bc1fc9d0 con 0x7f02bc1536f0 2024-09-21T06:12:18.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.119+0000 7f02b3400640 1 -- 172.21.15.50:0/3928610153 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f02ac0052a0 con 0x7f02bc1536f0 2024-09-21T06:12:18.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.119+0000 7f02b3400640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0298075510 0x7f0298077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:18.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.119+0000 7f02b3400640 1 -- 172.21.15.50:0/3928610153 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5027+0+0 (secure 0 0 0) 0x7f02ac12dc60 con 0x7f02bc1536f0 2024-09-21T06:12:18.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.119+0000 7f02b2000640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0298075510 0x7f0298077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:18.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.123+0000 7f02b2000640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0298075510 0x7f0298077a20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f02a4000f70 tx=0x7f02a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:18.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.123+0000 7f02b3400640 1 -- 172.21.15.50:0/3928610153 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f02ac0c7020 con 0x7f02bc1536f0 2024-09-21T06:12:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.299+0000 7f02bbe00640 1 -- 172.21.15.50:0/3928610153 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f02bc0b5260 con 0x7f02bc1536f0 2024-09-21T06:12:18.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.299+0000 7f02b3400640 1 -- 172.21.15.50:0/3928610153 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7f02ac0ac030 con 0x7f02bc1536f0 2024-09-21T06:12:18.414 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-09-21T06:12:18.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.303+0000 7f02bbe00640 1 -- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0298075510 msgr2=0x7f0298077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:18.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.303+0000 7f02bbe00640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0298075510 0x7f0298077a20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f02a4000f70 tx=0x7f02a4009040 comp rx=0 tx=0).stop 2024-09-21T06:12:18.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.303+0000 7f02bbe00640 1 -- 172.21.15.50:0/3928610153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02bc1536f0 msgr2=0x7f02bc14afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:18.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.303+0000 7f02bbe00640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02bc1536f0 0x7f02bc14afe0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f02ac098170 tx=0x7f02ac0a85a0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.303+0000 7f02bbe00640 1 -- 172.21.15.50:0/3928610153 shutdown_connections 2024-09-21T06:12:18.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.303+0000 7f02bbe00640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f02bc1fba60 0x7f02bc14c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.303+0000 7f02bbe00640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02bc160060 0x7f02bc13da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.303+0000 7f02bbe00640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0298075510 0x7f0298077a20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.303+0000 7f02bbe00640 1 --2- 172.21.15.50:0/3928610153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02bc1536f0 0x7f02bc14afe0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.303+0000 7f02bbe00640 1 -- 172.21.15.50:0/3928610153 >> 172.21.15.50:0/3928610153 conn(0x7f02bc0b4560 msgr2=0x7f02bc072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:18.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.303+0000 7f02bbe00640 1 -- 172.21.15.50:0/3928610153 shutdown_connections 2024-09-21T06:12:18.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.303+0000 7f02bbe00640 1 -- 172.21.15.50:0/3928610153 wait complete. 2024-09-21T06:12:18.416 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-21T06:12:18.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.579+0000 7f4bfd400640 1 -- 172.21.15.50:0/3984515920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf80b54f0 msgr2=0x7f4bf815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:18.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.579+0000 7f4bfd400640 1 --2- 172.21.15.50:0/3984515920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf80b54f0 0x7f4bf815bb60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009bc0 tx=0x7f4bf001cae0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.579+0000 7f4bfd400640 1 -- 172.21.15.50:0/3984515920 shutdown_connections 2024-09-21T06:12:18.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.579+0000 7f4bfd400640 1 --2- 172.21.15.50:0/3984515920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf80b54f0 0x7f4bf815bb60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.579+0000 7f4bfd400640 1 -- 172.21.15.50:0/3984515920 >> 172.21.15.50:0/3984515920 conn(0x7f4bf80b2040 msgr2=0x7f4bf80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:18.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.579+0000 7f4bfd400640 1 -- 172.21.15.50:0/3984515920 shutdown_connections 2024-09-21T06:12:18.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.579+0000 7f4bfd400640 1 -- 172.21.15.50:0/3984515920 wait complete. 2024-09-21T06:12:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.579+0000 7f4bfd400640 1 Processor -- start 2024-09-21T06:12:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.579+0000 7f4bfd400640 1 -- start start 2024-09-21T06:12:18.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bfd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf80b5c70 0x7f4bf8150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:18.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bfd400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4bf81527b0 0x7f4bf8150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:18.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bfd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf8151bf0 0x7f4bf8151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:18.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bfd400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf8169b30 con 0x7f4bf80b5c70 2024-09-21T06:12:18.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bfd400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf81699b0 con 0x7f4bf8151bf0 2024-09-21T06:12:18.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bfd400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4bf8169cb0 con 0x7f4bf81527b0 2024-09-21T06:12:18.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bf6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf80b5c70 0x7f4bf8150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:18.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bf6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf80b5c70 0x7f4bf8150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47258/0 (socket says 172.21.15.50:47258) 2024-09-21T06:12:18.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bf6800640 1 -- 172.21.15.50:0/405357746 learned_addr learned my addr 172.21.15.50:0/405357746 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:18.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bf7200640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf8151bf0 0x7f4bf8151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:18.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bede00640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4bf81527b0 0x7f4bf8150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:18.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bf6800640 1 -- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4bf81527b0 msgr2=0x7f4bf8150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:18.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bf6800640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4bf81527b0 0x7f4bf8150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bf6800640 1 -- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf8151bf0 msgr2=0x7f4bf8151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:18.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bf6800640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf8151bf0 0x7f4bf8151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:18.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bf6800640 1 -- 172.21.15.50:0/405357746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf80af800 con 0x7f4bf80b5c70 2024-09-21T06:12:18.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bf7200640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf8151bf0 0x7f4bf8151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:18.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bf6800640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf80b5c70 0x7f4bf8150690 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f4be800c450 tx=0x7f4be800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:18.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bef400640 1 -- 172.21.15.50:0/405357746 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be80052d0 con 0x7f4bf80b5c70 2024-09-21T06:12:18.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bef400640 1 -- 172.21.15.50:0/405357746 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4be8005470 con 0x7f4bf80b5c70 2024-09-21T06:12:18.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bfd400640 1 -- 172.21.15.50:0/405357746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf80afae0 con 0x7f4bf80b5c70 2024-09-21T06:12:18.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bef400640 1 -- 172.21.15.50:0/405357746 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be8012870 con 0x7f4bf80b5c70 2024-09-21T06:12:18.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bfd400640 1 -- 172.21.15.50:0/405357746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf8157d00 con 0x7f4bf80b5c70 2024-09-21T06:12:18.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.583+0000 7f4bfd400640 1 -- 172.21.15.50:0/405357746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf815bb60 con 0x7f4bf80b5c70 2024-09-21T06:12:18.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.587+0000 7f4bef400640 1 -- 172.21.15.50:0/405357746 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f4be8003600 con 0x7f4bf80b5c70 2024-09-21T06:12:18.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.587+0000 7f4bef400640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4bd4075510 0x7f4bd4077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:18.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.587+0000 7f4bef400640 1 -- 172.21.15.50:0/405357746 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5027+0+0 (secure 0 0 0) 0x7f4be8092d20 con 0x7f4bf80b5c70 2024-09-21T06:12:18.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.587+0000 7f4bede00640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4bd4075510 0x7f4bd4077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:18.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.587+0000 7f4bede00640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4bd4075510 0x7f4bd4077a20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4bf813c450 tx=0x7f4be0007cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:18.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.591+0000 7f4bef400640 1 -- 172.21.15.50:0/405357746 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4be805f1e0 con 0x7f4bf80b5c70 2024-09-21T06:12:18.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:18.831+0000 7f4bfd400640 1 -- 172.21.15.50:0/405357746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7f4bf815c6d0 con 0x7f4bf80b5c70 2024-09-21T06:12:18.891 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:12:18.894 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:12:19.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.558+0000 7f4bef400640 1 -- 172.21.15.50:0/405357746 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v708) ==== 189+0+0 (secure 0 0 0) 0x7f4be8063c10 con 0x7f4bf80b5c70 2024-09-21T06:12:19.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.562+0000 7f4bfd400640 1 -- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4bd4075510 msgr2=0x7f4bd4077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:19.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.562+0000 7f4bfd400640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4bd4075510 0x7f4bd4077a20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4bf813c450 tx=0x7f4be0007cd0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.562+0000 7f4bfd400640 1 -- 172.21.15.50:0/405357746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf80b5c70 msgr2=0x7f4bf8150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:19.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.562+0000 7f4bfd400640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf80b5c70 0x7f4bf8150690 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f4be800c450 tx=0x7f4be800c910 comp rx=0 tx=0).stop 2024-09-21T06:12:19.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.562+0000 7f4bfd400640 1 -- 172.21.15.50:0/405357746 shutdown_connections 2024-09-21T06:12:19.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.562+0000 7f4bfd400640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf8151bf0 0x7f4bf8151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.562+0000 7f4bfd400640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4bf81527b0 0x7f4bf8150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.562+0000 7f4bfd400640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4bd4075510 0x7f4bd4077a20 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.562+0000 7f4bfd400640 1 --2- 172.21.15.50:0/405357746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bf80b5c70 0x7f4bf8150690 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.562+0000 7f4bfd400640 1 -- 172.21.15.50:0/405357746 >> 172.21.15.50:0/405357746 conn(0x7f4bf80b2040 msgr2=0x7f4bf80cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:19.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.562+0000 7f4bfd400640 1 -- 172.21.15.50:0/405357746 shutdown_connections 2024-09-21T06:12:19.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.562+0000 7f4bfd400640 1 -- 172.21.15.50:0/405357746 wait complete. 2024-09-21T06:12:19.584 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T06:12:19.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.750+0000 7f4a58600640 1 -- 172.21.15.50:0/1179648443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a50074950 msgr2=0x7f4a5015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:19.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.750+0000 7f4a58600640 1 --2- 172.21.15.50:0/1179648443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a50074950 0x7f4a5015cf90 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009c80 tx=0x7f4a4801cda0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.750+0000 7f4a58600640 1 -- 172.21.15.50:0/1179648443 shutdown_connections 2024-09-21T06:12:19.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.750+0000 7f4a58600640 1 --2- 172.21.15.50:0/1179648443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a50074950 0x7f4a5015cf90 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.750+0000 7f4a58600640 1 --2- 172.21.15.50:0/1179648443 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a500b5a80 0x7f4a50074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.750+0000 7f4a58600640 1 -- 172.21.15.50:0/1179648443 >> 172.21.15.50:0/1179648443 conn(0x7f4a500708c0 msgr2=0x7f4a50070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:19.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.750+0000 7f4a58600640 1 -- 172.21.15.50:0/1179648443 shutdown_connections 2024-09-21T06:12:19.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.750+0000 7f4a58600640 1 -- 172.21.15.50:0/1179648443 wait complete. 2024-09-21T06:12:19.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.750+0000 7f4a58600640 1 Processor -- start 2024-09-21T06:12:19.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.750+0000 7f4a58600640 1 -- start start 2024-09-21T06:12:19.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a58600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a50074950 0x7f4a500b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:19.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a58600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a500b5a80 0x7f4a50148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:19.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a58600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a50157030 0x7f4a50151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:19.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a58600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5016a3c0 con 0x7f4a50157030 2024-09-21T06:12:19.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a58600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5016a240 con 0x7f4a50074950 2024-09-21T06:12:19.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a58600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5016a540 con 0x7f4a500b5a80 2024-09-21T06:12:19.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a56600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a50157030 0x7f4a50151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:19.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a56600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a50157030 0x7f4a50151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47276/0 (socket says 172.21.15.50:47276) 2024-09-21T06:12:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a56600640 1 -- 172.21.15.50:0/2349946193 learned_addr learned my addr 172.21.15.50:0/2349946193 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a55200640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a500b5a80 0x7f4a50148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a55c00640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a50074950 0x7f4a500b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a56600640 1 -- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a500b5a80 msgr2=0x7f4a50148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a56600640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a500b5a80 0x7f4a50148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a56600640 1 -- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a50074950 msgr2=0x7f4a500b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a56600640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a50074950 0x7f4a500b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a56600640 1 -- 172.21.15.50:0/2349946193 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5014a000 con 0x7f4a50157030 2024-09-21T06:12:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a55200640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a500b5a80 0x7f4a50148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a55c00640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a50074950 0x7f4a500b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a56600640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a50157030 0x7f4a50151680 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f4a4801b450 tx=0x7f4a48009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a4d600640 1 -- 172.21.15.50:0/2349946193 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a480bb070 con 0x7f4a50157030 2024-09-21T06:12:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a4d600640 1 -- 172.21.15.50:0/2349946193 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a48004d40 con 0x7f4a50157030 2024-09-21T06:12:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a58600640 1 -- 172.21.15.50:0/2349946193 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5014a2e0 con 0x7f4a50157030 2024-09-21T06:12:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a4d600640 1 -- 172.21.15.50:0/2349946193 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a480b2040 con 0x7f4a50157030 2024-09-21T06:12:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a58600640 1 -- 172.21.15.50:0/2349946193 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a501f9f10 con 0x7f4a50157030 2024-09-21T06:12:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.754+0000 7f4a58600640 1 -- 172.21.15.50:0/2349946193 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a5015cf90 con 0x7f4a50157030 2024-09-21T06:12:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.758+0000 7f4a4d600640 1 -- 172.21.15.50:0/2349946193 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f4a480be070 con 0x7f4a50157030 2024-09-21T06:12:19.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.758+0000 7f4a4d600640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a300754e0 0x7f4a300779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:19.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.758+0000 7f4a4d600640 1 -- 172.21.15.50:0/2349946193 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5027+0+0 (secure 0 0 0) 0x7f4a4812cf70 con 0x7f4a50157030 2024-09-21T06:12:19.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.758+0000 7f4a55c00640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a300754e0 0x7f4a300779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:19.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.758+0000 7f4a55c00640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a300754e0 0x7f4a300779f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4a440016b0 tx=0x7f4a44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:19.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.762+0000 7f4a4d600640 1 -- 172.21.15.50:0/2349946193 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4a48016990 con 0x7f4a50157030 2024-09-21T06:12:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 -- 172.21.15.50:0/2349946193 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f4a500d2260 con 0x7f4a50157030 2024-09-21T06:12:19.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a4d600640 1 -- 172.21.15.50:0/2349946193 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v57) ==== 59+0+2294 (secure 0 0 0) 0x7f4a480f9b20 con 0x7f4a50157030 2024-09-21T06:12:19.941 INFO:teuthology.orchestra.run.smithi050.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:12:19.941 INFO:teuthology.orchestra.run.smithi050.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:12:19.941 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:12:19.941 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_client 20/20 2024-09-21T06:12:19.941 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:12:19.941 INFO:teuthology.orchestra.run.smithi050.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:12:19.941 INFO:teuthology.orchestra.run.smithi050.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:12:19.941 INFO:teuthology.orchestra.run.smithi050.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:12:19.941 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3735.149299 2024-09-21T06:12:19.941 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4471.400731 2024-09-21T06:12:19.941 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3640.131623 2024-09-21T06:12:19.942 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3663.645087 2024-09-21T06:12:19.942 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3762.055672 2024-09-21T06:12:19.942 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3661.558528 2024-09-21T06:12:19.942 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3661.456695 2024-09-21T06:12:19.942 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3796.187852 2024-09-21T06:12:19.942 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds 20/20 2024-09-21T06:12:19.942 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:12:19.942 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced debug_ms 2/2 2024-09-21T06:12:19.942 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_frag true 2024-09-21T06:12:19.942 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:12:19.942 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:12:19.942 INFO:teuthology.orchestra.run.smithi050.stdout:mds dev mds_verify_scatter true 2024-09-21T06:12:19.943 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:12:19.943 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:12:19.943 INFO:teuthology.orchestra.run.smithi050.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:12:19.943 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced client_mount_timeout 600 2024-09-21T06:12:19.943 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_client 20/20 2024-09-21T06:12:19.943 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced debug_ms 1/1 2024-09-21T06:12:19.943 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:12:19.943 INFO:teuthology.orchestra.run.smithi050.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:12:19.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 -- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a300754e0 msgr2=0x7f4a300779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:19.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a300754e0 0x7f4a300779f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4a440016b0 tx=0x7f4a44009040 comp rx=0 tx=0).stop 2024-09-21T06:12:19.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 -- 172.21.15.50:0/2349946193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a50157030 msgr2=0x7f4a50151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:19.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a50157030 0x7f4a50151680 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f4a4801b450 tx=0x7f4a48009350 comp rx=0 tx=0).stop 2024-09-21T06:12:19.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 -- 172.21.15.50:0/2349946193 shutdown_connections 2024-09-21T06:12:19.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a300754e0 0x7f4a300779f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a50157030 0x7f4a50151680 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a500b5a80 0x7f4a50148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 --2- 172.21.15.50:0/2349946193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a50074950 0x7f4a500b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:19.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 -- 172.21.15.50:0/2349946193 >> 172.21.15.50:0/2349946193 conn(0x7f4a500708c0 msgr2=0x7f4a5015e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:19.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 -- 172.21.15.50:0/2349946193 shutdown_connections 2024-09-21T06:12:19.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:19.942+0000 7f4a58600640 1 -- 172.21.15.50:0/2349946193 wait complete. 2024-09-21T06:12:19.963 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:12:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.126+0000 7fe32aa00640 1 -- 172.21.15.50:0/3350110833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3240b54f0 msgr2=0x7fe32415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.126+0000 7fe32aa00640 1 --2- 172.21.15.50:0/3350110833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3240b54f0 0x7fe32415bb60 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009c80 tx=0x7fe31c01cce0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.126+0000 7fe32aa00640 1 -- 172.21.15.50:0/3350110833 shutdown_connections 2024-09-21T06:12:20.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.126+0000 7fe32aa00640 1 --2- 172.21.15.50:0/3350110833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3240b54f0 0x7fe32415bb60 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.126+0000 7fe32aa00640 1 -- 172.21.15.50:0/3350110833 >> 172.21.15.50:0/3350110833 conn(0x7fe3240b2040 msgr2=0x7fe3240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:20.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.126+0000 7fe32aa00640 1 -- 172.21.15.50:0/3350110833 shutdown_connections 2024-09-21T06:12:20.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.126+0000 7fe32aa00640 1 -- 172.21.15.50:0/3350110833 wait complete. 2024-09-21T06:12:20.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe32aa00640 1 Processor -- start 2024-09-21T06:12:20.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe32aa00640 1 -- start start 2024-09-21T06:12:20.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe32aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3240b5c70 0x7fe32414e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe32aa00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3241527b0 0x7fe32414f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe32aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3240aa4d0 0x7fe32414fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe32aa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe324169bd0 con 0x7fe3240b5c70 2024-09-21T06:12:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe32aa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe324169a50 con 0x7fe3240aa4d0 2024-09-21T06:12:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe32aa00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe324169d50 con 0x7fe3241527b0 2024-09-21T06:12:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe323e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3240b5c70 0x7fe32414e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe323e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3240b5c70 0x7fe32414e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47290/0 (socket says 172.21.15.50:47290) 2024-09-21T06:12:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe323e00640 1 -- 172.21.15.50:0/1983607625 learned_addr learned my addr 172.21.15.50:0/1983607625 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe328a00640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3240aa4d0 0x7fe32414fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe31b400640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3241527b0 0x7fe32414f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe323e00640 1 -- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3241527b0 msgr2=0x7fe32414f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe323e00640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3241527b0 0x7fe32414f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe323e00640 1 -- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3240aa4d0 msgr2=0x7fe32414fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe323e00640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3240aa4d0 0x7fe32414fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe323e00640 1 -- 172.21.15.50:0/1983607625 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32414f040 con 0x7fe3240b5c70 2024-09-21T06:12:20.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe323e00640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3240b5c70 0x7fe32414e6e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c3c0 tx=0x7fe31400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:20.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe320c00640 1 -- 172.21.15.50:0/1983607625 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314017070 con 0x7fe3240b5c70 2024-09-21T06:12:20.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe320c00640 1 -- 172.21.15.50:0/1983607625 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3140053c0 con 0x7fe3240b5c70 2024-09-21T06:12:20.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe32aa00640 1 -- 172.21.15.50:0/1983607625 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3240ac890 con 0x7fe3240b5c70 2024-09-21T06:12:20.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe320c00640 1 -- 172.21.15.50:0/1983607625 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314003860 con 0x7fe3240b5c70 2024-09-21T06:12:20.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.130+0000 7fe32aa00640 1 -- 172.21.15.50:0/1983607625 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3240ac5b0 con 0x7fe3240b5c70 2024-09-21T06:12:20.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.134+0000 7fe32aa00640 1 -- 172.21.15.50:0/1983607625 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe32415bb60 con 0x7fe3240b5c70 2024-09-21T06:12:20.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.134+0000 7fe320c00640 1 -- 172.21.15.50:0/1983607625 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fe31401a070 con 0x7fe3240b5c70 2024-09-21T06:12:20.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.134+0000 7fe320c00640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe3000754e0 0x7fe3000779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.134+0000 7fe320c00640 1 -- 172.21.15.50:0/1983607625 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5027+0+0 (secure 0 0 0) 0x7fe3140926e0 con 0x7fe3240b5c70 2024-09-21T06:12:20.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.134+0000 7fe31b400640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe3000754e0 0x7fe3000779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.138+0000 7fe31b400640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe3000754e0 0x7fe3000779f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe3241f9c80 tx=0x7fe30c009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:20.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.138+0000 7fe320c00640 1 -- 172.21.15.50:0/1983607625 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe31405eba0 con 0x7fe3240b5c70 2024-09-21T06:12:20.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.314+0000 7fe32aa00640 1 -- 172.21.15.50:0/1983607625 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe32415b620 con 0x7fe3240b5c70 2024-09-21T06:12:20.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.318+0000 7fe320c00640 1 -- 172.21.15.50:0/1983607625 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) ==== 122+0+0 (secure 0 0 0) 0x7fe3140635d0 con 0x7fe3240b5c70 2024-09-21T06:12:20.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.322+0000 7fe32aa00640 1 -- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe3000754e0 msgr2=0x7fe3000779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.322+0000 7fe32aa00640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe3000754e0 0x7fe3000779f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe3241f9c80 tx=0x7fe30c009060 comp rx=0 tx=0).stop 2024-09-21T06:12:20.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.322+0000 7fe32aa00640 1 -- 172.21.15.50:0/1983607625 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3240b5c70 msgr2=0x7fe32414e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.322+0000 7fe32aa00640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3240b5c70 0x7fe32414e6e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c3c0 tx=0x7fe31400c880 comp rx=0 tx=0).stop 2024-09-21T06:12:20.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.322+0000 7fe32aa00640 1 -- 172.21.15.50:0/1983607625 shutdown_connections 2024-09-21T06:12:20.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.322+0000 7fe32aa00640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3240aa4d0 0x7fe32414fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.322+0000 7fe32aa00640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe3241527b0 0x7fe32414f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.322+0000 7fe32aa00640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe3000754e0 0x7fe3000779f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.322+0000 7fe32aa00640 1 --2- 172.21.15.50:0/1983607625 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3240b5c70 0x7fe32414e6e0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.322+0000 7fe32aa00640 1 -- 172.21.15.50:0/1983607625 >> 172.21.15.50:0/1983607625 conn(0x7fe3240b2040 msgr2=0x7fe3240b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:20.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.322+0000 7fe32aa00640 1 -- 172.21.15.50:0/1983607625 shutdown_connections 2024-09-21T06:12:20.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.322+0000 7fe32aa00640 1 -- 172.21.15.50:0/1983607625 wait complete. 2024-09-21T06:12:20.341 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:12:20.341 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:12:20.453 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:12:20.544 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:12:20.544 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:12:20.544 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:20.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:12:20.553 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:12:20.553 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:20.567 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:12:20.567 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:12:20.567 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:20.607 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:12:20.608 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:12:20.608 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:20.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:12:20.619 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T06:12:20.619 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:12:20.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.810+0000 7fd6c2a00640 1 -- 172.21.15.50:0/2067024914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0003a60 msgr2=0x7fd6a0003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.810+0000 7fd6c2a00640 1 --2- 172.21.15.50:0/2067024914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0003a60 0x7fd6a0003ef0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac009b40 tx=0x7fd6ac01d220 comp rx=0 tx=0).stop 2024-09-21T06:12:20.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.810+0000 7fd6c2a00640 1 -- 172.21.15.50:0/2067024914 shutdown_connections 2024-09-21T06:12:20.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.810+0000 7fd6c2a00640 1 --2- 172.21.15.50:0/2067024914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0003a60 0x7fd6a0003ef0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.810+0000 7fd6c2a00640 1 -- 172.21.15.50:0/2067024914 >> 172.21.15.50:0/2067024914 conn(0x7fd6bc0b2040 msgr2=0x7fd6bc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:20.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.810+0000 7fd6c2a00640 1 -- 172.21.15.50:0/2067024914 shutdown_connections 2024-09-21T06:12:20.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.810+0000 7fd6c2a00640 1 -- 172.21.15.50:0/2067024914 wait complete. 2024-09-21T06:12:20.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.810+0000 7fd6c2a00640 1 Processor -- start 2024-09-21T06:12:20.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6c2a00640 1 -- start start 2024-09-21T06:12:20.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6c2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0003a60 0x7fd6bc151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6c2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6bc14c760 0x7fd6bc0b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6c2a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6bc14ccb0 0x7fd6bc0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6c2a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc0b5580 con 0x7fd6a0003a60 2024-09-21T06:12:20.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6c2a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc0b5400 con 0x7fd6bc14c760 2024-09-21T06:12:20.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6c2a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd6bc0b5700 con 0x7fd6bc14ccb0 2024-09-21T06:12:20.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6bbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0003a60 0x7fd6bc151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6bbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0003a60 0x7fd6bc151250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47300/0 (socket says 172.21.15.50:47300) 2024-09-21T06:12:20.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6bbe00640 1 -- 172.21.15.50:0/1276676702 learned_addr learned my addr 172.21.15.50:0/1276676702 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:20.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6c0a00640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6bc14ccb0 0x7fd6bc0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6b3400640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6bc14c760 0x7fd6bc0b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6bbe00640 1 -- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6bc14ccb0 msgr2=0x7fd6bc0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6bbe00640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6bc14ccb0 0x7fd6bc0b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6bbe00640 1 -- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6bc14c760 msgr2=0x7fd6bc0b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6bbe00640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6bc14c760 0x7fd6bc0b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6bbe00640 1 -- 172.21.15.50:0/1276676702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc1eb520 con 0x7fd6a0003a60 2024-09-21T06:12:20.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6bbe00640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0003a60 0x7fd6bc151250 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac01d050 tx=0x7fd6ac005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:20.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6b8c00640 1 -- 172.21.15.50:0/1276676702 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac0bb030 con 0x7fd6a0003a60 2024-09-21T06:12:20.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6b8c00640 1 -- 172.21.15.50:0/1276676702 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6ac0046d0 con 0x7fd6a0003a60 2024-09-21T06:12:20.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6c2a00640 1 -- 172.21.15.50:0/1276676702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6bc152d50 con 0x7fd6a0003a60 2024-09-21T06:12:20.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6b8c00640 1 -- 172.21.15.50:0/1276676702 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac004b20 con 0x7fd6a0003a60 2024-09-21T06:12:20.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.814+0000 7fd6c2a00640 1 -- 172.21.15.50:0/1276676702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6bc153310 con 0x7fd6a0003a60 2024-09-21T06:12:20.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.818+0000 7fd6c2a00640 1 -- 172.21.15.50:0/1276676702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6bc15f500 con 0x7fd6a0003a60 2024-09-21T06:12:20.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.818+0000 7fd6b8c00640 1 -- 172.21.15.50:0/1276676702 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fd6ac0be030 con 0x7fd6a0003a60 2024-09-21T06:12:20.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.818+0000 7fd6b8c00640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd6980754e0 0x7fd6980779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.818+0000 7fd6b8c00640 1 -- 172.21.15.50:0/1276676702 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5027+0+0 (secure 0 0 0) 0x7fd6ac12ebb0 con 0x7fd6a0003a60 2024-09-21T06:12:20.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.822+0000 7fd6b3400640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd6980754e0 0x7fd6980779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.822+0000 7fd6b3400640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd6980754e0 0x7fd6980779f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc154e00 tx=0x7fd6a4006db0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:20.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:20.822+0000 7fd6b8c00640 1 -- 172.21.15.50:0/1276676702 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd6ac016f60 con 0x7fd6a0003a60 2024-09-21T06:12:21.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6c2a00640 1 -- 172.21.15.50:0/1276676702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6bc06b250 con 0x7fd6a0003a60 2024-09-21T06:12:21.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6b8c00640 1 -- 172.21.15.50:0/1276676702 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+3905 (secure 0 0 0) 0x7fd6ac0fb060 con 0x7fd6a0003a60 2024-09-21T06:12:21.033 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:21.033 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":87,"btime":"2024-09-21T06:12:14:192948+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2643543096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2643543096},{"type":"v1","addr":"172.21.15.88:6836","nonce":2643543096}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3618269169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3618269169},{"type":"v1","addr":"172.21.15.88:6837","nonce":3618269169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/66879433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":66879433},{"type":"v1","addr":"172.21.15.50:6835","nonce":66879433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/2493163526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2493163526},{"type":"v1","addr":"172.21.15.50:6833","nonce":2493163526}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[]} 2024-09-21T06:12:21.034 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 87 2024-09-21T06:12:21.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6c2a00640 1 -- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd6980754e0 msgr2=0x7fd6980779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:21.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6c2a00640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd6980754e0 0x7fd6980779f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc154e00 tx=0x7fd6a4006db0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6c2a00640 1 -- 172.21.15.50:0/1276676702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0003a60 msgr2=0x7fd6bc151250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:21.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6c2a00640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0003a60 0x7fd6bc151250 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac01d050 tx=0x7fd6ac005850 comp rx=0 tx=0).stop 2024-09-21T06:12:21.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6c2a00640 1 -- 172.21.15.50:0/1276676702 shutdown_connections 2024-09-21T06:12:21.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6c2a00640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd6bc14ccb0 0x7fd6bc0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6c2a00640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6bc14c760 0x7fd6bc0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6c2a00640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd6980754e0 0x7fd6980779f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6c2a00640 1 --2- 172.21.15.50:0/1276676702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0003a60 0x7fd6bc151250 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6c2a00640 1 -- 172.21.15.50:0/1276676702 >> 172.21.15.50:0/1276676702 conn(0x7fd6bc0b2040 msgr2=0x7fd6bc159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:21.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.034+0000 7fd6c2a00640 1 -- 172.21.15.50:0/1276676702 shutdown_connections 2024-09-21T06:12:21.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.038+0000 7fd6c2a00640 1 -- 172.21.15.50:0/1276676702 wait complete. 2024-09-21T06:12:21.056 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T06:12:21.057 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T06:12:21.057 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:12:21.057 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:12:21.057 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T06:12:21.058 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T06:12:21.058 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:12:21.058 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:12:21.058 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T06:12:21.059 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T06:12:21.059 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:12:21.059 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:12:21.059 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T06:12:21.059 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T06:12:21.060 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:12:21.060 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:12:21.071 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:12:21.071 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T06:12:21.074 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:12:21.074 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T06:12:21.075 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:12:21.075 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T06:12:21.081 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:12:21.081 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T06:12:21.124 INFO:tasks.ceph.mds.b:Started 2024-09-21T06:12:21.124 DEBUG:teuthology.parallel:result is None 2024-09-21T06:12:21.125 INFO:tasks.ceph.mds.a:Started 2024-09-21T06:12:21.125 DEBUG:teuthology.parallel:result is None 2024-09-21T06:12:21.126 INFO:tasks.ceph.mds.d:Started 2024-09-21T06:12:21.126 DEBUG:teuthology.parallel:result is None 2024-09-21T06:12:21.127 INFO:tasks.ceph.mds.c:Started 2024-09-21T06:12:21.127 DEBUG:teuthology.parallel:result is None 2024-09-21T06:12:21.127 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T06:12:21.223 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-09-21T06:12:21.224 INFO:tasks.ceph.mds.d.smithi088.stdout:starting mds.d at 2024-09-21T06:12:21.224 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-09-21T06:12:21.226 INFO:tasks.ceph.mds.b.smithi088.stdout:starting mds.b at 2024-09-21T06:12:21.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.294+0000 7f8d86a00640 1 -- 172.21.15.50:0/2564596389 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d800b5c70 msgr2=0x7f8d8015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:21.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.294+0000 7f8d86a00640 1 --2- 172.21.15.50:0/2564596389 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d800b5c70 0x7f8d8015bb60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009bf0 tx=0x7f8d7001cb20 comp rx=0 tx=0).stop 2024-09-21T06:12:21.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.294+0000 7f8d86a00640 1 -- 172.21.15.50:0/2564596389 shutdown_connections 2024-09-21T06:12:21.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.294+0000 7f8d86a00640 1 --2- 172.21.15.50:0/2564596389 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d800b5c70 0x7f8d8015bb60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.294+0000 7f8d86a00640 1 -- 172.21.15.50:0/2564596389 >> 172.21.15.50:0/2564596389 conn(0x7f8d800b2040 msgr2=0x7f8d800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:21.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.294+0000 7f8d86a00640 1 -- 172.21.15.50:0/2564596389 shutdown_connections 2024-09-21T06:12:21.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.294+0000 7f8d86a00640 1 -- 172.21.15.50:0/2564596389 wait complete. 2024-09-21T06:12:21.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.294+0000 7f8d86a00640 1 Processor -- start 2024-09-21T06:12:21.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.294+0000 7f8d86a00640 1 -- start start 2024-09-21T06:12:21.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d86a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d800b4de0 0x7f8d80135900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:21.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d86a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d800b5c70 0x7f8d80138380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:21.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d86a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d801527b0 0x7f8d801388c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:21.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d86a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80169c60 con 0x7f8d801527b0 2024-09-21T06:12:21.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d86a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80169ae0 con 0x7f8d800b4de0 2024-09-21T06:12:21.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d86a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8d80169de0 con 0x7f8d800b5c70 2024-09-21T06:12:21.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d84a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d801527b0 0x7f8d801388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:21.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d84a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d801527b0 0x7f8d801388c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47346/0 (socket says 172.21.15.50:47346) 2024-09-21T06:12:21.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d84a00640 1 -- 172.21.15.50:0/594706592 learned_addr learned my addr 172.21.15.50:0/594706592 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:21.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d7fe00640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d800b4de0 0x7f8d80135900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:21.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d77400640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d800b5c70 0x7f8d80138380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d7fe00640 1 -- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d800b5c70 msgr2=0x7f8d80138380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d7fe00640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d800b5c70 0x7f8d80138380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d7fe00640 1 -- 172.21.15.50:0/594706592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d801527b0 msgr2=0x7f8d801388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d7fe00640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d801527b0 0x7f8d801388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d7fe00640 1 -- 172.21.15.50:0/594706592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8014ff50 con 0x7f8d800b4de0 2024-09-21T06:12:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d7fe00640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d800b4de0 0x7f8d80135900 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8d800b34f0 tx=0x7f8d70005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d7cc00640 1 -- 172.21.15.50:0/594706592 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d700bc070 con 0x7f8d800b4de0 2024-09-21T06:12:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d7cc00640 1 -- 172.21.15.50:0/594706592 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d700040b0 con 0x7f8d800b4de0 2024-09-21T06:12:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d86a00640 1 -- 172.21.15.50:0/594706592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d80159020 con 0x7f8d800b4de0 2024-09-21T06:12:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d7cc00640 1 -- 172.21.15.50:0/594706592 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d700b2040 con 0x7f8d800b4de0 2024-09-21T06:12:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d86a00640 1 -- 172.21.15.50:0/594706592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d8014d1b0 con 0x7f8d800b4de0 2024-09-21T06:12:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.298+0000 7f8d86a00640 1 -- 172.21.15.50:0/594706592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d8015bb60 con 0x7f8d800b4de0 2024-09-21T06:12:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.302+0000 7f8d7cc00640 1 -- 172.21.15.50:0/594706592 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f8d700bf070 con 0x7f8d800b4de0 2024-09-21T06:12:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.302+0000 7f8d7cc00640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8d5c075510 0x7f8d5c077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.302+0000 7f8d7cc00640 1 -- 172.21.15.50:0/594706592 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5027+0+0 (secure 0 0 0) 0x7f8d7012cf30 con 0x7f8d800b4de0 2024-09-21T06:12:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.302+0000 7f8d77400640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8d5c075510 0x7f8d5c077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:21.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.302+0000 7f8d77400640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8d5c075510 0x7f8d5c077a20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8d68007fd0 tx=0x7f8d68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:21.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.306+0000 7f8d7cc00640 1 -- 172.21.15.50:0/594706592 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8d700167d0 con 0x7f8d800b4de0 2024-09-21T06:12:21.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.494+0000 7f8d86a00640 1 -- 172.21.15.50:0/594706592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8d8014f490 con 0x7f8d800b4de0 2024-09-21T06:12:21.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.574+0000 7f8d7cc00640 1 -- 172.21.15.50:0/594706592 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v136) ==== 97+0+0 (secure 0 0 0) 0x7f8d700f93e0 con 0x7f8d800b4de0 2024-09-21T06:12:21.573 INFO:teuthology.orchestra.run.smithi050.stderr: removed all blocklist entries 2024-09-21T06:12:21.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.578+0000 7f8d86a00640 1 -- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8d5c075510 msgr2=0x7f8d5c077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:21.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.578+0000 7f8d86a00640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8d5c075510 0x7f8d5c077a20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8d68007fd0 tx=0x7f8d68009040 comp rx=0 tx=0).stop 2024-09-21T06:12:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.578+0000 7f8d86a00640 1 -- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d800b4de0 msgr2=0x7f8d80135900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.578+0000 7f8d86a00640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d800b4de0 0x7f8d80135900 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8d800b34f0 tx=0x7f8d70005b00 comp rx=0 tx=0).stop 2024-09-21T06:12:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.578+0000 7f8d86a00640 1 -- 172.21.15.50:0/594706592 shutdown_connections 2024-09-21T06:12:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.578+0000 7f8d86a00640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8d5c075510 0x7f8d5c077a20 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.578+0000 7f8d86a00640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d801527b0 0x7f8d801388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.578+0000 7f8d86a00640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8d800b5c70 0x7f8d80138380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.578+0000 7f8d86a00640 1 --2- 172.21.15.50:0/594706592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d800b4de0 0x7f8d80135900 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.578+0000 7f8d86a00640 1 -- 172.21.15.50:0/594706592 >> 172.21.15.50:0/594706592 conn(0x7f8d800b2040 msgr2=0x7f8d800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.578+0000 7f8d86a00640 1 -- 172.21.15.50:0/594706592 shutdown_connections 2024-09-21T06:12:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.578+0000 7f8d86a00640 1 -- 172.21.15.50:0/594706592 wait complete. 2024-09-21T06:12:21.598 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T06:12:21.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.758+0000 7f830b200640 1 -- 172.21.15.50:0/1027435959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f830c160060 msgr2=0x7f830c074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:21.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.758+0000 7f830b200640 1 --2- 172.21.15.50:0/1027435959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f830c160060 0x7f830c074240 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009900 tx=0x7f82f401c850 comp rx=0 tx=0).stop 2024-09-21T06:12:21.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.758+0000 7f830b200640 1 -- 172.21.15.50:0/1027435959 shutdown_connections 2024-09-21T06:12:21.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.758+0000 7f830b200640 1 --2- 172.21.15.50:0/1027435959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f830c160060 0x7f830c074240 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.758+0000 7f830b200640 1 --2- 172.21.15.50:0/1027435959 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f830c1536f0 0x7f830c15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.758+0000 7f830b200640 1 -- 172.21.15.50:0/1027435959 >> 172.21.15.50:0/1027435959 conn(0x7f830c0b4560 msgr2=0x7f830c0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:21.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.758+0000 7f830b200640 1 -- 172.21.15.50:0/1027435959 shutdown_connections 2024-09-21T06:12:21.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.758+0000 7f830b200640 1 -- 172.21.15.50:0/1027435959 wait complete. 2024-09-21T06:12:21.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830b200640 1 Processor -- start 2024-09-21T06:12:21.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830b200640 1 -- start start 2024-09-21T06:12:21.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f830c1536f0 0x7f830c0d1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:21.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830b200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f830c160060 0x7f830c1fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:21.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f830c160470 0x7f830c1fa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:21.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f830c16a490 con 0x7f830c160470 2024-09-21T06:12:21.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f830c16a310 con 0x7f830c1536f0 2024-09-21T06:12:21.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830b200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f830c16a610 con 0x7f830c160060 2024-09-21T06:12:21.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f830c160470 0x7f830c1fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f830c160470 0x7f830c1fa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47354/0 (socket says 172.21.15.50:47354) 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830a800640 1 -- 172.21.15.50:0/2505427063 learned_addr learned my addr 172.21.15.50:0/2505427063 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f8309e00640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f830c1536f0 0x7f830c0d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f8303e00640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f830c160060 0x7f830c1fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830a800640 1 -- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f830c160060 msgr2=0x7f830c1fa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830a800640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f830c160060 0x7f830c1fa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830a800640 1 -- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f830c1536f0 msgr2=0x7f830c0d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830a800640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f830c1536f0 0x7f830c0d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f830a800640 1 -- 172.21.15.50:0/2505427063 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c144960 con 0x7f830c160470 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.762+0000 7f8309e00640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f830c1536f0 0x7f830c0d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.766+0000 7f830a800640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f830c160470 0x7f830c1fa900 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f830400c730 tx=0x7f830400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.766+0000 7f8302000640 1 -- 172.21.15.50:0/2505427063 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304003b80 con 0x7f830c160470 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.766+0000 7f8302000640 1 -- 172.21.15.50:0/2505427063 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8304003d20 con 0x7f830c160470 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.766+0000 7f8302000640 1 -- 172.21.15.50:0/2505427063 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304005330 con 0x7f830c160470 2024-09-21T06:12:21.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.766+0000 7f830b200640 1 -- 172.21.15.50:0/2505427063 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c1452b0 con 0x7f830c160470 2024-09-21T06:12:21.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.766+0000 7f830b200640 1 -- 172.21.15.50:0/2505427063 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c145870 con 0x7f830c160470 2024-09-21T06:12:21.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.766+0000 7f830b200640 1 -- 172.21.15.50:0/2505427063 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f830c15d480 con 0x7f830c160470 2024-09-21T06:12:21.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.766+0000 7f8302000640 1 -- 172.21.15.50:0/2505427063 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f83040056e0 con 0x7f830c160470 2024-09-21T06:12:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.766+0000 7f8302000640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f82e80754e0 0x7f82e80779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:21.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.770+0000 7f8302000640 1 -- 172.21.15.50:0/2505427063 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4941+0+0 (secure 0 0 0) 0x7f8304093220 con 0x7f830c160470 2024-09-21T06:12:21.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.770+0000 7f8309e00640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f82e80754e0 0x7f82e80779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:21.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.770+0000 7f8309e00640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f82e80754e0 0x7f82e80779f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00bf40 tx=0x7f82fc00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:21.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:21.774+0000 7f8302000640 1 -- 172.21.15.50:0/2505427063 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f830c15d9c0 con 0x7f830c160470 2024-09-21T06:12:22.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 -- 172.21.15.50:0/2505427063 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f830c145040 con 0x7f830c160470 2024-09-21T06:12:22.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f8302000640 1 -- 172.21.15.50:0/2505427063 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v39) ==== 80+0+5304 (secure 0 0 0) 0x7f830405f6e0 con 0x7f830c160470 2024-09-21T06:12:22.011 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: "auth_dump": [ 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.a", 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mSKq0ARAA2egaHGYWhRUEq6oRl1CIMg==", 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.012 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.b", 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mwv0+DxAAY+eZPxv5uY9PnLl0asdr6w==", 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.c", 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5me6maCRAA4jvHgTLTKpb/mxzzG71SfQ==", 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:12:22.013 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.d", 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mzYnuFhAA/hEigjluXspzs/hyeWxBjQ==", 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.014 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.015 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.0", 2024-09-21T06:12:22.015 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDIX+5mrZF/NhAA33M6ummG4r4yXU8/YOL5WA==", 2024-09-21T06:12:22.015 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.015 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:12:22.015 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:12:22.015 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.015 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.015 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.015 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.015 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.1", 2024-09-21T06:12:22.015 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDMX+5myVz5DxAAMYfTPep6ubD7Cxnfa9dMCg==", 2024-09-21T06:12:22.015 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.2", 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDPX+5mA7ahIRAAf/osnY5l83p8ZWna0ErD2Q==", 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.3", 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDSX+5m+rDpNRAApGPZJIlAaA0DyjtPXPgoaQ==", 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.4", 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDYX+5mKL9SORAA21ITQUwUSoDUjXaD0Cwb3g==", 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.017 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.5", 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDcX+5minRqFBAAMJOBQNvJFD4TNdKbPMLBig==", 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.018 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.6", 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDfX+5m6HadJhAAAv670Tvyi4IKMByFPiBlAw==", 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.7", 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDjX+5muPrrAhAAuRB7Muamiht+f7LvSmI6UQ==", 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-09-21T06:12:22.019 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.0", 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw==", 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow r", 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.1", 2024-09-21T06:12:22.020 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mo94GJBAAMBSb1NPb2L93vzvnAh59kw==", 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.2", 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5m4IrnHxAAFjER3ywuS+FlbuA7bPZVqw==", 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:12:22.021 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.3", 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDHX+5mysQeKBAAq5c4bD3hbsU1H7JabaMjwQ==", 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.admin", 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mcUXKHhAAV1Rs0OMNVPeAsRiAVt67jA==", 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.x", 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5m2F9WOBAAZuQTUyR7oTT01CRD1jo+/Q==", 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:12:22.023 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.y", 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQDGX+5mEjC/MxAAHgwIu4CLDgFajwD2rL5WKA==", 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-21T06:12:22.024 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:12:22.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 -- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f82e80754e0 msgr2=0x7f82e80779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f82e80754e0 0x7f82e80779f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00bf40 tx=0x7f82fc00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 -- 172.21.15.50:0/2505427063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f830c160470 msgr2=0x7f830c1fa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f830c160470 0x7f830c1fa900 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f830400c730 tx=0x7f830400cbf0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 -- 172.21.15.50:0/2505427063 shutdown_connections 2024-09-21T06:12:22.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f82e80754e0 0x7f82e80779f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f830c160470 0x7f830c1fa900 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f830c160060 0x7f830c1fa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 --2- 172.21.15.50:0/2505427063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f830c1536f0 0x7f830c0d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 -- 172.21.15.50:0/2505427063 >> 172.21.15.50:0/2505427063 conn(0x7f830c0b4560 msgr2=0x7f830c0b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:22.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 -- 172.21.15.50:0/2505427063 shutdown_connections 2024-09-21T06:12:22.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.014+0000 7f830b200640 1 -- 172.21.15.50:0/2505427063 wait complete. 2024-09-21T06:12:22.034 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T06:12:22.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.194+0000 7ff8ac200640 1 -- 172.21.15.50:0/2696854148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8a40b5c70 msgr2=0x7ff8a415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.194+0000 7ff8ac200640 1 --2- 172.21.15.50:0/2696854148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8a40b5c70 0x7ff8a415bb60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff898009840 tx=0x7ff89801c760 comp rx=0 tx=0).stop 2024-09-21T06:12:22.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.194+0000 7ff8ac200640 1 -- 172.21.15.50:0/2696854148 shutdown_connections 2024-09-21T06:12:22.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.194+0000 7ff8ac200640 1 --2- 172.21.15.50:0/2696854148 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8a41527b0 0x7ff8a4074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.194+0000 7ff8ac200640 1 --2- 172.21.15.50:0/2696854148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8a40b5c70 0x7ff8a415bb60 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.194+0000 7ff8ac200640 1 -- 172.21.15.50:0/2696854148 >> 172.21.15.50:0/2696854148 conn(0x7ff8a40b2040 msgr2=0x7ff8a40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:22.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.194+0000 7ff8ac200640 1 -- 172.21.15.50:0/2696854148 shutdown_connections 2024-09-21T06:12:22.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.194+0000 7ff8ac200640 1 -- 172.21.15.50:0/2696854148 wait complete. 2024-09-21T06:12:22.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8ac200640 1 Processor -- start 2024-09-21T06:12:22.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8ac200640 1 -- start start 2024-09-21T06:12:22.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8ac200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8a40b4de0 0x7ff8a41f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8ac200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8a40b5c70 0x7ff8a4158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8ac200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8a41527b0 0x7ff8a414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8ac200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4168bd0 con 0x7ff8a40b4de0 2024-09-21T06:12:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8ac200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4168a50 con 0x7ff8a40b5c70 2024-09-21T06:12:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8ac200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a4168d50 con 0x7ff8a41527b0 2024-09-21T06:12:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8a9800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8a40b4de0 0x7ff8a41f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8a9800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8a40b4de0 0x7ff8a41f6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47368/0 (socket says 172.21.15.50:47368) 2024-09-21T06:12:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8a9800640 1 -- 172.21.15.50:0/2460449032 learned_addr learned my addr 172.21.15.50:0/2460449032 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8a8e00640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8a40b5c70 0x7ff8a4158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8aa200640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8a41527b0 0x7ff8a414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8a9800640 1 -- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8a41527b0 msgr2=0x7ff8a414c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8a9800640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8a41527b0 0x7ff8a414c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8a9800640 1 -- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8a40b5c70 msgr2=0x7ff8a4158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8a9800640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8a40b5c70 0x7ff8a4158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8a9800640 1 -- 172.21.15.50:0/2460449032 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a414a890 con 0x7ff8a40b4de0 2024-09-21T06:12:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8a9800640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8a40b4de0 0x7ff8a41f6820 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff89801cc40 tx=0x7ff898005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8a1600640 1 -- 172.21.15.50:0/2460449032 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8980b2040 con 0x7ff8a40b4de0 2024-09-21T06:12:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8a1600640 1 -- 172.21.15.50:0/2460449032 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8980041f0 con 0x7ff8a40b4de0 2024-09-21T06:12:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.198+0000 7ff8ac200640 1 -- 172.21.15.50:0/2460449032 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a414ab70 con 0x7ff8a40b4de0 2024-09-21T06:12:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.202+0000 7ff8a1600640 1 -- 172.21.15.50:0/2460449032 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898003600 con 0x7ff8a40b4de0 2024-09-21T06:12:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.202+0000 7ff8ac200640 1 -- 172.21.15.50:0/2460449032 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a414b130 con 0x7ff8a40b4de0 2024-09-21T06:12:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.202+0000 7ff8ac200640 1 -- 172.21.15.50:0/2460449032 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a415bb60 con 0x7ff8a40b4de0 2024-09-21T06:12:22.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.202+0000 7ff8a1600640 1 -- 172.21.15.50:0/2460449032 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7ff8980bd030 con 0x7ff8a40b4de0 2024-09-21T06:12:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.202+0000 7ff8a1600640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8840754e0 0x7ff8840779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.202+0000 7ff8a1600640 1 -- 172.21.15.50:0/2460449032 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4941+0+0 (secure 0 0 0) 0x7ff89812d000 con 0x7ff8a40b4de0 2024-09-21T06:12:22.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.206+0000 7ff8a8e00640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8840754e0 0x7ff8840779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.206+0000 7ff8a8e00640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8840754e0 0x7ff8840779f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff8a414d310 tx=0x7ff8900091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:22.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.210+0000 7ff8a1600640 1 -- 172.21.15.50:0/2460449032 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff8980c4020 con 0x7ff8a40b4de0 2024-09-21T06:12:22.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.386+0000 7ff8ac200640 1 -- 172.21.15.50:0/2460449032 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff8a4150b70 con 0x7ff8a40b4de0 2024-09-21T06:12:22.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.386+0000 7ff8a1600640 1 -- 172.21.15.50:0/2460449032 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v138) ==== 84+0+65 (secure 0 0 0) 0x7ff8a4075580 con 0x7ff8a40b4de0 2024-09-21T06:12:22.385 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:22.386 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:12:22.386 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:22.386 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-09-21T06:12:22.386 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-09-21T06:12:22.386 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:22.386 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:12:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.390+0000 7ff8ac200640 1 -- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8840754e0 msgr2=0x7ff8840779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.390+0000 7ff8ac200640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8840754e0 0x7ff8840779f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff8a414d310 tx=0x7ff8900091e0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.390+0000 7ff8ac200640 1 -- 172.21.15.50:0/2460449032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8a40b4de0 msgr2=0x7ff8a41f6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.390+0000 7ff8ac200640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8a40b4de0 0x7ff8a41f6820 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff89801cc40 tx=0x7ff898005db0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.390+0000 7ff8ac200640 1 -- 172.21.15.50:0/2460449032 shutdown_connections 2024-09-21T06:12:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.390+0000 7ff8ac200640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff8a41527b0 0x7ff8a414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.390+0000 7ff8ac200640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8a40b5c70 0x7ff8a4158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.390+0000 7ff8ac200640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff8840754e0 0x7ff8840779f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.390+0000 7ff8ac200640 1 --2- 172.21.15.50:0/2460449032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8a40b4de0 0x7ff8a41f6820 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.390+0000 7ff8ac200640 1 -- 172.21.15.50:0/2460449032 >> 172.21.15.50:0/2460449032 conn(0x7ff8a40b2040 msgr2=0x7ff8a40b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:22.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.390+0000 7ff8ac200640 1 -- 172.21.15.50:0/2460449032 shutdown_connections 2024-09-21T06:12:22.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.390+0000 7ff8ac200640 1 -- 172.21.15.50:0/2460449032 wait complete. 2024-09-21T06:12:22.409 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:12:22.409 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:12:22.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.574+0000 7f24cd000640 1 -- 172.21.15.50:0/887265124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c80b5a80 msgr2=0x7f24c8074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.574+0000 7f24cd000640 1 --2- 172.21.15.50:0/887265124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c80b5a80 0x7f24c8074410 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009900 tx=0x7f24b001c850 comp rx=0 tx=0).stop 2024-09-21T06:12:22.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.574+0000 7f24cd000640 1 -- 172.21.15.50:0/887265124 shutdown_connections 2024-09-21T06:12:22.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.574+0000 7f24cd000640 1 --2- 172.21.15.50:0/887265124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c80b5a80 0x7f24c8074410 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.574+0000 7f24cd000640 1 -- 172.21.15.50:0/887265124 >> 172.21.15.50:0/887265124 conn(0x7f24c80708c0 msgr2=0x7f24c8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:22.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.574+0000 7f24cd000640 1 -- 172.21.15.50:0/887265124 shutdown_connections 2024-09-21T06:12:22.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.574+0000 7f24cd000640 1 -- 172.21.15.50:0/887265124 wait complete. 2024-09-21T06:12:22.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.574+0000 7f24cd000640 1 Processor -- start 2024-09-21T06:12:22.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.574+0000 7f24cd000640 1 -- start start 2024-09-21T06:12:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24cd000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24c8074950 0x7f24c815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24cd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24c80b5a80 0x7f24c81fa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24cd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c8157030 0x7f24c81fa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24cd000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24c816b650 con 0x7f24c8157030 2024-09-21T06:12:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24cd000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f24c816b4d0 con 0x7f24c80b5a80 2024-09-21T06:12:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24cd000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f24c816b7d0 con 0x7f24c8074950 2024-09-21T06:12:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24c6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c8157030 0x7f24c81fa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24c6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c8157030 0x7f24c81fa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47388/0 (socket says 172.21.15.50:47388) 2024-09-21T06:12:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24c6800640 1 -- 172.21.15.50:0/245692534 learned_addr learned my addr 172.21.15.50:0/245692534 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24bfe00640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24c80b5a80 0x7f24c81fa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24c5e00640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24c8074950 0x7f24c815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24c6800640 1 -- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24c8074950 msgr2=0x7f24c815c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24c6800640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24c8074950 0x7f24c815c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24c6800640 1 -- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24c80b5a80 msgr2=0x7f24c81fa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24c6800640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24c80b5a80 0x7f24c81fa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24c6800640 1 -- 172.21.15.50:0/245692534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c81fb2f0 con 0x7f24c8157030 2024-09-21T06:12:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24c6800640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c8157030 0x7f24c81fa9d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f24c000c710 tx=0x7f24c000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24be000640 1 -- 172.21.15.50:0/245692534 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0003b80 con 0x7f24c8157030 2024-09-21T06:12:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24be000640 1 -- 172.21.15.50:0/245692534 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24c0003d20 con 0x7f24c8157030 2024-09-21T06:12:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24be000640 1 -- 172.21.15.50:0/245692534 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c000aab0 con 0x7f24c8157030 2024-09-21T06:12:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24cd000640 1 -- 172.21.15.50:0/245692534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c8157e80 con 0x7f24c8157030 2024-09-21T06:12:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24cd000640 1 -- 172.21.15.50:0/245692534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c80b2ec0 con 0x7f24c8157030 2024-09-21T06:12:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.578+0000 7f24cd000640 1 -- 172.21.15.50:0/245692534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c8074410 con 0x7f24c8157030 2024-09-21T06:12:22.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.582+0000 7f24be000640 1 -- 172.21.15.50:0/245692534 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f24c000ac50 con 0x7f24c8157030 2024-09-21T06:12:22.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.582+0000 7f24be000640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f24a4075510 0x7f24a4077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.582+0000 7f24be000640 1 -- 172.21.15.50:0/245692534 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4941+0+0 (secure 0 0 0) 0x7f24c0093100 con 0x7f24c8157030 2024-09-21T06:12:22.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.582+0000 7f24c5e00640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f24a4075510 0x7f24a4077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.582+0000 7f24c5e00640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f24a4075510 0x7f24a4077a20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f24b800b810 tx=0x7f24b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:22.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.586+0000 7f24be000640 1 -- 172.21.15.50:0/245692534 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f24c81fb2f0 con 0x7f24c8157030 2024-09-21T06:12:22.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:22.762+0000 7f24cd000640 1 -- 172.21.15.50:0/245692534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f24c80d2260 con 0x7f24c8157030 2024-09-21T06:12:23.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.582+0000 7f24be000640 1 -- 172.21.15.50:0/245692534 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v139) ==== 138+0+0 (secure 0 0 0) 0x7f24c8074d60 con 0x7f24c8157030 2024-09-21T06:12:23.580 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-09-21T06:12:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.586+0000 7f24cd000640 1 -- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f24a4075510 msgr2=0x7f24a4077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.586+0000 7f24cd000640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f24a4075510 0x7f24a4077a20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f24b800b810 tx=0x7f24b8008040 comp rx=0 tx=0).stop 2024-09-21T06:12:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.586+0000 7f24cd000640 1 -- 172.21.15.50:0/245692534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c8157030 msgr2=0x7f24c81fa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.586+0000 7f24cd000640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c8157030 0x7f24c81fa9d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f24c000c710 tx=0x7f24c000cbd0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.586+0000 7f24cd000640 1 -- 172.21.15.50:0/245692534 shutdown_connections 2024-09-21T06:12:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.586+0000 7f24cd000640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f24a4075510 0x7f24a4077a20 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.586+0000 7f24cd000640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c8157030 0x7f24c81fa9d0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.586+0000 7f24cd000640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24c80b5a80 0x7f24c81fa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.586+0000 7f24cd000640 1 --2- 172.21.15.50:0/245692534 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f24c8074950 0x7f24c815c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.586+0000 7f24cd000640 1 -- 172.21.15.50:0/245692534 >> 172.21.15.50:0/245692534 conn(0x7f24c80708c0 msgr2=0x7f24c815e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.586+0000 7f24cd000640 1 -- 172.21.15.50:0/245692534 shutdown_connections 2024-09-21T06:12:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.586+0000 7f24cd000640 1 -- 172.21.15.50:0/245692534 wait complete. 2024-09-21T06:12:23.613 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:12:23.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.786+0000 7fbfb4c00640 1 -- 172.21.15.50:0/1082287215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb01536f0 msgr2=0x7fbfb015d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:23.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.786+0000 7fbfb4c00640 1 --2- 172.21.15.50:0/1082287215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb01536f0 0x7fbfb015d480 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009900 tx=0x7fbfa001c850 comp rx=0 tx=0).stop 2024-09-21T06:12:23.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.786+0000 7fbfb4c00640 1 -- 172.21.15.50:0/1082287215 shutdown_connections 2024-09-21T06:12:23.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.786+0000 7fbfb4c00640 1 --2- 172.21.15.50:0/1082287215 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbfb0160060 0x7fbfb0074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.786+0000 7fbfb4c00640 1 --2- 172.21.15.50:0/1082287215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb01536f0 0x7fbfb015d480 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.786+0000 7fbfb4c00640 1 -- 172.21.15.50:0/1082287215 >> 172.21.15.50:0/1082287215 conn(0x7fbfb00b4560 msgr2=0x7fbfb00d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:23.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.786+0000 7fbfb4c00640 1 -- 172.21.15.50:0/1082287215 shutdown_connections 2024-09-21T06:12:23.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.786+0000 7fbfb4c00640 1 -- 172.21.15.50:0/1082287215 wait complete. 2024-09-21T06:12:23.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfb4c00640 1 Processor -- start 2024-09-21T06:12:23.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfb4c00640 1 -- start start 2024-09-21T06:12:23.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfb4c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbfb01536f0 0x7fbfb014b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:23.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfb4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb0160060 0x7fbfb0142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:23.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfb4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfb0160470 0x7fbfb00d1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:23.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfb4c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb016a490 con 0x7fbfb0160470 2024-09-21T06:12:23.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfb4c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb016a310 con 0x7fbfb0160060 2024-09-21T06:12:23.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfb4c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb016a610 con 0x7fbfb01536f0 2024-09-21T06:12:23.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfafe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfb0160470 0x7fbfb00d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:23.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfa6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb0160060 0x7fbfb0142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:23.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfaf400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbfb01536f0 0x7fbfb014b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:23.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfafe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfb0160470 0x7fbfb00d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47394/0 (socket says 172.21.15.50:47394) 2024-09-21T06:12:23.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.790+0000 7fbfafe00640 1 -- 172.21.15.50:0/2189603508 learned_addr learned my addr 172.21.15.50:0/2189603508 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:23.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfaf400640 1 -- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb0160060 msgr2=0x7fbfb0142d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:23.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfaf400640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb0160060 0x7fbfb0142d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfaf400640 1 -- 172.21.15.50:0/2189603508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfb0160470 msgr2=0x7fbfb00d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:23.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfaf400640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfb0160470 0x7fbfb00d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfaf400640 1 -- 172.21.15.50:0/2189603508 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb01fa800 con 0x7fbfb01536f0 2024-09-21T06:12:23.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfafe00640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfb0160470 0x7fbfb00d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:23.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfaf400640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbfb01536f0 0x7fbfb014b020 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00d2aa0 tx=0x7fbfa0005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:23.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfa7e00640 1 -- 172.21.15.50:0/2189603508 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa00ba020 con 0x7fbfb01536f0 2024-09-21T06:12:23.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfa7e00640 1 -- 172.21.15.50:0/2189603508 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfa0004320 con 0x7fbfb01536f0 2024-09-21T06:12:23.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfb4c00640 1 -- 172.21.15.50:0/2189603508 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb01faae0 con 0x7fbfb01536f0 2024-09-21T06:12:23.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfa7e00640 1 -- 172.21.15.50:0/2189603508 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa00b2040 con 0x7fbfb01536f0 2024-09-21T06:12:23.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfb4c00640 1 -- 172.21.15.50:0/2189603508 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb01fb0a0 con 0x7fbfb01536f0 2024-09-21T06:12:23.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfb4c00640 1 -- 172.21.15.50:0/2189603508 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb015d480 con 0x7fbfb01536f0 2024-09-21T06:12:23.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.794+0000 7fbfa7e00640 1 -- 172.21.15.50:0/2189603508 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fbfa0003860 con 0x7fbfb01536f0 2024-09-21T06:12:23.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.798+0000 7fbfa7e00640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbf8c075510 0x7fbf8c077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:23.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.798+0000 7fbfa7e00640 1 -- 172.21.15.50:0/2189603508 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(139..139 src has 1..139) ==== 5311+0+0 (secure 0 0 0) 0x7fbfa012d260 con 0x7fbfb01536f0 2024-09-21T06:12:23.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.798+0000 7fbfa6a00640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbf8c075510 0x7fbf8c077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:23.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.798+0000 7fbfa6a00640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbf8c075510 0x7fbf8c077a20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0149f90 tx=0x7fbf98012040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:23.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.802+0000 7fbfa7e00640 1 -- 172.21.15.50:0/2189603508 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbfa00c4060 con 0x7fbfb01536f0 2024-09-21T06:12:23.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:23.978+0000 7fbfb4c00640 1 -- 172.21.15.50:0/2189603508 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbfb00720a0 con 0x7fbfb01536f0 2024-09-21T06:12:24.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.598+0000 7fbfa7e00640 1 -- 172.21.15.50:0/2189603508 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v140) ==== 170+0+0 (secure 0 0 0) 0x7fbfa00164d0 con 0x7fbfb01536f0 2024-09-21T06:12:24.597 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-09-21T06:12:24.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.606+0000 7fbfb4c00640 1 -- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbf8c075510 msgr2=0x7fbf8c077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:24.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.606+0000 7fbfb4c00640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbf8c075510 0x7fbf8c077a20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0149f90 tx=0x7fbf98012040 comp rx=0 tx=0).stop 2024-09-21T06:12:24.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.606+0000 7fbfb4c00640 1 -- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbfb01536f0 msgr2=0x7fbfb014b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:24.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.606+0000 7fbfb4c00640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbfb01536f0 0x7fbfb014b020 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00d2aa0 tx=0x7fbfa0005790 comp rx=0 tx=0).stop 2024-09-21T06:12:24.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.606+0000 7fbfb4c00640 1 -- 172.21.15.50:0/2189603508 shutdown_connections 2024-09-21T06:12:24.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.606+0000 7fbfb4c00640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbf8c075510 0x7fbf8c077a20 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.606+0000 7fbfb4c00640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfb0160470 0x7fbfb00d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.606+0000 7fbfb4c00640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb0160060 0x7fbfb0142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.606+0000 7fbfb4c00640 1 --2- 172.21.15.50:0/2189603508 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbfb01536f0 0x7fbfb014b020 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.606+0000 7fbfb4c00640 1 -- 172.21.15.50:0/2189603508 >> 172.21.15.50:0/2189603508 conn(0x7fbfb00b4560 msgr2=0x7fbfb00715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:24.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.606+0000 7fbfb4c00640 1 -- 172.21.15.50:0/2189603508 shutdown_connections 2024-09-21T06:12:24.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.606+0000 7fbfb4c00640 1 -- 172.21.15.50:0/2189603508 wait complete. 2024-09-21T06:12:24.634 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:12:24.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.798+0000 7f80fca00640 1 -- 172.21.15.50:0/109984031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80f80b1dc0 msgr2=0x7f80f80b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:24.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.798+0000 7f80fca00640 1 --2- 172.21.15.50:0/109984031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80f80b1dc0 0x7f80f80b21d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009870 tx=0x7f80e001c760 comp rx=0 tx=0).stop 2024-09-21T06:12:24.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.798+0000 7f80fca00640 1 -- 172.21.15.50:0/109984031 shutdown_connections 2024-09-21T06:12:24.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.798+0000 7f80fca00640 1 --2- 172.21.15.50:0/109984031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80f80b1dc0 0x7f80f80b21d0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.798+0000 7f80fca00640 1 -- 172.21.15.50:0/109984031 >> 172.21.15.50:0/109984031 conn(0x7f80f806cdf0 msgr2=0x7f80f806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:24.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.798+0000 7f80fca00640 1 -- 172.21.15.50:0/109984031 shutdown_connections 2024-09-21T06:12:24.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.798+0000 7f80fca00640 1 -- 172.21.15.50:0/109984031 wait complete. 2024-09-21T06:12:24.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80fca00640 1 Processor -- start 2024-09-21T06:12:24.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80fca00640 1 -- start start 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80fca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80f80b1dc0 0x7f80f814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80fca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80f80b3130 0x7f80f815e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80fca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80f8155a00 0x7f80f81f9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80fca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8169ba0 con 0x7f80f80b3130 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80fca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8169a20 con 0x7f80f8155a00 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80fca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f80f8169d20 con 0x7f80f80b1dc0 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80f80b3130 0x7f80f815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80f80b3130 0x7f80f815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47400/0 (socket says 172.21.15.50:47400) 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80efe00640 1 -- 172.21.15.50:0/3632443244 learned_addr learned my addr 172.21.15.50:0/3632443244 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80f7e00640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80f8155a00 0x7f80f81f9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80f7400640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80f80b1dc0 0x7f80f814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80efe00640 1 -- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80f80b1dc0 msgr2=0x7f80f814d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:24.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80efe00640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80f80b1dc0 0x7f80f814d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80efe00640 1 -- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80f8155a00 msgr2=0x7f80f81f9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:24.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80efe00640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80f8155a00 0x7f80f81f9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80efe00640 1 -- 172.21.15.50:0/3632443244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f814fb10 con 0x7f80f80b3130 2024-09-21T06:12:24.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80efe00640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80f80b3130 0x7f80f815e480 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f80e0000c10 tx=0x7f80e00a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:24.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80ef400640 1 -- 172.21.15.50:0/3632443244 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e00b2040 con 0x7f80f80b3130 2024-09-21T06:12:24.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80ef400640 1 -- 172.21.15.50:0/3632443244 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80e00043c0 con 0x7f80f80b3130 2024-09-21T06:12:24.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80fca00640 1 -- 172.21.15.50:0/3632443244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f814fdf0 con 0x7f80f80b3130 2024-09-21T06:12:24.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80ef400640 1 -- 172.21.15.50:0/3632443244 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0003860 con 0x7f80f80b3130 2024-09-21T06:12:24.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.802+0000 7f80fca00640 1 -- 172.21.15.50:0/3632443244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f81503b0 con 0x7f80f80b3130 2024-09-21T06:12:24.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.806+0000 7f80fca00640 1 -- 172.21.15.50:0/3632443244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f814db70 con 0x7f80f80b3130 2024-09-21T06:12:24.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.806+0000 7f80ef400640 1 -- 172.21.15.50:0/3632443244 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f80e0003a00 con 0x7f80f80b3130 2024-09-21T06:12:24.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.810+0000 7f80ef400640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80d40754e0 0x7f80d40779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:24.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.810+0000 7f80ef400640 1 -- 172.21.15.50:0/3632443244 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5693+0+0 (secure 0 0 0) 0x7f80e012e0e0 con 0x7f80f80b3130 2024-09-21T06:12:24.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.810+0000 7f80f7400640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80d40754e0 0x7f80d40779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:24.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.810+0000 7f80f7400640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80d40754e0 0x7f80d40779f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f80e8007fd0 tx=0x7f80e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:24.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:24.814+0000 7f80ef400640 1 -- 172.21.15.50:0/3632443244 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f80e0016410 con 0x7f80f80b3130 2024-09-21T06:12:25.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.026+0000 7f80fca00640 1 -- 172.21.15.50:0/3632443244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f80f80b21d0 con 0x7f80f80b3130 2024-09-21T06:12:25.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.606+0000 7f80ef400640 1 -- 172.21.15.50:0/3632443244 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v90) ==== 174+0+0 (secure 0 0 0) 0x7f80e00fa620 con 0x7f80f80b3130 2024-09-21T06:12:25.612 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 20 and data pool 21 2024-09-21T06:12:25.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.618+0000 7f80fca00640 1 -- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80d40754e0 msgr2=0x7f80d40779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:25.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.618+0000 7f80fca00640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80d40754e0 0x7f80d40779f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f80e8007fd0 tx=0x7f80e8009040 comp rx=0 tx=0).stop 2024-09-21T06:12:25.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.618+0000 7f80fca00640 1 -- 172.21.15.50:0/3632443244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80f80b3130 msgr2=0x7f80f815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:25.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.618+0000 7f80fca00640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80f80b3130 0x7f80f815e480 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f80e0000c10 tx=0x7f80e00a57b0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.618+0000 7f80fca00640 1 -- 172.21.15.50:0/3632443244 shutdown_connections 2024-09-21T06:12:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.618+0000 7f80fca00640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80f8155a00 0x7f80f81f9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.618+0000 7f80fca00640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80d40754e0 0x7f80d40779f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.618+0000 7f80fca00640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80f80b3130 0x7f80f815e480 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.618+0000 7f80fca00640 1 --2- 172.21.15.50:0/3632443244 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80f80b1dc0 0x7f80f814d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.618+0000 7f80fca00640 1 -- 172.21.15.50:0/3632443244 >> 172.21.15.50:0/3632443244 conn(0x7f80f806cdf0 msgr2=0x7f80f80b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.618+0000 7f80fca00640 1 -- 172.21.15.50:0/3632443244 shutdown_connections 2024-09-21T06:12:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.618+0000 7f80fca00640 1 -- 172.21.15.50:0/3632443244 wait complete. 2024-09-21T06:12:25.651 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:12:25.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.810+0000 7f25d9200640 1 -- 172.21.15.50:0/2668417776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25d4160980 msgr2=0x7f25d4160d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:25.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.810+0000 7f25d9200640 1 --2- 172.21.15.50:0/2668417776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25d4160980 0x7f25d4160d90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009840 tx=0x7f25bc01c790 comp rx=0 tx=0).stop 2024-09-21T06:12:25.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.810+0000 7f25d9200640 1 -- 172.21.15.50:0/2668417776 shutdown_connections 2024-09-21T06:12:25.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.810+0000 7f25d9200640 1 --2- 172.21.15.50:0/2668417776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25d4160980 0x7f25d4160d90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.810+0000 7f25d9200640 1 -- 172.21.15.50:0/2668417776 >> 172.21.15.50:0/2668417776 conn(0x7f25d40d06e0 msgr2=0x7f25d40d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:25.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.814+0000 7f25d9200640 1 -- 172.21.15.50:0/2668417776 shutdown_connections 2024-09-21T06:12:25.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.814+0000 7f25d9200640 1 -- 172.21.15.50:0/2668417776 wait complete. 2024-09-21T06:12:25.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.814+0000 7f25d9200640 1 Processor -- start 2024-09-21T06:12:25.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.814+0000 7f25d9200640 1 -- start start 2024-09-21T06:12:25.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.814+0000 7f25d9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25d4070a30 0x7f25d40b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:25.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.814+0000 7f25d9200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25d40ce780 0x7f25d4167700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:25.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.814+0000 7f25d9200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25d4160980 0x7f25d4167c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:25.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.814+0000 7f25d9200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f25d416aef0 con 0x7f25d4070a30 2024-09-21T06:12:25.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.814+0000 7f25d9200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f25d416ad70 con 0x7f25d4160980 2024-09-21T06:12:25.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.814+0000 7f25d9200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f25d416b070 con 0x7f25d40ce780 2024-09-21T06:12:25.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.814+0000 7f25d1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25d4070a30 0x7f25d40b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:25.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25d1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25d4070a30 0x7f25d40b5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47410/0 (socket says 172.21.15.50:47410) 2024-09-21T06:12:25.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25d1e00640 1 -- 172.21.15.50:0/2257765893 learned_addr learned my addr 172.21.15.50:0/2257765893 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:25.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25cbe00640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25d40ce780 0x7f25d4167700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:25.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25d2800640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25d4160980 0x7f25d4167c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:25.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25d1e00640 1 -- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25d40ce780 msgr2=0x7f25d4167700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:25.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25d1e00640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25d40ce780 0x7f25d4167700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25d1e00640 1 -- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25d4160980 msgr2=0x7f25d4167c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:25.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25d1e00640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25d4160980 0x7f25d4167c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25d1e00640 1 -- 172.21.15.50:0/2257765893 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d41fd0d0 con 0x7f25d4070a30 2024-09-21T06:12:25.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25d1e00640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25d4070a30 0x7f25d40b5210 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f25c400c430 tx=0x7f25c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:25.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25ca000640 1 -- 172.21.15.50:0/2257765893 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4017070 con 0x7f25d4070a30 2024-09-21T06:12:25.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25ca000640 1 -- 172.21.15.50:0/2257765893 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25c40053c0 con 0x7f25d4070a30 2024-09-21T06:12:25.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25d9200640 1 -- 172.21.15.50:0/2257765893 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d4142640 con 0x7f25d4070a30 2024-09-21T06:12:25.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25ca000640 1 -- 172.21.15.50:0/2257765893 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4003860 con 0x7f25d4070a30 2024-09-21T06:12:25.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25d9200640 1 -- 172.21.15.50:0/2257765893 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d41fcb10 con 0x7f25d4070a30 2024-09-21T06:12:25.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25d9200640 1 -- 172.21.15.50:0/2257765893 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d4160d90 con 0x7f25d4070a30 2024-09-21T06:12:25.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.818+0000 7f25ca000640 1 -- 172.21.15.50:0/2257765893 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f25c401a070 con 0x7f25d4070a30 2024-09-21T06:12:25.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.822+0000 7f25ca000640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25b00754e0 0x7f25b00779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:25.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.822+0000 7f25ca000640 1 -- 172.21.15.50:0/2257765893 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5793+0+0 (secure 0 0 0) 0x7f25c4092800 con 0x7f25d4070a30 2024-09-21T06:12:25.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.822+0000 7f25cbe00640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25b00754e0 0x7f25b00779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:25.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.822+0000 7f25cbe00640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25b00754e0 0x7f25b00779f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0af000 tx=0x7f25bc0b2040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:25.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:25.826+0000 7f25ca000640 1 -- 172.21.15.50:0/2257765893 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f25c405ebb0 con 0x7f25d4070a30 2024-09-21T06:12:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.002+0000 7f25d9200640 1 -- 172.21.15.50:0/2257765893 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25d4160430 con 0x7f25d4070a30 2024-09-21T06:12:26.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25ca000640 1 -- 172.21.15.50:0/2257765893 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16837 (secure 0 0 0) 0x7f25c40635e0 con 0x7f25d4070a30 2024-09-21T06:12:26.004 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:26.004 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":141,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:12:25.031159+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:12:22.777798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T06:12:23.995303+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:12:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25d9200640 1 -- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25b00754e0 msgr2=0x7f25b00779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25d9200640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25b00754e0 0x7f25b00779f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0af000 tx=0x7f25bc0b2040 comp rx=0 tx=0).stop 2024-09-21T06:12:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25d9200640 1 -- 172.21.15.50:0/2257765893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25d4070a30 msgr2=0x7f25d40b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25d9200640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25d4070a30 0x7f25d40b5210 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f25c400c430 tx=0x7f25c400c8f0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25d9200640 1 -- 172.21.15.50:0/2257765893 shutdown_connections 2024-09-21T06:12:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25d9200640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25d4160980 0x7f25d4167c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25d9200640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25d40ce780 0x7f25d4167700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25d9200640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f25b00754e0 0x7f25b00779f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25d9200640 1 --2- 172.21.15.50:0/2257765893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25d4070a30 0x7f25d40b5210 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25d9200640 1 -- 172.21.15.50:0/2257765893 >> 172.21.15.50:0/2257765893 conn(0x7f25d40d06e0 msgr2=0x7f25d4145620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:26.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25d9200640 1 -- 172.21.15.50:0/2257765893 shutdown_connections 2024-09-21T06:12:26.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.006+0000 7f25d9200640 1 -- 172.21.15.50:0/2257765893 wait complete. 2024-09-21T06:12:26.027 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:12:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.186+0000 7f068f600640 1 -- 172.21.15.50:0/3986127338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06880b4de0 msgr2=0x7f06880b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.186+0000 7f068f600640 1 --2- 172.21.15.50:0/3986127338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06880b4de0 0x7f06880b51f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f06880703e0 tx=0x7f068001ccc0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.186+0000 7f068f600640 1 -- 172.21.15.50:0/3986127338 shutdown_connections 2024-09-21T06:12:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.186+0000 7f068f600640 1 --2- 172.21.15.50:0/3986127338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06880b4de0 0x7f06880b51f0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.186+0000 7f068f600640 1 --2- 172.21.15.50:0/3986127338 >> v2:172.21.15.88:3300/0 conn(0x7f06881527b0 0x7f0688074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.186+0000 7f068f600640 1 --2- 172.21.15.50:0/3986127338 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f06880b5c70 0x7f068815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.186+0000 7f068f600640 1 -- 172.21.15.50:0/3986127338 >> 172.21.15.50:0/3986127338 conn(0x7f06880b2040 msgr2=0x7f06880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.186+0000 7f068f600640 1 -- 172.21.15.50:0/3986127338 shutdown_connections 2024-09-21T06:12:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.186+0000 7f068f600640 1 -- 172.21.15.50:0/3986127338 wait complete. 2024-09-21T06:12:26.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068f600640 1 Processor -- start 2024-09-21T06:12:26.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068f600640 1 -- start start 2024-09-21T06:12:26.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068f600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06880b4de0 0x7f068814a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068f600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f06880b5c70 0x7f0688158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068f600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06881527b0 0x7f06880cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068f600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f068815f6b0 con 0x7f06880b4de0 2024-09-21T06:12:26.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068f600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f068815f530 con 0x7f06881527b0 2024-09-21T06:12:26.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068f600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f068815f830 con 0x7f06880b5c70 2024-09-21T06:12:26.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06880b4de0 0x7f068814a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06880b4de0 0x7f068814a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47434/0 (socket says 172.21.15.50:47434) 2024-09-21T06:12:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068cc00640 1 -- 172.21.15.50:0/490187991 learned_addr learned my addr 172.21.15.50:0/490187991 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068d600640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06881527b0 0x7f06880cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f0687e00640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f06880b5c70 0x7f0688158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068cc00640 1 -- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f06880b5c70 msgr2=0x7f0688158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068cc00640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f06880b5c70 0x7f0688158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068cc00640 1 -- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06881527b0 msgr2=0x7f06880cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068cc00640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06881527b0 0x7f06880cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068cc00640 1 -- 172.21.15.50:0/490187991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06881590f0 con 0x7f06880b4de0 2024-09-21T06:12:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068d600640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06881527b0 0x7f06880cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f0687e00640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f06880b5c70 0x7f0688158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068cc00640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06880b4de0 0x7f068814a8c0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f067c00c970 tx=0x7f067c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f0684c00640 1 -- 172.21.15.50:0/490187991 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c007b50 con 0x7f06880b4de0 2024-09-21T06:12:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f0684c00640 1 -- 172.21.15.50:0/490187991 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f067c007de0 con 0x7f06880b4de0 2024-09-21T06:12:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068f600640 1 -- 172.21.15.50:0/490187991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068814c660 con 0x7f06880b4de0 2024-09-21T06:12:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f0684c00640 1 -- 172.21.15.50:0/490187991 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c00f6b0 con 0x7f06880b4de0 2024-09-21T06:12:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.190+0000 7f068f600640 1 -- 172.21.15.50:0/490187991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06880cd2a0 con 0x7f06880b4de0 2024-09-21T06:12:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.194+0000 7f068f600640 1 -- 172.21.15.50:0/490187991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0688156a40 con 0x7f06880b4de0 2024-09-21T06:12:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.194+0000 7f0684c00640 1 -- 172.21.15.50:0/490187991 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f067c00f850 con 0x7f06880b4de0 2024-09-21T06:12:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.194+0000 7f0684c00640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0668075510 0x7f0668077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.194+0000 7f0684c00640 1 -- 172.21.15.50:0/490187991 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5793+0+0 (secure 0 0 0) 0x7f067c0939e0 con 0x7f06880b4de0 2024-09-21T06:12:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.194+0000 7f0687e00640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0668075510 0x7f0668077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.198+0000 7f0687e00640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0668075510 0x7f0668077a20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f06880cfe10 tx=0x7f0674005d20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:26.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.198+0000 7f0684c00640 1 -- 172.21.15.50:0/490187991 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f067c014030 con 0x7f06880b4de0 2024-09-21T06:12:26.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.378+0000 7f068f600640 1 -- 172.21.15.50:0/490187991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f068815fd50 con 0x7f06880b4de0 2024-09-21T06:12:26.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f0684c00640 1 -- 172.21.15.50:0/490187991 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16837 (secure 0 0 0) 0x7f067c0604f0 con 0x7f06880b4de0 2024-09-21T06:12:26.379 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:26.379 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":141,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:12:25.031159+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:12:22.777798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T06:12:23.995303+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:12:26.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f068f600640 1 -- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0668075510 msgr2=0x7f0668077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f068f600640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0668075510 0x7f0668077a20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f06880cfe10 tx=0x7f0674005d20 comp rx=0 tx=0).stop 2024-09-21T06:12:26.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f068f600640 1 -- 172.21.15.50:0/490187991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06880b4de0 msgr2=0x7f068814a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f068f600640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06880b4de0 0x7f068814a8c0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f067c00c970 tx=0x7f067c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:12:26.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f068f600640 1 -- 172.21.15.50:0/490187991 shutdown_connections 2024-09-21T06:12:26.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f068f600640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06881527b0 0x7f06880cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f068f600640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f06880b5c70 0x7f0688158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f068f600640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0668075510 0x7f0668077a20 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f068f600640 1 --2- 172.21.15.50:0/490187991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06880b4de0 0x7f068814a8c0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f068f600640 1 -- 172.21.15.50:0/490187991 >> 172.21.15.50:0/490187991 conn(0x7f06880b2040 msgr2=0x7f068815ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:26.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f068f600640 1 -- 172.21.15.50:0/490187991 shutdown_connections 2024-09-21T06:12:26.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.382+0000 7f068f600640 1 -- 172.21.15.50:0/490187991 wait complete. 2024-09-21T06:12:26.402 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:12:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.566+0000 7fd90a800640 1 -- 172.21.15.50:0/2134053580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8e8003fe0 msgr2=0x7fd8e8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.566+0000 7fd90a800640 1 --2- 172.21.15.50:0/2134053580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8e8003fe0 0x7fd8e8004810 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00c470 tx=0x7fd8ec01f6c0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.566+0000 7fd903e00640 1 -- 172.21.15.50:0/2134053580 reap_dead start 2024-09-21T06:12:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.566+0000 7fd90a800640 1 -- 172.21.15.50:0/2134053580 shutdown_connections 2024-09-21T06:12:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.566+0000 7fd90a800640 1 -- 172.21.15.50:0/2134053580 >> 172.21.15.50:0/2134053580 conn(0x7fd9040b2040 msgr2=0x7fd9040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.566+0000 7fd90a800640 1 -- 172.21.15.50:0/2134053580 shutdown_connections 2024-09-21T06:12:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.566+0000 7fd90a800640 1 -- 172.21.15.50:0/2134053580 wait complete. 2024-09-21T06:12:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd90a800640 1 Processor -- start 2024-09-21T06:12:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd90a800640 1 -- start start 2024-09-21T06:12:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd90a800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8e8003fe0 0x7fd9041e8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd90a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd904151bf0 0x7fd904154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd90a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd904152000 0x7fd904155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd90a800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd90415cd20 con 0x7fd904151bf0 2024-09-21T06:12:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd90a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd90415cba0 con 0x7fd904152000 2024-09-21T06:12:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd90a800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd90415cea0 con 0x7fd8e8003fe0 2024-09-21T06:12:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd8fbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd904151bf0 0x7fd904154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd8fbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd904151bf0 0x7fd904154e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47444/0 (socket says 172.21.15.50:47444) 2024-09-21T06:12:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd8fbe00640 1 -- 172.21.15.50:0/957155053 learned_addr learned my addr 172.21.15.50:0/957155053 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd903400640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8e8003fe0 0x7fd9041e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd903e00640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd904152000 0x7fd904155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd8fbe00640 1 -- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8e8003fe0 msgr2=0x7fd9041e8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd8fbe00640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8e8003fe0 0x7fd9041e8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd8fbe00640 1 -- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd904152000 msgr2=0x7fd904155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd8fbe00640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd904152000 0x7fd904155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.570+0000 7fd8fbe00640 1 -- 172.21.15.50:0/957155053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9041f72e0 con 0x7fd904151bf0 2024-09-21T06:12:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.574+0000 7fd8fbe00640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd904151bf0 0x7fd904154e30 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0263d0 tx=0x7fd8ec006fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.574+0000 7fd8fb400640 1 -- 172.21.15.50:0/957155053 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec0b3070 con 0x7fd904151bf0 2024-09-21T06:12:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.574+0000 7fd8fb400640 1 -- 172.21.15.50:0/957155053 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8ec006290 con 0x7fd904151bf0 2024-09-21T06:12:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.574+0000 7fd90a800640 1 -- 172.21.15.50:0/957155053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9041f75c0 con 0x7fd904151bf0 2024-09-21T06:12:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.574+0000 7fd8fb400640 1 -- 172.21.15.50:0/957155053 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec003600 con 0x7fd904151bf0 2024-09-21T06:12:26.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.574+0000 7fd90a800640 1 -- 172.21.15.50:0/957155053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9041f7b80 con 0x7fd904151bf0 2024-09-21T06:12:26.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.574+0000 7fd90a800640 1 -- 172.21.15.50:0/957155053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9041f7050 con 0x7fd904151bf0 2024-09-21T06:12:26.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.574+0000 7fd8fb400640 1 -- 172.21.15.50:0/957155053 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fd8ec0037a0 con 0x7fd904151bf0 2024-09-21T06:12:26.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.574+0000 7fd8fb400640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8e0075510 0x7fd8e0077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.574+0000 7fd8fb400640 1 -- 172.21.15.50:0/957155053 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5793+0+0 (secure 0 0 0) 0x7fd8ec130f30 con 0x7fd904151bf0 2024-09-21T06:12:26.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.578+0000 7fd903400640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8e0075510 0x7fd8e0077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.578+0000 7fd903400640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8e0075510 0x7fd8e0077a20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009c20 tx=0x7fd8f40091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:26.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.578+0000 7fd8fb400640 1 -- 172.21.15.50:0/957155053 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd9040b5730 con 0x7fd904151bf0 2024-09-21T06:12:26.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:26.802+0000 7fd90a800640 1 -- 172.21.15.50:0/957155053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd90415d430 con 0x7fd904151bf0 2024-09-21T06:12:27.582 INFO:tasks.ceph.mds.d.smithi088.stderr:2024-09-21T06:12:27.583+0000 7f7a58400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:12:27.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.626+0000 7fd8fb400640 1 -- 172.21.15.50:0/957155053 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v93) ==== 118+0+0 (secure 0 0 0) 0x7fd8ec0fda40 con 0x7fd904151bf0 2024-09-21T06:12:27.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.630+0000 7fd90a800640 1 -- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8e0075510 msgr2=0x7fd8e0077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:27.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.630+0000 7fd90a800640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8e0075510 0x7fd8e0077a20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009c20 tx=0x7fd8f40091e0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.630+0000 7fd90a800640 1 -- 172.21.15.50:0/957155053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd904151bf0 msgr2=0x7fd904154e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:27.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.630+0000 7fd90a800640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd904151bf0 0x7fd904154e30 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0263d0 tx=0x7fd8ec006fd0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.630+0000 7fd90a800640 1 -- 172.21.15.50:0/957155053 shutdown_connections 2024-09-21T06:12:27.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.630+0000 7fd90a800640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd904152000 0x7fd904155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.630+0000 7fd90a800640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8e0075510 0x7fd8e0077a20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.630+0000 7fd90a800640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd904151bf0 0x7fd904154e30 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.630+0000 7fd90a800640 1 --2- 172.21.15.50:0/957155053 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8e8003fe0 0x7fd9041e8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.630+0000 7fd90a800640 1 -- 172.21.15.50:0/957155053 >> 172.21.15.50:0/957155053 conn(0x7fd9040b2040 msgr2=0x7fd90415baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:27.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.630+0000 7fd90a800640 1 -- 172.21.15.50:0/957155053 shutdown_connections 2024-09-21T06:12:27.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.630+0000 7fd90a800640 1 -- 172.21.15.50:0/957155053 wait complete. 2024-09-21T06:12:27.654 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T06:12:27.654 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T06:12:27.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.822+0000 7facd7600640 1 -- 172.21.15.50:0/1751823465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facd01527b0 msgr2=0x7facd0074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:27.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.822+0000 7facd7600640 1 --2- 172.21.15.50:0/1751823465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facd01527b0 0x7facd0074e30 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7facd0072e30 tx=0x7facb801c800 comp rx=0 tx=0).stop 2024-09-21T06:12:27.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.822+0000 7facd7600640 1 -- 172.21.15.50:0/1751823465 shutdown_connections 2024-09-21T06:12:27.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.822+0000 7facd7600640 1 --2- 172.21.15.50:0/1751823465 >> v2:172.21.15.88:3300/0 conn(0x7facd00b4de0 0x7facd00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.822+0000 7facd7600640 1 --2- 172.21.15.50:0/1751823465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facd01527b0 0x7facd0074e30 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.822+0000 7facd7600640 1 --2- 172.21.15.50:0/1751823465 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7facd00b5c70 0x7facd015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.822+0000 7facd7600640 1 -- 172.21.15.50:0/1751823465 >> 172.21.15.50:0/1751823465 conn(0x7facd00b2040 msgr2=0x7facd00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:27.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.822+0000 7facd7600640 1 -- 172.21.15.50:0/1751823465 shutdown_connections 2024-09-21T06:12:27.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.822+0000 7facd7600640 1 -- 172.21.15.50:0/1751823465 wait complete. 2024-09-21T06:12:27.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd7600640 1 Processor -- start 2024-09-21T06:12:27.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd7600640 1 -- start start 2024-09-21T06:12:27.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd7600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facd00b4de0 0x7facd013b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:27.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd7600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facd00b5c70 0x7facd013bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:27.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd7600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7facd01527b0 0x7facd01541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:27.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd7600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7facd015f6b0 con 0x7facd00b4de0 2024-09-21T06:12:27.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd7600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7facd015f530 con 0x7facd00b5c70 2024-09-21T06:12:27.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd7600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7facd015f830 con 0x7facd01527b0 2024-09-21T06:12:27.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facd00b4de0 0x7facd013b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:27.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facd00b4de0 0x7facd013b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53298/0 (socket says 172.21.15.50:53298) 2024-09-21T06:12:27.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd4c00640 1 -- 172.21.15.50:0/2456917343 learned_addr learned my addr 172.21.15.50:0/2456917343 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:27.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facc7e00640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facd00b5c70 0x7facd013bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:27.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd5600640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7facd01527b0 0x7facd01541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:27.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd4c00640 1 -- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7facd01527b0 msgr2=0x7facd01541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:27.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd4c00640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7facd01527b0 0x7facd01541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd4c00640 1 -- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facd00b5c70 msgr2=0x7facd013bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:27.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd4c00640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facd00b5c70 0x7facd013bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd4c00640 1 -- 172.21.15.50:0/2456917343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd01552a0 con 0x7facd00b4de0 2024-09-21T06:12:27.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd4c00640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facd00b4de0 0x7facd013b590 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7facc00058d0 tx=0x7facc000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:27.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7faccd600640 1 -- 172.21.15.50:0/2456917343 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0017070 con 0x7facd00b4de0 2024-09-21T06:12:27.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7faccd600640 1 -- 172.21.15.50:0/2456917343 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facc0009d20 con 0x7facd00b4de0 2024-09-21T06:12:27.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7facd7600640 1 -- 172.21.15.50:0/2456917343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd014d9e0 con 0x7facd00b4de0 2024-09-21T06:12:27.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.826+0000 7faccd600640 1 -- 172.21.15.50:0/2456917343 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc00106c0 con 0x7facd00b4de0 2024-09-21T06:12:27.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.830+0000 7facd7600640 1 -- 172.21.15.50:0/2456917343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd014dfa0 con 0x7facd00b4de0 2024-09-21T06:12:27.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.830+0000 7facd7600640 1 -- 172.21.15.50:0/2456917343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd015fce0 con 0x7facd00b4de0 2024-09-21T06:12:27.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.830+0000 7faccd600640 1 -- 172.21.15.50:0/2456917343 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7facc001a070 con 0x7facd00b4de0 2024-09-21T06:12:27.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.830+0000 7faccd600640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7facac0754e0 0x7facac0779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:27.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.830+0000 7faccd600640 1 -- 172.21.15.50:0/2456917343 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7facc0093400 con 0x7facd00b4de0 2024-09-21T06:12:27.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.830+0000 7facc7e00640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7facac0754e0 0x7facac0779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:27.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.834+0000 7facc7e00640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7facac0754e0 0x7facac0779f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7facb80af000 tx=0x7facb801b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:27.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:27.834+0000 7faccd600640 1 -- 172.21.15.50:0/2456917343 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7facc005f960 con 0x7facd00b4de0 2024-09-21T06:12:28.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.050+0000 7facd7600640 1 -- 172.21.15.50:0/2456917343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7facd015e540 con 0x7facd00b4de0 2024-09-21T06:12:28.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.630+0000 7faccd600640 1 -- 172.21.15.50:0/2456917343 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v94) ==== 122+0+0 (secure 0 0 0) 0x7facc0064390 con 0x7facd00b4de0 2024-09-21T06:12:28.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.630+0000 7facd7600640 1 -- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7facac0754e0 msgr2=0x7facac0779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:28.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.630+0000 7facd7600640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7facac0754e0 0x7facac0779f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7facb80af000 tx=0x7facb801b040 comp rx=0 tx=0).stop 2024-09-21T06:12:28.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.630+0000 7facd7600640 1 -- 172.21.15.50:0/2456917343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facd00b4de0 msgr2=0x7facd013b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:28.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.630+0000 7facd7600640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facd00b4de0 0x7facd013b590 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7facc00058d0 tx=0x7facc000dde0 comp rx=0 tx=0).stop 2024-09-21T06:12:28.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.630+0000 7facd7600640 1 -- 172.21.15.50:0/2456917343 shutdown_connections 2024-09-21T06:12:28.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.630+0000 7facd7600640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7facd01527b0 0x7facd01541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:28.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.630+0000 7facd7600640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facd00b5c70 0x7facd013bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:28.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.630+0000 7facd7600640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7facac0754e0 0x7facac0779f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:28.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.630+0000 7facd7600640 1 --2- 172.21.15.50:0/2456917343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facd00b4de0 0x7facd013b590 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:28.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.630+0000 7facd7600640 1 -- 172.21.15.50:0/2456917343 >> 172.21.15.50:0/2456917343 conn(0x7facd00b2040 msgr2=0x7facd00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:28.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.630+0000 7facd7600640 1 -- 172.21.15.50:0/2456917343 shutdown_connections 2024-09-21T06:12:28.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.634+0000 7facd7600640 1 -- 172.21.15.50:0/2456917343 wait complete. 2024-09-21T06:12:28.655 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T06:12:28.656 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:12:28.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.818+0000 7f8f21c00640 1 -- 172.21.15.50:0/1160344912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0cda80 msgr2=0x7f8f1c0cde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:28.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.818+0000 7f8f21c00640 1 --2- 172.21.15.50:0/1160344912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0cda80 0x7f8f1c0cde90 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009c80 tx=0x7f8f1401ccb0 comp rx=0 tx=0).stop 2024-09-21T06:12:28.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.818+0000 7f8f21c00640 1 -- 172.21.15.50:0/1160344912 shutdown_connections 2024-09-21T06:12:28.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.818+0000 7f8f21c00640 1 --2- 172.21.15.50:0/1160344912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0cda80 0x7f8f1c0cde90 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:28.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.818+0000 7f8f21c00640 1 --2- 172.21.15.50:0/1160344912 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c0746e0 0x7f8f1c15c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:28.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.818+0000 7f8f21c00640 1 -- 172.21.15.50:0/1160344912 >> 172.21.15.50:0/1160344912 conn(0x7f8f1c0b41f0 msgr2=0x7f8f1c0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:28.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.818+0000 7f8f21c00640 1 -- 172.21.15.50:0/1160344912 shutdown_connections 2024-09-21T06:12:28.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.818+0000 7f8f21c00640 1 -- 172.21.15.50:0/1160344912 wait complete. 2024-09-21T06:12:28.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21c00640 1 Processor -- start 2024-09-21T06:12:28.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21c00640 1 -- start start 2024-09-21T06:12:28.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f1c0746e0 0x7f8f1c0cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:28.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c074c30 0x7f8f1c1514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:28.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0cda80 0x7f8f1c151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c169f80 con 0x7f8f1c0cda80 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c169e00 con 0x7f8f1c0746e0 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1c16a100 con 0x7f8f1c074c30 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0cda80 0x7f8f1c151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f1be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f1c0746e0 0x7f8f1c0cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0cda80 0x7f8f1c151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53306/0 (socket says 172.21.15.50:53306) 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21200640 1 -- 172.21.15.50:0/3388550601 learned_addr learned my addr 172.21.15.50:0/3388550601 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f13e00640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c074c30 0x7f8f1c1514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21200640 1 -- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c074c30 msgr2=0x7f8f1c1514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21200640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c074c30 0x7f8f1c1514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21200640 1 -- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f1c0746e0 msgr2=0x7f8f1c0cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21200640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f1c0746e0 0x7f8f1c0cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:28.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.822+0000 7f8f21200640 1 -- 172.21.15.50:0/3388550601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c0cb5d0 con 0x7f8f1c0cda80 2024-09-21T06:12:28.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.826+0000 7f8f21200640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0cda80 0x7f8f1c151a20 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f8f1401b3c0 tx=0x7f8f14009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:28.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.826+0000 7f8f18c00640 1 -- 172.21.15.50:0/3388550601 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f140bb070 con 0x7f8f1c0cda80 2024-09-21T06:12:28.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.826+0000 7f8f18c00640 1 -- 172.21.15.50:0/3388550601 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f14004d40 con 0x7f8f1c0cda80 2024-09-21T06:12:28.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.826+0000 7f8f21c00640 1 -- 172.21.15.50:0/3388550601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1c15f440 con 0x7f8f1c0cda80 2024-09-21T06:12:28.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.826+0000 7f8f18c00640 1 -- 172.21.15.50:0/3388550601 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f140b2040 con 0x7f8f1c0cda80 2024-09-21T06:12:28.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.826+0000 7f8f21c00640 1 -- 172.21.15.50:0/3388550601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1c15fa00 con 0x7f8f1c0cda80 2024-09-21T06:12:28.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.826+0000 7f8f21c00640 1 -- 172.21.15.50:0/3388550601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1c15c920 con 0x7f8f1c0cda80 2024-09-21T06:12:28.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.826+0000 7f8f18c00640 1 -- 172.21.15.50:0/3388550601 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f8f140be070 con 0x7f8f1c0cda80 2024-09-21T06:12:28.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.826+0000 7f8f18c00640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8ef8075510 0x7f8ef8077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:28.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.830+0000 7f8f18c00640 1 -- 172.21.15.50:0/3388550601 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f8f1412ceb0 con 0x7f8f1c0cda80 2024-09-21T06:12:28.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.830+0000 7f8f1be00640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8ef8075510 0x7f8ef8077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.830+0000 7f8f1be00640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8ef8075510 0x7f8ef8077a20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c004630 tx=0x7f8f0c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:28.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:28.834+0000 7f8f18c00640 1 -- 172.21.15.50:0/3388550601 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8f14016990 con 0x7f8f1c0cda80 2024-09-21T06:12:29.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 -- 172.21.15.50:0/3388550601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f1c0b4f30 con 0x7f8f1c0cda80 2024-09-21T06:12:29.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f18c00640 1 -- 172.21.15.50:0/3388550601 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5224 (secure 0 0 0) 0x7f8f140f92e0 con 0x7f8f1c0cda80 2024-09-21T06:12:29.048 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:29.048 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":94,"btime":"2024-09-21T06:12:28:632312+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4262887686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":4262887686},{"type":"v1","addr":"172.21.15.50:6835","nonce":4262887686}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":7721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3308178097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3308178097},{"type":"v1","addr":"172.21.15.88:6837","nonce":3308178097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/610421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":610421041},{"type":"v1","addr":"172.21.15.50:6834","nonce":610421041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:12:25.031119+0000","modified":"2024-09-21T06:12:28.054360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8632":{"gid":8632,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2222229719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2222229719},{"type":"v1","addr":"172.21.15.88:6835","nonce":2222229719}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8632,"qdb_cluster":[8632]},"id":10}]} 2024-09-21T06:12:29.048 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 94 2024-09-21T06:12:29.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 -- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8ef8075510 msgr2=0x7f8ef8077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8ef8075510 0x7f8ef8077a20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c004630 tx=0x7f8f0c00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 -- 172.21.15.50:0/3388550601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0cda80 msgr2=0x7f8f1c151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0cda80 0x7f8f1c151a20 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f8f1401b3c0 tx=0x7f8f14009350 comp rx=0 tx=0).stop 2024-09-21T06:12:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 -- 172.21.15.50:0/3388550601 shutdown_connections 2024-09-21T06:12:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8ef8075510 0x7f8ef8077a20 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f1c0cda80 0x7f8f1c151a20 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8f1c074c30 0x7f8f1c1514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 --2- 172.21.15.50:0/3388550601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f1c0746e0 0x7f8f1c0cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 -- 172.21.15.50:0/3388550601 >> 172.21.15.50:0/3388550601 conn(0x7f8f1c0b41f0 msgr2=0x7f8f1c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 -- 172.21.15.50:0/3388550601 shutdown_connections 2024-09-21T06:12:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.050+0000 7f8f21c00640 1 -- 172.21.15.50:0/3388550601 wait complete. 2024-09-21T06:12:29.071 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:12:29.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.230+0000 7f5f01200640 1 -- 172.21.15.50:0/4228350722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ee0004da0 msgr2=0x7f5ee0005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.230+0000 7f5f01200640 1 --2- 172.21.15.50:0/4228350722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ee0004da0 0x7f5ee0005230 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0b6600 tx=0x7f5ef401f390 comp rx=0 tx=0).stop 2024-09-21T06:12:29.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.230+0000 7f5efa800640 1 -- 172.21.15.50:0/4228350722 reap_dead start 2024-09-21T06:12:29.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.230+0000 7f5f01200640 1 -- 172.21.15.50:0/4228350722 shutdown_connections 2024-09-21T06:12:29.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.230+0000 7f5f01200640 1 -- 172.21.15.50:0/4228350722 >> 172.21.15.50:0/4228350722 conn(0x7f5efc0b2040 msgr2=0x7f5efc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.230+0000 7f5f01200640 1 -- 172.21.15.50:0/4228350722 shutdown_connections 2024-09-21T06:12:29.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.230+0000 7f5f01200640 1 -- 172.21.15.50:0/4228350722 wait complete. 2024-09-21T06:12:29.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5f01200640 1 Processor -- start 2024-09-21T06:12:29.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5f01200640 1 -- start start 2024-09-21T06:12:29.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5f01200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee0004da0 0x7f5efc1f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5f01200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5efc130570 0x7f5efc12db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5f01200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5efc0ae8e0 0x7f5efc0aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5f01200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc0b35c0 con 0x7f5efc130570 2024-09-21T06:12:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5f01200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc0b3440 con 0x7f5ee0004da0 2024-09-21T06:12:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5f01200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5efc0b3740 con 0x7f5efc0ae8e0 2024-09-21T06:12:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5ef3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5efc130570 0x7f5efc12db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5ef3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5efc130570 0x7f5efc12db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53334/0 (socket says 172.21.15.50:53334) 2024-09-21T06:12:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5ef3e00640 1 -- 172.21.15.50:0/3447932732 learned_addr learned my addr 172.21.15.50:0/3447932732 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5ef9e00640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee0004da0 0x7f5efc1f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5efa800640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5efc0ae8e0 0x7f5efc0aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5ef3e00640 1 -- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5efc0ae8e0 msgr2=0x7f5efc0aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5ef3e00640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5efc0ae8e0 0x7f5efc0aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5ef3e00640 1 -- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee0004da0 msgr2=0x7f5efc1f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5ef3e00640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee0004da0 0x7f5efc1f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.234+0000 7f5ef3e00640 1 -- 172.21.15.50:0/3447932732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5efc0af230 con 0x7f5efc130570 2024-09-21T06:12:29.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.238+0000 7f5ef3e00640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5efc130570 0x7f5efc12db00 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400eab0 tx=0x7f5ee400a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.238+0000 7f5ef2000640 1 -- 172.21.15.50:0/3447932732 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400ddb0 con 0x7f5efc130570 2024-09-21T06:12:29.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.238+0000 7f5ef2000640 1 -- 172.21.15.50:0/3447932732 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ee4004050 con 0x7f5efc130570 2024-09-21T06:12:29.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.238+0000 7f5ef2000640 1 -- 172.21.15.50:0/3447932732 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4004460 con 0x7f5efc130570 2024-09-21T06:12:29.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.238+0000 7f5f01200640 1 -- 172.21.15.50:0/3447932732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5efc13a560 con 0x7f5efc130570 2024-09-21T06:12:29.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.238+0000 7f5f01200640 1 -- 172.21.15.50:0/3447932732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5efc13ab20 con 0x7f5efc130570 2024-09-21T06:12:29.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.238+0000 7f5f01200640 1 -- 172.21.15.50:0/3447932732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5efc0cf690 con 0x7f5efc130570 2024-09-21T06:12:29.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.238+0000 7f5ef2000640 1 -- 172.21.15.50:0/3447932732 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f5ee4003600 con 0x7f5efc130570 2024-09-21T06:12:29.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.238+0000 7f5ef2000640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5ed8075510 0x7f5ed8077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.238+0000 7f5ef2000640 1 -- 172.21.15.50:0/3447932732 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f5ee4093ee0 con 0x7f5efc130570 2024-09-21T06:12:29.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.242+0000 7f5ef9e00640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5ed8075510 0x7f5ed8077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.242+0000 7f5ef9e00640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5ed8075510 0x7f5ed8077a20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5eec001ec0 tx=0x7f5eec0016d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.242+0000 7f5ef2000640 1 -- 172.21.15.50:0/3447932732 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5efc0af230 con 0x7f5efc130570 2024-09-21T06:12:29.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.418+0000 7f5f01200640 1 -- 172.21.15.50:0/3447932732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5efc0b4cd0 con 0x7f5efc130570 2024-09-21T06:12:29.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5ef2000640 1 -- 172.21.15.50:0/3447932732 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16824 (secure 0 0 0) 0x7f5ee4060420 con 0x7f5efc130570 2024-09-21T06:12:29.419 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:29.419 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":142,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:12:26.608700+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:12:22.777798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T06:12:23.995303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:12:29.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5f01200640 1 -- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5ed8075510 msgr2=0x7f5ed8077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5f01200640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5ed8075510 0x7f5ed8077a20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5eec001ec0 tx=0x7f5eec0016d0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5f01200640 1 -- 172.21.15.50:0/3447932732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5efc130570 msgr2=0x7f5efc12db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5f01200640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5efc130570 0x7f5efc12db00 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400eab0 tx=0x7f5ee400a6b0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5f01200640 1 -- 172.21.15.50:0/3447932732 shutdown_connections 2024-09-21T06:12:29.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5f01200640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5efc0ae8e0 0x7f5efc0aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5f01200640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5ed8075510 0x7f5ed8077a20 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5f01200640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5efc130570 0x7f5efc12db00 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5f01200640 1 --2- 172.21.15.50:0/3447932732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee0004da0 0x7f5efc1f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5f01200640 1 -- 172.21.15.50:0/3447932732 >> 172.21.15.50:0/3447932732 conn(0x7f5efc0b2040 msgr2=0x7f5efc140590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5f01200640 1 -- 172.21.15.50:0/3447932732 shutdown_connections 2024-09-21T06:12:29.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.422+0000 7f5f01200640 1 -- 172.21.15.50:0/3447932732 wait complete. 2024-09-21T06:12:29.442 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:12:29.443 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:12:29.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.602+0000 7fc5e1000640 1 -- 172.21.15.50:0/2973744856 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc5dc157030 msgr2=0x7fc5dc0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.602+0000 7fc5e1000640 1 --2- 172.21.15.50:0/2973744856 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc5dc157030 0x7fc5dc0b5a80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009840 tx=0x7fc5cc01c820 comp rx=0 tx=0).stop 2024-09-21T06:12:29.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.602+0000 7fc5e1000640 1 -- 172.21.15.50:0/2973744856 shutdown_connections 2024-09-21T06:12:29.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.602+0000 7fc5e1000640 1 --2- 172.21.15.50:0/2973744856 >> v2:172.21.15.88:3300/0 conn(0x7fc5dc160d50 0x7fc5dc074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.602+0000 7fc5e1000640 1 --2- 172.21.15.50:0/2973744856 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc5dc157030 0x7fc5dc0b5a80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.602+0000 7fc5e1000640 1 -- 172.21.15.50:0/2973744856 >> 172.21.15.50:0/2973744856 conn(0x7fc5dc0708c0 msgr2=0x7fc5dc070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.602+0000 7fc5e1000640 1 -- 172.21.15.50:0/2973744856 shutdown_connections 2024-09-21T06:12:29.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.602+0000 7fc5e1000640 1 -- 172.21.15.50:0/2973744856 wait complete. 2024-09-21T06:12:29.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.602+0000 7fc5e1000640 1 Processor -- start 2024-09-21T06:12:29.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.602+0000 7fc5e1000640 1 -- start start 2024-09-21T06:12:29.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5e1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc157030 0x7fc5dc14ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5e1000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5dc160d50 0x7fc5dc148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5e1000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc5dc161160 0x7fc5dc15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5e1000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc16a7d0 con 0x7fc5dc157030 2024-09-21T06:12:29.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5e1000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc16a650 con 0x7fc5dc160d50 2024-09-21T06:12:29.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5e1000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc16a950 con 0x7fc5dc161160 2024-09-21T06:12:29.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc157030 0x7fc5dc14ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc157030 0x7fc5dc14ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53346/0 (socket says 172.21.15.50:53346) 2024-09-21T06:12:29.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d9e00640 1 -- 172.21.15.50:0/1523903479 learned_addr learned my addr 172.21.15.50:0/1523903479 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:29.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d1400640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5dc160d50 0x7fc5dc148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5da800640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc5dc161160 0x7fc5dc15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d9e00640 1 -- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc5dc161160 msgr2=0x7fc5dc15c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d9e00640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc5dc161160 0x7fc5dc15c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d9e00640 1 -- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5dc160d50 msgr2=0x7fc5dc148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d9e00640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5dc160d50 0x7fc5dc148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d9e00640 1 -- 172.21.15.50:0/1523903479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc0b4c80 con 0x7fc5dc157030 2024-09-21T06:12:29.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d9e00640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc157030 0x7fc5dc14ab10 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009790 tx=0x7fc5cc0a64f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d2a00640 1 -- 172.21.15.50:0/1523903479 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc0a66e0 con 0x7fc5dc157030 2024-09-21T06:12:29.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5e1000640 1 -- 172.21.15.50:0/1523903479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc0b4f10 con 0x7fc5dc157030 2024-09-21T06:12:29.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d2a00640 1 -- 172.21.15.50:0/1523903479 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5cc004130 con 0x7fc5dc157030 2024-09-21T06:12:29.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5d2a00640 1 -- 172.21.15.50:0/1523903479 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc003600 con 0x7fc5dc157030 2024-09-21T06:12:29.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5e1000640 1 -- 172.21.15.50:0/1523903479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc14b1c0 con 0x7fc5dc157030 2024-09-21T06:12:29.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.606+0000 7fc5e1000640 1 -- 172.21.15.50:0/1523903479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc0d2260 con 0x7fc5dc157030 2024-09-21T06:12:29.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.610+0000 7fc5d2a00640 1 -- 172.21.15.50:0/1523903479 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fc5cc0c1020 con 0x7fc5dc157030 2024-09-21T06:12:29.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.610+0000 7fc5d2a00640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc5b8075680 0x7fc5b8077b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.610+0000 7fc5d2a00640 1 -- 172.21.15.50:0/1523903479 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7fc5cc12e330 con 0x7fc5dc157030 2024-09-21T06:12:29.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.610+0000 7fc5d1400640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc5b8075680 0x7fc5b8077b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.610+0000 7fc5d1400640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc5b8075680 0x7fc5b8077b90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0d12e0 tx=0x7fc5c40091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.614+0000 7fc5d2a00640 1 -- 172.21.15.50:0/1523903479 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc5cc016410 con 0x7fc5dc157030 2024-09-21T06:12:29.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.786+0000 7fc5e1000640 1 -- 172.21.15.50:0/1523903479 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5dc06b540 con 0x7fc5b8075680 2024-09-21T06:12:29.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.790+0000 7fc5d2a00640 1 -- 172.21.15.50:0/1523903479 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252499 (secure 0 0 0) 0x7fc5c4011b50 con 0x7fc5b8075680 2024-09-21T06:12:29.794 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:29.795 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:12:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.794+0000 7fc5e1000640 1 -- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc5b8075680 msgr2=0x7fc5b8077b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.794+0000 7fc5e1000640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc5b8075680 0x7fc5b8077b90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0d12e0 tx=0x7fc5c40091e0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.794+0000 7fc5e1000640 1 -- 172.21.15.50:0/1523903479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc157030 msgr2=0x7fc5dc14ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.794+0000 7fc5e1000640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc157030 0x7fc5dc14ab10 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009790 tx=0x7fc5cc0a64f0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.794+0000 7fc5e1000640 1 -- 172.21.15.50:0/1523903479 shutdown_connections 2024-09-21T06:12:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.794+0000 7fc5e1000640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc5dc161160 0x7fc5dc15c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.794+0000 7fc5e1000640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5dc160d50 0x7fc5dc148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.794+0000 7fc5e1000640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc5b8075680 0x7fc5b8077b90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.794+0000 7fc5e1000640 1 --2- 172.21.15.50:0/1523903479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc157030 0x7fc5dc14ab10 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.794+0000 7fc5e1000640 1 -- 172.21.15.50:0/1523903479 >> 172.21.15.50:0/1523903479 conn(0x7fc5dc0708c0 msgr2=0x7fc5dc0b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.794+0000 7fc5e1000640 1 -- 172.21.15.50:0/1523903479 shutdown_connections 2024-09-21T06:12:29.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:29.794+0000 7fc5e1000640 1 -- 172.21.15.50:0/1523903479 wait complete. 2024-09-21T06:12:29.815 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":621,"stamp":"2024-09-21T06:12:29.229918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1074804,"kb_used_data":15048,"kb_used_omap":12,"kb_used_meta":1059571,"kb_avail":753899916,"statfs":{"total":773094113280,"available":771993513984,"internally_reserved":0,"allocated":15409152,"data_stored":7425828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1085001264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.643275"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636164+0000","last_change":"2024-09-21T06:12:25.612914+0000","last_active":"2024-09-21T06:12:26.636164+0000","last_peered":"2024-09-21T06:12:26.636164+0000","last_clean":"2024-09-21T06:12:26.636164+0000","last_became_active":"2024-09-21T06:12:25.612390+0000","last_became_peered":"2024-09-21T06:12:25.612390+0000","last_unstale":"2024-09-21T06:12:26.636164+0000","last_undegraded":"2024-09-21T06:12:26.636164+0000","last_fullsized":"2024-09-21T06:12:26.636164+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:39.356625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618345+0000","last_change":"2024-09-21T06:12:25.612619+0000","last_active":"2024-09-21T06:12:26.618345+0000","last_peered":"2024-09-21T06:12:26.618345+0000","last_clean":"2024-09-21T06:12:26.618345+0000","last_became_active":"2024-09-21T06:12:25.612328+0000","last_became_peered":"2024-09-21T06:12:25.612328+0000","last_unstale":"2024-09-21T06:12:26.618345+0000","last_undegraded":"2024-09-21T06:12:26.618345+0000","last_fullsized":"2024-09-21T06:12:26.618345+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:57.830884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623653+0000","last_change":"2024-09-21T06:12:25.622728+0000","last_active":"2024-09-21T06:12:27.623653+0000","last_peered":"2024-09-21T06:12:27.623653+0000","last_clean":"2024-09-21T06:12:27.623653+0000","last_became_active":"2024-09-21T06:12:25.622427+0000","last_became_peered":"2024-09-21T06:12:25.622427+0000","last_unstale":"2024-09-21T06:12:27.623653+0000","last_undegraded":"2024-09-21T06:12:27.623653+0000","last_fullsized":"2024-09-21T06:12:27.623653+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:42.498937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.626161+0000","last_change":"2024-09-21T06:12:25.623359+0000","last_active":"2024-09-21T06:12:27.626161+0000","last_peered":"2024-09-21T06:12:27.626161+0000","last_clean":"2024-09-21T06:12:27.626161+0000","last_became_active":"2024-09-21T06:12:25.622631+0000","last_became_peered":"2024-09-21T06:12:25.622631+0000","last_unstale":"2024-09-21T06:12:27.626161+0000","last_undegraded":"2024-09-21T06:12:27.626161+0000","last_fullsized":"2024-09-21T06:12:27.626161+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:02.869029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618787+0000","last_change":"2024-09-21T06:12:25.607908+0000","last_active":"2024-09-21T06:12:26.618787+0000","last_peered":"2024-09-21T06:12:26.618787+0000","last_clean":"2024-09-21T06:12:26.618787+0000","last_became_active":"2024-09-21T06:12:25.607545+0000","last_became_peered":"2024-09-21T06:12:25.607545+0000","last_unstale":"2024-09-21T06:12:26.618787+0000","last_undegraded":"2024-09-21T06:12:26.618787+0000","last_fullsized":"2024-09-21T06:12:26.618787+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:33.338206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.635818+0000","last_change":"2024-09-21T06:12:25.625461+0000","last_active":"2024-09-21T06:12:26.635818+0000","last_peered":"2024-09-21T06:12:26.635818+0000","last_clean":"2024-09-21T06:12:26.635818+0000","last_became_active":"2024-09-21T06:12:25.624887+0000","last_became_peered":"2024-09-21T06:12:25.624887+0000","last_unstale":"2024-09-21T06:12:26.635818+0000","last_undegraded":"2024-09-21T06:12:26.635818+0000","last_fullsized":"2024-09-21T06:12:26.635818+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:55.033896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624534+0000","last_change":"2024-09-21T06:12:25.628168+0000","last_active":"2024-09-21T06:12:27.624534+0000","last_peered":"2024-09-21T06:12:27.624534+0000","last_clean":"2024-09-21T06:12:27.624534+0000","last_became_active":"2024-09-21T06:12:25.627258+0000","last_became_peered":"2024-09-21T06:12:25.627258+0000","last_unstale":"2024-09-21T06:12:27.624534+0000","last_undegraded":"2024-09-21T06:12:27.624534+0000","last_fullsized":"2024-09-21T06:12:27.624534+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:23.228842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.635687+0000","last_change":"2024-09-21T06:12:25.609541+0000","last_active":"2024-09-21T06:12:26.635687+0000","last_peered":"2024-09-21T06:12:26.635687+0000","last_clean":"2024-09-21T06:12:26.635687+0000","last_became_active":"2024-09-21T06:12:25.609220+0000","last_became_peered":"2024-09-21T06:12:25.609220+0000","last_unstale":"2024-09-21T06:12:26.635687+0000","last_undegraded":"2024-09-21T06:12:26.635687+0000","last_fullsized":"2024-09-21T06:12:26.635687+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:45.787384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618427+0000","last_change":"2024-09-21T06:12:25.616411+0000","last_active":"2024-09-21T06:12:26.618427+0000","last_peered":"2024-09-21T06:12:26.618427+0000","last_clean":"2024-09-21T06:12:26.618427+0000","last_became_active":"2024-09-21T06:12:25.616071+0000","last_became_peered":"2024-09-21T06:12:25.616071+0000","last_unstale":"2024-09-21T06:12:26.618427+0000","last_undegraded":"2024-09-21T06:12:26.618427+0000","last_fullsized":"2024-09-21T06:12:26.618427+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:23.228842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619302+0000","last_change":"2024-09-21T06:12:25.627766+0000","last_active":"2024-09-21T06:12:26.619302+0000","last_peered":"2024-09-21T06:12:26.619302+0000","last_clean":"2024-09-21T06:12:26.619302+0000","last_became_active":"2024-09-21T06:12:25.624288+0000","last_became_peered":"2024-09-21T06:12:25.624288+0000","last_unstale":"2024-09-21T06:12:26.619302+0000","last_undegraded":"2024-09-21T06:12:26.619302+0000","last_fullsized":"2024-09-21T06:12:26.619302+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:33.938443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.635905+0000","last_change":"2024-09-21T06:12:25.609517+0000","last_active":"2024-09-21T06:12:26.635905+0000","last_peered":"2024-09-21T06:12:26.635905+0000","last_clean":"2024-09-21T06:12:26.635905+0000","last_became_active":"2024-09-21T06:12:25.609167+0000","last_became_peered":"2024-09-21T06:12:25.609167+0000","last_unstale":"2024-09-21T06:12:26.635905+0000","last_undegraded":"2024-09-21T06:12:26.635905+0000","last_fullsized":"2024-09-21T06:12:26.635905+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:35.231184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618297+0000","last_change":"2024-09-21T06:12:25.609915+0000","last_active":"2024-09-21T06:12:26.618297+0000","last_peered":"2024-09-21T06:12:26.618297+0000","last_clean":"2024-09-21T06:12:26.618297+0000","last_became_active":"2024-09-21T06:12:25.608777+0000","last_became_peered":"2024-09-21T06:12:25.608777+0000","last_unstale":"2024-09-21T06:12:26.618297+0000","last_undegraded":"2024-09-21T06:12:26.618297+0000","last_fullsized":"2024-09-21T06:12:26.618297+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:40.121562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619113+0000","last_change":"2024-09-21T06:12:25.607648+0000","last_active":"2024-09-21T06:12:26.619113+0000","last_peered":"2024-09-21T06:12:26.619113+0000","last_clean":"2024-09-21T06:12:26.619113+0000","last_became_active":"2024-09-21T06:12:25.607316+0000","last_became_peered":"2024-09-21T06:12:25.607316+0000","last_unstale":"2024-09-21T06:12:26.619113+0000","last_undegraded":"2024-09-21T06:12:26.619113+0000","last_fullsized":"2024-09-21T06:12:26.619113+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:36.543983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.625198+0000","last_change":"2024-09-21T06:12:25.625307+0000","last_active":"2024-09-21T06:12:27.625198+0000","last_peered":"2024-09-21T06:12:27.625198+0000","last_clean":"2024-09-21T06:12:27.625198+0000","last_became_active":"2024-09-21T06:12:25.624933+0000","last_became_peered":"2024-09-21T06:12:25.624933+0000","last_unstale":"2024-09-21T06:12:27.625198+0000","last_undegraded":"2024-09-21T06:12:27.625198+0000","last_fullsized":"2024-09-21T06:12:27.625198+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:32.566609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618017+0000","last_change":"2024-09-21T06:12:25.618002+0000","last_active":"2024-09-21T06:12:26.618017+0000","last_peered":"2024-09-21T06:12:26.618017+0000","last_clean":"2024-09-21T06:12:26.618017+0000","last_became_active":"2024-09-21T06:12:25.617714+0000","last_became_peered":"2024-09-21T06:12:25.617714+0000","last_unstale":"2024-09-21T06:12:26.618017+0000","last_undegraded":"2024-09-21T06:12:26.618017+0000","last_fullsized":"2024-09-21T06:12:26.618017+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:43.136576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766063+0000","last_change":"2024-09-21T06:12:25.618664+0000","last_active":"2024-09-21T06:12:26.766063+0000","last_peered":"2024-09-21T06:12:26.766063+0000","last_clean":"2024-09-21T06:12:26.766063+0000","last_became_active":"2024-09-21T06:12:25.618261+0000","last_became_peered":"2024-09-21T06:12:25.618261+0000","last_unstale":"2024-09-21T06:12:26.766063+0000","last_undegraded":"2024-09-21T06:12:26.766063+0000","last_fullsized":"2024-09-21T06:12:26.766063+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:11.561785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619988+0000","last_change":"2024-09-21T06:12:25.619849+0000","last_active":"2024-09-21T06:12:26.619988+0000","last_peered":"2024-09-21T06:12:26.619988+0000","last_clean":"2024-09-21T06:12:26.619988+0000","last_became_active":"2024-09-21T06:12:25.619518+0000","last_became_peered":"2024-09-21T06:12:25.619518+0000","last_unstale":"2024-09-21T06:12:26.619988+0000","last_undegraded":"2024-09-21T06:12:26.619988+0000","last_fullsized":"2024-09-21T06:12:26.619988+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:48.182182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623621+0000","last_change":"2024-09-21T06:12:25.621753+0000","last_active":"2024-09-21T06:12:27.623621+0000","last_peered":"2024-09-21T06:12:27.623621+0000","last_clean":"2024-09-21T06:12:27.623621+0000","last_became_active":"2024-09-21T06:12:25.621433+0000","last_became_peered":"2024-09-21T06:12:25.621433+0000","last_unstale":"2024-09-21T06:12:27.623621+0000","last_undegraded":"2024-09-21T06:12:27.623621+0000","last_fullsized":"2024-09-21T06:12:27.623621+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:13.478353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636313+0000","last_change":"2024-09-21T06:12:25.613752+0000","last_active":"2024-09-21T06:12:26.636313+0000","last_peered":"2024-09-21T06:12:26.636313+0000","last_clean":"2024-09-21T06:12:26.636313+0000","last_became_active":"2024-09-21T06:12:25.613397+0000","last_became_peered":"2024-09-21T06:12:25.613397+0000","last_unstale":"2024-09-21T06:12:26.636313+0000","last_undegraded":"2024-09-21T06:12:26.636313+0000","last_fullsized":"2024-09-21T06:12:26.636313+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:44.609173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766479+0000","last_change":"2024-09-21T06:12:25.620384+0000","last_active":"2024-09-21T06:12:26.766479+0000","last_peered":"2024-09-21T06:12:26.766479+0000","last_clean":"2024-09-21T06:12:26.766479+0000","last_became_active":"2024-09-21T06:12:25.620034+0000","last_became_peered":"2024-09-21T06:12:25.620034+0000","last_unstale":"2024-09-21T06:12:26.766479+0000","last_undegraded":"2024-09-21T06:12:26.766479+0000","last_fullsized":"2024-09-21T06:12:26.766479+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:12.224272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619883+0000","last_change":"2024-09-21T06:12:25.627780+0000","last_active":"2024-09-21T06:12:26.619883+0000","last_peered":"2024-09-21T06:12:26.619883+0000","last_clean":"2024-09-21T06:12:26.619883+0000","last_became_active":"2024-09-21T06:12:25.627431+0000","last_became_peered":"2024-09-21T06:12:25.627431+0000","last_unstale":"2024-09-21T06:12:26.619883+0000","last_undegraded":"2024-09-21T06:12:26.619883+0000","last_fullsized":"2024-09-21T06:12:26.619883+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:57.631475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.625272+0000","last_change":"2024-09-21T06:12:25.622780+0000","last_active":"2024-09-21T06:12:27.625272+0000","last_peered":"2024-09-21T06:12:27.625272+0000","last_clean":"2024-09-21T06:12:27.625272+0000","last_became_active":"2024-09-21T06:12:25.622435+0000","last_became_peered":"2024-09-21T06:12:25.622435+0000","last_unstale":"2024-09-21T06:12:27.625272+0000","last_undegraded":"2024-09-21T06:12:27.625272+0000","last_fullsized":"2024-09-21T06:12:27.625272+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:57.326608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.625001+0000","last_change":"2024-09-21T06:12:25.624056+0000","last_active":"2024-09-21T06:12:27.625001+0000","last_peered":"2024-09-21T06:12:27.625001+0000","last_clean":"2024-09-21T06:12:27.625001+0000","last_became_active":"2024-09-21T06:12:25.623732+0000","last_became_peered":"2024-09-21T06:12:25.623732+0000","last_unstale":"2024-09-21T06:12:27.625001+0000","last_undegraded":"2024-09-21T06:12:27.625001+0000","last_fullsized":"2024-09-21T06:12:27.625001+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:39.068960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618057+0000","last_change":"2024-09-21T06:12:25.618510+0000","last_active":"2024-09-21T06:12:26.618057+0000","last_peered":"2024-09-21T06:12:26.618057+0000","last_clean":"2024-09-21T06:12:26.618057+0000","last_became_active":"2024-09-21T06:12:25.618160+0000","last_became_peered":"2024-09-21T06:12:25.618160+0000","last_unstale":"2024-09-21T06:12:26.618057+0000","last_undegraded":"2024-09-21T06:12:26.618057+0000","last_fullsized":"2024-09-21T06:12:26.618057+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:01.381247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766495+0000","last_change":"2024-09-21T06:12:25.609675+0000","last_active":"2024-09-21T06:12:26.766495+0000","last_peered":"2024-09-21T06:12:26.766495+0000","last_clean":"2024-09-21T06:12:26.766495+0000","last_became_active":"2024-09-21T06:12:25.609344+0000","last_became_peered":"2024-09-21T06:12:25.609344+0000","last_unstale":"2024-09-21T06:12:26.766495+0000","last_undegraded":"2024-09-21T06:12:26.766495+0000","last_fullsized":"2024-09-21T06:12:26.766495+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:29.672395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636385+0000","last_change":"2024-09-21T06:12:25.612562+0000","last_active":"2024-09-21T06:12:26.636385+0000","last_peered":"2024-09-21T06:12:26.636385+0000","last_clean":"2024-09-21T06:12:26.636385+0000","last_became_active":"2024-09-21T06:12:25.612225+0000","last_became_peered":"2024-09-21T06:12:25.612225+0000","last_unstale":"2024-09-21T06:12:26.636385+0000","last_undegraded":"2024-09-21T06:12:26.636385+0000","last_fullsized":"2024-09-21T06:12:26.636385+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:57.575899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766461+0000","last_change":"2024-09-21T06:12:25.630793+0000","last_active":"2024-09-21T06:12:26.766461+0000","last_peered":"2024-09-21T06:12:26.766461+0000","last_clean":"2024-09-21T06:12:26.766461+0000","last_became_active":"2024-09-21T06:12:25.630425+0000","last_became_peered":"2024-09-21T06:12:25.630425+0000","last_unstale":"2024-09-21T06:12:26.766461+0000","last_undegraded":"2024-09-21T06:12:26.766461+0000","last_fullsized":"2024-09-21T06:12:26.766461+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:18.861862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624866+0000","last_change":"2024-09-21T06:12:25.624809+0000","last_active":"2024-09-21T06:12:27.624866+0000","last_peered":"2024-09-21T06:12:27.624866+0000","last_clean":"2024-09-21T06:12:27.624866+0000","last_became_active":"2024-09-21T06:12:25.624433+0000","last_became_peered":"2024-09-21T06:12:25.624433+0000","last_unstale":"2024-09-21T06:12:27.624866+0000","last_undegraded":"2024-09-21T06:12:27.624866+0000","last_fullsized":"2024-09-21T06:12:27.624866+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:04.880942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.635775+0000","last_change":"2024-09-21T06:12:25.624642+0000","last_active":"2024-09-21T06:12:26.635775+0000","last_peered":"2024-09-21T06:12:26.635775+0000","last_clean":"2024-09-21T06:12:26.635775+0000","last_became_active":"2024-09-21T06:12:25.624308+0000","last_became_peered":"2024-09-21T06:12:25.624308+0000","last_unstale":"2024-09-21T06:12:26.635775+0000","last_undegraded":"2024-09-21T06:12:26.635775+0000","last_fullsized":"2024-09-21T06:12:26.635775+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:59.067650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.625488+0000","last_change":"2024-09-21T06:12:25.620357+0000","last_active":"2024-09-21T06:12:27.625488+0000","last_peered":"2024-09-21T06:12:27.625488+0000","last_clean":"2024-09-21T06:12:27.625488+0000","last_became_active":"2024-09-21T06:12:25.619973+0000","last_became_peered":"2024-09-21T06:12:25.619973+0000","last_unstale":"2024-09-21T06:12:27.625488+0000","last_undegraded":"2024-09-21T06:12:27.625488+0000","last_fullsized":"2024-09-21T06:12:27.625488+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:17.488083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617758+0000","last_change":"2024-09-21T06:12:25.624479+0000","last_active":"2024-09-21T06:12:26.617758+0000","last_peered":"2024-09-21T06:12:26.617758+0000","last_clean":"2024-09-21T06:12:26.617758+0000","last_became_active":"2024-09-21T06:12:25.624142+0000","last_became_peered":"2024-09-21T06:12:25.624142+0000","last_unstale":"2024-09-21T06:12:26.617758+0000","last_undegraded":"2024-09-21T06:12:26.617758+0000","last_fullsized":"2024-09-21T06:12:26.617758+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:48.699592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766305+0000","last_change":"2024-09-21T06:12:25.618988+0000","last_active":"2024-09-21T06:12:26.766305+0000","last_peered":"2024-09-21T06:12:26.766305+0000","last_clean":"2024-09-21T06:12:26.766305+0000","last_became_active":"2024-09-21T06:12:25.617536+0000","last_became_peered":"2024-09-21T06:12:25.617536+0000","last_unstale":"2024-09-21T06:12:26.766305+0000","last_undegraded":"2024-09-21T06:12:26.766305+0000","last_fullsized":"2024-09-21T06:12:26.766305+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:27.294981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618720+0000","last_change":"2024-09-21T06:12:25.615006+0000","last_active":"2024-09-21T06:12:26.618720+0000","last_peered":"2024-09-21T06:12:26.618720+0000","last_clean":"2024-09-21T06:12:26.618720+0000","last_became_active":"2024-09-21T06:12:25.611514+0000","last_became_peered":"2024-09-21T06:12:25.611514+0000","last_unstale":"2024-09-21T06:12:26.618720+0000","last_undegraded":"2024-09-21T06:12:26.618720+0000","last_fullsized":"2024-09-21T06:12:26.618720+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:05.454167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"141'4","reported_seq":19,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619730+0000","last_change":"2024-09-21T06:12:24.620583+0000","last_active":"2024-09-21T06:12:26.619730+0000","last_peered":"2024-09-21T06:12:26.619730+0000","last_clean":"2024-09-21T06:12:26.619730+0000","last_became_active":"2024-09-21T06:12:24.620281+0000","last_became_peered":"2024-09-21T06:12:24.620281+0000","last_unstale":"2024-09-21T06:12:26.619730+0000","last_undegraded":"2024-09-21T06:12:26.619730+0000","last_fullsized":"2024-09-21T06:12:26.619730+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:47.691713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618425+0000","last_change":"2024-09-21T06:12:25.612169+0000","last_active":"2024-09-21T06:12:26.618425+0000","last_peered":"2024-09-21T06:12:26.618425+0000","last_clean":"2024-09-21T06:12:26.618425+0000","last_became_active":"2024-09-21T06:12:25.611836+0000","last_became_peered":"2024-09-21T06:12:25.611836+0000","last_unstale":"2024-09-21T06:12:26.618425+0000","last_undegraded":"2024-09-21T06:12:26.618425+0000","last_fullsized":"2024-09-21T06:12:26.618425+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:50.732103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618272+0000","last_change":"2024-09-21T06:12:24.614480+0000","last_active":"2024-09-21T06:12:26.618272+0000","last_peered":"2024-09-21T06:12:26.618272+0000","last_clean":"2024-09-21T06:12:26.618272+0000","last_became_active":"2024-09-21T06:12:24.614159+0000","last_became_peered":"2024-09-21T06:12:24.614159+0000","last_unstale":"2024-09-21T06:12:26.618272+0000","last_undegraded":"2024-09-21T06:12:26.618272+0000","last_fullsized":"2024-09-21T06:12:26.618272+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:56.318730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.635935+0000","last_change":"2024-09-21T06:12:25.626929+0000","last_active":"2024-09-21T06:12:26.635935+0000","last_peered":"2024-09-21T06:12:26.635935+0000","last_clean":"2024-09-21T06:12:26.635935+0000","last_became_active":"2024-09-21T06:12:25.626605+0000","last_became_peered":"2024-09-21T06:12:25.626605+0000","last_unstale":"2024-09-21T06:12:26.635935+0000","last_undegraded":"2024-09-21T06:12:26.635935+0000","last_fullsized":"2024-09-21T06:12:26.635935+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:13.613982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.626208+0000","last_change":"2024-09-21T06:12:24.620901+0000","last_active":"2024-09-21T06:12:27.626208+0000","last_peered":"2024-09-21T06:12:27.626208+0000","last_clean":"2024-09-21T06:12:27.626208+0000","last_became_active":"2024-09-21T06:12:24.618875+0000","last_became_peered":"2024-09-21T06:12:24.618875+0000","last_unstale":"2024-09-21T06:12:27.626208+0000","last_undegraded":"2024-09-21T06:12:27.626208+0000","last_fullsized":"2024-09-21T06:12:27.626208+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:57.938112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618753+0000","last_change":"2024-09-21T06:12:25.616799+0000","last_active":"2024-09-21T06:12:26.618753+0000","last_peered":"2024-09-21T06:12:26.618753+0000","last_clean":"2024-09-21T06:12:26.618753+0000","last_became_active":"2024-09-21T06:12:25.616490+0000","last_became_peered":"2024-09-21T06:12:25.616490+0000","last_unstale":"2024-09-21T06:12:26.618753+0000","last_undegraded":"2024-09-21T06:12:26.618753+0000","last_fullsized":"2024-09-21T06:12:26.618753+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:44:43.974116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.626160+0000","last_change":"2024-09-21T06:12:24.620195+0000","last_active":"2024-09-21T06:12:27.626160+0000","last_peered":"2024-09-21T06:12:27.626160+0000","last_clean":"2024-09-21T06:12:27.626160+0000","last_became_active":"2024-09-21T06:12:24.619886+0000","last_became_peered":"2024-09-21T06:12:24.619886+0000","last_unstale":"2024-09-21T06:12:27.626160+0000","last_undegraded":"2024-09-21T06:12:27.626160+0000","last_fullsized":"2024-09-21T06:12:27.626160+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:46.739582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617866+0000","last_change":"2024-09-21T06:12:25.626295+0000","last_active":"2024-09-21T06:12:26.617866+0000","last_peered":"2024-09-21T06:12:26.617866+0000","last_clean":"2024-09-21T06:12:26.617866+0000","last_became_active":"2024-09-21T06:12:25.625938+0000","last_became_peered":"2024-09-21T06:12:25.625938+0000","last_unstale":"2024-09-21T06:12:26.617866+0000","last_undegraded":"2024-09-21T06:12:26.617866+0000","last_fullsized":"2024-09-21T06:12:26.617866+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:22.459130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766130+0000","last_change":"2024-09-21T06:12:24.619038+0000","last_active":"2024-09-21T06:12:26.766130+0000","last_peered":"2024-09-21T06:12:26.766130+0000","last_clean":"2024-09-21T06:12:26.766130+0000","last_became_active":"2024-09-21T06:12:24.618048+0000","last_became_peered":"2024-09-21T06:12:24.618048+0000","last_unstale":"2024-09-21T06:12:26.766130+0000","last_undegraded":"2024-09-21T06:12:26.766130+0000","last_fullsized":"2024-09-21T06:12:26.766130+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:06.581382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.625348+0000","last_change":"2024-09-21T06:12:25.632384+0000","last_active":"2024-09-21T06:12:27.625348+0000","last_peered":"2024-09-21T06:12:27.625348+0000","last_clean":"2024-09-21T06:12:27.625348+0000","last_became_active":"2024-09-21T06:12:25.632015+0000","last_became_peered":"2024-09-21T06:12:25.632015+0000","last_unstale":"2024-09-21T06:12:27.625348+0000","last_undegraded":"2024-09-21T06:12:27.625348+0000","last_fullsized":"2024-09-21T06:12:27.625348+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:56.559615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618399+0000","last_change":"2024-09-21T06:12:24.617857+0000","last_active":"2024-09-21T06:12:26.618399+0000","last_peered":"2024-09-21T06:12:26.618399+0000","last_clean":"2024-09-21T06:12:26.618399+0000","last_became_active":"2024-09-21T06:12:24.617506+0000","last_became_peered":"2024-09-21T06:12:24.617506+0000","last_unstale":"2024-09-21T06:12:26.618399+0000","last_undegraded":"2024-09-21T06:12:26.618399+0000","last_fullsized":"2024-09-21T06:12:26.618399+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:47.276272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617784+0000","last_change":"2024-09-21T06:12:25.608575+0000","last_active":"2024-09-21T06:12:26.617784+0000","last_peered":"2024-09-21T06:12:26.617784+0000","last_clean":"2024-09-21T06:12:26.617784+0000","last_became_active":"2024-09-21T06:12:25.608249+0000","last_became_peered":"2024-09-21T06:12:25.608249+0000","last_unstale":"2024-09-21T06:12:26.617784+0000","last_undegraded":"2024-09-21T06:12:26.617784+0000","last_fullsized":"2024-09-21T06:12:26.617784+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:19.272104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766105+0000","last_change":"2024-09-21T06:12:24.614660+0000","last_active":"2024-09-21T06:12:26.766105+0000","last_peered":"2024-09-21T06:12:26.766105+0000","last_clean":"2024-09-21T06:12:26.766105+0000","last_became_active":"2024-09-21T06:12:24.614326+0000","last_became_peered":"2024-09-21T06:12:24.614326+0000","last_unstale":"2024-09-21T06:12:26.766105+0000","last_undegraded":"2024-09-21T06:12:26.766105+0000","last_fullsized":"2024-09-21T06:12:26.766105+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:02.191560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617746+0000","last_change":"2024-09-21T06:12:25.609433+0000","last_active":"2024-09-21T06:12:26.617746+0000","last_peered":"2024-09-21T06:12:26.617746+0000","last_clean":"2024-09-21T06:12:26.617746+0000","last_became_active":"2024-09-21T06:12:25.608036+0000","last_became_peered":"2024-09-21T06:12:25.608036+0000","last_unstale":"2024-09-21T06:12:26.617746+0000","last_undegraded":"2024-09-21T06:12:26.617746+0000","last_fullsized":"2024-09-21T06:12:26.617746+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:58.284774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624874+0000","last_change":"2024-09-21T06:12:24.617782+0000","last_active":"2024-09-21T06:12:27.624874+0000","last_peered":"2024-09-21T06:12:27.624874+0000","last_clean":"2024-09-21T06:12:27.624874+0000","last_became_active":"2024-09-21T06:12:24.617445+0000","last_became_peered":"2024-09-21T06:12:24.617445+0000","last_unstale":"2024-09-21T06:12:27.624874+0000","last_undegraded":"2024-09-21T06:12:27.624874+0000","last_fullsized":"2024-09-21T06:12:27.624874+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:44.394243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619669+0000","last_change":"2024-09-21T06:12:25.618989+0000","last_active":"2024-09-21T06:12:26.619669+0000","last_peered":"2024-09-21T06:12:26.619669+0000","last_clean":"2024-09-21T06:12:26.619669+0000","last_became_active":"2024-09-21T06:12:25.614126+0000","last_became_peered":"2024-09-21T06:12:25.614126+0000","last_unstale":"2024-09-21T06:12:26.619669+0000","last_undegraded":"2024-09-21T06:12:26.619669+0000","last_fullsized":"2024-09-21T06:12:26.619669+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:52.286333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618530+0000","last_change":"2024-09-21T06:12:24.609257+0000","last_active":"2024-09-21T06:12:26.618530+0000","last_peered":"2024-09-21T06:12:26.618530+0000","last_clean":"2024-09-21T06:12:26.618530+0000","last_became_active":"2024-09-21T06:12:24.608867+0000","last_became_peered":"2024-09-21T06:12:24.608867+0000","last_unstale":"2024-09-21T06:12:26.618530+0000","last_undegraded":"2024-09-21T06:12:26.618530+0000","last_fullsized":"2024-09-21T06:12:26.618530+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:40.538811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619705+0000","last_change":"2024-09-21T06:12:25.618709+0000","last_active":"2024-09-21T06:12:26.619705+0000","last_peered":"2024-09-21T06:12:26.619705+0000","last_clean":"2024-09-21T06:12:26.619705+0000","last_became_active":"2024-09-21T06:12:25.618305+0000","last_became_peered":"2024-09-21T06:12:25.618305+0000","last_unstale":"2024-09-21T06:12:26.619705+0000","last_undegraded":"2024-09-21T06:12:26.619705+0000","last_fullsized":"2024-09-21T06:12:26.619705+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:35.783206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619160+0000","last_change":"2024-09-21T06:12:24.621430+0000","last_active":"2024-09-21T06:12:26.619160+0000","last_peered":"2024-09-21T06:12:26.619160+0000","last_clean":"2024-09-21T06:12:26.619160+0000","last_became_active":"2024-09-21T06:12:24.621083+0000","last_became_peered":"2024-09-21T06:12:24.621083+0000","last_unstale":"2024-09-21T06:12:26.619160+0000","last_undegraded":"2024-09-21T06:12:26.619160+0000","last_fullsized":"2024-09-21T06:12:26.619160+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:44.115082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766492+0000","last_change":"2024-09-21T06:12:25.613964+0000","last_active":"2024-09-21T06:12:26.766492+0000","last_peered":"2024-09-21T06:12:26.766492+0000","last_clean":"2024-09-21T06:12:26.766492+0000","last_became_active":"2024-09-21T06:12:25.613644+0000","last_became_peered":"2024-09-21T06:12:25.613644+0000","last_unstale":"2024-09-21T06:12:26.766492+0000","last_undegraded":"2024-09-21T06:12:26.766492+0000","last_fullsized":"2024-09-21T06:12:26.766492+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:59.227095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"141'6","reported_seq":21,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624793+0000","last_change":"2024-09-21T06:12:24.612261+0000","last_active":"2024-09-21T06:12:27.624793+0000","last_peered":"2024-09-21T06:12:27.624793+0000","last_clean":"2024-09-21T06:12:27.624793+0000","last_became_active":"2024-09-21T06:12:24.611940+0000","last_became_peered":"2024-09-21T06:12:24.611940+0000","last_unstale":"2024-09-21T06:12:27.624793+0000","last_undegraded":"2024-09-21T06:12:27.624793+0000","last_fullsized":"2024-09-21T06:12:27.624793+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:27.350833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623258+0000","last_change":"2024-09-21T06:12:25.632907+0000","last_active":"2024-09-21T06:12:27.623258+0000","last_peered":"2024-09-21T06:12:27.623258+0000","last_clean":"2024-09-21T06:12:27.623258+0000","last_became_active":"2024-09-21T06:12:25.632521+0000","last_became_peered":"2024-09-21T06:12:25.632521+0000","last_unstale":"2024-09-21T06:12:27.623258+0000","last_undegraded":"2024-09-21T06:12:27.623258+0000","last_fullsized":"2024-09-21T06:12:27.623258+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:18.527754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623331+0000","last_change":"2024-09-21T06:12:24.612768+0000","last_active":"2024-09-21T06:12:27.623331+0000","last_peered":"2024-09-21T06:12:27.623331+0000","last_clean":"2024-09-21T06:12:27.623331+0000","last_became_active":"2024-09-21T06:12:24.612200+0000","last_became_peered":"2024-09-21T06:12:24.612200+0000","last_unstale":"2024-09-21T06:12:27.623331+0000","last_undegraded":"2024-09-21T06:12:27.623331+0000","last_fullsized":"2024-09-21T06:12:27.623331+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:12.508843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617834+0000","last_change":"2024-09-21T06:12:25.615733+0000","last_active":"2024-09-21T06:12:26.617834+0000","last_peered":"2024-09-21T06:12:26.617834+0000","last_clean":"2024-09-21T06:12:26.617834+0000","last_became_active":"2024-09-21T06:12:25.615412+0000","last_became_peered":"2024-09-21T06:12:25.615412+0000","last_unstale":"2024-09-21T06:12:26.617834+0000","last_undegraded":"2024-09-21T06:12:26.617834+0000","last_fullsized":"2024-09-21T06:12:26.617834+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:49.430682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623339+0000","last_change":"2024-09-21T06:12:24.616333+0000","last_active":"2024-09-21T06:12:27.623339+0000","last_peered":"2024-09-21T06:12:27.623339+0000","last_clean":"2024-09-21T06:12:27.623339+0000","last_became_active":"2024-09-21T06:12:24.616024+0000","last_became_peered":"2024-09-21T06:12:24.616024+0000","last_unstale":"2024-09-21T06:12:27.623339+0000","last_undegraded":"2024-09-21T06:12:27.623339+0000","last_fullsized":"2024-09-21T06:12:27.623339+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:16.059748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617753+0000","last_change":"2024-09-21T06:12:25.609016+0000","last_active":"2024-09-21T06:12:26.617753+0000","last_peered":"2024-09-21T06:12:26.617753+0000","last_clean":"2024-09-21T06:12:26.617753+0000","last_became_active":"2024-09-21T06:12:25.608638+0000","last_became_peered":"2024-09-21T06:12:25.608638+0000","last_unstale":"2024-09-21T06:12:26.617753+0000","last_undegraded":"2024-09-21T06:12:26.617753+0000","last_fullsized":"2024-09-21T06:12:26.617753+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:33.409594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.635586+0000","last_change":"2024-09-21T06:12:24.609267+0000","last_active":"2024-09-21T06:12:26.635586+0000","last_peered":"2024-09-21T06:12:26.635586+0000","last_clean":"2024-09-21T06:12:26.635586+0000","last_became_active":"2024-09-21T06:12:24.608943+0000","last_became_peered":"2024-09-21T06:12:24.608943+0000","last_unstale":"2024-09-21T06:12:26.635586+0000","last_undegraded":"2024-09-21T06:12:26.635586+0000","last_fullsized":"2024-09-21T06:12:26.635586+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:33.783011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623209+0000","last_change":"2024-09-21T06:12:25.623446+0000","last_active":"2024-09-21T06:12:27.623209+0000","last_peered":"2024-09-21T06:12:27.623209+0000","last_clean":"2024-09-21T06:12:27.623209+0000","last_became_active":"2024-09-21T06:12:25.623154+0000","last_became_peered":"2024-09-21T06:12:25.623154+0000","last_unstale":"2024-09-21T06:12:27.623209+0000","last_undegraded":"2024-09-21T06:12:27.623209+0000","last_fullsized":"2024-09-21T06:12:27.623209+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:55.131765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"141'8","reported_seq":23,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766233+0000","last_change":"2024-09-21T06:12:24.608576+0000","last_active":"2024-09-21T06:12:26.766233+0000","last_peered":"2024-09-21T06:12:26.766233+0000","last_clean":"2024-09-21T06:12:26.766233+0000","last_became_active":"2024-09-21T06:12:24.608251+0000","last_became_peered":"2024-09-21T06:12:24.608251+0000","last_unstale":"2024-09-21T06:12:26.766233+0000","last_undegraded":"2024-09-21T06:12:26.766233+0000","last_fullsized":"2024-09-21T06:12:26.766233+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:44.810992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618956+0000","last_change":"2024-09-21T06:12:25.616851+0000","last_active":"2024-09-21T06:12:26.618956+0000","last_peered":"2024-09-21T06:12:26.618956+0000","last_clean":"2024-09-21T06:12:26.618956+0000","last_became_active":"2024-09-21T06:12:25.616357+0000","last_became_peered":"2024-09-21T06:12:25.616357+0000","last_unstale":"2024-09-21T06:12:26.618956+0000","last_undegraded":"2024-09-21T06:12:26.618956+0000","last_fullsized":"2024-09-21T06:12:26.618956+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:58.242236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619772+0000","last_change":"2024-09-21T06:12:24.607165+0000","last_active":"2024-09-21T06:12:26.619772+0000","last_peered":"2024-09-21T06:12:26.619772+0000","last_clean":"2024-09-21T06:12:26.619772+0000","last_became_active":"2024-09-21T06:12:24.606847+0000","last_became_peered":"2024-09-21T06:12:24.606847+0000","last_unstale":"2024-09-21T06:12:26.619772+0000","last_undegraded":"2024-09-21T06:12:26.619772+0000","last_fullsized":"2024-09-21T06:12:26.619772+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:05.896886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766622+0000","last_change":"2024-09-21T06:12:25.611430+0000","last_active":"2024-09-21T06:12:26.766622+0000","last_peered":"2024-09-21T06:12:26.766622+0000","last_clean":"2024-09-21T06:12:26.766622+0000","last_became_active":"2024-09-21T06:12:25.611097+0000","last_became_peered":"2024-09-21T06:12:25.611097+0000","last_unstale":"2024-09-21T06:12:26.766622+0000","last_undegraded":"2024-09-21T06:12:26.766622+0000","last_fullsized":"2024-09-21T06:12:26.766622+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:01.724361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618395+0000","last_change":"2024-09-21T06:12:24.614422+0000","last_active":"2024-09-21T06:12:26.618395+0000","last_peered":"2024-09-21T06:12:26.618395+0000","last_clean":"2024-09-21T06:12:26.618395+0000","last_became_active":"2024-09-21T06:12:24.614131+0000","last_became_peered":"2024-09-21T06:12:24.614131+0000","last_unstale":"2024-09-21T06:12:26.618395+0000","last_undegraded":"2024-09-21T06:12:26.618395+0000","last_fullsized":"2024-09-21T06:12:26.618395+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:55.551737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623591+0000","last_change":"2024-09-21T06:12:25.629107+0000","last_active":"2024-09-21T06:12:27.623591+0000","last_peered":"2024-09-21T06:12:27.623591+0000","last_clean":"2024-09-21T06:12:27.623591+0000","last_became_active":"2024-09-21T06:12:25.625307+0000","last_became_peered":"2024-09-21T06:12:25.625307+0000","last_unstale":"2024-09-21T06:12:27.623591+0000","last_undegraded":"2024-09-21T06:12:27.623591+0000","last_fullsized":"2024-09-21T06:12:27.623591+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:40.434350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766582+0000","last_change":"2024-09-21T06:12:24.618757+0000","last_active":"2024-09-21T06:12:26.766582+0000","last_peered":"2024-09-21T06:12:26.766582+0000","last_clean":"2024-09-21T06:12:26.766582+0000","last_became_active":"2024-09-21T06:12:24.617250+0000","last_became_peered":"2024-09-21T06:12:24.617250+0000","last_unstale":"2024-09-21T06:12:26.766582+0000","last_undegraded":"2024-09-21T06:12:26.766582+0000","last_fullsized":"2024-09-21T06:12:26.766582+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:27.371208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636066+0000","last_change":"2024-09-21T06:12:25.631701+0000","last_active":"2024-09-21T06:12:26.636066+0000","last_peered":"2024-09-21T06:12:26.636066+0000","last_clean":"2024-09-21T06:12:26.636066+0000","last_became_active":"2024-09-21T06:12:25.631223+0000","last_became_peered":"2024-09-21T06:12:25.631223+0000","last_unstale":"2024-09-21T06:12:26.636066+0000","last_undegraded":"2024-09-21T06:12:26.636066+0000","last_fullsized":"2024-09-21T06:12:26.636066+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:18.942133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618013+0000","last_change":"2024-09-21T06:12:24.616936+0000","last_active":"2024-09-21T06:12:26.618013+0000","last_peered":"2024-09-21T06:12:26.618013+0000","last_clean":"2024-09-21T06:12:26.618013+0000","last_became_active":"2024-09-21T06:12:24.616604+0000","last_became_peered":"2024-09-21T06:12:24.616604+0000","last_unstale":"2024-09-21T06:12:26.618013+0000","last_undegraded":"2024-09-21T06:12:26.618013+0000","last_fullsized":"2024-09-21T06:12:26.618013+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:16.480205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.625566+0000","last_change":"2024-09-21T06:12:25.621872+0000","last_active":"2024-09-21T06:12:27.625566+0000","last_peered":"2024-09-21T06:12:27.625566+0000","last_clean":"2024-09-21T06:12:27.625566+0000","last_became_active":"2024-09-21T06:12:25.621343+0000","last_became_peered":"2024-09-21T06:12:25.621343+0000","last_unstale":"2024-09-21T06:12:27.625566+0000","last_undegraded":"2024-09-21T06:12:27.625566+0000","last_fullsized":"2024-09-21T06:12:27.625566+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:41.546689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624627+0000","last_change":"2024-09-21T06:12:24.612100+0000","last_active":"2024-09-21T06:12:27.624627+0000","last_peered":"2024-09-21T06:12:27.624627+0000","last_clean":"2024-09-21T06:12:27.624627+0000","last_became_active":"2024-09-21T06:12:24.611693+0000","last_became_peered":"2024-09-21T06:12:24.611693+0000","last_unstale":"2024-09-21T06:12:27.624627+0000","last_undegraded":"2024-09-21T06:12:27.624627+0000","last_fullsized":"2024-09-21T06:12:27.624627+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:44.115082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624584+0000","last_change":"2024-09-21T06:12:25.627383+0000","last_active":"2024-09-21T06:12:27.624584+0000","last_peered":"2024-09-21T06:12:27.624584+0000","last_clean":"2024-09-21T06:12:27.624584+0000","last_became_active":"2024-09-21T06:12:25.627003+0000","last_became_peered":"2024-09-21T06:12:25.627003+0000","last_unstale":"2024-09-21T06:12:27.624584+0000","last_undegraded":"2024-09-21T06:12:27.624584+0000","last_fullsized":"2024-09-21T06:12:27.624584+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:07.547988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618975+0000","last_change":"2024-09-21T06:12:24.619226+0000","last_active":"2024-09-21T06:12:26.618975+0000","last_peered":"2024-09-21T06:12:26.618975+0000","last_clean":"2024-09-21T06:12:26.618975+0000","last_became_active":"2024-09-21T06:12:24.618807+0000","last_became_peered":"2024-09-21T06:12:24.618807+0000","last_unstale":"2024-09-21T06:12:26.618975+0000","last_undegraded":"2024-09-21T06:12:26.618975+0000","last_fullsized":"2024-09-21T06:12:26.618975+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:11.843425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624957+0000","last_change":"2024-09-21T06:12:25.614940+0000","last_active":"2024-09-21T06:12:27.624957+0000","last_peered":"2024-09-21T06:12:27.624957+0000","last_clean":"2024-09-21T06:12:27.624957+0000","last_became_active":"2024-09-21T06:12:25.614554+0000","last_became_peered":"2024-09-21T06:12:25.614554+0000","last_unstale":"2024-09-21T06:12:27.624957+0000","last_undegraded":"2024-09-21T06:12:27.624957+0000","last_fullsized":"2024-09-21T06:12:27.624957+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:40.434350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619300+0000","last_change":"2024-09-21T06:12:24.621062+0000","last_active":"2024-09-21T06:12:26.619300+0000","last_peered":"2024-09-21T06:12:26.619300+0000","last_clean":"2024-09-21T06:12:26.619300+0000","last_became_active":"2024-09-21T06:12:24.620084+0000","last_became_peered":"2024-09-21T06:12:24.620084+0000","last_unstale":"2024-09-21T06:12:26.619300+0000","last_undegraded":"2024-09-21T06:12:26.619300+0000","last_fullsized":"2024-09-21T06:12:26.619300+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:27.350833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624543+0000","last_change":"2024-09-21T06:12:25.627242+0000","last_active":"2024-09-21T06:12:27.624543+0000","last_peered":"2024-09-21T06:12:27.624543+0000","last_clean":"2024-09-21T06:12:27.624543+0000","last_became_active":"2024-09-21T06:12:25.626624+0000","last_became_peered":"2024-09-21T06:12:25.626624+0000","last_unstale":"2024-09-21T06:12:27.624543+0000","last_undegraded":"2024-09-21T06:12:27.624543+0000","last_fullsized":"2024-09-21T06:12:27.624543+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:29.265803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.622907+0000","last_change":"2024-09-21T06:12:24.612040+0000","last_active":"2024-09-21T06:12:27.622907+0000","last_peered":"2024-09-21T06:12:27.622907+0000","last_clean":"2024-09-21T06:12:27.622907+0000","last_became_active":"2024-09-21T06:12:24.611303+0000","last_became_peered":"2024-09-21T06:12:24.611303+0000","last_unstale":"2024-09-21T06:12:27.622907+0000","last_undegraded":"2024-09-21T06:12:27.622907+0000","last_fullsized":"2024-09-21T06:12:27.622907+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:03.617075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619690+0000","last_change":"2024-09-21T06:12:25.613627+0000","last_active":"2024-09-21T06:12:26.619690+0000","last_peered":"2024-09-21T06:12:26.619690+0000","last_clean":"2024-09-21T06:12:26.619690+0000","last_became_active":"2024-09-21T06:12:25.611897+0000","last_became_peered":"2024-09-21T06:12:25.611897+0000","last_unstale":"2024-09-21T06:12:26.619690+0000","last_undegraded":"2024-09-21T06:12:26.619690+0000","last_fullsized":"2024-09-21T06:12:26.619690+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:59.227095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619884+0000","last_change":"2024-09-21T06:12:24.621410+0000","last_active":"2024-09-21T06:12:26.619884+0000","last_peered":"2024-09-21T06:12:26.619884+0000","last_clean":"2024-09-21T06:12:26.619884+0000","last_became_active":"2024-09-21T06:12:24.620575+0000","last_became_peered":"2024-09-21T06:12:24.620575+0000","last_unstale":"2024-09-21T06:12:26.619884+0000","last_undegraded":"2024-09-21T06:12:26.619884+0000","last_fullsized":"2024-09-21T06:12:26.619884+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:18.264225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623693+0000","last_change":"2024-09-21T06:12:25.632519+0000","last_active":"2024-09-21T06:12:27.623693+0000","last_peered":"2024-09-21T06:12:27.623693+0000","last_clean":"2024-09-21T06:12:27.623693+0000","last_became_active":"2024-09-21T06:12:25.632081+0000","last_became_peered":"2024-09-21T06:12:25.632081+0000","last_unstale":"2024-09-21T06:12:27.623693+0000","last_undegraded":"2024-09-21T06:12:27.623693+0000","last_fullsized":"2024-09-21T06:12:27.623693+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:55.337729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"142'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.638091+0000","last_change":"2024-09-21T06:12:24.619812+0000","last_active":"2024-09-21T06:12:26.638091+0000","last_peered":"2024-09-21T06:12:26.638091+0000","last_clean":"2024-09-21T06:12:26.638091+0000","last_became_active":"2024-09-21T06:12:24.619450+0000","last_became_peered":"2024-09-21T06:12:24.619450+0000","last_unstale":"2024-09-21T06:12:26.638091+0000","last_undegraded":"2024-09-21T06:12:26.638091+0000","last_fullsized":"2024-09-21T06:12:26.638091+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:39.650262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618099+0000","last_change":"2024-09-21T06:12:25.628533+0000","last_active":"2024-09-21T06:12:26.618099+0000","last_peered":"2024-09-21T06:12:26.618099+0000","last_clean":"2024-09-21T06:12:26.618099+0000","last_became_active":"2024-09-21T06:12:25.627938+0000","last_became_peered":"2024-09-21T06:12:25.627938+0000","last_unstale":"2024-09-21T06:12:26.618099+0000","last_undegraded":"2024-09-21T06:12:26.618099+0000","last_fullsized":"2024-09-21T06:12:26.618099+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:04.505044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636248+0000","last_change":"2024-09-21T06:12:24.616468+0000","last_active":"2024-09-21T06:12:26.636248+0000","last_peered":"2024-09-21T06:12:26.636248+0000","last_clean":"2024-09-21T06:12:26.636248+0000","last_became_active":"2024-09-21T06:12:24.615933+0000","last_became_peered":"2024-09-21T06:12:24.615933+0000","last_unstale":"2024-09-21T06:12:26.636248+0000","last_undegraded":"2024-09-21T06:12:26.636248+0000","last_fullsized":"2024-09-21T06:12:26.636248+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:06.245427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636131+0000","last_change":"2024-09-21T06:12:25.623506+0000","last_active":"2024-09-21T06:12:26.636131+0000","last_peered":"2024-09-21T06:12:26.636131+0000","last_clean":"2024-09-21T06:12:26.636131+0000","last_became_active":"2024-09-21T06:12:25.623141+0000","last_became_peered":"2024-09-21T06:12:25.623141+0000","last_unstale":"2024-09-21T06:12:26.636131+0000","last_undegraded":"2024-09-21T06:12:26.636131+0000","last_fullsized":"2024-09-21T06:12:26.636131+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:15.949252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636060+0000","last_change":"2024-09-21T06:12:24.612148+0000","last_active":"2024-09-21T06:12:26.636060+0000","last_peered":"2024-09-21T06:12:26.636060+0000","last_clean":"2024-09-21T06:12:26.636060+0000","last_became_active":"2024-09-21T06:12:24.611844+0000","last_became_peered":"2024-09-21T06:12:24.611844+0000","last_unstale":"2024-09-21T06:12:26.636060+0000","last_undegraded":"2024-09-21T06:12:26.636060+0000","last_fullsized":"2024-09-21T06:12:26.636060+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:32.529980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766355+0000","last_change":"2024-09-21T06:12:25.633461+0000","last_active":"2024-09-21T06:12:26.766355+0000","last_peered":"2024-09-21T06:12:26.766355+0000","last_clean":"2024-09-21T06:12:26.766355+0000","last_became_active":"2024-09-21T06:12:25.632910+0000","last_became_peered":"2024-09-21T06:12:25.632910+0000","last_unstale":"2024-09-21T06:12:26.766355+0000","last_undegraded":"2024-09-21T06:12:26.766355+0000","last_fullsized":"2024-09-21T06:12:26.766355+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:57.191937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624785+0000","last_change":"2024-09-21T06:12:24.622020+0000","last_active":"2024-09-21T06:12:27.624785+0000","last_peered":"2024-09-21T06:12:27.624785+0000","last_clean":"2024-09-21T06:12:27.624785+0000","last_became_active":"2024-09-21T06:12:24.621522+0000","last_became_peered":"2024-09-21T06:12:24.621522+0000","last_unstale":"2024-09-21T06:12:27.624785+0000","last_undegraded":"2024-09-21T06:12:27.624785+0000","last_fullsized":"2024-09-21T06:12:27.624785+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:47.691713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619270+0000","last_change":"2024-09-21T06:12:25.624517+0000","last_active":"2024-09-21T06:12:26.619270+0000","last_peered":"2024-09-21T06:12:26.619270+0000","last_clean":"2024-09-21T06:12:26.619270+0000","last_became_active":"2024-09-21T06:12:25.624185+0000","last_became_peered":"2024-09-21T06:12:25.624185+0000","last_unstale":"2024-09-21T06:12:26.619270+0000","last_undegraded":"2024-09-21T06:12:26.619270+0000","last_fullsized":"2024-09-21T06:12:26.619270+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:48.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617941+0000","last_change":"2024-09-21T06:12:24.621829+0000","last_active":"2024-09-21T06:12:26.617941+0000","last_peered":"2024-09-21T06:12:26.617941+0000","last_clean":"2024-09-21T06:12:26.617941+0000","last_became_active":"2024-09-21T06:12:24.621031+0000","last_became_peered":"2024-09-21T06:12:24.621031+0000","last_unstale":"2024-09-21T06:12:26.617941+0000","last_undegraded":"2024-09-21T06:12:26.617941+0000","last_fullsized":"2024-09-21T06:12:26.617941+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:46.852585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618330+0000","last_change":"2024-09-21T06:12:25.611326+0000","last_active":"2024-09-21T06:12:26.618330+0000","last_peered":"2024-09-21T06:12:26.618330+0000","last_clean":"2024-09-21T06:12:26.618330+0000","last_became_active":"2024-09-21T06:12:25.611018+0000","last_became_peered":"2024-09-21T06:12:25.611018+0000","last_unstale":"2024-09-21T06:12:26.618330+0000","last_undegraded":"2024-09-21T06:12:26.618330+0000","last_fullsized":"2024-09-21T06:12:26.618330+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:00.574834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623036+0000","last_change":"2024-09-21T06:12:24.611596+0000","last_active":"2024-09-21T06:12:27.623036+0000","last_peered":"2024-09-21T06:12:27.623036+0000","last_clean":"2024-09-21T06:12:27.623036+0000","last_became_active":"2024-09-21T06:12:24.611245+0000","last_became_peered":"2024-09-21T06:12:24.611245+0000","last_unstale":"2024-09-21T06:12:27.623036+0000","last_undegraded":"2024-09-21T06:12:27.623036+0000","last_fullsized":"2024-09-21T06:12:27.623036+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:38.406677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618842+0000","last_change":"2024-09-21T06:12:25.607106+0000","last_active":"2024-09-21T06:12:26.618842+0000","last_peered":"2024-09-21T06:12:26.618842+0000","last_clean":"2024-09-21T06:12:26.618842+0000","last_became_active":"2024-09-21T06:12:25.606791+0000","last_became_peered":"2024-09-21T06:12:25.606791+0000","last_unstale":"2024-09-21T06:12:26.618842+0000","last_undegraded":"2024-09-21T06:12:26.618842+0000","last_fullsized":"2024-09-21T06:12:26.618842+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:29.880971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617564+0000","last_change":"2024-09-21T06:12:24.614354+0000","last_active":"2024-09-21T06:12:26.617564+0000","last_peered":"2024-09-21T06:12:26.617564+0000","last_clean":"2024-09-21T06:12:26.617564+0000","last_became_active":"2024-09-21T06:12:24.614043+0000","last_became_peered":"2024-09-21T06:12:24.614043+0000","last_unstale":"2024-09-21T06:12:26.617564+0000","last_undegraded":"2024-09-21T06:12:26.617564+0000","last_fullsized":"2024-09-21T06:12:26.617564+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:38.061082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624578+0000","last_change":"2024-09-21T06:12:25.627314+0000","last_active":"2024-09-21T06:12:27.624578+0000","last_peered":"2024-09-21T06:12:27.624578+0000","last_clean":"2024-09-21T06:12:27.624578+0000","last_became_active":"2024-09-21T06:12:25.626867+0000","last_became_peered":"2024-09-21T06:12:25.626867+0000","last_unstale":"2024-09-21T06:12:27.624578+0000","last_undegraded":"2024-09-21T06:12:27.624578+0000","last_fullsized":"2024-09-21T06:12:27.624578+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:06.793918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"141'3","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623565+0000","last_change":"2024-09-21T06:12:24.610884+0000","last_active":"2024-09-21T06:12:27.623565+0000","last_peered":"2024-09-21T06:12:27.623565+0000","last_clean":"2024-09-21T06:12:27.623565+0000","last_became_active":"2024-09-21T06:12:24.610544+0000","last_became_peered":"2024-09-21T06:12:24.610544+0000","last_unstale":"2024-09-21T06:12:27.623565+0000","last_undegraded":"2024-09-21T06:12:27.623565+0000","last_fullsized":"2024-09-21T06:12:27.623565+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:24.800893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":334,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618557+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:12:26.618557+0000","last_peered":"2024-09-21T06:12:26.618557+0000","last_clean":"2024-09-21T06:12:26.618557+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:12:26.618557+0000","last_undegraded":"2024-09-21T06:12:26.618557+0000","last_fullsized":"2024-09-21T06:12:26.618557+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509637,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132004,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239836,"statfs":{"total":96636764160,"available":96501592064,"internally_reserved":0,"allocated":2371584,"data_stored":1371054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509637,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141152,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230688,"statfs":{"total":96636764160,"available":96492224512,"internally_reserved":0,"allocated":1781760,"data_stored":780180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509636,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87000,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284840,"statfs":{"total":96636764160,"available":96547676160,"internally_reserved":0,"allocated":1773568,"data_stored":780056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509637,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136984,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234856,"statfs":{"total":96636764160,"available":96496492544,"internally_reserved":0,"allocated":1769472,"data_stored":780034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509636,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123096,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248744,"statfs":{"total":96636764160,"available":96510713856,"internally_reserved":0,"allocated":1773568,"data_stored":780124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509638,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163368,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208472,"statfs":{"total":96636764160,"available":96469475328,"internally_reserved":0,"allocated":1785856,"data_stored":781716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509636,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123688,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248152,"statfs":{"total":96636764160,"available":96510107648,"internally_reserved":0,"allocated":2379776,"data_stored":1372048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542341,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167512,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204328,"statfs":{"total":96636764160,"available":96465231872,"internally_reserved":0,"allocated":1773568,"data_stored":780616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:12:29.820 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:12:30.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.022+0000 7fa281200640 1 -- 172.21.15.50:0/3324716947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27c0b24d0 msgr2=0x7fa27c0b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.022+0000 7fa281200640 1 --2- 172.21.15.50:0/3324716947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27c0b24d0 0x7fa27c0b3130 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fa27c071ae0 tx=0x7fa27401cc80 comp rx=0 tx=0).stop 2024-09-21T06:12:30.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.022+0000 7fa281200640 1 -- 172.21.15.50:0/3324716947 shutdown_connections 2024-09-21T06:12:30.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.022+0000 7fa281200640 1 --2- 172.21.15.50:0/3324716947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27c0b24d0 0x7fa27c0b3130 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.022+0000 7fa281200640 1 --2- 172.21.15.50:0/3324716947 >> v2:172.21.15.88:3300/0 conn(0x7fa27c155a00 0x7fa27c15f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.022+0000 7fa281200640 1 -- 172.21.15.50:0/3324716947 >> 172.21.15.50:0/3324716947 conn(0x7fa27c06cdf0 msgr2=0x7fa27c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:30.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.022+0000 7fa281200640 1 -- 172.21.15.50:0/3324716947 shutdown_connections 2024-09-21T06:12:30.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.022+0000 7fa281200640 1 -- 172.21.15.50:0/3324716947 wait complete. 2024-09-21T06:12:30.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.022+0000 7fa281200640 1 Processor -- start 2024-09-21T06:12:30.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.022+0000 7fa281200640 1 -- start start 2024-09-21T06:12:30.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa281200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c155a00 0x7fa27c141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa281200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa27c15f5b0 0x7fa27c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa281200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27c0cfb60 0x7fa27c0aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa281200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c168af0 con 0x7fa27c0cfb60 2024-09-21T06:12:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa281200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c168970 con 0x7fa27c155a00 2024-09-21T06:12:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa281200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa27c168c70 con 0x7fa27c15f5b0 2024-09-21T06:12:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa27be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27c0cfb60 0x7fa27c0aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa27be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27c0cfb60 0x7fa27c0aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53362/0 (socket says 172.21.15.50:53362) 2024-09-21T06:12:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa27be00640 1 -- 172.21.15.50:0/883883391 learned_addr learned my addr 172.21.15.50:0/883883391 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa27b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c155a00 0x7fa27c141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa273e00640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa27c15f5b0 0x7fa27c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa27be00640 1 -- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa27c15f5b0 msgr2=0x7fa27c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa27be00640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa27c15f5b0 0x7fa27c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa27be00640 1 -- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c155a00 msgr2=0x7fa27c141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa27be00640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c155a00 0x7fa27c141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa27be00640 1 -- 172.21.15.50:0/883883391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27c152ed0 con 0x7fa27c0cfb60 2024-09-21T06:12:30.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa27be00640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27c0cfb60 0x7fa27c0aee20 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa274009bd0 tx=0x7fa2740092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:30.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa273400640 1 -- 172.21.15.50:0/883883391 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2740bb070 con 0x7fa27c0cfb60 2024-09-21T06:12:30.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa273400640 1 -- 172.21.15.50:0/883883391 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa274004d40 con 0x7fa27c0cfb60 2024-09-21T06:12:30.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa281200640 1 -- 172.21.15.50:0/883883391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27c155120 con 0x7fa27c0cfb60 2024-09-21T06:12:30.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa273400640 1 -- 172.21.15.50:0/883883391 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2740b2040 con 0x7fa27c0cfb60 2024-09-21T06:12:30.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa281200640 1 -- 172.21.15.50:0/883883391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27c154360 con 0x7fa27c0cfb60 2024-09-21T06:12:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.026+0000 7fa281200640 1 -- 172.21.15.50:0/883883391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27c153df0 con 0x7fa27c0cfb60 2024-09-21T06:12:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.030+0000 7fa273400640 1 -- 172.21.15.50:0/883883391 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fa2740be070 con 0x7fa27c0cfb60 2024-09-21T06:12:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.030+0000 7fa273400640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa2580754e0 0x7fa2580779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:30.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.030+0000 7fa273400640 1 -- 172.21.15.50:0/883883391 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7fa27412d6b0 con 0x7fa27c0cfb60 2024-09-21T06:12:30.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.030+0000 7fa27b400640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa2580754e0 0x7fa2580779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:30.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.030+0000 7fa27b400640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa2580754e0 0x7fa2580779f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa26c004630 tx=0x7fa26c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:30.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.034+0000 7fa273400640 1 -- 172.21.15.50:0/883883391 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa2740c5020 con 0x7fa27c0cfb60 2024-09-21T06:12:30.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.206+0000 7fa281200640 1 -- 172.21.15.50:0/883883391 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa27c0b3130 con 0x7fa2580754e0 2024-09-21T06:12:30.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.214+0000 7fa273400640 1 -- 172.21.15.50:0/883883391 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252499 (secure 0 0 0) 0x7fa27c0b3130 con 0x7fa2580754e0 2024-09-21T06:12:30.213 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:30.215 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:12:30.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.214+0000 7fa281200640 1 -- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa2580754e0 msgr2=0x7fa2580779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.214+0000 7fa281200640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa2580754e0 0x7fa2580779f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa26c004630 tx=0x7fa26c00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.214+0000 7fa281200640 1 -- 172.21.15.50:0/883883391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27c0cfb60 msgr2=0x7fa27c0aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.214+0000 7fa281200640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27c0cfb60 0x7fa27c0aee20 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa274009bd0 tx=0x7fa2740092c0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.218+0000 7fa281200640 1 -- 172.21.15.50:0/883883391 shutdown_connections 2024-09-21T06:12:30.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.218+0000 7fa281200640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa2580754e0 0x7fa2580779f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.218+0000 7fa281200640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27c0cfb60 0x7fa27c0aee20 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.218+0000 7fa281200640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa27c15f5b0 0x7fa27c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.218+0000 7fa281200640 1 --2- 172.21.15.50:0/883883391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c155a00 0x7fa27c141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.218+0000 7fa281200640 1 -- 172.21.15.50:0/883883391 >> 172.21.15.50:0/883883391 conn(0x7fa27c06cdf0 msgr2=0x7fa27c06ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:30.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.218+0000 7fa281200640 1 -- 172.21.15.50:0/883883391 shutdown_connections 2024-09-21T06:12:30.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.218+0000 7fa281200640 1 -- 172.21.15.50:0/883883391 wait complete. 2024-09-21T06:12:30.236 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":621,"stamp":"2024-09-21T06:12:29.229918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1074804,"kb_used_data":15048,"kb_used_omap":12,"kb_used_meta":1059571,"kb_avail":753899916,"statfs":{"total":773094113280,"available":771993513984,"internally_reserved":0,"allocated":15409152,"data_stored":7425828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1085001264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.643275"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636164+0000","last_change":"2024-09-21T06:12:25.612914+0000","last_active":"2024-09-21T06:12:26.636164+0000","last_peered":"2024-09-21T06:12:26.636164+0000","last_clean":"2024-09-21T06:12:26.636164+0000","last_became_active":"2024-09-21T06:12:25.612390+0000","last_became_peered":"2024-09-21T06:12:25.612390+0000","last_unstale":"2024-09-21T06:12:26.636164+0000","last_undegraded":"2024-09-21T06:12:26.636164+0000","last_fullsized":"2024-09-21T06:12:26.636164+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:39.356625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618345+0000","last_change":"2024-09-21T06:12:25.612619+0000","last_active":"2024-09-21T06:12:26.618345+0000","last_peered":"2024-09-21T06:12:26.618345+0000","last_clean":"2024-09-21T06:12:26.618345+0000","last_became_active":"2024-09-21T06:12:25.612328+0000","last_became_peered":"2024-09-21T06:12:25.612328+0000","last_unstale":"2024-09-21T06:12:26.618345+0000","last_undegraded":"2024-09-21T06:12:26.618345+0000","last_fullsized":"2024-09-21T06:12:26.618345+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:57.830884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623653+0000","last_change":"2024-09-21T06:12:25.622728+0000","last_active":"2024-09-21T06:12:27.623653+0000","last_peered":"2024-09-21T06:12:27.623653+0000","last_clean":"2024-09-21T06:12:27.623653+0000","last_became_active":"2024-09-21T06:12:25.622427+0000","last_became_peered":"2024-09-21T06:12:25.622427+0000","last_unstale":"2024-09-21T06:12:27.623653+0000","last_undegraded":"2024-09-21T06:12:27.623653+0000","last_fullsized":"2024-09-21T06:12:27.623653+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:42.498937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.626161+0000","last_change":"2024-09-21T06:12:25.623359+0000","last_active":"2024-09-21T06:12:27.626161+0000","last_peered":"2024-09-21T06:12:27.626161+0000","last_clean":"2024-09-21T06:12:27.626161+0000","last_became_active":"2024-09-21T06:12:25.622631+0000","last_became_peered":"2024-09-21T06:12:25.622631+0000","last_unstale":"2024-09-21T06:12:27.626161+0000","last_undegraded":"2024-09-21T06:12:27.626161+0000","last_fullsized":"2024-09-21T06:12:27.626161+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:02.869029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618787+0000","last_change":"2024-09-21T06:12:25.607908+0000","last_active":"2024-09-21T06:12:26.618787+0000","last_peered":"2024-09-21T06:12:26.618787+0000","last_clean":"2024-09-21T06:12:26.618787+0000","last_became_active":"2024-09-21T06:12:25.607545+0000","last_became_peered":"2024-09-21T06:12:25.607545+0000","last_unstale":"2024-09-21T06:12:26.618787+0000","last_undegraded":"2024-09-21T06:12:26.618787+0000","last_fullsized":"2024-09-21T06:12:26.618787+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:33.338206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.635818+0000","last_change":"2024-09-21T06:12:25.625461+0000","last_active":"2024-09-21T06:12:26.635818+0000","last_peered":"2024-09-21T06:12:26.635818+0000","last_clean":"2024-09-21T06:12:26.635818+0000","last_became_active":"2024-09-21T06:12:25.624887+0000","last_became_peered":"2024-09-21T06:12:25.624887+0000","last_unstale":"2024-09-21T06:12:26.635818+0000","last_undegraded":"2024-09-21T06:12:26.635818+0000","last_fullsized":"2024-09-21T06:12:26.635818+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:55.033896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624534+0000","last_change":"2024-09-21T06:12:25.628168+0000","last_active":"2024-09-21T06:12:27.624534+0000","last_peered":"2024-09-21T06:12:27.624534+0000","last_clean":"2024-09-21T06:12:27.624534+0000","last_became_active":"2024-09-21T06:12:25.627258+0000","last_became_peered":"2024-09-21T06:12:25.627258+0000","last_unstale":"2024-09-21T06:12:27.624534+0000","last_undegraded":"2024-09-21T06:12:27.624534+0000","last_fullsized":"2024-09-21T06:12:27.624534+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:23.228842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.635687+0000","last_change":"2024-09-21T06:12:25.609541+0000","last_active":"2024-09-21T06:12:26.635687+0000","last_peered":"2024-09-21T06:12:26.635687+0000","last_clean":"2024-09-21T06:12:26.635687+0000","last_became_active":"2024-09-21T06:12:25.609220+0000","last_became_peered":"2024-09-21T06:12:25.609220+0000","last_unstale":"2024-09-21T06:12:26.635687+0000","last_undegraded":"2024-09-21T06:12:26.635687+0000","last_fullsized":"2024-09-21T06:12:26.635687+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:45.787384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618427+0000","last_change":"2024-09-21T06:12:25.616411+0000","last_active":"2024-09-21T06:12:26.618427+0000","last_peered":"2024-09-21T06:12:26.618427+0000","last_clean":"2024-09-21T06:12:26.618427+0000","last_became_active":"2024-09-21T06:12:25.616071+0000","last_became_peered":"2024-09-21T06:12:25.616071+0000","last_unstale":"2024-09-21T06:12:26.618427+0000","last_undegraded":"2024-09-21T06:12:26.618427+0000","last_fullsized":"2024-09-21T06:12:26.618427+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:23.228842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619302+0000","last_change":"2024-09-21T06:12:25.627766+0000","last_active":"2024-09-21T06:12:26.619302+0000","last_peered":"2024-09-21T06:12:26.619302+0000","last_clean":"2024-09-21T06:12:26.619302+0000","last_became_active":"2024-09-21T06:12:25.624288+0000","last_became_peered":"2024-09-21T06:12:25.624288+0000","last_unstale":"2024-09-21T06:12:26.619302+0000","last_undegraded":"2024-09-21T06:12:26.619302+0000","last_fullsized":"2024-09-21T06:12:26.619302+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:33.938443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.635905+0000","last_change":"2024-09-21T06:12:25.609517+0000","last_active":"2024-09-21T06:12:26.635905+0000","last_peered":"2024-09-21T06:12:26.635905+0000","last_clean":"2024-09-21T06:12:26.635905+0000","last_became_active":"2024-09-21T06:12:25.609167+0000","last_became_peered":"2024-09-21T06:12:25.609167+0000","last_unstale":"2024-09-21T06:12:26.635905+0000","last_undegraded":"2024-09-21T06:12:26.635905+0000","last_fullsized":"2024-09-21T06:12:26.635905+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:35.231184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618297+0000","last_change":"2024-09-21T06:12:25.609915+0000","last_active":"2024-09-21T06:12:26.618297+0000","last_peered":"2024-09-21T06:12:26.618297+0000","last_clean":"2024-09-21T06:12:26.618297+0000","last_became_active":"2024-09-21T06:12:25.608777+0000","last_became_peered":"2024-09-21T06:12:25.608777+0000","last_unstale":"2024-09-21T06:12:26.618297+0000","last_undegraded":"2024-09-21T06:12:26.618297+0000","last_fullsized":"2024-09-21T06:12:26.618297+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:40.121562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619113+0000","last_change":"2024-09-21T06:12:25.607648+0000","last_active":"2024-09-21T06:12:26.619113+0000","last_peered":"2024-09-21T06:12:26.619113+0000","last_clean":"2024-09-21T06:12:26.619113+0000","last_became_active":"2024-09-21T06:12:25.607316+0000","last_became_peered":"2024-09-21T06:12:25.607316+0000","last_unstale":"2024-09-21T06:12:26.619113+0000","last_undegraded":"2024-09-21T06:12:26.619113+0000","last_fullsized":"2024-09-21T06:12:26.619113+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:36.543983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.625198+0000","last_change":"2024-09-21T06:12:25.625307+0000","last_active":"2024-09-21T06:12:27.625198+0000","last_peered":"2024-09-21T06:12:27.625198+0000","last_clean":"2024-09-21T06:12:27.625198+0000","last_became_active":"2024-09-21T06:12:25.624933+0000","last_became_peered":"2024-09-21T06:12:25.624933+0000","last_unstale":"2024-09-21T06:12:27.625198+0000","last_undegraded":"2024-09-21T06:12:27.625198+0000","last_fullsized":"2024-09-21T06:12:27.625198+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:32.566609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618017+0000","last_change":"2024-09-21T06:12:25.618002+0000","last_active":"2024-09-21T06:12:26.618017+0000","last_peered":"2024-09-21T06:12:26.618017+0000","last_clean":"2024-09-21T06:12:26.618017+0000","last_became_active":"2024-09-21T06:12:25.617714+0000","last_became_peered":"2024-09-21T06:12:25.617714+0000","last_unstale":"2024-09-21T06:12:26.618017+0000","last_undegraded":"2024-09-21T06:12:26.618017+0000","last_fullsized":"2024-09-21T06:12:26.618017+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:43.136576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766063+0000","last_change":"2024-09-21T06:12:25.618664+0000","last_active":"2024-09-21T06:12:26.766063+0000","last_peered":"2024-09-21T06:12:26.766063+0000","last_clean":"2024-09-21T06:12:26.766063+0000","last_became_active":"2024-09-21T06:12:25.618261+0000","last_became_peered":"2024-09-21T06:12:25.618261+0000","last_unstale":"2024-09-21T06:12:26.766063+0000","last_undegraded":"2024-09-21T06:12:26.766063+0000","last_fullsized":"2024-09-21T06:12:26.766063+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:11.561785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619988+0000","last_change":"2024-09-21T06:12:25.619849+0000","last_active":"2024-09-21T06:12:26.619988+0000","last_peered":"2024-09-21T06:12:26.619988+0000","last_clean":"2024-09-21T06:12:26.619988+0000","last_became_active":"2024-09-21T06:12:25.619518+0000","last_became_peered":"2024-09-21T06:12:25.619518+0000","last_unstale":"2024-09-21T06:12:26.619988+0000","last_undegraded":"2024-09-21T06:12:26.619988+0000","last_fullsized":"2024-09-21T06:12:26.619988+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:48.182182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623621+0000","last_change":"2024-09-21T06:12:25.621753+0000","last_active":"2024-09-21T06:12:27.623621+0000","last_peered":"2024-09-21T06:12:27.623621+0000","last_clean":"2024-09-21T06:12:27.623621+0000","last_became_active":"2024-09-21T06:12:25.621433+0000","last_became_peered":"2024-09-21T06:12:25.621433+0000","last_unstale":"2024-09-21T06:12:27.623621+0000","last_undegraded":"2024-09-21T06:12:27.623621+0000","last_fullsized":"2024-09-21T06:12:27.623621+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:13.478353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636313+0000","last_change":"2024-09-21T06:12:25.613752+0000","last_active":"2024-09-21T06:12:26.636313+0000","last_peered":"2024-09-21T06:12:26.636313+0000","last_clean":"2024-09-21T06:12:26.636313+0000","last_became_active":"2024-09-21T06:12:25.613397+0000","last_became_peered":"2024-09-21T06:12:25.613397+0000","last_unstale":"2024-09-21T06:12:26.636313+0000","last_undegraded":"2024-09-21T06:12:26.636313+0000","last_fullsized":"2024-09-21T06:12:26.636313+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:44.609173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766479+0000","last_change":"2024-09-21T06:12:25.620384+0000","last_active":"2024-09-21T06:12:26.766479+0000","last_peered":"2024-09-21T06:12:26.766479+0000","last_clean":"2024-09-21T06:12:26.766479+0000","last_became_active":"2024-09-21T06:12:25.620034+0000","last_became_peered":"2024-09-21T06:12:25.620034+0000","last_unstale":"2024-09-21T06:12:26.766479+0000","last_undegraded":"2024-09-21T06:12:26.766479+0000","last_fullsized":"2024-09-21T06:12:26.766479+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:12.224272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619883+0000","last_change":"2024-09-21T06:12:25.627780+0000","last_active":"2024-09-21T06:12:26.619883+0000","last_peered":"2024-09-21T06:12:26.619883+0000","last_clean":"2024-09-21T06:12:26.619883+0000","last_became_active":"2024-09-21T06:12:25.627431+0000","last_became_peered":"2024-09-21T06:12:25.627431+0000","last_unstale":"2024-09-21T06:12:26.619883+0000","last_undegraded":"2024-09-21T06:12:26.619883+0000","last_fullsized":"2024-09-21T06:12:26.619883+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:57.631475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.625272+0000","last_change":"2024-09-21T06:12:25.622780+0000","last_active":"2024-09-21T06:12:27.625272+0000","last_peered":"2024-09-21T06:12:27.625272+0000","last_clean":"2024-09-21T06:12:27.625272+0000","last_became_active":"2024-09-21T06:12:25.622435+0000","last_became_peered":"2024-09-21T06:12:25.622435+0000","last_unstale":"2024-09-21T06:12:27.625272+0000","last_undegraded":"2024-09-21T06:12:27.625272+0000","last_fullsized":"2024-09-21T06:12:27.625272+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:57.326608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.625001+0000","last_change":"2024-09-21T06:12:25.624056+0000","last_active":"2024-09-21T06:12:27.625001+0000","last_peered":"2024-09-21T06:12:27.625001+0000","last_clean":"2024-09-21T06:12:27.625001+0000","last_became_active":"2024-09-21T06:12:25.623732+0000","last_became_peered":"2024-09-21T06:12:25.623732+0000","last_unstale":"2024-09-21T06:12:27.625001+0000","last_undegraded":"2024-09-21T06:12:27.625001+0000","last_fullsized":"2024-09-21T06:12:27.625001+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:39.068960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618057+0000","last_change":"2024-09-21T06:12:25.618510+0000","last_active":"2024-09-21T06:12:26.618057+0000","last_peered":"2024-09-21T06:12:26.618057+0000","last_clean":"2024-09-21T06:12:26.618057+0000","last_became_active":"2024-09-21T06:12:25.618160+0000","last_became_peered":"2024-09-21T06:12:25.618160+0000","last_unstale":"2024-09-21T06:12:26.618057+0000","last_undegraded":"2024-09-21T06:12:26.618057+0000","last_fullsized":"2024-09-21T06:12:26.618057+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:01.381247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766495+0000","last_change":"2024-09-21T06:12:25.609675+0000","last_active":"2024-09-21T06:12:26.766495+0000","last_peered":"2024-09-21T06:12:26.766495+0000","last_clean":"2024-09-21T06:12:26.766495+0000","last_became_active":"2024-09-21T06:12:25.609344+0000","last_became_peered":"2024-09-21T06:12:25.609344+0000","last_unstale":"2024-09-21T06:12:26.766495+0000","last_undegraded":"2024-09-21T06:12:26.766495+0000","last_fullsized":"2024-09-21T06:12:26.766495+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:29.672395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636385+0000","last_change":"2024-09-21T06:12:25.612562+0000","last_active":"2024-09-21T06:12:26.636385+0000","last_peered":"2024-09-21T06:12:26.636385+0000","last_clean":"2024-09-21T06:12:26.636385+0000","last_became_active":"2024-09-21T06:12:25.612225+0000","last_became_peered":"2024-09-21T06:12:25.612225+0000","last_unstale":"2024-09-21T06:12:26.636385+0000","last_undegraded":"2024-09-21T06:12:26.636385+0000","last_fullsized":"2024-09-21T06:12:26.636385+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:57.575899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766461+0000","last_change":"2024-09-21T06:12:25.630793+0000","last_active":"2024-09-21T06:12:26.766461+0000","last_peered":"2024-09-21T06:12:26.766461+0000","last_clean":"2024-09-21T06:12:26.766461+0000","last_became_active":"2024-09-21T06:12:25.630425+0000","last_became_peered":"2024-09-21T06:12:25.630425+0000","last_unstale":"2024-09-21T06:12:26.766461+0000","last_undegraded":"2024-09-21T06:12:26.766461+0000","last_fullsized":"2024-09-21T06:12:26.766461+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:18.861862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624866+0000","last_change":"2024-09-21T06:12:25.624809+0000","last_active":"2024-09-21T06:12:27.624866+0000","last_peered":"2024-09-21T06:12:27.624866+0000","last_clean":"2024-09-21T06:12:27.624866+0000","last_became_active":"2024-09-21T06:12:25.624433+0000","last_became_peered":"2024-09-21T06:12:25.624433+0000","last_unstale":"2024-09-21T06:12:27.624866+0000","last_undegraded":"2024-09-21T06:12:27.624866+0000","last_fullsized":"2024-09-21T06:12:27.624866+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:04.880942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.635775+0000","last_change":"2024-09-21T06:12:25.624642+0000","last_active":"2024-09-21T06:12:26.635775+0000","last_peered":"2024-09-21T06:12:26.635775+0000","last_clean":"2024-09-21T06:12:26.635775+0000","last_became_active":"2024-09-21T06:12:25.624308+0000","last_became_peered":"2024-09-21T06:12:25.624308+0000","last_unstale":"2024-09-21T06:12:26.635775+0000","last_undegraded":"2024-09-21T06:12:26.635775+0000","last_fullsized":"2024-09-21T06:12:26.635775+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:59.067650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.625488+0000","last_change":"2024-09-21T06:12:25.620357+0000","last_active":"2024-09-21T06:12:27.625488+0000","last_peered":"2024-09-21T06:12:27.625488+0000","last_clean":"2024-09-21T06:12:27.625488+0000","last_became_active":"2024-09-21T06:12:25.619973+0000","last_became_peered":"2024-09-21T06:12:25.619973+0000","last_unstale":"2024-09-21T06:12:27.625488+0000","last_undegraded":"2024-09-21T06:12:27.625488+0000","last_fullsized":"2024-09-21T06:12:27.625488+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:17.488083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617758+0000","last_change":"2024-09-21T06:12:25.624479+0000","last_active":"2024-09-21T06:12:26.617758+0000","last_peered":"2024-09-21T06:12:26.617758+0000","last_clean":"2024-09-21T06:12:26.617758+0000","last_became_active":"2024-09-21T06:12:25.624142+0000","last_became_peered":"2024-09-21T06:12:25.624142+0000","last_unstale":"2024-09-21T06:12:26.617758+0000","last_undegraded":"2024-09-21T06:12:26.617758+0000","last_fullsized":"2024-09-21T06:12:26.617758+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:48.699592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766305+0000","last_change":"2024-09-21T06:12:25.618988+0000","last_active":"2024-09-21T06:12:26.766305+0000","last_peered":"2024-09-21T06:12:26.766305+0000","last_clean":"2024-09-21T06:12:26.766305+0000","last_became_active":"2024-09-21T06:12:25.617536+0000","last_became_peered":"2024-09-21T06:12:25.617536+0000","last_unstale":"2024-09-21T06:12:26.766305+0000","last_undegraded":"2024-09-21T06:12:26.766305+0000","last_fullsized":"2024-09-21T06:12:26.766305+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:27.294981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618720+0000","last_change":"2024-09-21T06:12:25.615006+0000","last_active":"2024-09-21T06:12:26.618720+0000","last_peered":"2024-09-21T06:12:26.618720+0000","last_clean":"2024-09-21T06:12:26.618720+0000","last_became_active":"2024-09-21T06:12:25.611514+0000","last_became_peered":"2024-09-21T06:12:25.611514+0000","last_unstale":"2024-09-21T06:12:26.618720+0000","last_undegraded":"2024-09-21T06:12:26.618720+0000","last_fullsized":"2024-09-21T06:12:26.618720+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:05.454167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"141'4","reported_seq":19,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619730+0000","last_change":"2024-09-21T06:12:24.620583+0000","last_active":"2024-09-21T06:12:26.619730+0000","last_peered":"2024-09-21T06:12:26.619730+0000","last_clean":"2024-09-21T06:12:26.619730+0000","last_became_active":"2024-09-21T06:12:24.620281+0000","last_became_peered":"2024-09-21T06:12:24.620281+0000","last_unstale":"2024-09-21T06:12:26.619730+0000","last_undegraded":"2024-09-21T06:12:26.619730+0000","last_fullsized":"2024-09-21T06:12:26.619730+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:47.691713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618425+0000","last_change":"2024-09-21T06:12:25.612169+0000","last_active":"2024-09-21T06:12:26.618425+0000","last_peered":"2024-09-21T06:12:26.618425+0000","last_clean":"2024-09-21T06:12:26.618425+0000","last_became_active":"2024-09-21T06:12:25.611836+0000","last_became_peered":"2024-09-21T06:12:25.611836+0000","last_unstale":"2024-09-21T06:12:26.618425+0000","last_undegraded":"2024-09-21T06:12:26.618425+0000","last_fullsized":"2024-09-21T06:12:26.618425+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:50.732103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618272+0000","last_change":"2024-09-21T06:12:24.614480+0000","last_active":"2024-09-21T06:12:26.618272+0000","last_peered":"2024-09-21T06:12:26.618272+0000","last_clean":"2024-09-21T06:12:26.618272+0000","last_became_active":"2024-09-21T06:12:24.614159+0000","last_became_peered":"2024-09-21T06:12:24.614159+0000","last_unstale":"2024-09-21T06:12:26.618272+0000","last_undegraded":"2024-09-21T06:12:26.618272+0000","last_fullsized":"2024-09-21T06:12:26.618272+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:56.318730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.635935+0000","last_change":"2024-09-21T06:12:25.626929+0000","last_active":"2024-09-21T06:12:26.635935+0000","last_peered":"2024-09-21T06:12:26.635935+0000","last_clean":"2024-09-21T06:12:26.635935+0000","last_became_active":"2024-09-21T06:12:25.626605+0000","last_became_peered":"2024-09-21T06:12:25.626605+0000","last_unstale":"2024-09-21T06:12:26.635935+0000","last_undegraded":"2024-09-21T06:12:26.635935+0000","last_fullsized":"2024-09-21T06:12:26.635935+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:13.613982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.626208+0000","last_change":"2024-09-21T06:12:24.620901+0000","last_active":"2024-09-21T06:12:27.626208+0000","last_peered":"2024-09-21T06:12:27.626208+0000","last_clean":"2024-09-21T06:12:27.626208+0000","last_became_active":"2024-09-21T06:12:24.618875+0000","last_became_peered":"2024-09-21T06:12:24.618875+0000","last_unstale":"2024-09-21T06:12:27.626208+0000","last_undegraded":"2024-09-21T06:12:27.626208+0000","last_fullsized":"2024-09-21T06:12:27.626208+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:57.938112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618753+0000","last_change":"2024-09-21T06:12:25.616799+0000","last_active":"2024-09-21T06:12:26.618753+0000","last_peered":"2024-09-21T06:12:26.618753+0000","last_clean":"2024-09-21T06:12:26.618753+0000","last_became_active":"2024-09-21T06:12:25.616490+0000","last_became_peered":"2024-09-21T06:12:25.616490+0000","last_unstale":"2024-09-21T06:12:26.618753+0000","last_undegraded":"2024-09-21T06:12:26.618753+0000","last_fullsized":"2024-09-21T06:12:26.618753+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:44:43.974116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.626160+0000","last_change":"2024-09-21T06:12:24.620195+0000","last_active":"2024-09-21T06:12:27.626160+0000","last_peered":"2024-09-21T06:12:27.626160+0000","last_clean":"2024-09-21T06:12:27.626160+0000","last_became_active":"2024-09-21T06:12:24.619886+0000","last_became_peered":"2024-09-21T06:12:24.619886+0000","last_unstale":"2024-09-21T06:12:27.626160+0000","last_undegraded":"2024-09-21T06:12:27.626160+0000","last_fullsized":"2024-09-21T06:12:27.626160+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:46.739582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617866+0000","last_change":"2024-09-21T06:12:25.626295+0000","last_active":"2024-09-21T06:12:26.617866+0000","last_peered":"2024-09-21T06:12:26.617866+0000","last_clean":"2024-09-21T06:12:26.617866+0000","last_became_active":"2024-09-21T06:12:25.625938+0000","last_became_peered":"2024-09-21T06:12:25.625938+0000","last_unstale":"2024-09-21T06:12:26.617866+0000","last_undegraded":"2024-09-21T06:12:26.617866+0000","last_fullsized":"2024-09-21T06:12:26.617866+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:22.459130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766130+0000","last_change":"2024-09-21T06:12:24.619038+0000","last_active":"2024-09-21T06:12:26.766130+0000","last_peered":"2024-09-21T06:12:26.766130+0000","last_clean":"2024-09-21T06:12:26.766130+0000","last_became_active":"2024-09-21T06:12:24.618048+0000","last_became_peered":"2024-09-21T06:12:24.618048+0000","last_unstale":"2024-09-21T06:12:26.766130+0000","last_undegraded":"2024-09-21T06:12:26.766130+0000","last_fullsized":"2024-09-21T06:12:26.766130+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:06.581382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.625348+0000","last_change":"2024-09-21T06:12:25.632384+0000","last_active":"2024-09-21T06:12:27.625348+0000","last_peered":"2024-09-21T06:12:27.625348+0000","last_clean":"2024-09-21T06:12:27.625348+0000","last_became_active":"2024-09-21T06:12:25.632015+0000","last_became_peered":"2024-09-21T06:12:25.632015+0000","last_unstale":"2024-09-21T06:12:27.625348+0000","last_undegraded":"2024-09-21T06:12:27.625348+0000","last_fullsized":"2024-09-21T06:12:27.625348+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:56.559615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618399+0000","last_change":"2024-09-21T06:12:24.617857+0000","last_active":"2024-09-21T06:12:26.618399+0000","last_peered":"2024-09-21T06:12:26.618399+0000","last_clean":"2024-09-21T06:12:26.618399+0000","last_became_active":"2024-09-21T06:12:24.617506+0000","last_became_peered":"2024-09-21T06:12:24.617506+0000","last_unstale":"2024-09-21T06:12:26.618399+0000","last_undegraded":"2024-09-21T06:12:26.618399+0000","last_fullsized":"2024-09-21T06:12:26.618399+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:47.276272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617784+0000","last_change":"2024-09-21T06:12:25.608575+0000","last_active":"2024-09-21T06:12:26.617784+0000","last_peered":"2024-09-21T06:12:26.617784+0000","last_clean":"2024-09-21T06:12:26.617784+0000","last_became_active":"2024-09-21T06:12:25.608249+0000","last_became_peered":"2024-09-21T06:12:25.608249+0000","last_unstale":"2024-09-21T06:12:26.617784+0000","last_undegraded":"2024-09-21T06:12:26.617784+0000","last_fullsized":"2024-09-21T06:12:26.617784+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:19.272104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766105+0000","last_change":"2024-09-21T06:12:24.614660+0000","last_active":"2024-09-21T06:12:26.766105+0000","last_peered":"2024-09-21T06:12:26.766105+0000","last_clean":"2024-09-21T06:12:26.766105+0000","last_became_active":"2024-09-21T06:12:24.614326+0000","last_became_peered":"2024-09-21T06:12:24.614326+0000","last_unstale":"2024-09-21T06:12:26.766105+0000","last_undegraded":"2024-09-21T06:12:26.766105+0000","last_fullsized":"2024-09-21T06:12:26.766105+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:02.191560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617746+0000","last_change":"2024-09-21T06:12:25.609433+0000","last_active":"2024-09-21T06:12:26.617746+0000","last_peered":"2024-09-21T06:12:26.617746+0000","last_clean":"2024-09-21T06:12:26.617746+0000","last_became_active":"2024-09-21T06:12:25.608036+0000","last_became_peered":"2024-09-21T06:12:25.608036+0000","last_unstale":"2024-09-21T06:12:26.617746+0000","last_undegraded":"2024-09-21T06:12:26.617746+0000","last_fullsized":"2024-09-21T06:12:26.617746+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:58.284774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624874+0000","last_change":"2024-09-21T06:12:24.617782+0000","last_active":"2024-09-21T06:12:27.624874+0000","last_peered":"2024-09-21T06:12:27.624874+0000","last_clean":"2024-09-21T06:12:27.624874+0000","last_became_active":"2024-09-21T06:12:24.617445+0000","last_became_peered":"2024-09-21T06:12:24.617445+0000","last_unstale":"2024-09-21T06:12:27.624874+0000","last_undegraded":"2024-09-21T06:12:27.624874+0000","last_fullsized":"2024-09-21T06:12:27.624874+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:44.394243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619669+0000","last_change":"2024-09-21T06:12:25.618989+0000","last_active":"2024-09-21T06:12:26.619669+0000","last_peered":"2024-09-21T06:12:26.619669+0000","last_clean":"2024-09-21T06:12:26.619669+0000","last_became_active":"2024-09-21T06:12:25.614126+0000","last_became_peered":"2024-09-21T06:12:25.614126+0000","last_unstale":"2024-09-21T06:12:26.619669+0000","last_undegraded":"2024-09-21T06:12:26.619669+0000","last_fullsized":"2024-09-21T06:12:26.619669+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:52.286333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618530+0000","last_change":"2024-09-21T06:12:24.609257+0000","last_active":"2024-09-21T06:12:26.618530+0000","last_peered":"2024-09-21T06:12:26.618530+0000","last_clean":"2024-09-21T06:12:26.618530+0000","last_became_active":"2024-09-21T06:12:24.608867+0000","last_became_peered":"2024-09-21T06:12:24.608867+0000","last_unstale":"2024-09-21T06:12:26.618530+0000","last_undegraded":"2024-09-21T06:12:26.618530+0000","last_fullsized":"2024-09-21T06:12:26.618530+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:40.538811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619705+0000","last_change":"2024-09-21T06:12:25.618709+0000","last_active":"2024-09-21T06:12:26.619705+0000","last_peered":"2024-09-21T06:12:26.619705+0000","last_clean":"2024-09-21T06:12:26.619705+0000","last_became_active":"2024-09-21T06:12:25.618305+0000","last_became_peered":"2024-09-21T06:12:25.618305+0000","last_unstale":"2024-09-21T06:12:26.619705+0000","last_undegraded":"2024-09-21T06:12:26.619705+0000","last_fullsized":"2024-09-21T06:12:26.619705+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:35.783206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619160+0000","last_change":"2024-09-21T06:12:24.621430+0000","last_active":"2024-09-21T06:12:26.619160+0000","last_peered":"2024-09-21T06:12:26.619160+0000","last_clean":"2024-09-21T06:12:26.619160+0000","last_became_active":"2024-09-21T06:12:24.621083+0000","last_became_peered":"2024-09-21T06:12:24.621083+0000","last_unstale":"2024-09-21T06:12:26.619160+0000","last_undegraded":"2024-09-21T06:12:26.619160+0000","last_fullsized":"2024-09-21T06:12:26.619160+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:44.115082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766492+0000","last_change":"2024-09-21T06:12:25.613964+0000","last_active":"2024-09-21T06:12:26.766492+0000","last_peered":"2024-09-21T06:12:26.766492+0000","last_clean":"2024-09-21T06:12:26.766492+0000","last_became_active":"2024-09-21T06:12:25.613644+0000","last_became_peered":"2024-09-21T06:12:25.613644+0000","last_unstale":"2024-09-21T06:12:26.766492+0000","last_undegraded":"2024-09-21T06:12:26.766492+0000","last_fullsized":"2024-09-21T06:12:26.766492+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:59.227095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"141'6","reported_seq":21,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624793+0000","last_change":"2024-09-21T06:12:24.612261+0000","last_active":"2024-09-21T06:12:27.624793+0000","last_peered":"2024-09-21T06:12:27.624793+0000","last_clean":"2024-09-21T06:12:27.624793+0000","last_became_active":"2024-09-21T06:12:24.611940+0000","last_became_peered":"2024-09-21T06:12:24.611940+0000","last_unstale":"2024-09-21T06:12:27.624793+0000","last_undegraded":"2024-09-21T06:12:27.624793+0000","last_fullsized":"2024-09-21T06:12:27.624793+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:27.350833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623258+0000","last_change":"2024-09-21T06:12:25.632907+0000","last_active":"2024-09-21T06:12:27.623258+0000","last_peered":"2024-09-21T06:12:27.623258+0000","last_clean":"2024-09-21T06:12:27.623258+0000","last_became_active":"2024-09-21T06:12:25.632521+0000","last_became_peered":"2024-09-21T06:12:25.632521+0000","last_unstale":"2024-09-21T06:12:27.623258+0000","last_undegraded":"2024-09-21T06:12:27.623258+0000","last_fullsized":"2024-09-21T06:12:27.623258+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:18.527754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623331+0000","last_change":"2024-09-21T06:12:24.612768+0000","last_active":"2024-09-21T06:12:27.623331+0000","last_peered":"2024-09-21T06:12:27.623331+0000","last_clean":"2024-09-21T06:12:27.623331+0000","last_became_active":"2024-09-21T06:12:24.612200+0000","last_became_peered":"2024-09-21T06:12:24.612200+0000","last_unstale":"2024-09-21T06:12:27.623331+0000","last_undegraded":"2024-09-21T06:12:27.623331+0000","last_fullsized":"2024-09-21T06:12:27.623331+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:12.508843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617834+0000","last_change":"2024-09-21T06:12:25.615733+0000","last_active":"2024-09-21T06:12:26.617834+0000","last_peered":"2024-09-21T06:12:26.617834+0000","last_clean":"2024-09-21T06:12:26.617834+0000","last_became_active":"2024-09-21T06:12:25.615412+0000","last_became_peered":"2024-09-21T06:12:25.615412+0000","last_unstale":"2024-09-21T06:12:26.617834+0000","last_undegraded":"2024-09-21T06:12:26.617834+0000","last_fullsized":"2024-09-21T06:12:26.617834+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:49.430682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623339+0000","last_change":"2024-09-21T06:12:24.616333+0000","last_active":"2024-09-21T06:12:27.623339+0000","last_peered":"2024-09-21T06:12:27.623339+0000","last_clean":"2024-09-21T06:12:27.623339+0000","last_became_active":"2024-09-21T06:12:24.616024+0000","last_became_peered":"2024-09-21T06:12:24.616024+0000","last_unstale":"2024-09-21T06:12:27.623339+0000","last_undegraded":"2024-09-21T06:12:27.623339+0000","last_fullsized":"2024-09-21T06:12:27.623339+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:16.059748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617753+0000","last_change":"2024-09-21T06:12:25.609016+0000","last_active":"2024-09-21T06:12:26.617753+0000","last_peered":"2024-09-21T06:12:26.617753+0000","last_clean":"2024-09-21T06:12:26.617753+0000","last_became_active":"2024-09-21T06:12:25.608638+0000","last_became_peered":"2024-09-21T06:12:25.608638+0000","last_unstale":"2024-09-21T06:12:26.617753+0000","last_undegraded":"2024-09-21T06:12:26.617753+0000","last_fullsized":"2024-09-21T06:12:26.617753+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:33.409594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.635586+0000","last_change":"2024-09-21T06:12:24.609267+0000","last_active":"2024-09-21T06:12:26.635586+0000","last_peered":"2024-09-21T06:12:26.635586+0000","last_clean":"2024-09-21T06:12:26.635586+0000","last_became_active":"2024-09-21T06:12:24.608943+0000","last_became_peered":"2024-09-21T06:12:24.608943+0000","last_unstale":"2024-09-21T06:12:26.635586+0000","last_undegraded":"2024-09-21T06:12:26.635586+0000","last_fullsized":"2024-09-21T06:12:26.635586+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:33.783011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623209+0000","last_change":"2024-09-21T06:12:25.623446+0000","last_active":"2024-09-21T06:12:27.623209+0000","last_peered":"2024-09-21T06:12:27.623209+0000","last_clean":"2024-09-21T06:12:27.623209+0000","last_became_active":"2024-09-21T06:12:25.623154+0000","last_became_peered":"2024-09-21T06:12:25.623154+0000","last_unstale":"2024-09-21T06:12:27.623209+0000","last_undegraded":"2024-09-21T06:12:27.623209+0000","last_fullsized":"2024-09-21T06:12:27.623209+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:55.131765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"141'8","reported_seq":23,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766233+0000","last_change":"2024-09-21T06:12:24.608576+0000","last_active":"2024-09-21T06:12:26.766233+0000","last_peered":"2024-09-21T06:12:26.766233+0000","last_clean":"2024-09-21T06:12:26.766233+0000","last_became_active":"2024-09-21T06:12:24.608251+0000","last_became_peered":"2024-09-21T06:12:24.608251+0000","last_unstale":"2024-09-21T06:12:26.766233+0000","last_undegraded":"2024-09-21T06:12:26.766233+0000","last_fullsized":"2024-09-21T06:12:26.766233+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:44.810992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618956+0000","last_change":"2024-09-21T06:12:25.616851+0000","last_active":"2024-09-21T06:12:26.618956+0000","last_peered":"2024-09-21T06:12:26.618956+0000","last_clean":"2024-09-21T06:12:26.618956+0000","last_became_active":"2024-09-21T06:12:25.616357+0000","last_became_peered":"2024-09-21T06:12:25.616357+0000","last_unstale":"2024-09-21T06:12:26.618956+0000","last_undegraded":"2024-09-21T06:12:26.618956+0000","last_fullsized":"2024-09-21T06:12:26.618956+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:58.242236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619772+0000","last_change":"2024-09-21T06:12:24.607165+0000","last_active":"2024-09-21T06:12:26.619772+0000","last_peered":"2024-09-21T06:12:26.619772+0000","last_clean":"2024-09-21T06:12:26.619772+0000","last_became_active":"2024-09-21T06:12:24.606847+0000","last_became_peered":"2024-09-21T06:12:24.606847+0000","last_unstale":"2024-09-21T06:12:26.619772+0000","last_undegraded":"2024-09-21T06:12:26.619772+0000","last_fullsized":"2024-09-21T06:12:26.619772+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:05.896886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766622+0000","last_change":"2024-09-21T06:12:25.611430+0000","last_active":"2024-09-21T06:12:26.766622+0000","last_peered":"2024-09-21T06:12:26.766622+0000","last_clean":"2024-09-21T06:12:26.766622+0000","last_became_active":"2024-09-21T06:12:25.611097+0000","last_became_peered":"2024-09-21T06:12:25.611097+0000","last_unstale":"2024-09-21T06:12:26.766622+0000","last_undegraded":"2024-09-21T06:12:26.766622+0000","last_fullsized":"2024-09-21T06:12:26.766622+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:01.724361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618395+0000","last_change":"2024-09-21T06:12:24.614422+0000","last_active":"2024-09-21T06:12:26.618395+0000","last_peered":"2024-09-21T06:12:26.618395+0000","last_clean":"2024-09-21T06:12:26.618395+0000","last_became_active":"2024-09-21T06:12:24.614131+0000","last_became_peered":"2024-09-21T06:12:24.614131+0000","last_unstale":"2024-09-21T06:12:26.618395+0000","last_undegraded":"2024-09-21T06:12:26.618395+0000","last_fullsized":"2024-09-21T06:12:26.618395+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:55.551737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623591+0000","last_change":"2024-09-21T06:12:25.629107+0000","last_active":"2024-09-21T06:12:27.623591+0000","last_peered":"2024-09-21T06:12:27.623591+0000","last_clean":"2024-09-21T06:12:27.623591+0000","last_became_active":"2024-09-21T06:12:25.625307+0000","last_became_peered":"2024-09-21T06:12:25.625307+0000","last_unstale":"2024-09-21T06:12:27.623591+0000","last_undegraded":"2024-09-21T06:12:27.623591+0000","last_fullsized":"2024-09-21T06:12:27.623591+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:40.434350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766582+0000","last_change":"2024-09-21T06:12:24.618757+0000","last_active":"2024-09-21T06:12:26.766582+0000","last_peered":"2024-09-21T06:12:26.766582+0000","last_clean":"2024-09-21T06:12:26.766582+0000","last_became_active":"2024-09-21T06:12:24.617250+0000","last_became_peered":"2024-09-21T06:12:24.617250+0000","last_unstale":"2024-09-21T06:12:26.766582+0000","last_undegraded":"2024-09-21T06:12:26.766582+0000","last_fullsized":"2024-09-21T06:12:26.766582+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:27.371208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636066+0000","last_change":"2024-09-21T06:12:25.631701+0000","last_active":"2024-09-21T06:12:26.636066+0000","last_peered":"2024-09-21T06:12:26.636066+0000","last_clean":"2024-09-21T06:12:26.636066+0000","last_became_active":"2024-09-21T06:12:25.631223+0000","last_became_peered":"2024-09-21T06:12:25.631223+0000","last_unstale":"2024-09-21T06:12:26.636066+0000","last_undegraded":"2024-09-21T06:12:26.636066+0000","last_fullsized":"2024-09-21T06:12:26.636066+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:18.942133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618013+0000","last_change":"2024-09-21T06:12:24.616936+0000","last_active":"2024-09-21T06:12:26.618013+0000","last_peered":"2024-09-21T06:12:26.618013+0000","last_clean":"2024-09-21T06:12:26.618013+0000","last_became_active":"2024-09-21T06:12:24.616604+0000","last_became_peered":"2024-09-21T06:12:24.616604+0000","last_unstale":"2024-09-21T06:12:26.618013+0000","last_undegraded":"2024-09-21T06:12:26.618013+0000","last_fullsized":"2024-09-21T06:12:26.618013+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:16.480205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.625566+0000","last_change":"2024-09-21T06:12:25.621872+0000","last_active":"2024-09-21T06:12:27.625566+0000","last_peered":"2024-09-21T06:12:27.625566+0000","last_clean":"2024-09-21T06:12:27.625566+0000","last_became_active":"2024-09-21T06:12:25.621343+0000","last_became_peered":"2024-09-21T06:12:25.621343+0000","last_unstale":"2024-09-21T06:12:27.625566+0000","last_undegraded":"2024-09-21T06:12:27.625566+0000","last_fullsized":"2024-09-21T06:12:27.625566+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:41.546689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624627+0000","last_change":"2024-09-21T06:12:24.612100+0000","last_active":"2024-09-21T06:12:27.624627+0000","last_peered":"2024-09-21T06:12:27.624627+0000","last_clean":"2024-09-21T06:12:27.624627+0000","last_became_active":"2024-09-21T06:12:24.611693+0000","last_became_peered":"2024-09-21T06:12:24.611693+0000","last_unstale":"2024-09-21T06:12:27.624627+0000","last_undegraded":"2024-09-21T06:12:27.624627+0000","last_fullsized":"2024-09-21T06:12:27.624627+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:44.115082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624584+0000","last_change":"2024-09-21T06:12:25.627383+0000","last_active":"2024-09-21T06:12:27.624584+0000","last_peered":"2024-09-21T06:12:27.624584+0000","last_clean":"2024-09-21T06:12:27.624584+0000","last_became_active":"2024-09-21T06:12:25.627003+0000","last_became_peered":"2024-09-21T06:12:25.627003+0000","last_unstale":"2024-09-21T06:12:27.624584+0000","last_undegraded":"2024-09-21T06:12:27.624584+0000","last_fullsized":"2024-09-21T06:12:27.624584+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:07.547988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618975+0000","last_change":"2024-09-21T06:12:24.619226+0000","last_active":"2024-09-21T06:12:26.618975+0000","last_peered":"2024-09-21T06:12:26.618975+0000","last_clean":"2024-09-21T06:12:26.618975+0000","last_became_active":"2024-09-21T06:12:24.618807+0000","last_became_peered":"2024-09-21T06:12:24.618807+0000","last_unstale":"2024-09-21T06:12:26.618975+0000","last_undegraded":"2024-09-21T06:12:26.618975+0000","last_fullsized":"2024-09-21T06:12:26.618975+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:11.843425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624957+0000","last_change":"2024-09-21T06:12:25.614940+0000","last_active":"2024-09-21T06:12:27.624957+0000","last_peered":"2024-09-21T06:12:27.624957+0000","last_clean":"2024-09-21T06:12:27.624957+0000","last_became_active":"2024-09-21T06:12:25.614554+0000","last_became_peered":"2024-09-21T06:12:25.614554+0000","last_unstale":"2024-09-21T06:12:27.624957+0000","last_undegraded":"2024-09-21T06:12:27.624957+0000","last_fullsized":"2024-09-21T06:12:27.624957+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:40.434350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619300+0000","last_change":"2024-09-21T06:12:24.621062+0000","last_active":"2024-09-21T06:12:26.619300+0000","last_peered":"2024-09-21T06:12:26.619300+0000","last_clean":"2024-09-21T06:12:26.619300+0000","last_became_active":"2024-09-21T06:12:24.620084+0000","last_became_peered":"2024-09-21T06:12:24.620084+0000","last_unstale":"2024-09-21T06:12:26.619300+0000","last_undegraded":"2024-09-21T06:12:26.619300+0000","last_fullsized":"2024-09-21T06:12:26.619300+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:27.350833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624543+0000","last_change":"2024-09-21T06:12:25.627242+0000","last_active":"2024-09-21T06:12:27.624543+0000","last_peered":"2024-09-21T06:12:27.624543+0000","last_clean":"2024-09-21T06:12:27.624543+0000","last_became_active":"2024-09-21T06:12:25.626624+0000","last_became_peered":"2024-09-21T06:12:25.626624+0000","last_unstale":"2024-09-21T06:12:27.624543+0000","last_undegraded":"2024-09-21T06:12:27.624543+0000","last_fullsized":"2024-09-21T06:12:27.624543+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:29.265803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.622907+0000","last_change":"2024-09-21T06:12:24.612040+0000","last_active":"2024-09-21T06:12:27.622907+0000","last_peered":"2024-09-21T06:12:27.622907+0000","last_clean":"2024-09-21T06:12:27.622907+0000","last_became_active":"2024-09-21T06:12:24.611303+0000","last_became_peered":"2024-09-21T06:12:24.611303+0000","last_unstale":"2024-09-21T06:12:27.622907+0000","last_undegraded":"2024-09-21T06:12:27.622907+0000","last_fullsized":"2024-09-21T06:12:27.622907+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:03.617075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619690+0000","last_change":"2024-09-21T06:12:25.613627+0000","last_active":"2024-09-21T06:12:26.619690+0000","last_peered":"2024-09-21T06:12:26.619690+0000","last_clean":"2024-09-21T06:12:26.619690+0000","last_became_active":"2024-09-21T06:12:25.611897+0000","last_became_peered":"2024-09-21T06:12:25.611897+0000","last_unstale":"2024-09-21T06:12:26.619690+0000","last_undegraded":"2024-09-21T06:12:26.619690+0000","last_fullsized":"2024-09-21T06:12:26.619690+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:59.227095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619884+0000","last_change":"2024-09-21T06:12:24.621410+0000","last_active":"2024-09-21T06:12:26.619884+0000","last_peered":"2024-09-21T06:12:26.619884+0000","last_clean":"2024-09-21T06:12:26.619884+0000","last_became_active":"2024-09-21T06:12:24.620575+0000","last_became_peered":"2024-09-21T06:12:24.620575+0000","last_unstale":"2024-09-21T06:12:26.619884+0000","last_undegraded":"2024-09-21T06:12:26.619884+0000","last_fullsized":"2024-09-21T06:12:26.619884+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:18.264225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623693+0000","last_change":"2024-09-21T06:12:25.632519+0000","last_active":"2024-09-21T06:12:27.623693+0000","last_peered":"2024-09-21T06:12:27.623693+0000","last_clean":"2024-09-21T06:12:27.623693+0000","last_became_active":"2024-09-21T06:12:25.632081+0000","last_became_peered":"2024-09-21T06:12:25.632081+0000","last_unstale":"2024-09-21T06:12:27.623693+0000","last_undegraded":"2024-09-21T06:12:27.623693+0000","last_fullsized":"2024-09-21T06:12:27.623693+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:55.337729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"142'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.638091+0000","last_change":"2024-09-21T06:12:24.619812+0000","last_active":"2024-09-21T06:12:26.638091+0000","last_peered":"2024-09-21T06:12:26.638091+0000","last_clean":"2024-09-21T06:12:26.638091+0000","last_became_active":"2024-09-21T06:12:24.619450+0000","last_became_peered":"2024-09-21T06:12:24.619450+0000","last_unstale":"2024-09-21T06:12:26.638091+0000","last_undegraded":"2024-09-21T06:12:26.638091+0000","last_fullsized":"2024-09-21T06:12:26.638091+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:39.650262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618099+0000","last_change":"2024-09-21T06:12:25.628533+0000","last_active":"2024-09-21T06:12:26.618099+0000","last_peered":"2024-09-21T06:12:26.618099+0000","last_clean":"2024-09-21T06:12:26.618099+0000","last_became_active":"2024-09-21T06:12:25.627938+0000","last_became_peered":"2024-09-21T06:12:25.627938+0000","last_unstale":"2024-09-21T06:12:26.618099+0000","last_undegraded":"2024-09-21T06:12:26.618099+0000","last_fullsized":"2024-09-21T06:12:26.618099+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:04.505044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636248+0000","last_change":"2024-09-21T06:12:24.616468+0000","last_active":"2024-09-21T06:12:26.636248+0000","last_peered":"2024-09-21T06:12:26.636248+0000","last_clean":"2024-09-21T06:12:26.636248+0000","last_became_active":"2024-09-21T06:12:24.615933+0000","last_became_peered":"2024-09-21T06:12:24.615933+0000","last_unstale":"2024-09-21T06:12:26.636248+0000","last_undegraded":"2024-09-21T06:12:26.636248+0000","last_fullsized":"2024-09-21T06:12:26.636248+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:06.245427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636131+0000","last_change":"2024-09-21T06:12:25.623506+0000","last_active":"2024-09-21T06:12:26.636131+0000","last_peered":"2024-09-21T06:12:26.636131+0000","last_clean":"2024-09-21T06:12:26.636131+0000","last_became_active":"2024-09-21T06:12:25.623141+0000","last_became_peered":"2024-09-21T06:12:25.623141+0000","last_unstale":"2024-09-21T06:12:26.636131+0000","last_undegraded":"2024-09-21T06:12:26.636131+0000","last_fullsized":"2024-09-21T06:12:26.636131+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:15.949252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.636060+0000","last_change":"2024-09-21T06:12:24.612148+0000","last_active":"2024-09-21T06:12:26.636060+0000","last_peered":"2024-09-21T06:12:26.636060+0000","last_clean":"2024-09-21T06:12:26.636060+0000","last_became_active":"2024-09-21T06:12:24.611844+0000","last_became_peered":"2024-09-21T06:12:24.611844+0000","last_unstale":"2024-09-21T06:12:26.636060+0000","last_undegraded":"2024-09-21T06:12:26.636060+0000","last_fullsized":"2024-09-21T06:12:26.636060+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:32.529980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.766355+0000","last_change":"2024-09-21T06:12:25.633461+0000","last_active":"2024-09-21T06:12:26.766355+0000","last_peered":"2024-09-21T06:12:26.766355+0000","last_clean":"2024-09-21T06:12:26.766355+0000","last_became_active":"2024-09-21T06:12:25.632910+0000","last_became_peered":"2024-09-21T06:12:25.632910+0000","last_unstale":"2024-09-21T06:12:26.766355+0000","last_undegraded":"2024-09-21T06:12:26.766355+0000","last_fullsized":"2024-09-21T06:12:26.766355+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:57.191937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624785+0000","last_change":"2024-09-21T06:12:24.622020+0000","last_active":"2024-09-21T06:12:27.624785+0000","last_peered":"2024-09-21T06:12:27.624785+0000","last_clean":"2024-09-21T06:12:27.624785+0000","last_became_active":"2024-09-21T06:12:24.621522+0000","last_became_peered":"2024-09-21T06:12:24.621522+0000","last_unstale":"2024-09-21T06:12:27.624785+0000","last_undegraded":"2024-09-21T06:12:27.624785+0000","last_fullsized":"2024-09-21T06:12:27.624785+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:47.691713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.619270+0000","last_change":"2024-09-21T06:12:25.624517+0000","last_active":"2024-09-21T06:12:26.619270+0000","last_peered":"2024-09-21T06:12:26.619270+0000","last_clean":"2024-09-21T06:12:26.619270+0000","last_became_active":"2024-09-21T06:12:25.624185+0000","last_became_peered":"2024-09-21T06:12:25.624185+0000","last_unstale":"2024-09-21T06:12:26.619270+0000","last_undegraded":"2024-09-21T06:12:26.619270+0000","last_fullsized":"2024-09-21T06:12:26.619270+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:48.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617941+0000","last_change":"2024-09-21T06:12:24.621829+0000","last_active":"2024-09-21T06:12:26.617941+0000","last_peered":"2024-09-21T06:12:26.617941+0000","last_clean":"2024-09-21T06:12:26.617941+0000","last_became_active":"2024-09-21T06:12:24.621031+0000","last_became_peered":"2024-09-21T06:12:24.621031+0000","last_unstale":"2024-09-21T06:12:26.617941+0000","last_undegraded":"2024-09-21T06:12:26.617941+0000","last_fullsized":"2024-09-21T06:12:26.617941+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:46.852585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618330+0000","last_change":"2024-09-21T06:12:25.611326+0000","last_active":"2024-09-21T06:12:26.618330+0000","last_peered":"2024-09-21T06:12:26.618330+0000","last_clean":"2024-09-21T06:12:26.618330+0000","last_became_active":"2024-09-21T06:12:25.611018+0000","last_became_peered":"2024-09-21T06:12:25.611018+0000","last_unstale":"2024-09-21T06:12:26.618330+0000","last_undegraded":"2024-09-21T06:12:26.618330+0000","last_fullsized":"2024-09-21T06:12:26.618330+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:00.574834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623036+0000","last_change":"2024-09-21T06:12:24.611596+0000","last_active":"2024-09-21T06:12:27.623036+0000","last_peered":"2024-09-21T06:12:27.623036+0000","last_clean":"2024-09-21T06:12:27.623036+0000","last_became_active":"2024-09-21T06:12:24.611245+0000","last_became_peered":"2024-09-21T06:12:24.611245+0000","last_unstale":"2024-09-21T06:12:27.623036+0000","last_undegraded":"2024-09-21T06:12:27.623036+0000","last_fullsized":"2024-09-21T06:12:27.623036+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:38.406677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618842+0000","last_change":"2024-09-21T06:12:25.607106+0000","last_active":"2024-09-21T06:12:26.618842+0000","last_peered":"2024-09-21T06:12:26.618842+0000","last_clean":"2024-09-21T06:12:26.618842+0000","last_became_active":"2024-09-21T06:12:25.606791+0000","last_became_peered":"2024-09-21T06:12:25.606791+0000","last_unstale":"2024-09-21T06:12:26.618842+0000","last_undegraded":"2024-09-21T06:12:26.618842+0000","last_fullsized":"2024-09-21T06:12:26.618842+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:29.880971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.617564+0000","last_change":"2024-09-21T06:12:24.614354+0000","last_active":"2024-09-21T06:12:26.617564+0000","last_peered":"2024-09-21T06:12:26.617564+0000","last_clean":"2024-09-21T06:12:26.617564+0000","last_became_active":"2024-09-21T06:12:24.614043+0000","last_became_peered":"2024-09-21T06:12:24.614043+0000","last_unstale":"2024-09-21T06:12:26.617564+0000","last_undegraded":"2024-09-21T06:12:26.617564+0000","last_fullsized":"2024-09-21T06:12:26.617564+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:38.061082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.624578+0000","last_change":"2024-09-21T06:12:25.627314+0000","last_active":"2024-09-21T06:12:27.624578+0000","last_peered":"2024-09-21T06:12:27.624578+0000","last_clean":"2024-09-21T06:12:27.624578+0000","last_became_active":"2024-09-21T06:12:25.626867+0000","last_became_peered":"2024-09-21T06:12:25.626867+0000","last_unstale":"2024-09-21T06:12:27.624578+0000","last_undegraded":"2024-09-21T06:12:27.624578+0000","last_fullsized":"2024-09-21T06:12:27.624578+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:24.586864+0000","last_clean_scrub_stamp":"2024-09-21T06:12:24.586864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:06.793918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"141'3","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:27.623565+0000","last_change":"2024-09-21T06:12:24.610884+0000","last_active":"2024-09-21T06:12:27.623565+0000","last_peered":"2024-09-21T06:12:27.623565+0000","last_clean":"2024-09-21T06:12:27.623565+0000","last_became_active":"2024-09-21T06:12:24.610544+0000","last_became_peered":"2024-09-21T06:12:24.610544+0000","last_unstale":"2024-09-21T06:12:27.623565+0000","last_undegraded":"2024-09-21T06:12:27.623565+0000","last_fullsized":"2024-09-21T06:12:27.623565+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:12:23.578986+0000","last_clean_scrub_stamp":"2024-09-21T06:12:23.578986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:24.800893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":334,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T06:12:26.618557+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:12:26.618557+0000","last_peered":"2024-09-21T06:12:26.618557+0000","last_clean":"2024-09-21T06:12:26.618557+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:12:26.618557+0000","last_undegraded":"2024-09-21T06:12:26.618557+0000","last_fullsized":"2024-09-21T06:12:26.618557+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509637,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132004,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239836,"statfs":{"total":96636764160,"available":96501592064,"internally_reserved":0,"allocated":2371584,"data_stored":1371054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509637,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141152,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230688,"statfs":{"total":96636764160,"available":96492224512,"internally_reserved":0,"allocated":1781760,"data_stored":780180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509636,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87000,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284840,"statfs":{"total":96636764160,"available":96547676160,"internally_reserved":0,"allocated":1773568,"data_stored":780056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509637,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136984,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234856,"statfs":{"total":96636764160,"available":96496492544,"internally_reserved":0,"allocated":1769472,"data_stored":780034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509636,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123096,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248744,"statfs":{"total":96636764160,"available":96510713856,"internally_reserved":0,"allocated":1773568,"data_stored":780124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509638,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163368,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208472,"statfs":{"total":96636764160,"available":96469475328,"internally_reserved":0,"allocated":1785856,"data_stored":781716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509636,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123688,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248152,"statfs":{"total":96636764160,"available":96510107648,"internally_reserved":0,"allocated":2379776,"data_stored":1372048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542341,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167512,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204328,"statfs":{"total":96636764160,"available":96465231872,"internally_reserved":0,"allocated":1773568,"data_stored":780616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:12:30.240 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:12:30.240 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T06:12:30.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.446+0000 7fad7cc00640 1 -- 172.21.15.50:0/3770386071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad780b5c70 msgr2=0x7fad7815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.446+0000 7fad7cc00640 1 --2- 172.21.15.50:0/3770386071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad780b5c70 0x7fad7815bb60 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fad78070710 tx=0x7fad6801c800 comp rx=0 tx=0).stop 2024-09-21T06:12:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.446+0000 7fad7cc00640 1 -- 172.21.15.50:0/3770386071 shutdown_connections 2024-09-21T06:12:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.446+0000 7fad7cc00640 1 --2- 172.21.15.50:0/3770386071 >> v2:172.21.15.88:3300/0 conn(0x7fad781527b0 0x7fad78074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.446+0000 7fad7cc00640 1 --2- 172.21.15.50:0/3770386071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad780b5c70 0x7fad7815bb60 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.446+0000 7fad7cc00640 1 -- 172.21.15.50:0/3770386071 >> 172.21.15.50:0/3770386071 conn(0x7fad780b2040 msgr2=0x7fad780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.446+0000 7fad7cc00640 1 -- 172.21.15.50:0/3770386071 shutdown_connections 2024-09-21T06:12:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.446+0000 7fad7cc00640 1 -- 172.21.15.50:0/3770386071 wait complete. 2024-09-21T06:12:30.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.446+0000 7fad7cc00640 1 Processor -- start 2024-09-21T06:12:30.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.446+0000 7fad7cc00640 1 -- start start 2024-09-21T06:12:30.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad7cc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fad780b4de0 0x7fad78158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:30.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad7cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad780b5c70 0x7fad780b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:30.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad7cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad781527b0 0x7fad780af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:30.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad7cc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fad78168a80 con 0x7fad781527b0 2024-09-21T06:12:30.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad7cc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fad78168900 con 0x7fad780b5c70 2024-09-21T06:12:30.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad7cc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fad78168c00 con 0x7fad780b4de0 2024-09-21T06:12:30.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad76800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad781527b0 0x7fad780af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:30.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad76800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad781527b0 0x7fad780af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53386/0 (socket says 172.21.15.50:53386) 2024-09-21T06:12:30.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad76800640 1 -- 172.21.15.50:0/3577625900 learned_addr learned my addr 172.21.15.50:0/3577625900 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:30.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad6d400640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad780b5c70 0x7fad780b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:30.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad75e00640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fad780b4de0 0x7fad78158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:30.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad76800640 1 -- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fad780b4de0 msgr2=0x7fad78158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad76800640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fad780b4de0 0x7fad78158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad76800640 1 -- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad780b5c70 msgr2=0x7fad780b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad76800640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad780b5c70 0x7fad780b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad76800640 1 -- 172.21.15.50:0/3577625900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad781564d0 con 0x7fad781527b0 2024-09-21T06:12:30.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad76800640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad781527b0 0x7fad780af140 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c6e0 tx=0x7fad7000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:30.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad6ea00640 1 -- 172.21.15.50:0/3577625900 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70003b80 con 0x7fad781527b0 2024-09-21T06:12:30.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad6ea00640 1 -- 172.21.15.50:0/3577625900 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad70003d20 con 0x7fad781527b0 2024-09-21T06:12:30.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad6ea00640 1 -- 172.21.15.50:0/3577625900 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70005330 con 0x7fad781527b0 2024-09-21T06:12:30.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad7cc00640 1 -- 172.21.15.50:0/3577625900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad781567b0 con 0x7fad781527b0 2024-09-21T06:12:30.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad7cc00640 1 -- 172.21.15.50:0/3577625900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad78156d70 con 0x7fad781527b0 2024-09-21T06:12:30.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.450+0000 7fad7cc00640 1 -- 172.21.15.50:0/3577625900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad78156260 con 0x7fad781527b0 2024-09-21T06:12:30.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.454+0000 7fad6ea00640 1 -- 172.21.15.50:0/3577625900 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fad700054d0 con 0x7fad781527b0 2024-09-21T06:12:30.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.454+0000 7fad6ea00640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fad54075510 0x7fad54077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:30.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.454+0000 7fad6ea00640 1 -- 172.21.15.50:0/3577625900 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7fad70092f80 con 0x7fad781527b0 2024-09-21T06:12:30.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.454+0000 7fad75e00640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fad54075510 0x7fad54077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:30.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.454+0000 7fad75e00640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fad54075510 0x7fad54077a20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fad680af000 tx=0x7fad6801b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:30.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.458+0000 7fad6ea00640 1 -- 172.21.15.50:0/3577625900 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fad7005f440 con 0x7fad781527b0 2024-09-21T06:12:30.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.702+0000 7fad7cc00640 1 -- 172.21.15.50:0/3577625900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fad7815bb60 con 0x7fad781527b0 2024-09-21T06:12:30.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.706+0000 7fad6ea00640 1 -- 172.21.15.50:0/3577625900 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v40) ==== 190+0+147 (secure 0 0 0) 0x7fad70063e70 con 0x7fad781527b0 2024-09-21T06:12:30.703 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-09-21T06:12:30.704 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQDHX+5mIxq4GxAAQoTPsyjf7H8KkLFK6skkYw== 2024-09-21T06:12:30.704 INFO:teuthology.orchestra.run.smithi050.stdout: caps mds = "allow" 2024-09-21T06:12:30.704 INFO:teuthology.orchestra.run.smithi050.stdout: caps mon = "allow r" 2024-09-21T06:12:30.704 INFO:teuthology.orchestra.run.smithi050.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T06:12:30.704 INFO:teuthology.orchestra.run.smithi050.stderr:updated caps for client.0 2024-09-21T06:12:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.706+0000 7fad7cc00640 1 -- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fad54075510 msgr2=0x7fad54077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.706+0000 7fad7cc00640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fad54075510 0x7fad54077a20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fad680af000 tx=0x7fad6801b040 comp rx=0 tx=0).stop 2024-09-21T06:12:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.706+0000 7fad7cc00640 1 -- 172.21.15.50:0/3577625900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad781527b0 msgr2=0x7fad780af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.706+0000 7fad7cc00640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad781527b0 0x7fad780af140 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c6e0 tx=0x7fad7000cba0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.706+0000 7fad7cc00640 1 -- 172.21.15.50:0/3577625900 shutdown_connections 2024-09-21T06:12:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.706+0000 7fad7cc00640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fad54075510 0x7fad54077a20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.706+0000 7fad7cc00640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad781527b0 0x7fad780af140 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.706+0000 7fad7cc00640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad780b5c70 0x7fad780b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.706+0000 7fad7cc00640 1 --2- 172.21.15.50:0/3577625900 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fad780b4de0 0x7fad78158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.706+0000 7fad7cc00640 1 -- 172.21.15.50:0/3577625900 >> 172.21.15.50:0/3577625900 conn(0x7fad780b2040 msgr2=0x7fad781559f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.710+0000 7fad7cc00640 1 -- 172.21.15.50:0/3577625900 shutdown_connections 2024-09-21T06:12:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.710+0000 7fad7cc00640 1 -- 172.21.15.50:0/3577625900 wait complete. 2024-09-21T06:12:30.728 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:12:30.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.890+0000 7f6b0fe00640 1 -- 172.21.15.50:0/718009877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b101536f0 msgr2=0x7f6b1015d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.890+0000 7f6b0fe00640 1 --2- 172.21.15.50:0/718009877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b101536f0 0x7f6b1015d480 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009900 tx=0x7f6b0001c670 comp rx=0 tx=0).stop 2024-09-21T06:12:30.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.890+0000 7f6b0fe00640 1 -- 172.21.15.50:0/718009877 shutdown_connections 2024-09-21T06:12:30.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.890+0000 7f6b0fe00640 1 --2- 172.21.15.50:0/718009877 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6b10160060 0x7f6b10074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.890+0000 7f6b0fe00640 1 --2- 172.21.15.50:0/718009877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b101536f0 0x7f6b1015d480 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.890+0000 7f6b0fe00640 1 -- 172.21.15.50:0/718009877 >> 172.21.15.50:0/718009877 conn(0x7f6b100b4560 msgr2=0x7f6b100d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:30.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.890+0000 7f6b0fe00640 1 -- 172.21.15.50:0/718009877 shutdown_connections 2024-09-21T06:12:30.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.890+0000 7f6b0fe00640 1 -- 172.21.15.50:0/718009877 wait complete. 2024-09-21T06:12:30.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.890+0000 7f6b0fe00640 1 Processor -- start 2024-09-21T06:12:30.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.890+0000 7f6b0fe00640 1 -- start start 2024-09-21T06:12:30.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b101536f0 0x7f6b10156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:30.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6b10160060 0x7f6b1014a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:30.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10160470 0x7f6b1014de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1016a440 con 0x7f6b101536f0 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1016a2c0 con 0x7f6b10160470 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0fe00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1016a5c0 con 0x7f6b10160060 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b101536f0 0x7f6b10156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b101536f0 0x7f6b10156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53404/0 (socket says 172.21.15.50:53404) 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0ea00640 1 -- 172.21.15.50:0/344808406 learned_addr learned my addr 172.21.15.50:0/344808406 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b07e00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6b10160060 0x7f6b1014a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0f400640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10160470 0x7f6b1014de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0ea00640 1 -- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6b10160060 msgr2=0x7f6b1014a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0ea00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6b10160060 0x7f6b1014a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0ea00640 1 -- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10160470 msgr2=0x7f6b1014de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0ea00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10160470 0x7f6b1014de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:30.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0ea00640 1 -- 172.21.15.50:0/344808406 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b10154660 con 0x7f6b101536f0 2024-09-21T06:12:30.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0ea00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b101536f0 0x7f6b10156b10 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f6b000ac040 tx=0x7f6b000aa640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:30.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b06a00640 1 -- 172.21.15.50:0/344808406 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b000aad80 con 0x7f6b101536f0 2024-09-21T06:12:30.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b06a00640 1 -- 172.21.15.50:0/344808406 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b000036f0 con 0x7f6b101536f0 2024-09-21T06:12:30.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0fe00640 1 -- 172.21.15.50:0/344808406 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b10156030 con 0x7f6b101536f0 2024-09-21T06:12:30.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b06a00640 1 -- 172.21.15.50:0/344808406 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00003b00 con 0x7f6b101536f0 2024-09-21T06:12:30.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0fe00640 1 -- 172.21.15.50:0/344808406 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b101565f0 con 0x7f6b101536f0 2024-09-21T06:12:30.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.894+0000 7f6b0fe00640 1 -- 172.21.15.50:0/344808406 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1015d480 con 0x7f6b101536f0 2024-09-21T06:12:30.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.898+0000 7f6b06a00640 1 -- 172.21.15.50:0/344808406 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f6b00003ca0 con 0x7f6b101536f0 2024-09-21T06:12:30.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.898+0000 7f6b06a00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6aec075510 0x7f6aec077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:30.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.898+0000 7f6b06a00640 1 -- 172.21.15.50:0/344808406 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f6b0012e710 con 0x7f6b101536f0 2024-09-21T06:12:30.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.898+0000 7f6b07e00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6aec075510 0x7f6aec077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:30.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.898+0000 7f6b07e00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6aec075510 0x7f6aec077a20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6b100d2680 tx=0x7f6af800a250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:30.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:30.902+0000 7f6b06a00640 1 -- 172.21.15.50:0/344808406 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6b1015d9c0 con 0x7f6b101536f0 2024-09-21T06:12:31.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 -- 172.21.15.50:0/344808406 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b100b54b0 con 0x7f6b101536f0 2024-09-21T06:12:31.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b06a00640 1 -- 172.21.15.50:0/344808406 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5224 (secure 0 0 0) 0x7f6b000fad10 con 0x7f6b101536f0 2024-09-21T06:12:31.112 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:12:31.113 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":94,"btime":"2024-09-21T06:12:28:632312+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4262887686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":4262887686},{"type":"v1","addr":"172.21.15.50:6835","nonce":4262887686}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":7721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3308178097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3308178097},{"type":"v1","addr":"172.21.15.88:6837","nonce":3308178097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/610421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":610421041},{"type":"v1","addr":"172.21.15.50:6834","nonce":610421041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:12:25.031119+0000","modified":"2024-09-21T06:12:28.054360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8632":{"gid":8632,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2222229719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2222229719},{"type":"v1","addr":"172.21.15.88:6835","nonce":2222229719}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8632,"qdb_cluster":[8632]},"id":10}]} 2024-09-21T06:12:31.113 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 94 2024-09-21T06:12:31.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 -- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6aec075510 msgr2=0x7f6aec077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6aec075510 0x7f6aec077a20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6b100d2680 tx=0x7f6af800a250 comp rx=0 tx=0).stop 2024-09-21T06:12:31.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 -- 172.21.15.50:0/344808406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b101536f0 msgr2=0x7f6b10156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b101536f0 0x7f6b10156b10 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f6b000ac040 tx=0x7f6b000aa640 comp rx=0 tx=0).stop 2024-09-21T06:12:31.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 -- 172.21.15.50:0/344808406 shutdown_connections 2024-09-21T06:12:31.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10160470 0x7f6b1014de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6b10160060 0x7f6b1014a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6aec075510 0x7f6aec077a20 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 --2- 172.21.15.50:0/344808406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b101536f0 0x7f6b10156b10 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 -- 172.21.15.50:0/344808406 >> 172.21.15.50:0/344808406 conn(0x7f6b100b4560 msgr2=0x7f6b1015e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 -- 172.21.15.50:0/344808406 shutdown_connections 2024-09-21T06:12:31.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.114+0000 7f6b0fe00640 1 -- 172.21.15.50:0/344808406 wait complete. 2024-09-21T06:12:31.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:12:25.031119+0000', 'modified': '2024-09-21T06:12:28.054360+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8632}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8632': {'gid': 8632, 'name': 'd', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6835/2222229719', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6834', 'nonce': 2222229719}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 2222229719}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8632, 'qdb_cluster': [8632]} 2024-09-21T06:12:31.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:12:31.537 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T06:12:31.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.698+0000 7f6ad7200640 1 -- 172.21.15.50:0/604132815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad815f5c0 msgr2=0x7f6ad815f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.698+0000 7f6ad7200640 1 --2- 172.21.15.50:0/604132815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad815f5c0 0x7f6ad815f9d0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009900 tx=0x7f6ad001c850 comp rx=0 tx=0).stop 2024-09-21T06:12:31.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.698+0000 7f6ad7200640 1 -- 172.21.15.50:0/604132815 shutdown_connections 2024-09-21T06:12:31.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.698+0000 7f6ad7200640 1 --2- 172.21.15.50:0/604132815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad815f5c0 0x7f6ad815f9d0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.698+0000 7f6ad7200640 1 --2- 172.21.15.50:0/604132815 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ad806dbc0 0x7f6ad806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.698+0000 7f6ad7200640 1 -- 172.21.15.50:0/604132815 >> 172.21.15.50:0/604132815 conn(0x7f6ad80b41f0 msgr2=0x7f6ad80b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.702+0000 7f6ad7200640 1 -- 172.21.15.50:0/604132815 shutdown_connections 2024-09-21T06:12:31.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.702+0000 7f6ad7200640 1 -- 172.21.15.50:0/604132815 wait complete. 2024-09-21T06:12:31.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.702+0000 7f6ad7200640 1 Processor -- start 2024-09-21T06:12:31.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.702+0000 7f6ad7200640 1 -- start start 2024-09-21T06:12:31.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.702+0000 7f6ad7200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ad806dbc0 0x7f6ad80997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.702+0000 7f6ad7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ad80d1140 0x7f6ad8099ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.702+0000 7f6ad7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad815f5c0 0x7f6ad8159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.702+0000 7f6ad7200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad8168ff0 con 0x7f6ad815f5c0 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.702+0000 7f6ad7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad8168e70 con 0x7f6ad80d1140 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.702+0000 7f6ad7200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad8169170 con 0x7f6ad806dbc0 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6ad6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad815f5c0 0x7f6ad8159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6ad6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad815f5c0 0x7f6ad8159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53422/0 (socket says 172.21.15.50:53422) 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6ad6800640 1 -- 172.21.15.50:0/3373830134 learned_addr learned my addr 172.21.15.50:0/3373830134 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6acd400640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ad80d1140 0x7f6ad8099ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6ad5e00640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ad806dbc0 0x7f6ad80997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6ad6800640 1 -- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ad806dbc0 msgr2=0x7f6ad80997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6ad6800640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ad806dbc0 0x7f6ad80997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6ad6800640 1 -- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ad80d1140 msgr2=0x7f6ad8099ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6ad6800640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ad80d1140 0x7f6ad8099ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6ad6800640 1 -- 172.21.15.50:0/3373830134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad815eba0 con 0x7f6ad815f5c0 2024-09-21T06:12:31.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6ad6800640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad815f5c0 0x7f6ad8159340 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009230 tx=0x7f6ad0005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6acea00640 1 -- 172.21.15.50:0/3373830134 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00b2040 con 0x7f6ad815f5c0 2024-09-21T06:12:31.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6acea00640 1 -- 172.21.15.50:0/3373830134 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ad0004130 con 0x7f6ad815f5c0 2024-09-21T06:12:31.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6ad7200640 1 -- 172.21.15.50:0/3373830134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad815c030 con 0x7f6ad815f5c0 2024-09-21T06:12:31.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6acea00640 1 -- 172.21.15.50:0/3373830134 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0003600 con 0x7f6ad815f5c0 2024-09-21T06:12:31.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.706+0000 7f6ad7200640 1 -- 172.21.15.50:0/3373830134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad815c5f0 con 0x7f6ad815f5c0 2024-09-21T06:12:31.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6acea00640 1 -- 172.21.15.50:0/3373830134 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f6ad00bf020 con 0x7f6ad815f5c0 2024-09-21T06:12:31.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ae4114000 1 Processor -- start 2024-09-21T06:12:31.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ae4114000 1 -- start start 2024-09-21T06:12:31.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6acea00640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ab40754e0 0x7f6ab40779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6acea00640 1 -- 172.21.15.50:0/3373830134 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f6ad012d5d0 con 0x7f6ad815f5c0 2024-09-21T06:12:31.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6acd400640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ab40754e0 0x7f6ab40779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ae4114000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 0x560c7ef8d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ae4114000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee3c950 0x560c7ef8dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ae4114000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7ef8e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ae4114000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560c7ee62410 con 0x560c7ee25ca0 2024-09-21T06:12:31.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ae4114000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560c7ed9fb40 con 0x560c7ea4c460 2024-09-21T06:12:31.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ae4114000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x560c7ed8ef70 con 0x560c7ee3c950 2024-09-21T06:12:31.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ad6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7ef8e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ad6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7ef8e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53430/0 (socket says 172.21.15.50:53430) 2024-09-21T06:12:31.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ad6800640 1 -- 172.21.15.50:0/2281500486 learned_addr learned my addr 172.21.15.50:0/2281500486 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:31.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ad5e00640 1 --2- 172.21.15.50:0/2281500486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 0x560c7ef8d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6acd400640 1 --2- 172.21.15.50:0/2281500486 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee3c950 0x560c7ef8dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6acd400640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ab40754e0 0x7f6ab40779f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6ac00058d0 tx=0x7f6ac0005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ad6800640 1 -- 172.21.15.50:0/2281500486 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee3c950 msgr2=0x560c7ef8dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ad6800640 1 --2- 172.21.15.50:0/2281500486 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee3c950 0x560c7ef8dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ad6800640 1 -- 172.21.15.50:0/2281500486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 msgr2=0x560c7ef8d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ad6800640 1 --2- 172.21.15.50:0/2281500486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 0x560c7ef8d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ad6800640 1 -- 172.21.15.50:0/2281500486 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7ed565b0 con 0x560c7ee25ca0 2024-09-21T06:12:31.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.710+0000 7f6ad6800640 1 --2- 172.21.15.50:0/2281500486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7ef8e120 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00f96f0 tx=0x7f6ad00f97c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89231bd69b6727b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ac6600640 1 -- 172.21.15.50:0/2281500486 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad011f1c0 con 0x560c7ee25ca0 2024-09-21T06:12:31.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ac6600640 1 -- 172.21.15.50:0/2281500486 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ad012d960 con 0x560c7ee25ca0 2024-09-21T06:12:31.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ac6600640 1 -- 172.21.15.50:0/2281500486 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00fe270 con 0x560c7ee25ca0 2024-09-21T06:12:31.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ae4114000 1 -- 172.21.15.50:0/2281500486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 msgr2=0x560c7ef8e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ae4114000 1 --2- 172.21.15.50:0/2281500486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7ef8e120 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00f96f0 tx=0x7f6ad00f97c0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ae4114000 1 -- 172.21.15.50:0/2281500486 shutdown_connections 2024-09-21T06:12:31.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ae4114000 1 --2- 172.21.15.50:0/2281500486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7ef8e120 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ae4114000 1 --2- 172.21.15.50:0/2281500486 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee3c950 0x560c7ef8dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ae4114000 1 --2- 172.21.15.50:0/2281500486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 0x560c7ef8d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ae4114000 1 -- 172.21.15.50:0/2281500486 >> 172.21.15.50:0/2281500486 conn(0x560c7ef12d90 msgr2=0x560c7ef42690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ae4114000 1 -- 172.21.15.50:0/2281500486 shutdown_connections 2024-09-21T06:12:31.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ae4114000 1 -- 172.21.15.50:0/2281500486 wait complete. 2024-09-21T06:12:31.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ae4114000 1 -- start start 2024-09-21T06:12:31.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.714+0000 7f6ae4114000 1 Processor -- start 2024-09-21T06:12:31.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 0x560c7ef94920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7ef94e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee3c950 0x560c7ef953a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560c7ed295f0 con 0x560c7ee25ca0 2024-09-21T06:12:31.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560c7ed92d80 con 0x560c7ea4c460 2024-09-21T06:12:31.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x560c7ed35520 con 0x560c7ee3c950 2024-09-21T06:12:31.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 10 client.0 _ll_register_callbacks cb 0x560c7ed39830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:31.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6acd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7ef94e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6acd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7ef94e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53440/0 (socket says 172.21.15.50:53440) 2024-09-21T06:12:31.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6acd400640 1 -- 172.21.15.50:0/549974160 learned_addr learned my addr 172.21.15.50:0/549974160 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:31.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ad6800640 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee3c950 0x560c7ef953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ad5e00640 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 0x560c7ef94920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6acd400640 1 -- 172.21.15.50:0/549974160 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee3c950 msgr2=0x560c7ef953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6acd400640 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee3c950 0x560c7ef953a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6acd400640 1 -- 172.21.15.50:0/549974160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 msgr2=0x560c7ef94920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6acd400640 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 0x560c7ef94920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6acd400640 1 -- 172.21.15.50:0/549974160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7ed2d190 con 0x560c7ee25ca0 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6acd400640 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7ef94e60 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0011490 tx=0x7f6ac00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf25270c43bb805f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6abaa00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6abaa00640 1 -- 172.21.15.50:0/549974160 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac001f840 con 0x560c7ee25ca0 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6abaa00640 1 -- 172.21.15.50:0/549974160 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f6ac0029b80 con 0x560c7ee25ca0 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 1 -- 172.21.15.50:0/549974160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560c7ed27110 con 0x560c7ee25ca0 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6abaa00640 1 -- 172.21.15.50:0/549974160 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f6ac0029d80 con 0x560c7ee25ca0 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 10 client.9411 fetch_fsmap learned FSMap version 94 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 1 -- 172.21.15.50:0/549974160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x560c7ee13f10 con 0x560c7ee25ca0 2024-09-21T06:12:31.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6abaa00640 1 -- 172.21.15.50:0/549974160 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560c7ed27110 con 0x560c7ee25ca0 2024-09-21T06:12:31.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6abaa00640 1 -- 172.21.15.50:0/549974160 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 94) ==== 3502+0+0 (secure 0 0 0) 0x7f6ac001f9e0 con 0x560c7ee25ca0 2024-09-21T06:12:31.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 10 client.9411 fetch_fsmap finished waiting for FSMap version 94 2024-09-21T06:12:31.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 10 client.9411 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:12:31.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 20 client.9411 populate_metadata read hostname 'smithi050' 2024-09-21T06:12:31.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.88:6834/2222229719,v1:172.21.15.88:6835/2222229719] conn(0x560c7ea317d0 0x560c7ef9ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 4 client.9411 mds_command: new command op to 8632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:31.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ae4114000 1 -- 172.21.15.50:0/549974160 --> [v2:172.21.15.88:6834/2222229719,v1:172.21.15.88:6835/2222229719] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c7eda1a90 con 0x560c7ea317d0 2024-09-21T06:12:31.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ad5e00640 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.88:6834/2222229719,v1:172.21.15.88:6835/2222229719] conn(0x560c7ea317d0 0x560c7ef9ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6ad5e00640 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.88:6834/2222229719,v1:172.21.15.88:6835/2222229719] conn(0x560c7ea317d0 0x560c7ef9ed00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27290fd2d4f7f5d0 server_cookie=eef57cdf31dffcb5 in_seq=0 out_seq=0 2024-09-21T06:12:31.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.718+0000 7f6abaa00640 10 client.9411 ms_handle_connect on v2:172.21.15.88:6834/2222229719 2024-09-21T06:12:31.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6abaa00640 1 -- 172.21.15.50:0/549974160 <== mds.0 v2:172.21.15.88:6834/2222229719 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560c7eda1a90 con 0x560c7ea317d0 2024-09-21T06:12:31.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6abaa00640 10 client.9411 handle_command_reply: tid=0 2024-09-21T06:12:31.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6ae4114000 1 client.9411 shutdown 2024-09-21T06:12:31.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6ae4114000 1 -- 172.21.15.50:0/549974160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 msgr2=0x560c7ef94e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6ae4114000 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7ef94e60 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0011490 tx=0x7f6ac00046d0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6ae4114000 1 -- 172.21.15.50:0/549974160 shutdown_connections 2024-09-21T06:12:31.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6ae4114000 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.88:6834/2222229719,v1:172.21.15.88:6835/2222229719] conn(0x560c7ea317d0 0x560c7ef9ed00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6ae4114000 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee3c950 0x560c7ef953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6ae4114000 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7ef94e60 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6ae4114000 1 --2- 172.21.15.50:0/549974160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 0x560c7ef94920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6ae4114000 1 -- 172.21.15.50:0/549974160 >> 172.21.15.50:0/549974160 conn(0x560c7ef12d90 msgr2=0x560c7ef49440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6abaa00640 0 client.9411 ms_handle_reset on v2:172.21.15.88:6834/2222229719 2024-09-21T06:12:31.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6ae4114000 1 -- 172.21.15.50:0/549974160 shutdown_connections 2024-09-21T06:12:31.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6ae4114000 1 -- 172.21.15.50:0/549974160 wait complete. 2024-09-21T06:12:31.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.722+0000 7f6ae4114000 20 client.9411 trim_cache size 0 max 16384 2024-09-21T06:12:31.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ae4114000 1 Processor -- start 2024-09-21T06:12:31.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ae4114000 1 -- start start 2024-09-21T06:12:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ae4114000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 0x560c7efff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ae4114000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee25ca0 0x560c7efffb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ae4114000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee3c950 0x560c7f0000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ae4114000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad8205610 con 0x560c7ee3c950 2024-09-21T06:12:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ae4114000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad82046a0 con 0x560c7ea4c460 2024-09-21T06:12:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ae4114000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad81fd760 con 0x560c7ee25ca0 2024-09-21T06:12:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ad6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee3c950 0x560c7f0000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ad6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee3c950 0x560c7f0000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53448/0 (socket says 172.21.15.50:53448) 2024-09-21T06:12:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ad6800640 1 -- 172.21.15.50:0/3934428042 learned_addr learned my addr 172.21.15.50:0/3934428042 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ad5e00640 1 --2- 172.21.15.50:0/3934428042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 0x560c7efff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6acd400640 1 --2- 172.21.15.50:0/3934428042 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee25ca0 0x560c7efffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ad6800640 1 -- 172.21.15.50:0/3934428042 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee25ca0 msgr2=0x560c7efffb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ad6800640 1 --2- 172.21.15.50:0/3934428042 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee25ca0 0x560c7efffb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ad6800640 1 -- 172.21.15.50:0/3934428042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 msgr2=0x560c7efff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ad6800640 1 --2- 172.21.15.50:0/3934428042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 0x560c7efff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ad6800640 1 -- 172.21.15.50:0/3934428042 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7edeadb0 con 0x560c7ee3c950 2024-09-21T06:12:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ad6800640 1 --2- 172.21.15.50:0/3934428042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee3c950 0x560c7f0000a0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00f9980 tx=0x7f6ad001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f63720bf61c83e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ac5c00640 1 -- 172.21.15.50:0/3934428042 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad011f1c0 con 0x560c7ee3c950 2024-09-21T06:12:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ac5c00640 1 -- 172.21.15.50:0/3934428042 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ad00f9500 con 0x560c7ee3c950 2024-09-21T06:12:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.742+0000 7f6ac5c00640 1 -- 172.21.15.50:0/3934428042 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00fa0f0 con 0x560c7ee3c950 2024-09-21T06:12:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 -- 172.21.15.50:0/3934428042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee3c950 msgr2=0x560c7f0000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 --2- 172.21.15.50:0/3934428042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee3c950 0x560c7f0000a0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00f9980 tx=0x7f6ad001c850 comp rx=0 tx=0).stop 2024-09-21T06:12:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 -- 172.21.15.50:0/3934428042 shutdown_connections 2024-09-21T06:12:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 --2- 172.21.15.50:0/3934428042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee3c950 0x560c7f0000a0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 --2- 172.21.15.50:0/3934428042 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ee25ca0 0x560c7efffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 --2- 172.21.15.50:0/3934428042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ea4c460 0x560c7efff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 -- 172.21.15.50:0/3934428042 >> 172.21.15.50:0/3934428042 conn(0x560c7ef12d90 msgr2=0x560c7ef8b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 -- 172.21.15.50:0/3934428042 shutdown_connections 2024-09-21T06:12:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 -- 172.21.15.50:0/3934428042 wait complete. 2024-09-21T06:12:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 -- start start 2024-09-21T06:12:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 Processor -- start 2024-09-21T06:12:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ea4c460 0x560c7f003390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7f0038d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ee3c950 0x560c7f003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560c7ed5aee0 con 0x560c7ee25ca0 2024-09-21T06:12:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560c7ee4e750 con 0x560c7ee3c950 2024-09-21T06:12:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x560c7ee13a80 con 0x560c7ea4c460 2024-09-21T06:12:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ae4114000 10 client.0 _ll_register_callbacks cb 0x7f6ac000fd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:31.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6acd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7f0038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ad5e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ea4c460 0x560c7f003390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6acd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7f0038d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53454/0 (socket says 172.21.15.50:53454) 2024-09-21T06:12:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6acd400640 1 -- 172.21.15.50:0/2877849810 learned_addr learned my addr 172.21.15.50:0/2877849810 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6ad6800640 1 --2- 172.21.15.50:0/2877849810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ee3c950 0x560c7f003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6acd400640 1 -- 172.21.15.50:0/2877849810 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ea4c460 msgr2=0x560c7f003390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6acd400640 1 --2- 172.21.15.50:0/2877849810 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ea4c460 0x560c7f003390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6acd400640 1 -- 172.21.15.50:0/2877849810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ee3c950 msgr2=0x560c7f003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6acd400640 1 --2- 172.21.15.50:0/2877849810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ee3c950 0x560c7f003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6acd400640 1 -- 172.21.15.50:0/2877849810 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7ed55ab0 con 0x560c7ee25ca0 2024-09-21T06:12:31.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6acd400640 1 --2- 172.21.15.50:0/2877849810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7f0038d0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f6ac001ffb0 tx=0x7f6ac0029f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27678665ddb3758e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.746+0000 7f6abaa00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-21T06:12:31.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6abaa00640 1 -- 172.21.15.50:0/2877849810 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0037070 con 0x560c7ee25ca0 2024-09-21T06:12:31.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 -- 172.21.15.50:0/2877849810 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad00f9500 con 0x560c7ee25ca0 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6abaa00640 1 -- 172.21.15.50:0/2877849810 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f6ac0005020 con 0x560c7ee25ca0 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6abaa00640 1 -- 172.21.15.50:0/2877849810 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f6ac0048050 con 0x560c7ee25ca0 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 10 client.9417 fetch_fsmap learned FSMap version 94 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 -- 172.21.15.50:0/2877849810 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x560c7ed34070 con 0x560c7ee25ca0 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6abaa00640 1 -- 172.21.15.50:0/2877849810 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ad00f9500 con 0x560c7ee25ca0 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6abaa00640 1 -- 172.21.15.50:0/2877849810 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 94) ==== 3502+0+0 (secure 0 0 0) 0x7f6ac004b020 con 0x560c7ee25ca0 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 10 client.9417 fetch_fsmap finished waiting for FSMap version 94 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 10 client.9417 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 20 client.9417 populate_metadata read hostname 'smithi050' 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 --2- 172.21.15.50:0/2877849810 >> [v2:172.21.15.88:6834/2222229719,v1:172.21.15.88:6835/2222229719] conn(0x560c7ea1f830 0x560c7efbf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 4 client.9417 mds_command: new command op to 8632 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 -- 172.21.15.50:0/2877849810 --> [v2:172.21.15.88:6834/2222229719,v1:172.21.15.88:6835/2222229719] -- command(tid 0: {"prefix": "status"}) -- 0x560c7ed9d540 con 0x560c7ea1f830 2024-09-21T06:12:31.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad5e00640 1 --2- 172.21.15.50:0/2877849810 >> [v2:172.21.15.88:6834/2222229719,v1:172.21.15.88:6835/2222229719] conn(0x560c7ea1f830 0x560c7efbf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad5e00640 1 --2- 172.21.15.50:0/2877849810 >> [v2:172.21.15.88:6834/2222229719,v1:172.21.15.88:6835/2222229719] conn(0x560c7ea1f830 0x560c7efbf730 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b70375e0ac54fef5 server_cookie=a9e6e6ffc4f63fd2 in_seq=0 out_seq=0 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6abaa00640 10 client.9417 ms_handle_connect on v2:172.21.15.88:6834/2222229719 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6abaa00640 1 -- 172.21.15.50:0/2877849810 <== mds.0 v2:172.21.15.88:6834/2222229719 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560c7ed9d540 con 0x560c7ea1f830 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6abaa00640 10 client.9417 handle_command_reply: tid=0 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 client.9417 shutdown 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 -- 172.21.15.50:0/2877849810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 msgr2=0x560c7f0038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 --2- 172.21.15.50:0/2877849810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7f0038d0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f6ac001ffb0 tx=0x7f6ac0029f20 comp rx=0 tx=0).stop 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 -- 172.21.15.50:0/2877849810 shutdown_connections 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 --2- 172.21.15.50:0/2877849810 >> [v2:172.21.15.88:6834/2222229719,v1:172.21.15.88:6835/2222229719] conn(0x560c7ea1f830 0x560c7efbf730 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 --2- 172.21.15.50:0/2877849810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c7ee3c950 0x560c7f003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 --2- 172.21.15.50:0/2877849810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560c7ee25ca0 0x560c7f0038d0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 --2- 172.21.15.50:0/2877849810 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x560c7ea4c460 0x560c7f003390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6abaa00640 0 client.9417 ms_handle_reset on v2:172.21.15.88:6834/2222229719 2024-09-21T06:12:31.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 -- 172.21.15.50:0/2877849810 >> 172.21.15.50:0/2877849810 conn(0x560c7ef12d90 msgr2=0x560c7ef8bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 -- 172.21.15.50:0/2877849810 shutdown_connections 2024-09-21T06:12:31.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 1 -- 172.21.15.50:0/2877849810 wait complete. 2024-09-21T06:12:31.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ae4114000 20 client.9417 trim_cache size 0 max 16384 2024-09-21T06:12:31.752 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:12:31.752 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:12:31.752 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-21T06:12:31.752 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 8632, 2024-09-21T06:12:31.752 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-21T06:12:31.752 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-21T06:12:31.752 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-21T06:12:31.752 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 6.1356202360000003, 2024-09-21T06:12:31.752 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 94, 2024-09-21T06:12:31.753 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 142, 2024-09-21T06:12:31.753 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 142, 2024-09-21T06:12:31.753 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 10.520701552 2024-09-21T06:12:31.753 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:12:31.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad7200640 1 -- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ab40754e0 msgr2=0x7f6ab40779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad7200640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ab40754e0 0x7f6ab40779f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6ac00058d0 tx=0x7f6ac0005860 comp rx=0 tx=0).stop 2024-09-21T06:12:31.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad7200640 1 -- 172.21.15.50:0/3373830134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad815f5c0 msgr2=0x7f6ad8159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad7200640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad815f5c0 0x7f6ad8159340 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009230 tx=0x7f6ad0005e70 comp rx=0 tx=0).stop 2024-09-21T06:12:31.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad7200640 1 -- 172.21.15.50:0/3373830134 shutdown_connections 2024-09-21T06:12:31.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad7200640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6ab40754e0 0x7f6ab40779f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad7200640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad815f5c0 0x7f6ad8159340 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad7200640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ad80d1140 0x7f6ad8099ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad7200640 1 --2- 172.21.15.50:0/3373830134 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ad806dbc0 0x7f6ad80997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad7200640 1 -- 172.21.15.50:0/3373830134 >> 172.21.15.50:0/3373830134 conn(0x7f6ad80b41f0 msgr2=0x7f6ad8068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad7200640 1 -- 172.21.15.50:0/3373830134 shutdown_connections 2024-09-21T06:12:31.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.750+0000 7f6ad7200640 1 -- 172.21.15.50:0/3373830134 wait complete. 2024-09-21T06:12:31.769 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T06:12:31.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.930+0000 7fbe27e00640 1 -- 172.21.15.50:0/1478750679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe200cefa0 msgr2=0x7fbe200cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.930+0000 7fbe27e00640 1 --2- 172.21.15.50:0/1478750679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe200cefa0 0x7fbe200cf3b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009840 tx=0x7fbe0c01c820 comp rx=0 tx=0).stop 2024-09-21T06:12:31.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.930+0000 7fbe27e00640 1 -- 172.21.15.50:0/1478750679 shutdown_connections 2024-09-21T06:12:31.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.930+0000 7fbe27e00640 1 --2- 172.21.15.50:0/1478750679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe200cefa0 0x7fbe200cf3b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.930+0000 7fbe27e00640 1 -- 172.21.15.50:0/1478750679 >> 172.21.15.50:0/1478750679 conn(0x7fbe200b2040 msgr2=0x7fbe200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.930+0000 7fbe27e00640 1 -- 172.21.15.50:0/1478750679 shutdown_connections 2024-09-21T06:12:31.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.930+0000 7fbe27e00640 1 -- 172.21.15.50:0/1478750679 wait complete. 2024-09-21T06:12:31.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.930+0000 7fbe27e00640 1 Processor -- start 2024-09-21T06:12:31.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.930+0000 7fbe27e00640 1 -- start start 2024-09-21T06:12:31.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe27e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe200749a0 0x7fbe20161420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe27e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbe200b3340 0x7fbe20167760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe27e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe200cefa0 0x7fbe201fc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe27e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2016b290 con 0x7fbe200cefa0 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe27e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2016b110 con 0x7fbe200749a0 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe27e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2016b410 con 0x7fbe200b3340 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe25e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe200cefa0 0x7fbe201fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe25e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe200cefa0 0x7fbe201fc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53460/0 (socket says 172.21.15.50:53460) 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe25e00640 1 -- 172.21.15.50:0/595484980 learned_addr learned my addr 172.21.15.50:0/595484980 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe24a00640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbe200b3340 0x7fbe20167760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe25400640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe200749a0 0x7fbe20161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe25e00640 1 -- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbe200b3340 msgr2=0x7fbe20167760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe25e00640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbe200b3340 0x7fbe20167760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe25e00640 1 -- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe200749a0 msgr2=0x7fbe20161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe25e00640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe200749a0 0x7fbe20161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe25e00640 1 -- 172.21.15.50:0/595484980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe200b0a20 con 0x7fbe200cefa0 2024-09-21T06:12:31.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe25e00640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe200cefa0 0x7fbe201fc430 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800caa0 tx=0x7fbe1800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe1d600640 1 -- 172.21.15.50:0/595484980 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18016020 con 0x7fbe200cefa0 2024-09-21T06:12:31.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe27e00640 1 -- 172.21.15.50:0/595484980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe20167ca0 con 0x7fbe200cefa0 2024-09-21T06:12:31.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe1d600640 1 -- 172.21.15.50:0/595484980 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe1800a950 con 0x7fbe200cefa0 2024-09-21T06:12:31.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe1d600640 1 -- 172.21.15.50:0/595484980 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800ada0 con 0x7fbe200cefa0 2024-09-21T06:12:31.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe27e00640 1 -- 172.21.15.50:0/595484980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe200b0f70 con 0x7fbe200cefa0 2024-09-21T06:12:31.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.934+0000 7fbe27e00640 1 -- 172.21.15.50:0/595484980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe201619e0 con 0x7fbe200cefa0 2024-09-21T06:12:31.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.938+0000 7fbe1d600640 1 -- 172.21.15.50:0/595484980 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fbe18019020 con 0x7fbe200cefa0 2024-09-21T06:12:31.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.938+0000 7fbe1d600640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbe000754e0 0x7fbe000779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.938+0000 7fbe1d600640 1 -- 172.21.15.50:0/595484980 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7fbe180933b0 con 0x7fbe200cefa0 2024-09-21T06:12:31.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.938+0000 7fbe25400640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbe000754e0 0x7fbe000779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.938+0000 7fbe25400640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbe000754e0 0x7fbe000779f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fbe14000c10 tx=0x7fbe14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:31.942+0000 7fbe1d600640 1 -- 172.21.15.50:0/595484980 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbe18020030 con 0x7fbe200cefa0 2024-09-21T06:12:32.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.154+0000 7fbe27e00640 1 -- 172.21.15.50:0/595484980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbe200cf3b0 con 0x7fbe200cefa0 2024-09-21T06:12:32.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.154+0000 7fbe1d600640 1 -- 172.21.15.50:0/595484980 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) ==== 53+0+73 (secure 0 0 0) 0x7fbe201430e0 con 0x7fbe200cefa0 2024-09-21T06:12:32.155 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:12:32.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.158+0000 7fbe27e00640 1 -- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbe000754e0 msgr2=0x7fbe000779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:32.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.158+0000 7fbe27e00640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbe000754e0 0x7fbe000779f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fbe14000c10 tx=0x7fbe14009040 comp rx=0 tx=0).stop 2024-09-21T06:12:32.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.158+0000 7fbe27e00640 1 -- 172.21.15.50:0/595484980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe200cefa0 msgr2=0x7fbe201fc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:32.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.158+0000 7fbe27e00640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe200cefa0 0x7fbe201fc430 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800caa0 tx=0x7fbe1800cb90 comp rx=0 tx=0).stop 2024-09-21T06:12:32.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.158+0000 7fbe27e00640 1 -- 172.21.15.50:0/595484980 shutdown_connections 2024-09-21T06:12:32.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.158+0000 7fbe27e00640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbe000754e0 0x7fbe000779f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:32.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.158+0000 7fbe27e00640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe200cefa0 0x7fbe201fc430 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:32.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.158+0000 7fbe27e00640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbe200b3340 0x7fbe20167760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:32.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.158+0000 7fbe27e00640 1 --2- 172.21.15.50:0/595484980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe200749a0 0x7fbe20161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:32.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.158+0000 7fbe27e00640 1 -- 172.21.15.50:0/595484980 >> 172.21.15.50:0/595484980 conn(0x7fbe200b2040 msgr2=0x7fbe200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:32.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.158+0000 7fbe27e00640 1 -- 172.21.15.50:0/595484980 shutdown_connections 2024-09-21T06:12:32.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32.158+0000 7fbe27e00640 1 -- 172.21.15.50:0/595484980 wait complete. 2024-09-21T06:12:32.177 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:12:32.177 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:12:32.177 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-21T06:12:32.177 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:12:32.177 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:12:32.177 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:12:32.177 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T06:12:32.177 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:12:32.177 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:12:32.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:32.177 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-21T06:12:32.187 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:12:32.187 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:12:32.187 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:12:32.187 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:12:32.187 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-21T06:12:32.187 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:12:32.187 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:12:32.187 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:12:32.187 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 41443sec preferred_lft 41443sec 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T06:12:32.188 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:8b:d0:0a:5d brd ff:ff:ff:ff:ff:ff 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:12:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T06:12:32.190 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:12:32.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:32.190 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:12:32.253 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-21T06:12:32.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:32.255 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-21T06:12:32.308 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-21T06:12:32.308 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:12:32.308 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-21T06:12:32.308 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T06:12:32.308 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-21T06:12:32.308 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:12:32.309 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:12:32.309 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-21T06:12:32.309 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:12:32.309 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:12:32.309 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:12:32.309 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-21T06:12:32.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:12:32.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:12:32.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:32.450 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T06:12:32.503 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:12:32.503 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:12:32.503 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:12:32.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:32.503 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-09-21T06:12:32.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:12:32.566 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:12:32.618 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:12:32.619 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:12:32.619 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:12:32.674 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T06:12:32.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:32.674 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:12:32.780 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:12:32.781 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:12:32.781 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:12:32.799 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:12:32.799 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:12:32.799 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:12:32.799 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:12:32.799 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:12:32.809 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T06:12:32.810 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T06:12:32.810 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-09-21T06:12:32.810 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.88:3300,172.21.15.88:3301" 2024-09-21T06:12:32.810 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T06:12:32.810 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790" 2024-09-21T06:12:32.810 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.88:6789,172.21.15.88:6790:/ 2024-09-21T06:12:32.810 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=3f44e77a-b7bc-4ef8-b058-60eaaa9147d6" will pass to kernel 2024-09-21T06:12:32.813 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:12:32.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:32.814 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:12:32.834 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:12:32.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:12:32.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:12:32.944 INFO:teuthology.orchestra.run.smithi050.stdout:client9426 2024-09-21T06:12:32.944 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:12:32.944 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:32.953 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T06:12:33.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.162+0000 7fb0b0400640 1 -- 172.21.15.50:0/2827668273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb090003c70 msgr2=0x7fb0900040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:33.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.162+0000 7fb0b0400640 1 --2- 172.21.15.50:0/2827668273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb090003c70 0x7fb0900040c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb09c001860 tx=0x7fb09c01c8e0 comp rx=0 tx=0).stop 2024-09-21T06:12:33.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.162+0000 7fb0ae400640 1 -- 172.21.15.50:0/2827668273 reap_dead start 2024-09-21T06:12:33.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.162+0000 7fb0b0400640 1 -- 172.21.15.50:0/2827668273 shutdown_connections 2024-09-21T06:12:33.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.162+0000 7fb0b0400640 1 -- 172.21.15.50:0/2827668273 >> 172.21.15.50:0/2827668273 conn(0x7fb0a80b2040 msgr2=0x7fb0a80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:33.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.162+0000 7fb0b0400640 1 -- 172.21.15.50:0/2827668273 shutdown_connections 2024-09-21T06:12:33.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.162+0000 7fb0b0400640 1 -- 172.21.15.50:0/2827668273 wait complete. 2024-09-21T06:12:33.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0b0400640 1 Processor -- start 2024-09-21T06:12:33.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0b0400640 1 -- start start 2024-09-21T06:12:33.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0b0400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb090003c70 0x7fb0a813c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:33.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0b0400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a813c600 0x7fb0a812d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:33.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0b0400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0a815b200 0x7fb0a815b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:33.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0b0400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a81688e0 con 0x7fb0a813c600 2024-09-21T06:12:33.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0b0400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8168760 con 0x7fb090003c70 2024-09-21T06:12:33.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0b0400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a8168a60 con 0x7fb0a815b200 2024-09-21T06:12:33.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0ad000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a813c600 0x7fb0a812d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:33.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0ad000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a813c600 0x7fb0a812d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53472/0 (socket says 172.21.15.50:53472) 2024-09-21T06:12:33.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0ad000640 1 -- 172.21.15.50:0/4133444222 learned_addr learned my addr 172.21.15.50:0/4133444222 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:33.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0ada00640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb090003c70 0x7fb0a813c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:33.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0ae400640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0a815b200 0x7fb0a815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:33.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0ad000640 1 -- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0a815b200 msgr2=0x7fb0a815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:33.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0ad000640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0a815b200 0x7fb0a815b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:33.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0ad000640 1 -- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb090003c70 msgr2=0x7fb0a813c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:33.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0ad000640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb090003c70 0x7fb0a813c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:33.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0ad000640 1 -- 172.21.15.50:0/4133444222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a815bb50 con 0x7fb0a813c600 2024-09-21T06:12:33.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0ada00640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb090003c70 0x7fb0a813c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:33.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.166+0000 7fb0ad000640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a813c600 0x7fb0a812d7f0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fb09400e590 tx=0x7fb09400ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:33.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.170+0000 7fb0a5600640 1 -- 172.21.15.50:0/4133444222 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094015070 con 0x7fb0a813c600 2024-09-21T06:12:33.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.170+0000 7fb0a5600640 1 -- 172.21.15.50:0/4133444222 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb094010980 con 0x7fb0a813c600 2024-09-21T06:12:33.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.170+0000 7fb0b0400640 1 -- 172.21.15.50:0/4133444222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a813cde0 con 0x7fb0a813c600 2024-09-21T06:12:33.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.170+0000 7fb0a5600640 1 -- 172.21.15.50:0/4133444222 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094010db0 con 0x7fb0a813c600 2024-09-21T06:12:33.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.170+0000 7fb0b0400640 1 -- 172.21.15.50:0/4133444222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a813d3a0 con 0x7fb0a813c600 2024-09-21T06:12:33.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.170+0000 7fb0b0400640 1 -- 172.21.15.50:0/4133444222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a813cb50 con 0x7fb0a813c600 2024-09-21T06:12:33.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.170+0000 7fb0a5600640 1 -- 172.21.15.50:0/4133444222 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fb094018070 con 0x7fb0a813c600 2024-09-21T06:12:33.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.170+0000 7fb0a5600640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0880754e0 0x7fb0880779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:33.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.170+0000 7fb0a5600640 1 -- 172.21.15.50:0/4133444222 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7fb094093c70 con 0x7fb0a813c600 2024-09-21T06:12:33.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.170+0000 7fb0ada00640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0880754e0 0x7fb0880779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:33.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.174+0000 7fb0ada00640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0880754e0 0x7fb0880779f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb09c001830 tx=0x7fb09c019720 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:33.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.174+0000 7fb0a5600640 1 -- 172.21.15.50:0/4133444222 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb0a815bb50 con 0x7fb0a813c600 2024-09-21T06:12:33.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.390+0000 7fb0b0400640 1 -- 172.21.15.50:0/4133444222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fb0a8074db0 con 0x7fb0a813c600 2024-09-21T06:12:33.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.714+0000 7fb0a5600640 1 -- 172.21.15.50:0/4133444222 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v95) ==== 136+0+0 (secure 0 0 0) 0x7fb0940606f0 con 0x7fb0a813c600 2024-09-21T06:12:33.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.718+0000 7fb0b0400640 1 -- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0880754e0 msgr2=0x7fb0880779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:33.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.718+0000 7fb0b0400640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0880754e0 0x7fb0880779f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb09c001830 tx=0x7fb09c019720 comp rx=0 tx=0).stop 2024-09-21T06:12:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.718+0000 7fb0b0400640 1 -- 172.21.15.50:0/4133444222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a813c600 msgr2=0x7fb0a812d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.718+0000 7fb0b0400640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a813c600 0x7fb0a812d7f0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fb09400e590 tx=0x7fb09400ea50 comp rx=0 tx=0).stop 2024-09-21T06:12:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.718+0000 7fb0b0400640 1 -- 172.21.15.50:0/4133444222 shutdown_connections 2024-09-21T06:12:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.718+0000 7fb0b0400640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0a815b200 0x7fb0a815b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.718+0000 7fb0b0400640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0880754e0 0x7fb0880779f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.718+0000 7fb0b0400640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a813c600 0x7fb0a812d7f0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.718+0000 7fb0b0400640 1 --2- 172.21.15.50:0/4133444222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb090003c70 0x7fb0a813c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.718+0000 7fb0b0400640 1 -- 172.21.15.50:0/4133444222 >> 172.21.15.50:0/4133444222 conn(0x7fb0a80b2040 msgr2=0x7fb0a80cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.718+0000 7fb0b0400640 1 -- 172.21.15.50:0/4133444222 shutdown_connections 2024-09-21T06:12:33.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.718+0000 7fb0b0400640 1 -- 172.21.15.50:0/4133444222 wait complete. 2024-09-21T06:12:33.739 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:12:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.898+0000 7fb4c2e00640 1 -- 172.21.15.50:0/3850316526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4bc160470 msgr2=0x7fb4bc074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.898+0000 7fb4c2e00640 1 --2- 172.21.15.50:0/3850316526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4bc160470 0x7fb4bc074780 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0ced40 tx=0x7fb4b401ccf0 comp rx=0 tx=0).stop 2024-09-21T06:12:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 -- 172.21.15.50:0/3850316526 shutdown_connections 2024-09-21T06:12:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 --2- 172.21.15.50:0/3850316526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4bc160470 0x7fb4bc074780 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 --2- 172.21.15.50:0/3850316526 >> v2:172.21.15.88:3300/0 conn(0x7fb4bc160060 0x7fb4bc074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 -- 172.21.15.50:0/3850316526 >> 172.21.15.50:0/3850316526 conn(0x7fb4bc0b4560 msgr2=0x7fb4bc0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 -- 172.21.15.50:0/3850316526 shutdown_connections 2024-09-21T06:12:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 -- 172.21.15.50:0/3850316526 wait complete. 2024-09-21T06:12:33.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 Processor -- start 2024-09-21T06:12:33.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 -- start start 2024-09-21T06:12:33.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4bc1536f0 0x7fb4bc1458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:33.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4bc160060 0x7fb4bc147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:33.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc160470 0x7fb4bc0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:33.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc1696e0 con 0x7fb4bc160060 2024-09-21T06:12:33.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc169560 con 0x7fb4bc160470 2024-09-21T06:12:33.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.902+0000 7fb4c2e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb4bc169860 con 0x7fb4bc1536f0 2024-09-21T06:12:33.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4bc160060 0x7fb4bc147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:33.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4bc160060 0x7fb4bc147f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53490/0 (socket says 172.21.15.50:53490) 2024-09-21T06:12:33.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c1000640 1 -- 172.21.15.50:0/4110874402 learned_addr learned my addr 172.21.15.50:0/4110874402 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:33.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c1a00640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4bc1536f0 0x7fb4bc1458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:33.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c1000640 1 -- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4bc1536f0 msgr2=0x7fb4bc1458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:33.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c2400640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc160470 0x7fb4bc0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:33.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c1000640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4bc1536f0 0x7fb4bc1458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:33.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c1000640 1 -- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc160470 msgr2=0x7fb4bc0d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:33.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c1000640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc160470 0x7fb4bc0d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:33.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c1000640 1 -- 172.21.15.50:0/4110874402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc146180 con 0x7fb4bc160060 2024-09-21T06:12:33.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c1a00640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4bc1536f0 0x7fb4bc1458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:33.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c2400640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc160470 0x7fb4bc0d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:33.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c1000640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4bc160060 0x7fb4bc147f60 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800e7c0 tx=0x7fb4a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:33.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4b9600640 1 -- 172.21.15.50:0/4110874402 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a800cc60 con 0x7fb4bc160060 2024-09-21T06:12:33.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4b9600640 1 -- 172.21.15.50:0/4110874402 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4a800ce00 con 0x7fb4bc160060 2024-09-21T06:12:33.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c2e00640 1 -- 172.21.15.50:0/4110874402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc147220 con 0x7fb4bc160060 2024-09-21T06:12:33.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4b9600640 1 -- 172.21.15.50:0/4110874402 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8005280 con 0x7fb4bc160060 2024-09-21T06:12:33.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c2e00640 1 -- 172.21.15.50:0/4110874402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4bc146740 con 0x7fb4bc160060 2024-09-21T06:12:33.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4c2e00640 1 -- 172.21.15.50:0/4110874402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4bc15d480 con 0x7fb4bc160060 2024-09-21T06:12:33.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.906+0000 7fb4b9600640 1 -- 172.21.15.50:0/4110874402 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fb4a8005420 con 0x7fb4bc160060 2024-09-21T06:12:33.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.910+0000 7fb4b9600640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb49c0754e0 0x7fb49c0779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:33.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.910+0000 7fb4b9600640 1 -- 172.21.15.50:0/4110874402 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7fb4a8093e60 con 0x7fb4bc160060 2024-09-21T06:12:33.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.910+0000 7fb4c1a00640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb49c0754e0 0x7fb49c0779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:33.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.910+0000 7fb4c1a00640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb49c0754e0 0x7fb49c0779f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0001630 tx=0x7fb4b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:33.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:33.914+0000 7fb4b9600640 1 -- 172.21.15.50:0/4110874402 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb4a809c030 con 0x7fb4bc160060 2024-09-21T06:12:34.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.114+0000 7fb4c2e00640 1 -- 172.21.15.50:0/4110874402 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fb4bc06b540 con 0x7fb49c0754e0 2024-09-21T06:12:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4b9600640 1 -- 172.21.15.50:0/4110874402 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fb4b00067e0 con 0x7fb49c0754e0 2024-09-21T06:12:34.116 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:12:34.116 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:12:34.116 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:12:34.116 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:12:34.116 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:12:34.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4c2e00640 1 -- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb49c0754e0 msgr2=0x7fb49c0779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:34.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4c2e00640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb49c0754e0 0x7fb49c0779f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0001630 tx=0x7fb4b0009040 comp rx=0 tx=0).stop 2024-09-21T06:12:34.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4c2e00640 1 -- 172.21.15.50:0/4110874402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4bc160060 msgr2=0x7fb4bc147f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:34.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4c2e00640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4bc160060 0x7fb4bc147f60 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800e7c0 tx=0x7fb4a800ec80 comp rx=0 tx=0).stop 2024-09-21T06:12:34.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4c2e00640 1 -- 172.21.15.50:0/4110874402 shutdown_connections 2024-09-21T06:12:34.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4c2e00640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc160470 0x7fb4bc0d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4c2e00640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb49c0754e0 0x7fb49c0779f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4c2e00640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4bc160060 0x7fb4bc147f60 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4c2e00640 1 --2- 172.21.15.50:0/4110874402 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb4bc1536f0 0x7fb4bc1458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4c2e00640 1 -- 172.21.15.50:0/4110874402 >> 172.21.15.50:0/4110874402 conn(0x7fb4bc0b4560 msgr2=0x7fb4bc14bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:34.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4c2e00640 1 -- 172.21.15.50:0/4110874402 shutdown_connections 2024-09-21T06:12:34.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.118+0000 7fb4c2e00640 1 -- 172.21.15.50:0/4110874402 wait complete. 2024-09-21T06:12:34.138 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T06:12:34.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.302+0000 7f18be200640 1 -- 172.21.15.50:0/4136832748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18b80cefa0 msgr2=0x7f18b80cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:34.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.302+0000 7f18be200640 1 --2- 172.21.15.50:0/4136832748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18b80cefa0 0x7f18b80cf3b0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f18b80b3040 tx=0x7f18a001c8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.302+0000 7f18be200640 1 -- 172.21.15.50:0/4136832748 shutdown_connections 2024-09-21T06:12:34.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.302+0000 7f18be200640 1 --2- 172.21.15.50:0/4136832748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18b80cefa0 0x7f18b80cf3b0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.302+0000 7f18be200640 1 --2- 172.21.15.50:0/4136832748 >> v2:172.21.15.88:3300/0 conn(0x7f18b80749a0 0x7f18b8074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.302+0000 7f18be200640 1 -- 172.21.15.50:0/4136832748 >> 172.21.15.50:0/4136832748 conn(0x7f18b80b2040 msgr2=0x7f18b80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:34.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.302+0000 7f18be200640 1 -- 172.21.15.50:0/4136832748 shutdown_connections 2024-09-21T06:12:34.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.302+0000 7f18be200640 1 -- 172.21.15.50:0/4136832748 wait complete. 2024-09-21T06:12:34.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.302+0000 7f18be200640 1 Processor -- start 2024-09-21T06:12:34.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.302+0000 7f18be200640 1 -- start start 2024-09-21T06:12:34.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18be200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18b80749a0 0x7f18b813d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:34.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18be200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18b80cefa0 0x7f18b81faa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18be200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18b8160330 0x7f18b8141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18be200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f18b816a060 con 0x7f18b8160330 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18be200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8169ee0 con 0x7f18b80749a0 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18be200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f18b816a1e0 con 0x7f18b80cefa0 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18b7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18b8160330 0x7f18b8141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18b7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18b8160330 0x7f18b8141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53510/0 (socket says 172.21.15.50:53510) 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18b7e00640 1 -- 172.21.15.50:0/327896561 learned_addr learned my addr 172.21.15.50:0/327896561 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18b7400640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18b80749a0 0x7f18b813d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18aea00640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18b80cefa0 0x7f18b81faa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18b7e00640 1 -- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18b80cefa0 msgr2=0x7f18b81faa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18b7e00640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18b80cefa0 0x7f18b81faa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18b7e00640 1 -- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18b80749a0 msgr2=0x7f18b813d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18b7e00640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18b80749a0 0x7f18b813d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18b7e00640 1 -- 172.21.15.50:0/327896561 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b80ab810 con 0x7f18b8160330 2024-09-21T06:12:34.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18b7400640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18b80749a0 0x7f18b813d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:34.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18aea00640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18b80cefa0 0x7f18b81faa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:34.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18b7e00640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18b8160330 0x7f18b8141370 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f18b000c760 tx=0x7f18b000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:34.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18afe00640 1 -- 172.21.15.50:0/327896561 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0003db0 con 0x7f18b8160330 2024-09-21T06:12:34.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18afe00640 1 -- 172.21.15.50:0/327896561 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18b000ce10 con 0x7f18b8160330 2024-09-21T06:12:34.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18be200640 1 -- 172.21.15.50:0/327896561 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b81faf80 con 0x7f18b8160330 2024-09-21T06:12:34.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18afe00640 1 -- 172.21.15.50:0/327896561 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b00052d0 con 0x7f18b8160330 2024-09-21T06:12:34.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18be200640 1 -- 172.21.15.50:0/327896561 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b8143780 con 0x7f18b8160330 2024-09-21T06:12:34.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.306+0000 7f18be200640 1 -- 172.21.15.50:0/327896561 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b8144ad0 con 0x7f18b8160330 2024-09-21T06:12:34.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.310+0000 7f18afe00640 1 -- 172.21.15.50:0/327896561 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f18b0005470 con 0x7f18b8160330 2024-09-21T06:12:34.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.310+0000 7f18afe00640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18940754e0 0x7f18940779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:34.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.310+0000 7f18afe00640 1 -- 172.21.15.50:0/327896561 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f18b0093180 con 0x7f18b8160330 2024-09-21T06:12:34.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.310+0000 7f18b7400640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18940754e0 0x7f18940779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:34.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.310+0000 7f18b7400640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18940754e0 0x7f18940779f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f18a8005fd0 tx=0x7f18a8005dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:34.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.314+0000 7f18afe00640 1 -- 172.21.15.50:0/327896561 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f18b81faf80 con 0x7f18b8160330 2024-09-21T06:12:34.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.490+0000 7f18be200640 1 -- 172.21.15.50:0/327896561 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f18b8074db0 con 0x7f18b8160330 2024-09-21T06:12:34.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.490+0000 7f18afe00640 1 -- 172.21.15.50:0/327896561 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) ==== 122+0+0 (secure 0 0 0) 0x7f18b80ab810 con 0x7f18b8160330 2024-09-21T06:12:34.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.494+0000 7f18be200640 1 -- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18940754e0 msgr2=0x7f18940779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:34.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.494+0000 7f18be200640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18940754e0 0x7f18940779f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f18a8005fd0 tx=0x7f18a8005dc0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.494+0000 7f18be200640 1 -- 172.21.15.50:0/327896561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18b8160330 msgr2=0x7f18b8141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:34.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.494+0000 7f18be200640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18b8160330 0x7f18b8141370 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f18b000c760 tx=0x7f18b000cc20 comp rx=0 tx=0).stop 2024-09-21T06:12:34.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.494+0000 7f18be200640 1 -- 172.21.15.50:0/327896561 shutdown_connections 2024-09-21T06:12:34.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.494+0000 7f18be200640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f18940754e0 0x7f18940779f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.494+0000 7f18be200640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18b8160330 0x7f18b8141370 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.494+0000 7f18be200640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f18b80cefa0 0x7f18b81faa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.494+0000 7f18be200640 1 --2- 172.21.15.50:0/327896561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18b80749a0 0x7f18b813d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.494+0000 7f18be200640 1 -- 172.21.15.50:0/327896561 >> 172.21.15.50:0/327896561 conn(0x7f18b80b2040 msgr2=0x7f18b815b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:34.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.494+0000 7f18be200640 1 -- 172.21.15.50:0/327896561 shutdown_connections 2024-09-21T06:12:34.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.494+0000 7f18be200640 1 -- 172.21.15.50:0/327896561 wait complete. 2024-09-21T06:12:34.513 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-21T06:12:34.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.674+0000 7ff738200640 1 -- 172.21.15.50:0/1472793523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7300749a0 msgr2=0x7ff730074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:34.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.674+0000 7ff738200640 1 --2- 172.21.15.50:0/1472793523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7300749a0 0x7ff730074db0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff7300cfc90 tx=0x7ff72401c8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.674+0000 7ff738200640 1 -- 172.21.15.50:0/1472793523 shutdown_connections 2024-09-21T06:12:34.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.674+0000 7ff738200640 1 --2- 172.21.15.50:0/1472793523 >> v2:172.21.15.88:3300/0 conn(0x7ff7300b3340 0x7ff7300b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.674+0000 7ff738200640 1 --2- 172.21.15.50:0/1472793523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7300749a0 0x7ff730074db0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.674+0000 7ff738200640 1 -- 172.21.15.50:0/1472793523 >> 172.21.15.50:0/1472793523 conn(0x7ff7300b2040 msgr2=0x7ff7300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:34.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.674+0000 7ff738200640 1 -- 172.21.15.50:0/1472793523 shutdown_connections 2024-09-21T06:12:34.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.674+0000 7ff738200640 1 -- 172.21.15.50:0/1472793523 wait complete. 2024-09-21T06:12:34.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff738200640 1 Processor -- start 2024-09-21T06:12:34.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff738200640 1 -- start start 2024-09-21T06:12:34.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff738200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7300749a0 0x7ff73013e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:34.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff738200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff7300b3340 0x7ff730141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:34.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff738200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7300cefa0 0x7ff73015ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:34.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff738200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff730169d20 con 0x7ff7300cefa0 2024-09-21T06:12:34.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff738200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff730169ba0 con 0x7ff7300749a0 2024-09-21T06:12:34.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff738200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff730169ea0 con 0x7ff7300b3340 2024-09-21T06:12:34.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff736200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7300cefa0 0x7ff73015ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:34.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff736200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7300cefa0 0x7ff73015ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53522/0 (socket says 172.21.15.50:53522) 2024-09-21T06:12:34.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff736200640 1 -- 172.21.15.50:0/3592149973 learned_addr learned my addr 172.21.15.50:0/3592149973 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:34.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff734e00640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff7300b3340 0x7ff730141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:34.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff735800640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7300749a0 0x7ff73013e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:34.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff736200640 1 -- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff7300b3340 msgr2=0x7ff730141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:34.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff736200640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff7300b3340 0x7ff730141840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff736200640 1 -- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7300749a0 msgr2=0x7ff73013e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:34.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff736200640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7300749a0 0x7ff73013e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff736200640 1 -- 172.21.15.50:0/3592149973 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73013f1b0 con 0x7ff7300cefa0 2024-09-21T06:12:34.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff735800640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7300749a0 0x7ff73013e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:34.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff736200640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7300cefa0 0x7ff73015ef30 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ff72800e970 tx=0x7ff72800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:34.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff72d600640 1 -- 172.21.15.50:0/3592149973 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728004d30 con 0x7ff7300cefa0 2024-09-21T06:12:34.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff72d600640 1 -- 172.21.15.50:0/3592149973 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff72800cb20 con 0x7ff7300cefa0 2024-09-21T06:12:34.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff72d600640 1 -- 172.21.15.50:0/3592149973 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728005020 con 0x7ff7300cefa0 2024-09-21T06:12:34.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff738200640 1 -- 172.21.15.50:0/3592149973 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7301423b0 con 0x7ff7300cefa0 2024-09-21T06:12:34.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.678+0000 7ff738200640 1 -- 172.21.15.50:0/3592149973 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7301407c0 con 0x7ff7300cefa0 2024-09-21T06:12:34.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.682+0000 7ff738200640 1 -- 172.21.15.50:0/3592149973 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff730142120 con 0x7ff7300cefa0 2024-09-21T06:12:34.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.682+0000 7ff72d600640 1 -- 172.21.15.50:0/3592149973 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7ff7280051c0 con 0x7ff7300cefa0 2024-09-21T06:12:34.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.682+0000 7ff72d600640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff7100754e0 0x7ff7100779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:34.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.682+0000 7ff72d600640 1 -- 172.21.15.50:0/3592149973 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7ff728093f60 con 0x7ff7300cefa0 2024-09-21T06:12:34.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.686+0000 7ff735800640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff7100754e0 0x7ff7100779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:34.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.686+0000 7ff735800640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff7100754e0 0x7ff7100779f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff7240af000 tx=0x7ff72401b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:34.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.686+0000 7ff72d600640 1 -- 172.21.15.50:0/3592149973 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff728016030 con 0x7ff7300cefa0 2024-09-21T06:12:34.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.862+0000 7ff738200640 1 -- 172.21.15.50:0/3592149973 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7ff730074db0 con 0x7ff7300cefa0 2024-09-21T06:12:34.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.862+0000 7ff72d600640 1 -- 172.21.15.50:0/3592149973 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v58) ==== 116+0+2 (secure 0 0 0) 0x7ff728060bf0 con 0x7ff7300cefa0 2024-09-21T06:12:34.863 INFO:teuthology.orchestra.run.smithi050.stdout:4 2024-09-21T06:12:34.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.866+0000 7ff738200640 1 -- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff7100754e0 msgr2=0x7ff7100779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:34.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.866+0000 7ff738200640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff7100754e0 0x7ff7100779f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff7240af000 tx=0x7ff72401b100 comp rx=0 tx=0).stop 2024-09-21T06:12:34.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.866+0000 7ff738200640 1 -- 172.21.15.50:0/3592149973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7300cefa0 msgr2=0x7ff73015ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:34.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.866+0000 7ff738200640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7300cefa0 0x7ff73015ef30 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ff72800e970 tx=0x7ff72800ee30 comp rx=0 tx=0).stop 2024-09-21T06:12:34.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.866+0000 7ff738200640 1 -- 172.21.15.50:0/3592149973 shutdown_connections 2024-09-21T06:12:34.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.866+0000 7ff738200640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff7100754e0 0x7ff7100779f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.866+0000 7ff738200640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7300cefa0 0x7ff73015ef30 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.866+0000 7ff738200640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff7300b3340 0x7ff730141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.866+0000 7ff738200640 1 --2- 172.21.15.50:0/3592149973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7300749a0 0x7ff73013e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:34.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.866+0000 7ff738200640 1 -- 172.21.15.50:0/3592149973 >> 172.21.15.50:0/3592149973 conn(0x7ff7300b2040 msgr2=0x7ff7300ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:34.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.866+0000 7ff738200640 1 -- 172.21.15.50:0/3592149973 shutdown_connections 2024-09-21T06:12:34.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:34.866+0000 7ff738200640 1 -- 172.21.15.50:0/3592149973 wait complete. 2024-09-21T06:12:34.886 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:12:35.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.050+0000 7fcc3a200640 1 -- 172.21.15.50:0/298583472 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc340b4de0 msgr2=0x7fcc340b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.050+0000 7fcc3a200640 1 --2- 172.21.15.50:0/298583472 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc340b4de0 0x7fcc340b51f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009c80 tx=0x7fcc2c01cd70 comp rx=0 tx=0).stop 2024-09-21T06:12:35.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.050+0000 7fcc3a200640 1 -- 172.21.15.50:0/298583472 shutdown_connections 2024-09-21T06:12:35.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.050+0000 7fcc3a200640 1 --2- 172.21.15.50:0/298583472 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc340b4de0 0x7fcc340b51f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.050+0000 7fcc3a200640 1 --2- 172.21.15.50:0/298583472 >> v2:172.21.15.88:3300/0 conn(0x7fcc340b5c70 0x7fcc3415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.050+0000 7fcc3a200640 1 -- 172.21.15.50:0/298583472 >> 172.21.15.50:0/298583472 conn(0x7fcc340b2040 msgr2=0x7fcc340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:35.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.050+0000 7fcc3a200640 1 -- 172.21.15.50:0/298583472 shutdown_connections 2024-09-21T06:12:35.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.050+0000 7fcc3a200640 1 -- 172.21.15.50:0/298583472 wait complete. 2024-09-21T06:12:35.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.050+0000 7fcc3a200640 1 Processor -- start 2024-09-21T06:12:35.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.050+0000 7fcc3a200640 1 -- start start 2024-09-21T06:12:35.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc3a200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc340b4de0 0x7fcc341499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc3a200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc340b5c70 0x7fcc340ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc3a200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc341527b0 0x7fcc3415a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc3a200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc34168920 con 0x7fcc340b4de0 2024-09-21T06:12:35.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc3a200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcc341687a0 con 0x7fcc341527b0 2024-09-21T06:12:35.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc3a200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fcc34168aa0 con 0x7fcc340b5c70 2024-09-21T06:12:35.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc33400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc340b4de0 0x7fcc341499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc33400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc340b4de0 0x7fcc341499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53538/0 (socket says 172.21.15.50:53538) 2024-09-21T06:12:35.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc33400640 1 -- 172.21.15.50:0/3736197418 learned_addr learned my addr 172.21.15.50:0/3736197418 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:35.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc33e00640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc341527b0 0x7fcc3415a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc2be00640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc340b5c70 0x7fcc340ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc33400640 1 -- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc340b5c70 msgr2=0x7fcc340ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc33400640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc340b5c70 0x7fcc340ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc33400640 1 -- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc341527b0 msgr2=0x7fcc3415a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc33400640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc341527b0 0x7fcc3415a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc33400640 1 -- 172.21.15.50:0/3736197418 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc340cf040 con 0x7fcc340b4de0 2024-09-21T06:12:35.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc33e00640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc341527b0 0x7fcc3415a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:35.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc2be00640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc340b5c70 0x7fcc340ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:35.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc33400640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc340b4de0 0x7fcc341499b0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400e970 tx=0x7fcc2400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:35.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc2b400640 1 -- 172.21.15.50:0/3736197418 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400cd20 con 0x7fcc340b4de0 2024-09-21T06:12:35.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc2b400640 1 -- 172.21.15.50:0/3736197418 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc24004020 con 0x7fcc340b4de0 2024-09-21T06:12:35.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc3a200640 1 -- 172.21.15.50:0/3736197418 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc341f69b0 con 0x7fcc340b4de0 2024-09-21T06:12:35.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc2b400640 1 -- 172.21.15.50:0/3736197418 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24004450 con 0x7fcc340b4de0 2024-09-21T06:12:35.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.054+0000 7fcc3a200640 1 -- 172.21.15.50:0/3736197418 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc341f6f70 con 0x7fcc340b4de0 2024-09-21T06:12:35.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.058+0000 7fcc3a200640 1 -- 172.21.15.50:0/3736197418 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc341f67c0 con 0x7fcc340b4de0 2024-09-21T06:12:35.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.058+0000 7fcc2b400640 1 -- 172.21.15.50:0/3736197418 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7fcc24003600 con 0x7fcc340b4de0 2024-09-21T06:12:35.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.058+0000 7fcc2b400640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc10075510 0x7fcc10077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.058+0000 7fcc2b400640 1 -- 172.21.15.50:0/3736197418 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7fcc24093c80 con 0x7fcc340b4de0 2024-09-21T06:12:35.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.058+0000 7fcc2be00640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc10075510 0x7fcc10077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.058+0000 7fcc2be00640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc10075510 0x7fcc10077a20 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c001400 tx=0x7fcc1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:35.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.062+0000 7fcc2b400640 1 -- 172.21.15.50:0/3736197418 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcc24060780 con 0x7fcc340b4de0 2024-09-21T06:12:35.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.238+0000 7fcc3a200640 1 -- 172.21.15.50:0/3736197418 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fcc340b51f0 con 0x7fcc340b4de0 2024-09-21T06:12:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.242+0000 7fcc2b400640 1 -- 172.21.15.50:0/3736197418 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v59)=0 v59) ==== 131+0+0 (secure 0 0 0) 0x7fcc24060080 con 0x7fcc340b4de0 2024-09-21T06:12:35.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.246+0000 7fcc3a200640 1 -- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc10075510 msgr2=0x7fcc10077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.246+0000 7fcc3a200640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc10075510 0x7fcc10077a20 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c001400 tx=0x7fcc1c009040 comp rx=0 tx=0).stop 2024-09-21T06:12:35.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.246+0000 7fcc3a200640 1 -- 172.21.15.50:0/3736197418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc340b4de0 msgr2=0x7fcc341499b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.246+0000 7fcc3a200640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc340b4de0 0x7fcc341499b0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400e970 tx=0x7fcc2400ee30 comp rx=0 tx=0).stop 2024-09-21T06:12:35.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.246+0000 7fcc3a200640 1 -- 172.21.15.50:0/3736197418 shutdown_connections 2024-09-21T06:12:35.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.246+0000 7fcc3a200640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc341527b0 0x7fcc3415a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.246+0000 7fcc3a200640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fcc340b5c70 0x7fcc340ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.246+0000 7fcc3a200640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fcc10075510 0x7fcc10077a20 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.246+0000 7fcc3a200640 1 --2- 172.21.15.50:0/3736197418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc340b4de0 0x7fcc341499b0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.246+0000 7fcc3a200640 1 -- 172.21.15.50:0/3736197418 >> 172.21.15.50:0/3736197418 conn(0x7fcc340b2040 msgr2=0x7fcc34150140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:35.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.246+0000 7fcc3a200640 1 -- 172.21.15.50:0/3736197418 shutdown_connections 2024-09-21T06:12:35.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.246+0000 7fcc3a200640 1 -- 172.21.15.50:0/3736197418 wait complete. 2024-09-21T06:12:35.264 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-21T06:12:35.265 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:35.263+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:35.265 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:35.263+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:35.265 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:35.263+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:35.265 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:35.263+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:35.265 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:35.263+0000 7f65a3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:35.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.430+0000 7f80eba00640 1 -- 172.21.15.50:0/2289859592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80cc003a10 msgr2=0x7f80cc003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.430+0000 7f80eba00640 1 --2- 172.21.15.50:0/2289859592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80cc003a10 0x7f80cc003e60 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f80d8001770 tx=0x7f80d801c830 comp rx=0 tx=0).stop 2024-09-21T06:12:35.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.430+0000 7f80eba00640 1 -- 172.21.15.50:0/2289859592 shutdown_connections 2024-09-21T06:12:35.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.430+0000 7f80eba00640 1 --2- 172.21.15.50:0/2289859592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80cc003a10 0x7f80cc003e60 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.430+0000 7f80eba00640 1 -- 172.21.15.50:0/2289859592 >> 172.21.15.50:0/2289859592 conn(0x7f80e40b2040 msgr2=0x7f80e40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:35.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.430+0000 7f80eba00640 1 -- 172.21.15.50:0/2289859592 shutdown_connections 2024-09-21T06:12:35.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.430+0000 7f80eba00640 1 -- 172.21.15.50:0/2289859592 wait complete. 2024-09-21T06:12:35.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.430+0000 7f80eba00640 1 Processor -- start 2024-09-21T06:12:35.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80eba00640 1 -- start start 2024-09-21T06:12:35.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80eba00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80cc003a10 0x7f80e414fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80eba00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e4159120 0x7f80e4150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80eba00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80e4151070 0x7f80e40ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80eba00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f80e40b5690 con 0x7f80e4159120 2024-09-21T06:12:35.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80eba00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f80e40b5510 con 0x7f80cc003a10 2024-09-21T06:12:35.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80eba00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f80e415f180 con 0x7f80e4151070 2024-09-21T06:12:35.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e4159120 0x7f80e4150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e4159120 0x7f80e4150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53558/0 (socket says 172.21.15.50:53558) 2024-09-21T06:12:35.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e3e00640 1 -- 172.21.15.50:0/3232628781 learned_addr learned my addr 172.21.15.50:0/3232628781 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e9a00640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80e4151070 0x7f80e40ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e9000640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80cc003a10 0x7f80e414fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e3e00640 1 -- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80e4151070 msgr2=0x7f80e40ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e3e00640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80e4151070 0x7f80e40ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e3e00640 1 -- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80cc003a10 msgr2=0x7f80e414fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e3e00640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80cc003a10 0x7f80e414fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e3e00640 1 -- 172.21.15.50:0/3232628781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e41f9040 con 0x7f80e4159120 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e9a00640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80e4151070 0x7f80e40ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e9000640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80cc003a10 0x7f80e414fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e3e00640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e4159120 0x7f80e4150b30 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f80d00117b0 tx=0x7f80d0011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e0c00640 1 -- 172.21.15.50:0/3232628781 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d001f070 con 0x7f80e4159120 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e0c00640 1 -- 172.21.15.50:0/3232628781 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80d000fe40 con 0x7f80e4159120 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80e0c00640 1 -- 172.21.15.50:0/3232628781 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d00126b0 con 0x7f80e4159120 2024-09-21T06:12:35.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80eba00640 1 -- 172.21.15.50:0/3232628781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e4151480 con 0x7f80e4159120 2024-09-21T06:12:35.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80eba00640 1 -- 172.21.15.50:0/3232628781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e41f9590 con 0x7f80e4159120 2024-09-21T06:12:35.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.434+0000 7f80eba00640 1 -- 172.21.15.50:0/3232628781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e4156dd0 con 0x7f80e4159120 2024-09-21T06:12:35.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.438+0000 7f80e0c00640 1 -- 172.21.15.50:0/3232628781 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f80d0022070 con 0x7f80e4159120 2024-09-21T06:12:35.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.438+0000 7f80e0c00640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80c40754e0 0x7f80c40779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.438+0000 7f80e0c00640 1 -- 172.21.15.50:0/3232628781 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f80d0093d90 con 0x7f80e4159120 2024-09-21T06:12:35.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.438+0000 7f80e9000640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80c40754e0 0x7f80c40779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.438+0000 7f80e9000640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80c40754e0 0x7f80c40779f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f80d8005000 tx=0x7f80d80195d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:35.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.442+0000 7f80e0c00640 1 -- 172.21.15.50:0/3232628781 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f80e4151480 con 0x7f80e4159120 2024-09-21T06:12:35.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.618+0000 7f80eba00640 1 -- 172.21.15.50:0/3232628781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f80e415f670 con 0x7f80e4159120 2024-09-21T06:12:35.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.622+0000 7f80e0c00640 1 -- 172.21.15.50:0/3232628781 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v60)=0 v60) ==== 136+0+0 (secure 0 0 0) 0x7f80d0060250 con 0x7f80e4159120 2024-09-21T06:12:35.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.626+0000 7f80eba00640 1 -- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80c40754e0 msgr2=0x7f80c40779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.626+0000 7f80eba00640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80c40754e0 0x7f80c40779f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f80d8005000 tx=0x7f80d80195d0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.626+0000 7f80eba00640 1 -- 172.21.15.50:0/3232628781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e4159120 msgr2=0x7f80e4150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.626+0000 7f80eba00640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e4159120 0x7f80e4150b30 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f80d00117b0 tx=0x7f80d0011c70 comp rx=0 tx=0).stop 2024-09-21T06:12:35.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.626+0000 7f80eba00640 1 -- 172.21.15.50:0/3232628781 shutdown_connections 2024-09-21T06:12:35.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.626+0000 7f80eba00640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80e4151070 0x7f80e40ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.626+0000 7f80eba00640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f80c40754e0 0x7f80c40779f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.626+0000 7f80eba00640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80e4159120 0x7f80e4150b30 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.626+0000 7f80eba00640 1 --2- 172.21.15.50:0/3232628781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80cc003a10 0x7f80e414fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.626+0000 7f80eba00640 1 -- 172.21.15.50:0/3232628781 >> 172.21.15.50:0/3232628781 conn(0x7f80e40b2040 msgr2=0x7f80e415b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:35.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.626+0000 7f80eba00640 1 -- 172.21.15.50:0/3232628781 shutdown_connections 2024-09-21T06:12:35.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.626+0000 7f80eba00640 1 -- 172.21.15.50:0/3232628781 wait complete. 2024-09-21T06:12:35.645 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-21T06:12:35.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.810+0000 7f284e600640 1 -- 172.21.15.50:0/2710527821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f284815c5d0 msgr2=0x7f2848073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.810+0000 7f284e600640 1 --2- 172.21.15.50:0/2710527821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f284815c5d0 0x7f2848073dc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2838009900 tx=0x7f283801c850 comp rx=0 tx=0).stop 2024-09-21T06:12:35.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.810+0000 7f284e600640 1 -- 172.21.15.50:0/2710527821 shutdown_connections 2024-09-21T06:12:35.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.810+0000 7f284e600640 1 --2- 172.21.15.50:0/2710527821 >> v2:172.21.15.88:3300/0 conn(0x7f2848074300 0x7f284815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.810+0000 7f284e600640 1 --2- 172.21.15.50:0/2710527821 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f284815c5d0 0x7f2848073dc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.810+0000 7f284e600640 1 -- 172.21.15.50:0/2710527821 >> 172.21.15.50:0/2710527821 conn(0x7f28480b2c40 msgr2=0x7f28480b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:35.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.810+0000 7f284e600640 1 -- 172.21.15.50:0/2710527821 shutdown_connections 2024-09-21T06:12:35.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.810+0000 7f284e600640 1 -- 172.21.15.50:0/2710527821 wait complete. 2024-09-21T06:12:35.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f284e600640 1 Processor -- start 2024-09-21T06:12:35.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f284e600640 1 -- start start 2024-09-21T06:12:35.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f284e600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2848074300 0x7f28480cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f284e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284815c5d0 0x7f28481f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f284e600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f284815f6b0 0x7f2848152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f284e600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f284806f980 con 0x7f284815c5d0 2024-09-21T06:12:35.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f284e600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28480b3d50 con 0x7f2848074300 2024-09-21T06:12:35.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f284e600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f284806fb00 con 0x7f284815f6b0 2024-09-21T06:12:35.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f283ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284815c5d0 0x7f28481f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f283ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284815c5d0 0x7f28481f97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53562/0 (socket says 172.21.15.50:53562) 2024-09-21T06:12:35.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f2847400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2848074300 0x7f28480cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f283ea00640 1 -- 172.21.15.50:0/4282287922 learned_addr learned my addr 172.21.15.50:0/4282287922 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:35.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f2847e00640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f284815f6b0 0x7f2848152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f283ea00640 1 -- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f284815f6b0 msgr2=0x7f2848152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f283ea00640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f284815f6b0 0x7f2848152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f283ea00640 1 -- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2848074300 msgr2=0x7f28480cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f283ea00640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2848074300 0x7f28480cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.814+0000 7f283ea00640 1 -- 172.21.15.50:0/4282287922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28481f9d30 con 0x7f284815c5d0 2024-09-21T06:12:35.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.818+0000 7f283ea00640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284815c5d0 0x7f28481f97f0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f283000e7c0 tx=0x7f283000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:35.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.818+0000 7f283fe00640 1 -- 172.21.15.50:0/4282287922 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283000ccc0 con 0x7f284815c5d0 2024-09-21T06:12:35.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.818+0000 7f283fe00640 1 -- 172.21.15.50:0/4282287922 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f283000ee70 con 0x7f284815c5d0 2024-09-21T06:12:35.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.818+0000 7f284e600640 1 -- 172.21.15.50:0/4282287922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284814a1a0 con 0x7f284815c5d0 2024-09-21T06:12:35.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.818+0000 7f283fe00640 1 -- 172.21.15.50:0/4282287922 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830004270 con 0x7f284815c5d0 2024-09-21T06:12:35.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.818+0000 7f284e600640 1 -- 172.21.15.50:0/4282287922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284814a760 con 0x7f284815c5d0 2024-09-21T06:12:35.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.818+0000 7f284e600640 1 -- 172.21.15.50:0/4282287922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284815f170 con 0x7f284815c5d0 2024-09-21T06:12:35.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.822+0000 7f283fe00640 1 -- 172.21.15.50:0/4282287922 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 55) ==== 100396+0+0 (secure 0 0 0) 0x7f2830003600 con 0x7f284815c5d0 2024-09-21T06:12:35.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.822+0000 7f283fe00640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2824075510 0x7f2824077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.822+0000 7f283fe00640 1 -- 172.21.15.50:0/4282287922 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f2830093fd0 con 0x7f284815c5d0 2024-09-21T06:12:35.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.822+0000 7f2847400640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2824075510 0x7f2824077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.822+0000 7f2847400640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2824075510 0x7f2824077a20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f28380af000 tx=0x7f283801b0f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:35.822+0000 7f283fe00640 1 -- 172.21.15.50:0/4282287922 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f28481f9d30 con 0x7f284815c5d0 2024-09-21T06:12:36.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.026+0000 7f284e600640 1 -- 172.21.15.50:0/4282287922 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f284806b540 con 0x7f2824075510 2024-09-21T06:12:36.036 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:36.035+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:36.037 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:36.035+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:36.037 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:36.035+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:36.037 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:36.035+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:36.037 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:12:36.035+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:36.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f283fe00640 1 -- 172.21.15.50:0/4282287922 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f28380a5a00 con 0x7f2824075510 2024-09-21T06:12:36.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f284e600640 1 -- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2824075510 msgr2=0x7f2824077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:36.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f284e600640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2824075510 0x7f2824077a20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f28380af000 tx=0x7f283801b0f0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f284e600640 1 -- 172.21.15.50:0/4282287922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284815c5d0 msgr2=0x7f28481f97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:36.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f284e600640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284815c5d0 0x7f28481f97f0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f283000e7c0 tx=0x7f283000ec80 comp rx=0 tx=0).stop 2024-09-21T06:12:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f284e600640 1 -- 172.21.15.50:0/4282287922 shutdown_connections 2024-09-21T06:12:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f284e600640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f284815f6b0 0x7f2848152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f284e600640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2824075510 0x7f2824077a20 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f284e600640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284815c5d0 0x7f28481f97f0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f284e600640 1 --2- 172.21.15.50:0/4282287922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2848074300 0x7f28480cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f284e600640 1 -- 172.21.15.50:0/4282287922 >> 172.21.15.50:0/4282287922 conn(0x7f28480b2c40 msgr2=0x7f28480b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f284e600640 1 -- 172.21.15.50:0/4282287922 shutdown_connections 2024-09-21T06:12:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.078+0000 7f284e600640 1 -- 172.21.15.50:0/4282287922 wait complete. 2024-09-21T06:12:36.098 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:12:36.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.266+0000 7ff1bb200640 1 -- 172.21.15.50:0/539781120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1bc0b3840 msgr2=0x7ff1bc072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:36.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.266+0000 7ff1bb200640 1 --2- 172.21.15.50:0/539781120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1bc0b3840 0x7ff1bc072030 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc071ae0 tx=0x7ff1b401cd50 comp rx=0 tx=0).stop 2024-09-21T06:12:36.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.266+0000 7ff1bb200640 1 -- 172.21.15.50:0/539781120 shutdown_connections 2024-09-21T06:12:36.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.266+0000 7ff1bb200640 1 --2- 172.21.15.50:0/539781120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1bc0b3840 0x7ff1bc072030 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.266+0000 7ff1bb200640 1 --2- 172.21.15.50:0/539781120 >> v2:172.21.15.88:3300/0 conn(0x7ff1bc0b5040 0x7ff1bc0cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.266+0000 7ff1bb200640 1 -- 172.21.15.50:0/539781120 >> 172.21.15.50:0/539781120 conn(0x7ff1bc06cdf0 msgr2=0x7ff1bc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:36.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.266+0000 7ff1bb200640 1 -- 172.21.15.50:0/539781120 shutdown_connections 2024-09-21T06:12:36.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.266+0000 7ff1bb200640 1 -- 172.21.15.50:0/539781120 wait complete. 2024-09-21T06:12:36.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.266+0000 7ff1bb200640 1 Processor -- start 2024-09-21T06:12:36.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.266+0000 7ff1bb200640 1 -- start start 2024-09-21T06:12:36.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1bb200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1bc0b3840 0x7ff1bc13c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:36.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1bb200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff1bc0b5040 0x7ff1bc0aa610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:36.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1bb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1bc0d0380 0x7ff1bc140360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:36.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1bb200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc167d50 con 0x7ff1bc0b3840 2024-09-21T06:12:36.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1bb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc167bd0 con 0x7ff1bc0d0380 2024-09-21T06:12:36.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1bb200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff1bc167ed0 con 0x7ff1bc0b5040 2024-09-21T06:12:36.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1bc0b3840 0x7ff1bc13c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:36.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1bc0b3840 0x7ff1bc13c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53582/0 (socket says 172.21.15.50:53582) 2024-09-21T06:12:36.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b9e00640 1 -- 172.21.15.50:0/1772919403 learned_addr learned my addr 172.21.15.50:0/1772919403 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:36.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b3e00640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff1bc0b5040 0x7ff1bc0aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:36.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1ba800640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1bc0d0380 0x7ff1bc140360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:36.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b9e00640 1 -- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff1bc0b5040 msgr2=0x7ff1bc0aa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:36.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b9e00640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff1bc0b5040 0x7ff1bc0aa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b9e00640 1 -- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1bc0d0380 msgr2=0x7ff1bc140360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:36.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b9e00640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1bc0d0380 0x7ff1bc140360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b9e00640 1 -- 172.21.15.50:0/1772919403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc1f6ae0 con 0x7ff1bc0b3840 2024-09-21T06:12:36.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b9e00640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1bc0b3840 0x7ff1bc13c800 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00dba0 tx=0x7ff1ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:36.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b2000640 1 -- 172.21.15.50:0/1772919403 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac004020 con 0x7ff1bc0b3840 2024-09-21T06:12:36.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b2000640 1 -- 172.21.15.50:0/1772919403 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1ac0042b0 con 0x7ff1bc0b3840 2024-09-21T06:12:36.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1bb200640 1 -- 172.21.15.50:0/1772919403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1bc1f6dc0 con 0x7ff1bc0b3840 2024-09-21T06:12:36.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1b2000640 1 -- 172.21.15.50:0/1772919403 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac005280 con 0x7ff1bc0b3840 2024-09-21T06:12:36.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1bb200640 1 -- 172.21.15.50:0/1772919403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1bc1f7380 con 0x7ff1bc0b3840 2024-09-21T06:12:36.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.270+0000 7ff1bb200640 1 -- 172.21.15.50:0/1772919403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1bc1f7760 con 0x7ff1bc0b3840 2024-09-21T06:12:36.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.274+0000 7ff1b2000640 1 -- 172.21.15.50:0/1772919403 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 56) ==== 100447+0+0 (secure 0 0 0) 0x7ff1ac01d020 con 0x7ff1bc0b3840 2024-09-21T06:12:36.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.274+0000 7ff1b2000640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff198075570 0x7ff198077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:36.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.274+0000 7ff1b2000640 1 -- 172.21.15.50:0/1772919403 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7ff1ac093750 con 0x7ff1bc0b3840 2024-09-21T06:12:36.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.274+0000 7ff1b3e00640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff198075570 0x7ff198077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:36.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.274+0000 7ff1b3e00640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff198075570 0x7ff198077a80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0735c0 tx=0x7ff1a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:36.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.278+0000 7ff1b2000640 1 -- 172.21.15.50:0/1772919403 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff1bc1f7760 con 0x7ff1bc0b3840 2024-09-21T06:12:36.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.490+0000 7ff1bb200640 1 -- 172.21.15.50:0/1772919403 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7ff1bc06b540 con 0x7ff198075570 2024-09-21T06:12:36.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.498+0000 7ff1b2000640 1 -- 172.21.15.50:0/1772919403 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7ff1a40050d0 con 0x7ff198075570 2024-09-21T06:12:36.497 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/0078a113-7228-4587-9847-47491cfce39e 2024-09-21T06:12:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.502+0000 7ff1bb200640 1 -- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff198075570 msgr2=0x7ff198077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.502+0000 7ff1bb200640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff198075570 0x7ff198077a80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0735c0 tx=0x7ff1a4009040 comp rx=0 tx=0).stop 2024-09-21T06:12:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.502+0000 7ff1bb200640 1 -- 172.21.15.50:0/1772919403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1bc0b3840 msgr2=0x7ff1bc13c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.502+0000 7ff1bb200640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1bc0b3840 0x7ff1bc13c800 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00dba0 tx=0x7ff1ac00a620 comp rx=0 tx=0).stop 2024-09-21T06:12:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.502+0000 7ff1bb200640 1 -- 172.21.15.50:0/1772919403 shutdown_connections 2024-09-21T06:12:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.502+0000 7ff1bb200640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1bc0d0380 0x7ff1bc140360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.502+0000 7ff1bb200640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff1bc0b5040 0x7ff1bc0aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.502+0000 7ff1bb200640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff198075570 0x7ff198077a80 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.502+0000 7ff1bb200640 1 --2- 172.21.15.50:0/1772919403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1bc0b3840 0x7ff1bc13c800 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.502+0000 7ff1bb200640 1 -- 172.21.15.50:0/1772919403 >> 172.21.15.50:0/1772919403 conn(0x7ff1bc06cdf0 msgr2=0x7ff1bc0b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:36.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.502+0000 7ff1bb200640 1 -- 172.21.15.50:0/1772919403 shutdown_connections 2024-09-21T06:12:36.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.502+0000 7ff1bb200640 1 -- 172.21.15.50:0/1772919403 wait complete. 2024-09-21T06:12:36.521 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:12:36.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.682+0000 7fae2ac00640 1 -- 172.21.15.50:0/507742911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2415c730 msgr2=0x7fae240b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:36.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.682+0000 7fae2ac00640 1 --2- 172.21.15.50:0/507742911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2415c730 0x7fae240b3710 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fae14009900 tx=0x7fae1401c8e0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.682+0000 7fae2ac00640 1 -- 172.21.15.50:0/507742911 shutdown_connections 2024-09-21T06:12:36.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.682+0000 7fae2ac00640 1 --2- 172.21.15.50:0/507742911 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fae240af520 0x7fae240af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.682+0000 7fae2ac00640 1 --2- 172.21.15.50:0/507742911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2415c730 0x7fae240b3710 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.682+0000 7fae2ac00640 1 -- 172.21.15.50:0/507742911 >> 172.21.15.50:0/507742911 conn(0x7fae240d06e0 msgr2=0x7fae240d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:36.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.686+0000 7fae2ac00640 1 -- 172.21.15.50:0/507742911 shutdown_connections 2024-09-21T06:12:36.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.686+0000 7fae2ac00640 1 -- 172.21.15.50:0/507742911 wait complete. 2024-09-21T06:12:36.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.686+0000 7fae2ac00640 1 Processor -- start 2024-09-21T06:12:36.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.686+0000 7fae2ac00640 1 -- start start 2024-09-21T06:12:36.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.686+0000 7fae2ac00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fae24073f90 0x7fae241fce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:36.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.686+0000 7fae2ac00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae240af520 0x7fae241602b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:36.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.686+0000 7fae2ac00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2415c730 0x7fae24166680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:36.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.686+0000 7fae2ac00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fae2406fba0 con 0x7fae2415c730 2024-09-21T06:12:36.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.686+0000 7fae2ac00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae2406fa20 con 0x7fae240af520 2024-09-21T06:12:36.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.686+0000 7fae2ac00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fae2406fd20 con 0x7fae24073f90 2024-09-21T06:12:36.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae28c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2415c730 0x7fae24166680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:36.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae28c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2415c730 0x7fae24166680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43824/0 (socket says 172.21.15.50:43824) 2024-09-21T06:12:36.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae28c00640 1 -- 172.21.15.50:0/3170363765 learned_addr learned my addr 172.21.15.50:0/3170363765 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:12:36.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae23e00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fae24073f90 0x7fae241fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:36.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae1be00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae240af520 0x7fae241602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:36.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae28c00640 1 -- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fae24073f90 msgr2=0x7fae241fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:36.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae28c00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fae24073f90 0x7fae241fce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae28c00640 1 -- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae240af520 msgr2=0x7fae241602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:36.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae28c00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae240af520 0x7fae241602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae28c00640 1 -- 172.21.15.50:0/3170363765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae24158430 con 0x7fae2415c730 2024-09-21T06:12:36.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae28c00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2415c730 0x7fae24166680 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00e970 tx=0x7fae1c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:36.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae20c00640 1 -- 172.21.15.50:0/3170363765 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c004d30 con 0x7fae2415c730 2024-09-21T06:12:36.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae20c00640 1 -- 172.21.15.50:0/3170363765 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae1c00cb20 con 0x7fae2415c730 2024-09-21T06:12:36.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae2ac00640 1 -- 172.21.15.50:0/3170363765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae24157670 con 0x7fae2415c730 2024-09-21T06:12:36.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae20c00640 1 -- 172.21.15.50:0/3170363765 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c005020 con 0x7fae2415c730 2024-09-21T06:12:36.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae2ac00640 1 -- 172.21.15.50:0/3170363765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae24157c30 con 0x7fae2415c730 2024-09-21T06:12:36.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae2ac00640 1 -- 172.21.15.50:0/3170363765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae240af930 con 0x7fae2415c730 2024-09-21T06:12:36.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.690+0000 7fae20c00640 1 -- 172.21.15.50:0/3170363765 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 56) ==== 100447+0+0 (secure 0 0 0) 0x7fae1c0051c0 con 0x7fae2415c730 2024-09-21T06:12:36.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.694+0000 7fae20c00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fae00075570 0x7fae00077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:36.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.694+0000 7fae20c00640 1 -- 172.21.15.50:0/3170363765 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7fae1c093ff0 con 0x7fae2415c730 2024-09-21T06:12:36.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.694+0000 7fae23e00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fae00075570 0x7fae00077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:36.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.694+0000 7fae23e00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fae00075570 0x7fae00077a80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fae140af000 tx=0x7fae1401b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:36.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.694+0000 7fae20c00640 1 -- 172.21.15.50:0/3170363765 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fae1c016030 con 0x7fae2415c730 2024-09-21T06:12:36.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.902+0000 7fae2ac00640 1 -- 172.21.15.50:0/3170363765 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fae2406b540 con 0x7fae00075570 2024-09-21T06:12:36.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.906+0000 7fae20c00640 1 -- 172.21.15.50:0/3170363765 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fae140040c0 con 0x7fae00075570 2024-09-21T06:12:36.906 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/0078a113-7228-4587-9847-47491cfce39e 2024-09-21T06:12:36.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.910+0000 7fae2ac00640 1 -- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fae00075570 msgr2=0x7fae00077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:36.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.910+0000 7fae2ac00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fae00075570 0x7fae00077a80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fae140af000 tx=0x7fae1401b120 comp rx=0 tx=0).stop 2024-09-21T06:12:36.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.910+0000 7fae2ac00640 1 -- 172.21.15.50:0/3170363765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2415c730 msgr2=0x7fae24166680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:36.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.910+0000 7fae2ac00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2415c730 0x7fae24166680 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00e970 tx=0x7fae1c00ee30 comp rx=0 tx=0).stop 2024-09-21T06:12:36.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.910+0000 7fae2ac00640 1 -- 172.21.15.50:0/3170363765 shutdown_connections 2024-09-21T06:12:36.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.910+0000 7fae2ac00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fae00075570 0x7fae00077a80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.910+0000 7fae2ac00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2415c730 0x7fae24166680 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.910+0000 7fae2ac00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae240af520 0x7fae241602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.910+0000 7fae2ac00640 1 --2- 172.21.15.50:0/3170363765 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fae24073f90 0x7fae241fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:36.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.910+0000 7fae2ac00640 1 -- 172.21.15.50:0/3170363765 >> 172.21.15.50:0/3170363765 conn(0x7fae240d06e0 msgr2=0x7fae240ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:36.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.910+0000 7fae2ac00640 1 -- 172.21.15.50:0/3170363765 shutdown_connections 2024-09-21T06:12:36.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:12:36.910+0000 7fae2ac00640 1 -- 172.21.15.50:0/3170363765 wait complete. 2024-09-21T06:12:36.929 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/0078a113-7228-4587-9847-47491cfce39e 2024-09-21T06:12:36.929 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:12:36.930 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:36.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:36.939 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0078a113-7228-4587-9847-47491cfce39e/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:12:46.615 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:12:46.615 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:12:46.615 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.61255 s, 112 MB/s 2024-09-21T06:12:46.616 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:12:46.616 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:46.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:46.625 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0078a113-7228-4587-9847-47491cfce39e/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:12:49.324 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:12:49.327 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:12:56.268 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:12:56.269 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:12:56.269 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.593 s, 112 MB/s 2024-09-21T06:12:56.270 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:12:56.271 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:12:56.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:12:56.280 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0078a113-7228-4587-9847-47491cfce39e/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-21T06:13:05.727 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records in 2024-09-21T06:13:05.727 INFO:teuthology.orchestra.run.smithi050.stderr:1024+0 records out 2024-09-21T06:13:05.727 INFO:teuthology.orchestra.run.smithi050.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.39554 s, 114 MB/s 2024-09-21T06:13:05.728 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-21T06:13:05.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.894+0000 7f3514000640 1 -- 172.21.15.50:0/4213327769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350c0b54f0 msgr2=0x7f350c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:05.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.894+0000 7f3514000640 1 --2- 172.21.15.50:0/4213327769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350c0b54f0 0x7f350c15bb60 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f3504009bc0 tx=0x7f350401cae0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.894+0000 7f3514000640 1 -- 172.21.15.50:0/4213327769 shutdown_connections 2024-09-21T06:13:05.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.894+0000 7f3514000640 1 --2- 172.21.15.50:0/4213327769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350c0b54f0 0x7f350c15bb60 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.894+0000 7f3514000640 1 -- 172.21.15.50:0/4213327769 >> 172.21.15.50:0/4213327769 conn(0x7f350c0b2040 msgr2=0x7f350c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:05.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.894+0000 7f3514000640 1 -- 172.21.15.50:0/4213327769 shutdown_connections 2024-09-21T06:13:05.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.894+0000 7f3514000640 1 -- 172.21.15.50:0/4213327769 wait complete. 2024-09-21T06:13:05.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3514000640 1 Processor -- start 2024-09-21T06:13:05.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3514000640 1 -- start start 2024-09-21T06:13:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3514000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350c0b5c70 0x7f350c140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3514000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f350c1527b0 0x7f350c0ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3514000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f350c0abbf0 0x7f350c151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3514000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f350c169bd0 con 0x7f350c0b5c70 2024-09-21T06:13:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3514000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f350c169a50 con 0x7f350c0abbf0 2024-09-21T06:13:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3514000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f350c169d50 con 0x7f350c1527b0 2024-09-21T06:13:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3511600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350c0b5c70 0x7f350c140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3511600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350c0b5c70 0x7f350c140b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56594/0 (socket says 172.21.15.50:56594) 2024-09-21T06:13:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3511600640 1 -- 172.21.15.50:0/159870931 learned_addr learned my addr 172.21.15.50:0/159870931 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3512000640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f350c0abbf0 0x7f350c151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:05.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3510c00640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f350c1527b0 0x7f350c0ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:05.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3511600640 1 -- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f350c1527b0 msgr2=0x7f350c0ab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:05.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3511600640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f350c1527b0 0x7f350c0ab6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3511600640 1 -- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f350c0abbf0 msgr2=0x7f350c151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:05.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3511600640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f350c0abbf0 0x7f350c151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3511600640 1 -- 172.21.15.50:0/159870931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350c1f9ad0 con 0x7f350c0b5c70 2024-09-21T06:13:05.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3511600640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350c0b5c70 0x7f350c140b70 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f350000c420 tx=0x7f350000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:05.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3509600640 1 -- 172.21.15.50:0/159870931 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500011790 con 0x7f350c0b5c70 2024-09-21T06:13:05.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3509600640 1 -- 172.21.15.50:0/159870931 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3500017070 con 0x7f350c0b5c70 2024-09-21T06:13:05.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3509600640 1 -- 172.21.15.50:0/159870931 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500011ce0 con 0x7f350c0b5c70 2024-09-21T06:13:05.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3514000640 1 -- 172.21.15.50:0/159870931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350c1f9db0 con 0x7f350c0b5c70 2024-09-21T06:13:05.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.898+0000 7f3514000640 1 -- 172.21.15.50:0/159870931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350c1fa370 con 0x7f350c0b5c70 2024-09-21T06:13:05.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.902+0000 7f3514000640 1 -- 172.21.15.50:0/159870931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f350c15bb60 con 0x7f350c0b5c70 2024-09-21T06:13:05.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.902+0000 7f3509600640 1 -- 172.21.15.50:0/159870931 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 57) ==== 100498+0+0 (secure 0 0 0) 0x7f350001c460 con 0x7f350c0b5c70 2024-09-21T06:13:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.902+0000 7f3509600640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f34ec075600 0x7f34ec077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.902+0000 7f3509600640 1 -- 172.21.15.50:0/159870931 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f3500093040 con 0x7f350c0b5c70 2024-09-21T06:13:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.906+0000 7f3510c00640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f34ec075600 0x7f34ec077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.906+0000 7f3510c00640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f34ec075600 0x7f34ec077b10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f350c1f9990 tx=0x7f34f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:05.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:05.910+0000 7f3509600640 1 -- 172.21.15.50:0/159870931 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f350005f5a0 con 0x7f350c0b5c70 2024-09-21T06:13:06.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.118+0000 7f3514000640 1 -- 172.21.15.50:0/159870931 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f350c0b25f0 con 0x7f34ec075600 2024-09-21T06:13:06.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3509600640 1 -- 172.21.15.50:0/159870931 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f350c0b25f0 con 0x7f34ec075600 2024-09-21T06:13:06.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3514000640 1 -- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f34ec075600 msgr2=0x7f34ec077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3514000640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f34ec075600 0x7f34ec077b10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f350c1f9990 tx=0x7f34f8009040 comp rx=0 tx=0).stop 2024-09-21T06:13:06.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3514000640 1 -- 172.21.15.50:0/159870931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350c0b5c70 msgr2=0x7f350c140b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3514000640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350c0b5c70 0x7f350c140b70 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f350000c420 tx=0x7f350000c8e0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3514000640 1 -- 172.21.15.50:0/159870931 shutdown_connections 2024-09-21T06:13:06.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3514000640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f350c0abbf0 0x7f350c151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3514000640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f350c1527b0 0x7f350c0ab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3514000640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f34ec075600 0x7f34ec077b10 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3514000640 1 --2- 172.21.15.50:0/159870931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350c0b5c70 0x7f350c140b70 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3514000640 1 -- 172.21.15.50:0/159870931 >> 172.21.15.50:0/159870931 conn(0x7f350c0b2040 msgr2=0x7f350c15a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:06.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3514000640 1 -- 172.21.15.50:0/159870931 shutdown_connections 2024-09-21T06:13:06.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.134+0000 7f3514000640 1 -- 172.21.15.50:0/159870931 wait complete. 2024-09-21T06:13:06.153 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-21T06:13:06.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.318+0000 7f5526a00640 1 -- 172.21.15.50:0/3955560517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55200b5c70 msgr2=0x7f552015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.318+0000 7f5526a00640 1 --2- 172.21.15.50:0/3955560517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55200b5c70 0x7f552015bb60 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f5520074b20 tx=0x7f551001c5f0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.318+0000 7f5526a00640 1 -- 172.21.15.50:0/3955560517 shutdown_connections 2024-09-21T06:13:06.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.318+0000 7f5526a00640 1 --2- 172.21.15.50:0/3955560517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55200b5c70 0x7f552015bb60 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.318+0000 7f5526a00640 1 -- 172.21.15.50:0/3955560517 >> 172.21.15.50:0/3955560517 conn(0x7f55200b2040 msgr2=0x7f55200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:06.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.318+0000 7f5526a00640 1 -- 172.21.15.50:0/3955560517 shutdown_connections 2024-09-21T06:13:06.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.318+0000 7f5526a00640 1 -- 172.21.15.50:0/3955560517 wait complete. 2024-09-21T06:13:06.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.318+0000 7f5526a00640 1 Processor -- start 2024-09-21T06:13:06.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.318+0000 7f5526a00640 1 -- start start 2024-09-21T06:13:06.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f5526a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55200b4de0 0x7f552015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:06.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f5526a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55200b5c70 0x7f55200aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:06.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f5526a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55201527b0 0x7f5520154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:06.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f5526a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5520169b60 con 0x7f55200b4de0 2024-09-21T06:13:06.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f5526a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f55201699e0 con 0x7f55201527b0 2024-09-21T06:13:06.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f5526a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5520169ce0 con 0x7f55200b5c70 2024-09-21T06:13:06.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f551fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55200b4de0 0x7f552015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f551fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55200b4de0 0x7f552015afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56618/0 (socket says 172.21.15.50:56618) 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f551fe00640 1 -- 172.21.15.50:0/1992162335 learned_addr learned my addr 172.21.15.50:0/1992162335 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f5517e00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55200b5c70 0x7f55200aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f5524a00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55201527b0 0x7f5520154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f551fe00640 1 -- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55200b5c70 msgr2=0x7f55200aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f551fe00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55200b5c70 0x7f55200aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f551fe00640 1 -- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55201527b0 msgr2=0x7f5520154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f551fe00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55201527b0 0x7f5520154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f551fe00640 1 -- 172.21.15.50:0/1992162335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55201f89b0 con 0x7f55200b4de0 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f5524a00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55201527b0 0x7f5520154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f551fe00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55200b4de0 0x7f552015afa0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f551001a220 tx=0x7f55100a6be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f551cc00640 1 -- 172.21.15.50:0/1992162335 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55100be070 con 0x7f55200b4de0 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f551cc00640 1 -- 172.21.15.50:0/1992162335 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55100a6d90 con 0x7f55200b4de0 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f5526a00640 1 -- 172.21.15.50:0/1992162335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55201f8c90 con 0x7f55200b4de0 2024-09-21T06:13:06.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f551cc00640 1 -- 172.21.15.50:0/1992162335 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55100052a0 con 0x7f55200b4de0 2024-09-21T06:13:06.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f5526a00640 1 -- 172.21.15.50:0/1992162335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55201f9250 con 0x7f55200b4de0 2024-09-21T06:13:06.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.322+0000 7f5526a00640 1 -- 172.21.15.50:0/1992162335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f552015bb60 con 0x7f55200b4de0 2024-09-21T06:13:06.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.326+0000 7f551cc00640 1 -- 172.21.15.50:0/1992162335 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 57) ==== 100498+0+0 (secure 0 0 0) 0x7f5510005440 con 0x7f55200b4de0 2024-09-21T06:13:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.326+0000 7f551cc00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54fc075600 0x7f54fc077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.326+0000 7f551cc00640 1 -- 172.21.15.50:0/1992162335 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f551012d090 con 0x7f55200b4de0 2024-09-21T06:13:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.326+0000 7f5517e00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54fc075600 0x7f54fc077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.326+0000 7f5517e00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54fc075600 0x7f54fc077b10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f552014f190 tx=0x7f5508009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.330+0000 7f551cc00640 1 -- 172.21.15.50:0/1992162335 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f55100ac030 con 0x7f55200b4de0 2024-09-21T06:13:06.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.530+0000 7f5526a00640 1 -- 172.21.15.50:0/1992162335 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f552006b540 con 0x7f54fc075600 2024-09-21T06:13:06.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f551cc00640 1 -- 172.21.15.50:0/1992162335 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f5508001450 con 0x7f54fc075600 2024-09-21T06:13:06.536 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/_nogroup/sv1/0078a113-7228-4587-9847-47491cfce39e 2024-09-21T06:13:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f5526a00640 1 -- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54fc075600 msgr2=0x7f54fc077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f5526a00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54fc075600 0x7f54fc077b10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f552014f190 tx=0x7f5508009040 comp rx=0 tx=0).stop 2024-09-21T06:13:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f5526a00640 1 -- 172.21.15.50:0/1992162335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55200b4de0 msgr2=0x7f552015afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f5526a00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55200b4de0 0x7f552015afa0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f551001a220 tx=0x7f55100a6be0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f5526a00640 1 -- 172.21.15.50:0/1992162335 shutdown_connections 2024-09-21T06:13:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f5526a00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55201527b0 0x7f5520154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f5526a00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f55200b5c70 0x7f55200aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f5526a00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f54fc075600 0x7f54fc077b10 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f5526a00640 1 --2- 172.21.15.50:0/1992162335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55200b4de0 0x7f552015afa0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f5526a00640 1 -- 172.21.15.50:0/1992162335 >> 172.21.15.50:0/1992162335 conn(0x7f55200b2040 msgr2=0x7f55200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f5526a00640 1 -- 172.21.15.50:0/1992162335 shutdown_connections 2024-09-21T06:13:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.538+0000 7f5526a00640 1 -- 172.21.15.50:0/1992162335 wait complete. 2024-09-21T06:13:06.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:06.558 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0078a113-7228-4587-9847-47491cfce39e 2024-09-21T06:13:06.558 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:13:06.568 INFO:teuthology.orchestra.run.smithi050.stdout:# file: volumes/_nogroup/sv1/0078a113-7228-4587-9847-47491cfce39e 2024-09-21T06:13:06.568 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.rbytes="3221225472" 2024-09-21T06:13:06.568 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:06.569 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-21T06:13:06.570 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99400 2024-09-21T06:13:06.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.782+0000 7f10fb600640 1 -- 172.21.15.50:0/1040546594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10f40b54f0 msgr2=0x7f10f415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.782+0000 7f10fb600640 1 --2- 172.21.15.50:0/1040546594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10f40b54f0 0x7f10f415bb60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009bc0 tx=0x7f10ec01ca00 comp rx=0 tx=0).stop 2024-09-21T06:13:06.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.782+0000 7f10fb600640 1 -- 172.21.15.50:0/1040546594 shutdown_connections 2024-09-21T06:13:06.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.782+0000 7f10fb600640 1 --2- 172.21.15.50:0/1040546594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10f40b54f0 0x7f10f415bb60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.782+0000 7f10fb600640 1 -- 172.21.15.50:0/1040546594 >> 172.21.15.50:0/1040546594 conn(0x7f10f40b2040 msgr2=0x7f10f40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:06.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.782+0000 7f10fb600640 1 -- 172.21.15.50:0/1040546594 shutdown_connections 2024-09-21T06:13:06.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.782+0000 7f10fb600640 1 -- 172.21.15.50:0/1040546594 wait complete. 2024-09-21T06:13:06.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.782+0000 7f10fb600640 1 Processor -- start 2024-09-21T06:13:06.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.782+0000 7f10fb600640 1 -- start start 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10fb600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10f40b5c70 0x7f10f40cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10fb600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f10f41527b0 0x7f10f41f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10fb600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10f414ad20 0x7f10f41f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10fb600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f10f4169b30 con 0x7f10f414ad20 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10fb600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f10f41699b0 con 0x7f10f40b5c70 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10fb600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f10f4169cb0 con 0x7f10f41527b0 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f9600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10f414ad20 0x7f10f41f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f8c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10f40b5c70 0x7f10f40cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f9600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10f414ad20 0x7f10f41f9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41050/0 (socket says 172.21.15.50:41050) 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f9600640 1 -- 172.21.15.50:0/3999175498 learned_addr learned my addr 172.21.15.50:0/3999175498 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10ebe00640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f10f41527b0 0x7f10f41f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f9600640 1 -- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f10f41527b0 msgr2=0x7f10f41f96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f9600640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f10f41527b0 0x7f10f41f96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f9600640 1 -- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10f40b5c70 msgr2=0x7f10f40cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f9600640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10f40b5c70 0x7f10f40cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f9600640 1 -- 172.21.15.50:0/3999175498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f414ec30 con 0x7f10f414ad20 2024-09-21T06:13:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f9600640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10f414ad20 0x7f10f41f9c30 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f10ec019610 tx=0x7f10ec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f1600640 1 -- 172.21.15.50:0/3999175498 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec0bb070 con 0x7f10f414ad20 2024-09-21T06:13:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f1600640 1 -- 172.21.15.50:0/3999175498 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10ec003df0 con 0x7f10f414ad20 2024-09-21T06:13:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10fb600640 1 -- 172.21.15.50:0/3999175498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f414a270 con 0x7f10f414ad20 2024-09-21T06:13:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10f1600640 1 -- 172.21.15.50:0/3999175498 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec0b2040 con 0x7f10f414ad20 2024-09-21T06:13:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10fb600640 1 -- 172.21.15.50:0/3999175498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f41582e0 con 0x7f10f414ad20 2024-09-21T06:13:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.786+0000 7f10fb600640 1 -- 172.21.15.50:0/3999175498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f415bb60 con 0x7f10f414ad20 2024-09-21T06:13:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.790+0000 7f10f1600640 1 -- 172.21.15.50:0/3999175498 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 57) ==== 100498+0+0 (secure 0 0 0) 0x7f10ec0be070 con 0x7f10f414ad20 2024-09-21T06:13:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.790+0000 7f10f1600640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10d0075630 0x7f10d0077b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.790+0000 7f10f1600640 1 -- 172.21.15.50:0/3999175498 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f10ec12cef0 con 0x7f10f414ad20 2024-09-21T06:13:06.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.790+0000 7f10f8c00640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10d0075630 0x7f10d0077b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:06.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.790+0000 7f10f8c00640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10d0075630 0x7f10d0077b40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f10e4000fc0 tx=0x7f10e4001c80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:06.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:06.794+0000 7f10f1600640 1 -- 172.21.15.50:0/3999175498 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f10ec0166b0 con 0x7f10f414ad20 2024-09-21T06:13:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.006+0000 7f10fb600640 1 -- 172.21.15.50:0/3999175498 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99400", "target": ["mon-mgr", ""]}) -- 0x7f10f4158580 con 0x7f10d0075630 2024-09-21T06:13:07.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.070+0000 7f10f1600640 1 -- 172.21.15.50:0/3999175498 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f10f4158580 con 0x7f10d0075630 2024-09-21T06:13:07.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.074+0000 7f10fb600640 1 -- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10d0075630 msgr2=0x7f10d0077b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.074+0000 7f10fb600640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10d0075630 0x7f10d0077b40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f10e4000fc0 tx=0x7f10e4001c80 comp rx=0 tx=0).stop 2024-09-21T06:13:07.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.074+0000 7f10fb600640 1 -- 172.21.15.50:0/3999175498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10f414ad20 msgr2=0x7f10f41f9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.074+0000 7f10fb600640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10f414ad20 0x7f10f41f9c30 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f10ec019610 tx=0x7f10ec009310 comp rx=0 tx=0).stop 2024-09-21T06:13:07.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.074+0000 7f10fb600640 1 -- 172.21.15.50:0/3999175498 shutdown_connections 2024-09-21T06:13:07.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.074+0000 7f10fb600640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10d0075630 0x7f10d0077b40 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.074+0000 7f10fb600640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10f414ad20 0x7f10f41f9c30 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.074+0000 7f10fb600640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f10f41527b0 0x7f10f41f96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.074+0000 7f10fb600640 1 --2- 172.21.15.50:0/3999175498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10f40b5c70 0x7f10f40cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.074+0000 7f10fb600640 1 -- 172.21.15.50:0/3999175498 >> 172.21.15.50:0/3999175498 conn(0x7f10f40b2040 msgr2=0x7f10f40b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:07.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.074+0000 7f10fb600640 1 -- 172.21.15.50:0/3999175498 shutdown_connections 2024-09-21T06:13:07.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.074+0000 7f10fb600640 1 -- 172.21.15.50:0/3999175498 wait complete. 2024-09-21T06:13:07.076 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:07.078+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:07.076 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:07.078+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:07.076 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:07.078+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:07.076 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:07.078+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:07.076 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:07.078+0000 7f65a7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:07.093 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone5031 2024-09-21T06:13:07.094 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:07.098+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:07.094 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:07.098+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:07.094 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:07.098+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:07.094 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:07.098+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:07.094 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:07.098+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:07.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.262+0000 7f6bb0c00640 1 -- 172.21.15.50:0/3551955358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bac1527b0 msgr2=0x7f6bac15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.262+0000 7f6bb0c00640 1 --2- 172.21.15.50:0/3551955358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bac1527b0 0x7f6bac15bb60 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009900 tx=0x7f6b9401c8b0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.262+0000 7f6bb0c00640 1 -- 172.21.15.50:0/3551955358 shutdown_connections 2024-09-21T06:13:07.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.262+0000 7f6bb0c00640 1 --2- 172.21.15.50:0/3551955358 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6bac0b4de0 0x7f6bac0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.262+0000 7f6bb0c00640 1 --2- 172.21.15.50:0/3551955358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bac1527b0 0x7f6bac15bb60 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.262+0000 7f6bb0c00640 1 -- 172.21.15.50:0/3551955358 >> 172.21.15.50:0/3551955358 conn(0x7f6bac0b2040 msgr2=0x7f6bac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:07.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.262+0000 7f6bb0c00640 1 -- 172.21.15.50:0/3551955358 shutdown_connections 2024-09-21T06:13:07.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.262+0000 7f6bb0c00640 1 -- 172.21.15.50:0/3551955358 wait complete. 2024-09-21T06:13:07.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6bb0c00640 1 Processor -- start 2024-09-21T06:13:07.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6bb0c00640 1 -- start start 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6bb0c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bac0b4de0 0x7f6bac0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6bb0c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6bac0b5c70 0x7f6bac14bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6bb0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bac1527b0 0x7f6bac14c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6bb0c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac168a00 con 0x7f6bac1527b0 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6bb0c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac168880 con 0x7f6bac0b4de0 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6bb0c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6bac168b80 con 0x7f6bac0b5c70 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6baa800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bac1527b0 0x7f6bac14c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6ba1400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6bac0b5c70 0x7f6bac14bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6ba9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bac0b4de0 0x7f6bac0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6baa800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bac1527b0 0x7f6bac14c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41082/0 (socket says 172.21.15.50:41082) 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6baa800640 1 -- 172.21.15.50:0/1463603100 learned_addr learned my addr 172.21.15.50:0/1463603100 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6ba1400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6bac0b5c70 0x7f6bac14bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:42846/0 (socket says 172.21.15.50:42846) 2024-09-21T06:13:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6baa800640 1 -- 172.21.15.50:0/1463603100 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6bac0b5c70 msgr2=0x7f6bac14bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6baa800640 1 --2- 172.21.15.50:0/1463603100 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6bac0b5c70 0x7f6bac14bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6baa800640 1 -- 172.21.15.50:0/1463603100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bac0b4de0 msgr2=0x7f6bac0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6baa800640 1 --2- 172.21.15.50:0/1463603100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bac0b4de0 0x7f6bac0cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6baa800640 1 -- 172.21.15.50:0/1463603100 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bac1f7a20 con 0x7f6bac1527b0 2024-09-21T06:13:07.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6baa800640 1 --2- 172.21.15.50:0/1463603100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bac1527b0 0x7f6bac14c280 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400e940 tx=0x7f6ba400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:07.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6ba2a00640 1 -- 172.21.15.50:0/1463603100 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4004d30 con 0x7f6bac1527b0 2024-09-21T06:13:07.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6ba2a00640 1 -- 172.21.15.50:0/1463603100 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ba400cb20 con 0x7f6bac1527b0 2024-09-21T06:13:07.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.266+0000 7f6ba2a00640 1 -- 172.21.15.50:0/1463603100 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4005020 con 0x7f6bac1527b0 2024-09-21T06:13:07.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.270+0000 7f6bb0c00640 1 -- 172.21.15.50:0/1463603100 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bac1f7180 con 0x7f6bac1527b0 2024-09-21T06:13:07.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.270+0000 7f6bb0c00640 1 -- 172.21.15.50:0/1463603100 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bac1f7740 con 0x7f6bac1527b0 2024-09-21T06:13:07.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.270+0000 7f6bb0c00640 1 -- 172.21.15.50:0/1463603100 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bac1f6ef0 con 0x7f6bac1527b0 2024-09-21T06:13:07.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.270+0000 7f6ba2a00640 1 -- 172.21.15.50:0/1463603100 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 57) ==== 100498+0+0 (secure 0 0 0) 0x7f6ba40051c0 con 0x7f6bac1527b0 2024-09-21T06:13:07.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.270+0000 7f6ba2a00640 1 --2- 172.21.15.50:0/1463603100 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6b88075600 0x7f6b88077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.270+0000 7f6ba2a00640 1 -- 172.21.15.50:0/1463603100 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f6ba4093cc0 con 0x7f6bac1527b0 2024-09-21T06:13:07.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.270+0000 7f6ba9e00640 1 --2- 172.21.15.50:0/1463603100 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6b88075600 0x7f6b88077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.274+0000 7f6ba9e00640 1 --2- 172.21.15.50:0/1463603100 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6b88075600 0x7f6b88077b10 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c007fd0 tx=0x7f6b9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:07.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.274+0000 7f6ba2a00640 1 -- 172.21.15.50:0/1463603100 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6ba4016030 con 0x7f6bac1527b0 2024-09-21T06:13:07.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.486+0000 7f6bb0c00640 1 -- 172.21.15.50:0/1463603100 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone5031", "target": ["mon-mgr", ""]}) -- 0x7f6bac0b51f0 con 0x7f6b88075600 2024-09-21T06:13:07.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:07.806+0000 7f6ba2a00640 1 -- 172.21.15.50:0/1463603100 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 58) ==== 100600+0+0 (secure 0 0 0) 0x7f6ba409e020 con 0x7f6bac1527b0 2024-09-21T06:13:08.076 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:08.078+0000 7f6541200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:08.076 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:08.078+0000 7f6541200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:08.076 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:08.078+0000 7f6541200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:08.076 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:08.078+0000 7f6541200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:08.076 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:08.078+0000 7f6541200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:08.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.178+0000 7f6ba2a00640 1 -- 172.21.15.50:0/1463603100 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6bac0b51f0 con 0x7f6b88075600 2024-09-21T06:13:08.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.182+0000 7f6bb0c00640 1 -- 172.21.15.50:0/1463603100 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6b88075600 msgr2=0x7f6b88077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:08.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.182+0000 7f6bb0c00640 1 --2- 172.21.15.50:0/1463603100 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6b88075600 0x7f6b88077b10 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c007fd0 tx=0x7f6b9c009040 comp rx=0 tx=0).stop 2024-09-21T06:13:08.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.182+0000 7f6bb0c00640 1 -- 172.21.15.50:0/1463603100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bac1527b0 msgr2=0x7f6bac14c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:08.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.182+0000 7f6bb0c00640 1 --2- 172.21.15.50:0/1463603100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bac1527b0 0x7f6bac14c280 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400e940 tx=0x7f6ba400ee00 comp rx=0 tx=0).stop 2024-09-21T06:13:08.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.182+0000 7f6bb0c00640 1 -- 172.21.15.50:0/1463603100 shutdown_connections 2024-09-21T06:13:08.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.182+0000 7f6bb0c00640 1 --2- 172.21.15.50:0/1463603100 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6b88075600 0x7f6b88077b10 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.182+0000 7f6bb0c00640 1 --2- 172.21.15.50:0/1463603100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bac1527b0 0x7f6bac14c280 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.182+0000 7f6bb0c00640 1 --2- 172.21.15.50:0/1463603100 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6bac0b5c70 0x7f6bac14bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.182+0000 7f6bb0c00640 1 --2- 172.21.15.50:0/1463603100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bac0b4de0 0x7f6bac0cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.182+0000 7f6bb0c00640 1 -- 172.21.15.50:0/1463603100 >> 172.21.15.50:0/1463603100 conn(0x7f6bac0b2040 msgr2=0x7f6bac154e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:08.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.182+0000 7f6bb0c00640 1 -- 172.21.15.50:0/1463603100 shutdown_connections 2024-09-21T06:13:08.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.182+0000 7f6bb0c00640 1 -- 172.21.15.50:0/1463603100 wait complete. 2024-09-21T06:13:08.209 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone34632 2024-09-21T06:13:08.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.478+0000 7fb605200640 1 -- 172.21.15.50:0/1512951861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60015f5c0 msgr2=0x7fb60015f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:08.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.478+0000 7fb605200640 1 --2- 172.21.15.50:0/1512951861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60015f5c0 0x7fb60015f9d0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fb6000ced20 tx=0x7fb5f801cd10 comp rx=0 tx=0).stop 2024-09-21T06:13:08.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.478+0000 7fb605200640 1 -- 172.21.15.50:0/1512951861 shutdown_connections 2024-09-21T06:13:08.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.478+0000 7fb605200640 1 --2- 172.21.15.50:0/1512951861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60015f5c0 0x7fb60015f9d0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.478+0000 7fb605200640 1 --2- 172.21.15.50:0/1512951861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb60006dbc0 0x7fb60006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.478+0000 7fb605200640 1 --2- 172.21.15.50:0/1512951861 >> v2:172.21.15.88:3300/0 conn(0x7fb6000d1140 0x7fb6000b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.478+0000 7fb605200640 1 -- 172.21.15.50:0/1512951861 >> 172.21.15.50:0/1512951861 conn(0x7fb6000b41f0 msgr2=0x7fb6000b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:08.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.478+0000 7fb605200640 1 -- 172.21.15.50:0/1512951861 shutdown_connections 2024-09-21T06:13:08.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.478+0000 7fb605200640 1 -- 172.21.15.50:0/1512951861 wait complete. 2024-09-21T06:13:08.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.478+0000 7fb605200640 1 Processor -- start 2024-09-21T06:13:08.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.478+0000 7fb605200640 1 -- start start 2024-09-21T06:13:08.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb605200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb60006dbc0 0x7fb60009a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:08.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb605200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6000d1140 0x7fb6000cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:08.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb605200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60015f5c0 0x7fb6000cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:08.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb605200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb600074dc0 con 0x7fb60015f5c0 2024-09-21T06:13:08.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb605200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb600074c40 con 0x7fb6000d1140 2024-09-21T06:13:08.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb605200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb600074f40 con 0x7fb60006dbc0 2024-09-21T06:13:08.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb5ffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60015f5c0 0x7fb6000cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:08.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb5ffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60015f5c0 0x7fb6000cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41104/0 (socket says 172.21.15.50:41104) 2024-09-21T06:13:08.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb5ffe00640 1 -- 172.21.15.50:0/4000436047 learned_addr learned my addr 172.21.15.50:0/4000436047 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:08.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb5ffe00640 1 -- 172.21.15.50:0/4000436047 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb60006dbc0 msgr2=0x7fb60009a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:08.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb5ffe00640 1 --2- 172.21.15.50:0/4000436047 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb60006dbc0 0x7fb60009a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb5ffe00640 1 -- 172.21.15.50:0/4000436047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6000d1140 msgr2=0x7fb6000cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:08.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb5ffe00640 1 --2- 172.21.15.50:0/4000436047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6000d1140 0x7fb6000cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb5ffe00640 1 -- 172.21.15.50:0/4000436047 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60015b0b0 con 0x7fb60015f5c0 2024-09-21T06:13:08.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb5ffe00640 1 --2- 172.21.15.50:0/4000436047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60015f5c0 0x7fb6000cc1e0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009bd0 tx=0x7fb5f8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:08.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb5f7400640 1 -- 172.21.15.50:0/4000436047 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80bc070 con 0x7fb60015f5c0 2024-09-21T06:13:08.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb5f7400640 1 -- 172.21.15.50:0/4000436047 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5f80042c0 con 0x7fb60015f5c0 2024-09-21T06:13:08.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb605200640 1 -- 172.21.15.50:0/4000436047 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6000af180 con 0x7fb60015f5c0 2024-09-21T06:13:08.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb5f7400640 1 -- 172.21.15.50:0/4000436047 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80b2040 con 0x7fb60015f5c0 2024-09-21T06:13:08.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.482+0000 7fb605200640 1 -- 172.21.15.50:0/4000436047 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6000af740 con 0x7fb60015f5c0 2024-09-21T06:13:08.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.486+0000 7fb605200640 1 -- 172.21.15.50:0/4000436047 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6000b0730 con 0x7fb60015f5c0 2024-09-21T06:13:08.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.486+0000 7fb5f7400640 1 -- 172.21.15.50:0/4000436047 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 58) ==== 100600+0+0 (secure 0 0 0) 0x7fb5f80046d0 con 0x7fb60015f5c0 2024-09-21T06:13:08.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.486+0000 7fb5f7400640 1 --2- 172.21.15.50:0/4000436047 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb5e0075720 0x7fb5e0077c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:08.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.486+0000 7fb5f7400640 1 -- 172.21.15.50:0/4000436047 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7fb5f812d5a0 con 0x7fb60015f5c0 2024-09-21T06:13:08.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.486+0000 7fb5ff400640 1 --2- 172.21.15.50:0/4000436047 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb5e0075720 0x7fb5e0077c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:08.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.490+0000 7fb5ff400640 1 --2- 172.21.15.50:0/4000436047 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb5e0075720 0x7fb5e0077c30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00058d0 tx=0x7fb5f0005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:08.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.490+0000 7fb5f7400640 1 -- 172.21.15.50:0/4000436047 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb5f8016990 con 0x7fb60015f5c0 2024-09-21T06:13:08.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:08.834+0000 7fb605200640 1 -- 172.21.15.50:0/4000436047 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone34632", "target": ["mon-mgr", ""]}) -- 0x7fb60006b540 con 0x7fb5e0075720 2024-09-21T06:13:09.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.394+0000 7fb5f7400640 1 -- 172.21.15.50:0/4000436047 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb5f000d960 con 0x7fb5e0075720 2024-09-21T06:13:09.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.398+0000 7fb605200640 1 -- 172.21.15.50:0/4000436047 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb5e0075720 msgr2=0x7fb5e0077c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.398+0000 7fb605200640 1 --2- 172.21.15.50:0/4000436047 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb5e0075720 0x7fb5e0077c30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00058d0 tx=0x7fb5f0005860 comp rx=0 tx=0).stop 2024-09-21T06:13:09.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.398+0000 7fb605200640 1 -- 172.21.15.50:0/4000436047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60015f5c0 msgr2=0x7fb6000cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.398+0000 7fb605200640 1 --2- 172.21.15.50:0/4000436047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60015f5c0 0x7fb6000cc1e0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009bd0 tx=0x7fb5f8003f60 comp rx=0 tx=0).stop 2024-09-21T06:13:09.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.398+0000 7fb605200640 1 -- 172.21.15.50:0/4000436047 shutdown_connections 2024-09-21T06:13:09.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.398+0000 7fb605200640 1 --2- 172.21.15.50:0/4000436047 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb5e0075720 0x7fb5e0077c30 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.398+0000 7fb605200640 1 --2- 172.21.15.50:0/4000436047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60015f5c0 0x7fb6000cc1e0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.398+0000 7fb605200640 1 --2- 172.21.15.50:0/4000436047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6000d1140 0x7fb6000cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.398+0000 7fb605200640 1 --2- 172.21.15.50:0/4000436047 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb60006dbc0 0x7fb60009a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.398+0000 7fb605200640 1 -- 172.21.15.50:0/4000436047 >> 172.21.15.50:0/4000436047 conn(0x7fb6000b41f0 msgr2=0x7fb60015a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:09.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.398+0000 7fb605200640 1 -- 172.21.15.50:0/4000436047 shutdown_connections 2024-09-21T06:13:09.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.398+0000 7fb605200640 1 -- 172.21.15.50:0/4000436047 wait complete. 2024-09-21T06:13:09.417 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone74693 2024-09-21T06:13:09.432 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:09.434+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:09.432 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:09.434+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:09.432 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:09.434+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:09.432 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:09.434+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:09.432 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:09.434+0000 7f65a6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:09.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.706+0000 7f1d90600640 1 -- 172.21.15.50:0/3994442902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d880b5c70 msgr2=0x7f1d8815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.706+0000 7f1d90600640 1 --2- 172.21.15.50:0/3994442902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d880b5c70 0x7f1d8815bb60 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f1d880cca90 tx=0x7f1d8001cb00 comp rx=0 tx=0).stop 2024-09-21T06:13:09.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.706+0000 7f1d90600640 1 -- 172.21.15.50:0/3994442902 shutdown_connections 2024-09-21T06:13:09.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.706+0000 7f1d90600640 1 --2- 172.21.15.50:0/3994442902 >> v2:172.21.15.88:3300/0 conn(0x7f1d880b4de0 0x7f1d880b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.706+0000 7f1d90600640 1 --2- 172.21.15.50:0/3994442902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d880b5c70 0x7f1d8815bb60 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.706+0000 7f1d90600640 1 -- 172.21.15.50:0/3994442902 >> 172.21.15.50:0/3994442902 conn(0x7f1d880b2040 msgr2=0x7f1d880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:09.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.706+0000 7f1d90600640 1 -- 172.21.15.50:0/3994442902 shutdown_connections 2024-09-21T06:13:09.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.706+0000 7f1d90600640 1 -- 172.21.15.50:0/3994442902 wait complete. 2024-09-21T06:13:09.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.710+0000 7f1d90600640 1 Processor -- start 2024-09-21T06:13:09.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.710+0000 7f1d90600640 1 -- start start 2024-09-21T06:13:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d90600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d880b4de0 0x7f1d88139c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d90600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1d880b5c70 0x7f1d880abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d90600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d881527b0 0x7f1d880ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d90600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1d88168a10 con 0x7f1d880b4de0 2024-09-21T06:13:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d90600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1d88168890 con 0x7f1d881527b0 2024-09-21T06:13:09.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d90600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1d88168b90 con 0x7f1d880b5c70 2024-09-21T06:13:09.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d8dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d880b4de0 0x7f1d88139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d8dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d880b4de0 0x7f1d88139c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41108/0 (socket says 172.21.15.50:41108) 2024-09-21T06:13:09.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d8dc00640 1 -- 172.21.15.50:0/1382367420 learned_addr learned my addr 172.21.15.50:0/1382367420 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:09.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d8dc00640 1 -- 172.21.15.50:0/1382367420 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1d880b5c70 msgr2=0x7f1d880abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d8dc00640 1 --2- 172.21.15.50:0/1382367420 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1d880b5c70 0x7f1d880abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d8dc00640 1 -- 172.21.15.50:0/1382367420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d881527b0 msgr2=0x7f1d880ac0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:13:09.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d8dc00640 1 --2- 172.21.15.50:0/1382367420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d881527b0 0x7f1d880ac0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d8dc00640 1 -- 172.21.15.50:0/1382367420 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d880aaad0 con 0x7f1d880b4de0 2024-09-21T06:13:09.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.714+0000 7f1d8dc00640 1 --2- 172.21.15.50:0/1382367420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d880b4de0 0x7f1d88139c90 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f1d80000f90 tx=0x7f1d800a5990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:09.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.718+0000 7f1d85600640 1 -- 172.21.15.50:0/1382367420 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d800bb070 con 0x7f1d880b4de0 2024-09-21T06:13:09.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.718+0000 7f1d85600640 1 -- 172.21.15.50:0/1382367420 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d80004e00 con 0x7f1d880b4de0 2024-09-21T06:13:09.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.718+0000 7f1d85600640 1 -- 172.21.15.50:0/1382367420 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d800b2040 con 0x7f1d880b4de0 2024-09-21T06:13:09.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.718+0000 7f1d90600640 1 -- 172.21.15.50:0/1382367420 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d880aadb0 con 0x7f1d880b4de0 2024-09-21T06:13:09.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.718+0000 7f1d90600640 1 -- 172.21.15.50:0/1382367420 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d880ab370 con 0x7f1d880b4de0 2024-09-21T06:13:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.718+0000 7f1d90600640 1 -- 172.21.15.50:0/1382367420 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d880b51f0 con 0x7f1d880b4de0 2024-09-21T06:13:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.722+0000 7f1d85600640 1 -- 172.21.15.50:0/1382367420 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 58) ==== 100600+0+0 (secure 0 0 0) 0x7f1d800be070 con 0x7f1d880b4de0 2024-09-21T06:13:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.722+0000 7f1d85600640 1 --2- 172.21.15.50:0/1382367420 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1d68075770 0x7f1d68077c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.726+0000 7f1d85600640 1 -- 172.21.15.50:0/1382367420 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f1d8012d850 con 0x7f1d880b4de0 2024-09-21T06:13:09.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.730+0000 7f1d85600640 1 -- 172.21.15.50:0/1382367420 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1d880b5730 con 0x7f1d880b4de0 2024-09-21T06:13:09.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.730+0000 7f1d8d200640 1 --2- 172.21.15.50:0/1382367420 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1d68075770 0x7f1d68077c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.746+0000 7f1d8d200640 1 --2- 172.21.15.50:0/1382367420 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1d68075770 0x7f1d68077c80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1d880ad340 tx=0x7f1d7c006bb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:09.878+0000 7f1d85600640 1 -- 172.21.15.50:0/1382367420 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 59) ==== 100651+0+0 (secure 0 0 0) 0x7f1d800f6100 con 0x7f1d880b4de0 2024-09-21T06:13:10.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:10.106+0000 7f1d90600640 1 -- 172.21.15.50:0/1382367420 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone74693", "target": ["mon-mgr", ""]}) -- 0x7f1d8806b540 con 0x7f1d68075770 2024-09-21T06:13:11.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.022+0000 7f1d85600640 1 -- 172.21.15.50:0/1382367420 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1d7c009280 con 0x7f1d68075770 2024-09-21T06:13:11.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.030+0000 7f1d90600640 1 -- 172.21.15.50:0/1382367420 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1d68075770 msgr2=0x7f1d68077c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:11.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.030+0000 7f1d90600640 1 --2- 172.21.15.50:0/1382367420 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1d68075770 0x7f1d68077c80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1d880ad340 tx=0x7f1d7c006bb0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.030+0000 7f1d90600640 1 -- 172.21.15.50:0/1382367420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d880b4de0 msgr2=0x7f1d88139c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:11.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.030+0000 7f1d90600640 1 --2- 172.21.15.50:0/1382367420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d880b4de0 0x7f1d88139c90 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f1d80000f90 tx=0x7f1d800a5990 comp rx=0 tx=0).stop 2024-09-21T06:13:11.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.030+0000 7f1d90600640 1 -- 172.21.15.50:0/1382367420 shutdown_connections 2024-09-21T06:13:11.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.030+0000 7f1d90600640 1 --2- 172.21.15.50:0/1382367420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d881527b0 0x7f1d880ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.030+0000 7f1d90600640 1 --2- 172.21.15.50:0/1382367420 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f1d880b5c70 0x7f1d880abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.030+0000 7f1d90600640 1 --2- 172.21.15.50:0/1382367420 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f1d68075770 0x7f1d68077c80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.030+0000 7f1d90600640 1 --2- 172.21.15.50:0/1382367420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d880b4de0 0x7f1d88139c90 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.030+0000 7f1d90600640 1 -- 172.21.15.50:0/1382367420 >> 172.21.15.50:0/1382367420 conn(0x7f1d880b2040 msgr2=0x7f1d88159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:11.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.030+0000 7f1d90600640 1 -- 172.21.15.50:0/1382367420 shutdown_connections 2024-09-21T06:13:11.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.030+0000 7f1d90600640 1 -- 172.21.15.50:0/1382367420 wait complete. 2024-09-21T06:13:11.056 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:11.058+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:11.056 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:11.058+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:11.056 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:11.058+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:11.056 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:11.058+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:11.056 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:11.058+0000 7f65a8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:11.088 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone90034 2024-09-21T06:13:11.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.382+0000 7f1119a00640 1 -- 172.21.15.50:0/4023052471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f111415aa50 msgr2=0x7f11140cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:11.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.382+0000 7f1119a00640 1 --2- 172.21.15.50:0/4023052471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f111415aa50 0x7f11140cd260 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1104009900 tx=0x7f110401c8e0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.382+0000 7f1119a00640 1 -- 172.21.15.50:0/4023052471 shutdown_connections 2024-09-21T06:13:11.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.382+0000 7f1119a00640 1 --2- 172.21.15.50:0/4023052471 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11140cdbb0 0x7f111415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.382+0000 7f1119a00640 1 --2- 172.21.15.50:0/4023052471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f111415aa50 0x7f11140cd260 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.382+0000 7f1119a00640 1 -- 172.21.15.50:0/4023052471 >> 172.21.15.50:0/4023052471 conn(0x7f11140b2040 msgr2=0x7f11140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:11.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.386+0000 7f1119a00640 1 -- 172.21.15.50:0/4023052471 shutdown_connections 2024-09-21T06:13:11.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.386+0000 7f1119a00640 1 -- 172.21.15.50:0/4023052471 wait complete. 2024-09-21T06:13:11.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.386+0000 7f1119a00640 1 Processor -- start 2024-09-21T06:13:11.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.386+0000 7f1119a00640 1 -- start start 2024-09-21T06:13:11.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1119a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11140cd7a0 0x7f1114157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1119a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11140cdbb0 0x7f111414d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1119a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111415aa50 0x7f11141554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1119a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1114165ff0 con 0x7f111415aa50 2024-09-21T06:13:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1119a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1114165e70 con 0x7f11140cd7a0 2024-09-21T06:13:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1119a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f1114166170 con 0x7f11140cdbb0 2024-09-21T06:13:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1113400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111415aa50 0x7f11141554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1113400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111415aa50 0x7f11141554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41124/0 (socket says 172.21.15.50:41124) 2024-09-21T06:13:11.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1113400640 1 -- 172.21.15.50:0/2627230928 learned_addr learned my addr 172.21.15.50:0/2627230928 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:11.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f110be00640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11140cdbb0 0x7f111414d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:11.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1112a00640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11140cd7a0 0x7f1114157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:11.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1113400640 1 -- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11140cdbb0 msgr2=0x7f111414d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:11.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1113400640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11140cdbb0 0x7f111414d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1113400640 1 -- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11140cd7a0 msgr2=0x7f1114157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:11.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1113400640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11140cd7a0 0x7f1114157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1113400640 1 -- 172.21.15.50:0/2627230928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1114156b70 con 0x7f111415aa50 2024-09-21T06:13:11.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.390+0000 7f1113400640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111415aa50 0x7f11141554c0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f110c00b9d0 tx=0x7f110c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:11.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.394+0000 7f110aa00640 1 -- 172.21.15.50:0/2627230928 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c019070 con 0x7f111415aa50 2024-09-21T06:13:11.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.394+0000 7f110aa00640 1 -- 172.21.15.50:0/2627230928 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f110c00cce0 con 0x7f111415aa50 2024-09-21T06:13:11.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.394+0000 7f110aa00640 1 -- 172.21.15.50:0/2627230928 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0146a0 con 0x7f111415aa50 2024-09-21T06:13:11.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.394+0000 7f1119a00640 1 -- 172.21.15.50:0/2627230928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1114156e50 con 0x7f111415aa50 2024-09-21T06:13:11.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.394+0000 7f1119a00640 1 -- 172.21.15.50:0/2627230928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1114157410 con 0x7f111415aa50 2024-09-21T06:13:11.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.398+0000 7f1119a00640 1 -- 172.21.15.50:0/2627230928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111414f8b0 con 0x7f111415aa50 2024-09-21T06:13:11.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.398+0000 7f110aa00640 1 -- 172.21.15.50:0/2627230928 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 59) ==== 100651+0+0 (secure 0 0 0) 0x7f110c01c070 con 0x7f111415aa50 2024-09-21T06:13:11.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.402+0000 7f110aa00640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10f00757e0 0x7f10f0077cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.402+0000 7f110aa00640 1 -- 172.21.15.50:0/2627230928 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5793+0+0 (secure 0 0 0) 0x7f110c09a030 con 0x7f111415aa50 2024-09-21T06:13:11.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.410+0000 7f1112a00640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10f00757e0 0x7f10f0077cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:11.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.410+0000 7f110aa00640 1 -- 172.21.15.50:0/2627230928 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f110c05f620 con 0x7f111415aa50 2024-09-21T06:13:11.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.442+0000 7f1112a00640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10f00757e0 0x7f10f0077cf0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f11040af000 tx=0x7f110401b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:11.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.782+0000 7f1119a00640 1 -- 172.21.15.50:0/2627230928 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone90034", "target": ["mon-mgr", ""]}) -- 0x7f1114158b50 con 0x7f10f00757e0 2024-09-21T06:13:11.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:11.958+0000 7f110aa00640 1 -- 172.21.15.50:0/2627230928 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 60) ==== 100753+0+0 (secure 0 0 0) 0x7f110c05b890 con 0x7f111415aa50 2024-09-21T06:13:12.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.682+0000 7f110aa00640 1 -- 172.21.15.50:0/2627230928 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1114158b50 con 0x7f10f00757e0 2024-09-21T06:13:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.682+0000 7f1119a00640 1 -- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10f00757e0 msgr2=0x7f10f0077cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.682+0000 7f1119a00640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10f00757e0 0x7f10f0077cf0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f11040af000 tx=0x7f110401b120 comp rx=0 tx=0).stop 2024-09-21T06:13:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.682+0000 7f1119a00640 1 -- 172.21.15.50:0/2627230928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111415aa50 msgr2=0x7f11141554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.682+0000 7f1119a00640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111415aa50 0x7f11141554c0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f110c00b9d0 tx=0x7f110c00be90 comp rx=0 tx=0).stop 2024-09-21T06:13:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.686+0000 7f1119a00640 1 -- 172.21.15.50:0/2627230928 shutdown_connections 2024-09-21T06:13:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.686+0000 7f1119a00640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f10f00757e0 0x7f10f0077cf0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.686+0000 7f1119a00640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111415aa50 0x7f11141554c0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.686+0000 7f1119a00640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f11140cdbb0 0x7f111414d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.686+0000 7f1119a00640 1 --2- 172.21.15.50:0/2627230928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11140cd7a0 0x7f1114157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.686+0000 7f1119a00640 1 -- 172.21.15.50:0/2627230928 >> 172.21.15.50:0/2627230928 conn(0x7f11140b2040 msgr2=0x7f11140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.686+0000 7f1119a00640 1 -- 172.21.15.50:0/2627230928 shutdown_connections 2024-09-21T06:13:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:12.686+0000 7f1119a00640 1 -- 172.21.15.50:0/2627230928 wait complete. 2024-09-21T06:13:12.721 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone44825 2024-09-21T06:13:13.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.018+0000 7f5a71a00640 1 -- 172.21.15.50:0/560277502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a6c0cefa0 msgr2=0x7f5a6c0cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:13.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.018+0000 7f5a71a00640 1 --2- 172.21.15.50:0/560277502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a6c0cefa0 0x7f5a6c0cf3b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009840 tx=0x7f5a5801c6b0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.018+0000 7f5a71a00640 1 -- 172.21.15.50:0/560277502 shutdown_connections 2024-09-21T06:13:13.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.018+0000 7f5a71a00640 1 --2- 172.21.15.50:0/560277502 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a6c0b3340 0x7f5a6c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.018+0000 7f5a71a00640 1 --2- 172.21.15.50:0/560277502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a6c0cefa0 0x7f5a6c0cf3b0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.018+0000 7f5a71a00640 1 -- 172.21.15.50:0/560277502 >> 172.21.15.50:0/560277502 conn(0x7f5a6c0b2040 msgr2=0x7f5a6c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:13.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.018+0000 7f5a71a00640 1 -- 172.21.15.50:0/560277502 shutdown_connections 2024-09-21T06:13:13.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.018+0000 7f5a71a00640 1 -- 172.21.15.50:0/560277502 wait complete. 2024-09-21T06:13:13.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.022+0000 7f5a71a00640 1 Processor -- start 2024-09-21T06:13:13.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.022+0000 7f5a71a00640 1 -- start start 2024-09-21T06:13:13.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a71a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a6c0749a0 0x7f5a6c159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:13.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a71a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a6c0b3340 0x7f5a6c13f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:13.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a71a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a6c0cefa0 0x7f5a6c13f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:13.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a71a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c169e10 con 0x7f5a6c0749a0 2024-09-21T06:13:13.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a71a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c169c90 con 0x7f5a6c0b3340 2024-09-21T06:13:13.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a71a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5a6c169f90 con 0x7f5a6c0cefa0 2024-09-21T06:13:13.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a6aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a6c0749a0 0x7f5a6c159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:13.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a6aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a6c0749a0 0x7f5a6c159280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41144/0 (socket says 172.21.15.50:41144) 2024-09-21T06:13:13.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a6aa00640 1 -- 172.21.15.50:0/1372911558 learned_addr learned my addr 172.21.15.50:0/1372911558 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:13.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a6aa00640 1 -- 172.21.15.50:0/1372911558 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a6c0cefa0 msgr2=0x7f5a6c13f710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:13:13.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a6aa00640 1 --2- 172.21.15.50:0/1372911558 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a6c0cefa0 0x7f5a6c13f710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a6aa00640 1 -- 172.21.15.50:0/1372911558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a6c0b3340 msgr2=0x7f5a6c13f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:13.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a6aa00640 1 --2- 172.21.15.50:0/1372911558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a6c0b3340 0x7f5a6c13f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a6aa00640 1 -- 172.21.15.50:0/1372911558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c139990 con 0x7f5a6c0749a0 2024-09-21T06:13:13.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.026+0000 7f5a6aa00640 1 --2- 172.21.15.50:0/1372911558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a6c0749a0 0x7f5a6c159280 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00c560 tx=0x7f5a5c00ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:13.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.030+0000 7f5a62a00640 1 -- 172.21.15.50:0/1372911558 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c017070 con 0x7f5a6c0749a0 2024-09-21T06:13:13.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.030+0000 7f5a71a00640 1 -- 172.21.15.50:0/1372911558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c139c70 con 0x7f5a6c0749a0 2024-09-21T06:13:13.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.030+0000 7f5a71a00640 1 -- 172.21.15.50:0/1372911558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c143460 con 0x7f5a6c0749a0 2024-09-21T06:13:13.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.030+0000 7f5a62a00640 1 -- 172.21.15.50:0/1372911558 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a5c003600 con 0x7f5a6c0749a0 2024-09-21T06:13:13.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.030+0000 7f5a62a00640 1 -- 172.21.15.50:0/1372911558 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c003a50 con 0x7f5a6c0749a0 2024-09-21T06:13:13.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.034+0000 7f5a71a00640 1 -- 172.21.15.50:0/1372911558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a440052a0 con 0x7f5a6c0749a0 2024-09-21T06:13:13.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.034+0000 7f5a62a00640 1 -- 172.21.15.50:0/1372911558 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 60) ==== 100753+0+0 (secure 0 0 0) 0x7f5a5c01a070 con 0x7f5a6c0749a0 2024-09-21T06:13:13.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.034+0000 7f5a62a00640 1 --2- 172.21.15.50:0/1372911558 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a4c0759a0 0x7f5a4c077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:13.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.034+0000 7f5a62a00640 1 -- 172.21.15.50:0/1372911558 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f5a5c0928b0 con 0x7f5a6c0749a0 2024-09-21T06:13:13.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.034+0000 7f5a6a000640 1 --2- 172.21.15.50:0/1372911558 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a4c0759a0 0x7f5a4c077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:13.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.038+0000 7f5a6a000640 1 --2- 172.21.15.50:0/1372911558 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a4c0759a0 0x7f5a4c077eb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f5a580af000 tx=0x7f5a58019450 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:13.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.042+0000 7f5a62a00640 1 -- 172.21.15.50:0/1372911558 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5a5c05f3b0 con 0x7f5a6c0749a0 2024-09-21T06:13:13.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:13.418+0000 7f5a71a00640 1 -- 172.21.15.50:0/1372911558 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone44825", "target": ["mon-mgr", ""]}) -- 0x7f5a44000c90 con 0x7f5a4c0759a0 2024-09-21T06:13:14.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.198+0000 7f5a62a00640 1 -- 172.21.15.50:0/1372911558 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5a44000c90 con 0x7f5a4c0759a0 2024-09-21T06:13:14.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.202+0000 7f5a71a00640 1 -- 172.21.15.50:0/1372911558 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a4c0759a0 msgr2=0x7f5a4c077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:14.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.202+0000 7f5a71a00640 1 --2- 172.21.15.50:0/1372911558 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a4c0759a0 0x7f5a4c077eb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f5a580af000 tx=0x7f5a58019450 comp rx=0 tx=0).stop 2024-09-21T06:13:14.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.202+0000 7f5a71a00640 1 -- 172.21.15.50:0/1372911558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a6c0749a0 msgr2=0x7f5a6c159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:14.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.202+0000 7f5a71a00640 1 --2- 172.21.15.50:0/1372911558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a6c0749a0 0x7f5a6c159280 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00c560 tx=0x7f5a5c00ca20 comp rx=0 tx=0).stop 2024-09-21T06:13:14.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.202+0000 7f5a71a00640 1 -- 172.21.15.50:0/1372911558 shutdown_connections 2024-09-21T06:13:14.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.206+0000 7f5a71a00640 1 --2- 172.21.15.50:0/1372911558 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5a6c0cefa0 0x7f5a6c13f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.206+0000 7f5a71a00640 1 --2- 172.21.15.50:0/1372911558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a6c0b3340 0x7f5a6c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.206+0000 7f5a71a00640 1 --2- 172.21.15.50:0/1372911558 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5a4c0759a0 0x7f5a4c077eb0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.206+0000 7f5a71a00640 1 --2- 172.21.15.50:0/1372911558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a6c0749a0 0x7f5a6c159280 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.206+0000 7f5a71a00640 1 -- 172.21.15.50:0/1372911558 >> 172.21.15.50:0/1372911558 conn(0x7f5a6c0b2040 msgr2=0x7f5a6c074db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:14.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.206+0000 7f5a71a00640 1 -- 172.21.15.50:0/1372911558 shutdown_connections 2024-09-21T06:13:14.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.206+0000 7f5a71a00640 1 -- 172.21.15.50:0/1372911558 wait complete. 2024-09-21T06:13:14.252 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone64946 2024-09-21T06:13:14.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.566+0000 7f6dbde00640 1 -- 172.21.15.50:0/3351042280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da0003a40 msgr2=0x7f6da0003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:14.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.566+0000 7f6dbde00640 1 --2- 172.21.15.50:0/3351042280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da0003a40 0x7f6da0003e90 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c730 tx=0x7f6dac01c700 comp rx=0 tx=0).stop 2024-09-21T06:13:14.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.566+0000 7f6dbd400640 1 -- 172.21.15.50:0/3351042280 reap_dead start 2024-09-21T06:13:14.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.566+0000 7f6dbde00640 1 -- 172.21.15.50:0/3351042280 shutdown_connections 2024-09-21T06:13:14.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.566+0000 7f6dbde00640 1 -- 172.21.15.50:0/3351042280 >> 172.21.15.50:0/3351042280 conn(0x7f6db806d3f0 msgr2=0x7f6db80b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:14.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.574+0000 7f6dbde00640 1 -- 172.21.15.50:0/3351042280 shutdown_connections 2024-09-21T06:13:14.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.574+0000 7f6dbde00640 1 -- 172.21.15.50:0/3351042280 wait complete. 2024-09-21T06:13:14.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.582+0000 7f6dbde00640 1 Processor -- start 2024-09-21T06:13:14.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.582+0000 7f6dbde00640 1 -- start start 2024-09-21T06:13:14.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.586+0000 7f6dbde00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6db81524c0 0x7f6db81fc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:14.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.586+0000 7f6dbde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6db81fce30 0x7f6db80ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:14.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.586+0000 7f6dbde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6db80aedd0 0x7f6db81ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:14.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.586+0000 7f6dbde00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81666f0 con 0x7f6db81fce30 2024-09-21T06:13:14.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.586+0000 7f6dbde00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6db8166570 con 0x7f6db80aedd0 2024-09-21T06:13:14.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.586+0000 7f6dbde00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6db8166870 con 0x7f6db81524c0 2024-09-21T06:13:14.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.586+0000 7f6dbd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6db80aedd0 0x7f6db81ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:14.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.586+0000 7f6dbd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6db80aedd0 0x7f6db81ffee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:35426/0 (socket says 172.21.15.50:35426) 2024-09-21T06:13:14.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.586+0000 7f6dbd400640 1 -- 172.21.15.50:0/2437279741 learned_addr learned my addr 172.21.15.50:0/2437279741 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:14.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.586+0000 7f6db7e00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6db81fce30 0x7f6db80ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:14.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.586+0000 7f6dbca00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6db81524c0 0x7f6db81fc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:14.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.590+0000 7f6dbd400640 1 -- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6db81524c0 msgr2=0x7f6db81fc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:14.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.590+0000 7f6dbd400640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6db81524c0 0x7f6db81fc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.590+0000 7f6dbd400640 1 -- 172.21.15.50:0/2437279741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6db81fce30 msgr2=0x7f6db80ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:14.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.590+0000 7f6dbd400640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6db81fce30 0x7f6db80ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.590+0000 7f6dbd400640 1 -- 172.21.15.50:0/2437279741 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db80af1e0 con 0x7f6db80aedd0 2024-09-21T06:13:14.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.590+0000 7f6db7e00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6db81fce30 0x7f6db80ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:14.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.590+0000 7f6dbca00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6db81524c0 0x7f6db81fc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:14.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.590+0000 7f6dbd400640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6db80aedd0 0x7f6db81ffee0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6db80b30e0 tx=0x7f6db000dc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:14.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.590+0000 7f6db4c00640 1 -- 172.21.15.50:0/2437279741 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db000de40 con 0x7f6db80aedd0 2024-09-21T06:13:14.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.590+0000 7f6dbde00640 1 -- 172.21.15.50:0/2437279741 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db81fd320 con 0x7f6db80aedd0 2024-09-21T06:13:14.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.594+0000 7f6dbde00640 1 -- 172.21.15.50:0/2437279741 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db8154d50 con 0x7f6db80aedd0 2024-09-21T06:13:14.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.594+0000 7f6db4c00640 1 -- 172.21.15.50:0/2437279741 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6db000bb10 con 0x7f6db80aedd0 2024-09-21T06:13:14.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.594+0000 7f6db4c00640 1 -- 172.21.15.50:0/2437279741 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0003980 con 0x7f6db80aedd0 2024-09-21T06:13:14.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.594+0000 7f6dbde00640 1 -- 172.21.15.50:0/2437279741 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d940052a0 con 0x7f6db80aedd0 2024-09-21T06:13:14.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.598+0000 7f6db4c00640 1 -- 172.21.15.50:0/2437279741 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 60) ==== 100753+0+0 (secure 0 0 0) 0x7f6db001b020 con 0x7f6db80aedd0 2024-09-21T06:13:14.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.598+0000 7f6db4c00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d980758d0 0x7f6d98077de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:14.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.602+0000 7f6dbca00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d980758d0 0x7f6d98077de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:14.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.610+0000 7f6db4c00640 1 -- 172.21.15.50:0/2437279741 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f6db005b900 con 0x7f6db80aedd0 2024-09-21T06:13:14.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.614+0000 7f6dbca00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d980758d0 0x7f6d98077de0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c700 tx=0x7f6dac0194a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:14.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.626+0000 7f6db4c00640 1 -- 172.21.15.50:0/2437279741 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6db80af1e0 con 0x7f6db80aedd0 2024-09-21T06:13:14.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:14.938+0000 7f6dbde00640 1 -- 172.21.15.50:0/2437279741 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone64946", "target": ["mon-mgr", ""]}) -- 0x7f6d94000c90 con 0x7f6d980758d0 2024-09-21T06:13:15.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.602+0000 7f6db4c00640 1 -- 172.21.15.50:0/2437279741 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6d94000c90 con 0x7f6d980758d0 2024-09-21T06:13:15.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.606+0000 7f6dbde00640 1 -- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d980758d0 msgr2=0x7f6d98077de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:15.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.606+0000 7f6dbde00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d980758d0 0x7f6d98077de0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c700 tx=0x7f6dac0194a0 comp rx=0 tx=0).stop 2024-09-21T06:13:15.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.606+0000 7f6dbde00640 1 -- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6db80aedd0 msgr2=0x7f6db81ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:15.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.606+0000 7f6dbde00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6db80aedd0 0x7f6db81ffee0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6db80b30e0 tx=0x7f6db000dc50 comp rx=0 tx=0).stop 2024-09-21T06:13:15.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.606+0000 7f6dbde00640 1 -- 172.21.15.50:0/2437279741 shutdown_connections 2024-09-21T06:13:15.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.606+0000 7f6dbde00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6db80aedd0 0x7f6db81ffee0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:15.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.606+0000 7f6dbde00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6d980758d0 0x7f6d98077de0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:15.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.606+0000 7f6dbde00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6db81fce30 0x7f6db80ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:15.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.606+0000 7f6dbde00640 1 --2- 172.21.15.50:0/2437279741 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6db81524c0 0x7f6db81fc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:15.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.606+0000 7f6dbde00640 1 -- 172.21.15.50:0/2437279741 >> 172.21.15.50:0/2437279741 conn(0x7f6db806d3f0 msgr2=0x7f6db81542f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:15.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.606+0000 7f6dbde00640 1 -- 172.21.15.50:0/2437279741 shutdown_connections 2024-09-21T06:13:15.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:15.606+0000 7f6dbde00640 1 -- 172.21.15.50:0/2437279741 wait complete. 2024-09-21T06:13:16.643 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99400 2024-09-21T06:13:16.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.954+0000 7f51aa000640 1 -- 172.21.15.50:0/1616701635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188004810 msgr2=0x7f5188004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.954+0000 7f51aa000640 1 --2- 172.21.15.50:0/1616701635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188004810 0x7f5188004c20 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f51a4075440 tx=0x7f519401f420 comp rx=0 tx=0).stop 2024-09-21T06:13:16.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.966+0000 7f51aa000640 1 -- 172.21.15.50:0/1616701635 shutdown_connections 2024-09-21T06:13:16.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.966+0000 7f51aa000640 1 --2- 172.21.15.50:0/1616701635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188004810 0x7f5188004c20 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.966+0000 7f51aa000640 1 -- 172.21.15.50:0/1616701635 >> 172.21.15.50:0/1616701635 conn(0x7f51a406e4a0 msgr2=0x7f51a406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:16.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.966+0000 7f51aa000640 1 -- 172.21.15.50:0/1616701635 shutdown_connections 2024-09-21T06:13:16.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.966+0000 7f51aa000640 1 -- 172.21.15.50:0/1616701635 wait complete. 2024-09-21T06:13:16.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.974+0000 7f51aa000640 1 Processor -- start 2024-09-21T06:13:16.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.974+0000 7f51aa000640 1 -- start start 2024-09-21T06:13:16.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.974+0000 7f51aa000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188004810 0x7f51a41f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.974+0000 7f51aa000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f51a41ed7d0 0x7f51a41edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.974+0000 7f51aa000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51a41456c0 0x7f51a4145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51aa000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4072980 con 0x7f5188004810 2024-09-21T06:13:16.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51aa000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4072800 con 0x7f51a41456c0 2024-09-21T06:13:16.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51aa000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f51a4072b00 con 0x7f51a41ed7d0 2024-09-21T06:13:16.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51a8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188004810 0x7f51a41f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51a8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188004810 0x7f51a41f86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44536/0 (socket says 172.21.15.50:44536) 2024-09-21T06:13:16.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51a8c00640 1 -- 172.21.15.50:0/491639536 learned_addr learned my addr 172.21.15.50:0/491639536 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:16.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51a8c00640 1 -- 172.21.15.50:0/491639536 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f51a41ed7d0 msgr2=0x7f51a41edbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51a8c00640 1 --2- 172.21.15.50:0/491639536 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f51a41ed7d0 0x7f51a41edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51a8c00640 1 -- 172.21.15.50:0/491639536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51a41456c0 msgr2=0x7f51a4145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51a8c00640 1 --2- 172.21.15.50:0/491639536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51a41456c0 0x7f51a4145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51a8c00640 1 -- 172.21.15.50:0/491639536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a4073450 con 0x7f5188004810 2024-09-21T06:13:16.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51a8c00640 1 --2- 172.21.15.50:0/491639536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188004810 0x7f51a41f86f0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f519000c700 tx=0x7f519000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:16.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51a0c00640 1 -- 172.21.15.50:0/491639536 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190017070 con 0x7f5188004810 2024-09-21T06:13:16.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51aa000640 1 -- 172.21.15.50:0/491639536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a4146010 con 0x7f5188004810 2024-09-21T06:13:16.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51a0c00640 1 -- 172.21.15.50:0/491639536 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5190005380 con 0x7f5188004810 2024-09-21T06:13:16.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51a0c00640 1 -- 172.21.15.50:0/491639536 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519000a8a0 con 0x7f5188004810 2024-09-21T06:13:16.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51aa000640 1 -- 172.21.15.50:0/491639536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a4143a90 con 0x7f5188004810 2024-09-21T06:13:16.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.978+0000 7f51aa000640 1 -- 172.21.15.50:0/491639536 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a40b3180 con 0x7f5188004810 2024-09-21T06:13:16.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.982+0000 7f51a0c00640 1 -- 172.21.15.50:0/491639536 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 60) ==== 100753+0+0 (secure 0 0 0) 0x7f519001a070 con 0x7f5188004810 2024-09-21T06:13:16.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.986+0000 7f51a0c00640 1 --2- 172.21.15.50:0/491639536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f51840758d0 0x7f5184077de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.986+0000 7f51a0c00640 1 -- 172.21.15.50:0/491639536 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f5190092ba0 con 0x7f5188004810 2024-09-21T06:13:16.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.986+0000 7f51a3e00640 1 --2- 172.21.15.50:0/491639536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f51840758d0 0x7f5184077de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.990+0000 7f51a3e00640 1 --2- 172.21.15.50:0/491639536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f51840758d0 0x7f5184077de0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f51940af000 tx=0x7f5188005840 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:16.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:16.998+0000 7f51a0c00640 1 -- 172.21.15.50:0/491639536 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f51a4146010 con 0x7f5188004810 2024-09-21T06:13:17.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:17.358+0000 7f51aa000640 1 -- 172.21.15.50:0/491639536 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99400", "target": ["mon-mgr", ""]}) -- 0x7f51a406b540 con 0x7f51840758d0 2024-09-21T06:13:17.458 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:17.457+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:17.458 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:17.457+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:17.458 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:17.457+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:17.458 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:17.457+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:17.458 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:17.457+0000 7f65a0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:19.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.194+0000 7f51a0c00640 1 -- 172.21.15.50:0/491639536 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f51940110f0 con 0x7f51840758d0 2024-09-21T06:13:19.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.198+0000 7f51aa000640 1 -- 172.21.15.50:0/491639536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f51840758d0 msgr2=0x7f5184077de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.198+0000 7f51aa000640 1 --2- 172.21.15.50:0/491639536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f51840758d0 0x7f5184077de0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f51940af000 tx=0x7f5188005840 comp rx=0 tx=0).stop 2024-09-21T06:13:19.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.198+0000 7f51aa000640 1 -- 172.21.15.50:0/491639536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188004810 msgr2=0x7f51a41f86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.198+0000 7f51aa000640 1 --2- 172.21.15.50:0/491639536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188004810 0x7f51a41f86f0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f519000c700 tx=0x7f519000cbc0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.198+0000 7f51aa000640 1 -- 172.21.15.50:0/491639536 shutdown_connections 2024-09-21T06:13:19.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.198+0000 7f51aa000640 1 --2- 172.21.15.50:0/491639536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51a41456c0 0x7f51a4145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.198+0000 7f51aa000640 1 --2- 172.21.15.50:0/491639536 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f51a41ed7d0 0x7f51a41edbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.198+0000 7f51aa000640 1 --2- 172.21.15.50:0/491639536 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f51840758d0 0x7f5184077de0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.198+0000 7f51aa000640 1 --2- 172.21.15.50:0/491639536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188004810 0x7f51a41f86f0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.198+0000 7f51aa000640 1 -- 172.21.15.50:0/491639536 >> 172.21.15.50:0/491639536 conn(0x7f51a406e4a0 msgr2=0x7f51a415cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:19.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.198+0000 7f51aa000640 1 -- 172.21.15.50:0/491639536 shutdown_connections 2024-09-21T06:13:19.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.198+0000 7f51aa000640 1 -- 172.21.15.50:0/491639536 wait complete. 2024-09-21T06:13:19.237 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone5031 2024-09-21T06:13:19.414 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:13:19.420 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.546+0000 7fa2a7200640 1 -- 172.21.15.50:0/1774057947 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2a80b4930 msgr2=0x7fa2a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.546+0000 7fa2a7200640 1 --2- 172.21.15.50:0/1774057947 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2a80b4930 0x7fa2a80b4d40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000a980 tx=0x7fa2a001db90 comp rx=0 tx=0).stop 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.546+0000 7fa2a7200640 1 -- 172.21.15.50:0/1774057947 shutdown_connections 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.546+0000 7fa2a7200640 1 --2- 172.21.15.50:0/1774057947 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2a80b4930 0x7fa2a80b4d40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.546+0000 7fa2a7200640 1 -- 172.21.15.50:0/1774057947 >> 172.21.15.50:0/1774057947 conn(0x7fa2a806cdf0 msgr2=0x7fa2a806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.546+0000 7fa2a7200640 1 -- 172.21.15.50:0/1774057947 shutdown_connections 2024-09-21T06:13:19.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.546+0000 7fa2a7200640 1 -- 172.21.15.50:0/1774057947 wait complete. 2024-09-21T06:13:19.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.550+0000 7fa2a7200640 1 Processor -- start 2024-09-21T06:13:19.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.550+0000 7fa2a7200640 1 -- start start 2024-09-21T06:13:19.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a80b4930 0x7fa2a8165c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a815b850 0x7fa2a814b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a7200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2a80c6950 0x7fa2a80c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a7200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8169430 con 0x7fa2a815b850 2024-09-21T06:13:19.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81692b0 con 0x7fa2a80b4930 2024-09-21T06:13:19.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a7200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a81695b0 con 0x7fa2a80c6950 2024-09-21T06:13:19.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a815b850 0x7fa2a814b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a815b850 0x7fa2a814b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44544/0 (socket says 172.21.15.50:44544) 2024-09-21T06:13:19.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a5400640 1 -- 172.21.15.50:0/2794980793 learned_addr learned my addr 172.21.15.50:0/2794980793 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:19.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a6800640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2a80c6950 0x7fa2a80c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a5400640 1 -- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2a80c6950 msgr2=0x7fa2a80c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a5400640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2a80c6950 0x7fa2a80c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a5400640 1 -- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a80b4930 msgr2=0x7fa2a8165c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a5400640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a80b4930 0x7fa2a8165c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a5400640 1 -- 172.21.15.50:0/2794980793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a814bec0 con 0x7fa2a815b850 2024-09-21T06:13:19.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a5400640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a815b850 0x7fa2a814b980 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c430 tx=0x7fa29400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:19.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa29e000640 1 -- 172.21.15.50:0/2794980793 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294011790 con 0x7fa2a815b850 2024-09-21T06:13:19.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa29e000640 1 -- 172.21.15.50:0/2794980793 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa294017070 con 0x7fa2a815b850 2024-09-21T06:13:19.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a7200640 1 -- 172.21.15.50:0/2794980793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a81f9130 con 0x7fa2a815b850 2024-09-21T06:13:19.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa29e000640 1 -- 172.21.15.50:0/2794980793 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294011cc0 con 0x7fa2a815b850 2024-09-21T06:13:19.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a7200640 1 -- 172.21.15.50:0/2794980793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81f96f0 con 0x7fa2a815b850 2024-09-21T06:13:19.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.554+0000 7fa2a7200640 1 -- 172.21.15.50:0/2794980793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a80b4d40 con 0x7fa2a815b850 2024-09-21T06:13:19.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.558+0000 7fa29e000640 1 -- 172.21.15.50:0/2794980793 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7fa29401c460 con 0x7fa2a815b850 2024-09-21T06:13:19.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.558+0000 7fa29e000640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa288075960 0x7fa288077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.558+0000 7fa29e000640 1 -- 172.21.15.50:0/2794980793 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7fa294093330 con 0x7fa2a815b850 2024-09-21T06:13:19.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.562+0000 7fa29e000640 1 -- 172.21.15.50:0/2794980793 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa29405fe50 con 0x7fa2a815b850 2024-09-21T06:13:19.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.578+0000 7fa2a5e00640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa288075960 0x7fa288077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.594+0000 7fa2a5e00640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa288075960 0x7fa288077e70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa298007fd0 tx=0x7fa298007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:19.918+0000 7fa2a7200640 1 -- 172.21.15.50:0/2794980793 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone5031", "target": ["mon-mgr", ""]}) -- 0x7fa2a806b540 con 0x7fa288075960 2024-09-21T06:13:21.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.017+0000 7fa29e000640 1 -- 172.21.15.50:0/2794980793 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa298003980 con 0x7fa288075960 2024-09-21T06:13:21.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.021+0000 7fa2a7200640 1 -- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa288075960 msgr2=0x7fa288077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.021+0000 7fa2a7200640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa288075960 0x7fa288077e70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa298007fd0 tx=0x7fa298007d00 comp rx=0 tx=0).stop 2024-09-21T06:13:21.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.021+0000 7fa2a7200640 1 -- 172.21.15.50:0/2794980793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a815b850 msgr2=0x7fa2a814b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.021+0000 7fa2a7200640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a815b850 0x7fa2a814b980 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c430 tx=0x7fa29400c8f0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.021+0000 7fa2a7200640 1 -- 172.21.15.50:0/2794980793 shutdown_connections 2024-09-21T06:13:21.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.021+0000 7fa2a7200640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa2a80c6950 0x7fa2a80c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.021+0000 7fa2a7200640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa288075960 0x7fa288077e70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.021+0000 7fa2a7200640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a815b850 0x7fa2a814b980 secure :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c430 tx=0x7fa29400c8f0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.021+0000 7fa2a7200640 1 --2- 172.21.15.50:0/2794980793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a80b4930 0x7fa2a8165c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.021+0000 7fa2a7200640 1 -- 172.21.15.50:0/2794980793 >> 172.21.15.50:0/2794980793 conn(0x7fa2a806cdf0 msgr2=0x7fa2a806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:21.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.021+0000 7fa2a7200640 1 -- 172.21.15.50:0/2794980793 shutdown_connections 2024-09-21T06:13:21.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.021+0000 7fa2a7200640 1 -- 172.21.15.50:0/2794980793 wait complete. 2024-09-21T06:13:21.075 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone34632 2024-09-21T06:13:21.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.333+0000 7f0caee00640 1 -- 172.21.15.50:0/3409580129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca80af5b0 msgr2=0x7f0ca806fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.333+0000 7f0caee00640 1 --2- 172.21.15.50:0/3409580129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca80af5b0 0x7f0ca806fb10 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8074c60 tx=0x7f0c9c01c770 comp rx=0 tx=0).stop 2024-09-21T06:13:21.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.333+0000 7f0caee00640 1 -- 172.21.15.50:0/3409580129 shutdown_connections 2024-09-21T06:13:21.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.333+0000 7f0caee00640 1 --2- 172.21.15.50:0/3409580129 >> v2:172.21.15.88:3300/0 conn(0x7f0ca8070050 0x7f0ca8072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.333+0000 7f0caee00640 1 --2- 172.21.15.50:0/3409580129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca80af5b0 0x7f0ca806fb10 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.333+0000 7f0caee00640 1 -- 172.21.15.50:0/3409580129 >> 172.21.15.50:0/3409580129 conn(0x7f0ca806d3f0 msgr2=0x7f0ca80b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:21.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.333+0000 7f0caee00640 1 -- 172.21.15.50:0/3409580129 shutdown_connections 2024-09-21T06:13:21.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.333+0000 7f0caee00640 1 -- 172.21.15.50:0/3409580129 wait complete. 2024-09-21T06:13:21.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.333+0000 7f0caee00640 1 Processor -- start 2024-09-21T06:13:21.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.333+0000 7f0caee00640 1 -- start start 2024-09-21T06:13:21.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0caee00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ca80730e0 0x7f0ca814bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0caee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca80af5b0 0x7f0ca814c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0caee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca815c6f0 0x7f0ca81fd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0caee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca8168f20 con 0x7f0ca815c6f0 2024-09-21T06:13:21.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0caee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca8168da0 con 0x7f0ca80af5b0 2024-09-21T06:13:21.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0caee00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca81690a0 con 0x7f0ca80730e0 2024-09-21T06:13:21.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0cae400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca815c6f0 0x7f0ca81fd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0cae400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca815c6f0 0x7f0ca81fd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44558/0 (socket says 172.21.15.50:44558) 2024-09-21T06:13:21.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0cae400640 1 -- 172.21.15.50:0/2863426560 learned_addr learned my addr 172.21.15.50:0/2863426560 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:21.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0cae400640 1 -- 172.21.15.50:0/2863426560 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ca80730e0 msgr2=0x7f0ca814bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0cae400640 1 --2- 172.21.15.50:0/2863426560 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ca80730e0 0x7f0ca814bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0cae400640 1 -- 172.21.15.50:0/2863426560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca80af5b0 msgr2=0x7f0ca814c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0cae400640 1 --2- 172.21.15.50:0/2863426560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca80af5b0 0x7f0ca814c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0cae400640 1 -- 172.21.15.50:0/2863426560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca80cd260 con 0x7f0ca815c6f0 2024-09-21T06:13:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0cae400640 1 --2- 172.21.15.50:0/2863426560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca815c6f0 0x7f0ca81fd580 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000b990 tx=0x7f0ca000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0ca5600640 1 -- 172.21.15.50:0/2863426560 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0019070 con 0x7f0ca815c6f0 2024-09-21T06:13:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0ca5600640 1 -- 172.21.15.50:0/2863426560 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ca000cc90 con 0x7f0ca815c6f0 2024-09-21T06:13:21.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0caee00640 1 -- 172.21.15.50:0/2863426560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca814c580 con 0x7f0ca815c6f0 2024-09-21T06:13:21.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.337+0000 7f0ca5600640 1 -- 172.21.15.50:0/2863426560 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00146a0 con 0x7f0ca815c6f0 2024-09-21T06:13:21.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.341+0000 7f0caee00640 1 -- 172.21.15.50:0/2863426560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca815d170 con 0x7f0ca815c6f0 2024-09-21T06:13:21.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.341+0000 7f0caee00640 1 -- 172.21.15.50:0/2863426560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca8072ba0 con 0x7f0ca815c6f0 2024-09-21T06:13:21.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.341+0000 7f0ca5600640 1 -- 172.21.15.50:0/2863426560 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f0ca001c070 con 0x7f0ca815c6f0 2024-09-21T06:13:21.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.345+0000 7f0ca5600640 1 --2- 172.21.15.50:0/2863426560 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c88075960 0x7f0c88077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.345+0000 7f0ca5600640 1 -- 172.21.15.50:0/2863426560 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f0ca0093290 con 0x7f0ca815c6f0 2024-09-21T06:13:21.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.349+0000 7f0ca5600640 1 -- 172.21.15.50:0/2863426560 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0ca005fe40 con 0x7f0ca815c6f0 2024-09-21T06:13:21.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.361+0000 7f0cada00640 1 --2- 172.21.15.50:0/2863426560 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c88075960 0x7f0c88077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.381+0000 7f0cada00640 1 --2- 172.21.15.50:0/2863426560 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c88075960 0x7f0c88077e70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0af000 tx=0x7f0c9c019510 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:21.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:21.705+0000 7f0caee00640 1 -- 172.21.15.50:0/2863426560 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone34632", "target": ["mon-mgr", ""]}) -- 0x7f0ca806b540 con 0x7f0c88075960 2024-09-21T06:13:22.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.485+0000 7f0ca5600640 1 -- 172.21.15.50:0/2863426560 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0c9c004020 con 0x7f0c88075960 2024-09-21T06:13:22.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.489+0000 7f0caee00640 1 -- 172.21.15.50:0/2863426560 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c88075960 msgr2=0x7f0c88077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.489+0000 7f0caee00640 1 --2- 172.21.15.50:0/2863426560 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c88075960 0x7f0c88077e70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0af000 tx=0x7f0c9c019510 comp rx=0 tx=0).stop 2024-09-21T06:13:22.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.489+0000 7f0caee00640 1 -- 172.21.15.50:0/2863426560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca815c6f0 msgr2=0x7f0ca81fd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.489+0000 7f0caee00640 1 --2- 172.21.15.50:0/2863426560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca815c6f0 0x7f0ca81fd580 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000b990 tx=0x7f0ca000be50 comp rx=0 tx=0).stop 2024-09-21T06:13:22.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.489+0000 7f0caee00640 1 -- 172.21.15.50:0/2863426560 shutdown_connections 2024-09-21T06:13:22.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.489+0000 7f0caee00640 1 --2- 172.21.15.50:0/2863426560 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0c88075960 0x7f0c88077e70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.489+0000 7f0caee00640 1 --2- 172.21.15.50:0/2863426560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca815c6f0 0x7f0ca81fd580 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.489+0000 7f0caee00640 1 --2- 172.21.15.50:0/2863426560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca80af5b0 0x7f0ca814c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.489+0000 7f0caee00640 1 --2- 172.21.15.50:0/2863426560 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0ca80730e0 0x7f0ca814bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.489+0000 7f0caee00640 1 -- 172.21.15.50:0/2863426560 >> 172.21.15.50:0/2863426560 conn(0x7f0ca806d3f0 msgr2=0x7f0ca80d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:22.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.489+0000 7f0caee00640 1 -- 172.21.15.50:0/2863426560 shutdown_connections 2024-09-21T06:13:22.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.489+0000 7f0caee00640 1 -- 172.21.15.50:0/2863426560 wait complete. 2024-09-21T06:13:22.544 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone74693 2024-09-21T06:13:22.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.829+0000 7f15cfe00640 1 -- 172.21.15.50:0/2455318050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15d015b6f0 msgr2=0x7f15d006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.829+0000 7f15cfe00640 1 --2- 172.21.15.50:0/2455318050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15d015b6f0 0x7f15d006e890 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009bc0 tx=0x7f15c801cba0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.829+0000 7f15cfe00640 1 -- 172.21.15.50:0/2455318050 shutdown_connections 2024-09-21T06:13:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.829+0000 7f15cfe00640 1 --2- 172.21.15.50:0/2455318050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15d015b6f0 0x7f15d006e890 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.829+0000 7f15cfe00640 1 -- 172.21.15.50:0/2455318050 >> 172.21.15.50:0/2455318050 conn(0x7f15d006d1f0 msgr2=0x7f15d006d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:22.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.829+0000 7f15cfe00640 1 -- 172.21.15.50:0/2455318050 shutdown_connections 2024-09-21T06:13:22.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.829+0000 7f15cfe00640 1 -- 172.21.15.50:0/2455318050 wait complete. 2024-09-21T06:13:22.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.833+0000 7f15cfe00640 1 Processor -- start 2024-09-21T06:13:22.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.833+0000 7f15cfe00640 1 -- start start 2024-09-21T06:13:22.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.833+0000 7f15cfe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15d006fb20 0x7f15d014fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.833+0000 7f15cfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15d015b6f0 0x7f15d00ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.833+0000 7f15cfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15d01502b0 0x7f15d00aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.833+0000 7f15cfe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f15d00cdea0 con 0x7f15d015b6f0 2024-09-21T06:13:22.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.833+0000 7f15cfe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f15d00cdd20 con 0x7f15d01502b0 2024-09-21T06:13:22.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.833+0000 7f15cfe00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f15d016b620 con 0x7f15d006fb20 2024-09-21T06:13:22.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15ce000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15d015b6f0 0x7f15d00ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15ce000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15d015b6f0 0x7f15d00ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44574/0 (socket says 172.21.15.50:44574) 2024-09-21T06:13:22.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15ce000640 1 -- 172.21.15.50:0/1515001369 learned_addr learned my addr 172.21.15.50:0/1515001369 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:22.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15cea00640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15d006fb20 0x7f15d014fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15ce000640 1 -- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15d006fb20 msgr2=0x7f15d014fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15ce000640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15d006fb20 0x7f15d014fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15ce000640 1 -- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15d01502b0 msgr2=0x7f15d00aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15ce000640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15d01502b0 0x7f15d00aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15ce000640 1 -- 172.21.15.50:0/1515001369 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d00b04b0 con 0x7f15d015b6f0 2024-09-21T06:13:22.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15ce000640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15d015b6f0 0x7f15d00ae8e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f15c000c430 tx=0x7f15c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:22.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15c6a00640 1 -- 172.21.15.50:0/1515001369 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0017070 con 0x7f15d015b6f0 2024-09-21T06:13:22.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15c6a00640 1 -- 172.21.15.50:0/1515001369 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15c00053e0 con 0x7f15d015b6f0 2024-09-21T06:13:22.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15cfe00640 1 -- 172.21.15.50:0/1515001369 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d015ebb0 con 0x7f15d015b6f0 2024-09-21T06:13:22.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15c6a00640 1 -- 172.21.15.50:0/1515001369 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0003860 con 0x7f15d015b6f0 2024-09-21T06:13:22.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15cfe00640 1 -- 172.21.15.50:0/1515001369 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d015e5f0 con 0x7f15d015b6f0 2024-09-21T06:13:22.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.837+0000 7f15cfe00640 1 -- 172.21.15.50:0/1515001369 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d015e3e0 con 0x7f15d015b6f0 2024-09-21T06:13:22.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.841+0000 7f15c6a00640 1 -- 172.21.15.50:0/1515001369 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f15c001a070 con 0x7f15d015b6f0 2024-09-21T06:13:22.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.841+0000 7f15c6a00640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15b0075960 0x7f15b0077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.841+0000 7f15c6a00640 1 -- 172.21.15.50:0/1515001369 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f15c00928c0 con 0x7f15d015b6f0 2024-09-21T06:13:22.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.841+0000 7f15cea00640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15b0075960 0x7f15b0077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.845+0000 7f15cea00640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15b0075960 0x7f15b0077e70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f15bc007fd0 tx=0x7f15bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:22.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:22.849+0000 7f15c6a00640 1 -- 172.21.15.50:0/1515001369 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f15c005f3f0 con 0x7f15d015b6f0 2024-09-21T06:13:23.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:23.197+0000 7f15cfe00640 1 -- 172.21.15.50:0/1515001369 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone74693", "target": ["mon-mgr", ""]}) -- 0x7f15d006b540 con 0x7f15b0075960 2024-09-21T06:13:24.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.937+0000 7f15c6a00640 1 -- 172.21.15.50:0/1515001369 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f15bc005020 con 0x7f15b0075960 2024-09-21T06:13:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.941+0000 7f15cfe00640 1 -- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15b0075960 msgr2=0x7f15b0077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.941+0000 7f15cfe00640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15b0075960 0x7f15b0077e70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f15bc007fd0 tx=0x7f15bc009040 comp rx=0 tx=0).stop 2024-09-21T06:13:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.941+0000 7f15cfe00640 1 -- 172.21.15.50:0/1515001369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15d015b6f0 msgr2=0x7f15d00ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.941+0000 7f15cfe00640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15d015b6f0 0x7f15d00ae8e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f15c000c430 tx=0x7f15c000c8f0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.941+0000 7f15cfe00640 1 -- 172.21.15.50:0/1515001369 shutdown_connections 2024-09-21T06:13:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.941+0000 7f15cfe00640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15d01502b0 0x7f15d00aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.941+0000 7f15cfe00640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f15b0075960 0x7f15b0077e70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.941+0000 7f15cfe00640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15d015b6f0 0x7f15d00ae8e0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.941+0000 7f15cfe00640 1 --2- 172.21.15.50:0/1515001369 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f15d006fb20 0x7f15d014fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.941+0000 7f15cfe00640 1 -- 172.21.15.50:0/1515001369 >> 172.21.15.50:0/1515001369 conn(0x7f15d006d1f0 msgr2=0x7f15d0159c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.941+0000 7f15cfe00640 1 -- 172.21.15.50:0/1515001369 shutdown_connections 2024-09-21T06:13:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:24.941+0000 7f15cfe00640 1 -- 172.21.15.50:0/1515001369 wait complete. 2024-09-21T06:13:25.004 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone90034 2024-09-21T06:13:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.297+0000 7f6351c00640 1 -- 172.21.15.50:0/2150605569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c0b5c70 msgr2=0x7f634c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:25.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.297+0000 7f6351c00640 1 --2- 172.21.15.50:0/2150605569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c0b5c70 0x7f634c15bb60 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f633c009840 tx=0x7f633c01c7f0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.297+0000 7f6351c00640 1 -- 172.21.15.50:0/2150605569 shutdown_connections 2024-09-21T06:13:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.297+0000 7f6351c00640 1 --2- 172.21.15.50:0/2150605569 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f634c0b4de0 0x7f634c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.297+0000 7f6351c00640 1 --2- 172.21.15.50:0/2150605569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c0b5c70 0x7f634c15bb60 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.297+0000 7f6351c00640 1 -- 172.21.15.50:0/2150605569 >> 172.21.15.50:0/2150605569 conn(0x7f634c0b2040 msgr2=0x7f634c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.297+0000 7f6351c00640 1 -- 172.21.15.50:0/2150605569 shutdown_connections 2024-09-21T06:13:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.297+0000 7f6351c00640 1 -- 172.21.15.50:0/2150605569 wait complete. 2024-09-21T06:13:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.301+0000 7f6351c00640 1 Processor -- start 2024-09-21T06:13:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.301+0000 7f6351c00640 1 -- start start 2024-09-21T06:13:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.301+0000 7f6351c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f634c0b4de0 0x7f634c0af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.301+0000 7f6351c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c0b5c70 0x7f634c13dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.301+0000 7f6351c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f634c1527b0 0x7f634c13d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.301+0000 7f6351c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f634c168a20 con 0x7f634c0b5c70 2024-09-21T06:13:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.301+0000 7f6351c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1688a0 con 0x7f634c1527b0 2024-09-21T06:13:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.301+0000 7f6351c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f634c168ba0 con 0x7f634c0b4de0 2024-09-21T06:13:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.301+0000 7f6343e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c0b5c70 0x7f634c13dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.301+0000 7f6343e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c0b5c70 0x7f634c13dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44596/0 (socket says 172.21.15.50:44596) 2024-09-21T06:13:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.301+0000 7f6343e00640 1 -- 172.21.15.50:0/1217664501 learned_addr learned my addr 172.21.15.50:0/1217664501 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:25.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.305+0000 7f6343e00640 1 -- 172.21.15.50:0/1217664501 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f634c0b4de0 msgr2=0x7f634c0af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:25.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.305+0000 7f6343e00640 1 --2- 172.21.15.50:0/1217664501 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f634c0b4de0 0x7f634c0af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.305+0000 7f6343e00640 1 -- 172.21.15.50:0/1217664501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f634c1527b0 msgr2=0x7f634c13d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:25.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.305+0000 7f6343e00640 1 --2- 172.21.15.50:0/1217664501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f634c1527b0 0x7f634c13d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.305+0000 7f6343e00640 1 -- 172.21.15.50:0/1217664501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634c0af590 con 0x7f634c0b5c70 2024-09-21T06:13:25.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.305+0000 7f6343e00640 1 --2- 172.21.15.50:0/1217664501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c0b5c70 0x7f634c13dc00 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f6334001ec0 tx=0x7f633400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:25.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.305+0000 7f6342a00640 1 -- 172.21.15.50:0/1217664501 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334017070 con 0x7f634c0b5c70 2024-09-21T06:13:25.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.305+0000 7f6342a00640 1 -- 172.21.15.50:0/1217664501 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6334003600 con 0x7f634c0b5c70 2024-09-21T06:13:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.305+0000 7f6342a00640 1 -- 172.21.15.50:0/1217664501 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334003a50 con 0x7f634c0b5c70 2024-09-21T06:13:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.305+0000 7f6351c00640 1 -- 172.21.15.50:0/1217664501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634c13e140 con 0x7f634c0b5c70 2024-09-21T06:13:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.305+0000 7f6351c00640 1 -- 172.21.15.50:0/1217664501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634c0b0670 con 0x7f634c0b5c70 2024-09-21T06:13:25.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.309+0000 7f6351c00640 1 -- 172.21.15.50:0/1217664501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f634c0008e0 con 0x7f634c0b5c70 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.313+0000 7f6342a00640 1 -- 172.21.15.50:0/1217664501 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f633401a070 con 0x7f634c0b5c70 2024-09-21T06:13:25.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.317+0000 7f6342a00640 1 --2- 172.21.15.50:0/1217664501 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6328075a30 0x7f6328077f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:25.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.317+0000 7f6342a00640 1 -- 172.21.15.50:0/1217664501 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f6334092e10 con 0x7f634c0b5c70 2024-09-21T06:13:25.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.329+0000 7f6342a00640 1 -- 172.21.15.50:0/1217664501 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6334021020 con 0x7f634c0b5c70 2024-09-21T06:13:25.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.333+0000 7f634aa00640 1 --2- 172.21.15.50:0/1217664501 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6328075a30 0x7f6328077f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:25.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.385+0000 7f634aa00640 1 --2- 172.21.15.50:0/1217664501 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6328075a30 0x7f6328077f40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f633c0af000 tx=0x7f633c019590 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:25.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:25.709+0000 7f6351c00640 1 -- 172.21.15.50:0/1217664501 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone90034", "target": ["mon-mgr", ""]}) -- 0x7f634c06b540 con 0x7f6328075a30 2024-09-21T06:13:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.965+0000 7f6342a00640 1 -- 172.21.15.50:0/1217664501 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f633c0a56b0 con 0x7f6328075a30 2024-09-21T06:13:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.969+0000 7f6351c00640 1 -- 172.21.15.50:0/1217664501 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6328075a30 msgr2=0x7f6328077f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.969+0000 7f6351c00640 1 --2- 172.21.15.50:0/1217664501 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6328075a30 0x7f6328077f40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f633c0af000 tx=0x7f633c019590 comp rx=0 tx=0).stop 2024-09-21T06:13:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.969+0000 7f6351c00640 1 -- 172.21.15.50:0/1217664501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c0b5c70 msgr2=0x7f634c13dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.969+0000 7f6351c00640 1 --2- 172.21.15.50:0/1217664501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c0b5c70 0x7f634c13dc00 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f6334001ec0 tx=0x7f633400c890 comp rx=0 tx=0).stop 2024-09-21T06:13:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.969+0000 7f6351c00640 1 -- 172.21.15.50:0/1217664501 shutdown_connections 2024-09-21T06:13:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.969+0000 7f6351c00640 1 --2- 172.21.15.50:0/1217664501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f634c1527b0 0x7f634c13d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.969+0000 7f6351c00640 1 --2- 172.21.15.50:0/1217664501 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6328075a30 0x7f6328077f40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.969+0000 7f6351c00640 1 --2- 172.21.15.50:0/1217664501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c0b5c70 0x7f634c13dc00 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.969+0000 7f6351c00640 1 --2- 172.21.15.50:0/1217664501 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f634c0b4de0 0x7f634c0af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.969+0000 7f6351c00640 1 -- 172.21.15.50:0/1217664501 >> 172.21.15.50:0/1217664501 conn(0x7f634c0b2040 msgr2=0x7f634c1545b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.969+0000 7f6351c00640 1 -- 172.21.15.50:0/1217664501 shutdown_connections 2024-09-21T06:13:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:26.969+0000 7f6351c00640 1 -- 172.21.15.50:0/1217664501 wait complete. 2024-09-21T06:13:27.021 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone44825 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.265+0000 7fe88be00640 1 -- 172.21.15.50:0/4013820353 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8840099e0 con 0x7fe874005eb0 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.265+0000 7fe893e00640 1 -- 172.21.15.50:0/4013820353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe874005eb0 msgr2=0x7fe874006380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.265+0000 7fe893e00640 1 --2- 172.21.15.50:0/4013820353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe874005eb0 0x7fe874006380 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fe88c15cda0 tx=0x7fe88401d110 comp rx=0 tx=0).stop 2024-09-21T06:13:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.269+0000 7fe891e00640 1 -- 172.21.15.50:0/4013820353 reap_dead start 2024-09-21T06:13:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.269+0000 7fe893e00640 1 -- 172.21.15.50:0/4013820353 shutdown_connections 2024-09-21T06:13:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.269+0000 7fe893e00640 1 -- 172.21.15.50:0/4013820353 >> 172.21.15.50:0/4013820353 conn(0x7fe88c0b2040 msgr2=0x7fe88c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.269+0000 7fe893e00640 1 -- 172.21.15.50:0/4013820353 shutdown_connections 2024-09-21T06:13:27.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.269+0000 7fe893e00640 1 -- 172.21.15.50:0/4013820353 wait complete. 2024-09-21T06:13:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe893e00640 1 Processor -- start 2024-09-21T06:13:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe893e00640 1 -- start start 2024-09-21T06:13:27.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe893e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe874005eb0 0x7fe88c1fc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:27.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe893e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe88c15a7b0 0x7fe88c1fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:27.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe893e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88c1fd430 0x7fe88c1416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:27.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe893e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c0b37d0 con 0x7fe88c1fd430 2024-09-21T06:13:27.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe893e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c0b3650 con 0x7fe874005eb0 2024-09-21T06:13:27.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe893e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe88c0b3950 con 0x7fe88c15a7b0 2024-09-21T06:13:27.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe891e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88c1fd430 0x7fe88c1416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:27.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe891e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88c1fd430 0x7fe88c1416f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45830/0 (socket says 172.21.15.50:45830) 2024-09-21T06:13:27.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe891e00640 1 -- 172.21.15.50:0/3757856861 learned_addr learned my addr 172.21.15.50:0/3757856861 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:27.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe891e00640 1 -- 172.21.15.50:0/3757856861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe88c15a7b0 msgr2=0x7fe88c1fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:27.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe891e00640 1 --2- 172.21.15.50:0/3757856861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe88c15a7b0 0x7fe88c1fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:27.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe891e00640 1 -- 172.21.15.50:0/3757856861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe874005eb0 msgr2=0x7fe88c1fc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:27.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe891e00640 1 --2- 172.21.15.50:0/3757856861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe874005eb0 0x7fe88c1fc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:27.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe891e00640 1 -- 172.21.15.50:0/3757856861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c1fd840 con 0x7fe88c1fd430 2024-09-21T06:13:27.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.273+0000 7fe891e00640 1 --2- 172.21.15.50:0/3757856861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88c1fd430 0x7fe88c1416f0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fe884002240 tx=0x7fe8840acb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:27.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.277+0000 7fe889600640 1 -- 172.21.15.50:0/3757856861 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8840b1070 con 0x7fe88c1fd430 2024-09-21T06:13:27.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.277+0000 7fe889600640 1 -- 172.21.15.50:0/3757856861 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8840b2040 con 0x7fe88c1fd430 2024-09-21T06:13:27.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.277+0000 7fe889600640 1 -- 172.21.15.50:0/3757856861 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8840ace20 con 0x7fe88c1fd430 2024-09-21T06:13:27.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.277+0000 7fe893e00640 1 -- 172.21.15.50:0/3757856861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c140650 con 0x7fe88c1fd430 2024-09-21T06:13:27.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.277+0000 7fe893e00640 1 -- 172.21.15.50:0/3757856861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c140c10 con 0x7fe88c1fd430 2024-09-21T06:13:27.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.277+0000 7fe893e00640 1 -- 172.21.15.50:0/3757856861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe88c15d690 con 0x7fe88c1fd430 2024-09-21T06:13:27.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.281+0000 7fe889600640 1 -- 172.21.15.50:0/3757856861 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7fe8840bf060 con 0x7fe88c1fd430 2024-09-21T06:13:27.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.281+0000 7fe889600640 1 --2- 172.21.15.50:0/3757856861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe86c075960 0x7fe86c077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:27.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.281+0000 7fe889600640 1 -- 172.21.15.50:0/3757856861 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7fe88412da60 con 0x7fe88c1fd430 2024-09-21T06:13:27.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.281+0000 7fe891400640 1 --2- 172.21.15.50:0/3757856861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe86c075960 0x7fe86c077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:27.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.285+0000 7fe891400640 1 --2- 172.21.15.50:0/3757856861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe86c075960 0x7fe86c077e70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe880001870 tx=0x7fe880001c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:27.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.285+0000 7fe889600640 1 -- 172.21.15.50:0/3757856861 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe88c1fd840 con 0x7fe88c1fd430 2024-09-21T06:13:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:27.641+0000 7fe893e00640 1 -- 172.21.15.50:0/3757856861 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone44825", "target": ["mon-mgr", ""]}) -- 0x7fe88c06b540 con 0x7fe86c075960 2024-09-21T06:13:28.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.093+0000 7fe889600640 1 -- 172.21.15.50:0/3757856861 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe880006050 con 0x7fe86c075960 2024-09-21T06:13:28.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.097+0000 7fe893e00640 1 -- 172.21.15.50:0/3757856861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe86c075960 msgr2=0x7fe86c077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:28.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.097+0000 7fe893e00640 1 --2- 172.21.15.50:0/3757856861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe86c075960 0x7fe86c077e70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe880001870 tx=0x7fe880001c00 comp rx=0 tx=0).stop 2024-09-21T06:13:28.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.097+0000 7fe893e00640 1 -- 172.21.15.50:0/3757856861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88c1fd430 msgr2=0x7fe88c1416f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:28.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.097+0000 7fe893e00640 1 --2- 172.21.15.50:0/3757856861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88c1fd430 0x7fe88c1416f0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fe884002240 tx=0x7fe8840acb70 comp rx=0 tx=0).stop 2024-09-21T06:13:28.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.101+0000 7fe893e00640 1 -- 172.21.15.50:0/3757856861 shutdown_connections 2024-09-21T06:13:28.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.101+0000 7fe893e00640 1 --2- 172.21.15.50:0/3757856861 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe86c075960 0x7fe86c077e70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.101+0000 7fe893e00640 1 --2- 172.21.15.50:0/3757856861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88c1fd430 0x7fe88c1416f0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.101+0000 7fe893e00640 1 --2- 172.21.15.50:0/3757856861 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe88c15a7b0 0x7fe88c1fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.101+0000 7fe893e00640 1 --2- 172.21.15.50:0/3757856861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe874005eb0 0x7fe88c1fc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.101+0000 7fe893e00640 1 -- 172.21.15.50:0/3757856861 >> 172.21.15.50:0/3757856861 conn(0x7fe88c0b2040 msgr2=0x7fe88c0b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:28.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.101+0000 7fe893e00640 1 -- 172.21.15.50:0/3757856861 shutdown_connections 2024-09-21T06:13:28.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.101+0000 7fe893e00640 1 -- 172.21.15.50:0/3757856861 wait complete. 2024-09-21T06:13:28.162 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone64946 2024-09-21T06:13:28.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.417+0000 7ff3bee00640 1 -- 172.21.15.50:0/3789932791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3a0005360 msgr2=0x7ff3a000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:28.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.417+0000 7ff3bee00640 1 --2- 172.21.15.50:0/3789932791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3a0005360 0x7ff3a000b9d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009ee0 tx=0x7ff3b001d630 comp rx=0 tx=0).stop 2024-09-21T06:13:28.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.417+0000 7ff3bee00640 1 -- 172.21.15.50:0/3789932791 shutdown_connections 2024-09-21T06:13:28.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.417+0000 7ff3bee00640 1 --2- 172.21.15.50:0/3789932791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3a0005360 0x7ff3a000b9d0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.417+0000 7ff3bee00640 1 -- 172.21.15.50:0/3789932791 >> 172.21.15.50:0/3789932791 conn(0x7ff3b80b4560 msgr2=0x7ff3b80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:28.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.417+0000 7ff3bee00640 1 -- 172.21.15.50:0/3789932791 shutdown_connections 2024-09-21T06:13:28.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.417+0000 7ff3bee00640 1 -- 172.21.15.50:0/3789932791 wait complete. 2024-09-21T06:13:28.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.421+0000 7ff3bee00640 1 Processor -- start 2024-09-21T06:13:28.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.421+0000 7ff3bee00640 1 -- start start 2024-09-21T06:13:28.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bee00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff3a0005360 0x7ff3b80d0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:28.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3b80c7860 0x7ff3b81f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:28.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3b81f55b0 0x7ff3b81f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:28.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b815d860 con 0x7ff3b80c7860 2024-09-21T06:13:28.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b815d6e0 con 0x7ff3b81f55b0 2024-09-21T06:13:28.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bee00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff3b815d9e0 con 0x7ff3a0005360 2024-09-21T06:13:28.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3b80c7860 0x7ff3b81f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:28.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3b80c7860 0x7ff3b81f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45846/0 (socket says 172.21.15.50:45846) 2024-09-21T06:13:28.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bd000640 1 -- 172.21.15.50:0/3379736963 learned_addr learned my addr 172.21.15.50:0/3379736963 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:28.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3be400640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3b81f55b0 0x7ff3b81f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:28.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bd000640 1 -- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff3a0005360 msgr2=0x7ff3b80d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:28.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bd000640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff3a0005360 0x7ff3b80d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bd000640 1 -- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3b81f55b0 msgr2=0x7ff3b81f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:28.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bd000640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3b81f55b0 0x7ff3b81f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bd000640 1 -- 172.21.15.50:0/3379736963 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b80d0960 con 0x7ff3b80c7860 2024-09-21T06:13:28.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bd000640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3b80c7860 0x7ff3b81f5070 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4003f40 tx=0x7ff3a400f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:28.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3b5600640 1 -- 172.21.15.50:0/3379736963 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4015070 con 0x7ff3b80c7860 2024-09-21T06:13:28.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3b5600640 1 -- 172.21.15.50:0/3379736963 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3a4011840 con 0x7ff3b80c7860 2024-09-21T06:13:28.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3b5600640 1 -- 172.21.15.50:0/3379736963 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4011c90 con 0x7ff3b80c7860 2024-09-21T06:13:28.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bee00640 1 -- 172.21.15.50:0/3379736963 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b80d0c40 con 0x7ff3b80c7860 2024-09-21T06:13:28.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.425+0000 7ff3bee00640 1 -- 172.21.15.50:0/3379736963 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b80d1200 con 0x7ff3b80c7860 2024-09-21T06:13:28.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.429+0000 7ff3bee00640 1 -- 172.21.15.50:0/3379736963 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b8160990 con 0x7ff3b80c7860 2024-09-21T06:13:28.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.429+0000 7ff3b5600640 1 -- 172.21.15.50:0/3379736963 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7ff3a40103c0 con 0x7ff3b80c7860 2024-09-21T06:13:28.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.433+0000 7ff3b5600640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff398075960 0x7ff398077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:28.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.433+0000 7ff3b5600640 1 -- 172.21.15.50:0/3379736963 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7ff3a4094060 con 0x7ff3b80c7860 2024-09-21T06:13:28.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.437+0000 7ff3bda00640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff398075960 0x7ff398077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:28.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.437+0000 7ff3bda00640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff398075960 0x7ff398077e70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac001700 tx=0x7ff3ac000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:28.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.437+0000 7ff3b5600640 1 -- 172.21.15.50:0/3379736963 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff3a4060b70 con 0x7ff3b80c7860 2024-09-21T06:13:28.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.757+0000 7ff3bee00640 1 -- 172.21.15.50:0/3379736963 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone64946", "target": ["mon-mgr", ""]}) -- 0x7ff3b806b540 con 0x7ff398075960 2024-09-21T06:13:28.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.909+0000 7ff3b5600640 1 -- 172.21.15.50:0/3379736963 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff3ac017070 con 0x7ff398075960 2024-09-21T06:13:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.909+0000 7ff3bee00640 1 -- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff398075960 msgr2=0x7ff398077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.909+0000 7ff3bee00640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff398075960 0x7ff398077e70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac001700 tx=0x7ff3ac000ee0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.909+0000 7ff3bee00640 1 -- 172.21.15.50:0/3379736963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3b80c7860 msgr2=0x7ff3b81f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.909+0000 7ff3bee00640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3b80c7860 0x7ff3b81f5070 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4003f40 tx=0x7ff3a400f7e0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.913+0000 7ff3bee00640 1 -- 172.21.15.50:0/3379736963 shutdown_connections 2024-09-21T06:13:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.913+0000 7ff3bee00640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3b81f55b0 0x7ff3b81f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.913+0000 7ff3bee00640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff398075960 0x7ff398077e70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.913+0000 7ff3bee00640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3b80c7860 0x7ff3b81f5070 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.913+0000 7ff3bee00640 1 --2- 172.21.15.50:0/3379736963 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff3a0005360 0x7ff3b80d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.913+0000 7ff3bee00640 1 -- 172.21.15.50:0/3379736963 >> 172.21.15.50:0/3379736963 conn(0x7ff3b80b4560 msgr2=0x7ff3b80b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.913+0000 7ff3bee00640 1 -- 172.21.15.50:0/3379736963 shutdown_connections 2024-09-21T06:13:28.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:28.913+0000 7ff3bee00640 1 -- 172.21.15.50:0/3379736963 wait complete. 2024-09-21T06:13:28.931 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99400 2024-09-21T06:13:29.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.093+0000 7fb10be00640 1 -- 172.21.15.50:0/4057782629 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb1040b54f0 msgr2=0x7fb10415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.093+0000 7fb10be00640 1 --2- 172.21.15.50:0/4057782629 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb1040b54f0 0x7fb10415bb60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009bf0 tx=0x7fb0fc01cad0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.093+0000 7fb10be00640 1 -- 172.21.15.50:0/4057782629 shutdown_connections 2024-09-21T06:13:29.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.093+0000 7fb10be00640 1 --2- 172.21.15.50:0/4057782629 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb1040b54f0 0x7fb10415bb60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.093+0000 7fb10be00640 1 -- 172.21.15.50:0/4057782629 >> 172.21.15.50:0/4057782629 conn(0x7fb1040b2040 msgr2=0x7fb1040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:29.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.093+0000 7fb10be00640 1 -- 172.21.15.50:0/4057782629 shutdown_connections 2024-09-21T06:13:29.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.093+0000 7fb10be00640 1 -- 172.21.15.50:0/4057782629 wait complete. 2024-09-21T06:13:29.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb10be00640 1 Processor -- start 2024-09-21T06:13:29.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb10be00640 1 -- start start 2024-09-21T06:13:29.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb10be00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb1040b5c70 0x7fb104150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb10be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1041527b0 0x7fb1040ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb10be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104159070 0x7fb1040acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb10be00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb104169bd0 con 0x7fb104159070 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb10be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb104169a50 con 0x7fb1041527b0 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb10be00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb104169d50 con 0x7fb1040b5c70 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb109400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb1040b5c70 0x7fb104150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb109e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104159070 0x7fb1040acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb109400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb1040b5c70 0x7fb104150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:47988/0 (socket says 172.21.15.50:47988) 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb109400640 1 -- 172.21.15.50:0/3636267114 learned_addr learned my addr 172.21.15.50:0/3636267114 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb108a00640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1041527b0 0x7fb1040ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb109400640 1 -- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1041527b0 msgr2=0x7fb1040ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb109400640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1041527b0 0x7fb1040ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb109400640 1 -- 172.21.15.50:0/3636267114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104159070 msgr2=0x7fb1040acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb109400640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104159070 0x7fb1040acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb109400640 1 -- 172.21.15.50:0/3636267114 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb104151200 con 0x7fb1040b5c70 2024-09-21T06:13:29.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb109e00640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104159070 0x7fb1040acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:29.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb109400640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb1040b5c70 0x7fb104150cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8001230 tx=0x7fb0f800c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb101600640 1 -- 172.21.15.50:0/3636267114 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8016020 con 0x7fb1040b5c70 2024-09-21T06:13:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb101600640 1 -- 172.21.15.50:0/3636267114 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0f8012920 con 0x7fb1040b5c70 2024-09-21T06:13:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb101600640 1 -- 172.21.15.50:0/3636267114 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8012d70 con 0x7fb1040b5c70 2024-09-21T06:13:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb10be00640 1 -- 172.21.15.50:0/3636267114 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1040aabb0 con 0x7fb1040b5c70 2024-09-21T06:13:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.097+0000 7fb10be00640 1 -- 172.21.15.50:0/3636267114 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1040ab170 con 0x7fb1040b5c70 2024-09-21T06:13:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.101+0000 7fb10be00640 1 -- 172.21.15.50:0/3636267114 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb10415bb60 con 0x7fb1040b5c70 2024-09-21T06:13:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.101+0000 7fb101600640 1 -- 172.21.15.50:0/3636267114 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7fb0f8019020 con 0x7fb1040b5c70 2024-09-21T06:13:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.101+0000 7fb101600640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0e4075960 0x7fb0e4077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.101+0000 7fb101600640 1 -- 172.21.15.50:0/3636267114 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7fb0f8093590 con 0x7fb1040b5c70 2024-09-21T06:13:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.101+0000 7fb108a00640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0e4075960 0x7fb0e4077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.105+0000 7fb108a00640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0e4075960 0x7fb0e4077e70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0007fd0 tx=0x7fb0f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.105+0000 7fb101600640 1 -- 172.21.15.50:0/3636267114 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb0f8020020 con 0x7fb1040b5c70 2024-09-21T06:13:29.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.321+0000 7fb10be00640 1 -- 172.21.15.50:0/3636267114 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99400", "target": ["mon-mgr", ""]}) -- 0x7fb10406b540 con 0x7fb0e4075960 2024-09-21T06:13:29.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.329+0000 7fb101600640 1 -- 172.21.15.50:0/3636267114 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fb0f00014a0 con 0x7fb0e4075960 2024-09-21T06:13:29.329 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:13:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "canceled", 2024-09-21T06:13:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:13:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:13:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:13:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:13:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:29.331 INFO:teuthology.orchestra.run.smithi050.stdout: "failure": { 2024-09-21T06:13:29.331 INFO:teuthology.orchestra.run.smithi050.stdout: "errno": "4", 2024-09-21T06:13:29.331 INFO:teuthology.orchestra.run.smithi050.stdout: "error_msg": "user interrupted clone operation" 2024-09-21T06:13:29.331 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:29.331 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:29.331 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:29.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.333+0000 7fb10be00640 1 -- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0e4075960 msgr2=0x7fb0e4077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.333+0000 7fb10be00640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0e4075960 0x7fb0e4077e70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0007fd0 tx=0x7fb0f0009040 comp rx=0 tx=0).stop 2024-09-21T06:13:29.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.333+0000 7fb10be00640 1 -- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb1040b5c70 msgr2=0x7fb104150cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.333+0000 7fb10be00640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb1040b5c70 0x7fb104150cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8001230 tx=0x7fb0f800c980 comp rx=0 tx=0).stop 2024-09-21T06:13:29.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.333+0000 7fb10be00640 1 -- 172.21.15.50:0/3636267114 shutdown_connections 2024-09-21T06:13:29.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.333+0000 7fb10be00640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0e4075960 0x7fb0e4077e70 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.333+0000 7fb10be00640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104159070 0x7fb1040acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.333+0000 7fb10be00640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1041527b0 0x7fb1040ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.333+0000 7fb10be00640 1 --2- 172.21.15.50:0/3636267114 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb1040b5c70 0x7fb104150cc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.333+0000 7fb10be00640 1 -- 172.21.15.50:0/3636267114 >> 172.21.15.50:0/3636267114 conn(0x7fb1040b2040 msgr2=0x7fb10415a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:29.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.333+0000 7fb10be00640 1 -- 172.21.15.50:0/3636267114 shutdown_connections 2024-09-21T06:13:29.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.333+0000 7fb10be00640 1 -- 172.21.15.50:0/3636267114 wait complete. 2024-09-21T06:13:29.352 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-21T06:13:29.352 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone5031 2024-09-21T06:13:29.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.513+0000 7f0b8e600640 1 -- 172.21.15.50:0/3324394195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b88157030 msgr2=0x7f0b880b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.513+0000 7f0b8e600640 1 --2- 172.21.15.50:0/3324394195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b88157030 0x7f0b880b5a80 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f0b880b1740 tx=0x7f0b7801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.513+0000 7f0b8e600640 1 -- 172.21.15.50:0/3324394195 shutdown_connections 2024-09-21T06:13:29.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.513+0000 7f0b8e600640 1 --2- 172.21.15.50:0/3324394195 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b88161160 0x7f0b88074950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.513+0000 7f0b8e600640 1 --2- 172.21.15.50:0/3324394195 >> v2:172.21.15.88:3300/0 conn(0x7f0b88160d50 0x7f0b88074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.513+0000 7f0b8e600640 1 --2- 172.21.15.50:0/3324394195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b88157030 0x7f0b880b5a80 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.513+0000 7f0b8e600640 1 -- 172.21.15.50:0/3324394195 >> 172.21.15.50:0/3324394195 conn(0x7f0b880708c0 msgr2=0x7f0b88070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:29.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.517+0000 7f0b8e600640 1 -- 172.21.15.50:0/3324394195 shutdown_connections 2024-09-21T06:13:29.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.517+0000 7f0b8e600640 1 -- 172.21.15.50:0/3324394195 wait complete. 2024-09-21T06:13:29.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.517+0000 7f0b8e600640 1 Processor -- start 2024-09-21T06:13:29.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.517+0000 7f0b8e600640 1 -- start start 2024-09-21T06:13:29.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.517+0000 7f0b8e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b88157030 0x7f0b881f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.517+0000 7f0b8e600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b88160d50 0x7f0b8814d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.517+0000 7f0b8e600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b88161160 0x7f0b8815e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.517+0000 7f0b8e600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0b88072230 con 0x7f0b88157030 2024-09-21T06:13:29.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.517+0000 7f0b8e600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b880720b0 con 0x7f0b88161160 2024-09-21T06:13:29.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.517+0000 7f0b8e600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0b880723b0 con 0x7f0b88160d50 2024-09-21T06:13:29.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b87400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b88157030 0x7f0b881f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b87400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b88157030 0x7f0b881f6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45874/0 (socket says 172.21.15.50:45874) 2024-09-21T06:13:29.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b87400640 1 -- 172.21.15.50:0/1131800902 learned_addr learned my addr 172.21.15.50:0/1131800902 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:29.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b87e00640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b88161160 0x7f0b8815e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b7ea00640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b88160d50 0x7f0b8814d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b87400640 1 -- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b88160d50 msgr2=0x7f0b8814d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b87400640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b88160d50 0x7f0b8814d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b87400640 1 -- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b88161160 msgr2=0x7f0b8815e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b87400640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b88161160 0x7f0b8815e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b87400640 1 -- 172.21.15.50:0/1131800902 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b88076900 con 0x7f0b88157030 2024-09-21T06:13:29.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b87400640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b88157030 0x7f0b881f6ae0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f0b7801af70 tx=0x7f0b78005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:29.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b7fe00640 1 -- 172.21.15.50:0/1131800902 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780b2040 con 0x7f0b88157030 2024-09-21T06:13:29.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b7fe00640 1 -- 172.21.15.50:0/1131800902 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b78004340 con 0x7f0b88157030 2024-09-21T06:13:29.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b7fe00640 1 -- 172.21.15.50:0/1131800902 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780052d0 con 0x7f0b88157030 2024-09-21T06:13:29.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b8e600640 1 -- 172.21.15.50:0/1131800902 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b88152880 con 0x7f0b88157030 2024-09-21T06:13:29.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b8e600640 1 -- 172.21.15.50:0/1131800902 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b88152e40 con 0x7f0b88157030 2024-09-21T06:13:29.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b8e600640 1 -- 172.21.15.50:0/1131800902 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b88072880 con 0x7f0b88157030 2024-09-21T06:13:29.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.521+0000 7f0b7fe00640 1 -- 172.21.15.50:0/1131800902 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f0b780bd060 con 0x7f0b88157030 2024-09-21T06:13:29.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.525+0000 7f0b7fe00640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b64075960 0x7f0b64077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.525+0000 7f0b7fe00640 1 -- 172.21.15.50:0/1131800902 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f0b7812c7c0 con 0x7f0b88157030 2024-09-21T06:13:29.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.525+0000 7f0b7ea00640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b64075960 0x7f0b64077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.525+0000 7f0b7ea00640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b64075960 0x7f0b64077e70 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0b8814a7f0 tx=0x7f0b70009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:29.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.529+0000 7f0b7fe00640 1 -- 172.21.15.50:0/1131800902 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0b780164b0 con 0x7f0b88157030 2024-09-21T06:13:29.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.745+0000 7f0b8e600640 1 -- 172.21.15.50:0/1131800902 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone5031", "target": ["mon-mgr", ""]}) -- 0x7f0b8806b540 con 0x7f0b64075960 2024-09-21T06:13:29.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.753+0000 7f0b7fe00640 1 -- 172.21.15.50:0/1131800902 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f0b7001b3f0 con 0x7f0b64075960 2024-09-21T06:13:29.754 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:29.754 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:13:29.754 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "canceled", 2024-09-21T06:13:29.755 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:13:29.755 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:13:29.755 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:13:29.755 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:13:29.755 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:29.755 INFO:teuthology.orchestra.run.smithi050.stdout: "failure": { 2024-09-21T06:13:29.755 INFO:teuthology.orchestra.run.smithi050.stdout: "errno": "4", 2024-09-21T06:13:29.755 INFO:teuthology.orchestra.run.smithi050.stdout: "error_msg": "user interrupted clone operation" 2024-09-21T06:13:29.755 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:29.755 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:29.755 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:29.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.757+0000 7f0b8e600640 1 -- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b64075960 msgr2=0x7f0b64077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.757+0000 7f0b8e600640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b64075960 0x7f0b64077e70 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0b8814a7f0 tx=0x7f0b70009500 comp rx=0 tx=0).stop 2024-09-21T06:13:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.757+0000 7f0b8e600640 1 -- 172.21.15.50:0/1131800902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b88157030 msgr2=0x7f0b881f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.757+0000 7f0b8e600640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b88157030 0x7f0b881f6ae0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f0b7801af70 tx=0x7f0b78005de0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.757+0000 7f0b8e600640 1 -- 172.21.15.50:0/1131800902 shutdown_connections 2024-09-21T06:13:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.757+0000 7f0b8e600640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b88161160 0x7f0b8815e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.757+0000 7f0b8e600640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0b88160d50 0x7f0b8814d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.757+0000 7f0b8e600640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f0b64075960 0x7f0b64077e70 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.757+0000 7f0b8e600640 1 --2- 172.21.15.50:0/1131800902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b88157030 0x7f0b881f6ae0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.757+0000 7f0b8e600640 1 -- 172.21.15.50:0/1131800902 >> 172.21.15.50:0/1131800902 conn(0x7f0b880708c0 msgr2=0x7f0b8814dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.757+0000 7f0b8e600640 1 -- 172.21.15.50:0/1131800902 shutdown_connections 2024-09-21T06:13:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.757+0000 7f0b8e600640 1 -- 172.21.15.50:0/1131800902 wait complete. 2024-09-21T06:13:29.777 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-21T06:13:29.778 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone34632 2024-09-21T06:13:29.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.941+0000 7f367ca00640 1 -- 172.21.15.50:0/3722875543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36780cefa0 msgr2=0x7f36780cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.941+0000 7f367ca00640 1 --2- 172.21.15.50:0/3722875543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36780cefa0 0x7f36780cf3b0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f3678160270 tx=0x7f366001c800 comp rx=0 tx=0).stop 2024-09-21T06:13:29.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.941+0000 7f367ca00640 1 -- 172.21.15.50:0/3722875543 shutdown_connections 2024-09-21T06:13:29.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.941+0000 7f367ca00640 1 --2- 172.21.15.50:0/3722875543 >> v2:172.21.15.88:3300/0 conn(0x7f36780b3340 0x7f36780b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.941+0000 7f367ca00640 1 --2- 172.21.15.50:0/3722875543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36780cefa0 0x7f36780cf3b0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.941+0000 7f367ca00640 1 -- 172.21.15.50:0/3722875543 >> 172.21.15.50:0/3722875543 conn(0x7f36780b2040 msgr2=0x7f36780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:29.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.941+0000 7f367ca00640 1 -- 172.21.15.50:0/3722875543 shutdown_connections 2024-09-21T06:13:29.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.941+0000 7f367ca00640 1 -- 172.21.15.50:0/3722875543 wait complete. 2024-09-21T06:13:29.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f367ca00640 1 Processor -- start 2024-09-21T06:13:29.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f367ca00640 1 -- start start 2024-09-21T06:13:29.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f367ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36780749a0 0x7f36781588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f367ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36780b3340 0x7f367813d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f367ca00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f36780cefa0 0x7f367813d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f367ca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3678169e10 con 0x7f36780b3340 2024-09-21T06:13:29.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f367ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3678169c90 con 0x7f36780749a0 2024-09-21T06:13:29.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f367ca00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3678169f90 con 0x7f36780cefa0 2024-09-21T06:13:29.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f366d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36780b3340 0x7f367813d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f366d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36780b3340 0x7f367813d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45890/0 (socket says 172.21.15.50:45890) 2024-09-21T06:13:29.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f366d400640 1 -- 172.21.15.50:0/1890285016 learned_addr learned my addr 172.21.15.50:0/1890285016 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:29.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f3675e00640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36780749a0 0x7f36781588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f3676800640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f36780cefa0 0x7f367813d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f366d400640 1 -- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f36780cefa0 msgr2=0x7f367813d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f366d400640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f36780cefa0 0x7f367813d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f366d400640 1 -- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36780749a0 msgr2=0x7f36781588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f366d400640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36780749a0 0x7f36781588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f366d400640 1 -- 172.21.15.50:0/1890285016 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367815b5b0 con 0x7f36780b3340 2024-09-21T06:13:29.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f366d400640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36780b3340 0x7f367813d1f0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f366001aeb0 tx=0x7f36600a6550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:29.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f366ea00640 1 -- 172.21.15.50:0/1890285016 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660004040 con 0x7f36780b3340 2024-09-21T06:13:29.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.945+0000 7f366ea00640 1 -- 172.21.15.50:0/1890285016 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36600042d0 con 0x7f36780b3340 2024-09-21T06:13:29.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.949+0000 7f367ca00640 1 -- 172.21.15.50:0/1890285016 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3678158de0 con 0x7f36780b3340 2024-09-21T06:13:29.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.949+0000 7f366ea00640 1 -- 172.21.15.50:0/1890285016 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36600044e0 con 0x7f36780b3340 2024-09-21T06:13:29.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.949+0000 7f367ca00640 1 -- 172.21.15.50:0/1890285016 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367813e840 con 0x7f36780b3340 2024-09-21T06:13:29.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.949+0000 7f367ca00640 1 -- 172.21.15.50:0/1890285016 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36780b3750 con 0x7f36780b3340 2024-09-21T06:13:29.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.949+0000 7f366ea00640 1 -- 172.21.15.50:0/1890285016 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f36600c1030 con 0x7f36780b3340 2024-09-21T06:13:29.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.949+0000 7f366ea00640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3654075b00 0x7f3654078010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.949+0000 7f366ea00640 1 -- 172.21.15.50:0/1890285016 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f366012da70 con 0x7f36780b3340 2024-09-21T06:13:29.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.949+0000 7f3675e00640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3654075b00 0x7f3654078010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.953+0000 7f3675e00640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3654075b00 0x7f3654078010 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f36680012f0 tx=0x7f3668009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:29.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:29.953+0000 7f366ea00640 1 -- 172.21.15.50:0/1890285016 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f36600c8060 con 0x7f36780b3340 2024-09-21T06:13:30.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.165+0000 7f367ca00640 1 -- 172.21.15.50:0/1890285016 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone34632", "target": ["mon-mgr", ""]}) -- 0x7f367806b540 con 0x7f3654075b00 2024-09-21T06:13:30.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f366ea00640 1 -- 172.21.15.50:0/1890285016 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f3668001b40 con 0x7f3654075b00 2024-09-21T06:13:30.175 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:30.175 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:13:30.175 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "canceled", 2024-09-21T06:13:30.175 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:13:30.175 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:13:30.175 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:13:30.175 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:13:30.175 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:30.176 INFO:teuthology.orchestra.run.smithi050.stdout: "failure": { 2024-09-21T06:13:30.176 INFO:teuthology.orchestra.run.smithi050.stdout: "errno": "4", 2024-09-21T06:13:30.176 INFO:teuthology.orchestra.run.smithi050.stdout: "error_msg": "user interrupted clone operation" 2024-09-21T06:13:30.176 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:30.176 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:30.176 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:30.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f367ca00640 1 -- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3654075b00 msgr2=0x7f3654078010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f367ca00640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3654075b00 0x7f3654078010 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f36680012f0 tx=0x7f3668009040 comp rx=0 tx=0).stop 2024-09-21T06:13:30.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f367ca00640 1 -- 172.21.15.50:0/1890285016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36780b3340 msgr2=0x7f367813d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f367ca00640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36780b3340 0x7f367813d1f0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f366001aeb0 tx=0x7f36600a6550 comp rx=0 tx=0).stop 2024-09-21T06:13:30.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f367ca00640 1 -- 172.21.15.50:0/1890285016 shutdown_connections 2024-09-21T06:13:30.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f367ca00640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f36780cefa0 0x7f367813d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f367ca00640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3654075b00 0x7f3654078010 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f367ca00640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36780b3340 0x7f367813d1f0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f367ca00640 1 --2- 172.21.15.50:0/1890285016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36780749a0 0x7f36781588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f367ca00640 1 -- 172.21.15.50:0/1890285016 >> 172.21.15.50:0/1890285016 conn(0x7f36780b2040 msgr2=0x7f36780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:30.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f367ca00640 1 -- 172.21.15.50:0/1890285016 shutdown_connections 2024-09-21T06:13:30.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.177+0000 7f367ca00640 1 -- 172.21.15.50:0/1890285016 wait complete. 2024-09-21T06:13:30.198 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-21T06:13:30.198 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone74693 2024-09-21T06:13:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.361+0000 7f6fb9600640 1 -- 172.21.15.50:0/4204212941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb415c5d0 msgr2=0x7f6fb4073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.361+0000 7f6fb9600640 1 --2- 172.21.15.50:0/4204212941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb415c5d0 0x7f6fb4073dc0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009900 tx=0x7f6fa401c8b0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.361+0000 7f6fb9600640 1 -- 172.21.15.50:0/4204212941 shutdown_connections 2024-09-21T06:13:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.361+0000 7f6fb9600640 1 --2- 172.21.15.50:0/4204212941 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6fb4074300 0x7f6fb415f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.361+0000 7f6fb9600640 1 --2- 172.21.15.50:0/4204212941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb415c5d0 0x7f6fb4073dc0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.361+0000 7f6fb9600640 1 -- 172.21.15.50:0/4204212941 >> 172.21.15.50:0/4204212941 conn(0x7f6fb40b2c40 msgr2=0x7f6fb40b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.361+0000 7f6fb9600640 1 -- 172.21.15.50:0/4204212941 shutdown_connections 2024-09-21T06:13:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.361+0000 7f6fb9600640 1 -- 172.21.15.50:0/4204212941 wait complete. 2024-09-21T06:13:30.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.361+0000 7f6fb9600640 1 Processor -- start 2024-09-21T06:13:30.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.361+0000 7f6fb9600640 1 -- start start 2024-09-21T06:13:30.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb9600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fb4074300 0x7f6fb41fae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb9600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6fb415c5d0 0x7f6fb41f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb9600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb415f6b0 0x7f6fb414f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb9600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb406f980 con 0x7f6fb415f6b0 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb9600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb40b3d50 con 0x7f6fb4074300 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb9600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb406fb00 con 0x7f6fb415c5d0 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fb4074300 0x7f6fb41fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb415f6b0 0x7f6fb414f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fabe00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6fb415c5d0 0x7f6fb41f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fb4074300 0x7f6fb41fae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:60460/0 (socket says 172.21.15.50:60460) 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb2a00640 1 -- 172.21.15.50:0/3947959570 learned_addr learned my addr 172.21.15.50:0/3947959570 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb3400640 1 -- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6fb415c5d0 msgr2=0x7f6fb41f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb3400640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6fb415c5d0 0x7f6fb41f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb3400640 1 -- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fb4074300 msgr2=0x7f6fb41fae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb3400640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fb4074300 0x7f6fb41fae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb3400640 1 -- 172.21.15.50:0/3947959570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb4076900 con 0x7f6fb415f6b0 2024-09-21T06:13:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb2a00640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fb4074300 0x7f6fb41fae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb3400640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb415f6b0 0x7f6fb414f210 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00c680 tx=0x7f6fac00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6faaa00640 1 -- 172.21.15.50:0/3947959570 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac003b80 con 0x7f6fb415f6b0 2024-09-21T06:13:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6faaa00640 1 -- 172.21.15.50:0/3947959570 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fac003d20 con 0x7f6fb415f6b0 2024-09-21T06:13:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6faaa00640 1 -- 172.21.15.50:0/3947959570 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac005330 con 0x7f6fb415f6b0 2024-09-21T06:13:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb9600640 1 -- 172.21.15.50:0/3947959570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb414e170 con 0x7f6fb415f6b0 2024-09-21T06:13:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb9600640 1 -- 172.21.15.50:0/3947959570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb414e730 con 0x7f6fb415f6b0 2024-09-21T06:13:30.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.365+0000 7f6fb9600640 1 -- 172.21.15.50:0/3947959570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb415f170 con 0x7f6fb415f6b0 2024-09-21T06:13:30.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.369+0000 7f6faaa00640 1 -- 172.21.15.50:0/3947959570 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f6fac006b60 con 0x7f6fb415f6b0 2024-09-21T06:13:30.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.369+0000 7f6faaa00640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f90075960 0x7f6f90077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:30.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.369+0000 7f6faaa00640 1 -- 172.21.15.50:0/3947959570 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f6fac0937c0 con 0x7f6fb415f6b0 2024-09-21T06:13:30.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.369+0000 7f6fb2a00640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f90075960 0x7f6f90077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:30.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.369+0000 7f6fb2a00640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f90075960 0x7f6f90077e70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6fa401af60 tx=0x7f6fa401b0f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:30.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.373+0000 7f6faaa00640 1 -- 172.21.15.50:0/3947959570 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6fac05fd70 con 0x7f6fb415f6b0 2024-09-21T06:13:30.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.585+0000 7f6fb9600640 1 -- 172.21.15.50:0/3947959570 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone74693", "target": ["mon-mgr", ""]}) -- 0x7f6fb414df20 con 0x7f6f90075960 2024-09-21T06:13:30.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.593+0000 7f6faaa00640 1 -- 172.21.15.50:0/3947959570 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f6fb414df20 con 0x7f6f90075960 2024-09-21T06:13:30.594 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:30.594 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:13:30.594 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "canceled", 2024-09-21T06:13:30.594 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:13:30.595 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:13:30.595 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:13:30.595 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:13:30.595 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:30.595 INFO:teuthology.orchestra.run.smithi050.stdout: "failure": { 2024-09-21T06:13:30.595 INFO:teuthology.orchestra.run.smithi050.stdout: "errno": "4", 2024-09-21T06:13:30.595 INFO:teuthology.orchestra.run.smithi050.stdout: "error_msg": "user interrupted clone operation" 2024-09-21T06:13:30.595 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:30.595 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:30.595 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:30.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.597+0000 7f6fb9600640 1 -- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f90075960 msgr2=0x7f6f90077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.597+0000 7f6fb9600640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f90075960 0x7f6f90077e70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6fa401af60 tx=0x7f6fa401b0f0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.597+0000 7f6fb9600640 1 -- 172.21.15.50:0/3947959570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb415f6b0 msgr2=0x7f6fb414f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.597+0000 7f6fb9600640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb415f6b0 0x7f6fb414f210 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00c680 tx=0x7f6fac00cb40 comp rx=0 tx=0).stop 2024-09-21T06:13:30.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.597+0000 7f6fb9600640 1 -- 172.21.15.50:0/3947959570 shutdown_connections 2024-09-21T06:13:30.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.597+0000 7f6fb9600640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6f90075960 0x7f6f90077e70 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.597+0000 7f6fb9600640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb415f6b0 0x7f6fb414f210 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.597+0000 7f6fb9600640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6fb415c5d0 0x7f6fb41f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.597+0000 7f6fb9600640 1 --2- 172.21.15.50:0/3947959570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fb4074300 0x7f6fb41fae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.597+0000 7f6fb9600640 1 -- 172.21.15.50:0/3947959570 >> 172.21.15.50:0/3947959570 conn(0x7f6fb40b2c40 msgr2=0x7f6fb406ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:30.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.597+0000 7f6fb9600640 1 -- 172.21.15.50:0/3947959570 shutdown_connections 2024-09-21T06:13:30.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.597+0000 7f6fb9600640 1 -- 172.21.15.50:0/3947959570 wait complete. 2024-09-21T06:13:30.617 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-21T06:13:30.617 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone90034 2024-09-21T06:13:30.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.781+0000 7f88f2200640 1 -- 172.21.15.50:0/4134822267 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f88ec157030 msgr2=0x7f88ec0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.781+0000 7f88f2200640 1 --2- 172.21.15.50:0/4134822267 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f88ec157030 0x7f88ec0b5a80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009840 tx=0x7f88dc01c820 comp rx=0 tx=0).stop 2024-09-21T06:13:30.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 -- 172.21.15.50:0/4134822267 shutdown_connections 2024-09-21T06:13:30.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 --2- 172.21.15.50:0/4134822267 >> v2:172.21.15.88:3300/0 conn(0x7f88ec160d50 0x7f88ec074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 --2- 172.21.15.50:0/4134822267 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f88ec157030 0x7f88ec0b5a80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 -- 172.21.15.50:0/4134822267 >> 172.21.15.50:0/4134822267 conn(0x7f88ec0708c0 msgr2=0x7f88ec070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:30.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 -- 172.21.15.50:0/4134822267 shutdown_connections 2024-09-21T06:13:30.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 -- 172.21.15.50:0/4134822267 wait complete. 2024-09-21T06:13:30.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 Processor -- start 2024-09-21T06:13:30.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 -- start start 2024-09-21T06:13:30.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f88ec157030 0x7f88ec144e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:30.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88ec160d50 0x7f88ec1f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:30.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec161160 0x7f88ec14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:30.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec16a780 con 0x7f88ec161160 2024-09-21T06:13:30.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec16a600 con 0x7f88ec160d50 2024-09-21T06:13:30.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.785+0000 7f88f2200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f88ec16a900 con 0x7f88ec157030 2024-09-21T06:13:30.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88ebe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec161160 0x7f88ec14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:30.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88ebe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec161160 0x7f88ec14b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45912/0 (socket says 172.21.15.50:45912) 2024-09-21T06:13:30.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88ebe00640 1 -- 172.21.15.50:0/495832786 learned_addr learned my addr 172.21.15.50:0/495832786 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:30.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88eb400640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f88ec157030 0x7f88ec144e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:30.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88eaa00640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88ec160d50 0x7f88ec1f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:30.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88ebe00640 1 -- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f88ec157030 msgr2=0x7f88ec144e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88ebe00640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f88ec157030 0x7f88ec144e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88ebe00640 1 -- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88ec160d50 msgr2=0x7f88ec1f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88ebe00640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88ec160d50 0x7f88ec1f72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88ebe00640 1 -- 172.21.15.50:0/495832786 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec1f7c60 con 0x7f88ec161160 2024-09-21T06:13:30.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88ebe00640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec161160 0x7f88ec14b580 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f88e400cb40 tx=0x7f88e400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:30.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88e3400640 1 -- 172.21.15.50:0/495832786 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4016020 con 0x7f88ec161160 2024-09-21T06:13:30.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88e3400640 1 -- 172.21.15.50:0/495832786 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88e400ab60 con 0x7f88ec161160 2024-09-21T06:13:30.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88f2200640 1 -- 172.21.15.50:0/495832786 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec1f7f40 con 0x7f88ec161160 2024-09-21T06:13:30.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88e3400640 1 -- 172.21.15.50:0/495832786 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4012420 con 0x7f88ec161160 2024-09-21T06:13:30.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88f2200640 1 -- 172.21.15.50:0/495832786 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec150310 con 0x7f88ec161160 2024-09-21T06:13:30.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88f2200640 1 -- 172.21.15.50:0/495832786 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ec1f79b0 con 0x7f88ec161160 2024-09-21T06:13:30.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.789+0000 7f88e3400640 1 -- 172.21.15.50:0/495832786 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f88e4019020 con 0x7f88ec161160 2024-09-21T06:13:30.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.793+0000 7f88e3400640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f88cc075990 0x7f88cc077ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:30.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.793+0000 7f88e3400640 1 -- 172.21.15.50:0/495832786 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f88e4093640 con 0x7f88ec161160 2024-09-21T06:13:30.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.793+0000 7f88eb400640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f88cc075990 0x7f88cc077ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:30.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.793+0000 7f88eb400640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f88cc075990 0x7f88cc077ea0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0af000 tx=0x7f88dc01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:30.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:30.797+0000 7f88e3400640 1 -- 172.21.15.50:0/495832786 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f88e409f080 con 0x7f88ec161160 2024-09-21T06:13:31.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.009+0000 7f88f2200640 1 -- 172.21.15.50:0/495832786 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone90034", "target": ["mon-mgr", ""]}) -- 0x7f88ec06b540 con 0x7f88cc075990 2024-09-21T06:13:31.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.257+0000 7f88e3400640 1 -- 172.21.15.50:0/495832786 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f88dc0a6570 con 0x7f88cc075990 2024-09-21T06:13:31.256 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:31.256 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:13:31.256 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "canceled", 2024-09-21T06:13:31.256 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:13:31.256 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:13:31.256 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:13:31.256 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:13:31.256 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:31.257 INFO:teuthology.orchestra.run.smithi050.stdout: "failure": { 2024-09-21T06:13:31.257 INFO:teuthology.orchestra.run.smithi050.stdout: "errno": "4", 2024-09-21T06:13:31.257 INFO:teuthology.orchestra.run.smithi050.stdout: "error_msg": "user interrupted clone operation" 2024-09-21T06:13:31.257 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:31.257 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:31.257 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.257+0000 7f88f2200640 1 -- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f88cc075990 msgr2=0x7f88cc077ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.257+0000 7f88f2200640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f88cc075990 0x7f88cc077ea0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0af000 tx=0x7f88dc01b060 comp rx=0 tx=0).stop 2024-09-21T06:13:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.257+0000 7f88f2200640 1 -- 172.21.15.50:0/495832786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec161160 msgr2=0x7f88ec14b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.257+0000 7f88f2200640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec161160 0x7f88ec14b580 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f88e400cb40 tx=0x7f88e400cc30 comp rx=0 tx=0).stop 2024-09-21T06:13:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.261+0000 7f88f2200640 1 -- 172.21.15.50:0/495832786 shutdown_connections 2024-09-21T06:13:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.261+0000 7f88f2200640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f88cc075990 0x7f88cc077ea0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.261+0000 7f88f2200640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec161160 0x7f88ec14b580 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.261+0000 7f88f2200640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88ec160d50 0x7f88ec1f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.261+0000 7f88f2200640 1 --2- 172.21.15.50:0/495832786 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f88ec157030 0x7f88ec144e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.261+0000 7f88f2200640 1 -- 172.21.15.50:0/495832786 >> 172.21.15.50:0/495832786 conn(0x7f88ec0708c0 msgr2=0x7f88ec15e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.261+0000 7f88f2200640 1 -- 172.21.15.50:0/495832786 shutdown_connections 2024-09-21T06:13:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.261+0000 7f88f2200640 1 -- 172.21.15.50:0/495832786 wait complete. 2024-09-21T06:13:31.280 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-21T06:13:31.280 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone44825 2024-09-21T06:13:31.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.445+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3350127697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd8004d50 msgr2=0x7f5fd80051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:31.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.445+0000 7f5ff7e00640 1 --2- 172.21.15.50:0/3350127697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd8004d50 0x7f5fd80051a0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4001860 tx=0x7f5fe401c880 comp rx=0 tx=0).stop 2024-09-21T06:13:31.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.445+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3350127697 shutdown_connections 2024-09-21T06:13:31.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.445+0000 7f5ff7e00640 1 --2- 172.21.15.50:0/3350127697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fd8004d50 0x7f5fd80051a0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:31.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.445+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3350127697 >> 172.21.15.50:0/3350127697 conn(0x7f5ff00b2040 msgr2=0x7f5ff00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:31.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.445+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3350127697 shutdown_connections 2024-09-21T06:13:31.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.445+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3350127697 wait complete. 2024-09-21T06:13:31.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.445+0000 7f5ff7e00640 1 Processor -- start 2024-09-21T06:13:31.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.445+0000 7f5ff7e00640 1 -- start start 2024-09-21T06:13:31.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff7e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fd8004d50 0x7f5ff01f8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:31.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ff0158160 0x7f5ff00b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:31.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ff0158570 0x7f5ff00b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:31.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff7e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff015bc40 con 0x7f5ff0158570 2024-09-21T06:13:31.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff7e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff015bac0 con 0x7f5ff0158160 2024-09-21T06:13:31.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff7e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff015bdc0 con 0x7f5fd8004d50 2024-09-21T06:13:31.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ff0158570 0x7f5ff00b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:31.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ff0158570 0x7f5ff00b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45924/0 (socket says 172.21.15.50:45924) 2024-09-21T06:13:31.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff5e00640 1 -- 172.21.15.50:0/3381279814 learned_addr learned my addr 172.21.15.50:0/3381279814 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:31.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff5400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fd8004d50 0x7f5ff01f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:31.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff4a00640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ff0158160 0x7f5ff00b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:31.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff5e00640 1 -- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fd8004d50 msgr2=0x7f5ff01f8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:31.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff5e00640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fd8004d50 0x7f5ff01f8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:31.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff5e00640 1 -- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ff0158160 msgr2=0x7f5ff00b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:31.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff5e00640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ff0158160 0x7f5ff00b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:31.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff5e00640 1 -- 172.21.15.50:0/3381279814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff01349c0 con 0x7f5ff0158570 2024-09-21T06:13:31.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff5e00640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ff0158570 0x7f5ff00b3880 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0072d70 tx=0x7f5fe800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:31.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5fed600640 1 -- 172.21.15.50:0/3381279814 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8004b30 con 0x7f5ff0158570 2024-09-21T06:13:31.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5fed600640 1 -- 172.21.15.50:0/3381279814 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fe8004cd0 con 0x7f5ff0158570 2024-09-21T06:13:31.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5fed600640 1 -- 172.21.15.50:0/3381279814 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8003be0 con 0x7f5ff0158570 2024-09-21T06:13:31.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3381279814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff0134ca0 con 0x7f5ff0158570 2024-09-21T06:13:31.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3381279814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff0135260 con 0x7f5ff0158570 2024-09-21T06:13:31.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.449+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3381279814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff015d160 con 0x7f5ff0158570 2024-09-21T06:13:31.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.453+0000 7f5fed600640 1 -- 172.21.15.50:0/3381279814 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f5fe8004030 con 0x7f5ff0158570 2024-09-21T06:13:31.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.453+0000 7f5fed600640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fd0075990 0x7f5fd0077ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:31.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.453+0000 7f5fed600640 1 -- 172.21.15.50:0/3381279814 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f5fe8093d40 con 0x7f5ff0158570 2024-09-21T06:13:31.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.453+0000 7f5ff5400640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fd0075990 0x7f5fd0077ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:31.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.453+0000 7f5ff5400640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fd0075990 0x7f5fd0077ea0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4001830 tx=0x7f5fe40196c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:31.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.457+0000 7f5fed600640 1 -- 172.21.15.50:0/3381279814 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5fe80602a0 con 0x7f5ff0158570 2024-09-21T06:13:31.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:31.669+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3381279814 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone44825", "target": ["mon-mgr", ""]}) -- 0x7f5ff00b42d0 con 0x7f5fd0075990 2024-09-21T06:13:36.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5fed600640 1 -- 172.21.15.50:0/3381279814 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f5ff00b42d0 con 0x7f5fd0075990 2024-09-21T06:13:36.259 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:36.259 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:13:36.259 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "canceled", 2024-09-21T06:13:36.259 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:13:36.259 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:13:36.259 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:13:36.259 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:13:36.259 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:36.259 INFO:teuthology.orchestra.run.smithi050.stdout: "failure": { 2024-09-21T06:13:36.259 INFO:teuthology.orchestra.run.smithi050.stdout: "errno": "4", 2024-09-21T06:13:36.259 INFO:teuthology.orchestra.run.smithi050.stdout: "error_msg": "user interrupted clone operation" 2024-09-21T06:13:36.260 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:36.260 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:36.260 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:36.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fd0075990 msgr2=0x7f5fd0077ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5ff7e00640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fd0075990 0x7f5fd0077ea0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4001830 tx=0x7f5fe40196c0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3381279814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ff0158570 msgr2=0x7f5ff00b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5ff7e00640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ff0158570 0x7f5ff00b3880 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0072d70 tx=0x7f5fe800f3c0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3381279814 shutdown_connections 2024-09-21T06:13:36.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5ff7e00640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5fd0075990 0x7f5fd0077ea0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5ff7e00640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ff0158570 0x7f5ff00b3880 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5ff7e00640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ff0158160 0x7f5ff00b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5ff7e00640 1 --2- 172.21.15.50:0/3381279814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5fd8004d50 0x7f5ff01f8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3381279814 >> 172.21.15.50:0/3381279814 conn(0x7f5ff00b2040 msgr2=0x7f5ff00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:36.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3381279814 shutdown_connections 2024-09-21T06:13:36.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.261+0000 7f5ff7e00640 1 -- 172.21.15.50:0/3381279814 wait complete. 2024-09-21T06:13:36.287 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-21T06:13:36.287 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone64946 2024-09-21T06:13:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.453+0000 7f6693c00640 1 -- 172.21.15.50:0/3527987029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f668c0b5c70 msgr2=0x7f668c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.453+0000 7f6693c00640 1 --2- 172.21.15.50:0/3527987029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f668c0b5c70 0x7f668c15bb60 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f667c009900 tx=0x7f667c01c820 comp rx=0 tx=0).stop 2024-09-21T06:13:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.453+0000 7f6693c00640 1 -- 172.21.15.50:0/3527987029 shutdown_connections 2024-09-21T06:13:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.453+0000 7f6693c00640 1 --2- 172.21.15.50:0/3527987029 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f668c1527b0 0x7f668c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.453+0000 7f6693c00640 1 --2- 172.21.15.50:0/3527987029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f668c0b5c70 0x7f668c15bb60 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.453+0000 7f6693c00640 1 -- 172.21.15.50:0/3527987029 >> 172.21.15.50:0/3527987029 conn(0x7f668c0b2040 msgr2=0x7f668c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.457+0000 7f6693c00640 1 -- 172.21.15.50:0/3527987029 shutdown_connections 2024-09-21T06:13:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.457+0000 7f6693c00640 1 -- 172.21.15.50:0/3527987029 wait complete. 2024-09-21T06:13:36.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.457+0000 7f6693c00640 1 Processor -- start 2024-09-21T06:13:36.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.457+0000 7f6693c00640 1 -- start start 2024-09-21T06:13:36.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.457+0000 7f6693c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f668c0b4de0 0x7f668c13fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:36.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.457+0000 7f6693c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f668c0b5c70 0x7f668c153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:36.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.457+0000 7f6693c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f668c1527b0 0x7f668c15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:36.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.457+0000 7f6693c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f668c168c70 con 0x7f668c0b4de0 2024-09-21T06:13:36.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6693c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f668c168af0 con 0x7f668c1527b0 2024-09-21T06:13:36.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6693c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f668c168df0 con 0x7f668c0b5c70 2024-09-21T06:13:36.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6691200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f668c0b4de0 0x7f668c13fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:36.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6691200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f668c0b4de0 0x7f668c13fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45948/0 (socket says 172.21.15.50:45948) 2024-09-21T06:13:36.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6691200640 1 -- 172.21.15.50:0/459155847 learned_addr learned my addr 172.21.15.50:0/459155847 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:36.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6683e00640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f668c0b5c70 0x7f668c153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:36.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6691c00640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f668c1527b0 0x7f668c15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:36.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6691200640 1 -- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f668c0b5c70 msgr2=0x7f668c153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6691200640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f668c0b5c70 0x7f668c153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6691200640 1 -- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f668c1527b0 msgr2=0x7f668c15a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6691200640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f668c1527b0 0x7f668c15a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6691200640 1 -- 172.21.15.50:0/459155847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c14bef0 con 0x7f668c0b4de0 2024-09-21T06:13:36.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6691c00640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f668c1527b0 0x7f668c15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:36.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6691200640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f668c0b4de0 0x7f668c13fa60 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f667c009850 tx=0x7f667c0a5860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:36.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6689600640 1 -- 172.21.15.50:0/459155847 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c0b2040 con 0x7f668c0b4de0 2024-09-21T06:13:36.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6689600640 1 -- 172.21.15.50:0/459155847 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f667c0043c0 con 0x7f668c0b4de0 2024-09-21T06:13:36.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6689600640 1 -- 172.21.15.50:0/459155847 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c005290 con 0x7f668c0b4de0 2024-09-21T06:13:36.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6693c00640 1 -- 172.21.15.50:0/459155847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c154930 con 0x7f668c0b4de0 2024-09-21T06:13:36.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6693c00640 1 -- 172.21.15.50:0/459155847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c0aaa00 con 0x7f668c0b4de0 2024-09-21T06:13:36.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6693c00640 1 -- 172.21.15.50:0/459155847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668c15bb60 con 0x7f668c0b4de0 2024-09-21T06:13:36.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.461+0000 7f6689600640 1 -- 172.21.15.50:0/459155847 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f667c005430 con 0x7f668c0b4de0 2024-09-21T06:13:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.465+0000 7f6689600640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6668075990 0x7f6668077ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.465+0000 7f6689600640 1 -- 172.21.15.50:0/459155847 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f667c12d060 con 0x7f668c0b4de0 2024-09-21T06:13:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.465+0000 7f6683e00640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6668075990 0x7f6668077ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.465+0000 7f6683e00640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6668075990 0x7f6668077ea0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f668c1fbb10 tx=0x7f66740091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:36.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.469+0000 7f6689600640 1 -- 172.21.15.50:0/459155847 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f668c075580 con 0x7f668c0b4de0 2024-09-21T06:13:36.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.681+0000 7f6693c00640 1 -- 172.21.15.50:0/459155847 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone64946", "target": ["mon-mgr", ""]}) -- 0x7f668c06b540 con 0x7f6668075990 2024-09-21T06:13:36.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6689600640 1 -- 172.21.15.50:0/459155847 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f6674011bb0 con 0x7f6668075990 2024-09-21T06:13:36.693 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:36.693 INFO:teuthology.orchestra.run.smithi050.stdout: "status": { 2024-09-21T06:13:36.693 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "canceled", 2024-09-21T06:13:36.693 INFO:teuthology.orchestra.run.smithi050.stdout: "source": { 2024-09-21T06:13:36.693 INFO:teuthology.orchestra.run.smithi050.stdout: "volume": "cephfs", 2024-09-21T06:13:36.693 INFO:teuthology.orchestra.run.smithi050.stdout: "subvolume": "sv1", 2024-09-21T06:13:36.693 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot": "ss1" 2024-09-21T06:13:36.693 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:36.693 INFO:teuthology.orchestra.run.smithi050.stdout: "failure": { 2024-09-21T06:13:36.693 INFO:teuthology.orchestra.run.smithi050.stdout: "errno": "4", 2024-09-21T06:13:36.693 INFO:teuthology.orchestra.run.smithi050.stdout: "error_msg": "user interrupted clone operation" 2024-09-21T06:13:36.694 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:36.694 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:36.694 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6693c00640 1 -- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6668075990 msgr2=0x7f6668077ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6693c00640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6668075990 0x7f6668077ea0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f668c1fbb10 tx=0x7f66740091e0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6693c00640 1 -- 172.21.15.50:0/459155847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f668c0b4de0 msgr2=0x7f668c13fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6693c00640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f668c0b4de0 0x7f668c13fa60 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f667c009850 tx=0x7f667c0a5860 comp rx=0 tx=0).stop 2024-09-21T06:13:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6693c00640 1 -- 172.21.15.50:0/459155847 shutdown_connections 2024-09-21T06:13:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6693c00640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f668c1527b0 0x7f668c15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6693c00640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f668c0b5c70 0x7f668c153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6693c00640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6668075990 0x7f6668077ea0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6693c00640 1 --2- 172.21.15.50:0/459155847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f668c0b4de0 0x7f668c13fa60 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6693c00640 1 -- 172.21.15.50:0/459155847 >> 172.21.15.50:0/459155847 conn(0x7f668c0b2040 msgr2=0x7f668c0cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:36.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6693c00640 1 -- 172.21.15.50:0/459155847 shutdown_connections 2024-09-21T06:13:36.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.697+0000 7f6693c00640 1 -- 172.21.15.50:0/459155847 wait complete. 2024-09-21T06:13:36.715 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-21T06:13:36.716 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-21T06:13:36.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.877+0000 7fd0ddc00640 1 -- 172.21.15.50:0/2361324210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0d81536f0 msgr2=0x7fd0d815d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.877+0000 7fd0ddc00640 1 --2- 172.21.15.50:0/2361324210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0d81536f0 0x7fd0d815d480 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80cf890 tx=0x7fd0c801c770 comp rx=0 tx=0).stop 2024-09-21T06:13:36.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.877+0000 7fd0ddc00640 1 -- 172.21.15.50:0/2361324210 shutdown_connections 2024-09-21T06:13:36.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.877+0000 7fd0ddc00640 1 --2- 172.21.15.50:0/2361324210 >> v2:172.21.15.88:3300/0 conn(0x7fd0d8074950 0x7fd0d80b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.877+0000 7fd0ddc00640 1 --2- 172.21.15.50:0/2361324210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0d81536f0 0x7fd0d815d480 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.877+0000 7fd0ddc00640 1 -- 172.21.15.50:0/2361324210 >> 172.21.15.50:0/2361324210 conn(0x7fd0d80b4560 msgr2=0x7fd0d80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:36.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.877+0000 7fd0ddc00640 1 -- 172.21.15.50:0/2361324210 shutdown_connections 2024-09-21T06:13:36.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.877+0000 7fd0ddc00640 1 -- 172.21.15.50:0/2361324210 wait complete. 2024-09-21T06:13:36.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.877+0000 7fd0ddc00640 1 Processor -- start 2024-09-21T06:13:36.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.877+0000 7fd0ddc00640 1 -- start start 2024-09-21T06:13:36.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0ddc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0d81536f0 0x7fd0d814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:36.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0ddc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0d8160060 0x7fd0d80b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:36.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0ddc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0d81fc730 0x7fd0d80b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:36.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0ddc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d8169d20 con 0x7fd0d81fc730 2024-09-21T06:13:36.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0ddc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d8169ba0 con 0x7fd0d81536f0 2024-09-21T06:13:36.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0ddc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d8169ea0 con 0x7fd0d8160060 2024-09-21T06:13:36.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0dd200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0d81fc730 0x7fd0d80b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:36.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0dd200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0d81fc730 0x7fd0d80b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55788/0 (socket says 172.21.15.50:55788) 2024-09-21T06:13:36.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0dd200640 1 -- 172.21.15.50:0/509310830 learned_addr learned my addr 172.21.15.50:0/509310830 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:36.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0dd200640 1 -- 172.21.15.50:0/509310830 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0d8160060 msgr2=0x7fd0d80b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0dd200640 1 --2- 172.21.15.50:0/509310830 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0d8160060 0x7fd0d80b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0dd200640 1 -- 172.21.15.50:0/509310830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0d81536f0 msgr2=0x7fd0d814f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0dd200640 1 --2- 172.21.15.50:0/509310830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0d81536f0 0x7fd0d814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0dd200640 1 -- 172.21.15.50:0/509310830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d814f8d0 con 0x7fd0d81fc730 2024-09-21T06:13:36.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0dd200640 1 --2- 172.21.15.50:0/509310830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0d81fc730 0x7fd0d80b2640 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000ea80 tx=0x7fd0d000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:36.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0d4c00640 1 -- 172.21.15.50:0/509310830 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0004b30 con 0x7fd0d81fc730 2024-09-21T06:13:36.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0d4c00640 1 -- 172.21.15.50:0/509310830 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0d0004cf0 con 0x7fd0d81fc730 2024-09-21T06:13:36.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0ddc00640 1 -- 172.21.15.50:0/509310830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d81fbc50 con 0x7fd0d81fc730 2024-09-21T06:13:36.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0d4c00640 1 -- 172.21.15.50:0/509310830 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0005280 con 0x7fd0d81fc730 2024-09-21T06:13:36.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0ddc00640 1 -- 172.21.15.50:0/509310830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d81fc210 con 0x7fd0d81fc730 2024-09-21T06:13:36.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0ddc00640 1 -- 172.21.15.50:0/509310830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d814f620 con 0x7fd0d81fc730 2024-09-21T06:13:36.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.881+0000 7fd0d4c00640 1 -- 172.21.15.50:0/509310830 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7fd0d0005420 con 0x7fd0d81fc730 2024-09-21T06:13:36.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.885+0000 7fd0d4c00640 1 --2- 172.21.15.50:0/509310830 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd0b4075960 0x7fd0b4077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:36.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.885+0000 7fd0d4c00640 1 -- 172.21.15.50:0/509310830 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7fd0d00942a0 con 0x7fd0d81fc730 2024-09-21T06:13:36.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.885+0000 7fd0d7e00640 1 --2- 172.21.15.50:0/509310830 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd0b4075960 0x7fd0b4077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:36.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.885+0000 7fd0d7e00640 1 --2- 172.21.15.50:0/509310830 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd0b4075960 0x7fd0b4077e70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80af000 tx=0x7fd0c8019510 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:36.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:36.889+0000 7fd0d4c00640 1 -- 172.21.15.50:0/509310830 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd0d0060d20 con 0x7fd0d81fc730 2024-09-21T06:13:37.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 -- 172.21.15.50:0/509310830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd0d80b5260 con 0x7fd0d81fc730 2024-09-21T06:13:37.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0d4c00640 1 -- 172.21.15.50:0/509310830 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1927 (secure 0 0 0) 0x7fd0d0060620 con 0x7fd0d81fc730 2024-09-21T06:13:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:37.128 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "3f44e77a-b7bc-4ef8-b058-60eaaa9147d6", 2024-09-21T06:13:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2024-09-21T06:13:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK", 2024-09-21T06:13:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2024-09-21T06:13:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "mutes": [] 2024-09-21T06:13:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 6, 2024-09-21T06:13:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2024-09-21T06:13:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: 0, 2024-09-21T06:13:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: 1, 2024-09-21T06:13:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: 2 2024-09-21T06:13:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:13:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2024-09-21T06:13:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "a", 2024-09-21T06:13:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "b", 2024-09-21T06:13:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "c" 2024-09-21T06:13:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:13:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 1053, 2024-09-21T06:13:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2024-09-21T06:13:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2024-09-21T06:13:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "squid", 2024-09-21T06:13:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "num_mons": 3 2024-09-21T06:13:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2024-09-21T06:13:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 144, 2024-09-21T06:13:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 8, 2024-09-21T06:13:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 8, 2024-09-21T06:13:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_up_since": 1726898177, 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 8, 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_in_since": 1726898167, 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 97 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 97, 2024-09-21T06:13:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 3, 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 4284, 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 17819565965, 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 36853923840, 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 736240189440, 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 773094113280, 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "read_bytes_sec": 564957138, 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "write_bytes_sec": 564668999, 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "read_op_per_sec": 139, 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "write_op_per_sec": 154 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 95, 2024-09-21T06:13:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "btime": "2024-09-21T06:12:33:714142+0000", 2024-09-21T06:13:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 10, 2024-09-21T06:13:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "up": 1, 2024-09-21T06:13:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "in": 1, 2024-09-21T06:13:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1, 2024-09-21T06:13:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [ 2024-09-21T06:13:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:13:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "filesystem_id": 10, 2024-09-21T06:13:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2024-09-21T06:13:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "d", 2024-09-21T06:13:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "up:active", 2024-09-21T06:13:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 8632 2024-09-21T06:13:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:13:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 3 2024-09-21T06:13:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2024-09-21T06:13:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-21T06:13:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "num_standbys": 1, 2024-09-21T06:13:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-21T06:13:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-21T06:13:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-21T06:13:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-09-21T06:13:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-21T06:13:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:13:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2024-09-21T06:13:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 26, 2024-09-21T06:13:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2024-09-21T06:13:33.265510+0000", 2024-09-21T06:13:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2024-09-21T06:13:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": {} 2024-09-21T06:13:37.136 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:37.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 -- 172.21.15.50:0/509310830 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd0b4075960 msgr2=0x7fd0b4077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 --2- 172.21.15.50:0/509310830 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd0b4075960 0x7fd0b4077e70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80af000 tx=0x7fd0c8019510 comp rx=0 tx=0).stop 2024-09-21T06:13:37.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 -- 172.21.15.50:0/509310830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0d81fc730 msgr2=0x7fd0d80b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 --2- 172.21.15.50:0/509310830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0d81fc730 0x7fd0d80b2640 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000ea80 tx=0x7fd0d000ef40 comp rx=0 tx=0).stop 2024-09-21T06:13:37.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 -- 172.21.15.50:0/509310830 shutdown_connections 2024-09-21T06:13:37.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 --2- 172.21.15.50:0/509310830 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd0b4075960 0x7fd0b4077e70 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 --2- 172.21.15.50:0/509310830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0d81fc730 0x7fd0d80b2640 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 --2- 172.21.15.50:0/509310830 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd0d8160060 0x7fd0d80b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 --2- 172.21.15.50:0/509310830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0d81536f0 0x7fd0d814f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 -- 172.21.15.50:0/509310830 >> 172.21.15.50:0/509310830 conn(0x7fd0d80b4560 msgr2=0x7fd0d815e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:37.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 -- 172.21.15.50:0/509310830 shutdown_connections 2024-09-21T06:13:37.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.133+0000 7fd0ddc00640 1 -- 172.21.15.50:0/509310830 wait complete. 2024-09-21T06:13:37.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:37.151 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone99400/0078a113-7228-4587-9847-47491cfce39e 2024-09-21T06:13:37.151 DEBUG:teuthology.orchestra.run.smithi050:> ) 2024-09-21T06:13:37.162 INFO:teuthology.orchestra.run.smithi050.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone99400/0078a113-7228-4587-9847-47491cfce39e': No such file or directory 2024-09-21T06:13:37.163 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T06:13:37.163 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T06:13:37.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.369+0000 7f1310800640 1 -- 172.21.15.50:0/316175331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f130815cbb0 msgr2=0x7f130815cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.369+0000 7f1310800640 1 --2- 172.21.15.50:0/316175331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f130815cbb0 0x7f130815cfc0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009870 tx=0x7f12f801c6d0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.369+0000 7f1310800640 1 -- 172.21.15.50:0/316175331 shutdown_connections 2024-09-21T06:13:37.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.369+0000 7f1310800640 1 --2- 172.21.15.50:0/316175331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f130815cbb0 0x7f130815cfc0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.369+0000 7f1310800640 1 -- 172.21.15.50:0/316175331 >> 172.21.15.50:0/316175331 conn(0x7f13080b2040 msgr2=0x7f13080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:37.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.369+0000 7f1310800640 1 -- 172.21.15.50:0/316175331 shutdown_connections 2024-09-21T06:13:37.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.369+0000 7f1310800640 1 -- 172.21.15.50:0/316175331 wait complete. 2024-09-21T06:13:37.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f1310800640 1 Processor -- start 2024-09-21T06:13:37.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f1310800640 1 -- start start 2024-09-21T06:13:37.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f1310800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1308072100 0x7f1308139570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:37.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f1310800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f13080cefa0 0x7f1308155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:37.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f1310800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f130815cbb0 0x7f1308155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f1310800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f130816b0f0 con 0x7f1308072100 2024-09-21T06:13:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f1310800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f130816af70 con 0x7f130815cbb0 2024-09-21T06:13:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f1310800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f130816b270 con 0x7f13080cefa0 2024-09-21T06:13:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f130de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1308072100 0x7f1308139570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f130de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1308072100 0x7f1308139570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55800/0 (socket says 172.21.15.50:55800) 2024-09-21T06:13:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f130de00640 1 -- 172.21.15.50:0/728352655 learned_addr learned my addr 172.21.15.50:0/728352655 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:37.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f130e800640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f130815cbb0 0x7f1308155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:37.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f130d400640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f13080cefa0 0x7f1308155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:37.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f130de00640 1 -- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f13080cefa0 msgr2=0x7f1308155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f130de00640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f13080cefa0 0x7f1308155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f130de00640 1 -- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f130815cbb0 msgr2=0x7f1308155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f130de00640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f130815cbb0 0x7f1308155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.373+0000 7f130de00640 1 -- 172.21.15.50:0/728352655 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130813f5a0 con 0x7f1308072100 2024-09-21T06:13:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.377+0000 7f130de00640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1308072100 0x7f1308139570 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f12fc001ec0 tx=0x7f12fc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.377+0000 7f1302000640 1 -- 172.21.15.50:0/728352655 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc017070 con 0x7f1308072100 2024-09-21T06:13:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.377+0000 7f1302000640 1 -- 172.21.15.50:0/728352655 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12fc003600 con 0x7f1308072100 2024-09-21T06:13:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.377+0000 7f1302000640 1 -- 172.21.15.50:0/728352655 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc003a50 con 0x7f1308072100 2024-09-21T06:13:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.377+0000 7f1310800640 1 -- 172.21.15.50:0/728352655 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130813f880 con 0x7f1308072100 2024-09-21T06:13:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.377+0000 7f1310800640 1 -- 172.21.15.50:0/728352655 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130813fe40 con 0x7f1308072100 2024-09-21T06:13:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.377+0000 7f1310800640 1 -- 172.21.15.50:0/728352655 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f130815cfc0 con 0x7f1308072100 2024-09-21T06:13:37.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.377+0000 7f1302000640 1 -- 172.21.15.50:0/728352655 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f12fc01a070 con 0x7f1308072100 2024-09-21T06:13:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.377+0000 7f1302000640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f12ec075960 0x7f12ec077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.377+0000 7f1302000640 1 -- 172.21.15.50:0/728352655 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f12fc0928e0 con 0x7f1308072100 2024-09-21T06:13:37.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.381+0000 7f130d400640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f12ec075960 0x7f12ec077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:37.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.381+0000 7f130d400640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f12ec075960 0x7f12ec077e70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f12f80af000 tx=0x7f12f801af10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:37.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.381+0000 7f1302000640 1 -- 172.21.15.50:0/728352655 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f12fc05ec90 con 0x7f1308072100 2024-09-21T06:13:37.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.581+0000 7f1310800640 1 -- 172.21.15.50:0/728352655 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f130806b540 con 0x7f12ec075960 2024-09-21T06:13:37.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1302000640 1 -- 172.21.15.50:0/728352655 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f12f80a65b0 con 0x7f12ec075960 2024-09-21T06:13:37.579 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:13:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:13:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephfs" 2024-09-21T06:13:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:37.580 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:13:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1310800640 1 -- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f12ec075960 msgr2=0x7f12ec077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1310800640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f12ec075960 0x7f12ec077e70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f12f80af000 tx=0x7f12f801af10 comp rx=0 tx=0).stop 2024-09-21T06:13:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1310800640 1 -- 172.21.15.50:0/728352655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1308072100 msgr2=0x7f1308139570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1310800640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1308072100 0x7f1308139570 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f12fc001ec0 tx=0x7f12fc00c890 comp rx=0 tx=0).stop 2024-09-21T06:13:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1310800640 1 -- 172.21.15.50:0/728352655 shutdown_connections 2024-09-21T06:13:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1310800640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f130815cbb0 0x7f1308155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1310800640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f13080cefa0 0x7f1308155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1310800640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f12ec075960 0x7f12ec077e70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1310800640 1 --2- 172.21.15.50:0/728352655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1308072100 0x7f1308139570 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1310800640 1 -- 172.21.15.50:0/728352655 >> 172.21.15.50:0/728352655 conn(0x7f13080b2040 msgr2=0x7f13080b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1310800640 1 -- 172.21.15.50:0/728352655 shutdown_connections 2024-09-21T06:13:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.585+0000 7f1310800640 1 -- 172.21.15.50:0/728352655 wait complete. 2024-09-21T06:13:37.603 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-21T06:13:37.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.773+0000 7fb29f000640 1 -- 172.21.15.50:0/3859602909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb298074b20 msgr2=0x7fb29815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.773+0000 7fb29f000640 1 --2- 172.21.15.50:0/3859602909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb298074b20 0x7fb29815cf90 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fb290009bc0 tx=0x7fb29001cba0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.773+0000 7fb29f000640 1 -- 172.21.15.50:0/3859602909 shutdown_connections 2024-09-21T06:13:37.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.773+0000 7fb29f000640 1 --2- 172.21.15.50:0/3859602909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb298074b20 0x7fb29815cf90 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.773+0000 7fb29f000640 1 --2- 172.21.15.50:0/3859602909 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb298157030 0x7fb2980b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.773+0000 7fb29f000640 1 -- 172.21.15.50:0/3859602909 >> 172.21.15.50:0/3859602909 conn(0x7fb2980708c0 msgr2=0x7fb298070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:37.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.773+0000 7fb29f000640 1 -- 172.21.15.50:0/3859602909 shutdown_connections 2024-09-21T06:13:37.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.773+0000 7fb29f000640 1 -- 172.21.15.50:0/3859602909 wait complete. 2024-09-21T06:13:37.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.773+0000 7fb29f000640 1 Processor -- start 2024-09-21T06:13:37.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.773+0000 7fb29f000640 1 -- start start 2024-09-21T06:13:37.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb29f000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb298157030 0x7fb298154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:37.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb29f000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb298160d50 0x7fb29813fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:37.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb29f000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb29814cf60 0x7fb29815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:37.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb29f000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb29816a800 con 0x7fb298157030 2024-09-21T06:13:37.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb29f000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb29816a680 con 0x7fb298160d50 2024-09-21T06:13:37.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb29f000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb29816a980 con 0x7fb29814cf60 2024-09-21T06:13:37.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb297e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb298157030 0x7fb298154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:37.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb297e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb298157030 0x7fb298154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55822/0 (socket says 172.21.15.50:55822) 2024-09-21T06:13:37.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb297e00640 1 -- 172.21.15.50:0/618848879 learned_addr learned my addr 172.21.15.50:0/618848879 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:37.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb29d000640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb29814cf60 0x7fb29815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:37.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb28fe00640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb298160d50 0x7fb29813fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:37.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb297e00640 1 -- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb29814cf60 msgr2=0x7fb29815bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb297e00640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb29814cf60 0x7fb29815bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb297e00640 1 -- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb298160d50 msgr2=0x7fb29813fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb297e00640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb298160d50 0x7fb29813fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb297e00640 1 -- 172.21.15.50:0/618848879 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2981568d0 con 0x7fb298157030 2024-09-21T06:13:37.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb28fe00640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb298160d50 0x7fb29813fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:37.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb297e00640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb298157030 0x7fb298154fa0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fb28800e7c0 tx=0x7fb28800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:37.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb294c00640 1 -- 172.21.15.50:0/618848879 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28800ccc0 con 0x7fb298157030 2024-09-21T06:13:37.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb294c00640 1 -- 172.21.15.50:0/618848879 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb28800ce80 con 0x7fb298157030 2024-09-21T06:13:37.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb29f000640 1 -- 172.21.15.50:0/618848879 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb298155830 con 0x7fb298157030 2024-09-21T06:13:37.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb294c00640 1 -- 172.21.15.50:0/618848879 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288005280 con 0x7fb298157030 2024-09-21T06:13:37.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb29f000640 1 -- 172.21.15.50:0/618848879 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb298155df0 con 0x7fb298157030 2024-09-21T06:13:37.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.777+0000 7fb29f000640 1 -- 172.21.15.50:0/618848879 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb29815cf90 con 0x7fb298157030 2024-09-21T06:13:37.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.781+0000 7fb294c00640 1 -- 172.21.15.50:0/618848879 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7fb288005420 con 0x7fb298157030 2024-09-21T06:13:37.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.781+0000 7fb294c00640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb274075960 0x7fb274077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:37.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.781+0000 7fb294c00640 1 -- 172.21.15.50:0/618848879 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7fb288094030 con 0x7fb298157030 2024-09-21T06:13:37.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.781+0000 7fb28fe00640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb274075960 0x7fb274077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:37.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.781+0000 7fb28fe00640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb274075960 0x7fb274077e70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb2981541f0 tx=0x7fb280009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:37.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.785+0000 7fb294c00640 1 -- 172.21.15.50:0/618848879 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb288060a30 con 0x7fb298157030 2024-09-21T06:13:37.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:37.993+0000 7fb29f000640 1 -- 172.21.15.50:0/618848879 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb29806b540 con 0x7fb274075960 2024-09-21T06:13:37.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.001+0000 7fb294c00640 1 -- 172.21.15.50:0/618848879 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+380 (secure 0 0 0) 0x7fb2800050d0 con 0x7fb274075960 2024-09-21T06:13:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:37.998 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:13:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:13:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone34632" 2024-09-21T06:13:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sv1" 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone90034" 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone74693" 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone99400" 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:13:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone64946" 2024-09-21T06:13:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:13:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone44825" 2024-09-21T06:13:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-21T06:13:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:13:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subvol_clone5031" 2024-09-21T06:13:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:38.000 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:13:38.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.005+0000 7fb29f000640 1 -- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb274075960 msgr2=0x7fb274077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.005+0000 7fb29f000640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb274075960 0x7fb274077e70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb2981541f0 tx=0x7fb280009040 comp rx=0 tx=0).stop 2024-09-21T06:13:38.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.005+0000 7fb29f000640 1 -- 172.21.15.50:0/618848879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb298157030 msgr2=0x7fb298154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.005+0000 7fb29f000640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb298157030 0x7fb298154fa0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fb28800e7c0 tx=0x7fb28800ec80 comp rx=0 tx=0).stop 2024-09-21T06:13:38.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.005+0000 7fb29f000640 1 -- 172.21.15.50:0/618848879 shutdown_connections 2024-09-21T06:13:38.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.005+0000 7fb29f000640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb29814cf60 0x7fb29815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.005+0000 7fb29f000640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb298160d50 0x7fb29813fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.005+0000 7fb29f000640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb274075960 0x7fb274077e70 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.005+0000 7fb29f000640 1 --2- 172.21.15.50:0/618848879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb298157030 0x7fb298154fa0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.005+0000 7fb29f000640 1 -- 172.21.15.50:0/618848879 >> 172.21.15.50:0/618848879 conn(0x7fb2980708c0 msgr2=0x7fb29815e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:38.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.005+0000 7fb29f000640 1 -- 172.21.15.50:0/618848879 shutdown_connections 2024-09-21T06:13:38.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.005+0000 7fb29f000640 1 -- 172.21.15.50:0/618848879 wait complete. 2024-09-21T06:13:38.021 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone34632 2024-09-21T06:13:38.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.189+0000 7f8fb5800640 1 -- 172.21.15.50:0/1681374394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fb0159130 msgr2=0x7f8fb015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.189+0000 7f8fb5800640 1 --2- 172.21.15.50:0/1681374394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fb0159130 0x7f8fb015bb90 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00d14d0 tx=0x7f8fa001c830 comp rx=0 tx=0).stop 2024-09-21T06:13:38.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.189+0000 7f8fb5800640 1 -- 172.21.15.50:0/1681374394 shutdown_connections 2024-09-21T06:13:38.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.189+0000 7f8fb5800640 1 --2- 172.21.15.50:0/1681374394 >> v2:172.21.15.88:3300/0 conn(0x7f8fb006cdf0 0x7f8fb006d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.189+0000 7f8fb5800640 1 --2- 172.21.15.50:0/1681374394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fb0159130 0x7f8fb015bb90 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.189+0000 7f8fb5800640 1 -- 172.21.15.50:0/1681374394 >> 172.21.15.50:0/1681374394 conn(0x7f8fb00b35f0 msgr2=0x7f8fb00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:38.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.189+0000 7f8fb5800640 1 -- 172.21.15.50:0/1681374394 shutdown_connections 2024-09-21T06:13:38.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.189+0000 7f8fb5800640 1 -- 172.21.15.50:0/1681374394 wait complete. 2024-09-21T06:13:38.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fb5800640 1 Processor -- start 2024-09-21T06:13:38.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fb5800640 1 -- start start 2024-09-21T06:13:38.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fb5800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8fb006cdf0 0x7f8fb0146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:38.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fb5800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fb006fc90 0x7f8fb015e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:38.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fb5800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8fb0159130 0x7f8fb01646c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:38.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fb5800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0168d00 con 0x7f8fb006fc90 2024-09-21T06:13:38.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fb5800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0168b80 con 0x7f8fb006cdf0 2024-09-21T06:13:38.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fb5800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb0168e80 con 0x7f8fb0159130 2024-09-21T06:13:38.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fa7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fb006fc90 0x7f8fb015e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:38.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fa7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fb006fc90 0x7f8fb015e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55840/0 (socket says 172.21.15.50:55840) 2024-09-21T06:13:38.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fa7e00640 1 -- 172.21.15.50:0/3096741807 learned_addr learned my addr 172.21.15.50:0/3096741807 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:38.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8faea00640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8fb006cdf0 0x7f8fb0146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:38.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8faf400640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8fb0159130 0x7f8fb01646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:38.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fa7e00640 1 -- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8fb0159130 msgr2=0x7f8fb01646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fa7e00640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8fb0159130 0x7f8fb01646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fa7e00640 1 -- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8fb006cdf0 msgr2=0x7f8fb0146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fa7e00640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8fb006cdf0 0x7f8fb0146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fa7e00640 1 -- 172.21.15.50:0/3096741807 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb015eb30 con 0x7f8fb006fc90 2024-09-21T06:13:38.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8faf400640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8fb0159130 0x7f8fb01646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:38.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fa7e00640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fb006fc90 0x7f8fb015e5f0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800d750 tx=0x7f8f9800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:38.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fa6a00640 1 -- 172.21.15.50:0/3096741807 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98017070 con 0x7f8fb006fc90 2024-09-21T06:13:38.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fa6a00640 1 -- 172.21.15.50:0/3096741807 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f98009d20 con 0x7f8fb006fc90 2024-09-21T06:13:38.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fa6a00640 1 -- 172.21.15.50:0/3096741807 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98005280 con 0x7f8fb006fc90 2024-09-21T06:13:38.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fb5800640 1 -- 172.21.15.50:0/3096741807 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb015d0a0 con 0x7f8fb006fc90 2024-09-21T06:13:38.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fb5800640 1 -- 172.21.15.50:0/3096741807 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb015d660 con 0x7f8fb006fc90 2024-09-21T06:13:38.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.193+0000 7f8fb5800640 1 -- 172.21.15.50:0/3096741807 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb006d200 con 0x7f8fb006fc90 2024-09-21T06:13:38.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.197+0000 7f8fa6a00640 1 -- 172.21.15.50:0/3096741807 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f8f98005420 con 0x7f8fb006fc90 2024-09-21T06:13:38.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.197+0000 7f8fa6a00640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f8c075960 0x7f8f8c077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:38.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.197+0000 7f8fa6a00640 1 -- 172.21.15.50:0/3096741807 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f8f980937b0 con 0x7f8fb006fc90 2024-09-21T06:13:38.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.197+0000 7f8faea00640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f8c075960 0x7f8f8c077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:38.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.197+0000 7f8faea00640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f8c075960 0x7f8f8c077e70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00af000 tx=0x7f8fa001b070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:38.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.201+0000 7f8fa6a00640 1 -- 172.21.15.50:0/3096741807 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8f9800de60 con 0x7f8fb006fc90 2024-09-21T06:13:38.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.409+0000 7f8fb5800640 1 -- 172.21.15.50:0/3096741807 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone34632", "target": ["mon-mgr", ""]}) -- 0x7f8fb006b540 con 0x7f8f8c075960 2024-09-21T06:13:38.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fa6a00640 1 -- 172.21.15.50:0/3096741807 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8fa00a6680 con 0x7f8f8c075960 2024-09-21T06:13:38.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fb5800640 1 -- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f8c075960 msgr2=0x7f8f8c077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fb5800640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f8c075960 0x7f8f8c077e70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00af000 tx=0x7f8fa001b070 comp rx=0 tx=0).stop 2024-09-21T06:13:38.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fb5800640 1 -- 172.21.15.50:0/3096741807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fb006fc90 msgr2=0x7f8fb015e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fb5800640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fb006fc90 0x7f8fb015e5f0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800d750 tx=0x7f8f9800dc10 comp rx=0 tx=0).stop 2024-09-21T06:13:38.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fb5800640 1 -- 172.21.15.50:0/3096741807 shutdown_connections 2024-09-21T06:13:38.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fb5800640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8fb0159130 0x7f8fb01646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fb5800640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8f8c075960 0x7f8f8c077e70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fb5800640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fb006fc90 0x7f8fb015e5f0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fb5800640 1 --2- 172.21.15.50:0/3096741807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8fb006cdf0 0x7f8fb0146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fb5800640 1 -- 172.21.15.50:0/3096741807 >> 172.21.15.50:0/3096741807 conn(0x7f8fb00b35f0 msgr2=0x7f8fb0075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:38.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fb5800640 1 -- 172.21.15.50:0/3096741807 shutdown_connections 2024-09-21T06:13:38.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.429+0000 7f8fb5800640 1 -- 172.21.15.50:0/3096741807 wait complete. 2024-09-21T06:13:38.440 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:38.441+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:38.440 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:38.441+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:38.440 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:38.441+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:38.440 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:38.441+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:38.440 INFO:tasks.ceph.mgr.x.smithi088.stderr:2024-09-21T06:13:38.441+0000 7f65a4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:38.448 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-21T06:13:38.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.617+0000 7f8deda00640 1 -- 172.21.15.50:0/3330358756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8de815c5d0 msgr2=0x7f8de8073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.617+0000 7f8deda00640 1 --2- 172.21.15.50:0/3330358756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8de815c5d0 0x7f8de8073dc0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f8de806ae60 tx=0x7f8dd801c890 comp rx=0 tx=0).stop 2024-09-21T06:13:38.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.617+0000 7f8deda00640 1 -- 172.21.15.50:0/3330358756 shutdown_connections 2024-09-21T06:13:38.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.617+0000 7f8deda00640 1 --2- 172.21.15.50:0/3330358756 >> v2:172.21.15.88:3300/0 conn(0x7f8de8074300 0x7f8de815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.617+0000 7f8deda00640 1 --2- 172.21.15.50:0/3330358756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8de815c5d0 0x7f8de8073dc0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.617+0000 7f8deda00640 1 -- 172.21.15.50:0/3330358756 >> 172.21.15.50:0/3330358756 conn(0x7f8de80b2c40 msgr2=0x7f8de80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:38.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.617+0000 7f8deda00640 1 -- 172.21.15.50:0/3330358756 shutdown_connections 2024-09-21T06:13:38.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.617+0000 7f8deda00640 1 -- 172.21.15.50:0/3330358756 wait complete. 2024-09-21T06:13:38.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.617+0000 7f8deda00640 1 Processor -- start 2024-09-21T06:13:38.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.617+0000 7f8deda00640 1 -- start start 2024-09-21T06:13:38.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8deda00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8de8074300 0x7f8de80aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:38.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8deda00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8de815c5d0 0x7f8de80ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:38.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8deda00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8de815f6b0 0x7f8de815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:38.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8deda00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8de806f980 con 0x7f8de815f6b0 2024-09-21T06:13:38.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8deda00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8de80b3d50 con 0x7f8de8074300 2024-09-21T06:13:38.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8deda00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8de806fb00 con 0x7f8de815c5d0 2024-09-21T06:13:38.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8de7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8de815f6b0 0x7f8de815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:38.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8de7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8de815f6b0 0x7f8de815a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55846/0 (socket says 172.21.15.50:55846) 2024-09-21T06:13:38.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8de7400640 1 -- 172.21.15.50:0/3929913899 learned_addr learned my addr 172.21.15.50:0/3929913899 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:38.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8de6a00640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8de8074300 0x7f8de80aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:38.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8ddfe00640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8de815c5d0 0x7f8de80ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:38.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8de7400640 1 -- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8de815c5d0 msgr2=0x7f8de80ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8de7400640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8de815c5d0 0x7f8de80ab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8de7400640 1 -- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8de8074300 msgr2=0x7f8de80aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8de7400640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8de8074300 0x7f8de80aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8de7400640 1 -- 172.21.15.50:0/3929913899 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de80ac660 con 0x7f8de815f6b0 2024-09-21T06:13:38.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8de7400640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8de815f6b0 0x7f8de815a9f0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f8de000c730 tx=0x7f8de000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:38.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8ddea00640 1 -- 172.21.15.50:0/3929913899 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0003b80 con 0x7f8de815f6b0 2024-09-21T06:13:38.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8ddea00640 1 -- 172.21.15.50:0/3929913899 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8de0003d40 con 0x7f8de815f6b0 2024-09-21T06:13:38.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8deda00640 1 -- 172.21.15.50:0/3929913899 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de80abb90 con 0x7f8de815f6b0 2024-09-21T06:13:38.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8ddea00640 1 -- 172.21.15.50:0/3929913899 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de00052d0 con 0x7f8de815f6b0 2024-09-21T06:13:38.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8deda00640 1 -- 172.21.15.50:0/3929913899 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de80ac380 con 0x7f8de815f6b0 2024-09-21T06:13:38.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.621+0000 7f8deda00640 1 -- 172.21.15.50:0/3929913899 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de815f170 con 0x7f8de815f6b0 2024-09-21T06:13:38.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.625+0000 7f8ddea00640 1 -- 172.21.15.50:0/3929913899 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f8de0005680 con 0x7f8de815f6b0 2024-09-21T06:13:38.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.625+0000 7f8ddea00640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8dc4075960 0x7f8dc4077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:38.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.625+0000 7f8ddea00640 1 -- 172.21.15.50:0/3929913899 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f8de0093d60 con 0x7f8de815f6b0 2024-09-21T06:13:38.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.625+0000 7f8de6a00640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8dc4075960 0x7f8dc4077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:38.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.625+0000 7f8de6a00640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8dc4075960 0x7f8dc4077e70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80af000 tx=0x7f8dd801b0d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:38.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.629+0000 7f8ddea00640 1 -- 172.21.15.50:0/3929913899 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8de00603a0 con 0x7f8de815f6b0 2024-09-21T06:13:38.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.837+0000 7f8deda00640 1 -- 172.21.15.50:0/3929913899 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8de80af2c0 con 0x7f8dc4075960 2024-09-21T06:13:38.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.853+0000 7f8ddea00640 1 -- 172.21.15.50:0/3929913899 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8dd80a5a40 con 0x7f8dc4075960 2024-09-21T06:13:38.848 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:38.848 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-21T06:13:38.848 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-21T06:13:38.848 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ss1" 2024-09-21T06:13:38.849 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-21T06:13:38.849 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-21T06:13:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.853+0000 7f8deda00640 1 -- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8dc4075960 msgr2=0x7f8dc4077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.853+0000 7f8deda00640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8dc4075960 0x7f8dc4077e70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80af000 tx=0x7f8dd801b0d0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.853+0000 7f8deda00640 1 -- 172.21.15.50:0/3929913899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8de815f6b0 msgr2=0x7f8de815a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.853+0000 7f8deda00640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8de815f6b0 0x7f8de815a9f0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f8de000c730 tx=0x7f8de000cbf0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.853+0000 7f8deda00640 1 -- 172.21.15.50:0/3929913899 shutdown_connections 2024-09-21T06:13:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.853+0000 7f8deda00640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8dc4075960 0x7f8dc4077e70 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.853+0000 7f8deda00640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8de815f6b0 0x7f8de815a9f0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.853+0000 7f8deda00640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8de815c5d0 0x7f8de80ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.853+0000 7f8deda00640 1 --2- 172.21.15.50:0/3929913899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8de8074300 0x7f8de80aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.857+0000 7f8deda00640 1 -- 172.21.15.50:0/3929913899 >> 172.21.15.50:0/3929913899 conn(0x7f8de80b2c40 msgr2=0x7f8de80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.857+0000 7f8deda00640 1 -- 172.21.15.50:0/3929913899 shutdown_connections 2024-09-21T06:13:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:38.857+0000 7f8deda00640 1 -- 172.21.15.50:0/3929913899 wait complete. 2024-09-21T06:13:38.871 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-21T06:13:39.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.041+0000 7f9888a00640 1 -- 172.21.15.50:0/66437445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9884072770 msgr2=0x7f98840cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.041+0000 7f9888a00640 1 --2- 172.21.15.50:0/66437445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9884072770 0x7f98840cfaa0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f98840ce210 tx=0x7f987c01cd80 comp rx=0 tx=0).stop 2024-09-21T06:13:39.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.041+0000 7f9888a00640 1 -- 172.21.15.50:0/66437445 shutdown_connections 2024-09-21T06:13:39.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.041+0000 7f9888a00640 1 --2- 172.21.15.50:0/66437445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9884072770 0x7f98840cfaa0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.041+0000 7f9888a00640 1 --2- 172.21.15.50:0/66437445 >> v2:172.21.15.88:3300/0 conn(0x7f9884072360 0x7f98840cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.041+0000 7f9888a00640 1 --2- 172.21.15.50:0/66437445 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9884155be0 0x7f988415c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.041+0000 7f9888a00640 1 -- 172.21.15.50:0/66437445 >> 172.21.15.50:0/66437445 conn(0x7f988406d920 msgr2=0x7f98840b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:39.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.041+0000 7f9888a00640 1 -- 172.21.15.50:0/66437445 shutdown_connections 2024-09-21T06:13:39.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.041+0000 7f9888a00640 1 -- 172.21.15.50:0/66437445 wait complete. 2024-09-21T06:13:39.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.041+0000 7f9888a00640 1 Processor -- start 2024-09-21T06:13:39.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9888a00640 1 -- start start 2024-09-21T06:13:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9888a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9884072360 0x7f9884142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9888a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9884072770 0x7f988406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9888a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9884155be0 0x7f988406f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9888a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f988415f730 con 0x7f9884072360 2024-09-21T06:13:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9888a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f988415f5b0 con 0x7f9884155be0 2024-09-21T06:13:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9888a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f988415f8b0 con 0x7f9884072770 2024-09-21T06:13:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9883400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9884072360 0x7f9884142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9883400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9884072360 0x7f9884142e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55858/0 (socket says 172.21.15.50:55858) 2024-09-21T06:13:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9883400640 1 -- 172.21.15.50:0/3381495646 learned_addr learned my addr 172.21.15.50:0/3381495646 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f987be00640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9884072770 0x7f988406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9883e00640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9884155be0 0x7f988406f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9883400640 1 -- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9884072770 msgr2=0x7f988406f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9883400640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9884072770 0x7f988406f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9883400640 1 -- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9884155be0 msgr2=0x7f988406f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9883400640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9884155be0 0x7f988406f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9883400640 1 -- 172.21.15.50:0/3381495646 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9884168120 con 0x7f9884072360 2024-09-21T06:13:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9883400640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9884072360 0x7f9884142e90 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f987400d780 tx=0x7f987400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f987b400640 1 -- 172.21.15.50:0/3381495646 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874017070 con 0x7f9884072360 2024-09-21T06:13:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f987b400640 1 -- 172.21.15.50:0/3381495646 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f987400de90 con 0x7f9884072360 2024-09-21T06:13:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f987b400640 1 -- 172.21.15.50:0/3381495646 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874004020 con 0x7f9884072360 2024-09-21T06:13:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9888a00640 1 -- 172.21.15.50:0/3381495646 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98840b0210 con 0x7f9884072360 2024-09-21T06:13:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.045+0000 7f9888a00640 1 -- 172.21.15.50:0/3381495646 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98840b04b0 con 0x7f9884072360 2024-09-21T06:13:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.049+0000 7f9888a00640 1 -- 172.21.15.50:0/3381495646 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f988415fd30 con 0x7f9884072360 2024-09-21T06:13:39.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.049+0000 7f987b400640 1 -- 172.21.15.50:0/3381495646 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f98740041c0 con 0x7f9884072360 2024-09-21T06:13:39.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.049+0000 7f987b400640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9860075960 0x7f9860077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.049+0000 7f987b400640 1 -- 172.21.15.50:0/3381495646 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f9874093800 con 0x7f9884072360 2024-09-21T06:13:39.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.049+0000 7f987be00640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9860075960 0x7f9860077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.049+0000 7f987be00640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9860075960 0x7f9860077e70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f98840b3020 tx=0x7f986c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.053+0000 7f987b400640 1 -- 172.21.15.50:0/3381495646 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9874020020 con 0x7f9884072360 2024-09-21T06:13:39.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.261+0000 7f9888a00640 1 -- 172.21.15.50:0/3381495646 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f988406b540 con 0x7f9860075960 2024-09-21T06:13:39.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.289+0000 7f987b400640 1 -- 172.21.15.50:0/3381495646 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f986c011b50 con 0x7f9860075960 2024-09-21T06:13:39.286 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:39.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.293+0000 7f9888a00640 1 -- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9860075960 msgr2=0x7f9860077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.293+0000 7f9888a00640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9860075960 0x7f9860077e70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f98840b3020 tx=0x7f986c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.293+0000 7f9888a00640 1 -- 172.21.15.50:0/3381495646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9884072360 msgr2=0x7f9884142e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.293+0000 7f9888a00640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9884072360 0x7f9884142e90 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f987400d780 tx=0x7f987400dc40 comp rx=0 tx=0).stop 2024-09-21T06:13:39.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.293+0000 7f9888a00640 1 -- 172.21.15.50:0/3381495646 shutdown_connections 2024-09-21T06:13:39.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.293+0000 7f9888a00640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9884155be0 0x7f988406f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.293+0000 7f9888a00640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f9884072770 0x7f988406f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.293+0000 7f9888a00640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9860075960 0x7f9860077e70 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.293+0000 7f9888a00640 1 --2- 172.21.15.50:0/3381495646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9884072360 0x7f9884142e90 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.293+0000 7f9888a00640 1 -- 172.21.15.50:0/3381495646 >> 172.21.15.50:0/3381495646 conn(0x7f988406d920 msgr2=0x7f98840cf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:39.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.293+0000 7f9888a00640 1 -- 172.21.15.50:0/3381495646 shutdown_connections 2024-09-21T06:13:39.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.293+0000 7f9888a00640 1 -- 172.21.15.50:0/3381495646 wait complete. 2024-09-21T06:13:39.309 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-21T06:13:39.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.477+0000 7f0775e00640 1 -- 172.21.15.50:0/4202746926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07701527b0 msgr2=0x7f0770074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.477+0000 7f0775e00640 1 --2- 172.21.15.50:0/4202746926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07701527b0 0x7f0770074e30 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f0770072e30 tx=0x7f075801c800 comp rx=0 tx=0).stop 2024-09-21T06:13:39.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.477+0000 7f0775e00640 1 -- 172.21.15.50:0/4202746926 shutdown_connections 2024-09-21T06:13:39.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.477+0000 7f0775e00640 1 --2- 172.21.15.50:0/4202746926 >> v2:172.21.15.88:3300/0 conn(0x7f07700b4de0 0x7f07700b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.477+0000 7f0775e00640 1 --2- 172.21.15.50:0/4202746926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07701527b0 0x7f0770074e30 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.477+0000 7f0775e00640 1 --2- 172.21.15.50:0/4202746926 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07700b5c70 0x7f077015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.477+0000 7f0775e00640 1 -- 172.21.15.50:0/4202746926 >> 172.21.15.50:0/4202746926 conn(0x7f07700b2040 msgr2=0x7f07700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:39.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.477+0000 7f0775e00640 1 -- 172.21.15.50:0/4202746926 shutdown_connections 2024-09-21T06:13:39.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.477+0000 7f0775e00640 1 -- 172.21.15.50:0/4202746926 wait complete. 2024-09-21T06:13:39.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.477+0000 7f0775e00640 1 Processor -- start 2024-09-21T06:13:39.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0775e00640 1 -- start start 2024-09-21T06:13:39.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0775e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07700b4de0 0x7f0770157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0775e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07700b5c70 0x7f0770157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0775e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07701527b0 0x7f07700ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0775e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f077015f700 con 0x7f07700b4de0 2024-09-21T06:13:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0775e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f077015f580 con 0x7f07700b5c70 2024-09-21T06:13:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0775e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f077015f880 con 0x7f07701527b0 2024-09-21T06:13:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f076ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07700b4de0 0x7f0770157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f076ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07700b4de0 0x7f0770157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55882/0 (socket says 172.21.15.50:55882) 2024-09-21T06:13:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f076ea00640 1 -- 172.21.15.50:0/1019221685 learned_addr learned my addr 172.21.15.50:0/1019221685 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0767e00640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07700b5c70 0x7f0770157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f076f400640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07701527b0 0x7f07700ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f076ea00640 1 -- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07701527b0 msgr2=0x7f07700ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f076ea00640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07701527b0 0x7f07700ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f076ea00640 1 -- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07700b5c70 msgr2=0x7f0770157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f076ea00640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07700b5c70 0x7f0770157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f076ea00640 1 -- 172.21.15.50:0/1019221685 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0770145e50 con 0x7f07700b4de0 2024-09-21T06:13:39.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f076ea00640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07700b4de0 0x7f0770157540 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f076000d7b0 tx=0x7f076000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:39.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0767400640 1 -- 172.21.15.50:0/1019221685 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760017070 con 0x7f07700b4de0 2024-09-21T06:13:39.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0767400640 1 -- 172.21.15.50:0/1019221685 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0760004330 con 0x7f07700b4de0 2024-09-21T06:13:39.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0767400640 1 -- 172.21.15.50:0/1019221685 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760005280 con 0x7f07700b4de0 2024-09-21T06:13:39.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0775e00640 1 -- 172.21.15.50:0/1019221685 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077014e240 con 0x7f07700b4de0 2024-09-21T06:13:39.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.481+0000 7f0775e00640 1 -- 172.21.15.50:0/1019221685 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0770158480 con 0x7f07700b4de0 2024-09-21T06:13:39.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.485+0000 7f0775e00640 1 -- 172.21.15.50:0/1019221685 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0770157fc0 con 0x7f07700b4de0 2024-09-21T06:13:39.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.485+0000 7f0767400640 1 -- 172.21.15.50:0/1019221685 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 61) ==== 100804+0+0 (secure 0 0 0) 0x7f0760005420 con 0x7f07700b4de0 2024-09-21T06:13:39.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.485+0000 7f0767400640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f074c075960 0x7f074c077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.485+0000 7f0767400640 1 -- 172.21.15.50:0/1019221685 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f0760093910 con 0x7f07700b4de0 2024-09-21T06:13:39.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.485+0000 7f0767e00640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f074c075960 0x7f074c077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.485+0000 7f0767e00640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f074c075960 0x7f074c077e70 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f07580af000 tx=0x7f07580a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:39.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.489+0000 7f0767400640 1 -- 172.21.15.50:0/1019221685 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0770157fc0 con 0x7f07700b4de0 2024-09-21T06:13:39.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.693+0000 7f0775e00640 1 -- 172.21.15.50:0/1019221685 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f077014e520 con 0x7f074c075960 2024-09-21T06:13:39.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.729+0000 7f0767400640 1 -- 172.21.15.50:0/1019221685 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f07580b3040 con 0x7f074c075960 2024-09-21T06:13:39.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.733+0000 7f0775e00640 1 -- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f074c075960 msgr2=0x7f074c077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.733+0000 7f0775e00640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f074c075960 0x7f074c077e70 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f07580af000 tx=0x7f07580a7040 comp rx=0 tx=0).stop 2024-09-21T06:13:39.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.733+0000 7f0775e00640 1 -- 172.21.15.50:0/1019221685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07700b4de0 msgr2=0x7f0770157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.733+0000 7f0775e00640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07700b4de0 0x7f0770157540 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f076000d7b0 tx=0x7f076000dbb0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.733+0000 7f0775e00640 1 -- 172.21.15.50:0/1019221685 shutdown_connections 2024-09-21T06:13:39.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.733+0000 7f0775e00640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f07701527b0 0x7f07700ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.733+0000 7f0775e00640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07700b5c70 0x7f0770157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.733+0000 7f0775e00640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f074c075960 0x7f074c077e70 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.733+0000 7f0775e00640 1 --2- 172.21.15.50:0/1019221685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07700b4de0 0x7f0770157540 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.733+0000 7f0775e00640 1 -- 172.21.15.50:0/1019221685 >> 172.21.15.50:0/1019221685 conn(0x7f07700b2040 msgr2=0x7f07700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:39.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.733+0000 7f0775e00640 1 -- 172.21.15.50:0/1019221685 shutdown_connections 2024-09-21T06:13:39.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.733+0000 7f0775e00640 1 -- 172.21.15.50:0/1019221685 wait complete. 2024-09-21T06:13:39.749 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone90034 2024-09-21T06:13:39.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.921+0000 7f8a90200640 1 -- 172.21.15.50:0/3312612270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a70004810 msgr2=0x7f8a70004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.921+0000 7f8a90200640 1 --2- 172.21.15.50:0/3312612270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a70004810 0x7f8a70004c20 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009b60 tx=0x7f8a7401d270 comp rx=0 tx=0).stop 2024-09-21T06:13:39.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.921+0000 7f8a90200640 1 -- 172.21.15.50:0/3312612270 shutdown_connections 2024-09-21T06:13:39.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.921+0000 7f8a90200640 1 --2- 172.21.15.50:0/3312612270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a70004810 0x7f8a70004c20 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.921+0000 7f8a90200640 1 -- 172.21.15.50:0/3312612270 >> 172.21.15.50:0/3312612270 conn(0x7f8a880708c0 msgr2=0x7f8a88070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:39.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.921+0000 7f8a90200640 1 -- 172.21.15.50:0/3312612270 shutdown_connections 2024-09-21T06:13:39.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.921+0000 7f8a90200640 1 -- 172.21.15.50:0/3312612270 wait complete. 2024-09-21T06:13:39.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.921+0000 7f8a90200640 1 Processor -- start 2024-09-21T06:13:39.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.921+0000 7f8a90200640 1 -- start start 2024-09-21T06:13:39.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a90200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a70004810 0x7f8a881f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a90200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a8814da50 0x7f8a88155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a90200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a88153f80 0x7f8a88154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a90200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8815d920 con 0x7f8a88153f80 2024-09-21T06:13:39.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a90200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8815d7a0 con 0x7f8a70004810 2024-09-21T06:13:39.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a90200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8a88071c00 con 0x7f8a8814da50 2024-09-21T06:13:39.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a8e200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a88153f80 0x7f8a88154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a8e200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a88153f80 0x7f8a88154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55904/0 (socket says 172.21.15.50:55904) 2024-09-21T06:13:39.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a8e200640 1 -- 172.21.15.50:0/577483818 learned_addr learned my addr 172.21.15.50:0/577483818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:39.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a8ce00640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a8814da50 0x7f8a88155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a8d800640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a70004810 0x7f8a881f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a8e200640 1 -- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a8814da50 msgr2=0x7f8a88155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a8e200640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a8814da50 0x7f8a88155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a8e200640 1 -- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a70004810 msgr2=0x7f8a881f9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a8e200640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a70004810 0x7f8a881f9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a8e200640 1 -- 172.21.15.50:0/577483818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a880b10f0 con 0x7f8a88153f80 2024-09-21T06:13:39.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a8e200640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a88153f80 0x7f8a88154390 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f8a700057f0 tx=0x7f8a800064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:39.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a85600640 1 -- 172.21.15.50:0/577483818 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8000f960 con 0x7f8a88153f80 2024-09-21T06:13:39.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a85600640 1 -- 172.21.15.50:0/577483818 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a80018070 con 0x7f8a88153f80 2024-09-21T06:13:39.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a85600640 1 -- 172.21.15.50:0/577483818 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8000fdc0 con 0x7f8a88153f80 2024-09-21T06:13:39.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a90200640 1 -- 172.21.15.50:0/577483818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a880b13d0 con 0x7f8a88153f80 2024-09-21T06:13:39.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a90200640 1 -- 172.21.15.50:0/577483818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a881f93a0 con 0x7f8a88153f80 2024-09-21T06:13:39.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.925+0000 7f8a90200640 1 -- 172.21.15.50:0/577483818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a880b0e60 con 0x7f8a88153f80 2024-09-21T06:13:39.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.929+0000 7f8a85600640 1 -- 172.21.15.50:0/577483818 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f8a8001d460 con 0x7f8a88153f80 2024-09-21T06:13:39.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.929+0000 7f8a85600640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a68075a20 0x7f8a68077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.929+0000 7f8a85600640 1 -- 172.21.15.50:0/577483818 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f8a80093aa0 con 0x7f8a88153f80 2024-09-21T06:13:39.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.929+0000 7f8a8d800640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a68075a20 0x7f8a68077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.929+0000 7f8a8d800640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a68075a20 0x7f8a68077f30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c000f50 tx=0x7f8a7c000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:39.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:39.933+0000 7f8a85600640 1 -- 172.21.15.50:0/577483818 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8a80060000 con 0x7f8a88153f80 2024-09-21T06:13:40.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.141+0000 7f8a90200640 1 -- 172.21.15.50:0/577483818 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone90034", "target": ["mon-mgr", ""]}) -- 0x7f8a8806b540 con 0x7f8a68075a20 2024-09-21T06:13:40.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.161+0000 7f8a85600640 1 -- 172.21.15.50:0/577483818 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8a7c0064b0 con 0x7f8a68075a20 2024-09-21T06:13:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.165+0000 7f8a90200640 1 -- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a68075a20 msgr2=0x7f8a68077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.165+0000 7f8a90200640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a68075a20 0x7f8a68077f30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c000f50 tx=0x7f8a7c000ee0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.165+0000 7f8a90200640 1 -- 172.21.15.50:0/577483818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a88153f80 msgr2=0x7f8a88154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.165+0000 7f8a90200640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a88153f80 0x7f8a88154390 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f8a700057f0 tx=0x7f8a800064f0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.165+0000 7f8a90200640 1 -- 172.21.15.50:0/577483818 shutdown_connections 2024-09-21T06:13:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.165+0000 7f8a90200640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f8a68075a20 0x7f8a68077f30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.165+0000 7f8a90200640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a88153f80 0x7f8a88154390 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.165+0000 7f8a90200640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f8a8814da50 0x7f8a88155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.165+0000 7f8a90200640 1 --2- 172.21.15.50:0/577483818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a70004810 0x7f8a881f9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.165+0000 7f8a90200640 1 -- 172.21.15.50:0/577483818 >> 172.21.15.50:0/577483818 conn(0x7f8a880708c0 msgr2=0x7f8a88070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:40.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.165+0000 7f8a90200640 1 -- 172.21.15.50:0/577483818 shutdown_connections 2024-09-21T06:13:40.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.165+0000 7f8a90200640 1 -- 172.21.15.50:0/577483818 wait complete. 2024-09-21T06:13:40.181 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone74693 2024-09-21T06:13:40.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.349+0000 7fa005800640 1 -- 172.21.15.50:0/4019637292 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0000b5c70 msgr2=0x7fa00015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:40.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.349+0000 7fa005800640 1 --2- 172.21.15.50:0/4019637292 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0000b5c70 0x7fa00015bb60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009840 tx=0x7f9ff001c680 comp rx=0 tx=0).stop 2024-09-21T06:13:40.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.349+0000 7fa005800640 1 -- 172.21.15.50:0/4019637292 shutdown_connections 2024-09-21T06:13:40.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.349+0000 7fa005800640 1 --2- 172.21.15.50:0/4019637292 >> v2:172.21.15.88:3300/0 conn(0x7fa0001527b0 0x7fa000074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.349+0000 7fa005800640 1 --2- 172.21.15.50:0/4019637292 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0000b5c70 0x7fa00015bb60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.349+0000 7fa005800640 1 -- 172.21.15.50:0/4019637292 >> 172.21.15.50:0/4019637292 conn(0x7fa0000b2040 msgr2=0x7fa0000b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:40.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.349+0000 7fa005800640 1 -- 172.21.15.50:0/4019637292 shutdown_connections 2024-09-21T06:13:40.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.349+0000 7fa005800640 1 -- 172.21.15.50:0/4019637292 wait complete. 2024-09-21T06:13:40.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.349+0000 7fa005800640 1 Processor -- start 2024-09-21T06:13:40.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7fa005800640 1 -- start start 2024-09-21T06:13:40.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7fa005800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0000b4de0 0x7fa000137600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:40.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7fa005800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0000b5c70 0x7fa0001f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:40.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7fa005800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0001527b0 0x7fa0001f9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:40.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7fa005800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa000168c50 con 0x7fa0000b5c70 2024-09-21T06:13:40.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7fa005800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa000168ad0 con 0x7fa0001527b0 2024-09-21T06:13:40.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7fa005800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa000168dd0 con 0x7fa0000b4de0 2024-09-21T06:13:40.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ff7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0000b5c70 0x7fa0001f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:40.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ff7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0000b5c70 0x7fa0001f7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55908/0 (socket says 172.21.15.50:55908) 2024-09-21T06:13:40.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ff7e00640 1 -- 172.21.15.50:0/228512814 learned_addr learned my addr 172.21.15.50:0/228512814 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:40.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ffea00640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0000b4de0 0x7fa000137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:40.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9fff400640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0001527b0 0x7fa0001f9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:40.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ff7e00640 1 -- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0000b4de0 msgr2=0x7fa000137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:40.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ff7e00640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0000b4de0 0x7fa000137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ff7e00640 1 -- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0001527b0 msgr2=0x7fa0001f9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:40.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ff7e00640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0001527b0 0x7fa0001f9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ff7e00640 1 -- 172.21.15.50:0/228512814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0001fa960 con 0x7fa0000b5c70 2024-09-21T06:13:40.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ff7e00640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0000b5c70 0x7fa0001f7380 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800e770 tx=0x7f9fe800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:40.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ff6a00640 1 -- 172.21.15.50:0/228512814 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800ccc0 con 0x7fa0000b5c70 2024-09-21T06:13:40.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7fa005800640 1 -- 172.21.15.50:0/228512814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0001fac40 con 0x7fa0000b5c70 2024-09-21T06:13:40.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ff6a00640 1 -- 172.21.15.50:0/228512814 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fe800ee80 con 0x7fa0000b5c70 2024-09-21T06:13:40.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7f9ff6a00640 1 -- 172.21.15.50:0/228512814 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8010670 con 0x7fa0000b5c70 2024-09-21T06:13:40.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.353+0000 7fa005800640 1 -- 172.21.15.50:0/228512814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0001f6b80 con 0x7fa0000b5c70 2024-09-21T06:13:40.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.357+0000 7fa005800640 1 -- 172.21.15.50:0/228512814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa00015bb60 con 0x7fa0000b5c70 2024-09-21T06:13:40.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.357+0000 7f9ff6a00640 1 -- 172.21.15.50:0/228512814 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f9fe8010810 con 0x7fa0000b5c70 2024-09-21T06:13:40.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.357+0000 7f9ff6a00640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9fdc0759f0 0x7f9fdc077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:40.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.357+0000 7f9ff6a00640 1 -- 172.21.15.50:0/228512814 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f9fe8093eb0 con 0x7fa0000b5c70 2024-09-21T06:13:40.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.357+0000 7f9ffea00640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9fdc0759f0 0x7f9fdc077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:40.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.357+0000 7f9ffea00640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9fdc0759f0 0x7f9fdc077f00 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00af000 tx=0x7f9ff001aec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:40.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.361+0000 7f9ff6a00640 1 -- 172.21.15.50:0/228512814 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9fe80602f0 con 0x7fa0000b5c70 2024-09-21T06:13:40.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.565+0000 7fa005800640 1 -- 172.21.15.50:0/228512814 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone74693", "target": ["mon-mgr", ""]}) -- 0x7fa00006b540 con 0x7f9fdc0759f0 2024-09-21T06:13:40.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.585+0000 7f9ff6a00640 1 -- 172.21.15.50:0/228512814 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ff0004130 con 0x7f9fdc0759f0 2024-09-21T06:13:40.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.589+0000 7fa005800640 1 -- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9fdc0759f0 msgr2=0x7f9fdc077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:40.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.589+0000 7fa005800640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9fdc0759f0 0x7f9fdc077f00 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00af000 tx=0x7f9ff001aec0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.589+0000 7fa005800640 1 -- 172.21.15.50:0/228512814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0000b5c70 msgr2=0x7fa0001f7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:40.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.589+0000 7fa005800640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0000b5c70 0x7fa0001f7380 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800e770 tx=0x7f9fe800ec30 comp rx=0 tx=0).stop 2024-09-21T06:13:40.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.589+0000 7fa005800640 1 -- 172.21.15.50:0/228512814 shutdown_connections 2024-09-21T06:13:40.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.589+0000 7fa005800640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0001527b0 0x7fa0001f9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.589+0000 7fa005800640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9fdc0759f0 0x7f9fdc077f00 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.589+0000 7fa005800640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0000b5c70 0x7fa0001f7380 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.589+0000 7fa005800640 1 --2- 172.21.15.50:0/228512814 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa0000b4de0 0x7fa000137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.589+0000 7fa005800640 1 -- 172.21.15.50:0/228512814 >> 172.21.15.50:0/228512814 conn(0x7fa0000b2040 msgr2=0x7fa0000b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:40.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.589+0000 7fa005800640 1 -- 172.21.15.50:0/228512814 shutdown_connections 2024-09-21T06:13:40.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.589+0000 7fa005800640 1 -- 172.21.15.50:0/228512814 wait complete. 2024-09-21T06:13:40.605 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99400 2024-09-21T06:13:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.773+0000 7f5d54a00640 1 -- 172.21.15.50:0/1974536456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d501527b0 msgr2=0x7f5d50074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.773+0000 7f5d54a00640 1 --2- 172.21.15.50:0/1974536456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d501527b0 0x7f5d50074e30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f5d50071cb0 tx=0x7f5d3801c800 comp rx=0 tx=0).stop 2024-09-21T06:13:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.773+0000 7f5d54a00640 1 -- 172.21.15.50:0/1974536456 shutdown_connections 2024-09-21T06:13:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.773+0000 7f5d54a00640 1 --2- 172.21.15.50:0/1974536456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d501527b0 0x7f5d50074e30 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.773+0000 7f5d54a00640 1 --2- 172.21.15.50:0/1974536456 >> v2:172.21.15.88:3300/0 conn(0x7f5d500b5c70 0x7f5d5015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.773+0000 7f5d54a00640 1 -- 172.21.15.50:0/1974536456 >> 172.21.15.50:0/1974536456 conn(0x7f5d500b2040 msgr2=0x7f5d500b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.773+0000 7f5d54a00640 1 -- 172.21.15.50:0/1974536456 shutdown_connections 2024-09-21T06:13:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.773+0000 7f5d54a00640 1 -- 172.21.15.50:0/1974536456 wait complete. 2024-09-21T06:13:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d54a00640 1 Processor -- start 2024-09-21T06:13:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d54a00640 1 -- start start 2024-09-21T06:13:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d54a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d500b4de0 0x7f5d5014f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d54a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5d500b5c70 0x7f5d501f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d54a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d501527b0 0x7f5d5013fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d54a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d50168b50 con 0x7f5d500b4de0 2024-09-21T06:13:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d54a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501689d0 con 0x7f5d501527b0 2024-09-21T06:13:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d54a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f5d50168cd0 con 0x7f5d500b5c70 2024-09-21T06:13:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d4de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d500b4de0 0x7f5d5014f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d4de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d500b4de0 0x7f5d5014f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55918/0 (socket says 172.21.15.50:55918) 2024-09-21T06:13:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d4de00640 1 -- 172.21.15.50:0/951860407 learned_addr learned my addr 172.21.15.50:0/951860407 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d47e00640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5d500b5c70 0x7f5d501f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d4de00640 1 -- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5d500b5c70 msgr2=0x7f5d501f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d4de00640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5d500b5c70 0x7f5d501f65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d4e800640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d501527b0 0x7f5d5013fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d4de00640 1 -- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d501527b0 msgr2=0x7f5d5013fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d4de00640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d501527b0 0x7f5d5013fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d4de00640 1 -- 172.21.15.50:0/951860407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d501f7610 con 0x7f5d500b4de0 2024-09-21T06:13:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d4e800640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d501527b0 0x7f5d5013fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d4de00640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d500b4de0 0x7f5d5014f510 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000c9f0 tx=0x7f5d4000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d46000640 1 -- 172.21.15.50:0/951860407 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40007bb0 con 0x7f5d500b4de0 2024-09-21T06:13:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d46000640 1 -- 172.21.15.50:0/951860407 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d40007e40 con 0x7f5d500b4de0 2024-09-21T06:13:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d54a00640 1 -- 172.21.15.50:0/951860407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d501f6b20 con 0x7f5d500b4de0 2024-09-21T06:13:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d46000640 1 -- 172.21.15.50:0/951860407 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000f690 con 0x7f5d500b4de0 2024-09-21T06:13:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.777+0000 7f5d54a00640 1 -- 172.21.15.50:0/951860407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d501f7330 con 0x7f5d500b4de0 2024-09-21T06:13:40.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.781+0000 7f5d54a00640 1 -- 172.21.15.50:0/951860407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5015bb60 con 0x7f5d500b4de0 2024-09-21T06:13:40.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.781+0000 7f5d46000640 1 -- 172.21.15.50:0/951860407 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f5d4000f830 con 0x7f5d500b4de0 2024-09-21T06:13:40.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.781+0000 7f5d46000640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5d2c075a20 0x7f5d2c077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:40.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.781+0000 7f5d46000640 1 -- 172.21.15.50:0/951860407 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7f5d40093d90 con 0x7f5d500b4de0 2024-09-21T06:13:40.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.781+0000 7f5d47e00640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5d2c075a20 0x7f5d2c077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:40.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.785+0000 7f5d47e00640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5d2c075a20 0x7f5d2c077f30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5d380af000 tx=0x7f5d3801b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:40.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:40.785+0000 7f5d46000640 1 -- 172.21.15.50:0/951860407 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5d40014030 con 0x7f5d500b4de0 2024-09-21T06:13:41.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.121+0000 7f5d54a00640 1 -- 172.21.15.50:0/951860407 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99400", "target": ["mon-mgr", ""]}) -- 0x7f5d5006b540 con 0x7f5d2c075a20 2024-09-21T06:13:41.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d46000640 1 -- 172.21.15.50:0/951860407 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5d38004230 con 0x7f5d2c075a20 2024-09-21T06:13:41.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d54a00640 1 -- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5d2c075a20 msgr2=0x7f5d2c077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d54a00640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5d2c075a20 0x7f5d2c077f30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5d380af000 tx=0x7f5d3801b040 comp rx=0 tx=0).stop 2024-09-21T06:13:41.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d54a00640 1 -- 172.21.15.50:0/951860407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d500b4de0 msgr2=0x7f5d5014f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d54a00640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d500b4de0 0x7f5d5014f510 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000c9f0 tx=0x7f5d4000ceb0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d54a00640 1 -- 172.21.15.50:0/951860407 shutdown_connections 2024-09-21T06:13:41.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d54a00640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d501527b0 0x7f5d5013fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d54a00640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f5d500b5c70 0x7f5d501f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d54a00640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5d2c075a20 0x7f5d2c077f30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d54a00640 1 --2- 172.21.15.50:0/951860407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d500b4de0 0x7f5d5014f510 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d54a00640 1 -- 172.21.15.50:0/951860407 >> 172.21.15.50:0/951860407 conn(0x7f5d500b2040 msgr2=0x7f5d500cecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:41.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d54a00640 1 -- 172.21.15.50:0/951860407 shutdown_connections 2024-09-21T06:13:41.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.137+0000 7f5d54a00640 1 -- 172.21.15.50:0/951860407 wait complete. 2024-09-21T06:13:41.154 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone64946 2024-09-21T06:13:41.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.325+0000 7fe88ac00640 1 -- 172.21.15.50:0/1662840446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8841527b0 msgr2=0x7fe88415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.325+0000 7fe88ac00640 1 --2- 172.21.15.50:0/1662840446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8841527b0 0x7fe88415bb60 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe884074b20 tx=0x7fe86c01c760 comp rx=0 tx=0).stop 2024-09-21T06:13:41.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.325+0000 7fe88ac00640 1 -- 172.21.15.50:0/1662840446 shutdown_connections 2024-09-21T06:13:41.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.325+0000 7fe88ac00640 1 --2- 172.21.15.50:0/1662840446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8841527b0 0x7fe88415bb60 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.325+0000 7fe88ac00640 1 -- 172.21.15.50:0/1662840446 >> 172.21.15.50:0/1662840446 conn(0x7fe8840b2040 msgr2=0x7fe8840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:41.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.325+0000 7fe88ac00640 1 -- 172.21.15.50:0/1662840446 shutdown_connections 2024-09-21T06:13:41.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.325+0000 7fe88ac00640 1 -- 172.21.15.50:0/1662840446 wait complete. 2024-09-21T06:13:41.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.325+0000 7fe88ac00640 1 Processor -- start 2024-09-21T06:13:41.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.325+0000 7fe88ac00640 1 -- start start 2024-09-21T06:13:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe88ac00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8840b4de0 0x7fe8840cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe88ac00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8840b5c70 0x7fe8841f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe88ac00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8841527b0 0x7fe8840cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe88ac00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe884169bd0 con 0x7fe8841527b0 2024-09-21T06:13:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe88ac00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe884169a50 con 0x7fe8840b4de0 2024-09-21T06:13:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe88ac00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe884169d50 con 0x7fe8840b5c70 2024-09-21T06:13:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe888c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8841527b0 0x7fe8840cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe888c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8841527b0 0x7fe8840cbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55930/0 (socket says 172.21.15.50:55930) 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe888c00640 1 -- 172.21.15.50:0/1022279378 learned_addr learned my addr 172.21.15.50:0/1022279378 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe883e00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8840b4de0 0x7fe8840cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe87be00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8840b5c70 0x7fe8841f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe888c00640 1 -- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8840b5c70 msgr2=0x7fe8841f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe888c00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8840b5c70 0x7fe8841f8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe888c00640 1 -- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8840b4de0 msgr2=0x7fe8840cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe888c00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8840b4de0 0x7fe8840cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe888c00640 1 -- 172.21.15.50:0/1022279378 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8840cf110 con 0x7fe8841527b0 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe888c00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8841527b0 0x7fe8840cbd10 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c730 tx=0x7fe87c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe880c00640 1 -- 172.21.15.50:0/1022279378 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c003b80 con 0x7fe8841527b0 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe880c00640 1 -- 172.21.15.50:0/1022279378 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe87c003d40 con 0x7fe8841527b0 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe88ac00640 1 -- 172.21.15.50:0/1022279378 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8841f9730 con 0x7fe8841527b0 2024-09-21T06:13:41.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe880c00640 1 -- 172.21.15.50:0/1022279378 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0052d0 con 0x7fe8841527b0 2024-09-21T06:13:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe88ac00640 1 -- 172.21.15.50:0/1022279378 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8840ce3d0 con 0x7fe8841527b0 2024-09-21T06:13:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.329+0000 7fe88ac00640 1 -- 172.21.15.50:0/1022279378 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8840ce1c0 con 0x7fe8841527b0 2024-09-21T06:13:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.333+0000 7fe880c00640 1 -- 172.21.15.50:0/1022279378 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fe87c005680 con 0x7fe8841527b0 2024-09-21T06:13:41.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.333+0000 7fe880c00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8600759f0 0x7fe860077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:41.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.333+0000 7fe880c00640 1 -- 172.21.15.50:0/1022279378 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7fe87c093300 con 0x7fe8841527b0 2024-09-21T06:13:41.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.333+0000 7fe883e00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8600759f0 0x7fe860077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:41.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.333+0000 7fe883e00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8600759f0 0x7fe860077f00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe8740098e0 tx=0x7fe874009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:41.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.337+0000 7fe880c00640 1 -- 172.21.15.50:0/1022279378 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe87c05f860 con 0x7fe8841527b0 2024-09-21T06:13:41.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.541+0000 7fe88ac00640 1 -- 172.21.15.50:0/1022279378 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone64946", "target": ["mon-mgr", ""]}) -- 0x7fe88406b540 con 0x7fe8600759f0 2024-09-21T06:13:41.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.565+0000 7fe880c00640 1 -- 172.21.15.50:0/1022279378 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe874009530 con 0x7fe8600759f0 2024-09-21T06:13:41.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.569+0000 7fe88ac00640 1 -- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8600759f0 msgr2=0x7fe860077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.569+0000 7fe88ac00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8600759f0 0x7fe860077f00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe8740098e0 tx=0x7fe874009060 comp rx=0 tx=0).stop 2024-09-21T06:13:41.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.569+0000 7fe88ac00640 1 -- 172.21.15.50:0/1022279378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8841527b0 msgr2=0x7fe8840cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.569+0000 7fe88ac00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8841527b0 0x7fe8840cbd10 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c730 tx=0x7fe87c00cbf0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.569+0000 7fe88ac00640 1 -- 172.21.15.50:0/1022279378 shutdown_connections 2024-09-21T06:13:41.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.569+0000 7fe88ac00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe8600759f0 0x7fe860077f00 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.569+0000 7fe88ac00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8841527b0 0x7fe8840cbd10 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.569+0000 7fe88ac00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe8840b5c70 0x7fe8841f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.569+0000 7fe88ac00640 1 --2- 172.21.15.50:0/1022279378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8840b4de0 0x7fe8840cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.569+0000 7fe88ac00640 1 -- 172.21.15.50:0/1022279378 >> 172.21.15.50:0/1022279378 conn(0x7fe8840b2040 msgr2=0x7fe884151a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:41.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.569+0000 7fe88ac00640 1 -- 172.21.15.50:0/1022279378 shutdown_connections 2024-09-21T06:13:41.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.569+0000 7fe88ac00640 1 -- 172.21.15.50:0/1022279378 wait complete. 2024-09-21T06:13:41.585 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone44825 2024-09-21T06:13:41.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.753+0000 7fbe35a00640 1 -- 172.21.15.50:0/4220424740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe300b4de0 msgr2=0x7fbe300b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.753+0000 7fbe35a00640 1 --2- 172.21.15.50:0/4220424740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe300b4de0 0x7fbe300b51f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fbe30070710 tx=0x7fbe2801ccc0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.753+0000 7fbe35a00640 1 -- 172.21.15.50:0/4220424740 shutdown_connections 2024-09-21T06:13:41.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.753+0000 7fbe35a00640 1 --2- 172.21.15.50:0/4220424740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe300b4de0 0x7fbe300b51f0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.753+0000 7fbe35a00640 1 --2- 172.21.15.50:0/4220424740 >> v2:172.21.15.88:3300/0 conn(0x7fbe301527b0 0x7fbe3015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.753+0000 7fbe35a00640 1 -- 172.21.15.50:0/4220424740 >> 172.21.15.50:0/4220424740 conn(0x7fbe300b2040 msgr2=0x7fbe300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:41.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.753+0000 7fbe35a00640 1 -- 172.21.15.50:0/4220424740 shutdown_connections 2024-09-21T06:13:41.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.753+0000 7fbe35a00640 1 -- 172.21.15.50:0/4220424740 wait complete. 2024-09-21T06:13:41.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe35a00640 1 Processor -- start 2024-09-21T06:13:41.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe35a00640 1 -- start start 2024-09-21T06:13:41.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe35a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe300b4de0 0x7fbe30155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:41.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe35a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbe300b5c70 0x7fbe301581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:41.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe35a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe301527b0 0x7fbe30158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:41.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe35a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbe30168920 con 0x7fbe301527b0 2024-09-21T06:13:41.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe35a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301687a0 con 0x7fbe300b4de0 2024-09-21T06:13:41.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe35a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fbe30168aa0 con 0x7fbe300b5c70 2024-09-21T06:13:41.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe2f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe301527b0 0x7fbe30158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:41.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe2f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe301527b0 0x7fbe30158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55942/0 (socket says 172.21.15.50:55942) 2024-09-21T06:13:41.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe2f400640 1 -- 172.21.15.50:0/2578641179 learned_addr learned my addr 172.21.15.50:0/2578641179 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:41.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe2ea00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe300b4de0 0x7fbe30155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:41.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe27e00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbe300b5c70 0x7fbe301581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:41.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe2f400640 1 -- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbe300b5c70 msgr2=0x7fbe301581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe2f400640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbe300b5c70 0x7fbe301581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe2f400640 1 -- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe300b4de0 msgr2=0x7fbe30155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe2f400640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe300b4de0 0x7fbe30155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe2f400640 1 -- 172.21.15.50:0/2578641179 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe30142480 con 0x7fbe301527b0 2024-09-21T06:13:41.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe2ea00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe300b4de0 0x7fbe30155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:41.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe27e00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbe300b5c70 0x7fbe301581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:41.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe2f400640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe301527b0 0x7fbe30158710 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009bd0 tx=0x7fbe280092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe26a00640 1 -- 172.21.15.50:0/2578641179 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280bb070 con 0x7fbe301527b0 2024-09-21T06:13:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe26a00640 1 -- 172.21.15.50:0/2578641179 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe28004d40 con 0x7fbe301527b0 2024-09-21T06:13:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe35a00640 1 -- 172.21.15.50:0/2578641179 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe301416c0 con 0x7fbe301527b0 2024-09-21T06:13:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe26a00640 1 -- 172.21.15.50:0/2578641179 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280b2040 con 0x7fbe301527b0 2024-09-21T06:13:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.757+0000 7fbe35a00640 1 -- 172.21.15.50:0/2578641179 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe30141c80 con 0x7fbe301527b0 2024-09-21T06:13:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.761+0000 7fbe35a00640 1 -- 172.21.15.50:0/2578641179 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe300b51f0 con 0x7fbe301527b0 2024-09-21T06:13:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.761+0000 7fbe26a00640 1 -- 172.21.15.50:0/2578641179 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fbe280be070 con 0x7fbe301527b0 2024-09-21T06:13:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.761+0000 7fbe26a00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbe0c075a20 0x7fbe0c077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.761+0000 7fbe26a00640 1 -- 172.21.15.50:0/2578641179 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5822+0+0 (secure 0 0 0) 0x7fbe2812d5e0 con 0x7fbe301527b0 2024-09-21T06:13:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.765+0000 7fbe2ea00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbe0c075a20 0x7fbe0c077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:41.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.765+0000 7fbe2ea00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbe0c075a20 0x7fbe0c077f30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbe200016b0 tx=0x7fbe20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:41.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:41.769+0000 7fbe26a00640 1 -- 172.21.15.50:0/2578641179 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbe28016970 con 0x7fbe301527b0 2024-09-21T06:13:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.133+0000 7fbe35a00640 1 -- 172.21.15.50:0/2578641179 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone44825", "target": ["mon-mgr", ""]}) -- 0x7fbe3006b540 con 0x7fbe0c075a20 2024-09-21T06:13:42.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.169+0000 7fbe26a00640 1 -- 172.21.15.50:0/2578641179 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbe20001c90 con 0x7fbe0c075a20 2024-09-21T06:13:42.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.173+0000 7fbe1fe00640 1 -- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbe0c075a20 msgr2=0x7fbe0c077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:42.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.173+0000 7fbe1fe00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbe0c075a20 0x7fbe0c077f30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbe200016b0 tx=0x7fbe20009040 comp rx=0 tx=0).stop 2024-09-21T06:13:42.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.173+0000 7fbe1fe00640 1 -- 172.21.15.50:0/2578641179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe301527b0 msgr2=0x7fbe30158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:42.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.173+0000 7fbe1fe00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe301527b0 0x7fbe30158710 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009bd0 tx=0x7fbe280092c0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.173+0000 7fbe1fe00640 1 -- 172.21.15.50:0/2578641179 shutdown_connections 2024-09-21T06:13:42.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.173+0000 7fbe1fe00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fbe0c075a20 0x7fbe0c077f30 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.173+0000 7fbe1fe00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe301527b0 0x7fbe30158710 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.173+0000 7fbe1fe00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fbe300b5c70 0x7fbe301581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.173+0000 7fbe1fe00640 1 --2- 172.21.15.50:0/2578641179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe300b4de0 0x7fbe30155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.173+0000 7fbe1fe00640 1 -- 172.21.15.50:0/2578641179 >> 172.21.15.50:0/2578641179 conn(0x7fbe300b2040 msgr2=0x7fbe3015a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:42.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.173+0000 7fbe1fe00640 1 -- 172.21.15.50:0/2578641179 shutdown_connections 2024-09-21T06:13:42.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.173+0000 7fbe1fe00640 1 -- 172.21.15.50:0/2578641179 wait complete. 2024-09-21T06:13:42.199 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone5031 2024-09-21T06:13:42.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.373+0000 7f4da5a00640 1 -- 172.21.15.50:0/3888868120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4da01527b0 msgr2=0x7f4da0074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:42.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.373+0000 7f4da5a00640 1 --2- 172.21.15.50:0/3888868120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4da01527b0 0x7f4da0074e30 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009900 tx=0x7f4d8801c820 comp rx=0 tx=0).stop 2024-09-21T06:13:42.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.373+0000 7f4da5a00640 1 -- 172.21.15.50:0/3888868120 shutdown_connections 2024-09-21T06:13:42.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.373+0000 7f4da5a00640 1 --2- 172.21.15.50:0/3888868120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4da01527b0 0x7f4da0074e30 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.373+0000 7f4da5a00640 1 --2- 172.21.15.50:0/3888868120 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4da00b5c70 0x7f4da015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.373+0000 7f4da5a00640 1 -- 172.21.15.50:0/3888868120 >> 172.21.15.50:0/3888868120 conn(0x7f4da00b2040 msgr2=0x7f4da00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:42.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.373+0000 7f4da5a00640 1 -- 172.21.15.50:0/3888868120 shutdown_connections 2024-09-21T06:13:42.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.373+0000 7f4da5a00640 1 -- 172.21.15.50:0/3888868120 wait complete. 2024-09-21T06:13:42.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.373+0000 7f4da5a00640 1 Processor -- start 2024-09-21T06:13:42.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.373+0000 7f4da5a00640 1 -- start start 2024-09-21T06:13:42.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4da5a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4da00b4de0 0x7f4da0157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:42.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4da5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4da00b5c70 0x7f4da0149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:42.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4da5a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4da01527b0 0x7f4da00b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4da5a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0168c50 con 0x7f4da00b5c70 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4da5a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0168ad0 con 0x7f4da01527b0 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4da5a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4da0168dd0 con 0x7f4da00b4de0 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d97e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4da00b5c70 0x7f4da0149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d97e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4da00b5c70 0x7f4da0149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55968/0 (socket says 172.21.15.50:55968) 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d97e00640 1 -- 172.21.15.50:0/1544747854 learned_addr learned my addr 172.21.15.50:0/1544747854 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d9f400640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4da01527b0 0x7f4da00b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d9ea00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4da00b4de0 0x7f4da0157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d97e00640 1 -- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4da00b4de0 msgr2=0x7f4da0157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d97e00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4da00b4de0 0x7f4da0157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d97e00640 1 -- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4da01527b0 msgr2=0x7f4da00b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d97e00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4da01527b0 0x7f4da00b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d97e00640 1 -- 172.21.15.50:0/1544747854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da00ce660 con 0x7f4da00b5c70 2024-09-21T06:13:42.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d9ea00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4da00b4de0 0x7f4da0157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:42.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d97e00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4da00b5c70 0x7f4da0149c00 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f4d880b0040 tx=0x7f4d880a5860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:42.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d96a00640 1 -- 172.21.15.50:0/1544747854 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d880b2040 con 0x7f4da00b5c70 2024-09-21T06:13:42.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4da5a00640 1 -- 172.21.15.50:0/1544747854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da00ce940 con 0x7f4da00b5c70 2024-09-21T06:13:42.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d96a00640 1 -- 172.21.15.50:0/1544747854 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d880044b0 con 0x7f4da00b5c70 2024-09-21T06:13:42.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4d96a00640 1 -- 172.21.15.50:0/1544747854 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88005290 con 0x7f4da00b5c70 2024-09-21T06:13:42.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4da5a00640 1 -- 172.21.15.50:0/1544747854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da00cef00 con 0x7f4da00b5c70 2024-09-21T06:13:42.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.377+0000 7f4da5a00640 1 -- 172.21.15.50:0/1544747854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da00ce3d0 con 0x7f4da00b5c70 2024-09-21T06:13:42.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.381+0000 7f4d96a00640 1 -- 172.21.15.50:0/1544747854 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f4d880be060 con 0x7f4da00b5c70 2024-09-21T06:13:42.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.381+0000 7f4d96a00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4d7c075a20 0x7f4d7c077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:42.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.381+0000 7f4d96a00640 1 -- 172.21.15.50:0/1544747854 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5878+0+0 (secure 0 0 0) 0x7f4d8812cf00 con 0x7f4da00b5c70 2024-09-21T06:13:42.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.381+0000 7f4d9ea00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4d7c075a20 0x7f4d7c077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:42.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.381+0000 7f4d9ea00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4d7c075a20 0x7f4d7c077f30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009b90 tx=0x7f4d900091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:42.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.385+0000 7f4d96a00640 1 -- 172.21.15.50:0/1544747854 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4da0075580 con 0x7f4da00b5c70 2024-09-21T06:13:42.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.585+0000 7f4da5a00640 1 -- 172.21.15.50:0/1544747854 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone5031", "target": ["mon-mgr", ""]}) -- 0x7f4da006b540 con 0x7f4d7c075a20 2024-09-21T06:13:42.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4d96a00640 1 -- 172.21.15.50:0/1544747854 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4d90011be0 con 0x7f4d7c075a20 2024-09-21T06:13:42.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4da5a00640 1 -- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4d7c075a20 msgr2=0x7f4d7c077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:42.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4da5a00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4d7c075a20 0x7f4d7c077f30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009b90 tx=0x7f4d900091e0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4da5a00640 1 -- 172.21.15.50:0/1544747854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4da00b5c70 msgr2=0x7f4da0149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:42.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4da5a00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4da00b5c70 0x7f4da0149c00 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f4d880b0040 tx=0x7f4d880a5860 comp rx=0 tx=0).stop 2024-09-21T06:13:42.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4da5a00640 1 -- 172.21.15.50:0/1544747854 shutdown_connections 2024-09-21T06:13:42.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4da5a00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4da01527b0 0x7f4da00b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4da5a00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4d7c075a20 0x7f4d7c077f30 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4da5a00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4da00b5c70 0x7f4da0149c00 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4da5a00640 1 --2- 172.21.15.50:0/1544747854 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4da00b4de0 0x7f4da0157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4da5a00640 1 -- 172.21.15.50:0/1544747854 >> 172.21.15.50:0/1544747854 conn(0x7f4da00b2040 msgr2=0x7f4da00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:42.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4da5a00640 1 -- 172.21.15.50:0/1544747854 shutdown_connections 2024-09-21T06:13:42.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.609+0000 7f4da5a00640 1 -- 172.21.15.50:0/1544747854 wait complete. 2024-09-21T06:13:42.626 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T06:13:42.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:13:42.682 INFO:teuthology.orchestra.run.smithi050.stdout:2 2024-09-21T06:13:42.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:13:42.689 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-21T06:13:42.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.901+0000 7f3b74200640 1 -- 172.21.15.50:0/2788259844 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b6c15f180 msgr2=0x7f3b6c15f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:42.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.901+0000 7f3b74200640 1 --2- 172.21.15.50:0/2788259844 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b6c15f180 0x7f3b6c15f590 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009c80 tx=0x7f3b6401cda0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.901+0000 7f3b74200640 1 -- 172.21.15.50:0/2788259844 shutdown_connections 2024-09-21T06:13:42.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.901+0000 7f3b74200640 1 --2- 172.21.15.50:0/2788259844 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b6c15f180 0x7f3b6c15f590 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.901+0000 7f3b74200640 1 --2- 172.21.15.50:0/2788259844 >> v2:172.21.15.88:3300/0 conn(0x7f3b6c155ad0 0x7f3b6c06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.901+0000 7f3b74200640 1 -- 172.21.15.50:0/2788259844 >> 172.21.15.50:0/2788259844 conn(0x7f3b6c0708c0 msgr2=0x7f3b6c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:42.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.905+0000 7f3b74200640 1 -- 172.21.15.50:0/2788259844 shutdown_connections 2024-09-21T06:13:42.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.905+0000 7f3b74200640 1 -- 172.21.15.50:0/2788259844 wait complete. 2024-09-21T06:13:42.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.905+0000 7f3b74200640 1 Processor -- start 2024-09-21T06:13:42.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.905+0000 7f3b74200640 1 -- start start 2024-09-21T06:13:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b74200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b6c0d29c0 0x7f3b6c157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b74200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b6c155ad0 0x7f3b6c13d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b74200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b6c15f180 0x7f3b6c1fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b74200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c16c4a0 con 0x7f3b6c15f180 2024-09-21T06:13:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b74200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c16c320 con 0x7f3b6c155ad0 2024-09-21T06:13:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b74200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3b6c16c620 con 0x7f3b6c0d29c0 2024-09-21T06:13:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b72200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b6c15f180 0x7f3b6c1fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b70e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b6c155ad0 0x7f3b6c13d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b72200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b6c15f180 0x7f3b6c1fcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55970/0 (socket says 172.21.15.50:55970) 2024-09-21T06:13:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b72200640 1 -- 172.21.15.50:0/2647932945 learned_addr learned my addr 172.21.15.50:0/2647932945 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:42.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b71800640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b6c0d29c0 0x7f3b6c157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:42.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b71800640 1 -- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b6c155ad0 msgr2=0x7f3b6c13d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:42.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b71800640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b6c155ad0 0x7f3b6c13d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b71800640 1 -- 172.21.15.50:0/2647932945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b6c15f180 msgr2=0x7f3b6c1fcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:42.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b71800640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b6c15f180 0x7f3b6c1fcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:42.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b71800640 1 -- 172.21.15.50:0/2647932945 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6c200950 con 0x7f3b6c0d29c0 2024-09-21T06:13:42.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.909+0000 7f3b72200640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b6c15f180 0x7f3b6c1fcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:42.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.913+0000 7f3b71800640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b6c0d29c0 0x7f3b6c157cc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000b410 tx=0x7f3b6000b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:42.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.913+0000 7f3b69600640 1 -- 172.21.15.50:0/2647932945 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60015020 con 0x7f3b6c0d29c0 2024-09-21T06:13:42.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.913+0000 7f3b69600640 1 -- 172.21.15.50:0/2647932945 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b60011940 con 0x7f3b6c0d29c0 2024-09-21T06:13:42.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.913+0000 7f3b69600640 1 -- 172.21.15.50:0/2647932945 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60011d90 con 0x7f3b6c0d29c0 2024-09-21T06:13:42.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.913+0000 7f3b74200640 1 -- 172.21.15.50:0/2647932945 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6c158200 con 0x7f3b6c0d29c0 2024-09-21T06:13:42.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.913+0000 7f3b74200640 1 -- 172.21.15.50:0/2647932945 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6c13e1d0 con 0x7f3b6c0d29c0 2024-09-21T06:13:42.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.913+0000 7f3b74200640 1 -- 172.21.15.50:0/2647932945 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b6c06e520 con 0x7f3b6c0d29c0 2024-09-21T06:13:42.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.917+0000 7f3b69600640 1 -- 172.21.15.50:0/2647932945 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f3b60018020 con 0x7f3b6c0d29c0 2024-09-21T06:13:42.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.921+0000 7f3b69600640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b4c0759f0 0x7f3b4c077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:42.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.921+0000 7f3b69600640 1 -- 172.21.15.50:0/2647932945 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(145..145 src has 1..145) ==== 5878+0+0 (secure 0 0 0) 0x7f3b600939a0 con 0x7f3b6c0d29c0 2024-09-21T06:13:42.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.921+0000 7f3b70e00640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b4c0759f0 0x7f3b4c077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:42.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.925+0000 7f3b70e00640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b4c0759f0 0x7f3b4c077f00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c14c9d0 tx=0x7f3b58009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:42.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:42.925+0000 7f3b69600640 1 -- 172.21.15.50:0/2647932945 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3b6005fe50 con 0x7f3b6c0d29c0 2024-09-21T06:13:43.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.257+0000 7f3b74200640 1 -- 172.21.15.50:0/2647932945 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f3b6c0d2f70 con 0x7f3b6c0d29c0 2024-09-21T06:13:43.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.257+0000 7f3b69600640 1 -- 172.21.15.50:0/2647932945 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v60)=0 v60) ==== 131+0+0 (secure 0 0 0) 0x7f3b60064880 con 0x7f3b6c0d29c0 2024-09-21T06:13:43.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.261+0000 7f3b74200640 1 -- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b4c0759f0 msgr2=0x7f3b4c077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:43.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.261+0000 7f3b74200640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b4c0759f0 0x7f3b4c077f00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c14c9d0 tx=0x7f3b58009060 comp rx=0 tx=0).stop 2024-09-21T06:13:43.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.261+0000 7f3b74200640 1 -- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b6c0d29c0 msgr2=0x7f3b6c157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:43.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.261+0000 7f3b74200640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b6c0d29c0 0x7f3b6c157cc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000b410 tx=0x7f3b6000b810 comp rx=0 tx=0).stop 2024-09-21T06:13:43.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.261+0000 7f3b74200640 1 -- 172.21.15.50:0/2647932945 shutdown_connections 2024-09-21T06:13:43.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.261+0000 7f3b74200640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3b4c0759f0 0x7f3b4c077f00 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.261+0000 7f3b74200640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b6c15f180 0x7f3b6c1fcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.261+0000 7f3b74200640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b6c155ad0 0x7f3b6c13d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.261+0000 7f3b74200640 1 --2- 172.21.15.50:0/2647932945 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3b6c0d29c0 0x7f3b6c157cc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.261+0000 7f3b74200640 1 -- 172.21.15.50:0/2647932945 >> 172.21.15.50:0/2647932945 conn(0x7f3b6c0708c0 msgr2=0x7f3b6c160750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:43.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.261+0000 7f3b74200640 1 -- 172.21.15.50:0/2647932945 shutdown_connections 2024-09-21T06:13:43.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.261+0000 7f3b74200640 1 -- 172.21.15.50:0/2647932945 wait complete. 2024-09-21T06:13:43.275 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T06:13:43.275 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:13:43.397 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T06:13:43.480 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T06:13:43.480 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:13:43.480 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:13:43.480 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:13:43.491 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:13:43.491 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:13:43.543 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T06:13:43.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.757+0000 7f4c58200640 1 -- 172.21.15.50:0/3300603575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c5015e290 msgr2=0x7f4c5015bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:43.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.757+0000 7f4c58200640 1 --2- 172.21.15.50:0/3300603575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c5015e290 0x7f4c5015bdf0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f4c500b3040 tx=0x7f4c4801cd80 comp rx=0 tx=0).stop 2024-09-21T06:13:43.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.757+0000 7f4c58200640 1 -- 172.21.15.50:0/3300603575 shutdown_connections 2024-09-21T06:13:43.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.757+0000 7f4c58200640 1 --2- 172.21.15.50:0/3300603575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c5015e290 0x7f4c5015bdf0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.757+0000 7f4c58200640 1 --2- 172.21.15.50:0/3300603575 >> v2:172.21.15.88:3300/0 conn(0x7f4c5015e8c0 0x7f4c5015ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.757+0000 7f4c58200640 1 -- 172.21.15.50:0/3300603575 >> 172.21.15.50:0/3300603575 conn(0x7f4c500b2040 msgr2=0x7f4c500b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:43.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.757+0000 7f4c58200640 1 -- 172.21.15.50:0/3300603575 shutdown_connections 2024-09-21T06:13:43.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.757+0000 7f4c58200640 1 -- 172.21.15.50:0/3300603575 wait complete. 2024-09-21T06:13:43.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.757+0000 7f4c58200640 1 Processor -- start 2024-09-21T06:13:43.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.757+0000 7f4c58200640 1 -- start start 2024-09-21T06:13:43.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c58200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5015b2d0 0x7f4c500cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:43.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c58200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c5015e290 0x7f4c501318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:43.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c58200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c5015e8c0 0x7f4c50140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:43.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c58200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501692a0 con 0x7f4c5015e8c0 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c58200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4c50169120 con 0x7f4c5015b2d0 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c58200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4c50169420 con 0x7f4c5015e290 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c56200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c5015e8c0 0x7f4c50140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c54e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c5015e290 0x7f4c501318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c55800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5015b2d0 0x7f4c500cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c56200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c5015e8c0 0x7f4c50140820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55982/0 (socket says 172.21.15.50:55982) 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c56200640 1 -- 172.21.15.50:0/316708755 learned_addr learned my addr 172.21.15.50:0/316708755 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c56200640 1 -- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c5015e290 msgr2=0x7f4c501318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c56200640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c5015e290 0x7f4c501318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c56200640 1 -- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5015b2d0 msgr2=0x7f4c500cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c56200640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5015b2d0 0x7f4c500cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c56200640 1 -- 172.21.15.50:0/316708755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c50144320 con 0x7f4c5015e8c0 2024-09-21T06:13:43.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c55800640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5015b2d0 0x7f4c500cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:43.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c56200640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c5015e8c0 0x7f4c50140820 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f4c4801d260 tx=0x7f4c48005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:43.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c4d600640 1 -- 172.21.15.50:0/316708755 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c480b2040 con 0x7f4c5015e8c0 2024-09-21T06:13:43.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c58200640 1 -- 172.21.15.50:0/316708755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c500ce3b0 con 0x7f4c5015e8c0 2024-09-21T06:13:43.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c4d600640 1 -- 172.21.15.50:0/316708755 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c480bb070 con 0x7f4c5015e8c0 2024-09-21T06:13:43.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c4d600640 1 -- 172.21.15.50:0/316708755 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48004240 con 0x7f4c5015e8c0 2024-09-21T06:13:43.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c58200640 1 -- 172.21.15.50:0/316708755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c500ce970 con 0x7f4c5015e8c0 2024-09-21T06:13:43.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.761+0000 7f4c58200640 1 -- 172.21.15.50:0/316708755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c500af2b0 con 0x7f4c5015e8c0 2024-09-21T06:13:43.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.765+0000 7f4c4d600640 1 -- 172.21.15.50:0/316708755 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f4c480043e0 con 0x7f4c5015e8c0 2024-09-21T06:13:43.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.765+0000 7f4c4d600640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c300759f0 0x7f4c30077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:43.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.765+0000 7f4c4d600640 1 -- 172.21.15.50:0/316708755 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5878+0+0 (secure 0 0 0) 0x7f4c4812df40 con 0x7f4c5015e8c0 2024-09-21T06:13:43.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.765+0000 7f4c55800640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c300759f0 0x7f4c30077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:43.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.765+0000 7f4c55800640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c300759f0 0x7f4c30077f00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009600 tx=0x7f4c440091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:43.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:43.769+0000 7f4c4d600640 1 -- 172.21.15.50:0/316708755 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4c48016970 con 0x7f4c5015e8c0 2024-09-21T06:13:44.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.121+0000 7f4c58200640 1 -- 172.21.15.50:0/316708755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4c5015ecd0 con 0x7f4c5015e8c0 2024-09-21T06:13:44.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c4d600640 1 -- 172.21.15.50:0/316708755 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) ==== 80+0+0 (secure 0 0 0) 0x7f4c480fa380 con 0x7f4c5015e8c0 2024-09-21T06:13:44.119 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-09-21T06:13:44.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c58200640 1 -- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c300759f0 msgr2=0x7f4c30077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:44.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c58200640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c300759f0 0x7f4c30077f00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009600 tx=0x7f4c440091e0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c58200640 1 -- 172.21.15.50:0/316708755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c5015e8c0 msgr2=0x7f4c50140820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:44.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c58200640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c5015e8c0 0x7f4c50140820 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f4c4801d260 tx=0x7f4c48005c00 comp rx=0 tx=0).stop 2024-09-21T06:13:44.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c58200640 1 -- 172.21.15.50:0/316708755 shutdown_connections 2024-09-21T06:13:44.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c58200640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4c300759f0 0x7f4c30077f00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c58200640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c5015e8c0 0x7f4c50140820 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c58200640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4c5015e290 0x7f4c501318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c58200640 1 --2- 172.21.15.50:0/316708755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5015b2d0 0x7f4c500cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c58200640 1 -- 172.21.15.50:0/316708755 >> 172.21.15.50:0/316708755 conn(0x7f4c500b2040 msgr2=0x7f4c501579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:44.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c58200640 1 -- 172.21.15.50:0/316708755 shutdown_connections 2024-09-21T06:13:44.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.125+0000 7f4c58200640 1 -- 172.21.15.50:0/316708755 wait complete. 2024-09-21T06:13:44.156 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:13:44.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:44.156 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:13:44.233 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:13:44.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:44.233 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:13:44.254 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:13:44.254 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:13:44.254 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:13:44.270 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:13:44.271 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:13:44.271 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:13:44.271 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:13:44.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:13:44.307 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T06:13:44.307 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:13:44.307 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:13:44.321 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:13:44.322 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:13:44.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.513+0000 7f7163600640 1 -- 172.21.15.50:0/2456795971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c0cefa0 msgr2=0x7f715c0cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:44.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.513+0000 7f7163600640 1 --2- 172.21.15.50:0/2456795971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c0cefa0 0x7f715c0cf3b0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f715c0b3040 tx=0x7f714801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.513+0000 7f7163600640 1 -- 172.21.15.50:0/2456795971 shutdown_connections 2024-09-21T06:13:44.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.513+0000 7f7163600640 1 --2- 172.21.15.50:0/2456795971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c0cefa0 0x7f715c0cf3b0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.513+0000 7f7163600640 1 --2- 172.21.15.50:0/2456795971 >> v2:172.21.15.88:3300/0 conn(0x7f715c0749a0 0x7f715c074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.513+0000 7f7163600640 1 -- 172.21.15.50:0/2456795971 >> 172.21.15.50:0/2456795971 conn(0x7f715c0b2040 msgr2=0x7f715c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:44.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.513+0000 7f7163600640 1 -- 172.21.15.50:0/2456795971 shutdown_connections 2024-09-21T06:13:44.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.513+0000 7f7163600640 1 -- 172.21.15.50:0/2456795971 wait complete. 2024-09-21T06:13:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.513+0000 7f7163600640 1 Processor -- start 2024-09-21T06:13:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.513+0000 7f7163600640 1 -- start start 2024-09-21T06:13:44.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7163600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f715c0749a0 0x7f715c145c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:44.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7163600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c0cefa0 0x7f715c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:44.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7163600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f715c160330 0x7f715c0b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:44.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7163600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f715c16a060 con 0x7f715c0cefa0 2024-09-21T06:13:44.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7163600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f715c169ee0 con 0x7f715c0749a0 2024-09-21T06:13:44.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7163600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f715c16a1e0 con 0x7f715c160330 2024-09-21T06:13:44.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f715be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c0cefa0 0x7f715c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:44.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f715be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c0cefa0 0x7f715c0b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56008/0 (socket says 172.21.15.50:56008) 2024-09-21T06:13:44.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f715be00640 1 -- 172.21.15.50:0/3557177815 learned_addr learned my addr 172.21.15.50:0/3557177815 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:44.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7160c00640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f715c0749a0 0x7f715c145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:44.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7161600640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f715c160330 0x7f715c0b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:44.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f715be00640 1 -- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f715c160330 msgr2=0x7f715c0b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:44.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f715be00640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f715c160330 0x7f715c0b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f715be00640 1 -- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f715c0749a0 msgr2=0x7f715c145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:44.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f715be00640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f715c0749a0 0x7f715c145c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f715be00640 1 -- 172.21.15.50:0/3557177815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c15fb40 con 0x7f715c0cefa0 2024-09-21T06:13:44.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7160c00640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f715c0749a0 0x7f715c145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:44.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f715be00640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c0cefa0 0x7f715c0b0ea0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f71480194d0 tx=0x7f7148005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:44.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7158c00640 1 -- 172.21.15.50:0/3557177815 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71480bc070 con 0x7f715c0cefa0 2024-09-21T06:13:44.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7158c00640 1 -- 172.21.15.50:0/3557177815 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7148005320 con 0x7f715c0cefa0 2024-09-21T06:13:44.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7163600640 1 -- 172.21.15.50:0/3557177815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715c13cd30 con 0x7f715c0cefa0 2024-09-21T06:13:44.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7158c00640 1 -- 172.21.15.50:0/3557177815 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71480b2040 con 0x7f715c0cefa0 2024-09-21T06:13:44.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7163600640 1 -- 172.21.15.50:0/3557177815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715c1fd030 con 0x7f715c0cefa0 2024-09-21T06:13:44.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.517+0000 7f7163600640 1 -- 172.21.15.50:0/3557177815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f715c1454c0 con 0x7f715c0cefa0 2024-09-21T06:13:44.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.521+0000 7f7158c00640 1 -- 172.21.15.50:0/3557177815 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f71480046d0 con 0x7f715c0cefa0 2024-09-21T06:13:44.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.521+0000 7f7158c00640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f713c0759e0 0x7f713c077ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:44.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.521+0000 7f7158c00640 1 -- 172.21.15.50:0/3557177815 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5878+0+0 (secure 0 0 0) 0x7f714812d140 con 0x7f715c0cefa0 2024-09-21T06:13:44.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.521+0000 7f7160c00640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f713c0759e0 0x7f713c077ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:44.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.521+0000 7f7160c00640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f713c0759e0 0x7f713c077ef0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7150009f30 tx=0x7f71500091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:44.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.525+0000 7f7158c00640 1 -- 172.21.15.50:0/3557177815 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f71480164b0 con 0x7f715c0cefa0 2024-09-21T06:13:44.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.745+0000 7f7163600640 1 -- 172.21.15.50:0/3557177815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f715c074db0 con 0x7f715c0cefa0 2024-09-21T06:13:44.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.745+0000 7f7158c00640 1 -- 172.21.15.50:0/3557177815 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5224 (secure 0 0 0) 0x7f71480f95f0 con 0x7f715c0cefa0 2024-09-21T06:13:44.741 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:44.741 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":95,"btime":"2024-09-21T06:12:33:714142+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4262887686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":4262887686},{"type":"v1","addr":"172.21.15.50:6835","nonce":4262887686}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":7721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3308178097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3308178097},{"type":"v1","addr":"172.21.15.88:6837","nonce":3308178097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/610421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":610421041},{"type":"v1","addr":"172.21.15.50:6834","nonce":610421041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:12:25.031119+0000","modified":"2024-09-21T06:12:28.054360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8632":{"gid":8632,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2222229719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2222229719},{"type":"v1","addr":"172.21.15.88:6835","nonce":2222229719}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8632,"qdb_cluster":[8632]},"id":10}]} 2024-09-21T06:13:44.741 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 95 2024-09-21T06:13:44.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.749+0000 7f7163600640 1 -- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f713c0759e0 msgr2=0x7f713c077ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:44.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.749+0000 7f7163600640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f713c0759e0 0x7f713c077ef0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7150009f30 tx=0x7f71500091e0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.749+0000 7f7163600640 1 -- 172.21.15.50:0/3557177815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c0cefa0 msgr2=0x7f715c0b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:44.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.749+0000 7f7163600640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c0cefa0 0x7f715c0b0ea0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f71480194d0 tx=0x7f7148005de0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.749+0000 7f7163600640 1 -- 172.21.15.50:0/3557177815 shutdown_connections 2024-09-21T06:13:44.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.749+0000 7f7163600640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f715c160330 0x7f715c0b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.749+0000 7f7163600640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f713c0759e0 0x7f713c077ef0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.749+0000 7f7163600640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c0cefa0 0x7f715c0b0ea0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.749+0000 7f7163600640 1 --2- 172.21.15.50:0/3557177815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f715c0749a0 0x7f715c145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:44.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.749+0000 7f7163600640 1 -- 172.21.15.50:0/3557177815 >> 172.21.15.50:0/3557177815 conn(0x7f715c0b2040 msgr2=0x7f715c0af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:44.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.749+0000 7f7163600640 1 -- 172.21.15.50:0/3557177815 shutdown_connections 2024-09-21T06:13:44.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:44.749+0000 7f7163600640 1 -- 172.21.15.50:0/3557177815 wait complete. 2024-09-21T06:13:44.764 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:13:45.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.009+0000 7fb6eb400640 1 -- 172.21.15.50:0/3130811034 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d80ad440 con 0x7fb6ec06e330 2024-09-21T06:13:45.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.009+0000 7fb6f3600640 1 -- 172.21.15.50:0/3130811034 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6ec06e330 msgr2=0x7fb6ec0b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.009+0000 7fb6f3600640 1 --2- 172.21.15.50:0/3130811034 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6ec06e330 0x7fb6ec0b51e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009900 tx=0x7fb6d801c8b0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.009+0000 7fb6f3600640 1 -- 172.21.15.50:0/3130811034 shutdown_connections 2024-09-21T06:13:45.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.009+0000 7fb6f3600640 1 --2- 172.21.15.50:0/3130811034 >> v2:172.21.15.88:3300/0 conn(0x7fb6ec070530 0x7fb6ec070940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.009+0000 7fb6f3600640 1 --2- 172.21.15.50:0/3130811034 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6ec06e330 0x7fb6ec0b51e0 secure :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009900 tx=0x7fb6d801c8b0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.009+0000 7fb6f3600640 1 -- 172.21.15.50:0/3130811034 >> 172.21.15.50:0/3130811034 conn(0x7fb6ec0b3a20 msgr2=0x7fb6ec0d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:45.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.009+0000 7fb6f3600640 1 -- 172.21.15.50:0/3130811034 shutdown_connections 2024-09-21T06:13:45.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.009+0000 7fb6f3600640 1 -- 172.21.15.50:0/3130811034 wait complete. 2024-09-21T06:13:45.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.013+0000 7fb6f3600640 1 Processor -- start 2024-09-21T06:13:45.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.013+0000 7fb6f3600640 1 -- start start 2024-09-21T06:13:45.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f3600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6ec070530 0x7fb6ec1fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f3600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6ec0b44a0 0x7fb6ec1474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f3600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6ec145450 0x7fb6ec145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f3600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec159180 con 0x7fb6ec145450 2024-09-21T06:13:45.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f3600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec159000 con 0x7fb6ec0b44a0 2024-09-21T06:13:45.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f3600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb6ec159300 con 0x7fb6ec070530 2024-09-21T06:13:45.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f0c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6ec070530 0x7fb6ec1fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6ec145450 0x7fb6ec145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6ec145450 0x7fb6ec145860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56020/0 (socket says 172.21.15.50:56020) 2024-09-21T06:13:45.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f1600640 1 -- 172.21.15.50:0/2959624453 learned_addr learned my addr 172.21.15.50:0/2959624453 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:45.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6ebe00640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6ec0b44a0 0x7fb6ec1474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f0c00640 1 -- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6ec0b44a0 msgr2=0x7fb6ec1474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f0c00640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6ec0b44a0 0x7fb6ec1474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f0c00640 1 -- 172.21.15.50:0/2959624453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6ec145450 msgr2=0x7fb6ec145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f0c00640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6ec145450 0x7fb6ec145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.017+0000 7fb6f0c00640 1 -- 172.21.15.50:0/2959624453 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec14b7f0 con 0x7fb6ec070530 2024-09-21T06:13:45.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.021+0000 7fb6f0c00640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6ec070530 0x7fb6ec1fcdc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80ac000 tx=0x7fb6d8005e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.021+0000 7fb6e8c00640 1 -- 172.21.15.50:0/2959624453 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d80b2040 con 0x7fb6ec070530 2024-09-21T06:13:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.021+0000 7fb6e8c00640 1 -- 172.21.15.50:0/2959624453 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6d8005320 con 0x7fb6ec070530 2024-09-21T06:13:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.021+0000 7fb6e8c00640 1 -- 172.21.15.50:0/2959624453 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d80046d0 con 0x7fb6ec070530 2024-09-21T06:13:45.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.021+0000 7fb6f3600640 1 -- 172.21.15.50:0/2959624453 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ec148050 con 0x7fb6ec070530 2024-09-21T06:13:45.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.021+0000 7fb6f3600640 1 -- 172.21.15.50:0/2959624453 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ec1fc2e0 con 0x7fb6ec070530 2024-09-21T06:13:45.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.025+0000 7fb6f3600640 1 -- 172.21.15.50:0/2959624453 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ec15da30 con 0x7fb6ec070530 2024-09-21T06:13:45.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.025+0000 7fb6e8c00640 1 -- 172.21.15.50:0/2959624453 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fb6d80a5a60 con 0x7fb6ec070530 2024-09-21T06:13:45.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.029+0000 7fb6e8c00640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb6cc0759e0 0x7fb6cc077ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.029+0000 7fb6e8c00640 1 -- 172.21.15.50:0/2959624453 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(145..145 src has 1..145) ==== 5878+0+0 (secure 0 0 0) 0x7fb6d812db20 con 0x7fb6ec070530 2024-09-21T06:13:45.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.029+0000 7fb6ebe00640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb6cc0759e0 0x7fb6cc077ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.029+0000 7fb6ebe00640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb6cc0759e0 0x7fb6cc077ef0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00058d0 tx=0x7fb6e0005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:45.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.033+0000 7fb6e8c00640 1 -- 172.21.15.50:0/2959624453 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb6d80164d0 con 0x7fb6ec070530 2024-09-21T06:13:45.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.361+0000 7fb6f3600640 1 -- 172.21.15.50:0/2959624453 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6ec15d750 con 0x7fb6ec070530 2024-09-21T06:13:45.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.361+0000 7fb6e8c00640 1 -- 172.21.15.50:0/2959624453 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5224 (secure 0 0 0) 0x7fb6d80a5d50 con 0x7fb6ec070530 2024-09-21T06:13:45.400 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:45.400 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":95,"btime":"2024-09-21T06:12:33:714142+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4262887686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":4262887686},{"type":"v1","addr":"172.21.15.50:6835","nonce":4262887686}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":7721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3308178097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3308178097},{"type":"v1","addr":"172.21.15.88:6837","nonce":3308178097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/610421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":610421041},{"type":"v1","addr":"172.21.15.50:6834","nonce":610421041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:12:25.031119+0000","modified":"2024-09-21T06:12:28.054360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8632":{"gid":8632,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2222229719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2222229719},{"type":"v1","addr":"172.21.15.88:6835","nonce":2222229719}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8632,"qdb_cluster":[8632]},"id":10}]} 2024-09-21T06:13:45.401 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 95 2024-09-21T06:13:45.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.365+0000 7fb6f3600640 1 -- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb6cc0759e0 msgr2=0x7fb6cc077ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.365+0000 7fb6f3600640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb6cc0759e0 0x7fb6cc077ef0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00058d0 tx=0x7fb6e0005860 comp rx=0 tx=0).stop 2024-09-21T06:13:45.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.365+0000 7fb6f3600640 1 -- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6ec070530 msgr2=0x7fb6ec1fcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.365+0000 7fb6f3600640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6ec070530 0x7fb6ec1fcdc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80ac000 tx=0x7fb6d8005e90 comp rx=0 tx=0).stop 2024-09-21T06:13:45.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.365+0000 7fb6f3600640 1 -- 172.21.15.50:0/2959624453 shutdown_connections 2024-09-21T06:13:45.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.365+0000 7fb6f3600640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb6cc0759e0 0x7fb6cc077ef0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.365+0000 7fb6f3600640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6ec145450 0x7fb6ec145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.365+0000 7fb6f3600640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6ec0b44a0 0x7fb6ec1474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.365+0000 7fb6f3600640 1 --2- 172.21.15.50:0/2959624453 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb6ec070530 0x7fb6ec1fcdc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.365+0000 7fb6f3600640 1 -- 172.21.15.50:0/2959624453 >> 172.21.15.50:0/2959624453 conn(0x7fb6ec0b3a20 msgr2=0x7fb6ec15cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:45.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.365+0000 7fb6f3600640 1 -- 172.21.15.50:0/2959624453 shutdown_connections 2024-09-21T06:13:45.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.365+0000 7fb6f3600640 1 -- 172.21.15.50:0/2959624453 wait complete. 2024-09-21T06:13:45.402 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:13:45.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.565+0000 7f57b3600640 1 -- 172.21.15.50:0/2703641796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57ac06cdf0 msgr2=0x7f57ac06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.565+0000 7f57b3600640 1 --2- 172.21.15.50:0/2703641796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57ac06cdf0 0x7f57ac06d200 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5794009840 tx=0x7f579401c790 comp rx=0 tx=0).stop 2024-09-21T06:13:45.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.565+0000 7f57b3600640 1 -- 172.21.15.50:0/2703641796 shutdown_connections 2024-09-21T06:13:45.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.565+0000 7f57b3600640 1 --2- 172.21.15.50:0/2703641796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57ac06cdf0 0x7f57ac06d200 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.565+0000 7f57b3600640 1 -- 172.21.15.50:0/2703641796 >> 172.21.15.50:0/2703641796 conn(0x7f57ac0b35f0 msgr2=0x7f57ac0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:45.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.565+0000 7f57b3600640 1 -- 172.21.15.50:0/2703641796 shutdown_connections 2024-09-21T06:13:45.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.565+0000 7f57b3600640 1 -- 172.21.15.50:0/2703641796 wait complete. 2024-09-21T06:13:45.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.569+0000 7f57b3600640 1 Processor -- start 2024-09-21T06:13:45.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.569+0000 7f57b3600640 1 -- start start 2024-09-21T06:13:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.569+0000 7f57b3600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ac06cdf0 0x7f57ac13fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.569+0000 7f57b3600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57ac06fc90 0x7f57ac140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.569+0000 7f57b3600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57ac159130 0x7f57ac152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.569+0000 7f57b3600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac16a550 con 0x7f57ac159130 2024-09-21T06:13:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.569+0000 7f57b3600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac16a3d0 con 0x7f57ac06cdf0 2024-09-21T06:13:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.569+0000 7f57b3600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f57ac16a6d0 con 0x7f57ac06fc90 2024-09-21T06:13:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.569+0000 7f57b1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57ac159130 0x7f57ac152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.569+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57ac06fc90 0x7f57ac140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.569+0000 7f57b1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57ac159130 0x7f57ac152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56030/0 (socket says 172.21.15.50:56030) 2024-09-21T06:13:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.569+0000 7f57b1600640 1 -- 172.21.15.50:0/214152082 learned_addr learned my addr 172.21.15.50:0/214152082 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57b0c00640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ac06cdf0 0x7f57ac13fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57a3e00640 1 -- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ac06cdf0 msgr2=0x7f57ac13fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57a3e00640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ac06cdf0 0x7f57ac13fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57a3e00640 1 -- 172.21.15.50:0/214152082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57ac159130 msgr2=0x7f57ac152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57a3e00640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57ac159130 0x7f57ac152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57a3e00640 1 -- 172.21.15.50:0/214152082 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac1597b0 con 0x7f57ac06fc90 2024-09-21T06:13:45.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57b1600640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57ac159130 0x7f57ac152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:45.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57a3e00640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57ac06fc90 0x7f57ac140330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f57940ac000 tx=0x7f57940a6740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:45.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57a9600640 1 -- 172.21.15.50:0/214152082 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57940053d0 con 0x7f57ac06fc90 2024-09-21T06:13:45.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57a9600640 1 -- 172.21.15.50:0/214152082 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5794005570 con 0x7f57ac06fc90 2024-09-21T06:13:45.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57b3600640 1 -- 172.21.15.50:0/214152082 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ac1530d0 con 0x7f57ac06fc90 2024-09-21T06:13:45.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57a9600640 1 -- 172.21.15.50:0/214152082 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794004290 con 0x7f57ac06fc90 2024-09-21T06:13:45.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57b3600640 1 -- 172.21.15.50:0/214152082 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ac159d30 con 0x7f57ac06fc90 2024-09-21T06:13:45.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57b3600640 1 -- 172.21.15.50:0/214152082 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57ac152e20 con 0x7f57ac06fc90 2024-09-21T06:13:45.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.573+0000 7f57a9600640 1 -- 172.21.15.50:0/214152082 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f57940b8030 con 0x7f57ac06fc90 2024-09-21T06:13:45.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.577+0000 7f57a9600640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5788075a20 0x7f5788077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.577+0000 7f57a9600640 1 -- 172.21.15.50:0/214152082 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(145..145 src has 1..145) ==== 5878+0+0 (secure 0 0 0) 0x7f579412d3b0 con 0x7f57ac06fc90 2024-09-21T06:13:45.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.577+0000 7f57b0c00640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5788075a20 0x7f5788077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.577+0000 7f57b0c00640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5788075a20 0x7f5788077f30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f579c007fd0 tx=0x7f579c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:45.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.577+0000 7f57a9600640 1 -- 172.21.15.50:0/214152082 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f57940bf070 con 0x7f57ac06fc90 2024-09-21T06:13:45.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.761+0000 7f57b3600640 1 -- 172.21.15.50:0/214152082 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57ac06d200 con 0x7f57ac06fc90 2024-09-21T06:13:45.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.761+0000 7f57a9600640 1 -- 172.21.15.50:0/214152082 <== mon.2 v2:172.21.15.88:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16986 (secure 0 0 0) 0x7f57940ac030 con 0x7f57ac06fc90 2024-09-21T06:13:45.758 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:45.758 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":145,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:13:41.843860+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:12:22.777798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T06:12:23.995303+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":145,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.29","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:13:45.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.765+0000 7f57b3600640 1 -- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5788075a20 msgr2=0x7f5788077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.765+0000 7f57b3600640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5788075a20 0x7f5788077f30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f579c007fd0 tx=0x7f579c009040 comp rx=0 tx=0).stop 2024-09-21T06:13:45.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.765+0000 7f57b3600640 1 -- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57ac06fc90 msgr2=0x7f57ac140330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.765+0000 7f57b3600640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57ac06fc90 0x7f57ac140330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f57940ac000 tx=0x7f57940a6740 comp rx=0 tx=0).stop 2024-09-21T06:13:45.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.765+0000 7f57b3600640 1 -- 172.21.15.50:0/214152082 shutdown_connections 2024-09-21T06:13:45.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.765+0000 7f57b3600640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f5788075a20 0x7f5788077f30 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.765+0000 7f57b3600640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57ac159130 0x7f57ac152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.765+0000 7f57b3600640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f57ac06fc90 0x7f57ac140330 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.765+0000 7f57b3600640 1 --2- 172.21.15.50:0/214152082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ac06cdf0 0x7f57ac13fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.765+0000 7f57b3600640 1 -- 172.21.15.50:0/214152082 >> 172.21.15.50:0/214152082 conn(0x7f57ac0b35f0 msgr2=0x7f57ac075da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:45.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.765+0000 7f57b3600640 1 -- 172.21.15.50:0/214152082 shutdown_connections 2024-09-21T06:13:45.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.765+0000 7f57b3600640 1 -- 172.21.15.50:0/214152082 wait complete. 2024-09-21T06:13:45.781 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:13:45.781 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:13:45.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.945+0000 7fb9ca000640 1 -- 172.21.15.50:0/56577399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b5c70 msgr2=0x7fb9c415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.945+0000 7fb9ca000640 1 --2- 172.21.15.50:0/56577399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b5c70 0x7fb9c415bb60 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009900 tx=0x7fb9b401c820 comp rx=0 tx=0).stop 2024-09-21T06:13:45.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.945+0000 7fb9ca000640 1 -- 172.21.15.50:0/56577399 shutdown_connections 2024-09-21T06:13:45.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.945+0000 7fb9ca000640 1 --2- 172.21.15.50:0/56577399 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9c41527b0 0x7fb9c4074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.945+0000 7fb9ca000640 1 --2- 172.21.15.50:0/56577399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b5c70 0x7fb9c415bb60 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.945+0000 7fb9ca000640 1 -- 172.21.15.50:0/56577399 >> 172.21.15.50:0/56577399 conn(0x7fb9c40b2040 msgr2=0x7fb9c40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:45.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.945+0000 7fb9ca000640 1 -- 172.21.15.50:0/56577399 shutdown_connections 2024-09-21T06:13:45.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.945+0000 7fb9ca000640 1 -- 172.21.15.50:0/56577399 wait complete. 2024-09-21T06:13:45.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.945+0000 7fb9ca000640 1 Processor -- start 2024-09-21T06:13:45.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.945+0000 7fb9ca000640 1 -- start start 2024-09-21T06:13:45.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9ca000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9c40b4de0 0x7fb9c4141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9ca000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b5c70 0x7fb9c4153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9ca000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c41527b0 0x7fb9c41541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9ca000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4168c50 con 0x7fb9c40b5c70 2024-09-21T06:13:45.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9ca000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4168ad0 con 0x7fb9c41527b0 2024-09-21T06:13:45.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9ca000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c4168dd0 con 0x7fb9c40b4de0 2024-09-21T06:13:45.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9bbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b5c70 0x7fb9c4153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9c3400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9c40b4de0 0x7fb9c4141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9bbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b5c70 0x7fb9c4153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56060/0 (socket says 172.21.15.50:56060) 2024-09-21T06:13:45.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9bbe00640 1 -- 172.21.15.50:0/96092663 learned_addr learned my addr 172.21.15.50:0/96092663 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:45.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9c3e00640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c41527b0 0x7fb9c41541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9bbe00640 1 -- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9c40b4de0 msgr2=0x7fb9c4141d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9bbe00640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9c40b4de0 0x7fb9c4141d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9bbe00640 1 -- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c41527b0 msgr2=0x7fb9c41541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9bbe00640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c41527b0 0x7fb9c41541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9bbe00640 1 -- 172.21.15.50:0/96092663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c4145570 con 0x7fb9c40b5c70 2024-09-21T06:13:45.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9c3400640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9c40b4de0 0x7fb9c4141d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:45.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9bbe00640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b5c70 0x7fb9c4153c70 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00d6a0 tx=0x7fb9ac00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:45.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9bb400640 1 -- 172.21.15.50:0/96092663 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac017070 con 0x7fb9c40b5c70 2024-09-21T06:13:45.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9bb400640 1 -- 172.21.15.50:0/96092663 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9ac004310 con 0x7fb9c40b5c70 2024-09-21T06:13:45.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9bb400640 1 -- 172.21.15.50:0/96092663 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac005280 con 0x7fb9c40b5c70 2024-09-21T06:13:45.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9ca000640 1 -- 172.21.15.50:0/96092663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c4142c50 con 0x7fb9c40b5c70 2024-09-21T06:13:45.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9ca000640 1 -- 172.21.15.50:0/96092663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c4143210 con 0x7fb9c40b5c70 2024-09-21T06:13:45.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.949+0000 7fb9ca000640 1 -- 172.21.15.50:0/96092663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c41429a0 con 0x7fb9c40b5c70 2024-09-21T06:13:45.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.953+0000 7fb9bb400640 1 -- 172.21.15.50:0/96092663 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fb9ac005420 con 0x7fb9c40b5c70 2024-09-21T06:13:45.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.953+0000 7fb9bb400640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb9a0075a20 0x7fb9a0077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.953+0000 7fb9bb400640 1 -- 172.21.15.50:0/96092663 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5878+0+0 (secure 0 0 0) 0x7fb9ac093810 con 0x7fb9c40b5c70 2024-09-21T06:13:45.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.953+0000 7fb9c3400640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb9a0075a20 0x7fb9a0077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.953+0000 7fb9c3400640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb9a0075a20 0x7fb9a0077f30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40af000 tx=0x7fb9b401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:45.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:45.957+0000 7fb9bb400640 1 -- 172.21.15.50:0/96092663 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb9ac020020 con 0x7fb9c40b5c70 2024-09-21T06:13:46.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.169+0000 7fb9ca000640 1 -- 172.21.15.50:0/96092663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9c415bb60 con 0x7fb9c40b5c70 2024-09-21T06:13:46.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.169+0000 7fb9bb400640 1 -- 172.21.15.50:0/96092663 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5224 (secure 0 0 0) 0x7fb9ac05fc40 con 0x7fb9c40b5c70 2024-09-21T06:13:46.165 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:46.165 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":95,"btime":"2024-09-21T06:12:33:714142+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4262887686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":4262887686},{"type":"v1","addr":"172.21.15.50:6835","nonce":4262887686}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":7721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3308178097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3308178097},{"type":"v1","addr":"172.21.15.88:6837","nonce":3308178097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/610421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":610421041},{"type":"v1","addr":"172.21.15.50:6834","nonce":610421041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:12:25.031119+0000","modified":"2024-09-21T06:12:28.054360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8632":{"gid":8632,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2222229719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2222229719},{"type":"v1","addr":"172.21.15.88:6835","nonce":2222229719}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8632,"qdb_cluster":[8632]},"id":10}]} 2024-09-21T06:13:46.166 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 95 2024-09-21T06:13:46.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.173+0000 7fb9ca000640 1 -- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb9a0075a20 msgr2=0x7fb9a0077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.173+0000 7fb9ca000640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb9a0075a20 0x7fb9a0077f30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40af000 tx=0x7fb9b401b060 comp rx=0 tx=0).stop 2024-09-21T06:13:46.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.173+0000 7fb9ca000640 1 -- 172.21.15.50:0/96092663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b5c70 msgr2=0x7fb9c4153c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.173+0000 7fb9ca000640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b5c70 0x7fb9c4153c70 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00d6a0 tx=0x7fb9ac00db60 comp rx=0 tx=0).stop 2024-09-21T06:13:46.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.173+0000 7fb9ca000640 1 -- 172.21.15.50:0/96092663 shutdown_connections 2024-09-21T06:13:46.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.173+0000 7fb9ca000640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c41527b0 0x7fb9c41541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.173+0000 7fb9ca000640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb9a0075a20 0x7fb9a0077f30 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.173+0000 7fb9ca000640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b5c70 0x7fb9c4153c70 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.173+0000 7fb9ca000640 1 --2- 172.21.15.50:0/96092663 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb9c40b4de0 0x7fb9c4141d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.173+0000 7fb9ca000640 1 -- 172.21.15.50:0/96092663 >> 172.21.15.50:0/96092663 conn(0x7fb9c40b2040 msgr2=0x7fb9c40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:46.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.173+0000 7fb9ca000640 1 -- 172.21.15.50:0/96092663 shutdown_connections 2024-09-21T06:13:46.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.173+0000 7fb9ca000640 1 -- 172.21.15.50:0/96092663 wait complete. 2024-09-21T06:13:46.188 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:13:46.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.357+0000 7fc099c00640 1 -- 172.21.15.50:0/1941895522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0940b5c70 msgr2=0x7fc09415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.357+0000 7fc099c00640 1 --2- 172.21.15.50:0/1941895522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0940b5c70 0x7fc09415bb60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc084009840 tx=0x7fc08401c7f0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.357+0000 7fc099c00640 1 -- 172.21.15.50:0/1941895522 shutdown_connections 2024-09-21T06:13:46.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.357+0000 7fc099c00640 1 --2- 172.21.15.50:0/1941895522 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc0940b4de0 0x7fc0940b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.357+0000 7fc099c00640 1 --2- 172.21.15.50:0/1941895522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0940b5c70 0x7fc09415bb60 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.357+0000 7fc099c00640 1 -- 172.21.15.50:0/1941895522 >> 172.21.15.50:0/1941895522 conn(0x7fc0940b2040 msgr2=0x7fc0940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:46.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.357+0000 7fc099c00640 1 -- 172.21.15.50:0/1941895522 shutdown_connections 2024-09-21T06:13:46.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.357+0000 7fc099c00640 1 -- 172.21.15.50:0/1941895522 wait complete. 2024-09-21T06:13:46.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.357+0000 7fc099c00640 1 Processor -- start 2024-09-21T06:13:46.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.357+0000 7fc099c00640 1 -- start start 2024-09-21T06:13:46.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc099c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc0940b4de0 0x7fc0941fb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc099c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0940b5c70 0x7fc0941f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc099c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0941527b0 0x7fc0941fa200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc099c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc094168970 con 0x7fc0940b5c70 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc099c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc0941687f0 con 0x7fc0941527b0 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc099c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc094168af0 con 0x7fc0940b4de0 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc08be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0940b5c70 0x7fc0941f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc08be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0940b5c70 0x7fc0941f9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56062/0 (socket says 172.21.15.50:56062) 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc08be00640 1 -- 172.21.15.50:0/650872810 learned_addr learned my addr 172.21.15.50:0/650872810 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc092a00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc0940b4de0 0x7fc0941fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc093400640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0941527b0 0x7fc0941fa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc08be00640 1 -- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc0940b4de0 msgr2=0x7fc0941fb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc08be00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc0940b4de0 0x7fc0941fb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc08be00640 1 -- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0941527b0 msgr2=0x7fc0941fa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc08be00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0941527b0 0x7fc0941fa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc08be00640 1 -- 172.21.15.50:0/650872810 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc094156e20 con 0x7fc0940b5c70 2024-09-21T06:13:46.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc08be00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0940b5c70 0x7fc0941f9cc0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00c730 tx=0x7fc07c00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:46.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc08aa00640 1 -- 172.21.15.50:0/650872810 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c016020 con 0x7fc0940b5c70 2024-09-21T06:13:46.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc08aa00640 1 -- 172.21.15.50:0/650872810 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc07c00a950 con 0x7fc0940b5c70 2024-09-21T06:13:46.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc08aa00640 1 -- 172.21.15.50:0/650872810 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00ada0 con 0x7fc0940b5c70 2024-09-21T06:13:46.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc099c00640 1 -- 172.21.15.50:0/650872810 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc094157100 con 0x7fc0940b5c70 2024-09-21T06:13:46.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc099c00640 1 -- 172.21.15.50:0/650872810 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0941576c0 con 0x7fc0940b5c70 2024-09-21T06:13:46.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.361+0000 7fc099c00640 1 -- 172.21.15.50:0/650872810 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0940b51f0 con 0x7fc0940b5c70 2024-09-21T06:13:46.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.365+0000 7fc08aa00640 1 -- 172.21.15.50:0/650872810 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fc07c019020 con 0x7fc0940b5c70 2024-09-21T06:13:46.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.365+0000 7fc08aa00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc0700759f0 0x7fc070077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:46.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.365+0000 7fc08aa00640 1 -- 172.21.15.50:0/650872810 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5878+0+0 (secure 0 0 0) 0x7fc07c093490 con 0x7fc0940b5c70 2024-09-21T06:13:46.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.365+0000 7fc092a00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc0700759f0 0x7fc070077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:46.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.365+0000 7fc092a00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc0700759f0 0x7fc070077f00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc0840af000 tx=0x7fc08401b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:46.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.369+0000 7fc08aa00640 1 -- 172.21.15.50:0/650872810 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc07c020020 con 0x7fc0940b5c70 2024-09-21T06:13:46.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.581+0000 7fc099c00640 1 -- 172.21.15.50:0/650872810 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc094156ba0 con 0x7fc0940b5c70 2024-09-21T06:13:46.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.581+0000 7fc08aa00640 1 -- 172.21.15.50:0/650872810 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5224 (secure 0 0 0) 0x7fc094156ba0 con 0x7fc0940b5c70 2024-09-21T06:13:46.577 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:46.577 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":95,"btime":"2024-09-21T06:12:33:714142+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4262887686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":4262887686},{"type":"v1","addr":"172.21.15.50:6835","nonce":4262887686}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":7721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3308178097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3308178097},{"type":"v1","addr":"172.21.15.88:6837","nonce":3308178097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/610421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":610421041},{"type":"v1","addr":"172.21.15.50:6834","nonce":610421041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:12:25.031119+0000","modified":"2024-09-21T06:12:28.054360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8632":{"gid":8632,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2222229719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2222229719},{"type":"v1","addr":"172.21.15.88:6835","nonce":2222229719}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8632,"qdb_cluster":[8632]},"id":10}]} 2024-09-21T06:13:46.578 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 95 2024-09-21T06:13:46.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.585+0000 7fc099c00640 1 -- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc0700759f0 msgr2=0x7fc070077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.585+0000 7fc099c00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc0700759f0 0x7fc070077f00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc0840af000 tx=0x7fc08401b030 comp rx=0 tx=0).stop 2024-09-21T06:13:46.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.585+0000 7fc099c00640 1 -- 172.21.15.50:0/650872810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0940b5c70 msgr2=0x7fc0941f9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.585+0000 7fc099c00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0940b5c70 0x7fc0941f9cc0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00c730 tx=0x7fc07c00cb30 comp rx=0 tx=0).stop 2024-09-21T06:13:46.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.585+0000 7fc099c00640 1 -- 172.21.15.50:0/650872810 shutdown_connections 2024-09-21T06:13:46.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.585+0000 7fc099c00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0941527b0 0x7fc0941fa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.585+0000 7fc099c00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc0700759f0 0x7fc070077f00 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.585+0000 7fc099c00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0940b5c70 0x7fc0941f9cc0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.585+0000 7fc099c00640 1 --2- 172.21.15.50:0/650872810 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc0940b4de0 0x7fc0941fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.585+0000 7fc099c00640 1 -- 172.21.15.50:0/650872810 >> 172.21.15.50:0/650872810 conn(0x7fc0940b2040 msgr2=0x7fc094159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:46.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.585+0000 7fc099c00640 1 -- 172.21.15.50:0/650872810 shutdown_connections 2024-09-21T06:13:46.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.585+0000 7fc099c00640 1 -- 172.21.15.50:0/650872810 wait complete. 2024-09-21T06:13:46.600 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:13:46.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.761+0000 7f6a87000640 1 -- 172.21.15.50:0/686918242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80160060 msgr2=0x7f6a80074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.761+0000 7f6a87000640 1 --2- 172.21.15.50:0/686918242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80160060 0x7f6a80074240 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009840 tx=0x7f6a6c01c790 comp rx=0 tx=0).stop 2024-09-21T06:13:46.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.761+0000 7f6a87000640 1 -- 172.21.15.50:0/686918242 shutdown_connections 2024-09-21T06:13:46.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.761+0000 7f6a87000640 1 --2- 172.21.15.50:0/686918242 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6a80160470 0x7f6a80074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.761+0000 7f6a87000640 1 --2- 172.21.15.50:0/686918242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80160060 0x7f6a80074240 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.761+0000 7f6a87000640 1 -- 172.21.15.50:0/686918242 >> 172.21.15.50:0/686918242 conn(0x7f6a800b4560 msgr2=0x7f6a800d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:46.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.761+0000 7f6a87000640 1 -- 172.21.15.50:0/686918242 shutdown_connections 2024-09-21T06:13:46.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.761+0000 7f6a87000640 1 -- 172.21.15.50:0/686918242 wait complete. 2024-09-21T06:13:46.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.765+0000 7f6a87000640 1 Processor -- start 2024-09-21T06:13:46.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.765+0000 7f6a87000640 1 -- start start 2024-09-21T06:13:46.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.765+0000 7f6a87000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6a801536f0 0x7f6a800d1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:46.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.765+0000 7f6a87000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a80160060 0x7f6a800d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:46.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.765+0000 7f6a87000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80160470 0x7f6a80152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.765+0000 7f6a87000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80169740 con 0x7f6a80160470 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.765+0000 7f6a87000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801695c0 con 0x7f6a80160060 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.765+0000 7f6a87000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6a801698c0 con 0x7f6a801536f0 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.765+0000 7f6a86600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80160470 0x7f6a80152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a86600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80160470 0x7f6a80152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33060/0 (socket says 172.21.15.50:33060) 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a86600640 1 -- 172.21.15.50:0/3844058427 learned_addr learned my addr 172.21.15.50:0/3844058427 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a85200640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a80160060 0x7f6a800d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a85c00640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6a801536f0 0x7f6a800d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a86600640 1 -- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6a801536f0 msgr2=0x7f6a800d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a86600640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6a801536f0 0x7f6a800d1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a86600640 1 -- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a80160060 msgr2=0x7f6a800d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a86600640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a80160060 0x7f6a800d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a86600640 1 -- 172.21.15.50:0/3844058427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a800d27a0 con 0x7f6a80160470 2024-09-21T06:13:46.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a86600640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80160470 0x7f6a80152da0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800d9a0 tx=0x7f6a7800de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:46.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a7d600640 1 -- 172.21.15.50:0/3844058427 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78017070 con 0x7f6a80160470 2024-09-21T06:13:46.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a7d600640 1 -- 172.21.15.50:0/3844058427 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a78004e20 con 0x7f6a80160470 2024-09-21T06:13:46.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a7d600640 1 -- 172.21.15.50:0/3844058427 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7800bd50 con 0x7f6a80160470 2024-09-21T06:13:46.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a87000640 1 -- 172.21.15.50:0/3844058427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a800d21e0 con 0x7f6a80160470 2024-09-21T06:13:46.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a87000640 1 -- 172.21.15.50:0/3844058427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a801525a0 con 0x7f6a80160470 2024-09-21T06:13:46.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a87000640 1 -- 172.21.15.50:0/3844058427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a800d1f50 con 0x7f6a80160470 2024-09-21T06:13:46.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.769+0000 7f6a7d600640 1 -- 172.21.15.50:0/3844058427 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f6a78003980 con 0x7f6a80160470 2024-09-21T06:13:46.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.773+0000 7f6a7d600640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6a600759f0 0x7f6a60077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:46.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.773+0000 7f6a7d600640 1 -- 172.21.15.50:0/3844058427 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5878+0+0 (secure 0 0 0) 0x7f6a78093ba0 con 0x7f6a80160470 2024-09-21T06:13:46.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.773+0000 7f6a85c00640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6a600759f0 0x7f6a60077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:46.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.773+0000 7f6a85c00640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6a600759f0 0x7f6a60077f00 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6a74007fd0 tx=0x7f6a74009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:46.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.777+0000 7f6a7d600640 1 -- 172.21.15.50:0/3844058427 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6a78020020 con 0x7f6a80160470 2024-09-21T06:13:46.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.953+0000 7f6a87000640 1 -- 172.21.15.50:0/3844058427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a8015d480 con 0x7f6a80160470 2024-09-21T06:13:46.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.953+0000 7f6a7d600640 1 -- 172.21.15.50:0/3844058427 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16986 (secure 0 0 0) 0x7f6a78003c70 con 0x7f6a80160470 2024-09-21T06:13:46.951 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":145,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:13:41.843860+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:12:22.777798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T06:12:23.995303+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":145,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.29","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:13:46.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.957+0000 7f6a87000640 1 -- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6a600759f0 msgr2=0x7f6a60077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.957+0000 7f6a87000640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6a600759f0 0x7f6a60077f00 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6a74007fd0 tx=0x7f6a74009040 comp rx=0 tx=0).stop 2024-09-21T06:13:46.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.957+0000 7f6a87000640 1 -- 172.21.15.50:0/3844058427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80160470 msgr2=0x7f6a80152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.957+0000 7f6a87000640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80160470 0x7f6a80152da0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800d9a0 tx=0x7f6a7800de60 comp rx=0 tx=0).stop 2024-09-21T06:13:46.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.957+0000 7f6a87000640 1 -- 172.21.15.50:0/3844058427 shutdown_connections 2024-09-21T06:13:46.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.957+0000 7f6a87000640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6a600759f0 0x7f6a60077f00 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.957+0000 7f6a87000640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80160470 0x7f6a80152da0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.957+0000 7f6a87000640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a80160060 0x7f6a800d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.957+0000 7f6a87000640 1 --2- 172.21.15.50:0/3844058427 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6a801536f0 0x7f6a800d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.957+0000 7f6a87000640 1 -- 172.21.15.50:0/3844058427 >> 172.21.15.50:0/3844058427 conn(0x7f6a800b4560 msgr2=0x7f6a80149870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:46.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.957+0000 7f6a87000640 1 -- 172.21.15.50:0/3844058427 shutdown_connections 2024-09-21T06:13:46.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:46.957+0000 7f6a87000640 1 -- 172.21.15.50:0/3844058427 wait complete. 2024-09-21T06:13:46.973 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T06:13:47.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.137+0000 7f5033400640 1 -- 172.21.15.50:0/1011969858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f502c0b54f0 msgr2=0x7f502c155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:47.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.137+0000 7f5033400640 1 --2- 172.21.15.50:0/1011969858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f502c0b54f0 0x7f502c155b50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5018009840 tx=0x7f501801c790 comp rx=0 tx=0).stop 2024-09-21T06:13:47.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.137+0000 7f5033400640 1 -- 172.21.15.50:0/1011969858 shutdown_connections 2024-09-21T06:13:47.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.137+0000 7f5033400640 1 --2- 172.21.15.50:0/1011969858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f502c0b54f0 0x7f502c155b50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.137+0000 7f5033400640 1 -- 172.21.15.50:0/1011969858 >> 172.21.15.50:0/1011969858 conn(0x7f502c0b2040 msgr2=0x7f502c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:47.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.137+0000 7f5033400640 1 -- 172.21.15.50:0/1011969858 shutdown_connections 2024-09-21T06:13:47.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.137+0000 7f5033400640 1 -- 172.21.15.50:0/1011969858 wait complete. 2024-09-21T06:13:47.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f5033400640 1 Processor -- start 2024-09-21T06:13:47.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f5033400640 1 -- start start 2024-09-21T06:13:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f5033400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f502c072100 0x7f502c0b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f5033400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f502c073a90 0x7f502c14db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f5033400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f502c14e0d0 0x7f502c0ac090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f5033400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f502c169ed0 con 0x7f502c073a90 2024-09-21T06:13:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f5033400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f502c169d50 con 0x7f502c14e0d0 2024-09-21T06:13:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f5033400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f502c16a050 con 0x7f502c072100 2024-09-21T06:13:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f502be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f502c073a90 0x7f502c14db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f502be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f502c073a90 0x7f502c14db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33076/0 (socket says 172.21.15.50:33076) 2024-09-21T06:13:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f502be00640 1 -- 172.21.15.50:0/2254211020 learned_addr learned my addr 172.21.15.50:0/2254211020 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:47.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f5031400640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f502c14e0d0 0x7f502c0ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:47.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f5030a00640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f502c072100 0x7f502c0b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:47.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f502be00640 1 -- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f502c072100 msgr2=0x7f502c0b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:47.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f502be00640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f502c072100 0x7f502c0b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f502be00640 1 -- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f502c14e0d0 msgr2=0x7f502c0ac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:47.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f502be00640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f502c14e0d0 0x7f502c0ac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f502be00640 1 -- 172.21.15.50:0/2254211020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502c152130 con 0x7f502c073a90 2024-09-21T06:13:47.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.141+0000 7f502be00640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f502c073a90 0x7f502c14db90 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f5018000c10 tx=0x7f50180a6f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:47.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.145+0000 7f5028c00640 1 -- 172.21.15.50:0/2254211020 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018005040 con 0x7f502c073a90 2024-09-21T06:13:47.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.145+0000 7f5028c00640 1 -- 172.21.15.50:0/2254211020 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50180051e0 con 0x7f502c073a90 2024-09-21T06:13:47.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.145+0000 7f5028c00640 1 -- 172.21.15.50:0/2254211020 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50180055d0 con 0x7f502c073a90 2024-09-21T06:13:47.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.145+0000 7f5033400640 1 -- 172.21.15.50:0/2254211020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502c0aa600 con 0x7f502c073a90 2024-09-21T06:13:47.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.145+0000 7f5033400640 1 -- 172.21.15.50:0/2254211020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f502c0aabc0 con 0x7f502c073a90 2024-09-21T06:13:47.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.145+0000 7f5033400640 1 -- 172.21.15.50:0/2254211020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f502c155b50 con 0x7f502c073a90 2024-09-21T06:13:47.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.145+0000 7f5028c00640 1 -- 172.21.15.50:0/2254211020 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f50180046d0 con 0x7f502c073a90 2024-09-21T06:13:47.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.145+0000 7f5028c00640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f500c0759f0 0x7f500c077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:47.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.145+0000 7f5028c00640 1 -- 172.21.15.50:0/2254211020 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5878+0+0 (secure 0 0 0) 0x7f501812d4d0 con 0x7f502c073a90 2024-09-21T06:13:47.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.149+0000 7f5030a00640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f500c0759f0 0x7f500c077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:47.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.149+0000 7f5030a00640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f500c0759f0 0x7f500c077f00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5020007fd0 tx=0x7f5020009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:47.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.149+0000 7f5028c00640 1 -- 172.21.15.50:0/2254211020 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f50180ac030 con 0x7f502c073a90 2024-09-21T06:13:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.365+0000 7f5033400640 1 -- 172.21.15.50:0/2254211020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f502c0b3740 con 0x7f502c073a90 2024-09-21T06:13:47.360 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-09-21T06:13:47.365+0000 7f0d45000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:13:47.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.369+0000 7f5028c00640 1 -- 172.21.15.50:0/2254211020 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v96) ==== 161+0+0 (secure 0 0 0) 0x7f5018016410 con 0x7f502c073a90 2024-09-21T06:13:47.363 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:13:47.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.373+0000 7f5033400640 1 -- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f500c0759f0 msgr2=0x7f500c077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:47.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.373+0000 7f5033400640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f500c0759f0 0x7f500c077f00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5020007fd0 tx=0x7f5020009040 comp rx=0 tx=0).stop 2024-09-21T06:13:47.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.373+0000 7f5033400640 1 -- 172.21.15.50:0/2254211020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f502c073a90 msgr2=0x7f502c14db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:47.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.373+0000 7f5033400640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f502c073a90 0x7f502c14db90 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f5018000c10 tx=0x7f50180a6f90 comp rx=0 tx=0).stop 2024-09-21T06:13:47.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.373+0000 7f5033400640 1 -- 172.21.15.50:0/2254211020 shutdown_connections 2024-09-21T06:13:47.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.373+0000 7f5033400640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f502c14e0d0 0x7f502c0ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.373+0000 7f5033400640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f500c0759f0 0x7f500c077f00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.373+0000 7f5033400640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f502c073a90 0x7f502c14db90 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.373+0000 7f5033400640 1 --2- 172.21.15.50:0/2254211020 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f502c072100 0x7f502c0b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.373+0000 7f5033400640 1 -- 172.21.15.50:0/2254211020 >> 172.21.15.50:0/2254211020 conn(0x7f502c0b2040 msgr2=0x7f502c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:47.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.373+0000 7f5033400640 1 -- 172.21.15.50:0/2254211020 shutdown_connections 2024-09-21T06:13:47.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.373+0000 7f5033400640 1 -- 172.21.15.50:0/2254211020 wait complete. 2024-09-21T06:13:47.388 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:13:47.505 INFO:tasks.ceph.mds.d.smithi088.stdout:starting mds.d at 2024-09-21T06:13:47.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.553+0000 7f4297800640 1 -- 172.21.15.50:0/152812128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429015d280 msgr2=0x7f429015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:47.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.553+0000 7f4297800640 1 --2- 172.21.15.50:0/152812128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429015d280 0x7f429015d690 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f427c009900 tx=0x7f427c01c8e0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.553+0000 7f4297800640 1 -- 172.21.15.50:0/152812128 shutdown_connections 2024-09-21T06:13:47.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.553+0000 7f4297800640 1 --2- 172.21.15.50:0/152812128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429015d280 0x7f429015d690 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.553+0000 7f4297800640 1 -- 172.21.15.50:0/152812128 >> 172.21.15.50:0/152812128 conn(0x7f42900b2040 msgr2=0x7f42900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:47.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.553+0000 7f4297800640 1 -- 172.21.15.50:0/152812128 shutdown_connections 2024-09-21T06:13:47.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.553+0000 7f4297800640 1 -- 172.21.15.50:0/152812128 wait complete. 2024-09-21T06:13:47.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4297800640 1 Processor -- start 2024-09-21T06:13:47.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4297800640 1 -- start start 2024-09-21T06:13:47.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4297800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429015d280 0x7f429013cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:47.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4297800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429015dbd0 0x7f4290163a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:47.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4297800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f429015e8c0 0x7f4290163f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:47.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4297800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4290169d00 con 0x7f429015d280 2024-09-21T06:13:47.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4297800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4290169b80 con 0x7f429015dbd0 2024-09-21T06:13:47.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4297800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4290169e80 con 0x7f429015e8c0 2024-09-21T06:13:47.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4294e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429015d280 0x7f429013cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:47.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4294e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429015d280 0x7f429013cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33086/0 (socket says 172.21.15.50:33086) 2024-09-21T06:13:47.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4294e00640 1 -- 172.21.15.50:0/1886926936 learned_addr learned my addr 172.21.15.50:0/1886926936 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:47.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f428fe00640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429015dbd0 0x7f4290163a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:47.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4295800640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f429015e8c0 0x7f4290163f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:47.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4294e00640 1 -- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f429015e8c0 msgr2=0x7f4290163f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:47.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4294e00640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f429015e8c0 0x7f4290163f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4294e00640 1 -- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429015dbd0 msgr2=0x7f4290163a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:47.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4294e00640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429015dbd0 0x7f4290163a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4294e00640 1 -- 172.21.15.50:0/1886926936 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42901629a0 con 0x7f429015d280 2024-09-21T06:13:47.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f428fe00640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429015dbd0 0x7f4290163a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:47.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4294e00640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429015d280 0x7f429013cec0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f428400c450 tx=0x7f428400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:47.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f428cc00640 1 -- 172.21.15.50:0/1886926936 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840052d0 con 0x7f429015d280 2024-09-21T06:13:47.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f428cc00640 1 -- 172.21.15.50:0/1886926936 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4284005470 con 0x7f429015d280 2024-09-21T06:13:47.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4297800640 1 -- 172.21.15.50:0/1886926936 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4290162c80 con 0x7f429015d280 2024-09-21T06:13:47.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f428cc00640 1 -- 172.21.15.50:0/1886926936 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284012870 con 0x7f429015d280 2024-09-21T06:13:47.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.557+0000 7f4297800640 1 -- 172.21.15.50:0/1886926936 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4290163240 con 0x7f429015d280 2024-09-21T06:13:47.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.561+0000 7f4297800640 1 -- 172.21.15.50:0/1886926936 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f429015d690 con 0x7f429015d280 2024-09-21T06:13:47.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.561+0000 7f428cc00640 1 -- 172.21.15.50:0/1886926936 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f4284003600 con 0x7f429015d280 2024-09-21T06:13:47.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.561+0000 7f428cc00640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f42700759f0 0x7f4270077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:47.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.561+0000 7f428cc00640 1 -- 172.21.15.50:0/1886926936 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5936+0+0 (secure 0 0 0) 0x7f4284092fd0 con 0x7f429015d280 2024-09-21T06:13:47.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.561+0000 7f428fe00640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f42700759f0 0x7f4270077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:47.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.565+0000 7f428fe00640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f42700759f0 0x7f4270077f00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f427c0af000 tx=0x7f427c01b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:47.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.565+0000 7f428cc00640 1 -- 172.21.15.50:0/1886926936 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f428405f420 con 0x7f429015d280 2024-09-21T06:13:47.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:47.777+0000 7f4297800640 1 -- 172.21.15.50:0/1886926936 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f429013d400 con 0x7f429015d280 2024-09-21T06:13:48.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.369+0000 7f428cc00640 1 -- 172.21.15.50:0/1886926936 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v97) ==== 104+0+0 (secure 0 0 0) 0x7f4284063e50 con 0x7f429015d280 2024-09-21T06:13:48.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.373+0000 7f4297800640 1 -- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f42700759f0 msgr2=0x7f4270077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:48.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.373+0000 7f4297800640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f42700759f0 0x7f4270077f00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f427c0af000 tx=0x7f427c01b120 comp rx=0 tx=0).stop 2024-09-21T06:13:48.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.373+0000 7f4297800640 1 -- 172.21.15.50:0/1886926936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429015d280 msgr2=0x7f429013cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:48.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.373+0000 7f4297800640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429015d280 0x7f429013cec0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f428400c450 tx=0x7f428400c910 comp rx=0 tx=0).stop 2024-09-21T06:13:48.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.373+0000 7f4297800640 1 -- 172.21.15.50:0/1886926936 shutdown_connections 2024-09-21T06:13:48.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.373+0000 7f4297800640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f429015e8c0 0x7f4290163f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.373+0000 7f4297800640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429015dbd0 0x7f4290163a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.373+0000 7f4297800640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f42700759f0 0x7f4270077f00 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.373+0000 7f4297800640 1 --2- 172.21.15.50:0/1886926936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f429015d280 0x7f429013cec0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.373+0000 7f4297800640 1 -- 172.21.15.50:0/1886926936 >> 172.21.15.50:0/1886926936 conn(0x7f42900b2040 msgr2=0x7f4290075f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:48.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.373+0000 7f4297800640 1 -- 172.21.15.50:0/1886926936 shutdown_connections 2024-09-21T06:13:48.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.373+0000 7f4297800640 1 -- 172.21.15.50:0/1886926936 wait complete. 2024-09-21T06:13:48.393 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:13:48.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.565+0000 7f4a92200640 1 -- 172.21.15.50:0/2052836624 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a8c069500 msgr2=0x7f4a8c15c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:48.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.565+0000 7f4a92200640 1 --2- 172.21.15.50:0/2052836624 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a8c069500 0x7f4a8c15c620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009840 tx=0x7f4a7401c760 comp rx=0 tx=0).stop 2024-09-21T06:13:48.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.565+0000 7f4a92200640 1 -- 172.21.15.50:0/2052836624 shutdown_connections 2024-09-21T06:13:48.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.565+0000 7f4a92200640 1 --2- 172.21.15.50:0/2052836624 >> v2:172.21.15.88:3300/0 conn(0x7f4a8c0d07a0 0x7f4a8c15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.565+0000 7f4a92200640 1 --2- 172.21.15.50:0/2052836624 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a8c069500 0x7f4a8c15c620 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.565+0000 7f4a92200640 1 -- 172.21.15.50:0/2052836624 >> 172.21.15.50:0/2052836624 conn(0x7f4a8c0b35f0 msgr2=0x7f4a8c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:48.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.565+0000 7f4a92200640 1 -- 172.21.15.50:0/2052836624 shutdown_connections 2024-09-21T06:13:48.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.565+0000 7f4a92200640 1 -- 172.21.15.50:0/2052836624 wait complete. 2024-09-21T06:13:48.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a92200640 1 Processor -- start 2024-09-21T06:13:48.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a92200640 1 -- start start 2024-09-21T06:13:48.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a92200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a8c071940 0x7f4a8c1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:48.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a92200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a8c0d07a0 0x7f4a8c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:48.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a92200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a8c150150 0x7f4a8c15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:48.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a92200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c1699b0 con 0x7f4a8c071940 2024-09-21T06:13:48.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a92200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c169830 con 0x7f4a8c0d07a0 2024-09-21T06:13:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a92200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8c169b30 con 0x7f4a8c150150 2024-09-21T06:13:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a8b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a8c071940 0x7f4a8c1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a8b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a8c071940 0x7f4a8c1583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33096/0 (socket says 172.21.15.50:33096) 2024-09-21T06:13:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a8b400640 1 -- 172.21.15.50:0/2486039498 learned_addr learned my addr 172.21.15.50:0/2486039498 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a8be00640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a8c150150 0x7f4a8c15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a82a00640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a8c0d07a0 0x7f4a8c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a8b400640 1 -- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a8c150150 msgr2=0x7f4a8c15a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a8b400640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a8c150150 0x7f4a8c15a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a8b400640 1 -- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a8c0d07a0 msgr2=0x7f4a8c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a8b400640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a8c0d07a0 0x7f4a8c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a8b400640 1 -- 172.21.15.50:0/2486039498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c14f250 con 0x7f4a8c071940 2024-09-21T06:13:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a82a00640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a8c0d07a0 0x7f4a8c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:48.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a8b400640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a8c071940 0x7f4a8c1583b0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00c730 tx=0x7f4a7c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:48.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a83e00640 1 -- 172.21.15.50:0/2486039498 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c0068a0 con 0x7f4a8c071940 2024-09-21T06:13:48.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a83e00640 1 -- 172.21.15.50:0/2486039498 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a7c006a40 con 0x7f4a8c071940 2024-09-21T06:13:48.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a92200640 1 -- 172.21.15.50:0/2486039498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8c14dcf0 con 0x7f4a8c071940 2024-09-21T06:13:48.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a83e00640 1 -- 172.21.15.50:0/2486039498 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c007030 con 0x7f4a8c071940 2024-09-21T06:13:48.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.569+0000 7f4a92200640 1 -- 172.21.15.50:0/2486039498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8c14e2b0 con 0x7f4a8c071940 2024-09-21T06:13:48.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.573+0000 7f4a92200640 1 -- 172.21.15.50:0/2486039498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a8c15c620 con 0x7f4a8c071940 2024-09-21T06:13:48.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.573+0000 7f4a83e00640 1 -- 172.21.15.50:0/2486039498 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f4a7c0071d0 con 0x7f4a8c071940 2024-09-21T06:13:48.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.573+0000 7f4a83e00640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a680759f0 0x7f4a68077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:48.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.573+0000 7f4a83e00640 1 -- 172.21.15.50:0/2486039498 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5936+0+0 (secure 0 0 0) 0x7f4a7c092f60 con 0x7f4a8c071940 2024-09-21T06:13:48.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.573+0000 7f4a82a00640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a680759f0 0x7f4a68077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:48.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.577+0000 7f4a82a00640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a680759f0 0x7f4a68077f00 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4a740af000 tx=0x7f4a7401afa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:48.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.577+0000 7f4a83e00640 1 -- 172.21.15.50:0/2486039498 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4a8c161130 con 0x7f4a8c071940 2024-09-21T06:13:48.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:48.757+0000 7f4a92200640 1 -- 172.21.15.50:0/2486039498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4a8c155350 con 0x7f4a8c071940 2024-09-21T06:13:49.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.532+0000 7f4a83e00640 1 -- 172.21.15.50:0/2486039498 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v148) ==== 181+0+0 (secure 0 0 0) 0x7f4a7c05fa80 con 0x7f4a8c071940 2024-09-21T06:13:49.530 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:13:49.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.536+0000 7f4a92200640 1 -- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a680759f0 msgr2=0x7f4a68077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.536+0000 7f4a92200640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a680759f0 0x7f4a68077f00 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4a740af000 tx=0x7f4a7401afa0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.536+0000 7f4a92200640 1 -- 172.21.15.50:0/2486039498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a8c071940 msgr2=0x7f4a8c1583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.536+0000 7f4a92200640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a8c071940 0x7f4a8c1583b0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00c730 tx=0x7f4a7c00cbf0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.536+0000 7f4a92200640 1 -- 172.21.15.50:0/2486039498 shutdown_connections 2024-09-21T06:13:49.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.536+0000 7f4a92200640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f4a8c150150 0x7f4a8c15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.536+0000 7f4a92200640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a8c0d07a0 0x7f4a8c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.536+0000 7f4a92200640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f4a680759f0 0x7f4a68077f00 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.536+0000 7f4a92200640 1 --2- 172.21.15.50:0/2486039498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a8c071940 0x7f4a8c1583b0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.536+0000 7f4a92200640 1 -- 172.21.15.50:0/2486039498 >> 172.21.15.50:0/2486039498 conn(0x7f4a8c0b35f0 msgr2=0x7f4a8c155b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:49.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.536+0000 7f4a92200640 1 -- 172.21.15.50:0/2486039498 shutdown_connections 2024-09-21T06:13:49.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.536+0000 7f4a92200640 1 -- 172.21.15.50:0/2486039498 wait complete. 2024-09-21T06:13:49.559 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:13:49.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.728+0000 7fa42ba00640 1 -- 172.21.15.50:0/3634721237 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa424073a90 msgr2=0x7fa424073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.728+0000 7fa42ba00640 1 --2- 172.21.15.50:0/3634721237 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa424073a90 0x7fa424073ea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009bc0 tx=0x7fa41c01cba0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.728+0000 7fa42ba00640 1 -- 172.21.15.50:0/3634721237 shutdown_connections 2024-09-21T06:13:49.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.728+0000 7fa42ba00640 1 --2- 172.21.15.50:0/3634721237 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa424073a90 0x7fa424073ea0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.728+0000 7fa42ba00640 1 --2- 172.21.15.50:0/3634721237 >> v2:172.21.15.88:3300/0 conn(0x7fa4240b54f0 0x7fa424155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.728+0000 7fa42ba00640 1 -- 172.21.15.50:0/3634721237 >> 172.21.15.50:0/3634721237 conn(0x7fa4240b2040 msgr2=0x7fa4240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:49.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.728+0000 7fa42ba00640 1 -- 172.21.15.50:0/3634721237 shutdown_connections 2024-09-21T06:13:49.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.728+0000 7fa42ba00640 1 -- 172.21.15.50:0/3634721237 wait complete. 2024-09-21T06:13:49.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa42ba00640 1 Processor -- start 2024-09-21T06:13:49.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa42ba00640 1 -- start start 2024-09-21T06:13:49.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa42ba00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa424072100 0x7fa42414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa42ba00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa424073a90 0x7fa42413caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa42ba00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa42413d030 0x7fa4241f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa42ba00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa424168e90 con 0x7fa424072100 2024-09-21T06:13:49.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa42ba00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa424168d10 con 0x7fa424073a90 2024-09-21T06:13:49.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa42ba00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa424169010 con 0x7fa42413d030 2024-09-21T06:13:49.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa429000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa424072100 0x7fa42414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa429000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa424072100 0x7fa42414e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33104/0 (socket says 172.21.15.50:33104) 2024-09-21T06:13:49.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa429000640 1 -- 172.21.15.50:0/655521244 learned_addr learned my addr 172.21.15.50:0/655521244 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:49.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa423e00640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa424073a90 0x7fa42413caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa429a00640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa42413d030 0x7fa4241f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa429000640 1 -- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa42413d030 msgr2=0x7fa4241f8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa429000640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa42413d030 0x7fa4241f8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa429000640 1 -- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa424073a90 msgr2=0x7fa42413caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa429000640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa424073a90 0x7fa42413caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.732+0000 7fa429000640 1 -- 172.21.15.50:0/655521244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42413da50 con 0x7fa424072100 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.736+0000 7fa423e00640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa424073a90 0x7fa42413caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.736+0000 7fa429000640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa424072100 0x7fa42414e050 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c430 tx=0x7fa41800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.736+0000 7fa420c00640 1 -- 172.21.15.50:0/655521244 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418011790 con 0x7fa424072100 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.736+0000 7fa42ba00640 1 -- 172.21.15.50:0/655521244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42413d770 con 0x7fa424072100 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.736+0000 7fa420c00640 1 -- 172.21.15.50:0/655521244 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa418017070 con 0x7fa424072100 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.736+0000 7fa420c00640 1 -- 172.21.15.50:0/655521244 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418011ce0 con 0x7fa424072100 2024-09-21T06:13:49.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.736+0000 7fa42ba00640 1 -- 172.21.15.50:0/655521244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42414d850 con 0x7fa424072100 2024-09-21T06:13:49.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.736+0000 7fa42ba00640 1 -- 172.21.15.50:0/655521244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa424073ea0 con 0x7fa424072100 2024-09-21T06:13:49.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.736+0000 7fa420c00640 1 -- 172.21.15.50:0/655521244 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fa41801c460 con 0x7fa424072100 2024-09-21T06:13:49.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.740+0000 7fa420c00640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4040759f0 0x7fa404077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.740+0000 7fa420c00640 1 -- 172.21.15.50:0/655521244 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5470+0+0 (secure 0 0 0) 0x7fa4180934a0 con 0x7fa424072100 2024-09-21T06:13:49.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.740+0000 7fa423e00640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4040759f0 0x7fa404077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.740+0000 7fa423e00640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4040759f0 0x7fa404077f00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa4240b59c0 tx=0x7fa4100091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:49.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.744+0000 7fa420c00640 1 -- 172.21.15.50:0/655521244 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa41805ff60 con 0x7fa424072100 2024-09-21T06:13:49.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:49.920+0000 7fa42ba00640 1 -- 172.21.15.50:0/655521244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa424075120 con 0x7fa424072100 2024-09-21T06:13:50.136 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:13:50.140 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:13:50.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.888+0000 7fa420c00640 1 -- 172.21.15.50:0/655521244 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v149) ==== 169+0+0 (secure 0 0 0) 0x7fa41805f860 con 0x7fa424072100 2024-09-21T06:13:50.886 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-09-21T06:13:50.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.892+0000 7fa42ba00640 1 -- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4040759f0 msgr2=0x7fa404077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:50.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.892+0000 7fa42ba00640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4040759f0 0x7fa404077f00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa4240b59c0 tx=0x7fa4100091e0 comp rx=0 tx=0).stop 2024-09-21T06:13:50.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.892+0000 7fa42ba00640 1 -- 172.21.15.50:0/655521244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa424072100 msgr2=0x7fa42414e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:50.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.892+0000 7fa42ba00640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa424072100 0x7fa42414e050 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c430 tx=0x7fa41800c8f0 comp rx=0 tx=0).stop 2024-09-21T06:13:50.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.896+0000 7fa42ba00640 1 -- 172.21.15.50:0/655521244 shutdown_connections 2024-09-21T06:13:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.896+0000 7fa42ba00640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa42413d030 0x7fa4241f8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.896+0000 7fa42ba00640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa424073a90 0x7fa42413caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.896+0000 7fa42ba00640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4040759f0 0x7fa404077f00 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.896+0000 7fa42ba00640 1 --2- 172.21.15.50:0/655521244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa424072100 0x7fa42414e050 secure :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c430 tx=0x7fa41800c8f0 comp rx=0 tx=0).stop 2024-09-21T06:13:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.896+0000 7fa42ba00640 1 -- 172.21.15.50:0/655521244 >> 172.21.15.50:0/655521244 conn(0x7fa4240b2040 msgr2=0x7fa4240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.896+0000 7fa42ba00640 1 -- 172.21.15.50:0/655521244 shutdown_connections 2024-09-21T06:13:50.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:50.896+0000 7fa42ba00640 1 -- 172.21.15.50:0/655521244 wait complete. 2024-09-21T06:13:50.921 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T06:13:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.088+0000 7fc57a000640 1 -- 172.21.15.50:0/1879735651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5741536f0 msgr2=0x7fc57415d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.088+0000 7fc57a000640 1 --2- 172.21.15.50:0/1879735651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5741536f0 0x7fc57415d480 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fc5740ced40 tx=0x7fc56801c830 comp rx=0 tx=0).stop 2024-09-21T06:13:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.088+0000 7fc57a000640 1 -- 172.21.15.50:0/1879735651 shutdown_connections 2024-09-21T06:13:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.088+0000 7fc57a000640 1 --2- 172.21.15.50:0/1879735651 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc574160470 0x7fc574074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.088+0000 7fc57a000640 1 --2- 172.21.15.50:0/1879735651 >> v2:172.21.15.88:3300/0 conn(0x7fc574160060 0x7fc574074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.088+0000 7fc57a000640 1 --2- 172.21.15.50:0/1879735651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5741536f0 0x7fc57415d480 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.088+0000 7fc57a000640 1 -- 172.21.15.50:0/1879735651 >> 172.21.15.50:0/1879735651 conn(0x7fc5740b4560 msgr2=0x7fc5740d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.088+0000 7fc57a000640 1 -- 172.21.15.50:0/1879735651 shutdown_connections 2024-09-21T06:13:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.088+0000 7fc57a000640 1 -- 172.21.15.50:0/1879735651 wait complete. 2024-09-21T06:13:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc57a000640 1 Processor -- start 2024-09-21T06:13:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc57a000640 1 -- start start 2024-09-21T06:13:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc57a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5741536f0 0x7fc574149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc57a000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc574160060 0x7fc57414ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc57a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc574160470 0x7fc57414cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc57a000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc574071db0 con 0x7fc574160470 2024-09-21T06:13:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc57a000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc574071c30 con 0x7fc5741536f0 2024-09-21T06:13:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc57a000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc574071f30 con 0x7fc574160060 2024-09-21T06:13:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc579600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc574160470 0x7fc57414cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc579600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc574160470 0x7fc57414cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33116/0 (socket says 172.21.15.50:33116) 2024-09-21T06:13:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc579600640 1 -- 172.21.15.50:0/1988802391 learned_addr learned my addr 172.21.15.50:0/1988802391 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc573e00640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc574160060 0x7fc57414ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc578c00640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5741536f0 0x7fc574149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc579600640 1 -- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc574160060 msgr2=0x7fc57414ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc579600640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc574160060 0x7fc57414ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc579600640 1 -- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5741536f0 msgr2=0x7fc574149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc579600640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5741536f0 0x7fc574149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc579600640 1 -- 172.21.15.50:0/1988802391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5741f6920 con 0x7fc574160470 2024-09-21T06:13:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc579600640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc574160470 0x7fc57414cf70 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00e940 tx=0x7fc56c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc570c00640 1 -- 172.21.15.50:0/1988802391 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c004d30 con 0x7fc574160470 2024-09-21T06:13:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc570c00640 1 -- 172.21.15.50:0/1988802391 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc56c00cb20 con 0x7fc574160470 2024-09-21T06:13:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc570c00640 1 -- 172.21.15.50:0/1988802391 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c005020 con 0x7fc574160470 2024-09-21T06:13:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc57a000640 1 -- 172.21.15.50:0/1988802391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5741f6c00 con 0x7fc574160470 2024-09-21T06:13:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.092+0000 7fc57a000640 1 -- 172.21.15.50:0/1988802391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5741f71c0 con 0x7fc574160470 2024-09-21T06:13:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.096+0000 7fc57a000640 1 -- 172.21.15.50:0/1988802391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5740724d0 con 0x7fc574160470 2024-09-21T06:13:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.096+0000 7fc570c00640 1 -- 172.21.15.50:0/1988802391 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fc56c0051c0 con 0x7fc574160470 2024-09-21T06:13:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.096+0000 7fc570c00640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc5540759f0 0x7fc554077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.096+0000 7fc570c00640 1 -- 172.21.15.50:0/1988802391 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7fc56c093d60 con 0x7fc574160470 2024-09-21T06:13:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.096+0000 7fc578c00640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc5540759f0 0x7fc554077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.100+0000 7fc578c00640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc5540759f0 0x7fc554077f00 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc5680af000 tx=0x7fc56801b070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.100+0000 7fc570c00640 1 -- 172.21.15.50:0/1988802391 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc56c016030 con 0x7fc574160470 2024-09-21T06:13:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.292+0000 7fc57a000640 1 -- 172.21.15.50:0/1988802391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fc574152ee0 con 0x7fc574160470 2024-09-21T06:13:51.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.296+0000 7fc570c00640 1 -- 172.21.15.50:0/1988802391 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v61) ==== 71+0+0 (secure 0 0 0) 0x7fc56c060220 con 0x7fc574160470 2024-09-21T06:13:51.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.300+0000 7fc57a000640 1 -- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc5540759f0 msgr2=0x7fc554077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.300+0000 7fc57a000640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc5540759f0 0x7fc554077f00 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc5680af000 tx=0x7fc56801b070 comp rx=0 tx=0).stop 2024-09-21T06:13:51.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.300+0000 7fc57a000640 1 -- 172.21.15.50:0/1988802391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc574160470 msgr2=0x7fc57414cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.300+0000 7fc57a000640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc574160470 0x7fc57414cf70 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00e940 tx=0x7fc56c00ee00 comp rx=0 tx=0).stop 2024-09-21T06:13:51.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.300+0000 7fc57a000640 1 -- 172.21.15.50:0/1988802391 shutdown_connections 2024-09-21T06:13:51.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.300+0000 7fc57a000640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc5540759f0 0x7fc554077f00 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.300+0000 7fc57a000640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc574160470 0x7fc57414cf70 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.300+0000 7fc57a000640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc574160060 0x7fc57414ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.300+0000 7fc57a000640 1 --2- 172.21.15.50:0/1988802391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5741536f0 0x7fc574149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.300+0000 7fc57a000640 1 -- 172.21.15.50:0/1988802391 >> 172.21.15.50:0/1988802391 conn(0x7fc5740b4560 msgr2=0x7fc5740b6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:51.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.300+0000 7fc57a000640 1 -- 172.21.15.50:0/1988802391 shutdown_connections 2024-09-21T06:13:51.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.300+0000 7fc57a000640 1 -- 172.21.15.50:0/1988802391 wait complete. 2024-09-21T06:13:51.317 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-21T06:13:51.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.480+0000 7f801dc00640 1 -- 172.21.15.50:0/866950554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80180b4de0 msgr2=0x7f80180b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.480+0000 7f801dc00640 1 --2- 172.21.15.50:0/866950554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80180b4de0 0x7f80180b51f0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f8018070710 tx=0x7f801001cae0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.480+0000 7f801dc00640 1 -- 172.21.15.50:0/866950554 shutdown_connections 2024-09-21T06:13:51.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.480+0000 7f801dc00640 1 --2- 172.21.15.50:0/866950554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80180b4de0 0x7f80180b51f0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.480+0000 7f801dc00640 1 --2- 172.21.15.50:0/866950554 >> v2:172.21.15.88:3300/0 conn(0x7f80181527b0 0x7f801815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.480+0000 7f801dc00640 1 -- 172.21.15.50:0/866950554 >> 172.21.15.50:0/866950554 conn(0x7f80180b2040 msgr2=0x7f80180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:51.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.480+0000 7f801dc00640 1 -- 172.21.15.50:0/866950554 shutdown_connections 2024-09-21T06:13:51.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.480+0000 7f801dc00640 1 -- 172.21.15.50:0/866950554 wait complete. 2024-09-21T06:13:51.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f801dc00640 1 Processor -- start 2024-09-21T06:13:51.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f801dc00640 1 -- start start 2024-09-21T06:13:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f801dc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80180b4de0 0x7f80181fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f801dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80180b5c70 0x7f80180cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f801dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80181527b0 0x7f8018154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f801dc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8018168970 con 0x7f80180b5c70 2024-09-21T06:13:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f801dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f80181687f0 con 0x7f80181527b0 2024-09-21T06:13:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f801dc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f8018168af0 con 0x7f80180b4de0 2024-09-21T06:13:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f800e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80180b5c70 0x7f80180cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f800e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80180b5c70 0x7f80180cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33128/0 (socket says 172.21.15.50:33128) 2024-09-21T06:13:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f800e000640 1 -- 172.21.15.50:0/367481508 learned_addr learned my addr 172.21.15.50:0/367481508 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f8016a00640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80180b4de0 0x7f80181fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f8017400640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80181527b0 0x7f8018154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f800e000640 1 -- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80180b4de0 msgr2=0x7f80181fc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f800e000640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80180b4de0 0x7f80181fc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f800e000640 1 -- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80181527b0 msgr2=0x7f8018154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f800e000640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80181527b0 0x7f8018154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f800e000640 1 -- 172.21.15.50:0/367481508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80181f9930 con 0x7f80180b5c70 2024-09-21T06:13:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f800e000640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80180b5c70 0x7f80180cf9e0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f800000ca20 tx=0x7f800000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f800f400640 1 -- 172.21.15.50:0/367481508 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000007c70 con 0x7f80180b5c70 2024-09-21T06:13:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f800f400640 1 -- 172.21.15.50:0/367481508 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f800000f450 con 0x7f80180b5c70 2024-09-21T06:13:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f800f400640 1 -- 172.21.15.50:0/367481508 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800000f8a0 con 0x7f80180b5c70 2024-09-21T06:13:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f801dc00640 1 -- 172.21.15.50:0/367481508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80181f8db0 con 0x7f80180b5c70 2024-09-21T06:13:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.484+0000 7f801dc00640 1 -- 172.21.15.50:0/367481508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80181f9370 con 0x7f80180b5c70 2024-09-21T06:13:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.488+0000 7f801dc00640 1 -- 172.21.15.50:0/367481508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8018142420 con 0x7f80180b5c70 2024-09-21T06:13:51.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.488+0000 7f800f400640 1 -- 172.21.15.50:0/367481508 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f800000fa40 con 0x7f80180b5c70 2024-09-21T06:13:51.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.488+0000 7f800f400640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ff40759f0 0x7f7ff4077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.488+0000 7f800f400640 1 -- 172.21.15.50:0/367481508 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f8000093c90 con 0x7f80180b5c70 2024-09-21T06:13:51.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.488+0000 7f8016a00640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ff40759f0 0x7f7ff4077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.492+0000 7f8016a00640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ff40759f0 0x7f7ff4077f00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f80080098e0 tx=0x7f8008009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:51.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.492+0000 7f800f400640 1 -- 172.21.15.50:0/367481508 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8018142420 con 0x7f80180b5c70 2024-09-21T06:13:51.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.732+0000 7f801dc00640 1 -- 172.21.15.50:0/367481508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7f80181f8b40 con 0x7f80180b5c70 2024-09-21T06:13:51.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.896+0000 7f800f400640 1 -- 172.21.15.50:0/367481508 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v776) ==== 186+0+0 (secure 0 0 0) 0x7f8000014030 con 0x7f80180b5c70 2024-09-21T06:13:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.900+0000 7f801dc00640 1 -- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ff40759f0 msgr2=0x7f7ff4077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.900+0000 7f801dc00640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ff40759f0 0x7f7ff4077f00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f80080098e0 tx=0x7f8008009060 comp rx=0 tx=0).stop 2024-09-21T06:13:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.900+0000 7f801dc00640 1 -- 172.21.15.50:0/367481508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80180b5c70 msgr2=0x7f80180cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.900+0000 7f801dc00640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80180b5c70 0x7f80180cf9e0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f800000ca20 tx=0x7f800000cee0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.900+0000 7f801dc00640 1 -- 172.21.15.50:0/367481508 shutdown_connections 2024-09-21T06:13:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.900+0000 7f801dc00640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80181527b0 0x7f8018154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.900+0000 7f801dc00640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f7ff40759f0 0x7f7ff4077f00 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.900+0000 7f801dc00640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80180b5c70 0x7f80180cf9e0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.900+0000 7f801dc00640 1 --2- 172.21.15.50:0/367481508 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f80180b4de0 0x7f80181fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.900+0000 7f801dc00640 1 -- 172.21.15.50:0/367481508 >> 172.21.15.50:0/367481508 conn(0x7f80180b2040 msgr2=0x7f80180b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.900+0000 7f801dc00640 1 -- 172.21.15.50:0/367481508 shutdown_connections 2024-09-21T06:13:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:51.900+0000 7f801dc00640 1 -- 172.21.15.50:0/367481508 wait complete. 2024-09-21T06:13:51.918 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone ... ok 2024-09-21T06:13:51.918 INFO:tasks.cephfs_test_runner: 2024-09-21T06:13:51.918 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-21T06:13:51.918 INFO:tasks.cephfs_test_runner:Ran 9 tests in 1014.204s 2024-09-21T06:13:51.918 INFO:tasks.cephfs_test_runner: 2024-09-21T06:13:51.918 INFO:tasks.cephfs_test_runner:OK 2024-09-21T06:13:51.918 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-21T06:13:51.931 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T06:13:51.968 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T06:13:51.968 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:13:51.968 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:13:51.977 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:13:51.977 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-21T06:13:51.977 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:13:52.026 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6221 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T06:13:52.026 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:13:52.026 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:13:52.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:13:52.036 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T06:13:52.036 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:13:52.090 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7237 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T06:13:52.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:52.090 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T06:13:52.099 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:13:52.099 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T06:13:52.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:52.100 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:13:52.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:52.165 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T06:13:52.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:52.569 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-09-21T06:13:52.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:52.698 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-21T06:13:52.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:52.709 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-09-21T06:13:53.136 INFO:teuthology.orchestra.run.smithi050.stderr:Cannot find device "ceph-brx" 2024-09-21T06:13:53.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:13:53.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:53.139 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T06:13:53.149 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T06:13:53.149 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T06:13:53.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:53.150 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T06:13:53.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:53.215 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T06:13:53.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:53.686 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-09-21T06:13:53.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:53.825 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T06:13:53.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:13:53.834 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-09-21T06:13:54.326 INFO:teuthology.orchestra.run.smithi088.stderr:Cannot find device "ceph-brx" 2024-09-21T06:13:54.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:13:54.329 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T06:13:54.344 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T06:13:54.344 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:13:54.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.520+0000 7f010e600640 1 -- 172.21.15.50:0/3432820629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f010815b2d0 msgr2=0x7f010815b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.520+0000 7f010e600640 1 --2- 172.21.15.50:0/3432820629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f010815b2d0 0x7f010815b6e0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f01080b3040 tx=0x7f00f001c8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.520+0000 7f010e600640 1 -- 172.21.15.50:0/3432820629 shutdown_connections 2024-09-21T06:13:54.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.520+0000 7f010e600640 1 --2- 172.21.15.50:0/3432820629 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f010815bc20 0x7f010815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.520+0000 7f010e600640 1 --2- 172.21.15.50:0/3432820629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f010815b2d0 0x7f010815b6e0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.520+0000 7f010e600640 1 --2- 172.21.15.50:0/3432820629 >> v2:172.21.15.88:3300/0 conn(0x7f010815e8c0 0x7f010815ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.520+0000 7f010e600640 1 -- 172.21.15.50:0/3432820629 >> 172.21.15.50:0/3432820629 conn(0x7f01080b2040 msgr2=0x7f01080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:54.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.520+0000 7f010e600640 1 -- 172.21.15.50:0/3432820629 shutdown_connections 2024-09-21T06:13:54.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.520+0000 7f010e600640 1 -- 172.21.15.50:0/3432820629 wait complete. 2024-09-21T06:13:54.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f010e600640 1 Processor -- start 2024-09-21T06:13:54.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f010e600640 1 -- start start 2024-09-21T06:13:54.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f010e600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f010815b2d0 0x7f0108131020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f010e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f010815bc20 0x7f0108131560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f010e600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010815e8c0 0x7f01081f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f010e600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0108168c60 con 0x7f010815bc20 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f010e600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f010815e5d0 con 0x7f010815e8c0 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f010e600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f0108168de0 con 0x7f010815b2d0 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f00fea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f010815bc20 0x7f0108131560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f00fea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f010815bc20 0x7f0108131560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33144/0 (socket says 172.21.15.50:33144) 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f00fea00640 1 -- 172.21.15.50:0/1937225629 learned_addr learned my addr 172.21.15.50:0/1937225629 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f0107400640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f010815b2d0 0x7f0108131020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f0107e00640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010815e8c0 0x7f01081f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f00fea00640 1 -- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f010815b2d0 msgr2=0x7f0108131020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f00fea00640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f010815b2d0 0x7f0108131020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f00fea00640 1 -- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010815e8c0 msgr2=0x7f01081f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f00fea00640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010815e8c0 0x7f01081f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f00fea00640 1 -- 172.21.15.50:0/1937225629 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01080c63f0 con 0x7f010815bc20 2024-09-21T06:13:54.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f00fea00640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f010815bc20 0x7f0108131560 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f00f00b0040 tx=0x7f00f0005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:54.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f00ffe00640 1 -- 172.21.15.50:0/1937225629 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f00b2040 con 0x7f010815bc20 2024-09-21T06:13:54.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f00ffe00640 1 -- 172.21.15.50:0/1937225629 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00f0004330 con 0x7f010815bc20 2024-09-21T06:13:54.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f00ffe00640 1 -- 172.21.15.50:0/1937225629 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0005290 con 0x7f010815bc20 2024-09-21T06:13:54.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f010e600640 1 -- 172.21.15.50:0/1937225629 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01080c66d0 con 0x7f010815bc20 2024-09-21T06:13:54.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.524+0000 7f010e600640 1 -- 172.21.15.50:0/1937225629 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01080c6c90 con 0x7f010815bc20 2024-09-21T06:13:54.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.528+0000 7f010e600640 1 -- 172.21.15.50:0/1937225629 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010815d280 con 0x7f010815bc20 2024-09-21T06:13:54.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.528+0000 7f00ffe00640 1 -- 172.21.15.50:0/1937225629 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f00f0005430 con 0x7f010815bc20 2024-09-21T06:13:54.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.528+0000 7f00ffe00640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f00e40759f0 0x7f00e4077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.528+0000 7f00ffe00640 1 -- 172.21.15.50:0/1937225629 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f00f012d720 con 0x7f010815bc20 2024-09-21T06:13:54.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.528+0000 7f0107400640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f00e40759f0 0x7f00e4077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.532+0000 7f0107400640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f00e40759f0 0x7f00e4077f00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009b40 tx=0x7f00f80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:54.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.532+0000 7f00ffe00640 1 -- 172.21.15.50:0/1937225629 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f00f00164b0 con 0x7f010815bc20 2024-09-21T06:13:54.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.708+0000 7f010e600640 1 -- 172.21.15.50:0/1937225629 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01080b2940 con 0x7f00e40759f0 2024-09-21T06:13:54.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.708+0000 7f00ffe00640 1 -- 172.21.15.50:0/1937225629 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13567 (secure 0 0 0) 0x7f01080b2940 con 0x7f00e40759f0 2024-09-21T06:13:54.706 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:54.706 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:13:54.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.712+0000 7f010e600640 1 -- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f00e40759f0 msgr2=0x7f00e4077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.712+0000 7f010e600640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f00e40759f0 0x7f00e4077f00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009b40 tx=0x7f00f80091e0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.712+0000 7f010e600640 1 -- 172.21.15.50:0/1937225629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f010815bc20 msgr2=0x7f0108131560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.712+0000 7f010e600640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f010815bc20 0x7f0108131560 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f00f00b0040 tx=0x7f00f0005d60 comp rx=0 tx=0).stop 2024-09-21T06:13:54.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.712+0000 7f010e600640 1 -- 172.21.15.50:0/1937225629 shutdown_connections 2024-09-21T06:13:54.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.712+0000 7f010e600640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010815e8c0 0x7f01081f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.712+0000 7f010e600640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f00e40759f0 0x7f00e4077f00 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.712+0000 7f010e600640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f010815bc20 0x7f0108131560 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.712+0000 7f010e600640 1 --2- 172.21.15.50:0/1937225629 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f010815b2d0 0x7f0108131020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.712+0000 7f010e600640 1 -- 172.21.15.50:0/1937225629 >> 172.21.15.50:0/1937225629 conn(0x7f01080b2040 msgr2=0x7f01080737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:54.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.712+0000 7f010e600640 1 -- 172.21.15.50:0/1937225629 shutdown_connections 2024-09-21T06:13:54.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.712+0000 7f010e600640 1 -- 172.21.15.50:0/1937225629 wait complete. 2024-09-21T06:13:54.729 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":670,"stamp":"2024-09-21T06:13:53.271057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":1212272,"kb_used_data":19296,"kb_used_omap":12,"kb_used_meta":1192819,"kb_avail":753762448,"statfs":{"total":773094113280,"available":771852746752,"internally_reserved":0,"allocated":19759104,"data_stored":11584150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1221447216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8768022855,"num_objects":-2117,"num_object_clones":0,"num_object_copies":-4234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2117,"num_whiteouts":0,"num_read":-3618,"num_read_kb":-14393448,"num_write":-7121,"num_write_kb":-17401977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-22,"num_bytes_recovered":-92274688,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.901522"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":340,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T06:13:41.867795+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:13:41.867795+0000","last_peered":"2024-09-21T06:13:41.867795+0000","last_clean":"2024-09-21T06:13:41.867795+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:13:41.867795+0000","last_undegraded":"2024-09-21T06:13:41.867795+0000","last_fullsized":"2024-09-21T06:13:41.867795+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509653,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145920,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225920,"statfs":{"total":96636764160,"available":96487342080,"internally_reserved":0,"allocated":2408448,"data_stored":1389632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509653,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149512,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222328,"statfs":{"total":96636764160,"available":96483663872,"internally_reserved":0,"allocated":1826816,"data_stored":806846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509652,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131464,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240376,"statfs":{"total":96636764160,"available":96502145024,"internally_reserved":0,"allocated":1822720,"data_stored":804856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509653,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167556,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204284,"statfs":{"total":96636764160,"available":96465186816,"internally_reserved":0,"allocated":1818624,"data_stored":804834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509652,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149512,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222328,"statfs":{"total":96636764160,"available":96483663872,"internally_reserved":0,"allocated":1822720,"data_stored":804924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509654,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157004,"kb_used_data":3708,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":94214836,"statfs":{"total":96636764160,"available":96475992064,"internally_reserved":0,"allocated":3796992,"data_stored":2775929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":156957125},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509652,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152024,"kb_used_data":4296,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94219816,"statfs":{"total":96636764160,"available":96481091584,"internally_reserved":0,"allocated":4399104,"data_stored":3374349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542358,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159280,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212560,"statfs":{"total":96636764160,"available":96473661440,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:13:54.729 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:13:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.892+0000 7f93a4800640 1 -- 172.21.15.50:0/4001943819 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f939c0b54f0 msgr2=0x7f939c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.892+0000 7f93a4800640 1 --2- 172.21.15.50:0/4001943819 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f939c0b54f0 0x7f939c15bb60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9398009bf0 tx=0x7f939801cbb0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.892+0000 7f93a4800640 1 -- 172.21.15.50:0/4001943819 shutdown_connections 2024-09-21T06:13:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.892+0000 7f93a4800640 1 --2- 172.21.15.50:0/4001943819 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f939c0b54f0 0x7f939c15bb60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.892+0000 7f93a4800640 1 -- 172.21.15.50:0/4001943819 >> 172.21.15.50:0/4001943819 conn(0x7f939c0b2040 msgr2=0x7f939c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.892+0000 7f93a4800640 1 -- 172.21.15.50:0/4001943819 shutdown_connections 2024-09-21T06:13:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.892+0000 7f93a4800640 1 -- 172.21.15.50:0/4001943819 wait complete. 2024-09-21T06:13:54.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a4800640 1 Processor -- start 2024-09-21T06:13:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a4800640 1 -- start start 2024-09-21T06:13:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a4800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f939c0b5c70 0x7f939c151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a4800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f939c1527b0 0x7f939c0c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a4800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f939c0c57f0 0x7f939c0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a4800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f939c169bd0 con 0x7f939c0b5c70 2024-09-21T06:13:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a4800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f939c169a50 con 0x7f939c1527b0 2024-09-21T06:13:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a4800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f939c169d50 con 0x7f939c0c57f0 2024-09-21T06:13:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f939c0b5c70 0x7f939c151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f939c0b5c70 0x7f939c151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33146/0 (socket says 172.21.15.50:33146) 2024-09-21T06:13:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a1e00640 1 -- 172.21.15.50:0/3066256415 learned_addr learned my addr 172.21.15.50:0/3066256415 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a1400640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f939c1527b0 0x7f939c0c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a2800640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f939c0c57f0 0x7f939c0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a1e00640 1 -- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f939c0c57f0 msgr2=0x7f939c0cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a1e00640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f939c0c57f0 0x7f939c0cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a1e00640 1 -- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f939c1527b0 msgr2=0x7f939c0c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a1e00640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f939c1527b0 0x7f939c0c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a1e00640 1 -- 172.21.15.50:0/3066256415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c158670 con 0x7f939c0b5c70 2024-09-21T06:13:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a1e00640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f939c0b5c70 0x7f939c151e10 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f939000c430 tx=0x7f939000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f9396000640 1 -- 172.21.15.50:0/3066256415 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390017070 con 0x7f939c0b5c70 2024-09-21T06:13:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f9396000640 1 -- 172.21.15.50:0/3066256415 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93900053c0 con 0x7f939c0b5c70 2024-09-21T06:13:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a4800640 1 -- 172.21.15.50:0/3066256415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939c158950 con 0x7f939c0b5c70 2024-09-21T06:13:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f9396000640 1 -- 172.21.15.50:0/3066256415 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390003860 con 0x7f939c0b5c70 2024-09-21T06:13:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a4800640 1 -- 172.21.15.50:0/3066256415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939c158f10 con 0x7f939c0b5c70 2024-09-21T06:13:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.896+0000 7f93a4800640 1 -- 172.21.15.50:0/3066256415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f939c15bb60 con 0x7f939c0b5c70 2024-09-21T06:13:54.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.900+0000 7f9396000640 1 -- 172.21.15.50:0/3066256415 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f939001a070 con 0x7f939c0b5c70 2024-09-21T06:13:54.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.900+0000 7f9396000640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9380075a20 0x7f9380077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.900+0000 7f9396000640 1 -- 172.21.15.50:0/3066256415 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f9390092db0 con 0x7f939c0b5c70 2024-09-21T06:13:54.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.900+0000 7f93a1400640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9380075a20 0x7f9380077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.900+0000 7f93a1400640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9380075a20 0x7f9380077f30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f938c007fd0 tx=0x7f938c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:54.904+0000 7f9396000640 1 -- 172.21.15.50:0/3066256415 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9390021020 con 0x7f939c0b5c70 2024-09-21T06:13:55.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.076+0000 7f93a4800640 1 -- 172.21.15.50:0/3066256415 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f939c06b540 con 0x7f9380075a20 2024-09-21T06:13:55.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f9396000640 1 -- 172.21.15.50:0/3066256415 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13567 (secure 0 0 0) 0x7f938c0133e0 con 0x7f9380075a20 2024-09-21T06:13:55.076 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:55.076 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:13:55.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f93a4800640 1 -- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9380075a20 msgr2=0x7f9380077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f93a4800640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9380075a20 0x7f9380077f30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f938c007fd0 tx=0x7f938c009040 comp rx=0 tx=0).stop 2024-09-21T06:13:55.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f93a4800640 1 -- 172.21.15.50:0/3066256415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f939c0b5c70 msgr2=0x7f939c151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f93a4800640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f939c0b5c70 0x7f939c151e10 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f939000c430 tx=0x7f939000c8f0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f93a4800640 1 -- 172.21.15.50:0/3066256415 shutdown_connections 2024-09-21T06:13:55.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f93a4800640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f939c0c57f0 0x7f939c0cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f93a4800640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f939c1527b0 0x7f939c0c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f93a4800640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f9380075a20 0x7f9380077f30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f93a4800640 1 --2- 172.21.15.50:0/3066256415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f939c0b5c70 0x7f939c151e10 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f93a4800640 1 -- 172.21.15.50:0/3066256415 >> 172.21.15.50:0/3066256415 conn(0x7f939c0b2040 msgr2=0x7f939c159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:55.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f93a4800640 1 -- 172.21.15.50:0/3066256415 shutdown_connections 2024-09-21T06:13:55.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.080+0000 7f93a4800640 1 -- 172.21.15.50:0/3066256415 wait complete. 2024-09-21T06:13:55.098 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":670,"stamp":"2024-09-21T06:13:53.271057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":1212272,"kb_used_data":19296,"kb_used_omap":12,"kb_used_meta":1192819,"kb_avail":753762448,"statfs":{"total":773094113280,"available":771852746752,"internally_reserved":0,"allocated":19759104,"data_stored":11584150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1221447216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8768022855,"num_objects":-2117,"num_object_clones":0,"num_object_copies":-4234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2117,"num_whiteouts":0,"num_read":-3618,"num_read_kb":-14393448,"num_write":-7121,"num_write_kb":-17401977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-22,"num_bytes_recovered":-92274688,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.901522"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":340,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T06:13:41.867795+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:13:41.867795+0000","last_peered":"2024-09-21T06:13:41.867795+0000","last_clean":"2024-09-21T06:13:41.867795+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:13:41.867795+0000","last_undegraded":"2024-09-21T06:13:41.867795+0000","last_fullsized":"2024-09-21T06:13:41.867795+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509653,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145920,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225920,"statfs":{"total":96636764160,"available":96487342080,"internally_reserved":0,"allocated":2408448,"data_stored":1389632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509653,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149512,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222328,"statfs":{"total":96636764160,"available":96483663872,"internally_reserved":0,"allocated":1826816,"data_stored":806846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509652,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131464,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240376,"statfs":{"total":96636764160,"available":96502145024,"internally_reserved":0,"allocated":1822720,"data_stored":804856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509653,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167556,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204284,"statfs":{"total":96636764160,"available":96465186816,"internally_reserved":0,"allocated":1818624,"data_stored":804834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509652,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149512,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222328,"statfs":{"total":96636764160,"available":96483663872,"internally_reserved":0,"allocated":1822720,"data_stored":804924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509654,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157004,"kb_used_data":3708,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":94214836,"statfs":{"total":96636764160,"available":96475992064,"internally_reserved":0,"allocated":3796992,"data_stored":2775929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":156957125},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509652,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152024,"kb_used_data":4296,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94219816,"statfs":{"total":96636764160,"available":96481091584,"internally_reserved":0,"allocated":4399104,"data_stored":3374349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542358,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159280,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212560,"statfs":{"total":96636764160,"available":96473661440,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:13:55.099 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T06:13:55.099 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:13:55.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.260+0000 7f6baa000640 1 -- 172.21.15.50:0/3367052366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba40b4de0 msgr2=0x7f6ba40b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.260+0000 7f6baa000640 1 --2- 172.21.15.50:0/3367052366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba40b4de0 0x7f6ba40b51f0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4071cb0 tx=0x7f6b9c01c620 comp rx=0 tx=0).stop 2024-09-21T06:13:55.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.260+0000 7f6baa000640 1 -- 172.21.15.50:0/3367052366 shutdown_connections 2024-09-21T06:13:55.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.260+0000 7f6baa000640 1 --2- 172.21.15.50:0/3367052366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba40b4de0 0x7f6ba40b51f0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.260+0000 7f6baa000640 1 --2- 172.21.15.50:0/3367052366 >> v2:172.21.15.88:3300/0 conn(0x7f6ba40b5c70 0x7f6ba415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.260+0000 7f6baa000640 1 -- 172.21.15.50:0/3367052366 >> 172.21.15.50:0/3367052366 conn(0x7f6ba40b2040 msgr2=0x7f6ba40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:55.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.260+0000 7f6baa000640 1 -- 172.21.15.50:0/3367052366 shutdown_connections 2024-09-21T06:13:55.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.260+0000 7f6baa000640 1 -- 172.21.15.50:0/3367052366 wait complete. 2024-09-21T06:13:55.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6baa000640 1 Processor -- start 2024-09-21T06:13:55.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6baa000640 1 -- start start 2024-09-21T06:13:55.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6baa000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ba40b4de0 0x7f6ba414be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6baa000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba40b5c70 0x7f6ba414d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6baa000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ba41527b0 0x7f6ba415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6baa000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41688b0 con 0x7f6ba40b5c70 2024-09-21T06:13:55.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6baa000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4168730 con 0x7f6ba40b4de0 2024-09-21T06:13:55.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6baa000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba4168a30 con 0x7f6ba41527b0 2024-09-21T06:13:55.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6b9aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba40b5c70 0x7f6ba414d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6b9aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba40b5c70 0x7f6ba414d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33162/0 (socket says 172.21.15.50:33162) 2024-09-21T06:13:55.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6b9aa00640 1 -- 172.21.15.50:0/3895828189 learned_addr learned my addr 172.21.15.50:0/3895828189 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:55.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6ba3400640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ba40b4de0 0x7f6ba414be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6ba3e00640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ba41527b0 0x7f6ba415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6b9aa00640 1 -- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ba41527b0 msgr2=0x7f6ba415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6b9aa00640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ba41527b0 0x7f6ba415a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6b9aa00640 1 -- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ba40b4de0 msgr2=0x7f6ba414be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6b9aa00640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ba40b4de0 0x7f6ba414be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6b9aa00640 1 -- 172.21.15.50:0/3895828189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba4151110 con 0x7f6ba40b5c70 2024-09-21T06:13:55.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6b9aa00640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba40b5c70 0x7f6ba414d7d0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00b440 tx=0x7f6b8c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:55.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6b9be00640 1 -- 172.21.15.50:0/3895828189 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c015020 con 0x7f6ba40b5c70 2024-09-21T06:13:55.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6b9be00640 1 -- 172.21.15.50:0/3895828189 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b8c011980 con 0x7f6ba40b5c70 2024-09-21T06:13:55.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6baa000640 1 -- 172.21.15.50:0/3895828189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba4157130 con 0x7f6ba40b5c70 2024-09-21T06:13:55.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6b9be00640 1 -- 172.21.15.50:0/3895828189 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c011dd0 con 0x7f6ba40b5c70 2024-09-21T06:13:55.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.264+0000 7f6baa000640 1 -- 172.21.15.50:0/3895828189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba41f69c0 con 0x7f6ba40b5c70 2024-09-21T06:13:55.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.268+0000 7f6baa000640 1 -- 172.21.15.50:0/3895828189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba40b51f0 con 0x7f6ba40b5c70 2024-09-21T06:13:55.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.268+0000 7f6b9be00640 1 -- 172.21.15.50:0/3895828189 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f6b8c018020 con 0x7f6ba40b5c70 2024-09-21T06:13:55.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.268+0000 7f6b9be00640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6b800759f0 0x7f6b80077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.268+0000 7f6b9be00640 1 -- 172.21.15.50:0/3895828189 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f6b8c0935b0 con 0x7f6ba40b5c70 2024-09-21T06:13:55.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.268+0000 7f6ba3400640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6b800759f0 0x7f6b80077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.268+0000 7f6ba3400640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6b800759f0 0x7f6b80077f00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6b94005fd0 tx=0x7f6b9400a590 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:55.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.272+0000 7f6b9be00640 1 -- 172.21.15.50:0/3895828189 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6b8c01f020 con 0x7f6ba40b5c70 2024-09-21T06:13:55.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.448+0000 7f6baa000640 1 -- 172.21.15.50:0/3895828189 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ba414e470 con 0x7f6b800759f0 2024-09-21T06:13:55.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.448+0000 7f6b9be00640 1 -- 172.21.15.50:0/3895828189 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13553 (secure 0 0 0) 0x7f6ba414e470 con 0x7f6b800759f0 2024-09-21T06:13:55.445 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:55.445 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:13:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.448+0000 7f6baa000640 1 -- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6b800759f0 msgr2=0x7f6b80077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.448+0000 7f6baa000640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6b800759f0 0x7f6b80077f00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6b94005fd0 tx=0x7f6b9400a590 comp rx=0 tx=0).stop 2024-09-21T06:13:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.448+0000 7f6baa000640 1 -- 172.21.15.50:0/3895828189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba40b5c70 msgr2=0x7f6ba414d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.448+0000 7f6baa000640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba40b5c70 0x7f6ba414d7d0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00b440 tx=0x7f6b8c00b900 comp rx=0 tx=0).stop 2024-09-21T06:13:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.452+0000 7f6baa000640 1 -- 172.21.15.50:0/3895828189 shutdown_connections 2024-09-21T06:13:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.452+0000 7f6baa000640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ba41527b0 0x7f6ba415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.452+0000 7f6baa000640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6b800759f0 0x7f6b80077f00 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.452+0000 7f6baa000640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba40b5c70 0x7f6ba414d7d0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.452+0000 7f6baa000640 1 --2- 172.21.15.50:0/3895828189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ba40b4de0 0x7f6ba414be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.452+0000 7f6baa000640 1 -- 172.21.15.50:0/3895828189 >> 172.21.15.50:0/3895828189 conn(0x7f6ba40b2040 msgr2=0x7f6ba4151d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:55.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.452+0000 7f6baa000640 1 -- 172.21.15.50:0/3895828189 shutdown_connections 2024-09-21T06:13:55.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.452+0000 7f6baa000640 1 -- 172.21.15.50:0/3895828189 wait complete. 2024-09-21T06:13:55.467 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":671,"stamp":"2024-09-21T06:13:55.271428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1208740,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":1192819,"kb_avail":753765980,"statfs":{"total":773094113280,"available":771856363520,"internally_reserved":0,"allocated":16097280,"data_stored":7762976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1221447216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-991841877,"num_objects":-259,"num_object_clones":0,"num_object_copies":-518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-259,"num_whiteouts":0,"num_read":-3622,"num_read_kb":-14393452,"num_write":-9009,"num_write_kb":-17402052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-22,"num_bytes_recovered":-92274688,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.737533"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":348,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T06:13:51.130416+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:13:51.130416+0000","last_peered":"2024-09-21T06:13:51.130416+0000","last_clean":"2024-09-21T06:13:51.130416+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:13:51.130416+0000","last_undegraded":"2024-09-21T06:13:51.130416+0000","last_fullsized":"2024-09-21T06:13:51.130416+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509654,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145976,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225864,"statfs":{"total":96636764160,"available":96487284736,"internally_reserved":0,"allocated":2457600,"data_stored":1413148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149560,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222280,"statfs":{"total":96636764160,"available":96483614720,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131512,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240328,"statfs":{"total":96636764160,"available":96502095872,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167600,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204240,"statfs":{"total":96636764160,"available":96465141760,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149552,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222288,"statfs":{"total":96636764160,"available":96483622912,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509655,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155124,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":94216716,"statfs":{"total":96636764160,"available":96477917184,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":156957125},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509653,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150136,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221704,"statfs":{"total":96636764160,"available":96483024896,"internally_reserved":0,"allocated":2457600,"data_stored":1413148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542358,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159280,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212560,"statfs":{"total":96636764160,"available":96473661440,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:13:55.468 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:13:55.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.632+0000 7f252f400640 1 -- 172.21.15.50:0/2630942422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25280cefa0 msgr2=0x7f25280cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.632+0000 7f252f400640 1 --2- 172.21.15.50:0/2630942422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25280cefa0 0x7f25280cf3b0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f252815a930 tx=0x7f251001c8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.632+0000 7f252f400640 1 -- 172.21.15.50:0/2630942422 shutdown_connections 2024-09-21T06:13:55.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.632+0000 7f252f400640 1 --2- 172.21.15.50:0/2630942422 >> v2:172.21.15.88:3300/0 conn(0x7f2528160330 0x7f2528160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.632+0000 7f252f400640 1 --2- 172.21.15.50:0/2630942422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25280cefa0 0x7f25280cf3b0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.632+0000 7f252f400640 1 --2- 172.21.15.50:0/2630942422 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25280749a0 0x7f2528074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.632+0000 7f252f400640 1 -- 172.21.15.50:0/2630942422 >> 172.21.15.50:0/2630942422 conn(0x7f25280b2040 msgr2=0x7f25280b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:55.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.632+0000 7f252f400640 1 -- 172.21.15.50:0/2630942422 shutdown_connections 2024-09-21T06:13:55.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.632+0000 7f252f400640 1 -- 172.21.15.50:0/2630942422 wait complete. 2024-09-21T06:13:55.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.632+0000 7f252f400640 1 Processor -- start 2024-09-21T06:13:55.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.632+0000 7f252f400640 1 -- start start 2024-09-21T06:13:55.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280749a0 0x7f25281f9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252f400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25280cefa0 0x7f25280b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2528160330 0x7f25281f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252f400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f25280b3ee0 con 0x7f2528160330 2024-09-21T06:13:55.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252f400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f25280b3d60 con 0x7f25280749a0 2024-09-21T06:13:55.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252f400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f25280b4060 con 0x7f25280cefa0 2024-09-21T06:13:55.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2528160330 0x7f25281f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2528160330 0x7f25281f9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33178/0 (socket says 172.21.15.50:33178) 2024-09-21T06:13:55.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252d400640 1 -- 172.21.15.50:0/349296423 learned_addr learned my addr 172.21.15.50:0/349296423 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:55.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252ca00640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280749a0 0x7f25281f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252d400640 1 -- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25280cefa0 msgr2=0x7f25280b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f251fe00640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25280cefa0 0x7f25280b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252d400640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25280cefa0 0x7f25280b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252d400640 1 -- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280749a0 msgr2=0x7f25281f9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252d400640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280749a0 0x7f25281f9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252d400640 1 -- 172.21.15.50:0/349296423 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252815c700 con 0x7f2528160330 2024-09-21T06:13:55.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252ca00640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280749a0 0x7f25281f9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:55.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f251fe00640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25280cefa0 0x7f25280b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:55.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252d400640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2528160330 0x7f25281f9ee0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f252000eab0 tx=0x7f2520009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:55.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f2525600640 1 -- 172.21.15.50:0/349296423 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252000ce40 con 0x7f2528160330 2024-09-21T06:13:55.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f2525600640 1 -- 172.21.15.50:0/349296423 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2520004c20 con 0x7f2528160330 2024-09-21T06:13:55.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f2525600640 1 -- 172.21.15.50:0/349296423 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520005020 con 0x7f2528160330 2024-09-21T06:13:55.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252f400640 1 -- 172.21.15.50:0/349296423 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2528161820 con 0x7f2528160330 2024-09-21T06:13:55.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252f400640 1 -- 172.21.15.50:0/349296423 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25281f9070 con 0x7f2528160330 2024-09-21T06:13:55.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.636+0000 7f252f400640 1 -- 172.21.15.50:0/349296423 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25280cf3b0 con 0x7f2528160330 2024-09-21T06:13:55.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.640+0000 7f2525600640 1 -- 172.21.15.50:0/349296423 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f25200051e0 con 0x7f2528160330 2024-09-21T06:13:55.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.640+0000 7f2525600640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2504075a20 0x7f2504077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.640+0000 7f2525600640 1 -- 172.21.15.50:0/349296423 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f2520094700 con 0x7f2528160330 2024-09-21T06:13:55.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.640+0000 7f252ca00640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2504075a20 0x7f2504077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.640+0000 7f252ca00640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2504075a20 0x7f2504077f30 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2518009a40 tx=0x7f25180091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:55.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.644+0000 7f2525600640 1 -- 172.21.15.50:0/349296423 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f25200c5290 con 0x7f2528160330 2024-09-21T06:13:55.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f252f400640 1 -- 172.21.15.50:0/349296423 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25280b27f0 con 0x7f2528160330 2024-09-21T06:13:55.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f2525600640 1 -- 172.21.15.50:0/349296423 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+13127 (secure 0 0 0) 0x7f2520027080 con 0x7f2528160330 2024-09-21T06:13:55.820 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:13:55.820 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":149,"fsid":"3f44e77a-b7bc-4ef8-b058-60eaaa9147d6","created":"2024-09-21T05:56:03.541029+0000","modified":"2024-09-21T06:13:50.890606+0000","last_up_change":"2024-09-21T05:56:17.919268+0000","last_in_change":"2024-09-21T05:56:07.467198+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:56:18.921197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62980dcb-fdf5-40fb-a2a6-a456110d5727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6810","nonce":1152796349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6814","nonce":1152796349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6822","nonce":1152796349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1152796349},{"type":"v1","addr":"172.21.15.50:6818","nonce":1152796349}]},"public_addr":"172.21.15.50:6810/1152796349","cluster_addr":"172.21.15.50:6814/1152796349","heartbeat_back_addr":"172.21.15.50:6822/1152796349","heartbeat_front_addr":"172.21.15.50:6818/1152796349","state":["exists","up"]},{"osd":1,"uuid":"56c27f67-47d4-4947-9c33-de2f8289a275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6801","nonce":3048348933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6803","nonce":3048348933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6807","nonce":3048348933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3048348933},{"type":"v1","addr":"172.21.15.50:6805","nonce":3048348933}]},"public_addr":"172.21.15.50:6801/3048348933","cluster_addr":"172.21.15.50:6803/3048348933","heartbeat_back_addr":"172.21.15.50:6807/3048348933","heartbeat_front_addr":"172.21.15.50:6805/3048348933","state":["exists","up"]},{"osd":2,"uuid":"5c555649-42c2-4954-8b63-c7894b971798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6825","nonce":229343709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6827","nonce":229343709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6831","nonce":229343709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":229343709},{"type":"v1","addr":"172.21.15.50:6829","nonce":229343709}]},"public_addr":"172.21.15.50:6825/229343709","cluster_addr":"172.21.15.50:6827/229343709","heartbeat_back_addr":"172.21.15.50:6831/229343709","heartbeat_front_addr":"172.21.15.50:6829/229343709","state":["exists","up"]},{"osd":3,"uuid":"6ed0b889-c077-488f-abb5-1918a490c12e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6811","nonce":599598977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6815","nonce":599598977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6823","nonce":599598977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":599598977},{"type":"v1","addr":"172.21.15.50:6819","nonce":599598977}]},"public_addr":"172.21.15.50:6811/599598977","cluster_addr":"172.21.15.50:6815/599598977","heartbeat_back_addr":"172.21.15.50:6823/599598977","heartbeat_front_addr":"172.21.15.50:6819/599598977","state":["exists","up"]},{"osd":4,"uuid":"8e78b0d0-0ebd-478c-9c0b-c0cdb8156f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6827","nonce":2441482427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6829","nonce":2441482427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6833","nonce":2441482427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2441482427},{"type":"v1","addr":"172.21.15.88:6831","nonce":2441482427}]},"public_addr":"172.21.15.88:6827/2441482427","cluster_addr":"172.21.15.88:6829/2441482427","heartbeat_back_addr":"172.21.15.88:6833/2441482427","heartbeat_front_addr":"172.21.15.88:6831/2441482427","state":["exists","up"]},{"osd":5,"uuid":"5a0bf4bb-c6d7-48f6-810b-23f01fd7a745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6803","nonce":3846197193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6805","nonce":3846197193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6809","nonce":3846197193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3846197193},{"type":"v1","addr":"172.21.15.88:6807","nonce":3846197193}]},"public_addr":"172.21.15.88:6803/3846197193","cluster_addr":"172.21.15.88:6805/3846197193","heartbeat_back_addr":"172.21.15.88:6809/3846197193","heartbeat_front_addr":"172.21.15.88:6807/3846197193","state":["exists","up"]},{"osd":6,"uuid":"a9bbb22e-9d25-49db-9bb4-93ba63f97bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6819","nonce":2341496649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6821","nonce":2341496649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6825","nonce":2341496649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2341496649},{"type":"v1","addr":"172.21.15.88:6823","nonce":2341496649}]},"public_addr":"172.21.15.88:6819/2341496649","cluster_addr":"172.21.15.88:6821/2341496649","heartbeat_back_addr":"172.21.15.88:6825/2341496649","heartbeat_front_addr":"172.21.15.88:6823/2341496649","state":["exists","up"]},{"osd":7,"uuid":"01f3517b-4862-4189-b00f-e2da775385d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6811","nonce":4250480352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6813","nonce":4250480352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6817","nonce":4250480352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":4250480352},{"type":"v1","addr":"172.21.15.88:6815","nonce":4250480352}]},"public_addr":"172.21.15.88:6811/4250480352","cluster_addr":"172.21.15.88:6813/4250480352","heartbeat_back_addr":"172.21.15.88:6817/4250480352","heartbeat_front_addr":"172.21.15.88:6815/4250480352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.576693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.174994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.727409+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.813797+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:15.006601+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.299537+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.708337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:56:14.806932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6835/2222229719":"2024-09-22T06:13:47.367175+0000","172.21.15.88:6834/2222229719":"2024-09-22T06:13:47.367175+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:13:55.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f252f400640 1 -- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2504075a20 msgr2=0x7f2504077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f252f400640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2504075a20 0x7f2504077f30 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2518009a40 tx=0x7f25180091e0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f252f400640 1 -- 172.21.15.50:0/349296423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2528160330 msgr2=0x7f25281f9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f252f400640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2528160330 0x7f25281f9ee0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f252000eab0 tx=0x7f2520009f70 comp rx=0 tx=0).stop 2024-09-21T06:13:55.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f252f400640 1 -- 172.21.15.50:0/349296423 shutdown_connections 2024-09-21T06:13:55.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f252f400640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2504075a20 0x7f2504077f30 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f252f400640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2528160330 0x7f25281f9ee0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f252f400640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f25280cefa0 0x7f25280b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f252f400640 1 --2- 172.21.15.50:0/349296423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280749a0 0x7f25281f9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f252f400640 1 -- 172.21.15.50:0/349296423 >> 172.21.15.50:0/349296423 conn(0x7f25280b2040 msgr2=0x7f252815e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:55.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.824+0000 7f252f400640 1 -- 172.21.15.50:0/349296423 shutdown_connections 2024-09-21T06:13:55.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:55.828+0000 7f252f400640 1 -- 172.21.15.50:0/349296423 wait complete. 2024-09-21T06:13:56.845 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T06:13:56.845 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:13:57.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.024+0000 7f4526600640 1 -- 172.21.15.50:0/2935265298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45200b5c70 msgr2=0x7f452015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.024+0000 7f4526600640 1 --2- 172.21.15.50:0/2935265298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45200b5c70 0x7f452015bb60 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f4520070710 tx=0x7f451001c800 comp rx=0 tx=0).stop 2024-09-21T06:13:57.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.024+0000 7f4526600640 1 -- 172.21.15.50:0/2935265298 shutdown_connections 2024-09-21T06:13:57.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.024+0000 7f4526600640 1 --2- 172.21.15.50:0/2935265298 >> v2:172.21.15.88:3300/0 conn(0x7f45201527b0 0x7f4520074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.024+0000 7f4526600640 1 --2- 172.21.15.50:0/2935265298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45200b5c70 0x7f452015bb60 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.024+0000 7f4526600640 1 -- 172.21.15.50:0/2935265298 >> 172.21.15.50:0/2935265298 conn(0x7f45200b2040 msgr2=0x7f45200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.024+0000 7f4526600640 1 -- 172.21.15.50:0/2935265298 shutdown_connections 2024-09-21T06:13:57.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.024+0000 7f4526600640 1 -- 172.21.15.50:0/2935265298 wait complete. 2024-09-21T06:13:57.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f4526600640 1 Processor -- start 2024-09-21T06:13:57.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f4526600640 1 -- start start 2024-09-21T06:13:57.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f4526600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45200b4de0 0x7f45200aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f4526600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45200b5c70 0x7f452013fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f4526600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45201527b0 0x7f452013c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f4526600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4520168b20 con 0x7f45201527b0 2024-09-21T06:13:57.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f4526600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f45201689a0 con 0x7f45200b5c70 2024-09-21T06:13:57.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f4526600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f4520168ca0 con 0x7f45200b4de0 2024-09-21T06:13:57.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45201527b0 0x7f452013c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45200b5c70 0x7f452013fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45200b5c70 0x7f452013fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.50:38878/0 (socket says 172.21.15.50:38878) 2024-09-21T06:13:57.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45201527b0 0x7f452013c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59778/0 (socket says 172.21.15.50:59778) 2024-09-21T06:13:57.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451ea00640 1 -- 172.21.15.50:0/3676448149 learned_addr learned my addr 172.21.15.50:0/3676448149 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:57.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451f400640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45200b4de0 0x7f45200aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451fe00640 1 -- 172.21.15.50:0/3676448149 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45200b4de0 msgr2=0x7f45200aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451fe00640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f45200b4de0 0x7f45200aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451fe00640 1 -- 172.21.15.50:0/3676448149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45200b5c70 msgr2=0x7f452013fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451fe00640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45200b5c70 0x7f452013fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451fe00640 1 -- 172.21.15.50:0/3676448149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45201401d0 con 0x7f45201527b0 2024-09-21T06:13:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451ea00640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45200b5c70 0x7f452013fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.028+0000 7f451fe00640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45201527b0 0x7f452013c9a0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f4520074950 tx=0x7f4518009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.032+0000 7f4517400640 1 -- 172.21.15.50:0/3676448149 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518019070 con 0x7f45201527b0 2024-09-21T06:13:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.032+0000 7f4517400640 1 -- 172.21.15.50:0/3676448149 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4518014440 con 0x7f45201527b0 2024-09-21T06:13:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.032+0000 7f4517400640 1 -- 172.21.15.50:0/3676448149 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518014890 con 0x7f45201527b0 2024-09-21T06:13:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.032+0000 7f4526600640 1 -- 172.21.15.50:0/3676448149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45200ad050 con 0x7f45201527b0 2024-09-21T06:13:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.032+0000 7f4526600640 1 -- 172.21.15.50:0/3676448149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4520156500 con 0x7f45201527b0 2024-09-21T06:13:57.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.032+0000 7f4526600640 1 -- 172.21.15.50:0/3676448149 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f452006b540 con 0x7f45201527b0 2024-09-21T06:13:57.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.032+0000 7f4517400640 1 -- 172.21.15.50:0/3676448149 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f451801c070 con 0x7f45201527b0 2024-09-21T06:13:57.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.032+0000 7f4517400640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45000759f0 0x7f4500077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.032+0000 7f4517400640 1 -- 172.21.15.50:0/3676448149 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f4518093af0 con 0x7f45201527b0 2024-09-21T06:13:57.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.032+0000 7f4517400640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] conn(0x7f450007b960 0x7f450007de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.032+0000 7f4517400640 1 -- 172.21.15.50:0/3676448149 --> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f450007e4e0 con 0x7f450007b960 2024-09-21T06:13:57.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.032+0000 7f4517400640 1 -- 172.21.15.50:0/3676448149 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f4518093ea0 con 0x7f45201527b0 2024-09-21T06:13:57.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.036+0000 7f451f400640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45000759f0 0x7f4500077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.036+0000 7f451ea00640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] conn(0x7f450007b960 0x7f450007de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.036+0000 7f451ea00640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] conn(0x7f450007b960 0x7f450007de10 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.036+0000 7f451f400640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45000759f0 0x7f4500077f00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f45100af000 tx=0x7f451001b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.036+0000 7f4517400640 1 -- 172.21.15.50:0/3676448149 <== osd.0 v2:172.21.15.50:6808/1152796349 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f450007e4e0 con 0x7f450007b960 2024-09-21T06:13:57.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f4526600640 1 -- 172.21.15.50:0/3676448149 --> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f452015bb60 con 0x7f450007b960 2024-09-21T06:13:57.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f4517400640 1 -- 172.21.15.50:0/3676448149 <== osd.0 v2:172.21.15.50:6808/1152796349 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f452015bb60 con 0x7f450007b960 2024-09-21T06:13:57.046 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:57.046 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:13:57.047 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:57.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f4526600640 1 -- 172.21.15.50:0/3676448149 >> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] conn(0x7f450007b960 msgr2=0x7f450007de10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f4526600640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.50:6808/1152796349,v1:172.21.15.50:6810/1152796349] conn(0x7f450007b960 0x7f450007de10 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f4526600640 1 -- 172.21.15.50:0/3676448149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45000759f0 msgr2=0x7f4500077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f4526600640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f45000759f0 0x7f4500077f00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f45100af000 tx=0x7f451001b040 comp rx=0 tx=0).stop 2024-09-21T06:13:57.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f4526600640 1 -- 172.21.15.50:0/3676448149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45201527b0 msgr2=0x7f452013c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f4526600640 1 --2- 172.21.15.50:0/3676448149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45201527b0 0x7f452013c9a0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f4520074950 tx=0x7f4518009ed0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f451fe00640 1 -- 172.21.15.50:0/3676448149 reap_dead start 2024-09-21T06:13:57.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f4526600640 1 -- 172.21.15.50:0/3676448149 shutdown_connections 2024-09-21T06:13:57.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f4526600640 1 -- 172.21.15.50:0/3676448149 >> 172.21.15.50:0/3676448149 conn(0x7f45200b2040 msgr2=0x7f45200cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f4526600640 1 -- 172.21.15.50:0/3676448149 shutdown_connections 2024-09-21T06:13:57.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.048+0000 7f4526600640 1 -- 172.21.15.50:0/3676448149 wait complete. 2024-09-21T06:13:57.065 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T06:13:57.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.228+0000 7f3f82e00640 1 -- 172.21.15.50:0/1191739098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f7c0b5c70 msgr2=0x7f3f7c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.228+0000 7f3f82e00640 1 --2- 172.21.15.50:0/1191739098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f7c0b5c70 0x7f3f7c15bb60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009900 tx=0x7f3f6c01c820 comp rx=0 tx=0).stop 2024-09-21T06:13:57.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.228+0000 7f3f82e00640 1 -- 172.21.15.50:0/1191739098 shutdown_connections 2024-09-21T06:13:57.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.228+0000 7f3f82e00640 1 --2- 172.21.15.50:0/1191739098 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f7c1527b0 0x7f3f7c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.228+0000 7f3f82e00640 1 --2- 172.21.15.50:0/1191739098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f7c0b5c70 0x7f3f7c15bb60 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.228+0000 7f3f82e00640 1 -- 172.21.15.50:0/1191739098 >> 172.21.15.50:0/1191739098 conn(0x7f3f7c0b2040 msgr2=0x7f3f7c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.228+0000 7f3f82e00640 1 -- 172.21.15.50:0/1191739098 shutdown_connections 2024-09-21T06:13:57.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.228+0000 7f3f82e00640 1 -- 172.21.15.50:0/1191739098 wait complete. 2024-09-21T06:13:57.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f82e00640 1 Processor -- start 2024-09-21T06:13:57.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f82e00640 1 -- start start 2024-09-21T06:13:57.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f82e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f7c0b4de0 0x7f3f7c1548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f82e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f7c0b5c70 0x7f3f7c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f82e00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f7c1527b0 0x7f3f7c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f82e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c168c50 con 0x7f3f7c0b4de0 2024-09-21T06:13:57.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f82e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c168ad0 con 0x7f3f7c0b5c70 2024-09-21T06:13:57.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f82e00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7c168dd0 con 0x7f3f7c1527b0 2024-09-21T06:13:57.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f7be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f7c0b4de0 0x7f3f7c1548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f7be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f7c0b4de0 0x7f3f7c1548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59792/0 (socket says 172.21.15.50:59792) 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f7be00640 1 -- 172.21.15.50:0/1728831144 learned_addr learned my addr 172.21.15.50:0/1728831144 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f73e00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f7c0b5c70 0x7f3f7c153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f80e00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f7c1527b0 0x7f3f7c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f7be00640 1 -- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f7c1527b0 msgr2=0x7f3f7c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f7be00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f7c1527b0 0x7f3f7c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f7be00640 1 -- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f7c0b5c70 msgr2=0x7f3f7c153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f7be00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f7c0b5c70 0x7f3f7c153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f7be00640 1 -- 172.21.15.50:0/1728831144 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c153f60 con 0x7f3f7c0b4de0 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f73e00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f7c0b5c70 0x7f3f7c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f7be00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f7c0b4de0 0x7f3f7c1548a0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0b0040 tx=0x7f3f6c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f78c00640 1 -- 172.21.15.50:0/1728831144 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b2040 con 0x7f3f7c0b4de0 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f78c00640 1 -- 172.21.15.50:0/1728831144 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f6c005030 con 0x7f3f7c0b4de0 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f82e00640 1 -- 172.21.15.50:0/1728831144 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c1f8a00 con 0x7f3f7c0b4de0 2024-09-21T06:13:57.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f78c00640 1 -- 172.21.15.50:0/1728831144 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c005420 con 0x7f3f7c0b4de0 2024-09-21T06:13:57.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.232+0000 7f3f82e00640 1 -- 172.21.15.50:0/1728831144 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c1f91f0 con 0x7f3f7c0b4de0 2024-09-21T06:13:57.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.236+0000 7f3f82e00640 1 -- 172.21.15.50:0/1728831144 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f7c15bb60 con 0x7f3f7c0b4de0 2024-09-21T06:13:57.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.236+0000 7f3f78c00640 1 -- 172.21.15.50:0/1728831144 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f3f6c0be060 con 0x7f3f7c0b4de0 2024-09-21T06:13:57.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.236+0000 7f3f78c00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f580759f0 0x7f3f58077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.236+0000 7f3f78c00640 1 -- 172.21.15.50:0/1728831144 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f3f6c12cce0 con 0x7f3f7c0b4de0 2024-09-21T06:13:57.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.236+0000 7f3f73e00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f580759f0 0x7f3f58077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.240+0000 7f3f73e00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f580759f0 0x7f3f58077f00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3f64005fd0 tx=0x7f3f64005f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.240+0000 7f3f78c00640 1 -- 172.21.15.50:0/1728831144 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3f6c136070 con 0x7f3f7c0b4de0 2024-09-21T06:13:57.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 -- 172.21.15.50:0/1728831144 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f3f7c06b540 con 0x7f3f580759f0 2024-09-21T06:13:57.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f78c00640 1 -- 172.21.15.50:0/1728831144 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3f6400b7f0 con 0x7f3f580759f0 2024-09-21T06:13:57.417 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T06:13:57.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 -- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f580759f0 msgr2=0x7f3f58077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f580759f0 0x7f3f58077f00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3f64005fd0 tx=0x7f3f64005f00 comp rx=0 tx=0).stop 2024-09-21T06:13:57.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 -- 172.21.15.50:0/1728831144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f7c0b4de0 msgr2=0x7f3f7c1548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f7c0b4de0 0x7f3f7c1548a0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0b0040 tx=0x7f3f6c004040 comp rx=0 tx=0).stop 2024-09-21T06:13:57.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 -- 172.21.15.50:0/1728831144 shutdown_connections 2024-09-21T06:13:57.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f3f7c1527b0 0x7f3f7c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f7c0b5c70 0x7f3f7c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f3f580759f0 0x7f3f58077f00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 --2- 172.21.15.50:0/1728831144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f7c0b4de0 0x7f3f7c1548a0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 -- 172.21.15.50:0/1728831144 >> 172.21.15.50:0/1728831144 conn(0x7f3f7c0b2040 msgr2=0x7f3f7c14d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 -- 172.21.15.50:0/1728831144 shutdown_connections 2024-09-21T06:13:57.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.420+0000 7f3f82e00640 1 -- 172.21.15.50:0/1728831144 wait complete. 2024-09-21T06:13:57.439 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T06:13:57.439 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:13:57.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.604+0000 7f396cc00640 1 -- 172.21.15.50:0/1617991526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f396815e8c0 msgr2=0x7f396815ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.604+0000 7f396cc00640 1 --2- 172.21.15.50:0/1617991526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f396815e8c0 0x7f396815ecd0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f39680cce50 tx=0x7f395801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.604+0000 7f396cc00640 1 -- 172.21.15.50:0/1617991526 shutdown_connections 2024-09-21T06:13:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.604+0000 7f396cc00640 1 --2- 172.21.15.50:0/1617991526 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f396815bc20 0x7f396815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.604+0000 7f396cc00640 1 --2- 172.21.15.50:0/1617991526 >> v2:172.21.15.88:3300/0 conn(0x7f396815b2d0 0x7f396815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.604+0000 7f396cc00640 1 --2- 172.21.15.50:0/1617991526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f396815e8c0 0x7f396815ecd0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.604+0000 7f396cc00640 1 -- 172.21.15.50:0/1617991526 >> 172.21.15.50:0/1617991526 conn(0x7f39680b2040 msgr2=0x7f39680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.608+0000 7f396cc00640 1 -- 172.21.15.50:0/1617991526 shutdown_connections 2024-09-21T06:13:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.608+0000 7f396cc00640 1 -- 172.21.15.50:0/1617991526 wait complete. 2024-09-21T06:13:57.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.608+0000 7f396cc00640 1 Processor -- start 2024-09-21T06:13:57.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.608+0000 7f396cc00640 1 -- start start 2024-09-21T06:13:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.608+0000 7f396cc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f396815b2d0 0x7f39681f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.608+0000 7f396cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f396815bc20 0x7f39681f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.608+0000 7f396cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f396815e8c0 0x7f39680ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.608+0000 7f396cc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f396815d7c0 con 0x7f396815e8c0 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.608+0000 7f396cc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39680b5460 con 0x7f396815bc20 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.608+0000 7f396cc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f396815d940 con 0x7f396815b2d0 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f3966800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f396815e8c0 0x7f39680ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f3966800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f396815e8c0 0x7f39680ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59800/0 (socket says 172.21.15.50:59800) 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f3966800640 1 -- 172.21.15.50:0/1665901333 learned_addr learned my addr 172.21.15.50:0/1665901333 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f3965e00640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f396815b2d0 0x7f39681f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f395fe00640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f396815bc20 0x7f39681f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f3966800640 1 -- 172.21.15.50:0/1665901333 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f396815b2d0 msgr2=0x7f39681f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f3966800640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f396815b2d0 0x7f39681f5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f3966800640 1 -- 172.21.15.50:0/1665901333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f396815bc20 msgr2=0x7f39681f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f3966800640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f396815bc20 0x7f39681f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f3966800640 1 -- 172.21.15.50:0/1665901333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39681fa6c0 con 0x7f396815e8c0 2024-09-21T06:13:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f395fe00640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f396815bc20 0x7f39681f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f3966800640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f396815e8c0 0x7f39680ce110 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f396000e940 tx=0x7f396000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f395e000640 1 -- 172.21.15.50:0/1665901333 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960004d30 con 0x7f396815e8c0 2024-09-21T06:13:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f395e000640 1 -- 172.21.15.50:0/1665901333 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f396000cb20 con 0x7f396815e8c0 2024-09-21T06:13:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f395e000640 1 -- 172.21.15.50:0/1665901333 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960005020 con 0x7f396815e8c0 2024-09-21T06:13:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f396cc00640 1 -- 172.21.15.50:0/1665901333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39681f66b0 con 0x7f396815e8c0 2024-09-21T06:13:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f396cc00640 1 -- 172.21.15.50:0/1665901333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39681f8230 con 0x7f396815e8c0 2024-09-21T06:13:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f396cc00640 1 -- 172.21.15.50:0/1665901333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f396806b540 con 0x7f396815e8c0 2024-09-21T06:13:57.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.612+0000 7f395e000640 1 -- 172.21.15.50:0/1665901333 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f39600051c0 con 0x7f396815e8c0 2024-09-21T06:13:57.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.616+0000 7f395e000640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39440759f0 0x7f3944077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.616+0000 7f395e000640 1 -- 172.21.15.50:0/1665901333 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f3960094420 con 0x7f396815e8c0 2024-09-21T06:13:57.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.616+0000 7f395e000640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] conn(0x7f394407b960 0x7f394407de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.616+0000 7f395e000640 1 -- 172.21.15.50:0/1665901333 --> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3960018ec0 con 0x7f394407b960 2024-09-21T06:13:57.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.616+0000 7f395e000640 1 -- 172.21.15.50:0/1665901333 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f39600947d0 con 0x7f396815e8c0 2024-09-21T06:13:57.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.616+0000 7f3965e00640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39440759f0 0x7f3944077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.616+0000 7f395fe00640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] conn(0x7f394407b960 0x7f394407de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.616+0000 7f3965e00640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39440759f0 0x7f3944077f00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f39580af000 tx=0x7f39580a8040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.616+0000 7f395fe00640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] conn(0x7f394407b960 0x7f394407de10 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.616+0000 7f395e000640 1 -- 172.21.15.50:0/1665901333 <== osd.1 v2:172.21.15.50:6800/3048348933 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3960018ec0 con 0x7f394407b960 2024-09-21T06:13:57.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f396cc00640 1 -- 172.21.15.50:0/1665901333 --> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f39681fa470 con 0x7f394407b960 2024-09-21T06:13:57.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f395e000640 1 -- 172.21.15.50:0/1665901333 <== osd.1 v2:172.21.15.50:6800/3048348933 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f39681fa470 con 0x7f394407b960 2024-09-21T06:13:57.628 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:57.628 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:13:57.628 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:57.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f396cc00640 1 -- 172.21.15.50:0/1665901333 >> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] conn(0x7f394407b960 msgr2=0x7f394407de10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f396cc00640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.50:6800/3048348933,v1:172.21.15.50:6801/3048348933] conn(0x7f394407b960 0x7f394407de10 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f396cc00640 1 -- 172.21.15.50:0/1665901333 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39440759f0 msgr2=0x7f3944077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f396cc00640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39440759f0 0x7f3944077f00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f39580af000 tx=0x7f39580a8040 comp rx=0 tx=0).stop 2024-09-21T06:13:57.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f396cc00640 1 -- 172.21.15.50:0/1665901333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f396815e8c0 msgr2=0x7f39680ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f396cc00640 1 --2- 172.21.15.50:0/1665901333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f396815e8c0 0x7f39680ce110 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f396000e940 tx=0x7f396000ee00 comp rx=0 tx=0).stop 2024-09-21T06:13:57.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f3966800640 1 -- 172.21.15.50:0/1665901333 reap_dead start 2024-09-21T06:13:57.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f396cc00640 1 -- 172.21.15.50:0/1665901333 shutdown_connections 2024-09-21T06:13:57.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f396cc00640 1 -- 172.21.15.50:0/1665901333 >> 172.21.15.50:0/1665901333 conn(0x7f39680b2040 msgr2=0x7f396815a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f396cc00640 1 -- 172.21.15.50:0/1665901333 shutdown_connections 2024-09-21T06:13:57.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.632+0000 7f396cc00640 1 -- 172.21.15.50:0/1665901333 wait complete. 2024-09-21T06:13:57.647 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T06:13:57.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.812+0000 7f2f08a00640 1 -- 172.21.15.50:0/3629087552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8003c20 msgr2=0x7f2ee8004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.812+0000 7f2f08a00640 1 --2- 172.21.15.50:0/3629087552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8003c20 0x7f2ee8004070 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4001860 tx=0x7f2ef401c8b0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.812+0000 7f2f03e00640 1 -- 172.21.15.50:0/3629087552 reap_dead start 2024-09-21T06:13:57.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.812+0000 7f2f08a00640 1 -- 172.21.15.50:0/3629087552 shutdown_connections 2024-09-21T06:13:57.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.812+0000 7f2f08a00640 1 -- 172.21.15.50:0/3629087552 >> 172.21.15.50:0/3629087552 conn(0x7f2f0406cdf0 msgr2=0x7f2f0406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.812+0000 7f2f08a00640 1 -- 172.21.15.50:0/3629087552 shutdown_connections 2024-09-21T06:13:57.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.812+0000 7f2f08a00640 1 -- 172.21.15.50:0/3629087552 wait complete. 2024-09-21T06:13:57.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.812+0000 7f2f08a00640 1 Processor -- start 2024-09-21T06:13:57.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.812+0000 7f2f08a00640 1 -- start start 2024-09-21T06:13:57.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f08a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8003c20 0x7f2f04148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f08a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f041fc7c0 0x7f2f0415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f08a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2f04072c20 0x7f2f04073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f08a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2f04167a10 con 0x7f2ee8003c20 2024-09-21T06:13:57.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f08a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2f040b3e60 con 0x7f2f041fc7c0 2024-09-21T06:13:57.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f08a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2f04167b90 con 0x7f2f04072c20 2024-09-21T06:13:57.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f03400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8003c20 0x7f2f04148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f03400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8003c20 0x7f2f04148d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59828/0 (socket says 172.21.15.50:59828) 2024-09-21T06:13:57.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f03400640 1 -- 172.21.15.50:0/4253141012 learned_addr learned my addr 172.21.15.50:0/4253141012 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:57.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f03e00640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2f04072c20 0x7f2f04073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2efbe00640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f041fc7c0 0x7f2f0415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f03400640 1 -- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2f04072c20 msgr2=0x7f2f04073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f03400640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2f04072c20 0x7f2f04073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f03400640 1 -- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f041fc7c0 msgr2=0x7f2f0415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f03400640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f041fc7c0 0x7f2f0415dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f03400640 1 -- 172.21.15.50:0/4253141012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f04073570 con 0x7f2ee8003c20 2024-09-21T06:13:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f03e00640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2f04072c20 0x7f2f04073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2efbe00640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f041fc7c0 0x7f2f0415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f03400640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8003c20 0x7f2f04148d30 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2ef401afb0 tx=0x7f2ef40a5ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2efb400640 1 -- 172.21.15.50:0/4253141012 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4001480 con 0x7f2ee8003c20 2024-09-21T06:13:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2efb400640 1 -- 172.21.15.50:0/4253141012 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ef40b4030 con 0x7f2ee8003c20 2024-09-21T06:13:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2efb400640 1 -- 172.21.15.50:0/4253141012 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40068d0 con 0x7f2ee8003c20 2024-09-21T06:13:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f08a00640 1 -- 172.21.15.50:0/4253141012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0415e1a0 con 0x7f2ee8003c20 2024-09-21T06:13:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f08a00640 1 -- 172.21.15.50:0/4253141012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f041fbce0 con 0x7f2ee8003c20 2024-09-21T06:13:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.816+0000 7f2f08a00640 1 -- 172.21.15.50:0/4253141012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f040b3540 con 0x7f2ee8003c20 2024-09-21T06:13:57.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.820+0000 7f2efb400640 1 -- 172.21.15.50:0/4253141012 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f2ef4003b30 con 0x7f2ee8003c20 2024-09-21T06:13:57.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.820+0000 7f2efb400640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ee00759f0 0x7f2ee0077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.820+0000 7f2efb400640 1 -- 172.21.15.50:0/4253141012 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f2ef412d930 con 0x7f2ee8003c20 2024-09-21T06:13:57.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.820+0000 7f2efbe00640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ee00759f0 0x7f2ee0077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.820+0000 7f2efbe00640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ee00759f0 0x7f2ee0077f00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2f040aaa40 tx=0x7f2eec0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:57.824+0000 7f2efb400640 1 -- 172.21.15.50:0/4253141012 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2f04073570 con 0x7f2ee8003c20 2024-09-21T06:13:57.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 -- 172.21.15.50:0/4253141012 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f2f0406b540 con 0x7f2ee00759f0 2024-09-21T06:13:57.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2efb400640 1 -- 172.21.15.50:0/4253141012 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2eec004020 con 0x7f2ee00759f0 2024-09-21T06:13:58.000 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T06:13:58.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 -- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ee00759f0 msgr2=0x7f2ee0077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ee00759f0 0x7f2ee0077f00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2f040aaa40 tx=0x7f2eec0091e0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 -- 172.21.15.50:0/4253141012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8003c20 msgr2=0x7f2f04148d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8003c20 0x7f2f04148d30 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2ef401afb0 tx=0x7f2ef40a5ca0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 -- 172.21.15.50:0/4253141012 shutdown_connections 2024-09-21T06:13:58.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f2f04072c20 0x7f2f04073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f041fc7c0 0x7f2f0415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f2ee00759f0 0x7f2ee0077f00 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 --2- 172.21.15.50:0/4253141012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8003c20 0x7f2f04148d30 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 -- 172.21.15.50:0/4253141012 >> 172.21.15.50:0/4253141012 conn(0x7f2f0406cdf0 msgr2=0x7f2f041661f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:58.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 -- 172.21.15.50:0/4253141012 shutdown_connections 2024-09-21T06:13:58.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.004+0000 7f2f08a00640 1 -- 172.21.15.50:0/4253141012 wait complete. 2024-09-21T06:13:58.023 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T06:13:58.024 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:13:58.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.188+0000 7fb8f6600640 1 -- 172.21.15.50:0/1427962314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f00b4de0 msgr2=0x7fb8f00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.188+0000 7fb8f6600640 1 --2- 172.21.15.50:0/1427962314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f00b4de0 0x7fb8f00b51f0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009840 tx=0x7fb8d801c820 comp rx=0 tx=0).stop 2024-09-21T06:13:58.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.188+0000 7fb8f6600640 1 -- 172.21.15.50:0/1427962314 shutdown_connections 2024-09-21T06:13:58.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.188+0000 7fb8f6600640 1 --2- 172.21.15.50:0/1427962314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f00b4de0 0x7fb8f00b51f0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.188+0000 7fb8f6600640 1 --2- 172.21.15.50:0/1427962314 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8f0072100 0x7fb8f0155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.192+0000 7fb8f6600640 1 -- 172.21.15.50:0/1427962314 >> 172.21.15.50:0/1427962314 conn(0x7fb8f00b2040 msgr2=0x7fb8f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:58.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.192+0000 7fb8f6600640 1 -- 172.21.15.50:0/1427962314 shutdown_connections 2024-09-21T06:13:58.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.192+0000 7fb8f6600640 1 -- 172.21.15.50:0/1427962314 wait complete. 2024-09-21T06:13:58.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.192+0000 7fb8f6600640 1 Processor -- start 2024-09-21T06:13:58.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.192+0000 7fb8f6600640 1 -- start start 2024-09-21T06:13:58.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.192+0000 7fb8f6600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8f0072100 0x7fb8f014efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.192+0000 7fb8f6600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f00b4de0 0x7fb8f0156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.192+0000 7fb8f6600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f015f0e0 0x7fb8f01505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.192+0000 7fb8f6600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01693d0 con 0x7fb8f015f0e0 2024-09-21T06:13:58.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.192+0000 7fb8f6600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f0169250 con 0x7fb8f00b4de0 2024-09-21T06:13:58.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.192+0000 7fb8f6600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f0169550 con 0x7fb8f0072100 2024-09-21T06:13:58.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f015f0e0 0x7fb8f01505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f015f0e0 0x7fb8f01505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59842/0 (socket says 172.21.15.50:59842) 2024-09-21T06:13:58.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8efe00640 1 -- 172.21.15.50:0/2282985480 learned_addr learned my addr 172.21.15.50:0/2282985480 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:58.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8ef400640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8f0072100 0x7fb8f014efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8e7e00640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f00b4de0 0x7fb8f0156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8efe00640 1 -- 172.21.15.50:0/2282985480 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8f0072100 msgr2=0x7fb8f014efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8efe00640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb8f0072100 0x7fb8f014efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8efe00640 1 -- 172.21.15.50:0/2282985480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f00b4de0 msgr2=0x7fb8f0156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8efe00640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f00b4de0 0x7fb8f0156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8efe00640 1 -- 172.21.15.50:0/2282985480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f006f080 con 0x7fb8f015f0e0 2024-09-21T06:13:58.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8efe00640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f015f0e0 0x7fb8f01505f0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800c730 tx=0x7fb8e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:58.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8e7400640 1 -- 172.21.15.50:0/2282985480 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8003b80 con 0x7fb8f015f0e0 2024-09-21T06:13:58.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8e7400640 1 -- 172.21.15.50:0/2282985480 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8e8003d40 con 0x7fb8f015f0e0 2024-09-21T06:13:58.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8e7400640 1 -- 172.21.15.50:0/2282985480 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80052d0 con 0x7fb8f015f0e0 2024-09-21T06:13:58.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8f6600640 1 -- 172.21.15.50:0/2282985480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f0157bc0 con 0x7fb8f015f0e0 2024-09-21T06:13:58.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8f6600640 1 -- 172.21.15.50:0/2282985480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f014fdf0 con 0x7fb8f015f0e0 2024-09-21T06:13:58.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8f6600640 1 -- 172.21.15.50:0/2282985480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb8f006b540 con 0x7fb8f015f0e0 2024-09-21T06:13:58.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.196+0000 7fb8e7400640 1 -- 172.21.15.50:0/2282985480 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fb8e8005680 con 0x7fb8f015f0e0 2024-09-21T06:13:58.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.200+0000 7fb8e7400640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb8cc0759f0 0x7fb8cc077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.200+0000 7fb8e7400640 1 -- 172.21.15.50:0/2282985480 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7fb8e8093e50 con 0x7fb8f015f0e0 2024-09-21T06:13:58.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.200+0000 7fb8e7400640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] conn(0x7fb8cc07b960 0x7fb8cc07de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.200+0000 7fb8e7400640 1 -- 172.21.15.50:0/2282985480 --> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb8e801de70 con 0x7fb8cc07b960 2024-09-21T06:13:58.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.200+0000 7fb8e7400640 1 -- 172.21.15.50:0/2282985480 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fb8e8094200 con 0x7fb8f015f0e0 2024-09-21T06:13:58.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.200+0000 7fb8ef400640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb8cc0759f0 0x7fb8cc077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.200+0000 7fb8e7e00640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] conn(0x7fb8cc07b960 0x7fb8cc07de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.200+0000 7fb8e7e00640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] conn(0x7fb8cc07b960 0x7fb8cc07de10 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:58.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.200+0000 7fb8ef400640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb8cc0759f0 0x7fb8cc077f00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009b40 tx=0x7fb8e00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:58.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.200+0000 7fb8e7400640 1 -- 172.21.15.50:0/2282985480 <== osd.2 v2:172.21.15.50:6824/229343709 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb8e801de70 con 0x7fb8cc07b960 2024-09-21T06:13:58.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.212+0000 7fb8f6600640 1 -- 172.21.15.50:0/2282985480 --> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb8f0157950 con 0x7fb8cc07b960 2024-09-21T06:13:58.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.212+0000 7fb8e7400640 1 -- 172.21.15.50:0/2282985480 <== osd.2 v2:172.21.15.50:6824/229343709 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb8f0157950 con 0x7fb8cc07b960 2024-09-21T06:13:58.211 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:58.211 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:13:58.211 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:58.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.216+0000 7fb8f6600640 1 -- 172.21.15.50:0/2282985480 >> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] conn(0x7fb8cc07b960 msgr2=0x7fb8cc07de10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.216+0000 7fb8f6600640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.50:6824/229343709,v1:172.21.15.50:6825/229343709] conn(0x7fb8cc07b960 0x7fb8cc07de10 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.216+0000 7fb8f6600640 1 -- 172.21.15.50:0/2282985480 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb8cc0759f0 msgr2=0x7fb8cc077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.216+0000 7fb8f6600640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb8cc0759f0 0x7fb8cc077f00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009b40 tx=0x7fb8e00091e0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.216+0000 7fb8f6600640 1 -- 172.21.15.50:0/2282985480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f015f0e0 msgr2=0x7fb8f01505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.216+0000 7fb8f6600640 1 --2- 172.21.15.50:0/2282985480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f015f0e0 0x7fb8f01505f0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800c730 tx=0x7fb8e800cbf0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.216+0000 7fb8efe00640 1 -- 172.21.15.50:0/2282985480 reap_dead start 2024-09-21T06:13:58.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.216+0000 7fb8f6600640 1 -- 172.21.15.50:0/2282985480 shutdown_connections 2024-09-21T06:13:58.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.216+0000 7fb8f6600640 1 -- 172.21.15.50:0/2282985480 >> 172.21.15.50:0/2282985480 conn(0x7fb8f00b2040 msgr2=0x7fb8f006e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:58.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.216+0000 7fb8f6600640 1 -- 172.21.15.50:0/2282985480 shutdown_connections 2024-09-21T06:13:58.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.216+0000 7fb8f6600640 1 -- 172.21.15.50:0/2282985480 wait complete. 2024-09-21T06:13:58.230 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T06:13:58.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.392+0000 7f6ad5c00640 1 -- 172.21.15.50:0/1134289015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad015b2d0 msgr2=0x7f6ad015b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.392+0000 7f6ad5c00640 1 --2- 172.21.15.50:0/1134289015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad015b2d0 0x7f6ad015b6e0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00b3040 tx=0x7f6ab801c8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.392+0000 7f6ad5c00640 1 -- 172.21.15.50:0/1134289015 shutdown_connections 2024-09-21T06:13:58.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.392+0000 7f6ad5c00640 1 --2- 172.21.15.50:0/1134289015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad015b2d0 0x7f6ad015b6e0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.392+0000 7f6ad5c00640 1 --2- 172.21.15.50:0/1134289015 >> v2:172.21.15.88:3300/0 conn(0x7f6ad015e8c0 0x7f6ad015ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.392+0000 7f6ad5c00640 1 -- 172.21.15.50:0/1134289015 >> 172.21.15.50:0/1134289015 conn(0x7f6ad00b2040 msgr2=0x7f6ad00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:58.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.392+0000 7f6ad5c00640 1 -- 172.21.15.50:0/1134289015 shutdown_connections 2024-09-21T06:13:58.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.392+0000 7f6ad5c00640 1 -- 172.21.15.50:0/1134289015 wait complete. 2024-09-21T06:13:58.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6ad5c00640 1 Processor -- start 2024-09-21T06:13:58.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6ad5c00640 1 -- start start 2024-09-21T06:13:58.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6ad5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ad015b2d0 0x7f6ad013b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6ad5c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ad015bc20 0x7f6ad0074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6ad5c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad015e8c0 0x7f6ad0074690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6ad5c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad015d990 con 0x7f6ad015e8c0 2024-09-21T06:13:58.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6ad5c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad00b5420 con 0x7f6ad015b2d0 2024-09-21T06:13:58.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6ad5c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad015db10 con 0x7f6ad015bc20 2024-09-21T06:13:58.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6acf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad015e8c0 0x7f6ad0074690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6acf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad015e8c0 0x7f6ad0074690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59860/0 (socket says 172.21.15.50:59860) 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6acf400640 1 -- 172.21.15.50:0/892823494 learned_addr learned my addr 172.21.15.50:0/892823494 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6acea00640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ad015b2d0 0x7f6ad013b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6ac6000640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ad015bc20 0x7f6ad0074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6acf400640 1 -- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ad015bc20 msgr2=0x7f6ad0074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6acf400640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ad015bc20 0x7f6ad0074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6acf400640 1 -- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ad015b2d0 msgr2=0x7f6ad013b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6acf400640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ad015b2d0 0x7f6ad013b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.396+0000 7f6acf400640 1 -- 172.21.15.50:0/892823494 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad013cbb0 con 0x7f6ad015e8c0 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.400+0000 7f6acf400640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad015e8c0 0x7f6ad0074690 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800b9e0 tx=0x7f6ac800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.400+0000 7f6ac7400640 1 -- 172.21.15.50:0/892823494 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8003b80 con 0x7f6ad015e8c0 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.400+0000 7f6ac7400640 1 -- 172.21.15.50:0/892823494 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ac8003e10 con 0x7f6ad015e8c0 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.400+0000 7f6ad5c00640 1 -- 172.21.15.50:0/892823494 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad013dce0 con 0x7f6ad015e8c0 2024-09-21T06:13:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.400+0000 7f6ac7400640 1 -- 172.21.15.50:0/892823494 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac800cb80 con 0x7f6ad015e8c0 2024-09-21T06:13:58.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.400+0000 7f6ad5c00640 1 -- 172.21.15.50:0/892823494 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad013d130 con 0x7f6ad015e8c0 2024-09-21T06:13:58.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.400+0000 7f6ad5c00640 1 -- 172.21.15.50:0/892823494 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad00b1c30 con 0x7f6ad015e8c0 2024-09-21T06:13:58.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.400+0000 7f6ac7400640 1 -- 172.21.15.50:0/892823494 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f6ac8006d10 con 0x7f6ad015e8c0 2024-09-21T06:13:58.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.400+0000 7f6ac7400640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6aac0759f0 0x7f6aac077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.400+0000 7f6ac7400640 1 -- 172.21.15.50:0/892823494 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f6ac8093190 con 0x7f6ad015e8c0 2024-09-21T06:13:58.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.404+0000 7f6acea00640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6aac0759f0 0x7f6aac077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.404+0000 7f6acea00640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6aac0759f0 0x7f6aac077f00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0004630 tx=0x7f6ac000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:58.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.408+0000 7f6ac7400640 1 -- 172.21.15.50:0/892823494 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6ac805f5c0 con 0x7f6ad015e8c0 2024-09-21T06:13:58.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.584+0000 7f6ad5c00640 1 -- 172.21.15.50:0/892823494 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f6ad006b540 con 0x7f6aac0759f0 2024-09-21T06:13:58.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.584+0000 7f6ac7400640 1 -- 172.21.15.50:0/892823494 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6ac0005020 con 0x7f6aac0759f0 2024-09-21T06:13:58.581 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T06:13:58.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.588+0000 7f6ad5c00640 1 -- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6aac0759f0 msgr2=0x7f6aac077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.588+0000 7f6ad5c00640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6aac0759f0 0x7f6aac077f00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0004630 tx=0x7f6ac000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.588+0000 7f6ad5c00640 1 -- 172.21.15.50:0/892823494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad015e8c0 msgr2=0x7f6ad0074690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.588+0000 7f6ad5c00640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad015e8c0 0x7f6ad0074690 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800b9e0 tx=0x7f6ac800bea0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.588+0000 7f6ad5c00640 1 -- 172.21.15.50:0/892823494 shutdown_connections 2024-09-21T06:13:58.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.588+0000 7f6ad5c00640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f6aac0759f0 0x7f6aac077f00 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.588+0000 7f6ad5c00640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ad015e8c0 0x7f6ad0074690 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.588+0000 7f6ad5c00640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f6ad015bc20 0x7f6ad0074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.588+0000 7f6ad5c00640 1 --2- 172.21.15.50:0/892823494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ad015b2d0 0x7f6ad013b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.588+0000 7f6ad5c00640 1 -- 172.21.15.50:0/892823494 >> 172.21.15.50:0/892823494 conn(0x7f6ad00b2040 msgr2=0x7f6ad00ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:58.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.588+0000 7f6ad5c00640 1 -- 172.21.15.50:0/892823494 shutdown_connections 2024-09-21T06:13:58.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.588+0000 7f6ad5c00640 1 -- 172.21.15.50:0/892823494 wait complete. 2024-09-21T06:13:58.604 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T06:13:58.604 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:13:58.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.772+0000 7fe11ae00640 1 -- 172.21.15.50:0/2165966840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1140cefa0 msgr2=0x7fe1140cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.772+0000 7fe11ae00640 1 --2- 172.21.15.50:0/2165966840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1140cefa0 0x7fe1140cf3b0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009840 tx=0x7fe0fc01c820 comp rx=0 tx=0).stop 2024-09-21T06:13:58.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.772+0000 7fe11ae00640 1 -- 172.21.15.50:0/2165966840 shutdown_connections 2024-09-21T06:13:58.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.772+0000 7fe11ae00640 1 --2- 172.21.15.50:0/2165966840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1140cefa0 0x7fe1140cf3b0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.772+0000 7fe11ae00640 1 -- 172.21.15.50:0/2165966840 >> 172.21.15.50:0/2165966840 conn(0x7fe1140b2040 msgr2=0x7fe1140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:58.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.772+0000 7fe11ae00640 1 -- 172.21.15.50:0/2165966840 shutdown_connections 2024-09-21T06:13:58.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.772+0000 7fe11ae00640 1 -- 172.21.15.50:0/2165966840 wait complete. 2024-09-21T06:13:58.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.772+0000 7fe11ae00640 1 Processor -- start 2024-09-21T06:13:58.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.772+0000 7fe11ae00640 1 -- start start 2024-09-21T06:13:58.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe11ae00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1140749a0 0x7fe114145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe11ae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1140b3340 0x7fe1140aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe11ae00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe1140cefa0 0x7fe114159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe11ae00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe11416b350 con 0x7fe1140749a0 2024-09-21T06:13:58.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe11ae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe11416b1d0 con 0x7fe1140b3340 2024-09-21T06:13:58.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe11ae00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fe11416b4d0 con 0x7fe1140cefa0 2024-09-21T06:13:58.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe113e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1140749a0 0x7fe114145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe113e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1140749a0 0x7fe114145b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59878/0 (socket says 172.21.15.50:59878) 2024-09-21T06:13:58.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe113e00640 1 -- 172.21.15.50:0/3105056844 learned_addr learned my addr 172.21.15.50:0/3105056844 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:58.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe10be00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1140b3340 0x7fe1140aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe118e00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe1140cefa0 0x7fe114159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe113e00640 1 -- 172.21.15.50:0/3105056844 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe1140cefa0 msgr2=0x7fe114159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe113e00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fe1140cefa0 0x7fe114159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe113e00640 1 -- 172.21.15.50:0/3105056844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1140b3340 msgr2=0x7fe1140aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe113e00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1140b3340 0x7fe1140aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe113e00640 1 -- 172.21.15.50:0/3105056844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1141460d0 con 0x7fe1140749a0 2024-09-21T06:13:58.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe113e00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1140749a0 0x7fe114145b90 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe10400c400 tx=0x7fe10400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:58.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe110c00640 1 -- 172.21.15.50:0/3105056844 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104017070 con 0x7fe1140749a0 2024-09-21T06:13:58.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe110c00640 1 -- 172.21.15.50:0/3105056844 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe104003600 con 0x7fe1140749a0 2024-09-21T06:13:58.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe110c00640 1 -- 172.21.15.50:0/3105056844 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104003a50 con 0x7fe1140749a0 2024-09-21T06:13:58.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe11ae00640 1 -- 172.21.15.50:0/3105056844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1140cec00 con 0x7fe1140749a0 2024-09-21T06:13:58.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe11ae00640 1 -- 172.21.15.50:0/3105056844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1141fcc80 con 0x7fe1140749a0 2024-09-21T06:13:58.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.776+0000 7fe11ae00640 1 -- 172.21.15.50:0/3105056844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe11406b540 con 0x7fe1140749a0 2024-09-21T06:13:58.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.780+0000 7fe110c00640 1 -- 172.21.15.50:0/3105056844 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fe10401a070 con 0x7fe1140749a0 2024-09-21T06:13:58.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.780+0000 7fe110c00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe0f00759f0 0x7fe0f0077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.780+0000 7fe110c00640 1 -- 172.21.15.50:0/3105056844 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7fe104092e90 con 0x7fe1140749a0 2024-09-21T06:13:58.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.780+0000 7fe110c00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] conn(0x7fe0f007b900 0x7fe0f007ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.780+0000 7fe110c00640 1 -- 172.21.15.50:0/3105056844 --> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0f007e480 con 0x7fe0f007b900 2024-09-21T06:13:58.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.780+0000 7fe110c00640 1 -- 172.21.15.50:0/3105056844 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fe1040935f0 con 0x7fe1140749a0 2024-09-21T06:13:58.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.780+0000 7fe10be00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe0f00759f0 0x7fe0f0077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.780+0000 7fe118e00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] conn(0x7fe0f007b900 0x7fe0f007ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.780+0000 7fe10be00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe0f00759f0 0x7fe0f0077f00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0af000 tx=0x7fe0fc01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:58.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.780+0000 7fe118e00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] conn(0x7fe0f007b900 0x7fe0f007ddb0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:58.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.780+0000 7fe110c00640 1 -- 172.21.15.50:0/3105056844 <== osd.3 v2:172.21.15.50:6809/599598977 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe0f007e480 con 0x7fe0f007b900 2024-09-21T06:13:58.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe11ae00640 1 -- 172.21.15.50:0/3105056844 --> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe1140cf3b0 con 0x7fe0f007b900 2024-09-21T06:13:58.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe110c00640 1 -- 172.21.15.50:0/3105056844 <== osd.3 v2:172.21.15.50:6809/599598977 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe1140cf3b0 con 0x7fe0f007b900 2024-09-21T06:13:58.792 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:58.792 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:13:58.792 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:58.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe11ae00640 1 -- 172.21.15.50:0/3105056844 >> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] conn(0x7fe0f007b900 msgr2=0x7fe0f007ddb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe11ae00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.50:6809/599598977,v1:172.21.15.50:6811/599598977] conn(0x7fe0f007b900 0x7fe0f007ddb0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe11ae00640 1 -- 172.21.15.50:0/3105056844 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe0f00759f0 msgr2=0x7fe0f0077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe11ae00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fe0f00759f0 0x7fe0f0077f00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0af000 tx=0x7fe0fc01b060 comp rx=0 tx=0).stop 2024-09-21T06:13:58.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe11ae00640 1 -- 172.21.15.50:0/3105056844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1140749a0 msgr2=0x7fe114145b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe11ae00640 1 --2- 172.21.15.50:0/3105056844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1140749a0 0x7fe114145b90 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe10400c400 tx=0x7fe10400c8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe118e00640 1 -- 172.21.15.50:0/3105056844 reap_dead start 2024-09-21T06:13:58.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe11ae00640 1 -- 172.21.15.50:0/3105056844 shutdown_connections 2024-09-21T06:13:58.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe11ae00640 1 -- 172.21.15.50:0/3105056844 >> 172.21.15.50:0/3105056844 conn(0x7fe1140b2040 msgr2=0x7fe1140b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:58.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe11ae00640 1 -- 172.21.15.50:0/3105056844 shutdown_connections 2024-09-21T06:13:58.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.796+0000 7fe11ae00640 1 -- 172.21.15.50:0/3105056844 wait complete. 2024-09-21T06:13:58.811 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T06:13:58.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.976+0000 7f53a0a00640 1 -- 172.21.15.50:0/2665504623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f539c0b5c70 msgr2=0x7f539c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.976+0000 7f53a0a00640 1 --2- 172.21.15.50:0/2665504623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f539c0b5c70 0x7f539c15bb60 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f538c009900 tx=0x7f538c01c8b0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.976+0000 7f53a0a00640 1 -- 172.21.15.50:0/2665504623 shutdown_connections 2024-09-21T06:13:58.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.976+0000 7f53a0a00640 1 --2- 172.21.15.50:0/2665504623 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f539c0b4de0 0x7f539c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.976+0000 7f53a0a00640 1 --2- 172.21.15.50:0/2665504623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f539c0b5c70 0x7f539c15bb60 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.976+0000 7f53a0a00640 1 -- 172.21.15.50:0/2665504623 >> 172.21.15.50:0/2665504623 conn(0x7f539c0b2040 msgr2=0x7f539c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:58.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.976+0000 7f53a0a00640 1 -- 172.21.15.50:0/2665504623 shutdown_connections 2024-09-21T06:13:58.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.976+0000 7f53a0a00640 1 -- 172.21.15.50:0/2665504623 wait complete. 2024-09-21T06:13:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.976+0000 7f53a0a00640 1 Processor -- start 2024-09-21T06:13:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.976+0000 7f53a0a00640 1 -- start start 2024-09-21T06:13:58.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f53a0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f539c0b4de0 0x7f539c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f53a0a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f539c0b5c70 0x7f539c0b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f53a0a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c1527b0 0x7f539c0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f53a0a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f539c168a20 con 0x7f539c0b4de0 2024-09-21T06:13:58.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f53a0a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1688a0 con 0x7f539c1527b0 2024-09-21T06:13:58.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f53a0a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f539c168ba0 con 0x7f539c0b5c70 2024-09-21T06:13:58.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5399e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f539c0b4de0 0x7f539c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5399e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f539c0b4de0 0x7f539c0cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59894/0 (socket says 172.21.15.50:59894) 2024-09-21T06:13:58.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5399e00640 1 -- 172.21.15.50:0/2448530250 learned_addr learned my addr 172.21.15.50:0/2448530250 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f539a800640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c1527b0 0x7f539c0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5399e00640 1 -- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f539c0b5c70 msgr2=0x7f539c0b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5393e00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f539c0b5c70 0x7f539c0b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5399e00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f539c0b5c70 0x7f539c0b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5399e00640 1 -- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c1527b0 msgr2=0x7f539c0b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5399e00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c1527b0 0x7f539c0b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5399e00640 1 -- 172.21.15.50:0/2448530250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c0b6330 con 0x7f539c0b4de0 2024-09-21T06:13:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f539a800640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c1527b0 0x7f539c0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5393e00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f539c0b5c70 0x7f539c0b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5399e00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f539c0b4de0 0x7f539c0cdc60 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f538c009850 tx=0x7f538c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5392000640 1 -- 172.21.15.50:0/2448530250 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c0b2040 con 0x7f539c0b4de0 2024-09-21T06:13:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5392000640 1 -- 172.21.15.50:0/2448530250 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f538c005320 con 0x7f539c0b4de0 2024-09-21T06:13:58.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5392000640 1 -- 172.21.15.50:0/2448530250 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c0046d0 con 0x7f539c0b4de0 2024-09-21T06:13:58.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f53a0a00640 1 -- 172.21.15.50:0/2448530250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c15c2c0 con 0x7f539c0b4de0 2024-09-21T06:13:58.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f53a0a00640 1 -- 172.21.15.50:0/2448530250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c0ce6e0 con 0x7f539c0b4de0 2024-09-21T06:13:58.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f53a0a00640 1 -- 172.21.15.50:0/2448530250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c0b51f0 con 0x7f539c0b4de0 2024-09-21T06:13:58.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.980+0000 7f5392000640 1 -- 172.21.15.50:0/2448530250 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f538c0bd060 con 0x7f539c0b4de0 2024-09-21T06:13:58.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.984+0000 7f5392000640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53780759f0 0x7f5378077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:58.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.984+0000 7f5392000640 1 -- 172.21.15.50:0/2448530250 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f538c12d510 con 0x7f539c0b4de0 2024-09-21T06:13:58.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.984+0000 7f5393e00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53780759f0 0x7f5378077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:58.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.984+0000 7f5393e00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53780759f0 0x7f5378077f00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f539c0b2850 tx=0x7f5384009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:58.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:58.988+0000 7f5392000640 1 -- 172.21.15.50:0/2448530250 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f538c0164d0 con 0x7f539c0b4de0 2024-09-21T06:13:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f53a0a00640 1 -- 172.21.15.50:0/2448530250 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f539c06b540 con 0x7f53780759f0 2024-09-21T06:13:59.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f5392000640 1 -- 172.21.15.50:0/2448530250 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5384001c60 con 0x7f53780759f0 2024-09-21T06:13:59.160 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T06:13:59.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f53a0a00640 1 -- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53780759f0 msgr2=0x7f5378077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f53a0a00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53780759f0 0x7f5378077f00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f539c0b2850 tx=0x7f5384009040 comp rx=0 tx=0).stop 2024-09-21T06:13:59.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f53a0a00640 1 -- 172.21.15.50:0/2448530250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f539c0b4de0 msgr2=0x7f539c0cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f53a0a00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f539c0b4de0 0x7f539c0cdc60 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f538c009850 tx=0x7f538c005760 comp rx=0 tx=0).stop 2024-09-21T06:13:59.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f53a0a00640 1 -- 172.21.15.50:0/2448530250 shutdown_connections 2024-09-21T06:13:59.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f53a0a00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c1527b0 0x7f539c0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f53a0a00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f539c0b5c70 0x7f539c0b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f53a0a00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53780759f0 0x7f5378077f00 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f53a0a00640 1 --2- 172.21.15.50:0/2448530250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f539c0b4de0 0x7f539c0cdc60 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f53a0a00640 1 -- 172.21.15.50:0/2448530250 >> 172.21.15.50:0/2448530250 conn(0x7f539c0b2040 msgr2=0x7f539c14d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:59.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.164+0000 7f53a0a00640 1 -- 172.21.15.50:0/2448530250 shutdown_connections 2024-09-21T06:13:59.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.168+0000 7f53a0a00640 1 -- 172.21.15.50:0/2448530250 wait complete. 2024-09-21T06:13:59.183 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T06:13:59.184 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:13:59.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.352+0000 7fc639600640 1 -- 172.21.15.50:0/533920179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc634157030 msgr2=0x7fc6340b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.352+0000 7fc639600640 1 --2- 172.21.15.50:0/533920179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc634157030 0x7fc6340b5a80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc624009840 tx=0x7fc62401c790 comp rx=0 tx=0).stop 2024-09-21T06:13:59.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.352+0000 7fc639600640 1 -- 172.21.15.50:0/533920179 shutdown_connections 2024-09-21T06:13:59.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.352+0000 7fc639600640 1 --2- 172.21.15.50:0/533920179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc634157030 0x7fc6340b5a80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.352+0000 7fc639600640 1 -- 172.21.15.50:0/533920179 >> 172.21.15.50:0/533920179 conn(0x7fc6340708c0 msgr2=0x7fc634070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:59.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.352+0000 7fc639600640 1 -- 172.21.15.50:0/533920179 shutdown_connections 2024-09-21T06:13:59.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.352+0000 7fc639600640 1 -- 172.21.15.50:0/533920179 wait complete. 2024-09-21T06:13:59.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.352+0000 7fc639600640 1 Processor -- start 2024-09-21T06:13:59.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.352+0000 7fc639600640 1 -- start start 2024-09-21T06:13:59.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc639600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc634157030 0x7fc634161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc639600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc634160d50 0x7fc634142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc639600640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc634151010 0x7fc6340d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc639600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc63416b8f0 con 0x7fc634157030 2024-09-21T06:13:59.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc639600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc63416b770 con 0x7fc634160d50 2024-09-21T06:13:59.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc639600640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fc63416ba70 con 0x7fc634151010 2024-09-21T06:13:59.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc632a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc634157030 0x7fc634161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc632a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc634157030 0x7fc634161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59904/0 (socket says 172.21.15.50:59904) 2024-09-21T06:13:59.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc632a00640 1 -- 172.21.15.50:0/122096539 learned_addr learned my addr 172.21.15.50:0/122096539 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc62be00640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc634160d50 0x7fc634142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc633400640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc634151010 0x7fc6340d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc632a00640 1 -- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc634151010 msgr2=0x7fc6340d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc632a00640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc634151010 0x7fc6340d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc632a00640 1 -- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc634160d50 msgr2=0x7fc634142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc632a00640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc634160d50 0x7fc634142df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc632a00640 1 -- 172.21.15.50:0/122096539 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc634161e40 con 0x7fc634157030 2024-09-21T06:13:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc62be00640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc634160d50 0x7fc634142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc633400640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fc634151010 0x7fc6340d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc632a00640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc634157030 0x7fc634161680 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fc624009790 tx=0x7fc6240a6460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc62aa00640 1 -- 172.21.15.50:0/122096539 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624004240 con 0x7fc634157030 2024-09-21T06:13:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc62aa00640 1 -- 172.21.15.50:0/122096539 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6240043e0 con 0x7fc634157030 2024-09-21T06:13:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc639600640 1 -- 172.21.15.50:0/122096539 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc634162120 con 0x7fc634157030 2024-09-21T06:13:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc62aa00640 1 -- 172.21.15.50:0/122096539 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6240052a0 con 0x7fc634157030 2024-09-21T06:13:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc639600640 1 -- 172.21.15.50:0/122096539 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6341646a0 con 0x7fc634157030 2024-09-21T06:13:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.356+0000 7fc639600640 1 -- 172.21.15.50:0/122096539 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc63406b540 con 0x7fc634157030 2024-09-21T06:13:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.360+0000 7fc62aa00640 1 -- 172.21.15.50:0/122096539 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fc624005440 con 0x7fc634157030 2024-09-21T06:13:59.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.360+0000 7fc62aa00640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc6100759f0 0x7fc610077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.360+0000 7fc62aa00640 1 -- 172.21.15.50:0/122096539 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7fc62412d9b0 con 0x7fc634157030 2024-09-21T06:13:59.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.360+0000 7fc62aa00640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] conn(0x7fc61007b900 0x7fc61007ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.360+0000 7fc62aa00640 1 -- 172.21.15.50:0/122096539 --> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc61007e480 con 0x7fc61007b900 2024-09-21T06:13:59.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.360+0000 7fc62aa00640 1 -- 172.21.15.50:0/122096539 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fc62412e0f0 con 0x7fc634157030 2024-09-21T06:13:59.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.360+0000 7fc62be00640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc6100759f0 0x7fc610077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.360+0000 7fc633400640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] conn(0x7fc61007b900 0x7fc61007ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.360+0000 7fc62be00640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc6100759f0 0x7fc610077f00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc6341fc8a0 tx=0x7fc61c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:59.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.360+0000 7fc633400640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] conn(0x7fc61007b900 0x7fc61007ddb0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:59.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.360+0000 7fc62aa00640 1 -- 172.21.15.50:0/122096539 <== osd.4 v2:172.21.15.88:6826/2441482427 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc61007e480 con 0x7fc61007b900 2024-09-21T06:13:59.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc639600640 1 -- 172.21.15.50:0/122096539 --> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc6340b5a80 con 0x7fc61007b900 2024-09-21T06:13:59.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc62aa00640 1 -- 172.21.15.50:0/122096539 <== osd.4 v2:172.21.15.88:6826/2441482427 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc6340b5a80 con 0x7fc61007b900 2024-09-21T06:13:59.372 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:59.372 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:13:59.372 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:59.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc639600640 1 -- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] conn(0x7fc61007b900 msgr2=0x7fc61007ddb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc639600640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:6826/2441482427,v1:172.21.15.88:6827/2441482427] conn(0x7fc61007b900 0x7fc61007ddb0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc639600640 1 -- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc6100759f0 msgr2=0x7fc610077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc639600640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fc6100759f0 0x7fc610077f00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc6341fc8a0 tx=0x7fc61c009040 comp rx=0 tx=0).stop 2024-09-21T06:13:59.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc639600640 1 -- 172.21.15.50:0/122096539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc634157030 msgr2=0x7fc634161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc639600640 1 --2- 172.21.15.50:0/122096539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc634157030 0x7fc634161680 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fc624009790 tx=0x7fc6240a6460 comp rx=0 tx=0).stop 2024-09-21T06:13:59.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc633400640 1 -- 172.21.15.50:0/122096539 reap_dead start 2024-09-21T06:13:59.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc639600640 1 -- 172.21.15.50:0/122096539 shutdown_connections 2024-09-21T06:13:59.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc639600640 1 -- 172.21.15.50:0/122096539 >> 172.21.15.50:0/122096539 conn(0x7fc6340708c0 msgr2=0x7fc63415bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:59.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc639600640 1 -- 172.21.15.50:0/122096539 shutdown_connections 2024-09-21T06:13:59.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.376+0000 7fc639600640 1 -- 172.21.15.50:0/122096539 wait complete. 2024-09-21T06:13:59.391 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T06:13:59.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.552+0000 7ff2c7200640 1 -- 172.21.15.50:0/3502512896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2c00cefa0 msgr2=0x7ff2c00cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.552+0000 7ff2c7200640 1 --2- 172.21.15.50:0/3502512896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2c00cefa0 0x7ff2c00cf3b0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00cfc90 tx=0x7ff2a801c800 comp rx=0 tx=0).stop 2024-09-21T06:13:59.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.552+0000 7ff2c7200640 1 -- 172.21.15.50:0/3502512896 shutdown_connections 2024-09-21T06:13:59.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.552+0000 7ff2c7200640 1 --2- 172.21.15.50:0/3502512896 >> v2:172.21.15.88:3300/0 conn(0x7ff2c00b3340 0x7ff2c00b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.552+0000 7ff2c7200640 1 --2- 172.21.15.50:0/3502512896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2c00cefa0 0x7ff2c00cf3b0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.552+0000 7ff2c7200640 1 -- 172.21.15.50:0/3502512896 >> 172.21.15.50:0/3502512896 conn(0x7ff2c00b2040 msgr2=0x7ff2c00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:59.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.556+0000 7ff2c7200640 1 -- 172.21.15.50:0/3502512896 shutdown_connections 2024-09-21T06:13:59.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.556+0000 7ff2c7200640 1 -- 172.21.15.50:0/3502512896 wait complete. 2024-09-21T06:13:59.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.556+0000 7ff2c7200640 1 Processor -- start 2024-09-21T06:13:59.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.556+0000 7ff2c7200640 1 -- start start 2024-09-21T06:13:59.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.556+0000 7ff2c7200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2c00749a0 0x7ff2c0144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.556+0000 7ff2c7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2c00b3340 0x7ff2c013ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.556+0000 7ff2c7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2c00cefa0 0x7ff2c013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.556+0000 7ff2c7200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0169cd0 con 0x7ff2c00b3340 2024-09-21T06:13:59.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.556+0000 7ff2c7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0169b50 con 0x7ff2c00cefa0 2024-09-21T06:13:59.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.556+0000 7ff2c7200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c0169e50 con 0x7ff2c00749a0 2024-09-21T06:13:59.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2b7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2c00b3340 0x7ff2c013ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2b7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2c00b3340 0x7ff2c013ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59922/0 (socket says 172.21.15.50:59922) 2024-09-21T06:13:59.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2b7e00640 1 -- 172.21.15.50:0/2456023446 learned_addr learned my addr 172.21.15.50:0/2456023446 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:59.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2bfe00640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2c00749a0 0x7ff2c0144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2c5200640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2c00cefa0 0x7ff2c013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2b7e00640 1 -- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2c00749a0 msgr2=0x7ff2c0144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2b7e00640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2c00749a0 0x7ff2c0144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2b7e00640 1 -- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2c00cefa0 msgr2=0x7ff2c013b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2b7e00640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2c00cefa0 0x7ff2c013b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2b7e00640 1 -- 172.21.15.50:0/2456023446 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c01402a0 con 0x7ff2c00b3340 2024-09-21T06:13:59.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2b7e00640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2c00b3340 0x7ff2c013ac60 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7ff2a801cce0 tx=0x7ff2a8005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:59.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2bcc00640 1 -- 172.21.15.50:0/2456023446 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80b2040 con 0x7ff2c00b3340 2024-09-21T06:13:59.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2bcc00640 1 -- 172.21.15.50:0/2456023446 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2a80bb070 con 0x7ff2c00b3340 2024-09-21T06:13:59.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2c7200640 1 -- 172.21.15.50:0/2456023446 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c0140580 con 0x7ff2c00b3340 2024-09-21T06:13:59.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2bcc00640 1 -- 172.21.15.50:0/2456023446 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8003600 con 0x7ff2c00b3340 2024-09-21T06:13:59.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2c7200640 1 -- 172.21.15.50:0/2456023446 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c0140b40 con 0x7ff2c00b3340 2024-09-21T06:13:59.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2c7200640 1 -- 172.21.15.50:0/2456023446 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c00b3750 con 0x7ff2c00b3340 2024-09-21T06:13:59.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.560+0000 7ff2bcc00640 1 -- 172.21.15.50:0/2456023446 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7ff2a80037a0 con 0x7ff2c00b3340 2024-09-21T06:13:59.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.564+0000 7ff2bcc00640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff29c075a20 0x7ff29c077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.564+0000 7ff2bcc00640 1 -- 172.21.15.50:0/2456023446 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7ff2a812e270 con 0x7ff2c00b3340 2024-09-21T06:13:59.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.564+0000 7ff2bfe00640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff29c075a20 0x7ff29c077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.564+0000 7ff2bfe00640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff29c075a20 0x7ff29c077f30 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0007fd0 tx=0x7ff2b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:59.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.568+0000 7ff2bcc00640 1 -- 172.21.15.50:0/2456023446 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff2a80163f0 con 0x7ff2c00b3340 2024-09-21T06:13:59.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.740+0000 7ff2c7200640 1 -- 172.21.15.50:0/2456023446 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7ff2c006b540 con 0x7ff29c075a20 2024-09-21T06:13:59.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2bcc00640 1 -- 172.21.15.50:0/2456023446 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff2b0001470 con 0x7ff29c075a20 2024-09-21T06:13:59.739 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T06:13:59.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2c7200640 1 -- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff29c075a20 msgr2=0x7ff29c077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2c7200640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff29c075a20 0x7ff29c077f30 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0007fd0 tx=0x7ff2b0009040 comp rx=0 tx=0).stop 2024-09-21T06:13:59.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2c7200640 1 -- 172.21.15.50:0/2456023446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2c00b3340 msgr2=0x7ff2c013ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2c7200640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2c00b3340 0x7ff2c013ac60 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7ff2a801cce0 tx=0x7ff2a8005d20 comp rx=0 tx=0).stop 2024-09-21T06:13:59.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2c7200640 1 -- 172.21.15.50:0/2456023446 shutdown_connections 2024-09-21T06:13:59.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2c7200640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2c00cefa0 0x7ff2c013b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2c7200640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7ff29c075a20 0x7ff29c077f30 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2c7200640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2c00b3340 0x7ff2c013ac60 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2c7200640 1 --2- 172.21.15.50:0/2456023446 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7ff2c00749a0 0x7ff2c0144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2c7200640 1 -- 172.21.15.50:0/2456023446 >> 172.21.15.50:0/2456023446 conn(0x7ff2c00b2040 msgr2=0x7ff2c0075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:59.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2c7200640 1 -- 172.21.15.50:0/2456023446 shutdown_connections 2024-09-21T06:13:59.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.744+0000 7ff2c7200640 1 -- 172.21.15.50:0/2456023446 wait complete. 2024-09-21T06:13:59.762 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T06:13:59.763 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:13:59.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.928+0000 7fd7d2400640 1 -- 172.21.15.50:0/788431958 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7cc0b4de0 msgr2=0x7fd7cc0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.928+0000 7fd7d2400640 1 --2- 172.21.15.50:0/788431958 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7cc0b4de0 0x7fd7cc0b51f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009900 tx=0x7fd7b401c8b0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.932+0000 7fd7d2400640 1 -- 172.21.15.50:0/788431958 shutdown_connections 2024-09-21T06:13:59.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.932+0000 7fd7d2400640 1 --2- 172.21.15.50:0/788431958 >> v2:172.21.15.88:3300/0 conn(0x7fd7cc15cc60 0x7fd7cc15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.932+0000 7fd7d2400640 1 --2- 172.21.15.50:0/788431958 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7cc0b4de0 0x7fd7cc0b51f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.932+0000 7fd7d2400640 1 -- 172.21.15.50:0/788431958 >> 172.21.15.50:0/788431958 conn(0x7fd7cc0b2040 msgr2=0x7fd7cc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:59.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.932+0000 7fd7d2400640 1 -- 172.21.15.50:0/788431958 shutdown_connections 2024-09-21T06:13:59.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.932+0000 7fd7d2400640 1 -- 172.21.15.50:0/788431958 wait complete. 2024-09-21T06:13:59.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.932+0000 7fd7d2400640 1 Processor -- start 2024-09-21T06:13:59.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.932+0000 7fd7d2400640 1 -- start start 2024-09-21T06:13:59.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.932+0000 7fd7d2400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7cc0738b0 0x7fd7cc1fc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.932+0000 7fd7d2400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7cc0b4de0 0x7fd7cc14f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.932+0000 7fd7d2400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7cc15cc60 0x7fd7cc0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7d2400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc168eb0 con 0x7fd7cc15cc60 2024-09-21T06:13:59.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7d2400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc168d30 con 0x7fd7cc0738b0 2024-09-21T06:13:59.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7d2400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd7cc169030 con 0x7fd7cc0b4de0 2024-09-21T06:13:59.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7cbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7cc15cc60 0x7fd7cc0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7cbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7cc15cc60 0x7fd7cc0cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59936/0 (socket says 172.21.15.50:59936) 2024-09-21T06:13:59.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7cbe00640 1 -- 172.21.15.50:0/2404792944 learned_addr learned my addr 172.21.15.50:0/2404792944 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:13:59.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7cb400640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7cc0738b0 0x7fd7cc1fc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7c3e00640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7cc0b4de0 0x7fd7cc14f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7cbe00640 1 -- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7cc0b4de0 msgr2=0x7fd7cc14f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7cbe00640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd7cc0b4de0 0x7fd7cc14f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7cbe00640 1 -- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7cc0738b0 msgr2=0x7fd7cc1fc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7cbe00640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7cc0738b0 0x7fd7cc1fc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7cbe00640 1 -- 172.21.15.50:0/2404792944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7cc1fc880 con 0x7fd7cc15cc60 2024-09-21T06:13:59.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7cbe00640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7cc15cc60 0x7fd7cc0cb630 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009fd0 tx=0x7fd7c400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:59.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7c3400640 1 -- 172.21.15.50:0/2404792944 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400ce20 con 0x7fd7cc15cc60 2024-09-21T06:13:59.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7c3400640 1 -- 172.21.15.50:0/2404792944 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7c4004b30 con 0x7fd7cc15cc60 2024-09-21T06:13:59.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7c3400640 1 -- 172.21.15.50:0/2404792944 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4005020 con 0x7fd7cc15cc60 2024-09-21T06:13:59.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7d2400640 1 -- 172.21.15.50:0/2404792944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7cc1fcb60 con 0x7fd7cc15cc60 2024-09-21T06:13:59.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7d2400640 1 -- 172.21.15.50:0/2404792944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7cc152ca0 con 0x7fd7cc15cc60 2024-09-21T06:13:59.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7d2400640 1 -- 172.21.15.50:0/2404792944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd7cc06b540 con 0x7fd7cc15cc60 2024-09-21T06:13:59.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.936+0000 7fd7c3400640 1 -- 172.21.15.50:0/2404792944 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fd7c40051c0 con 0x7fd7cc15cc60 2024-09-21T06:13:59.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.940+0000 7fd7c3400640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7a80759f0 0x7fd7a8077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.940+0000 7fd7c3400640 1 -- 172.21.15.50:0/2404792944 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7fd7c4094790 con 0x7fd7cc15cc60 2024-09-21T06:13:59.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.940+0000 7fd7c3400640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] conn(0x7fd7a807b960 0x7fd7a807de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:59.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.940+0000 7fd7c3400640 1 -- 172.21.15.50:0/2404792944 --> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7c400be50 con 0x7fd7a807b960 2024-09-21T06:13:59.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.940+0000 7fd7c3400640 1 -- 172.21.15.50:0/2404792944 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fd7c4094b40 con 0x7fd7cc15cc60 2024-09-21T06:13:59.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.940+0000 7fd7cb400640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7a80759f0 0x7fd7a8077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.940+0000 7fd7c3e00640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] conn(0x7fd7a807b960 0x7fd7a807de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:59.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.940+0000 7fd7cb400640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7a80759f0 0x7fd7a8077f00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc007fd0 tx=0x7fd7bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:59.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.940+0000 7fd7c3e00640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] conn(0x7fd7a807b960 0x7fd7a807de10 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:59.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.940+0000 7fd7c3400640 1 -- 172.21.15.50:0/2404792944 <== osd.5 v2:172.21.15.88:6802/3846197193 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7c400be50 con 0x7fd7a807b960 2024-09-21T06:13:59.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7d2400640 1 -- 172.21.15.50:0/2404792944 --> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd7cc0b51f0 con 0x7fd7a807b960 2024-09-21T06:13:59.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7c3400640 1 -- 172.21.15.50:0/2404792944 <== osd.5 v2:172.21.15.88:6802/3846197193 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd7cc0b51f0 con 0x7fd7a807b960 2024-09-21T06:13:59.952 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:13:59.952 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:13:59.952 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:13:59.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7d2400640 1 -- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] conn(0x7fd7a807b960 msgr2=0x7fd7a807de10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7d2400640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:6802/3846197193,v1:172.21.15.88:6803/3846197193] conn(0x7fd7a807b960 0x7fd7a807de10 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:59.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7d2400640 1 -- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7a80759f0 msgr2=0x7fd7a8077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7d2400640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd7a80759f0 0x7fd7a8077f00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc007fd0 tx=0x7fd7bc009040 comp rx=0 tx=0).stop 2024-09-21T06:13:59.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7d2400640 1 -- 172.21.15.50:0/2404792944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7cc15cc60 msgr2=0x7fd7cc0cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:59.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7d2400640 1 --2- 172.21.15.50:0/2404792944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7cc15cc60 0x7fd7cc0cb630 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009fd0 tx=0x7fd7c400ef10 comp rx=0 tx=0).stop 2024-09-21T06:13:59.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7cbe00640 1 -- 172.21.15.50:0/2404792944 reap_dead start 2024-09-21T06:13:59.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7d2400640 1 -- 172.21.15.50:0/2404792944 shutdown_connections 2024-09-21T06:13:59.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7d2400640 1 -- 172.21.15.50:0/2404792944 >> 172.21.15.50:0/2404792944 conn(0x7fd7cc0b2040 msgr2=0x7fd7cc1519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:59.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7d2400640 1 -- 172.21.15.50:0/2404792944 shutdown_connections 2024-09-21T06:13:59.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:13:59.956+0000 7fd7d2400640 1 -- 172.21.15.50:0/2404792944 wait complete. 2024-09-21T06:13:59.971 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T06:14:00.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.148+0000 7fa498000640 1 -- 172.21.15.50:0/1825587233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4900b4de0 msgr2=0x7fa4900b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.148+0000 7fa498000640 1 --2- 172.21.15.50:0/1825587233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4900b4de0 0x7fa4900b51f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa488009bc0 tx=0x7fa48801cb70 comp rx=0 tx=0).stop 2024-09-21T06:14:00.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.148+0000 7fa498000640 1 -- 172.21.15.50:0/1825587233 shutdown_connections 2024-09-21T06:14:00.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.148+0000 7fa498000640 1 --2- 172.21.15.50:0/1825587233 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4900b4de0 0x7fa4900b51f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.148+0000 7fa498000640 1 --2- 172.21.15.50:0/1825587233 >> v2:172.21.15.88:3300/0 conn(0x7fa4901527b0 0x7fa49015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.148+0000 7fa498000640 1 -- 172.21.15.50:0/1825587233 >> 172.21.15.50:0/1825587233 conn(0x7fa4900b2040 msgr2=0x7fa4900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:00.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.148+0000 7fa498000640 1 -- 172.21.15.50:0/1825587233 shutdown_connections 2024-09-21T06:14:00.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.148+0000 7fa498000640 1 -- 172.21.15.50:0/1825587233 wait complete. 2024-09-21T06:14:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.148+0000 7fa498000640 1 Processor -- start 2024-09-21T06:14:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa498000640 1 -- start start 2024-09-21T06:14:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa498000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4900b4de0 0x7fa49013f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa498000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4900b5c70 0x7fa49013fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa498000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4901527b0 0x7fa490140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa498000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa4900b4420 con 0x7fa4900b4de0 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa498000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa4900b42a0 con 0x7fa4900b5c70 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa498000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fa4900b45a0 con 0x7fa4901527b0 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa495600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4900b4de0 0x7fa49013f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa495600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4900b4de0 0x7fa49013f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59940/0 (socket says 172.21.15.50:59940) 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa495600640 1 -- 172.21.15.50:0/1734371554 learned_addr learned my addr 172.21.15.50:0/1734371554 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa496000640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4901527b0 0x7fa490140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa494c00640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4900b5c70 0x7fa49013fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa495600640 1 -- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4901527b0 msgr2=0x7fa490140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa495600640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4901527b0 0x7fa490140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa495600640 1 -- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4900b5c70 msgr2=0x7fa49013fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa495600640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4900b5c70 0x7fa49013fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa495600640 1 -- 172.21.15.50:0/1734371554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa490157b00 con 0x7fa4900b4de0 2024-09-21T06:14:00.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa494c00640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4900b5c70 0x7fa49013fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:00.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa495600640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4900b4de0 0x7fa49013f7f0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fa48400c730 tx=0x7fa48400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:00.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa48d600640 1 -- 172.21.15.50:0/1734371554 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484016020 con 0x7fa4900b4de0 2024-09-21T06:14:00.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa48d600640 1 -- 172.21.15.50:0/1734371554 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa48400a950 con 0x7fa4900b4de0 2024-09-21T06:14:00.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa498000640 1 -- 172.21.15.50:0/1734371554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa490157de0 con 0x7fa4900b4de0 2024-09-21T06:14:00.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa48d600640 1 -- 172.21.15.50:0/1734371554 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48400ada0 con 0x7fa4900b4de0 2024-09-21T06:14:00.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.152+0000 7fa498000640 1 -- 172.21.15.50:0/1734371554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49013f2d0 con 0x7fa4900b4de0 2024-09-21T06:14:00.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.156+0000 7fa498000640 1 -- 172.21.15.50:0/1734371554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4900b4b40 con 0x7fa4900b4de0 2024-09-21T06:14:00.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.156+0000 7fa48d600640 1 -- 172.21.15.50:0/1734371554 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fa484019020 con 0x7fa4900b4de0 2024-09-21T06:14:00.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.156+0000 7fa48d600640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4700759f0 0x7fa470077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.156+0000 7fa48d600640 1 -- 172.21.15.50:0/1734371554 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7fa4840934f0 con 0x7fa4900b4de0 2024-09-21T06:14:00.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.156+0000 7fa494c00640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4700759f0 0x7fa470077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.156+0000 7fa494c00640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4700759f0 0x7fa470077f00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa49015a8c0 tx=0x7fa47c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:00.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.160+0000 7fa48d600640 1 -- 172.21.15.50:0/1734371554 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa484020020 con 0x7fa4900b4de0 2024-09-21T06:14:00.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.336+0000 7fa498000640 1 -- 172.21.15.50:0/1734371554 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fa49006b540 con 0x7fa4700759f0 2024-09-21T06:14:00.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa48d600640 1 -- 172.21.15.50:0/1734371554 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa47c010ab0 con 0x7fa4700759f0 2024-09-21T06:14:00.335 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T06:14:00.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa498000640 1 -- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4700759f0 msgr2=0x7fa470077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa498000640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4700759f0 0x7fa470077f00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa49015a8c0 tx=0x7fa47c0091e0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa498000640 1 -- 172.21.15.50:0/1734371554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4900b4de0 msgr2=0x7fa49013f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa498000640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4900b4de0 0x7fa49013f7f0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fa48400c730 tx=0x7fa48400cbf0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa498000640 1 -- 172.21.15.50:0/1734371554 shutdown_connections 2024-09-21T06:14:00.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa498000640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fa4901527b0 0x7fa490140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa498000640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4900b5c70 0x7fa49013fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa498000640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fa4700759f0 0x7fa470077f00 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa498000640 1 --2- 172.21.15.50:0/1734371554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4900b4de0 0x7fa49013f7f0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa498000640 1 -- 172.21.15.50:0/1734371554 >> 172.21.15.50:0/1734371554 conn(0x7fa4900b2040 msgr2=0x7fa49015b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:00.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa498000640 1 -- 172.21.15.50:0/1734371554 shutdown_connections 2024-09-21T06:14:00.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.340+0000 7fa498000640 1 -- 172.21.15.50:0/1734371554 wait complete. 2024-09-21T06:14:00.357 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T06:14:00.358 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:14:00.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.524+0000 7fb08bc00640 1 -- 172.21.15.50:0/4032956372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb08415c730 msgr2=0x7fb0840b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.524+0000 7fb08bc00640 1 --2- 172.21.15.50:0/4032956372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb08415c730 0x7fb0840b3710 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fb078009930 tx=0x7fb07801c8e0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.524+0000 7fb08bc00640 1 -- 172.21.15.50:0/4032956372 shutdown_connections 2024-09-21T06:14:00.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.524+0000 7fb08bc00640 1 --2- 172.21.15.50:0/4032956372 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb084073f90 0x7fb0840743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.524+0000 7fb08bc00640 1 --2- 172.21.15.50:0/4032956372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb08415c730 0x7fb0840b3710 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.524+0000 7fb08bc00640 1 -- 172.21.15.50:0/4032956372 >> 172.21.15.50:0/4032956372 conn(0x7fb0840d06e0 msgr2=0x7fb0840d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:00.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.524+0000 7fb08bc00640 1 -- 172.21.15.50:0/4032956372 shutdown_connections 2024-09-21T06:14:00.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.524+0000 7fb08bc00640 1 -- 172.21.15.50:0/4032956372 wait complete. 2024-09-21T06:14:00.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb08bc00640 1 Processor -- start 2024-09-21T06:14:00.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb08bc00640 1 -- start start 2024-09-21T06:14:00.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb08bc00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb084073f90 0x7fb0840aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb08bc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0840af520 0x7fb0840cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb08bc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb08415c730 0x7fb0840cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb08bc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb08406f7c0 con 0x7fb08415c730 2024-09-21T06:14:00.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb08bc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb08406f640 con 0x7fb0840af520 2024-09-21T06:14:00.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb08bc00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb08406f940 con 0x7fb084073f90 2024-09-21T06:14:00.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb089c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb08415c730 0x7fb0840cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb089200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb084073f90 0x7fb0840aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb083e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0840af520 0x7fb0840cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb089200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb084073f90 0x7fb0840aaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3301/0 says I am v2:172.21.15.50:57274/0 (socket says 172.21.15.50:57274) 2024-09-21T06:14:00.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb089200640 1 -- 172.21.15.50:0/1185225728 learned_addr learned my addr 172.21.15.50:0/1185225728 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:14:00.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb089200640 1 -- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0840af520 msgr2=0x7fb0840cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb089200640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0840af520 0x7fb0840cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb089200640 1 -- 172.21.15.50:0/1185225728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb08415c730 msgr2=0x7fb0840cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb089200640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb08415c730 0x7fb0840cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb089200640 1 -- 172.21.15.50:0/1185225728 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb084153180 con 0x7fb084073f90 2024-09-21T06:14:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb089c00640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb08415c730 0x7fb0840cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb089200640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb084073f90 0x7fb0840aaaf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb078009850 tx=0x7fb0780a5620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb080c00640 1 -- 172.21.15.50:0/1185225728 <== mon.2 v2:172.21.15.88:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0780b2040 con 0x7fb084073f90 2024-09-21T06:14:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb080c00640 1 -- 172.21.15.50:0/1185225728 <== mon.2 v2:172.21.15.88:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0780043b0 con 0x7fb084073f90 2024-09-21T06:14:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb080c00640 1 -- 172.21.15.50:0/1185225728 <== mon.2 v2:172.21.15.88:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078005280 con 0x7fb084073f90 2024-09-21T06:14:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb08bc00640 1 -- 172.21.15.50:0/1185225728 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb084154020 con 0x7fb084073f90 2024-09-21T06:14:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.528+0000 7fb08bc00640 1 -- 172.21.15.50:0/1185225728 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb084151620 con 0x7fb084073f90 2024-09-21T06:14:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.532+0000 7fb08bc00640 1 -- 172.21.15.50:0/1185225728 --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb08406b540 con 0x7fb084073f90 2024-09-21T06:14:00.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.532+0000 7fb080c00640 1 -- 172.21.15.50:0/1185225728 <== mon.2 v2:172.21.15.88:3301/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fb078005420 con 0x7fb084073f90 2024-09-21T06:14:00.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.532+0000 7fb080c00640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0640759f0 0x7fb064077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.532+0000 7fb080c00640 1 -- 172.21.15.50:0/1185225728 <== mon.2 v2:172.21.15.88:3301/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7fb07812d0d0 con 0x7fb084073f90 2024-09-21T06:14:00.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.532+0000 7fb080c00640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] conn(0x7fb06407b960 0x7fb06407de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.532+0000 7fb080c00640 1 -- 172.21.15.50:0/1185225728 --> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0780b1e50 con 0x7fb06407b960 2024-09-21T06:14:00.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.532+0000 7fb080c00640 1 -- 172.21.15.50:0/1185225728 <== mon.2 v2:172.21.15.88:3301/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fb07812d480 con 0x7fb084073f90 2024-09-21T06:14:00.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.532+0000 7fb083e00640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0640759f0 0x7fb064077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.532+0000 7fb089c00640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] conn(0x7fb06407b960 0x7fb06407de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.532+0000 7fb083e00640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0640759f0 0x7fb064077f00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb0840cfc90 tx=0x7fb07000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:00.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.532+0000 7fb089c00640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] conn(0x7fb06407b960 0x7fb06407de10 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:00.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.536+0000 7fb080c00640 1 -- 172.21.15.50:0/1185225728 <== osd.6 v2:172.21.15.88:6818/2341496649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0780b1e50 con 0x7fb06407b960 2024-09-21T06:14:00.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.548+0000 7fb08bc00640 1 -- 172.21.15.50:0/1185225728 --> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb0840743a0 con 0x7fb06407b960 2024-09-21T06:14:00.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.548+0000 7fb080c00640 1 -- 172.21.15.50:0/1185225728 <== osd.6 v2:172.21.15.88:6818/2341496649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb0840743a0 con 0x7fb06407b960 2024-09-21T06:14:00.546 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:14:00.546 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:14:00.546 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:14:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.548+0000 7fb08bc00640 1 -- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] conn(0x7fb06407b960 msgr2=0x7fb06407de10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.548+0000 7fb08bc00640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:6818/2341496649,v1:172.21.15.88:6819/2341496649] conn(0x7fb06407b960 0x7fb06407de10 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.548+0000 7fb08bc00640 1 -- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0640759f0 msgr2=0x7fb064077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.548+0000 7fb08bc00640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0640759f0 0x7fb064077f00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb0840cfc90 tx=0x7fb07000b2e0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.548+0000 7fb08bc00640 1 -- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb084073f90 msgr2=0x7fb0840aaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.548+0000 7fb08bc00640 1 --2- 172.21.15.50:0/1185225728 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb084073f90 0x7fb0840aaaf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb078009850 tx=0x7fb0780a5620 comp rx=0 tx=0).stop 2024-09-21T06:14:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.548+0000 7fb089c00640 1 -- 172.21.15.50:0/1185225728 reap_dead start 2024-09-21T06:14:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.552+0000 7fb08bc00640 1 -- 172.21.15.50:0/1185225728 shutdown_connections 2024-09-21T06:14:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.552+0000 7fb08bc00640 1 -- 172.21.15.50:0/1185225728 >> 172.21.15.50:0/1185225728 conn(0x7fb0840d06e0 msgr2=0x7fb0840d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:00.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.552+0000 7fb08bc00640 1 -- 172.21.15.50:0/1185225728 shutdown_connections 2024-09-21T06:14:00.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.552+0000 7fb08bc00640 1 -- 172.21.15.50:0/1185225728 wait complete. 2024-09-21T06:14:00.566 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T06:14:00.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.732+0000 7f0494c00640 1 -- 172.21.15.50:0/4114942578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0474003a10 msgr2=0x7f0474003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.732+0000 7f0494c00640 1 --2- 172.21.15.50:0/4114942578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0474003a10 0x7f0474003e80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0480009b90 tx=0x7f048001d2b0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.732+0000 7f048e800640 1 -- 172.21.15.50:0/4114942578 reap_dead start 2024-09-21T06:14:00.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.732+0000 7f0494c00640 1 -- 172.21.15.50:0/4114942578 shutdown_connections 2024-09-21T06:14:00.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.732+0000 7f0494c00640 1 -- 172.21.15.50:0/4114942578 >> 172.21.15.50:0/4114942578 conn(0x7f04900b2040 msgr2=0x7f04900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:00.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.732+0000 7f0494c00640 1 -- 172.21.15.50:0/4114942578 shutdown_connections 2024-09-21T06:14:00.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.732+0000 7f0494c00640 1 -- 172.21.15.50:0/4114942578 wait complete. 2024-09-21T06:14:00.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.732+0000 7f0494c00640 1 Processor -- start 2024-09-21T06:14:00.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.732+0000 7f0494c00640 1 -- start start 2024-09-21T06:14:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0494c00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0474003a10 0x7f04901f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0494c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04900ce470 0x7f0490158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0494c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04901e7e30 0x7f04901f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0494c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f049015cf30 con 0x7f04900ce470 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0494c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f049015cdb0 con 0x7f04901e7e30 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0494c00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f049015d0b0 con 0x7f0474003a10 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0485400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04900ce470 0x7f0490158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0485400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04900ce470 0x7f0490158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59970/0 (socket says 172.21.15.50:59970) 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0485400640 1 -- 172.21.15.50:0/2998932540 learned_addr learned my addr 172.21.15.50:0/2998932540 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f048de00640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0474003a10 0x7f04901f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f048e800640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04901e7e30 0x7f04901f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0485400640 1 -- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0474003a10 msgr2=0x7f04901f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0485400640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0474003a10 0x7f04901f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0485400640 1 -- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04901e7e30 msgr2=0x7f04901f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0485400640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04901e7e30 0x7f04901f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0485400640 1 -- 172.21.15.50:0/2998932540 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04901f4d00 con 0x7f04900ce470 2024-09-21T06:14:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f048e800640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04901e7e30 0x7f04901f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:00.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0485400640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04900ce470 0x7f0490158e90 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f0478001de0 tx=0x7f0478012870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:00.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0486a00640 1 -- 172.21.15.50:0/2998932540 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478010890 con 0x7f04900ce470 2024-09-21T06:14:00.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0486a00640 1 -- 172.21.15.50:0/2998932540 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0478010a30 con 0x7f04900ce470 2024-09-21T06:14:00.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0486a00640 1 -- 172.21.15.50:0/2998932540 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478010e20 con 0x7f04900ce470 2024-09-21T06:14:00.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0494c00640 1 -- 172.21.15.50:0/2998932540 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04901e6d90 con 0x7f04900ce470 2024-09-21T06:14:00.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0494c00640 1 -- 172.21.15.50:0/2998932540 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04901e7350 con 0x7f04900ce470 2024-09-21T06:14:00.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0494c00640 1 -- 172.21.15.50:0/2998932540 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04900768c0 con 0x7f04900ce470 2024-09-21T06:14:00.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.736+0000 7f0486a00640 1 -- 172.21.15.50:0/2998932540 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f0478013450 con 0x7f04900ce470 2024-09-21T06:14:00.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.740+0000 7f0486a00640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f046c0759f0 0x7f046c077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.740+0000 7f0486a00640 1 -- 172.21.15.50:0/2998932540 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f0478097390 con 0x7f04900ce470 2024-09-21T06:14:00.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.740+0000 7f048de00640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f046c0759f0 0x7f046c077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.740+0000 7f048de00640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f046c0759f0 0x7f046c077f00 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f04800af000 tx=0x7f048001a050 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:00.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.744+0000 7f0486a00640 1 -- 172.21.15.50:0/2998932540 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f04901f4d00 con 0x7f04900ce470 2024-09-21T06:14:00.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.920+0000 7f0494c00640 1 -- 172.21.15.50:0/2998932540 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f049006b540 con 0x7f046c0759f0 2024-09-21T06:14:00.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.920+0000 7f0486a00640 1 -- 172.21.15.50:0/2998932540 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f04800b3040 con 0x7f046c0759f0 2024-09-21T06:14:00.917 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T06:14:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.920+0000 7f0494c00640 1 -- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f046c0759f0 msgr2=0x7f046c077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.920+0000 7f0494c00640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f046c0759f0 0x7f046c077f00 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f04800af000 tx=0x7f048001a050 comp rx=0 tx=0).stop 2024-09-21T06:14:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.920+0000 7f0494c00640 1 -- 172.21.15.50:0/2998932540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04900ce470 msgr2=0x7f0490158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.920+0000 7f0494c00640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04900ce470 0x7f0490158e90 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f0478001de0 tx=0x7f0478012870 comp rx=0 tx=0).stop 2024-09-21T06:14:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.920+0000 7f0494c00640 1 -- 172.21.15.50:0/2998932540 shutdown_connections 2024-09-21T06:14:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.920+0000 7f0494c00640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04901e7e30 0x7f04901f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.920+0000 7f0494c00640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f046c0759f0 0x7f046c077f00 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.920+0000 7f0494c00640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04900ce470 0x7f0490158e90 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.920+0000 7f0494c00640 1 --2- 172.21.15.50:0/2998932540 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f0474003a10 0x7f04901f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.924+0000 7f0494c00640 1 -- 172.21.15.50:0/2998932540 >> 172.21.15.50:0/2998932540 conn(0x7f04900b2040 msgr2=0x7f04900b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.924+0000 7f0494c00640 1 -- 172.21.15.50:0/2998932540 shutdown_connections 2024-09-21T06:14:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:00.924+0000 7f0494c00640 1 -- 172.21.15.50:0/2998932540 wait complete. 2024-09-21T06:14:00.940 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T06:14:00.940 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.104+0000 7fd8fee00640 1 -- 172.21.15.50:0/405789962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f815cd20 msgr2=0x7fd8f815d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.104+0000 7fd8fee00640 1 --2- 172.21.15.50:0/405789962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f815cd20 0x7fd8f815d130 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8160da0 tx=0x7fd8f001cd60 comp rx=0 tx=0).stop 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.104+0000 7fd8fee00640 1 -- 172.21.15.50:0/405789962 shutdown_connections 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.104+0000 7fd8fee00640 1 --2- 172.21.15.50:0/405789962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f815cd20 0x7fd8f815d130 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.104+0000 7fd8fee00640 1 --2- 172.21.15.50:0/405789962 >> v2:172.21.15.88:3300/0 conn(0x7fd8f81602a0 0x7fd8f81606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.104+0000 7fd8fee00640 1 --2- 172.21.15.50:0/405789962 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f80cd2e0 0x7fd8f8070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.104+0000 7fd8fee00640 1 -- 172.21.15.50:0/405789962 >> 172.21.15.50:0/405789962 conn(0x7fd8f80b2c40 msgr2=0x7fd8f80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.104+0000 7fd8fee00640 1 -- 172.21.15.50:0/405789962 shutdown_connections 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.104+0000 7fd8fee00640 1 -- 172.21.15.50:0/405789962 wait complete. 2024-09-21T06:14:01.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.108+0000 7fd8fee00640 1 Processor -- start 2024-09-21T06:14:01.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.108+0000 7fd8fee00640 1 -- start start 2024-09-21T06:14:01.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.108+0000 7fd8fee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f80cd2e0 0x7fd8f813a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.108+0000 7fd8fee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8f815cd20 0x7fd8f8159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.108+0000 7fd8fee00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f81602a0 0x7fd8f80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.108+0000 7fd8fee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f80cfff0 con 0x7fd8f80cd2e0 2024-09-21T06:14:01.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.108+0000 7fd8fee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f80cfe70 con 0x7fd8f815cd20 2024-09-21T06:14:01.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.108+0000 7fd8fee00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f80d0170 con 0x7fd8f81602a0 2024-09-21T06:14:01.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.108+0000 7fd8f7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f80cd2e0 0x7fd8f813a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8f7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f80cd2e0 0x7fd8f813a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59992/0 (socket says 172.21.15.50:59992) 2024-09-21T06:14:01.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8f7e00640 1 -- 172.21.15.50:0/2630484866 learned_addr learned my addr 172.21.15.50:0/2630484866 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:14:01.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8fce00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f81602a0 0x7fd8f80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8efe00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8f815cd20 0x7fd8f8159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8f7e00640 1 -- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f81602a0 msgr2=0x7fd8f80b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8f7e00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f81602a0 0x7fd8f80b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8f7e00640 1 -- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8f815cd20 msgr2=0x7fd8f8159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8f7e00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8f815cd20 0x7fd8f8159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8f7e00640 1 -- 172.21.15.50:0/2630484866 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f8157630 con 0x7fd8f80cd2e0 2024-09-21T06:14:01.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8fce00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd8f81602a0 0x7fd8f80b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:01.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8efe00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8f815cd20 0x7fd8f8159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:01.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8f7e00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f80cd2e0 0x7fd8f813a6c0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800e7c0 tx=0x7fd8e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8f4c00640 1 -- 172.21.15.50:0/2630484866 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800cc60 con 0x7fd8f80cd2e0 2024-09-21T06:14:01.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8f4c00640 1 -- 172.21.15.50:0/2630484866 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8e800ce20 con 0x7fd8f80cd2e0 2024-09-21T06:14:01.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8f4c00640 1 -- 172.21.15.50:0/2630484866 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8005280 con 0x7fd8f80cd2e0 2024-09-21T06:14:01.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8fee00640 1 -- 172.21.15.50:0/2630484866 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f8157910 con 0x7fd8f80cd2e0 2024-09-21T06:14:01.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8fee00640 1 -- 172.21.15.50:0/2630484866 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f8157ed0 con 0x7fd8f80cd2e0 2024-09-21T06:14:01.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8fee00640 1 -- 172.21.15.50:0/2630484866 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd8f806b540 con 0x7fd8f80cd2e0 2024-09-21T06:14:01.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.112+0000 7fd8f4c00640 1 -- 172.21.15.50:0/2630484866 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fd8e8005420 con 0x7fd8f80cd2e0 2024-09-21T06:14:01.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.116+0000 7fd8f4c00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8d40759f0 0x7fd8d4077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.116+0000 7fd8f4c00640 1 -- 172.21.15.50:0/2630484866 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7fd8e80945d0 con 0x7fd8f80cd2e0 2024-09-21T06:14:01.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.116+0000 7fd8f4c00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] conn(0x7fd8d407b960 0x7fd8d407de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.116+0000 7fd8f4c00640 1 -- 172.21.15.50:0/2630484866 --> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8d407e4e0 con 0x7fd8d407b960 2024-09-21T06:14:01.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.116+0000 7fd8f4c00640 1 -- 172.21.15.50:0/2630484866 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fd8e8094980 con 0x7fd8f80cd2e0 2024-09-21T06:14:01.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.116+0000 7fd8efe00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8d40759f0 0x7fd8d4077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.116+0000 7fd8fce00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] conn(0x7fd8d407b960 0x7fd8d407de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.116+0000 7fd8efe00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8d40759f0 0x7fd8d4077f00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8159a40 tx=0x7fd8e00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.116+0000 7fd8fce00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] conn(0x7fd8d407b960 0x7fd8d407de10 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.116+0000 7fd8f4c00640 1 -- 172.21.15.50:0/2630484866 <== osd.7 v2:172.21.15.88:6810/4250480352 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd8d407e4e0 con 0x7fd8d407b960 2024-09-21T06:14:01.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8fee00640 1 -- 172.21.15.50:0/2630484866 --> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd8f815d130 con 0x7fd8d407b960 2024-09-21T06:14:01.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8f4c00640 1 -- 172.21.15.50:0/2630484866 <== osd.7 v2:172.21.15.88:6810/4250480352 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd8f815d130 con 0x7fd8d407b960 2024-09-21T06:14:01.127 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8fee00640 1 -- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] conn(0x7fd8d407b960 msgr2=0x7fd8d407de10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8fee00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:6810/4250480352,v1:172.21.15.88:6811/4250480352] conn(0x7fd8d407b960 0x7fd8d407de10 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8fee00640 1 -- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8d40759f0 msgr2=0x7fd8d4077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8fee00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd8d40759f0 0x7fd8d4077f00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8159a40 tx=0x7fd8e00091e0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8fee00640 1 -- 172.21.15.50:0/2630484866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f80cd2e0 msgr2=0x7fd8f813a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8fee00640 1 --2- 172.21.15.50:0/2630484866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f80cd2e0 0x7fd8f813a6c0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800e7c0 tx=0x7fd8e800ec80 comp rx=0 tx=0).stop 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8fce00640 1 -- 172.21.15.50:0/2630484866 reap_dead start 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8fee00640 1 -- 172.21.15.50:0/2630484866 shutdown_connections 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8fee00640 1 -- 172.21.15.50:0/2630484866 >> 172.21.15.50:0/2630484866 conn(0x7fd8f80b2c40 msgr2=0x7fd8f806f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8fee00640 1 -- 172.21.15.50:0/2630484866 shutdown_connections 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.132+0000 7fd8fee00640 1 -- 172.21.15.50:0/2630484866 wait complete. 2024-09-21T06:14:01.147 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T06:14:01.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.312+0000 7fb0f6200640 1 -- 172.21.15.50:0/1099226017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4003a10 msgr2=0x7fb0d4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.312+0000 7fb0f6200640 1 --2- 172.21.15.50:0/1099226017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4003a10 0x7fb0d4003e60 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009b40 tx=0x7fb0e001d220 comp rx=0 tx=0).stop 2024-09-21T06:14:01.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.312+0000 7fb0f6200640 1 -- 172.21.15.50:0/1099226017 shutdown_connections 2024-09-21T06:14:01.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.312+0000 7fb0f6200640 1 --2- 172.21.15.50:0/1099226017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4003a10 0x7fb0d4003e60 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.312+0000 7fb0f6200640 1 -- 172.21.15.50:0/1099226017 >> 172.21.15.50:0/1099226017 conn(0x7fb0f00b2040 msgr2=0x7fb0f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:01.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.312+0000 7fb0f6200640 1 -- 172.21.15.50:0/1099226017 shutdown_connections 2024-09-21T06:14:01.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.312+0000 7fb0f6200640 1 -- 172.21.15.50:0/1099226017 wait complete. 2024-09-21T06:14:01.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.312+0000 7fb0f6200640 1 Processor -- start 2024-09-21T06:14:01.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.312+0000 7fb0f6200640 1 -- start start 2024-09-21T06:14:01.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0f6200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0d4003a10 0x7fb0f0142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0f6200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0f01ec980 0x7fb0f01e8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0f6200640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0f0140650 0x7fb0f00ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0f6200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f00b55d0 con 0x7fb0f01ec980 2024-09-21T06:14:01.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0f6200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f00b5450 con 0x7fb0d4003a10 2024-09-21T06:14:01.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0f6200640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f00b5750 con 0x7fb0f0140650 2024-09-21T06:14:01.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0eea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0f01ec980 0x7fb0f01e8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0eea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0f01ec980 0x7fb0f01e8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60000/0 (socket says 172.21.15.50:60000) 2024-09-21T06:14:01.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0eea00640 1 -- 172.21.15.50:0/3068302161 learned_addr learned my addr 172.21.15.50:0/3068302161 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:14:01.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0efe00640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0f0140650 0x7fb0f00ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0ef400640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0d4003a10 0x7fb0f0142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0eea00640 1 -- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0f0140650 msgr2=0x7fb0f00ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0eea00640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0f0140650 0x7fb0f00ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0eea00640 1 -- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0d4003a10 msgr2=0x7fb0f0142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0eea00640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0d4003a10 0x7fb0f0142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0eea00640 1 -- 172.21.15.50:0/3068302161 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f01e8c60 con 0x7fb0f01ec980 2024-09-21T06:14:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0eea00640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0f01ec980 0x7fb0f01e8720 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc001c50 tx=0x7fb0dc00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0e7400640 1 -- 172.21.15.50:0/3068302161 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc015020 con 0x7fb0f01ec980 2024-09-21T06:14:01.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0e7400640 1 -- 172.21.15.50:0/3068302161 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0dc010980 con 0x7fb0f01ec980 2024-09-21T06:14:01.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0f6200640 1 -- 172.21.15.50:0/3068302161 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f015a620 con 0x7fb0f01ec980 2024-09-21T06:14:01.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0e7400640 1 -- 172.21.15.50:0/3068302161 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc010d90 con 0x7fb0f01ec980 2024-09-21T06:14:01.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0f6200640 1 -- 172.21.15.50:0/3068302161 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f01e7960 con 0x7fb0f01ec980 2024-09-21T06:14:01.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.316+0000 7fb0f6200640 1 -- 172.21.15.50:0/3068302161 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f015f560 con 0x7fb0f01ec980 2024-09-21T06:14:01.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.320+0000 7fb0e7400640 1 -- 172.21.15.50:0/3068302161 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fb0dc018020 con 0x7fb0f01ec980 2024-09-21T06:14:01.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.320+0000 7fb0e7400640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0d00759f0 0x7fb0d0077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.320+0000 7fb0e7400640 1 -- 172.21.15.50:0/3068302161 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7fb0dc0939f0 con 0x7fb0f01ec980 2024-09-21T06:14:01.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.320+0000 7fb0ef400640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0d00759f0 0x7fb0d0077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.320+0000 7fb0ef400640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0d00759f0 0x7fb0d0077f00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00ad000 tx=0x7fb0e0019fc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.324+0000 7fb0e7400640 1 -- 172.21.15.50:0/3068302161 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb0dc01f030 con 0x7fb0f01ec980 2024-09-21T06:14:01.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.504+0000 7fb0f6200640 1 -- 172.21.15.50:0/3068302161 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fb0f006b540 con 0x7fb0d00759f0 2024-09-21T06:14:01.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.504+0000 7fb0e7400640 1 -- 172.21.15.50:0/3068302161 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb0e00050a0 con 0x7fb0d00759f0 2024-09-21T06:14:01.502 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T06:14:01.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.508+0000 7fb0f6200640 1 -- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0d00759f0 msgr2=0x7fb0d0077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.508+0000 7fb0f6200640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0d00759f0 0x7fb0d0077f00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00ad000 tx=0x7fb0e0019fc0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.508+0000 7fb0f6200640 1 -- 172.21.15.50:0/3068302161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0f01ec980 msgr2=0x7fb0f01e8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.508+0000 7fb0f6200640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0f01ec980 0x7fb0f01e8720 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc001c50 tx=0x7fb0dc00ea20 comp rx=0 tx=0).stop 2024-09-21T06:14:01.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.508+0000 7fb0f6200640 1 -- 172.21.15.50:0/3068302161 shutdown_connections 2024-09-21T06:14:01.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.508+0000 7fb0f6200640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fb0f0140650 0x7fb0f00ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.508+0000 7fb0f6200640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fb0d00759f0 0x7fb0d0077f00 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.508+0000 7fb0f6200640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0f01ec980 0x7fb0f01e8720 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.508+0000 7fb0f6200640 1 --2- 172.21.15.50:0/3068302161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0d4003a10 0x7fb0f0142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.508+0000 7fb0f6200640 1 -- 172.21.15.50:0/3068302161 >> 172.21.15.50:0/3068302161 conn(0x7fb0f00b2040 msgr2=0x7fb0f00af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:01.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.508+0000 7fb0f6200640 1 -- 172.21.15.50:0/3068302161 shutdown_connections 2024-09-21T06:14:01.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.508+0000 7fb0f6200640 1 -- 172.21.15.50:0/3068302161 wait complete. 2024-09-21T06:14:01.524 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:14:01.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.684+0000 7f53f1800640 1 -- 172.21.15.50:0/4174769384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec160af0 msgr2=0x7f53ec070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.684+0000 7f53f1800640 1 --2- 172.21.15.50:0/4174769384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec160af0 0x7f53ec070690 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0724a0 tx=0x7f53e401cd50 comp rx=0 tx=0).stop 2024-09-21T06:14:01.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.684+0000 7f53f1800640 1 -- 172.21.15.50:0/4174769384 shutdown_connections 2024-09-21T06:14:01.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.684+0000 7f53f1800640 1 --2- 172.21.15.50:0/4174769384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec160af0 0x7f53ec070690 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.684+0000 7f53f1800640 1 --2- 172.21.15.50:0/4174769384 >> v2:172.21.15.88:3300/0 conn(0x7f53ec15c7b0 0x7f53ec068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.684+0000 7f53f1800640 1 -- 172.21.15.50:0/4174769384 >> 172.21.15.50:0/4174769384 conn(0x7f53ec0b35f0 msgr2=0x7f53ec0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:01.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.684+0000 7f53f1800640 1 -- 172.21.15.50:0/4174769384 shutdown_connections 2024-09-21T06:14:01.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.684+0000 7f53f1800640 1 -- 172.21.15.50:0/4174769384 wait complete. 2024-09-21T06:14:01.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.688+0000 7f53f1800640 1 Processor -- start 2024-09-21T06:14:01.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.688+0000 7f53f1800640 1 -- start start 2024-09-21T06:14:01.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.688+0000 7f53f1800640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f53ec069330 0x7f53ec0afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.688+0000 7f53f1800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53ec15c7b0 0x7f53ec153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.688+0000 7f53f1800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec160af0 0x7f53ec157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.688+0000 7f53f1800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec169d60 con 0x7f53ec160af0 2024-09-21T06:14:01.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.688+0000 7f53f1800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec169be0 con 0x7f53ec15c7b0 2024-09-21T06:14:01.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.688+0000 7f53f1800640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f53ec169ee0 con 0x7f53ec069330 2024-09-21T06:14:01.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.688+0000 7f53eb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec160af0 0x7f53ec157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.688+0000 7f53eb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec160af0 0x7f53ec157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60016/0 (socket says 172.21.15.50:60016) 2024-09-21T06:14:01.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.688+0000 7f53eb400640 1 -- 172.21.15.50:0/328995709 learned_addr learned my addr 172.21.15.50:0/328995709 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:14:01.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53eaa00640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f53ec069330 0x7f53ec0afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53e3e00640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53ec15c7b0 0x7f53ec153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53eb400640 1 -- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f53ec069330 msgr2=0x7f53ec0afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53eb400640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f53ec069330 0x7f53ec0afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53eb400640 1 -- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53ec15c7b0 msgr2=0x7f53ec153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53eb400640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53ec15c7b0 0x7f53ec153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53eb400640 1 -- 172.21.15.50:0/328995709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec1f8c30 con 0x7f53ec160af0 2024-09-21T06:14:01.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53e3e00640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53ec15c7b0 0x7f53ec153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:01.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53eb400640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec160af0 0x7f53ec157ac0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f53e4019960 tx=0x7f53e4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53e2a00640 1 -- 172.21.15.50:0/328995709 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e40bb070 con 0x7f53ec160af0 2024-09-21T06:14:01.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53e2a00640 1 -- 172.21.15.50:0/328995709 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53e4004c50 con 0x7f53ec160af0 2024-09-21T06:14:01.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53f1800640 1 -- 172.21.15.50:0/328995709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ec1f8ed0 con 0x7f53ec160af0 2024-09-21T06:14:01.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53e2a00640 1 -- 172.21.15.50:0/328995709 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e40b2040 con 0x7f53ec160af0 2024-09-21T06:14:01.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53f1800640 1 -- 172.21.15.50:0/328995709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ec0b0360 con 0x7f53ec160af0 2024-09-21T06:14:01.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.692+0000 7f53f1800640 1 -- 172.21.15.50:0/328995709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ec068df0 con 0x7f53ec160af0 2024-09-21T06:14:01.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.696+0000 7f53e2a00640 1 -- 172.21.15.50:0/328995709 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f53e40be070 con 0x7f53ec160af0 2024-09-21T06:14:01.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.696+0000 7f53e2a00640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53c80759f0 0x7f53c8077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.696+0000 7f53e2a00640 1 -- 172.21.15.50:0/328995709 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f53e412d6d0 con 0x7f53ec160af0 2024-09-21T06:14:01.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.696+0000 7f53eaa00640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53c80759f0 0x7f53c8077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.696+0000 7f53eaa00640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53c80759f0 0x7f53c8077f00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0058d0 tx=0x7f53dc005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.696+0000 7f53e2a00640 1 -- 172.21.15.50:0/328995709 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f53e4016970 con 0x7f53ec160af0 2024-09-21T06:14:01.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.872+0000 7f53f1800640 1 -- 172.21.15.50:0/328995709 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53ec06b540 con 0x7f53c80759f0 2024-09-21T06:14:01.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.872+0000 7f53e2a00640 1 -- 172.21.15.50:0/328995709 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13384 (secure 0 0 0) 0x7f53dc00fad0 con 0x7f53c80759f0 2024-09-21T06:14:01.869 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:14:01.869 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:14:01.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.876+0000 7f53f1800640 1 -- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53c80759f0 msgr2=0x7f53c8077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.876+0000 7f53f1800640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53c80759f0 0x7f53c8077f00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0058d0 tx=0x7f53dc005860 comp rx=0 tx=0).stop 2024-09-21T06:14:01.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.876+0000 7f53f1800640 1 -- 172.21.15.50:0/328995709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec160af0 msgr2=0x7f53ec157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.876+0000 7f53f1800640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec160af0 0x7f53ec157ac0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f53e4019960 tx=0x7f53e4005b00 comp rx=0 tx=0).stop 2024-09-21T06:14:01.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.876+0000 7f53f1800640 1 -- 172.21.15.50:0/328995709 shutdown_connections 2024-09-21T06:14:01.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.876+0000 7f53f1800640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f53c80759f0 0x7f53c8077f00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.876+0000 7f53f1800640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec160af0 0x7f53ec157ac0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.876+0000 7f53f1800640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53ec15c7b0 0x7f53ec153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.876+0000 7f53f1800640 1 --2- 172.21.15.50:0/328995709 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f53ec069330 0x7f53ec0afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.876+0000 7f53f1800640 1 -- 172.21.15.50:0/328995709 >> 172.21.15.50:0/328995709 conn(0x7f53ec0b35f0 msgr2=0x7f53ec0b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:01.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.876+0000 7f53f1800640 1 -- 172.21.15.50:0/328995709 shutdown_connections 2024-09-21T06:14:01.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:01.876+0000 7f53f1800640 1 -- 172.21.15.50:0/328995709 wait complete. 2024-09-21T06:14:01.891 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":674,"stamp":"2024-09-21T06:14:01.272790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1208740,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":1192819,"kb_avail":753765980,"statfs":{"total":773094113280,"available":771856363520,"internally_reserved":0,"allocated":16097280,"data_stored":7762976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1221447216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.378986"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":348,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T06:13:51.130416+0000","last_change":"2024-09-21T05:56:20.565503+0000","last_active":"2024-09-21T06:13:51.130416+0000","last_peered":"2024-09-21T06:13:51.130416+0000","last_clean":"2024-09-21T06:13:51.130416+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:13:51.130416+0000","last_undegraded":"2024-09-21T06:13:51.130416+0000","last_fullsized":"2024-09-21T06:13:51.130416+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:56:18.921944+0000","last_clean_scrub_stamp":"2024-09-21T05:56:18.921944+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:48.170754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509655,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145976,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225864,"statfs":{"total":96636764160,"available":96487284736,"internally_reserved":0,"allocated":2457600,"data_stored":1413148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509655,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149560,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222280,"statfs":{"total":96636764160,"available":96483614720,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131512,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240328,"statfs":{"total":96636764160,"available":96502095872,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509655,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167600,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204240,"statfs":{"total":96636764160,"available":96465141760,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149552,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222288,"statfs":{"total":96636764160,"available":96483622912,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509656,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155124,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":94216716,"statfs":{"total":96636764160,"available":96477917184,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":156957125},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509654,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150136,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221704,"statfs":{"total":96636764160,"available":96483024896,"internally_reserved":0,"allocated":2457600,"data_stored":1413148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542359,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159280,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212560,"statfs":{"total":96636764160,"available":96473661440,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:14:01.894 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T05:56:18.921944+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=5, tm_min=56, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=6, tm_min=13, tm_sec=55, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T06:14:01.894 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T06:14:20.247 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:14:20.252 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:14:21.895 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:14:22.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.071+0000 7f39f5a00640 1 -- 172.21.15.50:0/3659540021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39f00749a0 msgr2=0x7f39f0074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.071+0000 7f39f5a00640 1 --2- 172.21.15.50:0/3659540021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39f00749a0 0x7f39f0074db0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f39f00726f0 tx=0x7f39e001c8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.071+0000 7f39f5a00640 1 -- 172.21.15.50:0/3659540021 shutdown_connections 2024-09-21T06:14:22.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.071+0000 7f39f5a00640 1 --2- 172.21.15.50:0/3659540021 >> v2:172.21.15.88:3300/0 conn(0x7f39f00cefa0 0x7f39f00cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.071+0000 7f39f5a00640 1 --2- 172.21.15.50:0/3659540021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39f00749a0 0x7f39f0074db0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.071+0000 7f39f5a00640 1 -- 172.21.15.50:0/3659540021 >> 172.21.15.50:0/3659540021 conn(0x7f39f00b2040 msgr2=0x7f39f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:22.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.071+0000 7f39f5a00640 1 -- 172.21.15.50:0/3659540021 shutdown_connections 2024-09-21T06:14:22.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.071+0000 7f39f5a00640 1 -- 172.21.15.50:0/3659540021 wait complete. 2024-09-21T06:14:22.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.071+0000 7f39f5a00640 1 Processor -- start 2024-09-21T06:14:22.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.071+0000 7f39f5a00640 1 -- start start 2024-09-21T06:14:22.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39f5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39f00749a0 0x7f39f0140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:22.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39f5a00640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f39f00cefa0 0x7f39f015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:22.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39f5a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39f0160330 0x7f39f015f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:22.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39f5a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f39f016a060 con 0x7f39f00749a0 2024-09-21T06:14:22.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39f5a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0169ee0 con 0x7f39f0160330 2024-09-21T06:14:22.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39f5a00640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f39f016a1e0 con 0x7f39f00cefa0 2024-09-21T06:14:22.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39eea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39f00749a0 0x7f39f0140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:22.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39eea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39f00749a0 0x7f39f0140150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59282/0 (socket says 172.21.15.50:59282) 2024-09-21T06:14:22.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39eea00640 1 -- 172.21.15.50:0/1564872231 learned_addr learned my addr 172.21.15.50:0/1564872231 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:14:22.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39e7e00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f39f00cefa0 0x7f39f015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:22.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39ef400640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39f0160330 0x7f39f015f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:22.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39eea00640 1 -- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f39f00cefa0 msgr2=0x7f39f015a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39eea00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f39f00cefa0 0x7f39f015a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39eea00640 1 -- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39f0160330 msgr2=0x7f39f015f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39eea00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39f0160330 0x7f39f015f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39eea00640 1 -- 172.21.15.50:0/1564872231 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f013e1d0 con 0x7f39f00749a0 2024-09-21T06:14:22.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39eea00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39f00749a0 0x7f39f0140150 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f39e00194d0 tx=0x7f39e0005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:22.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39e6a00640 1 -- 172.21.15.50:0/1564872231 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e00bc070 con 0x7f39f00749a0 2024-09-21T06:14:22.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39e6a00640 1 -- 172.21.15.50:0/1564872231 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39e0005320 con 0x7f39f00749a0 2024-09-21T06:14:22.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39e6a00640 1 -- 172.21.15.50:0/1564872231 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e00b2040 con 0x7f39f00749a0 2024-09-21T06:14:22.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39f5a00640 1 -- 172.21.15.50:0/1564872231 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f01fc7e0 con 0x7f39f00749a0 2024-09-21T06:14:22.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39f5a00640 1 -- 172.21.15.50:0/1564872231 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f01fcda0 con 0x7f39f00749a0 2024-09-21T06:14:22.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.075+0000 7f39f5a00640 1 -- 172.21.15.50:0/1564872231 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f013e3a0 con 0x7f39f00749a0 2024-09-21T06:14:22.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.079+0000 7f39e6a00640 1 -- 172.21.15.50:0/1564872231 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f39e00046d0 con 0x7f39f00749a0 2024-09-21T06:14:22.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.079+0000 7f39e6a00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39cc0759e0 0x7f39cc077ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:22.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.079+0000 7f39e6a00640 1 -- 172.21.15.50:0/1564872231 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f39e012cff0 con 0x7f39f00749a0 2024-09-21T06:14:22.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.079+0000 7f39e7e00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39cc0759e0 0x7f39cc077ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:22.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.079+0000 7f39e7e00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39cc0759e0 0x7f39cc077ef0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f39f015f5a0 tx=0x7f39d80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:22.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.083+0000 7f39e6a00640 1 -- 172.21.15.50:0/1564872231 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f39e00164b0 con 0x7f39f00749a0 2024-09-21T06:14:22.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.255+0000 7f39f5a00640 1 -- 172.21.15.50:0/1564872231 --> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39f006b540 con 0x7f39cc0759e0 2024-09-21T06:14:22.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.255+0000 7f39e6a00640 1 -- 172.21.15.50:0/1564872231 <== mgr.4100 v2:172.21.15.88:6800/3010204332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13391 (secure 0 0 0) 0x7f39d8011bb0 con 0x7f39cc0759e0 2024-09-21T06:14:22.255 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:14:22.255 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-21T06:14:22.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.259+0000 7f39f5a00640 1 -- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39cc0759e0 msgr2=0x7f39cc077ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.259+0000 7f39f5a00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39cc0759e0 0x7f39cc077ef0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f39f015f5a0 tx=0x7f39d80091e0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.259+0000 7f39f5a00640 1 -- 172.21.15.50:0/1564872231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39f00749a0 msgr2=0x7f39f0140150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.259+0000 7f39f5a00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39f00749a0 0x7f39f0140150 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f39e00194d0 tx=0x7f39e0005de0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.259+0000 7f39f5a00640 1 -- 172.21.15.50:0/1564872231 shutdown_connections 2024-09-21T06:14:22.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.259+0000 7f39f5a00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39f0160330 0x7f39f015f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.259+0000 7f39f5a00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f39f00cefa0 0x7f39f015a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.259+0000 7f39f5a00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f39cc0759e0 0x7f39cc077ef0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.259+0000 7f39f5a00640 1 --2- 172.21.15.50:0/1564872231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39f00749a0 0x7f39f0140150 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.259+0000 7f39f5a00640 1 -- 172.21.15.50:0/1564872231 >> 172.21.15.50:0/1564872231 conn(0x7f39f00b2040 msgr2=0x7f39f006e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:22.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.259+0000 7f39f5a00640 1 -- 172.21.15.50:0/1564872231 shutdown_connections 2024-09-21T06:14:22.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.259+0000 7f39f5a00640 1 -- 172.21.15.50:0/1564872231 wait complete. 2024-09-21T06:14:22.278 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":684,"stamp":"2024-09-21T06:14:21.278005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1208724,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":1192819,"kb_avail":753765996,"statfs":{"total":773094113280,"available":771856379904,"internally_reserved":0,"allocated":16097280,"data_stored":7762976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1221447216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003113"},"pg_stats":[{"pgid":"1.0","version":"149'74","reported_seq":358,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T06:14:01.844653+0000","last_change":"2024-09-21T06:14:01.844653+0000","last_active":"2024-09-21T06:14:01.844653+0000","last_peered":"2024-09-21T06:14:01.844653+0000","last_clean":"2024-09-21T06:14:01.844653+0000","last_became_active":"2024-09-21T05:56:20.563969+0000","last_became_peered":"2024-09-21T05:56:20.563969+0000","last_unstale":"2024-09-21T06:14:01.844653+0000","last_undegraded":"2024-09-21T06:14:01.844653+0000","last_fullsized":"2024-09-21T06:14:01.844653+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'74","last_scrub_stamp":"2024-09-21T06:14:01.844524+0000","last_deep_scrub":"149'74","last_deep_scrub_stamp":"2024-09-21T06:14:01.844524+0000","last_clean_scrub_stamp":"2024-09-21T06:14:01.844524+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:18.161355+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509659,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145968,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225872,"statfs":{"total":96636764160,"available":96487292928,"internally_reserved":0,"allocated":2457600,"data_stored":1413148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149560,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222280,"statfs":{"total":96636764160,"available":96483614720,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509658,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131512,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240328,"statfs":{"total":96636764160,"available":96502095872,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167600,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204240,"statfs":{"total":96636764160,"available":96465141760,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509658,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149552,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222288,"statfs":{"total":96636764160,"available":96483622912,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509660,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155124,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":94216716,"statfs":{"total":96636764160,"available":96477917184,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":156957125},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509658,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150128,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221712,"statfs":{"total":96636764160,"available":96483033088,"internally_reserved":0,"allocated":2457600,"data_stored":1413148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542363,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159280,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212560,"statfs":{"total":96636764160,"available":96473661440,"internally_reserved":0,"allocated":1863680,"data_stored":822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:14:22.279 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T06:14:22.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.439+0000 7f2047000640 1 -- 172.21.15.50:0/724710682 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20400b4de0 msgr2=0x7f20400b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.439+0000 7f2047000640 1 --2- 172.21.15.50:0/724710682 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20400b4de0 0x7f20400b51f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2038009c80 tx=0x7f203801cd70 comp rx=0 tx=0).stop 2024-09-21T06:14:22.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.439+0000 7f2047000640 1 -- 172.21.15.50:0/724710682 shutdown_connections 2024-09-21T06:14:22.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.439+0000 7f2047000640 1 --2- 172.21.15.50:0/724710682 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20400b4de0 0x7f20400b51f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.439+0000 7f2047000640 1 --2- 172.21.15.50:0/724710682 >> v2:172.21.15.88:3300/0 conn(0x7f20401527b0 0x7f204015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.439+0000 7f2047000640 1 -- 172.21.15.50:0/724710682 >> 172.21.15.50:0/724710682 conn(0x7f20400b2040 msgr2=0x7f20400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:22.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.439+0000 7f2047000640 1 -- 172.21.15.50:0/724710682 shutdown_connections 2024-09-21T06:14:22.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.439+0000 7f2047000640 1 -- 172.21.15.50:0/724710682 wait complete. 2024-09-21T06:14:22.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f2047000640 1 Processor -- start 2024-09-21T06:14:22.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f2047000640 1 -- start start 2024-09-21T06:14:22.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f2047000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20400b4de0 0x7f20401494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:22.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f2047000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20400b5c70 0x7f20401f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:22.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f2047000640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20401527b0 0x7f2040151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:22.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f2047000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2040168a00 con 0x7f20400b5c70 2024-09-21T06:14:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f2047000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2040168880 con 0x7f20400b4de0 2024-09-21T06:14:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f2047000640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7f2040168b80 con 0x7f20401527b0 2024-09-21T06:14:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f2037e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20400b5c70 0x7f20401f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f2037e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20400b5c70 0x7f20401f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59284/0 (socket says 172.21.15.50:59284) 2024-09-21T06:14:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f2037e00640 1 -- 172.21.15.50:0/3945271790 learned_addr learned my addr 172.21.15.50:0/3945271790 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:14:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f2045000640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20401527b0 0x7f2040151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.443+0000 7f203fe00640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20400b4de0 0x7f20401494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:22.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f2037e00640 1 -- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20401527b0 msgr2=0x7f2040151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f2037e00640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20401527b0 0x7f2040151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f2037e00640 1 -- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20400b4de0 msgr2=0x7f20401494f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f2037e00640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20400b4de0 0x7f20401494f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f2037e00640 1 -- 172.21.15.50:0/3945271790 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20401f9d20 con 0x7f20400b5c70 2024-09-21T06:14:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f2037e00640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20400b5c70 0x7f20401f6f60 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f202800e7f0 tx=0x7f202800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f203cc00640 1 -- 172.21.15.50:0/3945271790 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202800ccc0 con 0x7f20400b5c70 2024-09-21T06:14:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f203cc00640 1 -- 172.21.15.50:0/3945271790 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f202800ce80 con 0x7f20400b5c70 2024-09-21T06:14:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f2047000640 1 -- 172.21.15.50:0/3945271790 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20401fa000 con 0x7f20400b5c70 2024-09-21T06:14:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f203cc00640 1 -- 172.21.15.50:0/3945271790 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028005280 con 0x7f20400b5c70 2024-09-21T06:14:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f2047000640 1 -- 172.21.15.50:0/3945271790 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20401fa5c0 con 0x7f20400b5c70 2024-09-21T06:14:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f2047000640 1 -- 172.21.15.50:0/3945271790 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20400b51f0 con 0x7f20400b5c70 2024-09-21T06:14:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.447+0000 7f203cc00640 1 -- 172.21.15.50:0/3945271790 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7f2028005420 con 0x7f20400b5c70 2024-09-21T06:14:22.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.451+0000 7f203cc00640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f201c075a20 0x7f201c077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:22.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.451+0000 7f203cc00640 1 -- 172.21.15.50:0/3945271790 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7f2028093ec0 con 0x7f20400b5c70 2024-09-21T06:14:22.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.451+0000 7f203fe00640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f201c075a20 0x7f201c077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:22.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.451+0000 7f203fe00640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f201c075a20 0x7f201c077f30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2030007fd0 tx=0x7f2030009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:22.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.455+0000 7f203cc00640 1 -- 172.21.15.50:0/3945271790 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2028060270 con 0x7f20400b5c70 2024-09-21T06:14:22.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.627+0000 7f2047000640 1 -- 172.21.15.50:0/3945271790 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f204015a4c0 con 0x7f20400b5c70 2024-09-21T06:14:22.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.631+0000 7f203cc00640 1 -- 172.21.15.50:0/3945271790 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v62)=0 v62) ==== 123+0+0 (secure 0 0 0) 0x7f2028064ca0 con 0x7f20400b5c70 2024-09-21T06:14:22.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f203cc00640 1 -- 172.21.15.50:0/3945271790 <== mon.0 v2:172.21.15.50:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f202805fd80 con 0x7f20400b5c70 2024-09-21T06:14:22.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f2047000640 1 -- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f201c075a20 msgr2=0x7f201c077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f2047000640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f201c075a20 0x7f201c077f30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2030007fd0 tx=0x7f2030009040 comp rx=0 tx=0).stop 2024-09-21T06:14:22.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f2047000640 1 -- 172.21.15.50:0/3945271790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20400b5c70 msgr2=0x7f20401f6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f2047000640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20400b5c70 0x7f20401f6f60 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f202800e7f0 tx=0x7f202800ecb0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f2047000640 1 -- 172.21.15.50:0/3945271790 shutdown_connections 2024-09-21T06:14:22.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f2047000640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7f20401527b0 0x7f2040151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f2047000640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7f201c075a20 0x7f201c077f30 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f2047000640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20400b5c70 0x7f20401f6f60 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f2047000640 1 --2- 172.21.15.50:0/3945271790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20400b4de0 0x7f20401494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f2047000640 1 -- 172.21.15.50:0/3945271790 >> 172.21.15.50:0/3945271790 conn(0x7f20400b2040 msgr2=0x7f20400b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:22.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f2047000640 1 -- 172.21.15.50:0/3945271790 shutdown_connections 2024-09-21T06:14:22.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.635+0000 7f2047000640 1 -- 172.21.15.50:0/3945271790 wait complete. 2024-09-21T06:14:22.654 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:14:22.654 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T06:14:22.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.815+0000 7fd5e8400640 1 -- 172.21.15.50:0/3365801800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5c8004da0 msgr2=0x7fd5c8005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.815+0000 7fd5e8400640 1 --2- 172.21.15.50:0/3365801800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5c8004da0 0x7fd5c8005230 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800c640 tx=0x7fd5d801d280 comp rx=0 tx=0).stop 2024-09-21T06:14:22.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.815+0000 7fd5e6400640 1 -- 172.21.15.50:0/3365801800 reap_dead start 2024-09-21T06:14:22.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.815+0000 7fd5e8400640 1 -- 172.21.15.50:0/3365801800 shutdown_connections 2024-09-21T06:14:22.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.815+0000 7fd5e8400640 1 -- 172.21.15.50:0/3365801800 >> 172.21.15.50:0/3365801800 conn(0x7fd5e00b2040 msgr2=0x7fd5e00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:22.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.815+0000 7fd5e8400640 1 -- 172.21.15.50:0/3365801800 shutdown_connections 2024-09-21T06:14:22.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.815+0000 7fd5e8400640 1 -- 172.21.15.50:0/3365801800 wait complete. 2024-09-21T06:14:22.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e8400640 1 Processor -- start 2024-09-21T06:14:22.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e8400640 1 -- start start 2024-09-21T06:14:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e8400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5c8004da0 0x7fd5e00ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e8400640 1 --2- >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd5e01f9a70 0x7fd5e00aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e8400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e01e2500 0x7fd5e01f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e8400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e015d070 con 0x7fd5e01e2500 2024-09-21T06:14:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e8400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e015cef0 con 0x7fd5c8004da0 2024-09-21T06:14:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e8400640 1 -- --> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e015d1f0 con 0x7fd5e01f9a70 2024-09-21T06:14:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e6400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e01e2500 0x7fd5e01f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e6400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e01e2500 0x7fd5e01f5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59304/0 (socket says 172.21.15.50:59304) 2024-09-21T06:14:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e6400640 1 -- 172.21.15.50:0/2720446935 learned_addr learned my addr 172.21.15.50:0/2720446935 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-21T06:14:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e5a00640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5c8004da0 0x7fd5e00ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e5000640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd5e01f9a70 0x7fd5e00aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e6400640 1 -- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd5e01f9a70 msgr2=0x7fd5e00aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e6400640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd5e01f9a70 0x7fd5e00aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e6400640 1 -- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5c8004da0 msgr2=0x7fd5e00ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:22.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e6400640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5c8004da0 0x7fd5e00ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:22.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.819+0000 7fd5e6400640 1 -- 172.21.15.50:0/2720446935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e015a9e0 con 0x7fd5e01e2500 2024-09-21T06:14:22.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.823+0000 7fd5e6400640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e01e2500 0x7fd5e01f5730 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd5d801cd40 tx=0x7fd5d801d4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.823+0000 7fd5dd600640 1 -- 172.21.15.50:0/2720446935 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d80011d0 con 0x7fd5e01e2500 2024-09-21T06:14:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.823+0000 7fd5dd600640 1 -- 172.21.15.50:0/2720446935 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fd5d8003980 con 0x7fd5e01e2500 2024-09-21T06:14:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.823+0000 7fd5e8400640 1 -- 172.21.15.50:0/2720446935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e01f8f90 con 0x7fd5e01e2500 2024-09-21T06:14:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.823+0000 7fd5dd600640 1 -- 172.21.15.50:0/2720446935 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8003d90 con 0x7fd5e01e2500 2024-09-21T06:14:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.823+0000 7fd5e8400640 1 -- 172.21.15.50:0/2720446935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e01f9550 con 0x7fd5e01e2500 2024-09-21T06:14:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.823+0000 7fd5e8400640 1 -- 172.21.15.50:0/2720446935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e01f8d80 con 0x7fd5e01e2500 2024-09-21T06:14:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.823+0000 7fd5dd600640 1 -- 172.21.15.50:0/2720446935 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 62) ==== 100855+0+0 (secure 0 0 0) 0x7fd5d8006070 con 0x7fd5e01e2500 2024-09-21T06:14:22.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.823+0000 7fd5dd600640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd5c00759f0 0x7fd5c0077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:22.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.827+0000 7fd5dd600640 1 -- 172.21.15.50:0/2720446935 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5027+0+0 (secure 0 0 0) 0x7fd5d812dc00 con 0x7fd5e01e2500 2024-09-21T06:14:22.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.827+0000 7fd5e5a00640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd5c00759f0 0x7fd5c0077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:22.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.827+0000 7fd5e5a00640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd5c00759f0 0x7fd5c0077f00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009ba0 tx=0x7fd5d40091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:22.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:22.831+0000 7fd5dd600640 1 -- 172.21.15.50:0/2720446935 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd5d8017030 con 0x7fd5e01e2500 2024-09-21T06:14:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 -- 172.21.15.50:0/2720446935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5e0074e30 con 0x7fd5e01e2500 2024-09-21T06:14:23.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5dd600640 1 -- 172.21.15.50:0/2720446935 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+3908 (secure 0 0 0) 0x7fd5d80fa680 con 0x7fd5e01e2500 2024-09-21T06:14:23.044 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-21T06:14:23.044 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":97,"btime":"2024-09-21T06:13:48:369538+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/4262887686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":4262887686},{"type":"v1","addr":"172.21.15.50:6835","nonce":4262887686}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":7721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3308178097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3308178097},{"type":"v1","addr":"172.21.15.88:6837","nonce":3308178097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/610421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":610421041},{"type":"v1","addr":"172.21.15.50:6834","nonce":610421041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":9085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1278134214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":1278134214},{"type":"v1","addr":"172.21.15.88:6835","nonce":1278134214}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[]} 2024-09-21T06:14:23.045 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 97 2024-09-21T06:14:23.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 -- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd5c00759f0 msgr2=0x7fd5c0077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:23.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd5c00759f0 0x7fd5c0077f00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009ba0 tx=0x7fd5d40091e0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 -- 172.21.15.50:0/2720446935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e01e2500 msgr2=0x7fd5e01f5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e01e2500 0x7fd5e01f5730 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd5d801cd40 tx=0x7fd5d801d4d0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 -- 172.21.15.50:0/2720446935 shutdown_connections 2024-09-21T06:14:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:6800/3010204332,v1:172.21.15.88:6801/3010204332] conn(0x7fd5c00759f0 0x7fd5c0077f00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e01e2500 0x7fd5e01f5730 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] conn(0x7fd5e01f9a70 0x7fd5e00aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 --2- 172.21.15.50:0/2720446935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5c8004da0 0x7fd5e00ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 -- 172.21.15.50:0/2720446935 >> 172.21.15.50:0/2720446935 conn(0x7fd5e00b2040 msgr2=0x7fd5e00cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 -- 172.21.15.50:0/2720446935 shutdown_connections 2024-09-21T06:14:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-21T06:14:23.047+0000 7fd5e8400640 1 -- 172.21.15.50:0/2720446935 wait complete. 2024-09-21T06:14:23.067 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-21T06:14:23.068 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T06:14:23.068 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T06:14:23.068 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:23.080 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T06:14:23.080 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T06:14:23.080 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:23.138 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T06:14:23.138 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T06:14:23.138 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:23.154 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T06:14:23.154 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T06:14:23.154 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:23.220 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T06:14:23.220 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T06:14:23.220 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T06:14:23.221 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:27.557 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-21T06:14:29.322 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T06:14:29.322 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T06:14:29.322 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:29.413 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T06:14:29.413 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T06:14:29.414 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:33.564 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-21T06:14:33.565 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-21T06:14:35.515 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T06:14:35.515 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T06:14:35.515 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:35.598 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T06:14:35.598 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T06:14:35.599 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:39.371 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-21T06:14:39.371 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-21T06:14:41.700 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T06:14:41.701 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T06:14:41.701 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:41.777 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T06:14:41.777 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T06:14:41.777 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:44.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-21T06:14:44.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-21T06:14:47.879 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T06:14:47.880 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T06:14:47.880 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:47.976 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T06:14:47.977 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T06:14:47.977 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T06:14:47.977 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:48.039 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T06:14:48.039 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T06:14:48.039 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:50.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-09-21T06:14:50.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-21T06:14:50.307 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:14:50.310 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:14:54.140 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T06:14:54.140 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T06:14:54.141 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T06:14:54.141 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:54.164 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T06:14:54.164 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T06:14:54.164 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:54.185 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T06:14:54.185 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T06:14:54.185 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:14:54.249 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T06:14:54.249 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T06:14:54.250 DEBUG:teuthology.orchestra.run.smithi050:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-21T06:14:54.293 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:14:54.293 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T06:14:54.566 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:14:54.567 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T06:14:54.898 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:14:54.898 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T06:14:55.132 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:14:55.132 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T06:14:55.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-09-21T06:14:55.303 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T06:14:55.303 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T06:14:55.452 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T06:14:55.453 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T06:14:55.676 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T06:14:55.676 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T06:14:55.871 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T06:14:55.872 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T06:14:56.097 INFO:tasks.ceph:Archiving mon data... 2024-09-21T06:14:56.098 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513/data/mon.a.tgz 2024-09-21T06:14:56.098 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2024-09-21T06:14:56.108 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.NbLzu6bWSR 2024-09-21T06:14:56.108 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.NbLzu6bWSR 2024-09-21T06:14:56.482 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.NbLzu6bWSR 2024-09-21T06:14:56.555 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.NbLzu6bWSR is 2MB 2024-09-21T06:14:56.638 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.NbLzu6bWSR 2024-09-21T06:14:56.647 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513/data/mon.b.tgz 2024-09-21T06:14:56.647 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-09-21T06:14:56.656 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.EVzvQg0cWB 2024-09-21T06:14:56.656 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.EVzvQg0cWB 2024-09-21T06:14:57.012 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.EVzvQg0cWB 2024-09-21T06:14:57.089 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.EVzvQg0cWB is 2MB 2024-09-21T06:14:57.172 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.EVzvQg0cWB 2024-09-21T06:14:57.181 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513/data/mon.c.tgz 2024-09-21T06:14:57.181 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-09-21T06:14:57.229 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.PNR7YzHL3i 2024-09-21T06:14:57.229 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.PNR7YzHL3i 2024-09-21T06:14:57.582 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.PNR7YzHL3i 2024-09-21T06:14:57.661 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.PNR7YzHL3i is 2MB 2024-09-21T06:14:57.741 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.PNR7YzHL3i 2024-09-21T06:14:57.749 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T06:14:57.749 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T06:14:57.753 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T06:14:57.827 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T06:14:57.861 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T06:14:57.861 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T06:14:57.863 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513/remote/smithi050/crash 2024-09-21T06:14:57.863 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T06:14:57.892 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513/remote/smithi088/crash 2024-09-21T06:14:57.893 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T06:14:57.918 INFO:tasks.ceph:Shutting down logrotate 2024-09-21T06:14:57.919 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:14:57.941 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T06:14:58.012 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T06:14:58.033 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T06:14:58.051 INFO:tasks.ceph:Compressing logs... 2024-09-21T06:14:58.051 DEBUG:teuthology.orchestra.run.smithi050:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:14:58.081 DEBUG:teuthology.orchestra.run.smithi088:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:14:58.131 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28996.log 2024-09-21T06:14:58.131 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T06:14:58.132 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.28996.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-21T06:14:58.132 INFO:teuthology.orchestra.run.smithi088.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28996.log.gz 2024-09-21T06:14:58.133 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28972.log 2024-09-21T06:14:58.134 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29075.log 2024-09-21T06:14:58.135 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.28972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34484.log 2024-09-21T06:14:58.135 INFO:teuthology.orchestra.run.smithi088.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28972.log.gz 2024-09-21T06:14:58.135 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.29075.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T06:14:58.136 INFO:teuthology.orchestra.run.smithi088.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.29075.log.gz 2024-09-21T06:14:58.136 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.1.34484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34484.log.gz 2024-09-21T06:14:58.136 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29099.log 2024-09-21T06:14:58.137 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29020.log 2024-09-21T06:14:58.139 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.29099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29123.log 2024-09-21T06:14:58.139 INFO:teuthology.orchestra.run.smithi088.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.29099.log.gz 2024-09-21T06:14:58.139 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.29020.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29020.log.gz 2024-09-21T06:14:58.140 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34586.log 2024-09-21T06:14:58.140 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.29123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34502.log 2024-09-21T06:14:58.141 INFO:teuthology.orchestra.run.smithi088.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29123.log.gz 2024-09-21T06:14:58.141 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.3.34586.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.34586.log.gz 2024-09-21T06:14:58.142 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29147.log 2024-09-21T06:14:58.142 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.1.34502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34502.log.gz 2024-09-21T06:14:58.143 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T06:14:58.144 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.29147.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T06:14:58.144 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29147.log.gz 2024-09-21T06:14:58.145 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29044.log 2024-09-21T06:14:58.146 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34591.log 2024-09-21T06:14:58.147 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.29044.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T06:14:58.148 INFO:teuthology.orchestra.run.smithi088.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29044.log.gz 2024-09-21T06:14:58.148 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.3.34591.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.3.34591.log.gz -5 2024-09-21T06:14:58.148 INFO:teuthology.orchestra.run.smithi088.stderr: --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T06:14:58.149 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-21T06:14:58.154 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.d.log: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T06:14:58.154 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-21T06:14:58.155 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T06:14:58.158 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-mon.b.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T06:14:58.166 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T06:14:58.217 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.4.log: 94.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-21T06:14:58.233 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46171.log 2024-09-21T06:14:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51485.log 2024-09-21T06:14:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35584.log 2024-09-21T06:14:58.235 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46171.log.gz 2024-09-21T06:14:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54021.log 2024-09-21T06:14:58.235 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51485.log.gz 2024-09-21T06:14:58.236 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45222.log 2024-09-21T06:14:58.236 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35584.log.gz 2024-09-21T06:14:58.237 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55082.log 2024-09-21T06:14:58.237 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54021.log.gz 2024-09-21T06:14:58.238 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45222.log: gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.53549.log 2024-09-21T06:14:58.238 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45222.log.gz 2024-09-21T06:14:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.55082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53497.log 2024-09-21T06:14:58.239 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55082.log.gz 2024-09-21T06:14:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50496.log 2024-09-21T06:14:58.240 INFO:teuthology.orchestra.run.smithi050.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.53549.log.gz 2024-09-21T06:14:58.241 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46041.log 2024-09-21T06:14:58.241 INFO:teuthology.orchestra.run.smithi050.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.53497.log.gz 2024-09-21T06:14:58.242 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55056.log 2024-09-21T06:14:58.242 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50496.log.gz 2024-09-21T06:14:58.242 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44935.log 2024-09-21T06:14:58.243 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46041.log.gz 2024-09-21T06:14:58.243 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.55056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54203.log 2024-09-21T06:14:58.243 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55056.log.gz 2024-09-21T06:14:58.244 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38866.log 2024-09-21T06:14:58.244 INFO:teuthology.orchestra.run.smithi050.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.44935.log.gz 2024-09-21T06:14:58.245 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46381.log 2024-09-21T06:14:58.245 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54203.log.gz 2024-09-21T06:14:58.246 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49982.log 2024-09-21T06:14:58.246 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38866.log.gz 2024-09-21T06:14:58.247 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54047.log 2024-09-21T06:14:58.247 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46381.log.gz 2024-09-21T06:14:58.248 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40899.log 2024-09-21T06:14:58.248 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49982.log.gz 2024-09-21T06:14:58.248 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47262.log 2024-09-21T06:14:58.249 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54047.log.gz 2024-09-21T06:14:58.249 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37324.log 2024-09-21T06:14:58.249 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40899.log.gz 2024-09-21T06:14:58.250 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48556.log 2024-09-21T06:14:58.250 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47262.log.gz 2024-09-21T06:14:58.251 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54310.log 2024-09-21T06:14:58.251 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37324.log.gz 2024-09-21T06:14:58.252 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40182.log 2024-09-21T06:14:58.252 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48556.log.gz 2024-09-21T06:14:58.253 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47721.log 2024-09-21T06:14:58.253 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54310.log.gz 2024-09-21T06:14:58.254 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44792.log 2024-09-21T06:14:58.254 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40182.log.gz 2024-09-21T06:14:58.255 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33900.log 2024-09-21T06:14:58.255 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47721.log.gz 2024-09-21T06:14:58.255 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40811.log 2024-09-21T06:14:58.256 INFO:teuthology.orchestra.run.smithi050.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.44792.log.gz 2024-09-21T06:14:58.256 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41899.log 2024-09-21T06:14:58.256 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33900.log.gz 2024-09-21T06:14:58.257 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40811.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.23893.log 2024-09-21T06:14:58.257 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40811.log.gz 2024-09-21T06:14:58.258 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53863.log 2024-09-21T06:14:58.258 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41899.log.gz 2024-09-21T06:14:58.259 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph.tmp-client.admin.23893.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.23893.log.gz 2024-09-21T06:14:58.259 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54281.log 2024-09-21T06:14:58.260 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35839.log 2024-09-21T06:14:58.260 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53863.log.gz 2024-09-21T06:14:58.261 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51102.log 2024-09-21T06:14:58.261 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54281.log.gz 2024-09-21T06:14:58.261 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37272.log 2024-09-21T06:14:58.261 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35839.log.gz 2024-09-21T06:14:58.262 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46462.log 2024-09-21T06:14:58.262 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51102.log.gz 2024-09-21T06:14:58.263 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46527.log 2024-09-21T06:14:58.263 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37272.log.gz 2024-09-21T06:14:58.264 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50064.log 2024-09-21T06:14:58.264 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46462.log.gz 2024-09-21T06:14:58.265 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43462.log 2024-09-21T06:14:58.265 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46527.log.gz 2024-09-21T06:14:58.266 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54343.log 2024-09-21T06:14:58.266 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50064.log.gz 2024-09-21T06:14:58.267 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34148.log 2024-09-21T06:14:58.267 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43462.log.gz 2024-09-21T06:14:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39459.log 2024-09-21T06:14:58.269 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54343.log.gz 2024-09-21T06:14:58.269 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34148.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-21T06:14:58.269 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34148.log.gz 2024-09-21T06:14:58.270 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53941.log 2024-09-21T06:14:58.270 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39459.log.gz 2024-09-21T06:14:58.271 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54874.log 2024-09-21T06:14:58.272 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53837.log 2024-09-21T06:14:58.272 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53941.log.gz 2024-09-21T06:14:58.272 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40681.log 2024-09-21T06:14:58.272 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54874.log.gz 2024-09-21T06:14:58.272 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37175.log 2024-09-21T06:14:58.273 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53837.log.gz 2024-09-21T06:14:58.273 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37966.log 2024-09-21T06:14:58.274 INFO:teuthology.orchestra.run.smithi050.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.40681.log.gz 2024-09-21T06:14:58.274 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37175.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37175.log.gz 2024-09-21T06:14:58.274 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47474.log 2024-09-21T06:14:58.275 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45644.log 2024-09-21T06:14:58.275 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37966.log.gz 2024-09-21T06:14:58.276 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53197.log 2024-09-21T06:14:58.276 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47474.log.gz 2024-09-21T06:14:58.277 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43748.log 2024-09-21T06:14:58.277 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45644.log.gz 2024-09-21T06:14:58.278 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47236.log 2024-09-21T06:14:58.278 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53197.log.gz 2024-09-21T06:14:58.279 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50931.log 2024-09-21T06:14:58.279 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43748.log.gz 2024-09-21T06:14:58.280 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37123.log 2024-09-21T06:14:58.280 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47236.log.gz 2024-09-21T06:14:58.281 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38072.log 2024-09-21T06:14:58.281 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50931.log.gz 2024-09-21T06:14:58.281 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37123.log: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T06:14:58.282 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36573.log 2024-09-21T06:14:58.282 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37123.log.gz 2024-09-21T06:14:58.282 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36025.log 2024-09-21T06:14:58.282 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38072.log.gz 2024-09-21T06:14:58.284 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34367.log 2024-09-21T06:14:58.285 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36573.log.gz 2024-09-21T06:14:58.285 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45510.log 2024-09-21T06:14:58.285 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36025.log.gz 2024-09-21T06:14:58.286 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48079.log 2024-09-21T06:14:58.286 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34367.log.gz 2024-09-21T06:14:58.286 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35943.log 2024-09-21T06:14:58.286 INFO:teuthology.orchestra.run.smithi050.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.45510.log.gz 2024-09-21T06:14:58.287 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28991.log 2024-09-21T06:14:58.287 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48079.log.gz 2024-09-21T06:14:58.287 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50090.log 2024-09-21T06:14:58.288 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35943.log.gz 2024-09-21T06:14:58.288 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35637.log 2024-09-21T06:14:58.288 INFO:teuthology.orchestra.run.smithi050.stderr: 75.3% -- replaced with /var/log/ceph/ceph-client.admin.28991.log.gz 2024-09-21T06:14:58.289 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47288.log 2024-09-21T06:14:58.289 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50090.log.gz 2024-09-21T06:14:58.290 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50116.log 2024-09-21T06:14:58.290 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35637.log.gz 2024-09-21T06:14:58.291 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33402.log 2024-09-21T06:14:58.291 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47288.log.gz 2024-09-21T06:14:58.291 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42924.log 2024-09-21T06:14:58.292 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50116.log.gz 2024-09-21T06:14:58.292 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49770.log 2024-09-21T06:14:58.293 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33402.log.gz 2024-09-21T06:14:58.293 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54665.log 2024-09-21T06:14:58.293 INFO:teuthology.orchestra.run.smithi050.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.42924.log.gz 2024-09-21T06:14:58.294 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38098.log 2024-09-21T06:14:58.294 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49770.log.gz 2024-09-21T06:14:58.295 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40934.log 2024-09-21T06:14:58.295 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54665.log.gz 2024-09-21T06:14:58.296 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54952.log 2024-09-21T06:14:58.296 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.38098.log.gz 2024-09-21T06:14:58.297 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53052.log 2024-09-21T06:14:58.297 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40934.log.gz 2024-09-21T06:14:58.297 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38436.log 2024-09-21T06:14:58.298 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54952.log.gz 2024-09-21T06:14:58.298 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37583.log 2024-09-21T06:14:58.299 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53052.log.gz 2024-09-21T06:14:58.299 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44300.log 2024-09-21T06:14:58.299 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38436.log.gz 2024-09-21T06:14:58.300 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39537.log 2024-09-21T06:14:58.300 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37583.log.gz 2024-09-21T06:14:58.301 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45725.log 2024-09-21T06:14:58.301 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44300.log.gz 2024-09-21T06:14:58.302 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50220.log 2024-09-21T06:14:58.302 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39537.log.gz 2024-09-21T06:14:58.303 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46806.log 2024-09-21T06:14:58.303 INFO:teuthology.orchestra.run.smithi050.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.45725.log.gz 2024-09-21T06:14:58.303 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43592.log 2024-09-21T06:14:58.303 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.50220.log.gz 2024-09-21T06:14:58.304 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48608.log 2024-09-21T06:14:58.304 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46806.log.gz 2024-09-21T06:14:58.305 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33640.log 2024-09-21T06:14:58.305 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43592.log.gz 2024-09-21T06:14:58.306 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36428.log 2024-09-21T06:14:58.306 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48608.log.gz 2024-09-21T06:14:58.307 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47500.log 2024-09-21T06:14:58.307 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33640.log.gz 2024-09-21T06:14:58.307 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.36428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40546.log 2024-09-21T06:14:58.307 INFO:teuthology.orchestra.run.smithi050.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36428.log.gz 2024-09-21T06:14:58.308 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47500.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T06:14:58.309 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40546.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47500.log.gz 2024-09-21T06:14:58.309 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38345.log 2024-09-21T06:14:58.309 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40546.log.gz 2024-09-21T06:14:58.310 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38709.log 2024-09-21T06:14:58.311 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45103.log 2024-09-21T06:14:58.311 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38345.log.gz 2024-09-21T06:14:58.312 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36376.log 2024-09-21T06:14:58.312 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38709.log.gz 2024-09-21T06:14:58.313 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49930.log 2024-09-21T06:14:58.313 INFO:teuthology.orchestra.run.smithi050.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.45103.log.gz 2024-09-21T06:14:58.313 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43625.log 2024-09-21T06:14:58.314 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36376.log.gz 2024-09-21T06:14:58.314 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49930.log: gzip 86.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.49930.log.gz --verbose 2024-09-21T06:14:58.314 INFO:teuthology.orchestra.run.smithi050.stderr: -- /var/log/ceph/ceph-client.admin.36707.log 2024-09-21T06:14:58.315 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39355.log 2024-09-21T06:14:58.315 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43625.log.gz 2024-09-21T06:14:58.316 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37728.log 2024-09-21T06:14:58.316 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36707.log.gz 2024-09-21T06:14:58.317 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44079.log 2024-09-21T06:14:58.317 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39355.log.gz 2024-09-21T06:14:58.317 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37728.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37728.log.gz 2024-09-21T06:14:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44579.log 2024-09-21T06:14:58.318 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52451.log 2024-09-21T06:14:58.319 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44079.log.gz 2024-09-21T06:14:58.319 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40655.log 2024-09-21T06:14:58.319 INFO:teuthology.orchestra.run.smithi050.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.44579.log.gz 2024-09-21T06:14:58.320 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33588.log 2024-09-21T06:14:58.321 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52451.log.gz 2024-09-21T06:14:58.321 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46709.log 2024-09-21T06:14:58.321 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40655.log.gz 2024-09-21T06:14:58.322 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46433.log 2024-09-21T06:14:58.322 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33588.log.gz 2024-09-21T06:14:58.323 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39258.log 2024-09-21T06:14:58.323 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46709.log.gz 2024-09-21T06:14:58.323 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36547.log 2024-09-21T06:14:58.324 INFO:teuthology.orchestra.run.smithi050.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46433.log.gz 2024-09-21T06:14:58.324 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46553.log 2024-09-21T06:14:58.325 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39258.log.gz 2024-09-21T06:14:58.325 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42950.log 2024-09-21T06:14:58.326 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36547.log.gz 2024-09-21T06:14:58.326 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54479.log 2024-09-21T06:14:58.326 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46553.log.gz 2024-09-21T06:14:58.327 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48459.log 2024-09-21T06:14:58.327 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42950.log.gz 2024-09-21T06:14:58.328 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46631.log 2024-09-21T06:14:58.328 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54479.log.gz 2024-09-21T06:14:58.329 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47773.log 2024-09-21T06:14:58.329 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48459.log.gz 2024-09-21T06:14:58.330 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40455.log 2024-09-21T06:14:58.330 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46631.log.gz 2024-09-21T06:14:58.331 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44326.log 2024-09-21T06:14:58.331 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47773.log.gz 2024-09-21T06:14:58.331 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40455.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.40455.log.gz -5 2024-09-21T06:14:58.332 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42794.log 2024-09-21T06:14:58.332 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41191.log 2024-09-21T06:14:58.332 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44326.log.gz 2024-09-21T06:14:58.333 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44495.log 2024-09-21T06:14:58.334 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42794.log.gz 2024-09-21T06:14:58.334 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51184.log 2024-09-21T06:14:58.334 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41191.log.gz 2024-09-21T06:14:58.335 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33141.log 2024-09-21T06:14:58.335 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44495.log.gz 2024-09-21T06:14:58.336 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49904.log 2024-09-21T06:14:58.336 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51184.log.gz 2024-09-21T06:14:58.337 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37609.log 2024-09-21T06:14:58.337 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33141.log.gz 2024-09-21T06:14:58.338 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52510.log 2024-09-21T06:14:58.338 INFO:teuthology.orchestra.run.smithi050.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.49904.log.gz 2024-09-21T06:14:58.338 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37888.log 2024-09-21T06:14:58.339 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37609.log.gz 2024-09-21T06:14:58.339 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52510.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52510.log.gz 2024-09-21T06:14:58.340 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52008.log 2024-09-21T06:14:58.340 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51758.log 2024-09-21T06:14:58.341 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37888.log.gz 2024-09-21T06:14:58.341 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48686.log 2024-09-21T06:14:58.341 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52008.log.gz 2024-09-21T06:14:58.342 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36521.log 2024-09-21T06:14:58.342 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51758.log.gz 2024-09-21T06:14:58.343 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52423.log 2024-09-21T06:14:58.343 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48686.log.gz 2024-09-21T06:14:58.344 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28845.log 2024-09-21T06:14:58.344 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36521.log.gz 2024-09-21T06:14:58.345 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33939.log 2024-09-21T06:14:58.345 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52423.log.gz 2024-09-21T06:14:58.346 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28845.log.gz 2024-09-21T06:14:58.346 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46683.log 2024-09-21T06:14:58.346 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44027.log 2024-09-21T06:14:58.346 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33939.log.gz 2024-09-21T06:14:58.347 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38655.log 2024-09-21T06:14:58.347 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46683.log.gz 2024-09-21T06:14:58.348 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52267.log 2024-09-21T06:14:58.348 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44027.log.gz 2024-09-21T06:14:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38655.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.38655.log.gz 2024-09-21T06:14:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53601.log 2024-09-21T06:14:58.349 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42276.log 2024-09-21T06:14:58.350 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52267.log.gz 2024-09-21T06:14:58.350 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43657.log 2024-09-21T06:14:58.350 INFO:teuthology.orchestra.run.smithi050.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.53601.log.gz 2024-09-21T06:14:58.351 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55238.log 2024-09-21T06:14:58.352 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.42276.log.gz 2024-09-21T06:14:58.352 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43657.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43657.log.gz 2024-09-21T06:14:58.352 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52716.log 2024-09-21T06:14:58.353 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.55238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39329.log 2024-09-21T06:14:58.353 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55238.log.gz 2024-09-21T06:14:58.354 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36062.log 2024-09-21T06:14:58.354 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52716.log.gz 2024-09-21T06:14:58.354 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50470.log 2024-09-21T06:14:58.355 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39329.log.gz 2024-09-21T06:14:58.355 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44631.log 2024-09-21T06:14:58.356 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36062.log.gz 2024-09-21T06:14:58.356 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52846.log 2024-09-21T06:14:58.356 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50470.log.gz 2024-09-21T06:14:58.357 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50698.log 2024-09-21T06:14:58.357 INFO:teuthology.orchestra.run.smithi050.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.44631.log.gz 2024-09-21T06:14:58.358 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48212.log 2024-09-21T06:14:58.358 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52846.log.gz 2024-09-21T06:14:58.359 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37693.log 2024-09-21T06:14:58.359 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50698.log.gz 2024-09-21T06:14:58.359 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49475.log 2024-09-21T06:14:58.360 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48212.log.gz 2024-09-21T06:14:58.360 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49744.log 2024-09-21T06:14:58.361 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37693.log.gz 2024-09-21T06:14:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.49475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49475.log.gz 2024-09-21T06:14:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45130.log 2024-09-21T06:14:58.362 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53145.log 2024-09-21T06:14:58.363 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49744.log.gz 2024-09-21T06:14:58.363 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53627.log 2024-09-21T06:14:58.363 INFO:teuthology.orchestra.run.smithi050.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.45130.log.gz 2024-09-21T06:14:58.364 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45885.log 2024-09-21T06:14:58.364 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53145.log.gz 2024-09-21T06:14:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40284.log 2024-09-21T06:14:58.365 INFO:teuthology.orchestra.run.smithi050.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.53627.log.gz 2024-09-21T06:14:58.366 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45885.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45885.log.gz 2024-09-21T06:14:58.366 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35454.log 2024-09-21T06:14:58.366 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.40284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41769.log 2024-09-21T06:14:58.366 INFO:teuthology.orchestra.run.smithi050.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40284.log.gz 2024-09-21T06:14:58.367 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40520.log 2024-09-21T06:14:58.367 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35454.log.gz 2024-09-21T06:14:58.368 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40785.log 2024-09-21T06:14:58.368 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41769.log.gz 2024-09-21T06:14:58.369 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40520.log: gzip 87.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.40520.log.gz -- 2024-09-21T06:14:58.369 INFO:teuthology.orchestra.run.smithi050.stderr: /var/log/ceph/ceph-client.admin.37402.log 2024-09-21T06:14:58.369 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43280.log 2024-09-21T06:14:58.370 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40785.log.gz 2024-09-21T06:14:58.371 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35558.log 2024-09-21T06:14:58.371 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37402.log.gz 2024-09-21T06:14:58.371 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42453.log 2024-09-21T06:14:58.372 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43280.log.gz 2024-09-21T06:14:58.372 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53275.log 2024-09-21T06:14:58.372 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35558.log.gz 2024-09-21T06:14:58.373 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44657.log 2024-09-21T06:14:58.373 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.42453.log.gz 2024-09-21T06:14:58.374 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38762.log 2024-09-21T06:14:58.374 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53275.log.gz 2024-09-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53223.log 2024-09-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44657.log.gz 2024-09-21T06:14:58.376 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37992.log 2024-09-21T06:14:58.376 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38762.log.gz 2024-09-21T06:14:58.376 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37454.log 2024-09-21T06:14:58.377 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53223.log.gz 2024-09-21T06:14:58.377 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49826.log 2024-09-21T06:14:58.377 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37992.log.gz 2024-09-21T06:14:58.378 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44605.log 2024-09-21T06:14:58.378 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37454.log.gz 2024-09-21T06:14:58.379 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35506.log 2024-09-21T06:14:58.379 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49826.log: /var/log/ceph/ceph-client.admin.44605.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49826.log.gz 2024-09-21T06:14:58.380 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44605.log.gz 2024-09-21T06:14:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53707.log 2024-09-21T06:14:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51158.log 2024-09-21T06:14:58.381 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35506.log.gz 2024-09-21T06:14:58.381 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48381.log 2024-09-21T06:14:58.382 INFO:teuthology.orchestra.run.smithi050.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.53707.log.gz 2024-09-21T06:14:58.382 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47602.log 2024-09-21T06:14:58.383 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51158.log.gz 2024-09-21T06:14:58.383 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52633.log 2024-09-21T06:14:58.383 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48381.log.gz 2024-09-21T06:14:58.384 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.47602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39721.log 2024-09-21T06:14:58.384 INFO:teuthology.orchestra.run.smithi050.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47602.log.gz 2024-09-21T06:14:58.385 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37862.log 2024-09-21T06:14:58.385 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52633.log.gz 2024-09-21T06:14:58.386 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39721.log: gzip -5 87.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.39721.log.gz /var/log/ceph/ceph-client.admin.43358.log 2024-09-21T06:14:58.386 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-09-21T06:14:58.386 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51678.log 2024-09-21T06:14:58.387 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37862.log.gz 2024-09-21T06:14:58.387 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34794.log 2024-09-21T06:14:58.387 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43358.log.gz 2024-09-21T06:14:58.388 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28761.log 2024-09-21T06:14:58.388 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51678.log.gz 2024-09-21T06:14:58.389 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37754.log 2024-09-21T06:14:58.389 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34794.log.gz 2024-09-21T06:14:58.390 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28761.log: 76.8% -- replaced with /var/log/ceph/ceph-client.admin.28761.log.gz 2024-09-21T06:14:58.390 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43254.log 2024-09-21T06:14:58.391 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37149.log 2024-09-21T06:14:58.392 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43254.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37754.log.gz 2024-09-21T06:14:58.392 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34955.log 2024-09-21T06:14:58.392 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43254.log.gz 2024-09-21T06:14:58.393 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44001.log 2024-09-21T06:14:58.393 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37149.log.gz 2024-09-21T06:14:58.393 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34550.log 2024-09-21T06:14:58.393 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34955.log.gz 2024-09-21T06:14:58.394 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44001.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T06:14:58.394 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44001.log.gz 2024-09-21T06:14:58.395 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35226.log 2024-09-21T06:14:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34550.log: /var/log/ceph/ceph-osd.2.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34550.log.gz 2024-09-21T06:14:58.396 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42531.log 2024-09-21T06:14:58.397 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35148.log 2024-09-21T06:14:58.397 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35226.log.gz 2024-09-21T06:14:58.397 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47396.log 2024-09-21T06:14:58.398 INFO:teuthology.orchestra.run.smithi050.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.42531.log.gz 2024-09-21T06:14:58.398 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36326.log 2024-09-21T06:14:58.399 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35148.log.gz 2024-09-21T06:14:58.399 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49406.log 2024-09-21T06:14:58.400 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47396.log.gz 2024-09-21T06:14:58.400 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46119.log 2024-09-21T06:14:58.401 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.36326.log.gz 2024-09-21T06:14:58.401 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52345.log 2024-09-21T06:14:58.402 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49406.log.gz 2024-09-21T06:14:58.402 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46119.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46119.log.gz 2024-09-21T06:14:58.402 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44709.log 2024-09-21T06:14:58.403 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42872.log 2024-09-21T06:14:58.404 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52345.log.gz 2024-09-21T06:14:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44709.log: gzip -5 87.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.44709.log.gz -- 2024-09-21T06:14:58.404 INFO:teuthology.orchestra.run.smithi050.stderr: /var/log/ceph/ceph-client.admin.35997.log 2024-09-21T06:14:58.405 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35532.log 2024-09-21T06:14:58.405 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42872.log.gz 2024-09-21T06:14:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43202.log 2024-09-21T06:14:58.406 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35997.log.gz 2024-09-21T06:14:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39154.log 2024-09-21T06:14:58.408 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35532.log.gz 2024-09-21T06:14:58.408 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43202.log: gzip -5 --verbose -- 87.2% /var/log/ceph/ceph-client.admin.44735.log -- replaced with /var/log/ceph/ceph-client.admin.43202.log.gz 2024-09-21T06:14:58.408 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-09-21T06:14:58.409 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48660.log 2024-09-21T06:14:58.409 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39154.log.gz 2024-09-21T06:14:58.410 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44735.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.44735.log.gz 2024-09-21T06:14:58.410 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43488.log 2024-09-21T06:14:58.411 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39749.log 2024-09-21T06:14:58.411 INFO:teuthology.orchestra.run.smithi050.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.48660.log.gz 2024-09-21T06:14:58.412 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50194.log 2024-09-21T06:14:58.412 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43488.log.gz 2024-09-21T06:14:58.413 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35786.log 2024-09-21T06:14:58.413 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39749.log.gz 2024-09-21T06:14:58.414 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45963.log 2024-09-21T06:14:58.414 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50194.log.gz 2024-09-21T06:14:58.415 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33953.log 2024-09-21T06:14:58.415 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35786.log.gz 2024-09-21T06:14:58.416 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28888.log 2024-09-21T06:14:58.416 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45963.log.gz 2024-09-21T06:14:58.417 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49628.log 2024-09-21T06:14:58.417 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33953.log.gz 2024-09-21T06:14:58.418 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28888.log.gz 2024-09-21T06:14:58.418 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54822.log 2024-09-21T06:14:58.419 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36599.log 2024-09-21T06:14:58.419 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54822.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49628.log.gz 2024-09-21T06:14:58.420 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45429.log 2024-09-21T06:14:58.420 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54822.log.gz 2024-09-21T06:14:58.421 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46301.log/var/log/ceph/ceph-client.admin.36599.log: 2024-09-21T06:14:58.421 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36599.log.gz 2024-09-21T06:14:58.422 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42427.log 2024-09-21T06:14:58.422 INFO:teuthology.orchestra.run.smithi050.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.45429.log.gz 2024-09-21T06:14:58.423 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46301.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46301.log.gz 2024-09-21T06:14:58.423 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40602.log 2024-09-21T06:14:58.424 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53301.log 2024-09-21T06:14:58.424 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42427.log.gz 2024-09-21T06:14:58.425 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39068.log 2024-09-21T06:14:58.425 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40602.log.gz 2024-09-21T06:14:58.426 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53301.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T06:14:58.426 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53301.log.gz 2024-09-21T06:14:58.427 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33376.log 2024-09-21T06:14:58.427 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39068.log.gz 2024-09-21T06:14:58.427 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41555.log 2024-09-21T06:14:58.429 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52293.log 2024-09-21T06:14:58.429 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33376.log.gz 2024-09-21T06:14:58.430 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41555.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.41555.log.gz --verbose 2024-09-21T06:14:58.430 INFO:teuthology.orchestra.run.smithi050.stderr: -- /var/log/ceph/ceph-client.admin.46249.log 2024-09-21T06:14:58.431 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51433.log 2024-09-21T06:14:58.431 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52293.log: /var/log/ceph/ceph-client.admin.46249.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52293.log.gz 2024-09-21T06:14:58.432 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35812.log 2024-09-21T06:14:58.432 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46249.log.gz 2024-09-21T06:14:58.433 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34357.log 2024-09-21T06:14:58.433 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51433.log.gz 2024-09-21T06:14:58.434 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41243.log 2024-09-21T06:14:58.434 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35812.log: 87.5%/var/log/ceph/ceph-client.admin.34357.log: -- replaced with /var/log/ceph/ceph-client.admin.35812.log.gz 2024-09-21T06:14:58.435 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35174.log 2024-09-21T06:14:58.435 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34357.log.gz 2024-09-21T06:14:58.436 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39920.log 2024-09-21T06:14:58.436 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41243.log.gz 2024-09-21T06:14:58.437 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47448.log 2024-09-21T06:14:58.437 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35174.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35174.log.gz 2024-09-21T06:14:58.438 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53171.log 2024-09-21T06:14:58.438 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39920.log.gz 2024-09-21T06:14:58.439 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49602.log 2024-09-21T06:14:58.439 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47448.log.gz 2024-09-21T06:14:58.440 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55317.log 2024-09-21T06:14:58.441 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53171.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53171.log.gz 2024-09-21T06:14:58.441 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34162.log 2024-09-21T06:14:58.441 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49602.log.gz 2024-09-21T06:14:58.442 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.55317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35760.log 2024-09-21T06:14:58.442 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55317.log.gz 2024-09-21T06:14:58.443 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51916.log 2024-09-21T06:14:58.444 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34162.log: /var/log/ceph/ceph-client.admin.35760.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34162.log.gz 2024-09-21T06:14:58.444 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48634.log 2024-09-21T06:14:58.444 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35760.log.gz 2024-09-21T06:14:58.445 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37836.log 2024-09-21T06:14:58.445 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51916.log.gz 2024-09-21T06:14:58.446 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35695.log 2024-09-21T06:14:58.447 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48634.log.gz 2024-09-21T06:14:58.447 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49576.log 2024-09-21T06:14:58.448 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37836.log.gz 2024-09-21T06:14:58.448 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35695.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35695.log.gz 2024-09-21T06:14:58.448 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47344.log 2024-09-21T06:14:58.449 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51784.log 2024-09-21T06:14:58.450 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49576.log: /var/log/ceph/ceph-client.admin.47344.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49576.log.gz 2024-09-21T06:14:58.451 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38918.log 2024-09-21T06:14:58.451 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47344.log.gz 2024-09-21T06:14:58.452 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40130.log 2024-09-21T06:14:58.452 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51784.log.gz 2024-09-21T06:14:58.452 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54848.log 2024-09-21T06:14:58.453 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38918.log: /var/log/ceph/ceph-client.admin.40130.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38918.log.gz 2024-09-21T06:14:58.454 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51316.log 2024-09-21T06:14:58.454 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40130.log.gz 2024-09-21T06:14:58.455 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54848.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54848.log.gz 2024-09-21T06:14:58.455 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28949.log 2024-09-21T06:14:58.455 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.51316.log: /var/log/ceph/ceph-client.admin.36196.log 2024-09-21T06:14:58.457 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28949.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28949.log.gz 2024-09-21T06:14:58.457 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51316.log.gz 2024-09-21T06:14:58.457 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34734.log 2024-09-21T06:14:58.458 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49148.log 2024-09-21T06:14:58.458 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36196.log.gz 2024-09-21T06:14:58.459 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51264.log 2024-09-21T06:14:58.459 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.34734.log.gz 2024-09-21T06:14:58.460 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41873.log 2024-09-21T06:14:58.460 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49148.log.gz 2024-09-21T06:14:58.461 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46995.log 2024-09-21T06:14:58.461 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51264.log.gz 2024-09-21T06:14:58.462 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35402.log 2024-09-21T06:14:58.463 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41873.log.gz 2024-09-21T06:14:58.464 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29477.log 2024-09-21T06:14:58.464 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46995.log.gz 2024-09-21T06:14:58.464 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41977.log 2024-09-21T06:14:58.465 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35402.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35402.log.gz 2024-09-21T06:14:58.466 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.29477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36766.log 2024-09-21T06:14:58.466 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29477.log.gz 2024-09-21T06:14:58.467 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41977.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41977.log.gz 2024-09-21T06:14:58.467 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50392.log 2024-09-21T06:14:58.468 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35122.log 2024-09-21T06:14:58.468 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36766.log: /var/log/ceph/ceph-client.admin.50392.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36766.log.gz 2024-09-21T06:14:58.469 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33562.log 2024-09-21T06:14:58.469 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50392.log.gz 2024-09-21T06:14:58.470 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54926.log 2024-09-21T06:14:58.470 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35122.log.gz 2024-09-21T06:14:58.471 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.33562.log: /var/log/ceph/ceph-client.admin.45911.log 2024-09-21T06:14:58.471 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33562.log.gz 2024-09-21T06:14:58.472 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42100.log 2024-09-21T06:14:58.472 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54926.log.gz 2024-09-21T06:14:58.473 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47975.log 2024-09-21T06:14:58.473 INFO:teuthology.orchestra.run.smithi050.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45911.log.gz 2024-09-21T06:14:58.474 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42401.log 2024-09-21T06:14:58.474 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.42100.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42100.log.gz 2024-09-21T06:14:58.475 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46015.log 2024-09-21T06:14:58.475 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47975.log.gz 2024-09-21T06:14:58.476 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42609.log 2024-09-21T06:14:58.476 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42401.log.gz 2024-09-21T06:14:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49956.log 2024-09-21T06:14:58.477 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46015.log.gz 2024-09-21T06:14:58.478 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45671.log 2024-09-21T06:14:58.478 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.42609.log.gz 2024-09-21T06:14:58.479 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49956.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49956.log.gz 2024-09-21T06:14:58.479 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40576.log 2024-09-21T06:14:58.480 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5/var/log/ceph/ceph-client.admin.45671.log: --verbose -- /var/log/ceph/ceph-client.admin.55030.log 2024-09-21T06:14:58.481 INFO:teuthology.orchestra.run.smithi050.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.45671.log.gz 2024-09-21T06:14:58.481 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36088.log 2024-09-21T06:14:58.481 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40576.log.gz 2024-09-21T06:14:58.482 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.55030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38462.log 2024-09-21T06:14:58.482 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55030.log.gz 2024-09-21T06:14:58.483 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50841.log 2024-09-21T06:14:58.484 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36088.log: /var/log/ceph/ceph-client.admin.38462.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36088.log.gz 2024-09-21T06:14:58.484 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36897.log 2024-09-21T06:14:58.484 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38462.log.gz 2024-09-21T06:14:58.485 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50841.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50841.log.gz 2024-09-21T06:14:58.485 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36818.log 2024-09-21T06:14:58.486 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5/var/log/ceph/ceph-client.admin.36897.log: --verbose -- /var/log/ceph/ceph-client.admin.36954.log 2024-09-21T06:14:58.487 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36897.log.gz 2024-09-21T06:14:58.487 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36818.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36818.log.gz 2024-09-21T06:14:58.488 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44233.log 2024-09-21T06:14:58.488 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43332.log 2024-09-21T06:14:58.489 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36954.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36954.log.gz 2024-09-21T06:14:58.489 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.44233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44233.log.gz 2024-09-21T06:14:58.489 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33692.log 2024-09-21T06:14:58.491 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52950.log 2024-09-21T06:14:58.491 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43332.log.gz 2024-09-21T06:14:58.492 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54177.log 2024-09-21T06:14:58.492 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33692.log.gz 2024-09-21T06:14:58.493 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44105.log 2024-09-21T06:14:58.493 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52950.log.gz 2024-09-21T06:14:58.494 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34706.log 2024-09-21T06:14:58.494 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54177.log.gz 2024-09-21T06:14:58.495 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50567.log 2024-09-21T06:14:58.495 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44105.log.gz 2024-09-21T06:14:58.496 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45158.log 2024-09-21T06:14:58.496 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34706.log.gz 2024-09-21T06:14:58.497 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54375.log 2024-09-21T06:14:58.497 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50567.log.gz 2024-09-21T06:14:58.498 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51812.log 2024-09-21T06:14:58.498 INFO:teuthology.orchestra.run.smithi050.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45158.log.gz 2024-09-21T06:14:58.499 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49096.log 2024-09-21T06:14:58.500 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54375.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54375.log.gz 2024-09-21T06:14:58.500 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43436.log 2024-09-21T06:14:58.500 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51812.log.gz 2024-09-21T06:14:58.501 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49096.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49096.log.gz 2024-09-21T06:14:58.502 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49550.log 2024-09-21T06:14:58.502 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.43436.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49304.log 2024-09-21T06:14:58.504 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43436.log.gz 2024-09-21T06:14:58.504 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36248.log 2024-09-21T06:14:58.504 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49550.log.gz 2024-09-21T06:14:58.505 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49304.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.49304.log.gz 2024-09-21T06:14:58.505 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55134.log 2024-09-21T06:14:58.507 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43228.log 2024-09-21T06:14:58.507 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36248.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36248.log.gz 2024-09-21T06:14:58.507 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.55134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50957.log 2024-09-21T06:14:58.507 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55134.log.gz 2024-09-21T06:14:58.508 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37045.log 2024-09-21T06:14:58.508 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43228.log.gz 2024-09-21T06:14:58.509 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42304.log 2024-09-21T06:14:58.509 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50957.log.gz 2024-09-21T06:14:58.510 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53785.log 2024-09-21T06:14:58.510 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37045.log.gz 2024-09-21T06:14:58.511 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51652.log 2024-09-21T06:14:58.511 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42304.log.gz 2024-09-21T06:14:58.512 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37487.log 2024-09-21T06:14:58.512 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53785.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53785.log.gz 2024-09-21T06:14:58.513 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51418.log 2024-09-21T06:14:58.513 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51652.log.gz 2024-09-21T06:14:58.514 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43082.log 2024-09-21T06:14:58.514 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37487.log.gz 2024-09-21T06:14:58.515 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.51418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52794.log 2024-09-21T06:14:58.515 INFO:teuthology.orchestra.run.smithi050.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51418.log.gz 2024-09-21T06:14:58.516 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33848.log 2024-09-21T06:14:58.516 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43082.log.gz 2024-09-21T06:14:58.517 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52794.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52794.log.gz 2024-09-21T06:14:58.517 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46858.log 2024-09-21T06:14:58.518 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44969.log 2024-09-21T06:14:58.519 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33848.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33848.log.gz 2024-09-21T06:14:58.519 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54557.log 2024-09-21T06:14:58.519 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46858.log.gz 2024-09-21T06:14:58.520 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35865.log 2024-09-21T06:14:58.520 INFO:teuthology.orchestra.run.smithi050.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.44969.log.gz 2024-09-21T06:14:58.521 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33796.log 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54557.log.gz 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50418.log 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35865.log.gz 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33796.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33796.log.gz 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41691.log 2024-09-21T06:14:58.524 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51732.log 2024-09-21T06:14:58.525 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50418.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50418.log.gz 2024-09-21T06:14:58.525 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28984.log 2024-09-21T06:14:58.525 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41691.log.gz 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41217.log 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi050.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.51732.log.gz 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50809.log 2024-09-21T06:14:58.528 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28984.log.gz 2024-09-21T06:14:58.528 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47897.log 2024-09-21T06:14:58.529 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41217.log.gz 2024-09-21T06:14:58.529 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50809.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50809.log.gz 2024-09-21T06:14:58.530 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44248.log 2024-09-21T06:14:58.530 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40156.log 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47897.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47897.log.gz/var/log/ceph/ceph-client.admin.44248.log: 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40050.log 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44248.log.gz 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40156.log: gzip -5 --verbose 87.5% -- -- replaced with /var/log/ceph/ceph-client.admin.40156.log.gz 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi050.stderr: /var/log/ceph/ceph-client.admin.52144.log 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log/var/log/ceph/ceph-client.admin.40050.log: 2024-09-21T06:14:58.534 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40050.log.gz 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29309.log 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52144.log.gz 2024-09-21T06:14:58.536 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50593.log 2024-09-21T06:14:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.29309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43949.log 2024-09-21T06:14:58.537 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29309.log.gz 2024-09-21T06:14:58.538 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41094.log 2024-09-21T06:14:58.538 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50593.log.gz 2024-09-21T06:14:58.539 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45354.log 2024-09-21T06:14:58.540 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43949.log.gz 2024-09-21T06:14:58.541 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48868.log 2024-09-21T06:14:58.541 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41094.log.gz 2024-09-21T06:14:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47201.log 2024-09-21T06:14:58.542 INFO:teuthology.orchestra.run.smithi050.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.45354.log.gz 2024-09-21T06:14:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35891.log 2024-09-21T06:14:58.543 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48868.log.gz 2024-09-21T06:14:58.544 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52196.log 2024-09-21T06:14:58.544 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47201.log.gz 2024-09-21T06:14:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55186.log 2024-09-21T06:14:58.545 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35891.log.gz 2024-09-21T06:14:58.546 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41925.log 2024-09-21T06:14:58.546 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52196.log.gz 2024-09-21T06:14:58.547 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.55186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45780.log 2024-09-21T06:14:58.547 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55186.log.gz 2024-09-21T06:14:58.548 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42224.log 2024-09-21T06:14:58.548 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.41925.log.gz 2024-09-21T06:14:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36274.log 2024-09-21T06:14:58.549 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45780.log.gz 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49524.log 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42224.log.gz 2024-09-21T06:14:58.552 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46579.log 2024-09-21T06:14:58.552 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36274.log.gz 2024-09-21T06:14:58.554 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49038.log 2024-09-21T06:14:58.554 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49524.log.gz 2024-09-21T06:14:58.554 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48504.log 2024-09-21T06:14:58.554 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46579.log.gz 2024-09-21T06:14:58.555 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51890.log 2024-09-21T06:14:58.555 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49038.log.gz 2024-09-21T06:14:58.556 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42661.log 2024-09-21T06:14:58.556 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48504.log.gz 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41847.log 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51890.log.gz 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45858.log 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi050.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.42661.log.gz 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40104.log 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41847.log.gz 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45404.log 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45858.log.gz 2024-09-21T06:14:58.562 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46223.log 2024-09-21T06:14:58.562 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40104.log.gz 2024-09-21T06:14:58.563 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41165.log 2024-09-21T06:14:58.563 INFO:teuthology.orchestra.run.smithi050.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.45404.log.gz 2024-09-21T06:14:58.564 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38371.log 2024-09-21T06:14:58.564 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46223.log.gz 2024-09-21T06:14:58.565 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54505.log 2024-09-21T06:14:58.565 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41165.log.gz 2024-09-21T06:14:58.567 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39381.log 2024-09-21T06:14:58.567 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38371.log.gz 2024-09-21T06:14:58.568 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42635.log 2024-09-21T06:14:58.568 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54505.log.gz 2024-09-21T06:14:58.569 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45698.log 2024-09-21T06:14:58.569 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39381.log.gz 2024-09-21T06:14:58.570 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33770.log 2024-09-21T06:14:58.570 INFO:teuthology.orchestra.run.smithi050.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.42635.log.gz 2024-09-21T06:14:58.571 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40024.log 2024-09-21T06:14:58.571 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.45698.log.gz 2024-09-21T06:14:58.572 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53444.log 2024-09-21T06:14:58.572 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33770.log.gz 2024-09-21T06:14:58.574 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52924.log 2024-09-21T06:14:58.574 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40024.log.gz 2024-09-21T06:14:58.575 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36170.log 2024-09-21T06:14:58.575 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53444.log.gz 2024-09-21T06:14:58.576 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33277.log 2024-09-21T06:14:58.576 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52924.log.gz 2024-09-21T06:14:58.577 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51615.log 2024-09-21T06:14:58.577 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36170.log.gz 2024-09-21T06:14:58.578 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42250.log 2024-09-21T06:14:58.578 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33277.log.gz 2024-09-21T06:14:58.579 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53471.log 2024-09-21T06:14:58.579 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51615.log.gz 2024-09-21T06:14:58.580 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47027.log 2024-09-21T06:14:58.580 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42250.log.gz 2024-09-21T06:14:58.581 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38200.log 2024-09-21T06:14:58.581 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.53471.log.gz 2024-09-21T06:14:58.583 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53364.log 2024-09-21T06:14:58.583 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47027.log.gz 2024-09-21T06:14:58.583 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.38200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44430.log 2024-09-21T06:14:58.584 INFO:teuthology.orchestra.run.smithi050.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38200.log.gz 2024-09-21T06:14:58.585 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54770.log 2024-09-21T06:14:58.585 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53364.log.gz 2024-09-21T06:14:58.586 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45751.log 2024-09-21T06:14:58.586 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44430.log.gz 2024-09-21T06:14:58.587 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47643.log 2024-09-21T06:14:58.587 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54770.log.gz 2024-09-21T06:14:58.588 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46145.log 2024-09-21T06:14:58.588 INFO:teuthology.orchestra.run.smithi050.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.45751.log.gz 2024-09-21T06:14:58.589 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34654.log 2024-09-21T06:14:58.589 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47643.log.gz 2024-09-21T06:14:58.590 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45617.log 2024-09-21T06:14:58.590 INFO:teuthology.orchestra.run.smithi050.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.46145.log.gz 2024-09-21T06:14:58.591 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33614.log 2024-09-21T06:14:58.591 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34654.log.gz 2024-09-21T06:14:58.592 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38840.log 2024-09-21T06:14:58.593 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.45617.log.gz 2024-09-21T06:14:58.594 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37350.log 2024-09-21T06:14:58.594 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33614.log.gz 2024-09-21T06:14:58.595 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49878.log 2024-09-21T06:14:58.595 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38840.log.gz 2024-09-21T06:14:58.596 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47923.log 2024-09-21T06:14:58.596 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37350.log.gz 2024-09-21T06:14:58.597 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53759.log 2024-09-21T06:14:58.597 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49878.log.gz 2024-09-21T06:14:58.598 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42820.log 2024-09-21T06:14:58.598 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.47923.log.gz 2024-09-21T06:14:58.599 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51706.log 2024-09-21T06:14:58.599 INFO:teuthology.orchestra.run.smithi050.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.53759.log.gz 2024-09-21T06:14:58.600 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44228.log 2024-09-21T06:14:58.600 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42820.log.gz 2024-09-21T06:14:58.602 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42172.log 2024-09-21T06:14:58.602 INFO:teuthology.orchestra.run.smithi050.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.51706.log.gz 2024-09-21T06:14:58.602 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.44228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54531.log 2024-09-21T06:14:58.602 INFO:teuthology.orchestra.run.smithi050.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44228.log.gz 2024-09-21T06:14:58.603 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41665.log 2024-09-21T06:14:58.603 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42172.log.gz 2024-09-21T06:14:58.605 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45379.log 2024-09-21T06:14:58.605 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54531.log.gz 2024-09-21T06:14:58.606 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39128.log 2024-09-21T06:14:58.606 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz 2024-09-21T06:14:58.607 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28773.log 2024-09-21T06:14:58.607 INFO:teuthology.orchestra.run.smithi050.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.45379.log.gz 2024-09-21T06:14:58.608 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39128.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T06:14:58.608 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39128.log.gz 2024-09-21T06:14:58.609 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28773.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.53523.log -- replaced with /var/log/ceph/ceph-client.admin.28773.log.gz 2024-09-21T06:14:58.609 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-09-21T06:14:58.610 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44908.log 2024-09-21T06:14:58.612 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53523.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53523.log.gz 2024-09-21T06:14:58.612 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51238.log 2024-09-21T06:14:58.613 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44908.log.gz 2024-09-21T06:14:58.614 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51015.log 2024-09-21T06:14:58.615 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49012.log 2024-09-21T06:14:58.615 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51238.log.gz 2024-09-21T06:14:58.616 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51015.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51015.log.gz 2024-09-21T06:14:58.616 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48277.log 2024-09-21T06:14:58.617 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49012.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49012.log.gz 2024-09-21T06:14:58.618 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48027.log 2024-09-21T06:14:58.619 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48329.log 2024-09-21T06:14:58.619 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48277.log.gz 2024-09-21T06:14:58.620 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48027.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48027.log.gz 2024-09-21T06:14:58.621 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48329.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48329.log.gz 2024-09-21T06:14:58.621 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48745.log 2024-09-21T06:14:58.623 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54978.log 2024-09-21T06:14:58.624 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39894.log 2024-09-21T06:14:58.624 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48745.log.gz 2024-09-21T06:14:58.625 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40429.log 2024-09-21T06:14:58.625 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54978.log.gz 2024-09-21T06:14:58.626 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39894.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39894.log.gz 2024-09-21T06:14:58.627 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39485.log 2024-09-21T06:14:58.627 INFO:teuthology.orchestra.run.smithi050.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.40429.log.gz 2024-09-21T06:14:58.628 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42976.log 2024-09-21T06:14:58.629 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46093.log 2024-09-21T06:14:58.629 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39485.log.gz 2024-09-21T06:14:58.631 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42976.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42976.log.gz 2024-09-21T06:14:58.631 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40351.log 2024-09-21T06:14:58.632 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46093.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46093.log.gz 2024-09-21T06:14:58.632 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33484.log 2024-09-21T06:14:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41795.log 2024-09-21T06:14:58.634 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40351.log.gz 2024-09-21T06:14:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37071.log 2024-09-21T06:14:58.635 INFO:teuthology.orchestra.run.smithi050.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.33484.log.gz 2024-09-21T06:14:58.636 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37428.log 2024-09-21T06:14:58.636 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41795.log.gz 2024-09-21T06:14:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52066.log 2024-09-21T06:14:58.637 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37071.log.gz 2024-09-21T06:14:58.638 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37428.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37428.log.gz 2024-09-21T06:14:58.639 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35663.log 2024-09-21T06:14:58.639 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52066.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52066.log.gz 2024-09-21T06:14:58.639 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41529.log 2024-09-21T06:14:58.641 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52319.log 2024-09-21T06:14:58.641 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35663.log.gz 2024-09-21T06:14:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41529.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41529.log.gz 2024-09-21T06:14:58.643 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52319.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52319.log.gz 2024-09-21T06:14:58.644 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49200.log 2024-09-21T06:14:58.645 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52542.log 2024-09-21T06:14:58.646 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52371.log 2024-09-21T06:14:58.646 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49200.log.gz 2024-09-21T06:14:58.648 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52542.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52542.log.gz 2024-09-21T06:14:58.648 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52371.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52371.log.gz 2024-09-21T06:14:58.649 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36222.log 2024-09-21T06:14:58.650 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53390.log 2024-09-21T06:14:58.651 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42742.log 2024-09-21T06:14:58.651 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36222.log.gz 2024-09-21T06:14:58.653 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53390.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53390.log.gz 2024-09-21T06:14:58.653 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42742.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.42742.log.gz 2024-09-21T06:14:58.654 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53655.log 2024-09-21T06:14:58.655 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41438.log 2024-09-21T06:14:58.656 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49356.log 2024-09-21T06:14:58.657 INFO:teuthology.orchestra.run.smithi050.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.53655.log.gz 2024-09-21T06:14:58.658 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34154.log 2024-09-21T06:14:58.658 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41438.log.gz 2024-09-21T06:14:58.659 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49356.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49356.log.gz 2024-09-21T06:14:58.660 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28759.log 2024-09-21T06:14:58.660 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34154.log.gz 2024-09-21T06:14:58.661 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35096.log 2024-09-21T06:14:58.662 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42105.log 2024-09-21T06:14:58.662 INFO:teuthology.orchestra.run.smithi050.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28759.log.gz 2024-09-21T06:14:58.663 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28876.log 2024-09-21T06:14:58.664 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35096.log.gz 2024-09-21T06:14:58.664 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.42105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42105.log.gz 2024-09-21T06:14:58.665 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48433.log 2024-09-21T06:14:58.666 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50034.log 2024-09-21T06:14:58.666 INFO:teuthology.orchestra.run.smithi050.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.28876.log.gz 2024-09-21T06:14:58.667 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34354.log 2024-09-21T06:14:58.667 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48433.log.gz 2024-09-21T06:14:58.668 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50034.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50034.log.gz 2024-09-21T06:14:58.669 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34354.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34354.log.gz 2024-09-21T06:14:58.670 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50008.log 2024-09-21T06:14:58.671 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52898.log 2024-09-21T06:14:58.672 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53327.log 2024-09-21T06:14:58.673 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50008.log.gz 2024-09-21T06:14:58.673 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42120.log 2024-09-21T06:14:58.673 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52898.log.gz 2024-09-21T06:14:58.674 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53327.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53327.log.gz 2024-09-21T06:14:58.676 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53119.log 2024-09-21T06:14:58.676 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42120.log.gz 2024-09-21T06:14:58.676 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50987.log 2024-09-21T06:14:58.678 INFO:teuthology.orchestra.run.smithi088.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T06:14:58.678 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51942.log 2024-09-21T06:14:58.679 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53119.log.gz 2024-09-21T06:14:58.679 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45535.log 2024-09-21T06:14:58.680 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50987.log.gz 2024-09-21T06:14:58.680 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51942.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.51942.log.gz -5 2024-09-21T06:14:58.680 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52742.log 2024-09-21T06:14:58.681 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46407.log 2024-09-21T06:14:58.681 INFO:teuthology.orchestra.run.smithi050.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.45535.log.gz 2024-09-21T06:14:58.683 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40279.log 2024-09-21T06:14:58.683 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52742.log.gz 2024-09-21T06:14:58.684 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46407.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46407.log.gz 2024-09-21T06:14:58.685 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41743.log 2024-09-21T06:14:58.685 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.40279.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40279.log.gz 2024-09-21T06:14:58.685 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43774.log 2024-09-21T06:14:58.687 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50444.log 2024-09-21T06:14:58.687 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41743.log.gz 2024-09-21T06:14:58.688 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43774.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43774.log.gz 2024-09-21T06:14:58.689 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50444.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50444.log.gz 2024-09-21T06:14:58.689 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46327.log 2024-09-21T06:14:58.691 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48183.log 2024-09-21T06:14:58.692 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45328.log 2024-09-21T06:14:58.692 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46327.log.gz 2024-09-21T06:14:58.693 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39180.log 2024-09-21T06:14:58.693 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48183.log.gz 2024-09-21T06:14:58.695 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45328.log: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.45328.log.gz 2024-09-21T06:14:58.695 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45937.log 2024-09-21T06:14:58.696 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39180.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39180.log.gz 2024-09-21T06:14:58.696 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55291.log 2024-09-21T06:14:58.697 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40299.log 2024-09-21T06:14:58.697 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45937.log.gz 2024-09-21T06:14:58.699 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.55291.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55291.log.gz 2024-09-21T06:14:58.699 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34370.log 2024-09-21T06:14:58.700 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40299.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40299.log.gz 2024-09-21T06:14:58.700 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50776.log 2024-09-21T06:14:58.701 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53418.log 2024-09-21T06:14:58.702 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34370.log.gz 2024-09-21T06:14:58.703 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50776.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50776.log.gz 2024-09-21T06:14:58.704 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53418.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53418.log.gz 2024-09-21T06:14:58.704 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46605.log 2024-09-21T06:14:58.705 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34358.log 2024-09-21T06:14:58.707 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28829.log 2024-09-21T06:14:58.707 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46605.log.gz 2024-09-21T06:14:58.708 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34358.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34358.log.gz 2024-09-21T06:14:58.709 INFO:teuthology.orchestra.run.smithi050.stderr: -- /var/log/ceph/ceph-client.admin.39511.log 2024-09-21T06:14:58.709 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28829.log.gz 2024-09-21T06:14:58.709 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48001.log 2024-09-21T06:14:58.711 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42583.log 2024-09-21T06:14:58.711 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39511.log.gz 2024-09-21T06:14:58.712 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44352.log 2024-09-21T06:14:58.712 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48001.log.gz 2024-09-21T06:14:58.713 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42583.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42583.log.gz 2024-09-21T06:14:58.713 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38046.log 2024-09-21T06:14:58.714 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44352.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44352.log.gz 2024-09-21T06:14:58.715 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35610.log 2024-09-21T06:14:58.716 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36300.log 2024-09-21T06:14:58.716 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38046.log.gz 2024-09-21T06:14:58.717 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35610.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35610.log.gz 2024-09-21T06:14:58.717 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43056.log 2024-09-21T06:14:58.719 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36300.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36300.log.gz 2024-09-21T06:14:58.719 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48923.log 2024-09-21T06:14:58.720 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40629.log 2024-09-21T06:14:58.720 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43056.log.gz 2024-09-21T06:14:58.722 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48923.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48923.log.gz 2024-09-21T06:14:58.722 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40629.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40629.log.gz 2024-09-21T06:14:58.723 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33666.log 2024-09-21T06:14:58.724 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T06:14:58.725 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52607.log 2024-09-21T06:14:58.725 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33666.log.gz 2024-09-21T06:14:58.727 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47695.log 2024-09-21T06:14:58.728 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52607.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52607.log.gz 2024-09-21T06:14:58.728 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41406.log 2024-09-21T06:14:58.730 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47695.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47695.log.gz 2024-09-21T06:14:58.730 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49174.log 2024-09-21T06:14:58.730 INFO:teuthology.orchestra.run.smithi050.stderr: 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T06:14:58.731 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41406.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41406.log.gz 2024-09-21T06:14:58.731 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52688.log 2024-09-21T06:14:58.733 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44683.log 2024-09-21T06:14:58.733 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49174.log.gz 2024-09-21T06:14:58.734 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52688.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52688.log.gz 2024-09-21T06:14:58.735 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44683.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44683.log.gz 2024-09-21T06:14:58.735 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46962.log 2024-09-21T06:14:58.736 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47747.log 2024-09-21T06:14:58.738 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47813.log 2024-09-21T06:14:58.738 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46962.log.gz 2024-09-21T06:14:58.739 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34160.log 2024-09-21T06:14:58.739 INFO:teuthology.orchestra.run.smithi050.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.47747.log.gz 2024-09-21T06:14:58.740 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36733.log 2024-09-21T06:14:58.741 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47813.log.gz 2024-09-21T06:14:58.741 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34160.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34160.log.gz 2024-09-21T06:14:58.742 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33874.log 2024-09-21T06:14:58.743 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43919.log 2024-09-21T06:14:58.743 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36733.log.gz 2024-09-21T06:14:58.744 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41016.log 2024-09-21T06:14:58.744 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33874.log.gz 2024-09-21T06:14:58.745 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43919.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43919.log.gz 2024-09-21T06:14:58.745 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54453.log 2024-09-21T06:14:58.746 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41016.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41016.log.gz 2024-09-21T06:14:58.747 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39669.log 2024-09-21T06:14:58.748 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47669.log 2024-09-21T06:14:58.748 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54453.log.gz 2024-09-21T06:14:58.749 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40990.log 2024-09-21T06:14:58.749 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39669.log.gz 2024-09-21T06:14:58.750 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50724.log 2024-09-21T06:14:58.750 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47669.log.gz 2024-09-21T06:14:58.752 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40990.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40990.log.gz 2024-09-21T06:14:58.752 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54796.log 2024-09-21T06:14:58.753 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50724.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50724.log.gz 2024-09-21T06:14:58.753 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53249.log 2024-09-21T06:14:58.754 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47173.log 2024-09-21T06:14:58.755 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54796.log.gz 2024-09-21T06:14:58.756 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53249.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53249.log.gz 2024-09-21T06:14:58.757 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47173.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47173.log.gz 2024-09-21T06:14:58.757 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46780.log 2024-09-21T06:14:58.758 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33428.log 2024-09-21T06:14:58.760 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41068.log 2024-09-21T06:14:58.760 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46780.log.gz 2024-09-21T06:14:58.760 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33428.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33428.log.gz 2024-09-21T06:14:58.761 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33536.log 2024-09-21T06:14:58.762 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41068.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41068.log.gz 2024-09-21T06:14:58.762 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51838.log 2024-09-21T06:14:58.763 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37780.log 2024-09-21T06:14:58.763 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33536.log.gz 2024-09-21T06:14:58.764 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51838.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51838.log.gz 2024-09-21T06:14:58.766 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37780.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37780.log.gz 2024-09-21T06:14:58.766 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34602.log 2024-09-21T06:14:58.767 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53575.log 2024-09-21T06:14:58.769 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52820.log 2024-09-21T06:14:58.769 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34602.log.gz 2024-09-21T06:14:58.769 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53575.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.53575.log.gz 2024-09-21T06:14:58.771 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52820.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52820.log.gz 2024-09-21T06:14:58.771 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42479.log 2024-09-21T06:14:58.772 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45250.log 2024-09-21T06:14:58.774 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48131.log 2024-09-21T06:14:58.774 INFO:teuthology.orchestra.run.smithi050.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.42479.log.gz 2024-09-21T06:14:58.775 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51212.log 2024-09-21T06:14:58.775 INFO:teuthology.orchestra.run.smithi050.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.45250.log.gz 2024-09-21T06:14:58.776 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36625.log 2024-09-21T06:14:58.776 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48131.log.gz 2024-09-21T06:14:58.777 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51212.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51212.log.gz 2024-09-21T06:14:58.778 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33718.log 2024-09-21T06:14:58.778 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36625.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36625.log.gz 2024-09-21T06:14:58.779 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38319.log 2024-09-21T06:14:58.780 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47092.log 2024-09-21T06:14:58.781 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33718.log.gz 2024-09-21T06:14:58.781 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38319.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38319.log.gz 2024-09-21T06:14:58.782 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44855.log 2024-09-21T06:14:58.784 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47092.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47092.log.gz 2024-09-21T06:14:58.784 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55108.log 2024-09-21T06:14:58.784 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48842.log 2024-09-21T06:14:58.784 INFO:teuthology.orchestra.run.smithi050.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.44855.log.gz 2024-09-21T06:14:58.785 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.55108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49278.log 2024-09-21T06:14:58.785 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55108.log.gz 2024-09-21T06:14:58.786 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48842.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48842.log.gz 2024-09-21T06:14:58.787 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41347.log 2024-09-21T06:14:58.787 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49278.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49278.log.gz 2024-09-21T06:14:58.788 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28961.log 2024-09-21T06:14:58.789 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52118.log 2024-09-21T06:14:58.789 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41347.log.gz 2024-09-21T06:14:58.791 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28961.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28961.log.gz 2024-09-21T06:14:58.791 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41951.log 2024-09-21T06:14:58.792 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52118.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52118.log.gz 2024-09-21T06:14:58.792 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33310.log 2024-09-21T06:14:58.794 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47617.log 2024-09-21T06:14:58.794 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41951.log.gz 2024-09-21T06:14:58.794 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44821.log 2024-09-21T06:14:58.794 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33310.log.gz 2024-09-21T06:14:58.795 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47617.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47617.log.gz 2024-09-21T06:14:58.796 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43514.log 2024-09-21T06:14:58.797 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44821.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.44821.log.gz 2024-09-21T06:14:58.797 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35278.log 2024-09-21T06:14:58.798 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41503.log 2024-09-21T06:14:58.798 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43514.log.gz 2024-09-21T06:14:58.800 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47871.log 2024-09-21T06:14:58.800 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35278.log.gz 2024-09-21T06:14:58.801 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49252.log 2024-09-21T06:14:58.801 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41503.log.gz 2024-09-21T06:14:58.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41042.log 2024-09-21T06:14:58.802 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47871.log.gz 2024-09-21T06:14:58.803 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45485.log 2024-09-21T06:14:58.803 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49252.log.gz 2024-09-21T06:14:58.805 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41042.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41042.log.gz 2024-09-21T06:14:58.805 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38020.log 2024-09-21T06:14:58.806 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45485.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.45485.log.gz 2024-09-21T06:14:58.806 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34145.log 2024-09-21T06:14:58.808 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35041.log 2024-09-21T06:14:58.808 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38020.log.gz 2024-09-21T06:14:58.809 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34145.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34145.log.gz 2024-09-21T06:14:58.809 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38944.log 2024-09-21T06:14:58.810 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.2.35041.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35041.log.gz 2024-09-21T06:14:58.810 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51537.log 2024-09-21T06:14:58.811 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34153.log 2024-09-21T06:14:58.812 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38944.log.gz 2024-09-21T06:14:58.812 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51537.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51537.log.gz 2024-09-21T06:14:58.814 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34153.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34153.log.gz 2024-09-21T06:14:58.814 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45185.log 2024-09-21T06:14:58.815 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45454.log 2024-09-21T06:14:58.817 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51132.log 2024-09-21T06:14:58.817 INFO:teuthology.orchestra.run.smithi050.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.45185.log.gz 2024-09-21T06:14:58.817 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45454.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.45454.log.gz 2024-09-21T06:14:58.819 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51132.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51132.log.gz 2024-09-21T06:14:58.819 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42369.log 2024-09-21T06:14:58.820 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35428.log 2024-09-21T06:14:58.822 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48986.log 2024-09-21T06:14:58.822 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42369.log.gz 2024-09-21T06:14:58.823 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45806.log 2024-09-21T06:14:58.823 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35428.log.gz 2024-09-21T06:14:58.824 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46067.log 2024-09-21T06:14:58.824 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48986.log.gz 2024-09-21T06:14:58.825 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45806.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45806.log.gz 2024-09-21T06:14:58.826 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41717.log 2024-09-21T06:14:58.826 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46067.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46067.log.gz 2024-09-21T06:14:58.827 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33744.log 2024-09-21T06:14:58.828 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37914.log 2024-09-21T06:14:58.828 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41717.log.gz 2024-09-21T06:14:58.829 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55160.log 2024-09-21T06:14:58.830 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33744.log.gz 2024-09-21T06:14:58.830 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33951.log 2024-09-21T06:14:58.831 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37914.log.gz 2024-09-21T06:14:58.832 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.55160.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.55160.log.gz 2024-09-21T06:14:58.832 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50645.log 2024-09-21T06:14:58.833 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33951.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33951.log.gz 2024-09-21T06:14:58.833 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39284.log 2024-09-21T06:14:58.834 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43803.log 2024-09-21T06:14:58.834 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50645.log.gz 2024-09-21T06:14:58.836 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37665.log 2024-09-21T06:14:58.836 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39284.log.gz 2024-09-21T06:14:58.837 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46495.log 2024-09-21T06:14:58.837 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43803.log.gz 2024-09-21T06:14:58.838 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37665.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37665.log.gz 2024-09-21T06:14:58.839 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38736.log 2024-09-21T06:14:58.840 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46495.log.gz 2024-09-21T06:14:58.840 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45301.log 2024-09-21T06:14:58.841 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43306.log 2024-09-21T06:14:58.841 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38736.log.gz 2024-09-21T06:14:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45301.log: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.45301.log.gz 2024-09-21T06:14:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50142.log 2024-09-21T06:14:58.844 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43306.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43306.log.gz 2024-09-21T06:14:58.845 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53967.log 2024-09-21T06:14:58.845 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50142.log.gz 2024-09-21T06:14:58.846 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44404.log 2024-09-21T06:14:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35200.log 2024-09-21T06:14:58.848 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53967.log.gz 2024-09-21T06:14:58.849 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44404.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44404.log.gz 2024-09-21T06:14:58.850 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35200.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35200.log.gz 2024-09-21T06:14:58.850 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37940.log 2024-09-21T06:14:58.851 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48582.log 2024-09-21T06:14:58.853 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46275.log 2024-09-21T06:14:58.853 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37940.log.gz 2024-09-21T06:14:58.854 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33943.log 2024-09-21T06:14:58.854 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48582.log.gz 2024-09-21T06:14:58.855 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49800.log 2024-09-21T06:14:58.855 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46275.log.gz 2024-09-21T06:14:58.856 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33943.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33943.log.gz 2024-09-21T06:14:58.857 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33935.log 2024-09-21T06:14:58.857 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49800.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49800.log.gz 2024-09-21T06:14:58.858 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38788.log 2024-09-21T06:14:58.859 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46936.log 2024-09-21T06:14:58.859 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33935.log.gz 2024-09-21T06:14:58.861 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38788.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38788.log.gz 2024-09-21T06:14:58.861 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46936.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46936.log.gz 2024-09-21T06:14:58.862 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44053.log 2024-09-21T06:14:58.863 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52040.log 2024-09-21T06:14:58.864 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44378.log 2024-09-21T06:14:58.864 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44053.log.gz 2024-09-21T06:14:58.865 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39433.log 2024-09-21T06:14:58.866 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52040.log.gz 2024-09-21T06:14:58.867 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44378.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44378.log.gz 2024-09-21T06:14:58.867 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44181.log 2024-09-21T06:14:58.869 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52768.log 2024-09-21T06:14:58.869 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39433.log.gz 2024-09-21T06:14:58.869 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51366.log 2024-09-21T06:14:58.870 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44181.log.gz 2024-09-21T06:14:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52768.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52768.log.gz 2024-09-21T06:14:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34147.log 2024-09-21T06:14:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51366.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51366.log.gz 2024-09-21T06:14:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37810.log 2024-09-21T06:14:58.874 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40403.log 2024-09-21T06:14:58.875 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34147.log.gz 2024-09-21T06:14:58.875 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37810.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37810.log.gz 2024-09-21T06:14:58.876 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39232.log 2024-09-21T06:14:58.876 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40403.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40403.log.gz 2024-09-21T06:14:58.877 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38681.log 2024-09-21T06:14:58.878 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53104.log 2024-09-21T06:14:58.878 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39232.log.gz 2024-09-21T06:14:58.879 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52659.log 2024-09-21T06:14:58.879 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.38681.log.gz 2024-09-21T06:14:58.880 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.53104.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53104.log.gz 2024-09-21T06:14:58.880 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33822.log 2024-09-21T06:14:58.881 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52659.log: gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.28718.log 2024-09-21T06:14:58.881 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52659.log.gz 2024-09-21T06:14:58.883 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50168.log 2024-09-21T06:14:58.883 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33822.log.gz 2024-09-21T06:14:58.884 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28718.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28718.log.gz 2024-09-21T06:14:58.885 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39570.log 2024-09-21T06:14:58.885 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50168.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50168.log.gz 2024-09-21T06:14:58.886 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35917.log 2024-09-21T06:14:58.887 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35312.log 2024-09-21T06:14:58.887 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39570.log.gz 2024-09-21T06:14:58.888 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35917.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35917.log.gz 2024-09-21T06:14:58.888 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53995.log 2024-09-21T06:14:58.889 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35312.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35312.log.gz 2024-09-21T06:14:58.889 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36114.log 2024-09-21T06:14:58.890 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50619.log 2024-09-21T06:14:58.890 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53995.log.gz 2024-09-21T06:14:58.892 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36469.log 2024-09-21T06:14:58.892 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36114.log.gz 2024-09-21T06:14:58.893 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50619.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50619.log.gz 2024-09-21T06:14:58.893 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36495.log 2024-09-21T06:14:58.894 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51864.log 2024-09-21T06:14:58.894 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36469.log.gz 2024-09-21T06:14:58.895 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44131.log 2024-09-21T06:14:58.896 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36495.log.gz 2024-09-21T06:14:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51864.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51864.log.gz 2024-09-21T06:14:58.897 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44881.log 2024-09-21T06:14:58.898 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49226.log 2024-09-21T06:14:58.898 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44131.log.gz 2024-09-21T06:14:58.899 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40866.log 2024-09-21T06:14:58.900 INFO:teuthology.orchestra.run.smithi050.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.44881.log.gz 2024-09-21T06:14:58.901 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49226.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49226.log.gz 2024-09-21T06:14:58.901 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35046.log 2024-09-21T06:14:58.902 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40866.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40866.log.gz 2024-09-21T06:14:58.902 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42343.log 2024-09-21T06:14:58.904 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.2.35046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47422.log 2024-09-21T06:14:58.904 INFO:teuthology.orchestra.run.smithi050.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35046.log.gz 2024-09-21T06:14:58.905 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38973.log 2024-09-21T06:14:58.905 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42343.log.gz 2024-09-21T06:14:58.906 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34369.log 2024-09-21T06:14:58.906 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47422.log.gz 2024-09-21T06:14:58.908 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38973.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38973.log.gz 2024-09-21T06:14:58.908 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51511.log 2024-09-21T06:14:58.909 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34369.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34369.log.gz 2024-09-21T06:14:58.909 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T06:14:58.910 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39946.log 2024-09-21T06:14:58.911 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51511.log.gz 2024-09-21T06:14:58.913 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.39946.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39946.log.gz 2024-09-21T06:14:58.913 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48407.log 2024-09-21T06:14:58.915 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43975.log 2024-09-21T06:14:58.916 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48407.log: 90.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T06:14:58.916 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48407.log.gz 2024-09-21T06:14:58.917 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-21T06:14:58.917 INFO:teuthology.orchestra.run.smithi088.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T06:14:58.917 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43975.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43975.log.gz 2024-09-21T06:14:58.917 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46657.log 2024-09-21T06:14:58.919 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39998.log 2024-09-21T06:14:58.920 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46657.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46657.log.gz 2024-09-21T06:14:58.921 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39998.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.39998.log.gz 2024-09-21T06:14:58.921 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43722.log 2024-09-21T06:14:58.923 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45033.log 2024-09-21T06:14:58.924 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43722.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43722.log.gz 2024-09-21T06:14:58.925 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45033.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.45033.log.gz 2024-09-21T06:14:58.926 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38629.log 2024-09-21T06:14:58.927 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43540.log 2024-09-21T06:14:58.929 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38629.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.38629.log.gz 2024-09-21T06:14:58.929 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43540.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43540.log.gz 2024-09-21T06:14:58.930 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39602.log 2024-09-21T06:14:58.931 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34342.log 2024-09-21T06:14:58.933 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39602.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39602.log.gz 2024-09-21T06:14:58.934 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34342.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34342.log.gz 2024-09-21T06:14:58.934 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-21T06:14:58.935 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52397.log 2024-09-21T06:14:58.938 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.52397.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52397.log.gz 2024-09-21T06:14:58.938 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34931.log 2024-09-21T06:14:58.941 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.34931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.34931.log.gz 2024-09-21T06:14:58.941 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36871.log 2024-09-21T06:14:58.943 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28833.log 2024-09-21T06:14:58.947 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28833.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28833.log.gz 2024-09-21T06:14:58.947 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36871.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36871.log.gz 2024-09-21T06:14:58.947 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54073.log 2024-09-21T06:14:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54073.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54073.log.gz 2024-09-21T06:14:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36443.log 2024-09-21T06:14:58.953 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36443.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36443.log.gz 2024-09-21T06:14:58.953 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42003.log 2024-09-21T06:14:58.955 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34137.log 2024-09-21T06:14:58.956 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42003.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42003.log.gz 2024-09-21T06:14:58.958 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36987.log 2024-09-21T06:14:58.961 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34137.log: gzip/var/log/ceph/ceph-client.admin.36987.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34137.log.gz 2024-09-21T06:14:58.961 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40707.log 2024-09-21T06:14:58.961 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36987.log.gz 2024-09-21T06:14:58.963 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37019.log 2024-09-21T06:14:58.963 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40707.log.gz 2024-09-21T06:14:58.966 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45590.log 2024-09-21T06:14:58.966 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37019.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37019.log.gz 2024-09-21T06:14:58.967 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52976.log 2024-09-21T06:14:58.969 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45590.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.45590.log.gz 2024-09-21T06:14:58.970 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50905.log 2024-09-21T06:14:58.973 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52976.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52976.log.gz 2024-09-21T06:14:58.974 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50905.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50905.log.gz 2024-09-21T06:14:58.974 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36923.log 2024-09-21T06:14:58.976 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49706.log 2024-09-21T06:14:58.976 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36923.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36923.log.gz 2024-09-21T06:14:58.977 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47597.log 2024-09-21T06:14:58.981 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49706.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49706.log.gz 2024-09-21T06:14:58.981 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28901.log 2024-09-21T06:14:58.981 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.47597.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47597.log.gz 2024-09-21T06:14:58.982 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46754.log 2024-09-21T06:14:58.984 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41295.log 2024-09-21T06:14:58.987 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46754.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46754.log.gz 2024-09-21T06:14:58.987 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28901.log.gz 2024-09-21T06:14:58.987 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-09-21T06:14:58.988 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41295.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41295.log.gz 2024-09-21T06:14:58.991 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5/var/log/ceph/ceph-client.admin.42198.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-09-21T06:14:58.992 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40325.log 2024-09-21T06:14:58.994 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53915.log 2024-09-21T06:14:58.995 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40325.log.gz 2024-09-21T06:14:58.995 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48712.log 2024-09-21T06:14:59.001 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53915.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53915.log.gz 2024-09-21T06:14:59.002 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48712.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48712.log.gz 2024-09-21T06:14:59.002 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42846.log 2024-09-21T06:14:59.002 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42846.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42846.log.gz 2024-09-21T06:14:59.002 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36423.log 2024-09-21T06:14:59.004 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.36423.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36423.log.gz 2024-09-21T06:14:59.004 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48303.log 2024-09-21T06:14:59.007 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41585.log 2024-09-21T06:14:59.007 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48303.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48303.log.gz 2024-09-21T06:14:59.010 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41585.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41585.log.gz 2024-09-21T06:14:59.010 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54639.log 2024-09-21T06:14:59.013 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54639.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54639.log.gz 2024-09-21T06:14:59.013 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47839.log 2024-09-21T06:14:59.019 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47839.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47839.log.gz 2024-09-21T06:14:59.019 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43144.log 2024-09-21T06:14:59.019 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43144.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43144.log.gz 2024-09-21T06:14:59.019 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51975.log 2024-09-21T06:14:59.022 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51975.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51975.log.gz 2024-09-21T06:14:59.022 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42557.log 2024-09-21T06:14:59.025 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42557.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42557.log.gz 2024-09-21T06:14:59.025 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46197.log 2024-09-21T06:14:59.027 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40076.log 2024-09-21T06:14:59.028 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46197.log: /var/log/ceph/ceph-client.admin.40076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55004.log 2024-09-21T06:14:59.035 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40076.log.gz 2024-09-21T06:14:59.035 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46197.log.gz 2024-09-21T06:14:59.035 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.55004.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55004.log.gz 2024-09-21T06:14:59.035 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41269.log 2024-09-21T06:14:59.035 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41269.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41269.log.gz 2024-09-21T06:14:59.035 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43030.log 2024-09-21T06:14:59.037 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43030.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43030.log.gz 2024-09-21T06:14:59.038 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39972.log 2024-09-21T06:14:59.040 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48777.log 2024-09-21T06:14:59.041 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39972.log.gz 2024-09-21T06:14:59.043 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39777.log 2024-09-21T06:14:59.045 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48777.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48777.log.gz 2024-09-21T06:14:59.047 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42053.log/var/log/ceph/ceph-client.admin.39777.log: 2024-09-21T06:14:59.048 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39777.log.gz 2024-09-21T06:14:59.050 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42768.log 2024-09-21T06:14:59.051 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38570.log 2024-09-21T06:14:59.053 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42768.log: /var/log/ceph/ceph-client.admin.42053.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42053.log.gz 2024-09-21T06:14:59.054 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42768.log.gz 2024-09-21T06:14:59.056 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38570.log: 86.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51076.log 2024-09-21T06:14:59.056 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38570.log.gz 2024-09-21T06:14:59.057 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46353.log 2024-09-21T06:14:59.061 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51076.log: /var/log/ceph/ceph-client.admin.46353.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46353.log.gz 2024-09-21T06:14:59.062 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37519.log 2024-09-21T06:14:59.062 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51076.log.gz 2024-09-21T06:14:59.064 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50366.log 2024-09-21T06:14:59.065 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37519.log.gz 2024-09-21T06:14:59.068 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50366.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50366.log.gz 2024-09-21T06:14:59.068 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38814.log 2024-09-21T06:14:59.071 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38814.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38814.log.gz 2024-09-21T06:14:59.071 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35480.log 2024-09-21T06:14:59.072 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49484.log 2024-09-21T06:14:59.074 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35480.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35480.log.gz 2024-09-21T06:14:59.077 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.49484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49484.log.gz 2024-09-21T06:14:59.077 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44553.log 2024-09-21T06:14:59.079 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51050.log 2024-09-21T06:14:59.080 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44553.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44553.log.gz 2024-09-21T06:14:59.086 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51050.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51050.log.gz 2024-09-21T06:14:59.086 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37376.log 2024-09-21T06:14:59.086 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37376.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37376.log.gz 2024-09-21T06:14:59.087 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35252.log 2024-09-21T06:14:59.089 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35252.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35252.log.gz 2024-09-21T06:14:59.089 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35070.log 2024-09-21T06:14:59.092 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35070.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35070.log.gz 2024-09-21T06:14:59.092 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41613.log 2024-09-21T06:14:59.095 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35969.log 2024-09-21T06:14:59.095 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41613.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41613.log.gz 2024-09-21T06:14:59.098 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35969.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35969.log.gz 2024-09-21T06:14:59.098 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48894.log 2024-09-21T06:14:59.101 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48894.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48894.log.gz 2024-09-21T06:14:59.101 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54900.log 2024-09-21T06:14:59.104 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54900.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54900.log.gz 2024-09-21T06:14:59.104 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43002.log 2024-09-21T06:14:59.110 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43002.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43002.log.gz 2024-09-21T06:14:59.110 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40377.log 2024-09-21T06:14:59.110 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40377.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40377.log.gz 2024-09-21T06:14:59.110 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54744.log 2024-09-21T06:14:59.113 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54744.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54744.log.gz 2024-09-21T06:14:59.113 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54229.log 2024-09-21T06:14:59.114 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42898.log 2024-09-21T06:14:59.116 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54229.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54229.log.gz 2024-09-21T06:14:59.116 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48530.log 2024-09-21T06:14:59.116 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42898.log.gz 2024-09-21T06:14:59.119 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48530.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48530.log.gz 2024-09-21T06:14:59.119 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40759.log 2024-09-21T06:14:59.122 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40759.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.40759.log.gz 2024-09-21T06:14:59.122 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46910.log 2024-09-21T06:14:59.125 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46910.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46910.log.gz 2024-09-21T06:14:59.125 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45832.log 2024-09-21T06:14:59.126 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38148.log 2024-09-21T06:14:59.128 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45832.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45832.log.gz 2024-09-21T06:14:59.131 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38148.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38148.log.gz 2024-09-21T06:14:59.131 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34680.log 2024-09-21T06:14:59.134 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34680.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34680.log.gz 2024-09-21T06:14:59.134 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44274.log 2024-09-21T06:14:59.139 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44274.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2024-09-21T06:14:59.139 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28797.log 2024-09-21T06:14:59.139 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28797.log.gz 2024-09-21T06:14:59.140 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47118.log 2024-09-21T06:14:59.142 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44999.log 2024-09-21T06:14:59.143 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47118.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48105.log 2024-09-21T06:14:59.143 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-09-21T06:14:59.146 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44999.log: /var/log/ceph/ceph-client.admin.48105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42505.log 2024-09-21T06:14:59.147 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48105.log.gz 2024-09-21T06:14:59.147 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37635.log 2024-09-21T06:14:59.149 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42505.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.42505.log.gz 2024-09-21T06:14:59.150 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37635.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37635.log.gz 2024-09-21T06:14:59.151 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49330.log 2024-09-21T06:14:59.151 INFO:teuthology.orchestra.run.smithi050.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.44999.log.gz 2024-09-21T06:14:59.154 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50340.log 2024-09-21T06:14:59.155 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49330.log.gz 2024-09-21T06:14:59.156 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55212.log 2024-09-21T06:14:59.158 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50340.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50340.log.gz 2024-09-21T06:14:59.160 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5/var/log/ceph/ceph-client.admin.55212.log: --verbose -- /var/log/ceph/ceph-client.admin.42687.log 2024-09-21T06:14:59.161 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55212.log.gz 2024-09-21T06:14:59.163 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35343.log 2024-09-21T06:14:59.166 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35343.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35343.log.gz 2024-09-21T06:14:59.167 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49680.log 2024-09-21T06:14:59.167 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42687.log.gz 2024-09-21T06:14:59.170 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49680.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49680.log.gz 2024-09-21T06:14:59.170 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53681.log 2024-09-21T06:14:59.173 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53681.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.53681.log.gz 2024-09-21T06:14:59.173 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41321.log 2024-09-21T06:14:59.176 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41321.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41321.log.gz 2024-09-21T06:14:59.176 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52222.log 2024-09-21T06:14:59.179 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52222.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52222.log.gz 2024-09-21T06:14:59.179 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33954.log 2024-09-21T06:14:59.182 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44527.log 2024-09-21T06:14:59.182 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33954.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33954.log.gz 2024-09-21T06:14:59.185 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44527.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.44527.log.gz 2024-09-21T06:14:59.185 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52477.log 2024-09-21T06:14:59.188 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52477.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52477.log.gz 2024-09-21T06:14:59.188 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45076.log 2024-09-21T06:14:59.191 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51413.log 2024-09-21T06:14:59.191 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45076.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45076.log.gz 2024-09-21T06:14:59.193 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.51413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34628.log 2024-09-21T06:14:59.194 INFO:teuthology.orchestra.run.smithi050.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51413.log.gz 2024-09-21T06:14:59.197 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.34628.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34628.log.gz 2024-09-21T06:14:59.197 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35372.log 2024-09-21T06:14:59.199 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50522.log 2024-09-21T06:14:59.201 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.35372.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35372.log.gz 2024-09-21T06:14:59.203 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50522.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50522.log.gz 2024-09-21T06:14:59.203 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41373.log 2024-09-21T06:14:59.206 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38492.log 2024-09-21T06:14:59.206 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41373.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41373.log.gz 2024-09-21T06:14:59.209 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38492.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38492.log.gz 2024-09-21T06:14:59.209 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28925.log 2024-09-21T06:14:59.215 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28925.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28925.log.gz 2024-09-21T06:14:59.215 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39206.log 2024-09-21T06:14:59.215 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39206.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39206.log.gz 2024-09-21T06:14:59.215 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54427.log 2024-09-21T06:14:59.218 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54427.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54427.log.gz 2024-09-21T06:14:59.218 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39812.log 2024-09-21T06:14:59.221 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39812.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39812.log.gz 2024-09-21T06:14:59.221 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49122.log 2024-09-21T06:14:59.224 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49122.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49122.log.gz 2024-09-21T06:14:59.224 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43831.log 2024-09-21T06:14:59.227 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38598.log 2024-09-21T06:14:59.227 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43831.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43831.log.gz 2024-09-21T06:14:59.229 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38598.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38598.log.gz 2024-09-21T06:14:59.230 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34351.log 2024-09-21T06:14:59.233 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34351.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34351.log.gz 2024-09-21T06:14:59.233 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54718.log 2024-09-21T06:14:59.236 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54718.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54718.log.gz 2024-09-21T06:14:59.236 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37097.log 2024-09-21T06:14:59.237 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48951.log 2024-09-21T06:14:59.239 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37097.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37097.log.gz 2024-09-21T06:14:59.242 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48951.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48951.log.gz 2024-09-21T06:14:59.242 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28785.log 2024-09-21T06:14:59.244 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28785.log.gz 2024-09-21T06:14:59.244 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28973.log 2024-09-21T06:14:59.247 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29388.log 2024-09-21T06:14:59.247 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28973.log: 74.4% -- replaced with /var/log/ceph/ceph-client.admin.28973.log.gz 2024-09-21T06:14:59.249 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.29388.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29388.log.gz 2024-09-21T06:14:59.250 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43384.log 2024-09-21T06:14:59.252 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43384.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43384.log.gz 2024-09-21T06:14:59.253 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49852.log 2024-09-21T06:14:59.256 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49852.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.49852.log.gz 2024-09-21T06:14:59.256 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41120.log 2024-09-21T06:14:59.258 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41120.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41120.log.gz 2024-09-21T06:14:59.259 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46832.log 2024-09-21T06:14:59.261 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47550.log 2024-09-21T06:14:59.262 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46832.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46832.log.gz 2024-09-21T06:14:59.264 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47550.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47550.log.gz 2024-09-21T06:14:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45989.log 2024-09-21T06:14:59.267 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45989.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45989.log.gz 2024-09-21T06:14:59.267 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53099.log 2024-09-21T06:14:59.270 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.53099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53099.log.gz 2024-09-21T06:14:59.270 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50308.log 2024-09-21T06:14:59.275 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52872.log 2024-09-21T06:14:59.275 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50308.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50308.log.gz 2024-09-21T06:14:59.275 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43892.log 2024-09-21T06:14:59.275 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52872.log.gz 2024-09-21T06:14:59.278 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43892.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43892.log.gz 2024-09-21T06:14:59.279 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42146.log 2024-09-21T06:14:59.281 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33456.log 2024-09-21T06:14:59.282 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42146.log.gz 2024-09-21T06:14:59.283 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51589.log 2024-09-21T06:14:59.285 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33456.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33456.log.gz 2024-09-21T06:14:59.288 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5/var/log/ceph/ceph-client.admin.51589.log: --verbose -- /var/log/ceph/ceph-client.admin.38293.log 2024-09-21T06:14:59.289 INFO:teuthology.orchestra.run.smithi050.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.51589.log.gz 2024-09-21T06:14:59.291 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55267.log 2024-09-21T06:14:59.292 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38293.log.gz 2024-09-21T06:14:59.294 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.55267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33510.log 2024-09-21T06:14:59.295 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55267.log.gz 2024-09-21T06:14:59.296 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41821.log 2024-09-21T06:14:59.299 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33510.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33510.log.gz 2024-09-21T06:14:59.300 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41821.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41821.log.gz 2024-09-21T06:14:59.300 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54151.log 2024-09-21T06:14:59.302 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54151.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54151.log.gz 2024-09-21T06:14:59.303 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47949.log 2024-09-21T06:14:59.307 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54401.log 2024-09-21T06:14:59.307 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47949.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.47949.log.gz 2024-09-21T06:14:59.308 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50246.log 2024-09-21T06:14:59.309 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54401.log.gz 2024-09-21T06:14:59.310 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47144.log 2024-09-21T06:14:59.312 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50246.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50246.log.gz 2024-09-21T06:14:59.314 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29333.log 2024-09-21T06:14:59.317 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49654.log 2024-09-21T06:14:59.317 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47144.log.gz 2024-09-21T06:14:59.319 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.29333.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29333.log.gz 2024-09-21T06:14:59.321 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33947.log 2024-09-21T06:14:59.322 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49654.log.gz 2024-09-21T06:14:59.324 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48355.log 2024-09-21T06:14:59.327 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48355.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48355.log.gz 2024-09-21T06:14:59.328 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33947.log.gz 2024-09-21T06:14:59.328 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40232.log 2024-09-21T06:14:59.334 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40232.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40232.log.gz 2024-09-21T06:14:59.334 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48053.log 2024-09-21T06:14:59.334 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48053.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48053.log.gz 2024-09-21T06:14:59.334 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50750.log 2024-09-21T06:14:59.337 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50750.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50750.log.gz 2024-09-21T06:14:59.337 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44469.log 2024-09-21T06:14:59.340 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47370.log 2024-09-21T06:14:59.340 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44469.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44469.log.gz 2024-09-21T06:14:59.343 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47370.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47370.log.gz 2024-09-21T06:14:59.343 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38518.log 2024-09-21T06:14:59.346 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38241.log 2024-09-21T06:14:59.347 INFO:teuthology.orchestra.run.smithi050.stderr: 86.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40733.log 2024-09-21T06:14:59.355 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38241.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38241.log.gz 2024-09-21T06:14:59.355 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40733.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40733.log.gz 2024-09-21T06:14:59.355 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38518.log.gz 2024-09-21T06:14:59.355 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39864.log 2024-09-21T06:14:59.355 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39864.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39864.log.gz 2024-09-21T06:14:59.355 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50671.log 2024-09-21T06:14:59.355 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50671.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50671.log.gz 2024-09-21T06:14:59.356 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51290.log 2024-09-21T06:14:59.358 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39006.log 2024-09-21T06:14:59.360 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51290.log.gz 2024-09-21T06:14:59.360 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39407.log 2024-09-21T06:14:59.362 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39006.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39006.log.gz 2024-09-21T06:14:59.363 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48245.log 2024-09-21T06:14:59.364 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39407.log.gz 2024-09-21T06:14:59.365 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45565.log 2024-09-21T06:14:59.368 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48245.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48245.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28865.log 2024-09-21T06:14:59.368 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-09-21T06:14:59.370 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.45565.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.45565.log.gz 2024-09-21T06:14:59.371 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51563.log 2024-09-21T06:14:59.373 INFO:teuthology.orchestra.run.smithi050.stderr: 75.6% -- replaced with /var/log/ceph/ceph-client.admin.28865.log.gz 2024-09-21T06:14:59.374 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53811.log 2024-09-21T06:14:59.375 INFO:teuthology.orchestra.run.smithi050.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.51563.log.gz 2024-09-21T06:14:59.377 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42714.log 2024-09-21T06:14:59.381 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.42714.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.42714.log.gz 2024-09-21T06:14:59.382 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53811.log.gz 2024-09-21T06:14:59.382 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37201.log 2024-09-21T06:14:59.384 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37201.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37201.log.gz 2024-09-21T06:14:59.385 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54611.log 2024-09-21T06:14:59.390 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54611.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54611.log.gz 2024-09-21T06:14:59.391 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51459.log 2024-09-21T06:14:59.391 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.51459.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51459.log.gz 2024-09-21T06:14:59.391 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46884.log 2024-09-21T06:14:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.46884.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46884.log.gz 2024-09-21T06:14:59.393 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38215.log 2024-09-21T06:14:59.395 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43176.log 2024-09-21T06:14:59.397 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38215.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38215.log.gz 2024-09-21T06:14:59.397 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34576.log 2024-09-21T06:14:59.397 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43176.log.gz 2024-09-21T06:14:59.400 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34576.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34576.log.gz 2024-09-21T06:14:59.400 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47318.log 2024-09-21T06:14:59.403 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.47318.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47318.log.gz 2024-09-21T06:14:59.403 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28913.log 2024-09-21T06:14:59.406 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28913.log: 75.7% -- replaced with /var/log/ceph/ceph-client.admin.28913.log.gz 2024-09-21T06:14:59.406 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49070.log 2024-09-21T06:14:59.409 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.49070.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49070.log.gz 2024-09-21T06:14:59.409 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54099.log 2024-09-21T06:14:59.413 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28817.log 2024-09-21T06:14:59.413 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54099.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54099.log.gz 2024-09-21T06:14:59.413 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28817.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28817.log.gz 2024-09-21T06:14:59.413 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.28931.log 2024-09-21T06:14:59.415 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.28931.log: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.28931.log.gz 2024-09-21T06:14:59.415 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41639.log 2024-09-21T06:14:59.418 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.41639.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41639.log.gz 2024-09-21T06:14:59.418 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39695.log 2024-09-21T06:14:59.421 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39695.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39695.log.gz 2024-09-21T06:14:59.421 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34913.log 2024-09-21T06:14:59.424 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.34913.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.34913.log.gz 2024-09-21T06:14:59.424 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36844.log 2024-09-21T06:14:59.427 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36844.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36844.log.gz 2024-09-21T06:14:59.427 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52170.log 2024-09-21T06:14:59.430 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54125.log 2024-09-21T06:14:59.430 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52170.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52170.log.gz 2024-09-21T06:14:59.433 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54125.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54125.log.gz 2024-09-21T06:14:59.433 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53002.log 2024-09-21T06:14:59.436 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44761.log 2024-09-21T06:14:59.436 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53002.log.gz 2024-09-21T06:14:59.437 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37246.log 2024-09-21T06:14:59.441 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.44761.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.44761.log.gz 2024-09-21T06:14:59.441 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37246.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37246.log.gz 2024-09-21T06:14:59.442 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52092.log 2024-09-21T06:14:59.447 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.52092.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52092.log.gz 2024-09-21T06:14:59.447 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40837.log 2024-09-21T06:14:59.447 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40837.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40837.log.gz 2024-09-21T06:14:59.448 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38892.log 2024-09-21T06:14:59.450 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50275.log 2024-09-21T06:14:59.451 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38892.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36792.log 2024-09-21T06:14:59.451 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-09-21T06:14:59.454 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.50275.log: gzip/var/log/ceph/ceph-client.admin.36792.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39838.log 2024-09-21T06:14:59.455 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36792.log.gz 2024-09-21T06:14:59.456 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38544.log 2024-09-21T06:14:59.459 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.39838.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39838.log.gz 2024-09-21T06:14:59.459 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38544.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.38544.log.gz 2024-09-21T06:14:59.459 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38195.log 2024-09-21T06:14:59.460 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50275.log.gz 2024-09-21T06:14:59.462 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.38195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38195.log.gz 2024-09-21T06:14:59.462 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43566.log 2024-09-21T06:14:59.465 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43566.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43566.log.gz 2024-09-21T06:14:59.465 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48157.log 2024-09-21T06:14:59.468 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.48157.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48157.log.gz 2024-09-21T06:14:59.468 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43111.log 2024-09-21T06:14:59.471 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43111.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43111.log.gz 2024-09-21T06:14:59.471 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38267.log 2024-09-21T06:14:59.474 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53733.log 2024-09-21T06:14:59.474 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.38267.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38267.log.gz 2024-09-21T06:14:59.477 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53733.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.53733.log.gz 2024-09-21T06:14:59.477 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36144.log 2024-09-21T06:14:59.480 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.36144.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36144.log.gz 2024-09-21T06:14:59.480 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54583.log 2024-09-21T06:14:59.483 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54583.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54583.log.gz 2024-09-21T06:14:59.483 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43410.log 2024-09-21T06:14:59.486 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53889.log 2024-09-21T06:14:59.486 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43410.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43410.log.gz 2024-09-21T06:14:59.489 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.53889.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53889.log.gz 2024-09-21T06:14:59.489 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40962.log 2024-09-21T06:14:59.492 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.40962.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40962.log.gz 2024-09-21T06:14:59.492 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33948.log 2024-09-21T06:14:59.495 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.33948.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33948.log.gz 2024-09-21T06:14:59.495 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43866.log 2024-09-21T06:14:59.498 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37298.log 2024-09-21T06:14:59.498 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.43866.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43866.log.gz 2024-09-21T06:14:59.501 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.37298.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37298.log.gz 2024-09-21T06:14:59.501 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54255.log 2024-09-21T06:14:59.503 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.54255.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54255.log.gz 2024-09-21T06:14:59.775 INFO:teuthology.orchestra.run.smithi050.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T06:15:00.046 INFO:teuthology.orchestra.run.smithi050.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-21T06:15:01.386 INFO:teuthology.orchestra.run.smithi088.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-21T06:15:01.756 INFO:teuthology.orchestra.run.smithi088.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T06:15:02.646 INFO:teuthology.orchestra.run.smithi050.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T06:15:02.979 INFO:teuthology.orchestra.run.smithi050.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T06:15:03.312 INFO:teuthology.orchestra.run.smithi088.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T06:15:03.373 INFO:teuthology.orchestra.run.smithi050.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T06:15:03.402 INFO:teuthology.orchestra.run.smithi088.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi088.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T06:15:03.491 INFO:teuthology.orchestra.run.smithi088.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T06:15:03.493 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-09-21T06:15:03.493 INFO:teuthology.orchestra.run.smithi088.stderr:real 0m5.394s 2024-09-21T06:15:03.493 INFO:teuthology.orchestra.run.smithi088.stderr:user 0m26.991s 2024-09-21T06:15:03.493 INFO:teuthology.orchestra.run.smithi088.stderr:sys 0m2.262s 2024-09-21T06:15:03.753 INFO:teuthology.orchestra.run.smithi050.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T06:15:03.853 INFO:teuthology.orchestra.run.smithi050.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-21T06:15:03.855 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-09-21T06:15:03.855 INFO:teuthology.orchestra.run.smithi050.stderr:real 0m5.769s 2024-09-21T06:15:03.855 INFO:teuthology.orchestra.run.smithi050.stderr:user 0m25.280s 2024-09-21T06:15:03.855 INFO:teuthology.orchestra.run.smithi050.stderr:sys 0m3.189s 2024-09-21T06:15:03.856 INFO:tasks.ceph:Archiving logs... 2024-09-21T06:15:03.856 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513/remote/smithi050/log 2024-09-21T06:15:03.857 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:15:06.869 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513/remote/smithi088/log 2024-09-21T06:15:06.870 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:15:09.489 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T06:15:09.500 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T06:15:09.500 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:15:09.504 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:15:09.594 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T06:15:09.603 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T06:15:09.603 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:15:09.606 DEBUG:teuthology.orchestra.run.smithi088:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:15:09.619 INFO:teuthology.orchestra.run.smithi050.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T06:15:09.620 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================== 2024-09-21T06:15:09.620 INFO:teuthology.orchestra.run.smithi050.stdout:*hv01.front.sepi 67.205.162.81 3 u 52 64 377 0.088 -5.895 0.846 2024-09-21T06:15:09.620 INFO:teuthology.orchestra.run.smithi050.stdout:+hv02.front.sepi 63.231.80.2 3 u 48 64 377 0.087 -7.461 0.976 2024-09-21T06:15:09.620 INFO:teuthology.orchestra.run.smithi050.stdout:+hv03.front.sepi 74.6.168.72 3 u 43 64 377 0.079 -5.143 1.355 2024-09-21T06:15:09.620 INFO:teuthology.orchestra.run.smithi050.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-21T06:15:09.621 INFO:teuthology.orchestra.run.smithi088.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T06:15:09.622 INFO:teuthology.orchestra.run.smithi088.stdout:============================================================================== 2024-09-21T06:15:09.622 INFO:teuthology.orchestra.run.smithi088.stdout:+hv01.front.sepi 67.205.162.81 3 u 48 64 377 0.085 -5.797 0.567 2024-09-21T06:15:09.622 INFO:teuthology.orchestra.run.smithi088.stdout:*hv02.front.sepi 63.231.80.2 3 u 56 64 377 0.086 -6.095 0.465 2024-09-21T06:15:09.622 INFO:teuthology.orchestra.run.smithi088.stdout:+hv03.front.sepi 74.6.168.72 3 u 39 64 377 0.102 -5.726 0.578 2024-09-21T06:15:09.622 INFO:teuthology.orchestra.run.smithi088.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-21T06:15:09.623 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T06:15:09.634 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T06:15:09.634 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T06:15:09.685 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T06:15:09.735 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T06:15:09.790 INFO:teuthology.task.internal:Duration was 1873.407266 seconds 2024-09-21T06:15:09.790 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T06:15:09.880 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T06:15:09.880 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:15:09.884 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:15:09.983 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T06:15:09.984 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2024-09-21T06:15:09.984 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:15:10.006 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T06:15:10.006 DEBUG:teuthology.orchestra.run.smithi088:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:15:10.028 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T06:15:10.028 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:15:10.054 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:15:10.112 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T06:15:10.112 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:15:10.145 DEBUG:teuthology.orchestra.run.smithi088:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:15:10.235 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T06:15:10.246 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T06:15:10.246 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:15:10.266 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:15:10.286 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T06:15:10.296 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:15:10.309 DEBUG:teuthology.orchestra.run.smithi088:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:15:10.324 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2024-09-21T06:15:10.346 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = core 2024-09-21T06:15:10.364 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:15:10.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:15:10.389 DEBUG:teuthology.orchestra.run.smithi088:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:15:10.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:15:10.418 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T06:15:10.429 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T06:15:10.429 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513/remote/smithi050 2024-09-21T06:15:10.430 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:15:10.457 DEBUG:teuthology.misc:Transferring archived files from smithi088:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913513/remote/smithi088 2024-09-21T06:15:10.457 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:15:10.490 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T06:15:10.490 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:15:10.497 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:15:10.537 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T06:15:10.549 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T06:15:10.550 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T06:15:10.586 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T06:15:10.586 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:15:10.590 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:15:10.595 INFO:teuthology.orchestra.run.smithi050.stdout: 658143 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:15 /home/ubuntu/cephtest 2024-09-21T06:15:10.598 INFO:teuthology.orchestra.run.smithi088.stdout: 658142 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:15 /home/ubuntu/cephtest 2024-09-21T06:15:10.600 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T06:15:10.635 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T06:15:10.744 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} duration: 1873.4072661399841 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T06:15:10.744 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T06:15:10.827 INFO:teuthology.run:pass